Algorithms and Accountability: What Residential Homes Need to Know About AI in Children's Social Care

Artificial intelligence has arrived in children's social care in a way that is neither uniform nor well-understood, but which is now sufficiently widespread to require serious attention from anyone working with looked-after children. The tools in question are not science fiction. They include predictive risk-scoring systems used by some local authorities to identify children at risk of harm before an incident occurs; machine-learning models that analyse patterns across case records to produce recommendations about placement type or intervention intensity; automated flagging tools embedded in case management systems that surface certain combinations of information as requiring urgent review; and, increasingly, large language model tools used to draft assessments, summarise records, or generate care planning documents. None of these tools are inherently dangerous, and several of them address genuine problems — the administrative burden on social workers, the volume of case records that must be synthesised, the difficulty of identifying patterns across large datasets that human attention alone would miss. But all of them embed assumptions, produce outputs that carry authority they may not always deserve, and operate in a professional context in which the young people most affected by them have no meaningful visibility of, or voice in, how they are used. For residential homes, this matters in specific and practical ways.

The use of predictive analytics in children's social care has a history in England that predates the current wave of AI interest by at least a decade, but which has accelerated substantially since 2022. Tools used by some local authorities attempt to identify, from patterns in administrative data — previous referrals, housing records, benefit claims, criminal justice contacts — which families or children are at elevated risk of harm. The ethical and empirical objections to this approach are not primarily technical but structural. The data these models train on reflects the history of professional intervention, not the underlying distribution of harm: families who have been visited, assessed, and recorded are over-represented in the training data relative to families who have experienced equivalent difficulties without professional contact. The characteristics that appear predictive — poverty, welfare receipt, housing instability, prior involvement with services — are not causes of harm but correlates of it, and they are also correlates of race, of geography, of socioeconomic disadvantage. A model trained to predict future involvement with children's services will systematically over-predict risk in communities that have historically had more contact with those services, which are disproportionately Black and mixed-heritage communities. Research published by the Alan Turing Institute, by the Children's Commissioner's office, and by academic groups tracking the use of algorithmic tools in public services has documented this pattern across multiple implementations. For young people already in residential care — almost all of whom will have extensive records, multiple prior referrals, and profiles that any predictive model would flag as high-risk — the implications of encountering these tools at subsequent decision points in their lives are significant, and the implications for their children, if they later become parents, are more significant still.

The accountability gap that AI tools create in professional decision-making is a specific problem that residential workers should understand, because it will increasingly shape the professional landscape in which they operate. When a social worker uses a risk-scoring tool that recommends a particular level of intervention and acts on that recommendation, the question of who is responsible for the outcome of the decision becomes genuinely difficult. The social worker was not the person who designed or calibrated the model. The model was not the person who made the final decision. The organisation that deployed the model may carry some liability, but the causal chain between the algorithm's output and the decision made in its name is not always clear. This matters for residential homes when those decisions involve their young people: when a placement decision is influenced by an algorithmic recommendation, when a risk assessment generated in part by automated tools forms the basis of a restriction on a young person's movements or contact, when an AI-drafted care plan shapes what a home is expected to deliver without a human having constructed it from a genuine understanding of the young person's situation. The general data protection regulation, as it applies to automated decision-making, includes provisions that are in principle relevant here: individuals have rights in relation to decisions made solely by automated processes that significantly affect them, including in theory the right to know that such a process has been used and to request human review. In practice, the opacity of many algorithmic tools used in children's services — deployed via procurement contracts with providers who treat their models as commercially confidential — makes these rights difficult to exercise. Children in care, including those in residential homes, are particularly poorly positioned to exercise them without support. Advocacy organisations, independent reviewing officers, and residential staff who understand the landscape are currently among the few sources of such support that exist.

The relationship between AI tools and the case records that residential homes generate is a specific area of emerging concern that deserves more attention than it has received. Residential homes produce substantial documentation: daily logs, incident reports, keywork records, health summaries, Regulation 44 reports, care review contributions. This documentation travels — to local authorities, to commissioning systems, to case management platforms operated by third parties. As AI tools are embedded more deeply in those case management platforms, the records that residential homes write are increasingly likely to become inputs into automated analyses that the home cannot see, that the young person does not know about, and that may produce outputs that influence decisions about that young person's future. The language used in residential records — which practitioners who have thought carefully about how we write about children will recognise as a significant professional and ethical question — is now also a question with algorithmic dimensions. A record that uses deficit language, that describes a young person primarily in terms of their difficulties rather than their capabilities, that reflects an incomplete or uncharitable account of a particular incident, does not merely shape the views of a social worker who reads it. It potentially shapes the outputs of any automated system that processes it. This is not an argument for sanitising or inaccurate documentation, but it is an argument for the kind of intentional, balanced, young-person-centred recording that good homes have been developing for its own sake — and which now also matters for reasons that extend beyond immediate professional relationships.

What residential homes can do about artificial intelligence in children's social care is not nothing, though it is not yet extensive. Homes should know whether the local authorities placing young people with them are using any form of algorithmic or predictive tool as part of their case management or risk assessment processes. They can ask — in placement planning meetings, in LAC review settings, in the conversations that registered managers have with commissioners — what automated tools are being used, what data sources they draw on, and what their outputs are used to inform. They can ensure that keyworkers are equipped to explain to young people, in accessible terms, that their records form part of a system that goes beyond any one social worker's desk, and that young people have rights in relation to their data that they can exercise, with support, if they choose to. They can advocate, through provider forums and the emerging Regional Care Cooperative governance structures, for procurement and transparency standards that require algorithmic tools used in children's social care to be audited for bias and to produce explainable outputs. And they can resist, in their own daily practice, the tendency — common to any environment in which algorithmic outputs become normalised — to treat a machine-generated recommendation as a fact rather than a hypothesis. The young people in residential homes arrived there through a sequence of decisions made by human beings operating in professional systems with their own assumptions, pressures, and blind spots. Adding automated systems to that landscape does not remove the professional and ethical responsibilities of the people in it. It makes those responsibilities harder to discharge and more important than ever to take seriously. A home that understands the AI landscape is not better placed to fight it — but it is better placed to keep its young people at the centre of decisions that too often get made about them, rather than with them.