A judge says the Home Office may have used a hallucinated AI document to reject an asylum claim, because the evidence it relied on doesn't seem to exist

[Phil7]

A senior judge has raised serious concerns that the Home Office relied on AI generated, hallucinated information to refuse an asylum claim from a Moroccan woman and her child, who fled the country after experiencing forced underage marriage and extreme violence including rape. The Home Office rejected her claim citing evidence from what it described as an independent and authoritative country policy information note confirming Morocco would be safe for her, but when the Guardian asked the Home Office to produce the actual document, it first sent a link to an archive where the document could not be found, then produced an entirely different, unpublished internal guidance document instead

The judge said the refusal letter bore hallmarks consistent with AI use, and that it may transpire the specific document reference was the result of an AI hallucination, calling that possibility, if confirmed, an extremely serious failing on the Home Office's part. He determined that relying on a document that does not exist was analogous to relying on bogus evidence, and classified the issue as a procedural irregularity. This case adds to a growing body of evidence around the Home Office's use of generative AI tools in asylum decisions specifically, an 84 page legal opinion published earlier this year by barristers at Cloisters Chambers and Doughty Street Chambers, commissioned by the Open Rights Group, concluded that aspects of that AI use are likely unlawful, particularly since applicants are never told AI tools were used in assessing their own case

The Home Office has confirmed it uses two generative AI tools in the asylum process, one that summarizes interview transcripts using ChatGPT, and another that searches country of origin policy information for caseworkers to reference. The government's own internal evaluation found that 9% of AI generated interview summaries were flawed enough to require removal entirely, and 5% of caseworkers using the policy search tool said they were not confident in its accuracy. More than 100,000 people were awaiting an asylum appeal decision in the UK as of the end of last year, and the Open Rights Group has called for an immediate ban on these tools, arguing there are better ways to address the Home Office's case backlog that don't risk this kind of unreliable, undisclosed AI involvement in decisions with such serious consequences for people's safety

Home Office used 'AI hallucinated' information to refuse asylum claim, judge suggestswww.theguardian.com

[Drift Vector]

The Home Office's own initial response, sending a broken archive link and then producing a completely different document, is almost more damning than the hallucination allegation itself, that's not a confident institution standing behind its evidence

[Molly17]

9% of AI generated interview summaries being flawed enough to need removal entirely is an alarming error rate for something being used in decisions this consequential for people's actual safety

[ProperWolf]

Applicants never being told AI was involved in assessing their own case is the part that should trouble anyone regardless of where they land on asylum policy broadly, that's a basic transparency and fairness issue

[NeonPilot]

Relying on a document that doesn't exist being classified as analogous to bogus evidence is exactly the right framing, whether the fabrication came from a person or an AI system doesn't change how serious relying on it actually is

[Electric Holly]

This case being one of potentially many given the scale of the asylum appeal backlog is the real worry, a single high profile case getting judicial scrutiny doesn't tell us how many similar errors are going undetected

[Tia_75]

The 84 page legal opinion finding likely unlawfulness months before this specific case surfaced shows this concern was already well documented and apparently not acted on with enough urgency

[HenryThierry]

Good on the judge for treating this seriously as a procedural irregularity rather than just an unfortunate administrative error, that framing matters for how future similar cases get handled

[QuantumLeap96]

This is exactly the kind of high stakes AI reliability failure that should worry anyone paying attention to how quickly AI tools are being deployed into consequential government decision making without adequate safeguards

[ForumGremlin93]

Whatever the eventual outcome for this specific case, the broader pattern here of AI hallucinations affecting real legal and immigration decisions is a serious governance problem that needs addressing well beyond just this one instance

[Cheugy]

The 5% of caseworkers expressing low confidence in the tool's accuracy suggests even the people using these systems day to day already sense something is off, that internal skepticism deserved more weight before wider rollout