The High Court in England and Wales has recently examined the use of generative artificial intelligence tools by lawyers in litigation. Two cases were referred to the High Court after it became apparent that false citations appeared in legal documents that were being relied upon by lawyers. Intentionally submitting false material in court proceedings, with the knowledge that the information is false, will amount to a contempt of court.[1] This could attract a prison sentence of up to two years. The harshness of the punishment is for good reason, the deliberate interference with the administration of justice is a serious act that undermines public confidence in the legal process.
In the first case, Ayinde, counsel for the claimant submitted to the court five cases that did not exist. The citation number for one of the cases did exist but belonged to a different case. The remaining cases did not exist at all. The defendant’s solicitor contacted the claimant’s solicitor to inform him that he could not find the cases that were set out in the pleadings and requested copies of them. The claimant’s lawyers responded that there had been technical, cosmetic errors that could be corrected on the record. Counsel later explained in a court hearing for that case that she kept a paper and digital list of cases with their ratios and had “dragged and dropped” the reference for the first case into her pleadings.[2] In a witness statement to the High Court, she stated that the judge had misunderstood her, and that she did do all her research electronically but denied using artificial intelligence tools.[3]
The High Court noted that the descriptions of the cases in her submissions used Americanised spellings, which was at odds with the English spelling of those words in her correspondence.[4] It concluded that she had either deliberately included fake citations or had used generative artificial intelligence tools to produce her list of cases and draft submissions.[5] The court decided not to find her in contempt of court because of her very junior status (she had only recently qualified) and unresolved questions about the degree of supervision and training she had received. She was however referred to the regulator for investigation.
In the second case, Al-Haroun, the judge raised concerns about the authorities cited in witness statements and correspondence. Some were completely fictitious. One fake judgment was even attributed to the judge herself. Others, though the cases existed, either did not contain the passages quoted from them or did not support the legal propositions for which they were cited. The lawyers confessed that they had used publicly available artificial intelligence tools, legal search engines and online sources. They acknowledged their failure to check the accuracy of those results, but denied the false citations were designed to intentionally mislead the court. This was accepted by the High Court, who supported their self-referral to the regulator, without a finding of contempt.
The President of the King’s Bench Division, Dame Victoria Sharp, warned in the judgment that “freely available generative artificial intelligence tools, trained on a large language model such as ChatGPT are not capable of conducting reliable legal research”.[6] She highlighted the necessity for lawyers to check the accuracy of research undertaken if using AI tools, by checking them with authoritative sources.[7] While some legal guidance is available, she urged those in positions of leadership in the legal profession, and regulators, to implement practical and effective measures to ensure compliance with professional and ethical duties when using artificial intelligence.
Other jurisdictions are also facing similar problems. In the United States, it is occurring with increasing frequency. One of the first cases arose in 2023, when a lawyer was found to have used ChatGPT to submit materials that contained fictitious citations. In Mata v Avianca Inc[8] the lawyer filed documents to the United States District Court for the Southern District of New York in which seven cases cited did not exist. When asked for a copy of the cases, the lawyer used ChatGPT again to summarise the cases and sent the summaries instead of the full cases. The summaries were described as “gibberish” by the Judge. The lawyer had even asked ChatGPT if one of the cases cited by it was “real”, to which ChatGPT responded that it did exist, and was available on Westlaw and LexisNexis. This was of course untrue, as was later discovered.
Unfortunately, the wide press coverage of Mata did not provide enough of a cautionary tale for the legal profession. As recently as May 2025, a large US law firm was forced to apologise to a judge for filing court documents that contained a fake citation and incorrect attribution for an expert report, that was generated by an AI Chatbot.[9] There are many, many examples of cases where lawyers have used AI generated tools that create pleadings containing AI hallucinations in the form of fake citations, wrongly attributed legal principles, or completely fictious cases.[10] Legal drafting that is created by the combination of AI tools on official legal platforms, the premium ChatGPT service and Google Scholar can still lead to a mish mash of partly correct, partly incorrect citations and attribution of legal principles.[11]
The trend is not limited to lawyers. In Kohls v Ellison[12] an expert submitted evidence about ‘deepfakes’, and with what may appear ironic, used generative artificial intelligence to draft a report that cited non-existent academic articles. The judge in that case described how the “credentialed expert on the dangers of AI and misinformation [had] fallen victim to the siren call of relying too heavily on AI”.[13]
Increasingly, people representing themselves in court without lawyers, are relying on artificial intelligence tools to make their legal arguments. For example, the cases of SW Harber v Commissions for His Majesty’s Revenue and Customs[14]; Olsen v Finansiel Stabilitet A/S[15]; Zzaman v Commissioners for His Majesty’s Revenue and Customs[16] in England and Wales. The problem is that the use of ChatGPT and other AI tools to draft legal arguments can create incoherent pleadings that are insufficient for the legal process, slow down the litigation or lead to unsustainable legal grounds being relied upon.[17] This is an international phenomenon, occurring in courts in multiple jurisdictions. The cases of Saxena v Martinez Hernandez[18];Geismayr v The Owners, Strata Plan[19]; and GNX v Children’s Guardian[20] in the United States, Canada, and Australia respectively, all involved individuals using AI tools in their legal cases, without success.
Attempts to use AI outputs as evidence is also creeping into the courts and has been rightly rejected. In Rollo v Marstons Trading Ltd[21] the English employment tribunal rejected the use of a ‘conversation’ with ChatGPT as expert evidence. Mr Rollo attempted to rely on the ChatGPT responses to undermine the other party’s evidence about the difficulty of data extraction. In The Hague, a lawyer who used ChatGPT to generate a list of ‘economically comparable’ vehicles in a car registration tax case, failed to persuade the Court of Appeal to accept his AI-generated evidence.[22]
In England and Wales, the Bar Council issued new guidance last year on the factors that should be considered if using ChatGPT or any other AI software based on large language models.[23] This guidance warns that the concepts of truth and accuracy do not feature in the architecture of these models, which are designed to create sequential text but not to analyse the content of data. Further, it points out that hallucinations are not the only risk that using AI tools present. There may also be bias in the training data, lack of explainability, the accidental disclosure of confidential or privileged information, breach of data protection legislation and intellectual property infringement. The Law Society also has guidance on the use of generative AI tools, which includes a checklist for lawyers to use.[24] It is important to note that recent guidance issued by the judiciary recommends that legal research for new information should not be conducted using an AI tool.[25]
Aside from wasting the court’s time there are other serious consequences that can arise from the improper use of AI tools, or where those tools generate legal nonsense. As pointed out by the judge in the case Mata, it deprives the party of authentic precedents and arguments, it could harm the reputation of judges whose names are falsely attributed to bogus judgments, it tarnishes public opinion about the legal profession and it could lead to the future challenge of a judicial ruling based on doubts of authenticity.[26] A stark warning was given by Sharp P. in Ayinde that the court’s decision not to initiate contempt proceedings in that case should not be interpreted as setting a precedent: “lawyers who do not comply with their professional obligations in this respect risk severe sanction.”[27]
This article first appeared in the September 2025 issue of Business Law International (Vol 26, No 3), and is reproduced by kind permission of the International Bar Association, London, UK. © International Bar Association.
If you are interested in any further information or advice, please contact my clerks on 0203 179 2023 or clerks@millenniumchambers.com
This blog is maintained for information purposes only. It is not intended to be a source of legal advice and must not be relied upon as such. Copyright © 2016-2026 Melissa Stock. All rights reserved. The author does not give permission for extraction of data from this site.
[1] R v Weisz ex p Hector Macdonald Ltd [1951] 2 KB 611.
[2] At paragraph 45 of the judgment.
[3] At paragraphs 52-53 of the judgment.
[4] At paragraph 38 of the judgment.
[5] At paragraph 68 of the judgment.
[6] At paragraph 6 of the judgment.
[7] At paragraph 7 of the judgment.
[8] Case No. 22-cv-1461 (PKC), 2o23 WL 4114965 (SDNY 22 June 2023).
[9] Concord Music Group Inc v Anthropic PBC 5:24-cv-03811-EKL (SVK) 23 May 2025.
[10] This is not an exhaustive list, but see as examples the U.S. cases: Ex parte Lee 673 SW 3d 755 (Tex App Waco 19 July 2023), Lacey v State Farm General Insurance Co CV 24-5205 FMO (MAAx), 6 May 2025; United States v Hayes (E.D. Cal. Jan 17 2025); United States v Cohen 724 F Supp 3d 251 (SDNY 2024). In other jurisdictions too: Valu v Minister for Immigration and Multicultural Affairs (No 2) [2025] FedCFamC2G 95 (Australia); Wikeley v Kea Investments Ltd [2024] NZCA 609 (New Zealand); Zhang v Chen [2024] BCSC 285, Ko v Li [2025] ONSC 2766 (Canada); Rodney Chagas v Fabricio Petinelli Vieria Coutinho 0001082-77.2024.8.16.0075 Ap (Brazil); Mavundla v MEC 7940/2024P (South Africa).
[11] See the case USA v Burke 8:24-cr-00068-KKM-TGW (15 May 2025) and Law360’s article ‘Attorneys for Alleged Fox Hacker ‘Deeply Regret’ Fake AI Citations’ 20 May 2025: https://www.law360.com/pulse/articles/2342825/attys-for-alleged-fox-hacker-deeply-regret-fake-ai-citations.
[12] No 24-cv-3754 LMP/DLM (D Minn 10 January 2025).
[13] See page 8 of that judgment.
[14] [2023] UKFTT 1007 (TC).
[15] [2025] EWHC 42 (KB).
[16] [2025] UKFTT 00539 (TC).
[17] See Crypto Open Patent Alliance v Wright [2025] EWHC 1139 (Ch) at paragraph 36, and Reddan v An Bord Pleanala v Trustees of Nenagh Golf Club [2025] IEHC 172.
[18] D.Nev April 23, 2025.
[19] KAS 1970 [2025] BCCRT 217.
[20] [2025] NSWCATAD 117.
[21] ET 1600833 /2022.
[22] X BV in Z v Tax Inspector ECLI:NL:GHDHA:2024:771 BK-23/570.
[23] Available here: https://www.barcouncilethics.co.uk/wp-content/uploads/2024/01/Considerations-when-using-ChatGPT-and-Generative-AI-Software-based-on-large-language-models-January-2024.pdf
[24] Available here: https://www.lawsociety.org.uk/topics/ai-and-lawtech/generative-ai-the-essentials#h4-heading3-3
[25] Civil Procedure Rules 1PG.11 and found here: https://www.judiciary.uk/wp-content/uploads/2023/12/AI-Judicial-Guidance.pdf .
[26] See pages 1-2 of U.S. District Judge Castel’s judgment in Mata.
[27] Paragraph 69 of the judgment.
privacy law barrister 