A Kenyan Supreme Court judge has warned judges and lawyers about using artificial intelligence tools like ChatGPT in court submissions following a case in which fabricated legal citations were generated and presented in court.
Justice Isaac Lenaola, also the chairman of the Judiciary’s Committee on Innovation, described an instance where lawyers had submitted arguments with AI’s help, only for the presiding judge to discover that every legal authority cited in the submission was fictitious.
While generative AI tools like ChatGPT promise efficiency and ease in drafting legal documents, their inaccuracy in citing past cases and legal precedents poses serious risks to judicial integrity. Lenaola’s warning calls for regulatory clarity as Kenya weighs in on the legal and ethical challenges of integrating artificial intelligence into courtroom practice.
“I was reading a case where a judge was horrified because of what had happened to him,” Lenaola said at the Judiciary Digital Transformation Conference on Friday. “Lawyers filed submissions using AI. The language was beautiful, but when the judge cross-referenced the authorities, all of them were created by artificial intelligence.”
Lenaola urged court officials and lawyers to refrain from using AI tools until clear guidelines are established. He warned of the reputational risks to the bench if judges were to incorporate such material into their rulings unknowingly.
While AI tools can produce fluent and convincing prose, there are documented cases of false information, especially when generating sources or citing examples.
“Please, judges, lawyers who are here. Until we give you guidelines, please avoid AI for now,” he said. “Can you imagine the embarrassment if a judge delivers a judgment, then the AI tools tell him, this was not created by you, this is AI, and all the authorities are fake?”
Kenya is not the only country grappling with legal practitioners turning to AI in the courtroom. Courts in the US, Canada, and the UK have also encountered instances where lawyers used ChatGPT to draft submissions containing fabricated case law.
In 2023, a New York federal judge fined lawyers $5,000 after they admitted using ChatGPT to create a legal brief that included non-existent court opinions and fake quotes. In 2024, another New York lawyer was sanctioned for citing a non-existent case generated by AI.
In Kenya, the Judiciary has been exploring integrating technology, including digitising court records, expanding e-filing systems, and in virtual court sessions. However, Lenaola’s comments suggest a more cautious approach to generative AI.
