The challenge of AI in the courtroom

It’s finally happening. After proving it can write college essays and concoct ‘art’ at a human level of mediocrity, artificial intelligence is coming for the court system.

Back in January, Global News reported that two lawyers in British Columbia had spotted an AI-generated fake caselaw in a civil case before the B.C. Supreme Court. The lawyers said counsel for one of the parties in a high-net-worth family matter used ChatGPT to prepare legal briefs on her client’s behalf.

ChatGPT is generative AI, built to create original content. It also has a well-known tendency to “hallucinate” — to make stuff up — in response to user prompts. The lawyer who presented the phony case law reportedly apologized and left the court in tears. In her case, the error was caught in time. But what if no one had noticed?

“Generative AI systems should be used very carefully and certainly not as a research tool,” says Charles Gluckstein of Gluckstein Lawyers in Toronto. His firm uses generative AI tools in its practice, but with guardrails to keep confidential data secure.

He says lawyers have a duty to their clients to keep their information confidential and to the court to improve the administration of justice. Anyone who fails to review the tools they’re using risks misrepresenting clients and misleading the court, with dire consequences.

“The alarming thing is what might happen if false caselaw gets introduced and is never spotted as fake. It could build up over time and soil the jurisprudence the entire justice system depends on,” Gluckstein says.

Sadly, bogus caselaw isn’t the only problem. AI also presents unique threats to evidence integrity — threats the justice system may not be equipped to handle. Dealing with those threats is going to require diligence — and money.Maura Grossman is an adjunct professor at Osgoode Hall Law School and a research professor at the University of Waterloo’s computer science faculty. She’s represented Fortune 500 companies and has been studying the impact of AI on evidence law for years.

At a basic level, she says, AI-generated evidence is no different from other tech-based exhibits, like digital video — as long as everyone involved agrees on what they’re looking at.

“If the evidence in question is known by both sides to be the product of an AI system, it’s a much easier problem,” she says, as the courts have rules to deal with complex scientific or technical evidence.

“The bigger problem is when the provenance of the proposed evidence is in question. This has a qualitatively different impact because of the unique power of audiovisual evidence, especially in jury trials.”

AI-driven “deepfakes” can produce false audiovisual evidence — which can have a particularly strong impact on juries. In a 2023 paper, U.S.-based legal academic Rebecca A. Delfino wrote that studies have shown that jurors who hear oral testimony along with video testimony are 650 per cent more likely to retain the information.

“Indeed, studies have demonstrated that video evidence powerfully affects human memory and perception of reality,” she noted.

Grossman says juries are extremely vulnerable to manipulation by AI-generated material.

“Judges aren’t immune either,” she adds.

Right now, the only way to single out such deepfakes is through a combination of guesswork and expertise; one side questions the veracity of a piece of evidence and offers evidence of fakery, and the parties bring in one or more experts to go over the evidence in detail.

“The opponent has to present contradictory evidence and the court has to strike a balance between the potential value of the impugned material and the potential for prejudice,” says Grossman. “It’s about a preponderance of probabilities and balancing risk.”

That’s easy enough to do if, for example, one party in a custody dispute can prove that he was unconscious in a hospital bed when a recording of him allegedly threatening his ex-wife was made. Technology offers only a stopgap solution.

“The hard truth is that even if we get very good at detecting AI fakes very quickly, it’s going to get challenging again just as quickly,” said Abdi Aidid of the University of Toronto, who researches the impact of new technology on the law.

“The problem is that our detection technology is learning at more or less the same pace as the technology we use to create deepfakes. So we’re unlikely to acquire a technological advantage anytime soon.”

Aidid says lawyers aren’t expected to certify the provenance of evidence their clients give them. That means courts will have to depend on experts in digital manipulation to establish the veracity of exhibits and sort the fake evidence from the genuine. That’s going to drive up trial costs for all sides.

“We’ll have to place some new onus on litigants themselves, perhaps by greatly increasing the penalties for presenting false evidence,” he says.

The ideal approach would be some standard form for identifying the provenance of a piece of digital evidence — “watermarking” technology.

“If that approach can be made to work, the only issue would be with material from devices that are not equipped with the watermark,” says Grossman.

“But that seems to be the most promising solution for now.”

In the meantime, courts are going to have to suffer through a lot of bad-faith claims about what’s authentic and what isn’t.

In 2018, law professors Bobby Chesney and Danielle Citron coined the term “liar’s dividend” to describe defendants arguing in court that the evidence against them is AI-generated. Two defendants on trial for the January 6 riot in Washington D.C. claimed that video showing them at the Capitol could have been concocted by AI. Elon Musk tried the same gambit in a civil trial.

“Deepfakes open up new avenues for gamesmanship and can drive up the duration and cost of trials,” says Aidid.

They also could have a far more corrosive effect on jury trials. Grossman says once jurors get in the habit of questioning evidence merely because it could be generated by AI, they might start substituting their own biases and suspicions for facts.

“Jurors could become overly cynical about evidence and give weight to things that are not in evidence, such as whether the accused looks guilty,” she says.

“That worries me a great deal.”