How do you cross-examine an algorithm?

For years, there was a box on a back shelf of P’s house in Edmonton labelled “insurance.” (For reasons of privacy, we are using her initial only.) Her husband thought it contained paperwork, but P had filled it with a different kind of insurance: a physical record of smashed phones, broken eyeglasses, and photographs of the bruises his violent episodes had left on her body. 

“I had a circle of blue bruises around my mouth because he would cover my mouth and smother me,” she says.  

Photographs, videos, and audio recordings are highly persuasive to judges and juries. When crime occurs in private, with no witnesses, a court contest is a tussle where two stories compete to offer the most plausible explanation of the same facts. Photographs and audio recordings join seemingly unimpeachable objectivity with emotional impact. One study found that combining visual and oral testimony can increase information retention among jurors by 650 per cent. 

A crisis of trust

Criminal defence lawyer Emily Dixon says if a client shows her an exonerating photo or video, she isn’t expected to run analytic tests before submitting it into evidence. It’s reasonable to assume that a photo is real—for now. Yet the world is fast approaching a point where we can no longer believe our eyes or ears. The onset of artificial intelligence, in the justice system and beyond, is poised to overturn existing practices. 

Specialists can still spot the anomalies that distinguish AI-generated images from real photos. 

“But in a year, we won’t,” says digital forensics expert Simon Lavallée.

Maura R. Grossman is a lawyer who has long worked to promote the use of advanced technologies for legal tasks, such as document review. When it comes to the threat of deepfakes, however, she believes Canada’s evidence laws will require an overhaul. 

“Before, if I wanted to fake your signature, I had to have some talent,” she says. 

“In this day and age, you could make a deepfake of my voice in two minutes.” 

This means juries may increasingly be skeptical of all evidence, Grossman says.

For complainants like P, the erosion of this trust carries a high cost: left with only doubt, people may be tempted to rely on their instincts—indistinguishable from their desires, fears, and prejudices. 

In cases of domestic violence in particular, audio and visual evidence is a weighty counter to the social impulse to minimize. At work and with their friends, P says, her husband was funny and kind, but he was a very different person at home, “to the point that most people wouldn’t believe you.”

Recently, the Law Commission of Ontario (LCO), an independent body that pushes for law reform, concluded a nationwide study of what is shaping up to be a head-on collision between two epistemic systems: the criminal justice system, which rests on the elimination of reasonable doubt; and artificial intelligence, a factory for doubt dissemination. 

“The risk of impacts on people’s lives is catastrophic,” says Ryan Fritsch, the lawyer who headed the initiative. 

The project brought together police officers, defence attorneys, prosecutors, judges, and human rights advocates, with the aim of formulating a set of recommendations to deliver to the Ontario government later this year. 

Eroding transparency

The black mirror of deepfake evidence is not the only challenge the advent of artificial intelligence poses to the criminal justice system. Whether through the adoption of advanced analytics for risk prediction, the rise of predictive policing, or the use of large language models to summarize depositions or draft decisions, Canadian courts are about to grapple with a form of intelligence with mysterious workings that chip away at the law’s foundation of transparency. 

Internationally, Canadian lawmakers are already far behind in developing governance for the use of artificial intelligence within the justice system. As a result, there are few guardrails in place. 

Canada also finds itself faced with clashing examples. While Europe has taken a skeptical stance, sometimes outright banning the use of certain technologies, the United States has leaned into decision-making driven by complex algorithms and big data. 

Canada’s legal rules and procedures are designed for a pre-AI era. How will the public retain confidence in a system that rests on the painstaking articulation of reasoned logic as more and more of what happens in our courtrooms starts in a black box?

Attorney Rupert Ross used to work as a fishing guide, and in his book Dancing with a Ghost, he wrote about learning to predict lake conditions before leaving the shore. Standing at the edge of the dock, he would layer impressions of wind pattern, cloud cover, and temperature over his mental images of different spots on the water. It was as if each image were “a transparency of sorts, with all the variables sketched opaquely on its surface. Then similar images of past days at the same spot are slid under it.” If the feeling of the day contained a Proustian twinge of a previous day when he had found pickerel at a certain cove, he brought his angler clients to those waters.

Ross’s description reads like a paean to the subtleties of embodied memory, one of the quintessential characteristics that separates humans from machines. To the empirically minded, however, this is just another way of saying that Ross made predictions based on comprehensive historical data. And aren’t machines simply better at this? 

That’s why, P tells me, she’s excited about the prospect of criminal risk assessments being performed by artificial intelligence. It seemed to her that justice system workers had been overly swayed by her husband’s charm. 

“These people are trying to predict outcomes, going off their imagination, their feelings, their gut instinct,” she says. 

“And AI takes all of that out.”  

Since the 1990s, statistical risk assessments have been conducted within the prison systems of many jurisdictions in Canada and elsewhere to sort offenders by the level of security needed, and as evidence in parole board hearings. By correlating today’s data with outcomes from past cases—did prisoners who answered questions similarly go on to commit further crimes?—prisons had a data-driven system for deciding who was eligible for laxer security or early parole. The accuracy of computerized algorithmic models quickly rivalled the careful assessments of expert psychologists. 

But, according to correctional lawyer Simon Borys, prisons and parole boards don’t have to explain their decisions in the same granular detail that courts do. It was in the 2010s, when algorithmic risk assessment tools were incorporated into sentencing in the United States, that the higher standard for legal reasoning brought to light concerning qualities of these tools. 

In 2016, the American media outlet ProPublica published a landmark study that found racial bias in the Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), a widely used algorithmic risk-assessment tool. Studies of similar tools used in Canada, such as the Level of Service Inventory (LSI), have found comparable results. 

For a kaleidoscope of reasons—including the fact models rely on historical arrest data, and racialized communities are more heavily policed than white neighbourhoods—algorithmic predictions tend to overstate the likelihood of racialized people committing future crimes and understate these risks for white offenders. 

If a bigger mirror only shows more of a distorted reality, artificial intelligence powered by big data exaggerates the problems of actuarial risk prediction. Because we live in an unfair world in which people who are poor, mentally ill, or racialized are disproportionately arrested, simply belonging to one of these categories can make you look more likely to commit crimes. (Equivant, the company that currently owns COMPAS, disputes that the tool is racially biased, and some subsequent studies undertaken by independent bodies also found ProPublica’s results overblown.)

Anyone accused of a crime in Canada has a right to know what evidence is being used against them. They have a right to test that evidence, to hear the reasons why a judge or jury decides to deprive them of their liberty, and to appeal their decision if these reasons are insufficient. In one of the papers commissioned by the LCO, Gideon Christian, the University of Calgary’s research chair of AI and law, and lawyer Armando D’Andrea propose a novel problem: how do you cross-examine an algorithm? If a risk assessment is to be classified as expert opinion, usually the expert proffering it would be expected to make themselves available to the court. In the case of an AI tool, however, even if a software engineer who had designed the program could be persuaded to testify, the nature of machine learning would limit their ability to vouch for what the program was actually doing. 

“You should be able to explain why a person got the score and what the score means,” says Tim Brennan, who originally designed COMPAS. 

“An AI technique like random forests”—a statistical sampling method that involves hundreds of decision trees branching into further decision trees—“is a great way to do prediction. But you cannot, for your life, explain it to a judge.” 

Even if courts banned lawyers’ use of AI, they would be unable to prevent reports or assessments used in other spheres—by social workers, for example, or addiction counsellors—from being compiled using the technology. But the use of AI could mean these reports, currently a routine part of court proceedings, would no longer pass tests that make them admissible evidence. 

Fritsch, the lawyer for the LCO, says AI is weakest in an area the justice system prizes highly: the provision of clear documentation and reasoning. AI systems occlude information or processes, which may mean they do not provide the full disclosure the law requires. For defendants hoping to appeal their sentences, this missing information could be key to overturning decisions that rob them of their liberty or even their lives. The inner workings of AI software are proprietary information owned by private companies. In the United States, courts have shown a disturbing tendency to side with corporations that argue they can’t be compelled to disclose trade secrets—even when their products are being used to send people to death row. 

AI aid for self-repped litigants

In a 2025 case at the British Columbia Civil Resolution Tribunal, Vice-Chair Eric Regehr posited in his decision that both parties must have used AI to write their submissions. His reasoning was that “there is no way a human being” could have made some of the errors the submissions contained. Furthermore, he did not consider himself obliged to respond to every one of the specious arguments the software had thrown his way. 

“I accept that artificial intelligence can be a useful tool to help people find the right language to present their arguments, if used properly,” Regehr wrote. 

“However, people who blindly use artificial intelligence often end up bombarding the CRT with endless legal arguments.” 

The promise of AI for self-representation is tantalizing. To state the obvious: lawyers are expensive. Rates for a criminal defence attorney can reach $800 an hour, and many people who apply for legal aid are denied. Accordingly, the number of people representing themselves in court is growing. The increase has been most marked in areas of civil law, such as divorce and custody cases, but some Canadians accused of a criminal offence find themselves with no option but to mount their own defence. 

Depending on the nature of the case, people representing themselves in court using Microsoft CoPilot or ChatGPT may face opposing counsel using even more sophisticated tools. An inventory compiled in June 2025 found 638 generative AI tools available in the “legaltech” field, designed for everything from trawling legal databases to reviewing contracts, searching for patents, and drafting documents. Companies are developing specialized artificial intelligence tools for big firms that can pay for the most advanced tech. So for a self-represented litigant, ChatGPT may be better than nothing, but it’s the kiddie-pool version of the paid products available to a well-resourced lawyer on the opposing side. 

Within Canada, some non-profits are attempting to level the playing field. Beagle+ is a free legal chatbot trained on British Columbia law, powered by ChatGPT, and built by the People’s Law School, which provides free, plain-language legal education and information to the public. At Queen’s University, the Conflict Analytics Lab has built OpenJustice, an open-access AI platform drawing from Canadian, US, Swiss, and French law. Samuel Dahan, the law professor who founded the project, says the idea was originally to build a Canadian large language model from scratch. 

“The sad reality of universities right now is that there’s no university in the world that has the resources to build a language model,” he says. 

At a Legal AI Hackathon co-hosted with Stanford University in Toronto last February, the Conflict Analytics Lab saw teams devise programs that use OpenJustice to help pro bono organizations streamline client intake and to identify trends in jurisprudence that might help direct a class-action lawsuit.

While artificial intelligence seems poised to widen rather than narrow the divide in access to justice, Grossman suggests a key way the technology could serve the average person in the civil context. When a pair of earrings she bought online arrived, one post was bent. She contacted the seller, sent a photo, and asked if a replacement earring could be sent. 

“He basically responded, if you weren’t a moron, you would be able to fix this.” 

So Grossman made her case in the dialogue box of the website’s online dispute resolution system. It scanned her photo, looked at the value of the item (the earrings had cost her about $12, including shipping), tabulated comparable transactions, and made a determination: it awarded her $2.38. 

Instead of a customer dissatisfied with an earring, Grossman says to picture her as a tenant whose landlord is illegally withholding her security deposit. 

“I’m a janitor, and if I take a day off from work, that means my kids don’t eat, and I don’t know anything about the court system, and maybe I don’t speak English well, and maybe I’m not here legitimately,” she says. 

The online adjudication system might not be perfect; it could award her less than the full amount. But the case might be over in a week, with no requirement to miss work or bring attention to her immigration status. 

“It’s not perfect,” she acknowledges, “but rough justice is better than no justice.”

Simultaneous narrowing and widening

So far, Canada’s attempts to pass comprehensive legislation on artificial intelligence have failed. AIDA, the Artificial Intelligence and Data Act, died on the table when Parliament was prorogued in January of 2025. 

It was also controversial: Amnesty International objected that the legislation overlooked potential harms to vulnerable groups, and the Assembly of First Nations threatened to sue over the lack of consultation. Indigenous groups, such as the Quebec-Labrador regional body of the AFN Commission, have proactively published guides to regulating and developing artificial intelligence with respect for Indigenous data sovereignty, but at present these are only statements of moral principle rather than enforceable law. With the appointment of Evan Solomon as the inaugural minister of artificial intelligence and digital innovation last year, it’s certain that another legislative effort is coming soon.

The Federal Court has stated that it will not use AI or automated decision-making tools without first engaging in public consultation. When it comes to the criminal system, “the public needs to be giving social license to these kinds of things,” Fritsch says.

But what would meaningful consultation on AI look like when experts can’t even agree on what artificial intelligence is? The LCO materials discuss COMPAS as an artificially intelligent tool, but Equivant denies that COMPAS makes use of AI. “It does not ‘learn’ or change over time,” the company says on its website. Brennan says a tool similar to COMPAS, designed for female offenders, has incorporated AI for the past eight years. He later amended this statement, saying that the tool uses a “machine-learning-derived classifier” but is not itself an AI tool. 

The category of artificial intelligence seems to be simultaneously narrowing and widening. 

“When it’s like a spam filter, and we get used to it, we don’t call it AI; we just call it software,” Grossman says. 

“When it’s new and magical, and we don’t understand it, we call it AI.” 

This past summer, Karine Gentelet, a sociologist at the Université du Québec en Outaouais, piloted an alternative form of consultation on AI. 

“It’s not a consultation about AI,” she says. 

“It’s a consultation about how people want to be consulted.” 

Rather than the large-scale forums that governments tend to convene, often dominated by industry, these sessions were small and held at a Montreal community centre. Thirty to forty participants per session proposed definitions of basic concepts like “trust” and “transparency.” Gentelet’s team put up a poster showing an Excel spreadsheet of 168 artificial intelligence programs in development or in use by the province’s agencies, available on the government’s website. Descriptions were brief and often vague: “Service to citizens.” Clearly, this was someone’s idea of transparency, but it didn’t match what workshop participants had written down. They wanted to know exactly how information would be collected and used, and how the government planned to guarantee ethical guardrails.

When asked what she would tell a government commission that asked her what AI should and shouldn’t do, P didn’t need to think twice. 

“I would ask if there’s a way for a lot of those computer engineering students to come up with a system to take some of the burden off shelter workers,” she says. 

Harried workers were constantly on the phone, calling around to different facilities to find spots for women in need of a safe place to stay. Couldn’t it be more like the aggregators that find airline tickets? You could enter your location and any special needs (not all shelters are safe places for transgender women, P noted), and an algorithm could handle the rest. 

Fundamentally, citizens want the criminal justice system to deliver two conflicting outcomes: to keep them safe, and to respect individual liberties. Dial public safety all the way to 10, and we are all pre-emptively kept in solitary confinement to prevent us from causing each other harm. Dial liberty to the max, and there is no protection for the weak. Fairness is one name for the balance between these concerns. A cornerstone of our legal tradition is the 1769 doctrine known as Blackstone’s ratio: “It is better that 10 guilty persons escape than that one innocent suffer.” This is a moral rather than a mathematical maxim. 

But deep within the coded instructions that govern algorithmic tools, fairness does find mathematical expression—or rather, expressions. The balance in risk predictions lies between the likelihood of a formula returning false positives or false negatives: either imprisoning people unnecessarily or returning people to the community who go on to hurt others. 

When engineers attempt to tinker with the variables that produce racist or sexist results, they apply different formulas that are fair by different standards. “Fairness through unawareness” means using training data for machine learning that excludes variables like race or gender. “Counterfactual fairness” means a prediction would remain constant even if the demographic group to which an individual belongs were modified. At last count, there are about 20 different definitions of fairness circulating in the risk assessment field, offering different trade-offs that are statistically impossible to reconcile. 

In a 2024 paper in the journal Artificial Intelligence and Law, German researchers warned that by designing models encoded with particular trade-offs, software engineers and companies are usurping what should be the choices of citizens. The research team devised a code that shifted the balance struck in COMPAS between victims’ and offenders’ rights. The researchers acknowledged that legislators might not agree that the new balance reflected the state’s vision of fairness, but at least the choices being made were now out in the open, where citizens could debate them.   

Most people, as Gentelet anticipated, don’t have the technical knowledge to speak with sophistication about artificial intelligence. The participants in her workshop were not there to parse how random forests differ from bootstrap classification methods, or to wax eloquent about oob (out-of-bag) samples. But they knew what scared them, including the technology’s environmental costs. The advent of artificial intelligence, they told Gentelet, was increasingly presented as a kind of destiny: just one more ineluctable step on a path humanity must follow. The majority of people in the room had one pressing question: Was it an option to say no?