Why Canada needs AI regulation in criminal justice

Log in to listen to this article

After hearing from more than 1400 people across Ontario, the Law Commission of Ontario has released its final report calling for policy and legislative reform in AI and criminal law.

Among its 29 recommendations are calls to create a provincial criminal AI framework, new legislation specifically for AI and criminal law, a dedicated provincial authority to oversee AI systems, and a public AI registry.

The project, which began in 2025 with a series of issue papers and consultations, is focused on how AI is changing the criminal justice sector. As AI continues to expand, so does its use in criminal justice. The LCO recommends using a Canadian lens in evaluating and using the technology.

“There’s a very strong sense that we need to govern AI,” says Ryan Fritsch, LCO legal counsel and project lead of the AI in Criminal Justice project.

“There are self-governance models, but we need more comprehensive, coherent policies.”

The report looks at the criminal justice systems south of the border and in the United Kingdom, which are further ahead than Canada in adopting AI.

Criminal justice technology ranges from facial recognition software, surveillance drones, social media monitoring, bail and sentencing algorithms, AI-assisted police report drafting, and automated license plate readers for tracking vehicles.

In Canada, Crown prosecutors are allowed to use pre-approved AI prompts in Microsoft Copilot for drafting legal memos and conducting research. Courts have also been using AI to assist in writing judgments, conducting legal research, drafting documents, and generating transcripts.

In the United States, the most common AI tool in law enforcement is automated license plate readers from Flock Safety. In addition to the company’s mass video surveillance and gunfire locator systems, the technology scans license plates to find stolen vehicles and track them in real time. About 140,000 automated license plate readers are currently in use there, leading to significant pushback.

Earlier this year, Florida Gov. Ron DeSantis said the use of the cameras is out of control and that the public is right to be concerned about them. More than 50 cities across the U.S. cancelled their Flock licenses this year. The Los Angeles Police Department was among the forces that cancelled contracts with the company amid privacy concerns. However, several advocacy groups have claimed the department has shared Flock data with United States Immigration and Customs Enforcement.

The U.S. government has made it clear that it won’t adopt comprehensive AI reform after President Trump’s 2025 executive order called for a “minimally burdensome” national AI policy. That hasn’t stopped individual states from moving ahead, as 38 states adopted new legislation last year. Some were aimed at regulating facial recognition software while others clarified who is responsible for AI-generated content.

The U.K. has a long history of using technology in criminal law, especially for surveillance. In 2024, more than 4.6 million faces were scanned by live facial recognition technology in England and Wales. Amid calls for more regulatory oversight, a 2025 government report recommended data audits of AI tools, greater transparency in AI use, new regulations, and more human oversight.

Regulation means better innovation

As in Canada, local police services and agencies in the U.K. have been allowed to decide which AI tools to use. Marion Oswald, a law professor at Northumbria University, has been examining these issues as part of the PROBabLE Futures project, an initiative looking into AI and criminal justice in the U.K. She says Canada should avoid their model and create a central legal framework.

“Some people within police, government, and the commercial sector argue that more regulation would be anti-innovation and would hold the police back from using technology,” she says.

“There is now increased realization and acceptance that regulation actually makes innovation better, and ensures that police are using technology in a trustworthy way.”

Without human oversight, mistakes will be made. In West Midlands, UK, police used Microsoft Copilot to do a security check on an upcoming soccer match and decided that Maccabi Tel Aviv soccer fans should be banned. The problem was Copilot’s security assessment was an AI hallucination. As a result, West Midlands police chief Craig Guildford resigned.

Even as more people grasp the risks of AI, there’s still deference given to these systems. Although several companies are developing AI credibility tools to detect whether people are lying through voice or facial expressions, Oswald warns against giving credibility to systems that haven’t been proven effective.

“These tools have been promoted extensively in law enforcement despite their lack of scientific validity or independent evaluation,” she says.

“Like a polygraph, they often slip under the radar but cause considerable risks for fairness and miscarriage of justice.”

Having interconnected AI tools can make mistakes worse. This year, researchers from the University of Glasgow and Northumbria issued a report, The Need for Assessment of the Cascade Effects from Sequencing AI Systems in Policing, in which participants described using connected AI systems. One respondent described how a system changed a witness statement from “grabbed a bottle of whiskey and shoved it in their coat pocket,” to “the bottle of whiskey was picked up from the shelf and placed in the basket” — a mistake that changes the statement’s legal significance. Oswald says more work is needed to prevent these errors.

“Chaining of AI tools and systems is something that hasn’t yet been given sufficient thought,” she says.

“An error or uncertainty in an AI output creates serious risks for the accuracy of evidence and legal outcomes.”

Yet connected systems are among the most effective ways software companies sell products and build brand loyalty. Typically, vendors will offer discounts on a bundle of services rather than on a single software program and will require customers to purchase quickly.

That means police services may not go through a lengthy procurement process that includes risk assessments, determining how data would be stored and used, reviewing vendors’ training data, and allowing for community consultation.

“Many police services in Canada feel powerless to negotiate with vendors,” says Fritsch.

“If we had legislation around standard terms of use for all contracts, that would protect digital sovereignty.”

He says this is why having a Canadian approach to AI governance is important. The LCO report emphasizes that Canadians have protected Charter rights in criminal law. That means some technology, such as real-time facial recognition software, should never be used here, as it would infringe on privacy law.

There are also underlying issues of systemic discrimination and the possibility of AI tech disproportionately affecting Black, Indigenous and other radicalized minorities. Bail and sentencing software has been shown in the past to affect radicalized minorities disproportionately.

Transparency and new evidentiary rules

Another growing trend is police services embracing predictive policing, using AI tools that draw on other databases to predict future crime. In 2016, the Calgary Police Service began using Palantir Gotham, a predictive policing tool, but cancelled the contract a few years later. The Vancouver Police Department developed its own predictive policing software, GeoDASH, to track and predict criminal activity. The Saskatoon Police Service and the University of Saskatchewan developed a pilot predictive policing tool in 2019 to identify potential victims who could become missing persons.

The LCO report recommends banning any tools that make predictions about whether someone will commit a crime.

“We have to create a new system around these tools,” says Fritsch.

“When breathalyzer tests were created, we had to implement new rules about when to use them, how to use them efficiently, create new evidentiary rules, and create rules to ensure procedural fairness. We can do the same for AI.”

The federal government’s recent AI strategy doesn’t mention criminal law or any new legislation related to AI and criminal justice. To fill the gap, several law enforcement organizations have adopted AI guidelines. The Canadian Association of Chiefs of Police released principles for responsible use earlier this year. That comes as police forces in many cities, including Toronto, Calgary, Vancouver, Ottawa, and Winnipeg, are using facial recognition software, video analytics, and AI-assisted police reports. The Toronto Police Service published a list of AI tools it’s using, but it’s not clear when the list was last updated.

Fritsch says transparency is essential, which is why the LCO recommendations include a call for a public AI registry.

“The public can’t participate meaningfully in consultation without transparency,” he says

“We can activate the public and get meaningful consultation through mandatory assessment.”

As LCO staff continue to consult with police services, courts, lawyers, and the public, the big question is whether the report’s recommendations will be implemented.

“Tech doesn’t create the rules; government creates the rules,” says Fritsch.

“I’m optimistic that we can implement these recommendations and make changes to our justice system.”