Forecasting the Supreme Court’s work

Last year, Judge Ketanji Brown Jackson spent three days being grilled by the Senate Judiciary Committee on her nomination to the U.S. Supreme Court. While the questions from Republicans on the panel were mostly bog-standard midterm messages about violent crime and abortion, Sen. Josh Hawley’s people did a little research.

“In every single child porn case for which we can find records, Judge Jackson deviated from the federal sentencing guidelines in favor of child porn offenders,” the Missouri Republican tweeted before Jackson’s hearings began, citing nine cases from her time as a federal District Court judge.

Hawley’s statistical bombshell turned out to be a dud: Judge Jackson’s sentencing pattern was pretty mainstream and Hawley himself had voted to confirm judicial nominees with similar sentencing patterns. Numbers don’t lie. People do, however — and numbers stripped of context can be misleading.

Judicial analytics is what happens when modern information processing techniques like machine learning are applied to the work of courts, judges and lawyers. Companies offering judicial analytics services make some bold claims — that their products can help lawyers craft arguments with particular judges in mind, that they can track a judge’s “ruling tendencies” or predict how long it will take for a case to conclude. One product offers to uncover the “personal factors” that influence a particular judge’s decisions, including “net worth, education, work experience [and] political affiliation.”

“Talking to lawyers, we don’t get the sense that it’s in widespread use in Canada,” said Amy Salyzyn, associate professor at the University of Ottawa’s faculty of law. “We are seeing more interest in the technology, however, and more and more court data is getting digitized and the tech is getting better.”

Are the makers of judicial analytics tools overselling what they can do? In a new paper, Salyzyn and her associate professor colleague, Jena McGill, argue that while the technology may hold some promise for analyzing trial-level trends, it can’t predict which way the Supreme Court of Canada will jump.

Commercial judicial analytics tools track the work of individual judges. SCC judges always sit in a panel and their rulings are arrived at collectively, if not always through consensus.

“Although a justice may be formally listed as the author of a decision, the work product is not theirs alone,” McGill and Salyzyn wrote in their paper — part of a collection of articles on data and the Supreme Court entitled (De)Coding the Court.

“This’ group work’ dynamic is incompatible with using judicial analytics tools to profile individual Supreme Court judges … Any profiled justice will be joined by several (usually eight) others in hearing and deciding a case, all or any of whom may have competing or contradictory preferences to a profiled justice.”

Judicial analytics tools could have other uses in Supreme Court cases, McGill and Salyzyn argue. They could spot patterns in the judicial authorities and secondary information sources upon which the court relies — useful stuff for a lawyer drafting a submission. They could ballpark a leave-to-appeal submission’s chance of success. They could be used to support recusal motions. Such motions face a “heavy burden” of proof, said McGill and Salyzyn, but analytics could pinpoint patterns in a particular judge’s work that could support a claim of bias.

“Even if such evidence is not admitted on a recusal motion,” they wrote, “there could be negative impacts on public confidence in the court if it is otherwise publicized.” Which points to the underlying problem posed by judicial analytics — its potential as a political tool. Because the line between the judiciary and politics is far blurrier in the U.S., judicial confirmation hearings there tend to be zero-sum political contests. Every judge is seen as partisan and every judicial appointment is a win for one party and a loss for the other.

It’s not hard to envision judicial analytics tools being weaponized by political actors to undermine candidates for courts here in Canada — especially high-stakes appointments to the high court. Statistics about past decisions could be stripped of context and deployed to suggest a particular candidate is “soft on crime” or biased against a particular population.

“Anyone who uses these tools ought to be aware of their limits,” said McGill. “A naked statistic like ‘the number of appeals allowed by the SCC jumped 35% last year’ is meaningless without the right context. It doesn’t take into account occasions when the court had no choice but to grant leave to appeal.”

The impact of politicized analytics might not be felt equally by every candidate, said Salyzyn. “Women or racialized candidates for the judiciary tend to come under much greater political scrutiny than others,” she said. “There is the potential for these tools to be weaponized against such candidates, creating a chilling effect.”

There’s even a risk that judicial analytics could lead to a “flattening” effect in Supreme Court decisions. The SCC occupies a unique space as Canada’s apex court — it deals in novel legal analysis and is less bound by precedent than lower courts. Judicial analytics encourages litigants to engage with the Supreme Court based on its past actions, possibly making creative approaches to legal questions less likely.

“It could, by extension, lead to fewer novel arguments and fewer contemporary sources being cited,” said McGill. “The great strength of the common law is its ability to adapt to unprecedented situations.”

Ottawa-based criminal lawyer Michael Spratt isn’t keen on the technology. He said he’s convinced judicial analytics will be employed for political purposes in Canada — and the public’s trust in the system will suffer as a result.

“It’s importing an American mentality to a court system that is very different,” he said.

“We’re already seeing that emerge here in journalism, with reporters pointing out whether a particular judge in the news was appointed by a Conservative or a Liberal. It’s always a part of the coverage now. And it’s a lazy form of journalism because it doesn’t really tell you anything about the individual.”

Would he ever use the technology himself? Spratt said he sees the appeal — it’s easy for lawyers to become overloaded with information and any service promising to reveal hidden patterns in the data is going to have customers.

“I haven’t used them myself,” he said. “I think good lawyers would find them largely unnecessary. If you’re focused and prepared, if you know your facts and you’re familiar with the jurisprudence and the court, you don’t need it.”