Diversity in user testing

Denis Ram, along with his law school classmates in 2021, created an app to automate the police complaint process as part of a course at the University of Alberta Faculty of Law. The goal is to make the process more accessible and reduce barriers for people of colour to file police complaints. Through funding from the City of Calgary's Innovation Fund, Ram's team is getting ready to move on to the next step.

When Ram and his team at Complete Complaints Foundation (CCF), the new nonprofit managing the app, decided it was time to do user testing, they turned to IncluCity, a newly formed Calgary nonprofit focused on offering diverse user testing for tech companies.

One of the biggest blind spots in tech development is user testing. Most companies neglect to have a diverse group of users from different ethnic, accessibility and language backgrounds to test their products. Diverse testing helps design better outcomes for communities.

"Tech solutions are often created without input from the people using then," says Geoff Zakaib, President of IncluCity Calgary. "Many startups or mature organizations don't have time or budget to do usability testing in an inclusive way. Sometimes companies will hire a marketing firm to engage focus groups, but these groups are not representative of the people who will be using it like newcomers, seniors or youth."

After going through an initial round, the police complaints app will be revamped in plain language, to be available in eight languages, for users to have the option to have audio to listen to the questions and the ability to make the text larger. The app will also incorporate case law in each section to better educate the public about the police complaint process. The goal is for the app to replace the current electronic police complaint system and eventually have the app manage all police complaints in Alberta.

"We wanted to go through IncluCity because we wanted to make the app better, and we wanted to test viability," says Ram, executive director of CCF. "So far, the reviews have been positive."

IncluCity uses funding from private and public funding to pay testers. Because testing doesn't typically happen in an office setting between 9 to 5, IncluCity volunteers go where the testers are. Testers, who are paid, use their own devices at the location of their choice, which could be a local coffee shop or the library. Those with busy schedules can also test it during evenings or weekends. IncluCity will also pay for childcare or transportation costs.

"We realize that everyone interacts with technology whether they choose to or they're forced to," says Zakaib. "If we don't have their inclusive voices at the table, that's not good. We need to hear from different perspectives and their voices need to be heard."

IncluCity is modelled after Code for Canada's Gathering Residents to Improve Technology (GRIT) Inclusive Usability Testing program that's been running since 2017. Code for Canada works with public and private organizations, including companies in the fintech market.

The organization is rooted in the growing civic tech movement, where technologists, developers and public servants collaborate on community projects. For example, community organizers from Civic Tech Fredericton created RiverWatch, an app that provides a three-day flood forecast for residents. There are ten civic tech organizations across Canada.

Code for Canada began in 2017 when Dorothy Eng, a marine engineer, got a few friends together in Toronto for civic tech meetups.

"We were hosting one of our hackathons and people from the Ontario government show up and they're like, 'you work on projects for free?'," says Eng. "We said, 'yes,' we like to contribute our skills to make our communities better. We decided to make it more official. We got funding from Ontario and we want Code for Canada to be national."

Though user testing can be a "hard sell," organizations have to recognize its importance, says Eng. And though they may be tempted to rely on their employees, that can be problematic since they are likely to know the system.

"We help teams check their biases," says Eng. "Are people thinking of the people who will be using their technology? Are they thinking of the people they're designing for? Governments have the obligation to create services that are accessible for everyone."

Another option is for organizations to hire marketing companies to do user testing. The challenge is finding organizations that do diverse user testing in focus groups. PLATO Testing hopes to close that gap. The organization offers software testing services where trained Indigenous professional software testers can do usability testing, accessibility and compatibility testing.

The goal isn't to get everyone but to represent as many people as possible so technology can resonate with the community. It's a goal that IncluCity is focused on.

"We can define communities in so many ways," says Zakaib. "It can be overwhelming when considering the many communities who are often under-represented such as newcomers, seniors, youth and others that may be disabled, Indigenous or with lived-experience. There's not an easy way to include everybody but we need to take small steps towards inclusivity. That's why it's important to get started."