It’s true, though. All around the nation civic groups are paying ordinary citizens to test-drive government web sites and put new apps through their paces.
Known as Civic User Testing Groups, or CUTGroups, the phenomenon first appeared in Chicago in 2013. More recently activists have launched CUTGroup efforts in Oakland, Calif., Chattanooga, Tenn., Miami and Detroit. Momentum is building: The Chicago group recently put out a how-to book for other cities looking to participate.
Citizens typically get a $5 VISA gift card just for signing up to participate. Anyone who comes in to test a website or application gets a $20 gift card. Organizers say the program offers a force multiplier to city IT departments that may not have the bench strength to perform thorough user testing on every new tool they introduce.
“This is a way for people to provide a check, to see whether they are really keeping up with their community and with people’s needs and expectations. That kind of thing is well-known in retail, but the importance has not been as widely recognized in municipal services,” said Dan Duffy, assistant director of CUTGroup Miami.
Since launching in September 2016 the Miami effort has already enrolled some 500 testers and performed two tests, one a review of the county’s data website and the other a test of the city’s next-generation web site, presently under development.
“The sooner we can provide feedback, the more valuable it is, so ideally we would like to validate what is presently in development,” Duffy said. “The best value is always in us getting involved early with what the municipalities are doing.”
Miami’s CUTGroup is a project of the Code for Miami civic organization, and is funded with a $100,000 2016 Knight Cities Challenge grant.
CUTGroup Chicago operates under the umbrella of the Smart Chicago civic group and draws its funding from that group as well as from a Knight Foundation grant. The effort has attracted some 1,900 testers and run over 30 tests, according to Project Coordinator Sonja Marziano.
Beneficiaries of these tests span a wide range. The CUTGroup has worked with civic technology developers and has also tested a redesign of the county website. Testers have helped the city’s Department of Innovation and Technology try out an open data portal redesign.
The CUTGroup has worked with the county forest preserve authorities to try out a new trail-finder app and is also collaborating with the city’s public school system on efforts to make websites more mobile-friendly.
More than merely taking these websites and apps out for a spin, CUTGroup organizers have sought to make their testing an asset to the IT development process.
“It’s a very collaborative process with the partner. We go through why they want to do this test but also what are the pieces of the technology where they really need feedback,” Marziano said. “It may not be the entire site: Maybe it is just the sign-in process. So we really drill down into what they want to test.”
By taking this hands-on approach, “it helps us do the screening [of potential testers] and it helps us to design the test questions. Then we will look through past analytics, look at how people have previously used the site, and the design process can happen around all of that,” she said.
This kind of intensive engagement is typical of the CUTGroup approach. It’s true that most websites and apps could be tested remotely, with users logging their feedback on a form. But CUTGroup promoters say government gets a better end product with a more hands-on process.
“The real value is in the observation, having a tester and a proctor there together,” Duffy said. “We give the tester the objective and then we actively record what we observe. We note where they struggle. We ask them what they would have expected, what their preferences are.”
In addition to providing a thorough evaluation, CUTGroups also give government something that can be hard to come by under ordinary circumstances: access to a broad base of citizen end users.
“At the government level things may be tested internally or tested on a limited group,” Marziano said. “When we get 20 or 30 people together at the library, we can provide a very diverse group with people from different neighborhoods, different demographics and so on. It’s a chance for the agencies to go out into the community and interact with people, to talk directly to their potential customers.”
While Marziano agreed that early stage testing is typically the most productive, she stressed that government can always benefit from putting its websites and apps in the hands of citizens for evaluation.
“Our partner agencies don’t always have a deep technical staff, so sometimes the usability testing will be done to scope out future work. They may not be able to make immediate changes but we can still do the testing and identify what needs to be done, even if that means they will send it out to an outside developer at some future time,” she said. “Even if a site is already launched, that is fine, as long as there is a plan for how they will respond to the feedback that the test is giving them.”
