Pennsylvania’s GenAI Pilot: Efficiency by the Numbers

Pennsylvania’s yearlong pilot for state employee use of generative artificial intelligence technology resulted in demonstrable time savings, according to a report late last month.

The goal of the state’s Generative AI Pilot Program was to understand how state employees can use tools like GenAI safely, to expand their capacity in serving residents.

Participation was open to all Pennsylvania Office of Administration employees, a range of which were selected for the first cohort, with the others added on a rolling basis. Employees from several other agencies were invited to participate in other cohorts. Participants completed a survey on a regular basis for approximately the first eight weeks of the pilot, and an exit survey at the end.

Employees used the technology for various tasks, with 36 percent of pilot participants having used it for “writing assistance, text generation, or drafting emails.”

Their use of ChatGPT cut working time to under 15 minutes in 58 percent of cases. Per the participants’ exit surveys, they saved 95 minutes a day by using ChatGPT.

Those surveyed reported more tasks were made simple and very simple when using ChatGPT as opposed to task complexity without it.

However, the generative AI outputs often still required additional nuance, or to be corrected for inaccuracies, hallucinations and bias. The report described human nuance as “essential.”
"I need more creative examples and limits of what I am permitted to do," one pilot participant said in feedback to the state. As such, the report encourages organizations to provide employees with clear examples of what is and isn’t permitted when using generative AI tools.

Pennsylvania’s report indicates officials believe this to have been the first such pilot program in the nation. Leaders indicated they hope other states will use their playbook, and encouraged other government officials to reach out to learn more about Pennsylvania’s approach.