That’s the finding of a new report from the state comptroller’s office, an audit released just as New York’s newly installed chief artificial intelligence officer gets to work.
According to the report, the state’s “centralized guidance and oversight of agencies’ use of AI is inadequate and creates a risk that the technology could be used irresponsibly.”
The report focused on the use of AI in four agencies — the Office for the Aging, the Department of Corrections and Community Supervision, the Department of Motor Vehicles and the Department of Transportation — as well as New York’s “overall AI policy.”
The report covered the period from January 2019 through November 2024, and follows another AI audit released just more than two years ago that focused on New York City.
Many other public agencies across the country are busy coming up with their own policies regarding AI.
In general, this new report found that “guardrails” for AI systems have not kept pace with the technology’s deployment, leaving those programs at risk for data breaches and “inequalities” when it comes to service delivery and decision-making.
“New York state agencies are using AI to monitor prisoners’ phone calls, catch fraudulent driver’s license applications, assist older adults and support government services,” Thomas DiNapoli, the comptroller, said in a statement. “Our audit found insufficient central guidance and oversight to check that these systems are reliable and accurate, and no inventory of what AI the state is using.”
His report determined that while the state has an eight-page AI policy, it does not offer “detailed guidance,” but instead advises agencies to seek further information from federal policies. That, in turn, produces confusion among agencies about what constitutes the “responsible use of AI.”
The report also said too little staff training around artificial intelligence can increase noncompliance and potentially lead to “unintended uses and consequences” of the technology, which is quickly spreading through all layers of government.
More specifically, according to the comptroller’s office, the state’s AI policy has only two sentences that focus on bias management, and criticizes the state’s Office of Information Technology Services, or ITS — New York’s governing body for AI — for not having an inventory of the AI systems being used by state agencies.
As for those specific four agencies the report covered, they lack in-house policies and procedures “to govern how AI is authorized, developed or used, for ensuring the data is unbiased and reliable, or have formal requirements of human oversight.”
At the DMV, the agency “exempted” its facial recognition software from AI oversight, according to the report, contrary to a determination from ITS.
The corrections department, meanwhile, has no place for “addressing potential AI risks,” nor does its contract with the vendor of the inmate phone call monitoring to “address reducing biases to decrease the possibility that an inmate could be unfairly or unnecessarily subjected to further investigation.”
Neither the transportation department nor the other agencies studied for this report have done “periodic reviews or audits” of their AI to make sure they are reliable, free of biases and otherwise accurate, according to the comptroller’s report.
Last month, the state’s Office of Information Technology Services announced the appointment of Shreya Amin, a data and AI scientist with nearly two decades of experience, as the New York’s first chief artificial intelligence officer.
“I'm committed to leveraging AI to enhance services, facilitate operations and drive data-informed decisions, ensuring ethical guardrails, equity and transparency,” Amin said at the time via a statement. “Collaborating across agencies and with our communities, we will create meaningful improvements in the lives of our residents while positioning New York as a leader in government AI.”
This week, after the comptroller’s report was released, her office said it was already working on some of the issues raised in this newest AI audit.
Amin “is already working to build a strong AI governance framework, delivering additional education and training resources, and helping agencies maximize efficiencies, deliver more and better serve the public,” Scott Reif, chief communications officer for ITS, told Government Technology via email. “We recognize that AI is a powerful and ever-evolving tool, and that’s why we are committed to being smart, agile and evolving right along with it.”
The comptroller’s report calls for ITS to craft a stronger AI policy by issuing guidance that other agencies can follow and provide more training. Specific agencies should craft “AI governance structures and policies” as they coordinate with ITS.
The Department of Motor Vehicles also should “review its facial recognition system with ITS to determine if it’s complying with the state’s AI Policy,” according to the report.
