After Vermont Auditor Doug Hoffer and his team selected six state IT projects as their chosen sample size for review, they found that five of the six projects had significant cost increases and/or schedule delays. Two of the six selected projects were fully deployed by the deadline; three were partially deployed; and only one of the projects was completed on time.
“One of the projects of concern failed to deploy a usable system more than two years after it was expected, despite paying the vendor $2 million or 95 percent of its maximum contract amount,” Hoffer wrote in the audit report, adding, “The project had an unrealistic completion date and the contractor eventually stopped working on the project without addressing problems identified during testing.”
In two of the other projects, “costs more than doubled because of delays and deadline extensions of at least 18 months due to changes to functional requirement regulations, alterations to development or deployment approaches, and added security requirements,” the report noted.
In one example, about a month after a procedural contract was signed for the state’s DMV modernization project by the Agency of Digital Service (ADS), the Legislature changed a statute, setting new fee requirements. As a result, a product registration function had to be added to the project via a new contract and a second contractor to comply with the new statute.
ADS Secretary Shawn Nailor noted that many delays were not technology-driven and were, in part, affected by policy and regulatory changes.
“Oftentimes, the things that influence our schedule aren’t tech-related,” Nailor said. “They can be caused by federal or state legislation changes. In one case, the Legislature chose to do a DMV fee increase that required us to take tech resources away from the DMV modernization effort to implement those fees.”
Nailor also noted that during the time of the audit, approximately 60 other active ADS projects were being facilitated and each project varied in scope. From this larger pool of projects, state auditors only focused on a small sample for review.
According to the auditors’ report, five of the six IT projects had “poorly defined measures that did not include a baseline goal or outline the size, amount or degree of the change expected and ... most involved entities made limited efforts or plans to evaluate whether the new systems were meeting the projects’ measures.”
“Our world is changing so fast now technology-wise that we are purposely and continuously assessing project needs in each structural phase of a project,” Nailor said. “We could be 18 or 24 months into a project and the tech landscape has changed so significantly within that time, but we’re willing to make additional investments of either time and/or money to ensure that the final product is not obsolete the day it goes live.”
Hoffer and his team included various recommendations for the ADS in the audit including guidance that reads, “Before approving an IT project, ADS (should) work with client entities to ensure that measures include a baseline, quantify results and are relevant.”
According to Nailor, ADS has already made improvements based on the auditor’s recommendations and the agency plans to keep improving processes.
“We’re going to keep doing what is right without restricting ourselves just to meet an arbitrary cost or schedule estimate,” he said. “We’ve already addressed some aspects of the findings and state auditors removed a couple of items from the recommendations after improvements were implemented.”
“We’re going to continue to work on better estimating project timelines and scope by using funds for a comprehensive portfolio management solution that will help us track projects better,” Nailor added.
Vermont IT experienced a statewide overhaul over the last six years, beginning with an all-encompassing consolidation of its information technology efforts. In 2017, the state centralized its IT activities into the newly formed ADS, which led to the formation of the Enterprise Project Management Office (EPMO), which provides project management services via internal staff or contractors to state entities that wish to implement an IT project.
“Without the vision of Gov. Phil Scott in creating the ADS, we would not be in a position where they could do a comprehensive audit of all technology projects across the state,” Nailor said.
“We’ve only been an agency for six years, and while we would like to be more mature in these tech spaces and are working each day to be, we’ve come so far and we’re able to have comprehensive IT audits like this now that wouldn’t have been possible seven years ago.”
