When Bob McDonnell took office as governor of Virginia in early 2010, he brought with him renewed hope that the state's troubled IT outsourcing contract with Northrop Grumman could be salvaged. At the time, there was talk of scrapping the agreement -- a move that would cost the state hundreds of millions of dollars. But rather than sever ties with Northrop Grumman, McDonnell turned to a man who, seven years ago, was instrumental in forging the partnership.
On March 15, McDonnell signed legislation that restructured the Virginia Information Technologies Agency (VITA) and put the state's CIO directly under the governor's authority. Shortly thereafter, McDonnell appointed longtime Virginia lawmaker Sam Nixon to this post. Nixon, who has several decades of IT experience, was one of the lawmakers who helped write VITA into existence in 2003.
Now he was tasked with rescuing the colossal outsourcing deal that he helped set in motion.
Troubled Partnership
It's no secret that since Virginia and Northrop Grumman struck the IT outsourcing deal, both parties have encountered several high-profile problems. The 10-year, $2.3 billion contract was written such that Northrop Grumman would provide the state's IT services. But the project -- the largest of its kind among the nation's state governments -- has been plagued by delays, cost overruns and poor service.The partnership between Northrop Grumman and VITA -- created in 2003 when then-Gov. Mark Warner launched a major initiative to centralize and overhaul how the state invested in and managed its technology and networks -- originally was hailed as a bold new approach to large-scale outsourcing. But poor planning, ineffective governance, lackluster service delivery and unrealistic service expectations nearly caused the contract to self-destruct.
Despite the best intentions, in hindsight the many moving parts designed to help one another ended up doing the opposite. VITA was tasked with overseeing the contract between the state and Northrop Grumman; the Legislature-created Information Technology Investment Board was given independent authority to appoint the state CIO; and Northrop Grumman agreed to a contract that required too little accountability on its end and too little governance on Virginia's part.
The situation became a crisis in June 2009, when VITA fired then-CIO Lem Stewart. Reports at the time claimed that Stewart's dismissal came on the heels of his failure to pay Northrop Grumman on a multimillion dollar invoice. Stewart suggested in several presentations that the lack of payment was due to his belief that the vendor was not meeting its contractual obligations.
Stewart's firing set in motion a game of CIO musical chairs. After several temporary CIOs, private-sector software executive George Coulter was hired by the Information Technology Investment Board. Coulter's time at the helm was also short-lived due to a change in the governor's office.
Contract Changes
McDonnell followed the VITA restructuring with a restructuring of the Northrop Grumman contract. The move was greeted with both optimism and skepticism, as the new deal increased payments to the company by $105 million, but also added stringent performance requirements and accountability conditions.The contract modifications added clarity and scope that were missing from the original agreement, according to Nixon. "If we had known four years ago what we know today, we would have started out differently," he said.
As an author of the original VITA legislation, Nixon has intimate knowledge of the thought process behind the outsourcing arrangement. He said some of the deal's shortcomings could only be discovered once the plan was set into motion.
"There were things that only time and experience could foretell," he said. "I don't want to say there were flaws, but there were certainly gaps in a contract this complex, this huge that when it came time to operationalize and actually do those things, it wasn't until then that those gaps became apparent."
The contract modifications came after a 2009 legislative audit slammed the agreement between Virginia and Northrop Grumman. The Joint Legislative Audit and Review Commission pinned the blame on both parties. The audit found that Northrop Grumman's inadequate planning and VITA's inadequate understanding of agencies' needs led to most project delays. Among the audit's other findings: the complexities of state government were not adequately understood by Northrop Grumman; VITA's authority to address delays was inadequate; VITA and the Information Technology Investment Board failed to reach an agreement on how to address performance problems; and both the CIO and the governor lacked the needed authority and accountability.
The new contract provisions are designed to improve performance and boost efficiency. For instance, the amended contract calls for Northrop Grumman to create Rapid Response teams to quickly resolve individual IT issues; expedite replacement of damaged desktops and laptops by replacing equipment within 24 hours of repair determination; consolidate and strengthen performance-level standards, with a 15 percent increase in penalties if the company fails to perform on performance standards; and add new services to the contract, such as improved disaster recovery and enhanced security features.
The new contract also requires Virginia to release $15 million in previously withheld contract payments and includes $105 million in additional cash payments over nine years.
What Went Wrong
Virginia certainly is not alone in encountering problems during large-scale IT outsourcing endeavors. Like most enterprisewide projects, it was for a noble cause that Virginia began this journey. Yet like many grand visions, critical details in Virginia were overlooked or never perceived."Northrop Grumman and the commonwealth as a whole saw the value of moving toward a shared-services model to providing the containment of cost, stability, considering more mature services in our case [and] providing services that we didn't have before, but that there is a considerable amount of complexity related to the technology component of making that happen," Nixon said. "And I think that the rate and the scope of change, particularly the scope, was much broader, much more complex than anyone had anticipated. We had an extraordinarily diverse technology environment here that has grown over many decades.
"The state and Northrop Grumman decided to hold hands and together step into a puddle," Nixon continued. "Neither side knew just how deep that puddle was at the time."
One major oversight -- which happens surprisingly often in state enterprise projects -- was the failure to bring agencies into discussions. In Virginia, before the contract restructuring, agencies had little or no say in the outsourcing arrangements that would impact their operations.
Giving agencies a stronger voice was what drove the creation of a CIO Council -- a group comprising state agency CIOs that meets every other month -- during Coulter's brief tenure as state CIO.
"Those meetings are for us to communicate with them, but we also have a town council-style interaction with them where they can provide their concerns to us," Nixon said. "We're also standing up an executive-level council called the Information Technology Advisory Council, ITAC. And that's going to be composed of the business owners, not IT people, but business owners of government. That group will have yet another conduit into VITA, and what I hope would happen, my vision is that group will focus on enterprisewide applications and enterprisewide initiatives."
Also problematic for Virginia was that because the CIO reported to the Information Technology Investment Board and not the governor, authority over the project on Virginia's side was never clear.
"We thought [that] was a creative and unique way of governing IT," he said. "And that worked well for a long time, but it stopped working pretty quickly at the point in which conflict began to raise its head within the partnership."
Complicating matters was the fact that no entity had the authority to compel agencies to do what VITA was asking.
"All the people who were here before say the change in the governance model is a good thing," Nixon said. "And again, it's painful for me to sit here and say in retrospect that the old governance model was broken. ... I don't know that it was broken, but it definitely became broken along the way."
This was because once things became confrontational, there was no one to step in, call a time-out and send everyone to their rooms, he said. "We do have that now with the governor," Nixon added. "So I think it's an important difference and one that has been very helpful since I have been here."
Going Forward
Nixon isn't ready to say that the governance changes and contract restructuring are enough to ensure smooth sailing into the future. And many state lawmakers remained unconvinced that the changes are enough. Several talked to the Richmond Times-Dispatch, expressing their concerns in a June 21 article."There's still some nervousness among the [budget-writing] committee," Delegate Scott Lingamfelter said. Delegate Chris Jones told the paper he wonders what will happen when the contract runs out in 2019. And Delegate Johnny S. Joannou, though pleased Nixon was named CIO, cryptically likened the outsourcing deal to a "bowl of spaghetti that's messed up."
Delegate Steven Landes, a longtime lawmaker who knows Nixon well, talked to Public CIO about his concerns regarding the restructuring. He echoed Jones' sentiment, wondering if the state is painting itself into a corner.
"One of my colleagues, Chris Jones from Suffolk, asked at the last preparations meeting, which I thought was on target, what's going to happen with the hardware when the contract is finished?" Landes said. "Does Virginia own the hardware or does Northrop Grumman? Those are some of the things we're still going to have to work through. But I think we're getting there."
Landes went on to say that he believes Nixon is the right person for the job, especially given the way the governance model has changed. Yet Landes remains unconvinced that the outsourcing arrangement is in taxpayers' best interest.
"As elected representatives, it's our job to make sure money is spent appropriately and that the state contracts are good for the taxpayer," Landes said. "From the state's standpoint, we have responsibility for the problems that ensued, and I think Northrop Grumman does too. And I think early on one of my concerns was it didn't seem like Northrop Grumman thought that there was any real responsibility, they felt like they were meeting the tenets of the contract. But as a contractor with a government entity, there is a little higher level of accountability than just in the private sector. It's not just private entity money that you're dealing with or a group of stockholders, you're dealing with the taxpayers, you're dealing with the average citizen."
Nixon said the experience with IT outsourcing -- from conception to present -- has changed his perspective on the process, were he to do it over again.
"I'm not sure if I'm ready to change my views on outsourcing, but maybe it has changed my views on how we constructed or went about the outsourcing," he said. "We probably got ahead of ourselves, particularly with VITA, which was still a fairly new organization."
In other words, he said, it was like trying to live in a house at the same time it was being built -- at the same time Lowe's showed up with a bunch of lumber, but they didn't have any hammers or nails.
"There were just a lot of things about that that were not sequenced properly," Nixon said. "Looking back, it's really easy to see those things. Back then it wasn't nearly as apparent. It was all shiny and new and all looked like a good idea, but when it came time to actually making things happen, that's when it became very difficult, very quickly."
