Agile Acquisitions: Rethinking Public-Sector Purchasing

For years the state of California has been doing multi-year, multimillion-dollar software projects, only to find that many of them don’t produce the results the state had hoped for. They take too long to plan and execute, and cost more than expected. “Customers have one consistent thing they tell us: You gave me what I asked for, but it is not really what I wanted,” said Peter Kelly, chief deputy director for the California Health and Human Services Agency’s Office of Systems Integration (OSI).

Determined to break out of that cycle, the state looked for a significant project to which it could apply agile project management methodology and develop software more iteratively. Agile development breaks software projects up into short “sprints” of a few weeks, while business officials and IT teams work closely on refinements. The traditional and linear software development approach is referred to as “waterfall.” Each phase of a project is completed before moving to the next.

In late 2015 the Health and Human Services Agency decided that the replacement for a 20-year-old Child Welfare Services case management system would be the test bed for agile adoption. Did that require rethinking how to do procurement for the project and what type of RFPs would be developed? “It required rethinking everything,” Kelly said, “and procurement was definitely at the spearhead of the effort.”

Changing Requirements

Federal, state and local government IT departments are gaining more experience with agile processes and are starting to develop staff competencies to work in a more iterative fashion, but among the transition challenges, many have identified procurement as a particular stumbling block. In 2017 Accenture partnered with the National Association of State Chief Information Officers (NASCIO) to survey state CIOs and agency heads. From 53 completed responses, 70 percent said procurement was not set up for agile projects.

Keir Buckhurst, an Accenture managing director for the public service industry, said that in traditional waterfall project procurement, you have three sides to a triangle: time, resources and requirements. The requirements are fixed, and you can have variations in time and resources. “With agile, we are switching it to say time and resources are fixed, but the requirements can change and evolve. Therefore, how you contract is very different. That is the toughest thing for people to wrap their heads around.”

4 Tips from Agile Procurement Pioneers Peter Kelly, chief deputy director of the California Health and Human Services Agency’s Office of Systems Integration: Don’t assume legislation or regulation blocks you: “There was no regulation that we couldn’t work within the confines of, and no policy so tight we couldn’t work through it,” Kelly said. “That was a real surprise to California. Most people initially assumed this was a nonstarter.” Keir Buckhurst, managing director of Accenture’s public service industry: Spend less time in negotiations on requirements and more on governance and communications. Be clear about roles, responsibilities and communication. Define how long sprints are and what decision-making is allowed at the team level. “If a project is really going to be agile, you can’t have a process in which decisions get escalated to a steering committee that meets only every two weeks,” he said. “Decisions need to be made at ground level.” Scott Smith, IT procurement director of Washington state: If you do create a pre-approved vendor pool, recognize that you need to refresh it often. “Resumes can move from company to company. If you can’t react to that quickly, the value of the pool diminishes. That is a problem we are trying to solve now.” Ron Baldwin, CIO of Montana: Beware the agile definition gap. Not all vendors can execute on agile, despite what they might say in proposals. “Not until you get the team on the ground will you really find out whether they possess those skills or not — or whether their idea of agile and yours are the same. We have found ourselves having to train the vendor on how we execute a project in agile.”

Another challenge is getting the executives at authorizing agencies to understand that how they will be asked for funding is changing. “They are used to somebody saying, ‘I am going to need exactly this much money because I am going to deliver exactly this functionality on exactly this date,’” said Buckhurst. Now, project teams tell them how much they are going to spend, but not exactly what the solution will look like or when it will be done.

Some states are not just training IT and business teams about agile, but also doing that same training for their contract and procurement staff. In fact, executives in the state of Washington’s IT procurement office went through “scrum master” training. (The agile framework for iterative change is called “scrum,” and the person leading the collaborative effort is called the “scrum master.”) They are working to not only procure for agile software projects but also to make the procurement process itself more agile, according to Scott Smith, the state’s IT procurement director.

“You can’t be good at evaluating a vendor and their ability to do agile software without understanding agile yourself,” Smith said. “You are not evaluating what they are going to create. You are evaluating the how. For me, that is the big difference.”

Agency executives and legislatures have to learn to give up the illusion of control over software projects, he explained. “You thought you had control with waterfall, but you really didn’t. The idea that change is bad contributed directly to waterfall projects’ failure. In an agile environment, we use change to a customer’s competitive advantage.”

Smith gave one example of how he would like to procure differently for agile: Often, with software projects, you get to a point of diminishing returns. In other words, the internal customer looks at the rest of the work to be done and says, “Never mind, we can live without those.” In a traditional waterfall environment, both parties are contractually obligated to finish those out, which is probably a waste of time and effort for both parties, he said.

The vendor is not providing value in the customer’s eyes and the customer is paying for something they don’t care about, according to Smith. “I would like us to accelerate to that point as quickly as possible,” he said. “We could negotiate with the vendor and say if you get to that point early, you can have 50 percent of what is left on the table for doing no work. That incentivizes the vendor to get to that point as quickly as possible, which is to the customer’s advantage. That is the contractual nuance we are thinking about from a negotiation perspective.”

The fact that the Office of the CIO in Washington is promoting agile has made a huge difference for the procurement team, Smith said. “We and our customer agencies were on the same maturity path to understanding how to do these projects, and as my team has ramped up, the customers have come to recognize the value. It has been good for us not to be too far out in front of them, but having the CIO’s office driving it was important.”

Rethinking Procurement for Big Projects

In California, several agencies have been experimenting with agile on smaller projects, but the Child Welfare Services case management system was the first place the state got behind this approach on a major capital investment. Hoping to learn from past failures, the agency wants to break the large project into smaller pieces and deliver value more iteratively, rather than spend years on procurement and development.

Breaking up such a huge project meant rethinking procurement. For one thing, if you are going to engage in multiple procurements, there is no foregone conclusion that the same vendor will win all of them, Kelly noted. If the project involves multiple vendors, that has implications for systems integration. OSI had a 13-year history of bringing in a single vendor to be the systems integrator but decided to play the role of systems integrator itself for this project, which required adding new skill sets.

OSI sought consulting help on procurement from 18F, a federal office housed within the U.S. General Services Administration, and Code for America, a nonprofit that augments local governments’ efforts involving technological innovation.

Kelly said Dave Zvenyach, an 18F official, “led a conversation for a day and a half with more attorneys and acquisition specialists than I could count. Going through laws, policies, regulations and statutes, he helped debunk their notions that agile procurement couldn’t work.”

It turned out that in order to make the major shift to iterative procurements to do agile work, not a single law had to be changed. “There was no regulation that we couldn’t work within the confines of, and no policy so tight we couldn’t work through it,” Kelly said. “That was a real surprise to California. Most people initially assumed this was a nonstarter.”

But even with legal questions set aside, the state had to create a mechanism to procure faster. “If you are going to do lots of small procurements, you can’t work on a 12- to 24-month time frame,” Kelly said. “It doesn’t work.”

California chose to follow 18F’s example and create a pool of vendors pre-approved to do agile work who could respond quickly to smaller procurements. In 2016 the state gave vendors a problem to solve using software with examples of what they wanted them to demonstrate. They had 30 days to reply. More than 20 companies made submissions, and 11 vendors, both big and small, qualified based on the state’s criteria.

As work ramps up on the child welfare system, the procurement process has shifted away from language around specific products the agency wants vendors to build and more toward how the agency wants to work, according to Kelly. “The ‘how,’ for me, is so important. It is a shared ownership, a shared responsibility. It is a gigantic paradigm shift for all parties involved. The last year for us has been as much about establishing how we work as it has been about what we are building.”

Less-Than-Agile Vendors

For most states and localities, the issue around agile procurement is getting started and then scaling up. Joshua Karstens is in charge of transitioning Maine’s IT projects from waterfall to agile. As director of the project management office in Maine’s Office of Information Technology, he has overseen agile used for smaller projects. “We are looking at how to do it with much larger projects where we have to do RFPs and contract with vendors,” he said. “We usually spend months on these RFPs creating long lists of requirements, and they end up being inaccurate because the project changes as we move through it.” Now, the goal is to get the RFPs down from 200 pages to 10 pages and with a quick turnaround.

Karstens has his eye out for a large project that would be a good fit for agile. “If we are going to have hard lessons from it, I want the risk to be low,” he said. “If we are going to do RFPs in a new way and award contracts and statements of work for each iteration, we don’t want a lot of risk associated with it.”

But an even bigger issue is finding vendors who have the experience to work in an agile environment. The state of Montana has years of experience with agile methodology internally, but mixed experiences working with outside vendors on agile projects. The situation has forced agencies to rethink how they pay for application development.

The state’s Application Technology Services group has been doing agile exclusively for eight years, said Audrey Hinman, the group’s bureau chief. Her team creates memoranda of understanding with its internal customers, and agile has definitely changed the way they are written. “The memorandum puts more of the responsibility for watching the budget and timeline on the agencies themselves,” said Hinman. “We don’t do fixed bids anymore. We do give a detailed scope of the project and estimate what cost we think it will have, but all the terminology just refers to estimates. We put the responsibility for prioritization of the backlog and watching expenditures and timelines on the agency project manager,” she explained. After the fifth or sixth sprint, we are usually delivering usable code, but they could shut development off at any time after that fifth or sixth sprint.”

Montana CIO Ron Baldwin said he believes that using agile exclusively helps attract and retain talent. “Our staff are so used to agile in terms of how they conduct an IT development project, they wouldn’t know what to do other than agile,” he said. “Most of them recognize that it is a modern methodology that is working to reduce the risk of projects and making customers happy because they are delivering more sooner, and the ultimate product is much more in line with what the customer expected for the budget they established in the first place.”

But now, Montana is finding an agile gap exists with the technology firms that are supposed to build and deliver the tech solutions it needs. “Vendors, particularly those that have teams that have been delivering a particular solution for years in their area of expertise, don’t necessarily bring the agile skills you would think they would bring,” Baldwin said. “They can say they will do it when they respond in proposals, but not until you get the team on the ground will you really find out whether they possess those skills or not — or whether their idea of agile and yours are the same. We have found ourselves having to train the vendor on how we execute a project in agile.”

Hinman said Montana still sees large vendors not fully committing to agile. “They say they are going to do it, but half the methodology is still waterfall,” she said.