In 2017, Minnesota replaced its aging 1980s legacy DMV system, only to run into problems and end up with a price tag that was double the original cost. Chicago’s Office of Budget and Management pulled the plug last year on a new budget software system that did not work. The city has been forced to rely on its legacy system that is no longer supported by a vendor and has limited reporting capabilities.
IT systems fail for a variety of reasons. Sometimes the problem is lack of proper planning. In other cases, it can be scope creep, allowing the project’s requirements to change over time, forcing delays and budget increases. But a major cause for IT project trouble is the software development methodology known as “waterfall.” Considered the first life-cycle process model for complex IT software projects, waterfall caught on early because of its simplicity. Each phase of the project must be completed before it moves on to the next.
That was important because so much of software development in the early decades of government IT involved one-off, custom-built systems that could include research, new development, prototyping, modification, reuse, re-engineering, maintenance and other activities. Waterfall provided a process for building IT systems that couldn’t be bought off-the-shelf, but could contain specific features that were unique to a jurisdiction’s requirements.
But what makes waterfall so popular — it’s a simple, sequential approach to development — has also proven to be its weakness. Failure to complete a phase successfully often meant going back and having to start again, which frequently proved to be costly and time consuming. To avoid this scenario, the project team would try to anticipate all situations by setting detailed requirements ahead of time.
As government IT projects grew in complexity, however, this approach became unwieldy. A change in project scope (a common issue in government where new policies and regulations can upend requirements) often led to a surge in alterations for the system. Throughout the 1990s and into the start of the 21st century, the number of large, costly IT systems that failed to operate as promised, or didn’t work at all, grew in number. At the same time, government increasingly needed a method to develop new software platforms that could be deployed at a smaller scale and in shorter time frames. Not surprisingly, CIOs and the software vendors who worked in government began to look for a better answer.
The Iterative Era Arrives
By 2010, the number of failed or failing IT projects in the federal government had become so alarming that the Office of Management and Budget (OMB) began advocating for a software development cycle that could reduce risk while delivering functionality within weeks or months rather than years by developing pieces of the system in smaller chunks.The process, known as agile, first gained traction in 2001 as the Agile Manifesto, but didn’t reach widespread adoption in the public sector until OMB began to demand change. Unlike waterfall, agile sets deadlines for deliverables in weeks, reducing the scope and risk. The idea is to create software in iterations by dividing up functionality into segments. While initial planning is done at a high level regarding cost, scope and timing, these plans are supplemented by more specific iterations of the project. The status of the effort is evaluated through software demonstrations.
If a requirement isn’t addressed in the demonstration, it can be added to the next iteration. This contrasts with traditional project management where progress is assessed based on a review of data and predetermined checkpoints. Another distinguishing factor of agile is its emphasis on collaboration, with self-directed teams consisting of the customer and the developers, resulting in frequent and close interaction.
As agile has matured, new elements have been added to improve how it can be adopted and used to its best advantage in the government sector. Because agile is so different from waterfall project management, it requires not just a different set of instructions on how to use it, but a new kind of culture, argues Brian Derfer, chief technology officer for Agile Six, a firm that works with government agencies on software development projects.
“In the past five years, government has tried to adopt agile at the surface level, but it hasn’t really built an ecosystem to support agile and to help it thrive,” he said.
According to Derfer, an agile ecosystem includes how contracts are structured; how the culture and mindset of participants need to change in order to support agile; how IT infrastructure and architecture can be more synergistic to agile; and the role of governance and how it is applied.
Contracts have always been a challenge for IT firms that work with government, where lowest bids frequently trump quality. That’s been the problem when contractors have tackled IT projects using the waterfall methodology and has carried over to firms that want to use agile.
“Contracts need to be structured to be more outcome-based,” said Dan Levenson, chief strategy officer at Agile Six. “Traditional contracts are hundreds of pages long with explicit instructions on what can and cannot be done. Vendors are expected to just deliver on all requirements, even if they were wrong. There’s no room for feedback or response.”
Contracts that allow for quick iterations let agencies see value earlier in the process, argues Levenson. Iterative contracting also permits agencies to pay at each iteration and can include multiple vendors. There’s less likelihood of paying for unneeded labor costs and vendor lock-in, a common problem in government IT projects.
Spiral Brings Iterative to Complex Projects
Agile, while innovative and more flexible than the waterfall methodology, isn’t the perfect solution for every IT software project. Another development life-cycle model called spiral has emerged as a favorite among software engineers involved in large, expensive and complicated projects. Spiral’s advantage, according to its advocates, is how it manages risk in a project. Spiral gets its name from the coiled diagram of an iterative, spiral development process created by Barry Boehm, who came up with the concept in 1986.The general idea behind spiral is that “choices based on a project’s risks generate an appropriate process model for the project. Thus, the incremental, waterfall, prototyping and other process models are special cases of the spiral model that fit the risk patterns of certain projects,” according to Wikipedia.
The advantages of spiral include its flexibility to allow for changes after development has started; how it takes risk into consideration in each phase of development, reducing the chance of failure; and customer satisfaction, thanks to its feedback loop, which allows for demonstrations and evaluations by the customer during each phase of the development.
Not surprisingly, spiral has been adopted by federal agencies, most notably the Department of Defense and NASA. As far back as 1999, a contractor used spiral to modernize the Army’s logistics system by introducing IT products as they became available. In 2003, the Army used spiral again to develop a massive $14.9 billion system for its Future Combat program, in which each phase was tested and built little by little.
NASA evaluated spiral and pointed out that the model contains elements of waterfall and iterative, with each phase of the project having to spiral through a set of risk assessments, requirements analysis, design, coding, testing and evaluation. Each pass through the “spiral” allows developers and customers to assess risk and lessons learned before moving on to the next phase.
In a report published in 2004, NASA concluded that spiral, while more expensive to use when compared to other development models, provided a better-quality product that meets the user’s needs by allowing requirements to evolve and by providing functionality at each phase of the development.
Agility at the Enterprise Level
If spiral doesn’t sound like the right fit, a growing number of government agencies are trying out scaled agile framework (SAFe) as a means of applying agile methodologies, along with their specific advantages, to large, complex IT software development projects. While it can help teams of developers and customers align their goals around a project, SAFe is designed to take that to the enterprise level, driving alignment across an organization.The Centers for Medicare and Medicaid Services (CMS) has been using SAFe with several partners and teams of workers to modernize a series of programs involving data analysis, financial payments and oversight, according to Abisoye Ajibade, a deputy platform manager and agile coach at NewWave, a health IT firm.
What’s critical to success, said Ajibade, is having all the participants on the same page. Traditional agile isn’t so effective at bringing everybody together to deliver their input to the project at the right time. SAFe increases engagement, not only at the team level, but also at the program and portfolio levels.
“There’s a lot of complexity to what we are doing,” said Ajibade. “I have oversight across six different contractors, each with its own culture and way of doing things. SAFe allows us to plan on the commitment and delivery; we can make sure that everyone who is responsible for executing their job is aware of what others are doing that will impact how the product is delivered and tested before it gets pushed out.”
Even with the advances to software development that agile, spiral and SAFe bring to the government IT workplace, challenges remain on just how effective and successful individual projects will turn out. Using a relatively new software development model at CMS, with its administration and oversight of the country’s two largest health-care systems, can be quite daunting, even with the best resources. Ajibade said coordination and execution can be difficult at times.
Still, the demand for flexible, iterative development models continues to grow as technology becomes the DNA of how government operates. With more options on how to deliver the right technological solution within budgets and on time, government can breathe a sigh of relief.
