Dan Saaty, co-founder and chief technology officer at Decision Lens, says the tool works -- and the company is building a roster of high-profile clients. The Pentagon uses the tool to help allocate $50 billion in funding, and the federal CIO and the USDA use the tool to prioritize technology projects, he said. The National Football League’s Kansas City Chiefs and Green Bay Packers have used Decision Lens to help select players in the draft. Saaty says he’s even used it a few times to help people decide whether or not to get divorced.
Making complex decisions, like deciding which IT projects to fund, is difficult because there are many considerations and interested parties, but the human mind is not equipped to handle decisions like a computer. People are impulsive and emotional, and despite their best intentions, irrational.
“A project is brought forward,” Saaty said. “You give it a thumbs up or thumbs down like Julius Caesar, but you don’t really know the implications. When I give something a thumbs up, it means something else doesn’t get funded, and is the thing I’m doing really more valuable than the other thing?”
Decision Lens changed how the Washington State CIO’s office, which oversees IT across the state, approaches prioritization. The new approach makes the state more organized and transparent, and it’s encouraging agencies to follow directives, like being cloud-first, officials there say.
“I went into their innovation center and they showed me a whiteboard of all their considerations as they look at these projects -- like governor’s priorities, technology feasibility, alignment to their architecture, and how it helps the agencies meet their missions, so there’s this whole list of aspirations,” Saaty explained. “Their challenge was they didn’t know how to put it in a framework or methodology to evaluate the projects.”
Decision Lens puts an organization’s priorities into a transparent and interactive framework, so everyone can see why projects were ranked the way they were, and then they can proceed accordingly. The results aren't always obvious. When Washington state plugged in its weighted selection criteria last year, for example, officials were surprised to find a marijuana licensing and distribution system at the top of the list, Saaty said. No one would have thought to make that project the top priority, but it was cloud-based, secure, and it would make the state millions of dollars -- all priorities the office had specified.
“Can you imagine if [the state CIO] didn’t have this process, and he approached the state legislature and said, ‘Sorry guys. More important than healthcare, transportation, and education is pot.’ They would have laughed him out of the room,” Saaty said. “But he brought stakeholders from all across the state, so he got to own the process.”
The state CIO’s office is required by law to prioritize IT projects that go before the legislature. So far this year, that means the office is responsible for 120 requests totaling about $500 million. Trying to figure out which projects to prioritize was a brutal task, said Michael DeAngelo, Washington’s deputy chief information officer.
“The way we did it the first year was worse than visiting your dentist,” DeAngelo said. “Essentially, we used Excel and we came up with some criteria and we spent a lot of time as a team trying to figure out the criteria that had to be weighted, and we were doing all this in a vacuum. It was really difficult for us to come to a consensus on how one criteria might be 15 percent of the score and the next criteria might be 20 percent. Everyone had a different opinion on what the criteria should be and we literally spent weeks just debating those kinds of things.”
Once the criteria were established, the state spent more weeks arguing over how each project should be scored according to the criteria, DeAngelo explained. Compounding their problems was the fact that none of the agencies knew the CIOs office was doing this, and so agencies weren't figuring the selection criteria into each request. The technology office had to translate each request into their framework using a highly subjective process, DeAngelo said.
The switch to Decision Lens fixed all that. “The primary advantage was that we didn’t spent nearly as much time trying to come up with the weighting of things,” DeAngelo said. “We spent more time talking about the actual content of the decision, because the process in the tool recognizes that everyone has a different opinion on things and that’s OK. We just average them out and everyone has a chance to speak their mind.”
The first year the tool was used, many agencies resisted, DeAngelo said, but that stopped when they saw how the legislature reacted to the prioritized list made by Decision Lens.
“The legislature saw our list, and they made the decision about what they were going to fund or not fund and there was about an 85 percent alignment between what was at the top of our list and what the legislators ended up funding,” DeAngelo said.
Based on those results, agencies are paying much more attention to the criteria this year, he added. “It’s been instrumental in influencing the approach that agencies are taking as they are building their systems.”
For Washington, Decision Lens turned out not to just be a decision making tool, but a communication tool, too. Getting agencies to take a cloud-first approach has been a struggle in past years, DeAngelo said, but once they saw that including the cloud in their funding requests would gain them a few points, and they could see in an intuitive graphic how that helped their chances, agencies started adhering to those kinds of directives.
“One of the legislators fell in love with the tool and he thinks it should be used in all of their decisions,” DeAngelo said. “It had a lot of credibility in the finance communities because it’s very transparent, people know in advance what the criteria are, and the level of participation is extremely high in the process.”
Steve Allen, strategic transportation investments director for the Tennessee Department of Transportation – calls Decision Lens a powerful tool. The department has used Decision Lens for investment planning since 2013, and it's been valuable for guiding capital investments, he said, but it can't replace intuition and experience.
“For instance, there are values you can’t really put a measurement on,” Allen explained. “There’s not really a way to know you’ve got three projects already underway on that one route and you don’t want to add that other project even if it’s a really good project just for the simple constructability. There are other things like that that you couldn’t really place a score on that and be fair. So you try to pick metrics that are justifiable and quantifiable for every project.”
Although Allen says the department could make decisions without it, Decision Lens is valuable for providing transparency.
“It gives us the ability to let the legislative body, the local officials, the organizations like our metropolitan planning organization – it lets them know we actually have a methodology that uses data to include a value to a project,” Allen said. “We’ll probably continue using it for the rest of this administration. Every administration comes in and evaluates how you do business and this administration is really very business-oriented.”
Some legislatures, like the one in Washington state, have access not just to the prioritized list output by the tool, but access to the tool itself, so the output can be adjusted and filtered based on differing priorities. Some legislatures don’t want to take the Mayor’s priorities into account, for instance, so Decision Lens allows the user to quickly play with the output.
The company reports that a Decision Lens license costs about $200,000 to $300,000 for a state agency, and the cost is based on the size of the portfolio budget the tool is being applied to.
Saaty said Decision Lens could be reworked and sold to the public for making personal decisions -- everything from which car to buy to what career to pursue could be decided based on a list of weighted selection criteria provided by users. The company doesn't market the product to consumers, however. Decision Lens uses its own software to shape its business model, and the software says entering the consumer market is a bad decision.