These changes are disrupting numerous sectors where businesses used to create competitive advantages for themselves based on access to superior IT or exclusive data. With these advantages slipping away, one of the primary differentiators for organizations in data-intensive sectors will be access to talented, data-literate workers. This is particularly true for government agencies. To improve public services with data-driven technology, they’ll need to work harder than ever to recruit, hire and retain highly skilled data engineers and managers.
To understand the relative importance of these skills, consider the degree to which cloud architectures have commoditized computing. While Moore’s Law — the observation that the number of transistors on a chip will double roughly every two years — predicted the modern digital era of smartphones, tablets and wearables, the recently proposed Bezos’ Law — the observation that the cost of a unit of computing power in the cloud is reduced by 50 percent every three years — predicts that the cost of computing will eventually be non-limiting for most organizations.
Cloud computing has not only reduced costs, it also has given organizations flexibility to adapt their computing infrastructure to changing needs. This has democratized access to the latest technologies, putting new entrants on the same footing as long-established incumbents. When everyone has access to a Stradivarius, talent matters.
The emergence of big data has had a similar effect. Relatively cheap and plentiful access to massive amounts of information has begun to erode the strategic advantage that organizations with a data monopoly might have counted on in the past. Some of these advantages will remain indefinitely — Netflix, for example, knows more about the viewing habits of its customers than anyone else — but competitors now can mine other data sources to narrow this gap. Therefore, organizations can no longer rely only on proprietary data to stay ahead and must instead compete on talent.
Yet while access to the most talented, data-literate workers increasingly defines which organizations win in the data economy, there are too few qualified people in the labor market. McKinsey Global Institute has estimated a shortfall of 140,000 to 190,000 data scientists by 2018, as well as an even greater shortage of managers with the analytical skills needed in a big data world. Employment in data-intensive industries is geographically concentrated in certain states, putting other states at a disadvantage. This challenge will be especially pronounced for government agencies, which already have a problem recruiting the best and brightest from the private sector.
There are no easy solutions. Two well-known factors affecting employment decisions — compensation and culture — require flexible budgets and organizational change, neither of which plays to government’s strengths. But government should not give up. The UK’s Government Digital Service fundamentally rebuilt the nation’s public-sector strategy for IT, proving that disruptive innovation in government is possible.
Moreover, government agencies do have an advantage in that many of the problems they’re working on — like increasing access to affordable health care, improving the quality of schools, and making cities safer and cleaner — are the types of problems that attract the sharpest minds. While they may not be able to match the pay or benefits of Silicon Valley, they offer the chance to improve the world. In particular, this may appeal to millennials who prioritize working for the betterment of society over obtaining wealth. (Sorry, Dunkin’ Donuts, cutting infant mortality is more exciting than selling more doughnuts.)
In the long term, policymakers must fix the workforce pipeline so that skills better match employer needs in the private and public sectors. But in the short term, governments will be in fierce competition with the private sector for the best data scientists. They’ll need to use all available resources to bring in the human capital that can ensure the opportunities from the data revolution don’t pass them by.
