Beyond Duck-and-Cover

"The more we rehearse disaster, the safer we’ll be from the real thing. Life seems to work that way, doesn’t it?” This line from Don DeLillo’s novel White Noise satirizes an American obsession with preparation — and indeed, drills have long been an important feature of government’s emergency preparation work, from the duck-and-cover drills of the 1950s to the hurricane and wildfire evacuation drills that proved immensely important this last year.

However, at the Data-Smart Summit hosted by Harvard’s Civic Analytics Network in late 2017, city officials discussed an element of emergency drills that has long received inadequate attention. According to James McConnell, assistant commissioner for strategic data for New York City Emergency Management (NYCEM), the devastation of Hurricane Sandy in 2012 revealed flaws in the city’s data infrastructure and protocols. One reason for these problems? “The data component was not being fully tested in our drills,” McConnell said. 

Recognizing the inadequacy of its data infrastructure testing, the city began to implement a new type of preparation: data drills. Led by NYCEM and the Mayor’s Office of Data Analytics (MODA), data drills stress-test the city’s data protocols in an emergency.

Data drills may take many forms, but often consist of city officials and data experts gathering at a table, starting from a hypothetical emergency situation and proposing a series of response steps. Participants might ask questions about what data they would need to find, what agencies they would need to contact to access information and how they would use that data to resolve problems.

In these drills, “you want to ensure that there’s something specific you’re testing,” said Mitsue Iwata, project manager for MODA. Defining a piece of data infrastructure or protocol to address is critical to making the drill focused and productive.

However, the insights leaders derive from these drills aren’t limited to those questions they set out to test. Data drills are useful not only for testing the data infrastructure a city has put in place, but also for discovering potential gaps that the city hasn’t even considered.

At the summit, former NYC Chief Analytics Officer and Harvard Ash Center Fellow Amen Ra Mashariki used a story from his time in New York to illustrate the potential of data drills to reveal unknown information gaps. In 2015, following an outbreak of Legionnaires’ disease caused by infected cooling towers, MODA was tasked with identifying every cooling tower in the city. Because no database with this information existed, the project required a rigorous, time-consuming analysis of New York building data from various agencies. Naturally, the city saw fit to reform this process, and so MODA developed the Building Intelligence tool, a 360-degree reconciled database for buildings that could provide information more quickly and easily in the next emergency.

However, MODA’s work was not finished here. The city realized that in the next emergency, it would not simply be building data that agencies would require, but “something we don’t know we need,” Mashariki explained. He described this information as “unknown unknowns” — data that the government does not yet realize it doesn’t possess. “How do you collect data on anything that you might need?” Mashariki asked.

He pointed to data drills as an answer. When you simulate an emergency, you come upon challenges and data gaps that arise in real crises. As a result, you become aware of needs that you never would have predicted — and can then fill those needs before a real emergency strikes.

And it’s not only a matter of identifying useful data sets, but also of improving practices across the board in ways that you might have never thought necessary. “Data drills are a good way to bring agile practices into the city enterprise,” said Francoise Pickart, director of Risk and Analytics at the Department of Health and Mental Hygiene’s Office of Emergency Preparedness and Response. These drills constantly bring to the surface new problems and drive iterative design.

Data drills are a way for cities to more effectively prepare, informing investment not only in tools and processes that were needed in a prior emergency, but that also may be needed in a future crisis. According to Pickart, these drills teach cities “how to design for the next disaster, not the last one.”  

Chris Bousquet, a research assistant/writer at the Ash Center for Democratic Governance and Innovation at the Harvard Kennedy School, co-authored this column.