MetroLab’s Executive Director Ben Levine sat down with Karen Lightman, executive director of Carnegie Mellon University’s Metro21; Michael Madaio, a Ph.D. student in CMU’s Human-Computer Interaction Institute; and Thomas Cook, assistant chief for operations at the Pittsburgh Bureau of Fire. For more information, please contact MetroLab at info@metrolabnetwork.org.
Image 1: Burgh's Eye View: An interactive map that displays high-risk commercial properties in Pittsburgh to inform fire inspections. Image courtesy of Michael Madaio, Carnegie Mellon University
Ben Levine: Could you please describe what Metro21 is? How are projects like Fire Risk Analysis incorporated into the initiative?
Karen Lightman: Carnegie Mellon University’s Metro21: Smart Cities Initiative seeks to research, develop and deploy 21st-century solutions to the challenges facing metro areas. This approach leverages the research and development engine of the university and couples it with the “real-world” laboratory of the city, where technologies and methods can be tested for viability and scalability. The Fire Risk Analysis led by Michael Madaio — one of many projects under way at Metro21 — is a great example of Carnegie Mellon technology being applied to solving a real-world problem. Thanks to this work, the Pittsburgh Bureau of Fire can better deploy its resources, resulting in reduced fire damage and potential loss of life. This is good work at its best.
Ben Levine: Can you describe what the Fire Risk Analysis Project is? Who is involved?
Michael Madaio: The Fire Risk Analysis project is designed to help the Pittsburgh Bureau of Fire better prioritize their commercial property fire inspections by developing predictive models of building fire risk. I’m a Ph.D. student in CMU’s Human-Computer Interaction Institute, and I was the project lead of a team of two master’s students in CMU’s Heinz College, Bhavkaran Singh and Qianyi Hu. We worked closely with Lt. Jason Batts at the Bureau of Fire and Geoffrey Arnold at the Pittsburgh Department of Innovation and Performance, who helped us acquire and understand the data to most effectively address the bureau’s needs.
Levine: What is the project focused on? What motivated you to address this particular challenge?
Madaio: Municipal fire departments conduct regular fire safety inspections of commercial properties. Unfortunately, due to their capacity, most cities are unable to inspect every property and must make decisions about which properties to prioritize. As seen in some recent high-profile fire incidents, such as the “Ghost Ship” fire in Oakland and the Grenfell Tower fire in London, some high-risk properties fall through the cracks, often due to a lack of inspection capacity or insufficient data sharing across municipal agencies.
To better inform the prioritization of fire inspections, the Fire Risk Analysis project has developed a predictive model of the likelihood of a fire incident to occur in a commercial property in a given year using historical fire incident data from the Bureau of Fire together with commercial property data from a variety of municipal agencies. That likelihood is converted to a risk score for each commercial property, which has been deployed in an interactive map visualization used by Pittsburgh fire inspectors to assist in their planning and prioritization of inspections. Using these fire risk scores along with fire inspectors’ expert knowledge, we hope to reduce fire risk and improve fire safety in the city of Pittsburgh.
Levine: How has the Bureau of Fire been involved in the project? Why did you decide to partner with Carnegie Mellon University and what are you hoping to achieve?
Thomas Cook: The Pittsburgh Bureau of Fire is seeking operational efficiencies through the use of data to assist day-to-day decision-making. We plan to assess the applicability and accuracy of the model over time and are currently exploring how to integrate the data generated into our operations.
Levine: I know that students were involved with this project. Can you talk about the experience of getting students engaged in a real-world challenge?
Madaio: We work closely with students from Heinz College’s master’s of information systems management, who take a variety of courses in data analysis, machine learning and data visualization. A project like this allows these students to take the skills and methods they’re learning in their classes and apply them to an end-to-end problem working closely with a partner organization, much like they will experience on the job market. Unlike the “toy problems” often used (for good reason) in classes, there are unique challenges of working with data in a real-world setting.
For example, students must understand the workflow, policies and context around the resulting available data, since that will help them understand, for instance, what it means when data is missing, or when there are apparently erroneous values in the data. This impacts every part of the data science pipeline, from cleaning the data, to joining it with other data sets at an appropriate level of granularity, to the analysis and evaluation of the performance of the predictive model, to, of course, its implementation into a tool that our clients will actually use in their operations. Finally, having students negotiate the needs of the client with the reality of the data and the results from the model is an authentic educational experience that most classes can’t offer. Here, our students were able to experience the data science process as they might encounter it in the workplace when they graduate, and will have gained invaluable experience in understanding the needs of a client and developing models and tools that will be useful for them.
Levine: What was the most surprising thing you learned during this process?
Madaio: I was most pleasantly surprised by the willingness of our partners at the Bureau of Fire to dive into the data and use the results to inform their existing processes. As predictive models contain some uncertainty, we had a number of stimulating, enriching discussions with the bureau about the usefulness and limitations of predictive modeling, and effective ways to integrate risk scores into their existing departmental decision-making.
I was also pleasantly surprised at the level of data preparation and sophistication already in place at the Department of Innovation and Performance. Not every city has such a robust set of open data able to be analyzed and joined with data sets across municipal agencies, and it made that part of the data analysis pipeline go so much more smoothly than it might have otherwise.
Finally, I was surprised by how many of the results of the predictive model aligned with existing initiatives from the Bureau of Fire. For instance, some of the most important “predictive features” in the risk model included property types (e.g., high-rise apartments, senior living centers) already targeted by the bureau for priority fire inspection for risk reduction. This was a great indicator of the “face validity” of the model above and beyond more traditional machine learning model performance metrics, and confirmed the utility of the model in helping prioritize individual properties within those larger sets of property types.
Levine: Where will this project go from here?
Madaio: At this point, the model is running on the Bureau of Fire’s servers, updating with new data and “re-training” periodically, visualizing those risk scores in an interactive data dashboard and interactive map. We now have a team of students working to incorporate new data sources (e.g., “tax lien” data and other property data) to improve the model, as well as conducting various machine learning experiments with new model types that can better represent time in the model (e.g., how long since a violation occurred, etc.). We are also in the process of extending the risk model from commercial properties to single-family residential properties that make up the bulk of the properties in the city of Pittsburgh. Although the Bureau of Fire does not inspect residential properties, they do conduct regular “community risk reduction” efforts, such as fire safety education or smoke alarm awareness campaigns. With a risk model that identifies the census blocks, neighborhoods or communities at greatest risk of fire incidents, the bureau can better inform the strategic planning of their community risk-reduction initiatives. We also have provided an extensive technical report detailing our process on the Metro21 project page website, as well as made the code for the model open source on the city of Pittsburgh’s Github page, so that other municipalities can take advantage of this project to help their fire departments better understand fire risk and improve fire safety in high-risk properties in their cities.
About MetroLab: MetroLab Network introduces a new model for bringing data, analytics and innovation to local government: a network of institutionalized, cross-disciplinary partnerships between cities/counties and their universities. Its membership includes more than 35 such partnerships in the United States, ranging from midsize cities to global metropolises. These city-university partnerships focus on research, development and deployment of projects that offer technologically and analytically based solutions to challenges facing urban areas, including: inequality in income, health, mobility, security and opportunity; aging infrastructure; and environmental sustainability and resiliency. MetroLab was launched as part of the White House’s 2015 Smart Cities Initiative. Learn more at www.metrolabnetwork.org or on Twitter @metrolabnetwork
