Chicago Pioneers Open Source Analytics Platform

Funded with a $1 million grant from Bloomberg Philanthropies, Chicago’s SmartData project will build the first open-source, predictive analytics platform – aggregating and analyzing information to help leaders make smarter, faster decisions and prevent problems before they develop.  “The Mayors Challenge is all about finding great ideas that can spread to other cities,” said Jim Anderson of Bloomberg Philanthropies. “While several municipalities are working to harness the power of big data, Chicago will be the first city to do so open source, making it possible for this great idea to spread and empower other cities.” More information on the Bloomberg Philanthropies Mayors Challenge can be found at bloomberg.org/mayorschallenge.

The SmartData Platform

Chicago collects 7 million rows of data a day — data on everything from weather to traffic patterns to the location of libraries, schools, sidewalks and public parks. But this abundance of data in itself can't solve urban problems. Most of the data exists in separate systems, often in conflicting and confusing formats. So how can government managers use the data to make better decisions? They need a tool to help them collate, sift and analyze. They need the SmartData platform.

SmartData will give leaders a tool to search for relevant data and detect relationships, analyzing millions of lines of data in real-time. This will help make smarter, earlier decisions to address a wide range of urban challenges. Chicago residents will experience services delivered earlier, sometimes even before the problem is apparent. Officials will be able to target responses that will address a wide range of urban issues — from managing weather emergencies to reducing traffic
accidents. The SmartData platform turns "thinking" into "doing." It turns "reactive" into "proactive." At its core, it makes data-driven and effective government the norm and can fundamentally alter the way the city operates.

“Continuing to expand the use of analytics across our city is a top priority,” said Chief Information Officer Brenna Berman (pictured at left), who is overseeing the platform’s development. “With the SmartData Platform, we’ll not only be able to expand analytics, but develop a new method of data-driven decision making that can change how cities across the country operate.”

The Platform in Action

The project has two goals — first, to help city managers analyze trend data and engage in predictive problem-solving, and second, to share the platform with cities that cannot build the capacity themselves.  All software developed on this project will be open source and will be made available to other cities.

SmartData will allow policymakers to make sense of the city’s billions of lines of data stored in disparate systems. Managers will be able to find answers to specific questions without having to manually search for data, or even know where or how the data is stored. End users will not see the millions of lines of code, or the billions of records stored and searched for their benefit.  All a user will see is a simple search screen to perform the query. Results will be presented in easy-to-read formats, including geographical plots, customizable to the user’s preferences (police beats, school
districts, sanitation districts, etc.).

The predictive power of the tool is its ability to analyze relationships in the data at a speed and on a scale not previously possible. For example, the SmartData Platform could query data on traffic patterns and pedestrian activity for a certain section of the city, and then compare it against other city data, such as weather patterns, traffic signal times and streetlight access. By doing so, SmartData might develop a prediction of where intervention is needed to reduce pedestrian-traffic collisions. The city could optimize services of all kinds in this way, benefitting citizens and reducing
costs.

Enhancing City Operations

The first stage of the SmartData project is now complete. The first tool, called WindyGrid, presents a unified view of operational data for all public safety agencies on a single dashboard.  Varied types of data are displayed in a user-friendly single graphical interface, allowing the user to make queries and also receive automatic updates and alerts.

WindyGrid includes more than a dozen different types of data, including geospatially tagged 311 reports, 911 calls and public Tweets, emergency operations data, video feeds from surveillance cameras, and city bus location data.  Users can query data by type, time and distance from a given location and determine how they want results displayed, including a heat map to show concentrations of results.   

A new pilot project focused on rodent baiting is demonstrating how Chicago can apply predictive analytics to core city services that affect quality of life in the city’s neighborhoods.  Rodent complaints are one of the city’s top ten 311 inquiries. Traditionally, the city has reactively deployed rodent baiting teams in response to complaints or events, such as water main breaks, that are anecdotally linked to rodent activity. While Chicago’s 311 call center processes more than
600 types of calls, the pilot identifies and analyzes 31 key call types that can predict rodent activity spikes 7 days in advance. By doing this, the pilot will allow the City’s rodent baiting teams to deploy in an earlier, more targeted fashion to better prevent rodent outbreaks. Initial pilot results are promising, and have included the detection of rat infestations that would never have been found without the new algorithm. The pilot’s final evaluation will be based on both efficiency improvements and reduction in rodent complaints.

Building a Replicable Model

Any city seeking to create its own SmartData platform will benefit from the groundwork done in Chicago. Chicago is building an open-source data infrastructure and a set of algorithms that other cities can re-use with no startup software development costs. Other cities will be able to import the architecture and the predictive algorithms (both logic and source code) developed by Chicago and adjust it to accommodate their own 311 or related data sets. The technology team in Chicago is also creating an archive of instructive documents and templates that will give other cities a roadmap to develop their own predictive analytics projects.

What does the Mayors Challenge victory mean for Chicago? “Bloomberg’s support is crucial to helping us build the SmartData Platform and make it available to other cities,” Berman said.  “It reflects what is at the heart of this initiative—our commitment to make our city smarter and a better place to live for our residents.”
 
This story was originally published by Data-Smart City Solutions.