Open Data Success Requires Streamlining and Standardization

On President Obama's first day in office, he signed the Memorandum on Transparency and Open Government -- a movement that already has transformed the way government and citizens communicate with one another.

Shared government data sets present unlimited opportunity, said Yo Yoshida, CEO of the mobile commerce company Appallicious. But to really break open the potential of open data, streamlining and standardizing the data is key. 

“We haven’t begun to see the boundaries,” he said. “The public has very limited resources or access to government. Bringing that to the fingertips of people who are using this daily is creating a conversation and communication lines that weren’t there before.”

Opening Up Government Data

Making the data that government collects useful to citizens is more complex than one might initially think — it's not as simple as handing an Excel document to an application developer.

State Level Open Data, Sunshine-Style On Feb. 20, the Sunlight Foundation announced Open States, a portal to give citizens access to state-based government data such as names of state legislators and state legislation. “If you're interested in your state lawmaker, you'll be able to get notifications for their actions, a map of their district, voting records, committee assignments, campaign finance records from Influence Explorer, local news articles and contact information,” the organization writes in its blog describing the project. “If you're curious about a particular piece of legislation, Open States allows you to check on its status, find the sponsors, break down votes, view bill text and all supporting documents.” The Open States project, which took four years to create, also enables users to search for various categories of legislation across state lines, and will be updated over time. A large challenge in completing the project, according to the foundation, was obtaining all the information from the various state websites, where it’s located in different formats, and putting it all in a standard format.

National data standardization has yet to be established, so each city — and the various departments within cities — are using and tracking data differently. When companies, such as Appallicious, use government data to create applications for public use, they must determine how the data set is being used and how it can be imported.

In San Francisco, Chief Innovation Officer Jay Nath said the city is working with civic startup Socrata to leverage its open data products, ensuring that application developers can access data in multiple formats, such as CSV, XML or JSON.

“Our goal is to reduce friction,” Nath said, “so developers can focus on creating useful applications.”

San Francisco also is taking steps to establish open data standards. It led the effort to develop an application program interface for Open311. The application, now in more than 40 cities internationally, allows someone using a mobile device or computer to input information about a public service problem at a given location. This report is routed to the appropriate department, which can address the issue, and the information is available for anyone to access and contribute additional detail.

In the same vein, San Francisco Mayor Edwin Lee recently announced the city’s collaboration with Yelp to create an open data standard for restaurant hygiene scores. Called Local Inspector Value-entry Specification (LIVES), it allows municipalities to publish restaurant inspection information directly to a restaurant's Yelp page.

“I see a future where more and more companies partner with cities on developing open data standards and new solutions,” Nath said. “Through several hackathons, we’ve gathered hundreds of citizens to engage deeply with government in creating new solutions. We continue to grow our community and partners, and are excited to take open data to the next level with our proposed open data legislation.”

Part of the revision to San Francisco’s 2009 open data legislation would require each city department to identify an open data coordinator to establish plans, create implementation timelines and begin itemizing data collected. The creation of these positions would support companies, like the team at Appallicious, by providing data sets that are ready for use.

As a result of streamlined open data, Nath said he envisions applications that mash up data from multiple public and private sources to create a seamless experience. A tourism application, for example, could help visitors navigate the city via public transit, taxis, car or bike sharing, biking and walking.

“By unlocking civic data, we’re creating a new marketplace for developers and in turn making it easier to access civic information and services,” Nath said. “Open data is taking cities from the trailing edge to the leading edge. Citizens are using 21st-century technology in their everyday lives and yet their interface with government is stuck in the 20th century.”

The App Developers' Role

Application developers also must do their part to stay relevant. Rather than creating an app using a subscription-based model, Appallicious identified a more sustainable proposition — taking micropayments on transactions. The company developed San Francisco’s Recreation & Parks application, which provides public information on 1,200 department facilities. Each time a visitor transacts with the application, Appallicious receives a convenience fee as its payment for developing the app.

“Applications are not a static thing,” said Appallicious’ Yoshida, adding that the company is in its 20th iteration of the Recreation & Parks application, with another iteration coming in March. "We need to constantly be staying cutting edge, and we can’t do that if we just create one application and leave it for five years," he said. "For us to be able to take micropayments, it creates a sustainable environment so we can meet the demands of the consumer and stay up with technology.”

San Francisco has partnered with startup company Exygy to develop an application store to showcase all of the apps developed using open data. The city has a collection of applications organized into various categories — crime, dining, environment, local, maps, news, politics and transportation.

“This is the cusp of something so big that we can’t even put our heads around it,” Yoshida said. “It’s really going to change and revolutionize the world.”

Main image courtesy of cafwd.org