It’s unclear whether bad actors accessed the information collected by 211 Los Angeles County, which included an estimated 396,000 contact emails and 33,000 Social Security numbers, but the data has since been protected and its access restricted.
But those familiar with the incident, which took place earlier this spring, said it’s a reminder that local agencies need to be vigilant as they guard large amounts of public data collected by government and affiliated entities.
The incident involving 211 Los Angeles County, which operates a 24-hour hotline, came to light on March 14, according to a report from UpGuard, a cybersecurity company in Mountain View, Calif., whose cyber-risk team spotted the issue. The company had “no pre-existing relationship with L.A. County,” its PR Director Kelly Rethmeyer told Government Technology. She characterized the discovery as “basically an independent research project” by a team whose “whole mission is to find active data sets in unsecured websites.”
The finding, by Chris Vickery, director of UpGuard’s cyber-risk team, revealed “an Amazon Web Services (AWS) S3 cloud storage bucket at the subdomain ‘lacounty’” which had been “configured to be publicly and anonymously accessible.”
Not all files were publicly downloadable, UpGuard said in its report, but those that could be “included Postgres data backups and CSV (comma-separated values) exports of that data, with hundreds of thousands of rows of sensitive personal information.”
The company cited the 501(c)3 nonprofit’s “dedication to preserving the confidentiality of reports,” and characterized what appeared to have occurred as a “technical misconfiguration,” which it defined as “an internal problem that emanate(s) from within the information technology (IT) infrastructure of any enterprise.” No hacker or hack is necessary for “massive damage to occur,” UpGuard said, citing a Gartner estimate that from 70 percent to 99 percent of data breaches result from internal misconfiguration.
In a statement to GT, Los Angeles County CIO Bill Kehoe said the agency “has determined that certain personal information maintained by its 211 vendor on a cloud storage repository was vulnerable to a possible breach.”
“Once alerted to this issue by an outside security firm, the county promptly directed that access to that information be blocked. The county has a well-earned reputation for its aggressive commitment to protecting personal data from outside intrusions, whether that information is maintained by the county or its vendors,” Kehoe said.
“There was no compromise of their systems. This was not white hat hacking. There was no vulnerability exploited,” Rethmeyer said. “Someone misconfigured their cloud storage so it was publicly accessible. It’s extremely easy to make that mistake, where someone clicks a button and all of a sudden those records are accessible.”
Once the issue was identified, UpGuard took steps to notify county officials “immediately,” it said, first contacting 211 and emailing a “recommended contact,” and ultimately reaching a “member of information security.”
“Our contact at L.A. County 211 assured us the problem would be taken care of, and in less than 24 hours, UpGuard confirmed the bucket itself was no longer publicly accessible,” the company said.
Several CSV files in the bucket had personal information critical to 211 service, UpGuard reported, including names, email addresses and weakly-hashed passwords for 384 users. Nearly all those email addresses were for the @211LA.org domain, meaning that if passwords were decrypted, the 211 accounts could be vulnerable along with any individual reuse of those passwords.
Most of the data it discovered, UpGuard said, came from one file that contained “a massive amount of personally identifiable information (PII)” including actual call notes for more than 200,000 calls between 2010 and 2018.
The company described 211 as “a top of the funnel operation,” and pointed out that while centralizing reports and technology enables administration, collaboration and efficiency, it can also increase risk.
“If this dataset is not carefully handled, the magnitude of exposure is far greater than if it occurred at any of the more specialized links down the triage chain,” UpGuard said in the report, adding that the issue isn’t unique to 211 but one confronting “all organizations” using the cloud and Internet applications to store and process data.
“That’s what we’re trying to do is spread awareness of how pervasive this issue is. That’s the mission of this team,” Rethmeyer said. The lesson for governments, she said, is to be inwardly and outwardly vigilant, and mindful of how third-party entities are handling data.
“In today’s day and age, especially with the cloud, the transparency of data at this point, it’s very pervasive. Companies need to be more vigilant in making sure their business partners are taken care of,” she added.
That’s exactly what’s next for L.A. County, Kehoe said.
“In this case, the county will be closely monitoring strong assurances from the vendor that it has strengthened its data safeguards, as well as its policies, protocols, processes and oversight to avoid any future exposure of sensitive information,” the CIO said.
