The city is currently working to recover from the accidental March 2021 deletion of troves of police data. An audit into that incident unearthed a second unintentional deletion of police files — this time of 13.16 terabytes, according to the initial event analysis report, published in September.
The police department was impacted again in November, when aerial surveillance footage was leaked from one of its vendors’ systems
In its September report, the city’s Information and Technology Services (ITS) department detailed problems in ITS’s staff oversight and data governance and management practices. It stressed that the city’s increased use of unstructured data makes changes essential to prevent a repeat event.
“Without proper, fully implemented Data Governance in place, the city is at risk of further loss of data, inability to recover from onsite failures causing loss of data, disaster recovery requiring recovery of data, liabilities from inappropriate exposure of data, and inability to fully realize the analytical value of the data due to a lack of quality or inability to aggregate across departments and data sets,” the report noted.
ITS and the Dallas Police Department (DPD) work together to handle police data. The police offer “subject matter expertise” around data definition, collection and use and have oversight of who gets access to the data. ITS, meanwhile, helps ensure the department’s data use and storage are compliant with laws and regulations — including those governing personal identifiable information (PII) — and helps with DPD’s data storage and security.
THE FALLOUT
In the March incident, an ITS employee accidentally deleted around 22 TB of police data, and the city has since recovered roughly two-thirds of it.
That leaves 7.51 TB, or 4.1 million files, that the city believes are permanently lost. This came from the DPD’s K drive, which held general evidence as well as that “gathered by DPD detectives for prosecutable, adjudicated, on-going cases,” per the September report. The majority was for the Family Violence Unit.
The report also found a separate incident in which 13.16 TB of data, or 4.6 million files, were deleted from the police’s Fusion drive. This comprised digital evidence collected during “routine investigations,” ITS told Government Technology.
The city is continuing to search for any pieces of the missing data to reconstruct what it can. The district attorney has identified 1,000 cases to prioritize for recovery.
WHAT WENT WRONG?
The accident happened when ITS tried to migrate police data off a cloud server and to an on-site archive, to trim cloud expenses. Migrating sensitive data is known to be a risky practice, the report said, but the employee handling the request — and managers approving it — didn’t seem to fully realize this, which led to a failure to take appropriate precautions or follow best practices.
“Additional scrutiny was not placed upon the change requestor(s) to ensure changes could not cause grave harm to the city’s data or reputation,” the report said. “Technical changes hurried through the process with poor planning, scheduling, detail, and documentation do not identify all potential risk and are contrary to best practices or standards.”
The problem goes deeper than a few employees’ mistakes.
The report noted that the employee was using an administrator account that gave them more access privileges than they should have, which points to a need for better data management controls.
Some issues stem from a work culture that over-emphasized speediness, the report said: “ITS must improve it’s [sic.] environmental, managerial, and operational directives and documented expectations to engender quality delivery of service over current focus upon schedule dates.”
The department also lacks strong data governance and management procedures, the report said. The city’s data management strategy was not formally implemented, and, even if it had been, was out of date.
“By managing data through a best practice in an industry recognized manner, the risk of data loss would have been greatly reduced, posing almost zero risk to the city,” the report said.
Issues like these increased the likelihood of data loss and difficulties for recovering.
WHAT’S BEING DONE?
ITS proposed 13 recommendations for improving data management for the city, and is currently preparing a plan of actions and milestones that will attach specific timelines to each one, the department told GovTech in an emailed response.
Per the report, recommendations include having departments like DPD establish roles for “data stewards” to work with ITS’s data management team on ensuring departments are following data policies and getting their data management needs met.
ITS also should implement data governance and management control systems — a process now underway. By September, the city had selected a data management framework and established a steering committee charged with creating data management polices and standards, per the report.
The September document is also only the initial report, and further details will come as the city’s investigation continues.
LEAKED AERIAL FOOTAGE
Data concerns reared again in November, this time involving a DPD vendor.
On Nov. 5, the city learned that transparency-focused nonprofit the Distributed Denial of Secrets (DDoSecrets) had published “raw aerial video footage filmed from the Dallas Police Department’s (DPD) helicopters,” ITS told Government Technology.
DPD, also in an email, said that it “cannot confirm at this time how much video information was breached.” ITS said the city believes the figure is 1.8 TB.
DDoSecrets published a 600-hour, 1.9 TB data set containing aerial surveillance footage from both Dallas and Atlanta-area police helicopters, per its website.
The footage includes videos of crowds at the State Fair of Texas and Atlanta’s Mercedes Benz Stadium, as well as zoomed-in recordings of residents in their neighborhoods, according to Courthouse News Service.
Per the Dallas Observer: “Footage from the leak shows cameras following individuals at extremely close range as they go about their daily lives. In one sequence, the camera zeroes in on two men fixing a flat tire on Interstate 30. The text of their bumper stickers is legible… Other sequences show police aiming military-grade thermal imaging at apartment complexes, private residences and groups of people in their backyards.”
The incident has sparked concerns about why the footage was taken and how the sensitive information was secured.
An anonymous source provided DDoSecrets with the data, saying it was obtained from unsecured cloud infrastructure, DDoSecrets co-founder Emma Best reportedly said.
ITS said that a third-party vendor installs and maintains the digital recording systems used in DPD helicopters, and the leak impacted only those systems and their data.
“The exfiltration of data was limited to the third-party vendor’s system which contained no other DPD data beyond highly compressed, raw aerial video footage. This was confirmed to the city of Dallas by the vendor,” ITS said.
The DPD spokesperson emphasized that police still have access to the footage.
“It is important to note that this video data was not lost nor is it missing,” DPD said. “The department, city of Dallas IT Services, and the third-party vendor are working closely together in support of resolving this potential breach.”
