Police departments already have a lot of data available that could be helpful: use of force incidents, citizen complaints, praise of officers from other officers or the community. But making sense of that data is not always easy. For example, if two officers use force during stops twice in one month, are they both performing the same — and at equal risk of having a serious incident in the future? What if one officer made dozens of stops during that time and the other only made two — using force both times? Does it make a difference if they were patrolling in different areas, or if one was on the night shift and the other working during the day?
To sort through the data and draw the right conclusions from it, some departments are turning to performance management systems — also called early intervention systems — for data analysis.
Early intervention systems “don’t start from good beginnings,” said Brian Christenson, vice president of public sector solutions for Sierra-Cedar, which works with police departments to customize and implement its system, called Insight. Many times, departments create them as part of a consent decree with the U.S. Department of Justice after an investigation into problems. “But they set the stage for improvements.”
How Intervention Systems Work
The basic idea of an early intervention system is to take data from multiple sources — often about a dozen, though the Seattle Police Department just started using a system that draws on 17 different data sets — and analyze them so users can easily see unusual patterns.“Every piece of data around an officer’s interactions or job comes together and forms the foundation for what an early intervention system does,” said Garth Strandberg, vice president of justice and public safety solutions with Sierra-Cedar.
The Seattle system, for example, looks at data that comes from sources ranging from human resources to internal affairs. It integrates with the computer-aided dispatch system and tracks arrests, training and other incidents. The Seattle Police Department worked with Accenture to develop this data analytics platform, which was finished in January. The system followed a consent decree with the federal government, which required that the city proactively address the use of force by police officers.
“We’re looking at the whole picture of the officer,” said Jody Weis, Accenture’s director of public safety for North America. “That’s good for the officers, and I think it’s really good for the community.”
Once the system has the data, the key is to analyze it. How many times has an officer used force in the past six weeks? That number may be higher for a member of a SWAT team, for example, than for someone who patrols a relatively peaceful neighborhood. Tuning the system to account for these differences is key to making it work.
“It looks at the outliers,” Strandberg said. “Do we have an issue with an officer that requires training? The idea is to engage with the officer before it becomes a punitive action.”
For example, the system the Los Angeles Police Department uses analyzes each officer’s performance each night, sending an electronic action item to the supervisor’s inbox if an officer’s performance appears to vary from that of other officers doing similar work.
The current version of the LAPD’s system, which Sierra-Cedar put together and is called TEAMS II (for Training, Evaluation and Management System), went live in 2007. The system was originally mandated by a consent decree between the department and the U.S. Department of Justice in 2001.
The LAPD system looks at five issues: use of force, complaints, collisions, pursuits, and claims and lawsuits. It looks at how often an officer is involved in these incidents compared with other officers, as well as how often these occur compared with how many stops and arrests they make.
“We have guidelines that supervisors go through for the analysis,” said Cmdr. Regina Scott of the LAPD. Supervisors must determine whether there is a pattern of conduct that needs further action. “Everyone is looking at the same things in creating their evaluation.”
The analysis can end in different ways, including no action, counseling, or initiating disciplinary action. The supervisor then discusses the issue — and its disposition — with the officer involved, and includes that conversation in the final analysis, which is then reviewed by several higher levels of command staff.
Different departments may start out with different data sets and look at them in different ways.
For example, Seattle’s system identifies each officer’s chain of command on any given day, Weis said, so if an officer’s results are different depending on the supervisor, that will become clear.
The system charts the places and times of day when force is most likely to be used. It also tracks demographic information for officers and offenders, so racial patterns will become apparent. And since it also tracks training, it can identify if an officer is missing critical training.
In the system Sierra-Cedar built for the New Orleans Police Department, which has had the company’s Insight tool fully implemented for a few months as part of its response to a consent decree, a supervisor dashboard allows supervisors to “look at everybody in their command in 30 seconds and get a summary of what they have done over the past 24 hours,” Strandberg said. “The supervisors are where change happens and where it affects the people on the street, and giving them the right tools is probably the most impactful thing the system does.”
Overcoming Challenges
Although early intervention systems can provide valuable insights, departments face many challenges in setting them up and using them effectively:- Starting with the right data. “It’s kind of junk in, junk out,” Weis said. If names have not been entered in the same format in the past, for example, it may cause problems for the new system. “You can have the greatest analytical tools in the world,” but if the underlying data is incorrect or incomplete, the conclusions may be wrong. And departments may use these conclusions to drive policy.
The LAPD, too, put in a “significant effort” to gather all the data to import it into the system, Strandberg said.
And in New Orleans, the department realized that it would need to upgrade some of the databases, including the training and payroll systems, that would feed into the system. “The information had to be accurate and clean,” said New Orleans Police Superintendent Michael Harrison. The department also had to digitize some historical information to enter into the system.
Once the data is cleaned up, it’s important to develop policies to ensure future data entry is done consistently.
- Getting buy-in. Officers may initially be suspicious of a system that is used to track disciplinary issues. And for the system to reassure the community, the public needs to understand how it works.
The New Orleans Police Department did not want its officers to view the system as punitive: “It is not a disciplinary management tool, it is a human resource management tool,” Harrison said. “It is designed to alert supervisors when we see trends that need some type of supervisory intervention.”
The New Orleans system includes commendations and positive comments from the public, and it gives individual officers self-service access to information in their personnel files. This gives them “something of value that they weren’t getting before,” Christenson said.
Communication was also key — both with officers and with the community.
“We spent a lot of time working on messaging,” Harrison said.
In Los Angeles, there were concerns at the beginning that the system would generate extra work for supervisors, Scott said. But after the system had been in use for a couple of years, “what you saw was that it was a tool that assisted supervisors. It kept them focused. And the officers feel that we are now more of a transparent organization, an organization that holds its people accountable.”
- Making the right comparisons. The system will not produce valid results — and will make officers feel it is unfair — unless it compares officers to others doing similar work. “Analytics are built into the system to make sure we’re comparing apples to apples,” Harrison said.
Reaping the Benefits
Because many external factors change at the same time that performance management systems are implemented, it can be difficult to quantify the results of using the systems.However, departments using the systems cite benefits in a number of areas:
- Early intervention. Before LAPD had its early intervention system in place, supervisors were often unsurprised when they heard about officers getting in trouble — but nothing had been done to keep issues from escalating, Scott said. “Now what you see is a department that is very proactive. [The system] allows the command staff to move people or change their assignments – you can do that before an officer gets into any type of risk-based problem.”
- Staff retention. “It’s expensive to train an officer,” Strandberg said. “If you can intervene with training or communication or counseling — so your first conversation with that officer isn’t when they’re being terminated — that’s a metric they’re concerned with.”
- Better decision-making. The system “is making us a 21st-century police department, where we can know how our people are performing and our decisions are based on science and data,” Harrison said.
When officers are being considered for new assignments or promotions, everyone involved wants to be sure the decisions are fair. Without good data available, it was easy for those decisions to be based on memory, not on accurate data. “Now, we’re actually comparing apples to apples and not making those decisions in a vacuum,” Harrison said. This helps ease officers’ concerns about fairness and favoritism in making these assignments.
- Improvements in culture and management. Seattle Chief of Police Kathleen O’Toole said in a statement, “This integrated platform has improved our ability to track use of force and officer performance across multiple measures. These analytics not only promote accountability, but also enhance police operations by supporting proper management of personnel —helping SPD officers deliver high-quality policing to the communities they serve.”
In Los Angeles, Scott said, the system “has changed the culture of our department. We’re all on the same page; we’re all looking for risk issues.”
- Public trust and communication. In addition to being able to tell the public that the department has an early warning system to help identify officers with problematic behavior, the New Orleans system now has more accurate data available. “As a result, we have been able to release a number of data sets to the public, resulting in public engagement,” said Lamar Gardere, New Orleans CIO. “It’s increased transparency and increased accountability.”
Scott said that in Los Angeles, too, the system has increased openness. “We’re more professional, more open, and because of that we have built the public trust within the community.”
Public trust is critical because of how life-changing an encounter with the police can be.
“Police officers have the authority to take away your liberty, and they can under certain circumstances have the authority to use deadly force,” Weis said.
The Future
Early intervention systems are expandable and modifiable, and departments continue to work on them. They can add new data sources, for example, or tweak the way the system identifies trends that need action.“We constantly look at our assessments,” Scott said. Are there different ways to measure performance? Should they tweak the way they calculate when an action item gets triggered? “We’re looking to make the system even better with the future iteration of TEAMS III.”
