Predictive Policing Systems
How they work, the data they use, and who uses them
This document is a resource for understanding the current landscape of predictive policing systems in the United States. It is maintained by the staff at Upturn.
In our research to date, we’ve found that vendor marketing and general optimism are widespread, but objective empirical assessments of these systems are few and far between. Given that many law enforcement agencies appear to be adopting or considering these new systems, it’s important for everyone — including public officials, police leaders, advocates and community members — to engage with departments considering such systems. However, as the inventory shows, we simply need more information.
Too often, vendors make claims that are impossible or infeasible to verify, while hiding (and requiring their customers to hide) the information necessary to objectively assess these tools. And it remains hard to know which departments are currently using what systems. Though this inventory lists certain departments that are publicly noted to have used or tested a particular system, much of the adoption and testing of these systems is unknown to the public.
Apart from predictive policing, our team has worked extensively on one other policing technology: Body-worn cameras. Together with The Leadership Conference, we helped develop civil rights principles for body worn cameras, and have published a detailed national scorecard of body camera policies in the nation's largest cities. It's relatively straightforward to know whether, or how, a police force is using body cameras: they are physically visible to the public, every officer must be trained in how to use them (requiring some standard operating procedure), and the video that they generate intermittently appears in the media, in court, and other places where people will see it.
Predictive policing, by contrast, may be as simple as a new piece of software on a crime analyst's computer, in a central office. It may shape how patrols are deployed or how officers' activities are directed, but these changes are not necessarily visible to the public (or even, at times, to line officers themselves). Moreover, because predictive policing tools are often more software-based than hardware-based, they are relatively easy for a department to add, remove, or change. It is also easy for departments to try multiple systems in tandem, or in overlapping trials. These factors make it challenging to understand the landscape of predictive policing.
We need your help! Please feel free to get in touch if you have information to add to what’s here, about any of the dimensions we’re tracking: what data a system’s predictions are based on; where certain systems are being used; how certain systems are being implemented by a police department; or any relevant information about a system that’s not already described here. We can be reached at [email protected].
We are eager to hear from anyone — police, activists, community members, and the vendors themselves. If you are a vendor and can correct any errors found here or help us more fully describe how your system works, please do get in touch.
Additions, corrections, and feedback are welcome at [email protected]. Or, you can edit any page on github.
This information has been gathered from publicly available sources.
What is “predictive policing”?
There is no standardized definition, but a 2013 RAND report sponsored by the National Institute of Justice provides a useful starting point:
Predictive policing is the application of analytical techniques—particularly quantitative techniques—to identify likely targets for police intervention and prevent crime or solve past crimes by making statistical predictions.
Types of predictive policing, as identified in that report, include:
Methods for predicting crimes: These are approaches used to forecast places and times with an increased risk of crime.
Methods for predicting perpetrators’ identities: These techniques are used to create profiles that accurately match likely offenders with specific past crimes.
Methods for predicting victims of crimes: Similar to those methods that focus on offenders, crime locations, and times of heightened risk, these approaches are used to identify groups or, in some cases, individuals who are likely to become victims of crime.
Four primary findings
Basic Information is often unavailable.
Transparency is a problem. There are basic questions about any new predictive policing system:
- What is the system trying to predict?
- What data are the system’s predictions based on?
- What approach does the system use to make its predictions?
- How accurate are the predictions? (How is accuracy defined and measured?)
- How is the data analyzed?
- Which departments are using the system?
- How are departments using systems — in other words, what do they do with the predictions?
These questions are often hard to answer. While vendors may release marketing literature, one-pagers, webinars, or provide brief descriptions of their technology to the press, such sources often leave key questions unanswered, providing only a high-level overview of what their systems do or don’t do. For example, though many vendors say they use a police department’s historical crime data to make predictions, almost no vendor clarifies what specific historical crime data they use. (For example: Are arrests included in that data? Is the data set solely based on calls for service?) And while many systems aim to predict a type of crime at a certain place and time, few specify the exact universe of crimes their systems do or do not predict. (For example: Are drug crimes predicted?)
Large technology firms play a growing role.
Although freestanding, independently branded predictive policing systems garner significant public attention, larger technology vendors are playing a growing role in the predictive policing space — especially with their longstanding relationships with law enforcement. For example:
- IBM and Microsoft have each repurposed existing, large-scale data analysis products to serve predictive policing goals. (For example, Though it does not have a specific predictive policing technology that it’s selling to law enforcement agencies, Microsoft held a webinar in 2015 showing how its existing technologies could be used for predictive policing.)
- Motorola and Lexis Risk Solutions, two longtime leaders in providing information technology for law enforcement, have each acquired smaller firms that were focused on predictive policing (PublicEngines and Bair Analytics, respectively).
While several predictive policing systems trace their origin story to academia (PredPol, RTM, and Chicago’s Heat List/Strategic Subject List), most of the systems we investigated are the creations of private companies, and appear to have had little if any input from academic experts.
Most of today’s systems focus on places, not people.
Although there is significant public interest in new “person-based” techniques that attempt to predict who will commit crimes in the future (or in techniques that attempt to predict suspects in a case), we found that most systems in the field today are focused on geography, predicting the locations and times of future crimes.
Vendors define and measure accuracy differently.
In selling their systems to law enforcement, vendors make claims about their accuracy. These claims vary wildly. But a more important theme is that these claims lack specificity and relativity. Particularly, when vendors make a claim about accuracy, the most common reaction was “compared to what?” For example, one vendor claims to “predict more than twice as much crime in every city that we’ve looked at,” without revealing the baseline relative to which their product predicts twice as much. Simply put, vendors do not specify how they either define or measure accuracy, thus making it difficult to verify the claims they make.
What we did
We reviewed vendor promotional materials, research literature, videos/webinars, relevant scholarly literature, trade press directed at police and the vendor community, available public contracts and scope of work agreements between vendors and cities, or vendors and police departments,, local and national news reports, and relevant documents that have become public via FOIA requests.
Thanks
Change Log
You can look on Github to see exactly how this document has changed over time.
License
TK