Data-informed predictive policing was heralded as less biased. Is it?
“Dirty data” makes for biased predictions
The technology is “perpetuating and, in some cases, concealing some of the biased practices in the police department,” said Rashida Richardson, director of policy research atAI Now, a New York University institute exploring the effect of artificial intelligence on society, which has looked at predictive policing programs around the country.
“Dirty data,” she said, makes for bad predictions.
Police data is open to error by omission, she said. Witnesses who distrust the police may be reluctant to report shots fired, and rape or domestic violence victims may never report their abusers.
Because it is based on crime reports, the data fed into the software may be less an objective picture of crime than it is a mirror reflecting a given police department’s priorities. Law enforcement may crack down on minor property crime while hardly scratching the surface of white-collar criminal enterprises, for instance. Officers may intensify drug arrests around public housing while ignoring drug use on college campuses.
Recently, Richardson and her colleagues Jason Schultz and Kate Crawford examined law enforcement agencies that use a variety of predictive programs.
They looked at police departments, including in Chicago, New Orleans, and Maricopa County, Ariz., that have had problems with controversial policing practices, such asstop and frisk, or evidence of civilrights violations, including allegations ofracial profiling.
They found thatsince “these systems are built on data produced during documented periods of flawed, racially-biased, and sometimes unlawful practices and policies,” it raised “the risk of creating inaccurate, skewed, or systemically biased data.”
A seminal study from a few years earlier shows how this could play out with a specific type of crime—in this case, drug crime.
Using a simulation, researchers Kristian Lum, now at the University of Pennsylvania, and William Isaac, now at DeepMind, an artificial intelligence company, looked atwhether PredPol’s predictions would disproportionatelysteer officers into minority neighborhoods in Oakland, Calif.
Lum and Issac had estimated drug use in the city based on the results of a national survey and found it would likely be fairly evenly distributed across the diverse city.
Yet when they fed arrest data into PredPol’s publicly available algorithm, they found the hotspots it would have predicted were not at all evenly distributed.
The researchers found the algorithm would have sent police to Black neighborhoods at roughly twice the rate of White neighborhoods. The tool spat out risk areas where minorities lived—based not on drug use, but rather on places with prior police-reported drug crimes.
“Over-policing imposes real costs on these communities,” Lum and Issac concluded in their 2016 report. “Increased police scrutiny and surveillance have been linked to worsening mental and physical health; and, in the extreme, additional police contact will create additional opportunities for police violence in over-policed areas.”
Companies say claims of bias need careful attention
PredPol did not respond to requests for comment. But the company’s founders, UCLA anthropology professor Jeffrey Brantingham and George Mohler, a computer science professor now at Indiana University–Purdue University Indianapolis, have done their own study, which they say indicates their methods do not result in higher minority arrest rates.
In2018, they published a paperthat began with acknowledging their critics, including Lum and Issac: “Though all of these studies deal with hypothetical scenarios or thought experiments, they succeed in demonstrating that careful attention needs to be paid to whether predictive policing produces biased arrests.”
Brantingham, Mohler, and their co-author, Matthew Valasik, conducted a real-life experiment with the Los Angeles Police Department.
They compared the results between two groups: officers sent to PredPol-predicted crime hotspots and officers sent to hotspots picked by crime analysts, which served as a control group. They varied the groups each officer was assigned to day to day.
Brantingham, Mohler, and Valasik found therateof minority arrests was about the same whether the software or analysts made the predictions. What changed was thenumberof arrests in the PredPol hotspots. That number was higher.
The increase in arrests, they wrote, is “perhaps understandable given that algorithmic crime predictions are more accurate.”
They acknowledged, however, that their findings “do not provide any guidance on whether arrests are themselves systemically biased” and called for “careful policy development” to prevent bias in predictive policing.
Neither Brantingham nor Mohler responded to requests for comment.
Robert Cheetham, the CEO of Azavea, which developed Hunchlab, said it’s important to recognize the bias in the data, and that their software sought to geographically space out predictions so police patrols wouldn’t be so concentrated on one particular neighborhood. He said Hunchlab also included information beyond crime reports, such as lighting, major events, school schedules, and locations of bars—which all can relate to crime but are not based on a single source of potentially police-biased data.
The product also avoided forecasting certain crimes. Drug crimes and prostitution were out, as they are more heavily influenced by whom police choose to target. And they didn’t think the tools could be used for rape, which is often not reported and which isunlikely to be deterred by police patrol.
It’s unclear if this approach has led to more equitable patrol.
Some cities have scrapped programs
Beyond complaints about racism, some police watchdogs have another critique: the software may not be helpful.
Earlier this year, Chicago’s inspector generalpublished an investigationinto the in-house predictive policing models, the Strategic Subject List and the Crime and Victimization Risk Model, which are both designed to predict an individual’s risk of being a shooting victim or offender.
An earlier investigation of the Strategic Subject List by theChicago Sun-Timeshad found that 85 percent of people with the highest score were African American men and some had no violent criminal record whatsoever.
The Chicago Police Department told the paper that the scores were based on a variety of factors, including the number of times the person had been shot or assaulted and arrests for violence or drugs and gang membership, and that overall the risk scores helped police prioritize whom to issue warnings or offer aid to.
But the inspector general found the risk scores were unreliable and could potentially lead to harsher charging decisions by prosecutors.
Chicago ended upscrapping the program.
The LAPD alsoended one of its predictive policingefforts this year after its own internal review cast doubt on its efficacy.
The LAPD was the earliest adopter of PredPol, partnering with researchersa decade agoto help develop the technology. But the police commission’s inspector general, Mark Smith,foundthe data related to the program so unreliable it was difficult to draw conclusions.
“Unfortunately, the data was kind of messed up,” Smith told The Markup. When they went to measure if PredPol worked, he said, “we really couldn’t tell.”
A year after the report was released, the department announced it would bedropping PredPol.The announcement came mere weeks before the nation would erupt in protest after the killing of Floyd.
“The cost projections of hundreds of thousands of dollars to spend on that right now versus finding that money and directing that money to other more central activities is what I have to do,” police chief Michael Moore said.
This article wasoriginally published on The MarkupbyAnnie Gilbertsonand was republished under theCreative Commons Attribution-NonCommercial-NoDerivativeslicense.
