A recent Associated Press story on risk assessments, performed to determine the likelihood that someone involved in the criminal justice system will re-offend, contains several common misunderstandings. By taking a closer look at a few of these misconceptions, we hope to clarify some major points about risk assessment overall.
Misconception 1: Low Risk = No Risk
A common misconception is that people deemed to be at a low risk of reoffending have no risk of doing so. The AP report highlights stories of three individuals assessed as low risk who were ultimately rearrested for committing violent crimes. One story involved Milton Thomas, described by the AP as having been “in and out of Arkansas prisons since 2008 for nonviolent crimes, including check fraud.” Thomas told the reporters he had been evaluated low risk by the state of Arkansas.
Thomas is currently awaiting trial for the alleged sexual assault of a 70-year-old woman.
Assuming that Thomas actually was assessed as low risk (although the article seems to rely only on Thomas’s word only), it’s important to note that low risk does not mean no risk. In fact, we know with certainty that for every group of low-risk individuals, some portion will go on to reoffend, albeit at asignificantly lower rate than individuals in moderate- and high-risk groups. Risk assessments are absolutely, statistically better at determining risk than the old ways of doing things (e.g., basing decision on a “gut instinct”), which were often less useful than flipping a coin.
When risk assessments are performed on individuals, they really tell us what risk group someone belongs to, rather than their individual risk of re-offense.
Misconception 2: All Assessments Are Created (or Conducted) Equally
The AP story reports that “an Arkansas consultant said a majority of male parolees and probationers were classified as low risk by the state’s Department of Community Correction but 46 percent were rearrested within 18 months.”
Another common misconception is that all assessments are of equal quality and efficacy and are conducted with equal skill. If the AP story is correct and 46 percent of these low-risk individuals were rearrested, it is almost certain that the risk assessment tool is being used incorrectly or has not been validated on the population to which it is being applied. Or administrators may simply have set cut-off scores that are unlikely to create any meaningful distinction between low-risk and high-risk groups. There are many examples of states and jurisdictions where risk assessments are being used properly, are properly validated, and where low-risk groups have recidivism rates of 5 percent or less.
The AP story also states:
…before Thomas was released the parole board assessed him as a high risk to commit more crimes. But a second risk assessment, conducted by the state’s community supervision agency, found him to be a low risk, Thomas told the AP. Thomas said he has no recollection of answering questions from the lengthy survey.
Again, the story gets information directly from Thomas, but assuming the information is true, it seems likely the risk assessment was not conducted appropriately.
It’s important to note, too, that some assessments don’t measure general risk; rather, they measure propensity for specific types of behavior (domestic violence, for example). Because we don’t know much about the assessments Thomas underwent, it’s difficult to explain or verify what actually happened. Understanding that not all assessments are created (or conducted) equally, though, is crucial to understanding risk assessment overall.
Misconception 3: Risk Assessments Prevent Crime
Another frequent misconception is that risk assessments themselves prevent crime. The AP story identifies another extreme case in which a crime occurred despite the use of risk assessments, and they seem to suggest that risk assessments are somehow meant to prevent crime.
Had Vann scored higher on the Static-99R [a risk assessment], Texas would have sent postcards to the community where he was living—if he was living in Texas. But Vann moved to Indiana after his release.
About one year later, 19-year-old Afrika Hardy was found dead in a Motel 6 bathtub 20 miles southeast of Chicago.
Risk assessments by themselves do not prevent crime. And it’s a mistake to suggest not only that flawed risk assessments lead directly to tragic crimes, but also that sending postcards to a community—or any specific intervention—can directly prevent such crimes.
Misconception 4: Risk Assessments Are Responsible for Bad Data
It’s important to know that risk assessments are only as effective as the data being used to develop them.The AP story refers to a Florida teen who completed a risk-assessment questionnaire (the PACT) and was deemed low risk to commit another crime. One mitigating factor was the teen’s self-reported “good group of friends,” the story reads. But when a prosecutor took a closer look, she determined this group of friends had “attempted a drive-by shooting of [the teen’s] house because they said he owed them money.” The story continues:
The assessment “is completely flawed,” Schneider [the prosecutor] said in court. “They were obviously depending just on the information this young man was providing himself”…
The case is a perfect example of “garbage in, garbage out,” Schneider said in an interview. “I continue to see where the PACT does not reflect the reality.”
Schneider is absolutely correct in her “garbage in, garbage out” judgment. Yet, this doesn’t indicate that risk assessments themselves are flawed but that they can be fed bad data. Most structured assessments require the assessor to verify information in available records, including criminal history, disciplinary records from prison, assigned prison programming, past supervision, etc.
When jurisdictions are following best practices, bad data should rarely find their way into an assessment leaving us with increased confidence in the outcomes of these tools.
What The Story Gets Right
Despite containing several common misconceptions, the AP story still got a few things right. For instance, Adam Gelb, director of The Pew Charitable Trust’s Public Safety Performance Project, said:
“States and localities and all the jurisdictions that are working on risk assessment right now, they’re in different places with respect to their ability to implement a good risk assessment. But it’s absolutely critical that they do.”
He couldn’t be more correct. Not every jurisdiction that uses risk assessment uses it perfectly, but it’s crucial that every jurisdiction work toward that end. U.S. Rep. Bobby Scott (D-VA) told the AP:
“I think you ought to have some assessment and do the best you can and keep updating it based on the research. But you ought not be afraid of a system that’s working on average because of one anecdote.”
There will always be instances in which an individual judged to be low risk goes on to commit another crime—as mentioned above, we know this will be the case. This doesn’t invalidate risk assessment in general, though, which, as Rep. Scott says, are crucial components in a triaging system that does much good, though there continues to be productive discussion about how and where risk assessments can be best used.
Solomon Graves, the administrative services manager of the Arkansas Parole Board (not, as the report claims, a member of the state parole board), told the AP: “Over time the tools will become more dependable. ‘We’re never going to have a 100-percent predictive tool,’ he said. ‘We’ll never be there.’”
This is absolutely true: No tool will ever be 100 percent accurate. But Graves makes an important point—that continuing to improve assessments will only make them, and the system overall, better.