How do you know whether the young person in front of you in court is likely to commit another crime? Historically the American justice system has looked at somewhat subjective factors like, “what crime did you commit?” or “have I seen you in court before?” In the past twenty years, the justice system has attempted to standardize the assessment of an individual’s likelihood of recidivism, reduce subjectivity, and target intentions at those who need them most. The result has been the implementation of the aptly named “risk assessment tools,” a set of standardized questions, that aim to establish if an individual has the factors research has proven increase people’s propensity to commit crimes. The risk assessments are also used to target services and the appropriate interventions, like substance use disorder treatment or family counseling. Instead of guessing, courts can reliably estimate an individual’s risk of further criminal activity and develop plans to reduce that risk based on data and science.
But the reality is somewhat different.
In fact, risk assessments, and risk prediction algorithms, have been met with mixed reception in the past few years: Pennsylvania recently delayed the role out a predictive algorithm to determine need for pre-trial detention over concerns of racial bias, over 100 civil rights organizations recently signed a statement saying risk assessments shouldn’t be used in pre-trial hearings at all, and a 2016 study opined that for juveniles “scores on a juvenile risk assessment provide little direction for targeting treatment to reduce young people’s risk of reoffending.” And yet, the Juvenile Drug Treatment Court (JDTC) Guidelines, (and other similar evidence-based practices in juvenile justice) call for the use of risk assessments to determine the intensity of the intervention.
So, what’s the problem with risk assessments? In part, to quote a recent study from Law Professor Megan Stevenson, “there is ample research . . . suggesting that risk assessment tools should have beneficial effects,” but little research showing that they actually do. This might be because they’re not being used properly, consistently, or even at all in some places which claim to have implemented them. It also might be because the assessments aren’t able to capture the necessary information from the people being assessed.
Does this mean that risk assessments are just an empty promise? I don’t believe so. Risk assessments have the potential to make a significant improvement in the fairness of our justice system, but we’re still optimizing them and developing systems and people which support their use. But, even in this emerging stage of risk assessment implementation; I’ve worked with JDTCs who have made huge improvements to their program through thoughtful implementation of a carefully considered risk assessment tool.
Most JDTCs use a ‘paper & pen’ risk assessment, a tool which scores risk based upon the answers to a set of questions (examples include the GAIN and the YASI). These questions were developed to identify risk factors for young people, such as criminal history (when and what crimes the person committed in the past), and “criminal thinking” (a mindset that allows the person to justify their crimes). These tools also look for protective factors that mitigate risk. These include school involvement, family cohesion, and strengths and abilities. From all these factors, the tool generates a score of low, moderate, or high risk.
Seems simple, right? Ask question, get answer, assign risk score. But of course, you can’t ask a participant in an assessment: “rate your family’s cohesiveness from 1-10” and expect a useful answer. So, the assessments ask a set of questions which, when taken as a group, provide an estimate of the level of family cohesiveness. These questions may include “how often does your family eat together?” or “do you feel safe in your home?” etc. Over the course of time, we have found that some of these questions were not universally applicable or understood the same way by all groups of people. For example, the feeling of safety in one’s home can be influenced not only by family dynamics but also by the relative crime rates of the neighborhood or security of the building that one lives in. If risk assessments aren’t accurately capturing this information for all young people, they aren’t going to be of much use.
However, there are ways to mitigate this gap, the most important being validation. Validation is the process by which the risk assessment is tested on a specific population to determine if the information it captures is accurate and reflective of the ‘true risk level.’ The goal of validation is to enhance the tool such that it provides all the necessary information about a young person, in a consistent way, but asks for it in a way that reduces the potential for misinterpretation or bias.
Therefore, it is vitally important that juvenile courts use a tool which has been validated for their community. One of the common critiques of risk assessments is that these tools are meant to provide an objective view of a person without the risk of personal bias, but if the tools consistently label one group as higher risk than another based upon differing cultural norms, they are likely exacerbating the risk of bias, not reducing it. Acknowledging that different tools work better on different populations is not a critique, but a recognition that we need to be careful about which tool we use, how we use them, and the conclusions we draw from the results.
It is also hugely important to note that the risk assessment does so much more than provide a score. The wealth of information it captures can help us with the ‘why’ behind the risk level and give us priorities to work on with the young person to reduce their risk level. No one with a criminal history will ever have a risk of zero, much like a cancer survivor will always carry a higher risk for cancer than the general population. You can’t undo a traumatic event or re-write a criminal history, but you can promote the treatment and intervention which may help to mitigate the negative effects of these past events.
Which brings us back to where we started: using risk assessment tools properly. Risk assessments’ true value comes not from merely assigning a risk score, but in capturing the information we need to inform risk reduction interventions.
Risk assessments may not be the answer to everything, but for improving juvenile justice outcomes, I’d say they’re a risk worth taking.
Matt Collinson is a senior research specialist working on the OJJDP-funded Juvenile Drug Treatment Courts Training and Technical Assistance Initiative at the Justice Programs Office (JPO).