Logo RGB Utah copy

Protecting the Bill of Rights in Utah since 1958

Where Evidence-Based Practices Will Fail Us

As Utah's public agencies and criminal justice entities hum with enthusiasm about data-driven approaches and evidence-based practices (for example: here, here, here, here, you get the idea), I hate to be the one to point this out but...

...it is inevitable that Utah's favorite criminal justice buzz term - "Evidence-Based Practices" - will run into the buzz saw of reality at some point. And by reality, I mean decades of policies - in law enforcement, housing, education, and other areas of government involvement - that discriminate against certain Americans based on their race and class.

Luckily, I'm not the only one pointing this out. I highly 452182976recommend this excellent article in Time last week describing U.S. Attorney General Eric Holder's concerns about increased used of risk assessment tools, particularly in pre-sentencing processes. While this data-driven approach to criminal justice may save money, it is very likely to perpetuate terrible and unjustified racial disparities in our criminal justice system.

Before we get too far in the weeds, let's take a second to define "evidence-based practices' as the term is used in the criminal justice world.

An "evidence-based practice" or "EBP" is an approach or intervention that has been evaluated - with some academic or scientific rigor - and has shown a significant level of effectiveness. Referring to a practice as "evidence-based" implies that there is a definable outcome, that the outcome is measurable; and that the outcome is defined according to some "practical reality" rather than abstract notion (such as, "this program results in fewer offenders returning to prison," or "this practice results in higher satisfaction among victims").

For a quick overview of some basic "best practices" in criminal justice based on current research, check out this past ACLU of Utah blog post.

A WARNING TO UTAH POLICYMAKERS: don't confuse "evidence-based" with "gospel truth."

EBPs are those approaches that have SUGGESTED SOME EFFECTIVENESS - they have not been proven infallable! That effectiveness, and the factors that contribute to it, have to be defined (guessed at?) in some way by some person. Some people think that "reduced recidivism" means success. Others believe that "higher rates of employement" mean success. Assumptions must be made, at many points, to come up with these metrics. Not all evaluators or academics agree that certain metrics truly represent some improvement in the system.

EBPs are based on what we know AT THE MOMENT - they are not the be-all, end-all of knowledge!  The science and research around criminal justice practices are always evolving. We know more today than we knew twenty years ago. We are going to know much more twenty years from now. Knowing that a practice or approach has some supportive research is NOT the same as knowing

EPBs typically CONSIDER SOME VERY NARROW METRIC OF SUCCESS - and tend to ignore bigger, messier social and historical realities! That brings us back to my original point - the limits of EBPs that are most concerning to the ACLU of Utah, and other groups committed to the advancement of equal treatment and protection.

Some EBPs - such as the risk assessment tools mentioned by Attorney General Holder - are the subject of great debate for civil liberties-informed advocates of criminal justice reform. On the one hand, they have the potential to help certain groups of people who are currently highly victimized by the system. For example, using pre-sentencing risk assessment tools to set bail might reduce the number of low-risk offenders who are jailed for months simply because they are too poor to afford bail (even if set at just a few hundred dollars).

On the other hand, risk assessment tools used before sentencing could very realisticPrisonBlacks-e1287414502661ally punish defendants just for being black. Those who've read Michelle Alexander's The New Jim Crow or Alice Goffman's On the Run: Fugitive Life in an American City (even Rand Paul seems to get the problem of racial disparities!) will understand how easily a risk assessment tool can make racial discrimination seen quite scientific.

A risk assessment will usually ask about such "criminogenic factors" as: age at first arrest; criminal history; peer group or social networks; family stability; and so on. When viewed through the lens of historical housing policies that have concentrated certain racial groups in urban areas without many resources, and law enforcement policies that have concentrated surveillance on young men of color, it becomes easy to see how a "risk assessment" tool can become a "privilege assessment" tool without any malice on the part of correctional or court staff.

 That's not to say that evidence-based practices are all bad! On the contrary, practices supported by scientific research offer a lot of promise in repairing some obvious and persistent problems in our criminal justice system. It is critical, however, that as Utah moves forward with the Justice Reinvestment process, we all bear in mind the inevitable limits of EBPs - specifically, that they ask us to use data that purports to be objective, when that data is in fact the result of decades of discriminatory policies and practices. 

EBPs used by the courts and by the Department of Corrections will hopefully help reduce our prison population, which would be a major step in the right direction. But we will have to continue to work well beyond the Justice Reinvestment process to solve to problems of unequal treatment and protection under the law, for people of color and poor people.