Christopher T. Lowenkamp, Ph.D.
Kristin Bechtel, M.S.
University of Cincinnati
Overview of risk assessment development and predictive validity
Method
Results
Discussion
OFFENDER ASSESSMENT and classification have become common practice throughout correctional programming over the past two decades. In particular, one risk/ needs assessment tool, the Level of Service Inventory-Revised (LSI-R), has gained widespread popularity in correctional settings. While multiple studies have demonstrated the predictive validity of the LSI-R in various correctional settings and populations, research continues to stress the importance of examining the predictive validity of the LSI-R (Andrews, 1982; Andrews & Bonta, 1995; Bonta & Andrews, 1993). As posited by Gottfredson and Moriarty (2006), examining the predictive validity of a risk/needs assessment is paramount since the samples used in the development of a risk tool should be representative of the population that the instrument is intended to be used upon. Further, they suggest that findings generated from similar samples should be cautiously interpreted due to the possibility of overestimating the tool’s validity in predicting recidivism. Given this warning, agencies should consider examining the predictive validity of their targeted populations in order to determine the instrument’s reliability and validity with their offender base. The current study attempts to demonstrate the utility of such a practice by examining the predictive validity of the LSI-R on a sample of probationers and parolees in Iowa.
Overview of risk assessment development and predictive validity
History of risk assessment
Previously, risk assessment involved a professional, or clinical, judgment concerning an individual’s risk to recidivate. Typically, this was measured by the intuition or “gut feelings” of the practitioner from an offender’s self-report or through a file review of official records (Bonta, 1996; Gottfredson, Moriarty, 2006; Latessa, 2003-2004). Unlike actuarial risk assessments, these early predictions of risk were potentially based on subjective biases, rather than on standardized objective risk measures, and were ultimately difficult to replicate (Bonta, 1996, 2000). Given these associated problems, this form of assessment has failed to demonstrate its ability to effectively measure an individual’s likelihood for future offending. Hence, there is little research support for the predictive validity and reliability of clinical judgments (Bonta, 2000; Lowenkamp, Holsinger, & Latessa, 2001).
Predictive measures found in first generation instruments, such as the Burgess Scale, are primarily static risk factors. While these factors are at least based on objective and easy-to-replicate measures that do demonstrate some reliability, there are noted disadvantages to such tools (Latessa, 2003- 2004). First, they do not incorporate dynamic risk factors, which could be targeted for change. Second, reassessment would be futile, and could only register increases of risk levels (Latessa, 2003-2004).
Second generation instruments, such as the Salient Factor Score or the Wisconsin Client Management Classification System, are empirically supported, but the risk measures are not necessarily grounded in theory and, similar to the Burgess scale, most of the measures are static (Andrews, Bonta & Wormith, 2006). Third generation instruments, such as the LSI-R, are not only empirically supported, but also include dynamic risk factors that are theoretically derived (Andrews et al., 2006). These dynamic risk factors, also known as criminogenic needs, comprise the areas to target for change in the offender.
Developers of the third-generation risk assessments noted the importance of testing the reliability and validity of their instruments. In order to do this, samples of offending populations were needed to examine the predictive validity of these tools. Baseline measures for recidivism were generated to determine the targeted population’s recidivism rate. With these data, the predictive validity of the instrument was then evaluated for the specified offending group. As previously stated, tests of predictive validity should be repeated, even for similar offending groups, as slight variations from one sample to the next can potentially overestimate the tool’s reliability (Gottfredson & Moriarty, 2006).
Predictive validity of the LSI-R
The LSI-R is a standardized actuarial instrument that contains 54 items and produces a summary risk score that can be categorized into five risk levels. Based on the Multi Health Systems (MHS) cutoff scores, ranges have been designated that indicate an individual’s risk category. Specifically, the risk categories are: 1) Low, which ranges from a 0 to 13 overall risk score; 2) Low/Moderate, which ranges from 14 to 23 overall risk score; 3) Moderate, which ranges from 24 to 33 overall risk score; 4) Moderate/High, which ranges from 34-40 overall risk score; and 5) High, which ranges from 41 to 54. Higher risk levels reflect an increase in the propensity to commit future criminal acts. These 54 static and dynamic items are divided into 10 domains. The 10 criminogenic domains include criminal history, education/employment, financial, familial relationships, accommodations, leisure and recreation, companions, alcohol and drug use, emotional health and attitudes, and orientations (Andrews & Bonta, 1995). Information to score the LSI-R is primarily gathered through offender self-report during a structured interview and available collateral information, such as official court records.
Based on the principles of effective correctional intervention, and specifically the risk principle, offenders should be separated by their risk level (Andrews, Bonta & Hoge, 1990). Further, multiple studies and meta-analyses have repeatedly shown that the intensity and dosage of programming, supervision and services should be related to the offender’s risk level (Andrews et al., 1990; Andrews & Dowden, 1999, 2006; Dowden & Andrews, 1999a, 1999b, 2000; Lipsey & Wilson, 1998; Lowenkamp, Latessa, & Holsinger, 2006). Simply put, offenders that demonstrate a higher risk should receive the majority of services. Likewise, lower-risk offenders should be diverted from programming that includes a higher-risk population. Several studies have revealed that the lower-risk group’s recidivism rate is likely to increase under these circumstances (Andrews, Zinger, Hoge, Bonta, Gendreau & Cullen, 1990; Andrews & Dowden, 1999; Dowden & Andrews, 1999a, 1999b; Lipsey & Wilson, 1998; Lowenkamp & Latessa, 2005). Specifically, programming designed for the high-risk offending populations, such as residential treatment, should not be provided to the low-risk offender. These programs have been found to increase the associated recidivism rates of their lowerrisk clients (Lowenkamp & Latessa, 2005).
Numerous studies have established support for the validity of the LSI-R in various correctional settings and populations. Specifically, in a review of validation studies, Andrews and Bonta (1995) reported that the instrument was a valid predictor of offending outcomes for incarcerated populations as well as offenders in community and residential settings. In one recent study examining an incarcerated population, Simourd (2004) evaluated the predictive ability of the LSI-R on a sample of Canadian offenders to assess whether the instrument was valid for inmates serving lengthier incarcerations. He found that the LSI-R was an effective tool for predicting both general (r= .44) and violent recidivism (r= .26). Similarly, in a study examining the predictive validity of the LSI-R on jail inmates, Holsinger, Lowenkamp and Latessa (2004) found a positive correlation between the total risk score and recidivism (r= .40). Finally, meta-analytic reviews have also suggested that the LSI-R is a valid predictor of future recidivism for offending populations (Gendreau, Little & Goggin, 1996; Gendreau, Goggin & Smith, 2002).
Support for the predictive validity of the LSI-R has also been noted for samples comparing ethnicities, sex, and age (Andrews & Bonta, 1995).1 In one empirical test examining the predictive validity of the LSI-R on the sex of the offender, Lowenkamp et al. (2001) found it was a valid assessment for males (r= .22) and performed equally as well, and in some instances, better for females (r= .37).
As indicated, multiple evaluations have demonstrated the predictive ability of the LSI-R. However, one study sought to address a gap in research concerning practitioner training and adherence to the guidelines concerning the administration of the LSI-R (Whiteacre, 2004). Flores and colleagues (2006) explored the predictive validity of the LSI-R, focusing on the “implementation integrity” and how this may impact the tool’s ability to produce valid results on their targeted population. Specifically, they found a significant positive correlation (r= .21) between the total LSI-R score and future recidivism for practitioners trained on the LSI-R. This correlation increased to r= .25 when the instrument had been in use at that agency for at least three years. In contrast, the correlation between the LSI-R score and recidivism for untrained practitioners revealed an insignificant correlation (r=.08).
The current research explores the relevance of examining the predictive validity of the LSI-R on a sample of probationers and parolees from Iowa. This process is more commonly referred to as norming the tool on the targeted population in order to demonstrate that the LSI-R has predictive validity for this sample. In particular, this study will profile the offenders who have been assessed on the LSI-R to determine if this risk and needs instrument was able to predict recidivism for these two distinct groups. It should be noted that individuals on parole may have a higher criminogenic risk to recidivate than those who have been placed on community supervision. As such, the analyses for these groups will be conducted separately and the reported findings will be presented with the total sample and then both samples individually.
Method
Sample
This sample of offenders is comprised of probationers and parolees from the State of Iowa. Initial LSI-R assessments were completed between the dates of May 12, 2003 and November 21, 2003, leaving a total sample size of 1,145 cases. Specifically, the total sample included 902 initial probation assessments and 243 parole assessments.
Measures
Six primary independent variables were used to predict recidivism within the multivariate analyses for the total sample, as well as both the probation and parole samples separately. Demographic variables that were included in the current study were sex, race, age, and marital status. With the exception of age, which remained as a metric level of measurement, sex, race and marital status were coded dichotomously. Specifically, for the variable labeled sex, 0 = male and 1 = female. Race was coded as 0 = white and 1= nonwhite, and for the offender’s marital status, 0 = married and 1 = single. Two additional independent continuous variables included time at risk and the total risk score of the offender based on the initial assessment of the LSI-R. Time at risk was measured as the number of days from the start of supervision until the end of the follow-up period. Finally, the offender’s total LSI-R was measured as a limited metric ranging from 0 to 54. In order to provide further descriptive information regarding the offenders, these data included two continuous variables that examined the time to failure, which was measured in the number of days before a violation, and the days to recidivism, which was measured as the days until re-arrest for a felony or indictable misdemeanor. While each of these variables applies to the total sample as well as to the separate probation and supervision samples, an additional independent variable labeled as supervision status was included in the multivariate analysis for the total sample. Specifically, this dichotomous measure was coded as 0 = probation and 1 = parole.
Only one dependent variable, recidivism, was considered in the current study. This measure examined whether or not an offender was re-arrested based on a felony charge or an indictable misdemeanor. In particular, this variable was coded as 0 = no felony charge or indictable misdemeanor and 1 = yes, the offender experienced a felony charge or indictable misdemeanor.
Analysis
As described above, each analysis will report the findings for the entire sample as well as treating the probation and parole groups separately. This is done in an effort to assess the predictive validity of the LSI-R on both groups, which rather expectedly may have varying risk factors and scores attributed to receiving different sentencing or supervision status types. There are three components for this analysis. The first section will describe the offenders based on demographic data as well as profiling the offenders from their LSI-R scores. Within the first section of the analysis, the descriptive statistics for each of the independent demographic variables as well as the dependent variable are presented. The second section of this report will focus on the validation of the LSI-R tool in predicting future criminal behavior among the probation and parole samples. Both bivariate and multivariate analyses were conducted to determine how well the LSI-R performed in predicting future criminal behavior of probationers and parolees. In addition, a receiver operating characteristic analysis was conducted in order to address any potential bias that may have impacted the strength of the correlation coefficients from the bivariate analysis.
Results
Demographics
Table 1 describes the demographic characteristics for the total sample, and both supervision status types. While the majority of offenders were single, white males, with an average moderate LSI-R risk score of 25, it is interesting to note some of the differences between the probation and parole groups.2 At the start of supervision, probationers averaged 30 years of age, while the parolees were slightly over 33 years of age. The range for probationers’ ages was 16 to 66 years, while the parolees’ ages ranged from 19 to 60 years. In addition, the modal value for the probationers’ ages was 19, yet the parolees were most likely to be 23 years of age. This may not be surprising, since these ages may reflect that the more youthful group would initially receive probationary sentences, while those who have received prison sentences already are much more likely to be older and perhaps of higher risk due to establishing a prior offense history.3
Out of the total sample (N= 1,145), 428 (33.2 percent) offenders were found to have recidivated during the follow-up period. Based on supervision status, nearly 31 percent of the probationers recidivated, in comparison to almost 43 percent of the parolees. A comparison of these findings with the days-to-recidivism measure found that the probationers do fail faster than the parolees, but a higher percentage of parolees recidivated in comparison to the probationers.
Validation of the LSI-R
Three separate analyses contributed to examining the predictive validity of the LSI-R on this sample. First, for the purposes of setting up the multivariate models, bivariate correlations were calculated for both the probation and parole groups examining recidivism by total LSI-R score. Second, a receiver operating characteristic (ROC) analysis was also conducted, since this statistic is not biased by a sample’s selection ratio or base rates (Mossman, 1994). The correlation coefficient, as produced by the bivariate correlation analysis, can be impacted from two sources: 1) the selection ratio, which is the percentage of recidivists as determined by the risk/ needs assessment and 2) the base rate, which reflects the actual recidivists in the sample (Flores, Lowenkamp, Smith & Latessa, 2006, p. 47). Finally, the multivariate models were constructed to identify if the LSI-R overall score was a significant predictor of recidivism while controlling for other variables.
Table 2 depicts the results of the bivariate correlations, which indicate that the LSI-R is a significant predictor of recidivism. With the exception of female parolees and non-white parolees, the total LSI-R score was significantly correlated with recidivism, for both the total sample (r= .245, p< .01) as well as probation (r= .233, p< .01) and parole (r= .254, p< .01).4
The ROC analysis produced a curve for this sample of Iowa offenders that represents the ratio of true positives (those who actually recidivated) to false positives (those who did not recidivate). For the probation sample, the ROC analysis found a ratio of 276:626 offenders. This resulted in an area under the curve of .644 (p< .01). Regarding the parolees, the ROC analysis ratio produced was 104:139. This resulted in an area under the curve of .652 (p< .01). As described by Rice and Harris (1995), these values under the curve can be treated as percentages, since the value is based on a ratio. Therefore, for the probationers, there was a 64.4 percent chance that a randomly selected recidivist earned a higher LSI-R score than a randomly selected nonrecidivist. Similarly, this value would be 65.2 percent for the parolees.
Both the bivariate and ROC analyses have revealed that the LSI-R is a valid predictor of recidivism for a sample of probationers and parolees in Iowa. This final analysis will examine the predictive ability of the LSI-R while considering the effect of sex, race, age, marital status, supervision status, and time at risk. Table 3 illustrates the results from the multivariate logistical regression models. Regarding the entire sample, results from Table 3 reveal that sex, race, age, supervision status type, and total LSI-R score are significant predictors of recidivism. Parolees were more likely to experience recidivism than probationers. Finally, for the total LSI-R score, the higher the total risk score, the more likely that a case would result in recidivism. Similar significant results were noted when examining the model for probationers. However, the findings were not consistently demonstrated for the parole group model. Both sex and age became insignificant predictors of recidivism for the parole sample. Yet, race and total LSI-R score remained statistically significant predictors of recidivism and the coefficients remained in the expected direction.
Discussion
Overall, the sample is comprised of offenders who are of moderate risk based on the MHS cutoffs. Regarding the validation of the LSI-R for these two groups, all of the analyses, both bivariate and multivariate, as well as the ROC analysis, suggest that the total LSI-R score is significantly related to predicting future criminal activity. As such, this risk/needs assessment is a valid and valuable tool for both supervision status types.
There are several practical purposes for programs and facilities to norm and validate the instrument on their specific population. First, agencies can use the instrument to designate supervision levels for their clients. Second, upon an initial assessment, case managers can create individualized case plans for the offender that targets their specific criminogenic needs or risk factors. Third, and specific to the current study, agencies can develop appropriate cutoff scores with which to manage their offending population and provide appropriate and beneficial service delivery. Fourth, offenders can be reassessed during their supervision as well as at the conclusion to determine if the assigned treatment and services reduced a client’s risk factors associated with recidivism.
Ultimately, programs are more likely to demonstrate their efficacy when utilizing a standardized and objective risk measure, such as the LSI-R, which identifies an individual’s risk level as well as his or her criminogenic needs. Actuarial assessment practices are clearly relevant and needed within the current correctional climate. With proper implementation and interpretation, correctional programs will have the ability to appropriately allocate funding and resources that may increase appropriate placement and treatment effectiveness and potentially enhance public safety (Flores et al., 2006; Latessa & Lowenkamp, 2005).