Performance Measures for the
Criminal Justice System
Discussion Papers from the BJS-Princeton Project
by : John J. DiIulio, Jr. Geoffrey P. Alpert Mark H. Moore George F. Cole Joan Petersilia Charles H. Logan James Q. Wilson |
October 1993, NCJ-143505
U.S. Department of Justice
Office of Justice Programs
Bureau of Justice Statistics
Lawrence A. Greenfeld
Acting Director
Bureau of Justice Statistics - Princeton University
Study Group on Criminal Justice
Performance Measures
The goal of the BJS-Princeton Project is to engage the criminal justice community in a rigorous debate regarding appropriate measures and ways in which they can be effectively utilized by policymakers and practitioners.
Papers were prepared for Study Group review and dissemination by John J. DiIulio, Jr.
(project director), James Q. Wilson (project advisor), Mark H. Moore, Joan Petersilia,
Geoffrey P. Alpert, George F. Cole, and Charles H. Logan. The authors focus on selected
components of the criminal justice system and the utility of performance measures for
each. Other members of the study group are Norman A. Carlson, University of Minnesota;
Wayne Estelle, former warden, California Men's Colony; James Short, Washington State
University; and Steven K. Smith (project monitor), BJS.
This project is supported by BJS grant number 92-BJCX-0002 to Princeton University. The
contents of this document do not necessarily reflect the views or policies of the Bureau
of Justice Statistics or the U.S. Department of Justice.
Contents
Lawrence A. Greenfeld | Foreword |
John J. Dilulio, Jr. | Rethinking the Criminal Justice System: Toward a New Paradigm |
Charles H. Logan | Criminal Justice Performance Measures for Prisons |
Joan Petersilia | Measuring the Performance of Community Corrections |
George F. Cole | Performance Measures for the Trial Courts, Prosecution, and Public Defense |
Geoffrey Alpert | Measuring Police Performance in the New Paradigm of Mark H. Moore Policing |
John J. DiIulio, Jr. | Measuring Performance When There Is No Bottom Line |
James Q. Wilson | The Problem of Defining Agency Success |
Acknowledgments
A number of individuals actively contributed to the working group meetings, reviewed
papers and provided comments, including Pamela Casey and Sally Hillsman of the National
Center for State Courts; James P. Lynch, American University; Elizabeth McCaughey,
Manhattan Institute; Anne Piehl, Harvard University; Allen J. Beck, Tom Hester, and John
M. Dawson of BJS; and Steven D. Dillingham, former director of BJS and study group member.
Production was administered by Marilyn Marbrook and Yvonne Boston of BJS.
Foreword
Efficiency, and fairness are central goals for the administration of criminal justice
in the United States. Efficiency means economically applying available resources to
accomplish statutory goals as well as to improve public safety. Effectiveness refers to
carrying out justice system activities with proper regard for equity, proportionality,
constitutional protections afforded defendants and convicted offenders, and public safety.
Assuring equal treatment and handling of like offenders and giving equal weight to legally
relevant factors in sentencing represent the types of concerns generally expressed about
the fairness of the criminal justice system.
Unanimous agreement exists that the justice system ought to be efficient, effective, and
fair. Less accord, however, exists about how best to secure these essential qualities or
how to measure whether they have been achieved. Apart from the obvious problem of
determining the measurement criteria for a particular performance expectation, there is a
more difficult subsequent problem of determining what weight to give to the findings and
what changes need to be made to resolve the gap between expectation and performance.
Unlike marks on a ruler, criminal justice measures are not neutral standards but are
factors that enter into the processes being analyzed-identifying relative degrees of
improvement in fairness in sentencing, for example, would still indicate that the
sentencing process was giving weight to information not legally relevant.
The essays in this volume take a new tack as their authors address all three concerns. The
participants in the Bureau of Justice Statistics-Princeton Study Group focus attention on
the problem of measurement with respect to these fundamental expectations for the
administration of justice. After years of observing the justice system that they are now
writing about, the authors provide some new ideas to consider for improving the justice
system.
In the first essay, the director of the study group, John DiIulio, Jr., proposes a fresh way to understand or interpret the familiar elements in American criminal justice: prisons, community supervision, trial courts, and police. He establishes the theme that there must be a full and realistic accounting of the activities of criminal justice agencies. In calling for measures grounded in civic ideals, DiIulio also initiates the refrain that underlines the involvement of citizens in the work of those agencies.
Before the discussion curves back to DiIulio's and James Wilson's thoughts about the
challenge of measuring performance in public organizations, the individual essays both
widen the argument, taking in broader intellectual concepts, and narrow it, listing
specific performance measures. In some cases, as in George Cole's presentation of measures
for trial courts, the basic outlines of the activities remain untouched, while in other
cases, as in Mark Moore's and Geoffrey Alpert's consideration of measures for police, the
authors urge a departure from the familiar. In probation and parole, the rapid growth in
the community supervision population, according to Joan Petersilia, has forced control out
of the hands of corrections professionals. She believes that one way administrators can
regain the operational levers is to define a mission and to measure clearly how well the
agencies are achieving their goals.
BJS data: Delineating and testing performance measures
The participants in the BJS-Princeton Study Group reached their conclusions with few
direct references to the basic numbers and statistics that BJS has reported over the past
20 years. Their task was to start hammering out an institutional template of concepts. As
more persons enter the discussions about evaluating the administration of justice,
however, increased attention will have to be paid to existing data and the need for new
data.
A commonly held view, for example, is that the criminal justice system is chaotic and
rather poorly administers the statutory expectations placed upon it. BJS data reveal that
there may be more coherence between law and practice than is generally assumed:
Example: Over the last decade, States have reformed many of their criminal
sentencing laws, largely aiming to increase the likelihood of a prison sentence. BJS data
reveal that what the State and Federal lawmakers sought in these reforms has been
achieved-the odds of imprisonment given conviction for most crimes has increased. The
reforms effectively brought about three record high rates in the prison population: per
capita, per reported crime, and per arrest.
Example: Over the last decade, as well, a "War on Drugs" was waged, and
BJS data show that drug offenses now account for a larger share of convictions and
imprisonments than ever before. BJS data tell us, in other words, that the public's
legislative agenda produced the results that were being sought.
The criminal justice system is busy, with many millions of transactions taking place
annually. Spending for criminal justice activities accounts for just over 3 cents of every
dollar in public spending (about $74 billion)-less than 1% of Federal spending, more than
6% of spending by the States, and nearly 7% of local spending. Overall spending at all
levels of government for justice activities is about equal to spending on transportation
and just below that spent by government on hospitals and health.
The annual total for State and local justice translates into about $300
per capita of the nearly $9,000 annual per capita spent by government. While municipal and
county governments accounted for 53% of all justice spending, the States accounted for
34%, and the Federal government just under 13%.
From 1985 to 1990, corrections spending has grown faster and spending on police has grown
slower than any other components of the justice system. The following compare the
increases in per capita spending (in constant dollars) occurring between 1985 and 1990 to
the increases in workload:
Spending on police protection grew 8%, the number of Index crimes reported by law enforcement agencies increased 16%, and the number of arrests grew 19%
Example: Besides delivering what the public and their legislators demanded,
criminal justice agencies have maintained the quality of services provided. Between 1984
and 1990, BJS data on State prisons show-
Example: BJS data can be used to evaluate the fairness with which the system
is operating. Surveys conducted among representative samples of State prisoners in 1979,
1986, and 1991 reveal that while the offense composition has changed, with drug offenses
accounting for more than twice the percentage of inmates in 1991 compared to earlier
surveys, little has changed in the criminal histories of those confined. In all three
surveys well over 9 out of 10 prisoners were either violent offenders or recidivists with
prior sentences to confinement facilities or probation.
Research using a variety of sources including the National Crime Victimization Survey, the
FBI data on arrests, and national prisoner surveys, has revealed that for personal contact
crimes, the racial composition of offenders as identified by victims closely parallels the
racial distribution of those arrested and sentenced to prison for the same crimes.
Data from recent national polls indicate that our citizens hold the criminal justice
system in much lower esteem than they do most other public institutions. To operate a
justice
system that is effective, efficient, and fair challenges us to think anew, examine,
debate, and continuously measure and evaluate its practices. In the final analysis, how
these measures are used to reduce the disparity between expectation and practice is
fundamental to the credibility given to our collective ideal of justice.
Lawrence A. Greenfeld
Acting Director
Rethinking the Criminal Justice System: Toward a New Paradigm
By John J. DiIulio, Jr.
Overview: Beyond crime rates and recidivism rates
Rates of crime and recidivism have long served as critical measures for the performance of the Nation's criminal justice system. These measures represent the basic goals of public safety to which all components of the criminal justice system contribute. At the same time, however, rates of crime and recidivism are not the only, or necessarily the best, measures of what criminal justice institutions do.
Few police officers believe that their work solely determines crime rates in their jurisdiction. Few corrections officials believe that what they do chiefly determines recidivism rates. Likewise, most criminal court judges, prosecutors, public defenders, and other justice practitioners know from experience that the prevalence and severity of crime depend mainly on factors affecting individuals long before most are taken into custody. Most justice practitioners understand that they can rarely do for their clients what parents, teachers, friends, neighbors, clergy, big-genetic inheritances, or economic opportunities may have failed to do.1
Still, crime rates and recidivism rates are meaningful overall measures of the system's
performance in protecting public safety, and what justice practitioners do undoubtedly
affects crime and recidivism rates. For example, a National Academy of Sciences panel
concluded that rising imprisonment rates may have reduced crime rates in the Nation by 10%
to 20%. 2 Furthermore, numerous studies refute the once-fashionable idea that
"nothing works" in the rehabilitation of criminals, showing that, other things
being equal, offenders who participate in certain types of institutional or
community-based treatment programs are less likely to be repeat offenders than the
nonparticipants.3 While no evidence indicates that mere increases in police on
auto patrol cut crime rates, a growing body of evidence establishes that crime and
disorder are less common in neighborhoods where police get out of their cars and into
regular contact with citizens. 4
1. As James Q. Wilson and Richard J. Herrnstein have observed, a keen
knowledge of the constitutional and social factors that have been found to be associated
with criminal behavior "rivet(s) our attention on the earliest stages of the life
cycle," and reveals that "after all is said and done, the most serious offenders
are boys who begin their delinquent careers at a very early age"; see Wilson and
Herrnstein, Crime and Human Nature (New York: Simon and Schuster, 1985), pp.
508-509. 2. Alfred Blumstein, et al., eds., Criminal Careers and "Career
Criminals (Washington, D.C.: National Academy Press, 1986), p.6. |
Unquestionably, the justice system affects crime and recidivism rates. As James Q. Wilson has commented, given "the elasticity of crime or recidivism rates to feasible changes in police or correctional practices, how much of a change in these rates can be obtained at a given cost in money, liberty, etc.? Surely the answer is some number greater than zero. If it were zero, then we could abolish arrests and prisons with no adverse effects on society. Clearly, that is not something we would be inclined to try. It is true that the prevalence and severity of crime in society do not depend mainly on what justice practitioners do. But the real question is: What feasible changes in what institutions and practices will make the largest marginal changes in crime rates? Judged that way, it may turn out that arrest or imprisonment rates have bigger effects on marginal rates than any feasible change in family or school practices, because what one can feasibly change in family or school practices turns out to be pretty trivial."5
5. James Q. Wilson, commentary on the draft of the first BJS-Princeton Discussion paper. |
Toward a new paradigm
To evaluate the performance of police departments, correctional agencies, and other key
components of the justice system exclusively in terms of crime rates and recidivism rates
may cause observers to overlook other important contributions of the system's day-to-day
performance and can obscure the role that average citizens play in promoting secure
communities. A wide gap often exists between the general public's expectations for the
justice system and what most justice practitioners recognize as the system's actual
capacity to protect public well-being. This paper sketches an outline of a new paradigm
encompassing the criminal justice system's history, vision, purposes, and measures. Four
points of qualification, however, are in order.6
First, this call for a new paradigm is not motivated by a desire to design performance
measures that guarantee justice agency success. Rather, it represents an attempt to
develop realistic intermediate and long-range measures. Realistic measures account for the
daily activities of justice agencies and for the constraints under which they normally
operate. Realistic, however, does not mean easy to achieve. Indeed, the alternate measures
presented in subsequent papers in this volume are measures according to which many justice
institutions, programs, and practices now fail.
6. The preliminary ideas for these sections were presented by several members of the Study Group at the BJS/Justice Research Statistics Association (JRSA) 1992 National Conference held in New Orleans, Louisiana, September 23-25, 1992. The Study Group wishes to thank those BJS/JRSA conference participants who identified the need for the points of clarification and qualification that follow, especially Dr. Timothy Carr of the Georgia Department of Corrections; Professor George Cole of the University of Connecticut; Professor Robert Friedmann of Georgia State University; Professor Graeme Newman of the State University of New York at Albany; Dr. Sally Hillsman of the National Center for State Courts; and Professor Charles W. Thomas of the University of Florida. The Study Group's formal advisor, Professor James Q. Wilson of the University of California at Los Angeles, provided invaluable criticism of an earlier draft of this paper. |
Second, better performance measures do not act like magnets for better ways of meeting
goals. All performance measures have their limitations and may invite perverse and
unintended administrative consequences. Still, justice practitioners probably can learn
something about how to fashion and implement effective performance measures from the
experiences of other organizations, public and private.
Third, a paradigm is broader than a theory. A theory is a statement about the relationship
between two or more variables that is supposed to hold under specified conditions. 7
A new paradigm orients general understanding to historical, empirical, or normative
realities that a prevailing paradigm has arguably de-emphasized, devalued, or simply
ignored. In essence, to call for a new paradigm is to appeal for new concepts and
categories of thinking about a given subject.
Fourth, crime rates and recidivism rates are indeed important measures of the system's
performance, which ought to be continually used and refined. Even so, all citizens in a
democracy are responsible to some degree for the way in which society addresses the
problem of crime. In addition, justice agencies serve the public in myriad ways that are
indirectly related to crime control goals, and society should devise and implement
performance measures that respect this reality.
7. For a brief discussion of theory, see DiIulio, No Escape, pp. 213-225. |
History: Multiple, vague, and contradictory purposes
The history of the American criminal justice system is a history of swings in public
mood. Americans have long been ambivalent about the purposes of criminal justice.8
Among other things, they have wanted a criminal justice system that apprehends and visits
harm upon the guilty (punishment); makes offenders more virtuous, or at least
more law abiding (rehabilitation); dissuades would-be offenders from criminal
pursuits (deterrence); protects innocent citizens from being victimized by
convicted criminals (incapacitation); and enables most criminals to return as
productive citizens to the bosom of the free community (reintegration). They have
wanted the system to achieve these contradictory public goals without violating the public
conscience (humane treatment), jeopardizing the public law (constitutional
rights), emptying the public purse (cost containment), or weakening the
tradition of State and local public administration (federalism).
Because the competing public expectations cannot be easily met all at once, first one and
then another dominate public attention. Justice policymakers and practitioners have
generally allowed the institutional and programmatic pendulum to swing with the public
mood between different approaches to crime prevention and control.9
8. Portions of this section are drawn from John J. DiIulio,
Jr., "Crime," in Henry J. Aaron and Charles Schultze, eds., Setting Domestic
Priorities: What Can Government Do? (Washington, D.C.: Brookings Institution, 1992),
chapter 4. 9. As William G. Mayer has shown, between 1960 and 1965, public opinion on crime and punishment became more liberal, and between 1965 and 1988 it became increasingly conservative; see Mayer, "Shifting Sands of Public Opinion: Is Liberalism Back?," The Public Interest, no. 107 (Spring 1992), pp. 3-17. Mayer's analysis concurs well with trends in criminal justice program administration such as the rise, decline, and, in many jurisdictions, official or de facto abolition of paroling authorities. On parole, see Edward E. Rhine, William R. Smith, and Ronald W. Jackson, Paroling Authorities: Recent History and Current Practice (Laurel, MD: American Correctional Association, 1991). |
For example, between 1967 and 1992, the Federal Government fought two very different
wars on crime. The first war (1967-80) was against poverty; the second (1980-92) was
against criminals. In the first war the social and economic "root causes" of
crime were attacked; in the second war the likelihood that criminals would be detected,
arrested, prosecuted, convicted, and incarcerated was increased. The chief strategists in
the first war were persons who believed that the Federal Government should play a central
role in crime control. They emphasized the goals of offender rehabilitation,
reintegration, humane treatment, and constitutional rights. The chief strategists in the
second war were persons who believed that law enforcement was primarily a State and local
responsibility. They emphasized the goals of punishment, deterrence, cost containment, and
federalism.
Some justice practitioners have coped fairly successfully with such shifts in public
sentiment, but many have not.
Despite the conflicting and changeful public demands on them, some police commissioners
have been able to "make themselves accountable to the public by defining their
purposes in broad terms and then by trying to keep their own actions, and the actions of
their organizations, consistent with these broad purposes."10 Similarly,
some corrections commissioners have coped well by means of "a creative capacity to
translate broad societal expectations and policy decrees into administrative action."
11 Yet, the fact remains that these swings in public mood and policy have
fostered administrative instability, frustrated longterm planning, and bred bureaucratic
norms that insulate practitioners from what they sometimes view as a fickle, generally
unappreciative, and often hostile public.
Democratic vision: Citizens as co-producers of justice
In the light of this history, a moderating, democratic vision of the justice system's public purposes and limitations is both necessary and desirable. Such a vision emerges from the realization that all citizens have the right and the responsibility to participate in the system. Citizens are co-producers of justice.12 The ability of justice practitioners to do their daily work depends on the cooperation and support of citizens who are formally "outside" the system -a citizen willing to testify against a violent drug dealer; a community group that trusts and assists the police; relatives, friends, and employers who help to keep a community-based offender on the straight-and-narrow. Citizens, not judges, prosecutors, law enforcement officers, or corrections officials, are primarily responsible for the quality of life in their communities, including the prevalence and severity of crime within them.
10. Mark H. Moore, "Police Leadership: The Impossible Dream,"
in Erwin C. Hargrove and John C. Glidewell, eds., Impossible Jobs in Public Management
(Lawrence, KS: University Press of Kansas, 1990), p.98. 11. John J. DiIulio, Jr.,
"Managing a Barbed-Wire Bureaucracy: The Impossible Job of the Corrections
Commissioner," in Hargrove and Glidewell, Impossible Jobs, p.67. |
As many honest friends of democracy have argued down through the ages, democratic
citizens are wont to hold everyone but themselves accountable for public problems and to
become impatient when facile solutions do not produce immediate results.13
Citizens in a democracy must begin by holding themselves and their neighbors accountable
for public affairs. A democratic vision of the justice system, therefore, is anything but
a sop to public frustrations with crime and disorder. Citizens who expect judges, police,
and other justice officials to solve society's crime problems are unrealistic; citizens
should not expect the officials to succeed without the active cooperation and support of
the community.
13. For example, see Alexis de Tocqueville, Democracy in America, ed. Phillips Bradley, vol. 2 (New York: Vintage Books, 1945), and Walter Lippmann, Essays on The Public Philosophy (Boston: Little, Brown and Company, 1955). |
Criminal-justice purposes: Four civic ideals
This democratic vision supplies a rationale for identifying the major purposes of the system in terms of four civic ideals:
(1) Doing justice,
(2) Promoting secure communities,
(3) Restoring crime victims, and
(4) Promoting noncriminal options.
Justice can be defined as the quality of treating individuals according to their civic rights and in ways that they deserve to be treated by virtue of relevant conduct. Criminal justice is rights-respecting treatment that is deserved by virtue of criminal conduct as judged by the rule of law.14 Thus, doing justice implies at least four things: hold offenders fully accountable for their offenses, protect offenders' constitutional and legal rights, treat like offenses alike, and take into account relevant differences among offenders and offenses.
14. This definition of criminal justice was suggested by the Study Group's Professor Charles H. Logan of the University of Connecticut. |
Promoting secure communities means more than to achieve low crime rates. Rather, it means providing the security to life, liberty, and property that is necessary for communities to flourish. It means enabling citizens to pursue their collective life as they see fit without undue fear of having that life disrupted or destroyed. It means securing communities against criminals who assault, rape, rob, defraud, deal drugs, burglarize, extort, and murder, but it also means securing them against the community-sapping disorders that are commonly associated with crime and the fear of crime-disorders such as petty crime, public drunkenness, aggressive panhandling, loitering, graffiti, abandoned cars, broken windows, and abandoned buildings.15
Restoring victims means to honor the community's obligation to make victims of
crime and disorder whole again. The victims' rights organizations, manifestos, and laws
that have proliferated over the last decade or so generally reflect and embody this
long-overlooked goal.16 Victims of crime have a special claim upon the criminal
justice system's human and financial resources. Whatever else it may achieve, no system
that dishonors that claim can be considered legitimate.
Finally, promoting noncriminal options means that punishment for criminal
behavior should interfere as little as possible with the pursuit of noncriminal behavior.
Even in prison, offenders should have at least some opportunity to engage in meaningful,
constructive, and legitimate activities. Nor should government impose arbitrary
restrictions on employment or other legitimate activities by convicted offenders except
where justified as a form of punishment or where public safety is at risk. This is not to
say that society has any greater obligation toward the betterment of offenders than it
owes to nonoffenders. It is not even to say that government has an obligation toward the
betterment of offenders and nonoffenders alike. But one function of government is to
promote (not necessarily to provide) legitimate opportunities and to facilitate (not
necessarily to require or directly to reward) their pursuit.
15. A fine treatment of the relationship between disorder, crime, and the
fear of crime can be found in Wesley G. Skogan, Disorder and Decline: Crime and the
Spiral of Decay in American Neighborhoods (Berkeley and Los Angeles: University of
California Press, 1990). Also see the following: George L. Kelling, "Measuring What
Matters," The City Journal, Spring 1992:21-33; James Q. Wilson and George L.
Kelling, "Police and Neighborhood Safety: Broken Windows," Atlantic Monthly,
March 1982, pp. 29-38; and James Q. Wilson and John J. DiIulio, Jr., "Crackdown:
Saving the Next Generation From the Drug-and-Crime Epidemic," New Republic,
July 10, 1989, pp. 21-25. 16. knowledge about the physical pains, psychological traumas, and economic losses suffered by victims of crime, their families and friends, and the public remains shallow but is increasing. For a serviceable overview, see Albert R. Roberts, ea., Helping Crime Victims: Research, Policy, and Practice (Newbury Park, Calif.: Sage Publications, 1990). |
Realistic performance measures
These four civic purposes point beyond crime rates and recidivism rates and toward more
realistic ways of measuring the performance of justice institutions, programs, and
practices. By no means is this the first call for such measures. During both of the
Federal wars on crime from the 1960's through the 1980's, a number of well-intentioned
efforts were made to rethink the measures commonly used to evaluate the system's
performance. (See Selected sources of measurement topics, p. 17.) Few of these
efforts moved much beyond a rehashing of such concepts as crime rates and recidivism
rates, and none had a wide or lasting impact on the field. In conjunction with his work on
the Study Group, Logan has developed a set of performance measures for secure correctional
institutions.
The measures Logan proposes for prisons and jails have the virtue of not asking criminal
justice institutions to do what other social institutions are more responsible for doing
and in many cases what other social institutions have failed to do. They do not, for
example, ask our corrections officials to somehow "correct the incorrigible,
rehabilitate the wretched, deter the determined, restrain the dangerous, and punish the
wicked." 17 But they do demand that, with the human and financial
resources that society has provided, and with the requisite support of other social
institutions, the officials must "keep prisoners-keep them in, keep them safe, keep
them in line, keep them healthy, and keep them busy-and do it with fairness, without undue
suffering, and as efficiently as possible."18 That alone is asking a great
deal, but it is not asking too much.
17. Charles H. Logan, "Criminal Justice Performance Measures For
Prisons," Performance Measures for the Criminal Justice System,
BJS-Princeton Discussion Papers, p. 23. 18. Logan, "Criminal Justice Performance Measures," p. 25. |
By the same token, it makes little sense to measure police performance in terms of
crime or arrest rates. Geoffrey P. Alpert and Mark H. Moore have developed an expanded
range of policing measures. In anticipation of their contribution, it is worth
highlighting George L. Kelling's recent article, "Measuring What Matters: A New Way
of Thinking About Crime and Public Order." After documenting that the New York City
Police Department has been doing quite well in relation to such conventional measures as
crime rates, arrest rates, emergency response times, and incidence of corruption, Kelling
keenly observes:
But New Yorkers are not the least bit reassured by these statistical and relative
achievements.... These formal measures of police work have little to do with community
needs.... [A] significant reason disorder has been ignored is that professional criminal
justice ideology narrowly defines the appropriate business of police and criminal justice
agencies as dealing with serious crime-that is, index crimes. Crime response, and arrest
statistics, form a pillar of that ideology. Disorder does not appear in any FBI index;
therefore, it has not been a priority.19
19. Kelling, "Measuring What Matters," pp. 21-22. |
Conclusion: Toward a new paradigm?
Is it possible for justice officials to develop, implement, and organize themselves
around performance measures that go beyond conventional measures such as rates of crime
and recidivism? And can this be done for all components of the system-courts, prosecutors'
offices, police departments, institutional corrections, community-based corrections?
The papers that follow in this compendium will tackle these questions and offer specific,
detailed proposals for new measures consistent with the historical understanding,
democratic vision, and civic purposes outlined above. In addition, they spell out the
practical and policy implications of adopting the new paradigm and spotlight its
implications for how agencies allocate resources, conduct program evaluations, and so on.
It is worth noting that many of the most successful major corporations use multiple performance measures that give tremendous weight to "soft" indicators along with sales reports, inventory records, and other "hard" financial data. For example, McDonald's Corporation has measured performance not simply by the conventional bottom line of profits, but by a dozen or so measures that roving teams of inspectors apply-Are the floors clean? Are the salt shakers full? Are the cashiers greeting customers and wearing their uniforms correctly? and so on. McDonald's recognized that the profits made by their stores were conditioned by economic and other factors over which their franchisees had little or no direct control. But the store owners, managers, and staff could be and are held strictly accountable for other factors that might affect business. 20
Likewise, over the last decade, the United States military has made great strides in
developing reasonable and realistic measures of combat readiness and combat effectiveness.
Prodded by government and private studies that found a need for improvements in military
planning, and in the areas of weapons acquisition, combat training, and force deployment,
each branch of the military responded by revamping certain of its strategic doctrines and
practices, and by getting away from simple "bean counting" measures. While many
improvements have yet to be made, the military has begun to think about new and better
ways of linking its national security mission to meaningful performance standards and
objectives.21
20. David C. Rickert, McDonald's Corporation (condensed), Harvard
Business School, revised February 1982. 21. For interesting examples that relate to defense acquisition programs, see Glenn A. Kent, A Framework for Defense Planning (Santa Monica, Calif.: RAND, August 1989), and Glenn A. Kent and William E. Simons, A Framework for Enhancing Operational Capabilities (Santa Monica, Calif.: RAND, 1991). |
The performance measurement lesson that much of corporate America and the American
military have learned is one that the American justice system can also apply. Crime rates,
recidivism rates, and other conventional bottom-line measures must have better grounding
in community needs and must be joined to a realistic set of performance standards. The
Study Group hopes to provide a gentle, democratic shove in that direction, and to get
policymakers, practitioners, analysts, activists, and interested citizens thinking and
debating toward a new paradigm of the American justice system.
About the author
John J. DiIulio, Jr., is Professor of Politics and Public Affairs at Princeton
University and nonresident Senior Fellow at the Brookings Institution. He served as
founding director of Princeton's Center of Domestic and Comparative Policy Studies. He is
a member of the National Commission on State and Local Public Service and served as an
advisor to the National Performance Review. His most recent book is Improving
Government Performance: An Owner's Manual (Brookings, 1993).
Selected sources of measurement topics
Gordon P. Whitaker, Stephen Mastrofski, Elinor Ostrom, Roger B. Parks, and Stephen L. Percy, Basic Issues in Police Performance, National Institute of Justice, July 1982.
Joan E. Jacoby, Basic Issues in Prosecution and Public Defender Performance, National Institute of Justice, July 1982.
Thomas J. Cook and Ronald W. Johnson, with Ellen Fried, John Gross, Mary Wagner, and
James Eisenstein, Basic Issues in Courts Performance, National Institute of
Justice, July 1982.
Gloria Grizzle, Jeffrey S. Bass, J. Thomas McEwen, Deborah M. Galvin, Ann G. Jones,
Harriet D. Mowitt, and Ann D. Witte, Basic Issues in Corrections Performance,
National Institute of Justice, July 1982.
Martha R. Burt, Measuring Prison Results: Ways to Monitor and Evaluate Corrections Performance, National Institute of Justice, June 1981.
Sorrel Wildhorn, Marvin Lavin, and Anthony Pascal, Indicators of Justice: Measuring
the Performance of Prosecution, Defense, and Court Agencies Involved in Felony
Proceedings; A Guide to Practitioners, National Institute of Law Enforcement and
Criminal Justice, Law Enforcement Assistance Administration, U.S. Department of Justice,
May 1977.
Sorrel Wildhorn, Marvin Lavin, Anthony Pascal, Sandra Berry, and Stephen Klein, Indicators
of Justice: Measuring the Performance of Prosecution, Defense, and Court Agencies Involved
in Felony Proceedings; Analysis and Demonstration, National Institute of Law
Enforcement and Criminal Justice, Law Enforcement Assistance Administration, U.S.
Department of Justice, May 1977.
Jack D. Reynolds, Performance Measurement in Probation and Parole, Washington, D.C.: University Research Corporation, 1979.
Benjamin H. Renshaw, A Recommended Set of Indicators for Evaluating the Performance
of the Philadelphia Criminal Justice System. Philadelphia: Government Studies &
Systems, Inc., 1971.
Criminal Justice Performance Measures for Prisons
by Charles H. Logan
This is the second in a series of papers on how to measure and evaluate the performance
of various agencies within the American criminal justice system. The first paper argued
the need for new criminal justice performance measures in addition to such conventional
ones as crime rates, arrest rates, and recidivism rates.1 This paper extends
that argument and applies it to prisons. It starts with a very brief and general
definition of criminal justice from a retributive (or "just deserts")
perspective. Such a perspective is rights-based rather than utilitarian, which implies
that evaluative indicators and measures of criminal justice should focus more on the
satisfaction of certain standards, values, and constraints than on the production of
particular consequences. In Herbert Packer's terms, they should focus more on the
"DueProcess Model" than on the "Crime Control Model" of criminal
justice.2 The paper will then outline the "confinement model" of
imprisonment, which rests on a normative statement of mission for a prison or prison
system. Finally, the paper will offer a set of empirical indicators that can be used as
performance measures for prisons and that concentrate on the competent, fair, and
efficient administration of confinement as a form of deserved punishment. While based on
the deserts theory of criminal justice, these measures will be seen to be at least
somewhat sensitive also to such goal-based concerns as rehabilitation of inmates and
protection of society, albeit for reasons independent of those utilitarian justifications
of imprisonment.
1. John J. DiIulio, Jr., Rethinking the Criminal Justice System:
Toward a New Paradigm, BJS Discussion Paper, NCJ-139670 (Washington, D.C.: Bureau of
Justice Statistics, 1993). 2. Herbert L. Packer, The Limits of the Criminal Sanction (Stanford University Press, 1968). |
What is criminal justice?
Justice is the quality of treating individuals according to their rights and in ways that they deserve to be treated by virtue of relevant conduct. Criminal justice is rights respecting treatment that is deserved by virtue of criminal conduct.
This definition of justice is rights-based, rather than utilitarian or consequentialist. A rights-based theory of justice gives a central role to punishment as a morally necessary response to the violation of rights. To believe in rights is to believe in duties; those are alternative statements of the same concept. To believe in duties is to accept, implicitly but of logical necessity, the corollary of punishment. When we say that people have a duty to refrain from violating the rights of others, we are saying that there must be some sanction if they fail to meet that duty. Duties are given meaning by the consequences that attach to their nonfulfillment. Thus, the meaning of a duty, like that of any other norm, must be socially constructed through the attachment of sanctions to behavior. A norm (a rule, a law, a duty, a right) that had no sanction attached to its violation would be empty and without meaning.
Justice by this definition is backward-looking. It requires that we treat people
according to what they have done, not what they (or others!) might do in the future as a
result of how we treat them now. Justice requires that all persons, including offenders,
be treated as autonomous and responsible actors and as ends in themselves, not as means to
social ends.
Finally, a rights-based theory sees justice as a process, an ongoing property of criminal
sanctioning as it occurs, not as an expected outcome. Criminal justice is thus a value in
itself and not merely useful as a means to some other end. Sanctioning that is evaluated
as to its justice or injustice may, in addition, be evaluated in terms of its
consequences for other values, such as freedom, order, happiness, wealth, or welfare, but
those are separate concerns. This means that questions about the effectiveness or
efficiency of the criminal justice system in achieving various "goals" or
"purposes" should be kept separate from, and secondary to, an evaluation of the
performance of the justice system in its most basic mission: doing justice.3
3. Again, however, the measures of justice to be derived here from a normative and nonutilitarian model will be seen to overlap considerably with measures that might be derived independently from a utilitarian model. |
The nonutilitarian concept of prison performance
To date, most evaluation research on prisons has focused on utilitarian questions. What
are the goals of imprisonment? To what extent and at what cost are they achieved? How does
imprisonment compare to alternatives in these respects? If we want evaluations of prison
performance that are based on a normative rather than a utilitarian view of criminal
justice, we need to reframe our question. We might ask, for example, "To what values
do prisons commit themselves in their mission statements, and how well do they live up to
those values?"
Social scientists are not comfortable with the idea of applying the tools of measurement
directly to questions of value. That's why criminologists are attracted to utilitarianism,
because it allows them to treat evaluative research on prisons as if it were a purely
objective, scientific enterprise. In contrast, a court-appointed special master, who is
usually a lawyer rather than a social scientist, evaluates a prison mostly from a
formalistic rather than a utilitarian perspective. That is, the prison and its activities
are examined not as means to an end (rehabilitation or crime control) but in terms of
standards and criteria of "proper" performance, or conduct in fulfillment of
duty. Consider this statement by a prominent prison master:4
In summary, the ideal prison provides basic human services in a decent and healthful
physical environment. Such a prison abjures idleness and its consequent human
deterioration by offering constructive employment, programming, and recreational
activities to the greatest extent possible; it addresses the human needs of prisoners for
self-expression, faith, and maintenance of ties of importance to all human beings; it
ensures safety from random violence, rape, and exploitation of the weak by the strong; it
insulates decisions affecting the lives of prisoners from arbitrary chaos by adhering to
due process of law; and it infuses the institutional environment with constructive
expectations through use of positive incentives for hard work and good behavior.
That is not a bad statement of the mission of a prison, and it is probably one with which most correctional officials could agree. The most important point to note here is that it does not focus on ultimate goals, such as treatment or punishment, but on a set of abstract values and normative criteria against which to evaluate the day-to-day operation of a prison.
4. Vincent M. Nathan, "Correctional Health Care: The Perspective of a Special Master," The Prison Journal, vol. 65, No. 1 (Spring-Summer, 1985), pp. 73-82, at p. 76. |
A prison mission statement under the confinement model
We ask an awful lot of our prisons. We ask them to correct the incorrigible,
rehabilitate the wretched, deter the determined, restrain the dangerous, and punish the
wicked. We ask them to take over where other institutions of society have failed and to
reinforce norms that have been violated and rejected. We ask them to pursue so many
different and often incompatible goals that they seem virtually doomed to fail. Moreover,
when we lay upon prisons the utilitarian goals of rehabilitation, deterrence, and
incapacitation, we ask them to achieve results primarily outside of prison, rather than
inside. 5 By focusing on external measures, we set prisons up to be judged on
matters well beyond their direct sphere of influence.
If we do not want to set them up for failure, we must assign to prisons a function and a
mission that we might reasonably expect them to fulfill. This mission ought to be fairly
narrow and consistent in scope, and it ought to be special to prisons, rather than
conflated with the functions of other social institutions such as schools or welfare
agencies. It also ought to be achievable and measurable mostly within the prison itself.6
Finally, a prison's mission ought to have intrinsic, and not just instrumental,
value. That is, it should identify activities that have value in themselves, when they
meet certain standards and criteria of performance, not activities that have value only
if, when, and because they are effective in achieving some further goal.
5. For each of these three goals, the principal measure of its
achievement is the number of crimes avoided in the general community. This is true even of
incapacitation, because that refers not to maintaining order within prisons but to
avoiding the crimes that current prisoners might have continued to commit if they were not
in custody. Incapacitation thus reflects much more the performance of the police,
prosecutors, and judges who catch, select, and send offenders to prison than it does the
performance of those who simply hold them there. 6. This last requirement pretty much rules out crime control and rehabilitation, at least as criteria for evaluating the performance of particular prisons, if not as goals of imprisonment generally. |
The prison mission statement proposed here is based on a "just deserts" theory of criminal justice, one that calls for a punitive and purely retributive (that is, a non-utilitarian) response to criminal conduct. Punishment under such a theory does not need to take the form of incarceration, but since this paper is about prison performance measures, I will narrow the theory down to what I call the "confinement model" of imprisonment.
Under the confinement model, the essential purpose of imprisonment is to punish offenders-fairly and justly- through lengths of confinement proportionate to the gravity of their crimes. Thus the term, "confinement model," may be thought of as a shortened version of a clumsier but more explicit label: the "doing justice through confinement as a form of punishment model."
The mission of a prison under the confinement model can be summarized quite succinctly:
The mission of a prison is to keep prisoners-to keep them in, keep them safe, keep them
in line, keep them healthy, and keep them busy-and to do it with fairness, without undue
suffering, and as efficiently as possible.
The confinement mission of prisons is not as narrow as it may seem at first, nor is it
necessarily harsh or insensitive to the welfare of prisoners. It should be noted that
under the confinement model offenders are sent to prison as punishment, not for
punishment. It is not within the legitimate mission of a prison to attempt to add
to (any more than to avoid or to compensate for) the pain and suffering inherent
in being forcibly separated from civil society. Stated more positively, coercive
confinement carries with it an obligation to meet the basic needs of prisoners at a
reasonable standard of decency. Thus, measures of health care, safety, sanitation,
nutrition, and other aspects of basic living conditions are relevant. Furthermore,
confinement must meet constitutional standards of fairness and due process, so it is not
just the effectiveness and efficiency, but also the procedural justice with which
confinement is imposed that is important. In addition, programmatic activities like
education, recreation, and work can be seen as part of the conditions of confinement,
regardless of their alleged effects on rehabilitation. In short, confinement is much more
than just warehousing.
Under the confinement model, a prison does not have to justify itself as a tool of
rehabilitation or crime control or any other instrumental purpose at which an army of
critics will forever claim it to be a failure. It proclaims itself to be, first and
foremost, an agent of justice, and not necessarily an agent of either individual or social
change. It asks to be judged only on its performance in carrying out the sanction of
confinement-as-punishment; the effectiveness of that sanction may be a valid and important
question, but it is not relevant to the measurement of prison performance under the
confinement model.
What, then, are the relevant criteria?
Prison performance criteria under the confinement model
It might seem that measuring prison performance within a confinement model would be fairly simple-and indeed it is more straightforward than attempting to measure the success of rehabilitation, deterrence, or incapacitation (let alone the net effects of imprisonment on all three of these in combination)-but it is by no means easy. Still, the confinement model does facilitate performance measurement, because it focuses less on the achievement of ultimate and abstract goals and more on the fulfillment of delimited and immediate tasks. It shifts our attention away from hard-to-determine outcomes and toward more directly observable processes and adherence to measurable standards.
The confinement mission of a prison, as stated above, identifies eight distinct dimensions for prison performance measures: Security, Safety, Order, Care, Activity, Justice, Conditions, and Management. Each of these dimensions will be discussed briefly.
1. Security ("keep them in"). A secure facility is one that is impervious in either direction, outward or inward. Escapes are an obvious indicator of a lack of security, but inward penetration, of drugs or other contraband, also represents a breakdown of external security. Internal security would include control over movement of prisoners within the prison and control over internal movement of contraband, such as food or silverware from the dining hall, drugs from the infirmary, or tools from workshops.
2. Safety ("keep them safe"). Inmates and staff need to be kept safe, not only from each other but from various environmental hazards as well. Thus, measures of safety would include assault statistics, safety inspection results, and accidental injury reports.
3. Order ("keep them in line"). Prisons run on rules, and
the ability of prison administrators to enforce compliance is central to prison
performance. Allowing for variation in the nature of their populations, it seems proper to
evaluate prisons according to their ability to prevent disturbances, minimize inmate
misconduct, and otherwise preserve order inside their walls.
4. Care ("keep them healthy"). I use the term "care"
rather than "service" to cover the ministrations of such personnel as doctors,
dentists, psychiatrists, psychologists, and dieticians. The distinction is primarily one
of degree and entitlement. Convicts are entitled only to a very basic, minimal level of
personal care consistent with the principle that it is not the purpose of imprisonment to
inflict physical suffering. At a minimum, prisons have an obligation to try to prevent
suicide, malnutrition, exposure to the elements, and the spread of contagious diseases.
Beyond the level of very basic care, however, the simple fact of confinement does not
entitle convicts to levels of service or to degrees of personal welfare that exceed what
they are able to obtain with their own resources. Therefore, when rating prisons on this
dimension an evaluator might choose not to make distinctions beyond a certain level.
5. Activity ("keep them busy"). When evaluating prisons under a
rehabilitation model, heavy emphasis is usually given to inmate programs; under a
confinement model, programs are still relevant, but on a different basis. Programs can be
classified into five different types: work, training, education, recreation, and therapy.
All five types are relevant under a confinement model but in each case any rehabilitative
effect a program might have is not directly relevant to its evaluation. Therapeutic
programs are so closely associated with the rehabilitative ideal that they are difficult
to recast in terms of the confinement model. They can, however, be offered as a form of
"care," and evaluated according to the principles discussed under that
dimension. Programs of the other three types should be judged according to how much
opportunity they provide inmates to engage in constructive activity or enterprise.
"Constructive" activity is not defined here as "contributing to the
betterment of inmates" but as activity that is, on its face, consistent with the
orderly, safe, secure, and humane operation of a prison. Idleness and boredom can be seen
as wrong in themselves, from a work ethic standpoint, or as so fundamentally related to
mischief as to be undesirable for that reason. Either way, prison programs of work,
training, and education should be evaluated under the confinement model as forms of
constructive activity and as antidotes to idleness, not as methods of rehabilitations.7
Under a rehabilitation model, work, education, and training are seen as benefits that are offered to prisoners, or even forced upon them, in the hope that this will make them better and more law-abiding citizens. Under the confinement model, work, education, and training are not benefits; they are opportunities, available to prisoners who are willing to make productive use of them. Ideally, prisons would have, or would fit into, an economy in which inmates could earn money by producing goods and performing services having real value. Inmates might then seek education and training, not to impress a parole board or a prison counselor, but to be able to perform a more valuable and higher paying job.
7. While a confinement model may sometimes be in conflict with a rehabilitation model, it is not necessarily so. In the confinement model, it is desirable to keep inmates constructively busy, quite apart from the question of whether that does them any rehabilitative good. That does not mean, however, that it does not matter from some other perspective whether the programs have any rehabilitative effect. It would be very nice if prison programs had rehabilitative effects. However, when we say that the primary purpose of prison is to punish through confinement, we become more interested in the operation of these programs inside the prison gates and less concerned about their effects beyond. |
The availability of opportunities for education and employment should offset some of
the austerity of a prison organized around a strict confinement model. However, amenities,
privileges, and benefits that might be justified under a rehabilitation model as a
worthwhile investment of taxpayers" money should not be provided free to prisoners
under a confinement model. Any social benefits that are guaranteed to all citizens should
be provided to prisoners as well (within limits imposed by security needs), but beyond
that prisoners would have to earn or purchase them at their own expense. Examples would
include higher education, entertainment, and medical, dental, or psychological services
beyond the minimal levels entailed in the confinement model.
Some people believe that constructive activity should be more than just an opportunity
available to inmates; it should be a prisoner's obligation as well. Offenders, in this
view, should be held financially as well as morally responsible for their crimes and their
imprisonment. Thus, prisoners should be required to work, to make restitution to their
victims, to support their families, and to pay something toward the cost of their
incarceration. Financial responsibility is not inconsistent with the confinement model,
and could therefore be included under the dimension of activity. However, it is
independent of, rather than integral to, a prison's primary mission of
confinement-as-punishment.
6. Justice ("do it with fairness"). In measuring the
performance of justice within prisons, the propriety of the sentence may be taken for
granted; what remains to be judged is the fairness with which the sentence is
administered. Stated more broadly, governing with justice requires adherence to the rule
of law inside prisons just as it does on the outside. Rules ("laws") must be
clear,
sanctions for their violation must be specified in advance and applied consistently,
enforcement and adjudication must follow due process, and there should be provisions for
independent review of decisions. Relevant to this dimension would be procedures and
practices in imposing discipline and allocating good time, grievance procedures,
availability of and access to legal resources, and inmate perceptions of the fairness and
legitimacy of rules and their enforcement.
7. Conditions ("without undue suffering"). A confinement model
obviously requires some evaluation of the conditions of confinement. This broad term would
include such things as population density, food, clothing, bedding, noise, light, air
circulation and quality, temperature, sanitation, recreation, visitation, and
communication with the outside. As with the dimension of "care," evaluation of
living conditions and quality of life should not be completely linear (the more the
better, without limits). In principle, this dimension is curved, so that differences imply
improvements at the lower end but have declining or even negative merit ("too good
for them") above some higher point. Most prisons today, however, probably lie along
the middle range of this dimension, where comparison can be linear.
8. Management ("as efficiently as possible"). Quality of
management is probably the single most important source of variation in the first 7
dimensions of quality of confinement. As such, there may be some redundancy in evaluating
management as, itself, a separate component of prison performance. However, it is better
to over measure than to under-measure, and many management variables bear a strong enough
presumptive relationship to overall quality of institutional operation that they can be
used as indicators of otherwise hard to measure concepts.
For example, such management-related variables as staff morale, absenteeism, and turnover
are visible reflections of institutional stress and tension. Training levels may be both a
cause of quality (through increased staff competence) and a result of quality (as a
product of institutional concern with proper procedure in treatment and discipline of
inmates). Thus, various sorts of management information can be used as a measure as well
as an explanation of confinement quality. Good management is also a legitimate end in
itself. The public has an interest in seeing that the money it spends on imprisonment is
not wasted, through over-staffing, high turnover, or other management-related problems.
These eight dimensions-security, safety, order, care, activity, justice, conditions, and
management-are appropriate concerns of prison professionals under the confinement model of
imprisonment, and therefore constitute relevant focal points when measuring prison
performance. Moreover, they are relatively precise concepts susceptible to
operationalization and empirical measurement. First, however, each dimension must be
divided into its component parts, or subdimensions.
Subdimensions and empirical indicators
The eight dimensions of prison performance described above are not directly measurable in themselves. As abstract concepts, they must eventually be linked to more concrete and observable indicators before they can be transformed into performance measures. A first step in that direction is to break each of the eight general concepts down into an associated set of more specific subdimensions, as shown in Figure 1. These subdimensions can be defined operationally by linking them to relevant empirical indicators drawn from two types and sources of data: institutional records and surveys of staff and inmates.8
8. Field observations would be a third type and source of data. They are not discussed here because they are better suited to an intensive case study than to the systematic, comparative style of performance evaluation contemplated in this essay. |
Figure 1. Prison performance measures based on staff and inmate surveys and
institutional records
Security--Security procedures, Drug use, Significant incidents, Community exposure, Freedom of movement, Staffing adequacy | Justice--Staff fairness, Limited use of force, Grievances: number and type, The grievance process, The discipline process, Legal resources and access, Justice delays
|
Safety--Safety of inmates, Safety of staff, Dangerousness of inmates, Safety of environment, Staffing adequacy | Conditions--Space in living areas, Social density and privacy, Internal freedom of movement, Facilities and maintenance, Sanitation, Noise, Food, Commissary, Visitation, Community access |
Order--Inmate misconduct, Staff use of force, Perceived control, Strictness of enforcement | Management--Job satisfaction, Stress and burn-out, Staff turnover, staff and management relations, Staff experience, Education, Training, Salary and overtime, Staffing efficiency |
Care--Stress and illness, Health care delivered, Dental care, Counseling, Staffing for programs and services | Activity--Involvement in and evaluation of: Work and industry, Education and training, Recreation, Religious activities |
I
Institutional records data can be drawn from such sources as the following:
Significant incident logs
Disciplinary logs and files
Grievance logs and files
Inmate employment records
Education records
Health clinic logs
Psychologist logs
Personnel records
Most prisons maintain records like these and should be able to retrieve and calculate
summary information from them on a periodic basis. The Bureau of Justice Statistics
conducts a complete, nationwide census of all prisons and jails every 5 years. The form
used for this census requests information on facility characteristics, court orders or
consent decrees, inmate population size and characteristics, housing confinement space,
inmate education and work assignments, counseling and other programs, furloughs, staff
characteristics by function and payroll category, inmate medical facilities and HIV
testing, inmate deaths, and major incidents during the last year. For the 1990 prison
census there was an additional set of questions on inmate drug use, interdiction methods
such as drug screening for inmates and staff, and treatment programs. Ideally, the data
collected through these censuses would allow researchers to construct performance measures
for comparing prisons both cross-sectionally and through time. Unfortunately, respondent
burden makes it difficult to expand the BJS census forms to address more of these issues
in depth, at least as the forms are administered to all facilities. It might be feasible,
however, to extend the data collection effort for a limited sample of prisons. Examples of
performance measures based on institutional records are
offered in an appendix to this paper. Many of them could be derived directly from
questions already included on the BJS census form while others would require collecting
additional data.
Surveys of staff and inmates can be more costly than a census of facilities, but they have the advantage of being able to generate a far greater range of performance measures. Since the Bureau of Justice Statistics also does national surveys of prison and jail inmates, in addition to the facility census, it might be feasible for those surveys to include questions aimed at constructing performance measures. The Bureau of Prisons has developed what it calls the Prison Social Climate Survey (PSCS) to gather information useful in the management of its facilities. This survey is administered to staff at half of the Bureau's prisons every 6 months and there is a parallel version of the survey for inmates, though it is rarely used. The PSCS includes questions in four areas:
The inmate version of the PSCS also covers the first three areas, but
in place of Work environment it has a Services and programs section asking about medical
care, counseling, education, recreation, work, and religious programs. Elsewhere, it has
questions on staff competence, attitudes, and interactions; on the discipline process; and
on aspects of living conditions beyond those asked also of staff.
Most questions in the PSCS ask the respondent to answer in terms of
conditions prevailing during the last 6 months, so that repeated surveys can be used to
map changes through time. The numerous questions cover all of the dimensions of prison
performance and quality identified in this paper, albeit some dimensions more thoroughly
than others. Many of the survey-based indicators listed in the appendix are taken from the
staff and inmate versions of the PSCS.
Significantly, this paper's eight conceptual dimensions of prison performance, and the
prison mission statement that underlies them, were inspired in major part by analyzing the
management concerns of the Bureau of Prisons as revealed in its Prison Social Climate
Survey. That the dimensions and measures of prison performance recommended here should
come from observing the self-imposed tasks, standards, and evaluation criteria of
a well-regarded prison system is consistent with the primary theme of this essay: Prisons
should be evaluated according to that which it is reasonable and realistic to regard as
being within their sphere of influence, competence, and accountability.
Introducing an instrument for prison performance measurement
The final section of this paper is the Appendix, p. 42. It presents an organized set of empirical indicators that could be used as performance measures for prisons. These indicators are grouped into the eight dimensions that have been identified as relevant to prison performance and quality under a confinement model of imprisonment. Within each dimension, the indicators are further organized according to subdimensions, and the indicators based on official records are distinguished visually (by the use of italics) from those that are based on survey instruments. This set of performance measures has not been tested for validity, reliability, internal consistency, scalability, predictive weighting, discrimination, or other statistical properties, and has been used, in its current form, in only one study.9 What the indicators have to recommend them for further study is not so much their current methodological quality as the fact that they are based on a clear conceptual model.
9. Charles H. Logan, Well Kept: Comparing Quality of Confinement in a Public and a Private Prison (Washington, D.C.: National Institute of Justice, 1991). The Prison Social Climate Survey, on the other hand, has been tested for statistical qualities, refined, and used in various research projects conducted by the Office of Research and Evaluation of the Federal Bureau of Prisons. |
In concluding this paper, a few caveats on the application of these measures may be in order. First, the indicators are offered here without any instructions for coding, scoring, combining into scales, or interpreting results. The intent of this paper is to offer some grounded and specific criteria and empirical indicators for evaluating prison performance, but not to prejudge the question of just what standards should differentiate good, fair, or poor performance. Indeed, such standards probably cannot be articulated in abstract or absolute terms. Rather, prison performance should generally be viewed as a relative or comparative matter, and preferably within contexts that hold constant or adjust for such factors as the population type, security level, or other characteristics of the prisons being compared. In any case, the most important thing in developing performance measures is not the choice of cutting points to use as standards (such as 60 versus 58 square feet per cell), but the choice of the criteria themselves (such as space per inmate in living areas). It is by choosing the criteria for evaluating performance that we structure incentives and thus shape the behavior we wish to evaluate and, ultimately, improve.
Second, as suggested by the large number of indicators in the Appendix, the concept of prison performance is complex and multi-dimensional. No single indicator, nor even any small subset of indicators, should be taken too seriously by itself. Multiple indicators are required to capture the many tradeoffs that must be made between the various and sometimes conflicting criteria of quality in the operation of a prison. Since no organization can maximize all values at once, the more criteria and indicators we use, the more accurately we can reflect the total pattern of an institution's strengths and weaknesses. Moreover, some indicators must be viewed in conjunction with others in order to be interpreted fully. For example, data on the frequency of shakedowns must be combined with measures of inmate grievances, drug use, and amounts of contraband found, in order to say whether those shakedowns amount to harassing inmates, engaging in needless ritualism, or doing what is necessary to interdict illicit traffic. Indicators taken separately will always produce errors of measurement and interpretation. Using large numbers of indicators helps to smooth out and counterbalance these errors, so that an overall pattern of findings will be more reliable and meaningful than the results of any of the individual indicators alone.
Third, note that some indicators are used multiple times, to measure different concepts, sometimes with conflicting results. For example, freedom of movement for inmates is a negative indicator on the dimension of Security, but a positive indicator on the dimension of Conditions. That sort of ambiguity is a fact of life, so it is valid to include measures that are scored both positively and negatively, on different dimensions. More troublesome are indicators that can be interpreted either positively or negatively within the same dimension. Many potential indicators were discarded during the process of constructing the set of measures found in the Appendix because opposing interpretations seemed equally plausible. Even after that culling process, however, some of the remaining indicators can be interpreted, at least arguably, in either a positive or a negative direction. There is no way to avoid that problem completely. To restrict oneself to indicators that are absolutely clear and unambiguous would produce very limited and uninteresting reports. A better solution is to report findings in detail, thus allowing others to make differing interpretations, and to search for patterns rather than relying very much on single indicators.
Finally, it may be noted that some of the indicators are "harder" and some
"softer" than others. Most of the indicators based on official records produce
relatively objective, factual, and verifiable data, while the indicators based on inmate
and staff surveys are more subjective and perceptual. Researchers should be sensitive to
the need to look for conflicting results (for example, between staff and inmate
perceptions), should try to use the "harder" indicators to corroborate the
"softer" indicators, and may wish to consider weighting some indicators more
heavily than others.
Appendix
Criminal Justice Performance Measures for Prisons
Italicized items are based on official records. Others are based on surveys of
staff or inmates (or both, in which case the staff and inmate means are counted as
separate indicators). "Rate per capita-6" means "divided by total number of
inmates resident at some time during a 6-month reference period." Scale values are
omitted for all scale items ("rating of ...," "perception of ...,"
etc.).
Survey and official record measures of prison performance
Dimension 1: Security ("keep them in")
A. General
1. Rating of how the building design affects surveillance of inmates
B. Security procedures (6-month period)
1. Perceived frequency of shakedowns in the living area
2. Perceived frequency of body searches
3. Proportion of staff who have observed:
a. Any consequential problems within the
institution
b. Lax security
c. Poor assignment of staff
d. Inmate security violations
e. Staff ignoring inmate misconduct
f. Staff ignoring disturbances
g. Other problems
4. Number of cell or bunk area shakedowns conducted in a l-month
period
a. Rate per inmate
b. Proportion finding contraband
5. Number of urinalysis tests based on suspicion in a l-month period
a. Rate per inmate
b. Proportion testing positive for opiates
C. Drug use (6-month period)
1. Drug-related incidents, number and rate per capita-6
2. Discipline reports related to drugs or contraband, number and
rate per capita-6
D. Significant incidents (6-month period)
1. Significant incidents, total and rate per capita-6
a. Proportion of 6-month population involved in
any incidents
2. Escapes, number and rate per capita-6
E. Community exposure (6-month period)
1. Furloughs, number and rate per capita-6
F. Freedom of movement
1. Perceived freedom of movement for inmates: Day / Evening / Night
G. Staffing
1. Ratio of resident population to security staff
Dimension 2: Safety ("keep them safe")
A. Inmate safety (6-month period)
1. Perceived likelihood of an inmate being assaulted in his living area
2. Estimated rate (per 100 population) of armed assaults involving
inmates
3. Estimated rate (per 100 population) of assaults against inmates
without a weapon
4. Estimated rate (per 100 population) of sexual assaults upon inmates
5. Estimated rate (per 100 population) of instances inmate has been
pressured for sex
6. Inmates' perceived danger of being: a. killed or injured b. punched
or assaulted
7. Proportion of inmates who say they have been physically assaulted by
another inmate in a 6-month period
8. Proportion of inmates who say they have been physically assaulted by
staff in a 6-month period
9. Discipline reports that involved fighting or assault, number and
rate per capita-6
10. Significant incidents involving inmate injury, number and rate per
capita-6
B. Staff safety (6-month period)
1. Rating of how the building design affects staff safety
2. Perceived danger to male staff
3. Perceived danger to female staff
4. Rating of how often inmates use physical force against staff
5. Perceived likelihood that a staff member would be assaulted
6. Proportion of staff who say they have been assaulted by an inmate in
a 6-month period
7. Significant incidents involving staff injury, number and rate
per capita-6
C. Dangerousness of inmates
1. Proportion of inmates perceived to be extremely dangerous
2. Proportion of inmates perceived to be somewhat dangerous
3. Perceived frequency of inmate possession of weapons in living
quarters
D. Safety of environment (6-month period)
1. Perceived frequency of accidents: Housing Units / Dining Hall / Work
Environment
2. Perceived occurrence in housing units of clutter that could feed a
fire
E. Staffing adequacy
1. Proportion of staff and inmates who feel there are enough staff to
provide for safety of inmates: Day / Evening / Night
2. Proportion of staff who feel there are enough staff to provide for
their own safety: Day /Evening / Night
Dimension 3: Order ("keep them in line")
A. Inmate misconduct (6-month period)
1. Perceived frequency of physical force by inmates against staff
2. Perceived security of inmate personal property
3. Proportion of inmates who report being punished in the last 6
months: a. with a major sanction b. with a lesser sanction
4. Number of inmates written up, as proportion of 6-month
population
5. Discipline reports, total and rate per capita-6
a. Reports per inmate among those written up
6. Significant incidents of disturbance or incitement to riot,
number and rate per capita-6
B. Staff use of force (6-month period)
1. Perceived frequency that staff have used force against inmates over
a 6-month period
2. Significant incidents in which force was used, number and rate
per capita-6
3. Significant incidents in which restraint was used, number and rate
per capita-6
C. Perceived control
1. Agreement that staff know what goes on among inmates
2. Agreement that staff have caught and punished the "real
troublemakers"
3. Perceptions of how much control inmates have over other inmates:
Day / Evening / Night
4. Perceptions of how much control staff have over
inmates: Day / Evening / Night
D. Strictness of enforcement (6-month period)
1. Proportion of discipline reports that were:
a. Dismissed
b. Guilty of a minor report
c. Guilty of a major report
2. Proportion of minor report convictions that received a sanction
of:
a. Warning/reprimand
b. 5-10 extra hours of duty
c. 15-20 extra hours of duty
d. 25-30 extra hours of duty
3. Proportion of major report convictions that received a sanction of:
a. Segregation only
b. Loss of goodtime only
c. Segregation and loss of goodtime
4. Average number of goodtime days taken away
5. Average number of days to be spent in segregation
6. Proportion of major report sanctions
a. Suspended at committee level
b. Modified by warden
Dimension 4: Care ("keep them healthy")
A. Stress and illness (6-month period)
1. Inmate stress scale: average of 9 items reporting feelings of
mental, physical, and emotional strain
2. Average number of days an inmate was ill or injured
3. Average number of days an inmate was seriously ill enough that
medical help was needed but did not go to sick call
4. Significant incidents involving suicide attempts or self-injury,
number and rate per capita-6
5. Significant incidents requiring first aid or infirmary visit, number
and rate per capita-6
B. Health care delivered (6-month period)
1. Proportion of inmates who used medical facilities other than for
emergency problems
a. Proportion of those who used the facilities
who felt the problem was properly taken care of
2 . Proportion of inmates who reported having had emergency medical
treatment
a. Proportion of those who received emergency
medical treatment who felt that it was adequately handled
3. Clinical contacts, total and rate per capita-6
4. Sick calls, number and rate per capita-6
5. Medical appointments, number and rate per capita-6
6. Physicals and TB tests, number and rate per capita-6
7. Lab appointments, number and rate per capita-6
8. Miscellaneous clinic visits, number and rate per capita-6
C. Dental care (6-month period)
1. Proportion of inmates who received dental treatment a. Proportion of
those receiving dental treatment who felt it was
adequately handled
2. Dental visits, number and rate per capita-6
D. Counseling (6-month period)
1. The alcohol and drug counseling services have been satisfactory
(agree/disagree)
2. Other counseling services have been satisfactory (agree/disagree)
3. Proportion of inmates who report having participated in some kind of
counseling:
a. Drug/alcohol counseling
b.Therapy
4. Psychologist contact cases per capita for I month
5. Number of contact hours per contact case for I month
6. Proportion of inmates who were involved in the following programs:
a. Psychology/psychiatric; includes substance
abuse
b. Employment and pre-release counseling
7. Psychiatric visits (over a 6-month period), number and rate per
capita-6
E. Staffing for programs and services
1. Number of program or services delivery staff (FTE):
a. Medical clinicians
b. Education/work
c. Psychology/counseling
d. TOTAL
2. Number of inmates (average daily resident population) per FTE staff
position in programs or services:
a. Per medical clinician
b. Per education/work staff
c. Per psychologist/counselor
d. Per total program/service staff
3. Program or services delivery staff as a proportion of total staff
Dimension 5: Activity ("keep them busy")
A. General
1. Inmates usually have things to do to keep them busy
B. Work and industry involvement (6-month period)
1. involvement in prison industry, work release, or institutional
jobs:
a. Proportion of population eligible
b. Proportion working
2. Among eligible inmates, proportion involved in:
a. Prison industry
b. Work release
c. Institutional jobs
3. Average work hours per week among employed inmates
C. Work and industry evaluation (6-month period)
1. The work training program has been satisfactory (agree/disagree)
2. Have the vocational training courses provided skills that are
useful?
a. Perceived importance of learning the
information presented in class
b. Perceived understanding of the information
presented in class
3. Grievances that involved problems with work, number and rate per
capita-6
D. Education and training involvement (6-month period)
1. Proportion of inmates who report having participated in some
educational program
a. Educational
b. Social education/pre-release skills
2. Enrollment in education or vocational training classes:
a. Proportion of population eligible
b. Proportion enrolled
3. Among eligible inmates, proportion involved in the following
programs:
a. Adult basic education
b. Secondary education
c. College education courses
d. Vocational training
4. Average class hours per week among those in education or
vocational training programs
E. Education and training evaluation (6-month period)
1. The general education program has been satisfactory (agree/disagree)
2. Have the academic courses provided useful skills?
a. Perceived understanding of the information
presented in class
b. Perceived importance of the information
presented in class
F. Recreation (6-month period)
1. Recreational activities are satisfactory (agree/disagree)
2. Rating of how often prison recreational facilities are used
3. Rating of how often inmates are unable to use the recreational
facilities
G. Religious services (6-month period)
1. Religious services have been satisfactory (agree/disagree)
2. Rating of how often inmates attend religious services
Dimension 6: Justice ("do it fairly")
A. Staff fairness
1. Questions on aspects of staff fairness
(agree/disagree)
a. Staff let inmates know what is expected of
them
b. Staff are fair and honest
c. Inmates are written up without cause
2. Staff are too involved in their own interests to care about inmate
needs (agree/disagree)
B. Limited use of force (6-month period)
1. Staff use force only when necessary (agree/disagree)
2. Perceived frequency with which staff have used force against inmates
3. Significant incidents in which force was used, number and rate
per capita-6
4. Significant incidents in which restraints were used, number and rate
per capita-6
C. Grievance volume (6-month period)
1. Proportion of staff reporting having a grievance filed against them
in last 6 months
2. Proportion of inmates who reported filing a grievance against staff
or management
3. Inmates filing grievances, number and proportion of 6-month
population
4. Grievances filed, total and rate per capita-6
5. Number of grievances directed at individual staff
a. Proportion of all grievances
b. Rate per capita-6
D. The grievance process (6-month period)
1. Perceived effectiveness of the grievance procedure
2. Perceived benefits of the grievance procedure
3. Perceived effect of grievance procedure on the quality of life
4. Proportion of inmate grievants who report their grievance was taken
care of:
a. Completely
b. Partially
c. Not at all
5. Proportion of inmates who did not file a grievance, who cite the
following reasons:
a. They never had any major complaint
b. The problem was solved informally
c. They thought it would be useless
d. They were afraid of negative consequences
e. Other reasons
6. Proportion of all grievances that were appealed
E. The discipline process (6-month period)
1. Proportion of inmates receiving a major sanction
who felt it was a fair punishment
2. Proportion of inmates receiving a lesser sanction
who felt it was a fair punishment
3. Perception of how many maximum security inmates really belong there
4. Proportion of discipline guilty verdicts that were appealed
a. Minor reports
b. Major reports
5. Proportion of major report sanctions
a. Suspended at committee level
b. Modified by warden
F. Legal resources and legal access (6-month period)
1. Proportion of inmates who have used the law library
2. Proportion of inmates who feel the law library has
supplied adequate information
3. Proportion of inmates who feel the law library has not
supplied adequate information.
4. Grievances that involved legal resources or access, number and
rate per capita-6
G. Justice delayed (6-month period)
1. Average number of days from the date of the discipline report
until the hearing
2. Proportion of minor reports with hearings beyond 7-day limit
3. From date of grievance report until resolved by grievance officer:
a. Average number of days
b. Proportion beyond 20 days
4. From date of grievance report until resolution approved by warden:
a. Average number of days
b. Proportion beyond 27 days
Dimension 7: Conditions ("without undue suffering")
A. General
1. The administration is doing its best to provide good living
conditions (agree/disagree)
B. Crowding (6-month period)
1. Average resident population as percentage of capacity
2. Proportion of 6-month period in which capacity was exceeded
3. Average number of sq. ft. per inmate in housing units
4. Perceived occurrence of crowding in the housing units
5. Perceived occurrence of crowding outside the housing units
C. Social density and privacy
1. Proportion of inmates who were confined in:
a. Single-occupancy units of 60 sq. ft. or more
b. Multiple-occupancy units with 60 sq. ft. or
more per inmate
c. Multiple-occupancy units with less than 60
sq. ft. per inmate
2. Perceived amount of privacy within the sleeping area
3. Perceived amount of privacy in the shower and toilet area
D. Internal freedom of movement
1. Perceived freedom of movement for inmates: Day / Evening / Night
2. Proportion of inmates who were confined to housing units for over 10
hours per day
E. Facilities and maintenance (6-month period)
1. Residents vs. conveniences in living areas
a. Inmates per shower
b. Inmates per sink
c. Inmates per toilet
d. Inmates per telephone
e. Inmates per television
2. Grievances about maintenance, number and rate per capita-6
F. Sanitation (6-month period)
1. Perceived occurence of insects, rodents, or dirt in the housing
units
2. Perceived occurrence of insects, rodents, or dirt in the dining hall
3. Perceived occurrence of a bad odor or poor air circulation in the
housing units
G. Noise (6-month period)
1. Perceived noise level in the evening hours
2. Perceived noise level in the sleeping hours
H. Food (6-month period)
1. Quality of food at the institution
2. Variety of the food at the institution
3. Proportion of inmates who feel enough food is served for the main
course
4. Proportion of inmates who feel the appearance of the food is
appealing
5. Grievances involving food complaints, number and rate per
capita-6
I. Commissary (6-month period)
1. There is an adequate commissary selection (agree/disagree)
2. Proportion of inmates who reported:
a. No errors in their commissary account
b. Errors that were corrected
c. Errors that were not corrected
J. Visitation (6-month period)
1. Proportion of inmates who find it hard to arrange visits with family
and friends
2. Proportion of inmates reporting family and friends who find it hard
to arrange visits
3. Average number of visitors reported by inmates
4. Rating of the quality of visits
5. Perceived occurrence of too many people in the visiting area
6. Rating of how often it is hard to talk to a visitor because of noise
in the visiting area
7. Proportion of inmates who feel the visiting room has enough
furniture
8. Proportion of inmates who feel the visiting room has enough vending
machines
9. Grievances involving visitation and mail problems, number and
rate per capita-6
K. Community access (6-month period)
1. Furloughs, number and rate per capita-6
Dimension 8: Management ("as efficiently as possible")
A. Job satisfaction (6-month period)
1. Institution satisfaction index: average across 3 items expressing
positive feelings toward the institution
2. Proportion of staff who reported filing a grievance against
management
3. Proportion of staff who have not filed a grievance, who cite the
following reason:
a. Never had a major complaint
b. Problem was taken care of informally
c. Thought it would be useless
d. Afraid of negative consequences
e. Other reason
B. Stress and burn-out
1. Job stress index: average across 5 items regarding how often staff
experience stress on the job
2. Hardening-toward-inmates index: average across 3 items regarding how
often staff feel indifferent or harsh toward
inmates
3. Relating-to-inmates index: average across 7 items regarding how
often staff feel positive about the way they work with
inmates
C. Staff turnover
1. Staff on reference date divided into:
a. Vacancies on reference date
b. Terminations during previous 6 months
2. Termination rate divided by relevant BOP tenure-specific rate
D. Staff and management relations
1. Management and communication index: average across 10 items
expressing positive appraisals of the organization and
authority of management
2. Relationship-with-supervisor index: average across 6 items regarding
how positive staff feel toward their supervisor
3. Rating of how the building design affects communication among line
staff
4. Rating of how the building design affects communication between line
staff and supervisors
E. Staff experience
1. Average number of years worked at this institution
2. Average number of other facilities worked in prior to this facility
3. Average years in corrections
a. Total staff, minus services staff
b. Custody staff
c. Top administrators
F. Education
1. Average years of education (excluding services staff)
G. Training
1. Training index: average across 5 items regarding the effectiveness
and quality of the training program
H. Salary and overtime (6-month period)
1. Average salary (in $1,000's)
a. Total, minus services staff
b. Custody staff
c. Top administrators
2. Average number of overtime hours worked in a week
3. Average proportion of overtime compensated by:
a. Extra pay
b. Compensatory time
c. No compensation
I. Staffing efficiency
1. Number of resident inmates per FTE staff member
This paper is based on research conducted by the author with support from several
agencies of the U. S. Department of Justice: the Bureau of Justice Statistics (grant no.
92-BJ-CX-0001), the National Institute of Justice (grant no. 86-IJ-CX-0062-S 1), the
National Institute of Corrections, and the Federal Bureau of Prisons. The views are those
of the author and do not necessarily reflect Department of Justice policy.
About the author
Dr. Charles H. Logan is a Professor and Associate Head of Sociology at the University
of Connecticut, where he has taught since 1970. He has published widely on many criminal
justice issues, including the deterrent effects of arrest and imprisonment, evaluation of
correctional programs, the effects of parole, juvenile justice, jury selection, prison
performance measures, and the privatization of prisons. His major work in the latter area
is Private Prisons. Cons and Pros (Oxford University Press, 1990). In 1987-88 he
served as a professional staff member for the President's Commission on Privatization and
has been a Visiting Fellow to several agencies of the U.S. Department of Justice-including
the National Institute of Justice, the Bureau of Prisons, and the Bureau of Justice
Statistics. He is a member of the Corrections and Sentencing Committee of the American Bar
Association's Criminal Justice Section.
Measuring the Performance of Community Corrections
by Joan Petersilia
Introduction
The 1980's saw tremendous growth in community corrections' populations-from about 1.4
million persons at the start of the decade to 3.2 million by 1990-a more than 130%
increase. This increase was larger than that experienced by either prisons or jails over
the same time period (Hindelang et al., 1981; Jankowski, 1992). Today, 3 out of every 4
persons under correctional supervision in the United States are on some form of
community-based custody-mostly probation or parole-although community corrections also
includes halfway houses, residential centers, work furlough, and all other programs for
managing the offender in the community.1
1. Probation is often confused with parole. Probation is a sentence the offender serves in the community while under supervision; parole is the conditional release of an inmate from incarceration under supervision after a portion of the prison sentence has been served. |
Despite its wide usage, community corrections is often the subject of intense
criticism. Probation and parole suffer from a "soft on crime" image and, as a
result, maintain little public support. Their poor (and some believe, misunderstood)
public image leaves them unable to compete effectively for scarce public funds.
Nationally, community corrections receives less than 10% of State and local government
expenditures for correctional services, which includes jails and prisons (Flanagan and
Maguire, 1992). And their budgets are declining at a faster rate when compared to other
criminal justice components. Over the last decade in Los Angeles, for example, the county
Superior Court budget grew more than 200%, the sheriffs' and district attorneys' budgets
grew about 50%, while that of probation grew by a mere 10%-even though probation
populations more than doubled over this time period.
It is also true that those offenders being sentenced to probation and parole are more
serious than in the past-in terms of their crimes, prior criminal records, and substance
abuse histories (Petersilia and Turner, 1990). In New York, for example, 77% of
probationers are felons (not misdemeanants), and fully a third of active cases are people
who have been found guilty of violent crimes-yet these persons are supervised on caseloads
of several hundred persons (Los Angeles Times, February 7, 1993:A20). The Los Angeles
situation is even more critical. The L.A. County Probation Department, the largest in the
world, supervises 90,000 adult offenders-80% of all those convicted in the county. But
three-quarters of those offenders are monitored only by computer, by probation officers
with caseloads of about 1,000 each. As a result, Chief Probation Officer Barry Nidorf
estimates that his deputies have, on average, only 1 hour and 47 minutes per year to
devote to each probationer (Los Angeles Times, February 8, 1993:A12).
Why does community corrections fare so poorly? It is not that its services are
undervalued. Quite the contrary: every national study and commission beginning with the
President's Commission on Law Enforcement and Administration of Justice (1967) up to the
more recent President's National Drug Control Strategy (1990) has recommended
expanding community corrections. In fact, the National Advisory Commission on Criminal
Justice Standards and Goals (1973:311) referred to community corrections as the justice
system's "brightest hope." Public opinion polls also show wide support for
community-based sentencing for nonviolent offenders (Doble, 1987; Jacoby and Dunn, 1987).
But while there is general support for the concept of community sanctions,
current programs are seen as inadequate. Most of the commissions that endorse community
corrections go on to state that current programs are unable to provide effective offender
supervision or rehabilitation. Furthermore, their minimal supervision is not seen as
adequate punishment for most serious crimes. As DiIulio notes in No Escape: The Future
of American Corrections (1991:68),
Most Americans think that criminal sanctions that make little or no use of incarceration
fail to protect the public adequately, to deter would-be criminals, and to prevent
convicted offenders from finding new victims. Furthermore, they simply do not feel that
alternatives to incarceration are an adequate moral response to the pain and suffering
imposed upon innocent victims by often calculating and remorseless victimizers.
In their defense, "community corrections" remains an ambiguous concept. It is
a legal status, an alternative to incarceration, a service-delivery mechanism, and an
organizational entity. As an organizational entity, it has objectives and performs a wide
range of activities-some totally unrelated to offender supervision and/or treatment. One
survey found that probation departments were responsible for more than 50 different
activities, including court related civil functions (for example, step-parent adoption
investigations, minority age marriage investigations) (Fitzharris, 1979). The time-worn
controversy over whether community corrections (particularly probation) is punishment,
treatment, or an amalgam of both further confuses discussions of its mission. As David
Fogel (1984) observed: "probation lacks a forceful imagery that other occupations in
criminal justice can claim: police catch criminals, prosecutors try to get them locked up,
judges put them in prisons, wardens keep them there, but what do probation officers
do?"
So while it is true that study panels continually endorse the expansion of community
corrections, none has specified exactly what functions it should perform, or when and how
it should be used. Having been founded more than 150 years ago, community corrections
still has an unclear primary mission, with confusion about what activities contribute to
that mission and how best to assess their performance. Many observers have urged community
corrections to quantify what they do, with whom, and to what benefit (Cochran et al.,
1991; Petersilia et al., 1985; Clear and O'Leary, 1983; McAnany et al., 1984; Blair et
al., 1987).
Without a clear public mandate or an effective constituency, community corrections has
been vulnerable to political pressures. It has fallen prey to the weakness described by
the adage: "If you don't stand for something, you'll fall for anything." In the
1970's, when rehabilitation was in favor, it promised to rehabilitate. In the 1980's, when
the public mood turned tough-on-crime, it promised toughness. But with budget cuts,
observers have noted that community corrections ends up promising much but being able to
deliver very little (Clear and Byrne, 1992).
As DiIulio (1992) points out, to some extent these problems are endemic to all public
agencies. However, they seem particularly problematic in community corrections agencies
for two reasons. First, there is a serious lack of consensus-and in fact, widespread
disagreement- among community corrections staff as to goals and mission. The leadership,
as well as line staff, differ-both within and among departments-as to the importance of
the surveillance/control model versus the treatment/service model. Of course, supervision
usually combines elements of both, but studies have shown that supervision activities do
differ significantly depending on which model the staff endorses (Ellsworth, 1992). The
controversy over whether community corrections is punishment, treatment, or both has now
been enlivened. A number of large community corrections agencies (for example, Los
Angeles) have now openly embraced a surveillance model-quite different from the meshing of
the two major goals in previous years.
Thus, there is a major disagreement within the leadership of community
corrections about what the primary mission of their agencies is.
The second major difficulty-closely related to the first -is that community corrections has, by and large, never been able to show that it "works." Historically, recidivism rates-an offender's return to crime after some intervention-have been the gauge by which community corrections has been evaluated. And after hundreds of research studies (most poorly done), the weight of evidence shows that community corrections programs have not been able to reduce recidivism (Lipton et al., 1975; Gottfredson and Gottfredson, 1980; Petersilia and Turner, 1993). A recent report by the Bureau of Justice Statistics revealed that 43% of felons on State probation were rearrested for another felony within 3 years (Langan and Cunniff, 1992).
After a long history of being unable to affect recidivism significantly, probation and parole leaders have begun to question the outcome measure itself-not whether recidivism should be included at all, but whether it should be the sole or primary measure of their performance. They note that crime is the result of a long line of social ills- dysfunctional families, economic and educational deprivation, and so on-and these social problems are clearly beyond the direct influence of probation/parole agencies. As the BJS/Princeton Study Group concludes (1992):
to evaluate the system's performance chiefly in terms of recidivism measures is to exaggerate the system's ability to affect the prevalence and severity of crime in society, to miss other important measures of the system's day-to-day performance, and to obscure the role that citizens can and should play in promoting secure communities (DiIulio, 1992).
Corrections officials also argue persuasively that recidivism rates measure just one
function, while ignoring other critical probation/parole tasks, such as preparing
presentence investigations, collecting fines and fees, monitoring community service, and
so on. Adequate performance indicators should reflect the multitude of an agency's goals
and activities.
The American Probation and Parole Association (1992) has recently called for including
other intermediate outcomes in program evaluations. These would measure the offenders'
activities while on probation or parole supervision (for example, rates of employment,
drug use, participation in work and education). The association argues that programs do
affect offender behavior, and that the effects would be shown if these mediating outcomes
were measured.
Corrections practitioners also question whether their performance should be judged by how the offender behaves once he is off formal probation/parole supervision. After all, other components of the justice system are not judged by their ability to affect the future criminal behavior of offenders, and corrections should not be either. Charles Logan (1992) agrees and writes:
The police, prosecutor, and courts are judged by more proximate outcomes (such as arrest and conviction rates). It is only when the system switches to corrections, that changing the offenders' crime behavior becomes the primary measure of success.
An additional difficulty has recently emerged regarding the recidivism measure. Specifically, even if one accepts that recidivism is a useful measure of success, it is not clear which direction indicates success. If one believes that the major mission of community corrections is to protect the public-emphasizing the surveillance function-then perhaps increasing recidivism rates (for example, returns to prison) is a positive-not negative-performance indicator. If offenders are convicted of a crime and incarcerated, then public safety is being served. But if rehabilitation is the primary goal, as it has historically been, then decreasing recidivism indicates success. For example, a recent evaluation of intensive probation/parole supervision programs found that closer supervision increased technical violations and returns to jail and prison. The Los Angeles Times called the probation program a "dismal failure" since it failed to decrease recidivism. Barry Nidorf, Chief Probation Officer in Los Angeles County, used the higher recidivism figures to argue that the ISP program had succeeded. He wrote:
Reducing recidivism was not listed as the highest priority...holding offenders accountable was of primary importance...given the stated project goal of protecting the community, a significantly lower recidivism rate was not expected.
For years, probation has been measured only by recidivism rate...In today's environment, with over 70% of the probation caseload consisting of felony offenders, is it still realistic to use only this one criterion? Why is revocation and sentencing of a probation violator not considered a "success"? I believe it should be...(Nidorf, 1991a).
Without consensus about whether the major criminal justice outcome-recidivism-should be increasing or decreasing as a result of interventions, the research field is likely to become even more muddled.
It does appear that community corrections, more than other public agencies, has had difficulty articulating its mission and reaching agreement as to appropriate performance indicators. But as budgets shrink, public stakeholders will be increasingly asking what they are getting-in objective, measurable terms-for their dollar. Within this context, probation and parole will become increasingly vulnerable to budget cuts if they cannot clearly articulate what they do, with whom, and to what effect.
This paper is designed to address that need. Specifically, it attempts to-
1 ) articulate a mission statement for community corrections
2) identify the goals contained within the mission statement
3) specify methods or activities that address each goal
4) identify measurable performance indicators for each goal.
This paper is clearly just a first step and is meant to stimulate discussion and
further work. Other goals could be added to the mission statement (for example, educate
the public, prevent crime), and additional (or different) methods and performance
indicators could be substituted for those indicated. Community corrections agencies should
customize their mission statement, methods, and performance indicators so that they
reflect local resources and priorities.
Step 1: Specify a mission statement for community corrections
A necessary first step toward developing performance indicators is to articulate the
organization's goals and mission. As Peters and Waterman, authors of In Search of
Excellence, put it: "Figure out what your value system is. Decide what your
company stands for..." (1982: 227). The same principle applies to social, nonprofit
organizations. Before community corrections agencies can evaluate their performance, they
must first define what they are attempting to accomplish.
As noted above, probation and parole have historically had difficulty defining their
mission (McAnany et al., 1984; Petersilia et al., 1985). Different goals are often given
by administrators and staff within the same depart meet. And sometimes the same person
gives different messages to different audiences, depending on his or her perception of
their support for rehabilitation versus enforcement activities. In some instances,
agencies adopt a mission or goal statement that they think will sell their services to the
external community, but that staff do not sincerely support. And the goals of the staff
might not be shared by their principal sponsors or customers (the judiciary, county
funders, and the public, in the case of community corrections). The result is that the
mission statements of many agencies become public relations gimmicks as opposed to working
statements that assist internal agency operations.
DiIulio (1991) advises criminal justice agencies to develop mission statements that-
1 ) include only activities that the agency can reasonably and realistically be
expected to fulfill
2) are fairly narrow and consistent in scope
3) contain activities that are unique to the justice agency, rather than conflated with
functions of other social institutions such as schools and welfare agencies.
His major message is that the mission statement should not ask the agency to "do the impossible" and it should contain activities over which it has direct control, rather than seek the achievement of more distant ends.
Nidorf (1991b) adds an additional requirement: mission statements must be responsive to
expectations held by the community and by other justice agencies. He argues persuasively
that community corrections operates within the justice system, not independent
from it, and that its goals
and activities-and eventual survival-depend heavily on how well it meets the expectations
of the public and peer agencies (police, courts).
Several attempts have been made to develop mission statements for community
corrections, most recently by the American Correctional Association (1986) and the
California Corrections Policy Project (1992). These statements, as well as those of
earlier years (see Fitzharris, 1979; McAnany, 1984; Clear and O'Leary, 1983), all embrace
the notions of public protection and offender rehabilitation. For instance, the California
Corrections Policy Projects' statement is:
The mission of community corrections is to protect the community; support the rights of
victims; enforce court-ordered sanctions; and assist offenders to change (CCPP, 1992).
The statement of the American Correctional Association (ACA) is broader, simply stating that the community corrections mission as sanction is to "enhance social order and public safety" (1986:58).
The American Probation and Parole Association (APPA) (1992) says that the goals of community corrections are to: "protect the community; deter criminal and drug activity; punish and rehabilitate offenders."
In fact, all of the mission statements identified were quite similar in their mention
of public protection and offender rehabilitation. Some also included assisting the court
in sentencing decisions, public education, and aiding victims (Fitzharris, 1979).
For purposes of this paper, the author adopts the following mission statement, which comes
from reviewing the literature as well as talking with the leadership of the APPA, National
Association of Probation Executives (NAPE), and the California Probation/Parole and
Correctional Association (CPPCA).
The mission of community corrections is to assist the court and/or parole board
in assessing candidates' suitability for community placement; and once offenders are
placed in the community, to enforce the court-ordered sanctions, protect the community,
assist offenders to change, and support the rights of their victims.
Step 2: Identify methods designed to accomplish each identified goal
The major objective in developing a clear mission or goal statement is that it defines
what the agency is about, what it hopes to achieve. This makes the next step possible:
specifying the agency activities that are designed to achieve (or move toward achieving)
the various goals. The table on pp. 78-79 takes the mission statement adopted for this
paper, breaks it down into individual goals, and then identifies specific community
corrections methods or activities that are designed to accomplish each goal. For example,
the first goal pertains to advising the court about an offender's suitability for
community placement. Community corrections staff do this in two principal ways: conducting
presentence investigations, and conducting investigations of parole or probation
violations. Methods are linked to each of the five major goals contained in the study
group's original mission statement.
Step 3: Specify performance indicators for each method or activity
Once the agency has identified its goals and the methods it uses to address each goal,
it can specify objective (measurable) performance criteria that determine the extent to
which the activities are being performed and the goals are being achieved.
Identifying indicators of probation and parole success is not as easy as it sounds. As
Clear and O'Leary (1983:67) note, many probation and parole staff are like social service
workers and tend to frame the substance of their work in terms of broad attitudinal or
otherwise nonspecific changes. Officers frequently identify the outcomes in terms of
changes in the client-but in often vague or ambiguous terms such as "ability to deal
with authority," or "improved self-image." These broad changes are
difficult to define operationally because they are subject to individual judgment-one
officer's definition of progress might be quite different from another's-and because there
are few objective measures for determining whether the goal has been met. For purposes of
performance measurement, we need objective, measurable outcome statements that are tied to
specific targeted activities that are believed to be related to the mission.
Column 3 of Table 1 ("Performance Indicators") specifies performance indicators
for each of the major goals in the mission statement. For example, one of the major goals
of community corrections is to protect the public. This is done by monitoring the
offender's behavior (through officer contacts), drug and alcohol testing, and so forth. To
objectively measure whether these activities protect the public, one would record the
number and types of arrests that occur during supervision and the number of offenders who
abscond during supervision. Likewise, one goal is to "assist the offender to
change." "Success" in achieving this goal might be measured by the number
of drug-free or alcohol-free days, number of times attending treatment, and objective
tests of attitude change.
It is important to mention two things at this point:
1 ) the performance indicators listed in Table 1 are quite different from those
currently used and include much more than recidivism, and
2) the measurements (including recidivism) reflect only activities that occur while
the offender is formally on community corrections status, not beyond.
It may be quite interesting to track offenders for some time period after their
supervision has ended and record their recidivism, but those rates should not be used, as
they have historically, as the primary measure of probation and parole effectiveness.
Criminal behavior is motivated by social and other factors over which the justice system
has little, if any, direct control. Hawkins and Alpert (1989) suggest that to ask the
justice system to assume responsibility for post-program behavior is akin to asking high
schools to assume responsibility for post-graduation employment. Success in high school is
defined as completing the course of study (not dropping out) and attaining some level of
knowledge and skills assessed by grades and standardized tests. Schools do not follow up
their graduates to see if they slip back into ignorance or fail to hold a job after
leaving school. But corrections programs are judged not on their immediate impact but on
their long-term effects: does the person refrain from crime after formal supervision has
ended?
It is the belief of the BJS/Princeton Study Group that the performance of
community corrections agencies ought not to be evaluated mainly or exclusively in terms of
their effects on recidivism rates, and that the success of community corrections should
not be based on some postprogram assessment of behavior.
Why bother with performance indicators?
Community corrections agencies do not currently collect information on most of the performance indicators listed in the table on pp. 78-79. To do so, agencies would first need to define their mission, identify activities related to that mission, and collect relevant performance indicators. Given their current budget woes, why would they want to take on this additional task?
There are a number of reasons. Completing such an exercise would assist them in
prioritizing activities and allocating resources. When the agency implemented a new
program or policy, data would be readily available to monitor (or even project) the
program's impact. Collecting data on the performance indicators on pp. 78-79 should also
increase staff morale, since these indicators measure activities they do control and will
likely show some success. Also, going through the exercise of defining mission,
activities, and performance indicators helps the organization-top management and field
staff-reach consensus on key issues. It also helps identify those priority areas that have
few related program activities. The most important reason, however, for completing this
exercise and collecting the necessary data is a practical one: without such information,
community corrections remains vulnerable to continued budget cuts. It has been shown that
those
who can quantify what they do, with whom, and to what effect have a strong competitive
advantage in budget negotiations.
Donald Cochran, Commissioner of Probation in Massachusetts (1991:38), articulates the practical advantages of engaging in such an exercise:
Probation managers need to use information to accurately determine what work needs to be done, the outcomes the agency will produce if resources are added, or likely program outcomes and organizational practices if resources continue to be reduced...information and knowledge driven probation programs can and will be funded (Cochran, 1991:36, underline in original).
Ronald Corbett (1991) expands by noting:
in a time when agencies are increasingly called to define their existence, the absence
of a clearly articulated philosophy will leave agencies vulnerable, unable to offer a
compelling vision of their contribution to the common good in a manner that justifies
continued existence...an agency that cannot offer a clear and convincing statement of its
reason for being will not survive the rough and tumble of competition for shrinking tax
dollars.
When public agencies fail to define their mission internally, political influences are
more apt to define it for them. And when they fail to articulate how they should be
evaluated, outcome measurements such as recidivism rates will likely be imposed upon them.
Over time, community corrections policy has become heavily influenced, not by those
practitioners who are most knowledgeable about it, but instead by public opinion, fear,
and political hype. One of the biggest challenges now facing community corrections is to
regain control of its profession. One direct and effective way to do this is by defining a
mission and showing-in measurable terms- that the agency is achieving it. With more solid
information, community corrections should again be able to inspire the confidence of
policymakers and the public, and ultimately secure the dollars necessary to perform
adequately the activities that help accomplish their mission.
The mission of community corrections is to assist the court and/or parole board in
assessing candidates' suitability for community placement; and once offenders are placed
in the community, to enforce the court-ordered sanctions, protect the community, assist
offenders to change, and support the rights of their victims.
Community corrections: Goals, methods, and performance indicators
Goals | Methods/Activities | Performance Indicators |
1. Assess the offender's suitability for placement: Community corrections advises the court and/or parole board on suitability of community placement | Conduct presentence investigations (PSI) Conduct investigations of violation reports | Accuracy and completeness of PSI, Timeliness of revocation and termination hearings, Validity of classification/prediction instrument, Percent of offenders receiving recommended sentence or violation act, Percent of offenders recommended for community who violate |
2. Enforce court-ordered sanctions: The court permits the offender to remain in the community if he/she adheres to certain conditions, which always includes no new crime and may include financial obligations and prohibitions on alcohol/drug use, peer associations, unemployment, etc. | Monitor police arrest and investigaton reports, Monitor vicitme
restitution and court fees, Monitor community service Conduct personal contacts and other monitoring (such as electronic), Verify employment, school performance, peer associations, Test for drug and alcohol use |
Number of arrest and technical violations during supervision, Percent of
ordered payments collected, Number of hours/days performed community service, Number of
favorable discharge Number of days employed in vocational education or school, Drug-free and/or alcohol-free days during supervision |
References
American Correctional Association (1986). Public Policy for Corrections. College Park, Md.: American Correctional Association.
American Probation and Parole Association (1992). Working Draft on Intensive Supervision Probation/Parole Training Manual.
Blair, Louis H., Harry P. Hatry, Karan Bunn et al. (1971). Monitoring the Impacts
of Prison and Parole Services: An Initial Examination. Washington, D.C.: The Urban
Institute.
California Corrections Policy Project (1991). California Corrections Policies for the
Future: Mission and Policy Statement. Sacramento: California Corrections Policy
Project.
Clear, Todd R., and James M. Byrne (1992). "The Future of Intermediate Sanctions: Questions to Consider," in James M. Byrne, Arthur J. Lurigio, and Joan Petersilia (eds.), Smart Sentencing: The Emergence of Intermediate Sanctions. Newbury Park, Calif.: Sage Publications.
Clear, Todd R., and Vincent O'Leary (1981). Controlling the Offender in the Community. Lexington, Mass.: Lexington Books.
Cochran, Donald (1991). "Classification/Information Systems" in Donald
Cochran et al., Managing Probation with Scarce Resources: Obstacles and Opportunities.
Washington, D.C.: National Institute of Corrections.
Cochran, Donald et al. (1991). Managing Probation with Scarce Resources: Obstacles and
Opportunities. Washington, D.C.: National Institute of Corrections.
Corbett, Ronald P., Jr. (1991). "How Did We Get in this Mess?" in Donald Cochran et al., Managing Probation with Scarce Resources: Obstacles and Opportunities. Washington, D.C.: National Institute of Corrections.
DiIulio, John J., Jr. (1991). No Escape: The Future of American Corrections. New York: Basic Books, Inc.
-(1992). Rethinking the Criminal Justice System: Toward a New Paradigm. Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics.
Doble, John (1987). Crime and Punishment: The Public's View. New York: Public Agenda Foundation.
Ellsworth, Thomas (1992). "The Goal Orientation of Adult Probation Professionals: A Study of Probation Systems," in Thomas Ellsworth (ed.), Contemporary Community Corrections. Prospect Heights: Waveland Press, Inc.
Fitzharris, Timothy L. (1979). Probation in an Era of Diminishing Resources. The Foundation for Continuing Education in Corrections. Sacramento, Calif.: The California Probation, Parole, and Correctional Association.
Flanagan, Timothy J., and Kathleen Maguire (eds.) (1992). Sourcebook of Criminal
Justice Statistics, 1991. U.S. Department of Justice, Bureau of Justice Statistics.
Washington, D.C.: U.S. Government Printing Office.
Fogel, David (1984). "The Emergence of Probation as a Profession in the Service of
Public Safety: The Next Ten Years," in Patrick D. McAnany, Doug Thomson, and David
Fogel (eds.), Probation and Justice: Reconsideration of Mission. Cambridge,
Mass.: Oelgeschlager,n and Hain.
Gottfredson, Michael R., and Don M. Gottfredson (1980). Decisionmaking in Criminal
Justice. Cambridge, Mass.: Ballinger.
Hawkins, Richard, and Geoffrey P. Alpert (1989). American Prison Systems: Punishment and Justice. Englewood Cliffs, N.J.: Prentice Hall.
Hindelang, Michael J., Michael R. Gottfredson, and Timothy J. Flanagan (eds.) (1981). Sourcebook of Criminal Justice Statistics, 1980. Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics.
Jacoby, Joseph E., and Christopher S. Dunn (1987). "National Survey on Punishment
for Criminal Offenses." Unpublished paper. Bowling Green State University, Bowling
Green, Ohio.
Jankowski, Louis W. (1992). Correctional Populations in the United States, 1990.
Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics.
Langan, Patrick A., and Mark A. Cunniff (1992), Recidivism of Felons on Probation,
1986-89, Special Report. Washington, D.C.: U.S. Department of Justice, Bureau of
Justice Statistics.
Lipton, D., R. Martinson, and J. Wilks (1975). The Effectiveness of Correctional
Treatment and What Works: A Survey of Treatment Evaluation Studies. New York: Praeger
Publishers.
Logan, Charles H. (1992). "Criminal Justice Performance Measures for Prisons," Performance Measures for the Criminal Justice System. Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics.
McAnany, Patrick, Doug Thomson, and David Fogel (1984). Probation and Justice: Reconsideration of Mission. Cambridge, Mass.: Oelgeschlager,n and Hain.
National Advisory Commission on Criminal Justice Standards and Goals (1973). Report on Corrections. Washington, D.C.: U.S. Government Printing Office.
Nidorf, Barry (1991a). " 'Nothing Works' Revisited," Perspectives, Vol. 15, No. 3, pp. 12-13.
-(1991b). "external Relations," in Donald Cochran et al., Managing Probation with Scarce Resources: Obstacles and Opportunities. Washington, D.C.: National Institute of Corrections.
Peters, Thomas, and Robert Waterman, Jr. (1982). In Search of Excellence. New York: Harper & Row.
Petersilia, Joan, and Susan Turner (1990). Intensive Supervision Probation for
High-Risk Offenders: Findings from Three California Experiments. R-3936-NIJ/BJA.
Santa Monica, Calif.: RAND.
-(1993). "Intensive Probation and Parole Supervision: Research Findings and Policy
Implications," in Michael Tonry (ed.), Crime and Justice, Vol. 17. Chicago:
University of Chicago Press.
Petersilia, Joan, Susan Turner, James Kahan, Joyce Peterson(1985). Granting Felons
Probation: Public Risks and Alternatives. R-3186-NIJ. Santa Monica, Calif.: RAND.
President of the United States (1990). National Drug Control Strategy.
Washington, D.C.: U.S. Government Printing Office.
President's Commission on Law Enforcement and Administration of Justice (1967). The
Challenge of Crime in a Free Society. Washington, D.C.: U.S. Government Printing
Office.
About the author
Joan Petersilia is director of RAND's Criminal Justice Program and Associate Professor
in the Department of Criminology, Law and Society, School of Social Ecology, University of
California, Irvine. She has directed major studies in policing, sentencing, career
criminals, corrections, and racial discrimination. For the past several years, her work
has focused on the effectiveness of probation and parole.
Dr. Petersilia has served as president of the American Society of Criminology and of the
Association for Criminal Justice Research in California. She is a fellow of both the
American Society of Criminology and the Western Society of Criminology and has received
awards from the American Probation and Parole Association and the California Probation,
Parole, and Corrections Association.
She is coauthor of Smart Sentencing: The Emergence of Intermediate Sanctions
(1992). Her latest major publications include "Intensive Probation and
Parole" (1993); "Crime and Punishment in California: Full Cells, Empty
Pockets, and Questionable Benefits" (1992); Intensive Supervision for
High-Risk Probationers: Findings from Three California Experiments (1990); Expanding
Options for Criminal Sentencing (1987); Police Performance and Case Attrition
(1987); and The Influence of Criminal Justice Research (1987).
Performance Measures for the Trial Courts, Prosecution, and Public Defense
by George F. Cole
Introduction
Thurman Arnold has argued that the central ideals of every Western government are symbolized in its judicial system: In particular, that the criminal trial "overshadows all other ceremonies as a dramatization of the values of our spiritual government, representing the dignity of the State as an enforcer of law, and at the same time the dignity of the individual when he is an avowed opponent of the State, a dissenter, a radical, or even a criminal."1
Although most Americans never visit the criminal courts, those with firsthand experience-defendants, witnesses, jurors-and those who learn about court processes and trials through the media are greatly influenced by their perceptions of the quality of justice. Their observations would contribute to citizen education about the role of law in society, the legitimacy of the criminal justice system, and ultimately to citizen willingness to abide by the law.
1. Thurman Arnold, The Symbols of Government (New York: Harcourt, Brace and World, 1962), pp. 128-130. |
Adjudication-prosecution, defense, and courts-is the subsystem of criminal justice that
perhaps more than either law enforcement or corrections is the component that the public
expects to allocate justice. Judges, prosecutors, and defenders are all attorneys who have
sworn to uphold the ideals of justice and to act according to the canons of the legal
profession. Judicial decisions are more visible than actions in police stations, probation
offices, or prisons; hence, the work of the courts is more open to public scrutiny. Not
only do courts make decisions in individual cases according to the concept of due process
but judges are charged with ensuring that the other components of the system operate
according to the law. Thus, as Herbert Packer has emphasized, it is not enough that the
law be fair; it must be seen to be fair.2
How should the performance of the agencies of adjudication be measured? What should be the
standards expected of trial courts, prosecution and defense offices, and the individuals
who work in them? These are enduring questions and have been raised about American trial
courts throughout this century.
Beginning in the 1970's, a series of federally funded studies addressed the issue of
performance indicators. To varying degrees, the studies stressed measures emphasizing
process rather than results, efficiency rather than effectiveness, and program outcomes
rather than policy outcomes.3 As a consequence, we know a lot about conviction
rates, numbers of dismissals, percentages of guilty pleas, case processing times, and the
use of various sentencing options-yardsticks that say little about the quality of justice.
Few offices of prosecution, indigent defense, or courts have developed and incorporated
performance assessments to evaluate the quality of the behavior of officials as they
interact with citizens-be they defendants, victims, jurors, or the general public.
2. Herbert L. Packer, The Limits of the Criminal Sanction
(Stanford, Calif.: Stanford University Press, 1968), p. 69. 3. See Sorrel Wildhorn, Marvin Lavin, and Anthony Pascal, Indicators of Justice (Lexington, Mass.: Lexington Books, 1977); Joan E. Jacoby, Basic Issues in Prosecution and Public Defender Performance (Washington, D.C.: National Institute of Justice, 1982); Thomas J. Cook and Ronald W. Johnson, with Ellen Fried, John Gross, Mary Wagner, and James Eisenstein, Basic Issues in Courts Performance (Washington, D.C.: National Institute of Justice, 1982). See also National District Attorneys Association, National Prosecution Standards, 2nd ea., (Alexandria, Va.: National District Attorneys Association, 1991); Roberta Rovner-Pieczenik, Alan Rapoport, and Martha Lane, Evaluation Design for Public Defender Offices (Washington, D.C.: National Institute of Law Enforcement and Criminal Justice, 1977). |
Since 1987 the National Center for State Courts has been working on the development and implementation of measurable performance standards to be used in general jurisdiction State trial courts. The center's goal has been not measures of structure and process but of performance, "what courts actually do with the means at their disposal." 4 This paper first traces the court reform movement so as to describe the assumptions that have guided trial court evaluation for the past half century and then fully explores the performance standards developed by the center. It is not the intention to ignore efforts to measure prosecutor and indigent defense performance but, rather, to use the experience of the courts as a demonstration of the types of standards and measures that could be adopted by other agencies in the adjudication subsystem of criminal justice.
4. National Center for State Courts, Commission on Trial Court Performance Standards, Trial Court Performance Standards (Williamsburg, Va.: National Center for State Courts, 1990), p. 1. |
Reforming the courts
The 1906 speech by Roscoe Pound to the American Bar Association, entitled "The Causes of Popular Dissatisfaction with the Administration of Justice," has been referred to as "the spark that kindled the white flame of progress." 5 Pound pointed to organizational inadequacies of the judicial system and asserted that there were too many courts and, in consequence, duplication and inefficiency; further, that there was a great waste of judicial power because of rigid jurisdictional boundaries, poor use of resources, and the frequent granting of new trials.6
Pound's call to action fell on sympathetic ears among the reformers of the Progressive Era, who thought that structural changes would lead to a more efficient and equitable administration of justice. The American Bar Association and the American Judicature Society set out to press the States to remove party politics from judicial selection, increase funding for the courts, consolidate and simplify court structures, and create a force of professional administrators. Proponents of the reforms emphasized that their adoption would "serve the court's basic task of determining cases justly, promptly, and economically."7 Referred to as the "conventional wisdom of State court administration" by later scholars, the thrust of this orientation became "the accepted theory of State court organization and administration for almost 70 years."8
5. Roscoe Pound, "The Causes of Popular Dissatisfaction with the
Administration of Justice," reprinted in Journal of the American Judicature
Society, 20 (1937):178-187; John H. Wigmore, "Roscoe Pound's St. Paul Address of
1906," Journal of the American Judicature Society 20 (1937): 136. 6. See also Roscoe Pound, "Organization of Courts," Judicature 11 (1927): 69; Pound, "Principles and Outline of a Modern Unified Court Organization," Judicature 23 (1940):225. 7. American Bar Association, Commission on Standards of Judicial Administration, Standards Relating to Court Organization (Chicago: American Bar Association, 1974), p.1. 8. Geoff Gallas, "The Conventional Wisdom of State Court Administration: A Critical Assessment and an Alternative Approach," Justice System Journal 2 (Spring 1976): 35-55. Gallas cites as examples of the conventional wisdom such reports as American Bar Association Commission on Standards of Judicial Administration, Standards Relating to Court Organization (Chicago: American Bar Association, 1974) and National Advisory Commission on Criminal Justice Standards and Goals, Courts (Washington, D.C.: Government Printing Office, 1973). |
Among the problems affecting the judiciary, case processing delay in the criminal trial courts has been a continuing focus for concern and reform. The Sixth Amendment to the Constitution mandates a speedy trial, yet throughout this century there is much evidence that the processing of some cases can take months, even years. The voluminous literature on delay underscores that case processing time beyond that which is necessary for a fair resolution of the issues greatly undercuts the quality of justice. It is argued that "justice delayed is justice denied" because witnesses may move or die, memories may fade, and the accused incurs costs because of pretrial restrictions on freedom, loss of income, and expenses run up in mounting a defense. The accumulation of case delay is seen as producing backlogs that "waste court resources, needlessly increase lawyer fees, and create confusion and conflict in allocating judges' time."9
Case-processing time has become one of the major criteria of trial court performance,
akin to the use of arrest and recidivism rates in evaluating the police and corrections.
The speed with which cases are processed has been viewed as having a determinative impact
on numbers of cases handled, amount and types of resources required, and quality of
justice allocated. Attacking trial court delay has thus been a focus of reformers and
researchers. Goerdt notes that since 1976 there have been at least eight national research
projects and a number of smaller studies that have examined the pace of litigation. 10
Attention to the problem has led to the adoption of disposition-time standards by the
American Bar Association, the Conference of Chief Justices, and the Conference of State
Court Administrators.
In keeping with the conventional wisdom that the inadequacies of courts are related to
their structure, efforts were advanced during the past quarter century to reduce delay
through unification of court systems, centralization of management, and expansion of
resources. However, by the late 1970's research conducted by the National Center for State
Courts began to challenge the assumptions that delay could be explained by such factors as
court structure, size, caseload per judge, and the type of case calendaring. 11
For example, as noted by one major study of the effect of structure on trial court
operations, "Unification may provide the framework in which the quality of justice
can be enhanced, but it seems doubtful that unification has a direct impact upon
quality."12
The fact that "the link between a court's effectiveness and its pace of litigation is
taken as faith" was addressed by Gallas, who argued that judicial leadership must be
considered as a factor influencing performance.13 Other researchers argued that
delay-reduction efforts should focus on such factors as "more effective court
leadership, commitment to achievement of disposition time goals, early and continuous
court control over caseload, and increased judicial accountability for case
processing."14 Delay continued to be a primary measure of trial court
performance, but explanations for its existence and methods for its reduction shifted.
9. American Bar Association, National Conference of State Trial Judges, Standards
Relating to Court Delay Reduction (Chicago: American Bar Association, 1984), p. 5
(commentary to Sec. 2.50 of the Standards Relating to Trial Courts as amended in 1984).
Cited in Barry Mahoney et. al., Changing Times in Trial Courts (Williamsburg,
Va.: National Center for State Courts, 1988), p. 2. 10. John Goerdt, Examining
Court Delay: The Pace of Litigation in 26 Urban Trial Courts, 1987 (Williamsburg,
Va.: National Center for State Courts, 1989), p. 4. |
Nonstructural influences on court performance
Recognition that court performance is related to factors other than structure has
emerged through research completed during the past two decades. Even with the overall
similarity of organizational structures and formal processes of adjudication, differences
among the trial courts are discernible to anyone who has visited American courthouses.
Some courts sentence offenders to longer terms than do others. Delay and tight pretrial
release policies keep some accused in jail awaiting trial, whereas in other jurisdictions
similar defendants are out on bail or have their cases expeditiously resolved. Guilty
pleas make up 90% of dispositions in some communities but only 60% in others. Researchers
have found that there are differences in the processes and outputs of courts among various
cities and among courtrooms in the same city even when the formal structures, laws, and
procedures are the same. The concepts of "local legal culture" and
"courtroom workgroup" have been particularly useful in helping to understand
these variations.
The concept of local legal culture-norms shared by members of a particular court community
(judges, attorneys, clerks, bailiffs, and others) as to case handling and participants'
behavior in the judicial process-has been shown to have a decided impact on court
processes and outcomes.15 Through the local legal culture informal rules and
practices arise within particular settings, and the "way things are done"
differs from place to place. The customs and traditions of each jurisdiction seem to vary
because local practices are affected by such factors as size, politics, and population
characteristics.16
15. Thomas W. Church, Jr. "Examining Local Legal Culture," American
Bar Foundation Research Journal (Summer, 1985), p. 49 16. For a study that analyzes the role of local legal culture of nine communities see: James Eisenstein, Roy B. Flemming, and Peter F. Nardulli, The Contours of Justice: Communities and Their Courts (Boston: Little, Brown, 1988). |
Adjudication is also influenced by the fact that courtroom participants are organized as a workgroup.17 From this perspective, the reciprocal relationships of the judge, prosecutor, and defense attorney, along with those of the supporting cast (clerk, reporter, and bailiff), are necessary to complete the group's basic task: the disposition of cases within the context of the laws and the concept of justice. As pointed out by Nardulli, the "courtroom elite" controls the operations of the court and, hence, most directly bears upon court performance.18 Research has shown that the outputs of courtroom workgroups differ, not because of the formal structure and procedures but because of level of cohesiveness, the influence of participants' sponsoring organizations (prosecution, judiciary, public defense), and the nature of the leadership of the group. In essence, the workgroup operates in an environment in which the local legal culture, recruitment and selection processes, cases, and the socioeconomic, political, and legal structures of the broader community are conceived as having an impact on decisionmaking.19 Court performance is thus greatly influenced by these factors.
17. James Eisenstein and Herbert Jacob, Felony Justice (Boston:
Little, Brown, 1977). 18. Meter F. Nardulli, The Courtroom Elite: An Organizational Perspective on Criminal Justice (Cambridge, Mass.: Ballinger, 1978). 19. See Thomas J. Cook, Ronald W. Johnson et. al., Basic Issues in Courts Performance (Washington,D.C.: National Institute of Justice, 1982), p. 38. |
How should trial courts be measured?
By the mid-1980's it was apparent to judicial administrators, judges, and scholars that
the longtime focus on the structure of the courts and processing times and assumptions
about ways that performance should be looked at had collided with research depictions of
court operations as much under the sway of the local legal culture, inter-organizational
links, and the interactions among members of the courtroom workgroup. In addition, critics
argued that a concern for efficiency in case processing can lead to injustice and
undermine the judiciary's institutional integrity.20
At a 1986 international conference on judicial administration, calls were made for greater
attention to performance measures. Gallas declared that beyond the goal of reforming
organizational structure is the need for effective leadership.21 He argued that
evaluating trial courts requires examination of the links between judicial leadership and
court performance. To differentiate successful from unsuccessful courts, he and others
argued, it is necessary to go beyond disposition times to overall performance. Feeney
emphasized that evaluating performance incorporates a clear definition of goals, adequate
evaluative information, and meaningful standards for judging performance. "About the
only thing that is clear is that if the courts are to improve during the next thirty years
everyone associated with the courts- managers, judges, researchers, and others-must
radically increase the attention to evaluation of performance."22
20. Judith Resnik, "Managerial Judges and Court Delay: The Unproven
Assumptions," 23 Judges' Journal 8 (1984); H. Lee Sarokin, "Justice
Rushed Is Justice Ruined," 38 Rutgers Law Review, 431 (1986). 21. Geoff Gallas, "Judicial Leadership Excellence: A Research Prospectus," Justice System Journal 12 (1987), p. 54. 22. Floyd Feeney, "Evaluating Trial Court Performance," Justice System Journal, 12 (1987) p. 167. |
Developing new measures of trial court performance
In their insightful book, Reinventing Government, Osborne and Gaebler describe
some of the problems of measuring the performance of public organizations.23
They note that unlike private businesses, government agencies seldom focus on outcomes
because assessment is so difficult. Developing suitable measures may take years; they may
be appropriate to outputs but not outcomes; they may define outcomes too narrowly, so that
employees concentrate on only a few goals; or they may be so numerous that employees do
not know what to concentrate on.
Citing the experience of the National Center for State Courts, Osborne and Gaebler say
that public organizations that have tried to measure performance have learned that there
is (l) a vast difference between measuring process and measuring results; (2) a vast
difference between measuring efficiency and measuring effectiveness; and (3) an important
difference between "program outcomes" and broader "policy outcomes."
They also describe the difficulties of developing acceptable performance standards and
measurement tools and the resistance of public service providers to new approaches.
In 1987 the National Center for State Courts initiated the Trial Court Performance
Standards Project to develop measurable performance standards for the State trial courts
of general jurisdiction. The project was begun after researchers at the center recognized
the shortcomings of the focus on the structure and machinery of trial courts and moved on
to questions relating to the goals of trial courts. What is it that courts are trying to
do? What should courts be trying to accomplish with the resources at their disposal? What
is the role of judicial leadership in improving the trial courts? How do we identify
courts that are doing well?
23. David Osborne and Ted Gaebler, Reinventing Government (New York: Praeger, 1992); see especially Appendix B. There is a large literature in public administration on the measurement of public goods and services. See, for example, Vincent Ostrom, Charles M. Tiebout, and Robert Warren, "The Organization of Government in Metropolitan Areas: A Theoretical Inquiry," American Political Science Review, 55 (December, 1961): pp. 83 1 -842. |
With funding from the Bureau of Justice Assistance, the center created the Commission on Trial Court Performance Standards, which took on the arduous task of redirecting research attention to the outcome aspect- what trial courts actually accomplish-and to developing tools to assess outcomes according to acceptable criteria. The commission focused solely on institutional rather than individual performance and promoted the approach as a self-assessment tool for the trial courts.24
Over a 3-year period the commission and center staff developed a set of 22 standards in 5 areas to measure trial court performance, developed a comprehensive measurement system, field-tested the standards and the system, and gained acceptance of the new criteria by key judicial organizations and several States.25
24. The evaluation of the performance of individual judges has been
ongoing in many States for nearly two decades. Much of this effort has been spearheaded by
the American Bar Association Special Committee on Evaluation of Judicial Performance and
the Judicial Performance Evaluation Project of the National Center for State Courts. See
Susan Keilitz and Judith White McBride, "Judicial Performance Evaluation Comes of
Age," State Court Journal (Winter 1992): 4-13. 25. National Center for State Courts, Commission on Trial Court Performance Standards, Trial Court Performance Standards (Williamsburg, Va.: National Center for State Courts, 1990). |
Let us examine the five performance areas as they relate to the key concepts suggested
by the call in the first paper in this compendium for a new paradigm by which to
understand and measure the criminal justice system.26 The paradigm emphasizes
the important role played by citizens as co-producers of justice. It recognizes that
police, courts, and corrections require the cooperation and support of citizens if they
are going to be successful at their jobs. The paradigm also recognizes that like
successful private organizations, criminal justice agencies need to be measured according
to "soft" performance indicators: how citizens are treated, how they evaluate
performance, and the level of trust and confidence of citizens in their institutions.
The five performance areas for State trial courts decided upon are:
As noted by the commission, the groupings represent alternative ways of viewing the fundamental purposes of trial courts as they allocate justice in individual cases, resolve disputes, uphold and promote the rule of law, protect individuals from the arbitrary use of governmental power, and encourage behavior that adheres to societal norms as expressed in statutes and regulations. Most people probably would agree that these are the things that trial courts do and that the five areas summarize public expectations of court performance. But as the commission discovered, it is one thing to agree on designation of performance areas in the abstract but it is another thing to develop and define specific, measurable standards. Each of the five performance areas is summarized below and the specific standards associated with them are listed.27
27. See the appendix of the commission's report for a listing of each performance measure, the data collection methods, and the personnel to be included in the evaluation. |
Standard 1. Access to justice
In a democracy courts must be open and accessible. The concept of accessibility refers not only to physical structure but also to the procedures and the responsiveness of personnel to members of the public:
Standard 1.1 Public proceedings
The trial court conducts its proceedings and other public business
openly.
Standard 1.2 Safety, accessibility, and convenience
The trial court facilities are safe, accessible, and convenient to use.
Standard 1.3 Effective participation
All who appear before the trial court are given the opportunity to
participate effectively, without undue hardship or
inconvenience
Standard 1.4 Courtesy, responsiveness, and respect
Judges and other trial court personnel are courteous and responsive to
the public and accord respect to all with whom they
come in contact.
Standard 1.5 Affordable costs of access
The costs of access to the trial court's proceedings and
records-whether measured in terms of money, time, or procedures
that must be followed-are reasonable, fair, and affordable.
Standard 2. Expeditiousness and timeliness
The trial court should meet its responsibilities to all persons and agencies affected by its actions and activities in an expeditious and timely manner. Unnecessary delay causes injustice and hardship, and in turn diminished public trust and confidence in the court.
Standard 2.1 Case processing
The trial court establishes and complies with recognized guidelines for
timely case processing while at the same time keeping
current with its incoming caseload.
Standard 2.2 Compliance with schedules
The trial court disburses funds promptly, provides reports and
information according to schedules, and responds to requests
for information and other services on an established
schedule that assures their effective use.
Standard 2.3 Prompt implementation of law and procedure
The trial court promptly implements changes in law and procedure.
Standard 3. Equality, fairness, and integrity
The trial court should provide due process and equal protection of the law to all who
have business before it. Equality and fairness demand equal justice under law. Integrity
should characterize the nature and substance of the trial court's procedures and decisions
and the consequences of the decisions.
Standard 3.1 Fair and reliable judicial process
Trial court procedures faithfully adhere to relevant laws, procedural
rules, and established policies.
Standard 3.2 Juries
Jury lists are representative of the jurisdiction from which they are
drawn.
Standard 3.3 Court decisions and actions
The trial court gives individual attention to cases, deciding
them without undue disparity among l like cases and upon legally
relevant factors.
Standard 3.4 Clarity
Decisions of the trial court unambiguously address the issues presented
to it and make clear how compliance can be
achieved.
Standard 3.5 Responsibility for enforcement
The trial court takes appropriate responsibility for the enforcement of
its orders.
Standard 3.6 Production and preservation of records
Records of all relevant trial court decisions and actions are accurate
and properly preserved.
Standard 4. Independence and accountability
The judiciary must assert and maintain its distinctiveness as a separate branch of
government. Within the organizational structure of the judicial branch, the trial court
must establish its legal and organizational boundaries, monitor and control its
operations, and account publicly for its performance.
Standard 4.1 Independence and comity
The trial court maintains its institutional integrity and observes the
principle of comity in its governmental relations.
Standard 4.2 Accountability for public resources
The trial court responsibly seeks, uses, and accounts for its public
resources.
Standard 4.3 Personnel practices and decisions
The trial court uses fair employment practices.
Standard 4.4 Public education
The trial court informs the community of its programs.
Standard 4.5 Response to change
The trial court anticipates new conditions or emergent events and
adjusts its operations as necessary.
Standard 5. Public trust and confidence
The public's compliance with the law is dependent to some degree upon its respect for the courts. Ideally, public trust and confidence in trial courts stem from the many contacts citizens have with the courts.
Standard 5.1 Accessibility
The trial court and the justice it delivers are perceived by the public
as accessible.
Standard 5.2 Expeditious, fair, and reliable court
functions
The public has trust and confidence that the basic trial court
functions are conducted expeditiously and fairly and that its
decisions have integrity.
Standard 5.3 Judicial independence and accountability
The trial court is perceived to be independent, not unduly influenced
by other components of government, and accountable.
Implementing trial court performance evaluation
From its inception the commission recognized that trial court performance evaluation would require-
Having promulgated the goals and standards, as outlined above, the commission next
addressed the tasks of developing measures of performance, outlining procedures for data
collection, field testing the procedures, and implementing the new approach in trial court
demonstration sites.
The commission and its staff spent many hours wrestling with the operational definition of
the standards so that measurement criteria could be developed and data collection
generated. It was recognized that both quantitative and qualitative measures would be used
and that data collection efforts had to be feasible, given consideration of costs and the
availability of data needed for specific measures. Questions of implementing the process
colored these discussions. The Appendix provides examples of the measures, data collection
methods, and the persons to be used as evaluators. As can be seen, not only are members of
the judicial staff involved in the evaluation of a court's performance but qualitative
data are gathered from the consumers of justice-attorneys, jurors, litigants, witnesses,
and community leaders.
With distribution of copies of the Trial Court Performance Standards in November
1990, the commission sought and gained endorsement of the standards by judicial and court
management professional organizations, began training programs for persons who would
conduct the evaluations, and launched a series of demonstrations in New Jersey, Ohio,
Virginia, and Washington. Not unexpectedly, implementing the standards in the
demonstration States brought out the need to revise several of the measurement methods.
It is well recognized that instituting change in organizations can be difficult.28
Personnel are especially resistant when new standards are instituted and when measurement
bears directly on their performance. One way to deal with the resistance of individual
service providers is involve them in development of the measures and to couch
incorporation of the performance evaluation as part of an ongoing self-assessment. The
experience of the commission's work in the demonstration States should assist others with
the implementation of performance measures.
28. Osborne and Gaebler, Reinventing Government, pp. 355359, summarize the "received wisdom" of experts in the performance measurement field with regard to implementation. |
Summary
The work of the Commission on Trial Court Performance Standards of the National Center for State Courts serves as a useful illustration of the development of goals and evaluation methods more in keeping with a criminal justice paradigm that emphasizes citizens as co-producers of justice. As we have seen, the commission's standards-
It is hoped that incorporating these measures into the management of State trial courts
will not only provide an important management tool to make the judiciary more responsive
to the "democratic vision" but also guide leaders in other sections of the
criminal justice system- police, prosecution, defense, and corrections-in the development
of their own standards and measurement tools.
About the author
George F. Cole is Professor and Head of the Department of Political Science at The
University of Connecticut. A specialist on the administration of criminal justice, he has
published extensively on such topics as prosecution, the courts, and corrections. He has
been granted two awards under the Fulbright-Hays Program to conduct research in England
and Yugoslavia, and has held a fellowship at the National Institute of Justice. In
addition to his scholarly publications he is the author of the widely used introductory
textbooks, The American System of Criminal Justice, 6th edition (Brooks/Cole,
1992) and with Todd Clear, American Corrections, 3rd edition (Wadsworth, 1994).
Appendix: Selected measures and data collection methods
Measure | Object or Subject | Methods | Evaluators |
1.1.1 Open hearings | Court proceedings | Structured observation | Volunteers |
1.1.2 Understanding court calendar | Court calendar and proceedings | Structured observation | Volunteers |
1.2.3 Perceptions of courthouse security | Regular users of court | Questionnaire survey | Court staff |
1.2.4 Knowledge of emergency procedures | Court employees | Interviews | Court staff; graduate students |
1.4.3 Litigant treatment | Court proceedings | Structured observation; questionnaire | Volunteers |
1.5.1 Alternatives for the finacially disadvantaged | Court services, Court resources | Structutred observation; document and review; interviews; inventory | Data collection team |
3.1.2 Performance in applying the law | Attorneys; court employees | Questionnaire survey | Court staff |
3.2.1 Jury source list | Juror source list | Statistical analysis | Court staff |
3.3.3 Jury selection procedures | Juror selection | Observation | Court Staff |
4.5.2 Anticipation of charge | Potential issues | Questionnaire survey; Delphi technique; group review | Court staff; consultant |
5.(1-3).2 Public perceptions | General public | Telephone survey | Court staff; consultant; graduate student |
5.(1-3).2 Community leaders' perceptions | Local bar; law enforcement agencies; local Government; news media | Focus group | Professional group |
Measuring Police Performance in the New Paradigm of Policing 1
by Geoffrey P. Alpert and Mark H. Moore
Introduction
During the 1980's and 1990's there has been a resurgence of interest in community policing. As an outgrowth of police-community relations, the concept of community policing has become the goal, method, and guiding principle for police. Unfortunately, community policing remains a concept and philosophy in search of a process, without proper ways to document or evaluate its efforts. This essay in the BJS-Princeton series focuses on community-oriented policing and takes a new approach to the measurement and evaluation of police performance. Before outlining our paradigm of police performance measures we will review the conventional measures and why we believe a new way of thinking must direct our attention to new performance measures.
1. Portions of this essay have been taken from Alpert and Dunham, Policing Multi-Ethnic Neighborhoods (1988) and Policing Urban America (1992). |
Citizens and their elected representatives have long sought a bottom line to measure
police performance. The goals have been to reassure the public that hard-earned tax
dollars were being spent to achieve important results and to hold police managers
accountable for improving organizational performance. As police agencies matured, four
generally accepted accounting practices became enshrined as the key measures to evaluate
police performance. These include--
1) reported crime rates
2) overall arrests
3) clearance rates
4) response times.
As these measures became institutionalized over the years, investments were made in developing information systems to record police performance consistent with these measures. Statistical reports using these measures were routinely issued. Further, the media, overseers in city councils, and auditors in city managers' offices have all been primed to acknowledge and use these measures to compare police performance from year to year and to compare local accomplishments with those of other cities. For most practical purposes, these are the statistics by which police departments throughout the United States are now held accountable.
These measures remain critical as part of an overall system for measuring police performance. As currently used, however, these measures reflect an increasingly outmoded model of police tasks and fail to capture many important contributions that police make to the quality of life. More important, these measures may misguide police managers and lead them and their organizations towards purposes and activities that are less valuable than others that can be achieved with limited and diminishing resources.
Police performance measures should focus on a new model of policing that emphasizes
their charge to do justice, promote secure communities, restore crime victims, and promote
noncriminal options-the elements of an emerging paradigm of criminal justice (DiIulio,
1992: 10-12). The purpose of this paper is to describe how policing fits in with this new
paradigm, including implications for restructuring the overall objectives and measuring
the accomplishments of policing through police agency performance measures (Kelling,
1992).
The evolving strategy of policing
Historically, policing in America has been inspired and guided by a vision of professional law enforcement. This vision is a coherent strategy of policing defining the principal ends, means, and legitimating principles of the police enterprise (Wilson and McLaren, 1977).
Professional law enforcement: The dominant strategy of policing
In this vision, the primary, perhaps exclusive goals of the police are to reduce crime
and criminal victimization. Police seek to achieve this goal by arresting and threatening
to arrest those who violate the criminal law. They organize themselves to produce this
result by:
1 ) patrolling city streets hoping to detect and deter crime
2) responding rapidly to calls for service
3) conducting investigations after crimes have been committed to identify criminal
offenders and develop evidence to be used in prosecutions.
In essence, in the vision of professional law enforcement, the police are seen as the
all-important entry point to the criminal justice system-the gatekeeper managing the first
step in bringing the force of the criminal law to bear on offenders.
To deal effectively with serious crime and dangerous criminal offenders, specialized
skills are required. The police have had to learn how to use legitimate force with skill
and confidence. They have had to improve their ability to investigate and solve crimes to
reduce the chance that serious offenders could escape accountability. Thus, in search of
increased effectiveness in dealing with an increasingly challenging and urgent problem,
the police consciously narrowed their focus and refined their skills in responding to
serious crime and dangerous offenders. By relying on the techniques of patrol, rapid
response, and retrospective investigation, the police have been kept at the forefront of
community life and have been made available to anyone who needed them when a crime
occurred.
Limitations of professional law enforcement
Recently, enthusiasm for this strategy of professional policing has waned. The
professional policing model has been ineffective in reducing crime, reducing citizens'
fears, and satisfying victims that justice is being done. Indeed, recent research
indicates that a majority of the population believes that the crime problem has become
progressively worse during the past decade (Gallup, 1992, cited in Bureau of Justice
Statistics, 1992: 185). Similarly, citizens have lost confidence in the criminal justice
system to protect them (Cole, 1992: 23).
Such charges are, in many respects, unfair to the police. It is unreasonable to expect the
police to reduce crime all by themselves. Crime rates are affected by vast social,
economic, and political forces. No matter how professional, police cannot solve the
"root causes" of crime. They cannot be blamed for increasing unemployment,
increasing inequality, or eroding family structures (Bazelon, 1988). In addition, police
are dependent on the rest of the criminal justice system to give significance to arrests.
Toward a new paradigm of policing
Many police executives are beginning to think about and experiment with a strategy of
policing that differs from the professional model and emphasizes the development of a
strong relationship with the community. The essence of this new paradigm is that
police must engage in community-based processes related to the production and maintenance
of local human and social capital. The means by which these lofty goals are to be achieved
are through the development of strong relationships with institutions and individuals in
the community. While the specific elements of this new strategy of policing have
not been agreed upon or clearly delineated, the broad characteristics are reasonably
clear.
The major theme of building a strong relationship with the community has
two justifications. First, it is an important way to make enforcement more effective.
Second, it is a way to prevent crime and make the community co-producers of justice
(Skogan and Antunes, 1979).
One excellent example comes from the Metro-Dade Police Department (MDPD) in Miami, Fla. In
June 1992 the staff of the Northside Station of the MDPD conducted a survey of local
residents (mostly African-American) to determine if any public personalities or activities
could serve as common ground between the police and young males (Metro-Dade Police
Department, 1992). What emerged was a fascinating finding. The young respondents
identified local rap radio disc jockeys and rap music as personalities and activities that
interested them.
In March 1993 the police turned these empirical findings into action. They created a
series of "Jammin' with the Man" concerts. Local disc jockeys were invited to
hold concerts in local parks sponsored by the police. While the youths enjoyed the music
and festivities, the police were there, talking with the youths and encouraging them to
talk and work with the police to understand each other. Although more than 5,000 people
attended the first event, there were no negative incidents. The MDPD report concluded by
noting:
While Jammin' with the Man was originally intended to be a single step in a
process to improve police community relations, a step aimed particularly at young men,
[it] seems to have become part or all of the answer. It has also become an educational
experience for the community as they see police as agents of peace rather than
enforcers of law. More
importantly, it has demonstrated that the mere act of the police engaged in active
listening has the effect of empowering them and perhaps alleviating some of their sense of
alienation (Metro-Dade Police Department, 1993: 6).
In other words, this project provided an excellent vehicle for the police to create and
maintain positive contacts with members of the community they serve and to be seen in a
positive light. Further, by initiating and participating in activities the youths enjoyed,
the police had an opportunity to see youth in a positive light.
Dr. Trevor Bennett has classified the various ways to consider community policing and has
reduced them to three categories. First, he notes that there are arguments which refer to
the intrinsic "goodness" of the general relationship between police and the
community. Second, he recognizes relationships in which the police and the public work
together to achieve common and specified goals, including the shared responsibility for
crime control. Third, he acknowledges the need for police to take into consideration the
wishes and concerns of the community. In Bennett's words:
[A] workable definition of a community policing philosophy might include the following
basic elements: a belief or intention that the police should work with the public whenever
possible in solving local problems and a belief that they should take account of the
wishes of the public in defining and evaluating operational police policy (Bennett, 1992:
7).
A second theme emphasizes attacking the communities' problems on a broader
front-in effect, rejecting the exclusive focus on serious crime. The theme
emerging from research is that much fear of crime is independent of victimization and that
there are things the police can do to deal with fear (Bureau of Justice Statistics, 1988,
1992). Research findings and practice make clear that citizens use the police for many
purposes other than crime control and that things other than crime are principal concerns
(Alpert and Dunham, 1992: 2-3). Certainly, goals other than the reduction of serious crime
should be emphasized when it is realized that crime control is not the principal or only
objective of the police. In any case, the police cannot achieve the reduction of fear or
crime by themselves. What the police can achieve is the independent goal of public or
customer satisfaction.
A third theme emphasizes some important changes in the way the police visualize
their work and their methods. In the traditional strategy of policing, the key
unit of work is the "incident." That is, patrol officers respond to a
specific incident, and it is the incident that becomes the focus of a criminal
investigation. What we have recently learned, however, is that a large proportion of
incidents emerge from a relatively small number of situations and locations. Moreover,
analysis of the problems underlying many incidents reported to the police suggests that
the police might be able to imagine and mount different kinds of intervention (Goldstein,
1990).
The concept known as problem-oriented policing emphasizes involvement of the police in
community life. This strategy has police serve as community agents rather than adversaries
with the community. Study group member Professor James F. Short suggests that police
should not maintain their gatekeeper function and solve problems for the
community but should be involved in solving problems with community
support and assistance. In this way, police can help develop and promote a sense of
community (Short, 1990: 225-226). Professor Short makes a critical link from the 1990's
problem-oriented policing to the role of police in the Chicago Area Project during the
1940's. As he informs us, there are many similarities in police functioning then and what
we are suggesting for the future. The vision was-
[T]he police as a resource for the community, aiding local residents and working with
indigenous leaders to solve community problems, with special focus on the problems of
young people. The goal in each of these programs is to promote the achievement of
"functional communities," that is, communities in which family life, work,
religion, education, law enforcement, and other institutional areas reflect and reinforce
common values (Short, 1990: 226).
Although arrests of offenders remains an important tactic, the police repertoire must
be widened to include a variety of civil actions, mobilization of citizens and other
government agencies to change the conditions that generate crime or that will likely
escalate deteriorating conditions. For example, the strategy of "Weed and Seed"
is to eliminate drug-related crime and to restore economic vitality to inner cities
through multi-agency cooperation and the use of community empowerment and resident
involvement (Department of Justice, 1991). An important aspect of this third theme is that
the police should become pro-active, interactive, and preventative in their orientation
rather than rely solely on reacting and control.
A fourth theme focuses on changes in internal working relationships. That
is, police agencies need to examine the potential strengths and weaknesses of
decentralization of authority by seeking ways to guide discretion and police behavior
generally through increasing reliance on values rather than rules and strict methods of
accountability (Alpert and Smith, 1993). These ideas are central to the concepts of
community policing, problem-solving policing and smarter policing. Incorporating these
ideas into strategies of policing, we believe, would truly professionalize police rather
than treat them as blue-collar workers. In addition to making police work more effective,
these four strategies may increase job satisfaction-and most importantly-community
satisfaction (Greene, Alpert and Styles, 1992).
These four themes combine to form the overarching principle of changed police-community
relationships. Currently, police work revolves around serious crimes. The community
participates by becoming the eyes and ears of the police; however, this strategy keeps the
police outside and above the community. Police are summoned by the community through
individual requests for service, and those requests are evaluated primarily in terms of
whether an offense has been committed and a crime has been solved.
Creative, problem-oriented policing strategies place the community in a much different
position than they have been in the recent past. Under this new paradigm, police work is
oriented toward community satisfaction and the increase in human and social capital in the
community. Satisfaction is determined not only by the police response to individual calls,
but also by community members banding together to advise and consult with the police.
Further, community institutions play the most important roles in changing community
conditions that generate crime and in shaping police activities related to crime and other
community conditions. Placing police and the citizens in communication with community
leaders creates a dialogue and interaction. This removes the police from a hierarchical
position and has the effect of increasing the accountability of the police to the
community.
One of the crucial issues that must be faced by all concerned with community policing is
the assumption that there is a community to organize. Some cities and suburbs have
developed rapidly and have not formed what sociologists refer to as communities or
neighborhoods. Similarly, some precincts or reporting areas may not be contiguous with
natural neighborhoods or communities. Finally, some areas that have deteriorated or are in
the process of deteriorating may be difficult to organize. Areas needing organization the
least will be the easiest to assist, while less well-organized communities, particularly
underclass areas of the inner city, will be the hardest to organize (Alpert and Dunham,
1988). However, examples of difficult and complicated organization are available.
One example of this community-building comes from Judge Thomas Petersen in Dade County,
Fla. Judge Petersen was able to create a sense of community in several areas known for
their lack of community spirit or allegiance. Judge Petersen, with assistance from the
housing authority, law enforcement officials, and private industry, established three
community stores that sold essential items in housing projects. In each, the housing
authority found sufficient space and turned the space into grocery stores with supplies
donated by private industry. The shelves were stocked with no up-front costs. Further,
training for the people necessary to run the business was procured from professionals in
the grocery business. Those who were hired to run the store were in need of child care,
and the space and training for that service was provided by the housing authority.
After a short period of time, a group of people were working in the store, others were
working in the child care center, and all were removed from public assistance. More
important, however, was the sense of community created by the stores and child care
centers. The stores became a focal point of the projects, and residents, police, and
others involved in their establishment gained a mutual respect and trust for each other.
Residents who had been scared to talk to other residents began to realize the importance
of community spirit and the benefits of mutual assistance. The workers and residents began
to identify with the operation of the store, and when anyone began to cause trouble or
tried to sell drugs, the police were called immediately, and residents would point out the
offender and work with the police to do justice. After a short period, the stores earned
the reputation as establishments that would not only sell goods but also as the heart of
the housing projects, serving as a rumor control center, a place to get assistance from
others, and a place with respect for the police function.
This new-found respect for police spread very quickly through the projects and neighborhoods. Residents who once despised the police were now working with them to solve crimes and create an atmosphere where street criminals would not be tolerated. In many respects, Judge Petersen had created a community spirit that fit neatly into the community-oriented policing strategy (Petersen, 1993).
Implications for police performance measurement
As society and the police approach a new understanding of how each can contribute to the other, it is critical to develop new measures to determine how well the police perform. Measures of performance rely on the definition of what the police are expected to do and how they are expected to do it. The measures must not only reflect but also help to shape community expectations of the police. For example, consider how neatly the current enshrined measures of police performance fit the dominant current strategy of policing.
Current performance measures as a reflection of professional law enforcement
Recall that the current strategy of policing emphasizes crime control through arrests
and that arrests are produced by patrol, rapid response to calls for service, and
retrospective investigation. Current police performance measures are linked directly to
these tasks. First, the overall objective of police has traditionally been perceived to
reduce crime. It follows that the traditional measure of police performance is the level
of reported crime measured by the Uniform Crime Reports. Another police task is
apprehending offenders. This task is measured by arrests. Other traditional measures
related to the crime rate include the ability to solve crimes (clearance rates-a very
subjective measure) and the ability to get to crime scenes quickly (response times). These
existing measures fit the traditional policing strategy perfectly, and they have become
recognized as the important measures.
What is missing from these measures
Limitations of the traditional policing strategy are also represented by the current
performance measures. It is important that crime is measured in terms of reported crime,
rather than through victimization surveys. Indeed, the police long resisted the
development of criminal victimization surveys, concerned that they would reveal
differential reporting and would be too subjective. This emphasis on reported crime left
invisible many crimes such as domestic assault, child abuse, extortion by armed robbers
and drug gangs, and other crimes in communities that did not trust or have confidence in
the police (Bureau of Justice Statistics, 1992, and Federal Bureau of Investigation,
1992).
It is also important that the measures that could have revealed the fairness and economy
within which the authority of the police was deployed got less attention than the question
of police effectiveness. There was no routine expectation that the police would publish
data on patrol allocations, response times, or crime solution rates across neighborhoods.
Similarly, no serious efforts were made to develop statistical evidence on the incidence
of brutality, excessive use of force, discourtesy, or corruption. In principle, one could
have collected information about these things by soliciting civilian complaints and taking
them as indicators of problems, if not probative of individual officer misconduct (U.S.
Civil Rights Commission, 1981). Again, the argument, albeit flawed, was that unlike
official crime statistics, such information was suspect and too subjective. Thus, in this
area as elsewhere, the commitment to fairness and discipline in the use of authority was
less important than the claim of crime control effectiveness.
Further, there was no real way to capture the quality of the response that the police made to citizen calls other than those involving criminal offenses for which an arrest could be made. In fact, most of the operational indicators implicitly viewed responding to non-crime complaints as something to be avoided and resisted rather than taken seriously. Measures included a comparison between time out of service and time in-service. In-service meant being on patrol, while out-of-service included meal breaks but also included meeting citizens and responding to their calls for service. Similarly, time spent on high-priority calls was compared with time spent on "nuisance calls." The purpose was to reduce time on nuisance calls, despite the fact that it was these calls that could be used to build the relationship with the community that was necessary to make their current tactics effective in dealing with crime (Sparrow, Moore, and Kennedy, 1990).
Finally, there was no real way to account for or measure pro-active operations. The only way to do this was through monitoring specialized squads or units. Units were created to deal with particular problems, often on a temporary basis, without the establishment of a method to capture the nature or extent of the units' activities. Similarly, there was no attempt to determine how much of the organization's resources was being committed to such pro-active operations (Bureau of Justice Statistics, 1992).
Reforming police performance measures
Orienting the agency to the community
Several options exist to reform police performance measures. First, existing measures could be improved to live up to the challenge of professionalism. This would include audited clearance and arrest rates and the development of statistical evidence on the use of force and the incidence of brutality, discourtesy, and corruption, among others. Second, performance measures could be linked more closely to action in the community, including the level of centralization and community-level programs. Under this structure, programs must be established that encourage calls to the police and evaluate calls to the police for service as well as concerns regarding criminal behavior. Measures should also include-
Further, measures of the form and level of self-defense efforts by citizens and measures of trust and confidence in the police should be routinely taken and evaluated. Measures of the quality of service delivery by the police should be taken to improve departmental functioning and reveal the quality of individual officers as reported by the citizens with whom they come in contact (Furstenberg and Wellford, 1973, and U.S. Civil Rights Commission, 1981).
Encouraging pro-active problem solving
One of the biggest problems in accounting for the performance of police departments is
to capture what is accomplished during pro-active and problem-solving activities. One way
to measure this concept is to view each problem-solving initiative as a particular program
to be evaluated for its immediate impact. A second way to measure the impact is to view
each as equivalent to a criminal investigation or special operation. In this way, a file
is created, activities are monitored, and results recorded and evaluated.
The problem, of course, is that the problems come in different sizes. Size can be measured
in terms of-
1 ) total resources committed to the problem
2) amount of time taken to solve
3) the number of specialized resources required
4) the extent to which higher-ranking officers must mobilize and coordinate efforts within
and outside the department to deal with the problem
5) its importance and scale within the community.
One way to deal with these concerns is to develop a tailored program for individual areas.
In other words, do not assume that each community has the same concerns or problems or
that each community should respond similarly to certain problems. One product that would
result from the effort to create, deliver, and measure these community oriented programs
and surveys is a data base on which a department or a division within a large department
could customize a pro-active or interactive problem-solving approach. Pressure to build a
portfolio of problems solved successfully and improved attitudes toward the police could
become as intense as current pressures to maintain low crime rates and quick response
times.
Managing the transition to the new strategy
One of the most difficult problems faced by police managers in the short run is the
awkward period of transition to the new strategy. The new programs will not be up and
operating, and the new measurement systems will not be working and widely accepted. Yet
the police will still be accountable to the public. Thus, they will have to develop
measures that can keep them accountable during the transition.
One method is to identify the particular investments and efforts that are required to
implement the new strategy of policing and report progress on these activities. If new
training is required, they can report on the development of the new curriculum and the
number of participating officers. If the formation of community groups is identified as
important, that progress can be monitored and recorded. If the development of a new call
management system or a new scheduling system is required, that too, can be monitored. The
point is simply to identify and monitor the key organizational investments that are
required. Unfortunately, no data sets exist on which to begin an analysis. The Bureau of
Justice Statistics has compiled the most comprehensive data set (Law Enforcement
Management and Administration Statistics- LEMAS) but its elements do not include many of
the critical measures discussed in this paper (Reeves, 1992).
Toward a new strategy
The urgent need today in measuring police performance is to move away from a sterile
conversation about performance measurement as an abstract technical problem and to
understand it as a device that can be used managerially to shape the future of policing.
This is neither a question of the essential unchanging measures that finally capture the
value of policing nor a discussion of outcomes versus outputs nor a discussion of single
versus multiple measures of performance. Instead, it is a discussion about a strategy of
policing that will work in the future and how to measure its effects. Current measures of
policing are holding police departments in their current mold and are keeping them mired
in the past. These current measures need to be supplemented by innovative policing and new
ways to measure their successes.
Our suggestion has several organizational elements that must be added to the traditional
components already existing in many police departments. The police initiative must stress
the need to learn about the residents and business people in their neighborhoods and to
see them in situations that are not always defined as negative or at best neutral. This
increased role for the police must include two basic approaches. First, a method must be
devised to solicit information from members of the community. This method can incorporate
meetings or citizens' advisory and focus groups with the police and can be enhanced by
community surveys to determine attitudes and suggestions concerning the police and the
police role. Another important dimension of this information gathering is the analysis of
what Skogan has found to be measures of neighborhood decline and disorder (Skogan, 1990).
Second, the police must use this information to reduce isolation between police and the
citizens. The strategy is to assign officers for an extended period, supervised by command
staff and advised by community groups. This move toward stability will increase the
identification of an officer with the residents, geography, politics, and other issues in
a given neighborhood.
These operational elements require proper training, feedback mechanisms, and an institutionalized reward system. Additionally, it is important that these efforts are measured, analyzed, and evaluated by the police officers, command staff, and members of the public.
Neighborhood training
Neighborhood training involves two basic questions the police must answer according to
the needs of each community or neighborhood: what to do and how to do it. In other words,
the priority of police resources, whether fighting crime or providing social services,
changes from neighborhood to neighborhood. Police officers must identify these needs from
their own experiences and expectations, from the perspective of the consumers, and from
that of the police administration. Neighborhood training can effectively inform the
officer as to what he or she can expect from the residents, physical surroundings, or
other influences. This in-service training can introduce officers to community
characteristics while they are working the streets under a supervisor (in a way similar to
a field training officer). What to do can be determined by problem-solving techniques. How
to do it is the all-important style of policing that needs to be developed and supervised
by command staff.
Distinct differences may exist among officers, administrators, and citizens concerning
style. Matching the style of policing to community needs and requirements will improve
both the police and the community. This can be achieved through training based upon
knowledge of community values and beliefs as well as the attitudes and priorities of
police officers. A necessary aspect of this is the continuous dialogue between residents
and the police. Research on attitudes, expectations and evaluation of services of both the
police and the members of the community is critical.
Monitoring
The final component of this strategy includes institutionalized monitoring and a formal
reward system. This requires an ongoing system to monitor both the community and the
police. The needs of the community can be determined by periodic social surveys, which, if
linked to census data and local planning information, can inform officials of the changing
nature of a given neighborhood. While it is relatively easy to identify what constitutes
negative behavior, it is difficult to specify exemplary behavior. The proper use of good
research, including appropriate sampling and a panel design, could provide a clear
snapshot of the needs expressed by a given community. Police officers and administrators
can work together to identify critical questions and a research design that can answer
them. A Blue-Ribbon Committee studying the Miami Police Department concluded that while
crime fighting activities are important, service activities are equally as important in
term of the new paradigm. In the final report, the committee noted:
It is our concern that a minor organizational change can have a major impact on community
relations and on the Interrelationships between citizens and police. We believe that
confidence in the police will be enhanced if the police measure and make more visible the
activities they perform. Moreover, police work is usually rewarded by the gratitude
an officer receives from those whom he or she helps. Status in the department ,
promotions, raises, commendations, etc., rest largely on his or her crime-fighting
activities, the number of arrests, crimes he or she solves, etc. As a result, the
patrol officer may regard service calls as necessary evil (Overtown Blue Ribbon Committee,
1984: 199).
These creative data, together with traditional law enforcement information, will permit
the development and maintenance of neighborhood profiles. Analyzing and monitoring these
profiles can assist the police in improving their training, tactical decisions,
effectiveness and efficiency.
Rewarding the officers
Most police departments provide incentives for their officers. These include traditional promotions, merit increases, and "officer-of-the-month" recognition. Many departments offer several opportunities for their officers to receive or earn rewards. Traditionally, these rewards have been based upon aggressive actions that led to arrest(s), the capture of a dangerous felon, or some other heroic activity. These criteria for rewarding police officers are important and serve to encourage similar actions from others. Yet other types of police behavior deserve recognition but remain lost and hidden behind the visible, aggressive activities of police officers. Activities that should receive more attention include exemplary service to the community and the reduction or diffusion of violence. Those who provide meritorious service may be recognized but often their actions are lost behind the brave shooting incident or heroic rescue. The local community needs to recognize officers who serve their "beat" or neighborhood in an exemplary fashion. A "Best Cop on the Block" recognition would be an important reward, if provided by local residents or merchants. When an officer avoids a shooting or talks a suspect into custody, his or her superiors may not find out; if they do, the officer may be labeled as a "chicken" or one who cannot provide needed back-up to his fellow officers. Nonaggressive behavior that reduces violence needs to be reinforced, rewarded, and established as the model for other officers to copy.
An institutional reward system should be established for officers who avoid or reduce
violent situations and who avoid the use of force, especially deadly force, when avoidance
is justifiable. When command officers, from the chief to the sergeants, support and reward
violence reduction, private business and service groups can be enlisted to provide
symbolic and monetary rewards for such behavior. The institutional support for the
effective policing of a neighborhood can only encourage others to consider a change in
priorities and style. While this is only one aspect of a neighborhood intervention and
community evaluation model, it could serve as a successful step toward meeting the joint
needs of the citizens and the police.
Data on these activities should be collected, assessed and evaluated to help determine
police departments" performance to do justice and promote secure communities.
Summary and conclusion
Police departments around the country have instituted one or more of the foregoing
organizational components into community policing programs, but we are not aware of any
agency that has incorporated them all or that uses many of these nontraditional
performance measures. The components of the suggested program need coordination and
individual assessment as well as analysis as a total effect.
Effective neighborhood policing requires that police administrators acquire adequate
information on the specific neighborhood, including knowledge of the informal control
structure of the neighborhood, attitudes about the police, and policing strategies and
styles. This information can be obtained from citizen surveys, census data, community
advisory groups, and community leaders. After accumulating the information, police
administrators can decide how to deal with any incongruence between the neighborhood
context and police policies, strategies, and styles. Some of these differences can be
reduced by campaigns to educate the citizens and change public opinion and attitudes. In
other cases, discrepancies can be reduced by training programs for officers who are
assigned to the areas. The training can focus on neighborhood specific strategies,
appropriate styles for the specific neighborhood, and placing priorities on tasks
consistent with the neighborhood's expectations. Subsequent to appropriate
neighborhood-based training, police administrators need to create and institutionalize a
system of monitoring and rewarding police officers' behavior. The police officers assigned
to the neighborhood provide the final link integrating the formal control system of the
police with the informal system in the neighborhood. Officers must apply the training
principles appropriately through their use of discretion.
From data collected from the neighborhoods, a good plan for neighborhood intervention and community evaluation can bring modern police work in line with our modern world. Moore and Kelling (1983: 65) have previously summarized these ideas quite well:
Police strategies do not exist in a vacuum. They are shaped by important legal, political, and attitudinal factors, as well as by local resources and capabilities, all factors which now sustain the modern conception of policing. So there may be little leeway for modern police executives. But the modern conception of policing is in serious trouble, and a review of the nature of that trouble against the background of the American history of policing gives a clear direction to police forces that wish to improve their performance as crime fighters and public servants.
The two fundamental features of a new police strategy must be these: that
the role of private citizens in the control of crime and maintenance of public order be
established and encouraged, not derided and thwarted, and that the police become more
active, accessible participants in community affairs. The police will have to do
little to encourage citizens to participate in community policing, for Americans are well
practiced at undertaking private, voluntary efforts; all they need to know is that the
police force welcomes and supports such activity. Being more visible and accessible
is slightly more difficult, but hiring more "community relations" specialists is
surely not the answer. Instead, the police must get out of their cars, and spend
more time in public spaces such as parks and plazas, confronting and assisting citizens
with their private troubles. This is mundane, prosaic work but it probably beats
driving around in cars waiting for a radio call. Citizens would surely feel safer
and, perhaps, might even be safer.
Private citizens working together and through community institutions can have a profound
impact on policing. Those community organizations and police agencies that have developed
reciprocal relationships will enjoy more success than those attempting to work without the
benefit of the others' knowledge and information.
The maintenance and analysis of administrative statistics can provide community members
and police supervisors with performance outcomes that promote justice. Patrol officers can
be in the best position to understand the varied and changing needs of the community, and
with input from research and training, appropriate activities can be devised to do justice
and promote safe communities and develop a new meaning for the phrase "professional
policing."
Table 1
The mission of the police consists of many diverse activities, not objectives in themselves but which are directed toward the protection of life. Goals include doing justice, promoting secure communities, restoring crime victims, and promoting non-criminal options.
Police: Goals, methods, and performance indicators
Goals
Doing Justice. Treating citizens in an appropriate manner based upon their conduct.
Methods/activities
Balancing formal and informal social controls, responding to calls for service, patrolling tactics, issuing traffic tickets, conducting investigations, writing reports, making arrests, and assisting in criminal prosecutions.
Performance indicators
Nature and type of patrolling strategy, number of traffic tickets issued, known crimes
that are cleared by audit or arrest, quality of reports, analysis of who calls the police,
evaluation of policies emphasizing values over rules, time invested and quality of
investigations, number of known crimes cleared by conviction, arrests and arrests cleared
by conviction, cases released because of police misconduct, citizen complaints, lawsuits
filed, and results of dispositions and officer-initiated encounters.
Goals
Promoting secure communities, enabling citizens to enjoy a life without fear of crime or victimization.
Methods/activities
Preventing/deterring criminal behavior and victimization, problem-solving initiatives, training for community differences, assisting citizens by reducing fear of crime and victimization.
Performance indicators
Programs and resources allocated to crime prevention programs, inter-governmental
programs, resources, both time and dollars dedicated to problem-solving, rewards and
monitoring of police, public trust and confidence in police performance, public attitudes
toward police actions and public fear of crime, and home and business security checks.
Goals
Restoring crime victims, by restoring victims' lives and welfare as much as possible.
Methods/activities
Assisting crime victims to understand the criminal justice system, assisting crime
victims with their difficulties created by the victimization, assisting crime victims to
put their lives back together.
Performance indicators
Number of contacts with victims after initial call for assistance, types of assistance
provided to victims, including information, comfort, transportation, and referrals to
other agencies.
Goals
Promoting noncriminal options, by developing strong relationships with individuals in the community.
Methods/activities
Develop and assist with programs that strengthen relationships between police and
members of the community and among community members, increase human and social capital in
the community and linkages with private industry.
Performance indicators
Programs and resources allocated to strengthening relationships between police and the
community and among community members, including traditional community relations programs,
school programs and resources spent to meet with the public in a positive alliance.
Innovative programs to develop a sense of community, organizational measures of
decentralization, community storefront operations and officer contacts with citizens for
positive relations and feedback on performance are aspects of developing strong
relationships with members of the community.
References
Alpert, Geoffrey, and Roger Dunham. Policing Urban America. Prospect Heights, Ill. Waveland Press. 1992.
Alpert, Geoffrey, and Roger Dunham. Policing Multi-Ethnic Neighborhoods. New York: Greenwood Press. 1988.
Alpert, Geoffrey, and William Smith. "Developing Police Policy: Evaluating the Control Principle." American Journal of Police. Forthcoming, 1993.
Bazelon, David. Questioning Authority. New York: Knopf. 1988.
Bennett, Trevor. Community Policing in Britain. Paper presented to the International Conference on Community Policing. Institute of Criminology, University of Heidelberg, Heidelberg, Germany. September 1992.
Bureau of Justice Statistics. Crime and the Nation's Households, 1991. Washington, D.C.: Bureau of Justice Statistics. 1992.
Bureau of Justice Statistics. Sourcebook of Criminal Justice Statistics, 1991. Washington, D.C.: Bureau of Justice Statistics. 1992.
Bureau of Justice Statistics. Report to the Nation on Crime and Justice, 2nd. Ed. Washington, D.C.: Bureau of Justice Statistics. 1988.
Cole, George. The American System of Criminal Justice. Pacific Grove, Calif.:
Brooks/Cole Publishing Co. 1992.
Department of Justice. Operation Weed and Seed: Reclaiming America's Neighborhoods.
Washington, D.C.: U.S. Department of Justice. 1991.
DiIulio, John. Rethinking the Criminal Justice System: Toward a New Paradigm. Washington, D.C.: Bureau of Justice Statistics. December 1992.
Federal Bureau of Investigation. Crime in the United States 1991. Washington, D.C.: United States Department of Justice. 1992.
Furstenberg, Frank, and Charles Wellford. "Calling the Police: The Evaluation of Police Service." Law and Society Review 7:393-406 (1973).
Gallup, George. The Gallup Poll Monthly, Report No. 318. Princeton, N.J.: The Gallup Poll. 1992.
Goldstein, Herman. Problem-Oriented Policing. New York: McGraw-Hill. 1990.
Greene, Jack, Geoffrey Alpert, and Paul Styles. "Values and Culture in Two American Police Departments: Lessons from King Arthur." Journal of Contemporary Criminal Justice 8: 183-207 (1992).
Kelling, George. "Measuring What Matters: A New Way of Thinking About Crime and Public Order." The City Journal Spring 2:21-34 (1992).
Metro-Dade Police Department. Survey of African-American Males 15-30 Years of Age.
Metro-Dade Police Department. Miami: 1992.
Metro-Dade Police Department. "Jammin' with the Man" Project Summary.
Metro-Dade Police Department. Miami: 1993.
Moore, Mark, and George Kelling. "To Serve and to Protect: Learning from Police History," The Public Interest, 70: 49-65. 1983.
Overtown Blue Ribbon Committee. Final Report. City of Miami, 1984.
Petersen, Thomas. Personal communication. February 1993.
Reaves, Brian. Law Enforcement Management and Administrative Statistics, 1990: Data for Individual State and Local Agencies with 100 or More Officers. Washington, D.C.: Bureau of Justice Statistics. 1992.
Short, James F. Delinquency in Society. Englewood Cliffs, NJ: Prentice-Hall. 1990.
Skogan, Wesley. Disorder and Decline: Crime and the Spiral of Decay in American Neighborhoods. New York: Free Press. 1990.
Skogan, Wesley, and George Antunes. "Information, Apprehension and Deterrence: Exploring the Limits of Police Productivity." Journal of Criminal Justice 7 (1979).
Sparrow, M., M. Moore, and D. Kennedy. Beyond 911: A New Era for Policing. New York: Basic Books. 1990.
U.S. Civil Rights Commission. Who's Guarding the Guardians? Washington, D.C.:
USGPO. 1981.
Walker, Samuel. The Police In America. New York: McGraw-Hill. 1 992.
Whitaker, Catherine. Crime Prevention Measures. Washington, D.C.: Bureau of Justice Statistics. 1986.
Wilson, O.W., and Roy McClaren. Police Administration, 4th Edition. New York:
McGraw-Hill. 1977.
About the authors
Geoffrey P. Alpert is Professor of Criminal Justice at the University of South Carolina
and Research Professor, Institute of Public Affairs. He has been a Visiting Fellow at the
Bureau of Justice Statistics and has served as the Director of Research in the Georgia
Department of Corrections. He has extensive experience in criminal justice agencies and
research. For the past 10 years, he has focused his interests on evaluation research. His
research efforts include the evaluation of multi-ethnic policing, the police use of deadly
force, the impact of indicators of unrest on the prevention of civil disturbances, how
police pursuit driving policies affect outcome, the effect of social programs on keeping
high-risk students in school, and the effectiveness of tactical narcotics teams. He has
published more than a dozen books and monographs and has contributed to legal,
sociological, and criminal justice journals.
Mark Moore is the Daniel and Florence Guggenheim Professor of Criminal Justice Policy and
Management and Faculty Chairman of the Program in Criminal Justice, John F. Kennedy School
of Government, Harvard University. For a decade he served as the Founding Chairman of the
Kennedy School's Committee on Executive Programs. His intellectual interests lie in
criminal justice policy, in public management, and in particular, the intersection of the
two fields. He has led national "executive sessions" on the future of juvenile
justice, policy, and prosecution. In the domain of crime policy, he is the author of Buy
and Bust: The Effective Regulation of an Illicit Market in Heroin and the co-author
of Dangerous Offenders: Elusive Targets; From Children to Citizens: The Mandate for
Juvenile Justice; and Beyond 911: A New Era for Policing. In the domain of
public management, his co-authored works include Inspectors-General: Junkyard Dogs or
Man's Best Friend; Public Duties: The Moral Obligations of Public Officials; and Ethics
in Government: The Moral Challenges of Public Leadership.
Measuring Performance When There Is No Bottom Line
by John J. DiIulio, Jr.
The great scholar of American government, Wallace Sayre, promulgated the "law" that "public and private management are fundamentally alike in all unimportant respects." Many leading thinkers-Graham T. Allison, Jr. of Harvard University and James Q. Wilson of UCLA, to cite just two-have echoed this view. 1
As Wilson has noted, to a "much greater extent than private bureaucracies, government agencies (1) cannot lawfully retain and devote to the private benefit of their members the earnings of the organization, (2) cannot allocate the factors of production (land, labor, capital) in accordance with the preferences of administrators, and (3) must serve goals not of the organization's own choosing. Control over revenues, productive factors, and agency goals is all vested to an important degree in entities external to the organization-legislatures, courts, politicians, and interest groups."2
1. See Graham T. Allison, "Public and Private Management: Are They
Fundamentally Alike in All Unimportant Respects?" in Richard J. Stillman, III, ea., Public
Administration: Concepts and Cases, Fifth Edition (Boston: Houghton Mifflin Company,
1992), chapter 10, pp. 282-288, and James Q. Wilson, Bureaucracy: What Government
Agencies Do and Why They Do It (New York: Basic Books, 1989). 2. Wilson, Bureaucracy, p. 115. |
With few exceptions, the human and financial resources of public managers are contingent upon the goodwill of legislators and other policymakers, not the goodwill of customers and other consumers. Relative to private managers, public managers in civil service bureaucracies have little discretion in hiring, firing, and promotion decisions; and, their ability to innovate-to change the way work gets done-is far more constrained.
Unlike most private corporations, most government agencies have no market-test of
output. The managers of a smokestack company are out to turn a profit; the managers of the
Environmental Protection Agency are out to "protect the environment." The
executives of television networks contrive to generate dividends for shareholders; the
heads of the Federal Communications Commission contrive to regulate airwaves in "the
public interest." At most, public managers have proxies for outputs: for example,
increasing weapons stockpiles as a proxy for "enhancing combat readiness" and
"strengthening national defense."
Thus, as Allison observes, there is "little if any agreement on the standards and
measurement of performance to appraise a government manager, while various tests of
performance-financial return, market share, performance measures for executive
compensation-are well established in private business.... Governmental managers rarely
have a clear bottom line, while that of a private business manager is profit, market
performance, and survival."3 Or, in Wilson's words, "whereas business
management focuses on the 'bottom line' (that is, profits), government management focuses
on the 'top line' (that is, constraints)."4
Thus, the essential differences between public and private management are rooted in the
nature of their respective goals. Basically, there are two types of goals: operational and
nonoperational.5 An operational goal is an image of a desired future state of
affairs that can be compared unambiguously to an actual or existing state of affairs. A
nonoperational goal is an image of a desired future state of affairs that cannot be
compared unambiguously to an actual or existing state of affairs.
Some examples are in order. "Keeping America first in space technology" is a nonoperational goal; "Putting an American on Mars by the year 2010" is an operational goal. "Improving the quality of public education in America" is a nonoperational goal; "Increasing the average verbal and math SAT scores of public school students by 20% between the year 1992 and the year 2000" is an operational goal. "Making America's welfare system work" is a nonoperational goal; "Instituting work-based welfare programs that get 50% of all participants off the rolls in their first 7 years of operation" is an operational goal. "Reforming criminals" is a nonoperational goal; "Doubling the rate of inmate participation in prison industry programs" is an operational goal.
3. Allison, "Public and Private Management," pp. 285-286,
summarizing the ideas of John T. Dunlop. 4. Wilson, Bureaucracy, p. 115. 5. I am indebted to James Q. Wilson for this distinction. |
Generally speaking, public managers have nonoperational goals, while private managers
have operational goals. Moreover, public managers often work in the context of multiple
and contradictory nonoperational goals.
Correctional agencies and forestry services are two classic examples. For much of American
history, correctional agencies were authorized to "punish, deter, incapacitate, and
rehabilitate" convicted criminals Most employees of the Federal Bureau of Prisons,
for example, are "correctional officers" responsible for the "care and
custody" of prisoners. By the same token, most employees of the U.S. Forest Service
are "forest rangers" charged with "multiple-use management" (hunting,
fishing, timber cutting, timber sales, fire prevention, flood control). The correctional
officers are "social workers, prison guards, and managers" all rolled into one;
the forest rangers are "executives, woodsmen, and planners" all rolled into one.6
In both cases, the multiple and contradictory nonoperational nature of the agency's goals
renders inapplicable the much-misunderstood concept of efficiency.
Efficiency refers to a relationship between valued inputs and desired outputs: maximizing outputs from a given set of inputs, or minimizing the inputs necessary to achieve a given level of outputs. In many, perhaps most, government agencies, the relationship between valued inputs (people, money) and desired outputs (less crime, better public health) is ambiguous. Where goals are nonoperational, and the technologies necessary to achieve them are either uncertain, or completely unknown, or simply unavailable, the quest for a "bottom line" is a fool's quest. To the extent that goals are vague or inconsistent, the concept of efficiency is irrelevant.7 The concept of efficiency is thus irrelevant to many, if not most, public management tasks.
6. For a discussion of correctional management, see John J. DiIulio, Jr.,
Governing Prisons: A Comparative Study of Correctional Management (New York: Free
Press, 1987); No Escape: The Future of American Corrections (New York: Basic
Books, 1991); and Principled Agents: Leadership, Administration, and Culture in a
Federal Bureaucracy (New York: Oxford University Press, forthcoming). The best single
account of forestry management remains Herbert Kaufman, The Forest Ranger: A Study in
Administrative Behavior (Baltimore and London: The Johns Hopkins University Press,
1960). For a more recent look at the U.S. Forest Service, see Terrence J. Tipple and J.
Douglas Wellman, "Herbert Kaufman's Forest Service Thirty Years Later: From
Simplicity and Homogeneity to Complexity and Diversity," paper presented at the
Annual Meeting of the American Society for Public Administration, Los Angeles, Calif.,
April 10, 1990, 26 pp. 7. I am indebted to Edward C. Banfield of Harvard University for this insight. |
Defining and measuring nonoperational goals: What is to be done?
Because public management and private management are indeed fundamentally alike in all unimportant respects, it makes little sense to lament the fact that "government isn't run like a business," or to assume that government agencies are inherently less "efficient" than private firms.8 As Allison concludes, "the notion that there is any significant body of private management practices and skills that can be transferred directly to public management tasks in a way that produces significant improvements is wrong."9
8. For arguments and evidence that support this view, see Charles T.
Goodesell, The Case for Bureaucracy: A Public Administration Polemic, Second
Edition (Chatham, N.J.: Chatham House Publishers Inc., 1985), and James W. Fesler and
Donald F. Kettl, The Politics of the Administrative Process (Chatham, N.J.:
Chatham House, 1990). 9. Allison, "Public and Private Management," p. 294. |
It is yet possible, however, for public managers to strengthen the relationship between
administration and goals, inputs and outputs, process and performance. The multiple and
contradictory nonoperational character of most public management goals-the lack of a
market-test of output in much of the governmental sector-is neither an immovable barrier
to defining appropriate measures for the goals of public agencies, nor an inexhaustible
excuse for the failure to do so. As Wilson notes, if someone set out to measure "the
output of a private school, hospital, or security service, he or she would have at least
as much trouble as would someone trying to measure the output of a public school,
hospital, or police department. Governments are not the only institutions with ambiguous
products." 10 In recent years, in fact, a number of major public policy
reforms were launched as efforts to define appropriate measures for the goals of
government agencies.11 There is no reason why the field of criminal justice
cannot or should not follow suit.
10. Wilson, Bureaucracy, p. 373. "One example is welfare reform; see Richard P. Nathan, Social Science in Government: Uses and Misuses (New York: Basic Books, 1988). |
A first step in defining appropriate measures for the goals of the criminal justice system is for practitioners to ask four questions:
These may seem like an incredibly obvious set of conceptual questions to ask, but they
are ones that public managers have not asked with sufficient regularity or seriousness of
purpose. This is true even in public management sectors like defense, where there has been
no shortage of well-funded experimentation with different managerial approaches and
performance-appraisal techniques.12 It is even truer for a public management
sector like criminal justice, where the occasions for such experimentation have been
severely limited both by fiscal constraints and frankly, by intellectually hide-bound
management traditions.
A second step in this direction is to translate the agency's nonoperational goals into
measurable process and performance criteria.
In a recent volume, a group of public management specialists analyzed "impossible
jobs in public management."13 An "impossible job" is defined by-
12. For a discussion of the need for operational concepts in defense
planning, see Glenn A. Kent, A Framework for Defense Planning (Santa Monica,
Calif.: The RAND Corporation, August 1989), especially pp. 21-24. 13. Erwin C. Hargrove and John C. Glidewell, eds., Impossible Jobs in Public Management (Lawrence, Kansas: University Press of Kansas, 1990). |
An example from corrections: The Federal Bureau of Prisons
A good example of this strategy in action is the Federal Bureau of Prisons (BOP). Established in 1930 and situated in the United States Department of Justice, the BOP is a career civil service agency. It has long enjoyed a reputation as one of the finest correctional agencies in the country, and one of the best managed agencies of the Federal Government. Contrary to the "Club Fed" stereotypes of the agency, the BOP has not always enjoyed "a better class of criminals," been less crowded, spent more per inmate, or had a lower inmate-to-staff ratio than State systems of comparable size and complexity. Instead, its success in running safe, clean, programmatic, and cost effective prisons has been the hard-won product of management innovations made under five consecutive directors.
Historically, the core of the BOP management approach has been to translate its
multiple, contradictory, and nonoperational goals (punish, deter, incapacitate,
rehabilitate) into measurable process and performance criteria. For example, the agency's
fourth director, Norman A. Carlson (1970-87), was an award-winning Federal executive. He
self-consciously managed the agency around three criteria: safety, humanity, and
opportunity. Essentially, safety meant the incidence of things that either threatened
the physical and emotional well-being of inmates and staff, or placed the public at direct
risk, or both (assault, homicide, escape). Humanity meant the incidence of things that
made for decent living and working conditions behind bars (decent food, clean quarters,
recreation). Opportunity meant the incidence of things that enabled inmates to better
themselves (programs in remedial reading, job training, drug treatment). Thus, in the
recent agency publication Bureau of Prisons Goals for 1992 and Beyond, we read:
"The Federal Bureau of Prisons protects society by confining offenders in the
controlled environments of prisons and community-based facilities that are safe, humane,
and ... provide work and other self-improvement opportunities...."
Many correctional agencies have such mission statements. But few have actually managed
themselves around measurable process and performance criteria as the BOP has done from
1930 to the present. A good contemporary example is the agency's Key
lndicators/Strategic Support System (KI/SSS). Begun in 1983 and carried out under
Carlson's successor, J. Michael Quinlan (1987-92), the BOP has elaborated "safety,
humanity, and opportunity" into KI/SSS, a personal computer-based management
information system that provides managers at all levels with comprehensive, up-to-date,
system-wide information on a wide range of specific performance variables. Invariably,
such innovations begin with managers who-
For example, in 1988, as the KI/SSS was still being developed, participants at the
BOP's annual wardens' conference received a perceptive four-page progress report that read
in part: "Unlike private agencies and companies, the Bureau of Prisons is expected to
respond to a wide variety of overall objectives, often conflicting, often difficult to
obtain.... The Bureau has stated that its mission is to provide humane control of inmates
(and) ... opportunities to those inmates who choose to use them.... (W)e need to ...
translate that broad goal into more specific measures."
Between 1983 and 1992, they did so, and it helped the agency to manage successfully a
doubling in its prisoner population and a concomitant growth in the number of its staff
and facilities.
The measurement net: How tight a weave?
Putting the example of the BOP to one side, but continuing for the moment with our
focus on correctional agencies, the central question is "How do we define a
comprehensive set of appropriate measures for the goals of corrections that can be used on
a day-to-day basis by the field's practitioners?" The answer is far from
obvious, but some progress
has already been made. The Study Group's Professor Charles H. Logan of the University of
Connecticut has defined eight distinct "dimensions of the quality of
confinement" (security, safety, order, care, activity, justice, conditions,
management), each of which can be measured on four or more "subdimensions" via
staff and inmate surveys and institutional records.14 Logan's work dispels one
concern about the possibility of defining such measures and raises a second. On the one
hand, it dispels the worry that any such measurement scheme is bound to be based
exclusively on one or another moral or ideological view of the "ends of criminal
justice." His quality of confinement measurements encompass and satisfy every major
school of thought about "what prisons are for."
14. Charles H. Logan, "Criminal Justice Performance Measures for Prisons," Performance Measures for the Criminal Justice System (Washington, D.C.: Bureau of Justice Statistics, 1993). |
On the other hand, however, Logan mixes process and performance measurements in a way that may or may not represent the final state of the art. For example, his "security" dimension has six subdimensions: security procedures, freedom of inmate movement, community exposure, staffing adequacy, significant incidents, and drug use. Arguably, the first four of these security subdimensions are process measures, while the last two are performance measures. The same sort of indiscriminate mixing of process and performance measures is present in the BOP's aforementioned KI/SSS. Whether it is either intellectually worthwhile or practically useful to distinguish more cleanly between process and performance measures remains an open question.15
15. For a general discussion of this issue that argues in favor of distinguishing process from performance measures, see John J. DiIulio, Jr., "Recovering the Public Management Variable: Lessons from Schools, Prisons, and Armies," Public Administration Review, Volume 49, Number 2, March/April 1989, pp. 127-133. |
Be careful of what you measure, for you may (or may not) get it
But before one even begins to treat with such questions, at least four flags of caution
should be waved over any effort to define appropriate measures for the goals of criminal
justice agencies, or, for that matter, any governmental agencies.
First, it is naive to assume that what government agencies actually do on a day-to-day
basis can be much affected by how, whether, or to what extent their managers define goals.
In the preface to his study of administration in the Federal Bureau of Investigation (FBI)
and the Drug Enforcement Agency (DEA), Wilson observes that the tasks of government
workers "are supposedly chosen, defined, revised, or discarded as a result of efforts
by administrators to achieve organizational goals," but "tasks are defined by
forces far more profound, and accordingly more resistant to change...."16
Second, even if what government agencies actually did was a simple function of how their
goals were defined and measured, that would not necessarily be a good thing in all cases.
There is always a danger that measurement-driven government workers will, so to speak,
"set up the target in order to facilitate shooting," and give rise to perverse
and unintended administrative routines. Criminal justice agencies furnish many examples:
FBI agents of the Hoover era who padded "the stats" on bank robberies; DEA
agents in the 1970's who persisted in street-level "buy-and-bust" operations
rather than more demanding investigative chores that might not succeed.17
Third, it is often the most dedicated and caring government workers who are anywhere from
suspicious to downright dismissive of any attempt to define and apply such measures. For
example, a hard-working veteran correctional counselor at a maximum-security prison
dismissed the need for such measures and was heedless of what he judged to be the
pseudo-intellectual justifications for them: "Experience on the ground is the only
way to understand the effects of a place like this." An equally dedicated coworker of
his stated: "I believe in this way of doing things. I'd do it this way even if it
didn't make any difference."18 Similarly, a committed participant in a
major community policing demonstration project was perturbed by the suggestion that the
study would help to define appropriate measures for the goals of police agencies:
"It's not about measuring anything, it's about doing things differently."19
Fourth, even under the best of conditions, efforts to act on appropriate measurements of
agency goals can be blocked by personnel regulations and administrative behavior within
government. Unfortunately, there are many examples. In a recent study of the Federal
procurement process, Professor Steven Kelman of Harvard University studied managers who
sought to withhold contracts for poor performance and secure the best possible deal for
the Nation's taxpayers; paradoxically, they were defeated by procedural regulations
intended to promote competition in contracting.20 In another recent study,
Gerald J. Garvey of Princeton University had a similar tale to tell about change-oriented
management within Federal regulatory bureaucracies.21
None of these cautions, however, should be sufficient to discourage one from exploring new
and better ways of defining appropriate measures for the goals of criminal justice
agencies.
16. James Q. Wilson, The Investigators: Managing FBI and Narcotics
Agents (New York: Basic Books, 1978), p. 8. 17. Wilson, The Investigators, especially chapter 5. 18. Quoted in DiIulio, No Escape, p. 144. 19. Interviewed for John J. DiIulio, Jr., "Crime," in Henry J. Aaron and Charles L. Schultze, eds., Setting Domestic Priorities: What Can Government Do? (Washington, D.C.: The Brookings Institution, 1992), Chapter 4. 20. Steven Kelman, Procurement and Public Management: The Fear of Discretion and the Quality of Government Performance (Washington, D.C.: American Enterprise Institute Press, 1990). 21. Gerald J. Garvey, The Challenge of Organizational Change: Living and Dying in a Federal Bureaucracy (San Francisco, Calif.: Jossey-Bass, 1992). |
The Problem of Defining Agency Success
by James Q. Wilson
For the most part, I agree with what DiIulio says in the first essay in this compendium, and I am generally sympathetic to the Study Group's ideas on performance measures for prisons and police departments. But let me reformulate the question this way: How can government agencies with nonoperational goals best develop performance measures?
The best way to approach this issue is to ask how private organizations with
nonoperational goals (schools, colleges, hospitals) cope with the problem of defining
success. In principle, they need not define it, for in a competitive market, success will
be defined for them by survival or failure. The heads of most such organizations are
unwilling to wait for this Darwinian verdict; instead, they try to anticipate it by
looking at the following measures:
Two things need to be said about this list. First, the proxy measures at the bottom of the list are somewhat crude and quite easily misinterpreted. A very good inner-city high school may send few students to Princeton but may have many graduates who get jobs at Lockheed or the Bank of America. The number who do, however, may be unknown because no routine reporting system captures it and the employment decision may come a year or two after graduation. In a hospital, the percentage of admitted patients who get better, given a certain disease, may say more about the personal habits and self-care of the patients than about what the hospital does. As the government increasingly subjects private agencies to constraints experienced by public agencies, private organizations are forced to place greater weight on crude proxy measurements like the ones mentioned above because the government places great weight on them.
Second, the executives of private organizations have more opportunity to engage in personal exploration of their organizational environment-so-called "soaking and poking" (soaking up information and poking into corners)- than do their public counterparts. This greater opportunity is because of three factors:
The compensation and tenure of private executives are more variable than those of public executives, and so the former have a greater incentive to find ways of anticipating (and, if necessary, correcting) adverse market trends. Good executives can be found among public agencies, but they are rarer. For example, the principals of more effective public schools are almost invariably ones who engage in a great deal of soaking and poking and who devote a lot of effort to student and teacher motivation. The literature on this subject makes it clear that this is not routine behavior for principals and often requires for its existence strong, even eccentric, personalities.
The case of big city police agencies
Most of the efforts to improve performance measures for policing have concentrated on
finding either real measures of overall effectiveness or plausible proxy measures. Not
much has come of these efforts for reasons that should be obvious. There are no
"real" measures of overall success; what is measurable about the level of public
order, safety, and amenity in a given large city can only partially, if at all, be
affected by police behavior. (For example, if the murder or robbery rates go up, one
cannot assume that this is the fault of the police; if they go down, one should not
necessarily allow the police to take credit for it.) Proxy measures almost always turn out
to be process measures- response time, arrest rates, or clearance rates-that may or may
not have any relationship to crime rates or levels of public order.
In my view, the search for better measures of police performance is doomed to failure so
long as it focuses on city-wide or even precinct-wide statistics. Most police chiefs will
agree on this point, I think. No matter how we improve the Uniform Crime Reports (UCR) or
the National Crime Victimization Survey (NCVS), they will not tell us very much (and
certainly not very much in a timely fashion) about what difference the police make in the
lives of the citizens.
Nor can most of the other devices available to private firms be applied to policing. There
are no client payments or donor contributions; police appropriations tend to reflect
fiscal constraints and past funding levels modified, from time to time, by generalized
concerns over crime increases and police misconduct. Professional evaluations are not very
reliable; policing is not a profession with clear standards enforced by knowledgeable
peers, except on the rare occasions when an outside group is brought in to evaluate a
department in trouble. But small-scale soaking and poking may be applicable.
These considerations lead me to suggest that the better approach to defining police goals
and performance measures involves thinking small and from the bottom up.
Let me clarify this suggestion by asking what citizens want that is related to police
behavior. They want several things, but in the interests of brevity I will focus on just
one.1 People want to live in safe, orderly neighborhoods. Wesley Skogan has
shown in his recent book that there is a high degree of consensus across demographic
categories as to what a safe, orderly neighborhood is.2 It includes streets
free of drug dealers, rowdy juveniles, threatening derelicts, soliciting prostitutes, and
predatory criminals; buildings without graffiti or other signs of decay; no drive-by
shootings, and so forth.
The police ought to make the production of safer, more orderly neighborhoods (not lower
crime rates or more drug arrests or more traffic tickets) one of their goals. They ought
to design ways of assessing the conditions of neighborhoods before and after various
police interventions. They ought to use that assessment to modify their deployments and
tactics.
1. I omit fair play, civil behavior, prompt response, and many other
important factors. Taking all these into account would complicate my argument but not
change it. 2. Wesley G. Skogan, Disorder and Decline: Crime and the Spiral of Decay in Amercian Neighborhoods (Berkeley and Los Angeles: University of California Press, 1990). |
Adopting this view implies certain actions, none easy but none impossible:
--What is the level of disorder and danger?
--How amenable is that level to improvement by a given, feasible level
of police and public action? (Some neighborhoods
are almost
self-policing and require only police response to calls for service. Some are almost
beyond repair, given
feasible levels of
intervention, because they are highly transient or have other problems contributing to
decay. Some are
amenable to
improvement, provided existing resources are carefully used.)
--A landlord who allows his building to be used by drug dealers
--A deli or spa that allows teenagers to hang out in front of it in
ways others find threatening
--A bar that is the scene of frequent brawls
--A public housing project with poor maintenance and security
--A bus or subway stop that is dominated by aggressive panhandlers or
homeless people sleeping on public benches
--A residential burglary gang that is working the neighborhood
--A park that is the scene of muggings and rapes.
--Tracking calls for service from a specific address or its
immediate neighbors
--Hiring a resident to make regular observations of life on the street
by, for example, at stated intervals counting the number
of panhandlers, suspected drug dealers,
sleeping vagrants, soliciting prostitutes, and so on
--Conducting a telephone survey of residents (using random-digit
dialing to minimize sample selection bias) to assess their
perceptions before and after the
intervention.
These micro-measures are likely to be among the few valid measures of police
performance. They may well lead to conclusions quite at variance with city-wide, aggregate
data. For example, the "crime rate" might be getting worse at the same time that
the conditions of life in neighborhoods has measurably improved, or vice versa.4
For a thoughtful and more detailed proposal along these lines, the reader should consult a
recent essay by George Kelling, especially his account of Operation Crossroads in New York
City's Times Square.5
Just as important as the measures themselves will be the requirement that the police
agency define its operational goals as improving the conditions of life in specific
neighborhoods. The attentive reader will have noticed that this way of defining police
goals is similar to what many advocates of community-oriented policing (COP) propose. This
is not an accident. But my argument is not that COP has such mystical, ideological, or
historical importance that it ought to govern police decisions as a matter of principle.
My argument is that if the police are serious about defining goals and measuring progress
toward them, they will inevitably have to do so by identifying problems relevant to
citizen concerns at the neighborhood level, specifying possible solutions, and measuring
the effect of those strategies. This is what is meant by problem-oriented policing (POP)
which, in my view, is the heart of COP and what makes POP (and COP properly defined)
different from police-community relations.
Note the parallels between POP and the More Effective Schools (MES) literature. Both focus on small spaces (neighborhoods, individual schools); both require the identification of specific problems (unruly teenagers, gang fights, and so forth); both can be evaluated only by specifying significant changes in the behaviors of particular people (fewer fights, quieter classrooms); both try to improve the preconditions of better lives (safer neighborhoods, more orderly classrooms) without directly attacking ultimate outcomes (crime rates, SAT scores) which have multiple causes; both require that the executive (police captain, school principal) spend a lot of time observing and talking to the affected people. And, a final parallel: only a minority of police chiefs (like the minority of school officials) are likely to want to do these things.
3. I do not here define "neighborhood." This is an important
issue that cannot be resolved conceptually. In rough terms, a neighborhood is what people
think it is. The size and shape will vary from place to place and be affected by natural
boundaries (for instance, parks, street-car tracks), institutions (grammar schools or
shopping centers or churches), ethnic composition, and traffic patterns. I wish to stress
that the area should be relatively small and its boundaries easily understood by most
people living there. 4. Some will object that neighborhood measures ignore spillover effects. Up to a point, that is true. But such evidence as we have on burglary and robbery, drug-dealing, and other offenses suggests that the displacement of crime from one area to the next is usually much less than 100%. A lot will depend on the mobility of offenders, and that will be constrained by ethnic identity, profit opportunities, the desire for local "cover," the threats of rival gangs elsewhere, and the depressing effect of laziness and/or opportunism. 5. George Kelling, "Measuring What Matters," The City Journal (Spring 1992), 21-33. |
About the author
Since 1985 James Q. Wilson has been the James Collins Professor of Management at the
University of California at Los Angeles (UCLA). Before that he was for 26 years the
Shattuck Professor of Government at Harvard University. He is the author or coauthor of 13
books, including The Moral Sense, Thinking About Crime, Varieties of Police Behavior,
Political Organizations, Crime and Human Nature (with Richard J. Herrnstein),
Bureaucracy, and Essays on Character. He has in addition edited or contributed to books on
urban problems, government, regulation of business, and the prevention of delinquency
among children, including Crime and Public Policy, From Children to Citizens:
Families, Schools, and Delinquency Prevention (with Glenn Loury), Understanding
and Controlling Crime (with David Farrington and Lloyd Ohlin), and Drugs and
Crime (with Michael Tonry).
He has served on a number of national commissions concerned with public policy. He was
chairman of the White House Task Force on Crime in 1966, chairman of the National Advisory
Commission on Drug Abuse Prevention (1972-73), a member of the Attorney General's Task
Force on Violent Crime in 1981, a member of the President's Foreign Intelligence Advisory
Board (198590), and a director of The Police Foundation (1970-93).
He is currently chairman of the board of academic advisers of the American Enterprise
Institute.
He has been elected a member of the American Academy of Arts and Sciences and a fellow of the American Philosophical Society. In 1990 he received the James Madison award for distinguished scholarship from the American Political Science Association and in 1991-92 served as that organization's president. Educated at the University of Redlands (A.B., 1952) and the University of Chicago (Ph.D., 1959), he has received honorary degrees from five universities. He grew up in Long Beach, Calif., and attended public schools there. From 1953 to 1955, he served in the U.S. Navy. He is married to the former Roberta Evans; they have two children.