U.S. Department of Justice Office of Community Oriented Policing Services LaW EnforCEMEnT TECH GUIDE for Creating Performance Measures That Work A Guide for Executives and Managers Chart: CHRI Total Index Crime Arrests YTD 97,000 Total Drug Crime Arrests YTD 94,700 Percent Change Arrests YTD 3.6% Local Agency Inquiries 267 Chart: CRASH Accidents Year to Date 859 Change Year Over Year .90 Fatalities Year to Date 948 Change Year Over Year .113 Chart: Key Performance Indicators (KPIs) Missing Persons 12 Open Major Cases 78 IA Open Cases 21 Officers Assigned to JTTF 43 Arrest Rate 80.23% DA Filings 77.22% Conviction Rate 65.21% Copyright © 2006 SEARCH Group, Incorporated. The U.S. Department of Justice reserves a royalty-free, nonexclusive, and irrevocable license to reproduce, publish, or otherwise use, and to authorize others to use, this book for Federal Government purposes. This document may be freely distributed and used for noncommercial and educational purposes. No part of this book may be reproduced in any form, by any means (including electronic, photocopying, recording, or otherwise) for commercial purposes without the prior permission of the U.S. Department of Justice or the author(s). U.S. Department of Justice Office of Community Oriented Policing Services LaW EnforCEMEnT TECH GUIDE for Creating Performance Measures That Work A Guide for Executives and Managers By David J. Roberts This publication was supported by cooperative agreement #2003CKWXK054 awarded by the U.S. Department of Justice Office of Community Oriented Policing Services to SEARCH Group,Incorporated,7311 Greenhaven Drive,Suite 145,Sacramento,CA 95831. The opinions or recommendations contained herein are those of the author(s) and do not necessarily represent the official position or policies of the U.S. Department of Justice. References to specific organizations, products, or services should not be considered an endorsement of the product by the author(s) or the U.S. Department of Justice. Rather, the references are illustrations to supplement discussion of the issues. U.S. Department of Justice Office of Community Oriented Policing Services Office of the Director 1100 Vermont Avenue, NW Washington, D.C. 20005 Dear Colleague: With increasing frequency, law enforcement agencies are being called on to systematically examine the impact of their policing practices. Many are using performance measures to do so. Performance measures, when conceived as part of a broad management perspective, can provide officers, supervisors, and executives with an increased level of understanding that can result in more effective and efficient services. Developing valid and reliable performance measures are necessary for assessing and reporting on the impact of policing efforts, whether those efforts include adopting a new technology or adopting new community policing initiatives. The Law Enforcement Tech Guide for Creating Performance Measures That Work is intended to help readers develop their own insightful measures that can be relied on to make improvements to individual programs and initiatives, and to improve the effectiveness of a department’s overall operations. This publication emphasizes that performance evaluation is more than just an academic exercise conducted at a single point in time. It is a complete strategic approach that can become an intrinsic component of police management and, if practiced on an ongoing basis, has the potential to have numerous benefits. This Guide is one of the many resources that the Office of Community Oriented Policing Services (COPS) offers to law enforcement. In fact, it is one in a series of Law Enforcement Tech Guides that can be used as a stand-alone reference or used to complement the Law Enforcement Tech Guide: How to Plan, Purchase and Manage Technology. These tech guides and many of our other knowledge-based resources can be downloaded from www.cops.usdoj.gov, or they can be ordered free of charge by contacting the COPS Office Response Center at 800.421.6770 or via e-mail at askCOPSRC@usdoj.gov. I hope that you find the Law Enforcement Tech Guide for Creating Performance Measures That Work to be both informative and helpful, and I encourage you to share it with other law enforcement practitioners. Carl R. Peed Director Contents Acknowledgments .................................................................... x About the Author .................................................................... xi Section (About the Guide) About the Guide ....................................................................... 3 Assumptions About You and Your Agency ............................... 4 How this Guide Is Organized.................................................... 5 Definition of Icons ................................................................... 6 How to Use this Guide .............................................................. 7 Section (Eight Facts about Performance Measurement of Technology Projects) Eight Facts About Performance Measurement of Technology Projects ........................................................... 11 Part I: Performance Measurement Chapter 1 What Is Performance Measurement and Why Is It Important?............................................... 17 Why Measure?........................................................................ 18 Know Your Purpose ................................................................ 20 You Have To!........................................................................... 21 Why Performance Measures Fail ............................................ 22 Why People Are Afraid of Performance Measurement .......... 23 Connect Your Project to the Broader Mission......................... 23 Chapter 2 Establish an Integrated Performance Management Framework ............................................... 27 It’s Not Just About Whether Crime Went Up or Down........... 27 Balanced Scorecard Provides Template .................................. 28 The Seven Dimensions of a Balanced Scorecard .................... 28 Part II: Performance Management The Six Steps to Establishing a Performance-Based Management Program ......................................................... 31 Part I Checklist............................................................... 32 Chapter 3 Define Mission and Strategic Performance Objectives .. 37 Step One Checklist ......................................................... 45 Chapter 4 Establish a Performance Management Framework ....... 49 The ABCs of Performance Management ................................ 50 Major Elements in Creating a Performance Management Framework ..................................................... 50 Step Two Checklist ......................................................... 61 Chapter 5 Establish Accountability for Performance..................... 65 Who’s Accountable? ............................................................... 67 Step Three Checklist ...................................................... 70 Chapter 6 Develop a Data Collection Plan...................................... 73 Identify Information Requirements........................................ 75 Identify Information Sources.................................................. 76 Pitfalls to Avoid ...................................................................... 76 Define Data Collection Process............................................... 78 Define Data Collection and Reporting Frequency .................. 80 Identify Data Collection Costs ................................................ 80 Address Data Protection......................................................... 82 Ensure Data Quality ............................................................... 82 Complete a Trial Run.............................................................. 82 Step Four Checklist ........................................................ 83 Chapter 7 Analyze, Review, and Report Performance Data ........... 87 Data Analysis Strategies ......................................................... 88 Create an Executive Dashboard .............................................. 95 Step Five Checklist ......................................................... 97 Chapter 8 Use Performance Information to Drive Improvement.......................................................101 Shape Organizational Culture .............................................. 102 Make Information Broadly Available.................................... 103 Reengineer Business Processes ............................................. 103 Step Six Checklist ......................................................... 105 Part III: Performance Management in Everyday Policing Chapter 9 Build Performance Management into Everyday Policing..........................................................111 Executive Support and Organizational Commitment Required .......................................................111 Comprehensive System of Accountability and Responsibility Required ..................................................... 112 Flatten the Organization....................................................... 112 Identify Problems Early .........................................................113 Ensure Progress and Keep Projects on Target ........................113 Demonstrate Value ................................................................114 Part III Checklist .......................................................... 115 Conclusion ....................................................................116 Appendixes A. Bibliography .....................................................................119 B. Glossary........................................................................... 125 C. Sample Assessment Measures ......................................... 133 D. Examples of Performance Management in Justice Agencies ................................................................141 E. Endnotes.......................................................................... 157 Acknowledgments This publication was prepared by SEARCH, The National Consortium for Justice Information and Statistics, Francis X. (Paco) Aumand III, chair, and Ronald P. Hawley, executive director. The project director was Kelly J. Harris, deputy executive director of programs. David J. Roberts, Global Justice Consulting, wrote this publication. Twyla R. Putt, manager, corporate communications, and Linda Townsdin, senior writer/editor, edited this publication, and Jane L. Bassett, publishing specialist, provided layout and design. Chris Roebuck, webmaster, provided web site coordination. The federal project manager was Debra Cohen, Ph.D., of the U.S. Department of Justice (DOJ) Office of Community Oriented Policing Services (COPS). Suggested Citation Roberts, David J., Law Enforcement Tech Guide for Creating Performance Measures That Work: A Guide for Executives and Managers, Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2006. About Us SEARCH, The National Consortium for Justice Information and Statistics, is dedicated to improving the quality of justice and public safety through the use, management, and exchange of information; application of new technologies; and responsible law and policy, while safeguarding security and privacy. We assist local, tribal, county, regional, and state agencies and organizations—including: law enforcement and public safety; first responders; prosecution; defense; adjudication; detention; corrections and probation; and other disciplines, such as transportation, drivers’ licensing, vehicle registration, public health, and social services—through a broad array of activities, resources, and products. Our focus is on criminal history systems, integrated justice information systems, information technology (planning, purchasing, managing), and cybercrime investigation. Our services include in-house and onsite technical assistance and training, resource development (web sites, publications, white papers, conferences, workshops), public policy assistance, and model development (model legislation, standards and procedures, best practices) in these focus areas. SEARCH online resources provide information on law enforcement information technology (IT), integrated justice, justice software solutions, and IT acquisition at www.search.org. About the Author David J. Roberts is principal of Global Justice Consulting, an international consultancy focusing on justice information technology planning, integrated/joined-up justice initiatives, and Justice Information Exchange Model (JIEM) analysis. He previously served as director of the Justice & Public Safety Practice, Global Public Sector for Unisys Corporation, and for 17 years as deputy executive director of SEARCH, The National Consortium for Justice Information and Statistics. He has provided technical assistance in a host of local, state, and national jurisdictions in planning and implementing integrated/joined-up justice information sharing, directed Unisys in development of its integrated justice information sharing (IJIS) solution, and directed a series of national conferences, including the 2002, 1999, and 1996 Bureau of Justice Assistance/SEARCH Symposia on Integrated Justice Information Systems. Mr. Roberts has served as director of a variety of U.S. federally-funded justice IT projects, including the SEARCH project to identify and model information exchange (JIEM), and a joint Bureau of Justice Statistics/ FBI project on National Incident-Based Reporting System (NIBRS) implementation among law enforcement agencies. He is a frequent speaker on justice information technology and integrated/joined-up justice both in the United States and abroad, and a published author. Mr. Roberts holds graduate degrees in criminal justice from the State University of New York at Albany and Oklahoma City University. Performance Measures Tech Guide Review Committee SEARCH extends its deepest thanks and appreciation to the following members of the Performance Measures Tech Guide Review Committee, who participated in an advisory capacity during the preparation of this Guide. Colonel Kenneth Bouche Deputy Director, Information and Technology Command, Illinois State Police Major Piper Charles Charlotte-Mecklenburg (North Carolina) Police Department Sara Phillips Assistant Director, Mayor’s Office of Homeland Security, Boston Captain Jon Shane Newark City (New Jersey) Police Department We would also like to thank members of the Bureau of Justice Assistance and the U.S. Department of Justice’s Global Justice Information Sharing Initiative (Global) for lending their expertise in a review of this document. About the Guide A Library of Tech Guide Resources This Tech Guide on creating performance measures is intended to serve as a companion guide to Law Enforcement Tech Guide: How to plan, purchase and manage technology (successfully!). The original Tech Guide was published in 2002 by the U.S. Department of Justice Office of Community Oriented Policing Services (COPS) and was developed as a step-by-step guide to help law enforcement agencies as they implement new technologies. This Performance Measures Tech Guide is intended to complement and be used along with the original Law Enforcement Tech Guide. As such, this Guide makes frequent references to content in the original Tech Guide. It may help to keep the original Tech Guide close at hand so you can refer to particular pages and sections as needed. This Performance Measures Tech Guide is one of a series of four topic-specific Tech Guides funded by the COPS Office. The four companion Tech Guides that will form a comprehensive library of technology resources, along with the original Law Enforcement Tech Guide, are: -Law Enforcement Tech Guide for Small and Rural Police Agencies: A Guide for Executives, Managers, and Technologists -Law Enforcement Tech Guide for Creating Performance Measures That Work: A Guide for Executives and Managers -Law Enforcement Tech Guide for Communications Interoperability: A Guide for Interagency Communications Projects -Law Enforcement Tech Guide for Information Technology Security: How to Assess Risk and Establish Effective Policies See page 7 for details on how to download or order your own copy of the original Law Enforcement Tech Guide. About the Guide From the earliest days of law enforcement, there was an enduring belief that technology could address nearly every problem encountered. In contemporary times, simply computerizing something was sometimes perceived as modernizing it and thereby improving it. It was only after very specific experiences with automation and computerization, and with the practical experiences of projects that failed by many different measures (e.g., took much longer than expected, cost much more than projected, had a negative impact [i.e., changed] on business practices, or were terminated for failure to reach completion or achieve fundamental business objectives) that we began to recognize that computerization, in and of itself, was not a panacea. It was clear that automation would not succeed without careful project management. Moreover, even if the project succeeded, the consequences might not generate the program or agency impact that had been expected. For these and many other reasons, performance measurement, effective program management, continuous assessment of interim project milestones, and implementation of midcourse corrections where necessary, are crucial steps in effective planning and implementation of technology (indeed, of all) projects. It also underscores the importance of clearly defining measurable objectives up front and ensuring that projects we are planning and implementing are directly related to broad program and agency objectives, goals, and mission. In many cases, technology has proven successful. Radios and cell phones have made instant communication possible. Automated fingerprint identification systems (AFIS), DNA profiling, and other forensic techniques have helped in establishing positive identification and greatly assisted law enforcement in solving crimes that historically would not have been cleared. Mobile computers have armed officers on the street with the ability to instantly record and access a variety of information that traditionally would have taken days, weeks, or even years to discover. About the Guide As in other business and government endeavors, however, technology has not always achieved the results intended or promised. Decision makers and funding bodies have increasingly asked agencies to demonstrate the tangible business benefits of technology expenditures. Too often agencies cannot specifically articulate their claim of success, or even how such results were measured. In light of concerns over effective government and efficient expenditure of public funds, government funding bodies, political leaders, agency decision makers, and the taxpaying public are demanding greater accountability for information and technology systems implementation. This Guide responds to the need to define success in technology programs and beyond. It is part of the broader Law Enforcement Tech Guide series created by the U.S. Department of Justice Office of Community Oriented Policing Services (COPS)1 and SEARCH, The National Consortium for Justice Information and Statistics.2 It is designed to assist agencies in assessing and managing performance, particularly as it relates to measuring the impact of federally funded programs. Performance management, of course, is much more than simply assessing the impact of specific federally funded grant projects. As a consequence, the approach taken here is to define an Integrated Performance Management Framework that can be broadly applied in law enforcement at the enterprise level. The principles of this framework, however, can easily be scaled to address performance measurement issues at individual program and project levels, thereby providing guidance to managers in assessing and managing the performance of specific programs and initiatives, as well as relating these programs to broader agency objectives and mission. This Tech Guide is designed to introduce the role of performance measurement in contemporary law enforcement management, and to teach core dynamics of performance management, monitoring, and reporting. The structure of this Tech Guide follows the general flow presented in the original Law Enforcement Tech Guide,3 and is meant to be complementary to other Tech Guides in the series, addressing Communications Interoperability, Small and Rural Police Agencies, etc. Whenever possible, we have included real-life examples from police departments that have successfully implemented performance management in their jurisdictions. FYI: We tell you how to get your own copy of the original Law Enforcement Tech Guide on page 7. Assumptions About You and Your Agency This Performance Measures Tech Guide is geared to chiefs, sheriffs, commanders, managers, and operational staff. It focuses on addressing the core business objectives of law enforcement, and relates performance measurement, assessment, and monitoring from the perspective of building effective, efficient organizations that can consistently evaluate and achieve success. This Guide provides examples of law enforcement agencies around the nation that have demonstrated success in measuring and managing their performance on specific projects and as part of broader Page 5 management mandates. Highlights of these real examples are provided throughout the Guide. This Tech Guide presumes that law enforcement agencies have developed, or are actively developing, effective enterprisewide strategic business plans. As noted on page 19 of this Tech Guide, “It is only by clearly articulating the objectives of agencies, understanding the current environment within which they operate, establishing baseline measures on critical factors related to the overall success in meeting agency objectives, and constantly measuring the impact of agency actions taken to achieve defined objectives that we can be effective.” The strategic planning process is fundamental to establishing objectives and a management paradigm that will enable effective performance management. Readers are encouraged to familiarize themselves with the strategic planning principles outlined in the original Law Enforcement Tech Guide, and to build the Integrated Performance Management Framework presented here into their broader strategic planning process. How this Guide Is Organized This Guide is organized into three parts that reflect the key elements of the performance measurement process: Part I Performance Measurement Part II Performance Management Part III Performance Management in Everyday Policing If you already have some background on the subject of performance measures, you could easily skip to Part II, which gets right to the heart of the subject by offering six steps that guide you through the process. We believe it is important, however, to understand the context before moving on to the steps, so we recommend that you read all parts and chapters. They provide key guidance and real-life examples from police departments that discuss what works and what doesn’t. This Guide also includes tips, checklists, definitions, and other helpful how-to information. Each chapter, for example, begins with a section that answers the four “Ws” about the topic: What is it, Why do it, Who is involved, and When to do it. We have included appendixes with references to resources, definitions, assessment measures, and endnotes that will enable you to review agency web sites and publications for more information on specific issues and topics. Page 6 Definition of Icons Throughout this Guide, icons are used to draw your attention to important concepts, ideas, reference material, and, in some cases, warnings. Below are the icons and what they represent: Original Tech Guide Reference Original Tech Guide Icon The parent Tech Guide contains many useful tools, charts, and instructions for conducting various tasks. When you see this icon, you will be directed to a specific page, or range of pages, in the original Law Enforcement Tech Guide. Stop Sign Stop Sign Icon When you see a stop sign icon, pay particular attention, as it will indicate where others have encountered trouble in their projects. The stop sign indicates pitfalls to avoid. Grant Requirements Grant Requirements Icon Be accountable to your grants. This icon will alert you when grant requirements may come into play. Tips Tips Icon If we’ve heard or know of shortcuts or have useful ideas on how to tackle a particular issue, we’ll use this icon to let you know. Web Resource Web Resource Icon Whenever you see this icon, we have presented information that may be accessible via the Internet. International International Icon This icon symbolizes international performance management efforts, and alerts you to another valuable perspective on performance management initiatives. Performance Management Team Performance Management Team Icon The Performance Management Team is responsible for leading, implementing, and monitoring performance measurement. You will find specific words of advice and recommendations wherever this icon appears. Page 7 Executive Sponsor Executive Sponsor Icon The executive sponsor is the project spokesperson, decision maker, and leader. You will find recommendations and advice just for you when you see this icon in the margins. How to Use this Guide This Performance Measures Tech Guide is intended to provide strategies, best practices, recommendations, and ideas for establishing an Integrated Performance Management Framework in your agency or jurisdiction. This Guide should not be construed as legal advice for any specific factual situation. It is meant to serve as a guideline for situations generally encountered in planning environments. It does not replace or supersede any policies, procedures, rules, and ordinances applicable to your jurisdiction. It is not a substitute for effective legal counsel and should not be interpreted as a legal service. Sources of the Original “Law Enforcement Tech Guide” The U.S. Department of Justice Office of Community Oriented Policing Services (COPS) published the original Law Enforcement Tech Guide in 2002. It is available electronically from the COPS Office web site at http://www.cops.usdoj.gov/default. asp?Item=512. There it is broken down into its separate parts as Portable Document Format (PDF) files so you can download or read one at a time. If you’re anxious to download the whole document at once—all 14 megabytes—the complete version can be found at SEARCH’s web site: http://www.search.org/files/pdf/TECHGUIDE.pdf. Hard copy versions are distributed by the COPS Office. To request one, call the COPS Office Response Center at 800.421.6770 or e-mail askCOPSRC@usdoj.gov. Eight Facts About Performance Measurement of Technology Projects Eight Facts About Performance Measurement of Technology Projects 1. Performance measurement improves the management and delivery of products and services. 2. Performance measurement improves communications internally among employees, as well as externally between the organization and its customers and stakeholders. 3. Performance measurement helps justify programs and their costs. 4. Performance measurement demonstrates accountability and stewardship of taxpayer resources. 5. Performance measurement is a federal grant-funding program requirement. 6. Performance indicators are useful for diagnosing problems. 7. Performance indicators can be used to assess how well projects and activities are working in practice. 8. Valid and reliable performance indicators can be used to construct better understanding of the operation of the legal system, the relationship between the legal system and larger economic or social development goals, and the impacts of various kinds of intervention and reform. Read on to learn HOW to implement these eight facts. Part I: Performance Measurement “Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.” —H. James Harrington ChapTER 1 WhaT IS pERFORMaNCE MEaSUREMENT aND WhY IS IT IMpORTaNT? Page 17 Chapter: 1 What Is Performance Measurement and Why Is It Important? Serious crime in New York City declined 70.6 percent in the 13-year period from 1993 to 2006. Murder fell by 69.6 percent, robbery by 72.8 percent, rape by 54.0 percent, and felonious assault by 58.9 percent. Crime fell 4.78 percent in 2006 (through December 10), when compared to 2005 figures.4 Violent crime in Los Angeles declined 2 percent through December 16, 2006, when compared to the same period in 2005, and 29 percent when compared to the same period in 2004. Total Part I crime dropped 8 percent and 20 percent, respectively, over the same time periods.5 3-1-1 nonemergency call systems were designed in part to reduce overburdened 9-1-1 Emergency Call Centers, expedite response to legitimate emergencies, and provide greater opportunities for community policing. Research in several jurisdictions has shown mixed results and has demonstrated the importance of ongoing performance monitoring and assessment to track the impact of the system on law enforcement and to guide changes in operations and response.6 Police chiefs and sheriffs, like chief executives of any organization, are measured on results. For law enforcement executives, that means public assessment on a variety of factors: • Is crime going up or down? • Do citizens feel safe in their communities? Do they feel safer today than they did a year ago or 10 years ago? • Does the agency respond quickly to emergency situations? • Does the agency rank favorably when compared with other jurisdictions in the amount of crime reported (both in gross number of crimes and per capita crime rates), the proportion of cases solved, and the number of arrests made? Page 18 Part I: Performance Measurement • Does the public respect the agency and the officers on patrol? Do they feel they are treated with respect and courtesy? Are they satisfied with the results when they call for assistance? • Does the agency demonstrate efficiency of operations and conscientious use of public resources? The variety of factors that contribute to an overall assessment of the quality of the job done by law enforcement demonstrates the extraordinary breadth and depth of responsibility and authority we invest in this critical public agency. Certainly other factors can also influence the amount of crime in a given jurisdiction (such as unemployment, age and social structure of the community, and general economic trends and conditions), but law enforcement is generally held to task as the agency most responsible and, therefore, most accountable. And as noted above, measures of the volume of crime reported in a jurisdiction are but a single dimension in the complex array of factors that weigh in the broader assessment of the quality of work done by police.7 Law enforcement executives must juggle a broad assortment of responsibilities ranging from traffic enforcement to border and port security, from school safety programs to major incident response in terrorist attacks and natural disasters, from domestic disturbances to serial homicide investigations spanning years and multiple jurisdictions. And these responsibilities are only likely to expand given contemporary and evolving trends in homeland security and intelligence information sharing. Web Resource Icon Why Measure? Measuring performance, constantly assessing and monitoring critical performance metrics, and tailoring proactive response and follow-up are fundamental components of effective management in contemporary law enforcement—indeed, in any organization, public or private, government or industry, large or small.8 Building an effective performance management program within an agency requires a comprehensive management paradigm that does the following: • Defines agency objectives in clear and measurable terms • Incorporates routine and unambiguous measures of the processes initiated, the resources expended, and the outcome and impact of agency-sponsored projects and activities • Empowers managers and staff to develop new and innovative responses • Monitors and evaluates their impact and influence on critical measures of success. Page 19 Chapter 1: What Is Performance Measurement and Why Is It Important? CompStat (or COMpSTaT), shorthand for computer statistics, connotes the computer- generated statistical reports and geographic maps that were generated for weekly “crime meetings” that Commissioner Bratton and his staff convened at the New York City police Department (NYPD) shortly after he arrived in 1994. The program, of course, is significantly broader than simply generating statistical summaries and maps showing the location of crime. “Performance evaluation is more than an academic exercise, a matter of methodologies and numbers. How performance is measured affects not only what the public knows about the police, but also the character of police operations and the management climate. Because performance evaluations establish priorities, incentives, and requirements, they are much too important to be left to technicians. Performance measurement should be viewed as an integral, ongoing part of the management of policing.” —David Bayley.9 It is only by clearly articulating the objectives of agencies, understanding the current environment within which they operate, establishing baseline measures on critical factors related to the overall success in meeting agency objectives, and constantly measuring the impact of agency actions taken to achieve defined objectives that we can be effective. CompStat is perhaps the best known and most well-documented contemporary example of the power of performance measurement in law enforcement management. The CompStat paradigm relies on four key principles: 1. Timely and accurate intelligence. 2. Effective tactics. 3. Rapid deployment of personnel and resources. 4. Relentless follow-up and assessment to ensure that the problem has been solved.10 William Bratton, New York City Police Commissioner at the time the agency developed CompStat, described the program in the following way: “In the 6-week Compstat cycle, the effectiveness of every new tactic or program is rapidly assessed. Failed tactics don’t last long, and successful tactics are quickly replicated in other precincts. Gathering field intelligence, adapting tactics to changing field conditions, and closely reviewing field results are now continual, daily processes. The NYPD can make fundamental changes in its tactical approach in a few weeks rather than a few years.”11 “…the NYPD now has the technological capacity to identify crime patterns almost immediately, and our response can be virtually contemporaneous with evolving patterns. We also have significantly tightened our management controls over police activities, empowering officers and commanders at the local level while holding them accountable for their crime-fighting results. Officers and Page 20 Part I: Performance Measurement commanders are now guided by comprehensive and coordinated strategies and tactical plans that provide enough flexibility to permit the crafting of appropriate site-specific responses. We relentlessly follow up on their activities to ensure that problems are solved rather than displaced. We have also recognized and embraced the wisdom of Wilson and Kelling’s ‘broken windows’ theory and its emphasis on the criminogenic nature of quality-of-life offenses. We have convinced officers and commanders that serious crime as well as public fear of crime can be reduced by tending to these “minor” offenses and annoyances of urban life.”12 “The Chicago Police Department demands more than meeting performance metrics—we also demand a return on investment.” 13 —Ron huberman, chief of staff for Chicago Mayor Richard Daley (former deputy superintendent, Chicago police Department) See page 21 for an example of tangible return on investment. performance measurement, in simplest terms, is the comparison of actual levels of performance to pre-established target levels of performance. Know Your Purpose No matter the nature of the program, agency, initiative or project, you’re making an investment of time, money, resources, etc., for some purpose. Measuring progress in achieving the intended objectives is an elementary, if often complex and frequently overlooked, step in basic management and administration. Measuring progress in meeting agency, department, division, program, and/or project objectives is a core step in determining: 1. Whether the programs, projects, or other initiatives being undertaken are properly aligned with the mission and goals of the sponsoring organization. 2. Whether they produce tangible improvements in processes and outcomes that collectively support agency objectives. 3. What factors are most clearly associated with success and failure. 4. Whether the initiative is repeatable, extendable, or sustainable over time, under different circumstances, and in different locations. 5. Whether the initiative represents a fair return on investment (ROI). According to The Performance-Based Management Handbook, Volume 1, “Performance measurement, in simplest terms, is the comparison of actual levels of performance to pre-established target levels of performance. To be effective, performance must be linked to the organizational strategic plan. Performance-based management essentially uses performance measurement information to manage and improve performance and to demonstrate what has been accomplished. In other words, performance measurement is a critical component of performance-based management.”14 “All high-performance organizations whether public or private are, and must be, interested in developing and deploying effective performance management systems, since it is only through such systems that they can remain high-performance organizations.” From Serving the American Public: Best Practices in Performance Measurement, National Performance Review (1997) Page 21 Chapter 1: What Is Performance Measurement and Why Is It Important? 2 Return on Investment 15 (Illinois State Police [ISP]/Chicago Police Department [CPD]) In many public safety initiatives, much of the efficiencies are difficult to quantify. Attempting to put a cost savings on lives saved, crimes reduced, victims spared the agony of abuse, and communities experiencing a greater feeling of safety are often quantified simply as priceless. Although these priceless attributes will most certainly be developed with this project, the tangible saving to Illinois taxpayers can also be quantified in the tens of millions of dollars! The following categories demonstrate a variety of areas where significant tangible ROI is expected. Direct Savings from the Elimination of Redundant Development Direct Development Savings Through the review of bids submitted for a statewide RMS, the ISP estimates an independent automated records management system with a data warehouse component would cost at least $6 million. This project was put on hold because sufficient funds were not available to move forward with the development. I-CASE, the records management component of I-CLEAR, will cost approximately $2 million to complete. Reduction of future expenditures - $4 million In 2002 the ISP released a request for proposals (RFP) for the replacement of our existing Criminal History Records Information (CHRI) System. The only bidder that met the minimum mandatory requirements presented a bid for $10 million. This project was not executed because sufficient funds were not available to move forward with the development. The multi-year cost to expand CPD’s CHRI system and merge the existing CPD/ISP CHRI databases is estimated at $4 million. Reduction of future expenditures - $6 million Direct Labor Savings By 2007, 25 positions statewide that are responsible for keying and filing paper-based records can be eliminated. The positions are estimated to cost the ISP $50,000 each annually. Future annual savings - $1.25 million Source: Colonel Kenneth Bouche, ISP Grant Requirements Icon You Have To! Measuring performance in achieving project objectives is also a key part of federal grant-funding programs. The Office of Community Oriented Policing Services (COPS), for example, monitors its federally funded projects to ensure fiscal responsibility and achievement of program objectives. As noted in the COPS Office’s 2004 document Grant Monitoring Standards and Guidelines for Hiring and Redeployment: 22 Part I: Performance Measurement Federal regulations require that the Federal government monitor any financial assistance to ensure that funds are spent properly. All COPS grantees are required to participate in grant monitoring and auditing activities by the COPS Office, the DOJ’s Office of the Inspector General (OIG), the DOJ’s Office of Justice Programs’ Office of the Comptroller (OC), the U.S. General Accounting Office (GAO) [now the Government Accountability Office], and other duly authorized representatives of the Federal government. The purpose of monitoring is to ensure that grantees meet their programmatic and financial requirements by properly managing funds and following grant program requirements. In general, the COPS Office monitors program activities and financial activities. Monitoring activities include on-site program reviews, the COPS Count survey, Office-based Grant Reviews, and office-based complaint, financial, and legal reviews.16 Performance management is needed not only to appraise individual project effectiveness, demonstrate conformity with funding requirements, and achieve broader organizational mission and goals—it’s also about accountability, i.e., demonstrating good stewardship of public resources. Citizens entrust law enforcement with significant powers and support the institution with public funds. Accordingly, law enforcement has a duty to demonstrate that it exercises these powers responsibly and expends funds in a manner that is both efficient and effective. As a consequence, effective performance management also requires regular reporting of results to funding authorities, the general public, key stakeholders, and internal staff, including commanders and supervisors, as well as line and support staff. Why Performance Measures Fail 1. Lack of a viable and active strategic planning process within the organization. 2. Failure to change the police culture to “embrace” performance management and in particular, “accountability.” 3. Lack of sufficient funding and technical competency to support performance management, especially data collection and analysis on an ongoing basis. 4. Lack of knowledge, skills, and abilities and the organizational capacity to successfully operate effective performance management processes. 5. Inability to sustain the performance management process in the face of budget cuts and scarce or reduced resources. 6. Lack of user input. 7. Incomplete requirements and specifications. 8. Lack of executive support. 9. Unclear objectives. Page 23 Chapter 1: What Is Performance Measurement and Why Is It Important? 23 10. Unrealistic time frames.17 11. Initial establishment of too many measures, rather than selecting core measures. (*See sidebar to the left for an example of this point.) * For example: In the shaded box on page 29 regarding trial court performance standards, “the original version had 68 measures under the 5 areas cited, of which only about 18 could be economically collected on an ongoing basis. No court ever implemented all of them or even most of them. After almost 15 years of failure, the courts tried again with 10 core measures. Early evidence with pilot courts suggests that only 3 or 4 of those will ever be widely implemented on a routine basis.” —Thomas M. Clarke, ph.D., National Center for State Courts Resistance to performance measurement is often based on fear of accountability and concern about recrimination, repercussions, or reprimands due to the outcomes of measures. Why People Are Afraid of Performance Measurement 1. Fear of measurement and new systems. 2. Lack of common definitions and terms. 3. Inconsistent or weak buy-in, and lack of understanding. 4. Poorly defined and understood decisions and strategies that are not actionable, and not linked to individual actions. 5. Treating budget development separately from strategy development. 6. Measures set independently of a performance framework, or measures with no ownership. 7. Losing momentum. 8. No performance targets, targets set too high or too low, or lack of relevant data to establish targets. 9. Little or no strategic feedback. 10. Lack of meaningful employee involvement.18 Connect Your Project to the Broader Mission This Guide is primarily about building an Integrated Performance Management Framework as an essential component of an enterprisewide management structure for law enforcement. Additionally, this Guide addresses specifically how to plan, implement, and manage project-targeted performance assessment to meet the reporting needs of funding and authorizing agencies, but the broader context of performance management in law enforcement is the overriding theme. Performance measurement should not be viewed solely as a single project-related activity designed simply to evaluate the effectiveness of specific projects, though it may well begin that way. Assessing the performance of individual projects and other law enforcement initiatives is an important and often a required element of funding and management authorities who support new and innovative law enforcement programs. Nevertheless, performance management should be a viewed as a critical component in an overarching management paradigm that assesses the value of individual projects to meet defined, discrete project objectives, and relates the value of these individual projects to the broader mission of the agency. ChapTER 2 Establish an Integrated Performance Management Framework Page 27 Chapter 2: Establish an Integrated Performance Management Framework Many manuals and textbooks describe in detail research and evaluation methodologies and analysis techniques.19 What is presented here is a comprehensive management process that incorporates performance measurement, assessment, and monitoring into a broader Integrated Performance Management Framework for law enforcement and other justice agencies.20 To be effective, the Integrated Performance Management Framework suggested in this report must be an essential element of broader strategic planning for the law enforcement (or any other justice) agency. Notwithstanding this perspective, the core principles underlying this framework are equally relevant in measuring the performance of individual programs within law enforcement agencies, regardless of whether they are new patrol tactics, community outreach initiatives, or technology implementation projects. Each of the principles described in the Integrated Performance Management Framework can be scaled to address specific programs and aligned with the broader mission and objectives of the sponsoring law enforcement agency. Stop Sign Icon It’s Not Just About Whether Crime Went Up or Down As noted above, the job of law enforcement is significantly more complex than simply reducing crime and measuring performance on the single (though certainly important) dimension of the number of crimes reported in a jurisdiction. Even CompStat, which superficially may appear to focus predominantly on the volume of crime reported at the precinct level, is in reality also focused on improving the quality of life for city residents, efficient use of resources, and effective operations.21 The modern police chief and sheriff cannot stand solely on the volume of crime reported in their jurisdiction. There needs to be a balance among multiple measures reflecting the complex array of responsibilities of law enforcement agencies…in other words, a balanced scorecard for law enforcement. Page 28 BALANCED SCORECARD: a measurement- based strategic management system, originated by Robert kaplan and David Norton, which provides a method of aligning business activities to the strategy, and monitoring performance of strategic goals over time. For more information, see http://www. balanced scorecard.org/. Balanced Scorecard Provides Template The notion of a balanced scorecard was originally proposed by Kaplan and Norton.22 Business is not measured solely on the dimension of whether it has made a profit within a single quarter or a year, though many corporate executives have been harshly criticized in recent years for myopically focusing on short-term financial gains for personal benefit. Rather, the performance of a corporation must be measured across four interrelated business perspectives: 1. Financial – How do we look to stakeholders? 2. Customer – How well do we satisfy our internal and external customers’ needs? 3. Internal Business Process – How well do we perform at key internal business processes? 4. Learning and Growth – Are we able to sustain innovation, change, and continuous improvement? Just as business performance must be measured across multiple dimensions, law enforcement performance cannot be measured on the single dimension of the volume of crime reporting in a jurisdiction. If it were that simple, the quarterly and annual Uniform Crime Reports (UCR) of local, state, and federal agencies would be sufficient appraisals of the job done by law enforcement executives. Instead, the performance of law enforcement agencies must ultimately be evaluated as a complex of measures related to the varying roles and responsibilities associated with this very public institution. Understanding the complex nature of the job of law enforcement is important, as it relates to defining the mission of the agency, and assuring that individual projects undertaken are closely aligned with broader agency objectives. It is critical to understand the complex nature of responsibilities so that individual projects can be properly coordinated and aligned to assist the department in achieving the overarching agency responsibilities, and in understanding how performance management relates to the broader management paradigm for law enforcement. The Seven Dimensions of a Balanced Scorecard In Recognizing Value in Policing: The Challenge of Measuring Police Performance (2002), Mark H. Moore, et. al., have suggested that the balanced scorecard for law enforcement be comprised of seven different dimensions: 1. Reduce criminal victimization. 2. Call offenders to account. 3. Reduce fear and enhance personal security. Page 29 Text Box: Trial court performance standards focus on: a. access to Justice b. Expedition and Timeliness c. Equality, Fairness, and Integrity d. Independence and accountability e. public Trust and Confidence. See National Center for State Courts, Trial Court performance Standards and Measuring System, at http://www.ncsconline.org/D_Research/tcps/index.html (End of Text Box) 4. Guarantee safety in public spaces. 5. Use financial resources fairly, efficiently, and effectively. 6. Use force and authority fairly, efficiently, and effectively. 7. Satisfy customer demands/achieve legitimacy with those policed.23 Each can be measured individually. Collectively, they represent a broad assessment of the overall quality of work by the local law enforcement agency. Law enforcement authorities around the world are increasingly aware of the wide- ranging nature of their responsibilities and are implementing comprehensive programs designed to constantly and publicly measure performance across an array of activities.24 Developing a comprehensive performance management system is really part of effective police management. It’s more than simply evaluating the relative success of an individual program or project, though that is an important element. Clearly we must assess the impact of specific activities, projects, and initiatives and we must identify where problems exist, mount initiatives to address the problem identified, and evaluate the success of our efforts to determine whether they have been successful and a wise expenditure of resources. Web Resource Icon The SARA problem-solving process (short for Scanning, Analysis, Response and Assessment) is one example of a tool used by law enforcement agencies to measure performance. An integral component of Problem-Oriented Policing (POP), SARA is principally focused on evaluating individual projects or initiatives, assessing whether the problem identified in fact declined, and, if so, whether the response was the proximate cause of the decline or remediation.25 The model is elegant in its simplicity and is direct, which is not to say that it’s easy. Evaluating the performance of specific programs or projects is a complex process requiring careful planning before, during, and after implementation, rigorous Page 30 research design and methodology, the application of appropriate and sometimes sophisticated analytic techniques, and knowledgeable analysis and appraisal. SARA represents an important tool for police executives in assessing the value and efficacy of different responses to crime, and it serves as an effective methodology for project evaluation even beyond POP. Readers are encouraged to utilize the SARA process for project evaluation and should consider this tool as an effective research methodology for incorporation in the Integrated Performance Management Framework presented in this Guide. SARA takes proper account of both process evaluations and impact evaluations. Process evaluations focus on how the initiative was executed; the activities, efforts, and workflow associated with the response. As John E. Eck notes in Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers (2001), process evaluations ask whether the response occurred as planned, and whether all components worked as intended.26 Fundamentally, a process evaluation posits the question, “Are we doing the thing right?” Impact evaluations focus on the outcome (the what) of the initiative; the output (products and services) and outcome (results, accomplishment, impact).27 Did the problem decline or cease? And if so, was the response the proximate cause of the decline? Fundamentally, the impact evaluation posits the question, “Are we doing the right thing(s)?” Are we doing INPUT (resources, including cost and workforce) + PROCESS (activities, effort windows)= Are we Doing the thing Right? (How?) OUTPUT (products and services produced) + OUTCOME (results, accomplishments, impact)= Are we doing the RIGHT THINGS? (What?)28 Page 31 As useful as the SARA process is for individual POP and other project evaluations, and it is indeed a very useful and user-friendly project evaluation methodology, it does not represent a comprehensive performance management paradigm contemplated in this Tech Guide. For that we have borrowed from the six-step performance-based management program originally developed for the U.S. Department of Energy.29 The Six Steps to Establishing a Performance- Based Management Program In the six-step process portrayed in Figure 2.1 “the first step is to define the organization’s mission and to establish its performance objectives. The next step is to establish performance measures based on and linked to the outcomes of the strategic planning phase. Following that, the next steps are to do the work and then collect performance data (measures) and to analyze, review, and report that data. The last step is for management to use the reported data to drive performance improvement, i.e., make changes and corrections and/or ‘fine tune’ organizational operations. Once the necessary changes, corrections, or fine tuning have been determined, the cycle starts over again.”30 While establishing accountability for performance appears as Step 3 in the figure, in reality it pervades the entire organization and must be a fundamental component of the management process guiding the agency. Figure 2.1 The Six Steps to Establishing a Performance-Based Management Program Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 4: Establish a process/System for Collecting Data to assess performance Step 2: Establish an Integrated performance Measurement System Step 1: Define Organizational Mission and Strategic performance Objectives Step 3: Establish accountability for performance Page 32 Each step in the performance-based management program will be discussed in more detail in Part II and each will be related to common issues and programs addressed in local law enforcement agencies, using actual programs and projects as illustrations wherever possible. Part I Checklist 1. Agency leaders and managers must understand basic concepts associated with performance management: a. Performance measurement b. Return on investment (ROI) c. Balanced scorecard d. SARA (Scanning, Analysis, Response and Assessment) e. Process evaluations f. Impact evaluations. 2. Agency leadership should understand the role of performance measurement within strategic planning. 3. Agency leadership should understand from a broad perspective the role of performance management. 4. Agency leadership should understand local, state, and federal government reporting requirements regarding performance. Part II: Performance Management “The concept of performance measurement is straightforward: You get what you measure, and you can’t manage a project unless you measure it.” —From Performance-based Management, General Services Administration ChapTER 3 DEFINE MISSION aND STRaTEGIC pERFORMaNCE OBJECTIvES Page 37 Chapter 3: Define Mission and Strategic Performance Objectives What A document that clearly articulates the mission of the agency (or the individual project) and defines performance objectives in unambiguous, measurable, actionable terms. Performance objectives should be directly linked to organizational goals and objectives derived from the broader mission statement, and ideally, these will be drawn from the organization’s strategic plan. Why Tying performance objectives of organizational units and individual projects to broad agency mission statements ensures coherent and deliberate management direction and control, enables both internal and external stakeholders to observe the direct contribution of their efforts in achieving agency goals, and provides objective measures of progress and success, and facilitates course correction and realignment. Who Senior departmental leadership, with active input and communication with employees, stakeholders, and customers/clients. When Shortly after the decision-making structure is formed and prior to initiating major projects. Orangizational Chart Step 1: Define Organizational Mission and Strategic performance Objectives Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 4: Establish a process/System for Collecting Data to assess performance Step 2: Establish an Integrated performance Measurement System Step 3: Establish accountability for performance page 38 Part II: Performance Management “If you don’t know where you’re going, any plan will do.” —peter F. Drucker Legendary management authority Fundamentally, performance assessment and management is about effective strategic planning. Strategic planning is “a continuous and systematic process where the guiding members of an organization make decisions about its future, develop the necessary procedures and operations to achieve that future, and determine how success is to be measured.”31 As previously noted, the mission of law enforcement is complex, evolving, and extraordinarily expansive. Developing a coherent mission statement and defining strategic performance objectives is considerably more involved than simply posting a slogan “To Serve and To Protect” on the side of patrol vehicles. While such a brief mission statement may be sufficient to encapsulate the department’s values and reflect public sentiment on the role of law enforcement in contemporary society, it provides few clues on how to measure the success of the agency, the performance of its employees, or the value of specific operations. For that, significantly more detailed planning is required. Agencies often define their mission or vision as fundamental elements of their strategic planning process, and these are indeed important elements in establishing corporate/organizational direction and policies. Mission, vision, values, assumptions, and business strategies, however, are rather different elements, all of which are important in establishing organizational objectives and tactical plans. • Mission statements identify the overall purpose for which the organization is organized. (See the mission statements of five police departments on page 39.) • Vision statements describe the future business environment and the role of the organization within it. • Value statements reflect fundamental beliefs and values guiding the agency, the nature of its responsibilities, and the philosophy underlying its approach. • Assumptions are also frequently discussed in strategic planning efforts, describing business environmental conditions that are expected in the future. • Business strategies identify how objectives are to be accomplished, e.g., community-oriented policing.32 Original Tech Guide Reference Icon pages 51-62 Detailed guidance in developing these critical elements of the broader strategic plan for law enforcement is provided in the original Law Enforcement Tech Guide. Please see Chapter 3 and the 12-Step Program for Developing the Charter for more detail.33 Page 39 Chapter 3: Define Mission and Strategic Performance Objectives 39 Police Department Mission Statement Examples Philadelphia Police Department Mission: The mission of the Philadelphia Police Department is to work in a true partnership with our fellow citizens of Philadelphia to enhance the quality of life in our city. And, by raising the level of public safety through law enforcement, to reduce the fear and incidence of crime. In accomplishing these goals: service will be our commitment, honor and integrity our mandate. http://www.ppdonline.org/hq_mission.php Las Vegas Metropolitan Police Department Mission: The mission of the Las Vegas Metropolitan Police Department is to protect people, property and rights in our community. http://www.lvmpd.com/about/index.html Vancouver Police Department Mission: In fulfillment of its public trust, the VPD maintains public order, upholds the rule of law and prevents crime. http://www.city.vancouver.bc.ca/police/ Oakland (CA) Police Department Mission: The Mission of the Oakland Police Department is to provide competent, effective public safety services to all persons with the highest regard for human dignity through efficient, professional, and ethical law enforcement and crime prevention practices. http://www. oaklandpolice.com/ Washington, D.C., Metropolitan Police Department Mission: The mission of the Metropolitan Police Department is to prevent crime and the fear of crime, as we work with others to build safe and healthy communities throughout the District of Columbia. http://mpdc.dc.gov/mpdc/cwp/view,a,1230,q,537827,mpdcNav_ GID,1529,mpdcNav,|.asp The Washington, D.C., Metropolitan Police Department (MPDC) defined prevention as a key element of its mission. The department took the next step of identifying how it was going to address this mission-critical objective in tangible, actionable terms and compared and distinguished how the approaches relate to one another. MPDC: Three Approaches to Prevention MPDC operationalized its mission to “prevent crime and the fear of crime” by developing three key and interrelated approaches to crime and disorder (as shown in Figure 3.1). Used together, these approaches address immediate crime problems by focusing law enforcement efforts, building partnerships with community stakeholders to sustain success, and using the resources of government and other organizations to work on the underlying causes of crime in a community. The approaches are: 1. Focused Law Enforcement; 2. Neighborhood Partnerships; and 3. Systemic Prevention.34 Page 40 Part II: Performance Management Figure 3.1 MPDC: Comparing the Three Approaches 35 Method of Prevention: Focused Law Enforcement Focus: Repeat criminal offenders and regulation violators. Repeat criminal offenses. Communities in distress. Effects: Disruption or termination of chronic crime activity, reduction of fear, and building of community confidence in the police. Key Police Activities: Crime analysis, directed patrol, tactical, follow-up investigations, arrests, and partnership with other regulatory or law enforcement agencies. Method of Prevention: Neighborhood Partnerships Focus: Physical and social conditions that lead to chronic crime and disorder. Community building. Effects: Active involvement of community and other government services leading to neighborhood stabilization. Key Police Activities: PSA [Police Service Area] integrity, PSA meetings, problem solving, and partnership with other agencies and the community. Method of Prevention: Systemic Prevention Focus: Health, social, educational, and economic conditions of individuals, families, and communities. Effects: Individuals, families, and communities build a resistance to crime and violence. Key Police Activities: Sharing information with, working with, and being advocates for the government and nongovernment agencies that effectively serve people and communities. The three approaches, or “Methods of Prevention,” are focused on different causal, contributory, and remediation factors associated with crime and official response in their jurisdiction, and each calls for different law enforcement, other government agency, and community response. MPDC has described the “Effects” or outcomes expected in general terms and has identified the “Key Police Activities” that it (the Police Department) will undertake to support these initiatives (e.g., crime analysis, directed patrol, partnership with other agencies and the community, etc.). Page 41 Chapter 3: Define Mission and Strategic Performance Objectives These are important steps in mapping how an agency is going to operationalize broad mission and vision statements by identifying specific programs, projects, and initiatives to achieve results. Additionally, linking specific initiatives to agency-defined goals helps ensure that operations are tightly aligned with the agency’s mission and represent logical, practical steps to achieving intended results. It also has the positive effect of presenting a road map for employees, stakeholders, and customers (the general public, in the case of law enforcement) so that they can clearly observe the relationship between individual activities and the broader goals of the sponsoring organization. Moreover, MPDC identified intended consequences of the actions and squarely accepted responsibility, thereby laying groundwork for accountability, by identifying the “Key Police Activities” it would undertake as part of its strategic plan. Further elaboration is needed to: a) Define time frames for the activities. b) Identify information that is needed for successful planning and implementation, and assignment of persons responsible for data collection, analysis, and management. c) Specify measures that will be incorporated in evaluating expected outcomes and results to be achieved, and how success will be defined. 36 d) Assign responsibility for deployment and performance. e) Identify resources (e.g., money, people, equipment) that are needed, both for operations and for performance measurement and monitoring. MPDC has defined performance targets across the entire agency and has structured these targets on two important dimensions: 37 1. Strategic Goals, which are closely aligned with the Value Goals suggested by Moore, et. al. That is: a) reduce and prevent crime and criminal victimization; b) produce justice by calling offenders to account for their crimes; c) enhance the sense of safety and security in public spaces; d) use force and authority judiciously and fairly; e) ensure customer satisfaction; and f ) develop an organization that is competitive, professional, equitable, and equipped with state-of-art tools and systems. 2. Service Areas/Programs across the agency. Regional Field Operations, Investigative Field Operations, Special Field Operations, Public Safety Communications Center, Corporate Communications, Strategic Planning, Organizational Development and Crime Prevention, and Agency Management. The performance targets identified in the MPDC plan are precise and specific to the Service Area/Program, as the example on page 42 demonstrates: 38 Page 42 It is important to separate worthy goals that cannot be tightly controlled from ones that possibly can be. as an example, Measure #8 (average response time) is conceivably controllable, but Measure #11 (reduction in crashes with fatalities) may not be. Even if measures are worthy, they may only induce frustration if a law enforcement agency is unable to control the outcome in real life. Regional Field Operations Program The purpose of the Field Operations Program is to provide response, patrol, tactical, investigative, problem solving, security, and traffic safety services to residents, visitors, and commuters in Washington, D.C., so they can be safe and feel safe from crime and injury. Key Results Measures: 1. In FY 2004, reduce Part 1 violent crime by 2% over previous fiscal year. 2. In FY 2004, reduce Part 1 property crime by 2% over previous fiscal year. 3. Reduce the rate of sustained citizen allegations of police misconduct per 1,000 sworn officers by 2% in FY 2004. 4. Reduce by 2% from previous fiscal year, the percent of victims surveyed reporting they were victimized more than once in the past 3 months. 5. Reduce by 5% the annual average number of city blocks with 15 or more repeat calls for service within a month for public disorder in FY 2004. 6. Reduce by 5% the annual average number of city blocks with 12 or more repeat calls for service within a month for drug activity in FY 2004. 7. In FY 2004, reduce number of addresses with 3 or more repeat calls for service during the year for domestic violence by 2% from the previous fiscal year. 8. In FY 2004, achieve a 2% reduction in the average response time (in minutes) from time of dispatch for Priority One crime calls, in progress or that just occurred, in which an officer arrived on the scene. 9. In FY 2004, increase by 2% the percentage of victims of crime reporting that they were “very satisfied” or “somewhat satisfied” with the initial police services they received when they were victims of crime. 10. In FY 2004, maintain a 62% target for the percentage of lieutenants, sergeants, and officers assigned to the PSAs [Police Service Areas]. 11. In FY 2004, reduce by 3% from previous fiscal year the number of vehicle crashes with driver and pedestrian fatalities. 12. Reduce by 2% the ratio of Part I arrests of youth offenders to detentions or arrests of youth for all crimes in FY 2004. —Metropolitan Police Department Strategic Business Plan FY 2004-2005 (June 2003) Additionally, MPDC has further elaborated performance targets by Service Area/ Program, and has included critical elements in its overall performance-based management program by identifying the purpose of the activity, the services included, the measurement targets, responsible parties (i.e., both program and activity managers), and resources available (i.e., both budget and personnel) (Figure 3.2).39 Page 43 Chapter 3: Define Mission and Strategic Performance Objectives Figure 3.2 Metropolitan Police Department Performance Plan PROGRAM Activity: Activity Purpose Statement REGIONAL FIELD OPERATIONS (Regional Operations Command - Central) The purpose of the ROC Central Activity is to provide focused law enforcement, response to calls for service, neighborhood partnerships and problem-solving, traffic control, and systemic prevention services to the people who live and work in D.C. so that they can feel safe and be safe from crime and injury. PROGRAM Activity: Services that Comprise the Activity REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): Focused Law Enforcement Neighborhood Partnerships and Problem Solving • Asian Liaison Unit Responding to Calls for Service Traffic Control Systemic Prevention • School Resource Officers • Metropolitan Police Boys and Girls Clubs Officers District Station OperationsOffice of the Assistant Chief for ROC-Central PROGRAM Activity: Activity Performance Measures (Target and Measure) REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): Results (Key Result Measures are italicized) Part 1 violent crime Part 1 property crime Ratio of Part 1 arrests of youth offenders to detentions or arrests of youth for all crimes Rate of sustained citizen allegations of police misconduct per 1,000 sworn officers Percent of victims surveyed reporting they were victimized more than once in the past 3 months Annual average number of city blocks with 15 or more repeat calls for service within a month for public disorder Annual average number of city blocks with 12 or more repeat calls forservice within a month for drug activity Number of addresses with 3 or more repeat calls for service during the year for domestic violence Average response time (in minutes) from time of dispatch for Priority One crime calls, in progress or that just occurred, in which an officer arrived on the scene Percentage of victims of crime reporting that they were very satisfied or somewhat satisfied with the police services they received when they were victims of crime Percentage of lieutenants, sergeants, and officers assigned to the PSAs Outputs: No. of reports taken for Part 1 violent and property crimes Demand: No. of reported Part 1 violent and property crimes (based on average of last 5 years) Efficiency: Cost per reported Part 1 violent and property crime PROGRAM Activity: Responsible Program Manager REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): EAC Michael J. Fitzgerald PROGRAM Activity: Responsible Activity Manager REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): AC Brian Jordan PROGRAM Activity: FY 2004 Budget (Gross Funds) REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): $68,169,413 PROGRAM Activity: FTEs REGIONAL FIELD OPERATIONS (Regional Operations Command - Central): 1,032 Page 44 PERFORMANCE PLAN TEMPLATE Mission / Vision / Values / Assumptions Strategic Performance Objective 1......Define Operation/Activity.....................Title Purpose................................Describe purpose(s) of initiative Execution..............................Define how you’re going to do it Performance Target(s)..................Target(s) (e.g., reduce violent crime by 10%) Performance Measures...................How you’re going to measure it Owner..................................Person responsible/accountable Resources..............................Resources needed for this initiative Budget.................................Funding dedicated to this initiative FTE....................................Staffing dedicated to this initiative Strategic Performance Objective 2 Operation/Activity Purpose Execution Performance Target(s) Performance Measures Owner Resources Budget FTE The MPDC Strategic Business Plans for 2004-2007 represent a careful elaboration of strategic objectives that are effectively tied to the agency mission statement, program activities designed to achieve defined objectives, performance measures and targets that will be used in assessing progress in meeting the objectives, personnel who are responsible (and therefore accountable) for the operations, and resources available to support the program. These core elements of an effective performance-based management program, as highlighted in the performance plan template above, will be explored in more detail in following chapters. Page 45 Chapter 3: Define Mission and Strategic Performance Objectives While much of the discussion thus far has focused on building performance-based management capability at the agency level, the elements are equally relevant at the individual project level. If, for example, an agency is pursuing funding for a new records management system, the objectives of this automation solution must be tied directly back to the overarching mission of the sponsoring organization. Law enforcement does not seek new or upgraded automation of their records management process simply for the sake of automation, but rather to achieve the broader purposes implicit in the agency mission statement, and these links to agency mission should be explicitly stated. Objectives of this initiative might include improved efficiency (i.e., more rapid entry of incident reports, less duplicate data entry, less time required by uniformed officers to complete reports and therefore not on the street answering calls or on patrol, and less delay in making information available to investigators) and greater effectiveness (i.e., officers will be able to provide better service to the public by being on patrol, investigators better able to identify and apprehend offenders if they have immediate access to accurate and complete information on crime incidents, etc.). These objectives, in turn, should be carefully crafted, based on research on current operations and measured with precision. Step 1: Define Organizational Mission and Strategic performance Objectives Step One Checklist 1. Define agency: a. Vision b. Mission c. Values d. Assumptions e. Business strategies. 2. Understand current performance: a. Establish baseline measures of current operations b. Identify operational, political, and environmental factors that influence performance. 3. Define performance measures and targets. 4. Assign ownership for agency operations. 5. Allocate resources to support agency operations. ChapTER 4 ESTaBLISh a pERFORMaNCE MaNaGEMENT FRaMEWORk Page 49 Chapter: 4 Establish a Performance Management Framework What Create an Integrated Performance Management Framework as part of the strategic planning process and the management structure of the organization that clearly identifies the ongoing role of performance measurement, assessment, and monitoring, establishes responsibility and accountability, provides necessary resources and support, and empowers managers, employees, and the Performance Management Team. Why Performance management is not simply project-specific evaluation, but part of an overarching management philosophy and a critical component of organizational strategic planning; it must be institutionalized and provide an opportunity to engage employees, key stakeholders, and the general public. Who Senior departmental leadership and the Performance Management Team, with active input and communication with employees, stakeholders, and customers and clients (the general public). When As part of the strategic planning process for the agency. Focus: Step 2: Establish an Integrated performance Measurement System Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 4: Establish a process/System for Collecting Data to assess performance Step 1: Define Organizational Mission and Strategic performance Objectives Step 3: Establish accountability for performance Page 50 Part II: Performance Management “The urgent need today in measuring police performance is to move away from a sterile conversation about performance measurement as an abstract technical problem and to understand it as a device that can be used managerially to shape the future of policing.” —Geoffrey p. alpert and Mark h. Moore, in Performance Measures for the Criminal Justice System (1993)40 The ABCs of Performance Management Creating an Integrated Performance Management Framework within the broader management structure of the organization is an essential step in building a robust performance-based management system. Senior leadership within the agency must believe in the role of performance-based management and must champion the cause both within and outside of the agency. Measuring, constantly assessing, and continuously monitoring performance, and altering operations in response to findings of success and failure, are not separate operations independent of the operational management of the organization, but rather are part and parcel of the fundamental strategic planning process. As noted in The Performance-Based Handbook, Volume 1, “…performance measures give life to the mission, vision, and strategy by providing a focus that lets each employee know how they contribute to the success of the [organization] and its stakeholders’ measurable expectations.”41 Integration of the performance management framework within the organizational management structure of the agency enables the performance measures to serve as effective agents for change, a critical element in driving agency action based on ongoing performance monitoring. Major Elements in Creating a Performance Management Framework 1. Define the Relationship of Performance Measurement to the Strategic Planning Process As noted earlier in this Guide, strategic plans set the foundation for the organization by defining the mission, vision, and values that drive organizational operations, by establishing priorities and strategies for implementation, and by assigning responsibility and allocating resources. Performance measures are the tools that provide ongoing assessment of the impact and outcome of operations, as well as an appraisal of the efficiency and effectiveness of the processes surrounding the operations. Remember—performance measurement will look both at how we implemented an operation, as well as whether the operation produced the intended result. International Icon The Home Office in the United Kingdom, which oversees police services throughout England and Wales, has published a National Policing Plan, establishing strategic priorities for police services for 2003–2006.42 As part of the Plan, and the planning process, the Home Office developed a Policing Performance Assessment Framework (implemented in 2003–2004 and described in their 2004–2007 National Policing Plan 43) that provides a comprehensive structure and national performance targets for forces in assessing their performance relative to priorities established nationally and locally, as part of their strategic planning process (Figure 4.1). Page 51 Chapter 4: Establish a Performance Management Framework The National Policing Plan 2004–2007, Uk home Office: “The aim of the Policing Performance Assessment Framework is to … improv[e] and build on the current performance framework by: • Balancing comprehensive coverage with simplicity of concept and understanding. • Representing a common view of key stakeholders, being seen as balanced and fair. • Providing the information required to help deliver real improvements in policing. Figure 4.1 Performance Management Cycle: Priorities Objectives Performance Indicators Targets Assess Performance (1) Step 1 is to set priorities. National priorities are set out within th[e] National Policing Plan. (2) Step 2 is to determine objectives that will help to achieve these priorities. The key national objectives required to meet priorities are listed in th[e] Plan. (3) Step 3 is to establish performance indicators that will help to monitor performance in achieving objectives. (4) Step 4 is to set targets for performance improvement—what level of performance do we want to achieve? (5) Step 5 is to assess performance against the targets we have set ourselves. How well have we done and how can we do better? It is through steps 3–5 that the Policing Performance Assessment Framework is being applied.” 44 The UK Police Performance Assessment Framework represents a significant achievement in tying performance to strategic planning objectives set nationally and locally, to reflect individual community interests and circumstances. Moreover, by applying the framework to police services throughout England and Wales, the Home Office has effectively created national standards against which individual forces can assess their individual performance in a unified approach. Page 52 Similar efforts have not yet been undertaken nationally in the United States, though initiatives are presently underway to develop functional standards for key law enforcement applications,45 and correctional administrators have developed performance measures for correctional institutions, including outcome-based performance measures, key indicators, and resource materials to assist correctional administrators in implementing this comprehensive program. 46 For example, the COPS Office is developing a community policing implementation assessment tool that would operationalize community policing and allow agencies to measure and evaluate their implementation across various community policing elements. 47 This self-assessment instrument is being developed using knowledge obtained from existing literature, input and guidance from practitioners, experts, and other stakeholders, and results from a limited field testing and validation phase. This tool is being designed to help agencies gain a more objective and complete understanding of their community policing implementation efforts, and to support training efforts, strategic planning, performance benchmarking, and other management initiatives. Local and state law enforcement agencies can nevertheless build a comprehensive performance management framework as part of their overall management structure, and ensure that performance measurement is a critical component of their strategic planning process. Indeed, defining the role of performance measurement in providing feedback on the strategic planning process is an essential element in all planning models. 2.Build the Performance Management Team A formal Performance Management Team should be appointed to lead, implement, and monitor performance measurement. The team should be made up of people who actually do the work to be measured and people who are very familiar with the work to be measured. It is important that each person understands the task before him or her and his or her role in its accomplishment. Stop Sign Icon Remember: 1) Good measurement takes a commitment of staff resources, and 2) This assignment must be taken seriously and allowed to continue, especially in times of lean personal resources. Executive Sponsor Icon a. Senior management involvement Senior management of the organization must play a key role in introducing, championing, and promoting the performance management program. Leadership commitment to development and use of performance measures must be apparent and genuine, as must their commitment to accountability (discussed in more detail in Chapter 5, Establish Accountability for Performance). Page 53 Chapter 4: Establish a Performance Management Framework b. Employee involvement Employees must also have voice and investment in strategic planning, as well as the performance management process. They, after all, will directly contribute to the input, output, outcome, performance, process, and every other aspect of the organization’s operations, and in large measure, it is their behavior (their performance, if you will) that will determine the overall success of the agency in reaching defined objectives. Depending on their role in the organization, they will have direct knowledge of the operations of the agency, the needs and capabilities of the organization, the interests and concerns of customers and the general public, and insight into operational factors that may significantly contribute to success in achieving strategic objectives. Their buy-in is as critical as senior management’s and will ensure effective implementation. Their involvement should be in understanding the operational context of agency initiatives and in crafting effective measures for both process and impact evaluations. c. Technical support The Performance Management Team may well need technical support for research methodology, statistical analysis, and information technology. It is crucial that representatives of the information technology section of the agency be actively engaged as members of the team, to ensure that data needed for effective performance measurement, analysis, and management can be provided from current or planned information systems in a timely, effective, and efficient manner. Agency resources should be brought to the task, though smaller agencies may not have analysis units with sufficient internal resources to support the effort. In such cases, it may be necessary to contract with specialists in the community to augment internal staff. Operational managers and staff who understand the fundamental business processes and mission of the organization should lead the team, but technical team members have a crucial role to play in implementing the program in a way that will produce valid, reliable, and effective performance measurement. 3. Address Stakeholder/Customer Needs Since one of the primary objectives of implementing performance management is to satisfactorily address the needs of stakeholders and customers (in the case of law enforcement, these customers include the general public), stakeholder and customer needs should be properly considered in developing the broader Performance Management Framework. The framework should consider not only the nature of the information needed to address stakeholder and customer concerns, but also address reporting and communication capabilities. How will performance results be shared with stakeholders and customers? How will we ensure that the measures evolve over time to meet the evolving needs and concerns of stakeholders and customers? What feedback mechanisms are in place to ensure that the measures are valid and reliable? Part II: Performance Management Figure 4.2 UK’s Policing Performance Assessment Framework Organizational chart A: Citizen Focus This is an area affected by all other aspects of performance and reflects satisfaction with service delivery as well as overall trust and confidence Priorities set nationally and locally 1 Reducing Crime Where police activity helps to prevent and reduce crime 2 Investigating Crime Where police activity solves crime and brings offenders to justice 3 Promoting Public Safety Where police activity promotes public safety and reduces anti-social behaviour 4 Providing Assistance Where police respond to general requests for assistance and support B: Organisational Capability Outcomes are affected by the level of resources available and how they are deployed “Domain A assesses the overarching Citizen perspective reflecting the commitment of the whole policing community to delivering a citizen-focused service to the public. From 2004-05, the citizen’s perspective will be considered in the following three areas of the Policing Performance Assessment Framework: User Satisfaction; Public Confidence; and Fairness and Equality. Domains 1-4 focus on Operational Effectiveness and, taken together, provide a comprehensive view of policing activity. Both the crime (Domains 1 and 2) and noncrime (Domains 3 and 4) activities of policing are covered in the Policing performance Assessment Framework. Domain B is focused on Efficiency and Organisational Capability. Activity Based Costing has been introduced in all forces this year (2003-04) to help link performance and resources. Measures in this domain will also assess how well an organisation is equipped to meet the future demands of policing.” 48 International Icon The UK’s Policing Performance Assessment Framework reflects an overarching focus on services to citizens, priorities established nationally and locally, and recognizes the impact organizational capabilities and resources have on achieving results (Figure 4.2). Page 55 Chapter 4: Establish a Performance Management Framework 4. Understand Performance Measurement Terminology Performance measurement jargon can be confusing, and needs to be understood and agreed to by the Performance Management Team. Generally, performance measures are divided into five types: 1. Input Measures—Used to understand the human and capital resources used to produce the outputs and outcomes. These are the raw figures associated with the dimensions of performance being measured. For law enforcement performance, input measures may include crime incident reports filed by reporting area, beat, district, or precinct, calls received by dispatch (i.e., CAD records), or surveys of residents capturing their experiences, concerns, and perceptions of safety. 2. Process Measures—Used to understand the intermediate steps in producing a product or service. In law enforcement, for example, a process measure could be the number of neighborhood Crime Watch meetings completed as scheduled, the number of laptop computers distributed and used by officers, or the number of officers assigned to special units to address specific crime problems. 3. Output Measures—Used to measure the product or service provided by the system or organization and delivered to customers. Examples of output measures are typically found in crime analysis reports that contain details regarding the nature of crimes reported, geographic distribution (automated crime maps), time of occurrence, and the nature of injury and financial loss. Other examples are found in reports regarding the characteristics of arrestees (extent of prior involvement with the legal system, criminal history records, relationship to victim, etc.). 4. Outcome Measures—The expected, desired, or actual result(s) to which the outputs of the activities of a service or organization have an intended effect. The outcome of law enforcement initiatives might be a reduction in the number of violent crimes reported in a reporting area, beat, district, or precinct. It might be a decrease in the number of 9-1-1 calls received by dispatch and an increase in the number of calls received by a 3-1-1 call system. Or it might be survey results of a community that show an increase in the perception of personal safety. 5. Impact Measures—The direct or indirect effects or consequences resulting from achieving program goals. An example of an impact is the comparison of actual program outcomes with estimates of the outcomes that would have occurred in the absence of the program. Impact Measures may in fact be a composite of Outcome Measures. That is, the impact of a program to improve the safety of a community may well be measured by the following: • The decreasing volume and rate of reported violent and property crimes (outcomes) Page 56 Part II: Performance Management • The increasing number of arrests made by officers (outcome) • The enhanced perception of personal safety by community residents (outcome) • A reduction in the number of emergency (9-1-1) calls received by dispatch (outcome). How well a commercial business performs is often summarily noted by its bottom line performance (i.e., a comparison of revenues generated versus costs in producing those revenues). There are, of course, many other measures that also comprise appraisals of how successful a business is, but fundamentally people look at the bottom line associated with net revenue. Performance measurement more generally, however, is often classified along different dimensions associated with efficiency, effectiveness, productivity, etc. The University of California identifies five classifications of performance measures:49 1. Efficiency. 2. Effectiveness. 3. Quality. 4. Timeliness. 5. Productivity. A Measure of . . . Efficiency Measures . . . The ability of an organization to perform a task. And is Expressed as a Ratio of . . . Actual input/planned input A Measure of . . . Effectiveness Measures . . . The ability of an organization to plan for output from its processes. And is Expressed as a Ratio of . . . Actual output/planned output A Measure of . . . Quality Measures . . . Whether a unit of work was done correctly. Criteria to define “correctness” are established by the customer(s). And is Expressed as a Ratio of . . . Number of units produced correctly/ total number of units produced A Measure of . . . Timeliness Measures . . . Whether a unit of work was done on time. Criteria to define “on time” are established by the customer(s). And is Expressed as a Ratio of . . . Number of units produced on time/ total number of units produced A Measure of . . . Productivity Measures . . . The amount of a resource used to produce a unit of work. And is Expressed as a Ratio of . . . Outputs/inputs As noted earlier, the job of law enforcement is considerably more complex than simply measured by the volume or rate of crime reported in a jurisdiction, and effective performance measures should take into substantial account each classification noted above, including an assessment of the costs and resources needed to achieve desired impacts.50 Page 57 5. Manage Performance Measurement It is important to effectively manage the performance measurement process to ensure that measures at each management level are kept to a minimum, that objectives are clearly defined and understood, that costs associated with obtaining a particular measure are contained and worth the effort, that measures are comprehensive and related to the dimension or activity being assessed, and that multiple measures do not conflict (e.g., that uniformed officer time savings are not being counted in different or conflicting ways). 6. Accept Accountability for Measures Accountability is a fundamental precept in performance management and it connotes taking responsibility for actions and accepting consequences, both positive and negative. Agency leadership and staff alike must “buy in” to the Performance Management Framework, as well as the individual measures that comprise the framework. This means that they must be engaged and invested in the program, recognize the validity of the measures and the value derived from assessing performance and tailoring actions based on continuous monitoring and assessment. 7. Communicate Effective and ongoing communication is crucial in establishing and maintaining an effective performance management system. Communication should permeate all levels of the organization, and staff and management alike should be actively made aware of the results of performance management initiatives. Periodic reports, formal briefings, and visual display of performance metrics are tools frequently used to inform personnel of the status of the program. Performance dashboards, which quickly convey the status of performance appraisals in graphic terms, are useful tools in providing needed feedback to users, managers, stakeholders, and customers (Figure 4.3). Stop Sign Icon Dashboards are a critical step that often cost more than expected. If not done well, the measures are ignored. It’s helpful to post two or three critical measures daily in the work areas of frontline staff so they can monitor their performance. One can easily imagine a comparable dashboard conveying impact measures for a law enforcement agency, for example, assessing the volume of crime reported, arrests made, survey results of public safety perception, and budget figures. Page 58 Part II: Performance Management Figure 4.3 Sample Balanced Scorecard Performance Dashboard Fuel: Financial Customer Do we have the financial resources and stability to reach our destination? Do we have the backing of our stakeholders? Temp: Customer Are we addressing and meeting the needs of our customers? Are they cold (not participating)? Are they hot (complaining)? Odometer: Maturity How long has our organization been “on the road?” How old is our measurement system? Is it time for a checkup? RPM: Internal Business Process Learning and Growth Are our internal business processes operating efficiently and effectively? In what gear are we operating? Too fast? Speed: Learning and Growth Are we growing and improving at a sustainable peace? Are we moving too slow? Too Fast Chapter 4: Establish a Performance Management Framework 9 Perhaps one of the more familiar examples of performance dashboards in law enforcement is the “Crime Clock” annually published by the FBI in its “Crime in the United States” report (Figure 4.4).51 As with all measures used in assessing performance, it is critical that the measures used are regularly reviewed to ensure that they are valid, reliable and accurate. Regular reporting of measures in a dashboard, or in any other medium, will have little substantive value (and in fact may do significant harm) if they are inaccurately compiled or reported, or fail to reliably record or measure the phenomena or activity of study. Figure 4.4 Crime Clock Every 22.8 seconds One Violent Crime Every 31.8 minutes One Murder Every 5.6 minutes One Forcible Rape Every 1.3 minutes One Robbery Every 36.8 seconds One aggravated assault Every 3.0 seconds One Property Crime Every 14.6 seconds One Burglary Every 4.5 seconds One Larceny-Theft Every 25.0 seconds One Motor vehicle Theft More information on the use of performance dashboards and other reporting and communication tools will be presented in Step 5 (see Chapter 7, Analyze, Review, and Report Performance Data). 8. Know How to Check/Test Your Measures After you have developed your performance measures, you will need to check/test them for soundness (i.e., completeness, applicability, usefulness, etc.). Knowing how to perform these checks/tests and, thus, knowing what to look for in a performance measure, will help your team develop sound performance measures from the start. 9. Learn From Others Now that your Performance Management Team is organized and ready to develop its performance measures, take one last important step—look at what other organizations similar to yours have done and are doing with regard to performance measurement. The point is to eliminate your team’s “reinventing the wheel” and, thus, save valuable time and resources. The odds run high that you will be able to find other page 60 organizations to share useful information that your team can adopt and adapt to its particular circumstances. 10. What Do You Measure Yourself Against? Performance measurement is all well and good, but what constitutes “Excellent,” “Satisfactory,” or even “Good” performance? In other words, how good is “Good,” and how do we even know what constitutes “Good” performance? a. Historical Agency Trends Clearly your agency can measure current performance against historical agency performance—how well are you doing on any of a variety of measures relative to how you performed last week, last month, last year, or an average of the past 2, 3, 5, or 10 years? This indeed is how the Uniform Crime Reporting (UCR) program measures crime throughout the United States.52 The program has been in existence since 1929 and annually receives crime reports from nearly 17,000 law enforcement agencies throughout the nation.53 In addition to reporting on the volume of crime and nature of crime reported, the UCR program also captures information on clearances (i.e., cases closed by arrest or other means), characteristics of victims and those arrested, and figures and circumstances surrounding law enforcement officers killed or assaulted. b. Pre/Post Program Performance Agencies often also evaluate performance based on measures of critical factors prior to implementation of a program or project, and then again after the program/project has begun. This pre/post design is a fundamental component of classic program evaluation methodology.54 An essential element in any performance measurement process is in understanding the baseline of current operations. How well is the agency (or the program, project, or initiative) currently performing by whatever measures are appropriate? What is the current and historical volume or rate of crime reported in the jurisdiction? Is it increasing or staying constant? How safe do citizens feel in their community? Do citizens have confidence in law enforcement? Baseline measures will help the agency in identifying problem areas and will establish the foundation on which performance will be measured. More discussion of baseline measures follows on page 90. c. Government/Industry/Practitioner Standards International Icon Another option is to compare performance of an individual agency or initiative to standards that have been developed by government, industry, or practitioner groups. As previously noted, the Home Office has created a series of police performance standards, and these government standards now serve as a referential baseline for police forces throughout England and Wales.55 Page 61 Chapter 4: Establish a Performance Management Framework While few such standards presently exist for law enforcement in the United States, the need is increasingly recognized in a host of domains. Functional standards for law enforcement information systems are being drafted by a consortium of law enforcement practitioner organizations,56 and additional standards are in development addressing intelligence information sharing, and other critical areas of law enforcement.57 Step 2: Establish an Integrated performance Measurement System Step Two Checklist 1. Have you defined the relationship of performance management to the strategic planning process of the agency? 2. Have you built a strong Performance Management Team? 3. Have you determined stakeholder/customer needs? 4. Do you understand performance measurement terminology? 5. Do you understand the importance of managing performance measurement? 6. Have you established accountability for the measures? (See Chapter 5 for more information.) 7. Have you communicated with all levels of your organization? 8. Do you know how to check/test your measures? 9. Have you looked at other organizations’ efforts and learned from them? 10. Do you know what to measure against? Page 65 ChapTER 5 ESTaBLISh aCCOUNTaBILITY FOR pERFORMaNCE Chapter:5 Establish Accountability for Performance What Effective performance measurement and management requires explicit recognition and acceptance of accountability for actions by the agency and by all participants throughout the organization. Why Accountability reflects ownership and responsibility for actions and acceptance of consequences, and is results-oriented. It demonstrates investment in the process and requires reporting to management, other participants, and external stakeholders. Who Agency leadership, middle managers, line, and support staff. When Accountability should be addressed in the earliest stages of building the Performance Management Framework and in each component. Step 3: Establish accountability for performance Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 4: Establish a process/System for Collecting Data to assess performance Step 2: Establish an Integrated performance Measurement System Step 1: Define Organizational Mission and Strategic performance Objectives Page 66 Part II: Performance Management “Because the police are fundamental state institutions, because they produce much that is publicly valuable, because they use valuable public assets, and because they have the capacity to threaten as well as protect social welfare, it is natural for citizens and taxpayers to demand accountability from them. On one hand, citizens have the right to demand accountability. After all, it is their money and liberty that is being used by public police departments to make the community safe and just. As a matter of principle, then, police departments owe citizens an accounting of the resources they use to operate, and the results they produce. On the other hand, citizens and their representatives might also think it useful to demand accountability from their police departments. The demand for accountability becomes an important instrument for creating the pressures and incentives that lead to improved overall performance and fewer egregious errors in police operations.” 58 —Mark h. Moore The “Bottom Line” of Policing: What Citizens Should Value (and Measure!) in Police Performance (2003) Law enforcement agencies are public institutions—funded and authorized by the community for defined purposes. As a consequence, they have a duty to exercise that authority and expend public resources in responsible and appropriate ways that demonstrate fairness, a commitment to the rule of law and constitutional principles, and a conscientious regard for efficiency and effectiveness. And, as a public institution, they have a further duty to regularly report on their performance. Virtually every introductory text on management discusses fundamental principles of authority, responsibility, and accountability. These are different but related dimensions of management. • Authority refers to the power to influence or command thought, opinion, or behavior. • Responsibility means that one is liable to be called to account as the primary cause, motive, or agent. • Accountability, on the other hand, is an obligation or willingness to accept responsibility and to account for one’s actions. 59 Responsibility has been described as the obligation to perform, whereas accountability is the liability one assumes for ensuring that an obligation to perform—a responsibility—is fulfilled. Put another way, responsibility is the obligation to act in the public interest, whereas accountability is the obligation to answer for intended action Page 67 and, later on, for the results—the obligation to report.60 Effective performance management, therefore, necessarily includes explicitly building accountability into the broad management framework of the organization. Senior leadership, organizational managers, and line staff alike must recognize and exert their legitimate levels of authority to act, take responsibility for their actions, and accept and embrace accountability. In The Performance-Based Management Handbook, Volume 3: Establishing Accountability for Performance (2001), Will Artley has described accountability as having five levels (Figure 5.1).61 Figure 5.1 The Five Levels of Accountability 1. Stakeholder Accountability 2. Organizational Accountability 3. Team Accountability 4. Individual Accounability 5. Personal Accountability Executive Sponsor Icon Who’s Accountable? Stakeholders have the highest level of accountability, which is disconnected from the pyramid in Figure 5.1 to reflect the fact that stakeholders do not have day-to-day responsibility for operational actions within the agency or internal organizational accountabilities. They have provided input in defining the mission and goals of the agency and desired organizational outcomes, and they leave it to the organization to achieve them, and then hold the organization to account for results. Page 68 Part II: Performance Management “Accountability is a multidimensional concept and often a key enabler of success. To truly work, accountability has to be shared by managers and employees; further, your organization as a whole must be accountable to the customer and stakeholder.” —From Balancing Measures: Best Practices in Performance Management, National partnership for Reinventing Government Organizational accountability answers to/reports on what an organization actually accomplished in relation to what it planned to accomplish. There are two types of organizational accountability: 1. Internal Organizational Accountability—Internal organizational accountability refers to the establishment of the upward and downward flow of accountabilities between management and individuals and teams within the organization. 2. External Organizational Accountability—In external organizational accountability, the organization answers to/reports to its stakeholders on both its organizational performance and organizational behavior. Team accountability recognizes that most performance within an agency is accomplished by organizational teams comprising individual members of the agency. Individual team members share ownership of the activity and responsibility for its achievement. Individual accountability refers to an accountability relationship within a work setting. It applies to both parties in the relationship—an authority (management) and a delegatee (the worker). The authority is responsible for providing adequate direction, guidance, and resources as well as removing barriers to performance. The delegatee is responsible for fulfilling its responsibilities. In this relationship, both are accountable to each other, and the focus is on the individual answering and reporting about his or her accomplishments (or lack of accomplishment). Personal accountability is an accountability relationship with oneself. In this relationship, the person looks to himself or herself for personal results and asks, “What can I do to improve the situation and make a difference?” In personal accountability, the individual looks within for answers instead of pointing fingers and placing blame on external factors. Some of the key aspects of personal accountability are honesty, integrity, ethicalness, morality, and reliability. In law enforcement, accountability has been extensively discussed as it relates to CompStat. William Bratton, in his new assignment as chief of the Los Angeles Police Department, again implemented a CompStat process, in which accountability figures prominently: “Simply put, L.A. Compstat uses computer crime-mapping and intensive anti- crime strategy meetings for developing and implementing new tactics, for coordinating patrol, detective, narcotics enforcement, and other specialized unit operations, and for exacting command accountability in addressing crime and disorder problems throughout the City…. L.A. Compstat goes beyond simple measures of police activity. Although statistics, maps, and strategy sessions are Page 69 more visible aspects of L.A. Compstat, performance accountability is the driving force of this crime reduction process.” 62 As Will Artley noted in The Performance-Based Management Handbook, accountability is results-oriented. It focuses not on raw measures of inputs and outputs, but on outcomes—the results of actions. It requires measurement, monitoring, and regular reporting. Moreover, accountability is meaningless without consequences. If chiefs and sheriffs, and indeed their subordinates throughout the agency, do not see consequences for their actions—both positive reward for superior performance and negative consequences for substandard performance—little change can be expected or progress achieved.63 Accountability can be established at the organizational level by creating effective, departmentwide strategic plans, which include careful articulation of agency objectives, strategies for meeting these objectives, and mechanisms and procedures that will be used to measure performance. Section or organizational units in turn create specific performance plans, which refine and further elaborate agency objectives and articulate specific tactical plans to achieve objectives. Building performance agreements into annual or semiannual personnel performance evaluation plans, designed in partnership with management and the staff responsible for carrying out program activities, will help drive accountability down to line staff, and should include specific designation of both rewards for achieving objectives and penalties for failure to do so. Building an Integrated Performance Management Framework requires law enforcement to build accountability into the management and strategic planning process. The agency itself must explicitly recognize its accountability to the community and to external stakeholders; agency leadership, middle managers, line, and support staff must also recognize and accept accountability for their actions. Page 70 Part II: Performance Management Step 3: Establish accountability for performance Step Three Checklist 1. Has agency leadership clearly articulated accountability as a fundamental principle governing management and operations throughout the organization? 2. Is the agency sufficiently results-oriented, i.e., has the agency mission been defined in articulate, actionable, measurable objectives? 3. Have staff and organizational units been given both authority and responsibility to plan, develop, implement, and manage operations? 4. Have sufficient resources been made available to enable successful execution of agency plans? 5. Have agency management and staff reached consensus on strategic performance objectives, performance targets and measures, reporting methods, and consequences for actions? 6. Has the agency developed and articulated practices and mechanisms to demonstrate and enforce accountability, i.e., how specifically will the agency implement and enforce accountability? ChapTER 6 DEvELOp a DaTa COLLECTION pLaN Page 73 Chapter:6 Develop a Data Collection Plan What A comprehensive Data Collection Plan that identifies the information requirements to support performance management, the sources of relevant information, data collection processes, collection and reporting frequencies and costs, and responsibilities. Why Performance assessment requires careful data collection planning to ensure that the specific performance metrics that have been defined are a) based on information and data sources that are relevant to the program, project, or initiative, b) valid and reliable indicators of the dimension being measured, c) subject to quantification, d) reasonably available from an access and cost standpoint, and e) of sufficient quality to support the measurement process. Who Performance Management Team. When Coincident with, and immediately following, development of the Integrated Performance Management Framework; before any data collection takes place. Step 4: Establish a process/ System for Collecting Data to assess performance Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 2: Establish an Integrated performance Measurement System Step 1: Define Organizational Mission and Strategic performance Objectives Step 3: Establish accountability for performance Page 74 Part II: Performance Management Establishing a comprehensive plan for data collection is the logical progression from past activities of setting the agency mission, operationalizing that mission statement into strategic performance objectives, developing and implementing a performance management process, defining specific performance metrics and targets, and establishing responsibility and accountability for performance measurement. We must now take the next step and determine what specific data and information we’re going to collect to give life to these performance metrics. And that means developing a comprehensive plan that addresses these eight factors: 1. Information Requirements. 2. Information Sources. 3. Data Collection Processes. 4. Data Collection and Reporting Frequencies. 5. Data Collection Costs. 6. Data Protection. 7. Data Quality. 8. Trial Run. Although we often use the terms data and information interchangeably, they in fact mean rather different things. • Data generally refers to factual information organized for analysis or to make decisions; numerical information represented in a form suitable for analysis and processing; and values derived from scientific experiments. • Information, in contrast, refers to knowledge derived from study, experience or instruction, or a collection of facts or data.64 Performance management requires that we gather information about the operations of the law enforcement agency; the processes surrounding programs, projects, and initiatives undertaken in order to judge their effective implementation; the impact of these operations on key dimensions related to performance; and changing environmental factors that may influence our evaluation.65 To gain this information, we must first determine what information best reflects the performance objectives, metrics, and targets we are interested in and the best sources of this information. From there we can determine what specific data elements must be captured and a procedure and methodology for collection, reporting, and subsequent analysis. It is important to recognize that data collection, like performance management itself, is not a single, discrete or one-time function. Instead, it is part of an ongoing process that management (and staff alike) must embrace to continually examine, evaluate, monitor, change, and drive performance within an organization. Page 75 Chapter 6: Develop a Data Collection Plan Identify Information Requirements Begin by looking at the information that is required for the performance metrics or targets that have been identified. We don’t begin by looking at information that is available, but rather that which is needed in order to effectively measure the performance dimensions we’re addressing. We don’t want to be constrained by existing sources—we want to “think outside the box” and create measures that most accurately reflect the dimension contemplated by the specific performance metric, and this may require additional data collection (such as public opinion surveys) or access to other data sources. Mark Moore, in Recognizing Value in Policing, for example, identifies a broad range of statistical indicators associated with each of the seven performance dimensions he has proposed, and many come from different data sources (e.g., crime and clearance statistics captured by the law enforcement agency, assessments of community perceptions of fear, budgetary information, citizen complaints, settlements in liability suits, etc.).66 These are shown in Figure 6.1. Figure 6.1 (Chart) Performance Dimensions and Statistical Indicators PERFORMANCE DIMENSION: Reduce criminal victimization STATISTICAL INDICATOR: Report crime rates victimization rates PERFORMANCE DIMENSION: Call offender to account STATISTICAL INDICATOR: Clearance rates Conviction rates PERFORMANCE DIMENSION: Reduce fear and enhance personal security STATISTICAL INDICATOR: Reported change in levels of fear Reported changes in self-defense measures (levels and types) PERFORMANCE DIMENSION Guarantee safety in public spaces STATISTICAL INDICATOR: Traffic fatalities, injuries, and damage Increased utilization of parks and public spaces Increased property values PERFORMANCE DIMENSION Use financial resources fairly, efficiently, and effectively STATISTICAL INDICATOR: Cost per citizen Deployment efficiency/fairness Scheduling efficiency Budget compliance Overtime expenditures Civilianization PERFORMANCE DIMENSION Use force and authority fairly, efficiently, and effectively STATISTICAL INDICATOR: Citizen complaints Settlements in liability suits police shootings PERFORMANCE DIMENSION Satisfy customer demands/achieve legitimacy with those policed STATISTICAL INDICATOR: Satisfaction with police services Response times Citizen perceptions of fairness Page 76 Part II: Performance Management In addition, we may find that we will need to create composite measures for specific performance metrics that combine data from different sources (e.g., a measure of the relative safety of a community may best be expressed as an amalgamation of measures related to the violent crime rate in the community, the nature and rate of quality-of-life crimes, the number of vacant and deteriorating homes in the neighborhood, SES (socio-economic status) indicators, and the results of surveys of members of the community assessing their perception of personal safety). Performance Management Team Icon The Performance Management Team should identify specific information requirements for each of the strategic performance objectives defined in the Performance Management Plan and ensure that representatives of the information technology (IT) section of the agency are actively consulted and engaged in the process (if they are not already members of the team) to ensure that information systems will support the Plan. The template in Figure 6.2 presents a format that the Performance Management Team may use in identifying information requirements for each performance target and measure, including reference not only to the source, but also the intended analysis strategy. Identify Information Sources Information can be captured from a variety of sources to reflect on strategic performance objectives and to build out the specific performance measures identified. Best practice suggests gathering information from multiple sources for each performance objective. Internal agency information sources are perhaps the first place to look, and they may well include incident reports, arrest reports, records from the agency’s automated Records Management System (RMS), Computer-Aided Dispatch (CAD) records (both calls for service and detailed dispatch records showing the time the call was received, the unit assigned and the time of assignment, the time the unit arrived, and the location, nature and disposition of the call), crime analysis reports and maps (which are often derived from incident reports, but may be augmented with additional data, such as the mapping coordinates), or supplementary investigative reports. Stop Sign Icon Pitfalls to Avoid Although these internal sources should make access to the information easier (as it is all within the originating agency), there nevertheless may be turf issues between sections of the department that could potentially hinder immediate access. These issues should be properly addressed through the strategic planning and communication process. Moreover, the quality, timeliness, and/or accessibility of the data may pose yet another challenge to the Performance Management Team (e.g., some agencies may have substantial data-entry backlogs or inadequate data-editing programs, which delay access to current information or compromise the quality of data). The team will need to work closely with agency administrators in addressing Page 77 Chapter 6: Develop a Data Collection Plan Figure 6.2 Information Requirements Template Performance Target: Reduce violent Crime 10% Performance Measure: Number of violent crimes reported and rate per 1,000 population Data Source(s): Incident reports entered into RMS; census data Analysis: Raw number of offenses and rate per/1,000 Page 78 these challenges in order to capture the necessary information in a timely, efficient, and cost-effective manner. Performance Management Team Icon The Performance Management Team, however, should not limit itself to information sources resident within the law enforcement agency. Additional data sources may provide important information in constructing performance measures, including official city records (e.g., land use records, fire and emergency service dispatch records, official economic indicators for the jurisdiction and for individual neighborhoods and communities, motor vehicle records), surveys of the general public, victimization studies, to name a few. As noted above, developing effective performance measures may well require the amalgamation of multiple individual measures into composite metrics assessing law enforcement performance. Development of composite measures is a very complicated task, however, and not something that can be quickly or easily accomplished. Given the inherent complexity of police work, and the multifaceted nature of law enforcement responsibilities, it is unlikely that a single composite measure could validly reflect the broad range of law enforcement performance. More likely, agencies might develop a series of composite measures specifically targeting discrete areas of responsibility. For example, local law enforcement might develop a composite measure to assess the relative safety of a community, building on several independent metrics associated with the volume and nature of crime reported in the community, citizen surveys assessing their perception of personal safety, victimization statistics, and other relevant measures.67 Such composite measures could more accurately reflect the broad dimension of interest, the variety of programs and activities initiated, and the overarching goals and objectives of the agency. Define Data Collection Process Once data sources have been identified, the Performance Management Team should carefully craft the data collection process as part of the Data Collection Plan. The process will define what data will be collected, the source of the data, procedures that must be followed in gaining access and capturing the data, any cleaning or conversion required of the data, and data capture procedures and methodologies. If neighborhood surveys will be required, for example, the plan should provide detailed information regarding the construction, testing, administration, coding, and entry of the survey responses, as well as some discussion of the analyses planned. The plan must also provide a candid assessment of challenges the agency is likely to face in collecting the necessary information and include practical remedies to ensure timely access to the data. Challenges may be related to the nature and/or structure of the data and information systems on which the agency will rely (incident-level data may be available, for example, but it may contain insufficient detail regarding The Federal Government regularly computes a series of economic measures, known collectively as the “Leading Economic Indicators,” as a composite index of the economic status of the nation. Such measures include the Gross Domestic product (GDp), which includes, among other things, measures of consumer spending on goods, shipments of capital equipment other than aircraft, motor vehicle sales and inventories, manufacturing durables inventories, Federal Government outlays, and consumer, producer, and international prices. For more, see http://www.bea. gov/. Page 79 the nature of the offense or how officers responded or its disposition); officer or citizen attitudes regarding the department or the data collection process; and other factors that will make timely data collection difficult or compromise the integrity of information. In Using Analysis for Problem-Solving: A Guidebook for Law Enforcement, author Timothy Bynum referenced work completed by the Police Executive Research Forum (PERF) for the COPS Office. By administering a survey to 447 recipients of COPS Problem- Solving Partnership grants, PERF was able to identify the chief obstacles to police agencies in obtaining and analyzing information needed to solve crime and disorder problems, as shown in Figure 6.3. Figure 6.3 (Chart) Obstacles to Obtaining and Analyzing Data by 447 COPS Grantees68 TYPE OF OBSTACLE / NUMBER OF AGENCIES REPORTING OBSTACLE / (%) Difficulty obtaining data / 126 / (28.2) Difficulty organizing existing data / 108 / (24.2) public apathy / 80 / (17.9) Bureaucratic/funding/technological delays / 57 / (12.8) Other obstacles / 46 / (10.3) public resistance / 39 / (8.7) Officer resistance to problem-solving tactics / 26 / (5.8) Lack of support from mid- and upper-level management / 15 / (3.4) political pressure or interference / 12 / (2.7) Inaccurate, unreliable, or inconsistent data / 12 / (2.7) Conflict with partner / 9 / (2.0) Lack of available information / 9 / (2.0) Need to manually code or recode data / 3 / (0.7) Page 80 Part II: Performance Management Stop Sign Icon You may want to consider doing the cost/benefit review earlier in the process. Statistics show that a critical part of specifying the measures up front is assessing the cost of collecting the measures and doing a cost/benefit review. Since the difficulties of collecting the data are typically underestimated, pilots or prototypes are not inappropriate to estimate the costs and difficulties. Define Data Collection and Reporting Frequency How frequently data are collected will depend on a variety of factors, including availability (e.g., daily reports of crime incidents captured by law enforcement officers and civilian report-takers versus periodic administration of public opinion surveys, which must be carefully planned and administered), costs, processing requirements (e.g., some data may need additional review, coding, or manipulation), reporting requirements established by the agency or external mandates, and the needs of decision makers. CompStat meetings, for example, began as weekly meetings but evolved into two meetings each week and commanders returned every 6 weeks.69 As a consequence, agency staff had to ensure that data was captured, processed, analyzed, and reported according to very strict time constraints. Reporting frequency must also be addressed in completing the Data Collection Plan. Internal reporting schedules and management needs must be addressed (as in the CompStat example above), but external reporting requirements are also important considerations. State and federal funding agencies, for example, require periodic financial and program reports, and these administrative mandates must be incorporated in planning schedules.70 Periodic and regularly scheduled reports on the status of the program and performance findings to employees, management, other stakeholders, and the general public are also important to ensure continuing support, investment and buy-in to the process. Identify Data Collection Costs Costs associated with accessing, collecting, processing, and analyzing data must also be considered and rigorously monitored. As a general rule, the more comprehensive, complex, and detailed the data, the greater the potential costs associated with its collection, processing, and analysis. Capturing data already automated as part of the agency’s routine business practices (e.g., incident data collected and entered daily into the RMS) may not represent significant costs, but delays in entry of the data, or failure to capture all of the information necessary for the analyses contemplated may require alterations in the computer systems, secondary entry of the needed information, or perhaps even parallel processing on a more timely basis. Again, close coordination and collaboration with members of the agency’s IT staff will greatly assist the process of identifying data collection procedures and costs. CompStat, for example, used much of the information already captured by the New York City Police Department in crime incident reports. Because of delays in entry of the information into the agency’s RMS and limitations in the nature, content, and structure of the data automated, however, incident data was independently entered into other CompStat systems constructed for immediate analysis and reporting.71 Page 81 Chapter 6: Develop a Data Collection Plan 8 “An important point is access to data. We assume that the analysts get data, but it is not true in all organizations. Three of the agencies around my jurisdiction refused to participate with us when we sat down with them. The reason they refused is their people didn’t have the data. We assumed that a lot of these agencies are computerized or that data is already compiled. It is simply not. It is on 3-x-5 cards stored somewhere in shoe boxes that somebody is going to have to go through and pull out what they want.” –Bob heimberger73 Additionally, Commissioner Bratton had noted that even mapping crime data in the city represented a financial challenge in the early days of CompStat: “The first maps were handheld, with acetate overlays for each type of crime…. As a reflection of how tight our budget actually was, we could not afford to buy the acetate these maps were printed on. We had to get a grant of $10,000 from the Police Foundation.”72 Surveys of the general public take time and effort in design, development, planning, testing, administering, automation, and analysis, and these costs must be factored into the budget of the Data Collection Plan. In addition, some other sources of data may require expenditure of funds, which must similarly be weighed, as must the costs of data collection, mapping, analysis, and reporting tools. Increasingly, agencies are posting information on their web sites for broad distribution. This is an effective strategy for broad dissemination of information. This strategy, however, also requires technical expertise and support, and should be weighed in developing the Performance Management Plan. Agencies must carefully consider the costs of collecting information and ensure that only that information necessary to address the performance measures defined is collected. Page 82 Part II: Performance Management Protecting critical law enforcement systems and data is addressed in Law Enforcement Tech Guide for Information Technology Security: How to Assess Risk and Establish Effective Policies, published by the COpS Office. Address Data Protection Law enforcement agencies routinely capture a significant amount of very personal data in incident reports—information regarding the identity and address of victims, suspects, complainants, and witnesses; circumstances surrounding an incident; relationships between parties; statements made to officers, and so on. Recognizing the sensitive nature of information to which they have access, and developing rigorous procedures to protect this sensitive data must be of paramount importance to the agency and the Performance Management Team. Privacy and confidentiality of justice information has been the topic of considerable research and policy development in the past 20 years, and agency administrators and the Performance Management Team should be well aware of these developments and tailor their Data Collection Plan accordingly.74 Ensure Data Quality The quality of data collected is an obvious and not entirely incidental factor in developing a robust data collection strategy. Data quality is usually referred to in terms of accuracy, completeness, timeliness, and consistency. Accuracy typically refers to the extent to which data are free from mistake or error, the data conform to reality, and the degree to which the data conform to a standard or true value of the thing being measured. Completeness refers to the extent to which the data (or the record) contain all of the necessary parts or elements. Timeliness refers to the temporal nature of information and its immediate accessibility. Consistency reflects continuity of the measure insofar as it is measured in the same way over time and by different parties.75 The Performance Management Team will want to assess the quality of data as part of its development of the Data Collection Plan to ensure that the information will properly reflect the performance measures created. The team will need to ensure consistent definitions and formats, define proper uses of data, resolve any conflicting information, and develop and enforce appropriate data edit criteria. Complete a Trial Run Once the Data Collection Plan is fully elaborated, it should be tested. Piloting the data collection process will identify problem areas and enable the team to refine procedures to ensure smooth and effective operations. Moreover, the trial run will provide an opportunity to test analysis of the data to ensure that the measures can be collected and analyzed, and that they are appropriate candidates for use in addressing the performance objectives established. Page 83 Chapter 6: Develop a Data Collection Plan Step 4: Establish a process/System for Collecting Data to assess Performance Step Four Checklist 1. Has the agency drafted and tested a comprehensive Data Collection Plan to support performance management? 2. Have information requirements been identified for each strategic performance objective and key performance measure? 3. Have information sources been identified, tested, and agreements (where needed) developed and executed, to ensure that data needed to support performance management are available and accessible? 4. Has the agency clearly described the data collection processes that will be used in gathering, automating, and analyzing data for performance management? 5. Has the agency specified and tested data collection and reporting frequencies to ensure that the correct information can and will be collected in the time frames required to meet performance management requirements? 6. Have costs for data collection and analysis been accurately estimated and does the agency have the funding necessary to implement the plan? 7. How will privacy, security, and data integrity issues be addressed throughout the data collection, analysis, and reporting processes? Have these methodologies been tested to ensure their adequacy? 8. Has data quality been tested to ensure that all necessary sources will enable collection of timely, accurate, and complete information? Where data quality issues emerge, what processes are in place to improve data quality and ensure continuity? 9. Has the Performance Team completed a test run of the full Data Collection Plan, to ensure that all elements will work and the analyses will produce performance measures that meet the requirements of the Integrated Performance Management Framework? Page 87 ChapTER 7 Analyze, Review, and Report Performance Data Chapter:7 Analyze, Review, and Report Performance Data What A rational and thorough process that delineates steps and techniques for objectively calculating, analyzing, reviewing, and reporting measures assessing performance and the impact of programs, projects, and initiatives. Why A structured methodology for analysis and reporting is a necessary step to ensure objective and comprehensive assessment, to provide sufficient and accurate information for effective leadership and decision making, and to foster full transparency and accountability. Who Performance Management Team. When Following development of the Performance Management Framework, Data Collection Plan, and the collection of data. Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 6: Establish a process/System for Using performance Information to Drive Improvement Step 4: Establish a process/System for Collecting Data to assess performance Step 2: Establish an Integrated performance Measurement System Step 1: Define Organizational Mission and Strategic performance Objectives Step 3: Establish accountability for performance Page 88 Part II: Performance Management Use the numbers to help people to improve, not to judge people. Variation is a fact of life. What we want to do with performance measures is to understand the variation. We want to learn how to influence performance with elements of the process that we can control. To interpret what we see, we need to know about variation and how observations vary. —Will artley, et. al., The Performance- Based Management Handbook, Volume 1 (2001) We’ve defined our performance objectives and measures in line with the organization’s mission, created an overarching Performance Management Framework and empowered people to move forward, initiated activities to implement programs and bring about change, and defined what information we’re going to collect, how we’re going to collect it, and how frequently. Our next step is to implement the Data Collection Plan, analyze the information we’ve collected, and report results. Our objective is to answer a series of questions: 1. How does actual performance compare to the goal or target established? 2. If there is a significant difference, why and what corrective action is necessary? 3. Did our programs, initiatives, and activities operate properly and achieve the result intended? 4. Are new goals or measures needed? 5. How have existing conditions changed? Even before we get to these questions, however, we’re going to need a plan to tell us how to analyze the results. How do we aggregate information and score our performance measures? And how do we determine whether our initiatives met the performance targets established? Data Analysis Strategies Analyzing the data gathered in the preceding “Data Collection” stage is one of the most important steps in Performance Management. Analysis should be part of the ongoing and integrated process of measuring and monitoring performance, not an activity simply undertaken at the conclusion of a project. In addition, it should be as simple and direct as possible (though some analytic techniques are, by nature, rather complex). Assess the Quality of Data A useful first step in the analysis of data captured for performance measurement is a quick review of the quality of the data and the collection processes. Calculation of broad measures of data that will be used to evaluate performance should chart the general landscape of the information gathered to globally assess data accuracy, logical inconsistencies, or bias in the collection process; to ensure data comparability; and to validate the relevance of the information for the analyses planned. Initial review of the data will greatly assist the Performance Management Team in assessing the overall quality of the data and the collection processes. This initial review includes: • Computation of descriptive statistics (e.g., measures of central tendency, such as the mean, median, mode, minimum, maximum, and sum) on single data elements or performance measures. Page 89 • Simple cross-tabulations (e.g., showing the number of crimes reported by crime type, by month [or other unit of time] and/or by district, precinct, or other jurisdictional category). From these preliminary analyses, the team should evaluate whether the general measures make sense, i.e., are they generally consistent with historical trends, though, one hopes that they will show improvement or change in the intended direction. Even if no improvement in the performance attribute measured is noted, are the changes recorded logical and conceivable? Extraordinary changes or inconceivably large shifts in performance metrics, either positive or negative, may be artifacts of reporting or data collection errors, bias, or inconsistencies. If one of the performance targets for a project is to reduce quality of life crimes reported in a jurisdiction by 10 percent, for example, and the data show a shift of 95 percent (either more or less), the Performance Management Team should investigate to ensure that the data are indeed accurate and reflect reality. The team should thoroughly investigate any substantive anomalies observed to ensure accuracy in measuring, collecting, automating, and compiling the data, and to assess the operational and community context of the measurement. Stop Sign Icon The range of statistical techniques listed below might be daunting to the average practitioner. Moreover, the complexity of some techniques can become a real impediment to understanding and support by practitioners, even if they are technically appropriate. The key point to remember is that rigorous and professional analysis is needed. Use Analytic Methods Beyond general assessment of the performance measures established, the Performance Management Team will, in all likelihood, use a range of analytic tools in completing its analysis of data. Descriptive statistics measuring attributes of individual performance variables, comparison of statistics pre- and post-program implementation, more formal time-series and interrupted time-series analyses, and complex multivariate statistical tests, correlation coefficients, regression equations, and factor and cluster analysis may be appropriate techniques, depending on the nature of the issues involved and the complexity of the performance measures used. Detailed discussion of these techniques is well beyond the scope of this report, and readers are encouraged to consult authoritative references in determining which specific analytic methods to use. 76 Suffice it to say that the measures used should be appropriate for the phenomena being measured, take into consideration the limitations of the data and the requirements of the statistical tests used, and should also take into consideration the audience for whom reports are being prepared. Page 90 These COpS Office publications provide examples of performance measures: 311 for Non- Emergencies http://www. cops.usdoj. gov/mime/open. pdf?Item=314 Call Management and Community Policing: A Guidebook for Law Enforcement http://www. cops.usdoj. gov/mime/open. pdf?Item=913 Misuse and Abuse of 911 http://www. cops.usdoj. gov/mime/open. pdf?Item=470 Calibrate Baseline Measures A key element in performance assessment is understanding the current status of the agency, the community, or the problem we’re seeking to address and calibrating a baseline of current or past performance. Baseline measures on key performance dimensions identify past performance, the status quo, the current situation, and/or projections for the immediate future, given existing and anticipated circumstances. Crime rates, arrest figures, measures of officer attitude and behavior, surveys of community perception of personal safety, and any number of other issues can serve as effective baseline measures indicative of agency, program, and/or officer performance. Baseline measures, of course, will have already been created in Step 1 of this process (see page 37), where the Performance Management Team and agency leadership have defined performance targets for the agency, program, or initiative. Before we can define performance targets, of course, we must know where we presently stand on identified dimensions and have well-crafted metrics to establish these as current benchmarks. From there, we can identify what and how much we want to improve performance on these key dimensions (our performance targets), and how we intend to achieve these performance targets. Test Hypotheses Analyzing the data and answering the questions posed above is the effective equivalent of hypothesis testing, a fundamental component of rigorous scientific methodology.77 As Valerie J. Easton and John H. McColl note in Statistics Glossary (version 1.1): “Setting up and testing hypotheses is an essential part of statistical inference. In order to formulate such a test, usually some theory has been put forward, either because it is believed to be true or because it is to be used as a basis for argument, but has not been proved, for example, claiming that a new drug is better than the current drug for treatment of the same symptoms.” 78 In law enforcement, our hypotheses may relate to the impact of certain crime reduction strategies on actually reducing the occurrence or seriousness of crimes, the likelihood or prevalence of victimization, or the efficiency of operations. Many jurisdictions around the nation, for example, have implemented 3-1-1 nonemergency call systems to divert nonemergency calls from overloaded 9-1-1 Emergency Call Centers and improve response time for legitimate emergency calls for service, by decreasing the amount of time uniformed officers spend responding to those nonemergency 9-1-1 calls. Page 91 In this 3-1-1 example, the hypotheses that we have constructed are that establishing a 3-1-1 call system will: • Divert calls from overloaded 9-1-1 Emergency Call Centers • Improve response time for legitimate emergency calls for service • Enable other, nonsworn officers or even other local agencies to respond to incidents not requiring police presence per se, and thereby to increase the amount of time uniformed officers spend on police activities • Provide better public service, particularly in emergency incidents.79 The question then becomes, did it work? This, in turn, must be tested by several different measures: • Did the 3-1-1 call system divert calls that otherwise would have been directed to the 9-1-1 Emergency Call Center? • Did response time for legitimate emergency calls for service improve as a result? • Did the 3-1-1 call system function to reduce the number of incidents requiring dispatch of a uniformed officer? • Did the officers perceive value in the 3-1-1 call system (i.e., more time available for patrol and other police activities)? • Did the public perceive value in the 3-1-1 call system (i.e., were they satisfied with the response)? These outcomes can be measured in any number of ways. • Analysis of call volumes (both the number of calls and the type or nature of calls) received by both the 9-1-1 and 3-1-1 call systems • Analysis of dispatch records to determine response time, i.e., a measure of the length of time from when the call was received to when an officer was dispatched, the length of time from when the call was dispatched to when the officer arrived on scene, and the length of time from when the officer arrived on scene to when they cleared the scene and returned to active service. • Analysis of dispatch records to assess how many incidents required dispatching of officers, how many of the dispatches were generated out of 9-1-1 and how many out of 3-1-1 call systems, comparison of the types of calls to which officers were dispatched, and comparison of the outcome of dispatch incidents. • Analysis of the amount of free time and patrol and other police activity time officers had both before and after implementation of the 3-1-1 call system • Analysis of surveys of officer perceptions of the value and operations of the 3-1-1 call system Page 92 • Analysis of surveys of citizens regarding the value of the 3-1-1 call system and satisfaction with the level and quality of service, preferably both pre- and post-3-1-1 call system. Performance Management Team Icon Know What to Measure With baseline performance metrics established, the role of the Performance Management Team is to execute the Data Collection Plan and begin analysis of the data. Analysis, however, is no simple matter. It will, in all likelihood, range from general descriptive statistics (e.g., simple counts and charts displaying the number of crimes reported by crime type and location) to more elaborate pre- and post-program comparisons, time-series, and multivariate statistical analyses.80 Moreover, given the complexity of the issues necessarily involved in performance management and the variety of factors influencing actions and response, jurisdictions will need to measure performance on multiple dimensions. Given the broad range of objectives of the 3-1-1 call systems, to continue our example, we cannot measure performance simply by comparing the number of calls they receive with the number of calls received by 9-1-1 Emergency Call Centers and expect significant diversion. In addition, simply counting the number of calls received by each Call Center will probably not provide sufficient information to accurately assess the achievement of even the first objective, since a number of factors may significantly influence these raw numbers. Public education programs designed to inform citizens about newly established 3-1-1 call systems and describing differences between emergency incidents (which are legitimate candidates for 9-1-1 Call Centers) and nonemergency situations (which the 3-1-1 call systems are designed to address), for example, may nevertheless trigger an increase in calls to both 3-1-1 and 9-1-1 call systems simply as a result of increasing public attention and community outreach.81 Without sufficient understanding of these operational circumstances, the Performance Management Team may be confounded by results and misinterpret findings. Additionally, the nature of the calls to each Call Center may show differences over time, with the 9-1-1 Call Center receiving a larger proportion of their calls as “Priority One” (high priority) emergencies, underscoring again the importance of having detailed baseline statistics. Agencies also will need to measure details of calls received in each of the Call Centers in more detail to assess changes in the nature of requests for service, and must certainly understand the operational and community context surrounding program implementation. How the agency addresses calls for service in each Call Center may also have a significant impact on results achieved (e.g., if the jurisdiction continues to dispatch uniformed officers to both 3-1-1 and 9-1-1 calls, little time savings may actually be realized). Page 93 Chapter 7: Analyze, Review, and Report Performance Data 93 Pre-Intervention Post-Intervention Image of 3-1-1 call system analysis in Baltimore, Maryland, in Managing Citizen Calls to the Police: An Assessment of Non-Emergency Call systems, Lorraine Mazerolle, et. al., 2001) As the analyses of 3-1-1 call systems demonstrate, analysis must take into consideration the operational context of the initiatives. The findings have important tactical and policy implications that must be addressed if success is to be achieved. These are important considerations, as they will greatly assist agency leadership and operational managers in identifying lessons learned and in modifying programs and initiatives to achieve intended results. Moreover, it demonstrates the dynamic nature of performance management, i.e., it is not an abstract research endeavor, but rather a coordinated program to assess and monitor performance to determine what works and what doesn’t work and why, and to effectively craft operations to achieve the best results that are empirically and objectively measured. These examples from a single initiative demonstrate the complicated nature of performance measurement and the importance of developing a clear and coherent strategy for the analysis of data. Understanding the complex nature of the phenomena under study, whether it be crime, officer performance, public opinion, or technology implementation, and developing accurate measures, effective analytic methods, and the ability to enlighten the analysis with a keen awareness of operational context is essential to the performance management process. Page 94 Part II: Performance Management Original Tech Guide Icon Page 159 Data Presentation—How Will the Data Be Used and Reported? The Performance Management Team will want to give careful consideration to how data that is being analyzed will be used and reported—even before it completes its analyses. As Bob Roper and Teri B. Sullivan note in Measuring the Success of Integrated Justice: A Practical Approach (2004), “The key to displaying the results is to convert raw data into useable information. Delivering truckloads of output that is never read accomplishes very little.” 83 Understanding the primary audience(s) for reports, the intended use of the data, and the basic message to be communicated will greatly affect the presentation format (published report, brochure, web site posting, oral presentation, etc.) and distribution channels used. Tips Icon Chart Design: A Few Hints and Tips 84 • Less is more • Group bars to show relationships • Avoid three-dimensional graphics • Use grids in moderation • Choose colors carefully or avoid them altogether • Limit use of typefaces • Choose legible typefaces • Set type against an appropriate background • Use pattern fills with moderation —Source: The Performance-Based Management Handbook, Volume 5 Roper and Sullivan suggest four key principles in converting raw data into useable information: • Convert data from words to pictures and graphs when possible. People respond to visual images—as long as they are simple and intuitive. One state routinely illustrates the progress of a project by using a map, where green counties indicate jurisdictions that have implemented electronic warrants, and yellow counties indicate those where implementation is still in progress. • Use color to highlight the most important points. One state distributes a monthly progress report that lists each county and the percentage of felony court dispositions that are matched to arrests in the criminal history repository. Counties that meet the state standard are coded green; those that are significantly exceeding the current standard are coded blue; those that are making significant progress toward satisfying the state standard are coded Page 95 yellow; and those that need significant help in attaining the goals are coded red. No one wants to be coded red because of the associated public safety implications. • Publish the output regularly. Users become dependent on feedback in order to improve. Information that is out of sight is also out of mind. • Do not overwhelm the audience with too much information—keep results short and simple. A line graph can present a lot of information in a simple format. On a graph that superimposes a trend line and standards on the actual monthly disposition-matching rate, the user can see how the actual disposition rates are changing over time, where those rates are likely to be in 6 months, and how the actual rate compares to the standard. 85 Web Resource Icon In addition to printed reports and brochures (which can be mailed, distributed at agency offices, and made available in community libraries and other city, county, and state offices), information also can be presented on agency web sites. Increasingly, law enforcement agencies are posting a broad range of information on their web sites, including general crime statistics, sex offender registries, neighborhood watch organizations and tips, and crime maps showing the geographic distribution of crime by crime type.86 Indeed, both the New York City Police Department and Los Angeles Police Department post CompStat reports and information on their web sites for public review and consumption.87 In an age of increasingly sophisticated online users, and with public expectations driven by the ever-expanding array of services provided by commercial providers and government organizations, law enforcement and justice agencies would do well to post performance measures on agency web sites for broad dissemination. Create an Executive Dashboard In addition to posting full reports and summary statistics and graphics related to performance measurement on agency web sites, technology also enables the creation of executive dashboards, which typically integrate multiple measures across a variety of dimensions and incorporate graphic displays and use color and images to effectively portray performance in a comprehensive way. The Illinois State Police, for example, are testing development of an executive dashboard that incorporates key performance indicators (KPIs), crime and arrest statistics, automobile accidents, and homeland security alerts. See Figure 7.2. It is instructive to note that this dashboard presents a comprehensive array of information that assesses performance across multiple dimensions and effectively organizes the measures into specific domains. Additionally, the use of color is strategic, designed to draw the reader’s immediate attention to areas of concern and to instantly identify critical trends (red signifying problem areas and green representing positive change or results). The executive dashboard is directed toward the highest level of leadership for the overall organization as a quick look at ongoing performance measurement status. Page 96 Part II: Performance Management Web Resource Icon On-Line analytical processing (OLap) is a category of software technology that enables analysts, managers, and executives to gain insight into data through fast, consistent, interactive access to a wide variety of possible views of information that has been transformed from raw data to reflect the real dimensionality of the enterprise as understood by the user…. OLap helps the user synthesize enterprise information through comparative, personalized viewing, as well as through analysis of historical and projected data in various “whatif” data model scenarios.89 Image of an Sample of a Executive Dashboard chart 90 Building effective business intelligence tools, which are designed to constantly assess performance on multiple dimensions across the breadth of the enterprise, requires integration of critical agency information systems and implementation of online analytical processing (OLAP) technologies that provide real-time analysis and results.88 Ongoing performance monitoring and assessment should not require the agency to mount independent research initiatives in order to constantly address core agency mission and objectives, but rather should be part and parcel of the information technology planning and implementation process that is tightly aligned with the agency’s overarching strategic planning process. In this way, the Performance Management Framework becomes ingrained in the management culture of the organization, which in turn ensures that it will be effectively institutionalized. Moreover, by building business intelligence and performance measurement as embedded elements of the agency’s broader management and IT planning process, the quality and timeliness of information is likely to improve, given the operational investment in core agency information systems. Page 97 Chapter 7: Analyze, Review, and Report Performance Data 9 Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step Five Checklist 1. Has the agency developed a comprehensive data analysis and reporting strategy? 2. Have analytic techniques been identified and tested to ensure the adequacy of the data, the technical sufficiency of the measures, and the relevance of techniques to the performance measures used? 3. Have agency/jurisdiction baseline measures been established and validated in each key performance domain? 4. Are agency actions sufficiently related to strategic objectives and performance targets to enable development of hypotheses that can be tested and validated? 5. Are performance measures well defined, articulate, and subject to objective calculation? 6. Are analysis and reporting techniques and methodologies well understood, clearly defined, and appropriate? Has the agency stratified analysis techniques and reporting methodologies to address the variety of audiences to whom they must report, and to ensure that the needs of each audience are uniquely addressed? 7. Has the agency developed and tested a plan that will ensure ongoing reporting of performance measures? ChapTER 8 USE pERFORMaNCE INFORMaTION TO DRIvE IMpROvEMENT Page 101 Chapter 8: Use Performance Information to Drive Improvement What Establish a formal process to institutionalize performance management, to ingrain it in agency culture, and to drive improvement. Why Performance management is not a one-time, project-driven assessment of success, but rather is part of an integrated management paradigm to relentlessly pursue excellence, aggressively adopt best practices, and constantly measure and achieve efficiency and effectiveness. Who Agency leadership, with the investment and support of mid-level managers, line staff, external stakeholders, and consumers/customers. When Planning should occur throughout Steps 1 through 5 and implementation should coincide with completion and publication of initial performance measurement reports, continuing and evolving as the agency and the Performance Management Framework matures. Step 6: Establish a process/System to Use performance Information to Drive Improvement Step 5: Establish a process/System for analyzing, Reviewing, and Reporting performance Data Step 4: Establish a process/System for Collecting Data to assess performance Step 2: Establish an Integrated performance Measurement System Step 1: Define Organizational Mission and Strategic performance Objectives Step 3: Establish accountability for performance Page 102 Part II: Performance Management As noted earlier in this Guide, performance management is about more than simply evaluating the effectiveness of individual projects or initiatives within law enforcement agencies. It is fundamentally about changing the culture of the organization, and continuously driving performance improvement. To effectively do so, the performance management process must become institutionalized; it must be ingrained within the culture of the organization, and mechanisms must be put in place to ensure continuous operation and evolution to meet the evolving needs of the agency and the community within which it operates. Shape Organizational Culture Institutionalizing performance management requires agencies to create an organizational culture that values performance measurement and management, and provides ways for everyone—from the chief to the officer on the street, from the commander to the records clerk and the dispatcher—to participate. Aligning operations to meet broad agency vision and mission statements, and defining performance measures and targets requires the operational input of practitioners at every level of the organization. When William Bratton assumed his role as police commissioner of New York City in 1994, he found an agency culture that was concerned more with keeping out of trouble than focusing on reducing crime. A survey of nearly 8,000 police officers and commanders found the following: • “At the highest levels of the organization, the basic aim of the NYPD was not to bring down crime but to avoid criticism from the media, politicians, and the public…. • The greater the distance from headquarters, the lesser the trust from one rank to the next…. • Police officers believed the department had not backed them up, even when their actions were warranted. • The department was structured to protect its good name (and the careers of its senior executives) rather than to achieve crime-fighting goals. • The Internal Affairs Bureau was seen as intent on tripping up officers for minor infractions rather than rooting out real corruption. • The mayor and [Bratton’s] strongly voiced support for the department had encouraged people throughout the organization, but they were waiting to see what we should do.” 91 Stop Sign Icon Broad investment, support, and involvement by staff throughout the organization, together with key stakeholders and customers (in the case of law enforcement, the general public) is essential to establish and maintain credibility, and also ensures that the performance management framework is solidly grounded in reality and meets the Page 103 primary needs of the users. The chief or sheriff, command staff, and middle managers must recognize the value of performance management and accept accountability, as must line and support staff. Everyone in the agency must recognize value in the program and see change and legitimate response as a result, though this may be difficult during initiation. The program is bound for failure, however, if agency administrators institute performance management, collect and analyze data, and either fail to heed or “fudge” the results, refuse to hold persons or units accountable, or simply lose momentum and do not follow up. Moreover, building performance measurement into annual or semiannual personnel performance appraisals will underscore the importance and legitimacy of the overall effort, further embed the notion of individual accountability, demonstrate management commitment, and establish a foundation for the Performance Management Framework. Make Information Broadly Available The results of performance measurement should be broadly disseminated throughout the agency and beyond. Commanders, middle management, and line and support staff should be actively engaged in the performance management process, from planning and development through analysis and reporting. Results should be shared, as appropriate, to ensure continuous investment and validation of the program. Additionally, broad dissemination of findings will reinforce the agency’s relentless pursuit of performance improvement and help underscore the focus on agency, unit, and program objectives, and reinforce tight alignment with the organizational mission. Transparency is a critical factor in performance management, and the agency must legitimately be seen as open, honest, aboveboard, and accountable. Management’s response to performance measurement findings will be closely watched. Does management actively review and participate in the performance measurement process? Does management respond to findings of the research and make changes in operations and/or processes? If performance results do not drive action, little value will be realized in the broad performance management program. Additionally, routinely sharing information with external stakeholders and customers is an important step in building trust and demonstrating transparency. Demonstrating quick wins early in the process and throughout can indicate immediate application of performance measurement results. Tips Icon Reengineer Business Processes Performance measurement affords the agency an opportunity to closely examine business processes and identify duplication in data entry, redundant processing, and circuitous business processes that are evidence of the piecemeal automation practices endemic in many jurisdictions. Careful planning and attention to detail in research can reveal fundamental flaws in agency practices that can be corrected with Page 104 appropriate action and reengineering. Too often, agencies have simply automated a traditional process, rather than critically examining the dynamics of agency operations and adopting solutions that incorporate reengineering of business processes. Performance management is effective only if it leads to real, tangible, and quantifiable change. Measuring how an agency, a division, department or team, specific projects, or individual personnel perform is an important element in managing for success, but measurement alone is insufficient to achieve change or lasting results. The results of performance measurement must not only be accurately and properly tabulated and broadly shared, but they must also translate into action, and this is often the most difficult part of the equation. Change is often difficult for people, and organizational change is even more challenging in government agencies and particularly in paramilitary units like law enforcement with long established and well-structured chains of command. In addition, the organizational culture plays a significant role in shaping how the agency responds to change. Breaking down those cultural barriers can be one of the greatest challenges agency administrators face in implementing lasting, consequential change. The commitment to change must come from the top of the organization, and that commitment must be based on the results of performance measurement that clearly articulate the business impact of historical and current operations, and remedial actions or other recommendations that are guided by the results of the research. But to be effective, change must inevitably involve all levels of the organization, i.e., command staff, mid-level managers, and line staff throughout the agency must be actively involved and invested in the performance measurement process and in proposing and implementing new business practices. In William Bratton’s earliest days at the New York City Police Department, for example, he involved more than 300 people from all ranks throughout the Department, organized into 12 reengineering teams addressing “productivity, discipline, in-service training, supervisory training, precinct organization, building community partnerships, geographical and functional organizational structure, paperwork, rewards and career paths, equipment and uniforms, technology and integrity.” 92 He indicates that the teams made more than 600 recommendations, 80 percent of which were accepted. In a very real way he reengineered the entire agency using internal staff that had intimate knowledge of the operations, an investment in the success of the agency, and commitment to execute the changes proposed, which they themselves were instrumental in formulating. Page 105 Chapter 8: Use Performance Information to Drive Improvement 0 Bratton’s success in New York is due in no small measure to the management model he implemented there. This model included strong commitment from the top levels of the department; clear vision and a well-articulated mission for the agency; strategic alignment of departmental operations with the mission defined; detailed and continuous measurement; accountability; and reengineering of critical operations throughout the entire enterprise. The broad goal of the Performance Management Framework described here is to build a comprehensive management paradigm that will support objective appraisal and monitoring of performance and appropriate action to drive improvement based on results. Just as change for the sake of change, without accurate performance measurement, is inefficient and generally ineffective, performance measurement without substantive action is little more than research without business consequence. Step 6: Establish a process/System for Using performance Information to Drive Improvement Step Six Checklist 1. Has agency leadership created a culture within the organization that supports performance measurement, accountability, and action-orientation? 2. Is information regarding performance broadly available to users, managers, stakeholders, and other interested parties? 3. Do managers and staff “buy in” to the performance management framework? Do they support the overall effort and recognize the value and validity of the approach and its implementation? 4. Does the performance management framework provide opportunity and encourage reengineering of operations to achieve performance targets, and greater efficiency and effectiveness? Part III: Performance Management in Everyday Policing “What gets measured, gets done.” —Peter Drucker ChapTER 9 BUILD pERFORMaNCE MaNaGEMENT INTO EvERYDaY pOLICING Page 111 Chapter 9: Build Performance Management into Everyday Policing Police chiefs and sheriffs, indeed officials in every form of government organization directly or indirectly part of the justice enterprise, are public servants. As such, they have an affirmative duty to manage effectively, demonstrate efficient and responsible use of public resources, exercise the powers the public has vested in their agencies in fair and transparent ways, and regularly report to the public on the quality of performance of their agencies and staff. The Integrated Performance Management Framework outlined in this Guide provides a comprehensive and structured methodology for incorporating performance measurement into the broader management paradigm for law enforcement and justice organizations. Individual program evaluation is a necessary component of performance management, but discrete efforts to evaluate the relative effectiveness of individual programs, projects, or initiatives, while valuable, do not provide the required organizational foundation contemplated in performance management. Executive Sponsor Icon Executive Support and Organizational Commitment Required Effective performance management requires vision, investment, insight, and commitment from the highest levels of the organization. The chief or sheriff (or other justice agency executive officer) must be able to clearly articulate the vision and goals of the agency, and together with his or her management team, be able to translate those goals into tangible, measurable, actionable objectives that can drive the performance of everyone within the agency, and other external affiliated participants (e.g., other government agencies, stakeholders, and justice partners). Agency executives must also be willing to make the necessary investment in staff and technology to effectively implement the performance management framework. Too often management agrees to make fundamental change or initiate a major new program, only to undercut its efforts by failing to properly fund the initiative or to recognize the staff and/or technology commitment required to ensure success. It is better to hold off on implementation until management has made a commitment for adequate funding, staff, and technology resources than to implement performance Page 112 management in half measures. That doesn’t mean that performance management must languish unless and until the agency secures a financial windfall of either local or federal funding, but simply that the program will not likely succeed unless adequate resources are firmly committed, either through new funding opportunities or internal reinvestment or reallocation. It may also mean augmenting existing staff with external competencies in research methodology, statistical analysis, geographic mapping, and technological expertise. Sworn law enforcement officers may lack specific skills in conducting statistical analyses, administering community surveys, or building information technology solutions necessary for sophisticated performance measurement. The agency, therefore, should be willing to reach out to other government, university, or community resources to acquire needed skills. Comprehensive System of Accountability and Responsibility Required Performance management also requires a significant cultural investment by the agency. Measuring, monitoring, and managing performance should become deeply embedded within the culture of the organization. Establishing a foundation of accountability at all levels of the organization and committing to transparency in process and operations are necessary elements in building that culture. Making an agency results-oriented can be accomplished only by defining objectives in measurable terms, empowering those charged with achieving results with the necessary responsibility and resources to do so, measuring and monitoring the outcome and impact of activities, and holding responsible persons accountable for achieving desired results. This doesn’t mean that failure to achieve positive results will inevitably result in punitive action. Rather, a system of accountability will provide evidence that activities initiated were guided by knowledge, insight, research, best practice, and conscientious action. Additionally, the entire performance management process must be transparent—obvious, aboveboard, devoid of hidden agendas, clearly defined, and objectively applied. Flatten the Organization Performance management flattens the law enforcement organization in important respects. To be effective, performance measures should be calculated on current operations as a baseline and to identify trouble spots, and this information should be broadly communicated to all interested and involved parties. Understanding and communicating current performance metrics is essential in motivating participants to action, identifying problems in objective and measurable terms, and assisting in the development of appropriate responses. In addition to broadly sharing this information, management and staff should be empowered to assist in the formulation and implementation of effective and innovative solutions and in monitoring their performance. This is particularly important, given their individual Page 113 and collective accountability. Moreover, routine communication of performance and direct involvement in the performance management process furthers investment by all participants in the Performance Management Framework. Beyond these structural components regarding organizational alignment, performance management is also a critical element in effectively managing operations. Identify Problems Early Building an effective performance measurement system, which constantly monitors performance on a variety of factors, is an extraordinarily useful tool in identifying problems that emerge in very early stages of development. Rather than being reactive and responding to catastrophic events and implementing major course corrections, a proactive, results-oriented agency can effectively identify problems in the early stages of emergence by monitoring performance continuously. Identifying critical performance measures and tightly aligning performance monitoring solutions with agency information systems provides agencies with the opportunity to identify emerging problems before they fully manifest themselves. In addition to identifying the critical performance measures, agencies should implement performance dashboards that tightly integrate with operational information systems, and embed management mechanisms with the ability to trigger alerts when performance metrics exceed narrow performance parameters. Spikes in the amount of reported crime, significant shifts in key dimensions (e.g., locations, times of day, victim characteristics, and arrest figures), and other attributes might well be incorporated into these online solutions similar to supply chain management systems in the private sector, which can identify deviations in predicted commercial activity and respond immediately, adjusting inventory to better meet the needs of consumers. Performance management is about more than effectively responding to problems as they arise; it’s also about anticipating problems and effectively responding early. Ensure Progress and Keep Projects on Target Performance management can also be an effective tool in ensuring progress and keeping projects and activities on track. The FBI, for example, recently adopted a Life Cycle Management Directive governing information technology (IT) projects. As noted in a June 2005 Information Week article, “The goal is to centralize IT assessment, using seven ‘gates,’ or review points that serve as the mechanism for management control and direction, decisionmaking, coordination, and confirmation of successful performance. ‘Do the money and the time match? Are we at our performance goals? You have a measuring tool now,’ [FBI Chief Information Officer Zalmai] Asmi says. ‘Anytime it deviates 10 percent or more, that’s when we bring in project management.’ The bureau is evaluating the health of all 479 of its IT projects in order to provide data that will help FBI managers understand the costs, schedules, and risks associated with each project.”93 Page 114 Part III: Performance Management in Everyday Policing “The concept of performance measurement is straightforward: You get what you measure, and you can’t manage a project unless you measure it.” —From Performance- Based Management, General Services administration. “The law enforcement profession should recognize the need for better analytic capabilities in the policing profession. To help guide the movement toward intelligence-led policing, leaders of state and local agencies in the United States should examine models from other countries as well as identify best practices nationwide… “Issues related to standardizing data systems, documenting criteria, and implementing and evaluating the standards must be resolved. There are more than 18,000 state, local, and tribal law enforcement agencies in the United States, and in order to guarantee the success of intelligence-led policing initiatives, greater agreement on these issues must be reached.”94 —Stephan a. Loyka, et. al., Protecting Your Community from Terrorism, Volume 4 (2005) Implementation of the Life Cycle Management program at the FBI required more than building performance measurement into the agency—it also required significant cultural changes within the institution, structural changes to empower the CIO with significantly broader control over IT decisionmaking throughout the agency, and development of performance benchmarks for each of its 479 IT projects.95 Monitoring program and agency performance in real-time provides an opportunity to implement minor course corrections as problems emerge, rather than waiting for catastrophic failure and systemic incidents that could be predicted by paying adequate attention to early warning indicators. It means, of course, insightful analysis of factors related to success as well as failure, and tightly aligning performance measurement and monitoring capabilities with operational systems. Demonstrate Value Effective performance management is also a crucial element in measuring and monitoring the value of law enforcement and justice operations, and in supporting community policing, problem-oriented policing, and intelligence-led policing endeavors. By measuring the effectiveness of specific initiatives, and monitoring the pulse of the community and crime problems, performance measurement facilitates the development of best practices throughout the law enforcement domain. But even beyond law enforcement, performance management is crucial for information sharing and integration projects.96 Performance management helps to build an enterprisewide ability to demonstrate the tangible business value of sharing information, not only in meeting individual agency goals, but also in meeting justice enterprise goals: • Reduce crime, and prevent and respond to national security threats, natural disasters, and large-scale criminal incidents Page 115 • Improve the quality and equality of justice • Improve efficiency, effectiveness, and return on investment. Part III Checklist 1. Do agency leaders “buy in” to the Performance Management Framework? Are they committed and invested in planning and implementation, and do they accept accountability, performance measurement, and open reporting? 2. Has agency leadership provided the necessary staff, funding, and technology support to ensure successful implementation? 3. Are management and staff throughout the organization empowered to act and to innovate? 4. Do managers and staff have broad access to information to assess current levels of performance and to evaluate the impact of programs and initiatives on achieving objectives? 5. Does the performance management framework enable early identification of problems and ongoing assessment as part of a continuous business improvement strategy? 6. Are mechanisms in place to enable course correction when it is determined that projects are off target and/or failing to meet interim milestones? 7. Does the performance management framework demonstrate value of operations within the agency, and in broader terms across the justice enterprise? Page 116 Conclusion This Tech Guide has focused on performance measurement, particularly as it relates to law enforcement operations and information technology planning and implementation. Rather than proposing a narrowly constrained program evaluation methodology that agencies might use in assessing the achievement of objectives for a single project or initiative, we have instead elaborated an integrated performance management framework that can be incorporated within the organizational and management structure of law enforcement organizations. The principles and methodologies described in this Tech Guide have direct application in evaluating the success and the performance of individual projects, but the performance management framework suggested provides an operational and strategic foundation for effective planning, management, and operations throughout the entire agency. Performance management, as this Tech Guide has discussed, operates at many levels throughout the organization. Defining strategic objectives of the agency, and building operational and tactical plans for implementation requires a comprehensive understanding of current operations, and accurate and precise measures of key performance indicators. Planning and implementing projects that are tightly aligned to the strategic objectives of the agency require careful and continuous monitoring and measurement to ensure efficient operations, effective implementation, and adequate return on the investment of time and resources needed. The performance management planning that you have completed in working through the steps outlined in this Tech Guide will ensure that your agency is able to develop valid and reliable performance measures and incorporate these into your everyday operations. Appendix A: Bibliography Page 119 Appendix A: Bibliography Alpert, Geoffrey P., and Mark H. Moore, “Measuring Police Performance in the New Paradigm of Policing,” in Performance Measures for the Criminal Justice System, NCJ 143505 (Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics, 1993). Artley, Will, D.J. Ellison, and Bill Kennedy, The Performance-Based Management Handbook, Volume 1: Establishing and Maintaining a Performance-Based Management Program (Washington, D.C.: U.S. Department of Energy, 2001). Artley, Will, The Performance-Based Management Handbook, Volume 3: Establishing Accountability for Performance (Washington, D.C.: U.S. Department of Energy, 2001). Association of State Correctional Administrators (ASCA), Key Indicators of ASCA Performance Standards (with Counting Rules) (Middletown, CT: ASCA, 2005). ASCA, Performance-Based Measures Resource Manual (Middletown, CT: ASCA, 2005). Bayley, David H., “Measuring Overall Effectiveness,” in Lawrence T. Hoover (ed.), Quantifying Quality in Policing (Washington, D.C.: Police Executive Research Forum, 1996). Beschen, Darrell, Richard Day, Gretchen Jordan, and Howard Rohm, The Performance-Based Management Handbook, Volume 4: Collecting Data to Assess Performance (Washington, D.C.: U.S. Department of Energy, 2001). Blalock, Jr., Hubert M., Social Statistics (New York: McGraw-Hill, 2nd Edition, 1979). Boba, Rachel, Guidelines to Implement and Evaluate Crime Analysis and Mapping in Law Enforcement (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services (COPS), 2000), at http://www.cops.usdoj.gov/mime/open.pdf?item=614. Boba, Rachel, Problem Analysis in Policing (Washington, D.C.: Police Foundation, 2003), at http://www.cops.usdoj.gov/mime/open.pdf?Item=847. Bratton, William J., “Great Expectations: How Higher Expectations for Police Departments Can Lead to a Decrease in Crime,” in Robert H. Langworthy (ed.), Measuring What Matters: Proceedings from the Policing Research Institute Meetings, NCJ 170610 (Washington, D.C.: U.S. Department of Justice, National Institute of Justice and Office of Community Oriented Policing Services, July 1999). Page 120 Appendix A Bratton, William, with Peter Knobler, Turnaround: How America’s Top Cop Reversed the Crime Epidemic (New York: Random House, 1998). Bureau of Justice Statistics, Performance Measures for the Criminal Justice System, NCJ-143505 (Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics, 1993). Bynum, Timothy S., Using Analysis for Problem-Solving: A Guidebook for Law Enforcement (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2001), at http://www.cops.usdoj.gov/mime/open.pdf?Item=342. Center for Society, Law and Justice, A Catalog of Performance Measures for Justice IT Projects: An Accountability Model (Metairie, LA: University of New Orleans, no date), at http://www.cslj.net/PerformanceMeasure/files/accountabilityinventory.pdf. Center for Society, Law and Justice, The Logic Model Approach to Developing Performance Measures for Justice Integration and Information Sharing Projects (Metairie, LA: University of New Orleans, no date), at http://www.cslj.net/PerformanceMeasure/files/ userguidetologicmodelapporach.pdf. Clarke, Ronald V., and Herman Goldstein, Reducing Theft at Construction Sites: Lessons From a Problem-Oriented Project (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2003), at http://www.cops.usdoj.gov/mime/open.pdf?Item=804. Dadds, Vikki, and Tammy Scheide, “Police Performance and Activity Measurement,” Trends and Issues in Crime and Criminal Justice, No. 180 (Cauberra: Australian Institute of Criminology, November 2000), at http://www.aic.gov.au/publications/tandi/tandi180.html. Eck, John E., Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2001), at http://www.popcenter.org/Tools/tool-assessing.htm. Federal Bureau of Investigation, Crime in the United States 2003 (Washington, D.C.: FBI, 2004). Fridell, Lorie, By the Numbers: A Guide for Analyzing Race Data from Vehicle Stops (Washington, D.C.: Police Executive Research Forum, 2004), at http://www.cops.usdoj.gov/default.asp?Item=1476. Gee, Diana, Kirkland Jones, Doug Kreitz, Sergio Nevell, Bill O’Connor, and Bob Van Ness, The Performance-Based Management Handbook, Volume 6: Using Performance Information to Drive Improvement (Washington, D.C.: U.S. Department of Energy, 2001). Geerken, Michael, Center for Society, Law and Justice, Performance Measurement Tools for Justice Information Technology Projects: Key Issues in Developing Performance Measurement Systems for Justice Page 121 Bibliography Integration (New Orleans, LA: University of New Orleans, draft, no date), at http://www.cslj. net/PerformanceMeasure/files/performancemeasurementmonograph.pdf. General Accounting Office, Homeland Security: Challenges in Achieving Interoperable Communications for First Responders, GAO-04-231T (Washington, D.C.: GAO, 2003). Glass, Gene V., and Julian C. Stanley, Statistical Methods in Education and Psychology (New Jersey: Prentice Hall, 2nd Edition, 1970). Greenemeir, Larry, “Tech vs. Terrorism,” in Information Week, June 6, 2005. Harris, Kelly J., and William H. Romesburg, Law Enforcement Tech Guide: How to plan, purchase and manage technology (successfully!), A Guide for Executives, Managers and Technologists (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2002). Henry, Vincent E., The COMPSTAT Paradigm: Management Accountability in Policing, Business and the Public Sector (Flushing, NY: Looseleaf Law Publications, Inc., 2002). Jordan, Gretchen, Steven Prevette, and Steve Woodward, The Performance-Based Management Handbook, Volume 5: Analyzing, Reviewing, and Reporting Performance Data (Washington, D.C.: U.S. Department of Energy, 2001). Loyka, Stephan A., Donald A. Faggiani, and Clifford Karchmer, Protecting Your Community from Terrorism, Volume 4: The Production and Sharing of Intelligence (Washington, D.C.: Police Executive Research Forum, February 2005), at http://www.cops.usdoj.gov/mime/open.pdf?Item=1438. Mazerolle, Lorraine, Dennis Rogan, James Frank, Christine Famega, and John E. Eck, Managing Citizen Calls to the Police: An Assessment of Non-Emergency Call Systems, NCJ 199060, revised final report to the U.S. Department of Justice, National Institute of Justice, October 2001, at http://www.ncjrs.org/pdffiles1/nij/grants/199060.pdf. Mazerolle, Lorraine, Dennis Rogan, James Frank, Christine Famega, and John E. Eck, Managing Citizen Calls to the Police With 911/311 Systems, NIJ Research for Practice, NCJ 206256 (Washington, D.C.: U.S. Department of Justice, National Institute of Justice, February 2005), at http://www.ncjrs.org/pdffiles1/nij/206256.pdf. Mazerolle, Lorraine, Dennis Rogan, James Frank, Christine Famega, and John E. Eck, Calling 311: Guidelines for Policy-makers, NIJ Research for Policy, NCJ 206257 (Washington, D.C.: U.S. Department of Justice, National Institute of Justice, February 2005), at http://www.ncjrs.org/pdffiles1/nij/206257.pdf. Moore, Mark H., and Anthony A. Braga, “Measuring and Improving Police Performance: The Page 122 Appendix A Lessons of CompStat and its Progeny,” in Policing: An International Journal of Police Strategies & Management, Volume 26, No. 3, 2003. Moore, Mark H., (with Anthony Braga), The “Bottom Line” of Policing: What Citizens Should Value (and Measure!) in Police Performance (Washington, D.C.: Police Executive Research Forum, 2003). Moore, Mark H., David Thacher, Andrea Dodge and Tobias Moore, Recognizing Value in Policing: The Challenge of Measuring Police Performance (Washington, D.C.: Police Executive Research Forum, 2002). Office of Community Oriented Policing Services, Grant Monitoring and Standards Guidelines for Hiring and Redeployment (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2004), at http://www.cops.usdoj.gov/default.asp?Item=299. Roberts, David J., Concept for Operations for Integrated Justice Information Sharing (Lexington, KY: NASCIO, 2003). Roper, Bob, and Teri B. Sullivan, “Measuring the Success of Integrated Justice: A Practical Approach” in Information Systems Integration: A Library of SEARCH Resources for Justice and Public Safety Practitioners (Sacramento, CA: SEARCH Group, Inc., 2004). Shewhart, Walter A., Statistical Method from the Viewpoint of Quality Control (Washington, D.C.: Department of Agriculture, 1939 (reprinted by Dover Publications, 1986)). Skogan, Wesley G., “Measuring What Matters: Crime, Disorder, and Fear” in Measuring What Matters: Proceedings from the Policing Research Institute Meetings, Robert H. Langworthy (ed.), NCJ 170610 (Washington, D.C.: U.S. Department of Justice, National Institute of Justice and the Office of Community Oriented Policing Services, July 1999). Solomon, Shellie E., and Craig D. Uchida, Building A 3-1-1 System For Police Non-Emergency Calls: A Process and Impact Evaluation (Washington, D.C.: 21st Century Solutions, Inc., 2003). Tufte, Edward R., The Visual Display of Quantitative Information (New York: Graphics Press, 2001). Washington, D.C., Metropolitan Police Department, Metropolitan Police Department Strategic Business Plan FY 2004-2005 (Washington, D.C., August 2004), and Metropolitan Police Department Strategic Business Plan FY 2006–2007 (Washington, D.C., October 2005), available at http://mpdc.dc.gov/mpdc/cwp/view,a,1246,q,558981.asp. World Bank, Measuring the Performance of a Legal System, (no date), available at http://www.worldbank.org/publicsector/legal/performancebrief.htm. Wright, Kevin, with Judy Bisbee and Patricia L. Hardyman, Defining and Measuring Corrections Performance, Final Report (Middletown, CT: ASCA, 2003). Appendix B: Glossary Page 125 Appendix B: Glossary Source: Beschen, Darrell, Richard Day, Gretchen Jordan and Howard Rohm, The Performance- Based Management Handbook, Volume 4: Collecting Data to Assess Performance (Washington, D.C.: U.S. Department of Energy, 2001) at Appendix A. Accountability The obligation a person, group, or organization assumes for the execution of assigned authority and/or the fulfillment of delegated responsibility. This obligation includes: answering—providing an explanation or justification—for the execution of that authority and/or fulfillment of that responsibility; reporting on the results of that execution and/or fulfillment; and assuming liability for those results. Activity Actions taken by a program or an organization to achieve its objectives. Assessment An all-inclusive term used to denote the act of determining, through a review of objective evidence and witnessing the performance of activities, whether items, processes, or services meet specified requirements. Assessments are conducted through implementation of activities such as audits, performance evaluations, management system reviews, peer reviews, or surveillances, which are planned and documented by trained and qualified personnel. Baseline The initial level of performance at which an organization, process, or function is operating upon which future performance will be measured. Benchmarking 1) To measure an organization’s products or services against the best existing products or services of the same type. The benchmark defines the 100 percent mark on the measurement scale. 2) The process of comparing and measuring an organization’s own performance on a particular process against the performance of organizations judged to be the best of a comparable industry. Page 126 Appendix B Characteristics Any property or attribute of an item, process, or service that is distinct, describable, and measurable. Continuous Improvement 1) The undying betterment of a process based on constant measurement and analysis of results produced by the process and use of that analysis to modify the process. 2) Where performance gains achieved are maintained and early identification of deteriorating environmental, safety, and health conditions is accomplished. Corrective Action Actions taken to rectify conditions adverse to quality and, where necessary, to preclude repetition. Criteria The rules or tests against which the quality of performance can be measured. Goal 1) The result that a program or organization aims to accomplish. 2) A statement of attainment/achievement, which is proposed to be accomplished or attained with an implication of sustained effort and energy. Guideline A suggested practice that is not mandatory in programs intended to comply with a standard. The word “should” or “may” denotes a guideline; the word “shall” or “must” denotes a requirement. Impact Characterization of the outcome of a program as it relates to specific objectives. Item An all-inclusive term used in place of the following: appurtenance, sample, assembly, component, equipment, material, module, part, structure, subassembly, subsystem, unit, documented concepts, or data. Lessons Learned A “good work practice” or innovative approach that is captured and shared to promote repeat application. A lesson learned may also be an adverse work practice or experience that is captured and shared to avoid recurrence. Management All individuals directly responsible and accountable for planning, implementing, and assessing work activities. Page 127 Glossary 2 Measurement The quantitative parameter used to ascertain the degree of performance. Metric A standard or unit of measure. Objective A statement of the desired result to be achieved within a specified time. Occurrence An unusual or unplanned event having programmatic significance such that it adversely affects or potentially affects the performance, reliability, or safety of a facility. Outcome The expected, desired, or actual result to which outputs of activities of an agency have an intended effect. Outcome Measure An assessment of the results of a program activity or effort compared to its intended purpose. Output A product or service produced by a program or process and delivered to customers (whether internal or external). Output Measure The tabulation, calculation, or recording of activity or effort and can be expressed in a quantitative or qualitative manner. Performance-Based Management A systematic approach to performance improvement through an ongoing process of establishing strategic performance objectives; measuring performance; collecting, analyzing, reviewing, and reporting performance data; and using that data to drive performance improvement. Performance Indicator(s) 1) A particular value or characteristic used to measure output or outcome. 2) A parameter useful for determining the degree to which an organization has achieved its goals. 3) A quantifiable expression used to observe and track the status of a process. 4) The operational information that is indicative of the performance or condition of a facility, group of facilities, or site. Page 128 Appendix B Performance Measure A quantitative or qualitative characterization of performance. Performance Measurement The process of measuring the performance of an organization, a program, a function, or a process. Performance Objective 1) A statement of desired outcome(s) for an organization or activity. 2) A target level of performance expressed as a tangible, measurable objective, against which actual achievement shall be compared, including a goal expressed as a quantitative standard, value, or rate. Performance Result The actual condition of performance level for each measure. Process An ongoing, recurring, and systematic series of actions or operations whereby an input is transformed into a desired product (or output). Program Evaluation An assessment, through objective measurement and systematic analysis, of the manner and extent to which federal programs achieve intended objectives. Quality A degree to which a product or service meets customer requirements and expectations. Reengineering The radical redesign of current business processes with the intent of reducing cost and cycle time, resulting in increased customer satisfaction. Senior Management The manager(s) responsible for mission accomplishment and overall operations. Stakeholder Any group or individual who is affected by or who can affect the future of an organization, e.g., customers, employees, suppliers, owners, other agencies, Congress, and critics. Strategic Planning A process for helping an organization envision what it hopes to accomplish in the future; identify and understand obstacles and opportunities that affect the organization’s ability to achieve that vision; and set forth the plan of activities and resource use that will best enable the achievement of the goals and objectives. Page 129 Glossary Task A well-defined unit of work having an identifiable beginning and end that is a measurable component of the duties and responsibilities of a specific job. Total Quality Management 1) A management philosophy that involves everyone in an organization in controlling and continuously improving how work is done in order to meet customer expectations of quality. 2) The management practice of continuous improvement in quality that relies on active participation of both management and employees using analytical tools and teamwork. Validation An evaluation performed to determine whether planned actions, if implemented, will address specific issue(s) or objective(s). Verification 1) A determination that an improvement action has been implemented as designed. 2) The act of reviewing, inspecting, testing, checking, auditing, or otherwise determining and documenting whether items, processes, services, or documents conform to specified requirements. Appendix C: Sample Assessment Measures Page 133 Appendix C: Sample Assessment Measures These assessment measures were the product of COPS-funded workshops and discussions on problem-solving held in 1998 and 1999. The workshops were attended by 16 police agencies, all of which were recipients of the 1997 COPS Problem-Solving Partnership grant program. Subject matter experts were selected to facilitate the workshops, working with these agencies to generate a list of common outcome measures for specific crime and disorder problems. These lists represent the outcome measures agreed on by the police agencies attending these meetings. Additional measures for assessing the effectiveness of police response to crimes and disorder can be found in the COPS Office Problem-Oriented Policing Guides (POP Guide) series. The POP Guides provide law enforcement with problem-specific questions to help identify potential factors and underlying causes of specific problems, identify known responses to each problem, and provide potential measures to assess the effectiveness of problem-solving efforts.98 General Assessment Measures Traditional Assessment Measures • Fewer incidents of targeted crime • Fewer citizen complaints about targeted crime • Fewer calls for service about targeted crime Nontraditional Assessment Measures Community Impact • Reduced fear/concern about problem • Increased citizen satisfaction with the way the police are handling the problem • Increased legitimate usage of target problem area Financial Impact • Lower levels of financial loss related to problem/increased business profits in target area • Decreased insurance payouts related to problem • Less property damage associated with problem Page 134 Appendix C Displacement-Related • Positive displacement of problem (displaced to another location, but at a lower level, or a less harmful type of crime, etc.) • Diffusion of benefits (target problem was reduced, and related problems were reduced as well; for example, efforts focused on reducing public drinking problem also happened to reduce vandalism problem) Other Measures • Fewer repeat calls for service about specific problem locations; fewer calls for service about related problem locations • Fewer repeat victims • Fewer/less serious injuries related to problem • Increased usage of a particular service/product that would reduce the problem (e.g., use of public transportation by repeat DWI offenders) • Increased witness cooperation (particularly for intimidation problems) Assault (Nondomestic) Traditional Assessment Measures • Change in the number of reported attacks • Change in the number of arrests for assault • Change in the number of cases prosecuted for assault • Change in the number of clearances for assault • Change in the number of assault convictions • Change in calls for service for assault in target area/jurisdictionwide Nontraditional Assessment Measures Community Impact • Change in fear of attack among victims and nonvictims • Changes in citizen satisfaction with handling of assault problem • Changes in citizen perception of prevalence and seriousness of the assault problem • Increased patronage/use of the problem location (bar, park, alley, shopping center, etc.) Financial Impact • Decline/increase in the cost of victim medical bills directly associated with targeted assaults • Decline/increase in cost to local hospitals for treatment of uninsured assault victims • Decline/increase in the number and amount of insurance claims of victims • Decline/increase in the average number of days lost at work/school by victims Page 135 • Increased/decreased profits of businesses in the location of the assault problem Displacement-Related • Decrease in reported assaults in target area while increase in another neighborhood/area • Change in type of weapon used • Changes in victim/offender characteristics • Changes in relationship between victim and offender • Change in the peak times that nondomestic assaults are occurring • Change in number of homicides • Decline/increase in the number of related misdemeanor offenses reported • Changes in the dynamics of the assault (one-on-one, gang-fights, surprise attacks, etc.) Other Measures • Change in the number of admissions to the hospital for nondomestic assault injuries • Decline/increase in repeat victimizations • Decline/increase in repeat offending • Decline/increase in the number of assaults that result in severe injury or death • Changes in the number of primary offenses with assault as a secondary offense • Changes in the number of assaults that result in a sexual assault or homicide • Changes in the reaction of the victim to the offender (fight back/not, etc.) • Decline/increase in the repeat calls for service at a problem location Commercial Burglary Traditional Assessment Measures • Change in the number of incidents/calls for service for commercial burglary in target area • Change in the number of arrests for commercial burglary • Change in the clearance rate for commercial burglary cases • Change in the number of commercial burglary cases prosecuted • Change in the number of convictions for commercial burglary Nontraditional Assessment Measures Community Impact • Increased/decreased usage of effective and tailored burglary prevention techniques by businesses/building owners (e.g., target hardening, cash/merchandise lockups, increased visibility of entry and exit points, security guards, video surveillance, lighting) • Changes in business owners’ satisfaction with police handling of commercial burglaries • Changes in perception of prevalence of commercial burglaries Page 136 Appendix C • Changes in the reporting of commercial burglaries (first time and repeat burglaries) Financial Impact • Changes in the amount of insurance claims/payout for items stolen in commercial burglaries • Changes in the average loss to the business per burglary or total annual business losses • Changes in the cost of damage to burgled businesses (lost revenue) Displacement-Related • Changes in offender characteristics (e.g., age, gender, race, location of residence, economic status) • Changes in the time of day of break-ins • Changes in the type of businesses victimized (e.g., corporations, convenience stores, restaurants, bars, factories, retail) • Changes in the method of breaking/entering or transporting the stolen items • Changes in the type of property taken • Changes in the location of the burglary (one commercial area to another/one city to another) • Changes in the mechanisms of liquidating the stolen items • Change in the number of reported related crimes (e.g., vandalism, robbery, larceny, residential burglary) Other Measures • Change in the number of repeat burglary calls for service at the same business • Change in the level of retail tenant turnover due to burglary losses/number of vacant businesses in the target area • Change in repeat offending/arrests • Change in the recovery rate of property stolen in commercial burglaries • Change in the number of businesses in target area at high risk of burglary/exhibiting high- risk factors Underage Drinking Traditional Assessment Measures • Change in the number of calls for service for underage drinking/possession of alcohol • Change in the number of arrests for underage drinking/possession of alcohol • Change in the number of citizen complaints about underage drinking • Change in the number of underage drinking/possession of alcohol cases prosecuted • Change in the number of convictions for underage drinking/possession of alcohol Page 137 Sample Assessment Measures Nontraditional Assessment Measures Community Impact • Change in the number of complaints to policing agency/letters to editor regarding underage drinking and associated problems (disorderly conduct, vandalism) • Decline/increase in citizen’s fear of underage drinking and related problems (disorderly conduct, vandalism, loitering, DWI/DUI) • Increased legitimate use by community members of public areas where underage drinking occurs • Changes in citizen satisfaction with police handling of underage drinking and related problems • Changes in perception of prevalence of underage drinking/possession of alcohol and related problems Financial Impact • Changes in the costs of medical bills directly associated with injuries resulting from drinking (DWI/DUI, accidents, sexual assaults) • Changes in the costs associated with vandalism/property destruction perpetrated by underage drinkers • Changes in cost of medical bills associated with being a victim of an accident caused by an underage drinker • Changes in profits and patronage of businesses selling to underage consumers Displacement-Related • Changes in the number of reported related problems/crimes (DWI, disorderly conduct, vandalism, loitering, public urination, theft, sexual assaults) • Changes in the location of alcohol consumption • Changes in the time of day of underage drinking • Changes in characteristics of underage drinkers (age, gender, race, location of residence, economic status) • Changes in the method of purchasing alcohol by underage drinkers (falsified identification misrepresenting age, use of legal-aged adult to purchase alcohol) • Changes in the type of establishments selling alcohol to underage consumers (bar/ restaurant, grocery store, convenience store, state alcohol control board sales outlet/store) • Change in the location of the establishments underage consumers patronize to obtain alcohol (adjoining municipality, certain parts of town) Page 138 Appendix C Other Measures • Changes in the number of emergency calls/hospital admissions for accidents resulting from underage drinking (including alcohol poisoning, accidents related to location of drinking (swimming/boating, etc.), DWI, sexual assaults, injuries to persons victimized by underage drinkers) • Change in the number of repeat calls for service at specific locations related to underage drinking and associated problems • Changes in self-reports of levels of underage drinking • Decline/increase in repeat offending/arrests of underage drinkers • Changes in rates of school attendance/tardiness among target age group Appendix D: Examples of Performance Management in Justice Agencies Page 141 Appendix D: Examples of Performance Management in Justice Agencies Many jurisdictions throughout the nation and around the world are actively engaged in sophisticated and effective performance measurement and management programs. In an effort to demonstrate the principles and best practices of performance management in operational settings of justice agencies, brief case studies and examples are provided in this appendix. Beyond these brief summaries, the reader is encouraged to review additional detail regarding these programs in the references cited. COMPSTAT: New York City and Los Angeles COMPSTAT (computerized statistics) continues to operate in the police departments of New York City and Los Angeles, and in many other law enforcement agencies throughout the United States.98 It serves as one of the best examples of a comprehensive performance management program within law enforcement agencies by linking strategic planning directly to operations, by establishing and enforcing accountability, and by implementing innovative technologies to support extensive data collection, analysis, mapping, and reporting. By itself, COMPSTAT does not address all performance measurement requirements facing contemporary law enforcement agencies. Agencies still need to monitor and manage specific projects, for example, including internal administrative operations and information technology applications, which may only indirectly contribute to substantive changes in the nature or volume of crime within a jurisdiction and/or the agency’s response. It does, however, provide an important organizational and management structure that clearly values planning, research, ongoing performance measurement and program evaluation, and proactive response based on empirical research. COMPSTAT, New York City Police Department As previously discussed in this Guide, the New York City Police Department (NYPD) created the COMPSTAT process in 1994. COMPSTAT does not operate in a vacuum, nor does it function Page 142 Appendix D effectively solely within the department. To be sure, COMPSTAT drives police operations in very real and tangible ways, but it also relies on collaboration with other agencies and with the community as a whole. That process continues to operate within NYPD and detailed crime information is posted on the Department’s web site for the city, by borough, and by precinct.99 LA COMPSTAT and COMPSTAT Plus, Los Angeles Police Department In 2002 William Bratton became chief of the Los Angeles Police Department (LAPD). Not surprisingly, LAPD has implemented COMPSTAT (known there as LA COMPSTAT)100 and has augmented the program by including the use of more in-depth auditing methods, mentorship, and close collaboration in an expanded version known as COMPSTAT Plus.101 Neighborhood-specific crime maps and citywide statistics, which facilitate transparency of operations, are readily available on the LAPD web site and empower the community in fundamental respects.102 In addition, effective planning and implementation are made manifest in the publication of the LAPD Plan of Action, which demonstrates the intrinsic link among performance measurement and agency culture, management, and strategic planning.103 Police Performance Assessment Framework: Home Office, United Kingdom The United Kingdom has implemented a comprehensive performance management framework across the 43 police forces of England and Wales—known as the Policing Performance Assessment Framework (PPAF). Technology is an important factor in effectively collecting, analyzing, and reporting performance information. As noted by the Home Office: “During the past three years, a performance framework has been developed by the Home Office and Her Majesty’s Inspectorate of Constabulary (HMIC), with support from the Association of Police Authorities (APA) and the Association of Chief Police Officers (ACPO). The aim is to drive up performance in policing. This framework is called the Policing Performance Assessment Framework (PPAF). “In order to present a picture about how a force is performing, two assessments have been made in seven key performance areas [or domains: 1) Reducing Crime, 2) Investigating Crime, 3) Promoting Safety, 4) Providing Assistance, 5) Citizen Focus, 6) Page 143 Examples of Performance Management in Justice Agencies Resource Use, and 7) Local Policing]. These assessments are based on a combination of performance data and professional judgement. Assessments are awarded for the performance delivered by a police force in 2004/05 (by comparing a force to its peers), and also for direction (by comparing the performance achieved by a force in one year to that achieved by the same force in the previous year). “Assessments are made covering the full range of policing activity, with a focus on local policing issues, apart from counter-terrorism (the latter being excluded given the difficulty and sensitivity of assessments in this critical area of policing).”104 The seven key performance domains identified above are clearly defined and carefully measured using a detailed methodology that is universally applied in measuring performance of each of the 43 police forces in England and Wales.105 The Home Office, for example, provides detailed guidance in defining statutory performance indicators that comprise each of the seven key performance domains. For example, the performance domain Citizen Focus is a function of how well forces address user (citizen) satisfaction with the service of police as measured by a variety of metrics: Statutory Performance Indicators (SPI) Long Title and Short Title SPI 1a. Satisfaction of victims of domestic burglary, violent crime, vehicle crime and road traffic collisions with respect to making initial contact with the police. Satisfaction with making contact SPI 1b. Satisfaction of victims of domestic burglary, violent crime, vehicle crime and road traffic collisions with respect to action taken by the police. Satisfaction with action taken SPI 1c. Satisfaction of victims of domestic burglary, violent crime, vehicle crime and road traffic collisions with respect to being kept informed of progress. Satisfaction with being kept informed SPI 1d. Satisfaction of victims of domestic burglary, violent crime, vehicle crime and road traffic collisions with respect to their treatment by staff. Satisfaction with treatment by staff SPI 1e. Satisfaction of victims of domestic burglary, violent crime, vehicle crime and road traffic collisions with respect to the overall service provided. Satisfaction with overall service 106 The police forces are provided detailed instruction on how each SPI is measured, including reporting frequency, data source(s), definitions, and standardized computational formulas (see example in Figure D.1).107 The performance measures are defined each year by statute.108 Page 144 Appendix D Figure D.1: User Satisfaction Measures for Citizen Focus Domain USER SATISFACTION MEASURES SPI 1a Satisfaction with making contact SPI 1b Satisfaction with action taken SPI 1c Satisfaction with being kept informed SPI 1d Satisfaction with treatment by staff Domain.............Type of Measure Citizen Focus......User Satisfaction Measure Priority Component.Comparator Type Not proposed.......MSF average Existing/Revised/New...Level of Data Existing SpI.......Force Reporting Frequency..Data Periods Reported through aDR: quarterly, discrete analysis & use: rolling years: 1 apr to 30 Jun 06 1 Jul to 30 Sep 06 1 Oct to 31 Dec 06 1 Jan to 31 Mar 07 Proportionality.........Data Source / ADR Form Forces to collect at 16+1 ethnicity Forces to report to hO at 5+1 ethnicity Measure to be published for 1 category of ethnicity -Force survey following hO/aCpO/apa guidance Submit raw data using aDR 443 Measure to be reported For each SpI (1a to 1d), two figures: • of victims surveyed, the percentage satisfied (y); and • of victims surveyed, the percentage very/completely satisfied (z). Formulae These percentages will be calculated automatically on the aDR spreadsheet. The description below is provided for information. For each SpI calculate y and z using the appropriate survey question. y equals (a+b+c) divided (a + b + c + d + e + f + g) times 100 z equals (a+b) divided (a + b + c + d + e + f + g) times 100 where: a is total number of respondents who answered ‘completely satisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; b is total number of respondents who answered ‘very satisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; c is total number of respondents who answered ‘fairly satisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; d is total number of respondents who answered ‘neither satisfied nor dissatisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; Page 145 Examples of Performance Management in Justice Agencies Figure D.1: User Satisfaction Measures for Citizen Focus Domain, continued e is total number of respondents who answered ‘fairly dissatisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; f is total number of respondents who answered ‘very dissatisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions; and g is total number of respondents who answered ‘completely dissatisfied’ for domestic burglary, violent crime, vehicle crime and road traffic collisions. Definitions: • Domestic burglary is defined as any burglary of a dwelling, including aggravated and attempted burglary, (i.e. hO offence codes 28 and 29). • The violent crime survey sample will be drawn from victims of wounding, assault and personal robbery, (i.e. hO offence codes 8a, 105a, and 34B). • vehicle crime is defined as any theft of motor vehicle, including aggravated and attempted vehicle taking, and any theft from a vehicle (i.e. hO offence codes 37.2, 45 and 48). • Road traffic collisions are those attended by the police (regardless of whether injury was sustained). Notes: 1. ‘Don’t know’ and blank responses should be excluded from the calculation. 2. The data submitted should be for the surveys carried out within each data period (rather than for crimes and incidents reported during the period). 3. The measures should have a confidence interval of no more than ±4.0% at the 95% confidence level. 4. The measure will be reported as a whole number. 5. Sample sizes (as per the ppaF user satisfaction survey guidance) are designed to meet the required confidence limits using a full year’s data. 6. The above guidance should be read in conjunction with Definitions and Survey Guidance for PPAF Measures of User Satisfaction 2006/07, which will be available on the assessment Methods website: http://police.homeoffice.gov.uk/ performance-and-measurement/assessment-methods/ 7. These indicators relate to aggregate results for the 4 victim groups to avoid the potential distorting effect of over- or under-sampling particular groups. For the purposes of the aDR and the SpIs 1a-d, the aggregation of results from the four user groups to produce overall force results is carried out by giving the results from each group equal weight (i.e. a contribution of 25% to the total). 8. a kDI has been added from april 2006 which requires forces to provide data on the results for all five measures (ie 1a - 1e) for those who report anti-Social Behaviour. Survey guidance on the new aSB requirement can also be found on the assessment Methods website. 9. See also SpI 1e. The resulting figures are reported by force throughout England and Wales, and comparisons are made between comparable forces, as well as for the reporting force over time (looking at changes in current performance year to year) (see Figure D.2). Performance measures are also aggregated for all forces to generate national metrics.109 Page 146 Appendix D Figure D.2: Crime Reduction / Promoting Safety National Comparisons (UK) (Chart 1) REDUCING CRIME.........National average (2003/04) (2004/05)..........National average (2003/04 to 2004/05) 1. 4a Comparative risk of personal crime...........(7%) (6%).........Statistically significant reduction 2. 4b Comparative risk of household crime..........(20%) (18%).......Statistically significant reduction 3. 5a Domestic burglary rate1......................(18.0) (14.4).....20.1% reduction 4. 5b violent crime rate2..........................(21.0) (22.4).....6.9% increase 5. 5c Robbery rate2................................(1.9) (1.7).......12.3% reduction 6. 5d vehicle crime rate2..........................(16.8) (14.0).....16.9% reduction 7. 5e Life threatening and gun crime rate2.........(0.62) (0.63).....2.0% increase 1 Rate per 1,000 households. 2 Rate per 1,000 population. (Chart 2) PROMOTING SAFETY........National average (2003/04) (2004/05)..........National average (2003/04 to 2004/05) 1. 9a Road traffic safety (casualty rate)4.....(7.6) (6.9)............9% reduction 2. 10ai Residents’ fear of crime – burglary....(13%) (12%)............Change not statistically significant 3. 10aii Residents’ fear of crime – car........(15%) (13%)............Statistically significant reduction 4. 10aiii Residents’ fear of crime – violent...(16%) (16%)............Change not statistically significant 5. 10b perceptions of anti-social behaviour....(16%) (17%)............Change not statistically significant 4 Data available only for calendar years. Number of people killed or seriously injured in road traffic collisions per 100 million vehicle km travelled. Page 147 Examples of Performance Management in Justice Agencies Justice Network, Commonwealth of Pennsylvania The Pennsylvania Justice Network (JNET™) is a secure virtual system for the sharing of justice information by authorized users.110 JNET is a collaborative effort of municipal, county, state, bordering states, and federal justice agencies to build a secure integrated justice system. Prior to JNET, each state agency had its own computer systems and databases. This resulted in a fragmented justice environment in which information sometimes took days or weeks to get to the appropriate agencies. JNET helps solve this problem and represents an unprecedented leap forward in information sharing and cooperation among local, county, state, and federal agencies. JNET provides a common online environment whereby authorized users can access offender records and other justice information from participating agencies. Based on open Internet/World Wide Web technologies and standards, JNET links information from diverse hardware and software platforms under a common, web-browser interface. Firewalls protect agency networks and systems from unauthorized intrusion. JNET has avoided “turf issues,” which have traditionally plagued other integration efforts, by leveraging existing agency systems, recognizing and ensuring agency independence, and allowing agencies to maintain control of their information. Additionally, JNET provides a forum for each participating agency to collaborate and share ideas. A Steering Committee is composed of members from 17 Commonwealth agencies who are appointed by each respective agency head. (See Executive Order, which defines the JNET governance structure.)111 Each Steering Committee member has an opportunity to vote on how the JNET budget is allocated, how and what data are shared, and how policy and technical issues affecting their organization and integrated justice as a whole are handled. Steering Committee members chair and staff each of the various JNET subcommittees and are advocates for JNET in their respective agencies. It is through their collaboration, commitment, cooperation, and hard work that JNET is the most successful integrated justice system in the nation. JNET began in May 1997 with the founding vision to enhance public safety through the integration of justice information throughout the Commonwealth of Pennsylvania. JNET has adopted business practices that promote cost effectiveness, information sharing, and timely and appropriate access to information while recognizing the independence of each agency. Page 148 Appendix D Objectives One of the major benefits of JNET is safer communities.112 Community safety is enhanced because JNET provides law enforcement officers with immediate access to critical criminal justice information that helps them to perform their jobs more effectively. The availability of this information has enabled officers in the field to identify and apprehend suspected individuals, solve cases faster, and keep criminals off the street. JNET has also minimized the risk of releasing offenders who could pose a public threat. Benefits also include reduced costs associated with defendant and offender processing, reduced delays in processing criminal cases, and a lower risk of releasing offenders who could pose a public threat. JNET has improved prosecution efforts and reduced delays through timely access to case information. The availability of photos through JNET has also been used to prevent innocent people from being arrested due to misidentification. A mobile version of JNET has been initiated to enable state and local officers access to JNET through laptops in their vehicles. Mobile access to Pennsylvania Department of Transportation photos, vehicle registration information, and criminal history records information and photos will be a major benefit to law enforcement and will enhance public safety. Using photos and other information available through JNET, officers will be able to identify suspected individuals much more quickly. Mobile access to data also will help prevent the arrest of innocent individuals. JNET already has been used extensively in the field by criminal justice agencies to identify and apprehend suspected criminals. The system has improved prosecution efforts and has reduced delays through timely access to case information. For example: • The Pennsylvania State Police use JNET to access criminal history records information and to assist in investigations. • The Pennsylvania Department of Corrections uses JNET to keep inmate records current. • The Pennsylvania Board of Probation and Parole uses JNET to access criminal history records information, to assist in finding people fleeing the law (absconders), and to access Department of Corrections’ misconduct information. • The FBI uses JNET to access criminal history records information and to assist in investigations. • District justices use JNET to access statewide Administrative Office of Pennsylvania Courts’ information. • Counties use JNET to determine inmate location, to find address information when serving warrants, and to assist in finding absconders. Page 149 Examples of Performance Management in Justice Agencies Performance Management JNET takes a systematic approach to service delivery, building on effective governance, effective strategic planning, an annual business plan that is tightly aligned with the strategic plan, and a Project Management Office (PMO), which carefully monitors and measures projects and programs to ensure an effective project life cycle (Figure D.3).113 Figure D.3: Pyramid Chart JNET’s Systematic Approach to Service Delivery 1.Project Life Cycle: Delivery of projects in a systematic, structured method. Project assessment, planning,and execution. 2.Project Management Office: PMO serves as central point for tracking, reporting, and management on all formal projects 3.Critical Path Matrix: Lists current projects with description, sponsor, status, milestone, and delivery dates 4.Annual Business Plan: Out lines the projects for entire fiscal year and ensures alignment with Strategic Plan 5.Strategic Plan: Outlines Vision, Mission, and Strategic Goals 6.Governance: Provides Strategic Vision, Policy, and Budget Oversight Page 150 Appendix D The approach to addressing performance measurement relates vision to strategy and performance in the following ways (Figure D.4): Figure D.4: Performance Measure: Vision and Strategy (Org chart) Customer: How should we appear to our customers? How do we better meet our constituents’ needs and improve customer service? Internal Business Process: What new processes can we establish for a efficient, effective integrated justice organization? Costs Savings How can we explore cost effective solutions for JNET, our business partners and the Commonwealth? Vision & Strategy Infrastructure & Organizational Growth: How will we improve and meet changing requirements imposed on or requested of JNET? Performance management puts strong focus on ensuring all parts of the organization are aligned with the overall preferred results of the organization. JNET addresses the following questions in assessing the results expected from projects: • Do the preferred results contribute to achieving the organization’s preferred results? How? • Is there anything else the organization could be doing to contribute more directly to the identified goals? In addition, results are weighted to reflect prioritization of the organization’s strategic objectives, addressing the following questions: • How many strategic objectives does this measurement address? • How many agencies are involved or affected? • How great of an impact will this measure have? An example of the weighting and scoring of a project appears in Figure D.5. Page 151 Examples of Performance Management in Justice Agencies Figure D.5: Example Spreadsheet Project Weighing and Scoring Performance measures are closely monitored to assess the project’s progress to achieve preferred results. Progress is continually measured to identify problems as they emerge and remedial actions are planned in advance, should performance vary from what is projected. In addition, ongoing feedback to project managers and participants provides useful and timely information to improve performance. Page 152 Appendix D Criminal Justice Information Technology Office, United Kingdom The Criminal Justice Information Technology office (CJIT) in the United Kingdom has initiated a nationwide information sharing program for “Joining-Up Justice.” CJIT’s vision is “to increase public trust in the effectiveness of the criminal justice system by bringing more criminals to justice and putting the interests of law-abiding citizens first.” CJIT will achieve this by “delivering a modern and joined-up criminal justice system that will harness the latest information technology to reduce unnecessary paperwork, speed up processes, and improve the criminal justice experience for all.”114 Like justice information sharing initiatives throughout the United States, the Joined-Up Justice program administered by CJIT is designed to provide immediate and enterprise-wide information accessibility and information exchange. The UK government has invested £2.2 billion in improving the information systems of participating justice agencies (police, prosecution, courts, and corrections) and is building the ability to link these systems together (i.e., “joining them up”) for more effective information sharing and accessibility (see Figure D.6). The objectives of this large-scale program are to: • Help build a fairer and more just society through the delivery of swifter justice • Ensure the public is served in a way that meets their expectations—that information supports people within the criminal justice system rather than acting as a barrier • Deliver technology solutions that will benefit the whole criminal justice system by helping to improve performance across every criminal justice organisation • Step up the fight against crime.115 Page 153 Examples of Performance Management in Justice Agencies Figure D.6: Criminal Justice Information Sharing in the UK (Image of Organizational Web) The objectives of the benefits realization and performance management program are to apply consistent measures to all projects and programs funded from the criminal justice system (CJS) IT budget; robust measurement, to minimize double counting; comprehensive measures, to capture all forms of value that are added by the projects and programs, and applied, to ensure that performance is measured from business case to benefit realization. The investment of government funds to support the CJIT Joined-Up Justice program is structured according to well-established investment principles: • Funding is dependent upon adherence to the CJS IT governance arrangements. • Projects funded must deliver benefits to more than one agency. • Complete what is started subject to performance. • Projects must provide positive rates of return agreed with the recipients. • Options selected should provide value for money and be quickly achievable. • Consume your own smoke (i.e., departments are expected to fund overspends). • Continued funding depends on performance, including benefits realisation. • Critical issues need to be addressed before funding is confirmed. • Projects must meet industry best practice criteria for Attractiveness and Achievability, which are independently measured. Examples of the Attractiveness and Achievability analyses are shown in Figures D.7 and D.8. Page 154 Appendix D ATTRACTIVENESS ANALYSIS ECONOMIC ANALYSIS* Figure D.7: Attractiveness Analysis Image of Attractiveness Analysis Chart Figure D.8: Achievability Analysis Image of Achievability Analysis Chart Examples of other CJIT project measurement and accountability reports are available in a PowerPoint presentation, “Realising the Benefits of e-Government—the CJS IT Programme experience,” by Stephen Jenner, Portfolio Director, Criminal Justice IT, available at http:// www.search.org/events/past/2006symposium/presentations.asp. Appendix E: Endnotes Page 157 Appendix E: Endnotes 1 Other Guides in this series include the Law Enforcement Tech Guide: How to plan, purchase and manage technology (successfully!), A Guide for Executives, Managers and Technologists, as well as those addressing small and rural agencies, IT security, and communications interoperability. See http://www.search.org/programs/safety/tech-guide.asp for details on and links to these guides. Also see http://www.cops.usdoj.gov/ for more information about the Office of Community Oriented Policing Services (COPS Office) and links to the guides, as well. 2 In addition to the Tech Guides, SEARCH has also hosted a series of conferences and workshops addressing many of the issues associated with the Tech Guides. See http://www.search.org for more information about SEARCH, The National Consortium for Justice Information and Statistics. 3 Harris, Kelly J., and William H. Romesburg, Law Enforcement Tech Guide: How to plan, purchase and manage technology (successfully!), A Guide for Executives, Managers and Technologists (Washington, D.C.: U.S. Department of Justice COPS Office, 2002). 4 For detailed information regarding historical (i.e., 1993, 1997, and 2003 comparisons) and current crime reported throughout New York City in weekly, monthly, year to date, and 2-, 4-, and 12-year increments, see http://www.nyc.gov/html/nypd/pdf/chfdept/cscity.pdf. Similarly detailed information by precinct is available at http://www.nyc.gov/html/nypd/html/pct/ cspdf.html. “Serious crime” for reporting purposes includes murder, rape, robbery, felonious assault, burglary, grand larceny, and auto theft. While other factors, such as shifting population demographics, certainly play a role in explaining these unprecedented declines in reported crime, a fundamental shift in management philosophy and operations focusing on proactive policing and performance measurement is largely credited for this dramatic fall in crime. 5 For detailed information regarding departmentwide reported crime in Los Angeles, see http:// www.lapdonline.org/assets/pdf/cityprof.pdf. Part 1 Crimes include those typically reported in the Uniform Crime Reporting (UCR) program of the Federal Bureau of Investigation (FBI), including homicide, rape, robbery, aggravated assault, burglary, auto theft, burglary/theft from vehicle, and personal theft/other theft (larceny). 6 Mazerolle, Lorraine, Dennis Rogan, James Frank, Christine Famega, and John E. Eck, Managing Citizen Calls to the Police With 911/311 Systems, NIJ Research for Practice, NCJ 206256 (Washington, D.C.: U.S. Department of Justice, National Institute of Justice (NIJ), February 2005), at http://www.ncjrs.org/pdffiles1/nij/206256.pdf; and Solomon, Shellie E., and Craig D. Uchida, Building A 3-1-1 System For Police Non-Emergency Calls: A Process and Impact Evaluation Page 158 Appendix E (Washington, D.C.: 21st Century Solutions, Inc., 2003), at http://www.cops.usdoj.gov/Default.asp?1275. 7 For purposes of our discussion, the generic reference to “police” is meant to incorporate police, sheriffs and their deputies, and other duly authorized law enforcement agencies and officers. 8 Alpert, Geoffrey P., and Mark H. Moore, “Measuring Police Performance in the New Paradigm of Policing,” in Performance Measures for the Criminal Justice System, NCJ 143505 (Washington, D.C.: U.S. Department of Justice, Bureau of Justice Statistics, October 1993), pp. 119–124. Also see Roper, Bob, and Teri B. Sullivan, “Measuring the Success of Integrated Justice: A Practical Approach,” in Information Systems Integration: A Library of SEARCH Resources for Justice and Public Safety Practitioners (Sacramento, CA: SEARCH Group, Inc., 2004), at http://www.search.org/files/pdf/IntegrationLibrary.pdf. 9 Bayley, David H., “Measuring Overall Effectiveness,” in Quantifying Quality in Policing, Lawrence T. Hoover (ed.) (Washington, D.C.: Police Executive Research Forum (PERF), 1996), pp. 37–54. 10 Henry, Vincent E., The COMPSTAT Paradigm: Management Accountability in Policing, Business and the Public Sector (Flushing, NY: Looseleaf Law Publications, Inc., 2002), p. 318; and Bratton, William, with Peter Knobler, Turnaround: How America’s Top Cop Reversed the Crime Epidemic (New York: Random House, 1998), p. 224. Also see Moore, Mark H., and Anthony A. Braga, “Measuring and Improving Police Performance: The Lessons of CompStat and its Progeny,” in Policing: An International Journal of Police Strategies & Management, Volume 26, No. 3, 2003, pp. 439–453. 11 Bratton, William J., “Great Expectations: How Higher Expectations for Police Departments Can Lead to a Decrease in Crime,” in Measuring What Matters: Proceedings from the Policing Research Institute Meetings, Robert H. Langworthy (ed.), NCJ 170610 (Washington, D.C.: U.S. Department of Justice, NIJ and COPS Office, July 1999), p. 15. 12 Ibid., at p. 20. Wilson, James Q., and George L. Kelling, “Broken Windows: The Police and Neighborhood Safety,” Atlantic Monthly, Volume 249, No. 3, March 1982. For an overview of the CompStat process, also see Kelling, George, “How to Run a Police Department,” City Journal, Volume 5, No. 4, Autumn 1995. For a review and critique of the CompStat process, also see Moore, Mark H., David Thacher, Andrea Dodge, and Tobias Moore, Recognizing Value in Policing: The Challenge of Measuring Police Performance (Washington, D.C.: PERF, 2002). 13 For additional discussion of ROI associated with Chicago’s CLEAR system, see McKay, Jim, “Clearly a Hit,” in Government Technology, May 2004, at http://www.govtech.net/magazine/story.php?id=90188&issue=5:2004&vw=hl. 14 Artley, Will, D.J. Ellison, and Bill Kennedy, The Performance-Based Management Handbook, Volume 1: Establishing and Maintaining a Performance-Based Management Program (Washington, D.C.: U.S. Department of Energy, 2001), p. 4. Page 159 Endnotes 15 Colonel Kenneth Bouche, Illinois State Police, provided the direct savings examples in this section from examples used in the I-CLEAR Project to describe Return on Investment. For examples of indirect savings, visit the SEARCH Tech Guide Resources web site at http://www. search.org/programs/safety/tech-guide.asp. I-CLEAR is a law enforcement data warehouse accessible by all Illinois law enforcement. The project is a partnership between the Chicago Police Department and the Illinois State Police. See http://www.icjia.state.il.us/IIJIS/public/PDF/ ICLEARinterfaceSlides_IMB_02112004.pdf. 16 COPS Office, Grant Monitoring Standards and Guidelines for Hiring and Redeployment (Washington, D.C.: U.S. Department of Justice COPS Office, September 2004), p. 13, at http://www.cops.usdoj.gov/mime/open.pdf?Item=299. 17 Some of these reasons have been adapted from The CHAOS Report (West Yarmouth, MA: The Standish Group, 1994), at http://www.standishgroup.com/sample_research/chaos_1994_ 1.php, while others (notably 1–5) were suggested by Major Piper Charles, Charlotte-Mecklenburg (North Carolina) Police Department, who kindly served as one of the reviewers of this report. 18 Items 1–10 are taken from a PowerPoint presentations by Rohm, Howard, “Improve Public Sector Results With A Balanced Scorecard: Nine Steps To Success” (The Balanced Scorecard Institute, 2003) slide 34, at http://www.balancedscorecard.org/files/Improve_Public_ Sector_Perf_w_BSC_0203.swf. 19 Blalock, Jr., Hubert M., Social Statistics (New York: McGraw-Hill, 2nd Edition, 1979); Eck, John E., Assessing Responses to Problems: An Introductory Guide for Police Problem-Solvers (Washington, D.C.: U.S. Department of Justice Office of Community Oriented Policing Services, 2001), at http://www.popcenter.org/Tools/tool-assessing.htm; and Glass, Gene V. ,and Julian C. Stanley, Statistical Methods in Education and Psychology (New Jersey: Prentice Hall, 2nd Edition, 1970). 20 In addition to this Tech Guide, the U.S. Department of Justice has also funded a project to address Performance Measurement Tools for Information Technology Projects. For more information, see http://www.cslj.net/PerformanceMeasure/index.htm. 21 For more detailed discussion of the CompStat process and its implementation in New York, see Bratton with Knobler, supra, note 10, and Henry, supra, note 10. 22 Kaplan, Robert S., and David P. Norton, The Balanced Scorecard: Translating Strategy into Action (Boston: Harvard University School Press, 1996), and The Balanced Scorecard Institute, “What is the Balanced Scorecard?” at http://www.balancedscorecard.org/basics/bsc1.html. 23 Moore and Thacher, et. al., supra, note 12 at pp. 75–79. Page 160 Appendix E 24 See the Royal Canadian Mounted Police’s use of the balanced scorecard approach to strategic planning at http://www.rcmp.ca/dpr/rpp2002_a_e.htm. Also see how police agencies in the United Kingdom have taken a balanced scorecard approach from a national perspective: The National Policing Plan 2003–2006 (London: Home Office, November 2002) at http://police. homeoffice.gov.uk/news-and-publications/publication/national-policing-plan/nat_ police_plan02.pdf. Also see performance plans, strategic objectives, and targets for local forces in the United Kingdom: Cambridgeshire Constabulary, “Cambridgeshire Policing Plan 2004–2005,” at http://www.cambs.police.uk/caminfo/plansandreports/cpp0405/; Greater Manchester Police, “Performance Statistics,” at http://www.gmp.police.uk/mainsite/pages/ performancestats.htm; and Metropolitan Police Authority, Policing and Performance Plan 2005/06, at http://www.met.police.uk/about/plans.htm. 25 For general discussion of the SARA process, see http://www.popcenter.org/about- SARA.htm. For a more detailed discussion of program evaluation and POP using the SARA process, see Eck, supra note 19. For other program evaluation methodologies, see the Bureau of Justice Assistance (BJA) Center for Program Evaluation, at http://www.ojp.usdoj.gov/ BJA/evaluation/index.html, and KRA Corporation, “A Guide to Evaluating Crime Control of Programs in Public Housing” (Washington, D.C.: U.S. Department of Housing and Urban Development, Office of Policy Development and Research, April 1997), at http://www.ojp. usdoj.gov/BJA/evaluation/guide/documents/documentz.html. See also Goldstein, Herman, Problem-Oriented Policing (McGraw-Hill, 1990, and Temple University Press, 1990). 26 Eck, supra note 19 at page 3. 27 See Rohm, supra note 18 at slide 18. 28 Ibid. 29 Artley, et. al., supra note 14 at p. 5. Artley, et. al., acknowledge that this performance-based management paradigm largely follows the “Plan-Do-Check-Act Cycle” originally developed by Walter Shewhart in the 1930s at Bell Laboratories. See Shewhart, Walter A., Statistical Method from the Viewpoint of Quality Control (Washington, D.C.: Department of Agriculture, 1939; reprinted by Dover Publications, 1986). Also see http://www.hci.com.au/hcisite2/toolkit/pdcacycl.htm. 30 Artley, et. al., supra note 14 at p.4. 31 Ibid., at p. 7. 32 Ibid., at p. 10. 33 Harris and Romesburg, supra note 3 at pp. 51–62. 34 See MPDC’s “Three Approaches to Prevention” page at http://mpdc.dc.gov/mpdc/cwp/view,a,3,q,548390,mpdcNav_GID,1530.asp. Page 161 Endnotes 35 See MPDC’s “Comparing the Three Approaches” page at http://mpdc.dc.gov/mpdc/cwp/ view,a,3,q,548341,mpdcNav_GID,1530.asp. The MPDC describes the Police Service Area as follows: The PSA Team. Your Police Service Area (PSA) team, made up of the PSA lieutenant, and his or her sergeants and officers, carries out the most direct role in preventing crime and disorder in your community. The PSA lieutenant is the PSA manager. He or she is accountable for the quality of policing in the PSA 24-hours-a-day. The PSA lieutenant relies on the PSA sergeants to supervise the officers they are working with during their tours of duty. One of the most important goals of the PSA team is maintaining PSA integrity. This means responding to calls for service and getting to know the people, the resources and the problems in their PSA. The PSA team must be visible and accessible to the people who live and work in their PSA. The PSA team is supported by the Focused Mission Teams, the detectives, the Mobile Crime Unit, Major Narcotics Branch, the Office of Youth Violence, the Special Investigations Division, and other critical Department resources…. How to Work with Your PSA Team. Community participation in each PSA is vital. Residents, business people, and other community groups must get to know their PSA lieutenant and sergeants. Community members must know their officers who are out in the neighborhoods day-to-day. Community members also must get involved by attending their PSA’s monthly community meetings. Your PSA lieutenant is responsible for scheduling monthly community meetings, which are open to all members of the community, and held in a place that’s accessible for everyone. Meeting agendas are planned by your PSA lieutenant and community leaders and deal with issues identified by both police and community. See MPDC’s “PSA Teams” page at http://mpdc.dc.gov/mpdc/cwp/view,a,3,Q,548439,mpdcNav_GID,1530,.asp. 36 For further discussion of converting business goals to measurable performance objectives for integrated justice information sharing initiatives, see Roper and Sullivan, supra note 8 at pp. 12–15, and Roberts, David J., Concept for Operations for Integrated Justice Information Sharing (Lexington, KY: NASCIO, 2003), pp. 47–49. 37 Metropolitan Police Department Strategic Business Plan FY 2006–2007 (October 2005) and Metropolitan Police Department Strategic Business Plan FY 2004–2005 (June 2003), at pp. 6–7 and 13–15. The latest revisions of these reports are available at http://mpdc.dc.gov/mpdc/cwp/ view,A,1246,Q,555621.asp under “MPDC Strategic Business Plans” link. 38 Ibid., at p. 13. 39 Ibid., at p. 16. 40 Alpert and Moore, supra note 8 at p. 125. 41 Artley, et. al., supra note 14 at p. 13. 42 UK Home Office, The National Policing Plan 2003–2006, supra note 24. Page 162 Appendix E 43 The National Policing Plan 2004–2007 (London: Home Office, November 2003), at http:// police.homeoffice.gov.uk/news-and-publications/publication/national-policing-plan/natpolplan2004_7.pdf. 44 Ibid., p. 30. 45 See Law Enforcement Information Technology Standards Council (LEITSC) work being done by the International Association of Chiefs of Police (IACP), et. al. 46 The Association of State Correctional Administrators (ASCA) has published three volumes associated with outcome-based performance measurement for correctional institutions. Wright, Kevin, with Judy Bisbee and Patricia L. Hardyman, Defining and Measuring Corrections Performance, Final Report (Middletown, CT: ASCA, 2003); Key Indicators of ASCA Performance Standards (with Counting Rules), (Middletown, CT: ASCA, 2005); and Performance-Based Measures Resource Manual, (Middletown, CT: ASCA, 2005). All three reports are available at http://www.asca.net/publications.html. 47 The COPS Office is working with Caliber Associates and PERF to develop the Community Policing Implementation Assessment tool, which should be ready for use by the field in 2006. See the COPS web site www.cops.usdoj.gov for periodic updates. 48 This material has been slightly edited and taken from UK Home Office, The National Policing Plan 2004–2007, supra note 43. 49 Artley, et. al., supra note 14. 50 For a discussion of performance measures associated with justice integration and information sharing projects, see Center for Society, Law and Justice, Performance Measurement Tools for Justice Information Technology Projects: The Logic Model Approach to Developing Performance Measures for Justice Integration and Information Sharing Projects (Metairie, LA: University of New Orleans, no date), at http://www.cslj.net/PerformanceMeasure/files/ userguidetologicmodelapporach.pdf; and Center for Society, Law and Justice, A Catalog of Performance Measures for Justice IT Projects: An Accountability Model (Metairie, LA: University of New Orleans, no date), at http://www.cslj.net/PerformanceMeasure/files/accountabilityinventory.pdf. 51 Federal Bureau of Investigation, Crime in the United States 2003, Section I, Summary of the Uniform Crime Reporting Program (Washington, D.C.: FBI, 2004), p. 7, at http://www.fbi.gov/ucr/cius_03/pdf/03sec1.pdf. 52 Federal Bureau of Investigation, “Uniform Crime Reports,” at http://www.fbi.gov/ucr/ucr.htm. Page 163 Endnotes 53 The UCR program has undergone significant change over the past 76 years, including transferring the program from the IACP, where it originated, to the FBI, as well as a move to incident-based reporting (the National Incident-Based Reporting Program—NIBRS), from monthly aggregate reporting. For more information, see http://www.fbi.gov/ucr/ucr.htm#nibrs. 54 Eck, supra note 19. For other program evaluation methodologies, see BJA Center for Program Evaluation, supra note 25, and KRA Corporation, supra note 25. 55 UK Home Office, The National Policing Plan 2003–2006, supra note 24. 56 Functional standards for law enforcement were developed by LEITSC, a project funded by the Office of Justice Programs, U.S. Department of Justice: Standard Functional Specifications for Law Enforcement Computer Aided Dispatch (CAD) Systems (Washington, D.C.: LEITSC, 2006), and Standard Functional Specifications for Law Enforcement Records Management Systems (RMS) (Washington, D.C.: LEITSC, 2006). They are available at http://www.leitsc.org. 57 For more information on the National Criminal Intelligence Sharing Plan, see http://it.ojp.gov/topic.jsp?topic_id=93. 58 Moore, Mark H., (with Anthony Braga), The “Bottom Line” of Policing: What Citizens Should Value (and Measure!) in Police Performance (Washington, D.C.: PERF, 2003), p. 7. 59 Definitions derived from Merriam-Webster Online, at http://www.m-w.com/. 60 Artley, et. al., supra note 14 at pp. 5–6 [citations omitted]. 61 This chart and much of the discussion of accountability levels that follows is taken from Artley, et. al., supra note 14 at pp. 9–10. 62 Los Angeles Police Department, LAPD State of the Department: Plan of Action for the Los Angeles that Is and the Los Angeles that Could Be (Los Angeles: LAPD, 2004), pp. 20–21, emphasis added. 63 Artley, et. al., supra note 14 at pp. 7–8. 64 For more definitions of “data” and “information,” see Merriam-Webster OnLine, at http://www.m-w.com/. 65 Also see Abt Associates, Inc., Evaluating Drug Control and System Improvement Projects, Guidelines for Projects Supported by the Bureau of Justice Assistance (Washington, D.C.: U.S. Department of Justice, NIJ, 1989 (Reprint 1992)), pp. 5–7, at http://www.ojp.usdoj.gov/BJA/evaluation/ guide/documents/chapter_2_nij_guide.htm. For a discussion of police activity measurement, see Dadds, Vikki, and Tammy Scheide, “Police Performance and Activity Measurement,” Trends Page 164 Appendix E and Issues in Crime and Criminal Justice, No. 180 (Canberra: Australian Institute of Criminology, November 2000), at http://www.aic.gov.au/publications/tandi/tandi180.html, and Marx, Gary T., “Alternative Measures of Police Performance” in E. Viano (ed.), Criminal Justice Research (New York: Lexington Books, 1976), at http://web.mit.edu/gtmarx/www/alt.html. 66 Moore and Thacher, et. al., supra note 12 at p. 132. 67 See, e.g., Skogan, Wesley G., “Measuring What Matters: Crime, Disorder, and Fear” in Measuring What Matters: Proceedings from the Policing Research Institute Meetings, Robert H. Langworthy (ed.), NCJ 170610 (Washington, D.C.: U.S. Department of Justice, NIJ and COPS Office, July 1999), pp. 37–53. 68 Prepared by PERF through Cooperative Agreement #98-CK-WX-K001, awarded by the U.S. Department of Justice COPS Office. Cited in: Bynum, Timothy S., Using Analysis for Problem- Solving: A Guidebook for Law Enforcement (Washington, D.C.: U.S. Department of Justice COPS Office, 2001), p. 18, at http://www.cops.usdoj.gov/mime/open.pdf?Item=342. Based on 447 COPS Problem-Solving Partnerships (PSP) project grantees, as reported by author Ann C. Grant of PERF. 69 Bratton with Knobler, supra note 10 at p. 233. 70 See COPS monitoring and auditing requirements at http://www.cops.usdoj.gov/mime/open.pdf?Item=299. 71 For discussion of the technology requirements for CompStat and the operating environment initially present in NYPD, see Henry, supra note 10 at pp. 248–250. 72 Bratton with Knobler, supra note 10 at p. 234. 73 Boba, Rachel, Problem Analysis in Policing (Washington, D.C.: Police Foundation, 2003), p. 32, at http://www.cops.usdoj.gov/mime/open.pdf?Item=847. 74 See, e.g., the extensive array of publications by SEARCH on privacy policy and justice information and criminal history systems at http://www.search.org/services/publications/ default.asp?category=4 and http://www.search.org/services/publications/default. asp?category=9. 75 For more complete definitions, see Merriam-Webster OnLine, at http://www.m-w.com/. 76 Blalock, supra note 19; Glass and Stanley, supra note 19; and Jordan, Gretchen, Steven Prevette, and Steve Woodward, The Performance-Based Management Handbook, Volume 5: Analyzing, Reviewing, and Reporting Performance Data (Washington, D.C.: U.S. Department of Energy, 2001). Also see Eck, supra note 19. Page 165 Endnotes 77 Glass and Stanley, supra note 19 at pp. 271–291. 78 Easton,Valerie J., and John H. McColl, Statistics Glossary, version 1.1, Centre for Applied Statistics, Lancaster University, at http://www.cas.lancs.ac.uk/glossary_v1.1/hyptest. html#hypothtest. 79 Solomon and Uchida, supra note 6 at p. 29. Also see, Mazerolle, Lorraine, Dennis Rogan, James Frank, Christine Famega, and John E. Eck, Managing Citizen Calls to the Police: An Assessment of Non-Emergency Call Systems, NCJ 199060 (revised final report submitted to the U.S. Department of Justice, NIJ, October 2001), at http://www.ncjrs.org/pdffiles1/nij/grants/199060.pdf; and Mazerolle, et. al., Managing Citizen Calls to the Police With 911/311 Systems, supra note 6. 80 For detailed discussion of sophisticated statistical analyses and computations, which exceed the general scope of this Tech Guide, see Blalock, supra note 19, and Glass and Stanley, supra note 19. 81 This, in fact, was one of the findings observed by Mazerolle, et. al., in their analysis of the 3-1-1 Call Center in Baltimore, MD. See Mazerolle, et. al., Managing Citizen Calls to the Police: An Assessment of Non-Emergency Call Systems, supra note 79 at p. 4–7. 82 Ibid., at p. 4–6. 83 Roper and Sullivan, supra note 8 at p. 17. 84 Jordan, et. al., supra note 76 at pp. 41–43. For a particularly insightful discussion of effective and innovative techniques of displaying quantitative information, see Tufte, Edward R., The Visual Display of Quantitative Information (Cheshire, CT: Graphics Press, 2001). 85 Roper and Sullivan, supra note 8 at p. 17. 86 See, e.g., San Diego (CA) Police Department, “Police Department Crime Statistics and Maps,” at http://www.sandiego.gov/police/stats/index.shtml, and Baltimore (MD) Police Department, “Fighting Crime Online,” at http://141.157.54.34/bpdmaps/police.htm. 87 For NYPD CompStat reports, see http://www.nyc.gov/html/nypd/html/pct/cspdf.html, and for current crime statistics from the LAPD, see http://www.lapdonline.org/crime_maps_and_compstat. 88 For more detailed discussion of OLAP technologies, see http://www.olapreport.com/. 89 See “OLAP and OLAP Server Definitions” at http://www.moulton.com/olap/olap.glossary.html. Page 166 Appendix E 90 This draft sample executive dashboard was provided by Colonel Kenneth Bouche, Illinois State Police, June 2005. 91 Bratton with Knobler, supra note 10 at pp. 215–216. Further discussion of the culture of NYPD before and after CompStat, see Henry, supra note 10 at pp. 60–68. 92 Bratton with Knobler, supra note 10 at p. 215. 93 Larry Greenemeir, “Tech vs. Terrorism,” in Information Week, June 6, 2005, at p. 46. See http://www.informationweek.com/story/showArticle.jhtml?articleID=164300083. 94 Loyka, Stephan A., Donald A. Faggiani, and Clifford Karchmer, Protecting Your Community from Terrorism, Volume 4: The Production and Sharing of Intelligence (Washington, D.C.: PERF, February 2005), p. 44, at http://www.cops.usdoj.gov/mime/open.pdf?Item=1438. 95 Ibid., and Holmes, Allan, “Why the g-men aren’t IT men,” in CIO-Asia Magazine, June 15, 2005, at http://www.cio-asia.com/showpage.aspx?pagetype=2&articleid=1612&issueid=52. 96 For more detail regarding performance measurement and integrated justice information sharing initiatives, see Geerken, Michael, Center for Society, Law and Justice, Performance Measurement Tools for Justice Information Technology Projects: Key Issues in Developing Performance Measurement Systems for Justice Integration (Metairie, LA: University of New Orleans, draft, no date) at http:// www.cslj.net/PerformanceMeasure/files/performancemeasurementmonograph.pdf. 97 For more information about the Problem-Oriented Guides for Police series and other COPS Office publications, please call the COPS Office Response Center at 800.421.6770 or visit www.cops.usdoj.gov. 98 See Weisburd, David, Stephen D. Mastrofski, Rosann Greenspan, and James J. Willis, The Growth of Compstat in American Policing (Washington, D.C.: The Police Foundation, 2001); Willis, James J., Stephen D. Mastrofski, David Weisburd, and Rosann Greenspan, Compstat and Organizational Change in the Lowell Police Department: Challenges and Opportunities (Washington, D.C.: The Police Foundation, 2003). 99 For a general discussion of the COMPSTAT process within NYPD, see http://www.nyc. gov/html/nypd/html/chfdept/compstat-process.html. Weekly crime statistics at the city, borough, and precinct levels are posted at http://www.nyc.gov/html/nypd/html/pct/cspdf. html. 100 For general discussion of COMPSTAT within LAPD, see http://www.lapdonline.org/crime_ maps_and_compstat/content_basic_view/6363. Page 167 Endnotes 101 For a fairly detailed account of the COMPSTAT Plus process, see http://www.lapdonline. org/inside_the_lapd/content_basic_view/6364. 102 See http://www.lapdonline.org/crime_maps_and_compstat. 103 The LAPD Plan of Action is available for download at http://www.lapdonline.org/inside_ the_lapd/content_basic_view/6561. 104 http://police.homeoffice.gov.uk/performance-and-measurement/performanceassessment/?version=2. 105 http://police.homeoffice.gov.uk/performance-and-measurement/assessmentmethods/. 106 http://police.homeoffice.gov.uk/news-and-publications/publication/performance-andmeasurement/SPI_Technical_Guidelines_204.pdf?view=Binary, at p. 8. 107 Ibid., at pp. 10–12. 108 See, e.g., Statutory Indicators for 2004, 2005, and 2006 at http://police.homeoffice.gov. uk/performance-and-measurement/assessment-methods. 109 http://police.homeoffice.gov.uk/248458/master_table.pdf. 110 http://www.pajnet.state.pa.us/pajnet/cwp/view.asp?a=213&q=75743. 111 The executive order is available at http://www.oa.state.pa.us/oac/lib/oac/exec_ orders/1999-4.pdf. 112 http://www.pajnet.state.pa.us/pajnet/cwp/view.asp?a=213&q=159659. 113 This information and graphics was taken from Firestone, Chad, “Performance Management,” PowerPoint presentation at the 2006 Symposium on Justice and Public Safety Information Sharing: Effective Decisionmaking for a Safer America, Washington, D.C., March 14, 2006. Available at http://www.search.org/events/past/2006symposium/presentations.asp. 114 http://www.cjit.gov.uk. 115 From Jenner, Stephen, “Realising the Benefits of e-Government—The CJS IT Programme Experience,” PowerPoint presentation at the 2006 Symposium on Justice and Public Safety Information Sharing: Effective Decisionmaking for a Safer America, Washington, D.C., March 14, 2006. Available at http://www.search.org/events/past/2006symposium/presentations.asp. For More InForMatIon: U.S. Department of Justice Office of Community Oriented Policing Services 1100 Vermont Avenue, N.W. Washington, DC 20530 To obtain details on COPS programs, call the COPS Office Response Center at 800.421.6770 Visit COPS Online at www.cops.usdoj.gov