|
|
Office of the Assistant Secretary for Planning and Evaluation |
Policy Information Center |
In this chapter, the U.S. Department of Health and Human Services (HHS) highlights evaluations of general interest to the public health, health care services, and human services community, and illustrates the diversity of HHS evaluations completed in fiscal year (FY) 1999. Included are summaries of six evaluation projects selected by the HHS Evaluation Review Panel, a group of outside evaluation experts. The Panel reviewed 26 reports nominated by HHS agencies and selected the reports to be highlighted in this chapter on the basis of the following criteria:
These criteria are more fully described in the Appendix.
The six studies are organized under two headings: program effectiveness evaluations and environmental assessments. These headings represent two common uses of HHS evaluation resources. Program effectiveness evaluation provides a way to determine the impact of the Departments programs on achieving intended goals and objectives. Environmental assessment is the way we understand the forces of change in the health and human services environment that will influence the success of our programs and the achievement of our goals and objectives.
Young people under 26 may be at risk for HIV and other sexually transmitted diseases and need information on how to prevent it. The Prevention Marketing Initiative (PMI) was a 5-year demonstration of social marketing initiated by the Office of HIV/AIDS of the Centers for Disease Control and Prevention in 1993. Prevention marketing involves social marketing, the behavioral sciences, and community participation. Both local and national in scope, PMI has three major components: local demonstration sites, national health communications, and prevention collaborative partners such as health communication firms.
This study evaluates the implementation phase of local demonstration projects in five sites. During the five years of the project, researchers conducted two case studies involving prevention marketing HIV/STD programs for young people under the age of 26. The first, completed in 1996, reported on the interventions designed by the local sites. The second describes the experiences of the local sites and the collaborative national partners who provided technical assistance during the transition and implementation phases of the project.
The purpose of the PMI demonstrations was to test the efficacy of combining social marketing with elements of behavioral sciences and community participation in reducing the risk of AIDS in young people. The researchers applied the concept of prevention marketing to the demonstration sites as an experiment in demonstrating the prevention marketing process, including the skills and resources needed to engage the community effectively; measuring the behavioral effects of data-based prevention marketing interventions; and documenting the lessons learned that may inform future prevention-marketing efforts. In addition to describing the experiences of the project participants, information in the second case study also will provide context for further evaluation of both outcomes of site-based skills-building workshops and the community-wide level of exposure to media messages.
During the planning of the project, each site considered intervention components previously shown as effective, with each site tailoring intervention components to its target audience and prevention objectives. Sites chose multifaceted interventions using social marketing techniques to reach the targeted population delivered through workshops and outreach components. An initial task was to hire staff and build community participation among key organizations and individuals. One national partner, the Academy for Educational Development (AED), and its partner, Porter/Novell, provided intensive technical assistance, focusing on developing organizational structures, establishing procedures for managing potential resistance to prevention programs, and building community participation.
In the following year, demonstration sites received technical assistance in conducting formative research, considered a critical step in social marketing. It involves an assessment of local needs and resources, helping to define target audiences and behavioral objectives through focus groups and other research, such as community environment and epidemiological profiles. Some sites added development of community networks or the support of community leaders to their plans.
Local sites chose interventions to reduce levels of selected risk behaviors among their target audiences, adapt programs to their local needs, and plan for implementation during the next two years. AED continued to provide technical and managerial support through the end of the fifth year. In the final year of the project, sites focused on fielding their efforts, evaluating them, and planning for continuity of their efforts.
This study had two major objectives: to describe changes in organizational structures and processes that had occurred since the first case study was conducted in 1996, and to document experiences with the major components of PMI during its final two years. The components involved social marketing, community collaboration (including media relations), behavioral science, youth participation, and technical assistance. Collecting and analyzing qualitative data was guided by seven research questions designed to meet the major objectives.
The case study used a qualitative approach to capture the experiences and recommendations of the PMI participants. A two-person field team visited each site and conducted interviews with 64 people. They conducted 10 more interviews with the national partners. The team used these sources of information to evaluate the project: (1) interviews with PMI staff, volunteers, and implementation partners; (2) observations of meetings or activities that occurred during the data collection period; (3) interviews with such national partners as technical assistance providers and project officers; (4) a review of current site-based documents; and (5) the final report for the initial case study. The data collected and analyzed then were used to create a thematic database, case summaries, and the final report representing an integrated summary of findings across all sites.
All five PMI sites contained structural elements that remained the same throughout the project. These included a lead agency, an advisory committee, an on-site staff, a youth committee, and technical-assistance providers. The sites experienced high turnover in organizations and individuals as the PMI project moved from planning to implementation. The lead agency changed at all five sites because of a mismatch of PMI site objectives and those of the lead agencies. The sites selected new agencies to manage implementation of the interventions. The role of the advisory committees changed from planning to project oversight; however, several sites experienced declining attendance and interest when they failed to define a clear role for their advisory committees.
A significant change in the program plan was to engage subcontractors to field the intervention components and evaluate the interventions. One aspect of the intervention implementation seen by respondents as highly successful was the way the various components of programs worked together and complemented each other.
Several forms of evaluation were used to document the effects of PMI, including: (1) community- based telephone interview in Sacramento assessing the scope and effects of the program; (2) a workshop evaluation to assess the effectiveness of the program for participants 3) process evaluation including this case study; (4) monitoring and accountability, and (5) formal and informal evaluations conducted by the sites themselves.
The community-based outcomes study involved a random telephone survey of teenagers inquiring about their attitudes, knowledge gains, and sexual behavior as a result of the PMI project. Sacramento provided convincing evidence that exposure to the social-marketing campaign produced the desired results–increased condom use among sexually active teenagers. More than 60 percent of the teens in the Sacramento target area were exposed to the social-marketing campaign through multiple channels.
The Prevention Marketing Initiative is a model that may be replicated and modified to fit the needs of other communities. The final report lists numerous recommendations based on the findings of the case study and lessons learned from the experiences of the project participants that will be of value to future public health promotion efforts.
AGENCY SPONSOR: Centers for Disease Control and Prevention
FEDERAL CONTACT: Nancy Chalmers
PHONE NUMBER: 404-639-7085
PERFORMER ORGANIZATION: Battelle, Durham, NC
The relationship between the regulation of health claims on food labels and the actual effects on consumer behavior in searching for and using the information provided is the subject of this study. The researchers interviewed more than 1,400 shoppers to assess and record how consumers use food label information appearing in different areas of the label when evaluating a product and making a purchase decision. The results suggested that consumers are likely to read no further than the health and nutrient-content claims on the front of the package and to give greater weight to the information mentioned in them than that contained in the Nutrition Facts panel.
Health claims also were found to produce a halo effect, extending to other health aspects not mentioned in the claim, and, in the case of one of three food products tested, a magic-bullet effect, attributing inappropriate health benefits to the product. The study acknowledges and extends earlier research and discussion of consumer information search behavior and regulation of health claims on food labels by the Food and Drug Administration.
The authors wanted to better understand the extent of consumer reliance on health claims appearing on the front of food packages and their extended search to the back of the package to view the Nutrition Facts panel. The study sought to answer this question: “Do consumers rely only on claims when evaluating a product and making a purchase decision, and if so, how does this affect their nutritional judgments and the quality of their choices?”
The 1990 Nutrition Labeling and Education Act (NLEA) regulates health claims on food labels. NLEA was enacted to “assist consumers in maintaining healthy dietary practices” amid concerns that unregulated health claims might be confusing and deceptive.
Health claims first appeared on Kellogg products in 1984. The number and types of diet and disease linkages used in advertising and on food labels throughout the early 1990s rapidly increased following Kellogg’s initiative. Research indicated that health claims and product advertising since 1984 have increased consumer awareness of diet-disease linkages and given rise to new or reformulated food products with greater nutritional value. Some, however, believed that health claims and product advertising were incomplete or misleading and failed to provide accurate information to consumers about the nutritional content of the product.
The Food and Drug Administration issued implementing rules for NLEA in May 1994, limiting claims to those supported by “publicly available scientific evidence” and the types of products that display health claims to those that meet a set of global nutritional standards. New products following enactment of NLEA tended to be healthier by reducing the level of negative nutrients, such as fat. However, researchers have found it difficult to measure the effects of health claim regulations, given the NLEA requirement that the Nutrition Facts panel appear on food labels. As early as 1973, numerous studies have been conducted using experimental protocols to understand the effects of nutrition information on consumer behavior.
A total of 1,403 primary food shoppers were interviewed at eight United States metropolitan statistical areas located in the Northeast, Southeast, Midwest, and Southwest. Interviews were conducted with each subject following initial screening on the basis of the relative amount of shopping, age, literacy, and
ability to read small print. Participants were presented three-dimensional mock-ups of well-known products (with brand names removed) under three categories of products, cereals, lasagna, and yogurt. The researchers experimentally manipulated an independent variable, creating ten variations in label conditions. Participants were given time to examine the package of each product prior to being asked a set of prescribed questions. In each question series, the researcher observed whether the respondent looked at (a) only the front panel of the package, (b) only the Nutrition Facts panel, (c) both the front and the Nutrition Facts panels; or (d) neither panel. The researchers tested ten separate outcome predictions relating to the effects of the truncated information search and the presence of health or nutrient-content claims on consumer information search and processing behavior.
The results of inferential statistical tests suggested that the presence of a health claim and a nutrient claim on a food package has a significant effect on the relative degree to which consumers limit their search for nutrition facts to the front panel of food packages. Consumers who either truncate their information search to the front or view nutrient claims are significantly more positive in their summary judgments of products. These consumers also are more prone to give greater weight to the information cited in the health claims than to that presented in the Nutrition Facts panel.
Information search behavior affected product ratings and health evaluations, independently from the presence of a health claim. For example, those who look only at the front panel give significantly higher purchase intention ratings than those who consult the Nutrition Facts panel. Researchers discovered this effect to be as large as those associated with product type or education level. They also found that the presence of a claim on a food package significantly influenced the halo effect described earlier, and for one product tested, a magic-bullet effect.
The findings of the study are useful and of interest to several audiences. For the FDA, the findings support the rationale for regulation of health claims. Based on the data, the authors suggest that manufacturers may find it “advantageous to rely more heavily on” nutrient-content claims, because the regulations permit a greater number of products to qualify for nutrient-content than for health claims. The effect of the latter on consumer information search and evaluation were found to be similar to that of health claims. Findings from the study may increase public awareness and encourage consumers to read beyond health claims. It extends previous research efforts and also suggests further avenues for explanation of consumer search and product evaluation behavior.
AGENCY SPONSOR: Food and Drug Administration
FEDERAL CONTACT: Alan Levy
PHONE NUMBER: 202-205-9448
PERFORMER ORGANIZATION: Food and Drug Administration, Rockville, MD
This study investigated the costs and benefits that accrue as the result of substance abuse treatment. Using data from NTIES, the research team constructed and compared estimates of treatment costs, crime-related and health care costs, and the income of 5,264 substance abusers in the periods before and after treatment. The findings of the study indicate dramatic reductions in crime-related costs, modest reductions in health-care costs, and modest increases in the earnings of substance abusers in the period after treatment.
Despite federal expenditure of over $2 million and an overall national expenditure in excess of $12 billion, it is estimated that about 1 million substance abusers in need of publicly supported treatment do not receive services. Policymakers often face the difficult task of justifying to taxpayers the use of public funds to support substance abuse treatment. For the public and policymakers, the relevant question relates to the degree to which substance abuse treatment provides benefits for the rest of society. Thus, the economic soundness of relevant policies depends on the cost effectiveness of treatment. The primary purpose of this study was to assess the costs and benefits of alcohol and drug abuse treatment and determine the resulting economic benefits to society.
One of the major goals of the DHHS Center for Substance Abuse Treatment (CSAT) is to improve the lives of those affected by alcohol and other substance abuse, and through treatment, to reduce the adverse effects of substance abuse on individuals, families, communities, and the society at large. Toward this goal, CSAT has invested and continues to invest significant resources in the development and provision of effective substance abuse treatment and recovery services. In 1996, Federal expenditure accounted for almost $2.7 billion of the about $12.7 billion national expenditures for the diagnosis and treatment of alcohol and drug abuse. Issues related to the investment in or continuation of existing programs and the development of new programs generally center on their efficacy and the extent to which they are cost effective. The collection and systemic analysis of high-quality data about substance abuse treatment services, clients, and outcomes provide sound scientific evidence upon which to address such questions as which treatment approaches are cost-effective methods for curbing addiction and addiction-related behaviors and what kinds of treatment work for what groups of clients. The National Evaluation Data Services (NEDS) project was established to provide CSAT with data management and scientific support services across an array of programmatic and evaluation activities. One of the specific objectives of the NEDS project is to provide CSAT with the capability to use existing data to address policy-relevant questions about substance abuse treatment. The investigation of the costs and benefits of alcohol and drug abuse treatment and the economic benefits to society was undertaken in pursuit of this objective.
The research task for this study was accomplished by analyzing NTIES “client-level” data and service delivery (SDU) administrative data collected between FY1992 and FY1995 on 5,264 clients who were in treatment demonstration programs supported by CSAT. Using SDU’s administrative/ service data, the research team constructed estimates of treatment costs per client for each of the modalities and across all modalities. To measure benefits, the research team estimated and compared the average costs per client, in terms of crime-related and health-care cost associated with substance abusers, and the income and social welfare benefits received by substance abusers in the 12 months prior to and after treatment. The difference between pre-treatment and post-treatment costs was used to estimate the economic impact of substance abuse treatment.
The analysis of data revealed: 1) approximately 75 percent reductions in crime-related costs; 2) 11 percent reduction in health-care costs per client; and 3) 9 percent increase in the earnings of substance abusers in the period after treatment. No change was found in welfare payments and Supplemental Social Security Income. Overall, the results obtained indicate that treatment created an average benefit to society of $9,177 per client and an average benefit to the non-treated population of $12,477 per client. Deducting the cost of treatment per client resulted in average treatment benefits of $6,236 and $9,536 per client for society and the non-treated population, respectively.
The research team concluded that, based on these results, the total benefits of substance abuse treatment, in terms of avoided costs and increased earning, far exceeded the costs of treatment. As a recommendation, the researchers indicated that the next steps should include analyses to identify what kinds of treatment works for whom. The team intends, in future work, to link intensity and cost of services to outcomes to identify the cost effectiveness of different treatment services.
Because of the large numbers of incomplete responses to cost questions in the baseline and administrative survey questionnaires, and discrepancies between total revenues and sum of details reported by SDU's the researchers choose to proxy treatment costs with information on revenues. Nevertheless, there is no evident reason to believe that the outcomes of the study would be any different, especially in light of the special steps the researchers took to ensure the reliability of the information derived.
AGENCY SPONSOR: Substance Abuse and Mental Health Services Administration
FEDERAL CONTACT: Ron Smith
PHONE NUMBER: 301-443-7730
PERFORMER ORGANIZATION: Caliber Associates, Fairfax, VA
Analysis of Prescription Drug to Over-the-Counter (Rx-to-OTC) Switch Movement
This study identified the trend and some possible reasons for the Rx-to-OTC switch movement; estimated the price and income elasticities of OTC switch drugs; and described the relationship between the demand for OTCs and select market and demographic variables. The study also examined the impact of specific Rx-to-OTC switches on clinical practice patterns, costs and benefits associated with the OTC switch movement, and the levels and types of Medicaid coverage for OTCs in different states. To conduct the study, the researchers reviewed the literature, and conducted key informant group interviews and analyzed select data sets. The findings of the literature review indicated that a number of forces have had an impact on the Rx-to-OTC movement: the market expansion motives of pharmaceutical companies, the trend toward deregulation. the individual autonomy/self-help movement, and health care containment efforts. Key informant interviews revealed that the switch trend was viewed as a positive one. Other analysis revealed a general increase in the demand for OTCs; higher benefits associated with the OTC switch relative to costs, and different types and levels of coverage of OTCs in Medicare Plans, from state to state. The study enabled the researchers to identify a set of questions requiring research attention.
This study focused on the impact of “switched drugs” and the switch trend–that is, prescription drugs (Rx) which have been switched to over-the-counter (OTC) status, with emphasis on more recent switches and potential switches. The central aim of the study was to provide a comprehensive review and analysis of the Rx-to-OTC switch movement in order to inform policymakers and define relevant research questions.
Approximately 600 over-the-counter (OTC) products currently available use ingredients and dosages available only by prescription 20 years ago. While there has been a steady stream of OTC switches since the mid-1970s, the number of switches has accelerated in recent years. Between 1988 and 1994, there were 14 switches, while in the last three years thee were at least 19 switches. Some of the possible reasons cited for this trend include: 1) a growing emphasis on individual autonomy and self-help; 2) a trend toward deregulation in the U.S.; 3) health care cost containment efforts; and 4) pharmaceutical industry self-interest/profit. The issue of Rx-to-OTC switching deserves considerable attention because of the large number of people who could be affected by the trend, including consumers, pharmaceutical companies, physicians, pharmacists, and payers.
The project began with a review of the literature related to the Rx-to-OTC switch movement. Based on the results of the literature review and discussions with the Research Advisory Group, an interview guide was constructed and administered to seven key informants representing pharmacists, consumers, policymakers, physicians, pharmaceutical companies, and payers. The interview guide covered three areas related to the switch movement: understanding the trend, understanding the process for switching drugs, and policy issues. Sales data for three recent switch drugs (H2 blockers, vaginal anticandidals, nicotine replacement products) were analyzed to examine the impact of specific Rx-to-OTC switches on clinical practice patterns. Focusing on the OTC availability of vaginal anticandidals, the research team identified and compared the benefits and potential negative outcomes associated with the OTC switch, taking into account the likelihood of their occurrence. In addition, the team examined epidemiological and medical literature to ascertain what researchers do or do not know about occurrence and treatment of vaginitis. Using data on state Medicaid plan coverage, the researchers examined the levels and types of OTC coverage provided in different states, and compared the utilization of “alternative” prescriptions and overall costs per beneficiary for plans that do and do not cover specific OTCs.
The literature review identified the market expansion motive on the part of pharmaceutical companies; the individual autonomy/self-help movement; the trend toward deregulation in the U.S.; and health care cost containment efforts as four major forces responsible for the Rx-to-OTC switch movement. In general, key informant interviews revealed that: 1) the switch trend was viewed as a positive one, providing increased access to safe and effective drugs and cost savings, primarily through reduced office visits and work time lost; 2) the FDA process for reviewing potential switches was seen as adequate and the number and types of switches as appropriate; 3) drug safety and efficiency profiles as well as the ability of individuals to appropriately self-diagnose and self-medicate were seen as the critical elements in the timing of an OTC switch; 4) the best types and levels of OTC information and the translation of that information into consumer knowledge of appropriate OTC use wee cited as critical policy issues; and 5) insurance coverage of OTCs was not seen as an issue of significant policy concern at this time.
Analysis of the demand for OTC switch drugs (H2 blockers, nicotine replacement medications, vaginal anticandidals) revealed that: 1) the demand for H2 blockers and vaginal anticandidals was relatively elastic (i.e., sensitive to price), but the demand for nicotine replacement medications was relatively insensitive to price; 2) increases in income led consumers to purchase more of the OTC products; 3) lower income populations were more to OTCs; 4) OTCs may actually serve as a substitute for conventional medical care (physician and hospital care). Results of estimations of individual demand for specific OTC brands within the three categories were generally consistent with the researchers expectations that these OTC markets are characterized by monopolistic competition.
The results of the analysis of the impact of OTC switch drugs indicate that: 1) the average number of vaginitis visits per woman decreased significantly in the post switch period (1990-1994); 2) the OTC switch of the more potent version of hydrocortisone (1%) had little impact if any on the number of patients seeking physician care for dermatitis complaints; 3) prescription rates for clotrimazole and miconazole decreased significantly after these medication switches to OTC, but there did not appear to be a significant increase in the prescription rates for other “substitute medications,” 4) prescription rates for hydrocortisone fell after the medication went OTC; and 5) the only related dermatitis drugs that were prescribed significantly more in 1990 and 1985 were those used for the treatment of acne. The study revealed that the estimated benefits of OTC anticandidal availability per person ($61.96) outweighed the estimated costs ($3.85).
With regard to trends in Medicaid coverage of OTCs, the study revealed that: 1) the number of OTCs covered varied significantly (from 0 to 408) from state to state; 2) the average level of (Medicaid) OTC expenditures per state in 1997 was $296 million; 3) OTC expenditures represented approximately three percent of Medicaid pharmaceutical budgets; 4) in general, Medicaid cost per beneficiary for OTC substitute drugs did not appear to differ significantly between states that do or do not cover OTC vaginal anticandidals; and 5) Medicaid OTC costs rose 38 percent between 1994 and 1997, and increased slightly less than the increase in pharmaceutical costs per beneficiary (48%) over the same period.
Consistent with the exploratory nature of this study and the expectations of the researchers, the study generated several questions, such as: How do consumers make decisions to purchase OTCs versus seeking professional care? How do clinicians treat patients in the presence of OTC products and lack of insurance coverage? What is the impact of health outcomes? Is it cost effective for health care plans to cover OTCs? How will decision-making change as more products become OTC? These questions and others raised in the study and the gaps in our knowledge of anticandidals self-medication are some of the areas that must be addressed with further research.
AGENCY SPONSOR: Office of the Secretary, Office of the Assistant Secretary for Planning and Evaluation
FEDERAL CONTACT: Laina Bush
PHONE NUMBER: 202-260-7329
PERFORMER ORGANIZATION: Northwestern University, Evanston, IL
Under most Medicaid managed care plans, beneficiaries who do not select a plan are automatically assigned to one. In some states, as many as 60 to 75 percent of these people may have been automatically enrolled. Concerns about this high rate of automatic assignment and whether the enrollees were making an informed choice prompted this study, which describes the effects of enrollment and autoenrollment (automatic assignment) policies and practices under mandatory Medicaid managed care on federally qualified health centers (FQHCs), FQHC networks and their plans, and their patients.
This study, the second phase of a two-part research project, examined the enrollment policies and practices under mandatory Medicaid managed care (MMMC) in nine states, and the experiences of Medicaid beneficiaries enrolling in managed care. Researchers also assessed the effects of MMMC enrollment policies on federally qualified health centers (FQHCs) and their ability to survive managed care. The first phase of the study grew out of the desire to gain greater understanding of possible causes for the high rates of auto-enrollment (mandatory assignment to a health plan when beneficiaries do not voluntarily enroll) in state Medicaid managed care programs. Because Medicaid managed care enrollment is one of the least well understood aspects of managed care, researchers wanted to improve the level of understanding about emerging issues and identify implications for access and quality.
The number of Medicaid beneficiaries enrolled in mandatory managed care programs has grown rapidly since 1990, creating a need to learn more about the dynamics of all facets of managed care. There is an evident need to develop a better understanding of how various state enrollment policies are constructed, how the enrollment practices actually work, and the significance of these policies and practices for beneficiaries, plans, and providers.
Concerned about the high rates of automatic enrollment in Medicaid plans, researchers reviewed policies and practices in 34 states with MMMC programs and found that they varied greatly, with no discernible association between particular state autoenrollment policies and high levels of autoenrollment. Medicaid beneficiaries have little or no knowledge about managed care and state agencies have limited knowledge and experience with managed care delivery systems and are also hampered by budget constraints. How people are enrolled in Medicaid has important implications for them, including access to and continuity of care, the stability of their existing relationships with providers, and their overall satisfaction with managed care. Mandatory enrollment also affects healthcare providers and may help determine the structure of the managed care market. The way States approach mandatory enrollment may influence market entry by various types of managed care plans as well as their stability and survival.
The methodology for the project involved a series of case studies conducted in nine states. Researchers analyzed numerous descriptive variables when selecting states from which to draw the case studies among the 34 with significant Medicaid managed care activity. The variables included autoenrollment rates, use of enrollment brokers, how long MMMC had been implemented, and FQHC’s experience with changes in patient volume and revenue. The researchers used focus groups to gain a picture of the enrollment experiences of Medicaid beneficiaries under their states’ Medicaid managed care programs. They also examined the experiences of providers which traditionally serve beneficiaries, particularly federally qualified health centers, because they are especially challenged by MMMC. The team constructed five study questions with subquestions to address the study objectives and to identify major areas of inquiry for the research. It also conducted structured interviews onsite with state Medicaid officials, managed care plan officials, FQHC representatives, Medicaid beneficiaries, and community advocates.
Findings
Seven major findings emerged from this study:
Potential uses of the findings are to: Improve the quality of information on plans and providers given to beneficiaries to help them voluntarily choose a plan; Develop strategies for improving the enrollment process; Identify common interests among the states, managed care plans, FQHCs, and beneficiaries; Generate collaborative problem solving between public and private policymakers.
AGENCY SPONSOR: Health Resources Services Administration
FEDERAL CONTACT: Dana Jones
PHONE NUMBER: 301-594-4058
PERFORMER ORGANIZATION: George Washington University, Washington, DC
If drug abuse among criminal offenders is to be successfully tackled as a vital issue on the national agenda, policymakers need the best available data. In an effort to provide better estimates of national drug use among arrestees, the researchers in this study discovered that multiple capture rates (the rates at which individuals appear in repeated arrests on a database or are “rearrested”) were significantly higher than those reported in earlier literature on the subject. In this study, the investigators estimated drug use prevalence for various sizes of arrestee populations using state and national samples; evaluated the applicability to large-scale data sets, as well as to various subgroups, of a previously developed regression model for synthetic estimation; examined the predictive power, sensitivity, and robustness of the model; and conducted testing to increase the accuracy of model estimates. The overall aim of the research was to obtain better estimates of drug use prevalence to lend greater scientific support to policy decisions and allocations of resources. The researchers applied the direct projection, bootstrap, and logistic regression synthetic estimation techniques using the national Drug Use Forecasting (DUF) project data, California DUF-like survey data, national FBI Uniform Crime Report (UCR) data, the California Monthly Arrest and Citation Register (MACR) data, and the 1990 U.S. Census and 1996 Current Population survey information to produce scientifically defensible estimates of drug use prevalence among arrestees. For example, the study indicates that of the 1.17 million adults arrested in California in 1996, more than a fourth were arrested for drug-related offenses, and 65 percent had used drugs before their arrest.
Policymakers need reliable information about drug use among criminal justice populations to more effectively respond to problems associated with drug use and crime. The need for improved drug use prevalence estimation techniques has been emphasized by several government agencies. The previous work of researchers from the UCLA Drug Abuse Research Center (DARC), including an initial study of regression modeling for drug use prevalence estimation, resulted in the development of a regression approach to synthetic estimation of drug use prevalence. This comparative follow-on study was conducted to: 1) estimate drug use prevalence for various sizes of arrestee populations using data from three separate studies of California, Texas, and national samples; 2) evaluate the applicability of the previously developed regression model for synthetic estimation to large-scale data at local, state, and national levels, as well as to various subgroups defined by gender, race/ethnicity, age, and type of offense; 3) examine the predictive power, sensitivity, and robustness of the regression model for synthetic estimation; and 4) determine overall and subgroup multiple capture rates (repeated arrests) to increase the accuracy of model estimates.
The high number of drug users in the criminal justice population and the immense costs and consequences of drug-related activities to society have been central topics of recent policy discussions. Studies have shown that the prevalence of drug use among arrestees is far greater than expected on the basis of estimates derived from general population studies. Policymakers have emphasized the need to obtain better estimates of drug use prevalence to scientifically support policy decisions and allocation of resources. Without enumeration of actual drug use among criminal justice populations, policymakers must base their decisions on reliable estimates obtained from comprehensive approaches which compensate for incomplete information and encompass various drug-related phenomena. Typical applications of the synthetic estimation methods have been limited by the lack of consistent empirical evidence of relationships between drug use prevalence and the indicators selected as covariates for model inclusion. To improve prevalence estimation methods, the UCLA Drug Abuse Research Center (DARC) conducted an initial study of regression modeling to estimate drug use prevalence among arrestees, resulting in improved conceptualization and preliminary application of a regression model. This study is part of an ongoing effort to evaluate the applicability of the flexible regression model to small-scale, relatively large-scale, and large-scale data.
The project performed secondary analysis on data from five major sources: the national DUF (ADAM) project, the California DUF-like survey data, the national Uniform Crime Report (UCR) data provided by the FBI, the California Monthly Arrest and Citation Register (MACR) data, and the 1990 U.S. Census and 1996 Current Population Survey information. The national and California DUF data served as the calibrator to estimate the number of drug users among arrestees in the national and California MACR data. Prevalence estimation at the county, state, and national levels were generated in three relatively independent studies for California, Texas, and the nation. To yield prevalence estimates, the research team used a three-step logistic regression model and estimation procedure. First, they conducted a separate logistic regression analysis using DUF data as the calibrator for each of the 96 subgroups defined by gender, race/ethnicity, age, and crime type. Second, they used the regression coefficients estimated from the logistic regression to derive drug use rates at county, state, and national levels. Finally, they obtained estimates of the number of drug-using arrestees in all subgroups at county, state, or national levels by multiplying derived prevalence rates by the number of arrestees in county, state, or national arrestee populations. For comparison purposes, they used two other methods in addition to logistic regression: 1) Direct projection of drug use rates from DUF survey data to calculate drug use prevalence in arrestee populations represented in state-wide or national arrest data; 2) The Bootstrap Method to estimate drug use rates and their standard errors from the DUF survey data to estimate drug use prevalence in larger arrestee populations.
A major finding of this study was that multiple capture rates (the rate at which individuals appear multiple times in the same database, or are “re-arrested”) were significantly higher than those reported in the literature. Moreover, multiple capture rates dramatically differed by gender, race/ethnicity, and crime type. The overall recapture rate among California arrestees in 1996 was about 30 percent. The overall recapture rates were highest for drug-related offenses. The study showed that 1.27 million adults were arrested 1.64 million times in California, in 1996. More than a quarter of the total number were arrested for drug-related offenses, and 65 percent had used drugs within three days of their booking date. Among the number of those arrested in California, about 770,000 were projected to be drug users. The study showed that 863,000 adult arrests were reported in Texas in 1996. Based on the regression analysis performed, the projections for adult arrestees and drug users was 13 percent higher than those produced by the bootstrap and direct projection methods. The analysis of national-level data revealed that in 20 of the 23 DUF sites, more than 60 percent of adult male arrestees in 1996 were positive for at least one drug; in 21 of 23 sites, more than 50 percent of female arrestees tested positive for at least one drug.
The study indicated that the regression model needs to be further expanded to include a wider range of drug abuse indicators and associated variables to stabilize estimations at various administrative levels and for special groups.
The study provided compelling evidence that regression models of synthetic estimation are sensitive to, and capable of, high-confidence estimates of drug use prevalence at local and state levels for many different subpopulations, as well as for the nation as a whole. The clear, detailed findings related to estimates of drug-using arrestees and estimates for gender, race/ethnicity, age, and crime-type subgroups are very likely to be useful to policymakers and researchers who want a comprehensive picture of the prevalence of drug use among adult arrestees.
AGENCY SPONSOR: The Substance Abuse and Mental Health Services Administration
FEDERAL CONTACT: Ron Smith
PHONE NUMBER: 301-443-7730
PERFORMER ORGANIZATION: Caliber Associates, Fairfax, VA