Richard Barth, Eleanor Locklin-Brown, Stephanie Cuccaro-Alamin,
and Barbara Needell
[ Main Page of Report | Contents of Report ]
Printer friendly version (PDF format)
The Personal Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) significantly altered the way the United States provides assistance to its neediest citizens. The act eliminated the federal entitlement to services that existed under the Aid to Families with Dependent Children (AFDC) program and replaced it with the block grant program Temporary Assistance for
Needy Families (TANF). TANF provides temporary financial assistance while recipients make the mandatory transition from welfare to work. These efforts to move adult recipients toward self-sufficiency may have consequences for the well-being of their children (Collins and Aber, 1996; Zaslow et al., 1998). Potential implications include changes in children's health, safety, education, and social competence.
Whether these consequences are positive or negative depends on how reforms impact family income, parenting practices or parental stress, and access to services (Collins and Aber, 1996; Zaslow et al., 1998). For example, economic hardship related to loss of benefits or other supports may complicate families' efforts to provide supportive environments for their children (Knitzer and Bernard, 1997). Increased parental stress related to economic, employment, or child care difficulties also may lead to poor parent/child interactions or exacerbate existing mental health conditions such as depression or substance abuse, thereby increasing the risk of negative outcomes (Knitzer and Bernard, 1997; Zaslow et al., 1998). Child health and safety also might be compromised if TANF alters access to non-TANF services such as health and childcare.
In contrast, positive changes in these areas may be beneficial to children and families (Collins and Aber, 1996; Zaslow et al., 1998). Specifically, policy changes might lead to improved outcomes for children whose parents become employed successfully. Children might benefit from access to more comprehensive health care, opportunities to observe parents coping effectively with work demands, higher educational aspirations and achievement, and exposure to parental peers who are engaged in more prosocial activities.
PRWORA currently is being praised by some, and criticized by others, for moving nearly 1.7 million recipients from welfare to work. However, until the impact of these reforms on child well-being is known, such celebrations are premature. Even if reforms succeed in moving mothers from welfare to work, if this in turn has negative consequences for children, its effectiveness will need to be reevaluated in light of these costs.
Prior to passage of welfare-to-work legislation, more than 40 states received waivers to experiment with welfare-to-work programs. Experimental evaluations of these initiatives now under way will provide valuable information about possible effects of certain aspects of PRWORA. Most, however, focus primarily on adult outcomes such as changes in income, employment, family formation, and attitude, and cover only a limited number of child outcomes (Research Forum on Children, Families, and the New Federalism, 1999; Yaffe, 1998). Additionally, those child outcomes "typically lack depth and uniformity" (Yaffe, 1998). Several states (e.g., Connecticut, Florida, Iowa, and Minnesota) are looking at child outcomes resulting from parental participation in AFDC waiver conditions that approximate TANF and that eventually will yield child-level outcome data. The Administration for Children and Families (ACF), Office of the Assistant Secretary for Planning and Evaluation (ASPE) in the Department of Health and Human Services has implemented the Project on State-Level Child Outcomes to assist waiver states in using administrative data to expand these child outcome measures and make them comparable across states. Although this uniformity will allow for the assessment of different state models, their utility is limited by their small sample sizes. In particular, small sample sizes make subgroup comparisons difficult and prohibit evaluation of rare events such as foster care placement or child mortality. Current evaluations of state welfare-to-work initiatives under PRWORA suffer from similar limitations. Given these limitations, researchers increasingly are turning to administrative data to try to gauge the relationship between receiving income assistance services under TANF and child well-being.
The purpose of this paper is to assist researchers in addressing the following questions:
Data about social welfare programs for children are fragmented and incomplete and lack a cohesive framework or policy ownership. Given this, it is important to identify the priority of policy issues and the specific research questions about PRWORA and child well-being when research is conducted. These early steps will assist in then defining the population of interest, the appropriate methodology, and the data sources available (National Research Council, 1999).
Several key policy issues have been identified for states examining the impact of welfare reform (Child Trends, 2000a; National Conference of State Legislators, 1999). First is the need to understand the specific components of an individual state's welfare reform program. This is especially important given the diversity across states in the implementation of PRWORA and corresponding use of TANF funds. Specific components left up to the states include mandated work, time limits, and sanctions. The implementation of these components may impact the outcomes for families and children; for example, mandated work without childcare may lead to cases of child neglect. Also needed is an examination of the interaction between welfare reform--that is, the change from AFDC to TANF--and other social welfare programs such as Medicaid, Children's Health Insurance Program (CHIP), the Food Stamps Program, Special Supplemental Nutrition Program for Women, Infants, and Children (WIC), Child Protective Services, and foster care, which also may be undergoing changes. Although efforts over the past two decades have sought to delink AFDC to other social welfare programs, in practice and for individual families these programs remain interwoven. The eligibility, availability, and accessibility of these supplemental services can impact family outcomes in conjunction with, or separate from, TANF enrollment, exit, and reentry. Understanding TANF's impact on children requires thoughtful specification of the child well-being outcomes TANF might influence so that investigators can fashion data collection efforts to maximize measurement of the predicted impact. Research about child outcomes can guide questions about current implementation and future programming using the flexibility of TANF funds.
Child Trends, a nonprofit, nonpartisan research center, along with colleagues at the federal and state levels participating in the Project on State-Level Child Outcomes, have offered a conceptual framework that organizes and clarifies the pathways through which welfare reform can impact children. This framework is displayed in Figure 10-1.
In this conceptual framework, state policies regarding PRWORA will directly effect family income, employment, family formation outcomes such as marriage or out-of-wedlock births, and attitudes of the adults in the case, such as self-esteem or feelings of being in control of one's life. Changes in these four areas will further influence the family and child's psychological well-being, the stability of the child's home, involvement of the absent parent (through child support enforcement), use of health and human services, and consumption of goods and services spent on the child. As a single mother copes with job training, new employment, transportation problems, childcare dilemmas, and confusion about eligibility for supplemental services, the child's physical home environment, relationship to the parent, and time with the parent can be expected to change. Such changes in the capacity and day-to-day schedule of maternal caretaking very likely will, in turn, impact the child's educational experiences, health and safety, and social and emotional adjustment.
This conceptual framework evinces the multifaceted ways that welfare programs can influence a range of child outcomes. The framework offers a critical organizing logic about how welfare reform impacts children, one that can be used to shape research agendas and frame research questions. The framework was developed to guide survey researchers studying about welfare, but it also serves as a useful reference for the inclusion of administrative data in research initiatives. Administrative data are far more useful for estimating some components of this model than others. Specifically, administrative data rarely contain in-depth information about parental psychological well-being or home environment, parenting practices, or social and emotional adjustment of children. Survey research is much better suited to these components. Administrative data can, however, be used to estimate family stability and turbulence through the inclusion of information about household changes and movement in and out of foster care. Administrative data also can help explain the changes in the utilization of supplemental health and human services--especially Medicaid, CHIP, Food Stamps, or WIC--which could be an indirect result of welfare changes. Finally, administrative data have the potential to measure health and safety outcomes, including abuse and neglect, injury, and mortality.
The remaining sections of this paper are developed to assist researchers in defining indicators for domains of child well-being and to clarify substantive issues that must be considered in applying these indicators toward addressing key evaluation questions about the impact of welfare reform. The paper is divided into sections roughly corresponding to the constructs and domains of child well-being offered by Child Trends (1999): (1) health, (2) safety (child welfare), (3) education, and (4) social and emotional adjustment (juvenile justice). For each domain we will identify information that directly describes or reflects on child well-being. Each section also includes a description of key data that are available to inform us about children's outcomes. For each domain we discuss exemplary efforts to use these data for evaluation of welfare reform. Finally, we conclude by discussing some of the scientific sensibilities that should be respected in the use of such data during research on welfare reform, including a discussion of linkages between population surveys and administrative data.
Access to health care services is a central consideration in the assessment of welfare reform, as these reforms change existing relationships among income, employment, and insurance of health care services for poor families and children (Child Trends, 2000a; Darnell and Rosenbaum, 1997; Moffitt and Slade, 1997; Schorr, 1997). Measures of child health typically emphasize access to care as an important measure, recognizing that health care is necessary but not sufficient for positive child health outcomes (Gortmaker and Walker, 1984; Margolis et al., 1997; Andrulis, 1998).
Parents access health care for their children through several paths. Many children receive health insurance provided by their parent's employer. However, some children of working parents may not have employer-sponsored health plans, and children of nonworking parents certainly do not have this benefit. These children of low-income or nonworking parents are eligible for services paid by publicly funded programs such as Medicaid or the new CHIP. The PRWORA legislation did not significantly alter Medicaid eligibility, and CHIP is designed to reach more of these uninsured children. Yet in July 1999 an estimated 4.7 million uninsured children were eligible for Medicaid but not enrolled (Families USA, 1999). Many states are beginning to track children's enrollment in Medicaid and CHIP, implement outreach efforts to increase CHIP enrollment, and expand Medicaid and CHIP income-level guidelines (Families USA, 1999; Children's Defense Fund, 1998).
The actual health services the child receives are also major determining factors in child health status. Examples of services may include (1) preventive care such as immunizations or dental care; (2) diagnostic screening such as vision and hearing screening, or weight for height measures; and (3) treatment for chronic conditions and disability, with corresponding risk of secondary disability. State policies about welfare reform have the potential to change, positively or negatively, the family environment where health behaviors and health decisions are carried out (Willis and Kleigman, 1997; O'Campo and Rojas-Smith, 1998; Brauner and Loprest, 1999). For example, even if a child is enrolled in Medicaid or CHIP, PRWORA work requirements may constrain a parent's ability to access health care. When access to health care services is limited, either through limited availability or limited utilization of services, children's health could suffer. Alternatively, the work requirements could encourage the parent to secure a job that includes health insurance (gaining access to health care), which may mean the family is able to utilize more services.
Access and utilization of services are interesting for evaluation purposes because they are believed to contribute to the actual health of the child. However, direct measures of child health outcomes are also needed to measure the effects of welfare reform on children. Direct measures of child health outcomes are scarce, however. Often researchers have to rely on indicators of health status. Recent discussions about welfare reform and health suggest some indicators to measure child health status. Children in poverty are more likely to be undernourished, iron deficient, or lead exposed (Geltman et al., 1996). Several measures such as infant mortality, injury, and the use of preventive medical services can be good indicators of child health status (Pappas, 1998). Starfield's Child Health and Illness Profile (Starfield et al., 1993) combines several of these indicators into a bio psycho social developmental assessment but is not found in administrative data sets. Even in survey research, questions about child health status may be limited to asking parents to rate their child's health from excellent to poor (Child Trends, 2000b). Thus, when using administrative data about child health status, it is often necessary to use measures of health services as markers for positive outcomes such as immunizations, enrollment in health plans, or preventive screening, along with indicators of actual outcomes such as infant mortality, low birthweight, blood lead levels, or adolescent substance abuse.
Our purpose here is to identify a reasonably comprehensive set of child health indicators available in at least some administrative data that are relevant to changes in welfare policy because they address health access or status of children. Healthy People 2000, an initiative begun in 1990 by the U.S. Department of Health and Human Services, set health objectives for the nation, including child health status objectives (National Center for Health Statistics, 1996). Over the years, the initiative has prompted state and local communities to develop their own similar objectives and indicators of progress toward achieving them. As a result, the Healthy People 2000 effort has created a set of fairly common measurements of child health across a range of public and private health programs. For example, one of the Healthy People objectives is to reduce infant mortality. This supports the inclusion of infant mortality reduction as part of most state health objectives, and as part of many state and local programs targeted toward women and children. At the federal level, the Maternal and Child Health Bureau (MCHB) identified 18 of the Healthy People 2000 objectives that specifically relate to women and children. Of these, 15 are child health status indicators that can be used to measure impact of welfare reform (Maternal and Child Health Bureau, 1996). Table 10-1 presents these indicators, along with several others, as recommendations for measuring utilization of health services as well as child health status. For each indicator, we describe whether data generally are available at the individual level or aggregated to some larger population. We also identify suggested data sources for these indicators. Many of these data sources are being used in current research about child health (Vermont Agency of Human Services, 2000; Child Trends, 1999).
Of the data sources identified in Table 10-1, the core indicators come from Medicaid and vital statistics. The following two sections discuss these two sources of data, how they can be used in studies of welfare reform outcomes on children, and some methodological issues in their use.
| Indicator | Level | Data Sources |
|---|---|---|
| Medicaid eligibility/enrollment/services | Individual | Medicaid data files |
| CHIP eligibility/enrollment/services | Individual | CHIP data files |
| Number/percent uninsured | Population | State dept. of insurance |
| SSI benefits | Individual | SSA data |
| Infant mortality | Individual | Vital statistics |
| Low birth weight | Individual | Vital statistics |
| HIV infection among women with live births | Individual | Vital statistics |
| Prenatal care | Individual | Vital statistics |
| Newborn screening | Individual | Vital statistics Birth defects registry State data system for newborn screening |
| Early and Periodic Screening, Diagnosis, and Treatment (EPSDT) | Population | Medicaid services/payment data |
| Identification of hearing impairments | Individual Population |
State data system for newborn screening Program evaluation data |
| Immunizations | Individual Population |
Medicaid, state immunization registry Program evaluation data |
| Blood lead levels | Individual Population |
EPSDT, clinic record Program evaluation data |
| Dental caries | Individual | Medicaid Public health dept. |
| Unintentional injuries | Individual | Vital statistics, hospital discharge
School-Based health centers |
| Child homicide | Individual | Vital statistics |
| Adolescent suicides | Individual | Vital statistics |
| Adolescent substance use rates | Individual Population |
Vital statistics Hospital discharge/health deptartment School-based health centers Program evaluation data |
| STD rates among youth | Population | Hospital discharge Program evaluation data School-based health centers |
| Adolescent pregnancy rates | Individual | Vital statistics |
Data from Medicaid eligibility, enrollment, and claims and the new state CHIP can be linked to provide longitudinal tracking of a child or family's health care services or lack of services. For example, a state could track the Medicaid or CHIP enrollment of a child whose mother left AFDC. Since it is unlikely that many families leaving TANF will promptly go to jobs with sufficient health benefits or wages above the Medicaid and CHIP guidelines, that is, 200 percent of the federal poverty level in most states but 350 percent of the federal poverty level in some Medicaid expansion programs, lack of Medicaid coverage of a child in an AFDC/TANF leaver family may indicate that the child is at risk of having no health care coverage. If Early and Periodic Screening and Diagnostic Testing (EPSDT) services also are recorded in the Medicaid files, similar linkages with welfare data will allow tracking of the utilization of preventive services for these low-income children. Linked administrative data from AFDC/TANF and Medicaid have also been used as the sample frame for complementary survey research, which can gather indicators of health status or measures of health care utilization and provide more in-depth measures. For example, the Next Generation, a project conducted by Manpower Demonstration Research Corporation (2000), will use survey data from 10 studies to obtain a more comprehensive perspective about the effects of welfare reform on health outcomes. Variables about health will be measured through survey questions, but the project also will include the existing administrative data used in each of the 10 studies.
Using administrative data from Medicaid and CHIP (or other health-related supplemental services such as the Food Stamps Program or WIC) requires attention to a variety of considerations.
One must consider the populations in the data sets in relation to the population of interest for the study. Specifically:
A thorough understanding of the administrative data being used is necessary.
Data linkage and confidentiality issues also arise:
In addition to these concerns about administrative data, identification of the relevant research questions is critical in guiding the analysis plan and selection of relevant data sets. The question of whether regulations make health care services available to all children who need them could be answered with eligibility data. The question of whether children leaving TANF continue to get needed health services cannot be answered with eligibility or enrollment data. That question only can be answered with service utilization data. The question of whether children exiting TANF are continuing to get timely immunizations could be answered by Medicaid services data or by separate immunization registries within a state (Child Trends, 1999).
Another relevant research question to include would be whether the population of cases had changed since PRWORA was enacted. Will you study AFDC populations before PRWORA, or just those TANF cases after the legislation was implemented? This would require including AFDC and TANF cases in the research. Beyond analysis of the data about AFDC/TANF and Medicaid, the Food Stamps Program, or WIC, research should include questions about barriers to supplemental services for families exiting welfare. One possible barrier is the continued linking of welfare to these supplemental services, despite efforts over the past decade to delink regulations about the programs. In practice, and for individual families, these programs remain interwoven. Another barrier is the complicated eligibility rules for services to support families leaving TANF and the media about the program that might affect whether families think they are eligible or not (Ellwood, 1999). Finally, a research question of interest would be "Upon exiting TANF, do families drop supplemental services, add supplemental services, or maintain existing levels?"
As a second predominant type of administrative data, vital statistics systems also can be linked to welfare data sets to provide a range of child health data. Issues of interest in using birth and death certificate files are described as follows:
Other data sets may not be as easily linked to welfare data sets, yet they should be considered. Twenty-one states have comprehensive databases on hospital discharges (Pappas, 1998). These data can provide information about a wide variety of health concerns, such as child injuries, acute illnesses, and emergency room visits. These data sets may include measures of income, payment authorization, or actual welfare status. Injury data can be linked to welfare participation for individual-level analyses if they can be obtained from local hospital organizations.
Data from school-based health centers are available in fewer places, but could be expected to become more useful as school-based health clinics expand their reach. Although not yet widely available, school-based health centers are growing in coverage and in some states now blanket the state. Some states, such as Massachusetts, have initiated statewide systems of maintaining school health data. School-based clinics often are under the umbrella of a local hospital, and can serve as Medicaid providers under managed care contracts. These data will be most useful when they cover a large proportion of all youth in the area under study and when they provide additional information not available in the Medicaid data. This is the case in Colorado and Connecticut, which have extensive school-based health center networks (Koppelman and Lear, 1998).
Another source of data is programs funded under Title V, the Maternal and Child Health Block Grant (MCHB), which requires performance measurement for contracting and evaluation. State welfare reform evaluators should collaborate with Title V program staff to explore data linkage, inclusion of common data elements of welfare status and health across data sets, and other ways to share data and evaluate child health in the era of reform. For example, several states, including Kansas and Arizona, are implementing performance measurement systems in their Title V maternal and child health programs (Gabor et al., 1997; Grason and Nachbar, 1997). In a pilot project involving seven states, sponsored by the MCHB in 1998, core performance measurements are monitored. These measurements include: needs assessments, percentage of Medicaid-eligible children enrolled, standards of care for women and children, health insurance coverage, and cooperative agreements among state Medicaid, WIC, and other human service agencies. An emphasis on information systems development is also part of these pilot programs and should be explored for linkage with welfare reform evaluation. In another example, the Institute for Child Health Policy at the University of Florida, Gainesville is currently evaluating enrollment in its Healthy Kids programs of outreach to uninsured children, as well as the quality of services in the program for children with special health care needs (Reiss, 1999; Shenkman, 1999).
Efforts to promote and monitor state health objectives should include indicators of children's health according to welfare, employment, and/or income status. As state and local communities plan for future Healthy People 2010 objectives, the impact of continuing welfare reform should be part of future health objectives. Where monitoring systems exist or are planned, they should include either linkage to state and local welfare data sets or common data elements that would provide for evaluation. For example, child health status measures could be monitored regularly according to the following categories: employed families with private health coverage, employed families with Medicaid or CHIP coverage, employed families with no coverage, unemployed families with Medicaid or CHIP, unemployed families with no coverage. These categories could be applied across a range of child health measures: prenatal care, infant mortality, low birth-weight, immunizations, hearing and vision screening, specialist care for children with special health care needs, injuries, or teen pregnancy.
The Aspen Roundtable on Comprehensive Community Based Initiatives has addressed the issue of using administrative data and identified several useful sources for conducting small-area analysis (Coulton and Hollister, 1999). These data include Head Start records, emergency medical service records, immunization registries, and hospital discharge records. Aggregate data at the neighborhood level, combined with comparable welfare data aggregated to the same level, can answer research questions about selected high-risk neighborhoods within a county, within major metropolitan areas, or across a state. Table 10-2 lists several Web sites of organizations conducting these types of neighborhood-level analyses using small-area analysis (Child Trends, 2000,b).
| Web sites for Local Research in Welfare Reform | |
|---|---|
| United Way of Chittenden County, Vermont www.unitedwaycc.org | Chapin Hall Center for Children at the University of Chicago www.chapin.uchicago.edu |
| Social Assets and Vulnerabilities Indicators for Central Indiana (SAVI) www.savi.org | Center for the Study of Social Policy www.cssp.org |
| United Way of Central Indiana www.unitedwaycc.org | Community Building Resource Exchange of the Aspen Institute www.commbuild.org/ |
| National Governors' Association www.nga.org | Aspen Institute Roundtable on Comprehensive Community Initiatives www.aspenroundtable.org |
| Urban Strategies Council Oakland, CA www.urbanstrategies.org | Zero Population Growth www.zpg.org |
| The Center on Urban Poverty and Social Change at Case Western Reserve University Povertycenter.cwru.edu/ | |
Examples of current research using administrative data on Medicaid use, health care access, and other health outcomes to evaluate the impact of welfare reform on children's health are increasing. In spring 2000, a three-state study about children's movement among AFDC, Medicaid, and foster care was released by the Assistant Secretary for Planning and Evaluation (ASPE) of the U.S. Department of Health and Human Services. The study was conducted by Chapin Hall Center for Children at the University of Chicago, Center for Social Services Research at the School of Social Welfare of the University of California at Berkeley, the University of North Carolina School of Social Work at Chapel Hill, and the American Institutes for Research (2000). The study used administrative data from 1995 to 1996 from AFDC, Medicaid, and child welfare programs, obtained through close collaboration with state agencies responsible for these program areas. A baseline population was identified and entry cohorts for each program were used to track experiences of children over the period just prior to PRWORA. The study focused on research questions about transitions from AFDC, including:
Wide variation was found in these measures across Illinois, California, and North Carolina. Differences also were identified for the four measures across children's age groups. This study provides an example of (1) the need to collaborate with other state agencies when using administrative data; (2) the importance of defining the population of study, in this case entry cohorts for 1 year prior to PRWORA with no AFDC/TANF enrollment in the previous 2 years; and (3) the difficulty of generalizing across local areas when studying the characteristics and consequences of welfare programs.
South Carolina has also developed linking capacity of administrative data called CHILD LINK (South Carolina Department of Social Services 1999). This state system links the following data sets: AFDC/TANF, food stamps, Medicaid eligibility, Medicaid payments, work support program data, child protective services, foster care, juvenile justice, alcohol and substance abuse, and wage data. The purpose is to better understand the Medicaid utilization for children after a parent becomes employed and to determine whether, after a client leaves welfare, they use other services to help them through the transition period.
Finally, an inventory of administrative data sets was prepared by UC Data Archive and Technical Assistance of the University of California at Berkeley (1999). This inventory was the result of surveying 26 states about their use of administrative data sets and their capacity to link them. Ninety-five percent of the 26 states were linking AFDC/TANF, Medicaid eligibility, and Food Stamp Program data. Fifty percent were linking AFDC/TANF, Medicaid claims, Medicaid eligibility, and Food Stamp Program data.
The considerable overlap between welfare and child welfare service populations is well documented. Children from welfare families account for as much as 45 percent of those served by the child welfare system (American Humane Association, 1984). The strong association between welfare and child maltreatment may be due to a number of factors, including the stresses associated with poverty, the existence of concurrent risk factors such as mental illness and illicit drugs, and welfare recipients' more frequent contact with public authorities (Coulton et al., 1995; Gelles, 1992; Gil, 1971; Giovannoni and Billingsley, 1970; Wolock and Magura, 1996; Zuravin and DiBlasio, 1996).
Given the documented association between welfare and child maltreatment, a number of authors have reflected on the possible impacts of welfare reform on child welfare (Aber et al., 1995; Haskins, 1995; Meezan and Giovannoni, 1995; Wilson et al., 1995; Zaslow et al., 1995). Essentially all conclude that efforts to induce welfare mothers to self-sufficiency may impact rates of child maltreatment. Again, whether this impact is positive or negative depends in part on what effect reforms have on family income, parental stress, and access to services (Collins and Aber, 1996). For example, loss of benefits or other income supports such as Supplemental Security Income may strain a family's abilities to provide basic necessities such as food and shelter, causing increased neglect and homelessness, even abandonment (Collins, 1997; Knitzer and Bernard, 1997; Shook, 1998). Increased parental stress related to economic, employment, or childcare difficulties may also lead to increased rates of abuse (Knitzer and Bernard, 1997; Meezan and Giovannoni, 1995).
In contrast, positive changes in these areas may be favorable to children and families. For example, rates of abuse and neglect may decline if reforms reduce family's economic hardship. Additionally, gainful employment might improve the mental health of single mothers thereby decreasing the risk of child maltreatment (Garfinkel and McLanahan, 1986). Better access to mental health and drug services also might have similar effects. In addition to impacting the actual rates of maltreatment, the increased scrutiny by public authorities faced by TANF participants and their families might result in greater detection of previously unreported abuse and neglect. Whether positive or negative, these changes likely will be reflected in the number and types of maltreatment reports, the number of case investigations and substantiations, and the number of children placed in foster care.
Welfare reform also may affect the experiences of the children served by the child welfare system. With the passage of PRWORA, a family's economic circumstances become a critical component of the child welfare decision-making process. In particular, parental TANF status could influence the decision to remove a child from a sanctioned parent without any legitimate source of income, and if removed, the TANF status of potential kin caregivers might alter the subsequent placement decision (Zeller, 1998). For example, the proportion of kin placements might decline because kin caregivers might not be exempt from TANF requirements (Berrick et al., 1999; Geen and Waters, 1997; Boots and Green, 1999). Economic factors also might influence children's length of stay in foster care, placement stability, as well as their rates and types of exits from the system. Specifically, parental TANF status might facilitate or stall reunification efforts impacting the duration of children's out-of-home placements. Children placed with kin might experience placement disruptions if their TANF status changes. Although the impact of TANF noncompliance on reunification efforts is clear, compliance also might be problematic, with work making it difficult for parents to meet child welfare timelines such as visitation and court appearances (Knitzer and Bernard, 1997).
In addition to these potential impacts on exit rates, changes in a family's TANF status following reunification might lead to an increased likelihood of reabuse and child welfare system recidivism. Recent research on the child welfare experiences of families in Cleveland suggests that families that go on and off of welfare are more likely to fail in their attempts with reunification of their children than families that continuously receive welfare during the reunification period (Wells and Guo, in press). This, along with data from California (Needell et al., 1999) showing that AFDC families with breaks in AFDC receipt are more likely to become involved with child welfare services, suggests the substantial sensitivity of welfare families to changes in service circumstances.
Unlike the domain of child health, child welfare data traditionally have offered little uniform program participation data. Relevant data are collected only by state child protective service and foster care service departments. In some states (e.g., California) all child welfare administrative data are now entered into one data system. In most states, however, child abuse and neglect reporting and investigation data are gathered separately from data about foster care and adoption. The following section provides an overview of different configurations of these data sources that can be utilized to assess the impact of welfare reform on child maltreatment rates and children's experiences in and exits from the child welfare system. Access and confidentiality issues loom large when using such data to study vulnerable children. Readers should consult Brady and his colleagues (1999, this volume) for an in-depth review of these important topics.
Child Welfare Services Indicators
Most administrative data in the child welfare domain is composed of service event types and dates that can be configured to construct a variety of outcome indicators. The two most common configurations are descriptions of caseloads at a point in time (or several points in time) and longitudinal data analyses of individual service careers over time. In addition to program participation data, demographic data for the children and families under study (e.g., birthdate, ethnicity, home address or location) are also common elements found in these databases. When combined with these demographic data, caseload and longitudinal indicators can provide a source for estimating system performance and client status.
Caseload data provide a snapshot of welfare and child welfare at a specific point in time. They are usually used for program management purposes and can contribute to the assessment of system impacts by indicating covariation between subpopulations in welfare and child welfare. Broadly, for instance, caseload indicators of how many children of a certain age are leaving welfare and how many children of a certain age are entering foster care can provide some indication of whether the welfare exits might be contributing to increases in foster care. However, given the large size of the welfare caseload and the small numbers of children entering foster care, this relationship could not be adequately understood without individual-level data that linked welfare and child welfare histories.
Recently the ACF's Child Welfare Outcomes and Measures Project developed a set of outcome measures using point-in-time data from the Adoption and Foster Care Analysis and Reporting System (AFCARS) to assess state performance in operating child welfare programs. Outcomes include annual incidence of child maltreatment, types of exits from the child welfare system, timing of exits, and placement stability. Although point-in-time data also can be used to measure case status outcomes such as foster care length of stay; the resulting statistics are biased because they overrepresent children with longer stays in care and are not very sensitive to changes in entries to foster care because the newcomers to care are just a portion of the overall population. Thus, although point-in-time estimates are the easiest and least expensive configuration of administrative data, this inherent bias limits their usefulness until the individual records comprising these caseload data are reconfigured into longitudinal data.
Administrative data typically can be reconfigured into event-level files that record program participation histories. Depending on the scope of available data, these events may be restricted to foster care spells or placements, or may more broadly include child abuse reports, investigations, and services provided in the home. Working with entry cohorts provides the clearest evidence of changes in patterns of care that might be associated with changes in welfare programs because the interpretation of the outcomes does not have to disentangle the contributions of different service programs (e.g., AFDC and TANF). Using data that can be subset into entry cohorts captures the dynamics of both system entries and exits, and therefore provides a more accurate assessment of outcomes than caseload. Although free from the biases of point-in-time data, longitudinal data analyses often are preceded by considerable programming to reconfigure data into an even-level, longitudinal format, and to link welfare and child welfare files.
The Multistate Foster Care Data Archive provides an illustration of the complexity as well as promise that longitudinal data offer researchers trying to understand child welfare careers (and how they might be influenced by TANF). The archive is an initiative by the Children's Bureau of the U.S. Department of Health and Human Services that is designed to foster increased collaboration among states regarding administrative data collection in the child welfare services arena. Administered by the Chapin Hall Center for Children at the University of Chicago, the archive currently includes data from child welfare agencies in 11 states. The archive processes state data to make them comparable across state systems. To ensure data comparability, the project focuses on "a limited set of characteristics and events that have clear meaning in all jurisdictions" (Wulczyn et al., 1999:1). The core of the archive is two databases--one consisting of child records, including unique identifiers and demographic information, and a second event-level field that stores information on child welfare events of interest. This structure allows researchers to use the data in a longitudinal format to capture children's spells in child welfare as well as other experiences. Additionally, data can be configured to provide traditional point-in-time estimates of caseload flow over time.
A sufficiently comprehensive set of outcome indicators is shown in Box 10-1 (note that some indicators have a clearer theoretical relationship to welfare reform than others).
BOX 10-1
|
Depending on the purpose of the analysis, indicators can be derived from either point-in-time or longitudinal data. Indicators can be expressed as rates based on the number of people at risk in a state, county, and even zip code of the underlying populations, such as the foster care incidence (entry) rates and prevalence (caseload) rates by age and ethnicity. Benchmarks can be set for both caseload and longitudinal indicators, such as prevalence rate over time, or number and proportion of children who experience reabuse within a year of being reunified from foster care.
In anticipation of later analyses of the effects of welfare reform, researchers in several states have undertaken projects using linked longitudinal AFDC and child welfare data to better understand the overlap between these two programs. These projects serve as models of what will be possible with post-TANF data. In one such endeavor, the Child Welfare Research Center at the University of California at Berkeley undertook an analysis to identify the characteristics of poor families at risk of child maltreatment. Using data from the California Children's Services Archive, researchers constructed a longitudinal database of children entering AFDC between 1988 and 1995 using MediCal data in 10 counties. Probability-matching software was employed to link AFDC histories for these children with birth records, statewide foster care data, and child maltreatment reporting data. Results revealed substantial overlap between the welfare and child welfare populations, with approximately 27 percent of all 1990 child AFDC entrants having child welfare contact, within 5 years and 3 percent entering foster care. This indicates that the overlap between welfare and child protective services is large enough to allow modeling of changes over time and across program types, although analyses of transitions to foster care may be too few to allow powerful modeling. Both total time on aid as well as the number of spells on aid were associated with child welfare contact. Children who transitioned to the child welfare system were more likely to come from single-parent families, larger families, have low birthweight and late or no prenatal care (Needell et al., 1999).
A similar analysis was undertaken in Illinois at the University of Chicago's Chapin Hall Center for Children. Using linked longitudinal data from the state Department of Children and Family Services and the Division of Financial Support Services, Shook (1998) set out to identify baseline rates of maltreatment among children in the Illinois AFDC program between 1990 and 1995. She also identified risk factors for child welfare contact among this population. Risks were higher for children on nonparent cases, children from single-parent families, and white children. Of particular interest were the findings that transitions were more likely among children with sanctioned family grants, because child removals for neglect, lack of supervision, or risk of harm were more likely among sanctioned cases. In addition to helping to identify possible implications of TANF sanctions, the research highlights the use of linked administrative data in assessing the impact of welfare reform on child welfare.
Administrative data from child welfare records also have been combined with qualitative survey data to study the impact of welfare reform. For example, a study currently under way (a collaborative effort by The Urban Institute and The University of California at Berkeley's Center for Social Services Research and UC Data Archive and Technical Assistance, funded by the Stuart Foundation) will combine qualitative data of welfare recipients with data from their administrative welfare records and any available data on children in the home that exist in child welfare administrative data records. This "marriage" between administrative data and qualitative data holds great promise. Specifically, although administrative data can provide information to answer questions such as "How many? What proportion? How long?", other methods can shed some light as to "Why?"
The Center for Social Services Research at the University of California at Berkeley routinely has used child welfare administrative data to draw representative samples to study children using other research methods. For example, counties in the Bay Area Social Services Consortium have funded research to understand the educational needs of children in foster care. A random sample of caregivers drawn from administrative data records is being interviewed to gather detailed information about the children in their care. Similarly, case records of children have been reviewed to look at concurrent planning in child welfare. (Concurrent planning is the provision of an alternative permanent plan, such as adoption, simultaneously with efforts to return a child to his or her birthparent.) In both cases, the samples were drawn from administrative data. Such methods easily could be adapted to provide more in-depth analysis of critical welfare reform issues.
Despite the wide variety of outcome indicators that can be configured from child welfare administrative data, like all services data, child welfare data cover only those who receive services. Because child welfare data are available only for those abused and neglected children who come to the attention of public systems of care, changes to the undetected abuse rate that may result from welfare program changes cannot be assessed. Despite the hurdles associated with linking welfare and child welfare data, given the established association between poverty and maltreatment, child welfare advocates and policy makers must examine the impact of welfare reform on child welfare services. In particular, whether these changes increase the likelihood of maltreatment has important consequences for both the TANF families and children as well as the general social good. In addition to the immediate risk of physical harm and even death maltreated children face, longer term consequences include deficits in emotional and physical health, cognitive development, and socialization difficulties (Ammerman et al., 1986; Couch and Milner, 1993). Furthermore, observed relationships between childhood maltreatment and later criminal activity or abusive behavior also increase future consequences for both children and society (Gray, 1988; Jonson-Reid and Barth, 2000).
Educational success is a key indicator of a child's well-being and clearly is related to current and future economic and physical well-being (Barnett, 1998; Card and Krueger, 1998). Educational success can be affected by educational histories, parental work, and targeted efforts to address parents' educational needs. Indeed, some welfare programs (i.e., Job Opportunities and Basic Skills [JOBS]) have more provisions directed at parental education than others (i.e., TANF). Because of the strong relationship between education of parents and children, when welfare programs help recipients to improve their educational skills (Boudett and Friedlander, 1997), they can be expected to have an influence on the learning of their children.
Certainly, improved educational performance of children is one hope of TANF. Because TANF does not pay for substantial educational programs for parents, the benefits for the education of children would have to be by more indirect means. This process may take several forms. For example, if parents' employment efforts result in relocation to communities that have schools with higher achievement for low-income children, this could result in educational achievement. Or, by witnessing their parents' success at the worksite, children could be inspired to have higher standards for their own achievement.
A limited set of pre-TANF research studies indicates there may not be a simple, sizable effect of welfare participation on children's educational attainment. Hill and O'Neill (1994) found that parental AFDC participation has no effect on children's scores on a standardized test of vocabulary, given income. Currie and Duncan (1995) confirmed that their results hold up even when sibling comparisons are used to account for unobserved maternal background characteristics. Yet a recent analysis of National Longitudinal Study of Youth data that included access to other mother and child services found a relationship between program participation and children's learning (Yoshikawa, 1999). Although the evidence base for research on educational outcomes and welfare reform primarily comes from surveys, there is good reason to suggest the importance of using administrative records to study this relationship. This will be particularly fruitful as the availability and meaningfulness of educational records continue to improve.
Measures of educational success include data elements that describe the child's achievement as well as their receipt of services. Many of these data are now in electronic databases in the school districts, but the automation of educational records tends to begin with the high schools and trickle down to the elementary schools. Thus, elementary school grades are not as likely to be automated as middle school or high school grades. Standardized statewide test scores are now quite routinely required of all students, as are periodic achievement test scores during certain sentinel years. The variety and repetition of tests is becoming quite extensive. (As an illustration, Box 10-2 includes the testing schedule for students in North Carolina schools.)
BOX 10-2
|
BOX 10-3
|
Most, but not all, students take these tests. Exemptions may be given to students in special education, as determined by their Individual Education Program teams. Exemptions also may be given to students who are not following a standard course of study, such as those in alternative education or adolescent parenting programs.
Grade retention histories usually are available (or can be inferred from birthdates and grade levels). Educational reform is making grade retention data more valuable. Although widespread adherence to the principles of social promotion have dominated the nation's public schools for many years, legislation in many states (e.g., California, New York, North Carolina) is now discouraging social promotion. In the future, grade retention may indicate a child's true performance, not just a school's educational strategy regarding social promotion.
School services data also are obtainable, although the lack of standardization makes it difficult to assess change when students also change schools during the period under study. School attendance data also is likely to be automated, although comparisons across schools and, especially, unified school districts must be done with care because of different ways of administering the statewide definitions of attendance. Schools also have data about student's disciplinary actions--nearly always including suspensions or expulsions, but also including a variety of other disciplinary actions that are less severe. But caution is also needed in making comparisons about disciplinary actions in school. This is particularly true of suspensions, as some schools use them routinely and some schools use them only after considerable effort to mediate the problematic situation. Further, different rules typically apply to children receiving special education services and the proportion of children receiving special education services is, in turn, quite variable across schools. To assess the effects of welfare reform on the educational outcomes of children, even a minimum data set that included measures of academic achievement, absences, and suspensions would be useful (see Box 10-3).
A major impediment to using educational data to estimate the well-being of children is the Family Educational Rights and Privacy Act (FERPA). First enacted in 1974, FERPA gives parents the right to inspect and review their children's education records, request amendment of the records, and have some control over the disclosure of information from the records. At age 18, this right is transferred to the student. The act also restricts the release of school records or information from those records that could identify the student. Before releasing such records or information to a party outside the school system, the school first must obtain the consent of the student's parent. FERPA offers a key exception to the prior consent requirement. Specifically, educators may disclose information without prior consent if the disclosure is being made to organizations conducting studies for, or on behalf of education agencies or institutions in order to develop tests, administer student aid, or improve instruction (¤99.31(a)(6) of FERPA regulations). To meet this requirement, the researcher must have an agreement with the educational institution that the researcher is working on the institutions' behalf and that the study will create information that will improve instruction. Additionally, FERPA was amended in 1994 to permit nonconsensual disclosures of education records to officials in the state juvenile justice system and under certain special circumstances. Despite some loosening of restrictions, FERPA remains a significant barrier to data access for researchers.
Because educational attainment represents one of the few unambiguous outcomes that can be assessed with administrative records, substantial legislative efforts need to be taken to make this critical source of information about child well-being more available to researchers. Consideration should be given to amend FERPA to not require parental signatures for the routine use of administrative data in research in cases where students' rights to privacy are not jeopardized.
More must be done to allow educational data to be used by researchers to better understand the educational implications of other social programs. This would be a low-cost way to try to assist our highest risk children--who are most likely to be involved in multiple systems of care. FERPA has been revised to make it possible for schools to share information with correctional agencies. Under FERPA, schools may disclose information from law enforcement unit records (see ¤99.3 and ¤99.8 of FERPA regulations) without the consent of the parents or eligible students. This enables schools to give information to social services or juvenile justice agencies as long as the school district first creates and maintains a "law enforcement unit" that is officially authorized to (1) enforce federal, state, or local law, or (2) maintain the physical security and safety of schools in the district. Although this is a modest amendment of FERPA that may not have direct relevance to most researchers endeavoring to use administrative educational data (perhaps unless they are also studying law enforcement issues), it does suggest the willingness of Congress to modify FERPA for good cause. The "Solomon Amendment" of 1999 also limited the unintended implications of FERPA in order to deny aid to schools that either prohibit or prevent the Secretary of Defense from obtaining, for military recruiting purposes, access to directory information on students. The needs of researchers and policy makers to have good information about the educational outcomes of welfare reform (and other social programs) also are worthy of a FERPA amendment.
Perhaps because of the obstacles created by FERPA, there has been little work matching administrative records to welfare data to assess possible relationships between educational progress and welfare program participation. There are a few examples, however. Orthner and Randolph (1999) examined the impact of parental work and continuity of welfare receipt on the dropout rates of high school students in families in poverty. This work was accomplished by matching individual records of children who were enrolled in the JOBS program in Mecklenburg County with the administrative records from the Mecklenburg Unified School District. Some case record checking also was done of the paper school district records. Using event history analysis, they examined the risk of dropping out of school in light of potential effects on subsequent social and economic well-being. The data indicate that that consistency in parental employment (i.e., parents who worked in all quarters) and transitions off welfare are associated with lower rates of dropping out of high school. Longer spells on public assistance are associated with higher dropout rates.
Some community colleges have merged their data with the public welfare data to better understand the overlap between their student population and the welfare population (Community College Involvement in Welfare; www.aacc. nche.edu/research/welfare.htm/11/6/99). In a survey of 1,124 community colleges conducted by the American Association of Community Colleges, about 32 percent track students on public assistance. Among the primary reasons given for not tracking students were student confidentiality and privacy issues. Yet the disruption in the educational careers of welfare recipients that may occur with the end of the JOBS program and the institution of TANF could be an important outcome for young people. A straightforward way to study this overlap and the changes in this population is to merge administrative data from community colleges and TANF programs.
Although social competence and adjustment is a difficult dimension to study using administrative (or survey) data, one area that can be examined profitably is the involvement with the juvenile authorities that results when children and youth break adult laws. Parental welfare program participation has long been thought to be associated with criminal and juvenile justice involvement (e.g., Levinson, 1969). There are several reasons why changes in a parent's involvement with welfare could affect the likelihood of juvenile justice involvement by their children. Communities with high welfare participation also have high crime rates. In one study (Philips et al., 1999) these two factors were temporally linked, showing that mortality (particularly intentional mortality) was far higher during the first week of the month when welfare and other public assistance checks arrive. Some youth may be involved in such crimes. Households with parents who move off welfare into self-sufficiency will have additional resources that they could use to purchase a variety of services and activities that would occupy their children and help them avoid the hazards of "hanging out." At the same time, with parents working away from the home, there may be less supervision for those youth who do not become involved in other activities. Also, if families have their benefits cut, we know they often rely on "other family members" to assist them. Although this typically means other adult relatives, it is possible that youth would feel pressured to bring new resources into the household or to, at least, find resources that would allow them to be less dependent on their families for food, clothing, and entertainment.
The most common approach to assessing criminal justice involvement is to study "arrest records." This is the device used in most studies of the transition from child welfare programs to juvenile justice involvement (e.g., Widom, 1991; English and Widom, 1999). The potential drawback of arrest records is that they reflect the combined behaviors of juveniles and criminal justice systems. This is counterbalanced by the fact that they are generally considered to be more useful than conviction records because convictions or incarcerations are determined by so many other factors--especially for less violent crimes. Still, convictions or incarcerations can be used if the theoretical relationship between welfare participation and crime suggests there would be higher rates of major crimes. Incarcerations in state training programs have been shown to be sensitive enough to pick up differences between groups that did and did not obtain ongoing child welfare services following a child abuse investigation (Jonson-Reid and Barth, 2000).
Juvenile justice data also can be obtained from a variety of settings, depending on the geographic locus of the study. At the local level, youth often are remanded to juvenile detention and county camps and ranches. At the state level, they may attend a training school or youth authority program. In more populous counties, they generally have greater capacity to hold more youth who commit more serious offenses at the local level, whereas more rural areas may use the statewide facilities to a greater extent. Statewide facilities often have their own databases, which include substantial additional information collected about the child at intake. This makes such information particularly useful in trying to explain exit patterns and the path of services once in the training program.
Some juveniles are tried as adults and others may have their records sealed for a variety of offenses. Still, these remain the exception and they are unlikely to bias study results or affect interjurisdictional comparisons as long as reasonable sample sizes are maintained.
Although these authors were unable to identify any studies that have directly tested the relationship between parents' welfare participation and children's juvenile justice involvement, one important study matched juvenile justice data with survey data from the Moving to Opportunity (MTO) experiment. In the MTO, a total of 614 families living in high-poverty Baltimore neighborhoods were assigned into three different "treatment groups": experimental group families received housing subsidies, counseling, and search assistance to move to private-market housing in low-poverty census tracts (poverty rates under 10 percent); Section 8-only group families received private-market housing subsidies with no constraints on relocation choices; and a control group received no special assistance under the MTO. The impact of this "treatment" on juvenile arrests was then assessed (Ludwig et al., 1999). (The authors also tested models that used convictions instead of arrests and found similar results.)
The study also cast light on the utility of a variety of indicators of juvenile justice involvement, finding that false arrests are likely to be crime specific and disproportionately involve charges such as disorderly conduct, resisting arrest, and assaulting a police officer. Second, they replicated their analysis using convictions instead of arrests, assuming that these show less variation across neighborhoods in false convictions than arrests because juvenile prosecutions are handled at the county level and arrests are made by local police.
Despite the benefits of using linked longitudinal administrative data, the work is complex and the level of effort and skill required is easily underestimated. Linking across data systems poses many challenges. Linking is accomplished by matching unique identifying information such as Social Security numbers across data systems of interest. Even when "unique" identifiers exist in the data sources to be linked, probabilistic-matching software should be employed to link records across data systems to reduce matching errors. Readers should consult Lee and Goerge (1999) for an in-depth review of the advantages of probabilistic matching even when Social Security numbers are available in both data sets.
In addition to the complex logistics of linking files, new issues are posed by TANF reforms themselves. In particular, a model that thoroughly investigates the relationship between parental welfare paths and child well-being requires not only data on the timing of welfare receipt, but also an indication of the reason that aid ceased. Without an explanation of the reason for termination, it is difficult to distinguish between parents who left aid for gainful employment and those who were dropped from the rolls due to a sanction and/or failure to comply with regulations. In many cases, this information is lacking. Therefore, researchers may try to link welfare and child well-being data to parental employment data in an attempt to understand which families are leaving welfare for "positive" reasons.
Finally, most current evaluation efforts typically focus on examining the relationship between parental welfare careers and outcomes for children. Under TANF, however, children's and parents' welfare careers must be considered separately. In some states, such as California, sanctions and time limits will result in a decrease in only the parental portion of the welfare grant, with the child's portion maintained. Children might, then, move to another household assistance unit where the parent figure gets full benefits. Identifying and successfully tracking these parents and children may involve record linkage across cases and incorporate case flow dynamics that are quite complicated. Beyond receipt of TANF assistance, children's participation in other important programs such as Medicaid, the Food Stamp Program, and WIC also must be evaluated if we are to gain a comprehensive understanding of the impacts that reforms have on child well-being.
Using administrative data to evaluate welfare reform presents challenges and opportunities within each of the domains of child well-being. Child abuse and neglect data generally are available to the evaluation of welfare reform because both child welfare records and TANF data sets typically reside within the same governmental department at the local and state levels. However, developing appropriate measures of child well-being from administrative child welfare requires intricate programming of longitudinal data files: understanding of the differences between the child's experience and the system performance indicators; expertise with a range of sophisticated analysis methods; and understanding of many interpretations that administrative data might allow. In contrast, health measures of child well- being--for example, birthweight or immunization completeness--are more uniformly defined and there is more agreement about their implications. However, these data are less available to study welfare reform because they typically reside within government entities separate from departments where welfare data reside. When these data sources differ, issues of compatibility of data formats and definitions, linking of data, confidentiality, and ownership of data files call for collaborative efforts to evaluate welfare reform. Evaluation of impacts within juvenile justice and education include particularly acute challenges of data availability, as well as the need to create valid and reliable measures.
The authors of this paper have endeavored to increase readers' familiarity with needed indicators of child well-being and the administrative data sets that contain them. A secondary goal has been to alert readers to the ways that existing policies hamper access to the data necessary to make informed decisions. Obtaining permission to use administrative data for evaluation purposes is harder than it needs to be. Without substantial convergence around the purposes of using administrative data, this emerging technology is going to be a partial, piecemeal, and ephemeral aid to government. The technical solutions for linking are increasing (storage is more affordable, processing times are shorter, and matching software is better), but public support has not been built to encourage this linking. Issues of data access and confidentiality present the greatest barriers to full utilization of this resource.
Although the federal government is demanding more accountability from the states, and the states from the counties, there is little outcry from public officials to permit the broader use of administrative data to generate the information required to track the performance of human service agencies. Scandinavian countries are generating invaluable research using linked data across generations to understand, for example, the transmission of schizophrenia across generations and the likelihood that children born with birth defects will give birth to children with birth defects. Similarly, program participation data have been combined with information from driving records, educational attainment, military service, and marriage certificates to understand lifetime outcomes of family recomposition and participation in service programs. Researcher access to administrative data is beneficial, and more open access will permit individuals to educate themselves about what is contained in such databases, to use the information within those databases to conduct research for multiple purposes, and to reassure the public about the feasibility of using already gathered information for the public good.
Concerns over confidentiality continue to present a major barrier to linking administrative data to evaluate the effects of welfare reform on child well-being. Perhaps nowhere is there as much sensitivity concerning privacy and confidentiality as with records containing information about vulnerable children and parents who have been accused of violating social norms by abusing or neglecting their children. At the same time, electronic availability of information on individuals permits sophisticated research that was simply impossible in the past. How do we reconcile the need to provide privacy and confidentiality to individual patients while enabling public health researchers and policy makers to use available information to make the best decisions?
Although privacy and confidentiality of records about children's well-being are important, we suggest there are already adequate protections, incentives and disincentives, and policies and procedures, to preserve individual privacy. We already trust millions of individuals in our society to respect the confidentiality of information they encounter each day in the human services, child welfare, health care, law enforcement, juvenile justice, and education sysem, to name a few. We trust the individuals conducting research within each of these systems to maintain the confidentiality of records. Most of these data are collected without any explicit discussion of whether or how they will be used for research that might inform administration of the program. Yet we have generated the expectation that individuals not working for those institutions who obtain data from them in order to advance services research through data linking represent a risk to the confidentiality concerns of service recipients. The expectation that there is likely to be even a minimal risk of mishandling data lacks an evidentiary base. In our 10 years of experience using administrative data of the most sensitive kinds (including child abuse reports and juvenile justice records), we know of no violations of individual rights of persons in those data sets. Nor do we have any stories to tell about exceptional procedures we instituted to prevent such misuse. The handling of that information was simply very routine. Perhaps we need a more systematic effort to determine what real and imagined threats to confidentiality exist in data-linking efforts. Until we have evidence to the contrary, we should continue to maintain databases with adequate identifying information to support future research projects, and we should advocate for change in unwisely broad legislative or regulatory language that adversely affects interorganizational research.
We believe it is appropriate and indeed necessary to maintain personal identifying information on public health and child well-being databases, and that those identifiers should be available to facilitate linkage of electronic health databases to support research to improve the health of our population as well as to enhance the health of individuals. At the same time, we emphasize that availability of such identifiers is quite different from license to invade the privacy of individuals or disregard the need for strict confidentiality of the information held within medical records. We believe it is possible to reconcile all these goals.
We need to encourage constant conversation between investigators specializing in administrative data and those designing surveys so that the surveys can be used to help inform the interpretation of the administrative data. Survey researchers generally are not familiar with the needs for researchers to be provided with data that have adequate variables for matching. For example, data that tell us about the reasons why clients change service use patterns can be combined with information from administrative data about how often and when these service use patterns change. Furthermore, we must develop better strategies for making survey data available for linking with administrative data. A serious threat to this possibility is the assumption that if it is possible for the confidentiality of a data set to be compromised, it will. This leads to counterproductive strategies such as making it impossible to accurately match samples to their communities or counties of origin (thus obviating the possibility of exploring neighborhood or county effects).
Whereas linked administrative data can provide important information on the impact of welfare reform on child well-being, it is not a panacea and will not provide us with all the information we need to monitor welfare reform. We must be wary of the conclusions we draw from linked data because we often cannot determine whether an individual did not experience the outcome, was recorded as experiencing the outcome but could not be matched across data systems (e.g., if they moved across jurisdictional lines), or experienced the outcome but was not recorded as such. Even when the data are accurate, at best they help us monitor who appears to be affected by welfare reform, when those impacts occur, and where the impact is greatest or least. Sometimes we do not even know the direction of that change. For example, if more children per capita are reported for abuse and neglect under TANF than were reported under JOBS, this could mean that the smaller TANF caseloads have resulted in more opportunities for home visiting and better early identification of child abuse and neglect. As to why welfare reform affects children and families differentially, administrative data can only guide us as to the best places to look for those answers. Carefully designed representative samples can be drawn and subjected to other methods (e.g., surveys) that can build on the framework that a comprehensive administrative data analysis provides.
[ Go to Contents ]
Main Page of Report | Contents of Report
Home Pages:
Human Services Policy (HSP)
Assistant Secretary for Planning and Evaluation
(ASPE)
U.S. Department of Health & Human Services
(HHS)