ACYF-CB-IM-01-06

ACF Administration for Children and Families	U.S. DEPARTMENT OF HEALTH AND HUMAN SERVICES Administration for Children, Youth and Families
	1. Log No.: ACYF-CB-IM-01-06	2. Issuance Date: July 3, 2001
	3. Originating Office: Children's Bureau
	4. Key Words: Sampling Period, Child and Family Services Reviews

INFORMATION MEMORANDUM

TO:	State and Territorial Agencies Administering or Supervising the Administration of Titles IV-B and IV-E of the Social Security Act
SUBJECT:	Sampling Period for Selecting Cases for the Child and Family Services (CFS) Reviews
LEGAL AND RELATED REFERENCES:	The Adoption and Safe Families Act of 1997 (Public Law 105-89), Titles IV-B and IV-E of the Social Security Act, the Social Security Act Amendments of 1994 (Public Law 103-432), 45 CFR 1355
PURPOSE:	The purpose of this information memorandum (IM) is to announce a change in the method used to determine the time period from which the sample of cases for the onsite CFS reviews will be drawn. This change will be effective immediately for all CFS reviews scheduled for Federal fiscal years 2001 and 2002.

INFORMATION:

Background

In accordance with 45 CFR 1355.33(c), the CFS reviews require an onsite review of a random sample of foster care and in-home cases for certain outcomes. We did not regulate specific time frames from which to draw a sample, so we are providing guidance in this format. To date, the sample has been selected from Federal fiscal year (FY) 1999 for foster care and calendar year (CY) 1999 for in-home cases, as those were the time frames for the Adoption and Foster Care Reporting and Analysis System (AFCARS) and the National Child Abuse and Neglect Data System (NCANDS) data used in the reviews. We chose this approach so that the time period from which we select the case sample would coincide with the time period covered by the data profiles used in the statewide assessment. We have, however, encountered numerous problems in implementing this procedure:

AFCARS and NCANDS submission periods do not coincide, i.e., AFCARS data are submitted by fiscal year while NCANDS data are submitted by calendar year. We, therefore, have two different time periods for the cases being reviewed.
States have continually expressed concern that reviewing "old" cases going back to FY and CY 1999 will not reflect current practice. Similarly, we are concerned about overrepresentation of adoption and emancipation cases due to the older sample.
Since many of the cases included in the sample have been closed for some time, the first few States participating in the CFS reviews encountered problems in locating and gaining consent from families to be part of the review. As a result, preserving an oversample containing a sufficient number of cases necessary to resolve discrepancies between the onsite review and the statewide assessment could, in some instances, present problems in resolving discrepancies.

We believe that using a more recent sample of cases for the onsite reviews will address the latter two concerns noted above, but we must also account for other issues that are affected by the time period used to select the sample. In reviewing cases onsite, we adhere to the concept of a "period under review." That is, we review for case activities that occur within a specific time frame, beginning with the time period from which the sample is drawn and continuing to the time of the onsite review. In order to evaluate certain indicators used in the reviews, for example, recurrence of maltreatment, re-entries into foster care, and stability of foster care placements, we must be able to follow individual cases for several months in order to determine the effects of such indicators. Therefore, we must balance our desire to review for recent child welfare practice in States with our need to review cases over a time period that is sufficient to observe certain critical indicators.

In light of these issues, we have re-evaluated the sampling period used to select both in-home and foster care cases for the CFS onsite reviews and describe below the changes we intend to make immediately.

Sample Periods

(1) The time period from which we select the sample of 150 foster care cases will be the most recent six-month AFCARS submission period available that allows sufficient follow-up time to evaluate the performance indicators associated with the review. Because of the follow-up time needed to evaluate cases, the AFCARS submission period from which we will select the sample may not be the most recent AFCARS submission period immediately preceding the onsite review. In order to use the most recent AFCARS submission period that will provide the needed follow-up time, the specific AFCARS submission period from which we select the samples will vary from State-to-State, depending upon when the State's review is scheduled, as follows:

For States whose onsite CFS reviews will be conducted between May 7, 2001 and September 30, 2001*, the sampling period will coincide with the six-month AFCARS submission period from April 1, 2000 - September 30, 2000.
For States whose onsite CFS reviews will be conducted between October 1, 2001 and March 31, 2002, the sampling period will coincide with the six-month AFCARS submission period from October 1, 2000 - March 31, 2001.
For States whose onsite CFS reviews will be conducted between April 1, 2002 and September 30, 2002, the sampling period will coincide with the six-month AFCARS submission period from April 1, 2001 - September 30, 2001.

(2) The time period from which we select the sample of 150 in-home cases will coincide with the six-month AFCARS submission periods noted above. For the universe of in-home cases, States will submit, from the three State sites selected for the onsite review, a listing of cases that were open for in-home services for at least 60 days during that same six-month AFCARS submission period.

The time period that we define as the "period under review," referenced above, will be changed to reflect the more current dates of the sample of cases reviewed onsite. In the first three CFS reviews conducted in Federal FY 2001, the "period under review" for foster care cases began with Federal FY 1999 and ended with the date of the onsite review, while the "period under review" for in-home service cases began with calendar year 1999 and ended with the date of the onsite review. The "period under review" for both foster care and in-home service cases will now begin with the time frame covered by the six-month AFCARS submission period from which both samples will be selected, and end with the date of the onsite review.

Data Profile Periods

While it is necessary for us to change the sample period due to the reasons noted above, it is not possible for us to change the time period covered by the data used to complete the data profiles. We must continue to use data for the profiles that are less recent than the sampling period for the following reasons:

We prepare the foster care data profiles on the basis on an annual AFCARS file that is based on the two submission periods for a given fiscal year, for example, we used 1999a and 1999b submissions to create the 1999 annual file;
We use data from a third AFCARS submission period, the one following the fiscal year, to update data elements in the annual file that may not have been captured in the two submissions for the fiscal year, for example, we used the 2000a submission to update the 1999 annual file;
We allow States a period of time in which to correct and re-submit AFCARS data if they are notified that their submission is incorrect, and such corrections are necessary to assure that we have the most accurate data available for preparing the numerous data elements in the data profiles; and
States have a longer period of time to submit NCANDS data than they have to submit AFCARS data, which does not permit the use of more current safety data in developing the data profiles.

The issues noted above that prohibit us from using a more recent time period for the data profiles do not apply to our use of a more recent time period to select the sample of cases for the onsite review. We are able to use a more recent AFCARS submission period to select the sample of foster care cases for the following reasons:

We are selecting the sample from one six-month AFCARS submission period, rather than an annual file;
The six-month AFCARS submission from which the foster care sample is selected will be representative of a State's universe of foster care cases for that time period;
Data quality issues in the AFCARS submissions that must be addressed through subsequent corrective action that are important to the preparation of the data profiles are not relevant for purposes of selecting the sample; and
For sampling purposes, we are not concerned with the timing of NCANDS submissions, since we select foster care cases from AFCARS and the in-home cases are selected from a listing of cases provided by the State rather than from NCANDS submissions.

Therefore, the most recent AFCARS and NCANDS submissions that will be available for use in preparing the data profiles for the FY 2001 CFS reviews is FY 1999 annual AFCARS data and CY 1999 NCANDS data. For States under review in FY 2002, the data profiles will be based on FY 2000 annual AFCARS data and CY 2000 NCANDS data, which will be the most recent data submissions available for the profiles.

Although the time periods covered by the data profiles will no longer coincide with the time periods used to select the sample of cases reviewed onsite, we believe the advantages of selecting a more recent sample of cases for the onsite review outweigh the concerns about using different time periods. We believe that this procedure will work to the advantage of States in permitting the most recent perspective possible on child welfare practice in the States. While we are using both the aggregate data and the case reviews to make decisions about a State's substantial conformity on two of the outcomes, we are examining the two sets of information using separate criteria (i.e., a qualitative assessment of the cases versus a comparative analysis of the data to the national standards). As such, there is no direct interaction between the data for the sample and for the profiles. In the event of a discrepancy between the two sets of information, the regulation at 45 CFR 1355.33(d) and the procedures manual for the CFS reviews provide procedures for resolving the discrepancy, with one option being the submission of more recent data.

INQUIRIES TO: ACF Regional Offices

/ s /
______________________________________
James A. Harrell
Acting Commissioner
Administration on Children, Youth
and Families

* This time period includes all States to be reviewed in FY 2001 except Delaware, North Carolina, and Vermont, as those reviews were completed or samples had already been drawn at the time of the decision to make this change. Back