Detailed Information on the
Education - Neglected and Delinquent State Agency Program Assessment

Pursue legislative, regulatory, or administrative changes that would enable the Department to collect data on achievement once students leave institutions, including data on high school completion.

Enhance technical assistance to States to help ensure that they are providing the required services to help students transition successfully to further schooling or employment.

Use performance data to target services and monitoring on areas of greatest need.

Collect grantee data on key student outcomes such as educational attainment and using this for program improvement and targeting of technical assistance.

Establish ambitious targets for annual performance measures.

Measure: The percentage of neglected or delinquent students obtaining a secondary school diploma or diploma equivalent.

Explanation:This measure helps track progress made towards one of the program's purposes - to prevent at-risk youth from dropping out of school and to provide dropouts returning from correctional facilities for neglected and delinquent youth with a support system to ensure continued education.

Measure: The percentage of neglected or delinquent students earning high school course credits.

Explanation:This measure helps track the progress made towards improving educational services and opportunities for N/D students by using high school credits earned as a measure of the level of academic engagement of students aged 11-21 years in juvenile detention, juvenile corrections and programs for youth who are neglected. [Note - State collections of these data are fairly new and face several challenges as N/D institutions are run by different agencies in different States. ED will continue to validate and update data to ensure accuracy and comparability.]

Measure: The percentage of long-term neglected or delinquent students who improve reading skills as measured through State-approved assessments.

Explanation:This measure helps track the progress made towards improving educational services and opportunities for N/D students by tracking the percentage of long-term students who are neglected or delinquent who showed improvement on pre-post achievement test scores in reading. A long-term student is an N/D program participant who is in the program for more than 90 days. Improvement is measured by increases in grade-level achievement on State-approved pre and posttest instruments. State-approved assessments are not necessarily those assessments used for Title I-A accountability purposes. ED validates these measures with a variety of data quality checks, including, for example, ensuring results tables are complete and without anomalies, discrepancies, or errors. ED contacts States to request corrections if and when issues are identified. [Note - State collections of these data are fairly new and face several challenges as N/D institutions are run by different agencies in different States. ED will continue to validate and update data to ensure accuracy and comparability.]

Measure: Cost per high school diploma or equivalent.

Explanation:This measure attempts to measure program cost efficiencies by tracking the ratio of the number of participating students achieving a high school diploma or its equivalent to the cost of the program.

Measure: The percentage of long-term neglected or delinquent students who improve mathematics skills as measured through State-approved assessments. (New measure, added February 2008)

Explanation:This measure helps track the progress made towards improving educational services and opportunities for N/D students by tracking the percentage of long-term students who are neglected or delinquent who showed improvement on pre-post achievement test scores in mathematics. A long-term student is an N/D program participant who is in the program for more than 90 days. Improvement is measured by increases in grade-level achievement on State-approved pre and posttest instruments. State-approved assessments are not necessarily those assessments used for Title I-A accountability purposes. ED validates these measures with a variety of data quality checks, including, for example, ensuring results tables are complete and without anomalies, discrepancies, or errors. ED contacts States to request corrections if and when issues are identified. [Note - State collections of these data are fairly new and face several challenges as N/D institutions are run by different agencies in different States. ED will continue to validate and update data to ensure accuracy and comparability.]

Is the program purpose clear?

Explanation: The legislation specifies that the purpose of Title I, Part D is (1) to improve educational services and equal opportunities for youth in neglected or delinquent correctional institutions or in adult correctional institutions; (2) to provide N and D youth with the services needed to make a successful transition from institutionalization to further schooling or employment; and (3) to prevent at-risk youth from dropping out of school, and to provide dropouts and youth returning from correctional institutions for neglected or delinquent youth with a support system to ensure their continued education.

Evidence: SEC. 1401 of Title I Part D, Subpart 1.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the needs of youth who are neglected, delinquent (incarcerated), and at risk of educational failure (N or D). The population of N or D children is significant, and the services needed by these youth tend to be intensive and expensive. Also, research suggests that education may be the best way to stop a youth from cycling through the system. In 2000, about 2.4 million youth (under 18) were arrested (National Center for Juvenile Justice). Over 100,000 youth are held in residential placement on a given day (OJJDP, 2004). About 550,000 abused or neglected children were in out-of-home placements in 2001 (Child Welfare League of America).

Evidence: Studies from universities, States across the country, the Federal Bureau of Prisons, Educational Testing Service, Correctional Education Association, and the Criminal Justice Policy Council have shown that education reduces recidivism. Costs of incarceration range from $35,000 to $64,000 per youth/per year. In the 2004-2005 school year, the Title I, Part D program reached 391,182 students. Of these, approximately 110,115 received services from State agencies (SAs) and 281,067 received services from local educational agencies (LEAs).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Title I, Part D program is the only program specifically for the education of youth who are neglected or delinquent that leverages its funding to raise participants' academic achievement in core subjects (reading and mathematics). With the exception of funding from the Individuals with Disabilities Education Act, no other Federal program provides funding or educational services to youth who are neglected or delinquent for the period while the youth are in State institutions. No other program holds States and LEAs accountable for the education of N/D students. Funding redundancy is also reduced through the Title I, Part D supplement-not-supplant requirement, which ensures that Federal funds add to, rather than replace, State and local funding.

Evidence: The Department does not collect national data on the Federal share of providing educational services for this population. However, the Council of Juvenile Corrections Administrators (CJCA) collected information from 24 States for fiscal year 2002-03 that showed a Federal share ranging from less than one percent (Mississippi) to as much as 35 percent (South Dakota). The average proportion of Federal funds to support correctional eduaction was approximately 18 percent, according to the survey.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is generally well designed to meet its goals and is free of major flaws. For example, the requirement that State agencies align their curricula with State standards helps ensure that a small amount of Federal funds leverages higher standards and greater accountability from States and local institutions. Further, States are required to submit plans aligned with national goals, and subgrantees, as a part of their applications to States, align those plans with State and Federal goals. There is no evidence of an approach that would be more effective or efficient

Evidence: State plans and consolidated applications submitted to the Department.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The formula for how much funding each State receives is based on counts of youth who are neglected or delinquent. Once States receive funds, States then determine how best to allocate the funds through a specified, detailed application process based on need. Further, the program's nonregulatory guidance specifies the types of agencies, institutions and students that are eligible to receive program funds or services. The statute also includes a "supplement, not supplant" provision. States may only take up to 1 percent of their grant for administration.

Evidence: The formula uses the counts of children who are neglected or delinquent who are enrolled in educational programs for a specified amount of time per week to determine allocation of funds. State funding allocation plans describe how funds are targeted to those most in needs of services and State education plans are aligned with those of other State and local agencies. States require applications from SAs and LEAs, in which the applicants identify those students in greatest need. Based on data collected from the 2004-2005 school year, Subpart 1 funds reached 110,115 students in Title I, Part D programs; 7,803 students were in neglected programs, 26,622 were in juvenile correctional facilities, 53,632 were in juvenile detention facilities, and 22,058 were in adult corrections facilities. State administration caps are set in Sec. 1004(a)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term performance measures that focus on specific educational outcomes for youth who are N or D. The two measures are the number of long-term students who improve on reading as measured by appropriate pre- and post- tests during their enrollment in the program, and the number of long-term students who accrue high-school course credits while in the program. "Long-term students" are defined as students who are enrolled in the program for 90 or more consecutive days while in the institution. The Department collects annual performance data on these indicators and has set specific long-term goals for each measure. In addition, the Department annually collects data on the percentage of students who obtain a high school diploma or equivalent. All of these performance measures align with the goals of Title I, Part D.

Evidence: The Department collects information related to its long-term performance measures on an annual basis, including data on ten academic and vocational outcomes. The information is collected through the ESEA Consolidated State Perfromance Report (CSPR).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Department has identified annual targets and long-term goals for each of the program's long-term measures and has established baselines based on data collected on the 2004-2005 school year. The Department has set the annual target for each of the long-term measures at a 5 percent increase per year and the following long-term goals: 1) 100 percent of students will demonstrate improvement in reading by 2014 and 2) Over 85 percent of eligible students will accrue high school course credits by 2014.

Evidence: Each year, the Department identifies its annual performance targets through its Program Performance Plan for Title I, Part D and as part of fulfilling its reporting requirements under the Government Performance and Results Act. Data for these reports are collected annually through the Consolidated State Performance Report (CSPR).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program's long-term goals are focused on the percentage of students who demonstrate improvement in the area of reading and the percentage of students who earn high school course credits. As is the case in many education programs, the annual performance measures for the program reflect intended annual progress toward the long-term objectives. In addition, the Department uses an additional annual performance measure (the percentage of students earning a diploma or GED) and an efficiency measure (the cost per diploma or equivalent) to help monitor progress towards the program's goals.

Evidence: The Department has established measures and reporting forms for collecting all academic and vocational outcome data, as well as demographic and other data. The measures were integrated into the Department's Consolidated Performance Report forms for School Year 2004-05 and the PBDMI/EDEN system in order to collect and report the data.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines and annual targets for the long-term and annual measures are based on initial data collected for the 2004-05 school year. For the first outcome measure, the percentage of neglected or delinquent students obtaining a secondary school diploma or equivalent, the baseline is 10.5 percent of N or D students. For the second outcome measure, the percentage of neglected or delinquent students earning high school course credits, the baseline is 55.95 percent of N or D students. And for the third outcome measure, the percentage of neglected or delinquent high school students who improve academic skills, the baseline is 72.53 percent of N or D students showing improvement in reading assessments. Information on improvement on Math assessments will be added in future years. Annual targets for each measure have been set at a 5 percent increase per year. In addition, the Department set its annual targets for the program's efficiency measure, cost per high school diploma, at a 6 percent reduction per year starting from a baseline of $5,095 in 2005.

Evidence: The Program Performance Plan for Title I, Part D and the annual collection through the Consolidated State Performance Report.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Sec. 1414 of Title I, Part D requires that grantees (SEAs) align their goals and objectives with Federal program goals. Evidence of this alignment is observed in required State consolidated applications and in State plans (which must be submitted to ED for approval), and via monitoring visits/activities. Further, Sec. 1414(a)(2)(iii) specifies that subgrantees are required to align program goals with State goals.

Evidence: Grantees and sub-grantees submit plans and agree to meet program goals. Both desk monitoring and site visits are used to ensure that the plans at each level are aligned and in the process of being implemented. To help ensure accountability, the program manager holds regular meetings with key personnel with the National Evaluation and Technical Assistance Center for the Education of Children and Youth who are Neglected, Delinquent or At Risk (NDTAC), a Department contractor, and receives a detailed monthly report of the Center's activities. The Center's products are featured on the Department's web site: original articles (http://www.neglected-delinquent.org/nd/resources/spotlight/spotlight_archives.asp), toolkits (http://www.neglected-delinquent.org/nd/resources/toolkits/toolkit.asp); topic library (http://www.neglected-delinquent.org/nd/resources/library.asp), and events (http://www.neglected-delinquent.org/nd/events.asp).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: No recent independent evaluation has been undertaken of the N or D program. However, performance management data, especially long-term outcome data, will help determine program impact. In addition, some States and localities have initiated independent evaluations of programs for youth who are N or D. For example, an independent evaluation of Massachusetts's Department of Youth Services (Part D, Subpart 1 grantee) is being conducted by the University of Massachusetts at Amherst.

Evidence: Results from the UMass study are forthcoming (http://www.donahue.umassp.edu/press/news/dyseinit).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program cannot tie funding levels to achievement of specific outcomes. However, he budget materials for the program show the full cost of administering the program.

Evidence:

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Beginning with the program's national data collection in 2004-2005, the program made a significant strategic shift to better document the program's progress towards meeting its statutory purpose and goals. The expanded data collection also established an annual accountability process in which grantees receive information from which they can make program decisions and improvements. Grantees now annually provide performance data that are specifically aligned with the goals of the program. Program staff vet performance data elements and measures in a number of ways and solicit feedback from State Part D Coordinators, local program directors, and experts in the N/D field. Additionally, the measures and goals are reviewed annually for appropriateness by Department officials and performance management experts at the program's technical assistance center. Each of the new measures has baselines and targets set and data are now being collected annually rather than every three years.

Evidence: The Department provided help preparing grantees for the program's increasing focus on outcomes and performance. For example, to help States in collecting their data and fulfilling the Department's reporting request, the Center developed line by line reporting guides for both the 2004-2005 and 2005-2006 cycles (see http://www.neglected-delinquent.org/nd/data/datacorner.asp#CSPR). To help improve the quality of academic performance data used for one of the program's annual and long-term measures, the Center developed A Brief Guide for Selecting and Using Pre-Post Assessments (http://www.neglected-delinquent.org/nd/resources/spotlight/spotlight200604a.asp), hosted two Webinars, and developed several other web products (see http://www.neglected-delinquent.org/nd/data/datacorner.asp#Pre-Post). The Department's Consolidated State Performance Report contains details about the annual data collection and reporting requirements.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Under the program's reauthorization in NCLB, the first data were collected for the 2004-05 school year. State agencies and local educational agencies must describe how the program has affected the ability of participants to maintain and improve educational achievement, accrue school credits toward grade promotion or graduation, make the transition to a regular program or other education program operated by a local educational agency, complete secondary school (or its equivalent), obtain employment after leaving the facility or institution, and, as appropriate, participate in postsecondary education and job training programs. The Department regularly monitors State agencies' program implementation, including required data collection and reporting systems. Data will continue to be collected to inform program management and further implement program improvements.

Evidence: The Consolidated State Performance Report includes required data collection for N and D programs annually.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Grantees and subgrantees are held accountable through extensive fiscal reviews and Federal monitoring for spending and results. In addition, at the State and local levels, contractors are held accountable through planning and implementation requirements by the State educational agencies. At the Federal level, the contractor responsible for program technical assistance is held accountable through fiscal monitoring and regular performance assessments (e.g., regularly assessed customer satisfaction scores). As a part of the President's Management Agenda, the Department has implemented an agency-wide system--EDPAS--that links employee performance to progress on strategic planning goals. The N/D Program Manager has specific EDPAS Performance Standards for the Part D program such as: "maintains ongoing analysis of the educational services provided by States for N/D programs. Develops data collection instruments and instructions. Oversees the technical assistance and onsite monitoring by ED related to N/D statutory requirements for States."

Evidence: As of April 2007, 45 of the 52 grantees have undergone the Federal monitoring since 2002. EDPAS agreement for program manager.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: All Neglected or Delinquent program funds are obligated by the Department on July 1st of the fiscal year, the first day of availability. Grantees and subgrantees are subject to extensive fiscal reviews and Federal monitoring for spending to ensure that funds are used for the purposes of the program.

Evidence: GAPS system drawdowns demonstrate timely obligation of funds. As of April 2007, 45 of the 52 grantees have undergone Federal monitoring since 2002.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: A new protocol for ED monitoring combines N/D monitoring with that of other Title I programs, such as Even Start and Education for Children who are Homeless, resulting in efficiencies at the Department level. The Department has also established a formal efficiency measure: the cost per diploma or equivalent. A program cost of $5,095 per diploma is the baseline for this measure, with a 6 percent reduction in cost per year as the annual target. In addition, the program is staffed by less than one-half an FTE and monitoring and technical assistance are carried out through competitive sourcing.

Evidence: Information on the Department's Title I monitoring process, including grantee-specific reports, is available at http://www.ed.gov/admins/lead/account/monitoring/index.html. Efficiency measure and data are from the program performance plan and reports.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Department officials regularly collaborate with Federal program officers from other ED, Labor, and Justice programs. Program staff have volunteered their time and resources to present at regional conferences and "webinars" (web-based seminars) and invite the participation of representatives from other agencies for program events. Program managers are directly involved in the work of the Coordinating Council for Juvenile Justice and Delinquency Prevention, which includes major Federal partners serving this population, such as officials for the Department of Labor (DOL), Department of Justice (DOJ), and the Department of Health and Human Services (HHS). For example, program officers from ED and the Office of Juvenile Justice and Delinquency Prevention (OJJDP) collaborated to design the scope of work for a new DOJ center established by congressional earmark to provide technical assistance to State juvenile justice agencies on No Child Left Behind. Program staff also work with the Coordinating Council for Juvenile Justice and Delinquency Prevention, which includes major Federal partners serving this population and meets on a quarterly basis. This Council concluded that all programs should focus on re-entry and transition issues. The Council assists the Title I, Part D program to learn about resources provided by others to its population and about the initiatives undertaken and opportunities provided by various offices and agencies. For example, the Department of Labor recently announced funding to review transition needs for institutionalized youth. ED staff learned of the initiative through the Council and subsequently shared this information with N/D grantees.

Evidence: See: http://www.criminologycenter.fsu.edu/p/nclb-about.php; http://www.juvenilecouncil.gov/2006nationalconference/ and http://www.neglected-delinquent.org/nd/events.asp for more information on collaboration and coordination activities described in the Explanation section.

Does the program use strong financial management practices?

Explanation: The Department undertakes fiduciary monitoring of States, including monitoring the GAPS system for drawdowns and ensuring that when schools close, funds are shifted to other areas of need.

Evidence: GAPS reports; Monitoring reports; Monitoring results available at http://www.ed.gov/admins/lead/account/monitoring/index.html

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Monitoring revealed that States faced difficulty in the area of reservation of funds for transition services as well as in data collection. Program managers subsequently adjusted the focus of technical assistance efforts to address those needs by providing topic-specific training on transition strategies and innovations through a number of activities, including four regional conferences on transition.

Evidence: Monitoring reports; monitoring results available at http://www.ed.gov/admins/lead/account/monitoring/index.html

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The content of ED's monitoring is based on States' obligation to provide guidance and support to subgrantees, based on the requirements of the No Child Left Behind Act (NCLB). Monitoring implementation includes taking a close look at how States have instituted policies, systems, and procedures in order to ensure LEA, SA, and school compliance with the statute and regulations. The requirements and principles are embedded in the three monitoring areas of standards, assessment and accountability; instructional support; and fiduciary responsibilities. During visits, staff look at instructional practices, including how money is used to meet academic outcomes and whether generally accepted evaluation procedures are used to assess progress and the quality of programming. The SASA team provides a comprehensive monitoring report to the SEA within 45 business days of the on-site visit. The report contains recommendations, findings, and required actions. As of April 2007, 45 state monitoring visits have been completed.

Evidence: Consolidated reports, monitoring reports, and technical assistance center data.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The Department collects grantee performance data through the Consolidated State Performance Report. Until recently, data collection was required at least once every three years. Starting with the 2004-05 school year, the Department has collected data from grantees on an annual basis. The Department currently shares available data with local directors and makes data available to the public via a website developed by the NDTAC. Profile pages are available on the website with State-specific data. The Department has incorporated the 2004-05 CSPR into these pages and will provide annual updates with each new year of collected data. The center also released the first in an ongoing series of data briefs based upon the collected data.

Evidence: See http://www.neglected-delinquent.org/nd/data.asp.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data measuring performance on both of the two long-term measures failed to show progress in meeting annual targets. The percenatge of N or D students earning high school course credits fell from 55.95 percent in 2005 to 47.68 percent in 2006. The percentage of N or D high school students showing improvement in reading fell from 72.53 percent to 70.11 percent. The Department recognizes that States may have struggled in the first data collection to provide accurate student counts and will be identifying grantees in need of targeted TA in this area.

Evidence: Program Performance Plans and Reports.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program met the annual performance targets for two of the four performance measures in 2006. The percentage of N or D students who obtain a diploma or equivalent and the cost per high school diploma or equivalent exceeded the annual performance targets for 2006. The percentage of N or D students earning high school course credits and the percentage of N or D students showing improvement in reading failed to meet the annual performance targets for 2006.

Evidence: Program performance plan and report.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department recently developed an efficiency measure to quantitatively assess cost effectiveness of the program. This measure tracks the cost per high school diploma or equivalent by taking the ratio of the number of students who are N or D that earned a high school diploma or equivalent to the cost of the program. In the 2005-06 school year, the program cost per diploma fell to $4,421, a 13 percent decrease over the previous year.

Evidence: Program performance plan and report.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: No comparable programs serve this specific population of incarcerated youth.

Evidence:

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No recent independent evaluation has been undertaken of the N or D program. A statutorily required report to Congress has been compiled using data from the 2004-05 school year. This report includes indicators that could inform performance and effectiveness.

Evidence: