Detailed Information on the
Family Violence Prevention and Services Program Assessment

Working to develop appropriate national grantee-supported performance outcome measures to demonstrate improved efficiencies or cost effectiveness.

Designing and implementing local, state, and national evaluation models in consultation with stakeholders and experts to ascertain the effectiveness and efficiency of program activities.

Coordinating intra-agency efforts to improve violent-crime reducing services, specifically domestic violence prevention and intervention.

Working to develop appropriate national grantee-supported performance outcome measures to demonstrate improved efficiencies or cost effectiveness. Milestone: Implementing pilot programs to test the information collection activities related to shelter-specific outcome measures.

Coordinating efforts with other federal agencies to improve violent-crime reducing services. Milestone: Expansion of the Demonstration Projects in the Interagency Agreement between the Indian Health Service (IHS) and Administration for Children and Familes (ACF) to a total of 20 projects by September 2007.

Designing and implementing local, state, and national evaluation models in consultation with stakeholders and experts to ascertain the effectiveness and efficiency of program activities. Milestone: Finalize the Statement of Work (SOW) with National Institute of Justice(NIJ)/Department of Justice (DOJ) for the evaluation of the domestic violence emergency shelters.

Designing and implementing local, state, and national evaluation models in consultation with stakeholders and experts to ascertain the effectiveness and efficiency of program activities. Milestone: Evaluation of the shelter system will be initiated.

Coordinating efforts with other federal agencies to improve violent-crime reducing services.

Measure: Increase through training the capacity of the National Domestic Violence Hotline to respond to an increase in the average number and the type of calls per month (as measured by the average number of calls per month to which the hotline responds).

Explanation:

Measure: Shorten the average "wait time" (on calls to the National Domestic Violence Hotline) in order to increase the number of calls responded to and that provide needed information to callers.

Explanation:

Measure: Reduce the Family Violence Prevention Services Act (FVPSA) dollars spent per "bed night."

Explanation:Measure 14D originally considered the cost of "bed nights," or nights spent in a domestic violence shelter by adult females, adult males, or children. This measure will require some modification as it will change in the upcoming months. The program is seeking to concentrate on the maitenance of the quality of services, the cost of core services in battered women's shelters compared to the cost in other publicly funded shelters (if possible), and the impact of extensive volunteer commitments to any efficiency measure.

Is the program purpose clear?

Explanation: Shelter: Purpose is to assist states to provide shelter and related assistance for victims of family violence and their dependents. Hotline: To operate a national toll-free 24 hour, 365 day hotline to provide information and assistance to victims of domestic violence.

Evidence: Shelter:Section 302(1) and (2) of the Family Violence Prevention and Services Act: Hotline: Section 316 of P.L. 98-457 as amended.

Does the program address a specific and existing problem, interest or need?

Explanation: Shelter: The National Crime Victimization Survey indicates that there were more than 790,000 victims of intimate partner violence in 1999. Intimate murder accounts for about 9% of murders that occur nationwide. Hotline: The Hotline receives an average of over 13,000 calls each month from across the U.S. and its territories, with the majority of those calls from domestic violence victims/survivors. In 2003, the Hotline received its one millionth call.

Evidence: Shelter: Uniform Crime Reporting System, the National violence Against Women Survey, The National Crime Victimization Survey, and the National Family Violence Survey. Hotline: NDVH Semi-Annual Program Report, Sept. 30, 2003

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Shelters: FVPS is the only federal program with a dedicated funding stream for shelters for battered women and their dependents. The appropriation supplements not supplants other resources acquired by States through local and private contributions. States are required to provide 70% of their FV funding to shelters and 25% for related assistance. Shelter programs funded by States are not duplicative of DOJ's proposed Family Justice Centers--which are multi-agency DV service centers that do not have shelter components. Hotline: NDVH is the only national 24-hr, toll-free hotline that accomodates callers from the US and its territories and complements state-based hotlines.

Evidence: Shelters: FVP funding, as a percentage of states funding for DV services, has increased to approx. 36% over the past decade. However, in several of the less populated states the % of FVPSA funding may be closer to 80%. Hotline: NDVH Semi-Annual Program Report, Sept. 30, 2003.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Shelters: FV is a designated annual appropriation which supports state efforts to fund their shelter system. The formula grant nature of the program allows the States the flexibility to provide a combination of shelter stays/related assist. & non-residential services. While FV does not fund all sheters nor all of the shelters' services, it is estimated that the formula program contributes to the funding of between 1,300 to 1,600 shelters and safehomes annually. There are no direct cash benefits to the recipients. Some of the related services that can be provided through the FV funding include emergency transpo., emergency childcare, individ.counseling, & legal advocacy. Hotline: NDVH is efficiently run from an integrated phone and computer call-center in Texas and contintues to respond to the steadily increasing number of phone calls while ensuring a consistent quality of services. It employs bilingual advocates, technology for deaf and hearing-impaired callers, and access to translators in 139 languages. NDVH provides constant training for staff and volunteers.

Evidence: Shelters: Report to Congress, 1999-2001. Hotline: Section 316 FV statute; NDVH Semi annual Program Report, Sept. 30, 2003.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Shelter: State agencies administer shelter grants and monitor sub-state grantees. States also collaborate with the DV coalitions, which are the membership organizations of the shelters, to assure equitable distribution of grant funds in rural and urban areas (section 303(a)(C). Shelters only provide services to victims of family violence and their dependents and are estimated to house more than 300,000 woman and children during a program year and provide an array of core services and non-residential programs for families in abusive situations. Hotline: NDVH reports on the types of calls answered. In 2003, the majority of calls were from victims of DV and family and friends of victims.

Evidence: Hotline: Data from NDVH Semi annual Program Report, Sept 30, 2003

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Shelter: The DV community has established the Documenting Our Work (DOW) work group to consider the planning, implementation, measurement and effectiveness of the domestic violence services--including shelters. The effort to determine outcome measures is considered long-term and difficult. The discussion of outputs which might be considered as proxies, i.e., safety plan process (written or unwritten) fall short of the conceptualization and clarity sought for in the results of these programs. Hotline(YES): Build the capacity of the NDVH to receive and respond to an increase in the average number calls per month.

Evidence: Shelter: The GPRA FY 2002 Performance Report for the Tribes cites the technical assistance and information available to the Tribes to assist in increasing the number of sponsored family violence programs. Hotline: NDVH Semi-Annual Report, September 30,2003. See also FY2002 GPRA Performance Report.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation:  

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Shelters: This program has one measure: to increase the total number of tribal shelters operating on Trust Lands and Tribal reservations. While this is intended to address a laudable goal of providing increased assistance to underserved communities, it does not capture the larger purpose of the program. Hotline: Increase the number of calls answered, lower the number of calls dropped or on hold too long, and respond more comprehensively to sexual assault calls.

Evidence: Shelters: FY 2002 GPRA Performance Report and FVPS Report to Congress Fy 1999-2001. Hotline: NDVH Semi-Annual Program Report and FVPS Report to Congress FY 1999-2001.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Shelters: Baseline targets have been set for Tribes and Alaskan villages that are challenging. Hotline: Increases in the number of calls answered relates to the ability to diminish the hold time thus reducing the number of calls that are dropped.

Evidence: Shelters: FY 2002 GPRA Performance Report and FVPS Report to Congress FY 1999-2001. Hotline: NDVH Semi-Annual Program Report, September 30, 2003 and FVPS Report to Congress FY 1999-2001 and FY 2002 GPRA Performance Report.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Shelters: 2.5 The domestic violence coalitions and the national domestic violence organizations are considered partners in the effort to improve the long term effectiveness of family violence programs. The DOW is an example of the domestic violence community's partnering with the family violence program to provide and help shape improvements in the delivery of domestic violence services. The DOW group has been working for the past several years in recognition of the fact that effective planning and performance based outcomes for programs have to be ultimate goals. The quest for measurable outputs and outcomes do not overshadow the commitment to providing victims of abuse with the services they feel is required and that can be delivered, i.e., how women perceive the services they receive, if their needs were met or were ignored, what kinds of services did they need that weren't offered, the impact of the stay on a woman with children ' did they understand they weren't the reasons for the domestic violence; and if the children and women feel more safe. Hotline: The NDVH has several non-governmental partners that are committed to the annual and long-term goals of the project.

Evidence: Shelters: See FY 2002 GPRA Performance Report and FVPS Report to Congress FY 1999-2001. Hotline: NDVH Semi-annual Program Report, September 30, 2003. See also FY 2002 GPRA Performance Report and FVPS Report to Congress FY 1999-2001.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Shelters: A major study of the nation-wide shelter system is in its initial phase. Other national studies are indicative of the progress being made in DV and confirm the need for the continuation of the FVPS program. Hotline: Two studies have been done regarding the Hotline and its effectiveness.

Evidence: Shelters: See NIJ interagency agreement; See also FY 2002 GPRA Performance Report and FVPS Report to Congress FY 1999-2001. Hotline: A study by the University of Texas at Austin (1997), and a study by Macro Associates (1999).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Shelters: The program's budget is not performance based. Program budget is dependent upon appropriation committee decisions. Hotline: Budget is not performance based. It is dependent upon appropriation committee decisions.

Evidence: Shelters: See respective conference reports. Hotline: see respective conference reports.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Shelters: In the process of developing discretionary priorities, issues and concerns that arise from the field, papers, studies, annual reports, and discussions with state and local partners and nonprofit partners are considered. For example: it was through discussions with the National Council of Juvenile and Family Court Judges that FV was made aware of the troublesome intersection of child protection services and domestic violence service providers in the co-occurrence of child maltreatment and domestic abuse. In response to this issue FV issued grant awards to address this intersection either through projects that provided an exchange of information, joint awareness training, or the development of mutual protocols. Hotline: The Hotline data collection program collects, analyzes and disseminates national data on the nature, scope and impact of FV in the US.

Evidence: Shelters: FY 2002 GPRA Performance Report; June 2003, DOW Report. Hotline: FY 2002 GPRA Performance Report; NDVH Semi-annual Program Report, September 30,2003; Training and Call Protocols with Family Advocacy Program, DOD.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Shelters: The states report on an annual basis on the effectiveness of their sub-state grantees (shelters). State agencies and Tribes with shelter programs are required to undergo A-122 audits. Performance reports also identify programmatic and management issues. For example, coalitions raised the issues of cultural competency and the need to provide services to underserved communities. FV responded with program guidance about the need for services to communities underserved because of racial and other barriers such as the aged, the deaf community, and those restricted from services through language barriers. Hotline: Grantee provides semi-annual and annual reports. These reports enable programmatic and strategic requirements to be implemented.

Evidence: Shelters: See Reports to Congress, 1999-2001; State Administering Agency Reports; Section 303a(4) of the FVPSA; Program applications, Application checklist. Hotline: NDVH Semi-annual Program Report, September 30, 2003; Report to Congress, 1999-2001.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Shelters: Federal managers are held accountable through annual work plans and performance plans. Program managers at the state level are held accountable through audits, state agency assurances, program reports and assurances that the sub-state programs are being effective in carrying out the purposes of the grant. The sub state grantees have to provide the state agency with an assessment of their activities. Hotline: Managers are encouraged to correct deficiencies in the operation of the hotline. Managers are asked to reduce the wait time.

Evidence: Shelters: Federal manager work plans and employee performance plans; Annual State Grant Performance Reports (Section 303(a)(4) of the Family Violence statute; FVPS Report to Congress, 2001. Hotline: NDVH Semi-Annual Program Report, September 30, 2003; Report to Congress, 1999-2001.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Shelters: States receive a single annual appropriation and have a 2-year expenditure period. No extensions are provided. Hotline: Upon receipt and approval of the Hotline plan and application, funds are awarded to the grantee. The award is an annual award and the grantee reports on a semi annual basis.

Evidence: Shelters: FV statute, Section 304; Regs. CFR 92; SF-269 Financial Status Reports; Grants Award Terms and Conditions. Hotline: Grantees's FS 269; Regs. CFR 74 and Semi-Annual Program Reports; Grant Award Terms and Conditions.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Hotline: Has the ability to track their incoming calls and the technical ability to project cost effectivenss with additional IT improvements.

Evidence: Shelters: June 2003, DOW Report. Hotline: NDVH Semi Annual Program Report, September 30, 2003; Report to Congress, 1999-2001

Does the program collaborate and coordinate effectively with related programs?

Explanation: Shelters: FV has a number of collaborative efforts that involve offices within HHS and with DOJ. For example, 24 grants have been provided to TANF agencies to collaborate with DV providers and 26 grants to child protective service agencies. Hotline: Effectively collaborates and coordinates their activities with both public and private entities.

Evidence: Shelters: Interagency agreements with the Violence Against Women Office, DOJ (Greenbook); National Institute of Justice (Shelter System Evaluation); and the Indian Health Services (Health Care Response to DV); Report to Congress, 1999-2001. Hotline: NDVH Semi-Annual Pogram Report, Sept. 30, 2003; Report to Congress, 1999-2001.

Does the program use strong financial management practices?

Explanation: Shelters: Administrating agencies must comply with the Single Audit Act requirements. Agencies must submit financial status reports annually on how FVPS funds are used. Hotline: Administrating agencies must comply with the Single Audit Act requirements. Agency must submit financial status reports annually on how FVPS funds are used.

Evidence: Shelters: OMB Circular A-128; Departmental Grants Management Report Requirements; Financial Status Report SF-269. Hotline: OMB Circular A-128; FVPS Statute; FVPS Program Announcement.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Shelters: FV is reliant upon the coalitions to assist the individual shelters within their respective states with training and technical assistance to address their particular problems in program administration, management and program service delivery. The coalitions also do needs assessments that are applicable to the states shelter programs. Hotline: Has repsonded to several evaluations with changes in procedures and modifications in their protocol to accomodate calls from the Deaf community, individuals who have been sexually assaulted, and calls from members of the armed forces.

Evidence: Shelters: Section 311 , FV Statute; Report to Congress, 1999-2001; FY 2002 GPRA Performance Report. Hotline: NDVH Semi-annual Program Report; FY 2002 GPRA Performance Report; Training and Call Protocols with Family Advocacy Program, DO; Report to Congress, 1999-2001.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Shelter: An annual review of the adminsitering agencies FVPS plan application is conducted to determine completeness, document assurances and to determine compliance with the FVPS statute. Administrering agencies in turn document compliance of the sub-state grantees. Hotline: Annual review of the grantee's application and adherence to the grant requirements and legislative guidance.

Evidence: Shelters: Annual State FVPS plans, section 303(a) of the FVPSA statute, annual report documents. Hotline: Grantees application, annual reports, grantee conferences, and site visits.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Shelters: The program collects data from the administering agencies and makes it available through the Report to Congress. The public can obtain the executive summary of the report on the website and request the full report. Hotline: Performacne data may be found in the Report to Congress that can be made available to the public through the NRC.

Evidence: Shelters: Report to Congress, 1999-2001. Hotline: Report to Congress, 1999-2001 - made available through the NRC/ PCADV (www.pcadv.org).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Shelters: Long-term performance goals are being developed. Hotline: concentrated on reducing the "wait" on incoming calls, thus reducing the number of calls lost. Arranged for training of advocates on the Hotline to improve their handling of sexual assualt calls and calls from military persons.

Evidence: Shelters: Secondary analysis of 1993-99 data on dv, NCVS revised 8/5/99; FY 2002 GPRA Performance Report; June 2002, DOW Report. Hotline: NDVH Semi Annual Program Report; Report to Congress, 1999-2001; Training and Call Protocols with Family Advocacy Program, DOD.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Shelters: Annual performance goal for the Tribes is dependent on the extent of turnover and the possible drop-out of participating Tribes and villages. Hotline: Performance Goals are achieved through ongoing efforts to improve the call response.

Evidence: Shelters: Tribal reports, Tribal grantee list: GPRA reports. Hotline: NDVH Semi-Annual Program Report, Sept. 30, 2003; Report to Congress, 1999-2001; FY 2002 GPRA Performance Report.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Shelters: There are no efficiency measures for the Tribal shelters; this question does not seem applicable. Hotline: Performance target is to reduce wait time for incoming calls thus reducing the number of dropped calls.

Evidence: Shelters: N/A . Hotline: NDVH Semi-Annual Program Report, Sept/ 30, 2003; Report to Congress, 1999-2001; FY 2002 GPRA Performance Report.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Shelters: There are both public and private programs that are similar. The Shelters house more than 300,000 woman and children during a program year and provide an array of core services and non-residential programs for families in abusive situations. Hotline: There is only one NDVH taking crisis calls and referring them to individual services. Their efforts compare favorably with other national call lines established to assist individuals with particular problems or who need information.

Evidence: Shelters: Report to Congress, 1999-2001. GPRA Performance Report FY2002; State agency and Coalition reports FY2002; Special Outreach Reports FY2002, BJS Statistics, April 1984 & August 1995; individual studies and evaluations. Hotline: NDVH Semi-annual Program Report, Sept. 30, 2003; Report to Congress, 1999-2001; FY 2002 GPRA Performance Report.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Shelters: Evaluations of the shelter programs are done locally by State partners based on criteria that they have established that shelters have to meet. Because of limited followup due to staff constraints and the reluctance of previous clients, the results achieved criteria are elusive. Hotline: monitors their call status for delays and drops and all indications are that there dropped calls numbers are decreasing.

Evidence: Shelters: State annual reports and the assessment of the completion of shelters' objectives during the program year. Hotline: NDVH Semi annual Program Report, Sept. 30, 2003; Report to Congress, 1999-2001; FY 2002 GPRA Performance Report and the Assessment of NDVH, 1997 - University of Texas School of Social Work.