This is the accessible text file for GAO report number GAO-04-245 
entitled 'Grants Management: Despite Efforts to Improve Weed and Seed 
Program Management, Challenges Remain' which was released on March 24, 
2004.

This text file was formatted by the U.S. General Accounting Office 
(GAO) to be accessible to users with visual impairments, as part of a 
longer term project to improve GAO products' accessibility. Every 
attempt has been made to maintain the structural and data integrity of 
the original printed product. Accessibility features, such as text 
descriptions of tables, consecutively numbered footnotes placed at the 
end of the file, and the text of agency comment letters, are provided 
but may not exactly duplicate the presentation or format of the printed 
version. The portable document format (PDF) file is an exact electronic 
replica of the printed version. We welcome your feedback. Please E-mail 
your comments regarding the contents or accessibility features of this 
document to Webmaster@gao.gov.

This is a work of the U.S. government and is not subject to copyright 
protection in the United States. It may be reproduced and distributed 
in its entirety without further permission from GAO. Because this work 
may contain copyrighted images or other material, permission from the 
copyright holder may be necessary if you wish to reproduce this 
material separately.

Report to the Chairman, Committee on the Judiciary, House of 
Representatives:

United States General Accounting Office:

GAO:

March 2004:

GRANTS MANAGEMENT:

Despite Efforts to Improve Weed and Seed Program Management, Challenges 
Remain:

GAO-04-245:

GAO Highlights:

Highlights of GAO-04-245, a report to the Chairman, Committee on the 
Judiciary, House of Representatives 

Why GAO Did This Study:

The Weed and Seed program, within the Department of Justice’s Office of 
Justice Programs (OJP), aims to prevent and reduce violent crime in 
targeted neighborhoods, but it cannot optimize its effectiveness 
without sound management practices. In 1999, GAO made four 
recommendations to the Executive Office for Weed and Seed (EOWS) to 
improve the program’s management, including (1) developing adequate 
internal controls to fully document decisions, (2) improving program 
monitoring, (3) developing criteria for determining when sites have 
become self-sustaining and when to reduce or withdraw program funding, 
and (4) developing additional performance measures. GAO did this study 
to assess progress in implementing these recommendations.

What GAO Found:

Despite some progress toward addressing GAO’s recommendations aimed at 
improving program management, GAO’s review shows that EOWS has not 
fully implemented the management improvement recommendations GAO made 
in 1999. First, although EOWS has revised its internal controls to 
require that significant qualification and funding decisions be 
documented and readily available in the central grant files for review, 
EOWS has not always ensured that its policies and procedures were 
followed, for the grant files GAO reviewed. Second, EOWS reported 
taking a number of actions intended to improve program monitoring, such 
as mandating the timely submission of progress reports and adequate 
recording of site visits as GAO recommended. Nonetheless, GAO found 
that while EOWS was able to provide such documentation before its 
review ended, documentation was not available in some of the central 
grant files GAO reviewed. Thus, the documentation was not readily 
available for external reviewers, as required by OJP policies and GAO’s 
internal control standards. Third, GAO found that EOWS still lacks 
fully developed criteria to determine when sites become self-sustaining 
and when to reduce or withdraw Weed and Seed funds because of the level 
of sustainability, even though sustainability is a central goal of the 
program. At the time of GAO’s review, no site’s funding had been 
reduced or withdrawn because of sustainability during the 13 years of 
the program’s existence. Fourth, EOWS has not developed outcome 
performance measures that can be used to adequately track progress 
toward program outcomes of the Weed and Seed program. While EOWS has 
initiated studies on how to develop performance measures, at the time 
of GAO’s review, none of these studies had been completed. Without 
requirements to monitor improvements and assign accountability, 
progress will be difficult to achieve.

What GAO Recommends:

GAO recommends that the Attorney General require the Assistant Attorney 
General for OJP to ensure that the Executive Office for Weed and Seed 
fully implement the intent of GAO’s previous recommendations. 

In commenting on a draft of this report, the Department of Justice 
agreed to strengthen controls on maintaining documentation and take 
further steps to define and apply criteria for self-sustainability. 
Justice believes that the studies currently under way will help develop 
outcome measures. 

www.gao.gov/cgi-bin/getrpt?GAO-04-245.

To view the full product, including the scope and methodology, click on 
the link above. For more information, contact Laurie Ekstrand at (202) 
512-8777 or Ekstrandl@gao.gov.

[End of section]

Contents:

Letter:

Results in Brief:

Background:

Internal Controls Have Been Developed, but Challenges Remain:

EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but 
Challenges Remain:

Despite Some Steps, Challenges Remain in Developing Criteria for Self-
Sustainability:

Performance Measures Generally Did Not Track Program Outcomes:

Conclusions:

Recommendations for Executive Action:

Agency Comments and Our Evaluation:

Appendix I: Scope and Methodology:

Appendix II: GAO Site Visit Summary:

Appendix III: Proposed Legislation Could Affect Self-Sustainability:

Appendix IV: Executive Office for Weed and Seed Selected Activities:

Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003:

Appendix VI: Description of EOWS's Ongoing and Completed Studies:

Appendix VII: Comments from the Department of Justice:

Appendix VIII: GAO Contacts and Staff Acknowledgments:

GAO Contacts:

Staff Acknowledgments:

Related GAO Products:

Tables:

Table 1: EOWS Funding History for Fiscal Years 1999-2003:

Table 2: Sites' Weed and Seed Funding History--Fiscal Years 1999-2003:

Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh, 
Charleston, and North Charleston:

Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in 
Pittsburgh, Charleston, and North Charleston:

Abbreviations:

AAG: Assistant Attorney General:

COPS: Community Oriented Policing Services:

DFCSP: Drug-Free Communities Support Program:

EOWS: Executive Office for Weed and Seed:

FBI: Federal Bureau of Investigation:

FY: fiscal year:

GMS: grant management system:

GPRA: Government Performance and Results Act:

HUD: Housing and Urban Development:

JRSA: Justice Research and Statistics Association:

OJP: Office of Justice Programs:

OMB: Office of Management and Budget:

OR: official recognition:

United States General Accounting Office:

Washington, DC 20548:

March 24, 2004:

The Honorable F. James Sensenbrenner, Jr.: 
Chairman: 
Committee on the Judiciary: 
House of Representatives:

Dear Mr. Chairman:

The Department of Justice's (Justice) Weed and Seed program proposes to 
"weed" out crime from targeted neighborhoods and "seed" them with a 
variety of programs and resources to prevent crime from recurring. Weed 
and Seed sites have sponsored activities such as police bike patrols, 
community cleanups, youth and recreational activities, and computer 
training. According to Justice, a central tenet of the Weed and Seed 
program is for local Weed and Seed sites to develop partnerships with 
other federal, state, and local governments and private sector agencies 
to leverage federal Weed and Seed grant funds with additional resources 
from these partners to promote weeding and seeding activities. These 
additional resources are intended to help the sites achieve the goal of 
becoming self-sustaining after Weed and Seed funding ends and to 
sustain crime reduction and community revitalization activities to 
ensure stable communities.

This report responds to your request to assess the Executive Office for 
Weed and Seed's (EOWS) efforts to implement the management improvement 
recommendations we made in 1999.[Footnote 1] It is not intended to 
evaluate the overall management or results of the program. In our 
previous report on the Weed and Seed program, we recommended that EOWS 
improve its management of the Weed and Seed program. Specifically, we 
recommended that EOWS (1) develop adequate internal controls to ensure 
that the basis and the rationale for new and existing site 
qualification and funding decisions are always fully documented, (2) 
improve program monitoring to ensure that sites meet the grant 
requirement of submitting progress reports and that EOWS site visits 
are documented, (3) develop criteria for determining when sites are 
self-sustaining and when to reduce or withdraw program funding, and (4) 
develop additional performance measures that track program outcomes. 
This report summarizes our assessment of EOWS's efforts to address our 
recommendations.

To obtain information on EOWS's efforts, we reviewed relevant 
documents, including EOWS's policies and procedures, monitoring 
documentation, and agency staffing and budget data. We reviewed GAO, 
Congressional Research Service, and Justice's Office of Inspector 
General reports. In addition, we interviewed officials from EOWS, the 
Office of Justice Programs, other entities in the Justice Department, 
and researchers in performance measurement and evaluation at the 
Justice Research and Statistics Association and the Urban Institute in 
Washington, D.C. We visited and interviewed program staff at three Weed 
and Seed sites. We reviewed 30 randomly selected Weed and Seed grant 
files to identify the actions taken by EOWS for those grants to 
document its qualification and funding decisions and the steps EOWS 
took to ensure that grant documentation requirements were met. Since 
the files we reviewed were not representative of all EOWS grant files, 
we cannot project the results to the larger population. However, the 
information helps to identify the level of implementation of our 
recommendations for those grants. Additional information about the 
report's scope and methodology is presented in appendix I. We conducted 
this engagement in accordance with generally accepted government 
auditing standards.

Results in Brief:

Despite some progress toward addressing our recommendations aimed at 
improving program management, our review shows that EOWS has not fully 
implemented any of the management improvement recommendations we made 
in 1999. First, EOWS has revised its internal controls to require that 
significant qualification and funding decisions be documented and 
readily available in two types of centralized files: official 
recognition and official grant files. However, EOWS has not always 
ensured that the documentation was readily available and that its 
policies and procedures for internal controls were followed for the 
official recognition and official grant files we reviewed. Second, EOWS 
reported to us that it had taken a number of actions intended to 
improve program monitoring, such as mandating the timely submission of 
progress reports and adequate recording of site visits, as we 
recommended. Nonetheless, we found that while EOWS was able to provide 
such documentation before our review ended, documentation was not 
available in some of the central grant files we reviewed. Thus, the 
documentation was not readily available for EOWS management or external 
reviewers, as required by the Office of Justice Programs (OJP) policies 
and GAO's internal control standards. Third, we found that EOWS still 
lacks fully developed criteria we recommended that they develop to 
determine when sites have become self-sustaining and when to reduce or 
withdraw Weed and Seed funds because of the level of sustainability, 
even though sustainability is a central goal of the program. At the 
time of our review, no site's funding had been reduced or withdrawn 
because of sustainability during the 13 years of the program's 
existence. Fourth, EOWS began developing additional performance 
measures to better assess how well sites are meeting program 
objectives, as we recommended in 1999. However, our work showed that 
although EOWS collected data on a variety of activities taking place at 
Weed and Seed sites, they generally did not measure the extent to which 
grantees were weeding crime from neighborhoods and preventing it from 
recurring. While EOWS has initiated studies on how to develop 
additional performance measures, at the time of our review, none of 
these studies had been completed.

To further improve program management, we make a recommendation to the 
Attorney General for four actions to help ensure full implementation of 
the recommendations we made in our 1999 report. In commenting on a 
draft of this report, Justice partially agreed with our recommendations 
and mentioned actions being taken or planned in response.

Background:

The Weed and Seed program is a Justice discretionary grant program 
within OJP.[Footnote 2] The Assistant Attorney General for OJP is 
responsible for the overall management and oversight of offices within 
OJP, including EOWS. The Assistant Attorney General for OJP sets 
policies, promotes coordination among OJP bureaus and offices, and 
ensures that EOWS follows its policies and procedures. EOWS provides 
funding to grantees to help prevent and control crime and improve the 
quality of life in targeted high-crime neighborhoods across the 
country.[Footnote 3] It is a joint federal, state, and local program 
for coordinated law enforcement and neighborhood reinvestment. Federal 
program funding is to support Weed and Seed sites and to provide 
training and technical assistance.[Footnote 4]

The Weed and Seed program has grown since it began in fiscal year 1991 
with three pilot sites in Kansas City, Missouri; Omaha, Nebraska; and 
Trenton, New Jersey, and a relatively small investment of federal 
money. From fiscal year 1999 through 2003, the number of Weed and Seed 
sites increased from 163 to 221, while the total annual program budget 
generally increased from about $45 million to $70 million. See table 1 
for fiscal years 1999-2003 data on the Weed and Seed program, including 
EOWS's funding history. In fiscal year 2003, with a budget of about $70 
million, EOWS awarded grants to 221 Weed and Seed sites.

Table 1: EOWS Funding History for Fiscal Years 1999-2003:

Fiscal year: 1999; 
Unobligated balance carried forward from prior year: $4,122,668; 
Prior year recoveries and other reimbursable adjustments: $714,466; 
EOWS appropriation: $33,500,000; 
Funds from the asset forfeiture fund[A]: $6,500,000; 
Funds from the Housing and Urban Development (HUD)[B]: $0; 
Total funding available: $44,837,134; 
Number of funded sites: 163.

Fiscal year: 2000; 
Unobligated balance carried forward from prior year: $4,129,852; 
Prior year recoveries and other reimbursable adjustments: $2,066,682; 
EOWS appropriation: $33,500,000; 
Funds from the asset forfeiture fund[A]: $6,500,000; 
Funds from the Housing and Urban Development (HUD)[B]: $0; 
Total funding available: $46,196,533[C]; 
Number of funded sites: 177.

Fiscal year: 2001; 
Unobligated balance carried forward from prior year: $4,955,423; 
Prior year recoveries and other reimbursable adjustments: $1,269,708; 
EOWS appropriation: $33,925,200; 
Funds from the asset forfeiture fund[A]: $15,500,000; 
Funds from the Housing and Urban Development (HUD)[B]: $0; 
Total funding available: $55,650,331; 
Number of funded sites: 212.

Fiscal year: 2002; 
Unobligated balance carried forward from prior year: $584,278; 
Prior year recoveries and other reimbursable adjustments: $2,359,292; 
EOWS appropriation: $58,918,000; 
Funds from the asset forfeiture fund[A]: $0; 
Funds from the Housing and Urban Development (HUD)[B]: $10,000,000; 
Total funding available: $71,861,570; 
Number of funded sites: 218.

Fiscal year: 2003; 
Unobligated balance carried forward from prior year: $4,102,334; 
Prior year recoveries and other reimbursable adjustments: $2,713,913; 
EOWS appropriation: $58,542,000; 
Funds from the asset forfeiture fund[A]: $0; 
Funds from the Housing and Urban Development (HUD)[B]: $4,935,000; 
Total funding available: $70,293,247; 
Number of funded sites: 221. 

Source: OJP's Office of Budget and Management Services data. GAO did 
not verify the data.

Note: For EOWS funding history prior to 1999, see GAO/GGD-99-110.

[A] The proceeds from the asset forfeitures fund were used for federal 
and state law enforcement purposes. EOWS received these funds from 
Justice, under a reimbursable agreement, for the payment of various 
costs incurred by state and local law enforcement officers that 
participated in joint federal law enforcement operations with federal 
agencies.

[B] HUD made $10 million in additional funding available, under a 
reimbursable agreement, to help Weed and Seed sites reduce drug-related 
crimes in public housing.

[C] Does not add because of rounding.

[End of table]

EOWS is responsible for the national management and administration of 
the Weed and Seed program, including developing policy and providing 
guidance and oversight. EOWS currently administers the Weed and Seed 
program with a staff of 2 management officials, 10 grant monitors, 3 
support staff, 3 detailees,[Footnote 5] and 6 contractors.

Before a community can apply for Weed and Seed grant funding and become 
an eligible site, it must first apply for and gain official recognition 
from EOWS. In order to obtain official recognition, a potential Weed 
and Seed site must show in its application that it has a strategy for 
weeding and preventing crime. Once the application has been received, 
EOWS creates and maintains the official recognition files that are 
intended to include documentation such as the rationale for decisions 
to grant or deny official recognition to an applicant. Official 
recognition requires the U.S. Attorney in the area where the Weed and 
Seed site is to be located to organize a local steering committee made 
up of various federal, state, and local representatives, including 
residents, to be responsible for local administration of the 
program.[Footnote 6] For official recognition, a site is also required 
to develop a management plan, engage residents and other partners in 
its activities, and develop a comprehensive program to weed out crime 
and gang activity and seed the area with social services, economic 
services, and economic revitalization.

Weed and Seed program guidance requires that its sites show plans for 
addressing four required elements: (1) law enforcement; (2) community 
policing; (3) crime prevention and intervention, and substance abuse 
prevention, intervention, and treatment; and (4) neighborhood 
restoration. According to EOWS, law enforcement should attempt to 
eliminate the most violent offenders by coordinating and integrating 
the efforts of federal, state, and local law enforcement agencies in 
targeted high-crime neighborhoods. The objective of community policing 
is to raise the level of citizen and community involvement in crime 
prevention and intervention activities. Crime and substance abuse 
prevention, intervention, and treatment should include youth services, 
school programs, community and social programs, and support groups. 
Finally, neighborhood restoration should focus on assistance to 
distressed neighborhoods through economic and housing development. Weed 
and Seed sites fund a variety of law enforcement and community 
activities. For example, law enforcement-funded activities range from 
participation in a multijurisdictional task force to conducting bike 
patrols in the community. See appendix II for additional information on 
Weed and Seed activities at the sites GAO visited.

Applicants that are officially recognized are eligible to apply for 
funding for up to 5 years. OJP's Office of the Comptroller creates and 
maintains official grant files for sites that are awarded funding by 
EOWS. Applicants are rated based on the strategy they developed in 
response to the four required elements, as stated above.

In our 1999 report, we made four recommendations to help EOWS improve 
program monitoring and management. The first was to develop adequate 
internal controls to fully document significant qualification and 
funding decisions. We found that EOWS lacked internal controls 
requiring significant program management decisions be documented. 
Therefore, EOWS was not able to ensure that it was making the best 
decisions about allocating available funds. Our second recommendation 
was to improve program monitoring to ensure that sites met the grant 
requirement of submitting progress reports and that EOWS site visits 
were documented. We found that EOWS did not always ensure that local 
sites submitted progress reports and that grant monitors documented the 
results of site visits. Such documentation would help EOWS management 
and grant monitors determine how sites are meeting program objectives 
and how well sites are complying with grant requirements, and assist 
them in making future grant qualification decisions. Our third 
recommendation was to develop criteria for determining when sites have 
become self-sustaining and when to reduce or withdraw program funding. 
We found that although self-sustainability is central to the program, 
no site's funding had been reduced or withdrawn as a result of its 
efforts to become self-sustaining in the 9 years of the program's 
existence. Establishing such criteria is important because the Weed 
Seed program was founded on the premise that federal funding would 
continue for a finite period, after which a Weed and Seed site would be 
self-sustaining. Our fourth recommendation was to develop additional 
performance measures to track program outcomes. We found that EOWS's 
performance indicators generally tracked activities rather than program 
results; therefore, EOWS was not able to measure the success of the 
program.

EOWS generally agreed with three of the four recommendations presented 
in the report and discussed future actions it planned to take. EOWS 
officials disagreed with our recommendation on self-sustainability. 
They stated that developing criteria to ascertain self-sustainment is 
redundant since EOWS adopted a 5-year rule under which it could 
discontinue awarding funding to qualifying sites unless the sites 
expanded to an additional neighborhood site.

Internal Controls Have Been Developed, but Challenges Remain:

Although EOWS has developed internal controls intended to require that 
significant qualification and funding decisions be documented and 
readily available for review as we recommended in 1999, these policies 
and procedures are generally not being followed in the files we 
reviewed. In response to our 1999 recommendation on internal controls, 
EOWS established policies and procedures intended to ensure that 
significant qualification decisions were documented. In addition, OJP 
requires that qualification and funding decisions are to be documented 
in official recognition and official grant files. Our review of 20 
official recognition and 10 official grant files showed that some of 
EOWS's official recognition and official grant files were missing full 
documentation regarding the qualification and funding 
decisions.[Footnote 7] However, before our file review ended, EOWS 
officials produced further documentation, which they acknowledged was 
not in the official recognition and official grant files as required, 
but rather in the personal working files of grant monitors and thus not 
readily available to EOWS management and external reviewers. This lack 
of ready availability is not in keeping with EOWS's and OJP's policies 
and procedures, or with the Comptroller General's standards for 
internal controls.[Footnote 8] Without having official recognition and 
official grant files complete and readily available, it may delay and 
complicate EOWS officials' oversight of the documentation of 
qualification and funding decisions.

Policies and Procedures Have Been Developed to Document Significant 
Decisions:

Both EOWS and OJP policies and procedures have been developed to help 
ensure the documentation of significant decisions. In 2000, EOWS 
developed a policies and procedures guide in response to the 
recommendation we made in our 1999 report intended to ensure that 
significant qualification and funding decisions for new and existing 
sites[Footnote 9] were always fully documented, and further revised the 
guide in 2003. This guide requires that all documentation pertaining to 
official recognition decisions be kept in the official recognition 
files. EOWS is also required to follow the policies in the OJP Grants 
Management Policies and Procedures Manual, which requires that EOWS 
fully document program management decisions in both official 
recognition files and official grant files. The OJP manual specifically 
requires that the official recognition files and official grant files 
contain key documents such as ratings information, rejection letters, 
and applications.

Official Recognition and Official Grant Files Were Incomplete:

Despite EOWS's and OJP's efforts to require the documentation of 
qualification and funding decisions in official recognition and 
official grant files, some of EOWS's official recognition and official 
grant files we reviewed were incomplete at the time of our file review. 
In reviewing the official recognition files, we randomly selected 10 
files to review from the 94 files submitted for official recognition in 
fiscal year 2002. Additional information about how we selected the 
files is in appendix I. We found sufficient documentation in 7 of the 
10 files to determine the basis and rationale for decisions to award 
official recognition. However, in the remaining 3 files we were unable 
to determine the basis and rationale for such decisions because 
documentation was insufficient. For example, key documents such as the 
ratings information that EOWS grant monitors use to record their 
assessment of the official recognition applications did not contain the 
basis and rationale for the award decision.[Footnote 10] Additionally, 
OJP's grant policies and procedures require EOWS's grant monitors to 
prepare a rejection letter, informing applicants of reasons for 
rejection when funding is denied and place a copy of the rejection 
letter in the official recognition file. We reviewed all 10 official 
recognition files for the applicants that were eligible to apply for 
funding in fiscal year 2002 but were rejected. We found a rejection 
letter was missing in 7 of the 10 files.

We also reviewed documents in EOWS's official grant files to assess 
EOWS's efforts to comply with EOWS and OJP policies for fully 
documenting funding decisions. We randomly selected 10 official grant 
files to review from the 31 sites that were funded in fiscal year 2002. 
We found that none of the 10 official grant files fully documented 
funding decisions for Weed and Seed sites. For example, the 
application, a basic component of the official grant file, was missing 
in 8 of the 10 files.

Challenges Persist in Making Documentation Readily Available:

Several directives require that grant documentation be readily 
available. OJP's policies and procedures manual requires that 
documentation be kept readily available in the official recognition and 
official grant files, so that OJP and EOWS management can identify and 
resolve any problems or deficiencies in grantees' compliance with 
relevant policies and procedures. OJP's policies and procedures manual 
requires that all pertinent information that should be in the official 
recognition and the official grant files be kept in a centralized 
location to facilitate reviewing for completeness. In addition, the 
Comptroller General's guidance on internal controls in the federal 
government, Standards for Internal Control in the Federal Government, 
requires that the documentation for all transactions and significant 
events be readily available for examination. The Assistant Attorney 
General for OJP is responsible for holding EOWS accountable for 
following its policies and procedures. In reference to the ratings 
information and rejection letters missing from the official recognition 
files, EOWS produced the documents after our initial review, informing 
us that the documents were kept in the personal files of the grant 
monitors--rather than the official recognition files--a practice not in 
keeping with EOWS's and OJP's policies and procedures. In reference to 
the missing applications, EOWS officials also produced them after our 
initial review, but did not specify where they found them. Not having 
complete and readily available official recognition and official grant 
files may delay and make it more difficult for OJP and EOWS officials 
to perform management functions such as overseeing and assessing the 
documentation and oversight of qualification and funding decisions. 
While we eventually were able to obtain documentation showing the basis 
and rationale for EOWS decision making, the incomplete official 
recognition and official grant files made the review difficult and 
time-consuming.

In responding to a draft of this report, Justice stated that electronic 
documents maintained in its grant management system (GMS) are 
considered a part of the official recognition and official grant files 
and are readily available to EOWS, and that they should have been 
readily available to, and easily accessible by, GAO.

EOWS Has Taken Steps to Improve Program-Monitoring Documentation, but 
Challenges Remain:

In response to our 1999 recommendation, EOWS has taken steps to improve 
program monitoring, as with the documentation of qualification and 
funding decisions, but challenges remain in making the monitoring 
documentation readily available. While OJP requires progress reports 
and site visit reports to be included in the official grant files, our 
review in 2003 of 10 official grant files showed that some of EOWS's 
files were missing full documentation. However, before our review 
ended, EOWS provided further documentation that fully documented 
progress reports and site visits, which it acknowledges was not in the 
official grant files, and thus not readily available. This lack of 
readily available documentation is not in keeping with OJP's policies 
and procedures. Not having complete and readily available official 
grant files may delay and complicate EOWS officials' and external 
reviewers' assessment of whether EOWS's monitoring requirements are 
being followed.

Official Grant Files Were Incomplete:

Despite EOWS's efforts, some of the official grant files we reviewed 
were incomplete at the time of our file review. We randomly selected 10 
files from the 31 sites that were first funded in fiscal year 
2002[Footnote 11] to determine whether the sites had submitted the 
required progress reports from January 1, 2003, through June 30, 
2003.[Footnote 12] Of the 10 files we reviewed, 6 did not contain any 
of the required progress reports for the period. The remaining 4 files 
included the required progress reports. OJP's policies and procedures 
require EOWS to ensure that progress reports are included in the 
official grant files. However, when we asked EOWS about the missing 
progress reports, EOWS provided us with the requested documents. We 
asked, but EOWS did not specify where it found the missing 
documentation.

In addition, EOWS's grant monitors are required to conduct site visits 
a minimum of every 18 months and document their visits. This 
documentation is meant to convey to EOWS management officials how well 
sites are complying with grant requirements and is to be used by EOWS 
in making funding decisions. Grant monitors are to prepare a report of 
the visits and forward a copy to the Office of the Comptroller for 
inclusion in the official grant file. We reviewed the same 10 grant 
files discussed above to determine whether site visits were fully 
documented. We found that a site visit report was not yet due for 7 of 
the 10 files. In the remaining 3 files where a site visit was due, none 
of the 3 files contained documentation that the site visit had been 
conducted. We requested to speak to the responsible grant monitors to 
determine whether site visits had been conducted. EOWS officials told 
us that the monitors were no longer involved with the program but 
provided additional documentation for the 3 files we reviewed. The 
documentation showed that of the 3 sites, only 1 visit had been 
conducted; the other 2 were scheduled but had been delayed. While we 
only reviewed 3 files where a site visit was due, the lack of 
documentation in all 3 files, as well as the fact that only 1 of 3 site 
visits had been conducted, added to our concerns about the completeness 
of official grant files for program oversight.

Challenges Persist in Making Monitoring Documentation Readily 
Available:

As we have previously mentioned, OJP's policies and procedures manual 
and the Comptroller General's guidance on internal controls require 
that documentation should be kept readily available for examination. 
Subsequent to our finding missing progress reports and site visit 
reports, EOWS provided us with additional documentation that 
demonstrated that the progress reports and one of the site visit 
reports had been completed. When asked about where they found the 
missing progress reports and site visit reports, EOWS officials did not 
specify where they found the missing documentation.

Despite Some Steps, Challenges Remain in Developing Criteria for Self-
Sustainability:

EOWS has established a rule and set some activities for sites to 
complete to encourage them to become self-sustaining. However, it has 
still not fully developed criteria to determine when sites have become 
self-sustaining and when to reduce or withdraw Weed and Seed grant 
funds, as we recommended in 1999. To become self-sustaining, sites must 
leverage additional resources from sources other than EOWS to sustain 
the Weed and Seed program without EOWS funds. We found that EOWS has 
not reduced or withdrawn funds from any Weed and Seed sites for reasons 
related to becoming self-sustaining. This is important because without 
overall program funding increases, new sites cannot be funded unless 
funding can be reduced or discontinued from sites that have achieved 
self-sustainability.

EOWS Encourages Self-Sustainability, yet Needs Criteria to Make Funding 
Decisions:

In 1999, EOWS established a rule to encourage self-sustainability that 
was generally intended to limit Weed and Seed funding to a site to 5 
calendar years and require grantees to shift the majority of EOWS funds 
to a different site after 5 years.[Footnote 13] According to EOWS 
officials, EOWS also encouraged sites to become self-sustaining by 
establishing activities to be completed during each year of the 5-year 
grant.[Footnote 14] Program monitors are to assess sites' completion of 
these activities by recording results on a checklist.

We found that while EOWS undertook efforts to encourage self-
sustainability, it has not fully developed criteria to make funding 
decisions based on whether sites had achieved self-sustainability. In 
fiscal year 2002, 73 sites, which had completed 5 years of EOWS 
funding, reapplied for funding. EOWS approved an additional round of 5-
year funding to 67 of these sites. EOWS denied funding to 6 sites, but 
not for reasons related to self-sustainability. At the time of our 
review, no site's funding had been reduced or withdrawn because the 
site had achieved self-sustainability, even though EOWS has funded some 
sites since the early 1990s. In relation to the activities EOWS set to 
encourage sites to achieve self-sustainability, because these 
activities were established in 2003, we could not assess the sites' 
progress toward completing them. As we reported in 1999, without 
criteria, EOWS does not have a basis for determining when sites are 
self-sustaining and when to reduce or withdraw Weed and Seed funds.

EOWS officials told us that OJP is currently developing criteria for 
self-sustainability for EOWS and other OJP programs and that a report 
detailing the criteria may be completed in 2004. According to Justice, 
EOWS developed a new criterion for self-sustainability in the FY2003 
Competitive Application Kit regarding whether or not a full-time 
coordinator is funded by a reallocation of resources other than the 
Weed and Seed grant.

For information about proposed legislation that could affect self-
sustainability for Weed and Seed sites, similar to other Justice 
programs, see appendix III.

Performance Measures Generally Did Not Track Program Outcomes:

In 1999, we reported[Footnote 15] that while EOWS had developed various 
performance measures in an attempt to respond to the Government 
Performance and Results Act (GPRA) of 1993,[Footnote 16] EOWS's 
measures generally did not track program outcomes.[Footnote 17] GPRA 
seeks to shift the focus of federal management and decision making away 
from activities performed to outcomes, or the results of activities 
undertaken. Since our last report, EOWS has continued to refine and 
develop its performance measures, but we found that EOWS still 
generally collects site activity data rather than measuring 
outcomes.EOWS also generally does not use intermediate measures, which 
represent conditions believed to precede or contribute to achieving the 
ultimate outcomes that may be considered constructive steps toward 
measuring outcomes. While assessing success using outcomes and/or 
intermediate measures can be difficult, doing so is important to 
program management and to policy makers for understanding whether the 
program is achieving the intended results, and for identifying 
opportunities for improvement.

EOWS Generally Collects Activity Data Rather Than Outcome Performance 
Measures:

Our review showed that EOWS generally collects activity data, but has 
not developed outcome or intermediate measures that enable EOWS 
management to track the success of the Weed and Seed program in meeting 
its goals.[Footnote 18] As established in prior work by GAO, outcome 
measures help officials track the success of their programs in meeting 
program goals.[Footnote 19] EOWS mostly collects data about site 
activities, such as whether sites have foot patrols, safe havens, and 
provide job training. An intermediate measure, for example in relation 
to job training programs, might be the extent to which those who attend 
job training obtain employment. This intermediate measure rests on the 
assumption that individuals who are employed are less likely to commit 
crimes. See appendix V for activity data collected by EOWS.

EOWS officials told us that to measure its success in reducing violent 
crime, they collect data on the number of homicides and consider 
homicides the significant indicator for measuring performance outcomes. 
However, using the number of homicides as an outcome measure indicative 
of program success is problematic for several reasons. First, because 
homicides are relatively rare, even in high crime areas, homicide 
trends may be too unstable (fluctuate too much from year to year) to 
assess the success of the Weed and Seed program. Second, outcome 
measures, such as homicides, can present some methodological challenges 
because it is difficult to draw a direct causal link between the 
homicides and a program's work. Economic trends and other law 
enforcement initiatives could also be responsible for the observed 
outcomes. If homicides are used as a performance measure, any analysis 
should attempt to control for other factors influencing the outcomes. 
Third, given the broad nature of this program goal --to reduce violent 
crime, measuring outcomes in relation to only one type of violent crime 
also seems problematic because it is too narrowly focused. According to 
EOWS officials, homicide data was selected because of its reliability, 
and because gang-related homicides are a significant indicator of the 
success of a Weed and Seed strategy.

In February 2004,the Office of Management and Budget (OMB) issued a 
detailed assessment of the Weed and Seed program.[Footnote 20] 
According to OMB, Weed and Seed has not demonstrated results. OMB's 
report states that Weed and Seed lacks clear targets or timelines for 
reducing violent crime. Additionally, the report states that Weed and 
Seed has difficulty collecting data and developing aggregate measures 
for assessing program performance.

GAO Has Previously Reported on How Other Federal Programs Have 
Developed Effective Performance Measures, Despite Challenges:

Although the Weed and Seed program faces many challenges in developing 
outcome measures for its various activities, we have previously 
reported that other federal programs have developed effective 
performance measures even under difficult circumstances.[Footnote 21] 
These reports identified a variety of strategies that other federal 
programs have used to develop performance measures. For example, we 
reported that some federal programs utilized a mix of outcome and 
intermediate measures.[Footnote 22] This combination of measures 
allowed them to minimize the risk of not showing outcomes because of 
their limited control over external factors such as economic trends, 
which may prevent programs from achieving intended outcomes. 
Intermediate measures are also helpful to show progress when it is 
expected to take many years before the desired outcome is likely to be 
achieved. In such instances, progress toward program outcomes may be 
demonstrated through intermediate outcomes. We also reported that where 
measuring outcomes is, after careful consideration, deemed infeasible, 
intermediate measures can be used to track progress toward outcomes for 
programs such as Weed and Seed.[Footnote 23]

In addition, we have reported other ways federal agencies have used 
intermediate outcomes.[Footnote 24] For example, the National Highway 
Traffic Safety Administration uses the rate of safety belt use as an 
intermediate measure of its goal to reduce motor vehicle crashes and 
the resulting fatalities and injuries. As we reported, an increase in 
the use of safety belts is considered an intermediate outcome--not an 
end outcome--because it is desirable not in itself but because it has 
been demonstrated to contribute to the ultimate goal--reducing highway-
related fatalities and injuries.[Footnote 25]

Additional Performance Measures Are Being Developed for EOWS:

The Justice Research and Statistics Association and the Urban Institute 
are currently developing additional outcome performance measures under 
contracts awarded by EOWS in 2002 and 2003. According to EOWS 
officials, these research organizations have the expertise needed to 
develop outcome performance measures for the Weed and Seed program. We 
interviewed researchers from these organizations and spoke to EOWS 
officials about their research plans. However, none of the studies on 
developing additional performance measures had been completed at the 
time of our review, and it is too soon to tell whether the studies will 
produce outcome measures needed to adequately assess the Weed and Seed 
program.[Footnote 26] The Justice Research Statistics Association and 
other researchers were conducting studies on crime data and evaluation 
during our review; one of those studies was submitted to EOWS in 
November 2003. See appendix VI for a description of EOWS's ongoing and 
completed studies.

Conclusions:

We believe that 5 years has been ample time for OJP and EOWS to fully 
implement the recommendations we made in our 1999 report. EOWS has 
developed and partially implemented policies requiring the completion 
of documentation to support major decisions, such as qualification and 
funding decisions, and recorded monitoring information. But by failing 
to ensure that these documents are appropriately maintained in official 
recognition and grant files, the documents are not readily available to 
meet their purpose. That is, the intent of this type of internal 
control is to ensure that both management and external reviewers such 
as auditors can adequately perform their responsibilities, for example, 
reviewing work and making management decisions based on complete and 
accurate information. While EOWS was eventually able to produce the 
documentation we requested, failure to appropriately file the 
documentation made the process of using it very inefficient and, in 
effect, may have defeated the purpose of having it completed.

While EOWS may have moved forward in addressing its self-sustainability 
goals by recently developing an activity checklist for grantees to use 
to document actions that are intended to achieve self-sustainability, 
the activities do not constitute criteria for determining when sites 
should be considered self-sustaining and consequently have federal 
funds reduced or discontinued. One of the benefits of such criteria is 
to enable EOWS to determine when current projects are likely to be able 
to self-sustain so that available funding can be used to help reduce 
crime and achieve other benefits in other deserving communities. 
Because no sites have had their funding withdrawn because they were 
deemed self-sustaining during the 13-year life of this program, EOWS 
may be foregoing the opportunity to use the funds in another location 
where the need for federal funding is greater or EOWS may need less 
overall funding. Without criteria to determine when federal funds are 
no longer needed, EOWS also runs the risk of providing funds beyond 
what is needed to sustain some sites.

EOWS also needs performance measures that focus on program outcomes, 
and/or achievement of intermediate goals, so that it and those that 
provide oversight, such as the Congress and OMB, will be able to 
adequately assess the extent to which the program is achieving its 
goals. Management depends on this type of assessment to make the 
strategic and operations decisions needed to achieve the program's 
missions and goals. Congress and oversight agencies need this type of 
assessment so they can make funding decisions and help ensure that EOWS 
is in the best position and has the best tools to accomplish its 
mission. While EOWS recognizes the need for outcome performance 
measures and has funded studies to help develop them, it is unclear at 
this time whether these studies will be able to suggest the needed 
measures.

Recommendations for Executive Action:

We recommend that the Attorney General of the United States require the 
Assistant Attorney General for OJP to ensure that the Executive Office 
for Weed and Seed fully implement the intent of our previous 
recommendations by taking the following four steps:

* maintain the documentation of the basis and rationale for 
qualification and funding decisions in appropriate grant files;

* retain progress reports and site visit reports in official grant 
files;

* clearly define criteria to assess when sites are self-sustaining and 
apply the criteria to sites when making further funding decisions; and:

* develop outcome performance measures--or, where measuring outcome is, 
after careful consideration, deemed infeasible, intermediate measures-
-that can be used to adequately track progress toward program outcomes 
of the Weed and Seed program.

Agency Comments and Our Evaluation:

We provided a draft of this report to the Attorney General of the 
United States for review and comment. In a March 11, 2004, letter, the 
Assistant Attorney General (AAG) for OJP commented on the draft. Her 
written comments are summarized below and presented in their entirety 
in appendix VII. Based on Justice's comments, we modified the text of 
our report where appropriate and also incorporated Justice's technical 
comments into this report where appropriate.

In the AAG's response, the Justice Department generally agreed with our 
recommendations on maintaining the documentation of the basis and 
rationale for qualification and funding decisions in appropriate grant 
files and retaining progress reports and site visit reports in official 
grant files. Justice agreed in part with our recommendation on self-
sustainability criteria and fully agreed with our recommendation on 
outcome performance measures, and offered additional information to 
show it had moved forward in responding to our past recommendation.

Justice agreed that controls should be strengthened to ensure that the 
basis and rationale for qualification and funding decisions are 
documented in the centrally maintained official recognition (OR) files. 
In Justice's comments, the AAG explained that electronic documents 
maintained in its grant management system (GMS) are considered a part 
of the official recognition file and are readily available to EOWS, and 
that they should have been, but were not, readily available to, and 
easily accessible by, GAO. As of the current fiscal year, OJP will not 
only post all solicitations and receive all grant applications via GMS, 
but also receive and maintain all grant progress reports on GMS. In 
this way, there will be no opportunity to misplace official grant 
documents. Justice stated that OJP's Office of the Comptroller will 
implement procedures to ensure that external reviewers have ready 
access to GMS information.

Justice agreed with our second recommendation on retaining progress 
reports and site visit reports in official grant files. Justice noted 
that EOWS recently implemented a progress reporting module in GMS that 
now enables recipients of grants awarded through GMS to submit semi 
annual progress reports electronically. OJP is developing a monitoring 
module that will enable program monitors to record and report on site 
visits directly in GMS. In the interim, EOWS and the Office of the 
Comptroller will take steps to strengthen controls to ensure that 
copies of completed site visit reports are included in the official 
grant file.

Justice agreed in part with our third recommendation on self-
sustainability and noted that EOWS will further define the criteria to 
assess when sites are self-sustaining and apply the criteria when 
making funding decisions. Further, Justice said it is promoting a 
"graduation" process to bring sites to sustainability and cease 
providing Weed and Seed funding to those sites, thus making funding 
available to assist newer developing sites. However, Justice does not 
agree that EOWS has never set criteria for self-sustainability. Justice 
said that EOWS set a criterion for self-sustainability in the FY2003 
Weed and Seed Competitive Application Kit, which asked if the 
application provided for a full-time coordinator funded by reallocation 
of existing resources other than the Weed and Seed grant. According to 
Justice, applications were ranked against this criterion. According to 
EOWS officials, in March 2004, EOWS will announce an additional 
criterion specific to measuring self-sustainability. The new criterion 
will require grant applicants to identify other funding sources at a 
level five times the EOWS contribution. This criterion will be 
considered when making funding decisions.

We applaud EOWS's commitment to further develop criteria to assess when 
sites are self-sustaining and apply the criteria to sites when making 
further funding decisions. However, Justice's comments did not provide 
specific information about how the Application Kit criterion has been 
used to determine self-sustainability. We added Justice's belief that 
this criterion relates to self-sustainability to the text, but also 
note that over the 13-year history of the program, funding has never 
been reduced or withdrawn from a site because a site was deemed to be 
self-sustaining.

Justice agrees that developing outcome performance measures is 
important and states that EOWS has been working to develop measures 
that track progress toward program outcomes in the Weed and Seed 
Program. In a letter commenting on this draft report, Justice provided 
a summary of results from JRSA's study to support EOWS's use of the 
change in homicides as a measure of program success. According to 
Justice's summary, this study concluded that measuring reductions of 
homicides is an adequate performance measure of the Weed and Seed 
program because of its reliability and explicit selection as a goal by 
over 30 percent of the Weed and Seed sites participating in the Crime 
Pattern Study. In addition, Justice includes a 5 percent reduction in 
homicides in Weed and Seed sites as part of its Strategic Plan. With 
its comments, Justice included summaries of two recent research 
reports. One of these, Analysis of Homicide in Weed and Seed Sites, 
showed positive results in homicide trends in Weed and Seed sites when 
compared to host jurisdictions. Another report, Crime Pattern Analysis 
(Three-Top Crime Study), showed that after the third year of Weed and 
Seed program implementation, sites encountered a significant decrease 
in reported crimes.

Because the Crime Pattern Analysis study was not completed at the time 
of our review, we could not fully assess its methodological rigor and 
whether it would result in effective outcome performance measures. 
Although we believe that EOWS may be moving in the right direction 
regarding developing additional performance measures and we mention the 
studies in the report text, we also believe that homicides have 
shortcomings as described in the text when used as the only measure of 
the program's performance. In addition, our review of the Analysis of 
Homicide in Local Weed and Seed Sites study generated several concerns. 
First, the study used changes in the raw volume of homicides, instead 
of changes in homicide rates, as its indicator of success or failure of 
program sites. This method does not account for changes in the 
population as would be accounted for were a homicide rate measured. 
Second, the study used the larger host jurisdictions for specific Weed 
and Seed sites as the comparison locations to which the Weed and Seed 
sites are compared. In doing so, Weed and Seed sites' homicide data are 
included with host jurisdictions' homicide data, thus making the 
differences between them extremely difficult to interpret. It is also 
worth noting that this study evaluates Weed and Seed sites only on the 
basis of decreases in homicides, while Weed and Seed was intended to 
decrease crime more generally.

We are sending copies of this report to the Attorney General, 
appropriate congressional committees, and other interested parties. We 
will also make copies available to others upon request. In addition, 
the report will be available at no charge on GAO's Web site at http://
www.gao.gov. If you or your staff has any questions on this report, 
please call Weldon McPhail on (202) 512-8644 or me on (202) 512-8777.

Sincerely yours,

Signed by: 

Laurie E. Ekstrand, Director: 
Homeland Security and Justice Issues:

[End of section]

Appendix I: Scope and Methodology:

To determine what actions the Executive Office for Weed and Seed (EOWS) 
has taken to develop an adequate internal control to ensure full 
documentation of the basis and rationale for qualifying new and 
existing sites for funding, we reviewed (1) the criteria used to 
determine which new and existing sites should be qualified for funding, 
and (2) EOWS's policies and guidance. To gather this information, we 
interviewed officials from Justice and EOWS and reviewed pertinent 
documents, including EOWS policies and procedures, official recognition 
and grant applications, management oversight and monitoring 
documentation, and budget reports. In addition, we randomly selected 10 
of 94 fiscal year 2002 official recognition files submitted to EOWS for 
review. These 10 files included 5 files from new applicants and 5 files 
from existing sites that applied in 2002 and were funded in fiscal year 
2002. We reviewed 10 of the 14 official recognition files that EOWS 
decided not to fund in fiscal year 2002. We also reviewed 10 randomly 
selected official grant files from the 31 new and existing sites that 
were approved for funding in fiscal year 2002. Because of limited 
resources, we did not attempt to review all the files. While the small 
sample size prevents us from making reliable generalizations, the 30 
files we reviewed represented various types of files that EOWS 
maintains. We selected the files from a list that was provided to us by 
EOWS. To ensure the list of files was sufficient for the purpose of our 
review, we spoke to knowledgeable EOWS officials about the completeness 
and accuracy of the list.

To assess what steps EOWS has taken to improve program monitoring to 
ensure that sites meet the grant requirement of submitting progress 
reports, and that EOWS site visits are documented, we reviewed EOWS 
program grant guidance, including training offered, the EOWS monitoring 
guidance used by grant monitors when conducting site visits, and the 
grant files for the Weed and Seed sites that we visited: Pittsburgh, 
Pennsylvania; Charleston, South Carolina; and North Charleston, South 
Carolina. We selected these sites from the 221 sites funded by EOWS in 
fiscal year 2003. Based on our discussions with EOWS officials about 
the sites, we selected three sites, which although not representative 
of all sites, had received Weed and Seed grant funds since the early 
1990s, which enabled us to learn about their efforts to implement the 
Weed and Seed strategy over time. We also reviewed selected monitoring 
visit reports prepared by grant monitors for these sites and biannual 
progress reports submitted in fiscal years 2002 and 2003. We 
interviewed EOWS officials and grant coordinators at these 3 sites 
regarding procedures used for monitoring Weed and Seed sites. We also 
reviewed the same 10 randomly selected official grant files discussed 
above from the 31 new and existing sites that were funded in fiscal 
year 2002 to determine whether the progress reports and site visit 
reports were in the official grant files and readily available. The 
sites were funded under EOWS's competitive application and did not 
include sites funded under EOWS's continuation application.[Footnote 
27] While the 10 files we reviewed represented various types of files 
that EOWS maintains, the small sample size prevents us from making 
reliable generalizations about all official grant files. To ensure the 
list of files was sufficient for the purpose of our review, we spoke to 
a knowledgeable official about the completeness and accuracy of the 
list.

To assess what criteria, if any, EOWS has developed when sites are 
self-sustaining and when to reduce or withdraw program funding, we 
asked EOWS officials for any relevant information. Following repeated 
requests, EOWS officials did not provide any documentation on criteria 
that they had developed. They did, however, provide information on the 
five-year rule for self-sustainability and the sites' activities, which 
we reviewed.

To determine what additional performance measures EOWS has developed 
that track program outcomes, we interviewed officials from EOWS and the 
sites we visited. We reviewed pertinent documents, including EOWS 
policies and procedures, grant applications, and data collected 
pursuant to the Government Performance and Results Act of 1993 (GPRA). 
In relation to the studies EOWS commissioned to develop program-wide 
performance measures, we interviewed the authors of the studies under 
way to discuss their research plans. At the time of our review, none of 
the studies on developing additional performance measures had been 
completed.

We conducted our audit work between June and December 2003 in 
accordance with generally accepted government auditing standards.

[End of section]

Appendix II: GAO Site Visit Summary:

Table 2: Sites' Weed and Seed Funding History--Fiscal Years 1999-2003:

Fiscal year: 1999; 
Pittsburgh, Pa., grant funds: $0; 
Charleston, S.C., grant funds: $175,000; 
North Charleston, S.C., grant funds: $0.

Fiscal year: 2000; 
Pittsburgh, Pa., grant funds: $475,000; 
Charleston, S.C., grant funds: $125,000; 
North Charleston, S.C., grant funds: $175,000.

Fiscal year: 2001; 
Pittsburgh, Pa., grant funds: $675,000; 
Charleston, S.C., grant funds: $225,000; 
North Charleston, S.C., grant funds: $125,000.

Fiscal year: 2002; 
Pittsburgh, Pa., grant funds: $825,000[A]; 
Charleston, S.C., grant funds: $225,000; 
North Charleston, S.C., grant funds: $275,000.

Fiscal year: 2003; 
Pittsburgh, Pa., grant funds: $435,000[B]; 
Charleston, S.C., grant funds: $275,000; 
North Charleston, S.C., grant funds: $225,000.

Fiscal year: Total; 
Pittsburgh, Pa., grant funds: $2,410,000; 
Charleston, S.C., grant funds: $1,025,000; 
North Charleston, S.C., grant funds: $800,000. 

Source: GAO site visits and EOWS data.

[A] Pittsburgh's Weed and Seed funds for fiscal year 2002 were amended 
into the 2001 grant award.

[B] Pittsburgh's Weed and Seed funds for fiscal year 2003 were amended 
into the 2001 grant award.

[End of table]

Pittsburgh, Pa.

Background:

Pittsburgh established its first Weed and Seed site, the Hill District, 
in 1992. In 1995, Hazelwood became the second Pittsburgh site. In 1997 
and 1999, East Liberty and Homewood became the third and fourth Weed 
and Seed sites in Pittsburgh. Pittsburgh currently receives funds for 
three sites: East Liberty, Hazelwood, and Homewood. The sites are 
located between the Allegheny and Monongahela rivers on the eastern 
side of the city. The population of the target area is 10,897 in East 
Liberty, 15,849 in Hazelwood, and 9,283 in Homewood. In fiscal year 
2003, the Pittsburgh sites in total received $435,000 in Weed and Seed 
funding. See table 2 for the Weed and Seed funding history for 
Pittsburgh's three sites.

Activities:

Pittsburgh's weeding activities include funding a drug violence 
initiative headed by the Federal Bureau of Investigation (FBI) and a 
community-based crime prevention activity coordinated with the police. 
Pittsburgh's seeding activities include funding a drug education youth 
camp that provides children with leadership training and fitness tests, 
and a downtown revitalization program. See table 3 for examples of the 
types of activities funded by the Pittsburgh Weed and Seed program.

Leveraging Efforts:

An important goal of the Weed and Seed program is for sites to leverage 
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained 
from Pittsburgh Weed and Seed program staff, we identified several 
partnerships established by the Pittsburgh Weed and Seed program to 
leverage resources. These cooperative arrangements involved partners 
such as the University of Pittsburgh Medical Center and the Pittsburgh 
Harlequin Rugby Association. Table 4 provides specific examples of 
leveraging efforts that we identified.[Footnote 28] Pittsburgh Weed and 
Seed program staff estimate that they coordinated and leveraged 
approximately $1.1 million in 2002.

Charleston, S.C.

Background:

Charleston became a Weed and Seed site in the spring of 1992. The 
Charleston Weed and Seed target area includes approximately 8 square 
miles on the Charleston Peninsula and in the city of Charleston. The 
entire Weed and Seed site is located in Charleston's Renewal Community. 
The population of the target area, encompassing 13 neighborhoods, is 
roughly 17,000. In fiscal year 2003, the site received $275,000 in Weed 
and Seed funding. See table 2 for Charleston's Weed and Seed funding 
history.

Activities:

Charleston's weeding activities include supporting a youth court with a 
Weed and Seed juvenile investigator and funding community policing foot 
patrols. Charleston's seeding activities include supporting a police-
youth after-school boxing program, and boarding up abandoned and vacant 
buildings to improve the appearance of the area. See table 3 for 
examples of the types of activities funded by the Charleston Weed and 
Seed program.

Leveraging Efforts:

An important goal of the Weed and Seed program is for sites to leverage 
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained 
from Charleston, we identified several partnerships established by the 
Charleston Weed and Seed program to leverage additional resources. 
These cooperative arrangements involved partners such as the FBI and 
the College of Charleston's School of Education. Table 4 provides 
specific examples of leveraging efforts that we identified.[Footnote 
29] Charleston Weed and Seed program staff estimate that they were able 
to coordinate and leverage approximately $1 million in 2002.

North Charleston, S.C.

Background:

North Charleston was originally funded as a Weed and Seed site in 
August 1993. The city of North Charleston is located about 7 miles 
north of the city of Charleston. Today, North Charleston has four 
target neighborhoods: Union Heights, Chicora/Cherokee, Accabee, and 
Liberty Hill. The population of the target area is 13,606. In fiscal 
year 2003, the site received $225,000 in Weed and Seed funding. See 
table 2 for North Charleston's Weed and Seed funding history.

Activities:

North Charleston's weeding activities include participating in 
Operation Cease Fire with the U.S. Attorney's Office, to reduce gun 
crime in target neighborhoods, and partnering with local law 
enforcement to improve the bike patrol program. North Charleston's 
seeding activities include developing the Drug Education for Youth 
program and supporting an annual neighborhood cleanup. See table 3 for 
examples of the types of activities funded by the North Charleston Weed 
and Seed program.

Leveraging Efforts:

An important goal of the Weed and Seed program is for sites to leverage 
resources from sources other than EOWS in order to become self-
sustaining. During our site visit and through documentation obtained 
from North Charleston, we identified several partnerships established 
by the North Charleston Weed and Seed program to leverage resources. 
These cooperative arrangements involved partners such as the United 
States Air Force and the Medical University of South Carolina. Table 4 
provides specific examples of leveraging efforts that we 
identified.[Footnote 30] North Charleston Weed and Seed program staff 
estimate that they coordinated and leveraged approximately $3.3 million 
in 2002.

Table 3: Examples of Activities Funded by Weed and Seed in Pittsburgh, 
Charleston, and North Charleston:

Site: Pittsburgh, Pa; 
Program component: Law enforcement; 
Activity: Safe Streets program; 
Weed and Seed partner: U.S. Attorney's Office, Federal Bureau of 
Investigation (FBI); 
Description: Drug violence initiative headed by the FBI.

Program component: Site: Community policing; 
Activity: Site: Bike safety classes and registration; 
Weed and Seed partner: Site: Community residents and police officers; 
Description: Site: Community-based crime prevention activity 
coordinated with the police.

Program component: Site: Prevention, intervention, and treatment; 
Activity: Site: Drug Education For Youth (DEFY) Camp; 
Weed and Seed partner: Site: Boy Scouts of America, U.S. Air Force; 
Description: Site: About 60 children participated in leadership 
training and fitness tests.

Program component: Neighborhood restoration; 
Activity: Improved housing; 
Weed and Seed partner: Hazelwood Initiative, Inc., and the Second 
Avenue Main Street Program; 
Description: A community-wide effort to reestablish the local business 
district, revitalize a 200-acre area, reclaim historic buildings, and 
build new housing.

Site: Charleston, S.C; 
Program component: Law enforcement; 
Activity: Youth Court; 
Weed and Seed partner: South Carolina Bar Association, Young Lawyers 
Division, Charleston Enterprise Community; 
Description: First-time juvenile offenders are tried by their peers and 
given an opportunity to improve their behavior.

Program component: Site: Community policing; 
Activity: Site: Foot patrols; 
Weed and Seed partner: Site: Charleston Weed and Seed Patrol Officers, 
Charleston Police Department; 
Description: Site: Weed and Seed officers patrol the community on foot.

Program component: Site: Prevention, intervention, and treatment; 
Activity: Site: Police Athletic League Boxing; 
Weed and Seed partner: Site: Police Athletic League; 
Description: Site: Participants include junior and senior boxers and 
coaches.

Program component: Neighborhood restoration; 
Activity: Boarding up abandoned and vacant buildings; 
Weed and Seed partner: Charleston Police Department, Maintenance 
Division; 
Description: The city boarded up abandoned and vacant buildings in Weed 
and Seed neighborhoods.

Site: North Charleston, S.C; 
Program component: Law enforcement; 
Activity: Cease Fire program; 
Weed and Seed partner: U.S. Attorney's Office, North Charleston Police 
Department; 
Description: A gun control program that began in January 2003 to reduce 
gun crime in target areas.

Program component: Site: Community policing; 
Activity: Site: Bike patrol; 
Weed and Seed partner: Site: North Charleston Police Department; 
Description: Site: Two officers on bike patrol are dedicated 
exclusively to the Weed and Seed target area.

Program component: Site: Prevention, intervention, and treatment; 
Activity: Site: DEFY program; 
Weed and Seed partner: Site: Charleston Air Force Base; 
Description: Site: Weed and Seed staff was developing a DEFY program 
for participants from the Weed and Seed target area.

Program component: Neighborhood restoration; 
Activity: Neighborhood cleanup; 
Weed and Seed partner: Neighborhood residents; 
Description: An annual neighborhood cleanup was held in April 2003.

[End of table]

Source: GAO site visits and EOWS.

Table 4: Examples of Leveraging Efforts at Weed and Seed Sites in 
Pittsburgh, Charleston, and North Charleston:

Weed and Seed site: Pittsburgh, Pa; 
Type of partnership or cooperative arrangement: Federal government; 
Name of partner or cooperative arrangement: FBI; 
Description: The FBI leads the Safe Streets program in Pittsburgh to 
reduce street gang and drug related violence.

Type of partnership or cooperative arrangement: State government; 
Name of partner or cooperative arrangement: State police; 
Description: State Police officers are assisting the U.S. Attorney's 
Office in implementing Justice's Project Safe Neighborhoods (PSN) 
initiative to reduce gun violence.

Type of partnership or cooperative arrangement: Local government, 
university; 
Name of partner or cooperative arrangement: Allegheny County Health 
Department (ACHD) and University of Pittsburgh Medical Center (UPMC); 
Description: In March and April 2000, Pittsburgh began a partnership 
with ACHD, UPMC, and others to conduct medical and dental screening at 
community events. The first major effort of the partnership was a lead 
abatement project, which identified and cleaned contaminated dwellings. 
The Pittsburgh Urban Redevelopment Authority and the Housing Authority 
also participated in this effort.

Type of partnership or cooperative arrangement:  Local community; 
Name of partner or cooperative arrangement:  Pittsburgh Harlequins 
Rugby Association; 
Description:  The partnership with the Harlequins began in March 2001. 
The Harlequins established two teams of 20 children to play touch rugby 
while also incorporating training in teamwork, partnership, and 
discipline into the sport.

Weed and Seed site: Charleston, S.C; 
Type of partnership or cooperative arrangement: Federal government; 
Name of partner or cooperative arrangement: FBI; 
Description: The FBI leads the Charleston Safe Streets Task Force to 
reduce drug-related activities.

Type of partnership or cooperative arrangement: Local government; 
Name of partner or cooperative arrangement: Charleston Police 
Department; 
Description: Police officers train youth in law enforcement procedures 
and tactics as well as physical training. The goal of this program is 
to instill responsibility in participants and to prepare them for 
possible careers in law enforcement.

Type of partnership or cooperative arrangement: Local college; 
Name of partner or cooperative arrangement: College of Charleston's 
School of Education; 
Description: Graduate interns tutor students from Safe Havens.

Weed and Seed site: North Charleston, S.C; 
Type of partnership or cooperative arrangement: Federal government; 
Name of partner or cooperative arrangement: U.S. Air Force; 
Description: The DEFY program will be conducted year-round, through a 
partnership with the Air Force.

Type of partnership or cooperative arrangement: Local government; 
Name of partner or cooperative arrangement: North Charleston Police 
Department; 
Description: Police officers instruct youth on crime prevention during 
school breaks at Safe Havens.

Type of partnership or cooperative arrangement: Local university; 
Name of partner or cooperative arrangement: Medical University of South 
Carolina Crime Victims Center; 
Description: Through this partnership, information was distributed to 
Hispanic families in the Weed and Seed target area about a program 
designed to provide medical services to Hispanic families. 

Source: GAO site visits and EOWS.

Note: Grantees also receive funds from other federal programs and non-
federal funds.

[End of table]

[End of section]

Appendix III: Proposed Legislation Could Affect Self-Sustainability:

Proposed legislation could affect self-sustainability for Weed and Seed 
sites by mandating that EOWS impose more rigorous requirements on its 
grantees, similar to some other Justice programs. Unlike some other 
Justice discretionary grant programs, the law establishing the Weed and 
Seed program currently does not have a matching funding 
requirement,[Footnote 31] nor does it limit the maximum number of years 
a grantee can receive funding.[Footnote 32] H.R. 3036, a bill to 
reauthorize the Department of Justice for fiscal years 2004 through 
2006, was introduced on September 9, 2003. This bill includes a 
provision that would limit the federal contribution to 75 percent of 
the total Weed and Seed program costs. The bill would also impose 
limitations on the duration of grants funded under the Weed and Seed 
program, limiting to 10 the total number of fiscal years a grantee may 
receive grants. Additionally, the bill requires that in order for a 
grantee to be eligible for a grant, it must agree to formulate a timely 
and effective plan to independently sustain the Weed and Seed program 
when federal funding ends.

Currently there are other federal grant programs that require a funding 
match and a limit on the duration of grant awards.[Footnote 33] For 
example, the Drug-Free Communities Support Program (DFCSP) and 
Community Oriented Policing Services (COPS) both have matching 
requirements and clearly specify the number of years a grantee may be 
funded. DFCSP is a discretionary grant program with a budget of about 
$60 million in fiscal year 2003. DFCSP requires a dollar-for-dollar 
match, limits grantees to no more than 5 years of funding, and reduces 
awards by 25 percent in the fourth and fifth years. The COPS program, 
with a budget of about $929 million in fiscal year 2003, generally 
requires grantees to contribute 25 percent of the costs of the program. 
The federal share of a COPS grant that covers more than 1 year must 
decline year to year.[Footnote 34]

[End of section]

Appendix IV: Executive Office for Weed and Seed Selected Activities:

Year 1: The local U.S. Attorney's Office establishes a steering 
committee, which establishes clear roles and responsibilities for the 
site's organizational and management structure[A]; 
Year 2: The steering committee continually educates, guides, and 
reevaluates site management; 
Year 3: [Empty]; 
Year 4: [Empty]; 
Year 5: [Empty].

Year 1: The steering committee meets monthly and develops a job 
description for the site coordinator; 
Year 2: Steering committee meets at least monthly; 
Year 3: Steering committee and organizational structure continue to 
function. Steering committee meets at least quarterly; 
Year 4: [Empty]; 
Year 5: [Empty].

Year 1: The grantee and the site coordinator establish procedures for 
tracking grant funds and expenditures; 
Year 2: The grantee and/or local U.S. Attorney's Office evaluates the 
site coordinator's job performance, with input from the steering 
committee[B]; The site coordinator and the steering committee work 
together to ensure timely expenditure of grant funds; 
Year 3: [Empty]; 
Year 4: [Empty]; 
Year 5: [Empty].

Year 1: Based on the time line, goals, and objectives established by 
the site's official recognition,[C] the site begins implementing 
strategies and developing baseline data, addressing the four required 
program elements[D]; 
Year 2: The site's strategy to achieve measurable outcomes in the four 
required program areas (especially law enforcement/community policing) 
is on track. The site continues to collect data related to goals and 
objectives; 
Year 3: The site continues to implement strategy for achieving goals 
and objectives in all four required program elements; 
Year 4: [Empty]; 
Year 5: The site makes plans to continue community development efforts 
and strategic planning through various funding sources.

Year 1: The site begins to publicize the program, to recruit 
neighborhood leaders, and to mobilize the community; 
Year 2: The site continues program publicity and outreach to recruit 
neighborhood leaders and mobilize community; 
Year 3: The site continues program publicity and outreach efforts to 
recruit neighborhood leaders and to mobilize the community while 
identifying and building sustainable leadership among community 
members; 
Year 4: The site continues program publicity and outreach efforts to 
mobilize community and recruit new community leaders; 
Year 5: The site's community leaders are working to sustain efforts and 
continue community mobilization.

Year 1: The site begins coordinating its efforts with related local 
initiatives, particularly law enforcement resources and activities; 
Year 2: The site continues to coordinate its efforts with local 
initiatives, particularly programs with local parks and recreation 
departments; 
Year 3: The Weed and Seed effort begins to be included in city/
community plans; 
Year 4: Community leaders have established working relationships with 
local officials; 
Year 5: The Weed and Seed community is included in city/community 
planning.

Year 1: [Empty]; 
Year 2: The steering committee develops and implements the special 
emphasis program based on community needs; 
Year 3: The special emphasis program is fully operational; 
Year 4: [Empty];  
Year 5: [Empty].

Year 1: The site submits timely grant expenditure reports to Executive 
Office for Weed and Seed and Office of Justice Programs; 
Year 2: [Empty]; 
Year 3: [Empty]; 
Year 4: [Empty]; 
Year 5: [Empty].

Year 1: The site submits timely progress reports, providing program 
outcome information related to the achievement of the goals and 
objectives stated in the official recognition and grant applications; 
Year 2: [Empty]; 
Year 3: [Empty]; 
Year 4: [Empty]; 
Year 5: The site submits a final progress report.

Year 1: [Empty]; 
Year 2: The site communicates regularly with all partners, including 
U.S. Attorney's Office, Executive Office for Weed and Seed, and 
community members, using newsletters, faxes, neighborhood visits, 
e-mail, community meetings, and so forth; 
Year 3: [Empty]; 
Year 4: [Empty];  
Year 5: [Empty].

Year 1: The steering committee and the site coordinator evaluate site 
progress to determine site's technical assistance needs, if any; 
Year 2: [Empty]; 
Year 3: [Empty]; 
Year 4: The steering committee and site coordinator review previous 
years' grant applications to determine plans for final year of official 
recognition; 
Year 5: The steering committee determines if there is a need to 
continue the program in other neighborhoods based on the program.

Year 1: The site develops operating plans for next year; 
Year 2: [Empty]; 
Year 3: [Empty]; 
Year 4: [Empty]; 
Year 5: [Empty].

Year 1: The site begins developing strategies for long-term self-
sustainment; 
Year 2: [Empty]; 
Year 3: The site establishes a subcommittee to look for additional 
funding or in-kind resources to sustain the program; 
Year 4: Subcommittee continues to search for additional resources to 
sustain the program; 
Year 5: [Empty].

Year 1: [Empty];
Year 2: The site begins to develop plans for evaluating its programs; 
Year 3: The site identifies an evaluation tool and begins 
developing an evaluation based on Executive Office for Weed and Seed 
site evaluation literature; 
Year 4: The site undergoes program evaluation to assess overall 
effectiveness of official recognition strategy implementation; 
Year 5: By the end of the year, the site releases its full 
evaluation report, assessing the overall program accomplishments and 
its effectiveness. 

Source: EOWS data.

Note: Arrow indicates action from the previous year continues to the 
current year.

[A] A site is a geographically defined area ranging in size from 
several neighborhood blocks to several square miles. With the input of 
the U.S. Attorney's Office and the steering committee, each site 
develops and implements its own Weed and Seed strategy.

[B] A grantee is an entity that receives funding from EOWS to implement 
the Weed and Seed program. The grantee distributes the funds to sites 
and is responsible for ensuring that sites comply with the terms of the 
grant. A grantee may have more than one site.

[C] Official recognition is a designation given to a grantee by EOWS 
that signifies that EOWS has approved the grantee's Weed and Seed 
strategy in a specific site. A grantee must receive official 
recognition from EOWS before a grantee can apply for funding. Official 
recognition requires, among other things, that the site develop a 
management plan and a comprehensive strategy for implementing the Weed 
and Seed strategy.

[D] The four required program elements that sites are required to 
address are (1) law enforcement; (2) community policing; (3) crime and 
substance prevention, intervention, and treatment; and (4) neighborhood 
restoration.

[End of table]

[End of section]

Appendix V: Activity Data Collected by EOWS in Fiscal Year 2003:

Activity data collected by EOWS: 1. Does site include a 
multijurisdictional task force.

Activity data collected by EOWS: 2. Does site participate in Project 
Safe Neighborhoods.

Activity data collected by EOWS: 3. Does site have foot patrols.

Activity data collected by EOWS: 4. Does site have bike patrols.

Activity data collected by EOWS: 5. Does site have substations.

Activity data collected by EOWS: 6. Does site have crime watch.

Activity data collected by EOWS: 7. Do police participate in community 
meetings.

Activity data collected by EOWS: 8. Number of homicides for 3 preceding 
calendar years for Weed and Seed site.

Activity data collected by EOWS: 9. Number of homicides for 3 preceding 
calendar years for entire jurisdiction.

Activity data collected by EOWS: 10. Number of total drug arrests[A].

Activity data collected by EOWS: 11. Number of heroin drug arrests.

Activity data collected by EOWS: 12. Number of cocaine (not including 
crack) drug arrests.

Activity data collected by EOWS: 13. Number of crack cocaine drug 
arrests.

Activity data collected by EOWS: 14. Number of marijuana drug arrests.

Activity data collected by EOWS: 15. Number of methamphetamines only 
drug arrests.

Activity data collected by EOWS: 16. Number of safe haven facilities.

Activity data collected by EOWS: 17. Number of safe havens receiving 
EOWS funding.

Activity data collected by EOWS: 18. Does site provide academic courses 
and tutoring.

Activity data collected by EOWS: 19. Does site provide mentoring.

Activity data collected by EOWS: 20. Does site provide prevention 
education.

Activity data collected by EOWS: 21. Does site provide dispute 
resolution and mediation.

Activity data collected by EOWS: 22. Does site provide recreation and 
athletics.

Activity data collected by EOWS: 23. Does site provide job training.

Activity data collected by EOWS: 24. Does site provide job placement.

Activity data collected by EOWS: 25. Does site provide antidrug 
education.

Activity data collected by EOWS: 26. Does site provide community police 
co-located in safe haven.

Activity data collected by EOWS: 27. Does site provide safe corridors 
(school escorts for children).

Activity data collected by EOWS: 28. Does site provide summer day camp.

Activity data collected by EOWS: 29. Does site provide youth leadership 
training.

Activity data collected by EOWS: 30. Does site provide boys and girls 
club programs.

Activity data collected by EOWS: 31. Does site provide scouting 
programs.

Activity data collected by EOWS: 32. Does site provide military cadet 
training.

Activity data collected by EOWS: 33. Does site provide antigang 
education and training.

Activity data collected by EOWS: 34. Does site provide Communities in 
Schools programs.

Activity data collected by EOWS: 35. Does site provide performance or 
applied arts programs.

Activity data collected by EOWS: 36. Does site provide victim 
assistance programs.

Activity data collected by EOWS: 37. Does site provide community 
projects, such as cleanups.

Activity data collected by EOWS: 38. Does site provide general health-
screening services.

Activity data collected by EOWS: 39. Does site provide lead-poisoning-
screening service.

Activity data collected by EOWS: 40. Number of persons receiving safe 
haven services.

Activity data collected by EOWS: 41. Number of community development 
corporations within site area.

Activity data collected by EOWS: 42. Is site constructing and 
renovating housing developments.

Activity data collected by EOWS: 43. Is site constructing and 
renovating commercial developments.

Activity data collected by EOWS: 44. Is site constructing and 
renovating business and community partnerships.

Activity data collected by EOWS: 45. Number of community cleanups done 
using EOWS support.

Activity data collected by EOWS: 46. Do Weed and Seed activities relate 
to or involve Brownfields Assessment Demonstration Pilots.

Activity data collected by EOWS: 47. Do Weed and Seed activities relate 
to or involve Brownfields Cleanup and Revolving Load Fund Pilots.

Activity data collected by EOWS: 48. Do Weed and Seed activities relate 
to or involve Brownfields Showcase Communities.

Activity data collected by EOWS: 49. Do Weed and Seed activities relate 
to or involve Brownfields Tax Incentive.

Activity data collected by EOWS: 50. Source of non-EOWS funding[B].

Activity data collected by EOWS: 51. Amount of non-EOWS grant funding.

Activity data collected by EOWS: 52. Source and type of in-kind 
contributions.

Source: EOWS data.

[A] In addition to the number of drug arrests, EOWS also requests that 
each site report the number of sale, manufacturing, and possession drug 
arrests for each type of drug.

[B] Non-EOWS grant funding includes funding from other federal, state, 
local, and private sources.

[End of table]

[End of section]

Appendix VI: Description of EOWS's Ongoing and Completed Studies:

Title of study and research partner: A Comparison of Homicide Trends in 
Local Weed and Seed Sites Relative to Their Host Jurisdictions, 1996 to 
2001; 
Justice Research and Statistics Association (JRSA); 
Purpose of study: Compare homicides in Weed and Seed sites with 
homicides in the jurisdictions that sites are located in, to determine 
whether homicides will decline faster in Weed and Seed sites than in 
the remainder of the jurisdiction; 
When results are expected: Report was expected in January 2004 but was 
issued in November 2003.

Title of study and research partner: Performance Indicator Study; 
Urban Institute; 
Purpose of study: Develop additional performance indicators that will
be used by EOWS and Weed and Seed sites to evaluate program performance 
with respect to crime control; 
When results are expected: March 2004.

Title of study and research partner: Weed and Seed Crime Pattern Data 
Collection; 
Justice Research and Statistics Association; 
Purpose of study: Summarize the types of crime selected as "target" 
crimes by Weed and Seed sites (sites choose three crimes) to determine 
crime reduction strategies. Assess the degree of success of Weed and 
Seed sites by analyzing the percentage of reduction in crime compared 
with the percentage in the rest of the jurisdiction that they are 
located in; 
When results are expected: March 2004.

Title of study and research partner: Meta-Analysis Evaluation Justice 
Research and Statistics Association; 
Purpose of study: Summarize the completed evaluations of individual 
Weed and Seed sites. The analysis will include more than 80 studies. 
JRSA's preliminary analysis (March 3, 2003) includes 36 evaluations. Of 
the 36 evaluations, JRSA determined that 5 were process, 19 were 
impact, and 9 were both process and impact evaluations. The remaining 3 
evaluations were not identified as process or impact; 
Note: A process evaluation addresses whether the program is working as 
intended. An impact evaluation isolates the effects of a particular 
program or factor from all other potential contributing factors that 
could also effect change; 
When results are expected: Mid 2004.

Title of study and research partner: Local Pilot Evaluations Local 
researchers; 
Purpose of study: Conduct local evaluations of Weed and Seed sites. In 
fiscal year 2002, EOWS funded 13 local evaluations. Each site contracts
with its own researcher; 
When results are expected: Mid 2004. 

Source: EOWS and research organizations.

[End of table]

[End of section]

Appendix VII: Comments from the Department of Justice:

U.S. Department of Justice 
Office of Justice Programs 
Office of the Assistant Attorney General:

Washington, D.C. 20531 
MAR 11 2004:

Ms. Laurie E. Ekstrand:

Director, Homeland Security and Justice Issues, 
General Accounting Office:

441 G Street, N.W. Mail Stop 2440A 
Washington, DC 20548:

Dear Ms. Ekstrand:

This letter responds to the General Accounting Office (GAO) draft 
report entitled, "GRANTS MANAGEMENT: Despite Efforts to Improve Weed 
and Seed Program Management, Challenges Remain " (GAO-04-025). We 
agree, in part, with the recommendations, and we are in the process of 
implementing appropriate corrective actions to address the 
recommendations. However, we do not believe that GAO fully acknowledges 
in the draft report the steps the Executive Office for Weed and Seed 
(EOWS) has taken to address the recommendations concerning self-
sustainability criteria and outcome measures included in the prior GAO 
report on EOWS. The draft report recommendations are restated below in 
bold, followed by our response.

1. Maintain the documentation of the basis and rationale for 
qualification and funding decisions in appropriate grant files.

We agree that controls should be strengthened to ensure that the basis 
and rationale for qualification and funding decisions are documented in 
the centrally maintained Official Recognition (OR) files when 
applications are not funded. The Acting Director for the EOWS has 
directed the Program Managers to forward a copy of all documents 
supporting funding decisions to the centrally maintained OR files.

GAO also criticizes EOWS for having incomplete grant files because a 
paper version of the application was not in the grant file. The 
applications were submitted electronically through the Office of 
Justice Programs (OJP's) Grants Management System (GMS). GMS has been a 
high priority of mine since being named Assistant Attorney General for 
OJP in September, 2001. As of the current fiscal year, ON will not only 
post all solicitations and receive all grant applications via GMS, but 
also receive and maintain all grant progress reports on GMS. In this 
way, there will be no opportunity to misplace any 
official document relating to any grant. Documents maintained in GMS 
are considered part of the official grant file in an electronic format. 
In compliance with the government's e-gov efforts, we no longer 
maintain paper copies of documents retained in GMS. Those documents, 
however, are readily available in electronic format and should be 
available to outside reviewers as well.

For the ease of the review process, we agree that applications should 
have been readily available to, and easily accessible by, GAO. In the 
future, the OJP's Office of the Comptroller (OC) will implement 
procedures to ensure that external reviewers are provided: 1) read-only 
access to GMS information, in coordination with OJP's Office of the 
Chief Information Officer; or 2) hard copies of pertinent GMS 
information.

2. Retain progress reports and site visit reports in official grant 
files.

We agree with the recommendation. EOWS and OC will strengthen controls 
to ensure that copies of completed site visit reports are included in 
the official grant file maintained in OC. Additionally, as part of GMS, 
OJP is developing a monitoring module that will enable Program Managers 
to record and report on site visits directly in GMS. When that module 
is available, it will be the policy of ON to require that grant 
managers submit the reports in this manner, thus making the entire file 
fully accessible and fully contained in GMS.

Beginning with the semiannual progress reporting period that ended on 
December 31, 2003, ON implemented a progress reporting module in GMS 
that enables grantees to submit semiannual progress reports 
electronically. As stated in response to Recommendation 1, documents 
maintained in GMS are considered a part of the official grant file in 
an electronic format, and all grantees are now required to submit their 
semiannual progress reports electronically. Finally, OJP in 2002 
adopted a policy of withholding further fund drawn-down capability from 
any grantee who fails to file timely progress reports.

3. Clearly define criteria to assess when sites are self-sustaining and 
apply the criteria to sites when making further funding decisions.

We agree, in part, with the recommendation. EOWS will further define 
the criteria to assess when sites are self-sustaining and apply the 
criteria when making funding decisions. Further, we are promoting a 
"graduation" process to bring sites to sustainability and cease 
providing Weed and Seed funding to those sites, thus making funding 
available to assist newer developing sites.

However, we do not agree with GAO's assertion on page 12 of the draft 
report that EOWS has never set criteria for self-sustainability. EOWS 
set a criterion for self-sustainability in the FY2003 Weed and Seed 
Competitive Application Kit: "Does the application provide for a full-
time coordinator funded by reallocation of existing resources other 
than the Weed and Seed grant?" The applications were ranked against 
this criterion. In March 2004, EOWS will announce an additional 
criterion specific to measuring self-sustainability. This new criterion 
will require grant applicants to identify other funding sources at a 
level five times the EOWS contribution. This criterion will be 
considered when making funding decisions.

4. Develop outcome performance measures-or, where measuring outcome is, 
after careful consideration, deemed infeasible, intermediate measures-
that can be used to adequately track progress toward program outcomes 
of the Weed and Seed program.

We fully agree that developing outcome performance measures is 
important. This, too, is a high personal priority of mine, and OJP's 
Office of Budget and Management Services is working with every OJP 
grant program to develop outcome measures (and, in appropriate cases, 
interim measures). EOWS has been working to develop measures that track 
progress toward program outcomes in the Weed and Seed Program.

As early as 1997, EOWS enlisted the assistance of experts in the field, 
such as the Justice Research and Statistics Association (JRSA), who 
contracted with John O'Connell, Director, Statistical Analysis Center 
in Delaware, to advise on the appropriate outcome measures.

O'Connell concluded, in a recent study, that measuring reductions of 
homicides is an adequate performance measure of the Weed and Seed 
program because of its reliability and explicit selection as a goal by 
over 30 percent of the Weed and Seed Sites participating in the Crime 
Pattern Study. In addition, the Department of Justice includes a five 
percent reduction in homicides in Weed and Seed sites as part of its 
Strategic Plan.

On March 2, 2004, JRSA submitted the following summaries of two studies 
that were conducted recently: (1) Analysis of Homicide in Weed and Seed 
Sites; and (2) Crime Patterns Analysis (Three-Top Crime Study).

Evaluation; Analysis of Homicide in Weed and Seed Sites;
Summary; One of the overriding goals of the national Weed and Seed 
strategy is to reduce violence in some of the nation's most 
crime-ridden neighborhoods. As part of GPRA homicide data have been 
collected for Weed and Seed sites since 1996 as one measure of violence. 
For this multi-year analysis, the most recent GPRA reporting is for 
2001. Over this six year period 220 sites provided sufficient homicide 
information for both the Weed and Seed sites and their host 
jurisdictions (newer sites do not yet have enough history for 
analysis). Seventy-seven percent of the Weed and Seed sites had 
positive results in their homicide trends when compared to host 
jurisdictions. In most of the sites (122) homicides decreased by about 
50 percent (from an average of about 10 homicides per site in 1996 to 5 
per site in 2001). In 31 sites (14%), the number of homicides remained 
stable while the homicide statistics for their respective jurisdictions 
increased and in 17 sites (8%), homicides increased at a slower rate 
than their jurisdictions. These trends in the most serious of crimes 
were a significant accomplishment for Weed and Seed sites considering 
that homicides in the host jurisdictions (and the nation) increased in 
2000 and 2001.

Evaluation; Crime Pattern Analysis; (Three-Top Crime Study);
Summary; As a voluntary effort to provide more information regarding 
the extent of crime reduction in Weed and Seed sites, EOWS solicited 
comparative crime information in mid-2003 from all active Weed and Seed 
sites. The Weed and Seed sites were asked to submit information 
regarding the top three crime issues. To be included in the analysis 
crime information both prior to and following the implementation of 
their Weed and Seed operations had to be provided. By the fall of 2003, 
98 sites had submitted crime pattern information. Forty percent of the 
sites were able to provide crime data for both pre and post time 
periods. Forty-one percent of the sites had not implemented their Weed 
and Seed strategy long enough to conduct a pre/post crime analysis. The 
remainder of the sites had data inconsistencies that allow for only a 
partial analysis of the information provided. Preliminary analysis 
show, that as expected, reported crimes increase during the initial 
phases of a Weed and Seed operation but that after the third year of 
program implementation sites encounter a significant decrease in 
reported crimes within the Weed and Seed site areas. This decrease is 
apparent both in the total number of each crime reported for the Weed 
and Seed sites as well as in the proportion of city-wide or 
jurisdiction-wide crime accounted for by the Weed and Seed target area. 
Ninety-four percent of the sites are addressing violent crimes or 
homicide as one of their key crime initiatives.

[End of table]

OJP wishes to stress its full commitment to outcome measurement, even 
in cases, such as a Weed and Seed strategy, which, as the GAO report 
suggests, may not easily lend itself to measurement due to multiple 
factors potentially affecting those outcomes. We will continue to work 
towards this goal.

We would like this information and the above summaries included in 
GAO's report. As always, OJP appreciates the opportunity to comment on 
the draft report. Thank you for your continued cooperation and 
assistance.

Sincerely,

Signed by: 

Deborah J. Daniels, 
Assistant Attorney General:

cc: Robert Samuels, Acting Director Executive Office for Weed and Seed:

Cynthia J. Schwimer Comptroller, OJP:

LeToya A. Johnson Audit Liaison, OJP:

Vickie L. Sloan Audit Liasion, DOJ:

OAAG Executive Secretariat 
Control Number 20040254:

[End of section]

Appendix VIII: GAO Contacts and Staff Acknowledgments:

GAO Contacts:

Laurie Ekstrand (202) 512-2758 Weldon McPhail (202) 512-8644:

Staff Acknowledgments:

In addition to those named above, the following individuals made key 
contributions to this report: Lisa G. Shibata, Kirstin B. L. Nelson, 
Cady L. Summers, David Alexander, Sidney Schwartz, Kevin L. Jackson, 
Elizabeth Curda, Tom James, Denise Fantone, Jan Montgomery, Ann H. 
Finley, Katherine M. Davis, and Leo Barbour.

[End of section]

Related GAO Products:

Law Enforcement: Better Performance Measures Needed to Assess Results 
of Justice's Office of Science and Technology, GAO-04-198. Washington, 
D.C.: November 14, 2003.

Standards for Internal Control in the Federal Government. GAO/AIMD-00-
21.3.1. Washington, D.C.: November 1999.

Federal Grants: More Can Be Done to Improve Weed and Seed Program 
Management. GAO/GGD-99-110. Washington, D.C.: July 16, 1999.

Small Business Administration: Enhancements Needed for Loan Monitoring 
System Benchmark Study. GAO/AIMD-99-165. Washington, D.C.: May 14, 
1999.

Managing for Results: Measuring Program Results That Are Under Limited 
Federal Control. GAO/GGD-99-16. Washington, D.C.: December 11, 1998.

Managing for Results: Analytic Challenges in Measuring Performance. 
GAO/HEHS/GGD-97-138. Washington, D.C.: May 30, 1997.

Federal Grants: Design Improvements Could Help Federal Resources Go 
Further. GAO/AIMD-97-7. Washington, D.C.: December 18, 1996.

Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to 
Society. GAO/RCED-92-106. Washington, D.C.: May 1992.


FOOTNOTES

[1] U.S. General Accounting Office, Federal Grants: More Can Be Done to 
Improve Weed and Seed Program Management, GAO/GGD-99-110 (Washington, 
D.C.: July 1999).

[2] Discretionary grants are awarded to eligible grantees, most often 
on a competitive basis. 

[3] A grantee is an entity that receives funding from EOWS to implement 
the Weed and Seed program. The grantee distributes the funds to sites 
and is responsible for ensuring that sites comply with the terms of the 
grant. A grantee may have more than one site.

[4] A site is a geographically defined area ranging in size from 
several neighborhood blocks to several square miles. With the input of 
the local U.S. Attorney's Office and the site's steering committee, 
each site develops and implements its own Weed and Seed program.

[5] Two detailees are from the Federal Bureau of Investigation (FBI) 
and one is from the Department of the Interior. The FBI detailees work 
part-time at EOWS.

[6] There are 93 U.S. Attorneys throughout the United States, Puerto 
Rico, the Virgin Islands, Guam, and the Northern Mariana Islands. Each 
U.S. Attorney is the chief federal law enforcement officer of the 
United States within his or her particular jurisdiction. The U.S. 
Attorneys serve as the nation's principal litigators under the 
direction of the Attorney General. 

[7] When applicants apply for funding, EOWS creates and maintains 
official recognition files. If applicants receive funding, OJP creates 
and maintains official grant files.

[8] U.S. General Accounting Office, Standards for Internal Control in 
the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 
1999).

[9] Throughout the rest of this report, we refer to both new and 
existing sites as sites. 

[10] EOWS uses the information in the documents as the basis to award 
official recognition.

[11] The remaining 187 of the 218 sites received funding in fiscal year 
2002, but were initially funded prior to fiscal year 2002. 

[12] Progress reports describe Weed and Seed activities and the 
accomplishment of objectives in a site's funding application. Progress 
reports help EOWS officials determine how sites are meeting program 
objectives and assist them in making future grant decisions. Progress 
reports are due 30 days after June 30 and December 31, respectively.

[13] Grantees may have more than one site.

[14] EOWS refers to these activities as benchmarks. For a list of 
EOWS's benchmarks, see appendix IV.

[15] See GAO/GGD-99-110. 

[16] P.L. 103-62.

[17] Performance measures translate program goals into concrete, 
observable conditions that determine what data to collect to learn 
whether progress has been made toward achieving program goals. Such 
measures are meant to cover the key aspects of performance that will 
enable programs to assess accomplishments, make decisions, realign 
processes, and assign accountability.

[18] EOWS officials told us that the Weed and Seed program has three 
main goals: (1) reduce violent crime, (2) reduce drug crime, and (3) 
coordinate Weed and Seed funds with other resources. Such goals are 
related to the overall intent of the program to weed out crime from 
targeted neighborhoods and seed them with a variety of programs to 
prevent crime from recurring.

[19] U.S. General Accounting Office, Managing for Results: Analytic 
Challenges in Measuring Performance, GAO/HEHS/GGD-97-138 (Washington, 
D.C.: May 30, 1997). 



[20] OMB periodically assesses goals and results of federal programs, 
such as Weed and Seed, and reports on its findings. The purpose is to 
tie performance with the budget process.

[21] U.S. General Accounting Office, Managing for Results: Measuring 
Program Results That Are Under Limited Federal Control,GAO/GGD-99-16 
(Washington, D.C.: December 11, 1998) and GAO/HEHS/GGD-97-138.

[22] See GAO/GGD-99-16.

[23] U.S General Accounting Office, Law Enforcement: Better Performance 
Measures Needed to Assess Results of Justice's Office of Science and 
Technology, GAO-04-198 (Washington, D.C.: November 2003).

[24] See GAO/GGD-99-16.

[25] It is commonly accepted that safety belt use reduces fatality 
rates and the severity of injuries. See U.S. General Accounting Office, 
Highway Safety: Safety Belts Use Laws Save Lives and Reduce Costs to 
Society, GAO/RCED-92-106 (Washington, D.C.: May 1992)

[26] In November 2003 the Justice Research and Statistics Association 
completed a study comparing homicide trends in sites to their host 
jurisdictions. However, this study relied on homicide rates collected 
by EOWS, rather than developing additional performance measures. 

[27] Sites funded under EOWS's competitive application can be either 
the first Weed and Seed site in the jurisdiction that received official 
recognition or a new geographical area (that is, not contiguous with an 
existing or former target area) in a jurisdiction with an existing (or 
continuation) or former Weed and Seed site. Sites funded under EOWS's 
continuation application are in their second, third, fourth, or fifth 
year of funding and implementing the Weed and Seed program.

[28] GAO did not verify the funds leveraged by Weed and Seed sites.

[29] GAO did not verify the funds leveraged by Weed and Seed sites.

[30] GAO did not verify funds leveraged by Weed and Seed sites.

[31] A matching requirement requires grant recipients to contribute 
their own funds to obtain federal grant funds. The Weed and Seed 
program is authorized and funded by the Department of Justice annual 
appropriation acts. These acts have not included either a requirement 
for matching funds or a limitation on the maximum number of years a 
grantee may receive funding. See for example, Departments of Commerce, 
Justice, and State, the Judiciary, and Related Agencies Appropriations 
Act, 1998, P.L. 105-119, 111 Stat. 2440, 2454. 

[32] As discussed earlier, beginning in 1999, EOWS imposed a 5-year 
site expansion rule, under which a site is awarded funding for 5 years 
and may receive an additional 5 years of funding if the majority of 
funds are shifted to a new site. This is not a requirement imposed by 
the law.

[33] U.S. General Accounting Office, Federal Grants: Design 
Improvements Could Help Federal Resources Go Further, GAO/AIMD-97-7 
(Washington, D.C.: December 1996).

[34] Grants for hiring and rehiring may be renewed for up to 5 years. 
Grants for other purposes may not cover more than 3 years. 

GAO's Mission:

The General Accounting Office, the investigative arm of Congress, 
exists to support Congress in meeting its constitutional 
responsibilities and to help improve the performance and accountability 
of the federal government for the American people. GAO examines the use 
of public funds; evaluates federal programs and policies; and provides 
analyses, recommendations, and other assistance to help Congress make 
informed oversight, policy, and funding decisions. GAO's commitment to 
good government is reflected in its core values of accountability, 
integrity, and reliability.

Obtaining Copies of GAO Reports and Testimony:

The fastest and easiest way to obtain copies of GAO documents at no 
cost is through the Internet. GAO's Web site ( www.gao.gov ) contains 
abstracts and full-text files of current reports and testimony and an 
expanding archive of older products. The Web site features a search 
engine to help you locate documents using key words and phrases. You 
can print these documents in their entirety, including charts and other 
graphics.

Each day, GAO issues a list of newly released reports, testimony, and 
correspondence. GAO posts this list, known as "Today's Reports," on its 
Web site daily. The list contains links to the full-text document 
files. To have GAO e-mail this list to you every afternoon, go to 
www.gao.gov and select "Subscribe to e-mail alerts" under the "Order 
GAO Products" heading.

Order by Mail or Phone:

The first copy of each printed report is free. Additional copies are $2 
each. A check or money order should be made out to the Superintendent 
of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 
more copies mailed to a single address are discounted 25 percent. 
Orders should be sent to:

U.S. General Accounting Office

441 G Street NW,

Room LM Washington,

D.C. 20548:

To order by Phone: 	

	Voice: (202) 512-6000:

	TDD: (202) 512-2537:

	Fax: (202) 512-6061:

To Report Fraud, Waste, and Abuse in Federal Programs:

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470:

Public Affairs:

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S.

General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C.

20548: