Detailed Information on the
Firearms Programs -- Integrated Violence Reduction Strategy Assessment

ATF will develop an evaluation strategy which subjects the program to external, rigorous and thorough, regularly scheduled independent evaluations.

ATF will continue to work with the Department and OMB on legislative proposals regarding graduated sanctions for Federal Firearms Licensees (FFLs) when they are found to be non-compliant. Graduated sanctions for FFL non-compliance are included in the proposed "Violent Crime and Anti-Terrorism Act of 2007."

ATF will ensure that all financial management related material weaknesses identified are corrected in an expeditious and comprehensive manner.

ATF will develop ambitious and/or challenging targets for the annual performance measures associated with its firearms program.

Measure: Percentage of firearms investigations resulting in a referral for criminal prosecution

Explanation:This measure reflects the percentage of investigations within the firearms program area, in which a defendant was referred for criminal prosecution.

Measure: Percentage of firearms investigations involving the recovery of crime guns

Explanation:This measure reflects the percentage of investigations within the firearms program area, that involved the recovery of a crime gun (a crime gun is any firearm that is illegally possessed, used in a crime, or suspected to have been used in a crime).

Measure: Number of gang related defendants

Explanation:This measure captures the number of defendants in gang-related investigations that have resulted in cases perfected and forwarded for prosecution.

Measure: Number of defendants who are involved in firearms trafficking

Explanation:This measure captures the number of defendants in firearm(s) trafficking investigations that have resulted in cases being perfected and forwarded for prosecution.

Measure: Number of defendants in investigations referred for prosecution who are felons in possession of firearms

Explanation:This measure captures the number of defendants in investigations referred for prosecution under Title 18 of the United States Code, Section 922(g)(1). This section of law prohibits possession of a firearm by persons convicted of a crime punishable by imprisonment for a term exceeding one year.

Measure: Increase pawn brokers' compliance rate with Federal firearms laws and regulations

Explanation:This measure is intended to capture the effect of ATF's initiatives towards the strategic objective of increasing the rate of compliance with firearms laws and regulations. ATF conducted a sampling of the pawn broker Federal firearms licensees in FY 2005, and have since begun an enhanced outreach and inspection initiative with these licensees. This measure will either be "met" or "not met" based on another sampling in 2009.

Measure: Reduction in unaccounted firearms

Explanation:This measure seeks to reflect the impact of ATF's regulatory mission of improving the compliance of industry members (see Objective 1.2 in ATF Strategic Plan). The percentage reflects the total reduction in the number of firearms that are unaccounted for during re-inspections of certain Federal firearms licensees, as compared to the licensees' previous inspections. The measure focuses on the important outcome of ensuring that firearms can be accounted for by industry members, facilitating traces of firearms recovered in crimes.

Measure: Percent of firearms traces completed within nine days

Explanation:The National Tracing Center Division is tasked with accurately and efficiently performing firearms trace requests for Federal, State, Local and International law enforcement agencies to provide investigative leads. Timeliness is a critical component relative to the value of the investigative lead. This measure creates a time standard for completing a trace within nine days, and measures the success towards meeting this trace completion timeframe. The number is the percentage firearm traces completed, that are completed within nine days.

Is the program purpose clear?

Explanation: ATF's firearms authorities are based primarily on three Federal statutes: the Gun Control Act (GCA), the National Firearms Act (NFA) and the Arms Export Control Act (AECA). At the direction of the Attorney General, ATF enforces the Federal firearms laws and regulations, as well as provides support to Federal, State, and local law enforcement officials in their fight against crime and violence. ATF employs its Integrated Violence Reduction Strategy (IVRS), a comprehensive and integrated set of programs to enforce the firearms laws to remove violent offenders from communities, stop illegal firearms trafficking, keep prohibited persons from possessing firearms, and pursue outreach and prevention efforts. The stated goal of IVRS is to reduce violent firearms crime. ATF worked closely with the Department of Justice to develop the President's Project Safe Neighborhoods (PSN) initiative, which was implemented in 2001. PSN is the Administration's primary violent crime initiative with the stated purpose of preventing and deterring gun violence. IVRS is ATF's contribution to PSN.

Evidence: ATF's firearms program mission is clearly stated in the Bureau's Strategic Plan (2004-2009) which states the goal as: "Prevent violent crime involving firearms." The IVRS strategic plan further articulates this mission: "to identify, investigate, and recommend prosecution of violent firearms offenders and other prohibited individuals, stop illegal firearms traffickers through effective enforcement and regulation of the firearms industry, and promote community participation in an effort to prevent violent behavior." The IVRS strategic plan outlines tactics and performance measures that support ATF's mission. ATF's statutory responsibilities are defined in ATF Publication 5300.4: Federal Firearms Regulations Reference Guide 2005.

Does the program address a specific and existing problem, interest, or need?

Explanation: Violent crime remains a significant and complex domestic problem, stemming from a variety of causes that vary from region to region. A common element, however, is the relationship between firearms violence and the unlawful diversion of firearms out of commerce into the hands of prohibited persons. ATF's statutory responsibility to investigate firearms crime and regulate the firearms industry, along with assets including technology and information, are focused under IVRS to: remove violent offenders, including gang members, from our communities; keep firearms from those who are by law prohibited from possessing them; discourage, prohibit, and interrupt illegal weapons transfers in accordance with the law; and prevent firearms violence through community outreach.

Evidence: The 2005 crime statistics released by the Bureau of Justice Statistics reveal there were nearly 480,000 victims of violent crimes involving firearms. According to the Uniform Crime Report, 1.4 million violent crimes were committed nationwide and 68 percent of all murders in the United States were committed by firearms. Independent reports of violent crime indicate alarming increases in violent crime with firearms (see Police Executive Research Forum, Violent Crime in America: 24 Months of Alarming Trends).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: By virtue of its statutory authorities under the Gun Control Act (GCA), the National Firearms Act (NFA) and the Arms Export Control Act (AECA), as well as through the expertise it has developed, ATF is uniquely equipped to identify and investigate firearms cases that span jurisdictions and in which interstate firearms trafficking may be involved. The GCA seeks to prevent the misuse of firearms to commit violence and other crime by keeping firearms from designated categories of prohibited persons and ensuring compliance with control on commerce in firearm. The GCA also provides for a separate offense when a person uses or carries a firearm during and in relation to a Federal crime of violence or drug trafficking offense, with escalating penalties relating to the involvement of the firearm in the crime. The GCA also provides a mandatory minimum penalty for armed career criminals convicted on three separate previous occasions for violent felonies or serious drug offenses. While there is no specific offense referred to as "firearms trafficking" in the GCA, several sections are useful in addressing this national problem. For example it is unlawful for any person to engage in the business of importing, manufacturing, or dealing in firearms without a Federal firearms license and it is unlawful for any person to make a false statement or provide fictitious identification in connection with the purchase or attempted purchase of a firearm. The GCA requires any person in the business of dealing, manufacturing or importing firearms to be licensed and maintain records, and gives ATF licensing and inspection authority to ensure compliance. The GCA also places strict controls on machine guns. Under the NFA, ATF maintains a registry of a limited group of firearms considered to be especially dangerous (for example, machineguns, sawed-off shotguns, and destructive devices such as bazookas and bombs), and prohibits the transfer or making of NFA firearms without prior approval and payment of tax. ATF's efforts in these regards complement local efforts. IVRS leverages partnerships with other Federal, State and local law enforcement agencies to address the complexity of violent crime collaboratively. ATF is also the only law enforcement organization that has the ability to trace firearms and compare ballistics through its National Integrated Ballistics Information Network (NIBIN) and is the only Federal entity that has the authority to issue firearms licenses and hold firearms dealers accountable for firearms by means of product accountability records. Given the complexity of many investigations, however, instances of overlap are unavoidable. Attempts to avoid confliction with other agencies for investigations and events are standard field practice.

Evidence: ATF's programs are designed specifically to administer and enforce the Federal firearms laws and regulations for which it has primary responsibility. ATF works with other Federal, State, and local agencies through formal task forces, such as Violent Crime Impact Teams (VCITs) and informal collaboration and by providing technological assistance to ensure efforts are mutually supportive rather than redundant. Department wide coordination processes have been established with regards to new task force deployments with potential investigatory overlap (see AG Anti-Gang Coordination Committee memo and Threat Assessment & Law Enforcement Activity Recommendation (task force application) to assist in limiting overlap and reducing redundancy.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: ATF brings much value to America's fight against violent firearms-related crime and works closely with other law enforcement agencies and jurisdictions to accomplish its mission. ATF is, however, hampered somewhat in its ability to regulate the over 65,000 Federal Firearms Licensees (FFLS) due to the fact that there are no graduated sanctions available to ATF if a licensee if found to be non-compliant; it is either license revocation or nothing at all. ATF, therefore, must reserve revocations for the most serious offenses, while other less serious, but nonetheless important issues of non-compliance go unchecked. ATF could pursue legislative strategies to address some of the barriers to effective regulatory enforcement that it has identified.

Evidence: Numerous academics have supported the notion of a comprehensive approach (such as that embodied by the IVRS program) to reducing firearms violence (see attached publications).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: ATF targets firearms violence by decentralizing prioritization of IVRS activities to its field offices. ATF has established a performance-based mechanism upon which decisions are based, the Field Division Strategic Operating Plans (FDSOPs), to include prioritization and resource allocation among various IVRS programs. These plans require special agents in charge (SACs) to identify goals, resource needs, challenges and obstacles, and best practices in customized strategies for their field divisions, while at the same time allowing ATF Headquarters to better coordinate long-range planning and resource allocation efforts on a national scale. Resources are targeted so as to reach intended beneficiaries only. ATF has developed accompanying statistical reports - the Industry Operations Quarterly Data Report (QDR) and the Strategic Investigative Data Report (SIDR) to provide regular feedback to field divisions and expand Headquarters-level program assessment capabilities. At the field level, SACs engage in a management and accountability process modeled on COMPSTAT (comparative statistics) in which needs and performance form the basis of decisions regarding resource allocation.

Evidence: The Violent Crime Impact Team (VCIT) strategy applies technology, such as crime gun mapping and ballistics identification to identify local hot spots and to target, investigate and arrest the worst of the worst criminals. Specifically, the application process for participation in the VCIT program requires jurisdictions provide a detailed description of the problem, and how the task force will help tackle the specified problem. The effectiveness of the VCIT program was also designed to be measurable through a combination of output and outcome measures, including numbers of crime guns recovered, number of "worst of the worst" criminals arrested, and reductions in homicides committed with firearms. Utlimately VCIT implementation is based on the recommendation and concurrence of the local ATF SAC, U.S. Attorney, Police Chief and other participants at the local and national levels. Field Division Strategic Operating Plans (FDSOPs), Industry Operations Quarterly Data Report (QDR), Strategic Investigative Data Report (SIDR), COMPSTAT - the management and accountability process modeled on comparative statistics. COMPSTAT is based on four principles which have proven to be essential ingredients of an effective crime fighting strategy: accurate and timely intelligence, effective tactics, rapid deployment of personnel and resources, and relentless follow-up and assessment. The philosophy behind COMSTAT is one of accountability and commitment to solving problems. Resources are targeted so as to reach intended beneficiaries only.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ATF has three long-term performance measures that focus on outcomes and are directly tied to its firearms mission and goals. ATF has identified a long term measure that directly relates to its regulatory and outreach efforts. In FY 2004, ATF completed inspections of more than 1,200 randomly selected firearms dealers. ATF has utilized information from the random sampling, along with operational intelligence, to design programs that focus regulatory resources on those segments of the industry most likely to be operating contrary to the laws and regulations. Specifically, for example, ATF has increased the frequency of inspections of the pawnbroker population and has initiated an aggressive outreach program with the pawnbroker industry, which was found to have a significantly reduced level of compliance when compared to other industry members. In FY 2009, ATF will conduct another random sampling program of the pawnbroker population to determine to what level this aggressive inspection and outreach effort increased the rate of compliance.

Evidence: ATF has three long term measures, including: (1) the percentage of firearms investigations resulting in a referral for criminal prosecution; (2) the percentage of firearms investigations involving the recovery of crime guns; and (3) the increase of pawn brokers' compliance with Federal firearms laws and regulations. These measures directly and meaningfully tie to ATF's mission and purpose of enforcing Federal criminal laws and regulating the firearms and explosives industries. Specifically, these long term measures reflect (1) the percentage of investigations in which a defendant was referred for criminal prosecution, (2) that involve the recovery of a crime gun (a crime gun is any firearm that is illegally possessed, used in a crime, or suspected to have been used in a crime, and (3) the effectiveness of ATF's initiatives on increasing the rate of compliance with firearms laws and regulations. Two of these measures reflect the percentage of investigations that result in specific outcomes, while the other captures its progress towards reaching its regulatory and outreach objectives.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ATF has ambitious targets and timeframes to meet its long-term goals by 2012. For example, 62% of investigations must result in a referral for criminal prosecution - up from 57% in 2006, and 40% of investigations must result in the recovery of crime guns - up from 30% in 2006. Additionally by 2012, ATF must increase the compliance rate among the pawn broker Federal firearms licensees from 41% (in 2005) to 64%.

Evidence: Long term Outcome Measure: Percentage of firearms investigations resulting in a referral for criminal prosecution Year Target Actual 2005 N/A 55% 2006 N/A 57% 2007 57% TBD 2008 58% TBD 2009 59% TBD 2010 60% TBD 2011 61% TBD 2012 62% TBD Long-term Outcome Measure: Percentage of firearms investigations involving the recovery of crime guns Year Target Actual 2005 N/A 26% 2006 N/A 30% 2007 30% TBD 2008 32% TBD 2009 34% TBD 2010 36% TBD 2011 38% TBD 2012 40% TBD Long-term Outcome Measure: Increase pawn brokers' compliance rate with Federal firearms laws and regulations Year Target Actual 2005 Baseline 41% 2006 45% TBD 2007 50% TBD 2008 53% TBD 2009 57% TBD 2010 60% TBD 2011 62% TBD 2012 64% TBD

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ATF has a limited number of annual measures that address IVRS primary strategic objectives of reducing firearms-related violence, and preventing prohibited persons from possessing firearms. The annual measures are specific to criminal activities that are a priority to the Federal law enforcement agency responsible for enforcing the Federal firearms laws and regulations and include, the number of gang related defendants, the number of defendants who are involved in firearms trafficking, the number of defendants in investigations referred for prosecution who are felons in possession of firearms, and the percent of firearms traces completed within nine days. Additionally, ATF has a regulatory annual performance measure which focuses on the effects of ATF presence and the rate of repeat violations by FFLs (specifically unaccounted for firearms).

Evidence: In support of its long term goals of increasing the percentage of firearms investigations resulting in a referral for criminal prosecution and involving the recovery of crime guns, ATF has developed annual measures including: (1) capturing the number of defendants in gang-related investigations that have resulted in cases perfected and forwarded for prosecution; (2) capturing the number of defendants in firearm(s) trafficking investigations that have resulted in cases being perfected and forwarded for prosecution; and (3) capturing the number of defendants in investigations referred for prosecution under laws which prohibit the possession of a firearm by persons convicted of a crime. In support of its long term goal of increasing the rate of compliance with Federal firearms laws and regulations for pawn brokers, ATF has developed annual measures including: (1) reducing the number of unaccounted for firearms; and, (2) increasing its proficiency in completing firearms traces. ATF's annual measures are in direct support of its long term goals.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Annual performance measure targets associated with the long term goals are relatively static, however, baselines have been established for each of the annual measures and are seemingly easily achievable. Increases established in the two annual measures for the number of defendants in gang-related cases, as well as for defendants involved in firearms trafficking, support two major priorities of the agency, however, the annual measure targets are not considered ambitious.

Evidence: See the annual performance measures.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ATF partners at the Federal, State, and local law enforcement level are committed to reducing violent firearms-related crime. Whereas, ATF requires a commitment on behalf of its partners to its stated goals and objectives, there is no evidence that ATF requires its partners to measure or report on their progress towards the achievement of those long-term or annual goals. In various instances, ATF is called into criminal investigations by the FBI, DEA, Customs, and local law enforcement for the purpose of pursuing charges within ATF's unique jurisdiction or providing specialized expertise. However, ATF participation in criminal investigations is not solely confined to these areas and committment to ATF annual and long term program goals is not a prerequisite or requirement to obtain ATF assistance or participation in an investigation. The commitment from partner agencies is articulated and agreed to in Memorandums of Understanding (MOU) and Memorandums of Agreement. ATF Orders explicitly direct task forces to commit to "goals, objectives, and performance measures to reflect the outcome of the task force effort as required by the Government Performance and Results Act." Additionally, the Department of Justice has established procedures to coordinate task forces. ATF works closely with industry and trade association partners, such as the National Sports Shooting Foundation (NSSF) towards the goals of the firearms programs. Specifically, ATF is in a joint program with the NSSF, with shared costs, to promote FFL and public awareness. The "Don't Lie for the Other Guy" program is a joint venture to increase compliance among industry members. Where able, ATF incorporates performance goals into National Tracing Center (NTC) contracts which explicitly include performance standards that directly impact upon achieving the "percentage of firearms traces completed in nine days" efficiency measure. In addition, ATF's Federal Firearms Licensing Center, National Firearms Act Branch and Firearms and Explosives Imports Branch are all currently developing workload measures for incorporation into employee critical elements (evaluation criteria). These workload measures will enable ATF to detect any slowdown in the processing of licenses, permits, and NFA transfers/registrations. This will allow for redistribution of work to ensure more efficient and effective results.

Evidence: ATF has executed MOUs with local agencies participating in NIBIN and task forces At ATF's NTC, ATF utilizes contractors to assist in firearms tracing and has built into their contract the expectation of improving trace response times). ATF's programs are also an important and integrated component of the President's Project Safe Neighborhoods initiative. PSN sets clear priorities for all partners; specifically, (1) heightened enforcement of all federal laws against illegal gun traffickers, and the corrupt federal firearms licensees that supply them, with an emphasis on those gun traffickers who supply illegal firearms to violent organizations and to juveniles; and, (2) renewed aggressive enforcement of federal firearms laws against those persons prohibited from possessing firearms or who use firearms in furtherance of illegal activities, including those persons denied under the Brady Act. ATF's Violent Crime Impact Team (VCIT) program requires submission of crime data monthly and law enforcement activity information weekly. The current gang task force coordination process documents the commitment (to ATF annual and/or long term goals?) of all the partners in the effort prior to approval of establishing such a task force. With National Integrated Ballistics Information Network (NIBIN), agencies must report monthly to ATF on their achievements with the equipment. ATF utilizes memoranda of understanding (MOUs) to gain commitment from partners towards the goals of the programs.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Although ATF has had several Inspector General and General Accounting Office (GAO) reviews of its programs over the past several years resulting in performance recommendations that were subsequently implemented, it has not been subjected to external, rigorous and thorough, regularly scheduled independent evaluations that examine overall how well the program is accomplishing its mission and meeting its long-term goals and objectives.

Evidence: DOJ reviews have been performed on the implementation of individual components of ATF's firearms program, such as DOJ's Inspector General's reviews of VCIT and the firearms inspections program, although not on a routine or regular basis. Additionally, ATF is collaborating with academia through the National Institutes of Justice (NIJ) to evaluate VCIT. ATF previously partnered with academia in 2000 in a review of ATF's enforcement of Federal laws against firearms traffickers, resulting in the "Following the Gun" report. Additionally, under Project Safe Neighborhoods, NIJ has sponsored periodic evaluations of firearms violence reduction programs are conducted in which ATF is a component of the reviews.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The 2008 Congressional budget request shows money and personnel, but not direct and indirect costs allocated to the different initiatives within the Firearms program, related to the outcome impact of the requested incremental (enhancement) changes. ATF's firearms budget represents 72% of the overall funding for the Bureau. Requests for enhancements (funding increases) are linked to accomplishment of current the annual and long-term performance goals. (New long term performance goals and annual measures are not mentioned in FY 2008 budget submission, however are expected to be included in the 2009 budget request. Reference to direct and indirect costs required to attain various performance results, including applicable agency overhead, retirement, and other costs are referenced in ATF's budget submission to OMB, but not in ATF's Congressional Budget request.) IT budget requests (Exhibits 51, 53) advance ATF and DOJ strategic goals. ATF's IT Strategic Plan supports DOJ's goals and objectives, as well as the Attorney General's priorities.

Evidence: In FY 2008, the firearms initiatives are attributable to funding enchancements in one annual outcome performance measure, "Number of defendants who are involved in trafficking". As cited within the FY 2008 Congressional Budget budget request, "this initiative supports the Department of Justice's strategic goal two: Enforce Federal Laws and Represent the Rights of the American People. ATF projects that convictions of violent criminals and firearms traffickers will increase with the increase in additional staffing for PSN enforcement efforts."

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The IVRS Strategic plan outlines the strategic planning process. ATF, working with stakeholders, has taken steps to correct strategic planning deficiencies by recently developed three new long-term outcome measures for inclusion in the 2009 Budget. ATF worked to adopt specific, ambitious long-term performance goals which are complimented by a limited number of annual performance goals and workload measures (with baselines, targets and timeframes) that demonstrate progress toward achieving the long-term goals. In addition, ATF has produced an interim status update on recommendations contained in the OIG's report regarding ATF's Violent Crime Impact Team (VCIT) Strategy.

Evidence: The IVRS Strategic Plan includes a few highly focused objectives and lays out the tactics that will be used to reach those objectives. Furthermore, performance indicators and goals are outlined for several years. ATF developed three specific long term measures that largely reflect the intended outcome of both the criminal and regulatory enforcement missions of the agency. In support of these long term measures, ATF has a few specific workload measures as well as annual performance measures that monitor progress toward achieving the long term goals.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Information from the case management system and the financial management system is collected quarterly, with the exception of violent firearms crime data, which is collected yearly. ATF also conducts operational reviews of each office once every 3 years (beginning in FY07, every 4 years), in which information about the quality of ATF's services is received from stakeholders. Additionally, ATF regularly meets with industry trade associations and industry members to obtain feedback on improvements to ATF's efforts to regulate the industry, while minimizing disruptions to business operations. Examples of actions taken to improve program performance include: implementation of the FFL pawnbroker initiative based on industry sampling, a streamlined process to submit crime gun traces, field division COMPSTAT-type process, Brady Operations field referral criteria, and implementation of Access 2000, which enables ATF to have 24-hour access to the records of several major manufacturers. ATF uses opportunities to collect performance information from the regulated industry by attending regular industry events, such as the Shooting Hunting Outdoor Trade Show (SHOT), and quarterly and semi-annual meetings with industry trade associations.

Evidence: ATF management is provided with quarterly reports and internal operational reviews with stakeholders. Field Divisions receive updated information from a case management system reflecting outcomes (See Strategic Investigative Data Report) that is used to manage (adjust, realign, reallocate) personnel and resources. The case management data is used to assess field managers' progress towards achieving goals set forth in their Field Division strategic planning document and as documented in their SES work plans.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: ATF managers have performance-based service contracts and are tied directly to those performance results. ATF managers in the field are held accountable for results through their performance work plans. However, at this time there is no uniform method for holding program partners accountable based on performance data and contribution to the program's goals.

Evidence: The case management data is used to assess field managers' progress towards achieving goals set forth in their Field Division strategic planning document and as documented in their SES work plans. ATF has monthly reviews where reports on cost and schedule variances are presented (anything over 8% must go to the IT governance board for further explanation). ATF also has a policy on Earned Value Management, now being implemented in all contracts over $1 million in value. ATF Federal Managers' performance work plans are directly tied to these initiatives by using the Federal Acquisition Regulation EVM measures as standards. In a more specific example, NIBIN Program participants are audited (by whom and how often?) to ensure that the capital equipment provided to them is being utilized, and a procedure is being developed to reclaim unused equipment. (If they are found to be non-compliant, what accountability measures are imposed?)

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: In OIG Audit Report 07-12, January 2007, ATF received a repeat material weakness on its process for recording accounts payable due to the continued existence of errors and inaccuracies with the accrual process used to record accounts payable. The accounts payable weakness was first reported in FY 2004. In its Independent Auditors' Report on Compliance and Other Matters on ATF's FY 2006 Financial Statement, the auditors also noted that ATF's financial management systems did not substantially comply with applicable federal financial management systems requirements and federal accounting standards." In most instances, funds allocated to program divisions are obligated for their intended purpose in a timely manner. Supervisory review procedures are in place to ensure that funds are expended appropriately. Obligations, commitments, and receivers are reviewed quarterly for accuracy and completeness. Financial transactions are managed through a monthly accountability process and discrepancies and errors are addressed.

Evidence: ATF's financial management system allows for tracking of spending by project code, and regular reviews of expenditures are conducted. Funds are obligated on an annual basis except for no-year carry over accounts. From a total of all sources of $1.002B of that amount $18M was non-obligated, of that non-obligated balance $9M was no-year carry over accounts to FY 2007. Therefore the amount that was actually non-obligated is $9M which is less then 1% of funds that remained non-obligated.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ATF has developed and deployed a new Field Division Strategic Operating Plan process comprehensively across all field divisions. Additionally, ATF developed complimenting statistical reports to provide regular feedback with regards to measuring and achieving efficiencies and effectiveness in program execution. With regards to information technology, ATF has an Earned Value Management (EVM) policy. Based on that EVM policy (performance data), ATF is implementing a system for reasonable cost comparisons using historical cost data for IT projects. This enables ATF to use estimating methodologies and similar work breakdown schedules to determine the actual scope and reasonable expectations of efficiencies and cost effectiveness. Also, as part of its Capital Planning and Investment process, ATF's governance boards in the Evaluation Phase review major projects (in the Evaluation Phase Review) to determine how effectively these projects are managed. Measures considered include how well the team performed and how well the solution satisfied the business need. IT improvements are approved and monitored through a process led by the recommendations of the Information Resource Management Council, which reviews business cases (which are required for all initiatives) and monitors milestones of approved projects. Examples of such projects are improvements ATF has made in its operations involving firearms tracing, the comparison of ballistics information, and the referral of investigative information. ATF has one annual efficiency measure; increasing the percent of firearms traces completed within nine days. ATF's National Tracing Center Division (NTCD) is tasked with accurately and efficiently performing firearms trace requests for Federal, State, Local and International law enforcement agencies to provide investigative leads. Timeliness is a critical component relative to the value of the investigative lead. This measure creates a time standard for completing a trace within nine days, and measures the success towards meeting this trace completion timeframe. The number is the percentage firearm traces completed, that are completed within nine days. At present, the NTCD completes 68% of its firearms traces within 9 days. While this overall percentage is expected to remain constant through 2012, the actual number of firearms traces performed is expected to increase annually.

Evidence: ATF has added the capability to submit trace requests electronically, speeding up a process that was very time-consuming for local agencies and was the major barrier to 100% tracing of crime guns. Access 2000 enables ATF to have 24-hour access to the records of several major manufacturers. Referrals of NICS/Brady violators have been streamlined in response to an audit recommendation, so that information is provided to field offices more quickly. The ATF National Tracing Center closely monitors and measures efficiency though its annual accountability report. ATF Field Offices utilize the Strategic Investigative Data Report and Quarterly Data Report (QDR) which incorporates the Full Time Equivalents in each Field Division with the work product produced.

Does the program collaborate and coordinate effectively with related programs?

Explanation: In implementing the Administration's Project Safe Neighborhoods initiative, ATF actively collaborates with Federal, State, and local law enforcement and regulatory agencies, industry, and industry associations to accomplish shared missions.

Evidence: ATF regularly coordinates training programs and information seminars for partner agencies. ATF also provides firearms trafficking training and training in tracing procedures for Federal, State, local, and international agencies. ATF has signed memoranda of understanding with the FBI for the Joint Terrorism Task Force and the U.S. Customs Service for investigations. ATF has worked with the U.S. Attorney Offices to incorporate prosecutorial guidelines in the early stages of the review and referral of NICS/Brady violators for investigative follow-up. ATF coordinates with other Federal, State and local agencies by working in partnership through formal task forces and informal collaboration. Department-wide coordination processes have been established with regards to new task force deployments with potential investigatory overlap (see AG Anti-Gang Coordination Committee memo and Task Force Application). An example of collaboration with industry trade associations is ATF's outreach efforts with NSSF through "Don't Lie for the Other Guy" educational seminars.

Does the program use strong financial management practices?

Explanation: ATF has received three consecutive unqualified audit opinions on its financial operations (FY 2003-2006). ATF is carrying two repeat material weaknesses, one for information technology and one for the recording of accounts payable. These weaknesses are Bureau-wide and not specific to this program, however the firearms program constitutes 72% of the Bureau. The first material weakness addresses the need for ATF to implement effective controls over information systems general access controls and system software. The second material weakness addresses ATF's process for recording accounts payable due to the continued existence of errors and inaccuracies with the accrual process used to record accounts payable. Also, in OIG Report 07-12, on ATF's FY 2006 Audited Financial Statement, auditors stated again (FY 2004, FY 2005, FY 2006) that ATF's financial management systems did not substantially comply with applicable federal financial management systems requirements and federal accounting standards. With regard to the Chief Financial Officer's Audit for FY 2006, the material weakness consisted of several repeat issues, as well as a few new issues. ATF recognizes that ongoing action is warranted to prevent recurrence of problems, and that it must continue to implement corrective actions that will address problems at the root cause. ATF implements the Department's self-assessment of internal control in IT areas. ATF will also ensure that self-assessments are accurate and complete using the Department's Computer Security Assessment and Management (CSAM) program and the results are adequate to detect and correct weaknesses prior to external reviews and audits.

Evidence: ATF has implemented the requirements of OMB Circular A-123 which requires annual assessments of its internal controls. ATF has identified no internal control deficiencies beyond the material weaknesses for accounts payable. ATF has developed a new corrective action plan to eliminate this weakness. The accounts payable accrual will be based on independent verifications of invoices billed, unbilled and estimates from ATF's high dollar vendors on a quarterly basis. Contracting Officers and the Financial Management Division (FMD) will work closely with the vendors to obtain reliable reporting information. The accrual for the new headquarters building will be recorded quarterly based on procedures developed by FMD and approved by the auditors at the end of fiscal year 2006. Other accounts payables areas will be based on past billings in order to project estimates for recurring monthly charges such as utilities. There are no other weaknesses in ATF's financial management practices as evidenced in our unqualified audit opinion. ATF will aggressively monitor the implementation of this new plan to clear this accounts payable weakness.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: As required by OMB Circular A-123, ATF's Office of Inspection reviews all directorates (divisions, districts, administrative components, and other offices) on a 3-year cycle (beginning in FY07, the reviews will be done on a 4-year cycle). The review covers all aspects of management, including funds management, procedures, personnel management, and asset utilization. ATF also utilizes a log to track and follow up on all external audit findings and recommendations. A key management deficiency issue that requires ATF attention and action is the National Integrated Ballistics Information Network, or NIBIN. This program compares images of ballistic evidence (projectiles and cartridge casings) obtained from crime scenes and recovered firearms. As a result, this system has amassed a large ballistics image database filled with crime gun data from all over the country from which Federal, State, and local law enforcement agencies obtain intelligence information. At this point, however, there is only a single source provider of technology and equipment to support NIBIN. This is problematic in assuring system continuity, upgrades, and deployment to users on a cost-effective and efficient basis.

Evidence: In addition to the OMB Circular A-123 assessment described above, ATF also requires all of its directorates to do their own self assessments through a specific management control tool; the results of these self assessments are evaluated as part of the certification required under A-123 . The Office of Enforcement Programs and Services (EPS) does a thorough job of reviewing their internal controls on an annual basis in accordance with OMB, DOJ and ATF guidelines. For the FY 2006 Mid-year Internal Control Assessment Review, EPS was in compliance in most areas. Where necessary, EPS executed corrective action plans to address deficiencies. ATF has implemented recommendations in OIG reports and audits such as: resolved case identification numbers, established plan to enhance promotion of NIBIN, instituted purchase card refresher training, included a step to analyze financial statement audit results when determining risk of improper and erroneous payments.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: A new long-term outcome goal has been identified related to the pawnbroker industry(see Question 2.1). ATF has increased the frequency of inspections of the pawnbroker population and has initiated an aggressive outreach program with the pawnbroker industry, which was found to have a significantly reduced level of compliance when compared to other industry members. ATF has historic data to support progress towards the achievement of its long term goal to increase pawnbrokers' compliance with Federal firearm laws and regulations. However, the only long term measure that exceeded its target (2006) was in reduced unaccounted for firearms. The other two long-term measure display actuals as baselines only (not comparable to targets).

Evidence: For the previous three years (2004, 2005 and 2006) the percent reduction in unaccounted firearms (in recall inspections) has consistently been above the established goal).

Does the program (including program partners) achieve its annual performance goals?

Explanation: ATF has made considerable progress towards its annual performance goals. ATF met or exceeded each of the annual performance goals for 2006. In several instances, however, these are newly established performance goals and measures with baseline and targets informed by historical data. Whereas, ATF requires a commitment on behalf of its partners to its stated goals and objectives, there is no evidence that ATF requires its partners to measure or report on their progress towards the achievement of those long-term or annual goals.

Evidence: ATF has many newly development performance measures and goals. For existing goals, ATF demonstrated improvements and achievements. Most notably have been the increases in gang-related defendants, defendants in firearms trafficking investigations, as well as the efficiency measure relating to trace completions, while the unaccounted firearms measure has consistently outperformed the annual goal. ATF will work to ensure that its program partners commit to annual targets and achieve them as well. Annual Outcome Number of gang related defendants Year Target Actual 2003 Baseline 1,123 2004 Baseline 2,010 2005 Baseline 3,136 2006 3,800 3,821 2007 3,800 TBD 2008 3,800 TBD 2009 3,900 TBD 2010 3,950 TBD 2011 4,000 TBD 2012 4,050 TBD Annual Outcome Number of defendants who are involved in firearms trafficking Year Target Actual 2003 Baseline 2,380 2004 Baseline 2,957 2005 Baseline 3,253 2006 3,000 3,017 2007 3,000 TBD 2008 3,150 TBD 2009 3,300 TBD 2010 3,350 TBD 2011 3,400 TBD 2012 3,450 TBD Annual Output Number of defendants in investigations referred for prosecution who are felons in possession of firearms Year Target Actual 2003 Baseline 8,034 2004 Baseline 8,944 2005 Baseline 9,388 2006 8,700 8,655 2007 8,700 TBD 2008 8,700 TBD 2009 8,700 TBD 2010 8,700 TBD 2011 8,700 TBD 2012 8,700 TBD Annual Outcome Reduction in unaccounted firearms Year Target Actual 2003 Baseline 85% 2004 Baseline 90% 2005 Baseline 90% 2006 75% 91% 2007 91% TBD 2008 92% TBD 2009 92% TBD 2010 93% TBD 2011 93% TBD 2012 93% TBD Annual Efficiency Percent of firearms traces completed within nine days Year Target Actual 2003 Baseline 48% 2004 Baseline 61% 2005 Baseline 58% 2006 63% 68% 2007 67% TBD 2008 67% TBD 2009 68% TBD 2010 68% TBD 2011 68% TBD 2012 68% TBD

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: ATF has developed a new Field Division Strategic Operating Plan process. Between the FDSOP and the accountability mechanisms in place for field management, ATF carefully assess the programs outcomes in relation to the resources committed to these outcomes. ATF carefully applies its analysis of cost efficiency in its application to law enforcement operations (the bulk of activities in the firearms programs), and sets efficiency targets only with regard to those more process oriented firearms programs applications such as the National Tracing Center (percent of firearms traces completed within 9 days).

Evidence: Since FY 2000 ATF has significantly increased certain law enforcement "outputs" per agent. Additionally, the efficiency measure regarding the percentage of traces completed within 9 days has increased from 48% in 2003 to 68% in 2006, yet is not expected to improve beyond 68% through 2012. Beyond this high-level annual efficiency measure, ATF monitors processes closely, and has demonstrated significant improved efficiencies. From processing applications to make, register, or transfer National Firearms Act weapons, to numbers of crime guns traced, ATF closely monitors both the outputs (and outcomes) and the resources to generate these results. For example, the National Tracing Center's Productivity Report outlines significant improvements in efficiencies.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Many Federal, State and local law enforcement agencies investigate violent crime and ATF's focus and expertise on firearms trafficking and other firearms-related crime is an important component of those investigations. ATF has primary jurisdiction to investigate Federal firearms crimes while other law enforcement entities also investigate and prosecute firearms offenses. ATF helps identify and investigate firearms cases that span jurisdictions and in which interstate firearms trafficking may be involved. ATF has the ability to trace firearms and compare ballistics through its National Integrated Ballistics Information Network (NIBIN) and is to issue firearms licensees. ATF's efforts in these regards complement and support local law enforcement crime investigation efforts.

Evidence: ATF and its partner agencies combine their expertise and assets, within their defined jurisdictions, to apply a comprehensive strategy under which they consolidate each other's efforts to fight violent crime such as initiatives under Project Safe Neighborhoods (PSN). ATF coordinates with its other federal and industry partners, including using the FBI's NICS system to provide criminal background checks on those wanting to purchase firearms legally, to keep firearms out of the hands of criminals and help prevent violent gun crimes.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations of programs within IVRS (FFL inspections, VCIT and NIBIN) have not indicated major deficiencies. These reports did indicate a lack of the ability to assess effectiveness. As noted previously ATF has adopted a number of effectiveness measures. There are a number of academic reviews that support the strategies employed by ATF, and indicate that these strategies are working. However, there is a lack of relevant evaluations at the national program level to demonstrate overall program effectiveness.

Evidence: In FFL inspections and VCIT, OIG Reports indicated that there was insufficient documentation to support the assertion of achievements. ATF is working through NIJ to assess the effectiveness of the VCIT program. There are numerous case studies conducted by academia (such as Michigan State University) through the National Institue of Justice (NIJ) of targeted local projects in which ATF is the lead Federal law enforcement partner (under PSN), which indicate that the program is effective and achieving results. A study of NIBIN sponsored by the equipment manufacturer is currently underway which looks at the effectiveness of automated ballistics comparison. A final conclusion has not been made. ATF has also approached Defense Advanced Research Projects Agency (DARPA) and National Institute of Standards and Technology (NIST) to evaluate the possibility of alternate methods of automated ballistics comparisons; this evaluation is also not complete at the time of the report.