This is the accessible text file for GAO report number GAO-05-73 entitled 'Defense Inventory: DOD and Prime Contractors Adhered to Requirements in Selected Contracts for Overseeing Spare Parts Quality' which was released on December 20, 2004. This text file was formatted by the U.S. Government Accountability Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. This is a work of the U.S. government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. Because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Report to the Senate and House Subcommittees on Defense, Committees on Appropriations: United States Government Accountability Office: GAO: December 2004: Defense Inventory: DOD and Prime Contractors Adhered to Requirements in Selected Contracts for Overseeing Spare Parts Quality: GAO-05-73: GAO Highlights: Highlights of GAO-05-73, a report to the Senate and House Subcommittees on Defense, Committees on Appropriations: Why GAO Did This Study: In the 2004 Defense Appropriations Act, Congress mandated that GAO examine and report on the oversight of prime contractors by the Department of Defense (DOD) and the oversight of subcontractors by the prime contractors. Contract quality assurance oversight is intended to assess whether contractors are capable of and are providing supplies or services that meet contract quality and technical requirements. Providing effective oversight is challenging. DCMA recognizes that the risk of nonconforming parts reaching end users exists, given the diversity of contracts, parts, and products used to meet weapon systems requirements and uses a risk management process to guide its efforts. For fiscal year 2003, government quality assurance oversight was required for approximately 273,000 contracts. GAO determined (1) whether DOD provided quality assurance oversight and enforcement over its spare parts prime contractors, (2) if prime contractors provided quality assurance oversight over their subcontractors, and (3) how DOD held prime contractors accountable for overseeing the subcontractors’ work. To address these objectives, GAO judgmentally selected and reviewed 15 contracts awarded to 11 prime contractors by the services and the Defense Logistics Agency. In commenting on a draft of this report, DOD provided one technical comment, which GAO incorporated as appropriate. What GAO Found: GAO’s review of the 15 contracts showed that quality assurance personnel within the Defense Contract Management Agency (DCMA)—DOD’s primary organization for providing quality assurance oversight— generally followed established policies, guidance, regulations, and contract requirements in performing oversight and enforcement over spare parts prime contractors. This oversight ranged from conducting physical inspection of parts, such as testing the measurements and functions of a part to evaluating contractor production processes to observing the outer appearance and counting the number of parts for compliance with contract requirements. When one of the prime contractor’s processes and another contractor’s parts did not meet contract requirements, DCMA used its enforcement system by issuing requests for corrective action by the prime contractors. GAO found that the 11 prime contractors reviewed provided quality assurance oversight over their subcontractors’ work in accordance with industry standards and contract requirements. The contractors used at least two and up to four methods in providing quality assurance oversight over their subcontractors. These methods included evaluating potential subcontractors for placement on an Approved Supplier List, requiring certifications of parts and processes, testing parts and processes, and tracking and monitoring subcontractor’s performance. The primary methods of oversight were evaluating subcontractors for placement on an Approved Supplier List and requiring certifications that parts and processes conform to contractual specifications. Establishing an Approved Supplier List served to identify subcontractors capable of producing needed parts or processes in accordance with industry standards and contractual specifications. In GAO’s review of the 15 contracts, DCMA held prime contractors accountable for their subcontractors’ work by requiring that the prime contractors adhere to contract clauses concerning oversight responsibility. Most of the contracts included either clauses stating that the prime contractor shall provide supplies that conform to contract requirements or clauses related to other quality requirements. When nonconformance was reported, DCMA quality assurance personnel and the prime contractor determined if the deficiency was due to contractor nonconformance and assigned responsibility for corrective action. GAO identified one deficiency from the 15 contracts that the prime contractor was responsible for and DCMA held the prime contractor accountable for the part. While GAO did not identify any major deficiencies from the contracts and practices it reviewed, GAO recognizes that the risk of nonconforming spare parts reaching end users exists. Compliance by contractors, DCMA, and other DOD agencies with established internal controls helps mitigate against this risk. www.gao.gov/cgi-bin/getrpt?GAO-05-73. To view the full product, including the scope and methodology, click on the link above. For more information, contact William M. Solis at (202) 512-8365 or solisw@gao.gov. [End of section] Contents: Letter: Results in Brief: Background: DCMA Provided Quality Assurance Oversight and Enforcement over Contractors: Prime Contractors Adhered to Industry Standards in Providing Quality Assurance Oversight over Subcontractors: DCMA Held Prime Contractors Accountable by Requiring Adherence to Contract Clauses: Concluding Observations: Agency Comments and Our Evaluation: Appendix I: Scope and Methodology: Appendix II: DCMA Offices and Contractors Reviewed: Appendix III: Comments from the Department of Defense: Appendix IV: GAO Contact and Staff Acknowledgments: Tables: Table 1: Types of Inspections Performed by DCMA Quality Assurance Specialists for the Contractors We Reviewed: Table 2: Methods of Quality Assurance Oversight over Subcontractors Provided by the Prime Contractors We Reviewed: Abbreviations: DCMA: Defense Contract Management Agency: DLA: Defense Logistics Agency: DOD: Department of Defense: United States Government Accountability Office: Washington, DC 20548: December 20, 2004: The Honorable Ted Stevens: Chairman: The Honorable Daniel Inouye: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations United States Senate: The Honorable Jerry Lewis: Chairman: The Honorable John Murtha: Ranking Minority Member: Subcommittee on Defense: Committee on Appropriations: House of Representatives: Military personnel rely upon weapon systems such as aircraft and tanks to successfully perform their missions on the battlefield. These systems contain numerous individual parts and assemblies produced by many different contractors and subcontractors. For example, an engine for an F-15 aircraft has approximately 16,000 parts and 1,600 different part numbers.[Footnote 1] Although there is one prime contractor responsible for producing this engine, there are numerous subcontractors supplying the parts and assemblies required to build the engine under numerous contracts, and quality assurance is required throughout the production process. While the military services' contracting offices oversee a small percentage of the Department of Defense's (DOD) contracts for acquiring the parts, the Defense Contract Management Agency (DCMA), through its quality assurance specialists, provides quality assurance oversight over the majority of these contracts. Quality assurance oversight is defined as the various functions, including inspections, performed to determine whether a contractor has fulfilled the contractual obligations related to quality and quantity. This oversight is intended to assess whether contractors are capable of and are providing supplies or services that meet contract quality and technical requirements. Providing effective oversight is challenging, and DCMA recognizes that the risk of nonconforming parts reaching end users exists, given the diversity of contracts, parts, and products used to meet weapon systems requirements. For fiscal year 2003, DCMA was responsible for performing government quality assurance oversight over approximately 273,000 contract actions.[Footnote 2] To help mitigate risks inherent with spare parts acquisitions, DOD developed a risk management guide that discusses risk and risk management, examines risk management concepts, and provides a practical reference for dealing with acquisition risk. DCMA used this guide in developing its Risk Assessment and Management Program that is used to assess the risk of a contractor providing nonconforming parts. According to DCMA officials, for the majority of spare parts prime contractors that had contracts requiring oversight by DCMA and used subcontractors in providing the parts, DCMA relied upon the prime contractors to oversee their subcontractors. In accordance with the Federal Acquisition Regulation, DCMA is only responsible for providing quality assurance oversight over subcontractors when required to do so in the government's interest. When DCMA provides quality assurance oversight over subcontractors, the prime contractor is not relieved of any quality assurance oversight responsibilities under the contract. This report responds to the mandate in section 8143 of the 2004 Defense Appropriations Act that GAO examine and report on the oversight of prime contractors by DOD and the oversight of subcontractors by prime contractors.[Footnote 3] Our review focused on whether DCMA and spare parts prime contractors are fulfilling their quality assurance oversight responsibilities as required by regulations, policies, procedures, and contract provisions. Specifically, our objectives were to determine whether: 1. DCMA provided quality assurance oversight and enforcement over its spare parts prime contractors in accordance with established policies, procedures, and guidance; and: 2. prime contractors provided quality assurance oversight over their subcontractors' work related to producing spare parts in accordance with industry standards and contract requirements. We also reviewed how DCMA held prime contractors accountable for overseeing their subcontractors' work. To address our objectives, we reviewed regulations, policies, and procedures related to the quality assurance oversight over spare parts contractors provided by DCMA and prime contractors and the enforcement actions available to assure compliance. In addition, we reviewed 15 contracts awarded to 11 prime contractors and any reports on product quality deficiency for these contracts[Footnote 4]. Specifically: * To assess whether DCMA provided quality assurance oversight over the prime contractors, we compared the quality assurance provisions included in the 15 contracts to oversight actions performed by DCMA quality assurance specialists.[Footnote 5] In assessing whether DCMA used enforcement actions, we looked for instances of spare parts nonconformance related to these contracts and determined if DCMA levied any enforcement actions against the prime contractors. * To assess prime contractors' oversight of their subcontractors, we interviewed representatives from the 11 prime contractors and determined if the prime contractors used subcontractors. For those that used subcontractors, we held discussions with prime contractor personnel related to the level of quality assurance oversight they performed over the subcontractors and reviewed their processes for managing suppliers and subcontractors. We also gathered documentation from the prime contractors on test results of parts produced by subcontractors. * To assess how DCMA held prime contractors accountable for the work of their subcontractors, we reviewed the 15 contracts to determine if they included clauses holding the prime contractor responsible for the spare parts provided to the government. For the instances of nonconforming parts reported against these contracts, we reviewed how DCMA handled the reported nonconformance and whether DCMA held the prime contractor responsible for correcting the nonconformance. More detailed information about our scope and methodology is contained in appendix I. We conducted our review from November 2003 through October 2004 in accordance with generally accepted government auditing standards. Results in Brief: In our review of the 15 contracts, DCMA quality assurance specialists followed established policies, procedures, and guidance in performing quality assurance oversight and enforcement over the spare parts prime contractors. DCMA's oversight was provided through inspections that included: (1) physical inspection of parts such as testing the measurements and functions of parts, (2) evaluation of the prime contractors' processes to determine compliance with established contract requirements and production procedures, and (3) observation of the outer appearance of spare parts and taking a physical count of the parts to ensure that the proper number of parts are being shipped to the end user. When one of the prime contractor's processes and another contractor's parts did not meet contract requirements, DCMA used an enforcement system that involved issuing requests for corrective action by the prime contractors. Corrective actions involved requiring the prime contractor to make corrections or changes to the production process and demonstrating how processes would be improved to prevent further instances of nonconformance. According to DCMA, it was not necessary to levy severe enforcement actions such as penalties and debarment from conducting business with the federal government against any of the contractors that we reviewed. The prime contractors we reviewed provided quality assurance oversight over their subcontractors' work related to producing spare parts in accordance with accepted industry standards and contract requirements. Based on industry standards, prime contractors used at least two and up to four methods in providing quality assurance oversight over their subcontractors' work. These methods included (1) evaluating and visiting potential subcontractors for placement on a list of approved suppliers for a particular part, (2) requiring certifications from the subcontractors that the parts or processes were produced in accordance with established industry standards and contract requirements, (3) performing periodic testing of parts produced by subcontractors, and (4) tracking and monitoring approved subcontractors' past performance. The primary methods of oversight used were evaluating subcontractors for placement on the prime contractor's Approved Supplier List and requiring certifications that their parts and processes conform to contractual specifications. Establishing an Approved Supplier List served to identify qualified subcontractors capable of producing needed parts or processes in accordance with industry standards and contractual specifications. Industry standards, developed and voted upon by a group of technical experts representing standards organizations such as Aerospace Standards and International Organization of Standardization, allowed prime contractors flexibility in providing quality assurance oversight over subcontractors considering the type and size of the organization and the product(s) produced. In our review of the 15 contracts, DCMA held prime contractors accountable for overseeing their subcontractors' work by requiring that the prime contractors adhere to contract clauses concerning oversight responsibility. Most of the contracts we reviewed included either the Federal Acquisition Regulation clause stating that the prime contractor shall tender to the government for acceptance only supplies that conform with contract requirements or other clauses related to quality assurance requirements. When instances of product quality deficiency were reported, the DCMA quality assurance specialist and the prime contractor determined if the deficiency was due to contractor nonconformance and assigned responsibility for corrective action. For the 15 contracts we reviewed, we identified one deficiency that was determined to be the responsibility of the prime contractor and DCMA held the prime contractor accountable for the part. DCMA also performed reviews of subcontractors' certifications to the prime contractor that the parts and processes produced by the subcontractor were manufactured in accordance with the contract requirements. These reviews served as a check by DCMA to determine whether the prime contractors' quality assurance systems were adequate for oversight of their subcontractors' work. During our review, service officials provided us with examples of other contracts in which nonconforming parts reached end users. DCMA followed its procedures for evaluating the causes of these nonconforming parts. The reasons for the nonconformance varied for each part. We are not making any recommendations in this report. While we did not identify any major deficiencies from the contracts and practices we reviewed, we recognize that the risk of nonconforming spare parts reaching end users exists. Compliance by contractors, DCMA, and other DOD agencies with established internal controls helps mitigate against this risk. In written comments on a draft of this report, DOD provided one technical comment, which we incorporated as appropriate. The department's comments and our evaluation are on page 17 of this report. Background: The Federal Acquisition Regulation is the primary regulation for use by all federal agencies in acquiring supplies and services. Sections 46 and 52, which discuss and set forth quality assurance and contract clauses, respectively, provide guidance in determining quality assurance responsibilities. According to the Federal Acquisition Regulation, government contract quality assurance is defined as the various functions, including inspections, performed by the government to determine whether a contractor has fulfilled the contract obligations pertaining to quality and quantity. These inspections can either be conducted at the contractor's place of manufacture and production (source) or at the receiving location for the parts (destination). DOD's Quality Assurance Oversight for Contracts Is Provided Primarily by DCMA: DCMA is DOD's primary organization for performing quality assurance oversight for contracts and this oversight responsibility typically only covers prime contractors. DCMA's execution of its quality assurance responsibility is primarily through its source inspection program. Source inspections are defined as inspections at the point where goods are manufactured or assembled. There are three types of source inspections: physical inspection; contractor process reviews; and kind, count, and condition. Physical inspection involves inspecting parts by comparing the parts to a specification, drawing, or other instruction. Contractor process reviews are inspections of processes and procedures for establishing confidence that the procured parts will produce a desired outcome. Inspections involving kind, count, and condition are inspections intended to visually identify and verify the quantity and exterior appearance of a part to determine if it visually meets contract specifications. During fiscal year 2003, DCMA was responsible for government source inspection for approximately 273,000 contracts. The Federal Acquisition Regulation requires that contracts include inspection and other quality requirements that will protect the interest of the government. It also provides guidance in establishing which clauses to include in the various types of contracts. The clauses included in each contract dictate the type and level of quality assurance oversight to be performed. DCMA quality assurance specialists are expected to follow the inspection and acceptance provisions in each contract in determining whether they are required to perform quality assurance oversight. If the contract states that inspection and acceptance will be at destination, then DCMA does not perform any quality assurance oversight. End users within military units, such as Army battalions and Air Force squadrons, perform inspection and acceptance for these contracts. Questions have been raised about the efficiency of DCMA's quality assurance oversight program. In October 2003, the DOD Inspector General reported that of the 518 contracts requiring DCMA source inspection that they reviewed, at least 172 of the inspections provided either nominal or no value to the DOD quality assurance process.[Footnote 6] The DOD Inspector General also pointed out concerns in the DOD quality assurance program, including (1) ambiguity in the level and extent of requested source inspections; (2) inconsistent and unclear application of items defined as critical or having a critical application; (3) inconsistent implementation of inspection procedures for items considered commercial, off-the-shelf; and (4) arbitrary and inconsistent inspection procedures for items purchased from distributors. DCMA Provided Quality Assurance Oversight and Enforcement over Contractors: In our review of 15 contracts awarded to 11 contractors, DCMA provided quality assurance oversight and enforcement over these spare parts prime contractors. DCMA used three types of inspections to perform quality assurance oversight over the contractors. DCMA adhered to the Federal Acquisition Regulation, contract quality requirements, and DCMA guidance in providing quality assurance oversight over these prime contractors. Federal Acquisition Regulation clauses included in the contracts guide whether or not DCMA would perform quality assurance oversight. DCMA provides oversight for those contracts requiring inspection at the place of manufacture of the parts. Enforcement of spare parts quality and safety during the production process was achieved through the issuance of corrective action requests. DCMA Used Three Types of Inspections to Perform Quality Assurance Oversight over Contractors: DCMA quality assurance specialists followed established policies, procedures, and guidance in performing their oversight over the spare parts prime contractors in our review. As shown in table 1, the quality assurance specialists performed one or a combination of three types of inspections over prime contractors. The three types of inspections include: physical inspection to measure the dimensions of parts or to test the parts; process reviews; or an observation of the kind, count, and condition of the parts. At 7 of the 11 contractor locations , DCMA quality assurance specialists performed both process reviews and physical inspections. Table 1: Types of Inspections Performed by DCMA Quality Assurance Specialists for the Contractors We Reviewed: 1; Prime contractor: Contractor A; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. 2; Prime contractor: Contractor B; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. 3; Prime contractor: Contractor C; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. 4; Prime contractor: Contractor D; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. 5; Prime contractor: Contractor E; Physical inspection: No; Process reviews: No; Kind, count, & condition: Yes[A]. 6; Prime contractor: Contractor F; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: Yes. 7; Prime contractor: Contractor G; Physical inspection: Yes; Process reviews: No; Kind, count, & condition: No. 8; Prime contractor: Contractor H; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. 9; Prime contractor: Contractor I; Physical inspection: No; Process reviews: Yes; Kind, count, & condition: No. 10; Prime contractor: Contractor J; Physical inspection: No; Process reviews: No; Kind, count, & condition: Yes[B]. 11; Prime contractor: Contractor K; Physical inspection: Yes; Process reviews: Yes; Kind, count, & condition: No. Source: GAO analysis. [A] Various types of inspections were performed at subsidiaries of this contractor. [B] This contractor is a distributor of parts made by other manufacturers. [End of table] For physical inspection of spare parts, the quality assurance specialist selects a sample of the parts from various production runs and inspects them by comparing the parts to a specification, drawing, or other instruction. For example, at one location, the prime contractor had procured the part from a subcontractor and required the subcontractor to provide test data related to the part. The quality assurance specialist reviewed the test data and compared it to the part specifications. Also, the quality assurance specialist inspected the number imprinted on the parts, exterior painting, and workmanship of the received parts to ensure they complied with contract specifications. The second type of inspection involves evaluating the prime contractors' processes to determine compliance with established contract requirements and production procedures. During these inspections the quality assurance specialist assesses the prime contractor's processes and production line procedures in relation to established industry practices and provides the contractor an early opportunity to make corrections or improvements, if necessary. For example, at one prime contractor location, the quality assurance specialist monitored the prime contractor's key processes during the production, including the cleaning, painting, welding, and final quality inspection of the product. To help ensure a comprehensive quality assurance check, the quality assurance specialist performed his daily quality checks at different phases of the production process. Also, the quality assurance specialist reviewed the prime contractor's procedures for selecting its subcontractors to determine if the subcontractors were certified in accordance with the applicable industry standards. The third type of inspection consists of observing the kind, count, and condition of the parts. Observation of the kind of part includes visual identification of at least one part for each different part being procured under the contract and verifying the part number against the number required in the contract. Counting the parts involves visual confirmation of the contents of one package per line item and counting the number of packages received. The quality assurance specialists verify the physical appearance of the parts to assess their condition. For example, at one location, the quality assurance specialist performed a kind, count, and condition inspection to confirm that the contractor had the correct part by verifying the part number, checking to ensure the contractor had the proper quantity of parts, and inspecting the outward appearance of the parts. DCMA Adhered to the Federal Acquisition Regulation, Contract Quality Requirements, and DCMA Guidance in Providing Quality Assurance Oversight over Prime Contractors: DCMA's quality assurance oversight over the 11 spare parts prime contractors in our review was in accordance with the Federal Acquisition Regulation, contract quality requirements, and DCMA's One Book guidance. Each contract we reviewed designated the location of inspection and acceptance by contract line item. Contracts that were designated for inspection and acceptance at the contractor's place of manufacture and production received DCMA quality assurance oversight. When contracts are designated for inspection and acceptance at the receiving location for the parts, the end user is responsible for inspecting the procured part and the DCMA quality assurance specialist typically does not get involved with the contract. Of the 15 contracts we reviewed, DCMA provided quality assurance oversight for the 13 contracts that were designated for inspection and acceptance at source. The other two contracts were designated for inspection and acceptance at destination and did not require DCMA quality assurance oversight. In accordance with the One Book guidance, quality assurance specialists performed inspections and acceptance for its customers to ensure supplies were in compliance with contract requirements. According to a DCMA official, this guidance allows DCMA quality assurance specialists flexibility in providing quality assurance oversight over prime contractors. For example, the guidance does not include standard requirements for the number of tests, site visits, or inspections that the DCMA quality assurance specialists should perform while providing quality assurance oversight. When contracts were designated for inspection and acceptance at source, quality assurance specialists typically reviewed contract requirements, assessed the contractor's risks of producing nonconforming parts, determined what needed to be done to mitigate the risks, and applied quality assurance oversight, including inspections, based on the contractor's risk level. Enforcement of Spare Parts Quality and Safety during the Production Process Was Achieved through Corrective Action Requests: During the production process, when a prime contractor's processes or spare parts did not meet contract requirements, DCMA used an enforcement system that involved issuing requests for corrective action by the prime contractor. According to the Federal Acquisition Regulation, contractors must be given an opportunity to correct or replace nonconforming supplies. Contractors are notified about nonconformance through corrective action requests issued by DCMA or product quality deficiency reports issued by the end user. When there is contractual nonconformance during the production process, DCMA may issue the prime contractor a corrective action request to formally communicate the deficiency and request corrective action on the part of the prime contractor. When the prime contractor does not take corrective actions, contractual remedies available to procuring contracting officers include suspension of progress payments, termination for default, and penalties such as suspension or debarment from holding contracts with the government. Only 2 of the 11 prime contractors that we reviewed received corrective action requests related to the contracts in our review. According to DCMA officials, the remaining contractors did not warrant corrective action requests related to the contracts we reviewed. Our review of DCMA files related to these contractors also did not identify any need for corrective actions by the prime contractors. Corrective actions identified by the two contractors involved the prime contractor making corrections or changes to its production processes and demonstrating how processes would be improved to prevent further instances of nonconformance. For example, DCMA issued a corrective action request to a prime contractor identifying loose insulation, an inactive gear, poor painting quality, and parts that did not meet surface finish specification requirements as nonconforming items. The prime contractor corrected the nonconformance and DCMA accepted the product. To prevent reoccurrence of the deficiencies, the prime contractor reported that it had taken the following actions: (1) discontinued the use of material that caused the loose insulation, (2) instructed operators to ensure that gears were properly adjusted, (3) arranged a meeting with all contractor paint personnel advising them on the importance of attention to detail, and (4) agreed to provide their quality assurance representatives with acceptable and unacceptable finish samples as visual standards to meet customer expectations. For the other prime contractor, DCMA issued a corrective action request because the contractor used the wrong procedure to receive approval for a major waiver from contract requirements.[Footnote 7] The request for major waiver was supposed to go through an array of signatures and DCMA approvals, whereas a minor waiver could be submitted and approved electronically, requiring fewer signatures and approvals. However, the prime contractor downgraded the waiver request from major to minor without the appropriate concurrence and approval of DCMA and submitted the request through the electronic system. According to the prime contractor, their personnel had conflicting procedures describing how to process waivers. To prevent reoccurrence of incorrect processing of major waivers, the contractor planned to review its current procedures for processing waivers. Also, the contractor planned to train its quality assurance staff to ensure they understand the correct procedures for processing waivers. Prime Contractors Adhered to Industry Standards in Providing Quality Assurance Oversight over Subcontractors: The prime contractors in our review adhered to industry standards in providing quality assurance oversight over their subcontractors' work. Based on industry standards, prime contractors performed at least two or up to four methods to provide quality assurance oversight over their subcontractors, as shown in table 2. The primary methods of oversight used were evaluating subcontractors for placement on an Approved Supplier List and requiring certifications of parts and processes. Industry standards, such as the International Organization for Standardization 9001 and Aerospace Standards 9100, require that an organization have a quality management system in place to ensure that it will produce high quality products that will serve their intended purpose. The standards are broad and are intended to be applicable to all organizations, regardless of type, size, and product provided. Table 2: Methods of Quality Assurance Oversight over Subcontractors Provided by the Prime Contractors We Reviewed: 1; Prime contractor: Contractor A; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: No. 2; Prime contractor: Contractor B; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: No. 3; Prime contractor: Contractor C; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: No; Tracked and monitored subcontractors' performance: Yes. 4; Prime contractor: Contractor D; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: No; Tracked and monitored subcontractors' performance: Yes. 5; Prime contractor: Contractor E; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: Yes. 6; Prime contractor: Contractor F; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: Yes. 7; Prime contractor: Contractor G; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: Yes. 8; Prime contractor: Contractor H; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: Yes. 9; Prime contractor: Contractor I; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: Yes. 10; Prime contractor: Contractor J; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: No; Tracked and monitored subcontractors' performance: No. 11; Prime contractor: Contractor K; Evaluated potential subcontractors for their approved supplier list: Yes; Required certifications: Yes; Tested parts and processes produced: Yes; Tracked and monitored subcontractors' performance: No. Source: GAO analysis. [End of table] All of the prime contractors that we reviewed evaluated potential subcontractors prior to the contract award for placement on their Approved Supplier List. The purpose of establishing an Approved Supplier List is to identify qualified subcontractors capable of producing needed parts or processes in accordance with industry standards and contractual specifications. When evaluating a potential subcontractor for inclusion on their Approved Supplier List, some prime contractors periodically visited their subcontractors' production facilities, requested that subcontractors complete surveys containing questions regarding the subcontractors' capabilities and qualifications necessary to produce parts and processes, or examined information about the technical skills and qualifications of subcontractor personnel, past performance for producing similar products, and applicable certifications related to the subcontractors' operations. Six of the 11 prime contractors said they periodically visited potential subcontractors prior to contract award. All of the prime contractors that we reviewed required certifications such as independent, third-party certifications or certificates of conformance from their subcontractors to certify that their parts and processes are in accordance with contract requirements and industry standards.[Footnote 8] Independent, third-party certifications and certificates of conformance served as verification that the subcontractors could produce parts and processes that conformed to contractual specifications. Prime contractors required certifications from their subcontractors for different phases of the production process. For example, one contractor used steel to fabricate parts and required a certification from the steel subcontractor that the steel had been produced according to specifications. In this instance, the prime contractor did not use the steel provided by the subcontractor until they received the certificate of conformance verifying that the product was in accordance with industry standards and contractual requirements. Eight of the 11 prime contractors we reviewed periodically tested parts or processes produced by their subcontractors. Prime contractors tested the subcontractors' parts or processes at either the manufacturing site or the receiving point to determine whether products or processes met contractual specifications. For example, one prime contractor performed mechanical and electrical tests on all materials received from subcontractors to ensure that the materials met contract specifications. If the materials did not meet contract specifications, the prime contractor's review board, which included the government quality assurance specialist, made a determination concerning the disposition of the materials. Disposition options included using the material "as is" or scrapping it. Seven of the prime contractors that we reviewed tracked and monitored the performance of their subcontractors by establishing performance goals, assessing and rating the subcontractors' performance, or recommending corrective and preventive actions when subcontractors produced nonconforming parts. The seven prime contractors established performance goals and rated their subcontractors on a routine and periodic basis using various performance metrics, such as product quality and on-time deliveries. For example, one contractor kept track of the number of nonconforming parts provided by each subcontractor in relation to the total number of parts provided. When a subcontractor's nonconformance rate exceeded the prime contractor's acceptable goal, the prime contractor placed the subcontractor on probation. DCMA Held Prime Contractors Accountable by Requiring Adherence to Contract Clauses: In our review of the 15 contracts, DCMA held prime contractors accountable for overseeing their subcontractors' work by requiring that prime contractors adhered to contract clauses concerning oversight responsibility. When instances of nonconformance were reported through product quality deficiency reports, the DCMA quality assurance personnel and the prime contractor determined if the deficiency was due to contractor nonconformance and assigned responsibility for corrective action. For the 15 contracts we reviewed, one deficiency was determined to be the responsibility of the prime contractor and DCMA held the prime contractor accountable for the part. During our review, service officials provided us with examples of other contracts in which nonconforming parts reached end users. DCMA followed its procedures for evaluating the causes of these nonconforming parts. The reasons for the nonconformance varied for each part. Most of the 15 contracts we reviewed included Federal Acquisition Regulation clause 52.246-2, Inspection of Supplies--Fixed Price, which states that the prime contractor shall tender to the government for acceptance only supplies that have been inspected in accordance with the inspection system and found by the contractor to be in conformity with contract requirements. This contract clause also states that the contractor is not relieved of its oversight responsibility when government quality assurance over subcontractors is required. The remaining contracts included other quality clauses or did not specify quality requirements because the contractors had quality systems that had been previously approved by the procuring contracting officers. DCMA also performed reviews of subcontractors' certifications to the prime contractor that the parts and processes produced by the subcontractor were manufactured in accordance with the contract requirements. DCMA quality assurance specialists periodically reviewed third-party certifications and contractors' documents related to site visits, receiving inspections, and other oversight of subcontractors. These reviews provided a technique for DCMA to determine whether the contractors' quality assurance systems were adequate for oversight of subcontractors. For example, at one contractor location, the quality assurance specialist obtained copies of certifications that steel was produced and heat-treated in accordance with standards. The specialist also reviewed copies of test records maintained by the contractor during the production process. After parts are provided to end users within military units, such as Army battalions and Air Force squadrons, instances of nonconforming parts are reported through product quality deficiency reports. End users issue product quality deficiency reports to identify deficiencies in parts that may indicate nonconformance with contractual or specification requirements. When the end user identifies a nonconforming product, the user issues a product quality deficiency report that is sent to the applicable DCMA contract management office and distributed to the quality assurance specialist responsible for overseeing the contractor that produced the item. The quality assurance specialist notifies the contractor of the report. The contractor may request that the part be returned to its facility for testing to determine whether the problem that has been identified by the user can be duplicated. The DCMA quality assurance specialist and the prime contractor also determine if the deficiency was due to contractor nonconformance with contract requirements and assign responsibility for corrective action. DCMA writes the final disposition of the product quality deficiency report based on the results of the tests completed by the contractor and the assessment of who was responsible for the deficiency. In those cases where the deficiency was determined to be the responsibility of the prime contractor, DCMA held the prime contractor accountable for the part. If the cause of the deficiency was tied back to a subcontractor, DCMA held the prime contractor responsible for correcting the deficiency and ensuring that the subcontractor's processes were modified to correct the cause of the nonconformance. During our review, service officials provided examples of other contracts in which nonconforming parts reached end users for various reasons. DCMA and prime contractors followed their procedures in evaluating causes for the nonconformance. For example, in March 2004 one prime contractor was notified that five wiring harnesses, manufactured by one of their approved subcontractors, were defective. Investigations showed that the prime contractor's quality oversight system did not detect the problem because its personnel were unfamiliar with drawings, specifications, or electrical wiring harness fabrications. After the problem was identified, the prime contractor provided training related to drawings and proper manufacturing processes to its quality assurance personnel as well as subcontractor personnel. However, according to the prime contractor, the subcontractor still could not produce the wiring harnesses correctly. As a result, the prime contractor selected another subcontractor to manufacture the wiring harnesses. In another case, the contract was for a survival kit that included critical safety items. The nonconformance related to an O-ring lubricant that was allowing oxygen pressure to be released prematurely. Based on the deficiency report, the contractor began using another lubricant for the O-ring that eliminated the problem. For this contract, DCMA recommended that government source inspection be added at the subcontractor level because no prior government source inspection was required. Yet in another example, an axle component manufactured by a subcontractor broke on an aircraft landing gear because the dimensions of the axle component were incorrect. Incorrect dimensions resulted from the subcontractor's improper grinding process; yet, the subcontractor never reported the discrepancies to the prime contractor. Since the incident, the prime contractor has placed the subcontractor on probation within the prime contractor's quality approval system. The subcontractor will remain on probation until an audit is performed by the end users of the axle to verify that all corrective actions are in effect. Concluding Observations: DCMA and the prime contractors we reviewed utilized a number of processes to provide quality assurance oversight over the production of spare parts for the military. The processes included conducting physical inspections of parts produced by contractors, reviewing prime contractor's processes, evaluating potential subcontractors for placement on an Approved Supplier List, requiring certifications of parts and processes, testing parts and processes, and tracking and monitoring subcontractor's performance. These processes are founded upon contractual requirements, DCMA policies, and industry standards for quality assurance. In addition, there are enforcement procedures that DCMA uses when nonconforming spare parts reach end users. However, despite these quality assurance controls, some risk still exists. For example, while we did not identify any major deficiencies from the contracts and practices we reviewed, service officials provided examples of nonconforming parts related to contracts not included in our review that reached end users for various reasons. Furthermore, given the vast number of contracts and contractors involved in providing spare parts to the government, we recognize that the risk of nonconforming spare parts reaching end users exists. Compliance by contractors, DCMA, and other DOD agencies with established internal controls helps mitigate against this risk. Agency Comments and Our Evaluation: In written comments on a draft of this report, DOD provided one technical comment, which we incorporated as appropriate. DOD did not provide any additional comments. DOD's written comments are reprinted in their entirety in appendix III. We are sending copies of this report to the Secretary of Defense; the Secretaries of the Army, the Navy, and the Air Force; the Director of the Defense Contract Management Agency; the Director of the Defense Logistics Agency; and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov. Please contact me at (202) 512-8365 if you or your staffs have any questions concerning this report. Major contributors to this report are included in appendix IV. Signed by: William M. Solis, Director: Defense Capabilities and Management: [End of section] Appendix I: Scope and Methodology: To address our objectives, we judgmentally selected for review 11 contracts awarded to 11 prime contractors. During the course of our review, Defense Contract Management Agency (DCMA) and the prime contractors provided four additional contracts awarded to some of these same contractors that had nonconforming parts that reached end users, bringing the total number of contracts we reviewed to 15. Given the small number of contracts we reviewed, our results cannot be used to make inferences about the entire population of contracts requiring government quality assurance. We included the following kinds of contracts as candidates for our study: contracts, purchase orders, basic ordering agreements, delivery orders against existing contracts, and contract modifications for existing contracts. We included contracts that were large and small dollar value, from DCMA's East and West regions, and from the Army, Navy, Air Force, and the Defense Logistics Agency (DLA). Eleven of these contracts were judgmentally selected: three from information provided by the Navy and Air Force on nonconforming parts and 8 from DCMA. To select the eight contracts, we obtained a query from DCMA's Mechanization of Contract Administration Services system of contracts for October 1, 1999, through September 30, 2003. Because of the large number of contracts in the database and because we wished to examine recent contract quality assurance oversight practices, we sorted the query to only identify contracts for fiscal year 2003. From this query, we sorted the contracts into four groups according to whether they were associated with the Army, the Navy, the Air Force, or the Defense Logistics Agency. We randomly sampled contracts from each of these four groups. Then, we judgmentally selected a subset of contracts from our random samples in such a way to obtain a set of eight contracts that spanned the various military commands and DCMA's East and West regions. The breakout of the eight contracts included three Army, two Air Force, two Navy, and one Defense Logistics Agency. To assess the reliability of data from DCMA's Mechanization of Contract Administration Services system we (1) performed electronic testing of required data elements, (2) reviewed existing information about the data and the system, and (3) interviewed agency officials knowledgeable about the data. We determined DCMA's Mechanization of Contract Administration Services system data to be reliable for the purposes of our review. To assess whether DCMA provided quality assurance oversight and enforcement over its spare parts prime contractors in accordance with established policies, procedures, and guidance, we compared contract quality assurance provisions and requirements to oversight actions performed by DCMA for the 15 contracts. Specifically, we reviewed the Federal Acquisition Regulation, the DCMA One Book, and the contracts to determine quality assurance oversight responsibilities and enforcement actions available to assure contractor compliance. We compared these policies, procedures, and contract requirements to the three types of inspections performed by DCMA quality assurance specialists to assess if DCMA provided appropriate oversight. For each of the contracts, we met with officials at the DCMA contract management offices identified on the contracts to determine quality assurance oversight actions performed by DCMA personnel. As part of this assessment, we determined which of the three types of inspections were performed for each contractor. We sent letters to officials at the DOD contracting offices to obtain and review documentation related to the pre-award process, quality assurance requirements, and the contracting officers' interaction with DCMA prior to contract award. In assessing whether DCMA used enforcement actions, we reviewed the product quality deficiency reports included in our review to determine if DCMA levied enforcement actions against the prime contractor when necessary. We also reviewed prior DOD and GAO reports related to DCMA's execution of its quality assurance oversight over prime contractors and DOD's implementation of its deficiency reporting system. To assess whether prime contractors provided quality assurance oversight over their subcontractors' work related to producing spare parts and followed industry standards and contract requirements, we identified prime contractor quality assurance oversight actions performed over subcontractors. We reviewed Aerospace Standard 9100 and prime contractor quality manuals to determine requirements for establishing contractor quality management systems and ensuring that their subcontractors are providing quality parts.[Footnote 9] We visited and interviewed representatives at the prime contractor locations as shown in appendix II, and determined whether the prime contractors used subcontractors. For those that used subcontractors, we identified the level of quality assurance oversight performed by these prime contractors over the subcontractors. We discussed whether the prime contractors performed supplier ratings of their subcontractors, tested parts or processes provided by their subcontractors, or conducted site visits at their subcontractors' facilities. At the prime contractor facilities, we observed the prime contractors' processes for manufacturing or repairing parts to determine the quality assurance performed by the prime contractor throughout the production process. To assess how DCMA held prime contractors accountable for the work of their subcontractors, we reviewed the 15 contracts to determine if the contracts included clauses holding the prime contractor responsible for the spare parts provided to the government and whether instances of nonconformance had occurred. For the reported instances of nonconformance, we looked at whether DCMA held the prime contractor responsible for correcting the nonconformance and what types of actions were performed by DCMA. We also reviewed the Federal Acquisition Regulation to determine contract clauses that require the prime contractor to ensure that parts furnished to the government conform to contract requirements. We reviewed the contracts to determine if they included clauses that the prime contractor was responsible for the spare parts being provided. We also visited or obtained information from representatives at the following organizations: * U.S. Air Force, Office of the Assistant Secretary, Contracting Operations Division, Rosslyn, Va; * U.S. Air Force Materiel Command, Wright Patterson Air Force Base, Ohio; * U.S. Army Materiel Command Headquarters, Fort Belvoir, Va; * U.S. Army, Aviation and Missile Command, Redstone Arsenal, Redstone, Ala; * U.S. Army, Communications Electronics Command, Fort Monmouth, NJ; * U.S. Army, Research Development and Engineering Command, Armament Research, Development and Engineering Center, Rock Island, Ill; * U.S. Army, Research Development and Engineering Command, Edgewood Chemical Biological Center, Rock Island, Ill; * U.S. Army, Secretary of the Army for Acquisition, Logistics, and Technology, Arlington, Va; * U.S. Army, Tank Automotive and Armaments Command, Warren, Mich; * U.S. Navy, Naval Air Systems Command, Patuxent River Naval Air Station, Patuxent River, Md; * U.S. Navy, Office of the Assistant Secretary, Research, Development, and Acquisition, Washington, D.C; * U.S. Defense Logistics Agency Headquarters, Fort Belvoir, Va; * U.S. Defense Contract Management Agency Headquarters, Alexandria, Va; * Office of the Under Secretary of Defense for Acquisition, Technology, and Logistics, Alexandria, Va; and: * Aerospace Industries Association, Arlington, Va. We performed our review from November 2003 through October 2004 in accordance with generally accepted government auditing standards. [End of section] Appendix II: DCMA Offices and Contractors Reviewed: 1; DCMA Contract Management Offices: Boston; DCMA District: East; Prime contractor: Contractor K; Number of contracts: 1. 2; DCMA Contract Management Offices: Dayton; DCMA District: East; Prime contractor: Contractor H; Number of contracts: 2. 3; DCMA Contract Management Offices: Long Island; DCMA District: East; Prime contractor: Contractor G; Number of contracts: 1. 4; DCMA Contract Management Offices: Maryland; DCMA District: East; Prime contractor: Contractor C; Number of contracts: 2. 5; DCMA Contract Management Offices: Philadelphia; DCMA District: East; Prime contractor: Contractor B; Number of contracts: 3. 6; DCMA Contract Management Offices: Philadelphia; DCMA District: East; Prime contractor: Contractor F; Number of contracts: 1. 7; DCMA Contract Management Offices: South Florida; DCMA District: East; Prime contractor: Contractor J; Number of contracts: 1. Total East Region contracts: 11. 8; DCMA Contract Management Offices: Long Beach; DCMA District: West; Prime contractor: Contractor E; Number of contracts: 1. 9; DCMA Contract Management Offices: Los Angeles; DCMA District: West; Prime contractor: Contractor D; Number of contracts: 1. 10; DCMA Contract Management Offices: Phoenix; DCMA District: West; Prime contractor: Contractor A; Number of contracts: 1. 11; DCMA Contract Management Offices: Santa Ana-Irvine; DCMA District: West; Prime contractor: Contractor I; Number of contracts: 1. Total West Region contracts: 4. Source: GAO analysis. [End of table] [End of section] Appendix III: Comments from the Department of Defense: OFFICE OF THE UNDER SECRETARY OF DEFENSE: ACQUISITION TECHNOLOGY AND LOGISTICS: 3000 DEFENSE PENTAGON: WASHINGTON, DC 20301-3000: DEC 06 2004: Mr. William M. Solis: Director, Defense Capabilities and Management: U.S. Government Accountability Office: Washington, D.C. 20548-0001: Dear Mr. Solis: This is the Department of Defense (DoD) response to the GAO draft report, GAO-05-73, "DEFENSE INVENTORY: DoD and Prime Contractors Adhered to Requirements in Selected Contracts for Overseeing Spare Parts Quality," dated November 10, 2004 (Code 350460). The draft report did not contain any recommendations for the DoD. Page 7 of the report discussed the DoD Inspector General (DoDIG) audit regarding source inspection performed in 2003 and states that the DoDIG "pointed out weaknesses in the management of DCMA's inspection program." The DoDIG report highlighted concerns with DoD at the program office, procurement activity, and quality assurance activity levels and did not isolate weaknesses with DCMA activities. Therefore, the DoD recommends that the text "...pointed out weaknesses in the management of DCMA's inspection program" be changed to "...pointed out concerns in the DoD quality assurance program." My point of contact is Lt Col Nannette Benitez who can be reached at (703) 695-8567 or via e-mail at nannette.benitez@osd.mil. Signed by: Deidre A. Lee: Director, Defense Procurement and Acquisition Policy: [End of section] Appendix IV: GAO Contact and Staff Acknowledgments: GAO Contact: David A. Schmitt (757) 552-8124: Acknowledgments: In addition to the individual named above, Connie W. Sawyer, Jr; Tracy Whitaker; Leslie West; Renee McElveen; Minette Richardson; Kenneth Patton; Sidney Schwartz; and Douglas Cole made key contributions to this report. FOOTNOTES [1] A part number is a combination of numbers, letters, and symbols assigned by a designer and used by a manufacturer or vendor to identify a specific part or item of material. [2] Contract actions include issuing new contracts, purchase orders, or basic ordering agreements; modifying existing contracts; and issuing orders for parts included in existing contracts. In this report, we refer to all of these actions as contracts. [3] The act (Pub. L. No. 108-87, § 8143 (2003)) also mandated that the Secretary of Defense report by March 31, 2004, on (1) how to implement a system for tracking safety-critical parts so that parts discovered to be defective can be identified and found; (2) appropriate standards and procedures to ensure timely notification of contracting agencies and contractors about safety issues including parts that may be defective, and whether the Government Industry Data Exchange Program should be made mandatory; (3) efforts to find and test airplane parts that have been heat-treated by companies alleged to have done so improperly; and (4) whether contracting agencies and contractors have been notified about alleged improper heat treatment of airplane parts. The Secretary's report was submitted on June 30, 2004, and included numerous actions DOD had taken or plans to take to address the issues included in the mandate. [4] Details about our selection of the contracts are included in the scope and methodology section of this report. [5] DCMA has issued broad guidance, referred to as the One Book, to assist quality assurance specialists in executing their oversight responsibilities. [6] ACQUISITION: Government Source Inspections (D-2004-011) October 15, 2003. [7] A waiver is a deviation from contract requirements and can be considered major or minor. [8] Third-party certification is performed by a registered external auditing organization and provides verification and assurance that products or processes are consistent with established standards. [9] Aerospace Standard 9100 includes the International Organization for Standardization 9001 quality assurance requirements. GAO's Mission: The Government Accountability Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to e-mail alerts" under the "Order GAO Products" heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. Government Accountability Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: