U. S. DEPARTMENT OF ENERGY OFFICE OF INSPECTOR GENERAL AUDIT OF ARCHITECT AND ENGINEERING COSTS AT THE IDAHO NATIONAL ENGINEERING LABORATORY The Office of Inspector General wants to make the distribution of its reports as customer friendly and cost effective as possible. Therefore, this report will be available electronically through the Internet five to seven days after publication at the following alternative addresses: Department of Energy Headquarters Gopher gopher.hr.doe.gov Department of Energy Headquarters Anonymous FTP vm1.hqadmin.doe.gov Department of Energy Human Resources and Administration Home Page http://www.hr.doe.gov/refshelf.html Your comments would be appreciated and can be provided on the Customer Response Form attached to the report. This report can be obtained from the U.S. Department of Energy Office of Scientific and Technical Information P.O. Box 62 Oak Ridge, Tennessee 37831 Report Number: DOE/IG-0387 Western Regional Audit Office Date of Issue: March 1996 Albuquerque, NM 87185-5400 March 22. 1006 IG-1 INFORMATION: Report on "Audit of Architect and Engineering Costs at the Idaho National Engineering Laboratory" TO: The Secretary BACKGROUND: In September 1990, the Office of Inspector General issued the Departmentwide Audit of Architect and Engineering Design Costs (DOE/IG-0289) which concluded that the Departmentms architect and engineering (A/E) costs averaged more than twice that of private industry. The primary cause of the higher costs was the lack of Departmental A/E cost standards that would provide measurement criteria for controlling costs. Consistent with our prior Departmentwide audit, the purpose of this audit was to determine whether A/E services performed at the Idaho National Engineering Laboratory were economical. Specifically, we determined whether the costs for A/E services at the Laboratory were comparable to the cost standards for A/E services in industry and the State of Idaho. DISCUSSION: The Idaho Operations Office has begun to develop performance expectations and performance measures in order to make the management and operating contract more performance based. The audit disclosed, however, that additional opportunities exist to improve management control over the costs of A/E services. We found that A/E costs for 65 conventional construction projects were $5.8 million higher than comparable industry standards. Therefore, we recommended that the Manager, Idaho Operations Office take aggressive action to control the excessive cost of A/E services that were previously identified in the 1990 Departmentwide audit report and has continued at least through February 2, 1996. Specifically, actions to control the excessive cost of A/E services should include the establishment of expectations and performance measurements for Lockheed to ensure that: * Laboratory costs for A/E services are more in line with industry cost standards; * A/E services are awarded competitively based on technical competence and price; and, * A/E services are similar in quality and detail to industry standards. The Idaho Operations Office agreed with our recommendation. (Signed) John C. Layton Inspector General Attachment cc: Deputy Secretary Acting Under Secretary AUDIT OF ARCHITECT AND ENGINEERING COSTS AT THE IDAHO NATIONAL ENGINEERING LABORATORY TABLE OF CONTENTS Page SUMMARY .................................................. 1 PART I - APPROACH AND OVERVIEW ......................... 2 Introduction .................................. 2 Scope and Methodology .......................... 2 Background .................................... 3 Observations and Conclusions .................. 4 PART II - FINDING AND RECOMMENDATION ................... 6 A/E Costs at the Laboratory ................... 6 PART III - MANAGEMENT AND AUDITOR COMMENTS ............... 13 PART IV - APPENDIX ...................................... 18 U.S. DEPARTMENT OF ENERGY OFFICE OF INSPECTOR GENERAL OFFICE OF AUDIT SERVICES AUDIT OF ARCHITECT AND ENGINEERING COSTS AT THE IDAHO NATIONAL ENGINEERING LABORATORY Audit Report Number: DOE/IG-0387 SUMMARY The National Performance Review (NPR) report, Making Government Work Better and Cost Less, (September 1993) recommends performance measurement as a tool to help improve Government operations. One performance measurement that the NPR encourages is benchmarking an agencyms performance against standards used by private industry and other Government agencies. The objective of this audit was to determine whether architect and engineering (A/E) services performed at the Idaho National Engineering Laboratory (Laboratory) were economical when compared to cost standards for A/E services in industry and the State of Idaho (State). Our analysis of the Laboratoryms costs for A/E services found that A/E costs for 65 conventional construction projects were, in the aggregate, $5.8 million higher than comparable industry standards. This occurred because of inadequate internal controls over A/E design costs. Specifically, the Laboratory did not have a way to measure the performance of its design programs; the Idaho Operations Officems (Operations Office) policy for the selection of A/E services precluded price competition; and, design services for conventional construction at the Laboratory were in more detail than necessary. We recommended that the Manager, Idaho Operations Office, establish expectations and performance measurements for Lockheed Idaho Technologies Company (Lockheed) to ensure that Laboratory costs for A/E services are more in line with industry cost standards; individual A/E services are awarded competitively based on technical competence and price; and, A/E services are similar in quality and detail to industry standards. We estimate that the Operations Office could save as much as $2.5 million for A/E services on 19 future planned conventional construction projects by implementing our recommendation. Management generally concurred with the finding and recommendation presented in the report and has already initiated corrective actions in response to the recommendation. _______(Signed)____________ OFFICE OF INSPECTOR GENERAL PART I APPROACH AND OVERVIEW INTRODUCTION In September 1990 the Office of Inspector General (OIG) issued the Departmentwide Audit of Architect and Engineering Design Costs (DOE/IG-0289) which concluded that the Departmentms A/E costs averaged more than twice that of private industry. The primary cause of the higher costs was the lack of Departmental A/E cost standards that would provide measurement criteria for controlling costs. Consistent with our prior Departmentwide audit, the purpose of this audit was to determine whether A/E services performed at the Laboratory were economical. Specifically, we determined whether the costs for A/E services at the Laboratory were comparable to the cost standards for A/E services in industry and the State; and, whether A/E costs were reasonable. SCOPE AND METHODOLOGY The audit was conducted at the Operations Office and the Laboratory from June 15, 1995 through October 20, 1995. To accomplish the audit objectives, we interviewed key personnel and reviewed: * Federal and Departmental regulations as well as Lockheed policies and procedures for A/E services; * prior A/E audit reports issued by the OIG; * accounting records and other documentation of A/E costs and construction costs for conventional construction projects on the books in Fiscal Year 1995; * publications of A/E cost estimates that are widely used in industry; * State of Idaho construction projects; and, * Operations Office plans for construction of conventional buildings in the future. The scope of the audit was limited to A/E services acquired for conventional construction projects on Lockheedms accounting records during Fiscal Year 1995. These projects included those that were recently completed, are currently underway, or are soon to begin. Conventional construction projects, according to the Department of Energyms (Department) cost guidance, include warehouses, laboratories, office buildings, non-process related utilities, sewage, and water treatment facilities. Conventional construction does not mean the projects were necessarily simple, non-sophisticated, or standard, but that from a design point of view, prior industry experience exists (DOE COST GUIDE, Volume 6, November 1994, Chapter 25, "Guidelines for Engineering, Design, & Inspection Costs"). During Fiscal Year 1995, there were 65 conventional construction projects on Lockheed's accounting records that cost about $88.1 million to construct and $13 million for A/E services. We reviewed all of these projects in our audit. We compared A/E costs for the 65 Laboratory projects to an industry benchmark taken from two publications that are used extensively in the construction industry for estimating costs (see Appendix). These publications are: * Mean's Building Construction Cost Data, 1995 annual edition, published by RS Means & Company; and, * Marshall Valuation Service, January 1995 edition, published by Marshall and Swift Company. In addition, we compared A/E costs for 20 of the 65 Laboratory projects to a benchmark for the State that was taken from cost data of 7 recently completed State of Idaho construction projects (see Appendix). The audit was performed according to generally accepted Government auditing standards for performance audits and included tests of internal controls and compliance with laws and regulations to the extent necessary to satisfy audit objectives. Accordingly, the audit included an assessment of significant internal controls with respect to A/E services including the Operations Officems policies for the selection and authorization of A/E services. We relied on Lockheedms internal project listing and accounting system to provide the universe of conventional construction projects and performed limited tests of the accounting system to ensure the reliability of computer processed data. Because our audit was limited, it would not necessarily have disclosed all internal control deficiencies that may have existed at the time of our audit. An exit conference was held on January 25, 1996. BACKGROUND Prior to Fiscal Year 1995, the Laboratory was managed by five contractors. At the beginning of Fiscal Year 1995, the five contracts were consolidated into one contract which was awarded to Lockheed. Two of the former contractors, Westinghouse Idaho Nuclear Company and EG&G Idaho, Inc., (EG&G) had subcontracts with several external A/E firms for design services. However, the majority of design work was performed in-house by EG&Gms resident A/E organization. When Lockheed took over management of the Laboratory it assumed the subcontracts with the external A/E firms and the internal resident A/E organization. In addition, the Operations Office had a prime contract with another firm which was referred to as the Operations Officems miscellaneous A/E firm. This A/E firm generally performed the complicated, non- conventional design work, but sometimes was used to provide conventional design services as well. Lockheed manages approximately 580 Department owned buildings at the Laboratory. The ages of these buildings range from less than one year to more than 40 years. The Operations Office has embarked on an extensive construction program to replace, upgrade or modify buildings that are old, deteriorated or obsolete to satisfy current construction codes or new mission needs. Before construction begins, however, A/E services must be procured to produce the designs for the buildings. A/E services, as defined by the Federal Acquisition Regulations, are professional services associated with the design of real property. A/E services include surveying, consultation, plans and specifications, value engineering, design review, and other related services. OBSERVATIONS AND CONCLUSIONS Department Order 430.1, Life Cycle Asset Management, (August 24, 1995) requires all operations offices to develop site- specific performance expectations and performance measures for design services. The Idaho Operations Office is currently preparing these expectations and performance measurements and expects to complete these around May 1996. We commend Idaho for recognizing that performance measurement systems improve performance and reduce costs. In addition, other positive practices were noted during our audit. For instance, Lockheed has a cost control system in place that is intended to make internally developed A/E plans and specifications much like a fixed price contract. Specifically, the construction project manager and the leader of the A/E design team agree to a fixed price "task baseline agreement" that includes a scope of work, deliverables, and schedule for a design package. This appears to be an innovative practice that may have the potential to keep design costs down. In fact, for 22 percent of the conventional construction projects included in our review, the Laboratoryms design costs were less than industry estimates. However, opportunities exist to improve management control over the costs of A/E services. We found that the Laboratoryms costs for obtaining A/E services for 65 conventional construction projects were approximately $5.8 million higher than comparable industry standards and about $1.6 million more than the State benchmark for 20 (of the 65) comparable construction design projects. Laboratory costs were higher because of inadequate internal controls over A/E design costs. Specifically, the Laboratory did not have a way to measure the performance of its design programs; Idahoms policy for the selection of A/E services precluded price competition; and, design services for conventional construction at the Laboratory were in more detail than necessary. We estimate that the Laboratory could save as much as $2.5 million on 19 future planned conventional construction projects by implementing our recommendation. In our opinion, the finding in this report disclosed material internal control weaknesses that the Department should consider when preparing its yearend assurance memorandum on internal controls. PART II FINDING AND RECOMMENDATION A/E Costs at the Laboratory FINDING The NPR report recommends benchmarking performance to industry standards as a tool to help improve Government operations. Our analysis of the Laboratoryms costs for A/E services showed that A/E costs for 65 conventional construction projects were $5.8 million higher than comparable industry standards. Laboratory costs were higher because of inadequate internal controls over A/E design costs. Specifically, the Laboratory did not have a way to measure the performance of its design programs; Idahoms policy for the selection of A/E services precluded price competition; and design services for conventional construction at the Laboratory were in more detail than necessary. As a result, the Laboratory could save approximately $2.5 million for design services on 19 future conventional construction projects by implementing our recommendation. RECOMMENDATION We recommend that the Manager, Idaho Operations Office, together with Lockheed, take aggressive action to control the excessive cost of A/E services that were previously identified in a 1990 Departmentwide audit report and have continued at least through February 2, 1996. Specifically, actions to control the excessive cost of A/E services should include the establishment of expectations and performance measurements for Lockheed to ensure that: * Laboratory costs for A/E services are more in line with industry cost standards; * A/E services are awarded competitively based on technical competence and price; and, * A/E services are similar in quality and detail to industry standards. MANAGEMENT REACTION Management generally concurred with the finding and recommendation and initiated corrective action. Detailed management and auditor comments are provided in Part III of this report. DETAILS OF FINDING The NPR report recommends performance measurement as a tool to help improve Government operations. In fact, the report stated that "...if it doesnmt get measured it doesnmt get improved." One performance measurement that the NPR encourages is benchmarking an agencyms performance against standards used by private industry and other Government agencies. When benchmarking, the NPR pointed out that it is imperative that the agency create a level playing field by fully accounting for all costs so that the services can be compared in as fair a manner as possible. For these reasons we compared the Laboratoryms A/E costs against industry standards and State benchmarks rather than the more stringent Federal 6-percent rule observed by other Federal agencies. Department Order 430.1, Life Cycle Asset Management, requires that the planning, design, construction, and management of physical assets incorporate industry standards and performance objectives. This Order shall be implemented on a site-by-site basis through the establishment of site-specific performance expectations and performance measurements. The Operations Office is preparing these expectations and performance measurements and expects to complete them during the March through May 1996 timeframe. In our audit we used two benchmarks for performance measurement of A/E costs: an industry benchmark, and one for the State. The industry benchmark for 65 conventional construction projects at the Laboratory averaged 8.17 percent of construction costs, according to the industry publications. The State benchmark for design costs averaged 8.15 percent of construction costs. The Brooks Architect-Engineers Act of 1949 states that all contracts for A/E services should be competed based on technical competence alone. The Brooks Act has been cited by the Operations Office to back up the position that price should not be a selection criteria for A/E services. However, on June 27, 1988, the language in the Department of Energy Acquisition Regulation (DEAR) was changed to include that while adhering to the principle of selection based upon qualifications for A/E contracts let directly by the Government, "...this does not prelude the consideration of other factors, including cost or price..." for A/E contracts awarded by management and operating contractors (M&O). COMPARISON OF DESIGN COSTS TO INDUSTRY AND STATE STANDARDS Design costs for conventional construction projects at the Laboratory were significantly higher than comparable industry standards and State benchmarks. We compared actual A/E costs for 65 conventional construction projects worth about $88.1 million to A/E cost estimates found in industry publications. In addition, we also compared A/E costs for 20 of these buildings worth about $32 million to similar projects constructed by the State. Comparison to Industry Design costs for 65 conventional construction projects worth about $88.1 million were, in the aggregate, about $5.8 million higher than comparable industry standards. The actual cost of A/E services for these projects was about $13 million on projects that the industry standards estimate should have cost about $7.2 million. However, although the majority of projects had A/E costs that were higher than industry standards, some Laboratory A/E costs were lower than industry standards as illustrated in the following chart. COMPARISON OF LABORATORY A/E COSTS TO INDUSTRY ($ millions) Laboratory A/E Costs Relative to No. of Constr. Laboratory Industry Total Ind. Standards Projects Cost A/E A/E Differences Less than Industry 14 $19.1 $1.1 $1.5 $(.4) 1 to 2 Times More 30 47.5 5.3 3.9 1.4 2 to 3 Times More 9 8.6 1.9 .7 1.2 3 to 4 Times More 8 8.8 2.9 .8 2.1 More Than 4 Times 4 4.1 1.8 .3 1.5 Totals 65 $88.1 $13.0 $7.2 $5.8 Percent of Construction 14.7% 8.17% As shown above, 12 of 65 (or 18 percent) of these projects have more than three times the design costs when compared against industry standards. However, many of the 30 projects that were only one to two times industry standards have significant dollar differences over industry standards. For example: * A new medical facility cost $390,000 to design which was $75,000 more than the industry estimate of $315,000. * A new emergency response facility cost $445,000 to design which was $167,000 more than the industry estimate of $278,000. * A new transportation complex cost $798,000 to design which was $323,000 more than the industry estimate of $475,000. The industry benchmark has been criticized by Operations Office personnel for not taking into consideration the local economic peculiarities of eastern Idaho, where A/E firms may be more difficult to find than in a larger metropolis. In addition, management argued that Government is fundamentally different than private industry and, therefore, cannot be fairly compared to one another. To satisfy these concerns, we also compared the A/E costs of the Laboratory to construction projects that were locally designed and constructed by the State of Idaho. Comparison to the State of Idaho The Laboratory spent approximately $1.6 million more than the State benchmark for 20 conventional construction design services on comparable projects included in our audit. The Statems design costs were, on an average, 8.15 percent of construction costs compared to the Laboratoryms average of 13.2 percent for office buildings, laboratory facilities, communications centers, and other common use spaces. The Laboratoryms A/E costs were about $4.2 million, which was $1.6 million more than the State benchmark of $2.6 million for comparable construction projects. INTERNAL CONTROLS OVER DESIGN COSTS Design costs at the Laboratory were higher than industry and State standards because the Operations Office did not have adequate internal controls over A/E design costs. Specifically, the Operations Office did not measure the performance of its design programs; the Operations Office policy for the selection of A/E services precluded price competition; and designs for conventional construction projects were more detailed than necessary. Performance Measurement of A/E Costs The Laboratory did not compare its A/E costs against industry cost standards to measure its performance. This kind of performance measurement could have provided management with information to identify and correct problems that resulted in higher design costs. As the NPR report indicated, measurement is a requirement for improvement. In addition, even though Headquarters initiated a cost reduction plan that attempted to measure and track design costs, the Operations Office did not fully implement the Headquarters initiated Improvement Plan for Reducing Architect-Engineer Costs, dated August 1993. This plan was developed in response to an Office of Inspector General audit report on A/E costs in 1990. The improvement plan was designed to: 1) improve A/E cost estimates and project cost management; 2) identify incompetent contractors and inefficient practices; and, 3) provide early recognition and resolution of sitewide problems and adverse trends. To implement this plan, the Manager, Idaho Operations Office was to designate an individual with approval authority for A/E services and create a database to compile, analyze, and report on A/E costs at the Laboratory. The role of the approval authority included reviewing A/E cost proposals to ensure that A/E costs were kept to the lowest amount possible and providing quarterly reports on A/E costs to Headquarters. However, the Operations Office did not designate an approval authority for A/E costs; create a database to report on A/E costs at the Laboratory; or provide quarterly reports on A/E costs to Headquarters. In fact, management informed us that implementing the improvement plan was largely forgotten due to reorganizations and changing priorities. Competition for A/E Services The Operations Officems policy for the selection of A/E services precluded competition by giving Lockheedms resident A/E organization a virtual monopoly over design services. Rather than routinely competing A/E design work, Lockheed scheduled design work to keep its resident A/E organization fully employed. Only design work above the maximum capacity of the resident A/E organization was awarded to external A/E firms. This policy gave the resident A/E organization the ability to control its level of "sales" of A/E services to the Laboratory. As a result, the resident A/E was not subject to market forces including demonstrating that it was the most economical by price competition, or by demonstrating that it was the best qualified. In the framework of the NPR report, this may be considered a service monopoly. The NPR report section entitled "Making Service Organizations Compete" suggests that agencies should not provide services in-house unless the services can compete with private companies. Thus, the Operations Officems policy for A/E services is not in the spirit of the NPR nor does it provide for obtaining A/E services at the lowest cost. One example of how the Operations Officems policy for selection of A/E services precluded competition, when it was readily available, was with a new Laboratory transportation complex. During discussions as to who would design the transportation complex, the project file documentation indicated that the external A/E contractor could do the design more efficiently than Lockheedms resident A/E. The documentation demonstrated that the external A/E had recently designed three major transportation complexes, and would be able to assign personnel who designed these projects to the Laboratory project. However, the resident A/E was given the design project because of the Operations Officems policy for selecting A/E services even though the resident A/E had not designed a transportation complex in more than 30 years. In addition, when A/E work was forwarded to an external firm there was little incentive for the external A/E firm to accept lower prices for its A/E services because there was no price competition for individual design services. All of the external A/E contracts were task order contracts. A task order contract is awarded to an A/E firm for three to five years. Prices are negotiated for each task placed for A/E services. However, the Laboratoryms ability to negotiate a favorable arrangement is reduced by the fact that the A/E firm knows that it has already won the contract. Thus, the A/E has no incentive to make lower offers. On the contrary, the A/E has incentive to get as much as the Department is willing to pay. Clearly, an environment where each task is subject to price competition between several qualified firms would be in the Governmentms best interest. Amount of Detail of Design Designs for conventional construction projects were in more detail than necessary. Specifically, Lockheed personnel stated that A/E designs for conventional construction projects were in such detail that many were almost to the level of design required for a nuclear related project. We were not professionally qualified to verify whether this was accurate. However, we concluded that providing more detail in A/E designs than necessary for conventional construction would partially explain the higher A/E design costs at the Laboratory. The inclination for more detailed designs was exemplified by the fact that there were two "design reviews" performed at the Laboratory. The first design review was performed internally by the A/E before the plans, drawings, and specifications were released. This review was performed by individuals independent of the design team to ensure the quality and accuracy of the design plans. Afterward, when project management received the plans, another design review team was assembled to examine the plans, drawings, and specifications again. Double checking design plans may be appropriate for nuclear related construction projects due to the inherently more complicated design issues. However, double-checking the designs of conventional construction projects typically is not performed in industry. In fact, an industry customer generally relies on the A/Ems quality assurance. If design problems are discovered by the general contractor during construction, then the A/E fixes the problem. According to the Operations Office, more detailed design eliminates many ambiguities in the drawings that might result in change orders by the general contractor. Management's position is that spending more on design makes the total project less expensive by reducing the number of change orders during construction. The Operations Office did not have quantitative support for this position and we were unable to verify it. However, we concluded that the State and industry have at least as much incentive to reduce total construction costs as the Department. POTENTIAL SAVINGS ON DESIGN SERVICES We concluded that the Laboratory has an opportunity to save approximately $2.5 million on design services in the future. The Laboratory has 19 conventional construction projects worth an estimated $50.6 million planned for Fiscal Years 1996 through 1999. We estimate that in the future, if the Laboratory continues the same practices that led to the higher A/E costs than industry standards, it could spend $3.3 million more than industry estimates on these 19 future projects. However, due to uncertainties in our estimate and the Laboratoryms inability to reduce its A/E costs to industry standards immediately, we project that 75 percent of this, or approximately $2.5 million, can be saved if the Operations Office implements changes to reduce the Governmentms design costs. The Operations Office expressed concern that our estimate of savings may never be realized because budget constraints may reduce or change the scope, cost, schedule, funding year, or even the final determination of need for the proposed construction projects. However, given that neither the Operations Office nor the OIG can predict how budget limitations will affect the construction program, we believe that it is reasonable to base our estimate on those projects that are currently planned. While our estimate of potential cost savings is limited to conventional construction projects (because these were readily comparable to industry standards), it should also be noted that nonconventional projects may also benefit from similar cost standards and controls. In fact, nonconventional projects probably have an even greater potential for cost reduction since the A/E costs on these projects are higher. In addition to the potential cost savings, we believe that the use of cost standards and increased awareness of design costs could significantly decrease the risk for waste. Specifically, the lack of cost standards could lead to abuse when design costs are allowed to run, as in some projects, as much as five times the going rate. PART III MANAGEMENT AND AUDITOR COMMENTS The Manager, Idaho Operations Office generally concurred with the finding and recommendation. Some of managementms comments are included in Part II. Managementms response to the recommendation, a summary of additional management comments, and auditor responses are provided below. Recommendation. We recommend that the Manager, Idaho Operations Office, together with Lockheed, take aggressive action to control excessive A/E costs that were previously identified in a 1990 Departmentwide audit report and have continued at least through February 2, 1996. Specifically, this should include the establishment of expectations and performance measurements for Lockheed to ensure that: * Laboratory costs for A/E services are more in line with industry cost standards; * A/E services are awarded competitively based on technical competence and price; and, * A/E services are similar in quality and detail to industry standards. Management Comments. Management agreed with implementing expectations and performance measures. Management stated that this recommendation is in the process of being accomplished as part of implementing the new Department Life Cycle Asset Management (LCAM) Order. Presently, the LCAM is in draft and is scheduled to be implemented within the March to May 1996 timeframe. Auditor Comments. Managementms comments and actions are responsive to the recommendation. Summary of Additional Management Comments. Even though management agreed to implement the recommendation, and generally concurred with the finding, they had some concerns about the comparison. Each of management's concerns are discussed below as well as the auditor responses. Management Comments. Management stated that Federal requirements such as (a) the Federal budget process, (b) Federal and Departmental Acquisition Regulations, and (c) Energy Conservation Reports and independent energy reviews drive the Laboratoryms A/E costs higher than the State or private industry. Auditor Comments. Although Federal requirements add some cost to the procurement of A/E services that industry may not incur, this only partially explains why costs were so much higher. The three main reasons for higher A/E costs at the Laboratory were: a lack of a performance measurement system; little or no price competition on each A/E task; and, more detailed design than is generally performed in industry. By implementing our recommendation, the Operations Office would be taking action to lower A/E costs. Management Comments. Management stated that the primary difficulty of comparing industry and the State of Idaho costs to the Laboratoryms is developing a valid "apples to apples" comparison. Management believed that there were a number of factors that cannot be readily separated which makes it difficult to normalize and compare the data. Some of these factors include: a) The Laboratoryms accounting system may not include the same cost data as the State of Idaho or industry, even when the description of the data is similar. For example, design review cost data is not captured the same way by private industryms and the Laboratoryms accounting systems. b) Studies by Independent Project Assessment (IPA) for the Departmentms Office of Environmental Management have shown a major difference between industry and the Department in the amount of up-front planning costs that are not accounted for as design costs. c) Many conventional projects were constructed within nuclear areas which adds additional design considerations. Also, the industry standard adjustment for modifications to an existing facility may not be appropriate for these facilities. In addition, design costs are accrued for connecting buildings to external and underground utilities at the Laboratory because the Department owns the utilities, whereas in private industry the public utility may do the design and construction of external utilities and recover costs through rate charges or connection fees. Auditor Comments. Government Auditing Standards require that we exercise due professional care in establishing the scope, selecting the methodology, and choosing tests and procedures for an audit. To comply with this requirement and to eliminate the perception of an "apples-to-oranges comparison," all material adjustments that the auditor and management could think of were made to make the comparison as accurate as practicable. Where uncertainties existed as to the appropriateness of including data or costs in the comparison, we chose to eliminate or mitigate such data or costs. The comparison is described in more detail in the Appendix. In addition, we agree that conventional construction in nuclear areas require more design effort than in non-nuclear areas. However, construction and modifications to buildings in nuclear areas also require more construction costs. Thus, the percentage of design over construction would not be materially affected. The same is true for the design costs to connect site facilities to electrical, water and sewage utilities. That is, the Laboratory also pays for the construction of these connections, therefore, after agreement by Lockheed personnel, we concluded that the A/E costs as a percentage of construction would not be materially affected. Management Comments. Management stated that the unique nature of fire and life safety improvements make it difficult to properly compare to industry standards. However, this data was still used to support the conclusions of high design costs. Of the projects analyzed, 14 (22 percent) had A/E costs below industry standards. Of the 51 projects that exceeded industry standards, 14 involved fire and life safety project designs. If this class of projects were excluded from the audit due to the difficulty in quantifying, there would have been smaller differences between industry and Laboratory costs. Auditor Comments. Fire and life safety improvements are modifications to existing facilities to upgrade fire alarms, announcement/intercom systems, ceiling sprinklers and water supply, exits, and barriers. These improvements were accounted for differently than other modifications because rather than accumulating design and construction costs on a building by building basis, costs were accumulated for a number of buildings in a geographic area. Because design and construction costs for fire and life safety improvements were accumulated differently than design costs for other modifications, we proposed eliminating them from our comparison. However, at the request of Lockheed management we included the fire and life safety improvement projects in the comparison. Notwithstanding the difference in accounting for fire and life safety improvements, the Laboratoryms A/E costs were still significantly higher than industry standards. Specifically, if the 14 fire and life safety improvement projects were completely removed from the comparison, A/E costs for the remaining 51 projects would still be $2.5 million higher than industry standards -- or approximately $50,000 more per building. Management Comments. Managementms interpretation of the Brooks Act for Federal A/E selections is that selection must be based on capability to perform and does not permit competition based on price. In addition, management felt the M&O contract may also preclude some competition options. Management also stated that several court cases found the Government acted improperly when cost proposals were requested even where firms had first been found to be equally qualified. However, management understands that the Savannah River Operations Office has recently implemented a process, through its M&O contractor, which reportedly satisfies the Brooks Act and permits price competition following technical prequalification. The Operations Office stated that it will pursue this issue and its application at the Laboratory under the terms of the Lockheed M&O contract. Auditor Comments. Managementms interpretation that Federal A/E selections cannot be based on price is correct. However, the DEAR specifically allows the selection of A/E contractors by an M&O contractor to include price as part of the selection criteria. The court cases mentioned were all Federal A/E selections, not selections made by an M&O contractor. Only 1 of 65 A/E designs was performed by the Departmentms A/E contractor. The other 64 were performed by or contracted out by Lockheed, and thus, were specifically allowed to be competed using price as part of the selection criteria. Since virtually all conventional A/E jobs performed for the Laboratory were allowed to be competed based on technical qualifications and price, we recommended that the Laboratory begin to compete individual A/E tasks. Management Comments. The 1993 A/E cost improvement plan was formalized in the DOE Cost Estimating Guide, Volume 6. The requirements for an approval authority and quarterly reports are no longer required by this document but are the responsibility of the Departmentms Project Manager. Copies of this guide and a formal explanation were provided to the OIG during this audit. In addition, this applied to conventional line item projects of which we have few each year. Auditor Comments. Headquarters developed an action plan to address the problem of high A/E costs throughout the Department. Specifically, Headquarters asked that "...all the action items in the Improvement Plan be implemented as expeditiously as possible." The Operations Office did not give the plan sufficient management attention to timely implement the action items required in the plan. Although there was a 15 month interval between the time the Improvement Plan was issued and the Cost Estimating Guide was completed, the Operations Office had not implemented the required items. As for managementms statement that the improvement plan was formalized into the DOE Cost Estimating Guide, Volume 6, (issued November 1994), which rendered the Improvement Plan obsolete, we could not verify that this was true. On the contrary, the Headquarters official who coordinated implementation of the Improvement Plan indicated that all of the recommendations in it were still in effect. Management Comments. From managementms point of view it appeared that the Laboratory has improved since 1990. They pointed out that the 1990 audit found that the Departmentms A/E costs were more than twice (200 percent) that of industry. The results of this audit are that the Laboratoryms A/E costs are approximately 181 percent higher. Management felt this was significantly lower than the finding identified in 1990. Auditor Comments. In spite of this positive trend, more improvement is still needed. PART IV APPENDIX Details of Comparison of Laboratory A/E Costs to Private Industry and Idaho When we compared Laboratory A/E costs for conventional construction projects to industry standards and the State benchmarks we were conservative in our use of comparison data. For example, we excluded Title III costs (construction-in- progress inspection services performed by the A/E firm) from the Laboratory data but left them in the industry standard which reduced the difference between the costs. In addition, of the three industry publications obtained, we eliminated the one with the lowest estimates of A/E costs and settled for the two more conservative publications (Means and Marshall & Swift). Also, if we were unsure as to what industry building class a Laboratory project should be compared against, we always selected the more conservative choice. In addition, we continually coordinated with Lockheed on how to make the comparison as fair as possible. For example, Lockheed indicated that its costs included a "common support" burden that pays for security, fire protection, bus service to the site and other costs that were not considered normal business expenses for industry and the State. Therefore, we removed these expenses from the Laboratory data to make the comparison to industry and the State more comparable. In addition, Lockheed pointed out that non-design charges (such as project management, cost estimating, and design review) are input into the Title I and II design charge numbers which may make the true cost of design services appear higher than it is. We looked into this and found that project management and cost estimating costs were of such an immaterial amount that it required no adjustment. The design review costs, we determined, were simply a quality control procedure that we consider a bona fide part of producing the designs, plans, and specifications. Also, Lockheed indicated that building construction costs (the denominator) were understated and A/E design costs (numerator) were overstated because A/E design costs included design costs accrued for major equipment to be installed in buildings. We found that this was true and with Lockheedms input decided that it would be more reasonable to add equipment costs in with construction costs. The industry benchmark was taken from manuals that publish architectural and engineering costs as a percentage of construction cost for most types of construction projects, including warehouses, office buildings, laboratories, medical facilities, research facilities, factories, municipal buildings, and special use facilities. Management was concerned that the comparison included a number of relatively small projects. This may result in higher numbers when calculating A/E percentages due to economies of scale. Management is correct in stating that there are economies of scale associated with the procurement of A/E services. That is, the larger the construction project, the smaller the A/E cost estimate as a percentage of construction. Fortunately, our industry data took this into account. Specifically, cost information was also broken down by project size. For example, the A/E cost estimate for an office building that cost $100,000 was 11.7 percent; a $500,000 building had an 8.5 percent cost estimate; and a $1 million building had an 7.3 percent cost estimate. This allowed us to compare the industry A/E "should be" costs for specific types of facilities and for the specific sizes of construction projects. In addition, many of the Laboratory construction projects that we reviewed were modifications to existing facilities which require more design effort than "ground up" construction. We adjusted the industry figures on a project by project basis to account for the additional design costs associated with modifications in accordance with the guidance contained in the industry manuals. We used these publications to determine an industry estimate of design costs for each conventional construction project in our audit. We then took an average of the industry estimates to come up with an aggregate industry estimate for design costs. To formulate a benchmark for the State, we obtained construction and A/E cost data for seven recently completed State construction projects that were comparable to conventional projects at the Laboratory. These seven projects included three office buildings; three laboratory-classroom buildings; and one media center that appeared to be comparable to facilities at the Laboratory. The 20 related construction projects were similar in size and makeup to the State facilities and included five office buildings; six laboratory and laboratory related projects; and nine other projects that had comparable design issues. Report No. DOE/IG-0387 CUSTOMER RESPONSE FORM The Office of Inspector General has a continuing interest in improving the usefulness of its products. We wish to make our reports as responsive as possible to our customers' requirements, and therefore ask that you consider sharing your thoughts with us. On the back of this form, you may suggest improvements to enhance the effectiveness of future reports. Please include answers to the following questions if they are applicable to you: 1. What additional background information about the selection, scheduling, scope, or procedures of the audit or inspection would have been helpful to the reader in understanding this report? 2. What additional information related to findings and recommendations could have been included in this report to assist management in implementing corrective actions? 3. What format, stylistic, or organizational changes might have made this report's overall message more clear to the reader? 4. What additional actions could the Office of Inspector General have taken on the issues discussed in this report which would have been helpful? Please include your name and telephone number so that we may contact you should we have any questions about your comments. Name ____________________________ Date______________________ Telephone _______________________ Organization______________ When you have completed this form, you may telefax it to the Office of Inspector General at (202) 586-0948, or you may mail it to: Office of Inspector General (IG-1) Department of Energy Washington, D.C. 20585 ATTN: Customer Relations If you wish to discuss this report or your comments with a staff member of the Office of Inspector General, please contact Wilma Slaughter at (202) 586-1924. _______________________________ 11/ DOE Cost Guide, Volume 6, November 1994, Chapter 25 Guidelines for Engineering, Design, & Inspection Costs.