Detailed Information on the
Smithsonian Institution Facilities Capital Assessment

Improving cost estimates for facility projects by systematically documenting the requirements, scope of work, and desired schedule.

Improving the Facilities Condition Index (rates the condition of facilities) and the process for assessing the conditions of the Smithsonian buildings. SI has launched a parametric deferred maintenance model (developed for NASA) and has begun collecting data for several facilities in 2006. This change will put SI in line with FCI methodology used by other government agencies.

Measure: Major projects whose change from the 35% design cost estimate to final project completion cost is within ??15% tolerance per industry standards

Explanation:2004 projects include Hazy & NMAI; no major projects to be completed in 2005; 2006 projects include Patent Office Building (POB), Asia Trail Phase I at the National Zoo, Arts and Industries Building Relocation & VERITAS. For 2007, 1 of 3 targets was not met: POB Courtyard not completed until Nov 2007 and final construction costs were not available for comparison.

Measure: Progress in improving the average Facility Condition Index of Smithsonian Institution (SI) facilities

Explanation:A higher percentage shows improvement in the condition of SI facilities. The new method provides a clearer picture of building revitalization needs and provides a comparable benchmark with other Federal agencies.

Measure: Meet individual capital project milestones that result in safer environment, safer collections, state-of-the-art research space, and more and better visitor experiences.

Explanation:2004 projects include Hazy, NMAI, Pod 5, POB & NMAH; 2005 and 2006 projects include Pod 5, Patent Office Building (POB), National Museum of American History, National Museum of Natural History, Asia Trail at the National Zoo, Arts and Industries Building, VERITAS, National Museum of African American History and Culture (NMAAHC). 2007 projects include all of the 2006 projects plus National Air and Space Museum and Victor Building Relocation. 2005 delays were VERITAS (construction delayed by Native American tribe lawsuit) and Asia Trail (construction delayed 4 months by birth of Panda cub). In 2007, 3 of 10 targets (project milestones) were not met due to: (1) Asia Trail delayed due to slower than anticipated private fund raising; (2) NMAAHC Environmental Impact Statement delayed due to higher than anticipated public interest in project; (3) VERITAS continued delay due to Native American tribe lawsuit.

Measure: Percent of major projects within budget compared to estimate at construction start

Explanation:Control of change orders and scope changes during construction indicates efficient use of resources and avoidance of additional costs or delays

Measure: Percent of capital funds obligated compared to funds available

Explanation:Obligation rate is indicative of efficiency in initiating capital work in a timely manner to avoid higher cost and continued deterioration due to delays

Measure: Percent of major revitalization projects designed to 35% prior to request for construction funding

Explanation:Measures how efficiently SI develops accurate cost estimates (at the 35% design stage), which saves time and money (avoids cost escalation) by allowing early award of construction contracts. More accurate estimates provided early in the process also improves overall efficiency of the capital program by reducing the need to reallocate funds from other projects to cover inaccurate estimates. In 2007, lack of sufficient planning and design funds and changes in project priorities impacted the execution of construction projects.

Is the program purpose clear?

Explanation: The Smithsonian Institution's Facilities Capital Program has a well-defined purpose as expressed in a wide range of documents, and it is closely linked to SI's mission ("the increase and diffusion of knowledge"). The program's purpose is to ensure that there are appropriate facilities in good condition to support SI's mission through execution of an aggressive long-range revitalization program and limited construction of new facilities.

Evidence: Enabling Legislation (20 USC §53a); FY 2006 Congressional Budget Request (p191); Smithsonian Institution Annual Performance Plan - 2006; (Key Performance Indicator 5 and 5.1).

Does the program address a specific and existing problem, interest, or need?

Explanation: The Smithsonian is the principal repository of the nation's collective memory and is the world's largest museum and research complex. Updated and renovated facilities are necessary to advance SI's mission and address the condition of its buildings and infrastructure. The Capital Program is specifically focused on addressing SI's facility revitalization challenges as well as providing a vehicle for limited new construction.

Evidence: National Academy of Public Administration (NAPA) Report, July 2001; Government Accountability Office (GAO) Audit Report (GAO-05-369), May 2005; FY 2006 Congressional Budget Request; Report of the Commission on the Future of the Smithsonian Institution (May 1995).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: SI has unique facilities, many historic, that support research, education, public programs, and exhibitions that span many disciplines for millions of visitors from around the world. SI is solely responsibile for providing the facilities necessary to maintain leadership in these areas and for the upkeep of those facilities.

Evidence: Enabling Legislation (20 USC §53a); SI FY 2006 Performance Plan; NAPA Study (July, 2001); GAO Audit Report (May 2005); FY 2006 Congressional Budget Request (Five-year capital plan, pg 194).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: SI has strong capital management processes in place. Recent reviews by outside organizations, such as the GAO, confirm that the a new centralized organizational structure has clarified program responsibility and accountability. The SI has implemented a number of industry best practices, standardized cost estimating, and established strong priority and decision-making processes to use available resources most effectively. Changes to the Capital Program budget structure allow the flexibility needed to allocate available resources to the highest priority needs and achieve the maximum benefit in improving the overall condition of SI facilities with limited program resources.

Evidence: FY 2006 Congressional Budget Request; Budget Exhibit 300s for Facilities Capital projects (available on request); GAO Audit Report (May 2005); Capital Program realignment approved in FY 2004 appropriation; Project Management Handbook; Revised OFEO Organization Chart for 2005; Capital Planning Board Charter.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: SI Facilities Capital Program funds are effectively targeted through formal planning and financial control mechanisms such as the SI Capital Planning Board. NAPA and GAO have both confirmed that SI directs resources to the highest priority capital requirements.

Evidence: Capital Planning Board minutes; Annual Congressional Budget submissions; NAPA Study (July, 2001); GAO Audit Report (May 2005).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Institution developed a limited number of specific long-term performance measures that are used to determine program success. These measures focus on the program's priority to revitalize the rapidly deteriorating infrastructure. This directed focus reflects concern expressed by OMB and Congress that the Institution needs to commit program resources to this problem. Additionally, the program has efficiency measures that clearly evaluate progress in meeting the stated outcome in a timely and cost-efficient manner.

Evidence: The long-range performance goals and associated outcomes are documented in the measures tab. Additional measures employed by the program are in the FY 2006 SI Performance Plan and Operational Reviews with Goals-at-a-Glance (GAAGs).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Capital Program's set of requirements includes ambitious financial and programmatic targets and time frames to address SI's significant capital challenges by maximizing the use of available resources. Major projects are individually managed and tracked by professional project managers who work with clients and contractors to achieve the ambitious milestones and overcome financial and logistical restraints.

Evidence: SI Performance Plan; GAAGs; GAO Audit Report (May 2005); NAPA Study (2001); SI Master Planning Process; Project Management Plans for Major Projects with financial and time frame guidelines; Budget Exhibit 300s.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: SI's Annual Performance Plan contains specific annual performance measures that demonstrate incremental progress toward meeting long-term program goals, including overall funding, and the achievement of cost and schedule milestones for individual projects. Performance measures for the Capital Program are included in a variety of supporting documents that are time-tested and closely vetted with its customer base. Additionally, the status of each major project is reviewed monthly with senior staff, and reported directly to the Secretary every four months.

Evidence: The annual performance goals and associated outcomes are documented in the measures tab. Additional measures employed by the program are found in: Monthly Performance Metrics; SI Operational Reviews; Monthly project executive reviews; GAAGs; SI Performance Plans.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Institution has established a specific baseline in terms of budget, schedule, and scope for all projects of more than $5M which are undertaken as part of the Capital Program. SI's Annual Performance Plan contains annual measures with baselines and ambitious cost, schedule, and performance targets for these projects.

Evidence: Budget Exhibit 300s; Baseline for percentage of program obligated and baseline for percentage completed.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Detailed long-range plans for the Capital Program are developed and closely vetted with internal and external customers, stakeholders (e.g., National Capital Planning Commission (NCPC), Commission on Fine Arts (CFA), OMB, Congress), and contractors. In addition, contractors demonstrate their commitment to SI goals through the process of Value Engineering (VE). The VE process determines the best value solution to address the interests of all parties, and recognizes that all partners must have their goals/needs satisfied for project success. Processes used to strengthen collaboration include monthly project meetings with contractors and stakeholders for major projects and periodic sessions with the NCPC and the CFA to coordinate and address program plans.

Evidence: Contract specifications and deliverables; Partnership agreements; Long-range plans; GAO Audit Report (May 2005).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: During the past 10 years, there have been four substantive and comprehensive studies of SI facilities that have included reviews of the Capital Program. These studies evaluated the scope of the program, along with management processes and effectiveness. All confirmed that program management was proper, funding was spent effectively, and the customer needs were met. SI intends to continue seeking independent evaluations by third parties to further strengthen program management, processes, and results.

Evidence: NASA Study (July 2001); NAPA Study (July, 2001); GAO Audit Report (May 2005); Report of the Commission on the Future of the Smithsonian Institution (May 1995); Recent IG audits on the Capital Program (A-02-04, Project Mgmt, Steven F. Udvar-Hazy Center; A-02-05, Project Mgmt, National Museum of the American Indian Mall Museum; Project Mgmt, Patent Office Building).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: SI's Congressional Budgets follow the performance-based budgeting concept, linking SI plans to its funding requests. The Institution's request for Capital Program funds clearly links resources requested to long-range outcomes and annual progress on specific projects. Budget requests provide detailed descriptions and justifications for all projects that exceed $5M. The Institution prepares detailed Budget Exhibit 300s for major projects that reflect direct costs, and SI budget requests clearly show the full costs of necessary revitalization.

Evidence: SI Congressional Budget Requests; Budget Exhibit 300s; SI Performance Plans; Program Performance Reports; GAAGs.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Capital Program has a solid strategic planning structure that includes a comprehensive prioritization process - the Facility Revitalization and Construction Project Priority Assignment Matrix. Consistent with that process is a sound decision-making process that includes senior SI leadership who make up the Capital Planning Board. In addition, SI Secretary and Board of Regents approvals are required prior to submitting these requirements to OMB and Congress. Additionally, as a result of the 2001 NAPA study, a revised budget structure and changes in the capital organization structure were developed and subsequently approved in the FY 2004 appropriation. The recent GAO audit confirmed that the process appears to be moving in the right direction.

Evidence: NAPA report (2001); GAO Audit Report (May 2005); Facility Revitalization and Construction Project Priority Assignment Matrix; Capital Planning Board Charter.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The Institution's Capital Planning Board regularly evaluates issues related to cost, schedule, and risk in order to formulate a long-range capital plan. SI recently strengthened the program by adding a comprehensive cost engineering function that includes professional cost engineers who are in the business of cost analysis and methodologies. Options, alternatives, and trade-offs have become a regular component of the analyses and include risk assessment. Value engineering methodologies are also applied to major projects with appropriate trade-offs between costs, savings, and schedules.

Evidence: Establishment of cost engineering capability; OFEO FY 2005 Organizational Chart; Establishment of the value engineering capability; Project Management Handbook; Capital Planning Board Minutes.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: A comprehensive series of detailed performance metrics has been developed and collected over the past two years in the facilities management area, including Capital Program indicators. These indicators are tracked continually, are used to help manage the program, and link directly to both short- and long-range goals. Monthly status reviews are conducted with program partners for major projects. These reviews are used to identify issues and concerns, and help ensure that partners are vested, informed, and given the opportunity to provide input at critical junctures in the life cycle of each project. This process improves program performance and ensures that completed projects satisfy customers. Operational Reviews are conducted three times annually (with the highest level of SI leadership) and ensure that the overall direction and intent of the program are achieved.

Evidence: Monthly Project Manager meetings; GAAGs; SI Operational Reviews.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Accountability for the overall program is assigned to a specific senior leader. Responsibilities are further delegated to direct report project managers in performance plans that specifically assign responsibilities and accountability for achieving program goals. SI's contracting process specifically outlines contractor deliverables and includes expected results within the stated scope of the project and schedules. SI has developed a policy that contracting technical representatives are placed on site and are responsible for evaluating contractor performance against standards established in the contract. A formal evaluation process is used to address performance issues with contractors. Those who fail to perform to prescribed standards in the contract are held accountable within the limitations of contracting law. The Institution also has specific legislative authority that enables it to consider factors such as past performance and quality of work, in addition to price, when awarding contracts.

Evidence: Director, Project Manager, Resident Engineer, and Construction Manager Performance Plans; IG Reports for the Patent Office Building, National Museum of the American Indian, and the Udvar-Hazy Center.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: One of the Institution's annual performance measures evaluates how well funds that are available in a given year are tracked and obligated. A monthly obligation plan is established at the beginning of the fiscal year and the actual obligations are reported during monthly financial status meetings with senior leadership. Program managers closely monitor project execution on a monthly basis and corrective actions are taken to realign or reapply funds, when necessary. In each of the last three years, the Capital Program has met or exceeded it annual obligation targets. GAO's audit confirmed that federal appropriations are being used as intended with a satisfactory rate of obligation.

Evidence: NAPA Report (July, 2001); GAO Audit Report (May 2005); GAAGs; Capital Monthly Obligation Plans; Monthly Financial Status reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program's efficiency and effectiveness efforts have focused on strengthening its cost estimating and value engineering capabilities. These steps have significantly improved the accuracy of initial government cost estimates as well as those at the 35%, 65% and 95% stages of a project's design life cycle. The costing initiative alone has resulted in improved efficiency of design/costing efforts, while value engineering efforts have already resulted in substantial project savings. The program's benchmarking with industry-wide standards and organizations, such as the Construction Industry Institute (CII), provide efficiency measures that make a difference in program effectiveness. For example, 14 best practices are considered in program approaches. Comprehensive best-practice training is provided to program participants and implemented where practicable. A Project Definition Rating Index (PDRI) process is used to define projects adequately and effectively. Results of the PDRIs are tracked monthly in program performance metrics. Additionally, the program has developed a four-category project priority assignment matrix structure.

Evidence: Project Management Handbook; Value Engineering program; Project Definition Rating Index (PDRI) reports; Capital Program priority structure; GAO Audit Report (May 2005); Facility Revitalization and Construction Project Priority Assignment Matrix; Construction Industry Institute (CII) Handbooks and Publications citing standards and best practices.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Extensive efforts are made each year to develop a direct relationship between the facilities capital and maintenance programs of the Institution. There is also a direct relationship between the Capital Program and customer requirements. Meetings between customers, maintenance program staff, and Capital Program staff are held annually to ensure that programmatic priorities and requirements are considered and closely coordinated with the maintenance program. In addition, SI collaborates with other federal and local jurisdictions on their capital programs to ensure a coordinated and well-planned implementation of related requirements. Periodic sessions with the NCPC and the CFA are also used to coordinate and address program plans.

Evidence: Five-year Capital Plan; Mall-wide security program; VERITAS project; GAO Audit Report (2005 draft) on Mall-wide security; NCPC's Comprehensive Plan for the Nation's Capital: Federal Elements.

Does the program use strong financial management practices?

Explanation: The SI has consistently received an unqualified opinion on its financial statements. No material internal control weaknesses have been identified or reported by either internal or external sources. An Enterprise Resource Planning system (PeopleSoft) has been implemented and serves as the official accounting system. It is certified by the Joint Financial Management Improvement Program. Additionally, the SI has integrated financial and program information into day-to-day financial operations and made great strides in performance-based budgeting by restructuring its FY 2006 SI Performance Plan to link directly to the program structure of the Congressional Budget.

Evidence: Financial Statement Annual Audit Reports; 2003 SI Report on Internal Controls; FY 2006 SI Performance Plan.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: In its 2001 evaluation of SI's Facilities Management, NAPA identified 30 deficiencies and SI has taken prudent action to implement 25 of its recommendations. The remaining 5 recommendations require actions outside the of SI's control. In addition, SI's response included: an FY 2001 reorganization of the Office of Facilities Engineering and Operations (OFEO); a new FY 2004 budget structure; and an FY 2003 restructuring of the Capital Program organization. SI has also implemented a process to identify and articulate the full requirements of the Capital Program and report them in its Congressional budget process. In its recent audit, GAO confirmed that these changes are moving the Institution in the right direction to correct program deficiencies. Further, the Institution has an established Inspector General office, which routinely inspects and evaluates Capital Program performance and individual projects. Recent evaluations have shown successful results, with only minor deficiencies noted, and corrective actions have been taken when deemed appropriate.

Evidence: NAPA Report (July 2001); GAO Audit Report (May 2005); IG Reports (examples include: A-02-04, Project Mgmt, Steven F. Udvar-Hazy Center; A-02-05, Project Mgmt, National Museum of the American Indian Mall Museum; Project Mgmt, Patent Office Building).

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: The overall Capital Program is developed using a set of priorities, and the Capital Planning Board provides a sounding board for validating projects and priorities in the context of long range SI goals. All capital projects have well defined scopes of work and are designed using an established set of design criteria which meet or exceed all public building codes as well as requirements specific to SI (such as temperature/humidity tolerances for collection space). Contracts have clearly articulated deliverables, and quality control is maintained by on-site inspection. Cost, scope and schedule goals are clearly established for each project and closely monitored by management.

Evidence: Project Management Handbook; Capital Planning Board minutes; FY 2006 SI Performance Plan.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: SI is making progress in satisfying its long-term performance goals. Within the amounts appropriated annually, SI has consistently dedicated Capital Program resources to reduce the program's requirements. This has resulted in a reduction to the capital repair and revitalization backlog. To better quantify the backlog, and progress towards reducing it, SI is in the process of baselining and tracking an aggregate facilities condition index for the Institution. Specific project milestones have been met as well, reflecting the progress of the SI and its partners toward providing a safer environment and revitalized space for SI programs. The recent audit by GAO confirmed that 12 of 13 ongoing projects are on schedule and within budget.

Evidence: Measures tab; FY 2004 Program Performance Report; GAO Audit Report (May 2005).

Does the program (including program partners) achieve its annual performance goals?

Explanation: Since establishing annual goals for the Capital Program, SI has consistently reached the targets for its Capital Program. Specific examples include the amounts obligated and progress measured for major projects. For example, the SI has exceeded the 85% target for Capital Program annual obligations for the past four years. In addition, all but one major project measured since 2003 have met milestones for budget and schedule (as confirmed in the recent GAO audit). This notable achievement could not have been made without the substantial contribution of design and construction contract partners.

Evidence: FY 2004 Program Performance Report; GAO Audit Report (May 2005).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Implementing value engineering has translated into measurable cost savings and standardizing the cost-estimating process will achieve further efficiencies in the future. In addition, process improvements such as moving towards 35% design prior to funding requestl for major projects promotes efficiencies attributable to stable baselines and contract specifications. The program's ongoing benchmarking with industry-wide standards and organizations, such as the Construction Industry Institute also translate into continued efficiency improvements.

Evidence: GAO Audit Report (May 2005); FY 2004 Program Performance Report; Internal analysis of design construction and engineering; SI Performance Plan; Design Manager and Construction Manager output data; Project specific value engineering cost savings assessments (available on request).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Institution compares itself to private programs, industry-recognized standards, and best practices. For example, SI uses the Construction Industry Institute's (CII) PDRI process to evaluate projects at various stages. The PDRI process helps all stakeholders reach agreement/alignment on the scope of the project prior to the 35% design phase. We continually benchmark against standard-setting organizations for the industry such as CII, DOE, NASA, and the Federal Facilities Council (FFC). This enables SI to evaluate its processes and outcomes against established standards. SI employees are active members of these committees and help set the standards for programs industry-wide.

Evidence: CII membership; FFC membership; PDRI results; and Monthly Performance Metrics.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: In the past 10 years, there have been four substantive and comprehensive reviews of the Capital Program. These have looked both at the scope of the program and its management, effectiveness, and results. All of the evaluations confirmed that program management was proper, funding was spent effectively, customer needs were met, and overall the program was achieving intended results. SI intends to continue seeking third-party assistance, as needed, to help improve program effectiveness.

Evidence: NASA Study (July 2001); NAPA Study (July 2001); GAO Audit Report (May 2005); Commission on the Future (May 1995).

Were program goals achieved within budgeted costs and established schedules?

Explanation: SI has a strong success rate in achieving budget and schedule targets during the past several years. The recent GAO audit confirmed that 12 of 13 major projects are on time and within budget. Operational requirements and efficiencies are considered as part of the design process for capital projects and budgeted as part of the Operations & Maintenance (O & M) program. These costs/processes will be evaluated during our O&M PART in the future. Disposal of unused facilities is rarely an issue with Smithsonian's historic buildings, but it is programmed, where appropriate, as part of larger projects.

Evidence: GAO Audit Report (May 2005).