Template for Operational Analysis Report      

This template guides the service organization when preparing an Operational Analysis Report. It includes information that is the basis for the summarized performance information presented in the Exhibit 300 and at service level reviews. The template is used to document and report an investment’s operational analysis procedures, results, and future plans including scheduled activities and corrective actions. The types of information, methods for collecting and analyzing the information, and the level of detail will depend on the size, complexity, and requirements of each investment. Include all performance information critical to determining whether the investment is meeting mission and performance objectives. The template can be augmented with charts/graphs to illustrate results and trends. Operational analysis documentation must be available for the Joint Resources Council, the Operational Quality Council, and the Office of Management and Budget (OMB). 

For those investments using this template or completing the operational analysis process for the first time not all information requested in this template may be available.  In such chases Operational analysis teams should provide as much information as possible in each section of the template.  Questions about this template should go to the Operational Quality Council, Post Implementation Review Quality Officer.

Signature Page      

Operational Analysis Report

(Program Name)

Version #

Approved by:                  Signature                   Date:
Vice President or Director of the performing service organization

Approved by:                  Signature                   Date:
Vice President or Director of the operating service organization(s) (if different from the performing service organization)

Submitted by:                  Signature                   Date:
Appropriate preparing organization

Federal Aviation Administration

800 Independence Avenue SW

Washington, DC 20591

1 Identifying Information      

1.2: Program Office

1.3: Date of Operational Analysis

1.4 : Investment (Project) Manager

1.5: Investment Owner

1.6: Submission Date

1.7: Revision Number

1.8: Revision Date

2 Operational Analysis Description      

• The period of the analysis,
• Its scope (investment segments/subsystems, any specific locations, etc),
• Operational analysis team composition
• The overall process
• Assumptions related to the strategic/business, customer, technical or financial analyses conducted as part of the operational analysis. 

3 Strategic and Business/Mission Analysis      

3.1: Summary of Goals Supported by the Investment

Summarize the specific DOT and FAA investment supported by this investment. Use appropriate goals from the DOT Strategic Plan, the FAA Flight Plan, and Line of Business (LOB) business plans. For those investments that complete Exhibits 300, include performance metrics/indicators in the areas of “mission and business results” and “processes and activities” being used to monitor performance and ensure DOT and Flight Plan mappings are consistent with those in the Exhibit 300. 

3.2: Data Collection and Analysis Methods, Procedures, and/or Tools

Describe methods, procedures, and/or tools used to collect and analyze performance goal data.

3.3:  Performance Results

Describe overall results including any instances of exceeding performance goals as well as failures to meet them. Describe the causes and impact of positive or negative performance results. 

3.4: Planned Actions

Describe activities and actions planned as a result of exceeding performance expectations or to mitigate the impact in areas where performance goals are not being met. Include activities and actions external to the investment or asset such as infrastructure shortfalls.

4 Customer, User, Stakeholder Analysis/Assessment      

Regardless of which “customer results” performance indicators and goals are reported in the Exhibit 300, customer/user/stakeholder indicators/metrics should address: Does the investment reflect current user or customer functional or performance requirements? Has the investment exceeded expectations? How does the investment address the following operational indicators in relation to customer and stakeholder requirements/expectations and their ability to perform their functions:

• Efficiency
• Effectiveness
• Maintainability/supportability
• Productivity
• Usability
• Suitability
• Security
• Availability
• Reliability
• Energy usage
• Costs and cost savings

Clearly identify any impacts on performance caused by sources external to the investment or asset, such as infrastructure (facilities, environmental, telecommunications)or events outside the control of the investment and those responsible for it.   Include the information listed below.

4.1:  Customers, Users, Stakeholders, and Related Performance Goals and Metrics

Identify primary customers/users and stakeholders related to this investment and summarize customer/user/stakeholder requirements, performance and satisfaction goals, and related metrics supported by this investment including related Exhibit 300 performance metrics/indicators in the area of “Customer Results” or “Processes and Activities” (e.g., productivity, efficiency, errors, complaints, and timeliness).

4.2:   Data Collection and Analysis Methods, Procedures, and/or Tools

Describe methods, procedures, and/or tools used to collect and assess customer/user efficiency, system usability and suitability, and customer/user satisfaction and other data (e.g., performance data, including error data, surveys, user group meetings, customer focus groups, system data, etc.)

4.3:  Performance Results

Describe overall results on actual performance and the results of surveys or other user or customer inputs, and usage analysis/trends, including any instances of exceeding performance goals as well as failures to meet performance goals. Describe the causes and impact of positive or negative performance results. 

4.4:  Planned Actions

Describe activities and actions planned as a result of exceeding performance expectations or to mitigate the impact in areas where performance goals have not been met. Include activities and actions external to the investment or asset such as infrastructure shortfalls.

5 Technical Performance Analysis      

• Functional performance/effectiveness
• Unscheduled outages
• Maintenance and equipment outages
• Mean time between outages/failures
• Replaced component count for the reporting period
• Mean time to restore
• Corrective maintenance actions labor hours
• Operational availability
• Impact on interfacing technology/downstream users
• Functional performance/effectiveness (in relation to functional requirements/objectives)

The following technical performance goals should also be addressed either in this area or in the user/customer areas where most appropriate:

• Operational productivity measures (e.g., mean time to perform tasks/functions, human-system throughput
• Human-system error rates
• Training time to proficiency
• Staffing support requirements (including technical skills, aptitudes, and related numbers)

Clearly identify any impacts on performance caused by sources external to the investment or asset, such as infrastructure (facilities, environmental, telecommunications or events outside the control of the investment and those responsible for it. 

5.1:  Technical Goals and Metrics

Summarize the technical goals and related metrics supported by this investment, including related Exhibit 300 performance metrics/indicators in the area of “Technology”.

5.2:  Data Collection and Analysis Methods, Procedures, and/or Tools

Describe methods, procedures, and/or tools used to collect and assess technical performance data (e.g. system reports, change order requests, etc.)

5.3: Performance Results

Describe overall results including any instances of exceeding performance goals as well as failures to meet them.  Describe the causes and impact of positive or negative performance results.

5.4:  Planned Actions

Describe activities and actions planned as a result of exceeding performance expectations or to mitigate the impact in areas where performance goals have not been met. Include activities and actions external to the investment or asset such as infrastructure shortfalls.

6 Financial Performance Analysis      

Explain if this investment is meeting financial/cost and schedule goals. Discuss actual cost performance to date and any expected changes to annual estimates as well as total baseline costs. In addition to the costs of the investment, this financial section should address performance in relation to cost saving/avoidance goals and metrics (e.g., cost avoidance or savings goals and indicators/metrics in the processes and activities or technology areas of the Exhibit 300) and the return-on-investment and payback-period estimates of the cost/benefit analyses that were part of investment analysis. Clearly identify any impacts on performance caused by sources external to the investment or asset, such as infrastructure (facilities, environmental, telecommunications) or events outside the control of the investment and those responsible for it. Include the information listed below.

6.1:  Financial Goals and Metrics

6.1.1 Cost elements/metrics

Explain the elements that are part of the investment’s operations and maintenance costs. Some financial/cost metrics that should be considered by all programs include:

• Costs of maintenance per system/box
• Repair costs
• Replacement parts/component costs
• Training costs
• Staffing costs
• Software license/maintenance
• Application hosting environment (infrastructure)
• Application support (to meet new/changing requirements)

6.1.2:  Cost and benefits goals

Summarize financial goals and related metrics supported by this investment including related Exhibit 300 saving/avoidance performance goals and metrics (for example, cost avoidance or savings goals and indicators in the areas of processes and activities or technology or integrated logistics support) and the return on investment and payback period estimates of the cost/benefit analyses that were part of the investment analysis. Include information about the alternatives analysis. Include the following information where available. If it is too soon (insufficient portions of the investment are operational) to make determinations, provide a planned date when sufficient operational information will be available to validate benefit expectations including, where applicable, financial performance goals from Section I.D of the Exhibit 300):

• What is the date of the program’s most current alternatives-cost/benefit analysis (AA-CBA) document?
• What is the value of the quantifiable benefits reported in the AA-CBA for the selected alternative?  Is that still a valid projection? If not, do you have a current projection of the quantifiable lifecycle benefits for your program? If yes, please describe.
• In the AA-CBA document, assumptions were made to estimate the end-of-life date for the program.  What is the end-of-life date and is that date still valid?  If not, what is the revised end-of-life date?

Please describe how the program is measuring and recording the value of accumulated benefits (including financial performance goals from Section I.D of the exhibit 300). Please detail the estimated benefits accumulation schedule (by FY) for the program consistent with your alternatives-cost/benefit analysis:

Previous Year-1 and earlier:  $

Previous Year: $

Current Year: $

Budget Year: $

Budget Year + 1: $

Budget Year +2: $

Budget Year +3: $

Budget Year +4 and Beyond:  $        

6.2:  Data Collection and Analysis Methods, Procedures, and/or Tools

Describe methods, procedures, and/or tools used to collect and analyze cost, benefit, and schedule performance data.

6.3: Cost Drivers

Identify major cost drivers related to this investment and how they impact costs

6.4: Performance Results

Describe overall results including any instances of exceeding cost performance goals as well as failures to meet them. Describe the causes and impact of positive or negative performance results. 

6.5:  Planned Actions

Describe activities and actions planned as a result of exceeding performance expectations or to mitigate the impact in areas where performance goals have not been met.

7 Actual Performance in Relation to Established Exhibit 300 Baseline Performance Goals      

(Note: This section should only be completed by investments that complete Exhibit 300s and the information should be consistent with that reported in the Exhibit 300.) 

For past and current fiscal years, provide the Performance Goals and Measures table(s) with prior-year actual results and current-year interim results, if known. Complete Table 1 and/or Table 2 below. If the project collects, manages, or reports to other performance goals and measures, add rows to record those goals and measures.

7.1: Table 1 (Optional)

Use of Table 1 is optional. It should only be used for investments that report in an OMB Exhibit 300 if either 1) not all actual performance results have been reported yet, or 2) the investment team wants to include Table 1 information for historical purposes. OMB guidance indicates that information technology (IT) investments initiated prior to FY2005 and non-IT investments must use Table 1, while IT investments initiated in FY2005 or beyond must use Table 2.  Consequently, if Table 1 is included, consistent with the Exhibit 300, it should normally not include goals for periods beyond FY 2005. If in the OMB Exhibit 300, Table 1 includes goals past FY 2005, then this table need only cover performance goals reported in the Exhibit 300 up through the current fiscal year. Clearly identify any impacts on performance caused by sources external to the investment or asset, such as infrastructure (facilities, environmental, telecommunications) or events outside the control of the investment and those responsible for it.  

7.2: Table 2

This table need only cover performance goals reported in the Exhibit 300 up through the current fiscal year. 

8 Table for Operations and Maintenance (O&M) Activities/Milestones      

• This table should include only operations and maintenance milestones/activities that are part of an investment’s steady-state or AMS in-service management phase. This table compares the actual cost and schedule performance in relation to current baseline cost and schedule goals.For investments that report in the Exhibit 300 and have a completed Exhibit 300 Program Baseline approved by the Joint Resources Council or other designated Investment Review Board, the baseline information in these tables should be consistent with that in the approved Exhibit 300 baseline document. The level of detail should be the same or greater than in the Exhibit 300. At a minimum, provide one “milestone” line for each fiscal year that shows the total costs and completion date of the O&M activities for that fiscal year. Milestones/activities may also be broken out in sufficient detail for each year to show operations activities that are supposed to take place consistent with service-level agreements or plans. Investments that do not submit Exhibit 300s to OMB may tailor or streamline this table based on the requirements of the JRC or the appropriate subordinate Investment Review Board. The cost estimates would include maintenance, training, repair, disposition, and other operational costs, including FTE costs. The milestones listed on the table may consist of a rollup of past fiscal-year costs, one line for annual costs at least up through the budget year, and a roll up of future out-year costs beyond the budget year. They may also include sub-milestones with a breakout of activities and related costs during each fiscal year (such as equipment repair costs, replacements or parts, technical documentation updates, training activities, and related FTE costs). Investments without a baseline officially approved by the Joint Resources Council or subordinate Investment Review Board should still fill in the “Planned” schedule and cost column and provide a brief explanation at the bottom of the table about the source or basis of the planned cost and schedule data.
• Questions about this table should be referred to the Value Management Office or the PIR Quality Officer

Note:  Prior to NAS handoff, programs will be using F&E funds to cover O&M costs. These costs are subject to EVM and will therefore be included in the EVM reporting process. When NAS handoff is complete, the investment or the operational segment/milestones will become subject to operational analysis.

9 Analysis of Risk      

Briefly describe how risks and related mitigation activities are monitored and any significant changes to the risk environment/status or risk-mitigation actions. 

10 Innovation Analysis      

Addressing innovation in the operational analysis demonstrates that investment managers are monitoring the current state of and availability in the marketplace of cost-saving and performance-enhancing technologies, and are communicating with investment customers and stakeholders to address questions such as:

• How can we better meet customer needs?
• Could we meet these same customer needs at lower cost?
• How could this investment be combined with others to better meet our organization’s strategic goals?
• Is the current investment implementation going to meet its life expectancy targets?

Are the customer’s most recent performance standard projections capable of being met with the current investment implementation?   

10.1: Methods used for Innovation Analysis and Related Results 

Briefly describe methods used to periodically assess if this investment adheres to the current Departmental and FAA Enterprise Architecture and whether alternatives are available to this investment, including new technical solutions and other systems developed by other agencies. Include the results of E-Gov strategy reviews and updated analysis of alternatives, market research, benchmarking and other methods used to ensure innovative approaches and technical solutions are being sought and considered. Briefly describe the results of these analyses/studies/market research, etc., and any need to start considering alternatives to this investment. Also address steps taken to ensure workforce competency and training in relation to this investment.

10.2: What was the plan and schedule for performing technical refreshes in the current and next fiscal year?  What are the associated costs to complete technical refreshes?

10.3:  Are you planning a major systems upgrade at any time over the next four (4) years? If yes, during which fiscal year will this occur? Briefly describe why the upgrade is needed and the estimated cost of the upgrade.

11 Summary of Findings and Recommendations      

If any cost, schedule, or performance variance is 10 percent or more, in the sections that follow provide a complete analysis of the reasons for the cost overrun or performance gap with planned actions to correct the variance and information on techniques that generated the savings. 

11.1:  Summary Gap Analyses

For each area of the operational analysis (mission and business, customer/user/stakeholder, technical, and financial goals), describe significant performance gaps and their root cause, and what, if any, additional functionality, capacity, or performance is required. Include information about any gap in the investment’s ability to adhere to the current DOT and FAA Enterprise Architectures and to satisfy or comply with current customer, strategic, technical, and security requirements and standards. If any cost, schedule, or performance variance is plus/minus 10 percent or more, summarize the reasons for the cost overrun or performance gap with planned actions to correct the variance and share techniques that generated savings. Address lessons-learned, why the problems occurred (root cause), or how savings were realized. Explain if further analysis is planned to better determine variance causes and how to address them. 

11.2:  Opportunities/Alternatives for Improvements

Based on the gap analysis, identify opportunities or alternatives to address the gap by improving functionality or performance (effectiveness and or efficiency). These opportunities may include investing in technology compliant with the FAA and Departmental Enterprise Architectures, business process reengineering, or collaboration with another project. Discuss whether E-Gov initiatives can be leveraged. Describe if/how the project could deliver services more efficiently in a web-based environment.

11.3:  Conclusions and Recommendations

State if current benefits continue to justify the operational costs of the investment or whether it should be enhanced or terminated. Specifically recommend if existing asset should be 1) continued with no additional investment, 2) enhanced, 3) terminated, or 4) migrated to a similar asset and retired. If the asset is to be enhanced or terminated, summarize the actions to be taken this fiscal year.

11.4:  Program/Investment Owner Recommendation/Concurrence

12 Action Plan      

(If applicable) In the table below, list actions planned for each performance area analyzed and the status of these actions. Uncompleted In Service Decision (ISD) action plan activities should also be included.  Actions may include plans to conduct analyses for alternate technologies or obtain more information, as well as corrective actions to address positive or negative cost or performance variances.