TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

 

 

The Business Systems Modernization Program Has Achieved Mixed Success in Addressing Weaknesses Identified in Internal and External Studies

 

 

 

November 2005

 

Reference Number:  2006-20-003

 

 

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Phone Number   |  202-927-7037

Email Address   |  Bonnie.Heald@tigta.treas.gov

Web Site           |  http://www.tigta.gov

 

 

November 21, 2005

 

 

MEMORANDUM FOR CHIEF INFORMATION OFFICER

                                        

FROM:                            Michael R. Phillips /s/ Michael R. Phillips

                                         Deputy Inspector General for Audit

 

SUBJECT:                    Final Audit Report – The Business Systems Modernization Program Has Achieved Mixed Success in Addressing Weaknesses Identified in Internal and External Studies (Audit # 200520026)

 

This report presents the results of our review to determine whether the Internal Revenue Service (IRS) Business Systems Modernization (BSM) Challenges Plan actions have been or will be effective in improving and strengthening the overall performance of the modernization program by addressing study weaknesses.  The IRS Chief Information Officer (CIO) requested the Treasury Inspector General for Tax Administration perform a review of the effectiveness of the BSM Challenges Plan.  This audit is in response to the CIO’s request.

The IRS is currently engaged in an effort, known as BSM, to modernize its systems and associated processes.  When the current IRS Commissioner arrived at the IRS in May 2003, the Commissioner and the PRIME contractor[1] determined they needed fresh assessments from outside experts on the health of the BSM program.  As a result, in the summer of 2003, the IRS launched a comprehensive review comprising three studies of the BSM program.  In addition, the PRIME contractor launched an internal study.  To address the results of these studies, the IRS and the PRIME contractor developed a 48-point action plan,[2] known as the BSM Challenges Plan, designed to address the BSM-related study recommendations.

Synopsis

The BSM Challenges Plan was intended to address longstanding BSM weaknesses identified in internal and external studies.  Our review of 25 action items found that, for 19 (76 percent) of the action items, the IRS is achieving this objective by effectively taking actions that should help address the identified weaknesses.  For example, project managers from both the IRS and the PRIME contractor believe communications have improved following the reorganization of the modernization governance structure.  In addition, the IRS has hired a number of executives from outside the IRS to increase the management capability and skills available to the BSM program.

For the remaining six items, we identified additional actions we believe are needed to effectively address the identified weaknesses.  For example, the IRS has made improvements in the areas of configuration and change management and has initiated or planned an ambitious set of activities; however, it was difficult to determine all the activities that needed to be accomplished or a time period for the activities because an overall action plan had not been prepared.

In addition, the Associate CIO (ACIO), BSM, informed us the relationship between the IRS and the PRIME contractor was changing.  Under the new operating model, the PRIME contractor will be a solutions provider and the BSM Office will become the program integrator.  BSM Challenges Plan Action Item 30-07 (Identify key productivity and quality metrics[3] across the life cycle based on industry standards), Action Item 180-09 (Implement enhanced subcontractor management model), and other action items have been affected by the change in the IRS and PRIME contractor relationship.

Recommendations

To ensure study weaknesses and previous recommendations concerning change/configuration management[4] are addressed, we recommended the ACIO, Enterprise Services, 1) create an overall plan that includes defined tasks, responsible individuals, and estimated completion dates for implementing one process for managing change; 2) create an overall plan that includes defined tasks, responsible individuals, and estimated completion dates for implementing the standardized configuration management toolset; and 3) ensure corrective actions are completed by assigning executive-level responsibility for addressing configuration management deficiencies.[5]

To ensure study weaknesses are addressed as part of the new IRS/PRIME contractor operating model, we recommended the ACIO, BSM, 1) assume the responsibility for correcting the identified study weaknesses concerning project metrics, 2) consider enforcement of the “inspection clause”[6] on all new contracts and establish staffing to accomplish inspections, and 3) review additional action items closed due to the change in the IRS/PRIME contractor operating model to determine if any aspects of the closed action items should be addressed as part of the new BSM operating model.

Response

The CIO agreed with our recommendations.  The CIO responded the BSM Challenges Plan has evolved to support the BSM program’s goal of continuous improvement.  BSM officials have identified highest priority initiatives, taking into consideration actions from the BSM Challenges Plan as well as internal and external assessments of the needs of the BSM program.

To ensure study weaknesses and previous recommendations concerning change/configuration management are addressed, the IRS is creating a plan to implement one process for managing change requests and Requests for Information Services.  The IRS is also hiring a contractor to assist in identifying a solution that integrates configuration management components on a Modernization and Information Technology Services organization-wide scale and has assigned responsibility for addressing configuration management deficiencies.  To ensure study weaknesses are addressed as part of the new IRS/PRIME contractor operating model, the IRS has initiated a phased approach to identifying, collecting, and using productivity and quality metrics.  In addition, the IRS has completed actions to ensure contractors are held accountable for work items and commitments (e.g., performing code and documentation inspections) and the BSM Quality Assurance organization coordinates with subcontractor Quality Assurance organizations to ensure quality products are delivered.  Lastly, the IRS has implemented processes to define project roles and decision authorities and to conduct life cycle reviews, and is currently implementing a formal risk elevation process.  Management’s complete response to the draft report is included as Appendix VII. 

Copies of this report are also being sent to the IRS managers affected by the report recommendations.  Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

 

 

Table of Contents

 

Background

Results of Review

The Business Systems Modernization Office and the PRIME Contractor Have Completed Actions That Have Led to Improvements in the Business Systems Modernization Program; However, More Work Lies Ahead

Additional Work Is Required to Completely Address Six Closed Business Systems Modernization Challenges Plan Action Items

Recommendations 1 through 4:

Recommendations 5 and 6:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Independent Assessment of the Business Systems Modernization Challenges Plan Status

Appendix V – Enterprise Life Cycle Overview

Appendix VI – Business Systems Modernization Project Descriptions

Appendix VII – Management’s Response to the Draft Report

 

 

Background

 

The Internal Revenue Service (IRS) is currently engaged in an effort, known as Business Systems Modernization (BSM), to modernize its systems and associated processes.  To facilitate the success of its modernization efforts, the IRS hired the Computer Sciences Corporation as the PRIME contractor and integrator for the BSM program and created the BSM Office (BSMO) to guide and oversee the work of the PRIME contractor.[7]

While the IRS and PRIME contractor have deployed projects that provide value to taxpayers and have built the infrastructure needed to support these projects, the IRS and its contractors have drawn increased criticism due to continuing schedule delays and cost increases.  When the current IRS Commissioner arrived at the IRS in May 2003, the Commissioner and the PRIME contractor determined they needed fresh assessments from outside experts on the health of the BSM program.  As a result, in the summer of 2003, the IRS launched a comprehensive review comprising three studies of the BSM program.  In addition, the PRIME contractor launched an internal study.

The BSM program released the results of these studies early in Fiscal Year 2004.  The studies were commissioned to identify specific areas for improvement in the BSM program.  To address the results of these studies, the IRS and the PRIME contractor developed a 48-point action plan,[8] known as the BSM Challenges Plan, designed to address the BSM-related study recommendations.  The IRS later merged 10 action items, resulting in 38 action items being actively worked by the IRS.  On May 1, 2004, the BSM Challenges Plan Close-Out Report was released detailing the actions taken to address BSM challenges.

In the summer of 2004, the IRS Chief Information Officer (CIO) requested the Treasury Inspector General for Tax Administration (TIGTA) perform a review of the effectiveness of the BSM Challenges Plan.  In December 2004, we issued a report on the results of that audit.[9]  We determined key IRS executives and stakeholders quickly acknowledged BSM shortcomings and developed action items to address the studies’ recommendations and resolve longstanding BSM issues.  However, we determined the studies’ recommendations were not fully addressed by the BSM Challenges Plan; no measurement plan was created to determine if the actions taken resulted, or will result, in actual improvements in the BSM program; and many BSM Challenges Plan action items were closed before all significant activities were completed.

In August 2004, the Associate CIO (ACIO), Modernization Management, identified the key barriers to success in the BSM program and created a plan to address these barriers.  This plan included detailed, high-priority initiatives to deal with the root causes that created the barriers.

Since many of the BSM Challenges Plan action items were reopened following our initial review, this audit was initiated to review the effectiveness of the BSM Challenges Plan.  We analyzed the audit work that had been completed during other reviews of the BSM program that related to the action items, performed indepth work on the effectiveness of the six additional action items shown in Figure 1, and evaluated the status of action items as communicated by the IRS.

Figure 1:  BSM Challenges Plan Action Items Reviewed in Depth
 As Part of This Audit

 

 

Source:  BSM Challenges Plan and this report.

This audit was conducted while changes were being made at the BSM program level, including a significant change in the relationship between the IRS and the PRIME contractor.  Any changes that have occurred since we concluded our analyses in April 2005 are not reflected in this report. 

This review was performed at the BSMO facilities in New Carrollton, Maryland, during the period October 2004 through April 2005.  The audit was conducted in accordance with Government Auditing Standards.  Detailed information on our audit objective, scope, and methodology is presented in Appendix I.  Major contributors to the report are listed in Appendix II.

 

Results of Review

 

The Business Systems Modernization Office and the PRIME Contractor Have Completed Actions That Have Led to Improvements in the Business Systems Modernization Program; However, More Work Lies Ahead

 

The BSM Challenges Plan was intended to address longstanding BSM weaknesses identified in internal and external studies.  Our review of 25 action items found that, for 19 (76 percent) of the action items, the IRS is achieving this objective by effectively taking actions that should help address the identified weaknesses.  For the remaining six items, we identified additional actions we believe are needed to effectively address the identified weaknesses.  These items are discussed in detail later in the report. 

In addition, we did not draw any conclusions about the remaining 13 of 38 total BSM Challenges Plan action items because the items met 1 or more of the following 3 criteria: 

• The IRS reports the action items are open or involve ongoing activities.
• A Fiscal Year 2005 TIGTA audit is currently being conducted to review the action items.
• The TIGTA has not scheduled an audit of the action items and does not have enough information or background knowledge to be able to express an opinion.

Figure 2 summarizes our opinions on the closure status of BSM Challenges Plan action items.[12] 

 

Figure 2:  TIGTA Opinion on the Closure Status of BSM Challenges Plan Action Items

 

 Figure 2 was removed due to its size.  To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

 

The IRS has successfully closed 19 BSM Challenges Plan action items

 

We determined the IRS has effectively closed 19 BSM Challenges Plan action items.  Some examples of positive outcomes that resulted from closing these action items follow.

·         Action Item 30-04 (Clarify roles of committees as advisory) – The PRIME contractor’s internal study found the decision-making process was ambiguous, with multiple steering committees, review meetings, and individuals either recommending, agreeing, providing input, or making decisions.  Based on the responses to a TIGTA questionnaire (see Figure 3), project managers from both the IRS and the PRIME contractor believe communications have improved following the reorganization of the modernization governance structure.

 

Figure 3:  BSM Project Managers’ Responses to TIGTA Questionnaire Concerning the Extent to Which Communications Have Improved

 

Figure 3 was removed due to its size.  To see Figure 3, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

·         Action Item 180-05 (Strengthen IRS program and project manager cadre) – The IRS Oversight Board recommended the management cadre be significantly strengthened by increasing the experience and capabilities of the BSM team.  This would require a meaningful number of outside hires from organizations that have experience with large, complex programs.  The IRS has hired a number of executives from outside the IRS to increase the management capability and skills available to the BSM program.  

·         Action Item 90-17 (Establish a BSM Acquisition Executive) – One of the external studies recommended the IRS increase focus on performance-based acquisitions.  The executive the IRS hired for modernization to complete this Action Item has Department of Defense level 3 equivalent certification in contracting and program management and has created a plan aimed at addressing performance-based acquisitions.

In addition, other actions items, such as 180-15 (Reduce the number of projects being conducted at the same time), have been accomplished.  We believe focusing on a smaller set of projects will lead to improvements in the BSM program and have recommended this action in the past.

 

Additional Work Is Required to Completely Address Six Closed Business Systems Modernization Challenges Plan Action Items

 

In a program of the size and complexity of the BSM program, it is not unreasonable to expect that changes will take time to implement and additional actions will be needed to correct weaknesses.  Of the 25 BSM Challenges Plan action items we reviewed, there are 6 we believe require more work to completely address internal and external study weaknesses.  The remainder of this report will discuss these six action items.

 

Action Item 90-07 (Refine change request process)

 

The IRS internal study recommended improving immature management processes.  One of the processes singled out in the BSM Challenges Plan was the change request process.  BSM Challenges Plan Action Item 90-07 was initiated for this purpose.  In addition, some of the activities taken as a result of this Action Item respond to our previous recommendations.

The IRS has made improvements in the areas of configuration and change management and has initiated or planned an ambitious set of activities; however, it was difficult to determine all the activities that needed to be accomplished or a time period for the activities because an overall action plan had not been prepared.  For example:

• Improving the change process – The IRS and the PRIME contractor had developed ambitious proposals for unifying and controlling change requests and Requests for Information Services;[13] however, an overall plan including defined tasks, responsible individuals, and estimated completion dates had not been completed.
• Implementing a standardized configuration management[14] toolset – In a prior audit, we determined some Information Technology Services organizations used different configuration management software that did not comply with the Enterprise Architecture.[15]  We recommended the CIO develop a transition plan to implement standardized Enterprise Architecture-compliant configuration management software.[16]  We determined the IRS created a plan; however, it would be several years before it was fully implemented.  While there are many activities ahead, the plan did not include estimated dates when activities are planned to be accomplished or assign accountability for ensuring the activities are completed.
• Revising the Configuration Management Directive[17] – In a previous audit, we recommended the CIO modify the Modernization and Information Technology Services (MITS) Configuration Management Directive and procedures to 1) assign responsibility for ensuring MITS organization configuration management processes are implemented throughout the Information Technology Services organization and coordinated with the BSMO and configuration management deficiencies[18] are appropriately addressed and 2) establish governance policies, similar to those used by the BSMO, for defining the authority levels and threshold criteria to approve and control changes to the production environment in the Information Technology Services organization.[19]

While the majority of the corrective actions related to our prior recommendations have been completed successfully,[20] we determined the revised Configuration Management Directive did not establish executive-level responsibility for addressing configuration management deficiencies.  If executive-level responsibility were assigned for addressing configuration management deficiencies, higher visibility would be given to ensuring known configuration management problems are corrected. 

Without an overall plan that includes defined tasks, responsible individuals, and estimated completion dates, as required by sound management practices, there is a high risk significant tasks may not be completed.  An overall action plan had not been created because the IRS was hiring an executive to oversee highly complex information system initiatives such as configuration management.

Management Actions:  During our audit, the IRS hired the ACIO, Enterprise Services, to oversee the configuration management process.  During discussion of our concerns with the ACIO, Enterprise Services, we were informed a unified work request process group was created to develop a plan to unify change request and Request for Information Services processing.  The ACIO, Enterprise Services, provided a charter for the group and a draft project management plan to guide activities.  In addition, the ACIO, Enterprise Services, provided a draft set of objectives for the unified work request process initiative:

• Unify and improve effectiveness of fragmented work request processes.
• Integrate with investment prioritization and control processes.
• Integrate with change and configuration management processes.
• Incorporate flexibility and discipline for a wide range of work request types.

The ACIO, Enterprise Services, indicated the configuration management work request recommendations should be completed in the summer of 2005 and many of the processes should be in place by early fall of 2005.

 

Action Item 30-07 (Identify key productivity and quality metrics across the life cycle based on industry standards)

 

In February 2005, the ACIO, BSM, informed us the relationship between the IRS and the PRIME contractor was changing.  Under the new operating model, the PRIME contractor will be a solutions provider and the BSMO will become the program integrator.

One BSM Challenges Plan action item affected by the new operating model is Action Item 30-07.  The PRIME contractor was initially responsible for this Action Item, which involved developing and implementing project-level productivity and quality metrics for modernization projects across the entire life cycle.

To close Action Item 30-07, the PRIME contractor prepared a plan entitled PRIME Program Enterprise Life Cycle Key Productivity and Quality Measures Roll Out Plan.  Our review of this Plan determined:

• The Plan was never updated or completed.
• Only the suggested metrics for Milestone 4[21] were defined in detail.
• Documentation on stakeholders’ reviews of the PRIME contractor’s Plan was not available.  The PRIME Program Enterprise Life Cycle Key Productivity and Quality Measures Roll Out Plan stated the effectiveness of the Plan would be determined through stakeholder reviews.
• The BSM Key Productivity and Quality Measures Plan mentioned in the PRIME contractor’s Plan was never created.

The Deputy ACIO, Business Integration, informed us the recent change in the IRS/PRIME contractor operating model caused the PRIME contractor to dissolve its Estimation and Maintenance Office, which had created the PRIME contractor’s Plan.  The Deputy ACIO, Business Integration, also informed us she had begun an initiative on program-level metrics, unrelated to Action Item 30-07.  Based on our conversation, she also stated it might be useful for her staff to refer to the PRIME contractor’s Plan while developing and enhancing program-level metrics.  This research leads us to conclude study weaknesses were not fully addressed by activities taken under Action Item 30-07.

Management Action:  We communicated our concerns with Action Item 30-07 to the ACIO, BSM, on April 5, 2005.  The ACIO, BSM, responded by stating over 300 measures had been identified as candidates for the MITS organization performance scorecard.  For certain measures, standard contract language was being developed for use in future contracts.  In addition, the ACIO, BSM, indicated the IRS first wants to ascertain what data are already available to support useful metrics without specifically tasking contractors.  Once this is complete, the IRS would determine what needs to change.

 

Action Item 180-09 (Implement enhanced subcontractor management model)

 

Action Item 180-09 is also affected by the new IRS/PRIME contractor operating model.  The PRIME contractor’s internal study identified improved subcontractor accountability and delivery as an issue.  BSM Challenges Plan Action Item 180-09 was developed to resolve this issue.

The PRIME contractor provided a listing of improvement activities completed in response to this Action Item.  For example, the PRIME contractor conducted subcontractor performance evaluations and required additional data from subcontractors.  However, processes had not been established to ensure the activities were performed on a routine basis and did not involve trending of subcontractor performance.  Therefore, we were not confident the improvement activities would be repeated; without trending, it would not be possible to determine whether subcontractor performance was improving.

Based on the new operating model, the PRIME contractor activity for Action Item 180-09 was closed.  Without a PRIME contractor program office in place, the BSMO will need to ensure contractors are adequately overseeing their subcontractors.  One way to achieve this is through the use of inspection clauses, which can be incorporated into contracts to allow Federal Government employees to inspect the goods and services completed by subcontractors.

Management Action:  On February 22, 2005, we communicated our concerns with Action Item 180-09 to the ACIO, BSM, who responded he was interested in our concerns and would discuss the issue with the Office of Procurement.

 

Other action items affected by the change in the IRS/PRIME contractor operating model

 

The change in the IRS/PRIME contractor operating model also affected three additional BSM Challenges Plan action items.  All three were closed based on the new operating model.

• Action Item 30-03 (Enforce rapid escalation of issues) – An issue is defined as a situation or condition that either (1) currently has negative consequences for the program/project or (2) has 100 percent probability of having negative consequences for the program/project.  In a complex environment, it is important for issues to be raised for resolution as quickly as possible.
• Action Item 90-12 (Ensure projects strictly follow the Enterprise Life Cycle and appropriate alternatives within the Enterprise Life Cycle are selected) – Since projects differ in size and complexity, it is important to provide guidance to project personnel on choosing what activities from the Enterprise Life Cycle are most appropriate.
• Action Item 180-11 (Clarify roles between the IRS and the PRIME contractor) – This Action Item involved improving four processes:  change request versus defect[22] reporting resolution, issue resolution, integrated master schedule[23] management, and change request resolution.

If improvement activities for these Action Items do not continue, it is possible study weaknesses will persist.  Since the processes in the above three Action Items will remain important regardless of who the program integrator is, it would be prudent to determine how these actions are going to be managed with the BSMO as the new program integrator.

 

Recommendations

 

To ensure study weaknesses and previous recommendations concerning change/configuration management are addressed, the ACIO, Enterprise Services, should:

Recommendation 1:  Create an overall plan that includes defined tasks, responsible individuals, and estimated completion dates for implementing one process for managing change requests and Requests for Information Services.

Management’s Response:  The CIO agreed with this recommendation.  To ensure study weaknesses and previous recommendations concerning change/configuration management are addressed, the IRS is creating a plan to implement one process for managing change requests and Requests for Information Services.  The IRS expects to complete the plan by December 31, 2005, and implement a system by October 31, 2006.

Recommendation 2:  Create an overall plan that includes defined tasks, responsible individuals, and estimated completion dates for implementing the standardized configuration management toolset.

Management’s Response:  The CIO agreed with this recommendation.  The IRS is hiring a contractor to assist in identifying a solution that integrates configuration management components on a MITS organization-wide scale, with a tool solution that supports the integrated process.  The IRS expects to complete a plan to implement the configuration management toolset by April 2006.

Recommendation 3:  Ensure corrective actions are completed by assigning executive-level responsibility for addressing configuration management deficiencies.

Management’s Response:  The CIO agreed with this recommendation.  The IRS has already assigned responsibility for addressing configuration management deficiencies.  Significant issues identified during internal configuration audits will be assigned to the appropriate executive, and documentation issues identified during compliance assessment audits will be assigned to the appropriate project manager.

To ensure study weaknesses are addressed as part of the new IRS/PRIME contractor operating model, the ACIO, BSM, should:

Recommendation 4:  Assume the responsibility for correcting the identified study weaknesses concerning project metrics by:

a.       Identifying a group of standardized productivity and quality metrics needed for all projects across the entire life cycle, defining these metrics, ensuring the metrics are collected or calculated, and using the metrics to consistently manage projects.

b.      Considering the impact of project-level metrics on the effort by the Deputy ACIO, Business Integration, to develop program-level metrics.  Identification of a standard set of project-level metrics could collectively become program-level metrics or provide needed details for analysis in trends at the program level.

Management’s Response:  The CIO agreed with this recommendation.  The IRS has initiated a phased approach to identifying, collecting, and using productivity and quality metrics.  The IRS will continue to use existing metrics as it makes improvements in identifying, collecting, and using productivity and quality metrics.  The IRS is currently working to develop cost and schedule variance metrics and will then focus on establishing a performance measurement framework, targeting business value metrics first.

Recommendation 5:  Consider enforcement of the “inspection clause” on all new contracts and establish staffing to accomplish inspections.

Management’s Response:  The CIO agreed with this recommendation.  The IRS has completed actions to ensure contractors are held accountable for work items and commitments.  For example, the IRS is performing code and documentation inspections, as well as conducting testing.  In addition, the BSM Quality Assurance organization coordinates with subcontractor Quality Assurance organizations to ensure quality products are delivered.

Recommendation 6:  Review additional action items closed due to the change in the IRS/PRIME contractor operating model to determine if any aspects of the closed action items should be addressed as part of the new BSM operating model.

Management’s Response:  The CIO agreed with this recommendation.  The IRS is currently implementing a formal risk elevation management process, which includes the use of risk coordinators on each project and regular project and executive risk reviews.  The IRS has already implemented a process to define project roles and decision authorities.  This process is part of the Enterprise Life Cycle and will apply to any contractor with whom the IRS works.  Lastly, the IRS has implemented four processes to conduct life cycle reviews across all phases of the Enterprise Life Cycle.

 

Appendix I

 

Detailed Objective, Scope, and Methodology

 

The overall objective of this review was to determine whether the Internal Revenue Service (IRS) Business Systems Modernization (BSM) Challenges Plan actions have been or will be effective in improving and strengthening the overall performance of the modernization program by addressing study weaknesses.[24]  NOTE:  At the beginning of our review, we chose to review 6 action items in depth because the other 42 BSM Challenges Plan action items 1) had been merged into other action items, 2) were the subject of other Treasury Inspector General for Tax Administration audits, 3) had not progressed far enough to warrant a review, or 4) were low-priority audit items.

To accomplish this objective, we: 

I.                   Determined if the BSM Acquisition Executive had a documented plan to address study weaknesses (Action Item 90-17, Establish a BSM Acquisition Executive). 

II.                Determined if new project managers (Director, Internal Management, and Director, Infrastructure Management) had a documented plan to address study weaknesses (Action Item 180-05, Strengthen IRS program and project manager cadre). 

III.             Determined if IRS and contractor project managers indicated the new governance structure had resulted in clearer or more consistent instructions (Action Item 30-04, Clarify roles of committees as advisory).  NOTE:  To answer this question, we submitted a questionnaire to five IRS and five contractor project managers for three of the six tax administration/internal management projects (Customer Account Data Engine [two releases], e-Services, and Integrated Financial System) and one of the two core infrastructure projects (Infrastructure Shared Services)[25] funded in the Fiscal Year 2004 expenditure plan.

IV.             Determined if the activities being taken as part of BSM Challenges Plan Action Items 180-09 (Implement enhanced subcontractor management model), 90-07 (Refine change request process), and 30-07 (Identify key productivity and quality metrics[26] across the life cycle based on industry standards) effectively addressed the associated weaknesses identified in the recent studies. 

V.                Identified the results of previous audits concerning BSM Challenges Plan action items.

VI.             Evaluated the status of action items as communicated by the IRS.

 

Appendix II

 

Major Contributors to This Report

 

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary V. Hinkle, Director

Troy D. Paterson, Audit Manager

Paul M. Mitchell, Lead Auditor

Tina Wong, Senior Auditor

Steven W. Gibson, Auditor

Perrin T. Gleaton, Auditor

 

Appendix III

 

Report Distribution List

 

Commissioner  C

Office of the Commissioner – Attn:  Chief of Staff  C

Deputy Commissioner for Operations Support  OS

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Associate Chief Information Officer, Enterprise Services  OS:CIO:ES

Associate Chief Information Officer, Information Technology Services  OS:CIO:I

Associate Chief Information Officer, Management  OS:CIO:M

Director, Stakeholder Management  OS:CIO:SM

Deputy Associate Chief Information Officer, Business Integration  OS:CIO:B:BI

Deputy Associate Chief Information Officer, Program Management  OS:CIO:B:PM

Deputy Associate Chief Information Officer, Systems Integration  OS:CIO:B:SI

Chief Counsel  CC

National Taxpayer Advocate  TA

Director, Office of Legislative Affairs  CL:LA

Director, Office of Program Evaluation and Risk Analysis  RAS:O

Office of Management Controls  OS:CFO:AR:M

Audit Liaisons:

Associate Chief Information Officer, Business Systems Modernization  OS:CIO:B

Manager, Program Oversight Office  OS:CIO:SM:PO

 

Appendix IV

 

Independent Assessment of the Business Systems Modernization Challenges Plan Status

 

To obtain fresh and independent assessments from outside experts on the health of the Business Systems Modernization (BSM) program, the Internal Revenue Service (IRS) launched a comprehensive review consisting of three studies and a BSM benchmarking analysis.  In addition, the PRIME contractor[27] launched an internal study.  The studies resulted in 21 recommendations for improvement in the BSM program, 15 of which were similar to those made in Treasury Inspector General for Tax Administration (TIGTA) reports issued over several years.[28]  Key IRS executives and stakeholders reviewed the results of the four studies, quickly acknowledged BSM shortcomings, and developed actions to address the studies’ recommendations and resolve longstanding BSM issues.  Collectively, these actions became known as the BSM Challenges Plan.

Based on our analysis of actions taken and the latest IRS BSM Challenges Plan status report (dated June 6, 2005), we created the following categories to document our opinion on the closing of each BSM Challenges Plan action item:

• Agree (19) – Activities conducted as part of the BSM Challenges Plan meet one of the following criteria:  1) the action item effectively corrected weaknesses found in the studies, 2) the IRS reports it has completed or is in the process of completing corrective actions to all relevant TIGTA recommendations, or 3) activities have been completed, yet it will take time to determine the effectiveness of the action item.
• Did Not Determine (13) – Activities conducted as part of the BSM Challenges Plan meet one or more of the following three criteria:  1) the IRS reports the action item is open or involves ongoing activities, 2) a Fiscal Year (FY) 2005 TIGTA audit is currently being conducted to review the action item, and/or 3) the TIGTA has not scheduled an audit of the action item and does not have enough information or background knowledge to be able to express an opinion.
• More Work Needed (6) – Activities conducted as part of the BSM Challenges Plan have not completely addressed study weaknesses.

Details on our reasons for categorizing each BSM Challenges Plan action item are shown in Figure 1.

 

Figure 1:  TIGTA Opinion on the Status of BSM Challenges Plan Action Items

 

 

Source:  BSM Challenges Plan, previous TIGTA audit reports, the JAMES, and TIGTA analysis of the Current Status of BSM Challenges Plan dated June 6, 2005.

 

Appendix V

 

Enterprise Life Cycle Overview

 

The Enterprise Life Cycle (ELC) defines the processes, products, techniques, roles, responsibilities, policies, procedures, and standards associated with planning, executing, and managing business change.  It includes redesign of business processes; transformation of the organization; and development, integration, deployment, and maintenance of the related information technology applications and infrastructure.  Its immediate focus is the Internal Revenue Service (IRS) Business Systems Modernization (BSM) program.  Both the IRS and the PRIME contractor[61] must follow the ELC in developing/acquiring business solutions for modernization projects.

The ELC framework is a flexible and adaptable structure within which one plans, executes, and integrates business change.  The ELC process layer was created principally from the Computer Sciences Corporation’s Catalyst^® methodology.[62]  It is intended to improve the acquisition, use, and management of information technology within the IRS; facilitate management of large-scale business change; and enhance the methods of decision making and information sharing.  Other components and extensions were added as needed to meet the specific needs of the IRS BSM program.

 

ELC Processes

 

A process is an ordered, interdependent set of activities established to accomplish a specific purpose.  Processes help to define what work needs to be performed.  The ELC methodology includes two major groups of processes:

·         Life-Cycle Processes, which are organized into phases and subphases and which address all domains of business change.

·         Management Processes, which are organized into management areas and which operate across the entire life cycle.

 

Enterprise Life-Cycle Processes

 

The chart was removed due to its size.  To see the chart, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

 

Life-Cycle Processes

 

The life-cycle processes of the ELC are divided into six phases, as described below:

·         Vision and Strategy - This phase establishes the overall direction and priorities for business change for the enterprise.  It also identifies and prioritizes the business or system areas for further analysis.

·         Architecture - This phase establishes the concept/vision, requirements, and design for a particular business area or target system.  It also defines the releases for the business area or system.

·         Development - This phase includes the analysis, design, acquisition, modification, construction, and testing of the components of a business solution.  This phase also includes routine planned maintenance of applications.

·         Integration - This phase includes the integration, testing, piloting, and acceptance of a release.  In this phase, the integration team brings together individual work packages of solution components developed or acquired separately during the Development phase.  Application and technical infrastructure components are tested to determine whether they interact properly.  If appropriate, the team conducts a pilot to ensure all elements of the business solution work together.

·         Deployment - This phase includes preparation of a release for deployment and actual deployment of the release to the deployment sites.  During this phase, the deployment team puts the solution release into operation at target sites.

·         Operations and Support - This phase addresses the ongoing operations and support of the system.  It begins after the business processes and system(s) have been installed and have begun performing business functions.  It encompasses all of the operations and support processes necessary to deliver the services associated with managing all or part of a computing environment.

The Operations and Support phase includes the scheduled activities, such as planned maintenance, systems backup, and production output, as well as the nonscheduled activities, such as problem resolution and service request delivery, including emergency unplanned maintenance of applications.  It also includes the support processes required to keep the system up and running at the contractually specified level.

 

Management Processes

 

Besides the life-cycle processes, the ELC also addresses the various management areas at the process level.  The management areas include:

·         IRS Governance and Investment Decision Management - This area is responsible for managing the overall direction of the IRS, determining where to invest, and managing the investments over time.

·         Program Management and Project Management - This area is responsible for organizing, planning, directing, and controlling the activities within the program and its subordinate projects to achieve the objectives of the program and deliver the expected business results.

·         Architectural Engineering/Development Coordination - This area is responsible for managing the technical aspects of coordination across projects and disciplines, such as managing interfaces, controlling architectural changes, ensuring architectural compliance, maintaining standards, and resolving issues.

·         Management Support Processes - This area includes common management processes, such as quality management and configuration management that operate across multiple levels of management.

 

Milestones

 

The ELC establishes a set of repeatable processes and a system of milestones, checkpoints, and reviews that reduce the risks of systems development, accelerate the delivery of business solutions, and ensure alignment with the overall business strategy.  The ELC defines a series of milestones in the life-cycle processes.  Milestones provide for “go/no-go” decision points in the project and are sometimes associated with funding approval to proceed.  They occur at natural breaks in the process where there is new information regarding costs, benefits, and risks and where executive authority is necessary for next phase expenditures.

There are five milestones during the project life cycle: 

·         Milestone 1 - Business Vision and Case for Action.  In the activities leading up to Milestone 1, executive leadership identifies the direction and priorities for IRS business change.  These guide which business areas and systems development projects are funded for further analysis.  The primary decision at Milestone 1 is to select BSM projects based on both the enterprise-level Vision and Strategy and the Enterprise Architecture.

·         Milestone 2 - Business Systems Concept and Preliminary Business Case.  The activities leading up to Milestone 2 establish the project concept, including requirements and design elements, as a solution for a specific business area or business system.  A preliminary business case is also produced.  The primary decision at Milestone 2 is to approve the solution/system concept and associated plans for a modernization initiative and to authorize funding for that solution.

·         Milestone 3 - Business Systems Design and Baseline Business Case.  In the activities leading up to Milestone 3, the major components of the business solution are analyzed and designed.  A baseline business case is also produced.  The primary decision at Milestone 3 is to accept the logical system design and associated plans and to authorize funding for development, test, and (if chosen) pilot of that solution.

·         Milestone 4 - Business Systems Development and Enterprise Deployment Decision.  In the activities leading up to Milestone 4, the business solution is built.  The Milestone 4 activities are separated by two checkpoints.  Activities leading up to Milestone 4A involve further requirements definition, production of the system’s physical design, and determination of the applicability of fixed-price contracting to complete system development and deployment.  To achieve Milestone 4B, the system is integrated with other business systems and tested, piloted (usually), and prepared for deployment.  The primary decision at Milestone 4B is to authorize the release for enterprise-wide deployment and commit the necessary resources.

·         Milestone 5 - Business Systems Deployment and Postdeployment Evaluation.  In the activities leading up to Milestone 5, the business solution is fully deployed, including delivery of training on use and maintenance.  The primary decision at Milestone 5 is to authorize the release of performance-based compensation based on actual, measured performance of the business system.

 

Appendix VI

 

Business Systems Modernization Project Descriptions

 

The following is a list of Business Systems Modernization projects.

Custodial Accounting Project (CAP) – The CAP will be a single, integrated data repository of taxpayer account information, integrated with the general ledger[63] and accessible for management analysis and reporting.  The first release of the CAP will extract taxpayer account data from the Individual Master File (IMF) for the Taxpayer Account Subledger.[64]

Customer Account Data Engine (CADE) – The CADE is the foundation for managing taxpayer accounts in the IRS modernization plan.  It will consist of databases and related applications that will replace the IRS’ existing Master File processing systems and will include applications for daily posting, settlement, maintenance, refund processing, and issue detection for taxpayer tax account and return data.

Development, Integration, and Testing Environments (DITE) – The DITE is a consolidated administrative and oversight function for three related Business Systems Modernization infrastructure support environments:  the Solutions Development Laboratory, Virtual Development Environment, and Enterprise Integration and Test Environment.

e-Services – The e-Services project provides a set of web-based business products as incentives to third parties to increase electronic filing, in addition to providing electronic customer account management capabilities to all businesses, individuals, and other customers. 

Enterprise Integration and Test Environment (EITE) – The EITE provides a comprehensive integration and testing environment to support integration and testing of components from multiple projects.

Infrastructure Shared Services (ISS) – The ISS program’s goal is to deliver a fully integrated shared information technology infrastructure to include hardware, software, shared applications, data, telecommunications, security, and an enterprise approach to systems and operations management.

Integrated Financial System (IFS) – The IFS is intended to address administrative financial management weaknesses.  The first release of the IFS will include the Accounts Payable, Accounts Receivable, General Ledger, Budget Execution, Cost Management, and Financial Reporting activities.  A future IFS release will be needed to fully resolve all administrative financial management weaknesses. 

Solutions Development Laboratory (SDL) – The SDL provides an environment that permits the rapid installation and configuration of proposed or potential systems solutions and provides the flexibility to scale and respond to multiple, concurrent projects.

Virtual Development Environment (VDE) – The VDE provides a software development environment enabling geographically distributed projects and developers access to standardized tools, information, and services.

 

Appendix VII

 

Management’s Response to the Draft Report

 

The response was removed due to its size.  To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.