TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
System Updates and Control Improvements Are Needed to Ensure Contact Recording Will Provide an Accurate Assessment of Taxpayer Assistance Quality
July 9, 2007
Reference Number: 2007-40-115
This
report has cleared the Treasury Inspector General for Tax Administration
disclosure review process and information determined to be restricted from
public release has been redacted from this document.
Redaction Legend:
1 = Tax Return/Return Information
Phone Number |
202-927-7037
Email Address | Bonnie.Heald@tigta.treas.gov
Web Site |
http://www.tigta.gov
July 9, 2007
MEMORANDUM FOR COMMISSIONER, WAGE AND INVESTMENT DIVISION
FROM: (for) Michael R. Phillips /s/ Michael E. McKenney
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – System Updates and Control Improvements Are Needed to Ensure Contact Recording Will Provide an Accurate Assessment of Taxpayer Assistance Quality (Audit # 200640035)
The overall objective of this review was to assess the effectiveness of the Contact
Recording System[1] to determine whether it is functioning as
intended. This review was part of the
audit coverage of the Wage and Investment Division and was included in our
Fiscal Year 2007 Annual Audit Plan.
Impact on the Taxpayer
The Internal Revenue Service’s (IRS) Taxpayer Assistance Centers provide taxpayers with face‑to‑face assistance on tax matters. The IRS is implementing at the Taxpayer Assistance Centers a standardized quality measurement system that records taxpayers’ interactions with assistors to measure the quality of service it provides them. System updates and control improvements are needed to ensure the Contact Recording System is providing reliable data that can be used to accurately measure the quality of assistance provided to taxpayers.
Synopsis
In Fiscal Year 2004, the IRS began implementing the Embedded
Quality Business Integration as a standardized quality measurement system. Contact Recording is one of four components
that make up this quality measurement system. It is an innovative approach to quality
control for face‑to‑face interactions between assistors and
taxpayers. In Fiscal Years 2006 and 2007, the IRS
implemented the Contact Recording System in 126 of the 401 Taxpayer Assistance
Centers. As of December 31, 2006,
managers had reviewed 1,554 recordings to evaluate employee performance, and
Centralized Quality Review System Office reviewers had reviewed 1,635
recordings to evaluate the accuracy of the responses to tax law and tax account
questions.
Challenges remain before the Contact Recording System can be installed in the remaining 275 Taxpayer Assistance Centers. In Fiscal Year 2007, the Field Assistance Office lost funding needed to continue implementation. Because the IRS had begun developing the Taxpayer Assistance Blueprint (a 5-year plan for providing service to individual taxpayers), it did not want to commit funds to install the Contact Recording System in the remaining Taxpayer Assistance Centers until the Taxpayer Assistance Blueprint report was issued.[2]
The IRS has concentrated on installing the Contact Recording System in the 126 Taxpayer Assistance Centers and ensuring it generates sufficient recordings. Because the System is in its first year of implementation, it is reasonable to expect some problems. System patches are needed to ensure the System will produce accurate results. System software is prematurely deleting recordings, a database problem is preventing the automatic uploading of recordings, and flaws prevent all recordings from having an equal chance of being selected for quality review.
The Contact
Recording System is a significant improvement over the prior method of quality
review. However, controls need to
be in place to ensure the System is operating as intended, the System is not
subject to manipulation, and all recordings have the opportunity for selection. The IRS agreed these issues could affect the
quality results if patches are not installed but believes oversampling by at
least 5 percent minimizes the problems.
It also agreed these issues could affect the statistical reliability of
the data but believes the data from the
Contact Recording System currently provide reasonably representative samples.
Internal controls also need to be improved to ensure the Contact Recording System is operating as intended and providing accurate quality results. There is insufficient monitoring to ensure assistors are not opting out of recordings to avoid having the contacts selected for review, the System is not periodically validated to ensure all taxpayer interactions are being recorded, and the System was not designed to generate scheduled management information reports that could be used to monitor its operation.
Recommendations
After considering the Taxpayer Assistance Blueprint results, the Commissioner, Wage and Investment Division, should evaluate and ensure funding for modernizing the Taxpayer Assistance Centers is sufficient to continue installation of the Contact Recording System; implement systemic changes to ensure the System is operating effectively; ensure all desktop computers have sufficient capacity to handle System requirements and all recordings are included in the population to be sampled; continue the development, testing, and installation of the patches for problems already identified to ensure the System is operating effectively; conduct periodic validations of System data to ensure taxpayer and assistor interactions are being recorded; and develop reports and procedures to monitor the System to identify anomalies and errors that could affect the National Sample.[3]
Response
IRS management agreed with all recommendations. The IRS is committed to funding the Contact Recording System through completion; however, management cautioned that funding to remodel the Taxpayer Assistance Centers is not assured because the Wage and Investment Division competes with other business system modernization projects for funding. The IRS will annually monitor funding requests to upgrade its facilities through its budget process, to ensure compatibility required for the Contact Recording System. The IRS agreed to develop procedures to reduce “Memo Counts,”[4] explore systemic changes to prevent input of erroneous codes, and coordinate and conduct reviews that will meet the refreshment schedule to ensure adequate capacity for all resident software on desktop computers. The IRS has identified over 27 system problems; has resolved 18; and will ensure patches are developed, tested, and installed for the remainder. It will also develop a specific control to periodically validate that all taxpayer and assistor actions are recorded, which includes establishing acceptable tolerances and internal controls to ensure sufficient oversight. Finally, the IRS has recommended software enhancements to the vendor, will revise its procedures to address anomalies and errors that could affect the National Sample, has eliminated the opt-out script, and will develop reports that ensure audit trails are accessible and readable. Management’s complete response to the draft report is included as Appendix V.
Copies of
this report are also being sent to the IRS managers affected by the report
recommendations. Please contact me at
(202) 622-6510 if you have questions or Michael E. McKenney, Assistant
Inspector General for Audit (Wage and Investment Income Programs), at (202)
622-5916.
Internal Controls Are
Insufficient to Ensure the Contact Recording System Is Functioning As Intended
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix IV
– Contact Recording and the Embedded Quality Measurement Systems
Appendix V
– Management’s Response to the Draft Report
Abbreviations
|
EQBI |
Embedded Quality Business Integration |
|
IRS |
Internal Revenue Service |
|
MITS |
Modernization and Information Technology Services |
|
TIGTA |
Treasury Inspector General for Tax
Administration |
The Internal Revenue Service (IRS) provides taxpayers the option of obtaining face‑to‑face assistance at 401 Taxpayer Assistance Centers.[5] The IRS suggests that taxpayers visit the Taxpayer Assistance Centers when they have complex tax issues, need to resolve tax problems relating to their tax accounts, have questions about how the tax law applies to their individual income tax returns, or feel more comfortable talking with someone in person. Approximately 2,100 IRS employees (called assistors) work in the Taxpayer Assistance Centers providing taxpayers personal, face‑to‑face assistance with tax matters that includes interpreting tax laws and regulations, preparing certain individual tax returns, resolving inquiries on taxpayer accounts, accepting payments, and providing various other services designed to minimize the burden on taxpayers in satisfying their tax obligations.
Figure
1: The EQBI
Figure 1 was removed due to its size. To see Figure 1, please go to the Adobe PDF
version of the report on the TIGTA Public Web Page.
In Fiscal Year 2004,
the IRS began implementing a
standardized quality measurement system to measure the quality of taxpayer
service at its Taxpayer Assistance Centers.
This measurement system, called the Embedded Quality Business
Integration (EQBI), is a centralized
data collection tool that would use current Q-MATIC System technology[6] in conjunction with
the planned Contact Recording System once implemented. Its implementation addresses many concerns
raised by the Office of Management and Budget, the Government Accountability
Office, and the Treasury Inspector General for Tax Administration (TIGTA) on the accuracy of tax law answers and the need
for a quality measurement system with which to set goals and measure progress
toward achieving those goals.
Initially with the EQBI,
The Contact Recording System automatically records
the audio portion of a taxpayer contact and synchronizes it with an assistor’s
computer screen activity. The Q-MATIC
System signals the recording to begin at the start of each contact and turns it
off at the appropriate time. Group managers
critique a selection of contacts to evaluate individual employee performance
and identify training needs. Quality
reviewers use statistically valid samples of selected contacts to verify the
accuracy of assistor responses to taxpayers (which result in the Customer Accuracy
measure reported internally and externally), analyze trends, identify
opportunities for improvement, and assist the Field Assistance Office with quality
improvement efforts. Figure 2 presents a
depiction of the Contact Recording System and the Embedded Quality Measurement
System.[8]
Figure 2: The
Embedded Quality Measurement System With the Contact Recording System
Figure 2 was removed due to its size. To see Figure 2, please go to the Adobe PDF
version of the report on the TIGTA Public Web Page.
See Appendix IV for details of how the Contact Recording System is used to gather data for the Field Assistance Office responsible for the Taxpayer Assistance Center Program.
Data from the Embedded
Quality Review System are used to report Customer Accuracy,
Professionalism, and Timeliness quality measures to the IRS Commissioner
as part of the IRS’ balanced measures.[9] The Customer Accuracy measure is also
reported externally to IRS stakeholders (e.g., Congress and the Government
Accountability Office) and as part of the reporting requirement of the Government
Performance and Results Act of 1993.[10] For Fiscal Year 2007, a baseline sample of selected
contacts–tax accounts and tax law questions–was established during October
through December 2006. The Field
Assistance Office plans to report its Customer Accuracy measure based on the National
Sample[11] to its external
customers beginning in January 2007.
This review was performed at the Field Assistance Office in
the Wage and Investment Division Customer Assistance, Relationships, and
Education function in
Implementing the Contact Recording System Will Provide an Innovative Tool to Measure the Quality of Face-to-Face Assistance
The EQBI is an innovative approach to quality control for
face-to-face interactions between assistors and taxpayers. The Contact Recording System is an advanced tool with
which managers can monitor and improve quality. Managers no longer need to travel to the Taxpayer Assistance Centers and
wait to observe a specific taxpayer contact that meets the required sampling
criteria. Instead, the recording of the taxpayer
contact is selected automatically and sent to the manager for review. The IRS uses recorded contacts to critique
and improve the experience it provides taxpayers for accuracy, timeliness, and
professionalism.
Figure 3 illustrates how the IRS anticipates taxpayer and employee[12] satisfaction will improve dramatically and help it deliver business results, by comparing quality measurement with and without Contact Recording.
Figure 3:
Comparison of Embedded Quality With
and Without the Contact Recording System
|
Without Contact
Recording |
With Contact Recording |
|
Manager
or reviewer observing interaction. |
Interaction
recorded. |
|
Difference
of opinion about what occurred. |
Hear
the actual contact. |
|
One
perspective. |
Both
perspectives. |
|
Cannot
remember contact. |
Fresh
replay. |
|
Waiting
for a contact. |
Contact
available when the reviewer needs it. |
|
“Hovering”
required. |
No
more standing nearby and taking notes. |
|
Travel
to remote sites for monitoring. |
Travel
only for coaching and development. |
|
Interruptions
and distractions. |
Full
attention during playback. |
|
IRS
and TIGTA shopping.[13] |
No
need for shopping. |
|
Guesswork
on contact selection. |
Statistically
reliable samples. |
|
Subjective
reviews. |
Objective
reviews. |
|
Assumptions
on training needs. |
Training
needs identified. |
|
|
|
|
Made-up
training scenarios. |
Clear
examples of taxpayer interactions. |
|
Inconsistent
quantity and quality of feedback. |
High-quality
feedback and consistent quantity of reviews. |
Source: Field Assistance Office Frequently Asked Questions, updated July 2006.
Prior to
implementing the Contact Recording System, the IRS had created a Project Team to
identify the need, vendor, software, cost, and funding for the System. The Team consisted of an IRS manager,
employees from the Wage and Investment Division and the Modernization and
Information Technology Services (MITS) organization, and an outside contractor with
the sole responsibility for crafting a business case. The Project Team, along with the vendor, customized
commercial off‑the-shelf software with targeted funds from the MITS organization.
According to the Contact Recording System business case, the estimated cost to implement the System in the 400 Taxpayer Assistance Centers by Fiscal Year 2014 would be more than $12.2 million.[14] The Project Team is using a system development guide, widely used by the IRS to organize and manage the development and maintenance of a system. The guide consists of five milestones that represent the completion of key activities within the project’s life cycle. The milestones are used to measure progress and to provide a review point for executive or oversight approval.
The Contact Recording System has been installed in 126 Taxpayer Assistance Centers
The Field Assistance
Office conducted an extensive analysis of 400 Taxpayer Assistance Centers located
in 5 geographical Areas and identified the first 126 Centers in 37 States
in which to install the Contact Recording System. Selection of these Taxpayer Assistance
Centers was based on factors such as the size of the Centers (small, medium,
and large),[15] the number of taxpayers that had visited the
Centers in Fiscal Year 2005, and the number of assistors and
managers. The shaded States in Figure 4 illustrate
where the Contact Recording System has been installed in one or more of each State’s
Taxpayer Assistance Centers.
Figure 4: States With
Taxpayer Assistance Centers
in Which the Contact Recording System Is Installed
Figure 4 was removed due to its size.
To see Figure 4, please go to the Adobe PDF version of the report on the
TIGTA Public Web Page.
As of February 2007,
85 (54 percent) group managers and 733 (44 percent) assistors were using the
Contact Recording System software installed on 639 (51 percent) desktop
computers. All assistors’ desktop computers
tested have licensed software to operate the Contact Recording System. The assistors and managers have been trained
to use the System. The recordings are generally
audible and understandable, and the desktop screen activity is synchronized
with the audio portion of the contact.
Recordings are currently
being reviewed by managers to evaluate assistors’ performance and by reviewers
in the independent Centralized Quality Review System Office to report quality
accuracy rates to internal and external customers. From May 1 through December 31, 2006, more
than 446,000 recordings were generated.
As of December 31, 2006, managers from the 126 Taxpayer
Assistance Centers had reviewed 1,554 recordings to evaluate employee performance,
and the Centralized Quality Review System Office reviewers had reviewed 1,635
recordings to evaluate the accuracy of the responses to tax law and account
questions. The cost to implement the Contact
Recording System in the 126 Taxpayer Assistance Centers through December 31,
2006, was approximately $5 million.
Challenges Must Be Overcome Before the Contact Recording System Can Be Installed in All Taxpayer Assistance Centers
In Fiscal Year 2007,
the Field Assistance Office lost funding needed to continue implementation in
the remaining 275 Taxpayer Assistance Centers.
The IRS had begun developing the Taxpayer Assistance Blueprint (a
5-year plan for preferred service delivery to individual taxpayers within
available resources) and did not want
to commit funds to install the Contact Recording System in the remaining
Taxpayer Assistance Centers until the Taxpayer Assistance Blueprint report was
issued.[16]
Other IRS business
systems modernization projects have also been given priority over the Contact
Recording System. The Project Team is
using Fiscal Year 2006 funding to install and provide technical support for the
System in the 126 Taxpayer Assistance Centers.
This funding may not cover all expenses.
In March 2007, the funding for the technical support contracted to a
consulting firm to implement the Contact Recording System was scheduled to end. The Project Team is working to obtain funding
to continue the support through the end of Fiscal Year 2007.
Of the 275 Taxpayer Assistance Centers awaiting installation
of the Contact Recording System, 152 need modifications so they can accommodate
the System–from providing panel dividers for taxpayer and assistor privacy to
transforming offices to accept computers and a computer network. Additional funding will be needed for this
effort. Figure 5 illustrates the suitability of the
Figure 5: Suitability
and Cost to Remodel Remaining
Taxpayer Assistance Centers
|
Remaining |
Cost to Modernize |
Total Cost to Remodel |
Suitability of Offices for Contact Recording |
|
86 |
$5,000 |
$430,000 |
Suitable for
installation now. Remodel, if required. |
|
58
|
$25,000
for 56 Offices, |
$1,400,000 $100,000 |
Suitable for
installation but will have audio challenges. |
|
8 |
$125,000 |
$1,000,000 |
Not
suitable for installation until remodeling. |
|
Total Costs to Remodel 152 Offices |
$2,930,000 |
|
|
Source: Field Assistance Office Contact Recording Facilities Evaluation PowerPoint Presentation, dated August 2006.
Because funding is not assured for the remodeling of 152 Taxpayer Assistance Centers, there is no certainty the Contact Recording System will be installed in all Taxpayer Assistance Centers. Management’s support is necessary to commit the appropriate human and financial resources. The Director, Field Assistance Office, has expressed concern about the funding of the Contact Recording Project but is committed to making sure it is funded through completion. If funding is not restored, the Field Assistance Office will experience costly delays and will be unable to measure the quality of taxpayer service in the remaining 275 Taxpayer Assistance Centers.
Recommendation
Recommendation 1: After considering the Taxpayer Assistance Blueprint results, the Commissioner, Wage and Investment Division, should evaluate and ensure funding for remodeling the Taxpayer Assistance Centers is sufficient to continue installation of the Contact Recording System.
Management’s
Response: IRS
management agreed with the recommendation.
They will annually monitor, through the budget process, requests for
funding by the Real Estate and Facilities Management Office to upgrade the
Taxpayer Assistance Centers to ensure compatibility required for the Contact
Recording System. However, they cannot
assure funding for
System Patches Are Needed to Ensure the Contact Recording System Will Produce Accurate Quality Results
Because the Contact Recording System is in the first year of
implementation, it is reasonable to expect some problems, particularly because
the “off-the-shelf” software had to be adapted for the IRS. Modifications have been made to improve the System,
but patches have not been installed to fix all problems identified during this
audit and by the Project Team. As a
result, the reliability of the quality results is at risk.
Systemic controls do not prevent assistors from inputting incorrect codes[17] that cause recordings not to be included in the National Sample
The Contact Recording System uses codes to select the
contacts to be included in the National Sample for quality review. During testing at six Taxpayer Assistance
Centers, assistors input incorrect codes when completing the interactions. Although it appears the assistors used the
wrong codes in error, it is also possible to purposely use the wrong code to exclude
recordings from the National Sample.
The Contact Recording System did not prevent the input of
incorrect codes. For example, when
****1**** does not meet the criteria for the National Sample and therefore
would not be included in the selection process.
Also, the Field Assistance Office was not sufficiently monitoring the System
to ensure assistors were inputting the correct codes.
This concern has
been alleviated because the Field Assistance Office has significantly reduced the
different types of codes used to record taxpayer interactions. Yet, there are still no systemic controls to
ensure assistors are inputting correct codes.
System software is deleting recordings before the established dates
Recordings are to remain on the assistors’ desktop
computers for 10 business days before being deleted. Nevertheless,
the Contact Recording System software purged recordings earlier than 10 business
days. The premature deletion of
recordings prevents all recordings from having an equal chance of being included
in the National Sample.
The Project Team
identified this problem and has determined that in some instances the premature
purging was caused by insufficient space on desktop computers. Prior to loading the Contact Recording System
software on the assistors’ desktop computers, computer technicians conduct a
system check to ensure the computers have sufficient space to accommodate the
software and the recordings. However,
once the Contact Recording System software is loaded onto the desktop
computers, no additional checks are made to ensure adequate space remains to
handle the Contact Recording System requirements.
Assistors use
several programs to perform their duties and have space in which to store data on
their computers. The MITS organization can
install updates to existing programs onto the computers. These changes could affect the space needed
to ensure desktop computers have sufficient capacity to prevent premature
deletion of recordings. When we discussed this issue with Field Assistance
Office officials, they stated they could coordinate with the MITS organization
to submit a 5-year refreshment plan that would identify program upgrades to
mitigate the impact on the System.
The Project Team
determined that immediate uploading of the recordings alleviates the problem of
recordings being purged before 10 business days. However, the Project Team has been unable to determine
whether insufficient computer space is the only cause for the premature purging
and does not know how many computers have insufficient space.
A database problem is preventing the automatic uploading of recordings selected for the National Sample
Selected information, including the assistor’s
identification number, the date and time of the contact, and the geographic area
where the recording originated, is generally uploaded to the server at the
Brookhaven,
These suspended recordings either eventually upload automatically
or require the Project Team to intervene and push them to the reviewers. Neither we nor the Project Team were able to
determine the number of recordings that have required or require manual
intervention. However, manual
intervention introduces the potential for errors or manipulation. The
Project Team stated the risk is mitigated because it could request an error
report listing recordings that do not upload.
The Project Team has
worked with the vendor to develop a patch to correct the problem; however, the
patch has yet to be installed because it did not work during testing. The Project Team stated that actions are
underway to send programming engineers from the vendor to analyze the problem
and develop a patch to correct the problem.
Programming errors are affecting the statistical reliability of the National Sample
All recordings should have an equal chance of being
selected for the National Sample. Although
tests of the National Sample selection process showed the correct sample size
is being selected, flaws in the random sampling process prevent all recordings
from having an equal chance of being selected for quality review. IRS statisticians identified this issue
during their review of Contact Recording System data. There is an incorrect rounding of whole
numbers during the random start and skip interval segments of the selection
process. The statisticians recommended two patches to correct the errors. The Project Team tested the patches, but the
test was unsuccessful because of incompatibility problems with another Field
Assistance Office system upgrade. The patches
are waiting to be installed.
The Project Team agreed these issues could affect the quality results if the patches are not installed. However, the IRS is currently selecting 5 percent more recordings than needed for the National Sample, and the Project Team believes this minimizes the problems.
The Office of Statistics of Income also agreed these issues
could affect the statistical reliability of the data but believes (1) the data from the Contact Recording System currently
provide reasonably representative samples and (2) estimates will become more
reliable when the issues are resolved. The
Office of Statistics of Income is confident the current Contact Recording System
methodology is considerably closer to achieving this ideal than sampling
methodologies used in prior years.
The Contact Recording System in its current state is a significant improvement from prior years. However, every recording in the population of recordings should have an equal chance of selection. Controls need to be in place to ensure the System is operating as intended, the System is not subject to manipulation, and all recordings have the opportunity for selection.
Recommendations
The Commissioner, Wage and Investment Division, should:
Recommendation 2: Develop and implement systemic changes to the Contact Recording System to prevent input of erroneous codes that might preclude recordings from being selected for the National Sample.
Management’s Response: IRS management agreed with the recommendation. They will develop procedures to reduce “Memo Counts”[19] and explore making systemic changes to prevent input of erroneous codes that might preclude recordings from being selected for the National Sample in the next release of the Contact Recording System.
Recommendation 3: Develop a system to ensure all desktop computers have sufficient capacity to handle Contact Recording System requirements and to ensure all recordings are included in the population to be sampled, coordinate with the MITS organization to request a 5‑year refreshment plan of program updates to mitigate the impact on the Contact Recording System, and identify other causes of the premature purging of recordings.
Management’s Response: IRS management agreed with the recommendation. They will coordinate with the MITS organization to conduct a review, using the established refreshment schedule, of all resident software on desktop computers to ensure adequate capacity; research software that will systemically monitor free disk space; and test and install a patch to correct purge routine issues.
Recommendation 4: Develop, test, and install patches for problems already identified to ensure the Contact Recording System is operating effectively.
Management’s Response: IRS management agreed with the recommendation, stating they have identified over 27 system problems and resolved 18 of them. They will develop, test, and install patches to resolve the remaining problems.
Internal Controls Are Insufficient to Ensure the Contact Recording System Is Functioning As Intended
The Contact Recording System was designed as a tool to improve quality at the Taxpayer Assistance Centers. The System has been in operation only 11 months, and it is reasonable that there would be a learning curve. During this time, the Field Assistance Office and the Project Team have concentrated on installing the System and ensuring it generates a sufficient number of recordings for review to assess the quality of taxpayer service.
A control log was not maintained to track issues or recurring problems with the Contact Recording System during this implementation phase
The IRS
requires the use and maintenance of an issues log for all systems
development. The Project Team used a
system development guide to organize and manage the process of developing
and maintaining the Contact Recording System.
In addition, it started a
log in the pilot phase of the project but did not continue with it during this
initial implementation phase. Maintaining a control log would have allowed
the Team to track and monitor many of the issues identified during this
audit.
For example, recordings
were not being deleted from the server within 45 calendar days as required. Testing in 6 Taxpayer Assistance Centers
showed that group managers had recordings past the allowed 45 calendar days. This happened when the Field Assistance Office expanded
the retention period in May and June 2006 from 45 calendar days to 90
calendar days for testing. In June 2006, the IRS requested the
vendor to change the retention period back to 45 calendar days. However, this issue was not controlled or
monitored. The Project Team discovered
that the vendor had not changed the retention period and contacted the vendor
again in October 2006. In November 2006,
we verified that the retention period was changed back to 45 calendar days.
Internal controls have not been developed to monitor many aspects of the Contact Recording System
Internal controls, synonymous with management
controls, should provide reasonable assurance that stated objectives are being
achieved. Internal control comprises the
plans, methods, and procedures used to meet the goals and objectives and serves
as the first line of defense in safeguarding assets and preventing and
detecting errors and fraud.
Testing of the
Contact Recording System showed that it was not always operating as intended
and assistors were not always following procedures, even though it was
producing an expected number of recordings to be reviewed.
There are no controls to periodically
validate that all contacts are recorded
A comparison of recordings at 18 Taxpayer Assistance Centers to the number of taxpayer visits reported by the Q-MATIC System showed a 52 percent difference between the Q‑MATIC and the Contact Recording Systems for Tax Accounts visits and a 46 percent difference for Tax Law visits. The Field Assistance Office agreed there were differences but could not resolve the discrepancies with available Q-MATIC System reports.
The Project Team later conducted another test to determine
whether all taxpayer interactions were recorded on the Contact Recording
System. By comparing a download of raw
data from the Q‑MATIC System to a list of recordings from 1
There are no controls to
ensure assistors use the correct codes
As previously stated, there are no system controls to ensure assistors are using the correct codes. Furthermore, there are also no current procedures to monitor the System to ensure the assistors use the correct codes.
There are insufficient
controls to ensure assistors do not inappropriately use the “opt-out”
[20]
Testing in one of
six Taxpayer Assistance Centers showed an assistor mistakenly using the opt‑out
button to wrap up (end) visits with taxpayers.
When the opt-out button instead of the wrap‑up button is used,
recordings are not made available for the National Sample. In addition, assistors could inappropriately
use the opt-out button to avoid having the interaction recorded.
Although the Project
Team was conducting limited monitoring of opt-out use to determine the
percentage of taxpayers who chose not to be recorded, it was not monitoring the
opt-out recordings to ensure assistors were appropriately using the opt-out
button. The Contact Recording System
does not generate opt‑out reports; however, ad hoc reports can be
generated at the
Additionally, use of
a specific script is currently required to inform taxpayers why they are being
recorded and how the IRS will use the recording. This allows taxpayers the opportunity to continue
being recorded. Testing at 6 Taxpayer
Assistance Centers showed that, of 29 interactions reviewed, 8 (28
percent) were opted out of being recorded.
None of the assistors used the required script in any of the eight
opt-out recordings.
Neither the Project Team nor Taxpayer Assistance Center managers were monitoring the opt‑out recordings to ensure assistors were appropriately using the script or the opt-out button. However, the Field Assistance Office is considering no longer requiring the script. Signs are strategically placed at assistor workstations to alert the taxpayer that he or she is being recorded.
The Contact
Recording System does not generate management information reports needed to
monitor the System
The Contact
Recording System was not designed to generate management information
reports. However, systems should include
controls that will create, protect, and retain records to the extent needed to
enable the monitoring, analysis, investigating, and reporting of unlawful,
unauthorized, or inappropriate information system activity. Also, the system should ensure the actions of
all individual users can be traced. The
Contact Recording System has an audit trail;[21] however,
the Project Team has to request the audit trail from the vendor and it is not
very readable.
Although the Contact
Recording System does not generate scheduled reports, we performed searches and
extracted ad hoc reports to identify anomalies that affected the National
Sample and identified procedural errors.
These reports showed, for example, that assistors were inappropriately
using the opt-out function and using incorrect codes.
Also, the Contact Recording System is not easily accessible for third-party independent validations because the recordings are housed on individual desktop computers located throughout the IRS and remain on the computers for only a short period of time before they are purged. This makes it difficult to select an independent sample. However, because we have access to the recordings selected for the National Sample when signed on through an IRS computer, we are making no recommendation at this time. We are also pursuing access to all System data so we may be able to obtain access to all recordings to conduct periodic independent testing.
Recommendations
The Commissioner, Wage and Investment Division, should:
Recommendation 5: Conduct periodic validations of Contact Recording System data to ensure taxpayer and assistor interactions are being recorded. This includes establishing an acceptable tolerance for those instances for which differences between the Q-MATIC and the Contact Recording Systems cannot be reconciled.
Management’s Response: IRS management agreed with the recommendation. They will develop a specific control to validate on a periodic basis that all taxpayer and assistor interactions are being recorded. This includes establishing an acceptable tolerance for data that cannot be reconciled and implementing internal controls and procedures to ensure managerial review and oversight.
Recommendation 6: Develop reports and procedures to monitor the Contact Recording System to identify anomalies and errors that could affect the National Sample. The Commissioner should also ensure audit trails are accessible and readable and ensure assistors are using the required script, until a study can be conducted to evaluate the use of the script when taxpayers opt out of recordings.
Management’s Response: IRS management agreed with the recommendation. They have recommended changes to the vendor for software enhancements, have evaluated and eliminated the opt-out script, will revise established procedures to reflect the script’s elimination and to address anomalies and errors that could affect the National Sample, and will develop reports to ensure audit trails are accessible and readable.
Appendix I
Detailed Objective, Scope, and Methodology
The overall objective of this review was to assess the effectiveness of
the Contact Recording System[22] to determine whether it is functioning as
intended. This included the pilot and
its phases, selection and delivery of the recording for quality review, and
retention of the recording. To
accomplish our objective, we:
I. Reviewed the Contact Recording System to determine whether it functioned as intended. We visited six Taxpayer Assistance Centers. Five were judgmentally selected based on their proximity to the Atlanta, Georgia, TIGTA office. The sixth was selected because it was convenient to a Project Team meeting we attended. We verified whether desktop computers had licensed software; determined whether managers and assistors were trained; evaluated whether recordings were audible, understandable, and synchronized with the computer screen activity; and determined closing activity.
II. Evaluated the sufficiency of internal controls of the Contact Recording System.
A. Tested the reliability of the tax accounts and tax law recordings in the Contact Recording System during a 2-week period in October 2006. We judgmentally selected 1 of 5 Areas in the Field Assistance Office and reviewed all 18 Taxpayer Assistance Centers in that Area that had the Contact Recording System. We used a judgmental sample because we did not have the resources to do a statistical sample and did not plan to project our results. We compared the recordings to the Q‑MATIC System reports for those same weeks and Area.
B. Validated whether the algorithm in the Contact Recording System selected the correct sample size of recordings for the tax accounts, tax law, and tax return preparation product lines for three of five Areas for the week of June 26 through June 30, 2006.
C.
Reviewed the results of the Contact Recording
System pilot and system validation to evaluate
the effectiveness of the pilot and validation.
D. Reviewed the implementation plan for the Contact Recording System.
E. Reviewed the retention of recordings to determine whether they met the requirements for the Field Assistance Office Program and record retention requirements.
III. Determined whether the results of the Contact Recording System could be independently validated to ensure they provide an accurate assessment of the quality of taxpayer service.
IV. Determined whether the Contact Recording System data could be retained and used for trending and/or analyses.
Appendix II
Major Contributors
to This Report
Michael E. McKenney, Assistant Inspector
General for Audit (Wage and Investment Income Programs)
Augusta
R. Cook, Director
Frank
Jones, Audit Manager
Deborah
Drain, Lead Auditor
Tracy
Harper, Senior Auditor
Robert
Baker, Auditor
Andrea
Hayes, Auditor
Geraldine
Vaughn, Auditor
Gerard Cain, Information Technology Specialist
Appendix III
Acting
Commissioner C
Office of the Commissioner – Attn: Chief of Staff C
Deputy
Commissioner for Operations Support OS
Deputy
Commissioner for Services and Enforcement
SE
Chief,
Agency Wide Shared Services OS:A
Chief
Information Officer OS:CIO
Deputy
Commissioner, Wage and Investment Division
SE:W
Director, Customer Assistance, Relationships, and Education SE:W:CAR
Director, Real Estate and Facilities Management OS:A:RE
Director, Strategy and Finance, Wage and Investment Division SE:W:S
Chief, Performance Improvement, Wage and Investment Division SE:W:S:PI
Director, Field Assistance, Wage and Investment Division SE:W:CAR:FA
Senior Operations Advisor, Wage and Investment Division SE:W:S
Chief Counsel CC
National Taxpayer Advocate TA
Director, Office of Legislative Affairs CL:LA
Director, Office of Program Evaluation and Risk Analysis RAS:O
Office of Internal Control OS:CFO:CPIC:IC
Appendix IV
Contact Recording and the Embedded Quality
Measurement Systems
First. The assistor presses the button on the Q-MATIC System[23] to call the next taxpayer, which signals the recording to begin. Signs are strategically placed in the area after the taxpayer has been called to alert the taxpayer that he or she is being recorded. If the taxpayer declines to be recorded, the assistor will stop the recording by using the Q‑MATIC System “taxpayer opt-out” button to end the recording. If the taxpayer exhibits a learning deficiency, is visually impaired, or has a language barrier, the assistor will either explain or use an interpreter service to explain the purpose of the recording and ask the taxpayer if he or she agrees to continue. If the taxpayer has additional questions or concerns, the assistor will refer him or her to the manager.
Second. The Contact Recording System quality monitoring software captures voice elements of customer contacts and assistors’ corresponding computer desktop activities, including customer account activity. Each recording is categorized by the assistor’s name and Standard Employee Identifier.
Third.
When the taxpayer’s transaction is completed, the
assistor inputs the appropriate Q-MATIC System code, which automatically stops
the recording.
Fourth.
The selected data automatically upload to a server
at the Brookhaven,
Fifth.
The recording remains on the assistor’s computer
until it has been automatically selected and subsequently selected for review, by
either the manager for evaluating employee performance or the algorithm process
for measuring the quality of the
Sixth.
The National Sample[26] routine runs every
Sunday night and marks the recordings with “Sample” to indicate that they are
to be included in the sample. Once selected for
quality review, the recordings made Monday through Friday are downloaded on Sunday. During
the filing season,[27] if the workweek
included Saturday, the recordings made Saturday would also be downloaded on Sunday.
·
If the recording has not been selected for review
within 10 business days by the manager, it is deleted from the assistor’s
computer. If the contact is selected for
review, it will be maintained for 18 months.
·
If the recording is selected by the algorithm
process, the recording remains on the server for 45 calendar days. If the
quality reviewer has not selected the contact within 45 calendar days, the
recording is deleted from the server. If the contact is selected, it
will be maintained for 18 months on the server at the Brookhaven,
Appendix V
Management’s Response to the Draft Report
The response was removed due to its
size. To see the response, please go to
the Adobe PDF version of the report on the TIGTA Public Web Page.
[1] The Contact Recording System captures the audio portion of the assistor/taxpayer interaction, synchronized with computer screen activity, for replay and quality review.
[2] The Taxpayer Assistance Blueprint Phase II report was issued in April 2007.
[3] The National Sample is the designation for statistically selected contacts that are reviewed by the Centralized Quality Review System Office to verify the accuracy of assistor responses to taxpayers.
[4] A secondary code used to count the volume of work and hours performed by an assistor when assisting a taxpayer. It is used to track specific data that cannot be captured directly on the Q-MATIC System, which is an automated queuing system used to control the flow of taxpayers waiting for assistance.
[5] There
are 401 Taxpayer Assistance Centers now, with the addition of
[6] The Contact Recording
System captures the audio portion of the assistor/taxpayer interaction,
synchronized with computer screen activity, for replay and quality review. The Q-MATIC System is an automated queuing system used to control the
flow of taxpayers waiting for assistance.
[7] Improved Internal Controls and Contact Recording Are Needed to Ensure the Accuracy and Reliability of the Taxpayer Assistance Centers Quality Measurement System (Reference Number 2006-40-022, dated December 2005).
[8] The Embedded Quality Measurement System is a set of processes used to measure taxpayer assistance performance against a set of predetermined standards.
[9] The IRS uses three measures (employee satisfaction, customer satisfaction, and business results) to measure organizational and employee performance.
[10] Pub. L. No. 103-62, 107 Stat. 285 (codified as amended in scattered sections of 5 U.S.C., 31 U.S.C., and 39 U.S.C.).
[11] The National Sample is the designation for statistically selected contacts that are reviewed by the Centralized Quality Review System Office to verify the accuracy of assistor responses to taxpayers.
[12] The term employee includes managers and assistors.
[13] Shopping is defined as anonymous visits to Taxpayer Assistance Centers to evaluate the quality of assistor responses to taxpayer tax law questions.
[14] The
estimates for installing the Contact Recording System in the Taxpayer
Assistance Centers include 400 of the 401 Taxpayer Assistance Centers and do
not include the
[15] The sizes of Taxpayer Assistance Centers are determined by the number of assistors assigned. Small Centers have 1 assistor to 4 assistors, medium Centers have 5 assistors to 9 assistors, and large Centers have 10 assistors to 20 assistors.
[16] The Taxpayer Assistance Blueprint Phase II report was issued in April 2007.
[17] Contact Recording System codes include the reason or nature of the taxpayers’ visits to the Taxpayer Assistance Centers and are input by assistors after they have completed their interactions with the taxpayers.
[18] Campuses are the data processing arm of the IRS. The campuses process paper and electronic submissions, correct errors, and forward data to the Computing Centers for analysis and posting to taxpayer accounts.
[19] A secondary code used to count the volume of work and hours performed by an assistor when assisting a taxpayer. It is used to track specific data that cannot be captured directly on the Q-MATIC System.
[20] If the taxpayer declines to be recorded, guidelines require the assistor to stop the recording by using the Q‑MATIC System “taxpayer opt-out” button to end the recording.
[21] An audit trail is a chronological record of system activities that is sufficient to permit reconstruction, review, and examination of a transaction from inception to final results.
[22] The Contact Recording System captures the audio portion of the assistor/taxpayer interaction, synchronized with computer screen activity, for replay and quality review.
[23] The Q-MATIC System is an automated queuing system used to control the flow of taxpayers waiting for assistance.
[24] Campuses are the data processing arm of the IRS. The campuses process paper and electronic submissions, correct errors, and forward data to the Computing Centers for analysis and posting to taxpayer accounts.
[25] IRS employees (called assistors) in the Taxpayer Assistance Centers provide taxpayers personal, face‑to‑face assistance with tax matters that includes interpreting tax laws and regulations, preparing certain individual tax returns, resolving inquiries on taxpayer accounts, accepting payments, and providing various other services designed to minimize the burden on taxpayers in satisfying their tax obligations.
[26] The National Sample is the designation for statistically selected contacts that are reviewed by the Centralized Quality Review System Office to verify the accuracy of assistor responses to taxpayers.
[27] The filing season is January through mid-April of each year, when most individuals file their income tax returns.