Detailed Information on the
National Practitioner and Health Care Integrity/Protection Data Banks Assessment

Adopting regulations and making systems changes implementing Section 1921 of the Social Security Act to minimize the program's identified limitations and to operate at maximum efficiency.

Implementing new national survey to evaluate program effectiveness

Measure: Increase annually use of the NPDB for licensing and credentialing decision-making, operationalized as the number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB reports.

Explanation:This measure involves the number of times that NPDB information obtained as a result of a query led to the querying entity not granting a license or clinical privileges (or granting only a limited license or limited clinical privileges). This number represents the result of the interaction of the number of queries submitted by eligible entities with: 1. The proportion of queries which result in disclosure of reports ("matches" of queries to reports) 2. The number of adverse actions, malpractice payment, and other reports in the data banks; and 3. The degree to which these reports provide new information not otherwise known by querying entities. Experience has shown that the primary factors are the numbers of queries and reports, which drive the number of disclosures. Both data banks count the number of queries, reports, and disclosures. Information gained from surveys of querying entities is used to determine the proportion of disclosures which limit the ability of a practitioner to practice. *The targets for 1999 and 2000 were based on our 1995 national users survey and, as noted above were substantially increased based on results of the later survey. The 2000 target appears not to have been met; however, had the results of the new survey been available, the target would clearly have been exceeded. ** The 2003 and 2005 targets assumed a substantial increase in the number of decisions as a result of the implementation of Section 1921 Regulations, which which would have shifted queries from the HIPDB to the NPDB; however, this regulation was not implemented. Actual Performance demonstrates continued increase despite what proved to be an unwarranted target.

Measure: Increase annually the use of the HIPDB for licensing and fraud/abuse related decision-making, operationalized as the number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in HIPDB reports.

Explanation:This measure involves the number of times that HIPDB information obtained as a result of a query led to the querying entity not granting a license or clinical privileges (or granting only a limited license or limited clinical privileges). This number represents the result of the interaction of the number of queries submitted by eligible entities with: 1. The proportion of queries which result in disclosure of reports ("matches" of queries to reports) 2. The number of adverse actions, malpractice payment, and other reports in the data banks; and 3. The degree to which these reports provide new information not otherwise known by querying entities. Experience has shown that the primary factors are the numbers of queries and reports, which drive the number of disclosures. Both data banks count the number of queries, reports, and disclosures. Information gained from surveys of querying entities is used to determine the proportion of disclosures which limit the ability of a practitioner to practice. * The 2003 and 2005 targets assumed a substantial decrease in the number of decisions affected as a result of the implementation of Section 1921 Regulations, which which would have shifted queries from the HIPDB to the NPDB; however, this regulation was not implemented.

Measure: Increase annually the number of times information provided by the NPDB is considered useful by the querying entity which received it.

Explanation:

Measure: Increase annually the number of times information provided by the HIPDB is considered useful by the querying entity which received it.

Explanation:

Measure: Increase annually the number of queries for NPDB and HIPDB responded to within 240 minutes.

Explanation:06: maintain a response time within 240 minutes with 4,611,000 queries 07: maintain a response time within 240 minutes with 08: maintain a response time within 240 minutes with

Is the program purpose clear?

Explanation: The purpose of the NPDB is to ensure that licensing and credentialing authorities have accurate and complete information about practitioners' past damaging or incompetent performance when making decisions to license or privilege individual health care practitioners. The information collected and distributed allows licensure and privileging officials to make more informed decisions and, thereby, improve the quality of health care by eliminating or restricting the ability of incompetent or miscreant practitioners to practice. The purpose of the HIPDB is to make information related to health care fraud and abuse activities by practitioners, providers, and suppliers available to law enforcement, licensing, and health plan officials. These officials can use the information to avoid dealing with fraudulent or abusing practitioners, providers, and suppliers or to assist in their prosecution, thus helping to reduce fraud and abuse and improve quality of health care. The HIPDB was established as part of the fraud and abuse control efforts in the Health Insurance Portability and Accountability Act of 1996.

Evidence: Section 402 of the Healthcare Quality Improvement Act specifies the purpose of the NPDB. Section 221(a) of the Health Insurance Portability and Accountability Act of 1996 specifies the purpose of the HIPDB.

Does the program address a specific and existing problem, interest, or need?

Explanation: Concerning the NPDB, licensing and credentialing authorities continue to require a reliable source of information to confirm and augment information submitted by applicants for licensure and clinical privileges to ensure that "previous damaging or incompetent performance" is disclosed. Higher standards for verification of credentials, when combined with the increasing mobility of the health care workforce mean that it is more important than ever that an authoritative source of verification information is available to ensure that past damaging or incompetent performance is discovered before a practitioner is licensed or granted privileges. Concerning the HIPDB, law enforcement and licensing and health plan officials have a need for adverse action and fraud and abuse-related information on practitioners, providers, and suppliers to help them make more informed licensure, contracting, or prosecutorial decisions. Estimates of the level of losses related to fraud and abuse in health care range from $50,000,000,000 to $100,000,000,000 each year. Medicare and Medicaid fraud and abuse have been estimated at $33,000,000,000 a year. Estimates of Medicare fraud and abuse range up to 14 percent of all Medicare expenditures.

Evidence: Ten years after the NPDB became operational, a survey of licensing and credentialing authorities by the University of Illinois at Chicago and Northwestern University found that information provided by the NPDB was very influential in decision-making in over two-thirds of the cases in which licensing or credentialing authorities received reports of previous adverse actions or malpractice payments. There is also strong evidence that practitioners continue to have licensure and disciplinary actions taken against them and that they continue to be responsible for incidents which lead to malpractice payments. During 2005, there were over 4,000 State licensure actions, over 900 clinical privileges actions, over 1,200 Medicare/Medicaid exclusion actions, and over 17,000 malpractice payments reported to the NPDB. During 2005 the HIPDB received almost 1,400 reports of healthcare-related criminal convictions. In addition the HIPDB received almost 26,000 State licensure action reports, and almost 4,000 other reports.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There are no other programs in either the public (Federal, State or local), non-profit or private sectors that collect the range of information collected by the NPDB and the HIPDB. The stated purpose of the Data Banks, as articulated in their authorizing legislation, clearly identified the need for national systems to collect this type of information because a similar or duplicative system did not exist.

Evidence: Other than the NPDB, no national data collection system for all malpractice payments exists; malpractice insurers generally release this competitive information only as required by law. Similarly, other than the NPDB and HIPDB, no national repository of clinical privileges, health plan action, or professional society membership information exists. There are systems which contain voluntarily submitted and sometimes not comprehensive State licensure information for some types of practitioners. These systems are generally operated by federations of State licensing boards for a few professions. They do not normally contain information on malpractice payments, clinical privileges, professional society membership actions, civil judgments, criminal convictions, Medicare and Medicaid exclusions, etc.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The current design of both programs, NPDB and the HIPDB, includes the use of a contractor to administer the NPDB and HIPDB computer operations and the use of Federal staff to develop policy and manage the programs. It has been determined that the existing design of the program maximizes program efficiency and effectiveness as it allows for continual quality improvement in the functioning and responsiveness of databanks. Unlike the NPDB, the HIPDB statute does not allow (or require) hospitals to query its data banks, resulting in a dramaticly lower amount of user fees for the program, relative to those received by the NPDB. Federal government agencies, such as CMS and HHS OIG, are not required to pay the user fees for querying the HIPDB as they must do when querying the NPDB. This also results in decreased income for the HIPDB. HIPDB's program managers and contractor's have been able to keep the HIPDB functioning and achieving its goals and objectives, in part due to efforts that have increased data bank's efficiency over the past several years.

Evidence: Aside from the design flaw within the statute of the HIPDB, independently conducted surveys for both data banks have shown adequate to high levels of customer satisfaction with the systems.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: All NPDB revenue comes directly from NPDB queriers who submit and pay for queries (requests for copies of NDPB reports). Similarly, almost all HIPDB revenue comes directly from HIPDB query fees, although a small amount comes from the Health Care Fraud and Abuse Control (HCFAC) account to compensate for free queries from federal agencies (by law). No funds are appropriated for either program. All revenues are used for the NPDB and HIPDB programs' purposes and support the operation of the NPDB and HIPDB. The NPDB and HIPDB do not make any grants or otherwise subsidize any programs. Targeted "beneficiaries" of the NPDB and HIPDB programs are 1) the legally authorized queriers and reporters; 2) individual practitioners, providers, and suppliers, who are permitted to obtain a copy of their own NPDB and/or HIPDB record; and 3) researchers, who may obtain statistical data.

Evidence: Eligible reporters and queriers ("beneficiaries") are specified in law and regulations. The enabling statutes also require that the programs be funded from user fees. Eligible queriers submit over 3,500,000 queries each year to the NPDB and over 900,000 queries each year to the HIPDB. They also file over 25,000 reports to the NPDB and over 33,000 reports to the HIPDB each year. Researchers download over 3,000 copies of the NPDB Public Use Data File each year.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has developed two long-term measures.

Evidence: Long Term Measure 1: Increase the annual number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports. Long Term Measure 2: Increase the annual number of times information provided by the NPDB and HIPDB is considered useful by the querying entity which received it.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Data Banks programs' have ambitious targets and timeframes for their long-term measures (please see evidence section). Three experts within the medical credentialing field have attested to the ambitiousness of these targets: 1) Ms. Jodi Schirling, CPMSM, Manager of Corporate Credentialing for Nemours, past-President of the National Association of Medical Staff Services, and also Chair of the National Practitioner Data Bank's "Executive Committee" (a voluntary committee composed of industry representatives) stated: "I have reviewed the baseline and target data used in the OMB Program Assessment and Review Tool for the NPDB and HIPDB. I have been in the credentials field for almost 30 years. Based on my experience in credentialing, and as a Past President of the National Association Medical Staff Services, I think the data is meaningful and the targets are ambitious." 2) Christina W. Giles, CPMSM, MS, President of Medical Staff Solutions and a partner in Edge-U-Cate, LLC, stated "I have reviewed the baseline and target data used in the OMB Program Assessment and Review Tool for the NPDB and HIPDB. I have been in the credentials/medical staff services administration area for over 25 years. Based on my experience in the field of credentialing and my work experience as a consultant for the past ten years working with medical staffs across the country, and as faculty/teacher of this field for 20+ years, I would agree that the data identified is realistic and meaningful and the targets for the coming years are ambitious." 3) Susan J. Freeburn, RN, Director of the Credentials Verification Program of the Armed Forces Institute of Pathology's (AFIP) Department of Legal Medicine, and a credentialing leader for the Department of Defense, stated: "I have reviewed the data attached for the NPDB-HIPDB programs targets for long-term measures and find them both ambitious with reasonable timeframes. Looking at the past performance and projecting to the future, your goals look to be attainable. "As the Director of the AFIP, Department Legal Medicine's CVO for the past five years, we have witnessed the effect of a positive NPDB or HIPDB with the agencies that we are contracted to do PSV. The Data Bank's information has been an invaluable tool for finding malpractice and unethical behaviors in practitioners in all walks of the medical field."

Evidence: Long Term Measure 1: The annual number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports. Baseline: Based on matched query responses (not reports) NPDB HIPDB 2005 44,500 Decisions 1,120 Decisions Target for FY 2013: NPDB HIPDB 2013 48,700 Decisions 1,400 Decisions Long Term Measure 2: The annual number of times information provided by the NPDB and HIPDB is considered useful by the querying entity which received it. Baseline: NPDB HIPDB 2005 451,400 Useful Disclosures 11,400 Useful Disclosures Target for FY 2013: NPDB HIPDB 2013 489,000 Useful Disclosures 14,200 Useful Disclosures

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has developed two annual measures which are identical to its long-term measures except that data is collected, and progress toward achieving their long-term targets is monitored, annually.

Evidence: Annual Measure 1: The annual number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports: Annual Measure 2: The annual number to times information provided by the NPDB and HIPDB was considered useful by the querying entity which received it.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Data Banks programs' have ambitious targets and timeframes for their long-term measures. Three experts within the medical credentialing field have attested to this ambitiousness of these targets: 1) Ms. Jodi Schirling, CPMSM, Manager of Corporate Credentialing for Nemours, past-President of the National Association of Medical Staff Services, and also Chair of the National Practitioner Data Bank's "Executive Committee" (a voluntary committee composed of industry representatives) stated: "I have reviewed the baseline and target data used in the OMB Program Assessment and Review Tool for the NPDB and HIPDB. I have been in the credentials field for almost 30 years. Based on my experience in credentialing, and as a Past President of the National Association Medical Staff Services, I think the data is meaningful and the targets are ambitious." 2) Christina W. Giles, CPMSM, MS, President of Medical Staff Solutions and a partner in Edge-U-Cate, LLC, stated "I have reviewed the baseline and target data used in the OMB Program Assessment and Review Tool for the NPDB and HIPDB. I have been in the credentials/medical staff services administration area for over 25 years. Based on my experience in the field of credentialing and my work experience as a consultant for the past ten years working with medical staffs across the country, and as faculty/teacher of this field for 20+ years, I would agree that the data identified is realistic and meaningful and the targets for the coming years are ambitious." 3) Susan J. Freeburn, RN, Director of the Credentials Verification Program of the Armed Forces Institute of Pathology's (AFIP) Department of Legal Medicine, and a credentialing leader for the Department of Defense, stated: "I have reviewed the data attached for the NPDB-HIPDB programs targets for long-term measures and find them both ambitious with reasonable timeframes. Looking at the past performance and projecting to the future, your goals look to be attainable. "As the Director of the AFIP, Department Legal Medicine's CVO for the past five years, we have witnessed the effect of a positive NPDB or HIPDB with the agencies that we are contracted to do PSV. The Data Bank's information has been an invaluable tool for finding malpractice and unethical behaviors in practitioners in all walks of the medical field."

Evidence: Annual Measure 1: The annual number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports. Baseline: NPDB HIPDB 2005 44,500 Decisions 1,120 Decisions Annual performance targets for FY 2006 - 2008 are: NPDB HIPDB 2006 45,025 Decisions 1,155 Decisions 2007 45,550 Decisions 1,190 Decisions 2008 46,075 Decisions 1,225 Decisions Annual Measure 2: The annual number to times information provided by the NPDB and HIPDB is considered useful by the querying entity which received it. Baseline: NPDB HIPDB 2005 451,400 Useful Disclosures 11,400 Useful Disclosures Annual performance target for FY 2006 - 2008: NPDB HIPDB 2006 456,100 Useful Disclosures 11,750 Useful Disclosures 2007 460,800 Useful Disclosures 12,100 Useful Disclosures 2008 465,500 Useful Disclosures 12,450 Useful Disclosures

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The NPDB- HIPDB operating contractor is the only formal partner. The contractor's performance is monitored with weekly and monthly reports. Performance measures are established in the contract, and the contractor is rewarded with bonus payments if performance targets are exceeded and fined if performance targets are not met. The contractor is dedicated to meeting or exceeding the performance targets.

Evidence: In general, over the past five years the contractor has met performance goals. Measures and standards for performance goals include (1) a website for reporting and querying accessible and functioning at least 94 percent of the time (less than 100 percent to allow for weekend downtime for maintenance and improvements), (2) an average response time for computer matched queries of no more than 3.75 hours, and (3) an erroneous disclosure rate for queries on practitioners of no more than 0.15 percent. Six other measures are also used, including one for "customer (i.e., PDBB) satisfaction." If the contractor's performance is below the standard, money is subtracted from the performance incentive award. If the contractor substantially exceeds the standard, money is added to the performance incentive award. An example of when the contractor did not meet expectations and PDBB's action concerns the second quarter in FY '03, when several research-related products were delivered which did not meet government quality expectations for accuracy and had to be redone by the contractor after PDBB discovered data errors. As a result, PDBB reduced the amount that the contractor receives as a Performance Fee Incentive Award for that quarter. Performance metrics include up-time for the querying and reporting web site, response time for query responses, time for human resolution of doubtful matches, time for report processing notification, erroneous disclosure rates, customer service center telephone wait time, and customer satisfaction.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The NPDB and HIPDB regularly conduct independent, unbiased, scientific evaluations of the programs. Approximately every 5 to 6 years since its opening, the NPDB has contracted with an independent researcher to conduct national scope surveys of NPDB reporters and queriers. The surveys are designed to assess user satisfaction with the NPDB, to determine how NPDB information is being used for decision-making, and to determine what kind of program improvements might be beneficial. The most recent of these surveys was completed in 2001 and was conducted jointly by Northwestern University Institute for Health Services Research and Policy Studies and the University of Illinois at Chicago Health Policy Center Survey Research Laboratory. The next survey will be conducted beginning in 2006. In the interim, the NPDB and HIPDB have contracted for independent American Customer Satisfaction Index surveys. The HIPDB ACSI survey was conducted in 2002 and the NPDB ACSI survey was conducted in 2003. The HIPDB survey found reporter satisfaction at "68" and querier satisfaction at "76." The average for all federal programs was "70.2" and for all private sector programs was "73.1." The NPDB survey found that satisfaction for queriers was "78" and for reporters was "76." The federal agency average that year was "70.9" and the private sector average was "73.8." The Data Bank programs are among the highest rated federal government programs. The program expects to again conduct ACSI surveys in 2010 or 2011.

Evidence: The NPDB and HIPDB programs' evaluations, past and future, are as follows: 2001: Assessment of overall satisfaction with reporting and querying processes of the National Practitioner Data Base (NPDB) - by Northwestern University's Institute for Health Services Research and Policy Studies and University of Illinois at Chicago's Health Policy Center, Survey Research Lab. 2002: HIPDB American Customer Satisfaction Index Survey (ACSIS) 2003: NPDB American Customer Satisfaction Index Survey (ACSIS) 2006: Award of contract for large national sample survey of NPDB and HIPDB users and non-users. Final report expected in 2008. 2010 - 2011: American Customer Satisfaction Index Surveys of NPDB and HIPDB users. 2012: Award of contract for large scale national sample survey of NPDB and HIPDB users and non-users. Final report expected in 2014. This schedule is a continuation of the schedule for national surveys and ACSI surveys previously conducted for the NPDB and later the HIPDB.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Within the Congressional Justification document both Data Banks programs provide a description of the total costs associated with operating their respective data banks. However, the programs do not explicitly tie their budgets, each of which are funded almost entirely through user fee collections, to their ability to accomplish their annual and long-term goals. The relationship between the two programs' annual and long-term targets and their user fee levels is not clear.

Evidence: The budget justifications for the Data Banks programs' activities are included in the Health Resources Administration Fiscal Year 2007 Justification of Estimates for Appropriation Committees.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: No strategic planning deficiencies have been identified in the last five years.

Evidence:

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The agency regularly collects information for program management and improvement purposes. This information includes: reporting and querying volumes, fee payments, calls to the NPDB and HIDPB Customer Assistance Center, timeliness of processing, and operating problems. In addition, program and contract officials solicit comments from stakeholder and customer organizations as well as users on program performance and policies and provide suggestions for improvements. The information collected is used to adjust program priorities, allocate resources, or other appropriate management actions. The program has also obtained baseline data to set meaningful targets for its annual and long term performance measures. These data are obtained from surveys conducted approximately every five years to assess user satisfaction with the program and its operations and to learn what could be improved.

Evidence: Based on the information collected from these various mechanisms, the program has made numerous management improvements including improved electronic data collection "forms", improvements to the Internet-based querying and reporting processes, and modifications and clarifications of Data Bank policies.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The NPDB and HIPDB have a formal schedule for system enhancements as reflected by new system software and web releases planned for specific dates. The NPDB and HIPDB operating contract establishes quantitative performance metrics which are evaluated quarterly, via the Quarterly Performance Metrics Report. Penalties are assessed or bonuses are awarded to the contractor depending on compliance with the standards established in the contract. Government Managers operate under a pass/fail system that clearly outlines their specific duties. Modest financial rewards also exist for outstanding performance. Annual and bi-annual reviews are conducted to ensure acceptable standards of performance are being met. Both the NPDB-HIPDB Program and Project Managers have and continue to receive "Outstanding" performance rating for the overall management of the NPDB-HIPDB program.

Evidence: 1) The Quarterly Performance Metrics Report This report contains two parts: Part I - contains the performance summary that lists each metric and the contractor's level of performance for the quarter. It also contains a brief written explanation of any factors that caused performance to fail to meet expectations. Part II - contains a breakdown, by metric, of the incentive payment due and penalties assessed for the quarter. 2) HHS Performance Evaluations Plan

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The NPDB and HIPDB programs are operated from user fee collections. These user fees are primarily used to operate, maintain, and enhance these Data Banks. User query payments are made via Credit Card or Electronic Funds Transfer (EFT) through a U.S. Department of Treasury contract with Mellon Bank. Program Management monitors on a weekly basis NPDB and HIPDB financial transactions, and on a monthly basis performs the financial reconciliation between Mellon Bank and the contractor. In addition, all program income (i.e., query fee collections) and expenses (i.e., administrative costs) information is reviewed and audited on a monthly basis.

Evidence: It is the responsibility of the NPDB-HIPDB Project Manager to manage the day-to-day financial efforts of NPDB and HIPDB Operations (i.e., contractual obligations) and Administration (i.e., program staff). The NPDB has several processes in-place to track how the user fee income is managed and spent. 1. On a monthly basis a "Statement of Income and Expenses" report is generated for the NPDB and HIPDB. This report shows monthly contract and Government administrative costs and User Fee Income collected. Every line item cost that is recorded during that given month is reviewed for validity before it is incorporated into the cumulative monthly reports. 2. There also exists a "NPDB Expenditures & Funding Requirements" document that outlines all contractual and administrative financial obligations for past, present, and future NPDB requirements. Periodically, these reports are briefed to HRSA Management and the Data Banks stakeholders.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The NPDB and HIPDB Program Staff is continuously looking for methods for improving system efficiencies and reducing operational costs. One significant approach that is used is to insist on reliable, repeatable, and managed processes to ensure that the system and any improvements are designed and developed with high quality are on schedule and are within budget. To date, several improvements in efficiency and cost effectiveness have been implemented (see evidence for a list of these improvements). In addition, the program has an efficiency measure -- The length of time it takes the NPDB and HIPDB to process a query and return results to the querier. The challenge to the NPDB and HIPDB is to maintain query response time within the 240 minute threshold even though the task of matching queries to reports becomes more difficult each year as both the number of queries and the number of reports in the NPDB and HIPDB continue to increase substantially each year. Data Banks' efficiency measure: Produce increasing amounts of output in the same amount of time. Numbers of queries responded to within 240 minutes Baseline 2005 4,414,000 queries 2006 Target: 4,611,000 queries 2007 Target: 4.701,000 queries 2008 Target: 4,792,000 queries

Evidence: Examples of cost savings and improved efficiencies that have taken place during the management of the NPDB and the HIPDB include: 1. Cost savings resulting from the elimination of a dedicated electronic mailbox system and communications contract. Users no longer need to e-mail queries and reports through a dedicated network service provider paid for by the NPDB. NPDB and HIPDB customers now use the Internet for all data bank communications at no additional cost to them beyond their normal Internet connection charge and at a significant communications cost savings to the NPDB and HIPDB. 2. Cost savings resulting from virtual elimination of paper processing. The Internet-based system was redesigned to virtually eliminate paper processing. Users can now not only query and report electronically, they can also update their registration information on-line, retrieve electronic billing information, access documentation and help files, and perform simplified self-queries. 3. Cost savings from all electronic payment. The system provides for query payment only by credit card or Electronic Funds Transfer (EFT). This eliminates the need to accept and process checks, and has significantly reduced both costs and the need to collect unpaid checks. 4. Improved accuracy of information reported and submitted for query match. The web interface enables improved validation of information submitted to the Data Banks. Validation rules are in effect for both queries and reports. Improved validation helps to ensure that all required information is provided and properly formatted before a report or query can be submitted. This improves report data quality.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Since by law there is some overlap in the information collected by the National Practitioners Data Bank and the Health Care Integrity Data Banks, as well as some overlap in the organizations that provide or access this information, efficient collaboration and coordination is between the two Data Banks is paramount. The NPDB supplies computer operations for the HIPDB through the NDPB operating contract. Reporters required to report file only one report which is automatically placed in the correct data bank or banks as required based on the information they supply in the report. Reporters do not have to have a detailed understanding of the laws' requirements to ensure that they report to the correct data bank. Queriers eligible for both data banks also may submit only one query which is routed to both data banks. In addition, program management staff operate the "Secretarial Review" process for resolving disputed reports jointly for the two programs, applying separate regulations as necessary. Practitioners with reports in both data banks need file only one dispute to open a case relative to both data banks. Operating metrics used to assess contractor performance are also assessed jointly for both data banks. System improvements for one data bank also are implemented for the other data bank. To the limited extent that there are programs elsewhere in the federal government, State and local governments as well as the private sector, the Data Banks work well with these programs. For example, in the federal sector, the HHS Office of Inspector General (OIG) collects and discloses actions it takes against practitioners authority to bill the Medicare and Medicaid programs, through an on-line database. These actions also are reported to the Data Banks by the OIG. The Data Banks set up an efficient system for the OIG to report this information to the Data Banks using the same database the IG uses for its systems. In another example, the Data Banks work with the National Association of State Insurance Commissioners (NAIC) to compare its statistical data to NPDB statistical data on malpractice payments to assess reporting compliance.

Evidence: 1) The electronic reporting interface does not ask reporters eligible to report to both the NPDB and the HIPDB which data bank they are reporting to. Instead it automatically files reports with the correct data bank or data banks depending on the information in the report being submitted. 2) The NPDB and HIPDB now contain almost 31,000 Medicare/Medicaid exclusion reports. 3) The NPDB routinely purchases annual "Supplement A to Schedule T" data from the NAIC to compare to NPDB malpractice payment reporting information.

Does the program use strong financial management practices?

Explanation: In 2005, HHS received a material control weakness for its financial systems and processes. HRSA contributes to the material internal control weakness identified in the 2005 HHS audit. HHS is in the process of resolving these weaknesses by replacing existing accounting systems within HHS with the Unified Financial Management System (UFMS). UFMS is scheduled to be operational for HRSA in October 2006.

Evidence: Since 2003, HRSA has been not been included in a consolidated HHS audit. In a 2005 audit of HHS, Ernest and Young found a material weakness in HHS financial systems and processes. In particular, the audit found: Documentation regarding significant accounting events, recording of non-routine transactions and post-closing adjustments, as well as correction and other adjustments made in connection with data conversion issues must be strengthened. Processes to prepare financial statements need improvement. Financial systems are not FFMIA compliant. Weaknesses were identified in Department/Operating Division Periodic Analysis, Oversight and Reconciliations In addition, the audit found PSC's DFP CORE accounting system, which supports the activities of HRSA, did not facilitate the preparation of timely financial statements and did not have an efficient mechanism in place to compile accounting statements.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Unified Financial Management System (UFMS) will improve funds control and monitoring and provide real-time data. In addition to streamlining the accounting process, HHS monitors funds received through annual Independent Financial Audits from grantees.

Evidence: To address management deficiencies, HRSA developed a baseline assessment of grantees to provide information about the overall strengths and weaknesses within the program. In 2005, HRSA implemented a web-based data collection system through the Electronic Handbook on the HRSA GEMS site to improve the data quality and elements collected. HRSA also held a TA conference call with consultants presenting elements of health care and business plans to incorporate program planning and provide HRSA program staff concrete information for grantee goals.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The historical performance data collected by the program for both measures supports the conclusion that the NPDB and HIPDB will reach their long term goals. No targets were previously adopted for the Useful Disclosures measure shown in this PART review. Targets for "decisions affected" have been modified over the years based on results of the most recent survey of users and delays in implementation of Section 1921 regulations, which will likely shift queries from the HIPDB to the NPDB. Targets for Decisions Affected were established for the NPDB as early as for FY '99 (before the HIPDB was opened) and were based on results from the NPDB's first national survey of users. The first target goal was 10,400 decisions affected. The target was substantially adjusted upward for FY '01 and beyond based on results of the NPDB's second national user survey and querying volume targets. Historical data that indicates that both data banks have made good progress in achieving its first long-term goal, "Increasing the number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports" since 1999: Year NPDB NPDB HIPDB HIPDB Decisions Affected Actual Decision Affected Actual Target Performance Target Performance 1999 10,400 10,800 n/a n/a 2004 42,700 43,800 800 1,000 2005 48,600* 44,500 560* 1,120 2013 48,700 1,400 *Target assumed substantial increase based on implementation of Section 1921 Regulations, which did not happen. Actual Performance shows continued increase despite what proved to be an unwarranted target. Historical data that indicates that both data banks have made good progress in achieving its second long-term goal "Increasing the number of times information provided by the NPDB and HIPDB is considered useful by the querying entity which received it" since 2001: NPDB HIPDB 2001 ACTUAL 393,100 Useful Disclosures 8,300 Useful Disclosures 2005 ACTUAL 451,400 Useful Disclosures 11,400 Useful Disclosures 2013 TARGET 489,000 Useful Disclosures 14,200 Useful Disclosures

Evidence: 1) Decisions affected targets (long-term/annual measure 1) for NPDB and HIPDB in 2005 were set based on proposed schedules for adoption of Section 1921 regulations. However, the regulations were not adopted as planned. Because of the assumed implementation of the Section 1921 regulations, the 2005 target for the NPDB was set much too high and the 2005 target for the HIPDB was set too low. In fact, actual 2005 performance of both data banks continued to show healthy improvement. American Customer Satisfaction Index Results for the HIPDB 2) American Customer Satisfaction Index Results for the NPDB

Does the program (including program partners) achieve its annual performance goals?

Explanation: The historical performance data collected by the program for both measures supports the conclusion that the NPDB and HIPDB have largely achieved their annual performance goals. Targets for "decisions affected" have been modified over the years based on results of the most recent survey of users and delays in implementation of Section 1921 regulations, which will likely shift queries from the HIPDB to the NPDB. Targets for Decisions Affected were established for the NPDB as early as for FY '99 (before the HIPDB was opened) and were based on results from the NPDB's first national survey of users. The first target goal was 10,400 decisions affected. The target was substantially adjusted upward for FY '01 and beyond based on results of the NPDB's second national user survey and querying volume targets. Historical data indicating that both data banks have largely achieved their goals for their first annual measure, "Increasing the number of licensing and credentialing decisions which limit practitioners' ability to practice because of information contained in NPDB and HIPDB reports" over the past 7 years: Year NPDB NPDB HIPDB HIPDB Decisions Affected Actual Decision Affected Actual Target Performance Target Performance 1999 10,400 10,800 n/a n/a 2000 13,350 11,050 700 675 2001 38,000 38,700 1,000 820 2002 39,750 39,800 836 810 2003 47,385* 39,900 1,200 850 2004 42,700 43,800 800 1,000 2005 48,600* 44,500 560* 1,120 *Target assumed substantial increase based on implementation of Section 1921 Regulations, which did not happen, (therefore the targets for NPDB where higher and those for HIPDB lower). Actual Performance shows continued increase despite what proved to be an unwarranted target. Historical data indicating that both data banks have largely achieved their goals for their second annual measure, "Increasing annually the number of times information provided by the NPDB and HIPDB is considered useful by the querying entity which received it" over the past 5 years: NPDB HIPDB 2001 393,100 Useful Disclosures 8,300 Useful Disclosures 2002 403,600 Useful Disclosures 8,200 Useful Disclosures 2003 404,500 Useful Disclosures 8,600 Useful Disclosures 2004 444,200 Useful Disclosure 10,200 Useful Disclosures 2005 451,400 Useful Disclosures 11,400 Useful Disclosures The program had not created targets for this measure prior to this year; however their annual data on this measure does indicate that both data banks have made yearly progress on this measure since 2001. (The only indication of a lack of progress came for the HIPDB in 2003, but this was reversed in 2004 and 2005.)

Evidence: 1) Decisions affected targets (long-term/annual measure 1) for the NPDB in 2003 and the NPDB and HIPDB in 2005 were set based on proposed schedules for adoption of Section 1921 regulations. However, the regulations were not adopted as planned. Because of the assumed implementation of the Section 1921 regulations, the 2005 target for the NPDB was set much too high and the 2005 target for the HIPDB was set too low. In fact, actual 2005 performance of both data banks continued to show healthy improvement. 2) American Customer Satisfaction Index Results for the NPDB

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Over the past year, the NPDB and the HIPDB have achieved improved efficiencies and cost effectiveness. The Data Banks are financed by user fees paid by queriers. There are no appropriations for either the NPDB or HIPDB. Since over 2/3 of NPDB queries and all HIPDB queries are submitted voluntarily, the NPDB and HIPDB must be operated efficiently in order to keep user fees affordable for queriers. Query fees have been raised and lowered over the years to reflect the cost of operating the Data Banks, but current query fees are less than half of the previous highest level. In addition, both the NPDB and the HIPDB have been able to achieve the efficiency measure of maintaining the length of time it takes the databanks to process a query and return results to the querier. For both the NPDB and the HIPDB, the amount of time was 240 minutes despite an increase since 2003 of almost 10 percent in the annual number of queries. Data Banks' Efficiency Measure with historical data: Produce increasing amounts of output in the same amount of time. 1991: 810,000 queries (NPDB only) (within 10 to 60 days of submission of a query) Numbers of queries (NPDB and HIPDB) responded to within 240 minutes 2003: 4,044,000 queries responded to within 240 minutes 2004: 4,329,000 queries responded to within 240 minutes 2005: 4,414,000 queries responded to within 240 minutes

Evidence: By law hospitals are required by Section 425 of the Health Care Quality Improvement Act of 1986 (42 USC 11135) to query the NPDB on all new applicants and once every two years on all practitioners with medical staff privileges. All other queriers, who submit over 2/3 of the queries to the NPDB and all HIPDB queries, have no legal requirement to query. Various enhancements throughout each Fiscal Year enable the Data Banks to use improvements in information technology to fulfill their goals. For example, over the past year, the NPDB and HIPDB have achieved improved efficiencies and cost effectiveness through (1) increasing the quality of the report data in the system while reducing the effort incurred by users to submit and maintain reports by eliminating duplicate reports, improving report/query matching processes, and implementing industry standard XML data transfer as a supplement to the pioneering data transfer format implemented before the industry standard XML was developed, (2) eliminating or further reducing the already minimal use of paper in the NPDB and HIPDB's processes, (3) adding upgraded web browser support (4) enhancing system security to implement the results of a HRSA OIT audit and to follow the guidance of NIST Special Publication 800-53 and the FISMA, and (5) implementing pay.gov, a more robust credit card transaction processing system than was previously used.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The NPDB and HIPDB are the only programs collecting and disseminating malpractice payment and adverse licensure, privileges, membership, judgment, and other adverse action information on practitioners, providers, and suppliers in the US. There are no other comparable programs, including government, private or non-profit. Information on Federal exclusion actions can be obtained, although in a labor intensive way, from the Federal Register and the OIG web site. For clinical privileges actions, professional society membership actions, malpractice payments, State exclusion actions, and adjudicated actions, information could possibly be obtained from the entity which took the action if the querying entity knows where to look and is willing to spend considerable time and effort gathering the information. This generally requires the practitioner of interest to disclose his or her past history. The failure of some practitioners to honestly reveal this information, of course, is a primary reason Congress established the data banks. For some professions State board organizations, such as the Federation of State Medical Boards, collect and make available information on licensure sanctions of licensed practitioners. They do this under varying circumstances and at varying cost. The amount of information available to licensing and credentialing authorities depends to a large degree on individual State laws. However, this information is limited to information on actions taken in the individual State. The Data Banks are the only source of this information on a national basis, which is necessary as health care providers today often move to, and look for work in, different States. As a practical matter there is no single alternative source to the data banks for the information they contain.

Evidence: American Customer Satisfaction Index Results for the HIPDB

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The NPDB has regularly sponsored independent, scientific evaluations of the programs involving national scope surveys of NPDB customers (reporters and queriers). The most recent survey was completed in 2001 and was conducted jointly by Northwestern University Institute for Health Services Research and Policy Studies and the University of Illinois at Chicago Health Policy Center Survey Research Laboratory. The independent evaluation results demonstrate that the NPDB is effective in achieving the desired results. Key findings were that 91.76 percent of queriers referring to a specific sampled report they had received from the NPDB on a practitioner found that information provided by the NPDB in the report was useful in their licensing and/or privileging consideration of that practitioner. For 57.41 percent of the reports received from the NPDB, the report was considered "useful" because it was the querier's "basic source for malpractice payment, licensure, clinical privileges, and exclusion information." For 9.04 percent of the reports, the queriers "decision regarding the practitioner [would] have been different if [the entity] had not received the NPDB response." This represents a significant percentage given the nature of the adverse information in the NPDB, because it indicates that almost 1 out of 10 practitioners who otherwise would have escaped notice of their previous bad acts or malpractice were discovered because of NPDB reports and subsequently were not licensed or privileged or only granted restricted licenses or privileges explicitly because of the NPDB's information, just as Congress intended. PDBB contracted with the independent American Customer Satisfaction Index (ACSI) program to conduct a standard ACSI user satisfaction survey for the HIPDB in 2002. The ACSI survey provided useful information and allowed the program to compare satisfaction with the HIPDB to the NPDB and to public satisfaction with other federal and private sector programs. It also allowed PDBB to identify major areas on which improvement efforts should be focused. The ACSI survey, however, is a relatively standardized, limited sample, limited question survey. It is not designed to provide detailed information on whether individual HIPDB matched query responses made a difference in decision making or on how information from specific matched query responses was used. Such detailed specific HIPDB information will be gathered in the national user survey of NPDB and HIPDB users which is scheduled for completion in mid-2008.

Evidence: 2001: Assessment of overall satisfaction with reporting and querying processes of the National Practitioner Data Base (NPDB) - by Northwestern University's Institute for Health Services Research and Policy Studies and University of Illinois at Chicago's Health Policy Center, Survey Research Lab. 2002: HIPDB American Customer Satisfaction Index Survey (ACSIS) 2003: NPDB American Customer Satisfaction Index Survey (ACSIS) 2006: Award of contract for large national sample survey of NPDB and HIPDB users and non-users. Final report expected in 2008.