SECTION I: INTRODUCTION

Efforts to modernize the American health care system have accelerated over the last five years, due in large part to several landmark studies revealing the startling toll of medical and medication errors. In 1999, the Institute of Medicine estimated that as many as 7,000 people die each year from medication errors alone, accounting for 1 out of 131 ambulatory deaths.[i] In hospitals, the average patient is subjected to at least one medication error per day.[ii] A recent study by the Center for Information Technology Leadership showed that 8.8 million Adverse Drug Events (ADEs) occur each year in ambulatory care. Also from this study came the even more troubling statistic that one quarter, or 3 million, of these errors were preventable.[iii]

Medication-related errors cost money in addition to costing lives. Preventable ADEs occurring in hospitals cost the American health care system $3.5 billion per year, [iv] while those in ambulatory settings amount to upwards of $887 million.[v] In addition, a paper-based system may create costs due to inefficient workflows.[vi] For example, illegible handwriting is a widespread problem that not only causes errors but also uses staff time to determine the physician’s intent. For this and other reasons, almost 30% of prescriptions require pharmacy call backs, resulting in 900 million prescription-related telephone calls annually.[vii]

In order to address these concerns with safety and efficiency, scholars, health experts, and industry leaders have supported the switch from a paper to an electronic system of prescribing. E-prescribing is considered safer^{^[viii]} because it ensures that meaningful and relevant data are communicated to the people who need it, when they need it.[ix]^,[x] For pharmacists, e-prescribing can better communicate the prescriber’s intent, eliminating issues with illegible handwriting or confusing directions. For prescribers, e-prescribing systems can include clinical decision support (CDS) systems that check the patient’s medical history and provide information about possible allergies, drug-drug interactions and dosing issues.^{^[xi]} Systems can also check a patient’s insurance coverage, notifying the prescriber when a drug is not covered or requires prior authorization from the insurer.

As a result of these improvements over a paper-based system, experts predict e-prescribing systems can avoid more than 2 million ADEs annually, of which 130,000 are life-threatening.[xii] In addition to reducing the medical spending associated with treating these ADEs, such computer systems could generate other savings. For example, e-prescribing has the potential to allow providers to make more informed decisions about clinically appropriate and cost-effective medications.[xiii] According to the Center for Information Technology Leadership, an additional cost savings of $2.7 billion would result from e-prescribing’s ability to reduce clinicians’ phone time.[xiv] The e-Health Initiative recently estimated that widespread adoption of e-prescribing could save the United States healthcare system $27 billion per year.[xv]

Because of e-prescribing’s proven potential to reduce medication errors and the cost of medical care, Congress mandated in the Medicare Prescription Drug Improvement and Modernization Act (MMA) of 2003 that all plans participating in the new Medicare prescription drug benefit (Part D) support an e-prescribing program. Although prescribers are not required to participate, the plans must have a system in place for those who do want to use e-prescribing technology. In its requirement that all Part D plans support e-prescribing, the MMA also required that all such programs follow federal standards promulgated by the Secretary of Health and Human Services. This report analyzes the readiness and potential impact of several proposed standards.

A Recent History of the Implementation of E-prescribing

Over the last four years, several private organizations, states, and regional collaboratives have sought to implement and evaluate the effectiveness of integrated e-prescribing systems:

§ In 2002, Tufts Health Plan of Massachusetts and Advance PCS (now Caremark) conducted a year-long pilot study of integrated e-prescribing. The study involved over 100 clinicians, and found that e-prescribing had positive effects on patient safety, cost, pharmacy and prescriber efficiency, and user satisfaction.[xvi]

§ In 2003, Blue Cross Blue Shield of Massachusetts, in conjunction with various industry partners, launched the eRx Collaborative. The Collaborative’s goal was to deploy e-prescribing systems in the offices of 3,400 Massachusetts physicians.[xvii] After two years, the program had allowed for over three million prescriptions to be transmitted electronically. With regards to safety, by 2005 more than 5,500 prescriptions per month were being changed as a result of warning messages built into the system.[xviii]

§ In 2003, the Rhode Island Quality Institute started a program to implement statewide e-prescribing throughout the state of Rhode Island. Aided by the state’s appropriation of $20 million towards Health Information Exchange, as well as underwriting from e-prescribing network SureScripts and vendor LighthouseMD, the public, private, and academic partnership was met with great success. In 2006, Rhode Island was ranked first in the country in e-prescribing.[xix]

§ In 2003, the Massachusetts Medical Society developed a strategic plan for implementing interoperable e-prescribing throughout the state of Massachusetts. Later, in 2005, the society subsidized e-prescribing for its members. Massachusetts was recently ranked third in e-prescribing, according to pharmacy groups.[xx]^-[xxi]

§ In 2004, Wellpoint, Inc. of Indiana introduced the Physician Quality and Technology Initiative (PQTI). The program gave e-prescribing and administrative software to over 19,000 physicians in HMOs and PPOs in California, Georgia, Missouri and Wisconsin. Although the majority of physicians enrolled in this program use only the administrative software program, the preliminary results surrounding e-prescribing are encouraging. As of July 2005, over 90,000 prescriptions had been written electronically, and offices using the e-prescribing utility were spending 75% less time on administrative work.[xxii]

§ In 2004, CareFirst Blue Cross Blue Shield, Maryland’s largest insurer, decided to provide DrFirst’s Rcopia to 500 physicians. CareFirst and DrFirst also gave wireless handheld devices developed by spring and palmOne Inc. to participating physicians.[xxiii] After the program’s first year, it was estimated that over $1.3 million in cost savings could be directly attributed to reduced prescribing errors.[xxiv]

§ In 2005, Blue Cross Blue Shield of Michigan and the big three American automakers joined forces with local pharmacy benefit managers and HMOs to launch the Southeast Michigan e-prescribing Initiative (SEMI). Over 17,000 physicians had the opportunity to participate in the program, which successfully developed an interoperable e-prescribing system throughout the region.[xxv] Between August 2005 and April 2006, Henry Ford Medical Group reported over 588,000 electronic scripts in total, with 70,000 prescriptions canceled due to drug-drug interaction warnings and over 4,500 prescriptions canceled due to allergy alerts.[xxvi] In addition, over the course of the entire project, researchers found a net reduction in pharmacist initiated calls, as well as a savings in physician telephone time. Together, these efficiencies could save an estimate $2.9 billion per year. Other findings included increased formulary compliance rates and increased generic dispensing for physicians who used e-prescribing.[xxvii]

§ In 2005, the state of Delaware launched a pilot e-prescribing program with 100 physicians, using the system Rcopia, developed by DrFirst. Results from the pilot study will be published in late 2006.[xxviii] [xxix] As of June 2006, more than 75,426 prescriptions had been written electronically.[xxx]

§ In 2005, Sierra Health Services, it subsidiaries Health Plan of Nevada and Southwest Medical Associates (SMA), and the Clark County Medical Society of Nevada funded a program to provide all 5,000 physicians in the state with e-prescribing software. All Nevada physicians were eligible to receive a free license for Allscripts’ Touchworks Rx+ application.[xxxi] Since the implementation of the application, use of generic drugs among patients increased from 59% of prescriptions written to 65%; this increase translates into an annual cost savings of $5 million.[xxxii] In addition, callbacks from pharmacies declined and patient satisfaction increased.[xxxiii]

§ In January 2006, Blue Cross Blue Shield of North Carolina started “ePrescribe,” an initiative that provided funding for e-prescribing software, hardware, and support for 1,000 physicians in North Carolina.[xxxiv]

§ In May 2006, L.A. Care, the largest Medicaid HMO in the country, began an e-prescribing pilot program. The Los Angeles-based health care payer purchased hardware and software from Zix Corporation and provided it free of charge to participating physicians. The initial pilot study involved 50 physicians, but may ultimately include as many as 100.[xxxv]

§ In October 2006, New Hampshire Governor John Lynch announced the goal of having all doctors in his state prescribing electronically by 2008. According to experts, New Hampshire has a good chance of achieving this goal, due in part to its small size, as well as the fact that in 2006, New Hampshire already possessed a fairly advanced health information system—nearly 75% of the state’s primary care physicians used EHRs, and up to 80% of the state’s pharmacies had e-prescribing capabilities.[xxxvi]

§ In January 2007, a coalition of technology companies and healthcare organizations calling themselves the National e-prescribing Patient Safety Initiative announced a program aimed at providing free e-prescribing to every physician in America. The coalition is led by Allscripts and Dell, and includes Aetna, Cisco Systems, Fujitsu Computers of America, Google, Microsoft, Sprint Nextel, SureScripts, Wellpoint, and Wolters Kluwer Health. Through web-based software available for free at www.natinoalerx.com, the coalition hopes to appeal to small practice physicians who otherwise would not wish to purchase an e-prescribing system.[xxxvii]

§ In January 2007, TennCare, Tennessee’s Medicaid program, received a $674,200 HHS grant to launch an electronic prescription system linking rural doctors and pharmacies. The main goals of the program are to reduce pharmacy costs and increase patient safety by providing physicians in rural communities with computers to transmit prescriptions to local pharmacies.[xxxviii]

§ In February 2007, five of Florida’s largest health plans announced a joint effort to encourage the State’s physicians to use e-prescribing. The collaborative, named “e-Prescribe Florida” involves Blue Cross Blue Shield of Florida, Av-Med, Cigna, Humana, and UnitedHealthcare.[xxxix]

The momentum is building in the industry to drive adoption of e-prescribing as a result of the MMA legislation and the desire to create efficiencies, reduce costs and improve patient safety.

The Need for Standards

In spite of the progress made by numerous projects, the adoption of e-prescribing technology remains limited. According to industry surveys, only 5% to 18% of doctors use any form of e-prescribing.[xl] Fewer than 3 percent of all prescriptions are written with integrated e-prescribing systems that realize the most significant benefits of e-prescribing. [xli] Ideally, these integrated systems communicate clearly, securely, and easily across key steps in the drug delivery chain – from accessing information vital to choosing medications, to ordering medications, to dispensing drugs, to payment.[xlii]^.[xliii]

Prescribers, pharmacy dispensers, software vendors, insurers, and patients must work together in order for the integrated e-prescribing system to become a reality. In order to share critical information across various health care settings, systems must be able to interoperate with one another. The inability for multiple systems to share information with a standard format and vocabulary has been a hurdle to effective implementation of e-prescribing.[xliv] The few data standards that are available often are not published with sufficient precision to be implemented in a way that can be constructed as a true “standard.”[xlv] As a result, participants in the e-prescribing chain may have to use more time-consuming “workarounds” to transfer information and accomplish other functions of e-prescribing.[xlvi] Solutions like the manual re-entry of data lessen the potential safety and economic benefits of using e-prescribing in the first place.

To address this situation, the National Committee on Vital and Health Statistics (NCVHS) was called upon by the MMA to develop recommendations for uniform standards to enable e-prescribing in ambulatory care. A standard is a published specification that establishes a common language, and contains a technical specification or other specific criteria and is designed to be used consistently, as a rule, a guideline, or a definition. The specific data and supportive process must “match” at both the source and destination computer systems -- which is only achievable with adherence to using the same standards. Thus, standards are the fundamental building blocks essential for the widespread adoption of e-prescribing as well as other health information technologies (HIT).

NCVHS identified and evaluated three types of e-prescribing standards as necessary requirements to support e-prescribing. Message format standards provide communication protocols and data content requirements (including those that support medication decision making). Terminologies ensure data comparability and interoperability. Identifiers for all relevant entities within the e-prescribing process allow for clearer tracking and communication.

The MMA requires that all Part D plans follow federal standards promulgated by the Secretary of Health and Human Services when they implement e-prescribing systems. Compliance with the standards is also required for prescribers and dispensers that choose to send or receive prescription-related information electronically for covered Part D drugs for Part D eligible individuals.[xlvii] Although prescribers and pharmacies will not be required to use these same standards for their other patients, the adoption of standards for Part D will pave the way for more integrated systems across the board.

Proposed Standards

When HHS started to circulate rules proposing standards for e-prescribing, the rules differentiated between those that were available for immediate adoption and those that needed further testing (See 70 FR 6256 (February 4, 2005) and 70 FR 67573 (November 7, 2005), to be codified at 42 CFR 423). The criteria for immediate adoption included:

§ Approval by an ANSI-accredited SDO to assure consideration of industry requirements.

§ Implementations among multiple partners to assure interoperability.

§ Recognition by key stakeholders to assure industry recognition of a single standard.

Three standards met these criteria, and have been adopted as “foundation” standards for the new Part D requirement (see Exhibit 1).

In addition, HHS identified several areas in which standards are needed, but no single standard has been widely adopted. The MMA called for the Secretary of Health and Human Services to propose such standards and to sponsor pilot sites to evaluate them.[xlviii] The Secretary selected six such new or emerging standards, called “initial” standards (see Exhibit 1).

Exhibit 1. Description of Initial and Foundation Standards and Focus of their Testing

Name	Standard Description	Testing Requirements
Initial Standards
NCPCP Formulary and Benefits Standard Version 1.0	Displays the formulary status and alternative drugs as well as co-pays and other status information. NCPDP has developed a standard using RxHub protocol.	Determine if it should be adopted as a standard
NCPDP SCRIPT Medication History	Includes the status, provider, patient, coordination of benefit, repeatable drug request, and response segments of SCRIPT.	Determine readiness of the NCPDP's standard medication history message
NCPDP SCRIPT: Fill Status Notification	Informs when Rx filled, not filled, or partially filled. Includes provider, patient, and drug segments of SCRIPT message. Not yet generally used.	Assess the business value and clinical utility
Structured and Codified SIG	Indication, dose, dose calculation, dose restriction, route, frequency, interval, site, administration time and duration, stop	Test structured and codified SIGs (patient instructions) developed through standards development organization efforts
RxNorm-Clinical drug terminology (Versions 8/2/2006 and 12/21/2006)	A clinical drug nomenclature that provides standard names for clinical drugs and for dose forms as administered. It also provides links from clinical drugs to their active ingredients, drug components, and most related brand names.	Determine whether RxNorm terminology translates to NDC for new prescriptions, renewals and changes
Prior authorization messages	Requires header information, requester, subscriber, utilization management, and other relevant information for prior authorization requests	Determine functionality of new versions of the ASC X12N 275/278. (with HL7 attachment)
Foundation Standards
NCPDP Telecommunications	HIPAA standard for eligibility communications between retail pharmacy dispensers and payers/PBMs.	Determine interoperability with new standards.
ASC X12N-270/271	HIPAA standard for eligibility and benefits communications between dentists, professionals, institutions, and health plans.	Determine interoperability with new standards.
NCPDP SCRIPT Standard Version 5, Release 0: New Change Renewal Cancellation	Provides for the exchange of new prescriptions, changes, renewals, and cancellation notifications. Each function has varying degrees of industry experience.	Determine interoperability with new standards.

Long Description: The exhibit list the six new or emerging standards, called “initial” standards which include NCPDP SCRIPT, Medication History, NCPDP SCRIPT: Fill Status Notification, Structured and Codified SIG, RxNorm-Clinical drug terminology, and Prior authorization messages.

Pilot Sites

During calendar year 2006, the initial standards were tested in five sites (see Exhibit 2). The sites included a variety of different settings, including long term care and small and large physician practices, to determine whether the standards are ready for broad adoption. Sites were asked to determine whether the initial standards allow participants to effectively and unequivocally communicate necessary information between all participants in the transaction, such as the pharmacy, pharmacy benefits manager (PBM), router, plan and prescriber. They were also asked to explore how the initial standards worked with the more accepted foundation standards. Pilot sites also tracked outcomes in their projects that could be attributed more generally to the use of e-prescribing, such as a reduction in medical errors.

Exhibit 2. List of E-prescribing Pilot Sites

Organization Name	Project Name
RAND Corporation	Test of Medicare’s initial e-prescribing standards in the New Jersey e-prescribing Action Coalition
Brigham and Woman’s Hospital	E-prescribing using a Community Utility: The e-prescribing Gateway
Achieve Healthcare Information Technologies, LP (LTC)	LTC e-prescribing Standards Pilot Study
Ohio KePRO/UHMP	A Practice-Based Pilot Test of Emerging e-prescribing Standards
SureScripts, LLC	Maximizing the Effectiveness of e-prescribing Between Physicians and Community Pharmacies

Long Description: Provides a list of the five pilot sites which include: RAND Corporation, Brigham and Woman’s Hospital, Achieve Healthcare Information Technologies, LP (LTC), Ohio KePRO/UHMP, and SureScripts, LLC.

The Secretary of Health and Human Services will consider the results of these pilot sites prior to promulgation of a final set of uniform standards, which are due by April, 2008. The use of this pilot test data and findings will allow for a set of standards that benefits all interested stakeholders: clinicians, medication dispensers, Part D Sponsors, and most importantly, patients and their families. This report provides a look at the results of the five pilot sites.

Objectives of the Evaluation

The evaluation team was charged with providing the information needed to make informed decisions regarding the initial standards’ functionality, interoperability with foundation standards, and impact on workflow, clinical, and other outcomes. Specifically, the primary objectives of the evaluation are to:

1) systematically collect and interpret the evidence (e.g., testing methods and findings) reported by e-prescribing pilot sites;

2) determine the initial standards that are reported to be functional (and those which are not),

3) document the benefits, challenges, and technical considerations for mass implementation of the initial standards in different settings, and

The following sections describe the proposed standards (Section II) and the pilot sites (Section III) in more detail. Section IV then describes the approach taken by the evaluation team to critically collect data from pilot sites and the methods used to evaluated their findings and assess the strength of their claims. In Section V, are detailed descriptions of the pilots' experiences with the initial standards. Section VI presents the findings to date on the other outcomes of e-prescribing in these pilots. The final section of this report synthesizes the conclusions and recommendations to be drawn from the analysis of findings reported by the pilot sites.

	Achieve LTC	Brigham & Women’s	Ohio KePRO/ UHMP	RAND	SureScripts
*Settings*
1-2 physician offices		yes	yes	yes	yes
Small offices		yes	yes	yes	yes
Larger offices		yes	yes	yes	yes
Academic clinics		yes		yes	yes
Long term care	yes
*Stakeholders*
Prescribers	yes	yes	yes	yes	yes
Pharmacists	yes	yes	yes	yes	yes
Patients				yes	yes
RNs	yes				yes
Technology Vendors	yes	yes	yes	yes	yes
MAs					yes
Others	LPN/NP/PA		yes		yes

Project Name	Award Amount Grant Number	PI	Lead Organization (State)	Software Vendor(s)	Network Provider(s)	Pharmacy System	Other Partners (State)	Medication History	Formulary & Benefit	Fill Status	Prior Authorization	SIG	RxNorm	Telecommunications	ANSI X12N 270/271	NCPDP SCRIPT	Test Environment/Setting
A Practice-based Pilot Test of Emerging e-prescribing Standards	$896K U18HS 016389	Barich, D.	Ohio KePRO/UHMP (OH)	Instant DX, NDC Health (practice management)	RxHub, SureScripts	CVS, Walgreens, Rite-Aid	QualChoice (OH), Aetna (CT), University of Minnesota Division of Health Services Research (MN), MGMA Center for Research (CO), Partners Healthcare (MA), Richard Pham(MN)	X	X	X	X	X	X		X		Impact in small practices.
E-prescribing Using A Community Utility: e-prescribing Gateway	$1M U18HS 16377	Rothschild, J.	Brigham and Women's Hospital (MA)	Brigham and Women’s Hospital	RxHub, SureScripts	“Community of Pharmacy Chains”	CareGroup Health System (MA), MA-Share (MA)	X	X	X	X	X	X		X	X	Impact on Adverse Drug Events (ADEs).
Long Term Care e-prescribing Standards Pilot Study	$1.1M U18HS 16378	Bordelon, M.	Achieve Healthcare Information Technology (MN)	Achieve Healthcare Information Technology	RxHub	Preferred Choice Pharmacy of Benedictine Health System	Benedictine Health System, Preferred Choice Pharmacy (MN), RNA Health Information Systems (OH), Prime Therapeutics (MN), Blue Cross Blue Shield of MN		X	X	X			X	X	X	Impact in long term care setting.
Maximizing Effectiveness of e-prescribing Between Physicians and Community Pharmacies	$1.9M U18HS 16394	Lapane, K.	Brown Medical School (RI)	AllScripts, MedPlus/Quest Diagnostics, DrFirst, Gold Standard, ZixCorp	SureScripts	Ahold (Giant and Stop & Shop), Albertsons (Sav-On and Osco), Brooks, CVS, Duane Reed, Rite Aid, Walgreens, Wal-Mart, Kerr Drugs, Longs Drugs	SureScripts (VA), Midwestern University (AZ), Chain Pharmacy Advisory Council (VA), Independent Pharmacy Advisory Council (VA)	X	X	X		X	X	X	X	X	Impact on pharmacy workflow. Interface with multiple vendors, practice sites.
Pilot Testing of E-prescribing Standards	$1.8M U18HS 16391	Bell, D.	RAND Corporation (CA)	AllScripts, InstantDx, and iScribe	RxHub, SureScripts	Walgreens (IL)	Horizon Blue Cross Blue Shield of New Jersey (NJ), Caremark Rx (TX)	X	X	X	X	X	X			X	Impact on prescriber workflow. Technical expert panel review of standards.

SECTION IV : EVALUATION METHODS

The evaluation team was charged with making informed recommendations regarding initial standards’ functionality, interoperability with foundation standards, and their impact on workflow, clinical, and other outcomes. Specifically, the primary objectives of the evaluation are to: 1) systematically collect and interpret the evidence (e.g., testing methods and findings) reported by eRx pilot sites; 2) determine the initial standards that are reported to be functional (and those which are not), and 3) document the benefits, challenges, and technical considerations for mass implementation of the initial standards in different settings. This section describes the methods followed by the NRC evaluation team and consultants to carry out these tasks.

Research Questions

The team identified key questions for each of the components of the evaluation: both the functionality of the standards as well as their outcomes and likely impacts.

Functionality. In evaluating the functionality of the standards, the key question was whether the standards allow participants to effectively and unequivocally communicate the necessary information between all participants in the transaction, such as the pharmacy, PBM, router, plan and prescriber. Component questions included:

§ Are the right data being sent?

§ Are the data usable and accurate?

§ Are the data well-understood at all points of the transaction?

§ Can all appropriate drugs and other therapies be ordered via e-prescribing?

§ Do the initial standards work well together and with the foundation standards?

§ What workarounds were used, and how can the standards be improved to address them?

§ How long does it take to conduct each transaction using the initial standards?

Outcomes and Impacts. In addition to testing the basic functionality of the standards, pilot tests studied a wide variety of outcomes, including the following:

§ Does eRx increase the use of on-formulary medications and generics?

§ Does eRx improve the rate of potential inappropriate prescribing (e.g. Beers criteria)?

§ Does eRx affect the rate of hospital and emergency department use?

§ Does eRx affect the number of medication errors and adverse drug events? Does it reduce the rate of hospitalizations and emergency department visits associated with adverse drug events?

§ Does eRx improve workflow in prescriber offices (fewer interactions with pharmacies, freeing up support staff time for other functions, more time available for patient interaction)?

§ What are the uptake and dropout rates among prescribers?

§ Does eRx affect patient satisfaction?

Data Collection Activities

The NRC evaluation team systematically collected both qualitative and quantitative data from various sources to inform the recommendations to CMS. Exhibit 11 shows the different types of data collected or reviewed, and the various data sources. Qualitative data were collected by conducting: 1) semi-structured interviews with pilot site staff, 2) observational site visits to pilot sites using a tailored protocol, and 3) unstructured interviews with key informants. Quantitative data (collected by the pilot sites themselves) included technical testing results which measure the functionality and interoperability of foundation and initial standards, and findings from the various evaluations which investigated different impacts and outcomes of e-prescribing. These data were collected by reviewing pilot site documents including the proposals, presentations, and progress and final reports.

Exhibit 11. Description of Data Sources Used to Collect Both Primary and Secondary Evaluation Data

Data Source	Type	Data Collected/Reviewed (primary/secondary)
A. Review of Pilot Site Documents	Qualitative Quantitative	· Status of standards testing (s) · Status of evaluation (s) · Barriers to testing/evaluation (s) · Preliminary findings (s)
B. Structured PI Conference Calls	Qualitative	· Status of standards testing (p) · Status of evaluation (p) · Dates for site visit (p) · Agenda for site visit (p)
C. Pilot Site Visits	Qualitative Quantitative	· Status of standards testing (p) · Status of evaluation (p) · Barriers to testing/evaluation (p) · Preliminary findings (p) · Certification process (p) · Data collection methods (p) · Workarounds, modification (p)
D. Key Informant Interviews	Qualitative	· Experience with initial standards in different settings (p) · Certification process (p) · Workarounds, modifications (p)

Key: (P)= Primary data; (S)= Secondary data.
Review of Pilot Site Documents. The evaluation team reviewed various types of documents produced by pilot sites. These documents included their grant proposal, quarterly progress reports, final project report, and all publications and presentations produced during the grant period.[l] The objective was to collect information regarding their approach to testing the initial standards, note changes to their proposals, and to begin to assess the strength of their testing and evaluation methods. The document reviews also served as a source of information which was used to generate project specific questions which were included in the tailored site visit protocol.

Structured Telephone Calls with Key Pilot Site Staff. Prior to the site visit, the evaluation team contacted key pilot site staff (e.g., the PI, evaluator, project coordinator, etc.) to discuss various aspects of the project. Specifically, the evaluation team was interested in assisting the pilot sites to prepare for the upcoming site visit by reviewing the overall objectives and expectations for the visit. During this initial call pilot sites were asked to provide an overall status of the project, as well as to discuss any preliminary or final results. In addition, the initial call served to finalize a date for the site visit, and to prioritize the agenda. These calls were about 45 to 60 minutes in duration.

Pilot Site Visits. The NRC evaluation team conducted one day, on-site visits to all five of the eRx pilot sites. The purpose of the site visits was to: 1) collect test and evaluation data not yet reported in progress or final reports, 2) document barriers and challenges to standards’ implementation and testing, 3) begin to collect information to assess testing methods and evaluation approaches, and 4) discuss preliminary findings from data analyses conducted by pilot sites to date. Site visits were conducted by NRC evaluation team members and consultants following a site visit protocol which is included in Appendix A.

Key Informant Interviews. The evaluation team also conducted brief unstructured telephone interviews with key informants. These calls were made after all site visits were completed and served to validate the information which was learned during the site visits and from reviewing the pilot sites’ final reports. Key informants included individuals with direct industry experience working with various aspects of e-prescribing, including: prescribers, dispensers (e.g., pharmacy staff), software vendors, and individuals who have developed and evaluated e-prescribing programs and applications.

Data Analysis

Rather than taking pilot site results at face value, the evaluation team used information collected from each of three domains to help inform CMS recommendations regarding the overall functionality of initial standards: 1) pilot site characteristics, 2) standards testing and evaluation results, and 3) assessment of pilot site testing and evaluation methods (see Exhibit 12). By reviewing the findings across all three domains, the evaluation team was able to assess pilot site results within the context of the settings and methods within which the testing occurred. This exercise allowed NRC evaluation staff to reach informed conclusions regarding how each pilot site-level result should be used in developing recommendations for CMS.

Exhibit 12 . Overview of Evaluation Framework

Long Description: The exhibit shows how data from three domains helped inform CMS recommendations regarding the overall functionality of initial standards: 1) pilot site characteristics, 2) standards testing and evaluation results, and 3) assessment of pilot site testing and evaluation methods.

Pilot Site Characteristics

Each project was implemented using different software vendors, with different types of partners, and in different settings and conditions (such as prescription volume). All five of the eRx pilot sites engaged in the testing of different combinations of foundation and initial standards and transactions. In addition, each pilot site conducted evaluative studies focusing on the impacts of eRx on different outcomes of interest to CMS/AHRQ as expressed in the RFA, as well as of interest internally to their organization.

Given the great variability both between and within pilot sites, this domain was designed to provide a descriptive overview of the eRx pilot sites along three distinct characteristics of each of the projects: 1) initial and foundation standards tested, 2) partners, software vendors, and network providers involved in the project, and 3) testing environment and healthcare setting within which the standards were tested. Specifically, the NRC evaluation team collected descriptive information from all pilot sites in order to answer key evaluation questions, including: 1) did the pilot sites collectively test all the initial standards? 2) did the testing encompass all possible transactions, settings, and conditions? 3) are there any transactions, settings, and conditions that were not tested?, and 4) are there any other gaps in testing or in evaluation of impacts?

Pilot Site Evaluation Findings

The framework used to lead the evaluation of eRx pilot sites was designed to systematically collect and assess three types of findings: 1) data generated by pilot sites’ testing of foundation and initial standards, 2) findings from evaluations conducted by pilot sites to assess the impacts of the implementation of initial standards on workflow and other outcomes of interest, and 3) secondary data collected by the NRC evaluation team. The objective of this domain is not to validate the data reported by the pilot sites (as would be done in a meta-evaluation) but to aggregate and compare their collective findings both within and across pilot sites.

There are three primary factors used to determine the functionality of an initial standard, its ability to: 1) effectively communicate electronic prescriptions, 2) interoperate with foundation standards, and 3) have positive impacts on workflow and other outcomes. Each factor has a series of questions that the pilot sites were required to address in their research. The pilot sites reported findings for these questions and their responses were collected and analyzed to critically assess each of the initial standards.

1. Effective Communication of Electronic Prescriptions. A fundamental requirement for determining that an initial standard is ready for mass implementation is whether the standard has the ability to effectively and reliably transmit electronic prescriptions to and from prescribers and dispensers. In addition to collecting both live and lab test data from pilot sites, the evaluation team also collected data on other critical measures of a standard=s ability to communicate, such as: 1) foundation standards with which each of the initial standards was tested, 2) versions of the initial standards tested, 3) total number of transactions completed, and 4) percentage of failed transactions (e.g., communication errors). The overall objective was to determine if the mandatory or required data elements within a standard were being transmitted accurately at all points of the transaction, how long it took to conduct each transaction, and whether there were any volumes at which an eRx was not properly transmitted (e.g., artifact failure).

Exhibit 13 shows a detailed list of factors and corresponding measures or data elements which were reported by pilot sites, and used by the NRC evaluation team to make recommendations to CMS.

Exhibit 13. Summary of Key Factors and Measures/Data Elements Required by the RFA and Used by the Evaluation Team to Inform the Recommendations to CMS

Domain	Factors	Measure/Data Element
1. Pilot Site Characteristics	Comprehensive Coverage of Initial Standards and Transactions Tested	· Type of prescribers · Type of dispensers · Software vendors · Network providers	· Test setting (live or lab) · Setting of eRx implementation · Standards/transactions tested · Type of pharmacy system
2. Results of Technical Testing and Evaluation Findings	Effective Communication	· Type and number of prescribers · Type and number of dispensers · Number of e-prescriptions · Total number of errors (i.e., incomplete transactions) · Elapsed time to complete transactions	· More measures from certification process
Interoperability	· Number of workarounds · Time required to achieve certification · Number of communication errors	· Modifications necessary for certification
Impacts and Other Outcomes	Clinical · Use of on-formulary medication and generics · Change in rate of potential inappropriate prescribing · Rate of medication errors post eRx · Rate of adverse drug events post eRx · Changes in the rate of hospital and emergency department use associated with ADEs · Reliability of prescriptions Economic · Reduction in: costs, calls, data entry and processing · Security improvements	Workflow (changes) · Prescriber’s office flow · Verbal orders · Prescriber uptake/dropout rate · Prescription renewal rates · New prescription rates · Fill status rates · Number of callbacks (pharmacy to prescriber) · Patient satisfaction · Physician satisfaction · Reduction in processing times
3. Assessment of Pilot Site Testing and Evaluation Methods	Timing of Testing and Data Collection	· Volume of eRx · Transaction certification date · Date of query (survey, focus group, site visit)	· Prescriber uptake/dropout rate
Testing Conditions	· Setting (live or lab) · Volume of eRx
Data Convergence	· Results of pilot sites’ testing of initial standards · Key informants’ reported experiences with standards · Literature review of standards’ performance

Domain

Factors

Measure/Data Element

1. Pilot Site Characteristics

Comprehensive Coverage of Initial Standards and Transactions Tested

· Type of prescribers

· Type of dispensers

· Software vendors

· Network providers

· Test setting (live or lab)

· Setting of eRx implementation

· Standards/transactions tested

· Type of pharmacy system

2. Results of Technical Testing and Evaluation Findings

Effective Communication

· Type and number of prescribers

· Type and number of dispensers

· Number of e-prescriptions

· Total number of errors (i.e., incomplete transactions)

· Elapsed time to complete transactions

· More measures from certification process

Interoperability

· Number of workarounds

· Time required to achieve certification

· Number of communication errors

· Modifications necessary for certification

Impacts and Other Outcomes

Clinical

· Use of on-formulary medication and generics

· Change in rate of potential inappropriate prescribing

· Rate of medication errors post eRx

· Rate of adverse drug events post eRx

· Changes in the rate of hospital and emergency department use associated with ADEs

· Reliability of prescriptions

Economic

· Reduction in: costs, calls, data entry and processing

· Security improvements

Workflow (changes)

· Prescriber’s office flow

· Verbal orders

· Prescriber uptake/dropout rate

· Prescription renewal rates

· New prescription rates

· Fill status rates

· Number of callbacks (pharmacy to prescriber)

· Patient satisfaction

· Physician satisfaction

· Reduction in processing times

3. Assessment of Pilot Site Testing and Evaluation Methods

Timing of Testing and Data Collection

· Volume of eRx

· Transaction certification date

· Date of query (survey, focus group, site visit)

· Prescriber uptake/dropout rate

Testing Conditions

· Setting (live or lab)

· Volume of eRx

Data Convergence

· Results of pilot sites’ testing of initial standards

· Key informants’ reported experiences with standards

· Literature review of standards’ performance

2. Initial Standards' Interoperability with Foundation Standards. Another factor in determining if an initial standard is ready for mass implementation is whether it is interoperable[li] with established or foundation standards. The objective of this domain is to determine in which settings the initial standards interoperate (and do not interoperate) with foundation standards. Evaluation staff collected information from pilot sites on whether any modifications (either to the initial or foundation standards) were required in order to make them interoperable, as well as the time and expense required to make the necessary modifications.[lii] A distinction was made between an initial standard’s limitations which were attributed to the standard itself, and those which could be attributed to the organization implementing the standard. In the latter case, specific descriptions of the implementation challenges were documented and described. In addition to communication related measures, Exhibit 13 shows some of the measures used to determine a standard's interoperability with foundation standards, such as: time required to attain certification for each specific transaction, number of modifications required before certification was granted, etc.

3. Impacts and Other Outcomes. A third critical factor in evaluating the initial standards’ functionality is the impact they have on workflow (both at the practice and pharmacy levels) as well as other clinical and economic impacts. Examples of impacts of eRx implementation on workflow include: prescriber satisfaction with eRx, dispenser satisfaction with eRx, prescriber uptake and dropout, pharmacy callbacks, and eRx renewals and fills. In addition, the evaluation team also considered the impacts eRx has on clinical outcomes, including: change in medical errors, adverse drug events (ADEs), rate of potential inappropriate prescribing, and economic impacts such as reductions in costs related to data entry and processing, and reductions in calls.

Assessment of Pilot Site Testing and Evaluation Methods

The methodologies used to assess the initial standards varied by pilot site. All sites used multiple approaches, including interviews with technical staff responsible for implementing the standard, measurements such as transaction times that are obtained from certification processes, live transactions encompassing an end-to-end prescribing process, simulation of data transactions in laboratory settings and through a Delphi expert panel process. Different interventions yielded varying results creating some difficulty in evaluating overall experience, but produced interesting information on which intervention methods work best under specific conditions.

The evaluation team had to efficiently gage the strength of the testing methods and research designs used by pilot sites to test the initial standards, and evaluate impacts and other outcomes. To make this determination, the NRC evaluation team used three questions to guide their decision making. First, was the timing between the testing and data collection appropriate to produce accurate findings? Second, were the conditions under which the testing was conducted appropriate and feasible? Finally, was there concordance in the findings and conclusions reached by pilot sites which tested the same initial standards and transactions?

1. Timing of Testing and Data Collection. Pilot sites followed aggressive project timelines which required that they obtain certification on various eRx applications, train staff on their use, and conduct their testing and evaluation of the various standards. Given these constraints, the evaluation team questioned whether prescribers or eRx users had sufficient time to form opinions regarding their eRx experience before they were queried either via survey or site visits. In other words, did users have enough time to familiarize themselves with, and form opinions about eRx? Users might have formed negative opinions about eRx because of their limited exposure to eRx. Conversely, their limited experience with eRx might not have allowed them to experience problems that only arise when certain levels of eRx are reached.

2. Appropriateness of Testing Conditions and Methods. In addition to the timing of the testing, the evaluation team also assessed the various testing conditions and methods used by pilot sites to test the standards. For instance, an important measure in this assessment included the volume of e-prescriptions delivered by pilot sites. The primary question used to assess the test conditions was to determine if the standards were tested under a volume of prescriptions which reflect the volume the standard would experience in real-world settings. For the evaluations, the team focused on the sample sizes and data collection methods used to make claims regarding the standard’s impact on workflow and other clinical outcomes.

3. Convergence of Evidence. AHRQ funded a total of five pilot sites to test the initial standards in different settings, and using different software vendors, network providers, and pharmacy systems. Therefore, in order to recommend to CMS that an initial standard is ready for mass implementation, findings to the various tests conducted by the pilot sites should be somewhat similar or converge on a final outcome concerning specific standards. In addition, pilot site findings were compared to the qualitative opinions expressed by industry key informants.

SECTION V:
EVALUATION FINDINGS: INITIAL STANDARDS

The five pilot sites included in this evaluation proposed to test six standards for e-prescribing as required in application to the Grant. Each standard is a published specification that establishes a common language, and contains technical specifications designed to be used consistently, as a rule, a guideline, or a definition. Effective, interoperable standards will be the fundamental building blocks essential for the widespread adoption of e-prescribing as well as other health information technologies (HIT). However, it is necessary for organizations to implement standards and use them in production to fully comprehend what is needed for a viable standard that addresses the key functionality for e-prescribing. Thus, these pilot sites have provided valuable information about how to move the initial standards toward that goal.

Information standards have two components: the content and the format. A key factor to successful industry adoption will depend on how the standards have been implemented. A standard for all practical purposes can function properly, transmitting the right data to the right location, in the right format; however, some implementations may not be a perfect use of a standard. A vendor or organization may use the standard in a way that it was not intended for - to meet the need of a particular business application. Based on this misuse of a standard or variant interpretation of the implementation process, it would be difficult to name this as a foundation standard ready for widespread industry use. In this section, we highlight findings about implementation of standards as well as the functioning of the standards themselves.

To evaluate the accuracy, usability, completeness, interoperability, and speed of each initial standard transaction in transmitting the necessary data, pilot sties implemented different methodologies which utilized multiple approaches including interviews with technical staff responsible for implementing the standard, measurements such as transaction times that are obtained from certification processes, live transactions encompassing an end-to-end prescribing process, simulation of data transactions in laboratory settings, and through a Delphi expert panel process. Different interventions yielded varying results, creating some difficulty in assessing overall experience. However, the sites produced interesting information on which intervention methods work best under specific conditions.

This section provides a summary that different standards tested by pilots, and whether those tests included full implementation or exploratory inquiries into the functionality of the standard. Also presented are findings on each standard from each of the pilot sites, identifying the overall result as well as the particular issues that each site raised with each standard.

Pilot Site Coverage of the Standards

Each pilot site engaged in the testing of different combinations of foundation and initial standards as outlined in Exhibit 14. Each of the initial standards was tested by four or five of the pilot sites. However, the type of testing varied from site to site.

Exhibit 14. Summary of Standards Tested and Methodology by Pilot Site

	ACHIEVE	BRIGHAM & WOMEN’S	Ohio KePRO/UHMP	RAND	SURESCRIPTS
*Initial Standards*
NCPDP SCRIPT Medication History Standard v8.1	No	Yes-live	Yes-live	Yes-live	Yes-live
NCPDP Formulary & Benefits Standard v1.0	Yes-live	Yes-live	No	Yes-live	Yes-live
NCPDP SCRIPT Fill Status Notification v8.1	Yes-live	Yes-evaluate only	Yes-live	Yes-panel focus grp	Yes-using Med Hx
Prior Authorization ANSI X12N278/275 with HL7 Attachment	Yes-live	Yes-lab	Yes-live	Yes-live	No
NCPDP Structure & Codified SIG Standard v1.0	No	Yes-lab	Yes-lab	Yes-lab	Yes-lab
NLM’s RxNorm Drug nomenclature	No	Yes-lab v.12/21/06	Yes-lab v.12/21/06	Yes-lab v.12/21/06	Yes-lab v.8/2/06
*Foundation Standards*
NCPDP Telecom	Yes-live	No	No	No	Yes-live
ANSI X12N 270/271	Yes-live	Yes-live	Yes-live	No	Yes-live
NCPDP SCRIPT: New, Refill, Change, Cancel, Renew	Yes-live	Yes-evaluate only	No	Yes-live	Yes-live

Pilots tested some standards in a “live” environment, with prescribers generating an electronic prescription and transmitting that prescription electronically to a pharmacy. For example, the medication history, formulary and benefits, and prior authorization standards were tested predominantly in this way. Some pilots chose to evaluate several standards in a “lab” environment using presentations to expert panels, workgroups, interview and survey techniques, as well as other tools for analyzing the adequacy of the standard’s content and usability. For example, the structured and codified SIG standard and the RxNorm standard were tested exclusively in this way.

The foundation standards were included in several pilot sites, even though they are not considered to be initial standards that require testing. The purpose of including standards is to ensure interoperability with initial standards. Most pilots utilized these standards. Some proposed to report on the results of that use (indication of yes) and others opted not to specifically report findings (indication of no). The testing methodology utilized by one of the pilots, “evaluate only” represents mapping of the structure and content of the initial standards to the foundation standards to identify potential interoperability issues.

Results of Standards Testing

This section describes the results reported by each of the pilot sites. The results are presented, by standard, on a pilot-by-pilot basis.

Formulary & Benefits

Formulary & Benefits data standards must provide a uniform means for Pharmacy benefit payers (including health plans and PBMs) to communicate a range of formulary and benefit information to prescribers via point-of-care (POC) systems. The standard covers a range of formulary and benefit data, including information on 1) General formulary data (for example, therapeutic classes and subclasses); 2) Formulary status of individual drugs (i.e., which drugs are covered); 3) Preferred alternatives (including, but not limited to restrictions that may impact whether the plan will cover a drug being considered, such as quantity limits and need for prior authorization); and 4) Co-payment (that is, not just the single co-payment amount for the drug being considered, but the co-payments for one drug option versus another). The standard’s goal is to enable the prescriber to consider this of information during the prescribing process, making the most appropriate drug choice for the patient without extensive back-and-forth administrative activities with the pharmacy or the health plan.

This NCPDP standard transaction version 1.0 was implemented live in all pilot sites where technology vendors were certified prior to production. This standard works in tandem with the eligibility request and response (ANSI ASC X12N 270/271). Once the individual is identified, the appropriate drug benefit coverage is then located and transmitted to the requestor.

MA-SHARE- Brigham & Women’s

Did the standard work? When eligibility data were complete, the Formulary and Benefits standard did support download of four types of formulary-related data including: 1) Formulary Status List, 2) Coverage List 3) Copay List and Alternatives List. No interoperability issues were reported in the standard itself. The conclusions of the pilot testing found that Formulary and Benefits standard adequately supports the transfer of formulary and benefits data from the data provider to the data consumer and recommend its use under Medicare Part D e-prescribing.

Issues raised: This pilot project found that successful use of the Formulary and Benefits request required successful use of one of the foundation standards, the eligibility standard ANSI x12N 271 and accurately identifying the right individual. The response to an eligibility request does not always include some information required for retrieving formulary data including: formulary ID, alternatives list ID and Copay List ID. The next version of this standard (v5010) is expected to alleviate some of these challenges.

Variation in the amount and level of detail provided by the various PBMs did add complexity to design and development. For example, some plans represent their formularies at the level of explicit NDC codes while others use representative NDCs. Implementation of the standard proved to be the most challenging as there was considerable variation and change in certification requirements of the PBMs/payers. Again, this is not a problem with the structure of the standard, but a fundamental weakness in the implementation of that standard.

RAND

Did the standard work? In general, the standard worked, but there were some implementation factors that are probably limiting the benefits this standard could deliver.

Issues raised: RAND experienced issues with having the formulary and benefit functionality tied to a successful eligibility transaction. This pilot’s observation was that this makes patient identification a key component. A patient unique medical ID that could be transmitted to verify eligibility would eliminate failures because of differences in demographic information which is currently used to find a successful match.

This pilot found the lack of industry standard drug identifiers to be a critical issue in the successful deployment and acceptance of the Formulary and Benefits standard. Market participants use drug databases from a number of companies that each has its own proprietary drug identification method. A primary operation in the use of the standard is to match the drug selected for a prescription to the appropriate drug in the formulary data. The greatest near-term potential for improving use of the standard is the further development of RxNorm to serve as the preferred drug identifier. The Formulary and Benefits standard currently has fields to support the use of RxNorm, so no additional modifications to the existing standard would be required.

Many health plans vary their co-payments and quantity limits based on type of pharmacy. For example, it is common for mail order pharmacies to dispense a three-month supply, often at lower co-pay than retail pharmacy. Currently the standard does not differentiate by type of pharmacy, so it cannot provide this type of information to the prescriber.

The Formulary and Benefits standard is quite broad and there are a number of complex data relationships supported by the standard. This complexity creates a certain level of confusion as to how to properly use the data and leads to implementation issues. One of the business functions supported by the Formulary and Benefits standard is the alerting of physicians that prior authorization (PA) is required. This is an interoperability issue as many prior authorizations are only required after a limitation is hit. However, the standard only allows definition of PA or limitations as “standalone” rules when they really need to be considered together. The standard needs to support different types of situations with use of Prior Authorization like the ability to indicate a PA is required (depending on the benefit) after a quantity limitation, step care or any other type of message involving benefit limits.

The standard does provide a mechanism for co-pay changes based on an individual’s accumulated use of their benefit. The most prevalent example of this type of coverage is the ‘doughnut hole’ that exists in Medicare Part D. The technology standard is not the issue; the major dependency will be access to the source information in a timely manner.

SURESCRIPTS

All existing implementations in this pilot used a hosted model, where eligibility information is provided through a nightly data exchange and formulary information is updated periodically as needed.

Did the standard work? It was the assessment of this initiative that overall, the standard works as it is intended and provides great value. The Formulary files were created as specified by the NCPDP Formulary and Benefits standard.

Issues raised: The only concern relative to formulary specifically, is the large amount of optional data (flexibility of the standard) that makes it hard to anticipate what an end user is going to want or find acceptable. There is a minimal amount of data that is required, but a lot that is optional.

ACHIEVE

Achieve built formulary benefits by modifying the computerized physician order entry (CPOE) part of the Achieve Matrix® EHR software system to include eligibility information for residents. This information is provided to Achieve by RxHub through a X12N 270/271 transaction. The Matrix CPOE drug lookup feature was enhanced to present indicators that show what drugs will and will not be covered by the resident payer. If a drug requires prior authorization, it is alerted to the prescriber during the transaction.

Did the standard work? In the two treatment facilities where this standard was tested, the transaction was able to provide coverage information for 84 out of the 196 residents (43%) using the eligibility information available from RxHub. This 43% coverage percentage was much higher than the investigators had anticipated. No changes were made to these standards for the LTC environment.

Issues raised: During the pilot, additional pharmacy coverage plans were made available from MediMedia’s InfoScan formulary list to allow the nursing facility the ability to assign one of InfoScan’s over 4,000 formularies to the 57% of the residents that did not have an RxHub insurance plan automatically assigned. However, InfoScan’s manual coverage assignment process was not utilized during the pilot because the value of this coverage information was not regarded as worth the effort to manually assign a resident’s coverage to an InfoScan formulary.

Ohio-KePRO/UHMP

The Formulary and Benefits Standard was not specifically tested during the pilot phase as it was their understanding that this was named a foundation standard. Eligibility was utilized in the testing of other standards like Medication History and creation of a new prescription. When a UHMP prescriber begins to write an electronic prescription for a patient within OnCallData™ (POC application), their prescription benefit eligibility is automatically checked against RxHub’s Master Patient Index (MPI). More than a quarter million (299,857) prescription benefit eligibility checks were generated in OnCallData™ during the pilot period from January through December 2006. The majority of these checks (58.6 percent) came back positive meaning that the greater part of the prescriptions created within OnCallData™ were informed by eligibility-based formulary. Based on these results it would be fair to state that the standard works as intended for the purposes of supporting an e-prescribing process.

CONCLUSIONS:

Overall, pilot site test findings show that the commonly-used parts of the batch NCPDP Formulary and Benefits standard have been successfully implemented among a variety of e-prescribing partners, and that this implementation may deliver some value in approximating patients’ drug coverage. However, technical and implementation factors prevent the realization of much of the standard’s potential benefits. The industry would likely be capable of adopting at least some of its features as an Initial Standard for e-prescribing in 2008, but any such mandate may need to further specify how the standard should be used and how it should evolve based on additional research and development.

The difficulty is not necessarily caused by the standard itself, but is based on the fact that formulary and benefits data is complex and difficult to comprehend due to its clinical nature. Implementing the data into the user interface requires input from someone with a level of clinical knowledge. In addition, different payers have different business rules that they may require the POC vendor to follow. In addition, the large volume of data that is supplied by the PBMs/Payers due to the number and complexity of formularies and benefits maintained adds another level of difficulty in the implementation.

Determining the actual coverage for a particular prescription is based not only on relatively constant data like the formulary placement of a drug for a particular plan but also on patient-specific factors such as the patient’s year-to-date drug expenditures, which can change over time. A good example of this is Medicare Part D, which has a maximum benefit payout and then the patient must pay for the next prescriptions out of pocket until another tier is reached and the benefit then pays again (i.e. the Part D “doughnut hole”). One solution to this challenge would be to provide patient-specific benefit information with a real time transaction standard. This option would require the need to migrate the formulary standard to a messaging standard and exchanging at least portions of formulary files as a real time transaction. Additional work and evaluation is needed to determine which portions of the Formulary and Benefits standard would be suitable for a real time transaction set.

The pilots did identify the need for full eligibility data to enable proper use of the formulary and benefits function. The next version of the eligibility standard ANSI x12N 271 (version 5010) is expected to alleviate some of these challenges. Another solution would be a patient unique medical ID (identifier) that could be transmitted to verify eligibility.

Medication History

The Medication History standard provides a uniform means for prescribers, dispensers, and payers to communicate about a listing of drugs that have been prescribed or claimed for a patient within a certain timeframe. Other information about medication history may include the pharmacy that filled the prescription and the physician that wrote the prescription. This standard is relatively mature and widely adopted by the prescribing industry. It has been shown to be useful in preventing medication errors, adverse drug events, as well as understanding medication management compliance.

Methods utilized by the pilot sites included testing of medication history from: only dispensed medication sources (retail pharmacy through SureScripts), only payer/PBM sources (through RxHub), as well as a mixture of both prescription sources. Results demonstrate there is a difference in how the standard is implemented based on the source of the medication history.

MA-SHARE- Brigham & Women’s

Testing of the Medication History standard included several components obtaining live medication history data (pharmacy-based dispensed prescription data) to compare against prescribed medications. The pilot also mapped the Medication History standard to the New Prescription standard to identify potential interoperability issues.

Did the standard work? Overall, the Medication History standard was found to be adequate for transmitting accurate prescribing information. The pilot determined that industry maturity and adoption of the Medication History standard strongly supports the use. The pilot testing of this standard uncovered a few issues with completeness and quality of the medication history data sourced from pharmacy-based dispensed database. There were a large number of existing records that did not comply with the standards.

Issues raised: Linking of medication history transactions is cumbersome in the absence of a unique Rx identifier that is known to both the prescriber and pharmacy. Coverage of medication history data from various sources differs considerably based on legal restrictions, readiness of pharmacies to contribute data, restrictions imposed by health plans, lack of data sources for certain patients, and various other factors. For example some State laws prohibiting provisioning sensitive patient data, including data related to HIV/AIDS and mental health conditions, applies to payer-sourced medication data but not to pharmacy-sourced medication history. Thus the pilot was able to retrieve medication history from dispensed medications without applying filters. Also noted as problematic in the implementation of this standard are the varying vendor requirements for exchanging prescriber and pharmacy data and the lack of standardization for presentation of the information via system interfaces.

RAND

Did the standard work? Overall, the Expert Panel evaluation resulted in members feeling that the structure, format and code lists associated with the medication history function of NCPDP SCRIPT are sufficient.

Issues raised: The optional nature of many data elements in the standard allow valuable information to be left out of the medication history records. These data elements include: 1) prescriber identity, 2) directions for use, 3) quantity dispensed, and 4) dispensing pharmacy.

As in the Brigham and Women’s pilot, there was general agreement that it would be beneficial to receive a recognizable “Prescriber ID” or “Prescriber Name” with each history transaction. This would assist a physician in recognizing their own prescriptions as well as those written by other physicians in their office. This data is also useful in matching a history record back to the original prescription record. If it cannot be determined who wrote a prescription in the history, it is difficult to be certain that a history record is reflecting a particular record in the physician’s system. The prescriber identifier is an optional data element and many times not present in data generated from claims data. In addition even when a prescriber ID is present, there has not been a standard physician ID that is carried throughout the prescribing transaction process. DEA numbers have routinely been used in claims processing to identify physicians but the DEA number is not sufficient for this purpose and is not always present. The NPI number may help in this case but it is not yet clear whether the NPI will resolve this issue. Research needs to be done to determine if the NPI is an appropriate identifier for this purpose and if current NPI implementation in the marketplace will produce useable identification data.

Another important challenge of the current implementations of the medication history function of NCPDP SCRIPT is that the ability to request a medication history for a particular patient is controlled by the requirement to perform an eligibility check against the master patient index of the medication history provider. If the eligibility check is successful then a history can be requested.

The current medication history standard does not sufficiently handle the returning of prescription records containing compound drugs. Specific to this issue, one of the Expert Panelists stated that “Compounds are not adequately represented by the industry. Today, all processors (including pharmacies and PBMs) are forced to use made-up NDC numbers to represent compound drugs, as there is not an industry identifier used to represent compounds.”

The return of duplicate history records also has been stated as an issue. Duplicate records will lead to improper DUR alerts such as duplicate therapy warnings when, in fact, the multiple records are for the same prescription. This is not a limitation of the standard itself per se, but of the way the request is submitted, therefore, POC vendors have to be aware that a duplicate record could be returned, and come up with an appropriate strategy for handling this situation.

The most prevalent issue reported is the inability for a POC vendor’s system to properly identify a drug returned in a history record. These issues arise because there is no standard drug identifier, because there are multiple drug database companies licensing their proprietary databases into the POC marketplace. In addition the medication history standard does not require any drug identifier to be present. A possible solution to this issue is the adoption of RxNorm as the standard drug identifier.

SURESCRIPTS

Did the standard work? In general the pilot found that the standard is structured well and suits itself to the exchange of information.

Issues raised: The challenge is how providers and vendors are making sense out of the data.

There are additional data sets that could be exchanged that would provide additional value to prescribers when viewing medication history. In general, SureScripts considers the retail pharmacy medication history data to be comprehensive and more current than the data available from claims adjudication process. For example, the current standard accommodates populating a field for medication instructions, but current pharmacy practice is that this information is not being sent to payer/PBM in the adjudication process. Other data fields available from the pharmacy data are allergy information (if available from pharmacy), over the counter medications, and any “cash-pay” prescriptions.

ACHIEVE did not test this standard.

Ohio-KePRO/UHMP

Prescriber’s initiating an e-prescription for a patient within OnCallData™, have the ability to view a patient’s paid prescription history (alternately referred to as “Medication History”) by pressing a button on the website titled “PBM/Retail History.” This capability has been in production at UHMP since before the project began, but it has changed over the course of the study period. At the beginning of the study, only insurance paid prescription drug data from RxHub was being transferred. Thus, if a patient paid cash for a drug, and a claim was not submitted to their insurer/PBM, the prescription would not appear in OnCallData’s Medication History display. However, beginning in October 2006, OnCallData™ began pulling dispensed prescription information via SureScripts and merged it with paid prescription claims data from RxHub.

Did the standard work? Medication History Standard had been in production prior to and during the pilot phase, working as it is intended—transmitting and generating a large volume of medication history information to the POC software (OnCallData) from the payer/PBM source via RxHub. The testing was focused on promoting and driving the usage of this information by physicians—not a standards issue per se but an implementation challenge.

Issues raised: While Medication History has been available during the entire study period, it has not been viewed very often. This is likely due to the fact that a majority of users were not aware that is was available. Modifications were made to the user-interface, as it was not intuitive to the physicians where on the application this information was found. Also, a huge challenge for the vendor application is how to reconcile and present medication history data that is coming from multiple sources, i.e. paid claims via RxHub and dispensed medications via SureScripts.

CONCLUSIONS:

Medication history is available from a number of sources, but not one provides a comprehensive complete listing. SureScripts data includes dispensed medications from retail and independent pharmacies; RxHub provides medication history from PBM/payer sources but not the entire universe of commercial PBM/payers. This source of history is claims-based and may not include information on self-pay or the uninsured population. The lack of a universal source for this information has limited clinician’s willingness to access medication history believing that the information is not complete enough to provide real value. To promote widespread adoption of this standard it is recommended that it will be necessary to reconcile data from a large number of sources to provide complete enough information to prove useful.

For complete records, interoperability is required with the NCPDP Telecommunications 5.1 standard and with HL7-based prescription orders. Medication history records that are created from pharmacy claims data rely upon the data transmitted from the dispensing pharmacy to the PBM during the claim submission process using the NCPDP Telecommunications 5.1 standard. There were no apparent conflicts between the NCPDP SCRIPT medication history standard and the NCPDP Telecommunications standard. It was generally agreed that there are appropriate and available data elements in the Telecom standard to support the data needed to populate the medication history records. However, it will take a coordinated effort between all involved parties to agree on implementation rules.

One of the intermediaries who are actively providing medication history services has implemented a mapping between records of HL7-based prescription orders and NCPDP SCRIPT-based medication history records. Although representatives of the intermediary reported minor mapping issues, overall they are satisfied with the interoperability of the two standards.

Fill Status Notification

The Fill Status Notification is part of the NCPDP SCRIPT standard, but it was not named a foundation standard due to lack of industry experience. The standard covers notification from a pharmacy to a prescriber when the prescription has been dispensed (medication picked up by patient), partially dispensed (partial amount of medication picked up by the patient), or not dispensed (medication not picked up by patient) and medication returned to stock.

Pilot testing of this transaction and functionality varied widely among each of the pilot sites. One approach used an alternative standard, “Medication History” message type to proxy the result of a Fill Status request. Several of the pilots evaluated the Prescription Fill Status transaction in lab environments.

MA-SHARE- Brigham & Women’s

Testing of Fill Status Notification was conducted via interviews to understand workflows and issues associated with the transaction to assess the business and clinical usefulness as well as to detect any potential implementation challenges.

Issues raised: The NCPDP SCRIPT v8.1 standard does not require a unique identifier that can be maintained through the life cycle of the prescription. If the original prescription was received by the pharmacy through e-prescribing, the original transaction number can be returned in the RxFill message to assist in matching. However, if the pharmacy received the prescription in a non-electronic format, no transaction ID will be available so other data elements must be used for automated matching. This adds additional complexity and resources on both the prescriber and pharmacy workflow. Data elements such as a unique patient or record ID might allow a fully automated match between RxFill transaction and patient record.

The low level of industry adoption and resulting lack of best practice experience makes the fill status notification a less desirable feature of e-prescribing.

RAND

Testing of Fill Status was initiated in this pilot by first reviewing the list of issues as stated in the NCPDP White Paper on this subject.

Did the standard work? The RxFILL format and structure of the NCPDP SCRIPT standard does work as it is intended, however due to implementation and unresolved business issues this standard continues to be rarely used in the industry and does not appear ready to be included among Medicare’s Initial Standards for e-prescribing.

Issues raised: Since RxFill is a “push” from the pharmacy to the prescriber, pharmacy software systems are critical to this transaction. The recipient of an RxFill transaction is the point-of-care (POC) vendor system used by the prescriber. It was suggested that work would need to be done as far as integration between point-of-care and pharmacy systems, and that there would need to be development of a reconciliation tool. The biggest effort is in matching the RxFill transaction with the original prescription in cases when the originating SCRIPT reference number is not provided. (This is an optional field in the RXFILL transaction.)

Prescribers who participated in focus groups raised additional concerns about potential new burdens that Fill Status alerting could place on their time and office staff time, especially from alerts that would imply the need for outreach to the patient beyond the counseling that they would normally undertake during the patient visit. Prescribers were genuinely interested in whether their patients were taking and following prescribed treatment regimens and if notification of NOT filled were presented they would want to take action based on that information.

Another concern raised by the focus group participants was new medico-legal liability that could result from the existence of non-adherence alerting. Confidentiality and privacy issues around RxFILL should also be better addressed by the industry. The pilot concluded that additional research on both patient- and physician-level opt-in or opt-out needs to be conducted.

One of the expert panelists who has experience with multiple POC vendor systems observed that “modifications are needed universally to implement RxFill, as the transaction was not previously utilized by the industry.” It is recommended that further research be undertaken to identify specific circumstances in which medication adherence could provide a sufficient return, in terms of health or cost savings in reduced service use, to make the financial incentive for physicians and pharmacies sufficient to develop and support RxFILL

SURESCRIPTS

The SureScripts pilot tested the RxFILL message type in conjunction with the medication history message type. This process entailed an intermediary receiving daily dispensed medication updates from community pharmacies and then generating and sending an RXFILL message to the prescriber where it is then mapped to the appropriate medication in the patient’s drug profile and accordingly displayed. The primary focus of this pilot testing was to evaluate and provide a business case for the full scale implementation and deployment of this message type, and exploring the acceptability and potential value of presenting information about patients’ adherence to their medication regimens based on data from the fill status transaction compared to using the medication history transaction alone.

Did the standard work? The preliminary results conclude that the standard from a technical level does work.

Issues raised: SureScripts hypothesizes that the medication history message includes all the information contained in the RxFILL message and thus considers it rather redundant to implement the latter. Benefits from use of this transaction include addressing patient compliance including monitoring the use of controlled substances, managing chronic conditions and identifying reasons for non-compliance such as inability to pay or misunderstanding of dosing instructions. Some of the barriers included additional workload and potential liability issues. There still remain some key challenges in solving issues around the implementation of this standard. Additional analysis will help answer some of these outstanding questions.

ACHIEVE

Several modifications were needed to the NCPDP SCRIPT v8.1 to meet the e-prescribing needs of the LTC environment. Fill Status is an example of modifications made to allow this transaction to be useful in a LTC setting.

Did the standard work? Yes, however anecdotal feedback from the treatment facilities indicated that the Fill Status content was not referenced enough during the pilot to ascertain the potential value for this standard in the LTC environment.

Issues raised: Fill Status supports outbound messaging from the pharmacy to the provider. The pharmacy sends a FILL status to the nursing home when an order is ‘filled’ in the RNA pharmacy application. The FILL is utilized in this environment to indicate what exact packaged drug is coming, with the exact instructions that will be on the label. The Fill Status conveys changes made by the pharmacy such as generic substitutions, strength changes, and SIG (direction) variances. A slight change was made to this standard which sends pharmacy fill information from the pharmacy to the nursing facility. The pilot added a Not Filled Reason Code and a Date with a text reason to the Fill Status SCRIPT to enable the pharmacy to indicate why they are not filling an order because communicating this information was important to the facility and the pharmacy.

Ohio-KePRO/ UHMP

Did the standard work? RxFill transactions were successfully tested in a production setting (final transaction volume reporting pending), albeit the messages were not generated by pharmacy systems directly. Instead, the messages were generated indirectly – by SureScripts – after the transfer of prescription data to SureScripts’ prescription history repository by participating pharmacies.

Issues raised: The requirement of an intermediating entity (other than merely a transaction routing / certifying entity) created an additional moving part for RxFill that may have directly contributed to both initial failed attempts to put RxFILL into production at our practices. In the first instance, a critical data element (NDC code of the dispensed drug) was inadvertently omitted from the RxFILL transactions by SureScripts; in the second, SureScripts failed to capture the names of the providers participating in the Ohio test. Under normal pre-production testing circumstances, both of these problems would likely have been detected before RxFILL was moved into production. These two problems thus have no relevance to the RxFILL standard itself. Nonetheless, the pilot’s experience suggests that using a prescription repository as an intermediating entity in RxFILL transaction generation creates additional potential failure points for RxFill transactions and highlights the need for especially vigilant transaction testing / certification between trading partners.

With or without this intermediating repository, the pilot found a significant lack of interoperability between NEWRX and RXFILL, largely in the form of a missed opportunity to use an originating order number (beginning with the NEWRX generated by the e-prescribing application) for loop closure when that e-prescribing application receives the corresponding RXFILL notice for that original prescription some hours or days later. Such originating order numbers are routinely used for loop closure in laboratory test ordering and resulting back to the ordering system.

A unique prescription order number is always created by the e-prescribing vendor, but it remains an optional component in NEWRX and does not exist at all in RXFILL (see Sec 6.4.8, p 29 of NCPDP Pilot Guidance Document). This should not be a required field in RXFILL, since RXFILL must be able to handle prescriptions not originating from an e-prescribing application in the first place (e.g., hand-written prescription or prescription printed from a computer and hand-carried to the pharmacy).

The intermediating repository complicates this issue further in that it creates yet another interoperability point that resides in between NEWRX and RXFILL. It is unclear what messaging standard is being used by pharmacies to transmit prescription data to SureScripts’ repository, but it is likely a SureScripts’ proprietary standard (the pilot is waiting for confirmation that this is indeed the case).

Another complication accentuated by (but not necessarily exclusive to) the intermediating repository is the issue of the RXFILL trigger: dispensed vs. picked-up. The clinical purpose of a RXFILL message is to let a prescriber know whether or not a prescription has been picked up, not whether or not the prescription was dispensed to a shelf to await pickup. According to SureScripts’ implementation practices, pharmacies can only send prescription data to SureScripts’ repository either when the prescription is dispensed or when it is picked up, but not both. The pilot was expecting that all three participating pharmacies – CVS, Walgreens and Rite Aid – would be sending prescription data to SureScripts’ repository only when picked up, and that RXFILL messages would only be generated by data received with a picked-up flag. However, they only recently learned (2^nd week in January) that one of the three pharmacies – Rite Aid – was sending dispensed data instead of picked-up data to SureScripts’ repository.

While the issues raised are not problems with the RxFILL standard itself, they did expose the vulnerability of assumption-based, rather than transaction-based, NoFill alerting to false positive alerts. In particular, any breakdown in the arrival of the RxFILL transaction itself or critical data elements within the RxFILL transaction will lead to false positive (e.g., NoFill alerting). This emphasizes the necessity of the RxFILL/NoFill messaging to occur directly between the pharmacy and the e-prescribing application, perhaps based on a return-to-stock event.

Although the missing NDC codes caused difficulties with matching the original prescription order in OnCallData’s database with the RxFILL message from SureScripts in this test, the NDC number is not a necessary part of the RxFILL messaging. Instead, if the prescription order number were a required part of the NEWRX standard the order number could be used to match to the RxFILL messages thereby eliminating the need to match on NDC number, patient name, etc. This is not an inoperability problem between the foundation standard and the initial standard because the order number is RXFILL/ NoFill notification function easier.

CONCLUSIONS

The standard is clearly sufficient to support the activities of pharmacy sending messages to the prescriber as to the status of a prescription. The challenges encountered are not related to the structure and format of the standard, but in its implementation. Currently there are pharmacy computer systems and technologies available today to track patient pick-up. But, in the majority of pharmacies today, the pharmacy system in use either does not have this capability, or, the pharmacy has not yet made the additional investment in technology required to do so.

RxFILL is intended as a transaction that would encourage adherence and compliance with medication therapy. As promising as the transaction is for that function, the pilots’ experiences and observations tend to indicate there is no marketplace demand for RxFILL. There are business challenges that also must be overcome prior to widespread implementation of this transaction. There was significant concern among prescribers about new medico-legal liability that could result from the existence of non-adherence alerting. Confidentiality and privacy issues around RxFILL should also be better addressed by the industry. Additional research on both patient- and physician-level opt-in or opt-out needs to be conducted.In order to fully implement this standard, pharmacy management systems would need to design and develop the capabilities to track, trigger, and send the RxFILL Status messages and the data supporting them as well as implement these changes and modify workflows as needed.

Prior Authorization

The prior authorization standard incorporates real-time prior authorization functionality in the ASC X12N 278 Health Care Services Review transaction. There were two models to consider – solicited and unsolicited. Under the solicited model, the prescriber requests questions from the health plan or PBM. Under the unsolicited model, the questions and criteria reside on the point-of-care software systems and the clinician knows all the questions needed for a particular drug before ending the Prior Authorization (PA) request. All pilot sites selected the unsolicited model.

The specific process for the unsolicited model of electronic prior authorization is as follows:

1) Payers and PBMs publish drug-specific prior authorization requirements using the NCPDP Formulary & Benefits file specification;

2) Prescribing systems use those prior authorization flags to alert prescribers of authorization requirements;

3) Prescribers provide needed information in the format of an electronic prior authorization request;

4) Prescribing systems submit electronic prior authorization requests to Payer/PBMs using the XI2 278 transaction, including appropriate patient information (diagnosis/conditions);

5) Payer/PBMs respond using the 278 response, and potentially note the authorization result in the claim adjudication system.

The pilot sites examined various approaches to assessing the potential impact of a standardized electronic prior authorization (e-PA) on the prescriber’s workflow, changes in prescribing behaviors and perceptions of access to appropriate medications both in lab environments and live implementations.

MA-SHARE- Brigham & Women’s

This pilot developed a four-pronged approach to comprehensively test e-PA standards, which involved an electronic test harness, a data map, interviews with IT professionals at provider groups, and interviews and a focus group with prescribers and other key stakeholders. As part of the testing, they devised 120 test scenarios to execute in the test harness and mapped fields from the 278 and 275 transactions to evaluate the coherence and completeness of the e-PA standards. The purpose of the data map was threefold: (1) to identify redundant elements, i.e., elements that were included in both transactions, (2) to determine which elements would be required for implementation, and (3) to assess which elements would be necessary to link transactions together.

Did the standard work? Using the HL7 PA Attachment requires the use of Logical Observation Identifiers, Names and Codes (LOINC) codes. The LOINC standard does not contain all of the questions that payers require to conduct prior authorization. Further, the LOINC standard forms ask questions not currently required by the payers. The LOINC standard should become more flexible to adapt better to the needs of the payers.

Issues raised: Prior Authorization is a very complex standard to implement for IT professionals. It requires the understanding of four different standards and multiple payer requirements. There are inconsistencies between the 278 and 275 standards which need to be addressed. The element names should be made consistent where the content is the same and unnecessary redundancies should be eliminated. The e-PA standards should delineate an electronic process for patient notification when coverage is changed or when a final decision on a prior authorization request has been rendered. Also, the e-PA standards need to address quality care dosing (QCD) overrides. Currently, there is no way for a physician to justify QCD overrides using the LOINC standard. The task group should incorporate a field for QCD justification into its existing forms or the 278 transaction if QCD will always require justification. To maximize efficiency of physician workflow the documentation for e-PA standards should provide guidance on how to prioritize PA requests electronically to denote urgency.

RAND

The aim of the pilot with respect to prior authorization testing was to assess the potential impact of standardized e-PA on prescribers’ work processes, prescribing patterns, and perceptions of access to appropriate medications. To provide criteria to iScribe and Allscripts, investigators built a ‘file transfer protocol’. Both Allscripts and iScribe built working modules within their point-of-care software that enable prescribers and their staff to request prior authorization (PA) electronically, and this was certified by Caremark. RxHub modified its electronic prescription routing system to validate and transmit e-PA transactions using the X12N 278 and 275 and PA Attachment standards. They also certified Allscripts and iScribe on all transactions for format and content. RxHub also built a portal that Caremark’s PA staff utilized to approve or deny e-PA requests. Allscripts, iScribe, RxHub and Caremark compared the HL7 PA Attachments to the Horizon PA forms. The coalition compared Horizon’s PA forms (mostly yes/no questions) against the HL7 PA Attachment that had been completed via a consensus-based standards development process, and represented them in the user interface.

The HL7 PA Attachment requires the use of LOINC codes. During the pilot, investigators asked HL7 to create custom-question LOINC codes to support the majority of Caremark’s PA questions, which could not be represented with existing LOINC codes.

Did the standard work? The pilot project recommended that additional work be done to evaluate its effectiveness and usability for widespread industry use.

Issues raised: Investigators agree with the NCPDP task group that HIPAA-named prior authorization standard – the X12N 278 v4010 – is not adequate to support drug prior authorization because it was designed for service or procedure PA, not for medication PA. Work-arounds designed for the X12 278 v4010 were not sufficient to facilitate e-PA because it does not support the transmittal of information required by the health plan or PBM to make a decision as to whether to approve or deny the request.

The X12N 278 v5010 assumes that there will also be an X12N 275 (wrapper) and an HL7 PA Attachment using either the solicited or unsolicited model. The HL7 PA Attachment was designed to define the allowable and/or required content for the PA request and the structure for the infrastructure for the transmission of that content so that a health plan or PBM can approve or deny the request. Investigators observed that this is a complex model that requires expertise with multiple standards and standards development organizations, which may limit adoption.

The predefined therapeutic categories did not support the PA requirements of Horizon, which creates PA forms on a drug basis. The same data is required in multiple places of the different transactions, creating confusion for both the POC software vendors, physicians and back-end receiving system. Files were required to pre-populate the physician system with data for the unsolicited model. The unsolicited model requires an initial load of a question for a drug or drug class and then a method to update (by either updating individual questions or completely replacing all questions). This capability was not built into the unsolicited model.

The need to use LOINC codes may delay the process of creating new questions and criteria for months, whereas Horizon expects Caremark to implement new questions and criteria within days.

“If the questions themselves are standardized, there is no clear process for a quick and easy way to update questions. Health plans and other clients review and update PA criteria on an ongoing basis and the standards need to accommodate that process.” noted Caremark investigators.

The underlying assumption of the PA model was that the industry would be willing to standardize questions. Caremark investigators noted why this is unlikely:

“Health plans can be very particular about the wording of the PA questions – these questions go through multiple reviews by both the PBM clinical team and the health plan/client clinical team. Trying to standardize the questions themselves will expend more effort with little assistance in the adoption of e-PA. Since not all prescribers are going to be connected, health plans will need to support both an electronic and paper process. These two processes must present the same criteria. By trying to standardize specific wording, these standards are effectively asking health plans to standardize on the way they implement PA across the board.”

Additional LOINC codes were needed for custom questions.

One of the challenges of the HL7 PA Attachment is that it does not support content rules (including conditionality) or question sequence. Thus, vendors are not able to make questions mandatory. Making fields mandatory would ensure that the information required is completed and reduce the need for the back-and-forth that currently takes place between PA reps and prescribers. Recommendations of the investigators suggest that a separate field should be provided to accommodate the question number. Vendors should not be required to assume that sequence number and question number are synonymous.

The standards do not support the addition of open-ended comments to PA requests. A free text comments field is needed to allow prescribers to add information they feel is important to the PA request but that may not be covered by the questions. Some of the information presented to prescribers in the PA forms includes educational information about the medication or directions for completing the form rather than questions. It is important for the standards to support such information. The ability to provide information (rather than just questions that need to be answered) on the PA form would be helpful whenever there is a need for additional instructions. It would also be helpful to be able to provide a title for each electronic PA form (e.g. “Horizon Celebrex PA Form”) – mimicking what is on the top of the current paper PA form.

The PBM’s unique member ID and cardholder ID would be valuable to have for both back-end processing and for display purposes. Both the X12 278 and X12 275 require the use of an ID, but not the member ID and the PBM Unique ID. “Member ID is critical for the PA technicians to ensure that they have the correct member for the PA,” noted Caremark investigators.

The proposed standards assume that vendors have ICD-9 codes and that they will be used to answer questions. Currently there are issues with ICD-9s, including the level of specificity of ICD-9 codes is inconsistent across PA categories, combining the request for diagnosis and co-morbidities in one question, and questions on broad conditions.

SURESCRIPTS did not test this standard.

ACHIEVE

Achieve integrates the electronic prior authorization (e-PA) transaction into the Matrix system using the X12 278 standard. The prior authorization transaction allows a prescriber to request authorization during the prescribing process and tracks authorization responses with the order for future reference. During the pilot, trading partners employ the following prior authorization automation messages and processes: 1) Payer / PBMs will publish drug-specific prior authorization requirements using the NCPDP Formulary & Benefits file specification; 2) Prescribing systems key off those prior authorization flags to alert prescribers of authorization requirements; 3) Prescribing systems submit electronic prior authorization requests to Payer / PBMs using the X12 278 transaction, including appropriate patient information (diagnosis/conditions); 4) Payer / PBMs respond using the 278 response, and potentially note the authorization result in their claim adjudication system. Prior Authorization for the provider & pharmacy using X12 278/275 with HL7, LOINC, & XML attachments.

Did the standard work? The pilot project reports, “Since its implementation, we have demonstrated that this electronic prior authorization process does work, and the real-time PA request status for an order is available to the nursing facility software application (for the prescribers & agents of the prescriber) and to the pharmacy software application. The pilot testing demonstrated that the electronic Prior Authorization process could be implemented in a practical and effective manner, which all required information could be communicated electronically, the required time of the prescriber could be minimized, and the process status could be made visible to the facility and LTC pharmacy through minor adaptations to their existing systems. During the two months of implementation, there were only a limited number of electronic PA request submissions; therefore further research is required to thoroughly demonstrate its value in the LTC environment.

Ohio-KePRO/ UHMP

The PA test was architected according to RxHub’s “unsolicited” model. Under this model, Anthem converted the questions for the eight drugs to be tested into the format designated by RxHub, so that the questions could be incorporated into and transferred with the Anthem formulary file via the Formulary File transfer initial standard. Formulary matching during a prescribing session is predicated upon a prior successful eligibility check utilizing the ASC X12N 270/271 standard) presenting a natural opportunity to assess interoperability between a foundation (X12 270/271 eligibility check/response) and two initial standards; Formulary and Benefits and Prior Authorization.

Did the standard work? Yes, however the PA team felt that the conversion of the PA questions to the format required for inclusion in the formulary data file was “not an easy task”, even for only the 8 drugs involved in a health plan/payer may have and how often they change, and then multiply by the number of plans, it becomes a massive task.”

Issues raised: Although the standard works, the implementation was complicated by the need to comply with four different standard implementation guides – some of which were in various stages of completeness and usability. Streamlining of the inputs would be desirable and more efficient, as the same data is required in multiple places (278, 275 and PA Attachment) Some accommodations need to be made as the “predefined therapy categories” currently do not meet the requirements of payer forms, does not support the unsolicited model well and there is an inability to standardize questions. Some of the workarounds/modifications made to satisfy the standard’s implementation included; a requirement for conditionality of questions – modified and tested conditionality in this pilot, new requirement for ‘check lists’, development of additional LOINC codes to accommodate custom questions (most questions are customized) and the need to allow for comments or additional text.

CONCLUSIONS

The e-PA standard has the potential to improve operational efficiencies for providers by standardizing payer processes. With the current paper process, provider groups face challenges such as losing forms, manually researching detailed patient information, and staying abreast of the latest payer requirements, as well as timeliness of responses to enable appropriate treatment. The proposed e-PA process could facilitate tracking of authorizations, automatically populate relevant patient information in applications, and simplify the overall system. That being said, some changes to the e-PA process will be required to further increase the usability of the standards and the efficiency of implementation. Based on the technical evaluation of electronic prior authorization (e-PA), it is recommended:

§ There should be a standard-based means to support e-PA within the context of e-prescribing, with the intent of some day having transactions that support a fully automated, real-time process.

§ Because the combination of the X12N 278, X12N 275 and HL7 PA Attachment is cumbersome, confusing and requires expertise that may limit adoption, one standard transaction should be considered – one that is specifically designed for medication e-PA. This standard should be built with the assumption that criteria can be pre-loaded into point-of-care (POC) software systems (the unsolicited model) and should be a) organized by drug, b) support content logic (conditionality), numbering of questions and cardinality, c) provide for educational information and directions, d) support open-ended questions and e) uniquely identify the patient.

§ The NCPDP Formulary & Benefit Standard should be leveraged to transmit criteria to the point-of-care (POC) software vendor. (Note: An NCPDP task group has already been formed to develop a new file type within the NCPDP Formulary and Benefit standard to support this.)

§ The focus for standard e-PA transactions should be on the format and infrastructure of the transaction rather than the wording and standardization of the questions themselves.

§ Additional research should be done on electronic prior authorization (e-PA) to a) test the recommendations above, b) determine the return on investment of modifying the existing e-PA infrastructure, c) better understand cost of managing two systems (electronic & traditional), d) provide patient-specific PA status. More health plans, PBMs, physicians and categories should be involved, so that additional perspectives on how criteria can be analyzed and the return on investment can be calculated more broadly than just for one study participant.

It would be appropriate to make these modifications and evaluate prior to widespread adoption as a final standard for Medicare Part D e-prescribing program.

Structured and Codified SIG

Patient instructions for taking medications are placed at the end of a prescription. These are called the signatura, commonly abbreviated SIG. Structured and codified SIGs will enhance patient safety, although it is also recognized that free text capability must be preserved for special circumstances. NCPDP, HL7, and others are working on addressing structured SIG components and plan to seek broad industry participation. At the time of the pilots’ initial start date, the likelihood that the proposed standard would be balloted and adopted by NCPDP was not a near–term prospect. As a result, the pilot sites agreed to test NCPDP’s proposed Structured and Codified SIG concept in a laboratory setting.

Each of the pilot sites chose a different approach to testing this standard. Approaches included; review of the proposed SIG standard, identification of test cases, using live transactions and selecting samples of prescriptions with a wide variety of sigs, recreating each test case in laboratory environment, and then developing a test harness that would include functions of an electronic information exchange application. Another approach was to analyze an initial sample (significant in number) with an attempt to represent each distinct sig using the proposed standard’s 128 data fields. This effort has been a cooperative, coordinated approach to provide more comprehensive and robust information regarding the appropriateness and usefulness of the proposed standard for wide-industry use.

MA-SHARE- Brigham & Women’s

Discussions with providers and IT professionals during the pilot phase demonstrated that Structured and Codified Sig addresses a need in the medical community. With additional development, the proposed standard may provide a controlled vocabulary that reflects prescribing thinking, offers structure and simplicity, and improves communications between prescribers and pharmacies.

Did the standard work? Structured and Codified SIG’s highly flexible design, coupled with a lack of explicit guidance around the standard’s implementation, results in a system that is both complex to execute and difficult to understand. Currently, the standard requires both free text and codified SIG to be sent to the pharmacy. This may cause inconsistencies in the data that is transmitted. The need to mandate the transmission of both SIG formats should be re-evaluated.

Issues raised: Because of the highly flexible design of Structured and Codified Sig, coupled with the lack of explicit guidance around the standard’s implementation, this results in a system that may prove difficult to understand and complex to execute. The standard employs 14 segment types, each of which is further broken down into subsegments. It also has over 1,300 terms in its database. The SNOMED terms contained in the standard’s database are hard to classify into the 14 segment types, further complicating the interpretation of the Structured and Codified SIG. Providers have indicated that tabbing through multiple fields, each with a large number of options available (due to high flexibility mentioned earlier) creates additional burdens on workflow and is often cumbersome.

Furthermore, the database of SNOMED terms is not wholly adequate for devising usable SIGs. Providers regularly use ranges, dates, diagnostic codes, and lab test results in their SIGs. The standard either does not support these items, or it does not provide sufficient guidance on how to enter this information into a Structured and Codified SIG format. The documentation on Structured and Codified SIG should incorporate – or more clearly define - how to use such elements when deploying the standard. The Structured and Codified SIG standard does not currently allow for prompt revision of its fields in the event of new methods of drug administration. Methods to update the standard more efficiently should continue to be explored. Based on the conclusions from this pilot, they do not recommend its use as a final standard for Medicare Part D e-prescribing in its current state.

RAND

Did the standard work? Overall, the pilot found low agreement among attempts to represent the same prescription information, suggesting that the standard is unlikely to be ready for adoption as a requirement for e-prescribing in 2008.

Issues raised: The pilot recommends that more examples be added to the SIG format Implementation Guide to show the types of data that are intended to be mapped into each field. In addition, correct or further explanations of the apparent inconsistencies in field definitions and examples are needed in the Guide. For example, the Implementation Guide suggests the word “every” in a manner that contains “every x hours” should be mapped to the Frequency field whereas, intuitively, it would seem the word “every” should be mapped to the Frequency Units Text field. Additional research is needed to identify instances in which confusion about field names and uses leads to misinterpretation of the prescriber’s instructions, which in turn could lead to drug therapy mismanagement and jeopardize patient safety.

Field names containing both the words “units” and “text” caused confusion for the mapping reviewers and should be simplified. These included Dose Units Text (we recommend renaming the field to Dose Text), Frequency Units Text (we recommend renaming the field to Frequency Text), Interval Value Units Text (rename to Interval Value Text), Dose Maximum Value Units Text and Dose Maximum Variable “Units” Text (rename to Dose Maximum Value Text and Dose Maximum Variable Text).

Additional definitions and examples are needed to clarify the intent and use of the Indication Segment and how its application is different from the Administration Timing, Frequency, and Interval segments. The Free Text String Indicator should be clarified, and examples should be provided in the Implementation Guide to avoid misinterpretation of the prescriber’s instructions to the patient.

SURESCRIPTS

The SureScripts pilot tested codified SIG in controlled stimulated lab environment. The plan was to analyze and document the mapping and conversion of prescriber entered free text SIGs into appropriate SIG codes, assess the exact display of the prescriber in the pharmacy and evaluate the feasibility of usage for all message types. The pilot conducted a survey tool to give an assessment of various scenarios around SIG implementation eliciting reactions to overall perceptions of the approach of guiding the standards development, reactions to specific coding scenarios, for example complexity for coding instructions from simplest to most complicated), and the perceptions of administrative burden related to operationalizing the proposed standard.

Did the standard work? Preliminary conclusions suggest the direction that NCPDP went was too complex and too difficult to implement in its current format. Experts can’t agree on how to use it. Investigators for this pilot came to the conclusion that it’s not ready for widespread use.

Issues raised: Several of the respondents reported they can accommodate the SIG with utilizing between 6-8 fields. A survey of knowledge base vendors revealed multiple NDC matches. These should be addressed before implementation. Results of the pharmacy-to-prescriber study are not yet available.

Ohio-KePRO/UHMP

Issues raised: Of the 45 fields represented for the mapping exercise, 10 (22%) were not used by any reviewer for any Sig. These unused fields were the “rate of administration” and “rate unit of text” fields from the dose segment, all six of the fields in the dose calculation segment, and “multiple route modifier” and “indication value units.” Among all 41 Sigs that were mapped, there were no instances in which any two reviewers agreed on the representation across all segments and fields. When Sigs contained multiple dosing and/or multiple frequencies, such as “1 to 2 tablets” or “every 4 to 6 hours”, none of the reviewers correctly identified the proper use of the modifier fields for variable dosing or variable frequency. Also in these cases, the Sig Sequence Position was not utilized as described in the Structured and Codified Sig Format Implementation Guide. None of the reviewers correctly utilized the values for the Free Text String Indicator field.

CONCLUSIONS

In summary, the Structured and Codified SIG format needs additional work with reference to field definitions and examples, field naming conventions and clarifications of field use where new codes are recommended, such as the SIG Free Text Indicator field. Such research will improve adoption of e-prescribing, in general, and use of the Structured and Codified SIG format, specifically.

It is imperative that the prescriber’s instructions for medication use be translated exactly into e-prescribing and pharmacy practice management systems to realize the full value of these technologies in reducing medication errors, decreasing healthcare costs and improving patient safety. With additional development, the standard may provide a controlled vocabulary that reflects prescriber thinking, offers structure and simplicity, and improves communications between prescribers and pharmacies. It is not recommend for use for Medicare Part D e-prescribing in its current state.

RxNorm

RxNorm, a standardized nomenclature for clinical drugs, is produced by the National Library of Medicine. In this context, a clinical drug is a pharmaceutical product given to (or taken by) a patient with a therapeutic or diagnostic intent. In RxNorm, the name of a clinical drug combines its active ingredients, strengths, and form. It provides links from clinical drugs, both branded and generic, to their active ingredients, drug components (active ingredient + strength), and related brand names. NDCs (National Drug Codes) for specific drug products (where there are often many NDC codes for a single product) are linked to that product in RxNorm.

RxNorm follows a standard format in the naming of clinical drugs. Drugs named in disparate ways in various other vocabularies are normalized according to RxNorm’s naming conventions. There are specific rules and naming conventions used in RxNorm. The SCD—the semantic clinical drug, or normalized form of the generic drug name—always contains the ingredient(s), the strength, and the dose form, in that order. The SBD—the semantic branded drug, follows a similar convention, with the addition of the brand name in brackets at the end of the name.

RxNorm links its names to many of the drug vocabularies commonly used in pharmacy management and drug interaction software. The Logical Observation Identifier Names and Codes (LOINC^®)database provides a universal code system for reporting laboratoryand other clinical observations. Its purpose is to identifyobservations in electronic messages such as Health Level Seven(HL7) observation messages, so that when hospitals, health maintenanceorganizations, pharmaceutical manufacturers, researchers, andpublic health departments receive such messages from multiplesources, they can automatically file the results in the rightslots of their medical records, research, and/or public healthsystems.[liii]

Currently, there are multiple systems using different databases to uniquely identify drugs. RxNorm is an attempt to create one standard format for drug names, with links from clinical drugs to their active ingredients, drug components, and most related brand names. An RxNorm name should exist for every strength and dose of every available combination of clinically significant ingredients.

RxNorm terminology is being evaluated in context to the NCPDP SCRIPT Standard for new prescriptions, renewals, and changes. RxNorm (versions 8/2/06 and 12/21/06) were included in the 2006 pilot sites to determine how well the RxNorm clinical drug, strength, and dosage information can be translated from the prescriber’s system into an NDC at the dispenser’s system that represents the prescriber’s intent. This translation has required the participation of intermediary drug knowledge base vendors as the RxNorm is not yet fully mapped. The pilot sites that tested this standard did so in a laboratory setting, specifically to gain understanding of the completeness and accuracy of RxNorm for representing a sample of new and prescriptions and renewal request that were actually transmitted between prescriber’s offices and pharmacies. Also implemented as part of the pilot testing was use of work process modeling, demonstrating the effects of using RxNorm in SCRIPT new prescriptions and renewal requests, semi-structured survey instrument, as well as Expert Panel process for evaluation.

MA-SHARE- Brigham & Women’s

This project implemented an RxNorm database, and then used a sample of medication history for approximately 6,000 BIDMC patients to match NDC codes from the sample to either RxNorm (version 12/21/06) Semantic Clinical Drug (SCD) or Semantic Branded Drug (SBD) strings, as appropriate. Next, a pharmacist compared a sample of the strings obtained from RxNorm with those drug names from medication history. Finally, they examined the interoperability of the RxNorm standard with the Formulary and SCRIPT 8.1 NewRx standards to determine the coherence of the standard. This entailed assessing whether Formulary and NewRx contained fields that were available for, and compatible with, RxNorm codes.

Did the standard work? RxNorm has the potential to simplify e-prescribing, create efficiencies, and reduce dependence on NDCs. If the standard were used both within payer formularies and within provider groups, it could decrease the complexities currently inherent in formulary lookup. The pilot testing concludes that the dictionary standard requires further evaluation and refinement before it can be deployed in a live setting. Conclusion is that this standard is not recommended for use in Medicare Part D e-prescribing in its current state.

Issues raised: RxNorm documentation requires further development to provide examples on RxNorm usage within a provider setting, as well as a set of more concrete examples on how to trace data within the RxNorm RRF files. It should also include a list of RxNorm’s limitations. In addition, there is no central repository containing a list of all NDC codes, nor is there a reference guide that indicates all of the NDCs associated with a particular drug. Currently, there are errors in the RxNorm database that cause some NDCs to be linked to ingredients rather than drugs. Although the database is designed for users to look up NDCs, roughly 12% of NDCs in this pilot’s test sample could not be matched with an SCD or an SBD. Improving the linking mechanism may help reduce errors and improve the lookup process.

The significance of medication packaging and standardized dosage to prescription drugs requires further assessment. Currently, RxNorm text strings do not reflect packaging information and rely on normalized dosing. Physicians, payers, and patients should be consulted to determine how packaging affects patient care, formulary status, and pricing. Additionally, pharmacists could provide input regarding how they would select a particular package based on the RxNorm code they receive from the physician’s office.

If RxNorm is to be expanded internationally, differences in terminology between the US and other countries will need to be resolved. Currently, a number of medications have varying brand names and generic names in different parts of the world. More research will be required to determine which terms to include in its database. The RxNorm documentation should provide guidance on how to use the dictionary standard when prescribing compounded drugs. The documentation does not currently address this issue.

Finally, a strategy for more widespread adoption of the RxNorm dictionary standard could be devised. One of the primary uses of RxNorm is for formulary lookup services. However, the standard does not contain terms relating to non-drug therapeutic devices such as wheelchairs and heart stents. Providers would like to be able to look up the formulary status for these types of devices.

RAND

This pilot evaluated the readiness of RxNorm (version 12/21/06) for use in e-prescribing transactions by assessing (a) the completeness of its Semantic Clinical Drug (SCD) concepts for representing a large sample of new prescriptions and renewal requests transmitted between a point of care e-prescribing system and retail pharmacies, and (b) the agreement between the Semantic Clinical Drug (SCD) concepts that were independently selected for each medication in this sample by two drug knowledge base vendors.

Did the standard work? Investigators reported, “Although our analysis of data collected in the evaluation of RxNorm is ongoing, our findings so far would indicate that RxNorm holds considerable promise as an interlingua for representing clinical drugs in e-prescribing transactions. However, members of our expert panel had varying levels of experience with it and overall they lacked confidence that they will be able to incorporate it as a requirement by 2008.”

Issues raised: RxNorm is intended to provide a single SCD identifier for each clinically distinct drug that is currently available by prescription. Testing of this standard reveal cases in which NDC codes did not match to an SCD indicating there has either been an error in matching to the correct RxNorm concept or an error with RxNorm itself, with more than one term being available for the same clinical drug concept (i.e. unresolved synonymy).

SURESCRIPTS

The focus of this pilot testing was to verify the accuracy of the RxNorm database as a cross reference to textual Medication Name/Strength/Form and to NDC number, verify the completeness of the RxNorm database to fully represent the pharmacist’s medication dispensing selection, evaluate the use of RxNorm as an additional verification tool to compare the prescriber’s original intent to the pharmacist’s dispending decision, and as a potential replacement in the future for the textual drug name used to communicate the prescriber’s intent in electronic transactions today. RxNorm (version 8/2/06) values were applied to a set of real prescription records in the test so that review and analysis can be conducted regarding the accuracy and viability of RxNorm for future potential use in a live environment. SureScripts will create the de-identified Original Prescription File which will be sent to prescriber vendor. The next step is to gather the mapped files from prescriber vendor and the participating pharmacy chain. These files will be processed / analyzed independently by SureScripts to cross reference and the two files will then have their corresponding prescription records compared to evaluate RxNorm. A Final Comparison File will be created to document the key values from both original files side by side for easy analysis and documentation of test results.

Did the standard work? Final analysis and results are still pending, however, early assessment is that there are still some critical outstanding issues that must be addressed and additional work must be completed before RxNorm is ready for widespread implementation and adoption. There is potential for RxNorm to decrease complexity representing clinical drugs in e-prescribing transactions, but additional work needs to be done before deployment in a live setting.

Issues raised: Accuracy and mapping issues were cited as being problematic. There were multiple SCD RxNorm concepts mapping to the same NDC code—this problem tends to occur when pooling information from multiple data sources or multiple RxNorm concepts are similar such that there is ambiguity as to how to make an accurate match.

Ohio-KePRO/ UHMP

Did the standard work? Additional work with mapping and matching issues needs to be resolved before this is ready for widespread implementation.

Issues raised: One of the problems cited in this pilot’s experience was reconciling brand/ingredient mismatching which stems from the granularity of the ingredient concept. In certain cases, the salt form is important to know to distinguish clinical concepts. Also, while one brand (barring reformulation) links with only one generic set of ingredients, an ingredient may link to more than one brand (e.g. Tylenol links specifically to acetaminophen, but not vice-versa). Another challenge is that RxNorm does not cover OTCs. Since OTCs are valid medication concepts, they should be scheduled for inclusion in some future version. Also cited as a deficiency is the conceptual limit of “medication” such as insulin. Inputting different dosage forms of insulin (pen, injectable, infusion formulation) return as “injectable.” In this case, although the ingredient and brand are the same, the way that the dosage form is delivered to the patient may be different. RxNorm’s design philosophy is designed on how “a clinician may order for a patient or administered” device differences (such as pens and cartridges) would constitute a clinical difference. This is one of the situations where the available granularity is not sufficient to describe clinical reality. These entries may be slated for local editing for situation-specific environments.

CONCLUSIONS

RxNorm (version 12/21/06) has the potential to simplify e-prescribing, create efficiencies, and reduce dependence on NDCs. If the standard were used both within payer formularies and within provider groups, it could decrease the complexities currently inherent in formulary lookup. However, the dictionary standard requires further evaluation and refinement before it can be deployed in a live setting. The evaluation team does not recommend its use for Medicare Part D e-prescribing in its current state.

NCDPD SCRIPT Standard v8.1

The NCDPD SCRIPT Standard is a foundation standard used for transmission of basic information about e-prescriptions, including not only new orders but also change, renewal, and cancellation of existing prescriptions. Achieve tested this standard and found that modifications were required in order to ensure accurate transmission of the data in the LTC setting. However, through partner agreement, “work-arounds” were identified and implemented. The pilot site submitted to NCPDP a formal request in the form of a DERF (Data Element Request Form) to modify this standard as needed.

Issues raised: In LTC, a prescription order typically remains an open order with no end date or a date far in the future. At times, a prescriber has the need to modify this order and notify the pharmacy. The changes would include the significant change of dose, form, strength, or route, or the modifications of frequency, or minor change related to the order. The prescriber system will always send a CANCEL and a NEWRX, regardless of the type of change. This process differs from the Change Request (RXCHG), because it is initiated from the prescriber not the pharmacy. With the request coming from the prescriber, there is no need for a response approving the request. The pharmacy, upon reviewing these changes, would determine if the original order needs to be cancelled or modified.

In the LTC environment there is a need to send a refill request from a facility to a pharmacy. An example use case is when a medication supply for a resident is running low (2-3 doses) and a new supply is needed from the pharmacy, the nurse needs a way to notify the pharmacy that a refill for the medication is needed. Typically, the physician is not involved in this process until the end of the month when all of the resident’s orders are signed in batch.

The proposed solution is to include the long term care flow of facility to pharmacy wherever the refill is currently mentioned as a message from the pharmacy to the prescriber. There is a need to maintain three separate identifiers for the refill request and response. Currently, the Census Update Transaction is originated by the facility in a long term care environment. The transaction notifies the pharmacy about census events. The transaction can be used in three cases - to notify the pharmacy of a new resident, a change to demographic information of a resident, or the discharge of a resident. In LTC these changes to the patient status happen regularly. They are not necessarily tied to a prescription, but they may drive processes at the pharmacy. Because of this, a message type is necessary to convey patient information “decoupled” from any prescription information.

SECTION VI: EVALUATION FINDINGS - OTHER OUTCOMES

While pilot sites were all expected to test the same initial standards, they were given flexibility to test different outcomes of e-prescribing. These outcomes included workflow issues, such as callbacks from the pharmacy to the prescriber, patient safety measures such as medication errors and adverse drug events, and other measures important to e-prescribing implementation, such as prescriber and patient satisfaction (see Exhibit 15).

Many of the pilot sites were continuing their analyses at the time this report was being prepared. For the purposes of this report, “completed” outcomes are those for which pilot sites included information in their final report. “Planned” outcomes were in the process of being addressed as of the last site visit, but pilot sites have not provided results. “Incomplete” implies that part, but not all of an outcome has been evaluated.

Exhibit 15. E-prescribing Outcomes Tested by Pilot Sites

	Achieve- LTC	Brigham & Women’s	Ohio KePRO/ UHMP	RAND	Surescripts
Prescriber uptake and satisfaction	Completed	Completed	Completed	Completed	Completed
Prescriber workflow changes	Completed	Planned	Completed	Completed	Completed
Workflow changes relating to verbal orders	Completed				Completed
Callbacks (pharmacy to prescriber)	Incomplete	Planned	Completed	Incomplete	Planned
Patient Satisfaction					Completed
Use of medication history functions			Completed	Completed	Completed
Changes in prescription renewal rates					Completed
Changes in new prescription rates	Completed	Completed			Completed
Inappropriate prescribing rates	Completed	Planned	Completed	Completed	Completed
Medication Errors		Incomplete	Planned	Completed	Completed
Adverse drug events		Incomplete	Completed		Planned
Hospitalizations and ED visit rates				Completed
Use of on-formulary/generic medications	Incomplete		Completed	Completed	Completed
Change in fill status rates				Completed	Completed
Benefit related to processing	Completed
Improved security and reliability	Completed

E-prescribing Outcomes

In the sections that follow, we summarize the results of each outcome studied, subcategorized by pilot team. We then provide concluding remarks based on these results, and taking into account any methodological or other limitations.

Prescriber Uptake and Satisfaction

Any assessment of care provider order entry functionality—with or without e-prescribing, would be incomplete without a discussion about the overall adoption rate and sustained use by clinicians. All five pilot sites tracked prescriber uptake and/or satisfaction.

Achieve

In the long-term facility environment, physicians generated a very small number of orders directly. Instead, agents of the prescriber (22 RNs and 38 LPNs) entered the orders (Exhibit 16). Nurses accounted for 95% of all orders entered into the e-prescribing environment. In their facilities, nurse practitioners and physician assistants were involved in the process. Increasing CPOE use by prescribers is one of the keys to positively affecting the quality of care and resident safety because the CPOE clinical alerts should be presented to the prescriber in order to more proactively affect the resident’s medication management. In their environment, the primary responsibility of agents of the prescriber has traditionally been to accurately record prescriber orders – not to evaluate clinical alerts from the CPOE system. Therefore, this pattern of adoption was not viewed favorably.

Exhibit 16. Achieve Orders Entered by Staff Type

Both Treatment Facilities	May	Jun	July	Aug	Sept	Oct	Nov	Dec	Total	% Total
Charge Nurse	0	76	571	813	700	491	739	647	4037	56.47%
Charge Nurse Limited	0	0	0	0	0	196	204	195	595	8.32%
Clinical/Financial	0	0	0	0	0	1	0	1	2	0.03%
Corporate Clinical Manager	0	13	0	0	0	3	0	0	16	0.22%
Director of Nursing	0	54	117	267	251	203	238	172	1302	18.21%
LPN	0	9	247	199	226	64	41	30	816	11.41%
MDS Coordinator	0	0	12	4	4	2	0	2	24	0.34%
NP/PA	0	0	0	35	80	66	45	71	297	4.15%
Pharmacist	0	0	27	0	0	0	0	0	27	0.38%
Unit Coordinator	0	0	13	0	0	0	0	0	13	0.18%
Unknown	0	1	5	1	7	1	5	0	20	0.28%
Total NEWRX Scripts	0	153	992	1319	1268	1027	1272	1118	7149	100.00%

Nursing Staff Satisfaction

Participants were asked to rate their level of satisfaction with the e-prescribing system on a scale from 0-10 with 0 being “very dissatisfied” and 10 being “very satisfied”. Nursing staff ratings ranged from 0 to 8 with an overall mean of 4.15 indicative of a slightly dissatisfied rating. The ratings by the staff of the two facilities differed with the rating at one facility staff as neutral (5.16) and at the other facility as somewhat dissatisfied (3.28).

A variation in the perspective of the staff was evident in that some described the use of the e-prescribing system as a hassle and others indicated it made their work easier. There were several reasons cited for their perception of being a hassle. The majority of the reasons were related to the failure and dependability of the system. The system was perceived to add additional steps to the medication order process as verification or follow up on the submission of an order was necessary. There was a sense of being unsure if the e-prescribing system actually transmitted the order necessitating the follow up. As described later, uncertainty about transmission also negatively impacted pharmacists, as well. One staff stated “it’s a hassle; we’re already putting in doctor’s orders in Matrix so why not leave it that way. You do it, then you have to go back and recheck and a lot of time it wasn’t working and then I had to make phone calls and call other floors to see what they did to correct it.”

Training appeared to improve user perceptions of the system, though this was not a focus of the evaluation. Several indicated that having all staff trained in the process would have been beneficial as those who did not attend needed to learn from others, or did not even attempt to use. Return demonstrations were noted as a recommendation to the training process.

Pharmacy Staff Satisfaction

Participants were asked to rate their level of satisfaction with the e-prescribing system on a scale from 0-10 with 0 being “very dissatisfied” and 10 being “very satisfied”. Pharmacy staff ratings were a 3 and a 0. Although the sample size is small (2 providers), these ratings indicated that these two staff members are very dissatisfied with the system.

Several aspects of the e-prescribing system were identified by the respondents as making their jobs more difficult. Processing refills that were rejected was identified as the biggest problem of the system being very time consuming. Combination orders, specifically Warfarin and Prednisone tapers, were mentioned as very difficult to process using the system. There was a lack of trust that orders were entered correctly or completely, especially for new admissions.

Brigham & Women’s Hospital (BWH)

Among 217 eligible physicians in the intervention clinic, 22 (11%) of attending physicians agreed to participate in the study although more may have utilized the e-prescribing module. During the site visit, Brigham investigators noted challenges enrolling clinicians primarily due to the prevalence of e-prescribing before the study and the lack of perceived additional value to participating in the study. In general, their response to e-prescribing was very similar to other groups, ranging from enthusiastic to more reserved. There were common themes among the reserved prescribers, including the challenges of future orders (refills to be taken to the pharmacist in a month), and conditional orders (prescriptions that are to be filled only if some other event warrants.)

Ohio-KePRO/ UHMP

The Ohio project leveraged a network of multispecialty practices throughout the Northeast Ohio region called the University Hospitals Medical Practices. Using a combination of a lightweight e-prescribing tool and a $500 incentive for physician adoption, they created a study group of 25 practices (130 physicians) and a control group of 22 practices (77 physicians.) The median practice size was 3.5 physicians, with some 1-person practices and many larger practices. Although the number of sites above was enrolled, not all physicians had to adopt e-prescribing.

As was the case with almost all pilot sites, adoption was increasing through the end of the pilot. Using their monthly audit reports, about two thirds of the physicians (about 100 out of 130) at the 25 study practices were e-prescribing for at least 150 prescriptions per month, either directly or via surrogates. Once adopted, dropout was extremely unusual.

A total of 47 medical group practices were included in this study. Since the main variable of interest was the influence of e-prescribing of drugs on costs and quality of care, they started with 25 UHMP practices that had adopted e-prescribing and matched them with practices that have not adopted these technologies. Practices were matched on size, specialty mix and urban location. A total of 22 matched practices were recruited for the study resulting in the 47-practice sample. Some of the UHMP practices did not have full use of this technology by all of their physicians but those practices were classified as adopters. When the unit of analysis was physician level, the physician not using e-prescribing in UHMP practices were treated the same as physician in the matched practice.

The practices in their sample ranged in size from 1 to 9 FTE physicians. The e-prescribing practices were slightly larger than the others and although all of these practices were selected because they provided primary care, there was a mix of physician specialties and one practice had at least some sub-specialists. The practice sample was also potentially confounded by post hoc differences electronic information capacity, decision-making process, and staffing. These variables could not be controlled for and may affect the validity of comparisons

Characteristics of Adopters

Multivariate analysis of prescriber survey results was conducted to determine any differences in the culture of practices that could predict successful adoption of e-prescribing systems. These data are shown below.

These data indicate that practices that have adopted e-prescribing have cultures that place more value on information (have an information mentality), are more cohesive, have high levels of organizational trust, are more adaptive and have a culture that emphasizes the group over the individual physician (less autonomous). The adaptive variable is only significant at the P = .18 level but the Ohio team report it because with this small sample size, it is an important finding. These are important findings for those that are attempting to expand the adoption of these technologies to other medical groups since it points to cultural traits that might influence the success of their efforts.

Exhibit 17. Differences in the Cultures of Study and Control Group Practices

Adopt	Coef.	Std. Err.	z	P> z	[95% Conf. Interval]
Collegiality	-1.976903	1.772185	-1.12	0.265	-5.45032	1.496515
Information	5.21644	2.417896	2.16	0.031	.4774503	9.955429
Quality	.0364183	2.001662	0.02	0.985	-3.886767	3.959604
Management Style	.2041121	.9703696	0.21	0.833	-1.697777	2.106002
Cohesiveness	3.986776	2.487916	1.60	0.109	-.8894499	8.863002
Adaptive	3.378013	2.571605	1.31	0.189	-1.66224	8.418266
Autonomy	-4.72278	2.16749	-2.18	0.029	-9.970982	-.4745781
Business	1.240313	1.653426	0.75	0.453	-1.000343	4.480969
_cons	-17.4548	9.265015	-1.88	0.060	-35.61389	.7042966

RAND

This pilot group utilized two main e-prescribing systems as a part of the Horizon Cross Blue Shield of NJ. In 2003, after Horizon joined with Caremark Rx to provide PBM services, Horizon providers were invited to adopt iScribe e-prescribing technology. Later, Allscripts’ TouchScript system and InstantDx’s OnCallData were added to the tools available to participants. As of November, 2006, 770 providers had iScribe installed, and 150 had AllScripts installed. These totals do not include an additional 80 from the iScribe group and 48 from the Allscripts group who disenrolled or withdrew from the program and were replaced by prescribers from the Horizon program’s waiting list.

As a part of their evaluation, RAND performed site visits to 12 of the offices that were planning to install the two e-prescribing systems under study. Site visits of the 12 physician offices were disappointing: 2 sites cancelled their installation, while an additional 4 sites had no prescribers (only staff) using the e-prescribing system, leaving 6 sites with at least some prescribers using the system. Over half of all prescribers studied continued to use paper. Reasons given for using paper are described in Exhibit 18.

Exhibit 18. Reasons Given for Continuing to Use Paper for Prescribing (RAND)

	Strongly disagree	Disagree	Neutral	Agree	Strongly Agree
I get too busy	10%	17%	7%	35%	31%
I can't use the PDA because of technical problems (e.g. network connectivity)	3%	3%	6%	37%	51%
Patients were not in the PDA	5%	8%	5%	47%	36%
Pharmacies don't reliably receive and process the prescriptions I send electronically	8%	13%	33%	36%	10%
System interfered with established office workflow	16%	34%	22%	22%	7%
System takes too much of my time	15%	24%	19%	30%	13%
System takes too much of my staff's time	24%	32%	30%	9%	6%

Central themes from prescriber interviews focused on the time it takes to implement and train providers as well as concern that prescriptions would be lost or that patients might lose track of what medications they are taking. In some cases, specific hardware (such as personal digital assistants) did not function as expected.

Recruitment for the Horizon E-prescribe Program was targeted at physicians who had been responsible for more than 500 Horizon BCBSNJ prescription claims in the previous year. Physicians who volunteered for the program without having been recruited were also allowed to enroll. iScribe’s recruitment targeted offices with 5 or fewer physicians, Allscripts targeted those with more than 5 physicians, and InstantDx targeted a smaller set of offices that had implemented a specific practice management system that their product was designed to interface with. Due to a higher then expected response rate, the program was expanded to allow 770 iScribe enrollees, and additional volunteers were put on a waiting list. After enrollment, participants were screened for appropriate Internet access and practice management systems, had the necessary wireless router and PDA hardware installed, and then received training on the system before being considered active. Exhibit 19 shows activations by month.

Exhibit 19. E-prescribing Participation by Month in the Horizon BCBSNJ E-Prescribe Program

Short Description: E-prescribing Participation by Month in the Horizon BCBSNJ E-Prescribe Program

Long Description: The exhibit shows the number of participants in e-prescribing program at Horizon BCSNJ. The graph shows that participating grew steadily into late 2006, but did suffer some withdrawals from 80 participants in iScribe and 48 from Allscripts.

Despite their efforts, 80 iScribe enrollees and 48 Allscripts enrollees withdrew from the program, either before or after activation. The reasons that enrollees gave Horizon for withdrawal included an inability to make the system work for their practice, switching to a full EHR system, and switching to a different e-prescribing program that was being sponsored by Aetna.

SureScripts

Physician focus groups provided valuable insight in the SureScripts study. On the whole, physician use was inconsistent. Comments by physicians cite many of the same reasons listed by other pilot sites, including time, expediency (it is often easier to call the pharmacy than to find a computer) and environmental (not all exam rooms have computers.)

Conclusions

All of the pilot sites had reasonable adoption of e-prescribing. It was apparent from preliminary data that adoption and retention was generally good, though drop outs were not uncommon. Prescribing rates for all sites reporting suggest validity for subsequent analyses. There is an obvious overlap between the finding that e-prescribing adoption occurs across clinical sites and subsequent changes in workflow that has been described by Johnson[liv] and Bell[lv] previously.

Factors associated with order entry system satisfaction have been well described and consistently link adoption with workflow integration. The role of RNs, LPNs, and office staff—who often function as an agent of the prescriber in all sites studied—was unanticipated, but was clearly part of successful workflows. This was particularly true for the long-term care facility, but was equally pervasive in clinics of all sizes. The extent to which e-prescribing systems adapt to this dynamic might be a critical factor in the adoption of e-prescribing in many environments. Some of the reasons given for prescriber drop-out may further support the need for better alignment of roles and e-prescribing functionality. In addition, the fact that adoption of e-prescribing varies depending on organizational culture warrants consideration for people trying to understand why some prescribers adopt and some do not.

Prescriber workflow changes and workflow related to verbal orders; benefits related to processing and data entry

One hope for e-prescribing is that it will improve workflow for both prescribers and pharmacists. Widespread adoption of e-prescribing will require that prescribers realize these improvements in workflow, or that other perceived benefits of e-prescribing are large enough to counteract any negative impact on workflow.

Achieve

An early observation in the Achieve pilot was that physician adoption would be minimal. The majority (94% or more) of prescriptions were managed by RNs and LPNs on site, who drafted orders for physicians to sign. With the implementation of CPOE, there was no noticeable change in prescriber workflow. However, there was an expected change in errors, primarily due to the decrease in transcribing handwritten orders to dispensable pharmacy orders.

During the site visit to Achieve, interviews with the charge nurses generated some important local observations about the impact of e-prescribing on their workflow.

§ Nurse as an agent model works technically in e-prescribing

§ Patient safety alerts are largely ignored when the nurse is the agent

§ Data entry errors can still happen

§ Combination orders create a challenge

Two of the many facility tasks that were measured and analyzed by the research team demonstrated possible effects due to e-prescribing. The communication tasks, which included all time related to telephone, written, or in person communication with either the pharmacy or the physician/NA/PA, for the treatment and comparison facilities were analyzed for the three data collection points (pre-pilot baseline, after phase 1 & 2). It was noted that the comparison facilities did not change over time; however, there was a significant decrease in average times for the treatment facilities – which had smaller average times after phase 2. Secondly, fax communication times for the treatment facilities were significantly lower than the comparison facilities after phase 2.

The e-prescribing system did not appear to impact the efficiency of pharmacy tasks (sorting, insurance rejections, new admissions, new/changed orders, and refills;) however, the number of occurrences requiring fax sorting was reduced after the implementation of e-prescribing, and there was a moderate reduction after Phase 2 in the typical time spent per medication, for new admissions, and a reduction in the variation in time spent per medication for insurance rejections.

Pharmacy workflow appeared to be negatively impacted by e-prescribing. For example:

1. The traditional paper process and the e-prescribing process are two totally separate functions within the pharmacy. The two processes did not integrate until after the prescriptions were processed through the system.

2. Pharmacists were concerned about accuracy in the nurse as agent model and about how additional questions and processing time for combination orders impacted efficiency.

3. Anecdotal comments suggest that the pharmacists made more clarification calls to the facilities and had more rework with combination orders (tapers or other orders with multiple dosing directions.) When the pharmacy receives a new order that’s been sent in this manner, a pharmacist must manually re-combine the individual messages before entering into the dispensing system. Combination orders can cause similar problems at the pharmacy when they are changed or discontinued.

The impact of e-prescribing on pharmacy practice is as dramatic a change as moving from typed labels and documentation on hard copy prescriptions to using computers for prescription processing. Not only is it a significant work process change, but also a major professional practice change – a true paradigm shift. Changes of this magnitude take time to adjust to and accept. When evaluating the pharmacy staff satisfaction levels with e-prescribing, a number of factors should be kept in mind.

Observational Study

New orders incorporated a number of tasks that were diverse including: Delete Order, Enter Order, Review Order, and Sign off order. An analysis model that incorporated number of medications as a covariate was used in the group comparison. A significant group effect (Prob 0.0306) was found with the average time for the Treatment group being approximately 15 seconds longer than the Comparison group.

All of the tasks for new orders were analyzed. For the task Entering Orders there was a significant relationship between the number of medications and time, thus number of medications was used as a covariate in the analysis. A significant facility effect (Prob = 0.0095) was found but no difference was found between groups. Treatment Facility 1 was found have a slightly higher mean time for entering orders than the other facilities. Comparison Facility 4 had the lowest mean time for entering orders. From analysis of the plot of time by facility, the amount of time required to enter orders was found to increase positively with the number of medications - yet after 15 medications the time leveled off.

Scanning of refills took place only in the treatment facilities as part of the e-prescribing system. A Wilcoxon Rank Sum test was conducted to compare the typical time spent in scanning at the two treatment facilities. Significant differences in typical amount of time for scanning between the two facilities were found with Treatment Facility 1 being higher. This finding was considered in the analysis that follows.

Analysis was conducted to determine if differences in the typical amount of time required for Labeling vs. Scanning across the groups/periods/facilities existed. This analysis required modifications to the initial nested linear model that had been developed because Task (Labeling vs. Scanning) was included in the model. Labeling was done in both Groups, but Scanning was only done in the Treatment group. Thus, a new variable called GroupTask was created for use in the model. Ideally, interaction terms would have been included in this model; however, due to the sparseness of the data this was not possible.

Marginal differences between Facilities in the amount of time required to complete labeling and scanning (Prob = 0.0532) were found. No statistical differences between the treatment and comparison groups for the label/scan task were found. A comparison of the mean and distribution by task and facility indicated that the label task required greater time at Treatment Facility 1 than did the scanning at the same facility. The mean time for scanning at the two treatment sites was less than the mean time for the labeling task at Comparison Facility 3 and Treatment Facility 1 (pre e-prescribing). Labeling at Comparison Facility 4 has the shortest mean time and probably reflects the limited documentation of this task at the site.

Other Variables

Further analysis was conducted to compare the treatment and comparison groups for time on task using data categories. The small number of occurrences of several of the data code tasks necessitated combining tasks to create categories. The data code tables below show the categories created for the research and their frequency distribution. The distribution of the “time” variable strongly skewed right so a transformation of time to a log₁₀ was used in the analysis. A nested linear model that considered all effects treated as fixed effects, and number of medications as a covariate was used in the analysis process for variables of interest. Analysis also considered differences between facilities (1-4) and groups (treatment and comparison).

The distribution of these categorized tasks is presented in the bar graph and frequency tables below.

Task Category: Communication

This task category combines the time spent on tasks that related to phone, written, or in person communication with pharmacy, physician/NP or other. Analysis indicated that Treatment Facility 1 had a statistically higher average time than other facilities at Baseline yet the average time dropped dramatically from baseline to Phase 2. See the graph below for comparison of facilities.

Further analysis of the Treatment and Comparison groups (instead of individual facilities) was completed. The time for the comparison group did not change over time; however, there is a significant decrease in average times for the Treatment group.

Task Category: Fax Communication

Fax communication represented numerous observations at all facilities. The time for communication via fax was not related to number of medications eliminating number of medications as a covariate in the model. Differences between facilities and differences between the Treatment and Comparison groups were found. In addition, the group by period interaction was marginally significant suggesting the effect of group may be different for the different periods. The Treatment group had a significantly lower typical Communication Fax time than the Comparison group at Phase 2. All other differences were not statistically important.

Other Task Categories

No effect of the e-prescribing system on other task categories that were compared over time between the treatment and comparison groups was found. These task categories included: 1) Communication: Other (which includes Communication: Person, Communication: Phone, and Communication: Written), 2) Communication: Pharmacy, 3) Communication: Physician, 4) Deleting orders, 5) Entering orders, 6) Reviewing orders, and 7) Verifying orders.

Pharmacy Data Analysis

Although there was not enough evidence to suggest that the e-prescribing system affected the typical time spent on any of the tasks, there were several observations worth noting.

1. The number of occurrences requiring fax sorting was reduced after the implementation of e-prescribing system, with only 2 observations noted in Phase 2 as compared with 7 observations at baseline.

2. There was a moderate reduction at Phase 2 in typical time spent per medication and a reduction in the variation in time spent per medication for insurance rejections. At Baseline (n=4) the mean time was 2.14 minutes (SD=1.37) while at Phase 2 (n=4) the mean time was 1.05 minutes (SD=0.48).

3. There was an increase in time for spent per medication from baseline (n=8; mean = 0.55; SD=0.20) to Phase 1 (n=10; mean 0.74; SD 0.37) and Phase 2 (n=10; mean =0.74; SD 0.24) for new admissions. Aside from the outlier present at Phase 1, the variation is similar across the period.

4. There was a slight decrease in typical time spent per medication and a moderation decrease in the variation in time spent per medication from baseline to Phase 1 and 2 for new/change orders. See exhibit below for means and standard deviations.

Further analysis was conducted to determine the effect of number of medications for time spent for the above mentioned tasks. Time spent on task was significantly associated with number of medications in a positive direction indicating that as the number of medication increases so does the time spent on the task (except for the task of insurance rejections). Yet there was no evidence to suggest that the e-prescribing system has any impact on the relationship that exists between number of medication and time spent on a task.

BWH

As with other groups, BWH quickly confronted the reality of nurses’ role in renewals/prescribing. First, many Rx renewals are processed by nurses who are not authorized Rx signers. BIDMC had to make a policy decision as to whether electronic refills done by nurses should be viewed like calling Rxs to a pharmacy (no MD signature required) or like documenting an order (MD signature required). They opted for the latter and built a queue to allow MDs to sign and electronically route Rxs written by nurses. Second, in order to meet the technical requirements for the NewRx message, they had to modify Rx data entry so as to collect information in a slightly more structured format. This had little to no impact for new medications. For renewals, which are usually done with a single click, each medication had to be modified in order to collect the additional information. This modification is required only once to convert to the new format and takes anywhere from 15 seconds to a minute per medication, depending on whether physicians use the “quick pick” function. Physician reaction varied widely. Some thought it was a very minor issue, while others considered it a major barrier to entry.

Their clinical group also noted some barriers to their existing workflow with prescribing, including:

§ Conditional prescribing--where patients are given prescriptions to fill at a later date only under certain conditions

§ Future prescribing--as when providing renewals for 3-month Rxs that could not be automatically renewed and to carry the patient until the next scheduled visit, 6-12 months later).

§ The inability to e-prescribe narcotics and other Schedule II-V drugs created some work flow challenges but appeared to be accepted by patients and clinicians.

Pharmacy workflow was impacted negatively in several pharmacy chains studied. Despite current thinking, the vast majority of pharmacy chains stores in the Boston area did not carry out true e-prescribing. Pharmacies had the capability but in reality reported to us that they generally printed out the eRx routed through the Rx Gateway and re-entered the data in their pharmacy system. Therefore the intervention period was essentially not different in practice from the baseline (CPOE without electronic transmission) period at the level of the pharmacy. Only 1 pharmacy chain, estimated with 10% or less of the eRxs in their study, conducted end-to-end e-prescribing such that the Medication Hx and Rx-for-Dispensed were not affected by data-reentry at the pharmacy.

Ohio-KePRO/UHMP

Site visits were conducted at 25 e-prescribing primary care practices and 22 matched non-e-prescribing practices in Northeast Ohio. In addition, surveys were given to all clinicians with prescriptive authority. This team was unable to do pre/post e-prescribing comparisons, since the primary group of study practices was already e-prescribing before study began. Unfortunately, the pilot team did not anticipate the role of surrogates in this process; therefore, they were not studied.

This finding has significant implications for the flow of formulary and other decision support messaging. It also appears to impact prescriber familiarity with some available e-prescribing functions, such as transferred prescription history. It also has implications for how some physicians use the printed daily e-prescribing audit logs - instead of reviewing these logs to make sure that no unauthorized prescriptions have been sent under their name, they are mostly interested in making sure that the prescriptions they did authorize have been entered correctly.

Although sites embraced e-prescribing in general with no significant change in workflow, the research team noted that offices had to adjust to accommodate cases where prescription messaging to pharmacies resulted in “lost” (and therefore unavailable) prescriptions at the pharmacy early in the implementation. These failures appear to be overwhelmingly due to pharmacist error rather than technical transaction problems; there seemed to be considerable variation between practices with respect to their ability to successfully resolve these problems.

Those sites using e-prescribing that is not integrated into an EMR appear to have no change in paper-based prescription renewal workflow within the practice: electronically received renewal requests are usually printed and attached to paper charts and used both as an authorization request and (subsequently) response communication vehicle, and marked up by pen, and used as documentation for the paper chart. The fact that renewal requests can be received electronically and the responses re-entered and transmitted electronically (in spite of all the paper in between) seems to add value from the e-prescribing practice’s perspective.

One of the most important findings of the project was the high rate of surrogate-based e-prescribing. From August through November 2006, 77 percent of e-prescriptions entered into OnCallData™ were entered by someone other than the authorizing prescriber (also referred to as “surrogate” entry). Furthermore, surrogate-based adoption did not appear to be a transitional stage leading to direct-use adoption patterns by e-prescribers who start out as surrogate-based. This finding challenges a prevalent tenet of e-prescribing adoption, namely that the best way to achieve e-prescriber adoption is by engaging surrogates first. While engaging surrogates around e-prescribing appeared to be a remarkably winning strategy for driving practice adoption, if the surrogate-based workflow made sense for a practice at the beginning of an implementation, then it worked for a reason and tended to persist.

How renewal requests are received differed considerably for the UHMP and control practices. Importantly, as shown in Exhibit 20 below, the UHMP practices reported that, on average, 40.8 percent of renewal requests originating from community pharmacy came in by phone, 27.9 percent by fax, and 31.0 percent via an electronic request directly into the e-prescribing software, compared with 59.6 percent by phone, 39.1 percent by fax, and 0 percent by e-prescribing at the control practices.

Exhibit 20. How Prescription Renewal Requests are Received

How Prescription Renewal Requests are Received from Community Pharmacies
	Type of Prescription System
	UHMP Practices	Control Practices	P value
Mean percent community pharmacy requests received by phone	40.8%	59.6%	0.274
Mean percent community pharmacy requests received by fax	27.9%	39.1%	0.620
Mean percent community pharmacy requests received by e-prescribing	31.0%	0.0%	0.000
Mean percent community pharmacy requests received by e-mail	0.0%	0.0%	0.069
Mean percent community pharmacy requests received by US mail	0.2%	1.4%	0.001

E-prescribing technologies may also affect if the practice utilizes a dedicated prescription voice mailbox to receive renewal requests. While not statistically significant, 60.0 percent of e-prescribing practices reported having a dedicated voice mailbox to receive renewal requests, compared to 33.3 percent of non-e-prescribing practices. Another apparent difference in workflow involves the staff member who is primarily responsible for prescription renewal calls. For the UHMP practices, 65.2 percent reported a Medical Assistant (MA) is responsible for prescription renewal calls and 17.4 percent reported that the front desk/office manager has this responsibility. This compares to the control group who reported 35 percent used a MA for this task and 40 percent reported that the front desk / office manager has this responsibility. Site visits and interviews with UHMP practices noted that MAs managed the e-prescribing system on behalf of most physicians. Due to the interaction with e-prescribing application, it is logical that the MA would also have responsibility for renewal phone calls, although implementation of e-prescribing by other practices could have different results.

There were also some striking differences with respect to internal renewal request processing. For instance, UHMP practices reported that the patient’s paper medical chart is pulled in order to authorize a prescription refill less often, on average, than the control practices (81.5 percent UHMP, 98.0 percent control, significant at .001). The specific reasons for the medical chart pull are shown in Exhibit 21. The most highly scored (3= most common, 2=2nd less common, 1=3rd less common, 0=all others) items for UHMP were “Need to verify last visit date, Pap smear, BP measurement, etc” (mean score of .96), “Lack of general familiarity with the patient (other than need to check last visit)” (mean score .52) and “The chart will be needed for documentation after authorization anyway” (mean score .52). This contrasts with the control practices who rated “The chart will be needed for documentation after authorization anyway” (mean score 1.19) as the most important reason for the chart pull, followed by “Need to verify last visit date, Pap smear, BP measurement, etc” (mean score of 1.00), and “The nature of the drug(s) being requested (e.g., birth control pills, narcotics)” (mean score of .48).

Exhibit 21. Reasons to Pull Patient’s Medical Chart

Reason to Pull Patient's Medical Chart to Authorize Prescription Refill - Mean Score (3= most common, 2=2nd less common, 1=3rd less common, 0=all other)
	Type of Prescription System
	UHMP Practices	Control Practices
Need to verify last visit date, Pap smear, BP measurement, etc	0.96	1.00
Lack of general familiarity with the patient (other than need to check last visit)	0.52	0.33
Uncertainty about what exactly is being requested	0.20	0.24
Authorizing physician asked for the chart after initially reviewing the request w/o it	0.12	0.14
The nature of the drug(s) being requested (e.g., birth control pills, narcotics)	0.32	0.48
The chart will be needed for documentation after authorization anyway	0.52	1.19

Medical record documentation differs due to the e-prescribing technologies available to UHMP. Only 9.6 percent of UHMP practices responded, “Physician writes a note in the chart” (compared to 35.4 percent of the control practices, significant at .000) and 34.4 percent of UHMP practices responded, “MA or other staff writes a note in the chart” (compared to 55.4 percent of the control, not significant). More importantly, for UHPC practices, 25.2 percent responded, “Copy of internal renewal messaging form placed in chart” (compared to 9.2 percent of the control, significant at .003) and another 17.6 responded “Renewal note is printed from e-prescribing application and placed in chart” (compared to 0 percent of the control, significant at .000).

Exhibit 22. Phoned-in Renewal Requests

How Phoned-in Renewal Requests are Documented in Patient's Medical Chart
	Type of Prescription System
	UHMP Practices	Control Practices	Significance
Physician writes a note in the chart	9.60%	35.40%	0.000
MA or other staff writes a note in the chart	34.40%	55.40%	0.402
Copy of internal renewal messaging form placed in chart	25.20%	9.15%	0.003
Renewal note is printed from e-prescribing application and placed in chart	17.60%	0.00%	0.000
Other means	13.60%	0.00%	0.000

The overall impact on workflow and effort is summarized by the respondents who were asked to use a 7 point Likert scale to: “Rate the relative resource intensity of managing prescription renewal requests and responses from start to finish for each of the following ways a renewal request is received.” UHMP and control practices rated phoned-in requests similarly in terms of resource intensity (5.12 for UHMP practices compared to 5.14 for control practices, significant at .041). However, UHMP practices rated fax requests as much easier in resource use (3.40 for UHMP compared to 4.14 for control practices, significant at .004). Most importantly, the UHMP practices rated requests that arrived by an e-prescribing application as the least resource intensive (2.83 for e-prescribing practices compared to no score for the control group).

RAND

RAND researchers engaged in the process of “work process modeling” to refine a model for e-prescribing (Exhibit 23). This model takes into account the information flow related to the new standards, and makes explicit workflow requirements that must be support by e-prescribing vendors.

Exhibit 23. RAND Model of E-prescribing Work Process

Long Description: The exhibit illustrates RAND’s model for e-prescribing. The model takes into account the information flow related to the new standards, and makes explicit workflow requirements that must be support by e-prescribing vendors.

Site visits to offices revealed that only 50% of the enrolled offices continued to rely solely on authorized prescribers to interact with e-prescribing systems. Visits to these offices were illuminating. In some offices, e-prescribing caused a shift in the responsibility either to or away from nurses. Prescription renewals, in particular, were more commonly delegated to office staff in the presence of e-prescribing.

SureScripts

Among prescribers or their agents who adopted e-prescribing, obtaining prior approvals, responding to refill requests, and resolving pharmacy callbacks were all done more efficiently with e-prescribing than before. Both groups perceived a greater than 50% reduction in time to manage refill requests, and significant time savings in managing pharmacy call backs. The amount of time saved varied with each vendor system, but all systems appeared to save time when resolving prescription problems. Close to 90% of surveyed providers perceived that efficiency of care was improved with e-prescribing, while approximately 75% believed that patient safety and quality of care were improved.

In terms of impact on relationship with patients, providers were approximately evenly split in their assessment, with just over 50% perceiving that communications with patients were much or somewhat better, and a similar number perceiving that the overall relationship with patients was much or somewhat better with e-prescribing.

SureScripts clinicians had significantly varying use of all e-prescribing features, as shown below. The most frequently used features included generating new and refill prescriptions, approving new and refill prescriptions, reviewing the medication history, updating the medication list, entering and revising patient information, reviewing prescription reference information, reviewing formulary information, and constructing a favorites list.

Exhibit 24. Least-Used Features of E-prescribing (SureScripts)

Short Description: Least-Used Features of E-prescribing (SureScripts)

Long Description: The exhibit lists the least used e-prescribing features used by prescribers. It shows that physicians had significantly varying use of all e-prescribing features. The most frequently used features included generating new and refill prescriptions, approving new and refill prescriptions, reviewing the medication history, updating the medication list, entering and revising patient information, reviewing prescription reference information, reviewing formulary information, and constructing a favorites list.

SureScripts is completing their analysis of interview and survey data that will be used to construct a workflow diagram that will include the fate of verbal orders in the e-prescribing era. These data are not yet available.

Conclusions

The main finding of all the sites is that the role of surrogates was underappreciated in prescribing workflow generally, and particularly post e-prescribing adoption. In the paper-based prescribing era, many medications were called into pharmacies without physician intervention. In the e-prescribing era, where e-prescribing can eliminate phone time and cost, the same office-based personnel now take on a larger role. Although this improves office efficiency, the impact on patient safety is not established. One project site (Achieve) noted that this workflow might compromise the attention given to drug-allergy alerts. This phenomenon deserves additional study and should become a part of the education provided to e-prescribing physicians.

Another important finding from these groups is that in almost no setting did e-prescribing replace the need for paper-based prescribing. Factors important in this observation include the inability to manage future orders electronically, the inability to submit Schedule II controlled substance prescriptions using the SCRIPT standards, and the time pressures that prescribers feel. The impact of this finding on workflow is seen in the highly variable use of e-prescribing features.

One of the sites with the strongest study design for assessing workflow change reported notable reductions in workflow as a result of e-prescribing. However, in those groups with preexisting electronic health records or e-prescribing, workflow transformation might have occurred before the beginning of the study and would also not likely be found. Additional longitudinal studies will be needed to explore this question in more depth.

Implementation of e-prescribing had the potential to dramatically change pharmacy workflow, in several cases with negative consequences. For example, two sites mentioned that pharmacies were unable to transfer received SCRIPT messages to their pharmacy system. Patient concerns (see section below) almost always included challenges at the pharmacy related to pharmacy workflow of electronically-submitted new prescriptions. In short, additional studies of outpatient pharmacies, with communication to pharmacy information systems vendors are needed.

In analyses, particularly those by Ohio KePRO/UHMP, pilot sites identified a shift from phone-based to e-prescribing based refill and renewal prescriptions. Based on the provider practice surveys, the new e-prescribing workflows for refills and renewals may generate efficiencies for small physician offices. In addition, these findings suggest that in the current form, e-prescribing tools may decrease the reliance on verbal orders and phone transmission of those orders. However, the use of surrogates may be associated with a concomitant increase in “verbal orders”—none of the sites have clearly outlined how providers are involved in the cognitive aspects of prescribing when surrogates act as their agents.

Changes in Number of Callbacks from Pharmacy to Prescribers

Almost 30% of prescriptions require pharmacy call backs, resulting in 900 million prescription-related telephone calls annually.[lvi] Many of these calls are for issues that should not arise in a well-implemented e-prescribing system, such as clarification of handwriting or renewal requests. Four pilot sites included a study of callback rates.

Achieve

Anecdotes from the pharmacists and nurses are that the system has dramatically reduced call backs during new admissions.

BWH

The following exhibit provides the preliminary analyses of the observation study of office workflow management Rx callbacks / renewals.

	Baseline Period		Intervention Period
	Clinic A	Clinic C	Clinic A	Clinic C
Observation Hours	41.7	41.3	31.4	36.3
Observation hours related to Rx activity	29.8	26.9	22.5	25.1
Total Rxs processed n (%)	1364	519	788	775
New non-narcotic	833 (61)	171 (33)	401 (51)	117 (15)
New narcotic	229 (17)	23 (4)	88 (11)	12 (2)
Existing Rx	219 (16)	260 (50)	278 (35)	383 (49)
Rx type not selected	83 (6)	65 (13)	21 (3)	263 (34)
Rxs processed per hour	45 / hour	19.3 / hr	35 / hour	31 / hour
Call Type %	IN OUT	IN OUT	IN OUT	IN OUT
Phone	64 6	16 26	48 8	7 19
Email	7 7	1 0	12 11	1 0
Fax	12 0	14 12	11 0	9 6
Paper Message	0 0	7 13	1 0	22 12
In-person request	0 0	7 2	1 0	11 13
Entry not selected	3	3	9	1
Contact type %
Patient	64	38	53	56
Pharmacy	14	30	14	18
Health plan	0	7	0	1
Ordering prescriber	13	8	22	10
Other provider	4	5	1	8
Entry Not selected	5	12	9	6
Reason for Call
Renewal/refill	78	57	61	59
Rx clarification	16	20	28	34
New Rx	2	7	1	2
Prior auth. / confirm medical necessity	1	12	0	1
Incorrect prescription	0	0	1	3
Other	2	3	0	0
Tasks for Rx Request/ Other Workflow Activities %
Lookup patient medical Hx in webOMR	37	21	35	23
Perform patient care related activities	5	13	6	5
Receive / review and document RX requests	22	24	13	24
Contact patients	5	8	4	10
Clarify orders with prescribers	13	6	11	5
Review standard guidelines	3	7	1	17
Insurance approval	0	4	0	2
Update patient med list in WebOMR	0	4	0	3
Transmit order to pharmacy	1	5	6	5
Update Rxs to new eRx format	NA	NA	10	NA
Sending eligible eRx to MD queue for sign	NA	NA	2	NA
Looking up and entering pharmacy info	NA	NA	1	NA

Ohio-KePRO/UHMP

The Ohio team planned to evaluate phone call and fax tally sheets used by the staff at the practices to record 5 consecutive weekday counts of prescription-related phone calls. Data recorded for each call includes:

§ Estimated length of call

§ Incoming vs. outbound

§ Call to/from patient, pharmacy

§ Purpose of call (renewal, formulary, clarification)

§ Chart required

Tally sheets that were returned had suspicious data quality—results must be viewed only as pilot data deserving more careful study.

Exhibit 25. Number of Incoming and Outgoing Calls per Physician per Day

	Practice Type	Mean	Std. Deviation	Std. Error	t-statistic	Sig. Level
Incoming Calls	UHMP	5.48	4.40	0.43	2.59	0.010
	Control	4.14	2.81	0.28
Outgoing Calls	UHMP	1.53	1.07	0.13	-5.38	0.000
	Control	3.59	3.08	0.36

The e-prescribing practices had significantly more incoming calls and significantly fewer outgoing calls compared to the control group practices. The higher average number of incoming calls may be associated with callbacks from the patients, who upon arrival at the pharmacy, are told by the pharmacy staff that they have no prescription for them. Instances such as this were described during many site visits to UHMP practices. While some of the practices have resolved the problem through pharmacy education, it remains a problem. Additionally, calls from the patients and/or pharmacies regarding the absence of a prescription may have been higher during the timeframe of data collection as many of the practices were still in the earlier stages of e-prescribing during the Summer of 2006.

Next, the characteristics of incoming and outgoing calls were examined. Practice staff, via the tally sheets, was asked the amount of time spent on the phone dealing with the prescription-related phone call, the source or destination of the call, the prescription issue that was being called about and whether or not a chart pull was necessary to process the call. See the exhibits below.

Exhibit 26. Differences in Time Spent on Phone

	INCOMING CALLS*				OUTGOING CALLS*
	UHMP		Control		UHMP		Control
	Number	Percent	Number	Percent	Number	Percent	Number	Percent
Time Spent on Phone
< 2 minutes	1,029	66.1	898	62.4	157	53.8	547	63.2
2 to 5 minutes	465	29.9	499	34.7	101	34.6	276	31.9
> 5 minutes	63	4.1	42	2.9	34	11.6	43	5.0
	1,557	100.0	1439	100.0	292	100.0	866	100.0
* Chi-square significant at p<.01

The distribution of the time spent on the phone for both incoming and outgoing calls was statistically different between the UHMP and control practices. While the UHMP practices had significantly more incoming calls (Exhibit 26), a higher proportion of these calls were under two minutes, compared to the controls (66% vs. 62%). Conversely, the UHMP practices had significantly fewer outgoing calls (Exhibit 27) but a larger proportion of these calls took more than two minutes (46.2%) compared to the control group practices 36.9%.

Exhibit 27. Differences in the Source or Destination of Phone Calls

	INCOMING CALLS				OUTGOING CALLS*
	UHMP		Control		UHMP		Control
	Number	Percent	Number	Percent	Number	Percent	Number	Percent
Source/Destination
Patient	1,094	69.9	1,020	68.5	92	31.7	160	19.1
Pharmacy	435	27.8	440	29.5	173	59.7	634	75.7
PBM	36	2.3	30	2.0	25	8.6	43	5.1
	1,565	100.0	1,490	100.0	290	100.0	837	100.0
* Chi-square significant at p<.01

The difference in the distribution of the source / destination of prescription-related calls was statistically significant for outgoing calls only. The UHMP and control group practices were remarkably similar in terms of the source of incoming calls: roughly 70% were from patients, 29% from pharmacies and 2% from PBMs. For outgoing calls, the e-prescribing practices had a smaller percentage of calls to pharmacies (60%) compared to almost 76% in the control group practice. This is likely due to the fact that the practice staff can enter the prescription and send it electronically to the pharmacy rather than having to call it in. This is one of the biggest gains in efficiency related to e-prescribing.

Exhibit 28. Differences in Charts Requested to Process Prescription-Related Phone Calls

	INCOMING CALLS
	UHMP		Control
	Number	Percent	Number	Percent
Chart Requested?
No	153	10.6	151	10.5
Yes	1,293	89.4	1,287	89.5
	1,446	100.0	1,438	100.0
Note: Chart pull information requested for incoming calls only

The majority of incoming, prescription-related phone calls required a chart pull in both the UHMP and control group practices. The distribution is nearly identical between the practice types and not statistically different, but may be related to cultural differences between the two groups.

RAND

Using their online prescriber survey, with a 58% overall response rate, there was no significant difference in prescribers’ perceptions of the number of calls related to prescription drug coverage among those who had implemented e-prescribing vs. controls who were on the waiting list.

SureScripts

The range of pharmacy faxes and call backs, as well as patient calls regarding prescriptions per week, was wide.

Conclusions

Collection of data about callbacks is difficult, as these pilots discovered. However, it appears that e-prescribing may represent a component of cost-shifting—providing more efficiency in practice settings, but with unknown and possible deleterious effects at the dispensing end of the continuum. More studies need to be done to evaluate this phenomenon, given both the data about the rarity of true “end-to-end” e-prescribing, and the theme of negative impact on the workflow of pharmacists at this transition point.

Patient Satisfaction

Patient satisfaction will be another important component driving e-prescribing. Even if e-prescribing improves prescriber workflow, if patients report problems when they go to pick up a prescription, prescriber adoption may be limited. Only one site, SureScripts, included this outcome in their study.

SureScripts

SureScripts addressed patient satisfaction using a combination of focus groups and site visits to office practices (performance analysis). Provider groups surveyed represented the geographic range of their study population, as well as a range of office practice sizes and medical specialties.

Patients either mildly or strongly preferred e-prescribing over paper prescriptions. When surveyed, 54% of patients were very satisfied, 29% were moderately satisfied, 14% were somewhat dissatisfied, and 3% were dissatisfied with e-prescribing as dispensed at the pharmacy. Odds ratios showed that adults under 65 years of age were 1.7 times more likely to strongly prefer e-prescribing.

However, two vendors in their study received less favorable patient opinions, with 80% of one vendor’s patents preferring paper. Comments from physicians about patient preferences reflected this variation, with comments ranging from patients who were unhappy to see their physician use the PDA (“Oh no, not that thing; that didn’t work last time.”) to patients who had bad experiences where prescriptions “do not go through.” Researchers noted that the provision of paper was associated with a higher rate of pharmacy pick up in their sample.

Conclusions

Although only from one site, findings suggest that while most patients are satisfied with e-prescribing, the process has challenges. Many of the patients most in need of e-prescribing, such as geriatric patients, may find pharmacy-related challenges especially onerous, given the difficulties they may face troubleshooting the system if the pharmacist is unwilling to make phone calls, check multiple queues, or check underneath the fax machine. It is not clear from these results where problems occurred for the 17% of surveyed patients who were not satisfied. Identifying whether they had problems with the process (using a PDA, for example), the routing mechanism (faxes that did not get received by the pharmacy), or the order fulfillment stage (pharmacies who do not check for electronically transmitted prescriptions, or who don’t fill the prescription until the patient presents to pick it up) might help improve patient satisfaction in the future. It is clear that more data about patient perspectives is necessary.

Use of Medication History Functions

The availability of a patient’s medication history can enable prescribers and pharmacists to prevent medical errors by checking for redundant drugs and drug-drug interactions. Three sites specifically tracked how frequently prescribers accessed medication history information via the e-prescribing system, or asked them how useful they found this information.

Ohio-KePRO/ UHMP

During the site visit to KePRO, the researchers commented that the study was limited by a fairly obtuse integration of this functionality within the e-prescribing system, resulting in very few providers being aware of it. The team was able to generate quantitative and qualitative data about their implementation of Medication History.

Exhibit 29. Use of Medication History, by Month (Ohio KePRO/UHMP)

Year 2006	June	July	Aug	Sept	Oct
Medication History Checked	12324	10447	13063	9962	12464
Medication History Viewed	117	122	134	129	488

Follow-up phone interviews revealed that only one of the 9 test practices--an entirely surrogate-based OnCallData™ practice--actually followed through with routinely pulling up transferred prescription histories at patient encounters and printing them. An interview with the practice manager at that practice revealed that the MAs at the practice strongly resisted the workflow involved (in stark contrast to their rapid embracing of OnCallData™ e-prescribing workflow generally), and the physicians for whom the reports were being printed did not find them useful enough to demand that their MAs continue to access and print them. An interview with the physician Medical Director at that practice confirmed the report by the office manager. This physician placed high conceptual value on transferred prescription history both during the baseline site visit interview in September and during the phone interview in January. However, he did not feel that he and his partners had been adequately trained about how to use the OnCallData™ reports and the reports were largely ignored.

Phone interviews with the Medical Director internists at each of the other two internal medicine practices in the nine practice test group revealed somewhat different findings. Neither of these other internal medicine practices sustained any printing of prescription history reports during November. However, both of these internists were direct OnCallData™ users and were aware of and had used the transferred prescription history feature even before our test. Both had strong positive feelings about the feature. Results of these three interviews are summarized in Exhibit 30 below.

	Control Clinics	e-prescribing Clinic
Total Number of Dispensing Errors	22	37	190	19	58
Dose errors	3 (13.6)	7 (18.9)	34 (17.8)	1 (5.2)	4 (6.9)
Frequency errors	4 (18.2)	8 (21.6)	30 (15.8)	4 (21.1)	12 (20.7)
Product strength errors	1 (4.5)	1 (2.7)	35 (18.4)	1 (5.2)	4 (6.9)
As directed when inappropriate	4 (18.1)	1 (2.7)	30(15.8)	2 (10.4)	3 (5.1)
Brand name not dispensed when indicated	0	1 (2.7)	1 (0.6)	0	0
Route error	3 (13.6)	0	2 (1.1)	0	0
Incorrect dosage form (IR vs ER)	0	1 (2.7)	3 (1.6)	0	0
Duration of therapy	0	0	3 (1.6)	1 (5.2)	0
PRN missing or added	3 (13.6)	2 (5.4)	7 (3.7)	1 (5.2)	4 (6.9)
“See attached directions” written on label	0	0	0	1 (5.2)	1(.7)
Methadone “for pain” missing on label	0	0	0	0	1 (1.7)
Oral instructions for a topical prescription	0	1 (2.7)	0	0	0
Errors suggesting prescription not presented at pharmacy
Total	4 (18.2)	15(40.5)	42 (22.1)	8 (42.1)	29 (50)
Previous dispensing Hx matches current fill	4	11	19	5	21
Dose and frequency	0	2	10	1	2
Strength and dose	0	0	5	0	1
Strength and frequency	0	2	1	0	2
Strength, dose, and frequency	0	0	5	2	0
Unclear directions	0	0	2	0	3

Ohio-KePRO/ UHMP

The Ohio team conducted an analysis of their health plan data over the period of 1/1/04 through 6/30/06. Data pulled include drugs, diagnoses, patient age categories, patient gender and laboratory procedure codes, and are being evaluated for drug-age errors (Beers), drug-condition (including pregnancy), drug-drug, drug-gender, failure to monitor, and omission errors.

Exhibit 43. Adverse Drug Events by Practice Type, All Data Sources Combined

All data combined	Number of ADE hits	Total Encounters OR Rx	% of ADE hits
UHMP eRx	5,343	2,941,920	0.18%*
UHMP eRx - PRE	3,197		0.11%**
UHMP eRx - POST	2,146		0.07%**
UHMP non eRx	1,484	861,938	0.17%
Control Group	1,831	522,249	0.35%*
MD not in any study group	5,648	2,806,099	0.20%
Data could not be assigned to group	4,825	3,238,022	0.15%
TOTAL	19,131	10,370,228	0.18%

*Difference between UHMP eRx and control group physicians is statistically significant (Chi-square=612.8, p<.0001).

**Difference between UHMP eRx PRE and POST is statistically significant using McNemar’s Test (S=2932291.2, p<.0001).

Looking at all data sources combined, it appears that control group physicians (non- e-prescribers) had a greater number of ADE hits then the UHMP e-prescribing physicians. 0.37% compared to 0.18%, respectively. The data also show that the UHMP e-prescribing physicians had fewer ADE hits after they began e-prescribing (0.11% pre compared to 0.07% post). Both of these differences were statistically significant.

Exhibit 44. ADEs by Practice Type, Concept Data

Concept™ Data*	Number of ADE hits	Total Encounters	% of ADE hits
UHMP eRx	2,694	1,994,302	0.14%
UHMP eRx - PRE	1,471		0.07%**
UHMP eRx - POST	1,223		0.06%**
UHMP non eRx	1,130	687,976	0.16%
Control Group		0
MD not in any study group	400	479,078	0.08%
TOTAL	4,224	3,161,356	0.13%
*Concept ™ is the practice management system in use at UHMP. It does not contain data for control group physicians.

**Difference between UHMP eRx PRE and POST is statistically significant using McNemar’s Test (S=1989165.0, p<.0001).

The percentage of ADE hits by practice type in the individual data sets (Exhibits 57 - 60) appeared to be similar. However, using McNemar’s test to compare the UHMP e-prescribers during the periods before and after e-prescribing, the differences were statistically significant at p <0.0001. This was also true when looking at the ADE rates for the UHMP e-prescribers compared to the control group physicians using Chi-Square test (p <.0001) for each data set with the exception of QualChoice where the difference was not statistically significant (p=0.08)

Access to patient’s medical records was not available during the study. Because of this, there is no way to confirm that these ADE hits were indeed true hits and if they were due to a medication error (preventable adverse drug events) or were non-preventable adverse drug events (no error). Due to time constraints, the number of drug pregnancy trigger hits and drug procedure hits were unable to be examined, as originally planned.

SureScripts

In the SureScripts pilot, clinicians were asked how often they overrode different alerts. Responses appear to be highly system dependent, ranging from approximately 30% to 100% of respondents saying at least “sometimes.” In terms of dose checks, responses were less varied, with a range from 70-100% saying at least sometimes. Virtually all respondents (over 80 to 100% ) said they overrode drug-drug interaction alerts at least sometimes. Comments noted that most drug-drug interactions were already known by the clinician, and that “everything interacts with everything” making an overwhelming amount of alerting. Providers were aware of the potential for an important alert to be missed, but recognized that the volume of alerts made that more likely.

During the 312 pharmacy observation shifts, pharmacists reviewed a total of 2,690 e-prescription orders (new = 83.0%, refill =17.0%) and intervened 102 times for an overall intervention rate of 3.8%. The rate at which pharmacists identified problems on new e-prescriptions was found to be nearly twice that of refills at 4.1% and 2.2%. The problems were recognized by pharmacists (65.7%), the pharmacy’s computer system (14.7%), the patient (6.9%) and the physician (4.9%). The most common reason for pharmacists’ interventions was to supplement omitted information (31.9%), especially missing directions. Other common problems included insufficient dose (9.7%) and excessive dose (8.0%). The most common response to e-prescribing problems was to contact the prescriber (64.1%), followed by consulting the patient’s profile or medication history (12.8%) and interviewing the patient or the patient’s representative (9.4%). In most cases, the e-prescription order was changed and the prescription was ultimately dispensed to the patient (56%). In 15% of cases, the e-prescription order was eventually dispensed as initially written following contact with and clarification by the prescriber. In 10% of cases the prescription was not dispensed. An additional 12% of prescription issues remained unresolved when the pharmacist reported these data. Twenty percent were dispensed with different directions.

RAND

Claims Data. In a preliminary analysis of potential drug-drug interactions (DDIs), RAND identified incident claims for medications among the 25 most-severe DDIs[lxi] that occurred when the same patient had a claim for a simultaneous supply of the other medication in the DDI pair. In the RAND data set of 2.8 million prescription claims for patients of e-prescribing and control physicians, RAND detected a total of 1780 such potentially-severe DDI events, for an overall rate of 6.4 events per 10,000 prescriptions. The DDI event rate increased overall from the pre- to the post- EPS period (from 5.6 to 7.8 events per 10,000 Rx, chi square P<.0001). Attributing each event to the prescriber of the second (incident) prescription, 153 were attributable to EPS prescribers (7.4 per 10,000 Rx), vs. 1021 to control-group prescribers (6.4 per 10,000 Rx), and 606 to prescribers in neither group (6.3 per 10,000 Rx, P=.001). The pre- to post- increase did not differ significantly among groups.

Conclusions

Data from this section are not conclusive. The two main sources of data about ADEs in the pilot sites arose from analyses done at BWH. The third analysis, conducted by RAND is still in a preliminary state. Additional analysis is expected by these groups, and will be necessary to draw any conclusions from the results presented thus far.

Use of On-formulary Medications and Generics

Health plans and pharmacy benefit managers use a variety of cost incentives to steer utilization to cost-effective drugs. However, these incentives are most effective if they are communicated to the prescriber. Because each health plan has different incentives, physicians rarely know what a particular patient will pay for one drug compared to its alternatives. In a paper-based system, the onus is generally on a patient to complain about high drug costs and ask their provider to consider lower-cost alternatives. E-prescribing creates the possibility for providers to see information about a patient’s drug coverage, and which drugs are preferred by the patient’s insurer, at the time of prescribing. An e-prescribing system may also remind the prescriber when generic alternatives are available to a brand-name drug. This information can allow prescribes to make more cost-effective decisions for each patient. Four sites planned studies of this behavior, but preliminary data are available from only one of those sites.

Achieve

Achieve has not yet completed their analysis of on-formulary and generic prescribing. They have planned a retrospective look back at the end of the study for each of the data collection periods. The pharmacy will keep an electronic record of these events to be used at the end of the study. 100% of all events will be tabulated during each of the measure periods. Cost will be tabulated based on cost of non-formulary prescriptions to the facility.

Ohio-KePRO/ UHMP

Formulary Compliance data for Ohio are shown below. As can be seen, formulary compliance is at or above 90% for all three groups. There were no statistically significant differences.

Exhibit 45. Differences in Formulary Compliance, 3 Months of Health Plan Data

Practice Type	eRx Status	Formulary Rx	Total Rx	% Formulary
UHMP	No	6,698	7,442	90.00
UHMP	Yes	32,493	35,924	90.45
Control	No	12,305	13,535	90.91

While the majority (58%) of prescriptions written in OnCallData™ are informed by eligibility-based formulary, the rate of formulary compliance is not higher among e-prescribing doctors. This is not surprising in light of the fact that most formulary switches occur when the patient takes the script to the pharmacy. If the pharmacist discovers, during the pre-adjudication process, that the prescribed drug is not on the patient’s formulary, the pharmacist advises the patient and typically tells them to contact their physician or contacts the physician’s office directly.

Ohio pharmacists are required to substitute generics for brand name drugs, when available, unless the prescriber expressly indicates on the prescription that substitution should not occur or the patient requests the brand name drug. As such, the research team did not expect e-prescribing to have a significant impact on generic substitution rates.

Exhibit 46. Differences in Generic Dispensing Rates, 3 Months of Health Plan Data

Practice Type	eRx Status	Generic Rx	Total Rx	% Generic
UHMP	No	4,100	7,442	55.09
UHMP	Yes	18,733	35,924	52.15*
Control	No	7,609	13,535	56.22*

*Significant main effects, the mean difference is significant at the .05 level.

The control group physicians had a slightly higher proportion of generic prescriptions (56.2%) compared to the e-prescribing UHMP physicians (52.12%). One Way ANOVA showed significant differences between the three groups on the proportion of drugs that were generic (F=5.72, p=.000). Specifically, the difference between the e-prescribing UHMP (A Yes) and the control group practices was significant (Bonferroni Post Hoc Mean Difference [B-A Yes]=-.08, p=.000). The control group practices had a significantly higher percent of generic drugs than the UHMP e-prescribing practices. Differences between UHMP e-prescribing and non-prescribing practices were not significant. Please note that this data set is only for 3 months.

Some of the difference in generic dispensing rates may be due to the fact that a higher proportion of UHMP physicians’ patients requested the brand name drug (6%) compared to only 4% for the control group. These numbers trended downward over time from 8% for the UHMP physicians in 2004 to 4% in the 2006 data. Similarly, the proportion of control group patients requesting the brand name drug fell from 5% in 2004 to 3% in the January through September 2006 data.

Using the 3 months of health plan data (described above), the research team looked at the proportion of prescriptions that were “single source” meaning that they were for brand name drugs that had no generic equivalent.

Exhibit 47. Single Source Dispensing Rates, 3 Months of Health Plan Data

Practice Type	eRx Status	Single Source Rx	Total Rx	% Single Source
UHMP	No	3,000	7,442	40.31
UHMP	Yes	15,564	35,924	43.32*
Control	No	5,353	13,535	39.55*

* Significant main effects, the mean difference is significant at the .05 level.

As can be a larger proportion of UHMP prescriptions were for single source drugs; 43.3% for the e-prescribing UHMP physicians compared to 39.6% for the control group. One Way ANOVA showed significant differences between the three groups on the proportion of drugs that were single source (F=4.59, p=.011). Specifically, the difference between the e-prescribing UHMP (A Yes) and the control group practices was significant (Bonferroni Post Hoc Mean Difference [B-A Yes]=-.065, p=.010). The UHMP e-prescribing practices had a significantly higher percent of single source drugs than the control group practices. Differences between UHMP e-prescribing and non-prescribing practices were not significant, which was not surprising given the results of the generic analyses.

The fact that a larger proportion of UHMP prescriptions are for single source drugs and more UHMP patients request brand names at the pharmacy may explain some of the cost differences observed between the UHMP and control group physicians (see Cost Analysis Section). It is highly unlikely that these differences are due to e-prescribing, as a larger percentage of patients requested the brand name drugs in 2004, before the majority of UHMP practices were e-prescribing

Exhibit 48. Cost of Brand and Generic Anticholestorimia Drugs

	Brand		Generic
	Crestor	Lipitor	Lovastatin	Simvastatin	Zocor	Pravastatin
Average Cost	$ 86.68	$ 96.60	$ 30.01	$ 110.65	$ 143.32	$ 97.80

Cost Differences by Practice Type:

Post hoc tests revealed that, in all the years examined, the control group physicians had significantly lower average prescription claim costs compared to both types of UHMP practices. And, the UHMP e-prescribing practices had significantly lower average cost per prescription claim compared to their non-e-prescribing counterparts. The exception to this was for 2004 to 2006 time period; in this case, the difference between the two types of UHMP practices was NOT significant.

Exhibit 49. Cost Differences by Practice Type

	2004^a		2005^b		2006 (Jan.-Sep.)^c		2004-2006^d
Practice Type/eRx Status	N	Ave. Cost	N	Ave. Cost	N	Ave. Cost	N	Ave. Cost
1. UHMP - No	16,621	$54.09	17,210	$56.63	8,865	$60.17	42,696	$56.33
2. UHMP - Yes	96,608	$52.67	100,566	$53.37	51,041	$56.74	248,215	$53.79
3. Control	63,210	$48.09	63,744	$48.69	32,833	$50.61	159,787	$48.85
^aF=543.41, p=.000; Bonferroni Mean Difference (1-2) p=.000, (1-3) p=.000 (2-3) p=.000
^bF=174.45, p=.000; Bonferroni Mean Difference (1-2) p=.004, (1-3) p=.000 (2-3) p=.000
^cF=214.62, p=.000; Bonferroni Mean Difference (1-2) p=.000, (1-3) p=.000 (2-3) p=.000
^dF=158.45, p=.000; Bonferroni Mean Difference (1-3) p=.005 (2-3) p=.002

The fact that these cost differences existed in 2004, before the majority of UHMP practices began e-prescribing, suggests that difference in average cost between the practice types is not due to e-prescribing. Rather, the difference in cost may be partially explained by the fact that UHMP physicians tend to prescribe more single source drugs and fewer generics, and a larger percentage of UHMP patients request the brand name drug at the pharmacy.

RAND

RAND’s analysis of prescription claims focused on patients of 319 primary care physicians who had activated e-prescribing during calendar year 2005 (to allow for all to have at least 6 months of claims after e-prescribing). The study found a significant temporal increase in the use of generic meds for new ACE inhibitor prescriptions, but the effect was limited to those with the highest extent of e-prescribing use. Patient income also was significantly inversely associated with a higher likelihood of a generic medication claim.

Prescribers in the RAND survey had mixed perceptions about the use of formulary information, as shown in the exhibit below. There was uncertainty expressed about the accuracy and completeness of the formulary information available through e-prescribing, with 53% of respondents either believing that “drug coverage information appears to be missing in the system for those with prescription benefits?” or not being confident about its presence.

Exhibit 50. Perceptions of Formulary Data in E-prescribing Software (RAND)

The drug coverage information in the e-prescribing system…	Strongly disagree	Disagree	Neutral	Agree	Strongly Agree
Is clear and understandable	3%	18%	32%	38%	9%
Helped me to manage cost for my patients	6%	17%	37%	35%	4%
Reduced the need to change prescriptions because of coverage problems	2%	25%	39%	29%	5%
Reduced the number of calls to my office from pharmacies and patients regarding coverage problems	5%	25%	41%	24%	4%
Overall, saves me time	6%	23%	41%	28%	3%
Overall, reduces costs for my office	6%	25%	50%	17%	1%
Overall, I am satisfied with the drug coverage information	7%	18%	38%	33%	4%

Formulary and benefit information was attended to by some, but not all providers, both because of prior attitudes as disclosed during interviews and because of skepticism about the accuracy of reference material. At one site, an office manager and pharmacist each had systems that disagreed about the coverage of a specific patient’s medications.

Claims data analysis. Results from multivariate logistic regression modeling showed that, controlling for an underlying trend toward more generic use over time, high users of the EPS had a significant increase in their likelihood of prescribing a generic ACEi after eRx activation vs. before (OR = 2.8, p < 0.01), whereas no significant difference was found for low or medium eRx users. Generic drug use was also significantly associated with patient income (OR = 0.95 for each $10,000 increase in annual household income) and Hispanic race as estimated by patient zip code data (OR = 0.69). Even when the model controlled for these significant factors, no significant association was found between generic pharmacy claims and patient age, patient gender, or black patient race.

SureScripts

Frequency of using formulary and benefit functionality did not change during the follow-up period; however most reported they found formulary and benefit information at least somewhat useful in making medication decisions at the point of prescribing and discussing and evaluating costs with patients. While 72% of participants reported no change in the accuracy of formulary and benefit information, 25% reported improvements in accuracy after the integration of supplemental SureScripts provision of formulary and benefit information.

Exhibit 51. SureScripts Physician Formulary and Benefits Survey

Long Description: The exhibit shows the responses to the question: In the past 2 to 4 weeks, have you noticed a change in the amount of information contained regarding the formulary provided by your e-prescribing software? The chart shows the frequency of respondents who indicated an “increase” a “decrease” and “no change.” Responses are shown for Vendors A, C, D, E-state 1, E-state 2, and F, as well as overall.

Conclusions

Data about the role of e-prescribing on the use of on-formulary medications and generics is still very preliminary. There does appear to be concern about the accuracy of the information being returned to the e-prescribing system, which has led to a variable reliance on these data among clinicians. Additional results will be helpful in understanding the quality (real or perceived) of this information. Generic prescribing, on the other hand, may be most impacted by any prescribing tool that shows generic alternatives for brand name medications. This process is not reliant on patient formulary information; in fact, e-prescribing user interfaces that include an option to automatically allow generic substitution may be sufficient in some settings to assist with generic prescribing. The data provided by RAND support this belief. It is likely that further data from these sites will not demonstrate the marginal contribution of end-to-end e-prescribing over ambulatory CPOE.

Change in Fill Status Rates

Health care providers can use automated information about whether a patient has filled a prescription to track medication compliance and follow-up with patients who may not be taking a prescribed medicine. Three sites tracked outcomes related to fill status, but data are only available from one site at this time.

Ohio-KePRO/ UHMP

UHMP Physicians were significantly more interested than control prescribers in being notified about a no-fill event (6.18 on 1 – 7 scale vs. 5.23; p < 0.01). Baseline interest scores among the UHMP respondents were higher for no-fill alerting than for fill notification (6.18 vs. 5.78, not tested for significance), but interest level scores were nonetheless quite high for both.

No-fill event notices were unable to be implemented for the pilot group, due to major technical hurdles with the implementation approach. Instead, when an incoming RxFill message was successfully matched to an original earlier outbound prescription, the fill status of the prescription was appended to the history of that prescription, and was retrievable upon lookup by users. The post-intervention survey was confounded by incomplete implementation of NoFill messages, making many questions irrelevant. The very small number of responses was mostly from providers who had never seen the RxFill information. These providers continued to want notification about fill events.

However, for those who did receive some no-fill messages, initial enthusiasm waned once the workflow realities became clearer. One physician, who saw an early no-fill message on her patient, commented that she had “ignored it” because it was “very difficult” and “too much work” to respond to it.

Exhibit 52. Interest in Continuing RXFILL by Selected Physician

	Number	Yes	No
Were you aware that you could look up (in ONCALLDATA™) the filled status of any prescription sent to CVS, Rite Aid, or Walgreens during this test?	3	1	2
	Number	Range	Average
Would you recommend continuing this filled status function, even if the NoFill alerts are turned off?	3	6 to 7	6.5
(1 = definitely not to 7 = definitely yes)

Exhibit 53. Interest in Continuing RXFILL by Practice Manager

	Number	Yes	No
Were you aware that you could look up (in ONCALLDATA™) the filled status of any prescription sent to CVS, Rite Aid, or Walgreens during this test?	3	2	1
	Number	Range	Average
Would you recommend continuing this filled status function, even if the NoFill alerts are turned off?	1*	7	7.0
(1 = definitely not to 7 = definitely yes)

*Due to changes in the survey instrument, this question was only on one of the practice manager surveys.

SureScripts

Prior to implementing the RXFILL functionality, Surescripts assessed opinions on the value of this new feature among focus group participants (clinicians and their staff). Of the sixty-three percent of comments about potential usefulness of such a feature, 22% were negative (legal trap, don’t want responsibility, burdensome, “ignorance is bliss”) and 15% were equivocal (might be useful on a selected basis, seems useful but could become overwhelming). Among those who were not enthusiastic about RXFILL, concerns focused primarily on potential liability and additional work generated for clinicians and staff. Although some also conceded that it could provide good clinical information that would probably improve quality of care, most were concerned about the large volume of work that might result. Figures D3 through D6 from the SureScripts report show the survey results after implementation.

Exhibit 54. Figures D3 – D6 from SureScripts Report

Figures D3 – D6 from SureScripts Report

Long Description: The exhibit shows responses to the following four questions: Some e-prescribing software has the capability of alerting the physician when the patient has NOT picked up a prescription. How useful do you think this information would be to your practice?; If the e-prescribing software alerted you to when patients did NOT pick up prescriptions that would have serious medical consequences if not taken, what would you do?; How concerned are you about liability if you know a patient did not pick up a prescription?; and, The medication history option also provides information regarding which medications have been picked up at pharmacy, would you prefer to have the fill info list on active prescription list, check fill information using the prescription history option, or have specific messages alerting you when prescriptions have been filled?

Among clinician respondents, 47% found the functionality very useful and an additional 42% found it somewhat useful. Only 9% of prescribers indicated that they would not want this information for any medications. Eighty percent indicated that if a medically necessary medication was not filled, they would call the patient. Regarding notification that a patient has NOT picked up a prescription, more than half of clinicians indicated that cardiovascular (66%), antidiabetics (64%), anticoagulants (61%), antidepressants (60%), antibiotics (59%), and antipsychotics (53%) were important to be informed about. Forty-four percent were very concerned about liability issues. Fifty-four percent wanted fill information on the active medication lists, 20% desired alerting messages when a prescription was filled, and 26% preferred the information be available via medication history.

RAND

An expert panel was able to discuss several aspects of the RxFill’s problems and potential benefits. Our panelists representing smaller, community pharmacists indicated that at least some would need to implement new software systems to capture the dispensing and return-to-stock events needed to trigger RxFill messages. POC prescribing system vendors also indicated that reconciling RxFill messages with the original prescription could also be challenging because the originating SCRIPT reference number is an optional field. Although the transaction might provide information that prescribers could use to improve medication adherence, several panelists observed that there is no marketplace demand for RxFill. As one participant put it, even “if a physician wants it, who is going to pay for it?”

One panelist suggested that using the RxFill (dispensed) message alone could suffice for non-adherence alerting, since the lack of a message within some specified time interval could be used to trigger an alert. However, another panelist observed that the existence of opt-out mechanisms would undermine this mechanism, even if implementation of RxFill message was mandated saying, “If patients are opting-in or opting-out … then [if] the physician doesn’t get a ‘filled’ response what does the physician know? Maybe I opted out. They can’t really determine that it was filled, and they can’t determine that it wasn’t filled.” Panelists were also concerned about mechanisms for letting patients opt-in or opt-out of providing this information. One said, “The process of setting-up and maintaining the [opt-in or opt-out] indicator would be significant. Numerous interfacing systems would need to change to allow for modification of this indicator.” However, another said, “that’s something that can be designed for and I think that having a patient opt in or out of this is probably something on which we should do more research.”

Fill Status Alerting Focus Groups. Allscripts users who participated in focus groups had significant concerns about the new burdens that Fill Status alerting could place on their time and their offices. Prescribers were generally interested in whether their patients are taking what is prescribed and they want to take action based on this information. However, most indicated that their adherence monitoring activities were limited to the patient visit and that they would not have time to telephone patients about failures to fill or refill prescriptions. Time was also cited as a barrier to discussing adherence during the patient visit. A minority of providers had some familiarity with the existing feature in the Allscripts web interface for browsing the patient’s Med Hx records.

There was significant concern among prescribers about new medico-legal liability that could result from the existence of non-adherence alerting. Although some debated whether the ignorance that generally exists today would be any protection, one participant summed up the concern by saying “I can imagine, people, lawyers using it in instances to say ‘You were aware Mrs. S wasn’t taking her medications. Why didn’t you take greater steps to encourage her compliance before she had this stroke?”

Conclusions

Fill status use was extremely limited, due primarily to difficult implementation of this standard in both practices and community pharmacies. This observation does not appear to bode well for the adoption of the standard in the short term. Fill status analyses thus far are widely varying. Surescripts is arguably quite biased in their assessment of this outcome, given their reliance on refill requests in their business model. As mentioned by the RAND expert panel, satisfaction with RxFill is likely to be somewhat dependent on the implementation of fill status messages in e-prescribing tools, as well as other reasons patients have for untimely pick up of prescribed medications[lxii] [lxiii] [lxiv].

Improved Security and Reliability of Prescriptions

Migrating to an electronic system of prescribing has the potential to eliminate uncertainties and insecurities that are currently ubiquitous in the paper-based environment. A robust password-protected architecture for e-prescribing can prevent protected health information from slipping into the wrong hands and enhance health officials’ ability to track the trade of narcotic drugs.

Achieve

E-prescribing Security and eSignature Infrastructure – Long Term Care Alternate Model. The diagram below pictorially represents the LTC perspective and how security is invoked in the LTC environment.

Exhibit 55. E-prescribing Security and eSignature Infrastructure – Long Term Care Alternate Model

Long Description: The exhibit shows a flow chart that outlines Achieve’s process for ensuring security in e-prescribing transactions. A prescriber will prescribe from a desktop, tabletPC, or PDA; send the message over a secure POC network to a POC server with a firewall; the message will continue over a secure network to a router server with a firewall; the message will continue to a pharmacy central server with a firewall; and finally, the message will be routed to pharmacy computers over a secure pharmacy internal network.

The diagram shows that the industry has implemented various touch points to mitigate technical risks. The following information provides more detailed explanation of mitigating technical security risk using this e-prescribing model in the LTC environment.

§ Controlled Substance Routing From Prescriber System – Prescriptions for controlled substances will be routed directly from the prescriber system through the router to the pharmacy. The router will send a parallel message to the facility to update the facility’s electronic record.

§ Controlled Substance Routing From Facility CPOE System – Prescriptions for controlled substances will be routed directly from the facility system through the router to the pharmacy. The router will not send a parallel message back to the prescriber system.

§ User Authentication - All users are authenticated before being granted access to any application, database or network involved in e-prescribing.

§ System Authentication – All inter-network communications (Prescriber to LTC Facility, LTC Facility to Router, Router to Pharmacy) are subject to authentication based upon IP address, ID and password authentication prior to opening a secure channel.

§ Wireless devices accessing Prescriber or LTC Facility networks will do so via secure connections and require user authentication.

§ Prescriber devices directly accessing a LTC Facility CPOE systems do so via a secure connection and require user authentication before access is granted.

§ Use of PHI (protected health information) is always done in accordance with HIPAA standards for the purpose of treatment, payment, or healthcare operations.

§ Router operations adhere to all applicable HIPAA security guidelines.

§ Router performs internal assessments using security scanning tools for network and system security.

§ Router maintains only enough information to allow for routing, auditing and support.

§ Router may not view or modify eRx except when translating from one messaging standard to another (e.g. HL7 to NCPDP)

§ Pharmacy stores a cross-reference table containing DEA number and their unique IDs (assigned by the POC server).

§ Pharmacist may contact POC or prescriber at any time to verify the authenticity of the eRx.

§ Prescriber, LTC Facility, Router and Pharmacy maintain transaction logs that may be used for auditing purposes.

Conclusions

Since Achieve was the only pilot to report on security and privacy issues arising from their e-prescribing project, findings in this area are clearly limited. However, the security and authentication architecture Achieve developed shows that the LTC industry is taking steps towards implementing secure electronic systems.

SECTION VII: CONCLUSIONS AND RECOMMENDATIONS

This report assesses the findings of five pilot sites that were active through the end of 2006. There are two levels of final findings that we present in this section: first, recommendations regarding the appropriateness of specific standards for adoption as part of the Medicare Part D program and second, conclusions regarding the types of clinical and workflow outcomes that may be affected by e-prescribing.

Because the projects ended so recently, pilot sites are still analyzing data and coming to their own conclusions about the success of the proposed standards and the outcomes and impacts of e-prescribing. At this stage we believe there is enough information available to make preliminary recommendations regarding standards adoption on the part of CMS and that it is unlikely that these recommendations will change.

We are less able to draw definitive conclusions regarding outcomes associated with e-prescribing as pilot sites are still working through their own results and drafting their conclusions on these issues. However, we do have some preliminary conclusions related to outcomes. These are items that were so striking from the preliminary findings that they are unlikely to change when all data are available. In the paragraphs below we outline our draft conclusions and recommendations, first as they relate to standards being tested by the pilots and second as they relate to outcomes to be expected from e Prescribing.

Standards Testing

Results from standards testing described in the present report include testing of proposed initial standards as well as their interoperability with foundation standards that are already in use in the industry. While overall there is no new information regarding the usefulness of foundation standards, the pilot conducted by Achieve did elucidate several modifications to the NCPDP Script 8.1 standard that would benefit their use in the LTC setting. These modifications were described in the standards testing discussion as well as in the Executive Summary and the importance of considering and addressing these modifications represents an important conclusion in this study. Testing of the initial standards themselves produced varying results. Based on the data available so far, three standards appear to be well-specified, and the evaluation team recommends them for adoption: Formulary and Benefits, Medication History, and Fill Status Notification. However, for each of these standards, there are implementation issues that need attention:

§ Formulary and Benefits. Several pilots found that the functionality of this standard was limited by the ability to match eligibility data for patients. This is largely attributable to 1) the large volume of data that is maintained and supplied by PBMs/Payers, 2) the challenge of implementing this large volume of data so that it is clinically relevant and usable, and 3) ensuring that actual coverage of a patient takes into consideration additional factors like patient’s year-to-date drug expenditures. Some of these limitations will be addressed in the next version (version 5010) of the eligibility standard.

§ Medication History. Although the standard worked technically, only a small fraction of prescribers used the function or knew it was available. More research must be done to determine the optimal way to display and maintain this list. One issue that arose is that for prescribers to find this function useful, it must create a very complete list by pulling information from a wide variety of sources. The experience from the pilots suggests that more work is necessary before the standards driven medication history information can be reconciled effectively from multiple sources and be displayed in a user-friendly manner within the e-prescribing system.

§ Fill Status Notification. Again, although this standard worked as a standard, pilots had trouble implementing it. An important barrier is that pharmacies are not always able to track fill status at the point of sale. Although the capability of tracking fill status exists, this has not been implemented in many of the pharmacy systems today. Based on evaluation of the pilots, there was no strong message from prescribers indicating that the fill status information is particularly useful to their practice.

While pilot sites working with each of these standards acknowledge important implementation and workflow issues need to be addressed prior to achieving the full benefits of adoption, adoption of the standard by CMS will likely encourage stakeholders to continue to work through these issues and will create incentive for new e-prescribing efforts to work with these standards.

The three other standards are not as far along in their development. Testing of these standards was more limited, and often based only in a lab or hypothetical environment. Even with this more limited testing, pilot sites had significant concerns with each standard. The evaluation team recommends that they undergo further development and testing before adoption:

§ Prior Authorization. A final PA standard should support a fully automated, real-time e-PA process. This standard should be built with the assumption that criteria can be pre-loaded into point-of-care (POC) software systems (the unsolicited model). The current standard should be improved to be a) organized by drug, b) support content logic (conditionality), numbering of questions and cardinality, c) provide for educational information and directions, d) support open-ended questions, e) uniquely identify the patient, and f) provide patient-specific PA status. The focus should be on providing an infrastructure and format for e-PA, but it is unlikely that health plans will agree to standardized forms or questions for the PA request.

§ Structured and Codified SIG. Further development is needed with reference to field definitions and examples, field naming conventions and clarifications of field use where new codes are recommended, such as the SIG Free Text Indicator field. With additional development, the standard may provide a controlled vocabulary that reflects prescriber thinking, offers structure and simplicity, and improves communications between prescribers and pharmacies. In addition the pilots reported that simplification of the standard is required and that the implementation guides provide examples to avoid misinterpretation.

§ RxNorm. The RxNorm dataset is not yet complete and requires further refinement to include all medications. When it is complete, RxNorm has the potential to simplify e-prescribing, create efficiencies, and reduce dependence on NDCs.

In addition to implementation challenges these standards may face, each has problems that are internal to the standard itself. In the case of Prior Authorization some reorganization and refinement of the standard is necessary to support business needs of key stakeholders. For structured and codified SIG additional information is needed as part of the standard itself to increase the usability, and RxNorm is currently incomplete as a dataset (v. 12/21/06). Each of these standards holds promise, but, the experience from the pilots suggests that they should be revised prior to adoption by Medicare Part D.

Outcomes and Impacts

Data on outcomes are even more limited at this point in time than data on the functionality of the initial standards. As many have acknowledged, measuring changes in clinical and economic outcomes following the implementation of e-prescribing is a challenging task and the data that are just now emerging from the pilots are not fully digested and understood well enough at this stage to offer definitive conclusions. To date, however, the pilot sites have learned a great deal that should impact e-prescribing standards implementation.

Economic impact of e-prescribing standards is significant. Our estimate is that rulemaking associated with the adoption of initial standards for e-prescribing is “economically significant” relative to federal legislative definitions and that costs associated with technology adoption and business practice change will be incurred by prescribing providers as well as pharmacies and payers. Each of these groups may also incur benefits through simplification and efficiency with the likelihood of system-wide economic benefits increasing with improved adoption. Importantly, however, we note that beneficial impacts will require not only use of standards, but adoption of specific technology and changes to business practices among, providers, payers and pharmacies. For example, pilot’s report that many, if not most retail pharmacies have not yet adopted systems that are fed directly by standards-based electronic messaging. This means that in these cases e-prescriptions come through via fax on electronic inbox and must be re-entered by pharmacy staff offering a slight improvement in pharmacy efficiency relative to paper prescriptions.

Role of office staff and surrogates. Successful e-prescribing involved not only authorized prescribers, but their staff. In these pilots, though vaguely described, a large number of prescriptions were generated by agents of the prescriber. This finding is consistent with other published literature, as noted above. Additional studies evaluating these standards, and other implementation efforts, must take these agents of the prescriber into account. Workflows for newly implemented features, such as RxFill and NoFill, may well be better aligned with the jobs of surrogates. The role of surrogates is particularly important when understanding the impact of e-prescribing on ADEs. The presence of standards-driven information at the point of care can only impact clinical outcomes to the extent that a licensed prescriber is at the ready to receive this information and incorporate it into a prescription decision at the point of care.

Knowledge and attitudes that impact adoption. E-prescribing adoption does not appear to be well integrated into the thinking of many practitioners. While there was enough uptake in each pilot to support the project, there were also non-adopters and drop-outs. Even among e-prescribers, in almost no setting did e-prescribing replace the need for paper-based prescribing. Data from the RAND analysis, and data from Ohio KePRO, suggest that clinicians identify their existing workload, PDA technical difficulties, and pharmacy readiness as major barriers to the sustained adoption of e-prescribing. In addition, physician office culture may play a large role in determining who adopts and who does not. Unfortunately, it is not clear whether there is a knowledge deficit about safe prescribing or other primary focal point for education that could shift their willingness to adopt e-prescribing. Data about ambulatory prescribing are still scant in the published literature. The results of these pilot sites will provide a comprehensive set of data about reasons for failed adoption of e-prescribing. CMS should encourage the publication of these results, followed by more in-depth analysis of the knowledge and attitudes that appear to impact adoption, with a goal of AHRQ/CMS-funded studies to address these issues.

Impact of e-prescribing at pharmacies. The evaluation team found that for the most part, pilot sites were unprepared for the challenges that pharmacies would have when managing e-prescription messages. The pilots reported significant problems being shifted “downstream” and requiring the cooperation of pharmacists and patients to reconcile, including “lost” prescriptions and a lack of clarity about which prescriptions to fill. In addition, the workflow changes required at the pharmacy level were significant, with a number of sites reporting additional data entry tasks when e-prescribing was implemented. These findings however, should be tempered by the fact that most pilot sites had a partial or not fully electronic prescribing system in place, e.g., e-faxing. These partial e-prescribing systems may lead to temporary reductions in workflow, affecting some aspects of pharmacy work which could ultimately be resolved with full-scale implementation. CMS might consider convening a panel of pharmacy chains to elucidate the problems (both workflow and technical) being anticipated to result from the adoption of the initial standards and to discuss strategies to mitigate these issues.

Impact of e-prescribing on prescriber behaviour. Through observational studies, site visits, and interviews, the pilot sites were able to characterize how e-prescribing changes the number and type of drugs providers prescribe. In the LTC setting, e-prescribing may enable physicians to manage the myriad of drugs taken by their patients, helping to reduce the number of redundant or unnecessary prescriptions. One pilot site found a statistically significant reduction in this vein. In addition, e-prescribing may affect prescriber behaviour by encouraging them to switch patients’ medications from brand-name to lower-priced generic drugs. Finally, new prescriber workflows associated illustrated by the pilots show that paper-based prescribing may not necessarily be eliminated after the introduction of e-prescribing.

Overall, the work of the pilots demonstrated the potential for effective standards-based implementation of e-prescribing as it relates to three of the initial standards: Formulary and Benefits, Medication History and Fill Status Notification. In testing, these standards have been proven to be an effective means of supporting structured transmission of information crucial to achieving the benefits associated with e-prescribing. While the evaluation team has determined that there is a need to modify the standards, there continues to be significant issues with respect to their implementation. The three additional standards being tested, while holding promise, require additional work to the standard itself prior to adoption for government programs.

Prescription History	N	Range	Average
When available, how would you rate the usefulness of these prescription histories, in general?	3	6	6.0
(1 = not useful to 7 = very useful)
When available for you to review, what impact did the prescription histories generally have on time spent during an encounter?	3	3 to 6	4.3
(1 = took more time to 7 = saved time)
When available, was reviewing these prescription histories worth the effort?	3	7	7.0
(1 = definitely no 7 = definitely yes)
Would you recommend continuing to print prescription histories for review?	2	0 to 6	3.0
(0 = not applicable, 1 = definitely not 7 = definitely yes)

Reasons Prescribers Found Printed Prescription Histories Useful
Reason	N
Able to look up information that patient could not provide	2
Able to identify drug seeking patients	3
Able to identify compliance issues	3
Help build medication list for new patients	1*
Help update medication lists during visits	1*
*Due to changes in the survey instrument, this question was only on one of the prescriber surveys.

Resident Order Analysis of Both Treatment Facilities
	May	December	Decrease	% Decrease
Average Active Orders/Resident/mos	13	12.8	0.2	1.54%
Number of residents with 9 or more active orders/month:
	May	December		% Decrease
# of residents with 9+ orders	188	198
Total residents	234	254
% of residents with 9+ orders	80.34%	77.95%		2.39%

Practice Type/eRx Status	Drug-Drug	High Dose	Total DUR	Total Rxs	DUR Rate/1,000 Rx
UHMP - No	4	29	33	10,874	3.03
UHMP - Yes	54	121	175	65,357	2.68
Control	13	22	35	8,991	3.89
TOTAL	71	172	243	85,222	2.85

Specialty	Practice Type/eRx Status		Drug-Drug	High Dose	Total DUR	Total Rxs	DUR Rate/1,000 Rx
Family Medicine	UHMP - No		0	4	4	606	6.60
	UHMP - Yes		5	13	18	7,331	2.46
	Control		10	5	15	5,001	3.00
	Subtotal		15	22	37	12,938	2.86

Internal Medicine	UHMP - No		4	17	21	8,962	2.34
	UHMP - Yes		49	92	141	55,886	2.52
	Control		2	12	14	3,423	4.09
	Subtotal		55	121	176	68,271	2.58

Pediatrics	UHMP - No		0	8	8	1,306	6.13
	UHMP - Yes		0	16	16	2,140	7.48
	Control		1	5	6	567	10.58
	Subtotal		1	29	30	4,013	7.48
		TOTAL	71	172	243	85,222	2.85

Medication Safety	Baseline Period		Intervention Period
	Control Site n (%)	Intervention Site n (%)	Intervention Site Only consented ePrescribers n (%)
Medication Errors	50 (0.4)	89 (1.4)	137 (1.7)
Near Misses	130 (1)	70 (1.1	96 (1.2)
Significant	53 (0.4)	35 (0.6)	45 (0.5)
Serious	68 (0.5)	32 (0.5)	48 (0.6)
Life-Threatening	9 (0.07)	3 (0.05)	1 (0.01)
ADEs*	6 (.04)	2 (0.03)	1 (0.01)

The [medication history] in [system 1 \| system 2]	Strongly Disagree	Disagree	Neutral	Agree	Strongly Agree
Is complete for most patients	10%	12%	39%	39%	0%
Helps me to identify and address patients' adherence problems	6%	12%	43%	39%	0%
Saves me time	10%	6%	60%	20%	4%
Overall, improves the quality of my prescribing	6%	8%	45%	37%	4%
Overall, I am satisfied with the medication history list in the e-prescribing system	8%	14%	41%	37%	0%

ADE Monitor	Baseline Period n (%)	Intervention Period n (%)		All Periods n (%)
	All Clinics	Control Clinics	Intervention Clinics
All Rxs (meds only)	25126	3121	4929	33176
No ADE Hit	21938 (87.3)	2685 (86.0)	4205 (85.3)	28828 (86.9)
ADE (+) Hit	3188 (12.7)	436 (14.0)	724 (14.7)	4348 (13.1)
Actual ADE	NA	NA	NA	NA

Medication/Dispensing Hx	Control Clinics		Intervention Clinics
Period	Baseline n (%)	Intervention n (%)	Baseline n (%)	Intervention n (%)
Electronically prescribed	no	no	no	YES	no
Rx in webOMR	5093	6354	19670	2179	7726
Reviewed Rx	1313 (25.8)	2723 (42.8)	9695 (49.2)	1574 (72.2)	3974 (51.4)
Dispensing Hx available	525(40.0)	584(21.4)	4603(47.8)	633(40.2)	953(24.0)
No error	497(94.7)	539(92.3)	4371(95.9)	608 (96.1)	875 (91.8)
Prescribing error corrected	6 (1.1)	8 (1.3)	42 (0.9)	6 (0.9)	20 (2.1)
Dispensing error	22 (4.1)	37(6.3)¹	190 (4.1)	19 (3.0)²	58 (6.1)²

	Control Clinics			E-prescribing Clinic
	n (%)	n (%)	p value	n (%)	n (%)	p value	n (%)	p value
Period	Baseline	Intervention		Baseline	Intervention		Intervention
Electronically prescribed	no	No		no	Yes		no
Prescriptions in webOMR	5093	6354		19670	2179		7726
Excluded prescriptions (see above)	3780 (74.2)	3631 (57.1)		9975 (50.7)	605 (27.8)		3752 (48.6)
Prescriptions for Review	1313 (25.8)	2723 (42.8)		9695 (49.2)	1574 (72.2)		3974 (51.4)
Dispensing history not available upon review	788(60.0)	2139(78.5)		5092(52.5)	941(60.0)		3021(76.0)
History available for analysis	525(40.0)	584(21.4)		4603(47.8)	633(40.2)		953(24.0)
No error	497(94.7)	539(92.3)		4371(95.9)	608 (96.1)		875 (91.8)
Prescribing error corrected	6(1.1)	8(1.3)		42(0.9)	6(0.9)		20 (2.1)
Dispensing error	22(4.1)	37(6.3)	0.110	190(4.1)	19(3.0)	0.005¹	58(6.1)	0.004¹

SECTION I: INTRODUCTION

Pilot Sites

Prescription Fill Status Notification

Prior Authorization

Ohio KePRO/ UHMP

SECTION V: EVALUATION FINDINGS: INITIAL STANDARDS

MA-SHARE- Brigham & Women’s

RAND

SURESCRIPTS

ACHIEVE

Ohio-KePRO/UHMP

CONCLUSIONS:

MA-SHARE- Brigham & Women’s

RAND

SURESCRIPTS

ACHIEVE did not test this standard.

Ohio-KePRO/UHMP

CONCLUSIONS:

MA-SHARE- Brigham & Women’s

RAND

SURESCRIPTS

Ohio-KePRO/ UHMP

CONCLUSIONS

MA-SHARE- Brigham & Women’s

RAND

ACHIEVE

CONCLUSIONS

MA-SHARE- Brigham & Women’s

RAND

SURESCRIPTS

CONCLUSIONS

MA-SHARE- Brigham & Women’s

RAND

SURESCRIPTS

Achieve

Nursing Staff Satisfaction

Pharmacy Staff Satisfaction

Brigham & Women’s Hospital (BWH)

Ohio-KePRO/ UHMP

RAND

SureScripts

Conclusions

Task Category: Communication

Task Category: Fax Communication

Other Task Categories

BWH

Ohio-KePRO/UHMP

RAND

SureScripts

Conclusions

Achieve

BWH

Ohio-KePRO/UHMP

RAND

Conclusions

SureScripts

Conclusions

Ohio-KePRO/ UHMP

RAND

SureScripts

Achieve/ LTC

BWH

Conclusions

Achieve

Ohio-KePRO/UHMP

SureScripts

Conclusions

BWH

No error

Corrected Rx for review

No error

Ohio-KePRO/ UHMP

SureScripts

RAND

Conclusions

Achieve

Ohio-KePRO/ UHMP

Cost Differences by Practice Type:

RAND

SureScripts

SECTION V:
EVALUATION FINDINGS: INITIAL STANDARDS