Design: Inpatient nursing units in an academic health system were the setting for the study. The study comprised before-and-after comparisons between phase 1, pre-implementation of POE (pre-POE) and phase 2, post-implementation of POE (post-POE) and, within phase 2, a comparison of POE and the combination of POE plus eMAR. Length of stay and cost were compared pre- and post-POE for a period of 10 to 12 months across all services in the respective hospitals.

Measurements: Comparisons were made pre- and post-POE for the time intervals between initiation and completion of pharmacy (pre-POE, n=46; post-POE, n=70), radiology (pre-POE, n=11; post-POE, n=54), and laboratory orders (without POE, n=683; with POE, n=1,142); timeliness of countersignature of verbal order (University Hospitals [OSUH]: pre-POE, n=605; post-POE, n=19,225; James Cancer Hospital (James): pre-POE, n=478; post-POE, n=10,771); volume of nursing transcription errors (POE with manual MAR, n=888; POE with eMAR, n=396); length of stay and total cost (OSUH: pre-POE, n=8,228; post-POE, n=8,154; James: (pre-POE, n=6,471; post-POE, n=6,045).

Results: Statistically significant reductions were seen following the implementation of POE for medication turn-around times (64 percent, from 5:28 hr to 1:51 hr; p<0.001), radiology procedure completion times (43 percent, from 7:37 hr to 4:21 hr; p<0.05), and laboratory result reporting times (25 percent, from 31:3 min to 23:4 min; p=0.001). In addition, POE combined with eMAR eliminated all physician and nursing transcription errors. There were 43 and 26 percent improvements in order countersignature by physicians in OSUH and James, respectively. Severity-adjusted length of stay decreased in OSUH (pre-POE, 3.91 days; post-POE, 3.71 days; p=0.002), but not significantly in James (pre-POE, 3.68 days; post-POE, 3.61 days; p=0.356). Although total cost per admission decreased significantly in selected services, it did not change significantly across either institution (OSUH: pre-POE, $5,697; post-POE, $5,661; p=0.687; James: pre-POE, $6,427; post-POE, $6,518; p=0.502).

Conclusion: Physician order entry and eMAR provided the framework for improvements in patient safety and in the timeliness of care. The significant cultural and workflow changes that accompany the implementation of POE did not adversely affect acuity-adjusted length of stay or total cost. The reductions in transcription errors, medication turn-around times, and timely reporting of results supports the view that POE and eMAR provide a good return on investment.

Institutions embarking on the implementation of POE require tangible evidence that this technology will indeed enhance patient care and improve institutional efficiencies. While most available literature highlights benefits realized by a few institutions using homegrown systems,⁵ we show the benefits of implementing a vendor-based POE system, Invision 24 with graphical user interface (Siemens Medical Solutions Health Services Corp., Malvern, Pennsylvania) on various aspects of work flow and efficiency, enhanced patient safety, and reduced length of stay and costs.

Previous studies have consistently demonstrated the beneficial effects of computerized POE on the reduction of medical errors.^8–11 Computerized POE offers complete and accurate information, automatic dose calculations, and clinical decision support at the point of care, including drug–drug interaction and allergy checking, as well as ordering supported by evidence-based best practice. These features help ensure the equitable delivery of quality care, minimize human error, improve medication management, facilitate reporting and decision making, and improve resource utilization while offering timely access to information and supporting compliance management. There is a paucity of information in the literature about the additional safety benefits resulting from combining POE and electronic medication administration record (eMAR).

After a multi-year planning and development process, POE was deployed simultaneously on a large scale across all inpatient clinical areas except the maternal-fetal area. Health care providers, including residents and nurses, were engaged in the POE initiative from the very beginning, in the hope of creating leaders who would champion the positive attributes of POE.¹² These processes were determined to be critical to the successful implementation and the realization of benefits.

Our POE system is a commercially available vendor-based system that was extensively modified with vendor-provided tools to meet the needs of the physician user. The POE system supports the ability of the clinician to write any order, consult, or requisition electronically. Orders are interfaced bidirectionally with our laboratory and radiology systems and unidirectionally with our dietary system. Orders are printed directly to pharmacy and the other ancillary areas.

Numerous clinical decision support tools are integrated into the ordering pathways. These include, but are not limited to, allergy checking, drug interaction, order duplicate checking, corollary order checking, weight-based dosing, and drug route restriction. In addition, a number of clinical alerts run simultaneously during the ordering process through a rules engine application. More than 450 evidenced-based order sets, designed to meet the needs of all clinical specialties from oncology and rehabilitation to intensive care (both medical and surgical), are available to facilitate and expedite electronic order entry as to support best practice.

To measure the immediate tangible benefits following the implementation of POE, The Ohio State University Health System (OSUHS) established metrics for the following criteria—patient care quality and safety, work flow efficiency, regulatory compliance, and cost. The POE implementation objectives of OSUHS parallel the Institute of Medicine’s six dimensions of care, which define high-quality patient care as safe, patient centered, effective, timely, efficient, and equitable.¹³ These objectives guided planning and implementation strategies of our electronic medical record, including POE and eMAR, throughout the entire process.¹²

To evaluate the realization of these goals, the OSUHS formally collected data on patient care practices through chart review and the direct observation of physicians as they prepared written orders, prior to implementing the computerized POE system. This process provided the hospital with comparison data against which to accurately measure the effects of the new computerized system. Following POE implementation, data were obtained electronically from the POE system and a computerized information warehouse.

The objective of the study was to quantify and document the immediate tangible benefits on patient care practices that were realized after the implementation of POE and eMAR at the OSUHS.

To meet the institutional needs for rapid system evaluation, a broad approach was adopted instead of an in-depth analysis. Three different methods of data collection were necessary because of the complexities of criteria measured. Because of the complexity of the time and motion studies, the narrative is supplemented with a table (Table 1 ).

Table 1 Summary of Processes Measured

The Ohio State University Health System is an academic medical center that comprises four hospitals dedicated to physical rehabilitation, behavioral health, oncology, and general medical, surgical, and obstetric care. Within the scope of POE implementation, these hospitals include high-acuity areas, such as surgical, medical, and bone marrow intensive care units.

Physician order entry was introduced at OSUHS from February through May 2000. The dedicated surgical transplant unit was the first to go live, in February 2000, followed by the oncology environment of the James Cancer Hospital (James), in April 2000. The Ohio State University Hospitals (OSUH) went live with POE in May 2000, and Dodd Physical Rehabilitation Center (Dodd) followed a little later in April 2001.

On inpatient care units where POE is implemented, all orders are processed through the system, with 80 percent being entered by physicians and the rest by nursing or other licensed care providers.¹³ In addition, an eMAR documentation application that is incorporated into POE was implemented on the organ transplant service and in all areas of the James and Dodd.

Medication Turn-around Times A medication event was defined as the completion of the entire cycle, from the physician order (manual or electronic) through medication administration. In the study of medication turn-around times, additional measurements specific to pharmacy were made. These included the time orders were received and verified by pharmacists, the time orders were entered into the pharmacy system, and the time medications were actually administered to the patients.

Measurements were made only for medications in which the timely administration was critical to the patient outcome. These included the first dose of intravenous medications that were dispensed by central pharmacy. PRN medications, medication available on “unit dispensing system,” and oral medications that were routinely scheduled were excluded.

Radiology Procedure Completion Times A radiology event was defined from physician order (manual or electronic) through completion of the procedure. Times associated with receipt of radiology examination requests as well as times associated with scheduling and completion of procedures were collected for radiology orders. Targeted radiology procedures included chest and abdominal x-ray films and abdominal ultrasound scans, determined to be the highest volume services ordered. All other radiology procedures were excluded from the study. Those procedures that did not have the required time elements were excluded from the analysis.

Laboratory Result Turn-around Time (With and Without POE) Surgical and medical intensive care units were selected for this study. At the time of the study, POE was implemented on the surgical intensive care unit (SICU) but not the medical intensive care unit (MICU). All laboratory orders were included for a 2-mo period, from July to August 2000.

In our POE application, the entire process from order entry to result reporting is automated. The order interfaces to the laboratory system and generates the laboratory label, which is necessary for the laboratory to test and result the specimen correctly. Without POE, the order requires completion of a manual requisition and manual entry of patient and specimen information by laboratory technicians. This generates the label that facilitates the remainder of the process.

The study analyzed the time between receipt of the specimen in the laboratory and electronic posting of the result. In the MICU (no POE), receipt time was when the paper requisition and specimen were received and manually entered into the laboratory system. In the SICU (with POE), receipt time was the time the specimen was received in the laboratory.

Medication Transcription Errors (With and Without eMAR) Two surgical units that had implemented POE were studied. However, one unit had also implemented eMAR, which is created from the orders in POE. The data were collected between December 2000 and January 2001 from the eMAR (area A) and manual MAR (area B). The audit included evaluation of incorrect order, order transcribed incorrectly, order not transcribed, and medication administered and documented as given without corresponding order in POE.

Orders Requiring Physician Countersignature Pre-POE data were collected manually by Medical Information Management staff through the evaluation of a random sample of charts over 3 days, from Nov 28 to 30, 1999. Following implementation, all patient orders were evaluated electronically from the POE system, from Jul 9 through Aug 7, 2000. Orders requiring physician countersignature included both verbal and telephone orders communicated to licensed staff. The total number of orders requiring countersignature vs. the number that were countersigned was calculated.

Length of Stay and Total Costs Pre-POE and post-POE data were obtained through the centralized information warehouse. The length of stay, case mix index, and total costs for each patient were obtained from this database. The case mix index, or the acuity index, is the weight assigned by Health Care Financing Administration (HCFA) to each diagnosis-related group.¹⁴ This value is used to weight reimbursement to determine the intensity of resource use and indirectly determine the severity of illness by diagnosis. The pre-POE and post-POE time periods were identical for each medical service studied, varying between 10 and 12 months, depending on implementation dates.

Measurements and outcomes are summarized in Table 2 .

Table 2 Summary of Measurements and Outcomes

Medication Turn-around Times The results of measurements of 46 medication events before POE implementation were compared with 70 medication events following POE implementation. A statistically significant (p<0.001) 64 percent reduction in medication turn-around time was observed. The medication turn-around time decreased from 5 hr 28 min pre-POE to 1 hr 51 min post-POE (Figure 1left ). The two key phases that saw improvement were communication of the order to pharmacy (which includes the steps between physician ordering and dispensing by pharmacy) and administration of the dispensed medication to the patient (which includes the steps between dispensing by pharmacy and administration of the medication to the patient). The first phase decreased from 3 hr 57 min to 33 min, and the second phase decreased from 3 hr 16 min to 1 hr 22 min (Figure 1right ).

Figure 1 Medication turn-around times. Left, Overall, from order entry to medication administration in the transplant service. Right, Two phases, from order entry to pharmacy, and from pharmacy to medication administration in the transplant service. Black columns, (more ...)

Radiology Procedure Completion Times The measurements of 11 pre-POE radiology events were compared with those of 54 post-POE events. There was a statistically significant reduction (43 percent, p<0.05) in the interval from the time of radiology request to the time of completion of the procedure following the implementation of POE (from 7 hr 37 min to 4 hr 21 min) (Figure 2 ).

Figure 2 Radiology turn-around times, from order entry to procedure completion, in the transplant service. Black column, pre-POE (n=11; Jan 15–Feb 15, 2000); white column, post-POE (n=54; May 1–30, 2000).

Laboratory Result Turn-around time From July to August 2000, the SICU had 1,142 laboratory orders and the MICU 683 laboratory orders.

Comparison of laboratory turn-around times for the two units revealed a 25 percent reduction in result reporting time (p=0.001). The average result reporting time in the SICU, using POE, was 23 min 4 sec, whereas in the MICU, where manual order entry was used, the result reporting time was 31 min 3 sec (Figure 3 ).

Figure 3 Pathology and clinical laboratory result report times for manual order entry in the medical intensive care unit (n=683, black column) and physician order entry in the surgical intensive care unit (n=1,142, white column).

Medication Transcription Errors From area A (POE and eMAR), 396 medication orders originating from 25 patients were analyzed and were compared with 888 medication orders from 80 patients in area B (POE and manual MAR). In area B, 11.3 percent of orders had transcription errors, whereas transcription errors were completely eliminated in area A. Of the transcription errors in area B, 4.6 percent of orders were not transcribed, 3.5 percent were transcribed incorrectly, and 1.2 percent were transcribed without a corresponding physician order (Figure 4 ). In this study, 1.9 percent of orders had errors occurring because of incorrect order writing by the physician.

Figure 4 Number and type of transcription errors recorded during the audit. A, Incorrect order—the order was written with incorrect elements by the physician. B, Transcribed without order—documentation of administration is present without a corresponding (more ...)

Order Requiring Physician Countersignature Before POE implementation, 605 patient orders were analyzed for OSUH, and 478 were analyzed for the James. After POE implementation, 19,225 patient orders were analyzed for OSUH and 10,771 were analyzed for the James. There were statistically significant (p<0.001) 43.1 and 26.2 percent increments in physician countersignature of non-physician-entered orders in OSUH and James, respectively (Figure 5 ).

Figure 5 Cosigned verbal orders at OSUH (graph on left) and James (graph on right), showing percentage of signed verbal orders (black columns) and unsigned verbal orders (white columns) before POE implementation (Nov 28–30, 1999) and after (Jul 9–Aug (more ...)

Length of Stay and Total Cost per Admission There was a statistically significant decrease in patient acuity-adjusted length of stay following implementation of POE in OSUH in the heart service (pre-POE, 3.35 days; post-POE, 2.87 days; p<0.001) and organ transplant service (pre-POE, 4.71 days; post-POE, 4.02 days; p=0.002) (Figure 6 ). There were no statistically significant changes in length of stay following the implementation of POE in nephrology, general surgery, cardiothoracic surgery, general medicine, or gastroenterology. When all the services in the OSUH were combined, severity-adjusted length of stay decreased from 3.91 to 3.71 days (p=0.002).

Figure 6 The acuity-adjusted lengths of stay at OSUH and James over 10 to 12 months, pre-POE (black columns) and post-POE (white columns) in these key service areas: TR, transplant; HRT, heart; T/S, cardiothoracic surgery; BMT, bone marrow transplant; HEM, hematology-oncology; (more ...)

In the oncology environment, there was a statistically significant decrease in patient acuity-adjusted length of stay following implementation of POE in neurology/neurosurgery (pre-POE, 2.21 days; post-POE, 2.02 days; p=0.045), but not in bone marrow transplant (pre-POE, 5.48 days; post- POE, 4.28 days; p=0.076) (Figure 6 ).

There was a statistically significant increase in length of stay in gynecology/oncology (pre-POE, 3.01 days; post-POE, 3.35 days; p=0.024) but no statistically significant change in length of stay in surgical oncology or in thoracic surgery. When all the services in the James were combined, the reduction in severity-adjusted length of stay of 3.68 to 3.61 days (p=0.356) was not significant.

At the OSUH, total costs for the heart transplant service (pre-POE, $5,264; post-POE, $4,871; p=0.013) and organ transplant service (pre-POE, $8,382; post-POE, $7,711; p=0.043) showed a statistically significant decrease (Figure 7 ), whereas costs for general surgery (pre-POE, $4,995; post-POE, $5,567; p=0.008) showed a statistically significant increase. There were no statistically significant changes in other services.

Figure 7 The acuity-adjusted total costs at OSUH and James over 10 to 12 months, pre-POE (black columns) and post-POE (white columns) in these key service areas: HRT, heart; T/S, cardiothoracic surgery; TR, transplant; HEM, hematology-oncology; SON, surgical oncology; (more ...)

In the oncology environment, services such as surgical oncology (pre-POE, $6,087; post-POE, $5,631; p=0.008) and neurology/neurosurgery (pre-POE, $5,600; post-POE, $5,125; p=0.045) showed statistically significant reductions in total costs, whereas the gynecology/oncology service (pre-POE, $5,046; post-POE, $5,821; p<0.001) showed a statistically significant increase in total costs and thoracic surgery (pre=POE, $5,181; post=POE, $5,946; p = 0.055) showed a nonsignificant increase. There were no significant changes in other oncology-related services.

When all the services were combined, severity-adjusted total cost per admission did not change significantly in either OSUH (pre-POE, $5,697; post-POE, $5,661; p=0.687) or in the James (pre-POE, $6,427; post-POE, $6,518; p=0.502)).

This study confirms the premise that POE can be an effective tool for the delivery of health care. Previous studies have simultaneously compared nursing units with and without POE or, in some cases, established a baseline prior to implementation.^8,9 We have compared the effects of POE before and after implementation on the same nursing unit and thus compared similar patient populations. Although there has been speculation as to the combined effects of POE and eMAR, this is the first report that demonstrates them.

Despite the major cultural change and introduction of new technologies, we did not experience any significant negative results from implementation. In fact, work flow accuracy and efficiency were actually enhanced in many instances (Figures 1 to 3 ). The observed increase in cost or length of stay in specific services is being further investigated. Potential variables that may influence the successful outcome of POE implementation include technical system design, extent of education and training, diversity of the patient population, clinician users, and method of POE deployment.

The accurate administration of medication requires the completion of four interdependent steps—order placement by the physician, transcription by the nurse, verification and dispensing by the pharmacy, and administration by the nurse.^1,16

Errors introduced at any step of the process can be unwittingly transmitted to the next step. The percentage of medication errors occurring in the four stages of the medication process have been categorized: Physician ordering constitutes 39 to 49 percent of the reported errors, nursing administration 26 to 38 percent, medication transcription 11 to 12 percent, and pharmacy verification and dispensing 11 to 14 percent.^1,16 Physician order entry affects physician ordering and pharmacy verification and dispensing by the decision support tools incorporated into the ordering process and the subsequent clarity of the printed order. The eMAR addresses the nursing transcription phase of the medication cycle.

The combined effects of POE and eMAR in our system account for improvements in all phases of the medication cycle, including ordering, nursing transcription, pharmacy verification, and dispensing and administration. In addition, the incorporation of clinical decision support tools in our system, such as screening for drug allergies and drug–drug interactions, reduces error reduction and prevents adverse events.

The implementation of an integrated online MAR with POE led to complete elimination of nursing transcription errors, while POE with manual medication charting resulted in a transcription error rate of 11.3 percent (Figure 4 ). The implementation of POE greatly decreases but does not eliminate order transcription. If left as a paper process, the MAR still requires order transcription. When online eMAR and POE are integrated, the need for order transcription is eliminated. Thus, the entire process of entering, communicating, transcribing, and documenting the order becomes electronic. Total elimination of transcription leaves little room for errors associated with interpretation and translation. In this study, unlike other studies, we examined the effects on nursing transcription and administration errors associated with the implementation of online eMAR integrated with POE.

In our study, the time that it took for an order to be communicated to pharmacy was significantly reduced by POE, from 3:57 to 0:33 hr (Figure 1left ). We expected to see increased efficiency at this stage in the process, since the multiple steps required by manual medication ordering and subsequent delivery of the paper order to pharmacy are eliminated with POE. The medication order verification and dispensing process in pharmacy obviously remains manual, providing an opportunity for error. Unexpectedly, we discovered that POE also shortened the dispensing-to-administration phase of the medication cycle, from 3 hr 16 min pre-POE to 1 hr 22 min post-POE (Figure 1right ).

In retrospect, we think that POE provided the tools for timely and accurate patient care. The nursing work lists, nursing order notification displays, and the eMAR served to provide a heightened awareness among the nursing staff of the need for timely medication administration. The use of POE also increased nursing expectations for drug delivery, as the orders were immediately visible to nurses.

Academic medical centers have a wide diversity of caregivers—such as registered nurses, medical students, physician assistants, and other licensed clinicians—who have varied and limited order-writing authority. Countersignature of such orders is a significant issue, particularly from the perspective of regulatory requirements, such as those of JCAHO. The National Coordinating Council for Medication Error Reporting and Prevention (NCC MERP) reports that as many as 25 percent of reported errors arise from confusion over the similarity of drug names. This can be especially true with verbal orders. The NCC MERP has a number of initiatives and recommendations to reduce medication errors associated with verbal orders. Two of their recommendations involving verbal orders are that verbal orders should be written and signed by the individual receiving the order and should also identify the prescribing physician and that verbal orders should be documented with the patient’s medical record and should be reviewed and countersigned by the prescriber as soon as possible.¹⁷

Physician order entry records the name of the ordering physician and the user taking the verbal order electronically, fulfilling the first requirement. It promotes heightened awareness by providing built-in reminders to the physician when orders remain to be countersigned and also prevents the physicians from writing a discharge order if there are orders that remain to be countersigned. Statistically significant improvements in the physician countersignature rates of verbal and telephone orders at both OSUH and the James were noted following implementation of POE. A higher degree of compliance was seen at the James both pre-POE and post-POE, because of a large number of chemotherapy orders. The mandatory countersignature of chemotherapy orders is of particular importance because of the greater risks involved in the administration of cytotoxic agents. This has created a culture of compliance at the James, which explains the different improvement rates between the OSUH (43.1 percent) and the James (26.15 percent).

Previous studies have reported that up to 30 percent of all hospitalized patients experienced delays in their care, with the average length of the delay being 2.9 days. This represents 17 percent of all hospital days. The delays are attributable to delays in decision making by physicians while they wait for results, in the scheduling of diagnostic tests, and in discharge planning.¹⁸ In addition, delays in therapeutic or prophylactic administration of antibiotics have a major effect on clinical outcomes. For example, the timing of prophylactic antibiotics for abdominal or cardiothoracic surgery within a narrow window has a significant effect on the incidence of postoperative infection, length of stay, and cost.^19,20 Patient outcomes can be significantly enhanced by greater timeliness of care if physicians have access to critical laboratory results, showing that information technologies that facilitate the transmission of important patient data can potentially improve the quality of care.²¹

Following the implementation of POE, we demonstrated reductions in the time between the order and administration of the medication, the time for completion of radiology procedures, and the time for reporting the results of laboratory tests. Despite the improvement we were unable to demonstrate a consistent effect on length of stay or cost.

Because of a number of limitations, the benefits outlined above may not be generalizable to other institutions. Variations in POE applications may contribute to difficulty in data comparison across different institutions. Because of the nature of our studies, pre-POE data collection of all required data points was resource intensive with respect to time and personnel. This resulted in smaller sample sizes, especially in the radiology turn-around time study. Pharmacy and radiology turn-around time studies were done on one nursing unit and would need to be repeated in other areas.

We recognize that length of stay and cost are affected by a number of variables, including patient acuity, seasonal variations, and clinical environment. Case mix index adjustment of both length of stay and total cost should correct for patient acuity variations, and the length of the study comparison (10 to 12 mo) should minimize seasonal variations.

The initial positive effects observed immediately following the implementation of POE can be viewed as the first-stage benefits. Our objective was to move toward the Institute of Medicine goals for patient care quality and safety. The data indeed demonstrate that the introduction of POE with eMAR improved health care in a manner that was safe and timely.

The broad approach to this study was favored over an in-depth analysis, to allow the institution to validate the promise of POE. The sharing of this information not only facilitated our continued deployment but also resulted in increased acceptance by our clinical users. These methods are repeatable and could serve as benchmarks for any institution embarking on this initiative.

It is important to note that physician order entry provides a historical record for ease of analysis and long-term observation of trends. Although we used the data to assess the process of care in this study, it is equally applicable to disease management.

The immediate benefits of POE are abundantly clear. We are at the beginning of appreciating the long-term benefits of POE with the power and tools available to us to facilitate clinical practice changes. We see POE as an integral application for us to facilitate continuous quality improvement by decision support mechanisms and to monitor effects and compliance.

At OSUHS, we have shown that POE and eMAR have enhanced patient care by improving turn around time, reducing transcription errors, and improving verbal order countersignature by physicians. This study supports the Institute of Medicine recommendations for use of POE and eMAR to enhance patient care quality and safety.

The authors appreciate the valuable assistance with data collection provided by the Departments of Pharmacy, Radiology, Laboratory, Medical Information Management, and Quality and Operations and the School of Allied Health. They thank Siemens Medical Solutions for their ongoing support. They also thank Steven Suing for assistance with the post-POE reports and Jean Johnson and Liz Marshall for editorial assistance prior to submission of the manuscript.