Background
New South Wales is the most populous state in Australia, with over 7
million people in an area larger than Texas. Sydney, the state capital, has
over 3 million people. Health services are dominated by the state-administered
public health service, providing integrated hospital and community services to
the population. It is a large operation, with a budget of approximately $5
billion (US) and around 100,000 employees. The cities boast a number of large,
highly specialized hospitals, and there are a range of units down to quite
small rural institutions. Medical staff are mostly contracted, although there
are a substantial number of salaried staff specialists in the major hospitals.
Services are normally provided free to the population. A smaller private
health system also exists alongside the public system, supported by an
insurance industry. General practitioners are principally supported by federal
funding.
The health system has a geographically divisionalized structure. Its chief
executive is the Director-General who reports directly to the NSW Minister for
Health. A central Department of Health reports to the Director-General. It has
responsibility for corporate issues, including state wide funding and policy.
Provision of hospital-based care is the province of the Area Health Services
(AHSs) and rural Districts (for simplicity we refer to them all as Areas in
this study). Area chief executives and their boards report to the
Director-General but have a significant level of independence in respect to
the organization and provision of health care services in their geographic
area. Areas typically include one large teaching hospital and several smaller
community care units. The hospitals have the form of a professional
bureaucracy in that clinical staff, especially medical staff, act with a
considerable level of autonomy.
Management of IT within the NSW health system conforms to the federal
structure (Fig. 3). The
department has a central information management group, which establishes
technical and information standards and manages the state IT strategy. In the
past it has funded a considerable amount of hardware and has written a number
of core applications which have been implemented in most Areas. Areas also
have IT staff who maintain local operations, support software, develop smaller
applications and provide policy support for their own administrations. Some of
the hospitals also had their own centres of expertise.
| Figure 3 Federal structure applied to IT at NSW Health. |
With this structure, the NSW health system has been able to take advantage
of economies of scale in providing core systems. At the start of the period
covered by this case study (1989-1995), many basic administrative and clinical
IT services had been developed over the previous decade, and many institutions
relied heavily on them. However, there was considerable dissatisfaction with
the capabilities of the existing systems. They were not very flexible and were
not able to provide the management information required or the sophistication
of the clinical services that were being demanded. A number of the larger
Areas had already built supplementary applications to meet their own special
needs.
The New IT Strategy
In 1989 a new strategy was embarked upon; its objective was better resource
management. There was pressure to contain costs, and because the funding came
from the public purse, the political imperative was to retain existing
services and develop new ones where possible. A new minister of health felt
that better information was needed to enable the proper management of the
health system. It was also believed that the technology would streamline
operations and help clinicians provide better services. At the same time,
there was an increasing awareness that the federal government might impose new
reporting requirements, using case-mix. This reinforced the need for a new IT
strategy.
A group of IT strategists based in one of the hospitals had for some time
been arguing for the purchase of a “state-of-the-art” system. When
a major consulting company conducted an IT strategy formulation exercise, it
recommended the purchase of a new suite of systems, starting with financial,
pathology, and clinical applications. A “best of breed” policy was
adopted to select the best system available in each of these areas, with an
integration strategy to link these systems. This strategy was argued in terms
of economy, in relation to both the costs of development and the economies of
scale in a state-wide implementation. The advice was adopted, and a sum of
over $110 million (US) was allocated, with further internal funds added at a
later stage. The intention was to support both more effective clinical care
and better management information. The “best of breed” policy made
effective integration a crucial element in gaining the benefits. The program
was overseen by an Information Services Steering Committee consisting of
policymakers, CEOs of involved Areas and IT specialists.
Accountability was to be the hallmark of the selection. A request for
tenders was issued internationally, and a search team toured to find what was
available. Applicants were screened to form a short list. With the final short
listed offerings, a set of scenarios was established, and the systems were
trialed with set-piece scripts. Vendors were given only a limited time to
demonstrate the capabilities of their systems. Great care was taken to ensure
that each system was trialed under the same conditions and judged by the same
criteria. The short time-frame for the project excluded any substantial
consultation, but the selection team of central policymakers and site IT
specialists included doctors and managers.
The selection program was considered rigorous, and the systems finally
chosen were generally agreed to be the best available systems that met the
criteria. The PAS/clinical system in particular had been implemented in
approximately 100 sites in the United States and in several in Europe. Its
principal functions were patient administration, records management (but not
electronic medical record), order communication (order entry and results
reporting), and clinic scheduling. It also had a general report generation
facility.
Over the next year, contracts were drawn up and signed. During that time,
the Area administrations had been invited to bid for the opportunity for
participating in pilot site trials of the system. Funding was to be shared
between the central department and the sites: one third from the central
department, and two thirds from the Area, half of which was to be a loan. The
incentive for participating as a pilot site was that other Areas would have to
pay a considerably greater portion of the funding. However, despite this
support from the Department, the Areas were reluctant to make the commitment
required. They were under continuing financial constraint, and management had
little basis to assess the cost-benefit of the system. The Department had to
increase its contribution to 80% before the Areas would agree to participate
in pilot studies.
Implementing the PAS/Clinicals System
The PAS/clinicals system was to be tested in three different types of site:
a major metropolitan hospital, a major rural hospital, and a group of
hospitals. The last was to demonstrate that the system could integrate a
number of institutions within an Area. As the program proceeded there were
concerns about the integration strategy. None of the three pilot sites was
also a pilot for both the pathology and financial systems. To demonstrate the
viability of this integration they engaged two further pilot sites in which
the integration strategy could be tested. These sites were two major
metropolitan referral hospitals. There were now five pilot sites.
The vendor worked directly with the pilot sites to implement the system and
had the responsibility of achieving successful implementation given a certain
support infrastructure from the sites. The implementation commenced with a
“localization” process to adapt the system to Australian, and
specifically NSW, conditions. Both the vendor and the Department wanted to
minimize these changes to enable a rapid implementation, and they intended
that the sites would adapt their operations as much as possible to fit in with
the system. The vendor, in addition, wanted to retain the integrity of their
common international system. However, the sites took a very different
approach. They had not been party to the development of the strategy and had
had little involvement in the selection of the system. Further, they had
little besides sales presentations on which to base their expectations. Their
priority was to ensure that the system would serve them effectively. When they
saw the system, they found a considerable difference between what the system
provided and what they considered they needed. The sites argued for more
changes than the central IT staff and the vendor wanted. The result was a
continuing conflict between the centre and Areas, the resolution of which
required compromises on both sides. The sites were also disturbed by the cost
of many of the changes. Making the bulk of these changes delayed the project
by a year. Because of the compromises that had to be made in the changes
requested, there were still substantial deficiencies that had to be worked
around.
The concepts of benefits varied considerably. The original strategy was
based on a very general benefits definition. The sites were expected to
justify their pilot proposal with a more detailed specification of benefits.
This was done by each site independently, as their proposals were competitive.
These benefits were defined both in general terms and by many specific
details. However, under the pressure of implementation, little attention was
paid to these original benefit definitions.
Implementation was undertaken in a number of stages. The first was the
patient administration system (PAS) which handled the registration, admission,
and transfer of patients, as well as medical records and sundry other tasks.
Other stages were the order communication system (OC) handling clinical order
entry (OE) and results reporting (RR), and the clinic scheduling (CS) systems.
The PAS system was the basic module on which the others depended, but it was
not expected to provide functionality that was not already substantially
provided by the existing system, apart from functionality in areas such as
outpatients. It was the order communication and the clinical scheduling
systems that were expected to provide the main benefits through improved
management information and clinical support.
The implementation of the PAS generally went off successfully, with the
rural hospital going first, and one of the hospital group, a medium-sized
metropolitan hospital, going several months later. There were teams of people
allocated to the programs, and it was undertaken with considerable attention
to training and the preparation of manuals. Some members of the teams were
very stretched, but there was a high level of dedication that pulled them
through. The people involved were mainly full-time staff of the hospital, and
they felt themselves committed to making it work. There were some significant
benefits with the improvement of admissions processes and greater involvement
by the nurses in managing patient movements. There were, however, significant
problems. Training requirements were substantial, and there was difficulty in
getting time off for nurses to undertake training. Not all nurses were able to
adapt to the use of the computer, which created an extra load for the other
staff. There was a continuing need to train new staff and update staff for
system changes. The report generator was not suitable for managers to use, and
they had to wait on programmers to write the reports that they wanted. There
were also problems reported in the consistency of the data concerning bed
allocations and active patient lists. Nevertheless, the PAS system was
installed in four of the five hospitals and was considered to be operating
successfully.
The order communication system turned out to be a very much greater
challenge. There were two major reasons for this:
- Order entry required a substantial involvement of medical staff.
- The principal gain for clinicians lay in the reporting of pathology
results. This required an effective integration with the pathology system.
In the existing procedures, medical staff had to sign all orders on paper
forms. The forms were then conveyed to the relevant department for action. The
forms had the patient name on a sticker, and the clinician's task was merely
to name the test, note any clinically relevant details, and sign it. This
could be easily done at the patient's bedside during rounds. There were,
however, problems in delays and in the legibility of the orders.
The new system involved the clinician entering the test directly into the
computer using an electronic signature. The clinician, then, had to go to a
terminal and enter the test by using the OC system. The system was considered
rather user unfriendly by the clinicians. They had to pass through four levels
of log-in, some with rather obscure passwords. The screen navigation system
was complex, with nonintuitive commands and a variety of confusing shortcuts.
There were up to 11 screens and up to 43 key strokes required to order 1 test.
Despite promises, the link with pathology had not been achieved in most
hospitals, so the bulk of results were not getting through to the sources
immediately. There was no connection between order entry and results
reporting, so the clinician could not assess the past results while ordering
tests. Further, the character-based screens seemed antiquated in comparison
with Windows-based systems which some clinicians were familiar with.
A task that previously took about 20 seconds was now taking several minutes
and was causing considerable frustration. It took a lot of time for people who
used it frequently, and it was difficult to master for those who used it
infrequently. There did not appear to be any significant gain for medical
staff, and any gain that could be argued was mainly in enabling management to
monitor clinical activity. Given the continuing financial constraints, medical
staff members were not sympathetic to the expenditure of funds on these
systems nor of the burden of learning and operation it placed on them.
To make matters worse, some hospitals already had reasonably effective
results reporting systems, so clinicians were actually losing
functionality.
There were three types of medical staff involved:
- Staff specialists. These were the only doctors who were actually employees
of the hospital.
- Visiting Medical Officers (VMOs) were specialists who were contracted by
the hospital to treat patients. Many ordered tests only on an occasional basis
when assistance was not available.
- Resident Medical Officers (RMOs) were trainee doctors who rotated amongst
the hospitals. These staff carried out most of the ordering and were generally
too hard pressed to involve themselves in planning.
Most medical staff were very busy and had limited time to undertake
training. Some had high expectations of information technology, and those who
were computer literate were familiar with the conveniences of a Windows-based
interface. It was not too surprising, therefore, that there were strong
complaints by medical staff, leading to collective protest in several
cases.
Other modules were having difficulty. In particular the clinic scheduling
system had been declared inadequate for major hospitals, and an alternative
was being sought. There were operational problems as well. Technical staff
found the system very cumbersome. It required 24-hour, 7-days-a-week support
and it had to be taken down about 1 hour each day. They found it difficult to
get adequate documentation on the system, or to get reasonable explanations
for the problems that they were experiencing. They also found that data
storage was complex, confusing, and redundant. Overall, they found themselves
very dependent on the technical support from the suppliers in the United
States.
Despite all these problems, there were significant successes. Some
departments, such as radiology and allied health had much improved order
delivery, and they much appreciated the legibility and the timeliness of the
orders. Orders could be placed at any computer in the hospital and small
clinics were also able to use the clinic scheduling system effectively. Some
innovative centres were able to use the management reporting system to extract
useful analysis. By adapting the allocation of some of the fields, useful
information on clinical services was also available.
Of the four sites that implemented PAS, the first of these, the rural
hospital, implemented the OC for a period of 15 months and the second, a
smaller urban hospital, implemented it for 6 weeks. The clinic scheduling
system was implemented only in the rural hospital. There were other partial
implementations as well. However it was only the rural hospital which had
implemented all the principal systems and realized most of those benefits that
were achieved. This site was considered by many to be a success.
As the program proceeded, the level of dissension gradually increased. In
particular, medical staff were becoming more resistant to having to use the
system. Managers saw continuing maintenance and support costs and little
evidence of cost savings or the resolution of the problems. Finally, with a
change in state government, it was decided to withdraw the system. Although
the rural hospital would have liked to continue, a single implementation was
not commercially viable. Fortunately, the previous system was still available
and had been upgraded to provide most of the critical functionality, so the
hospitals were able to adopt this system with relative ease.
The losses were substantial and took several forms. There was the financial
cost, the delay in the strategy with opportunity costs, and the considerable
distrust generated in central IT initiatives. There were significant assets
retained, however, in communication infrastructure, hardware installed, and
computer familiarity among the staff. There was also much learned about the
complexities of large-scale implementation, particularly in terms of
organizational issues, and about the organizations themselves.