pmc logo imageJournal ListSearchpmc logo image
Logo of annsurgJournal URL: redirect3.cgi?&&auth=08cF-0_1JCD-W4HdoVThNkKK6PWfPUp_12TLACA98&reftype=publisher&artid=1448959&article-id=1448959&iid=130653&issue-id=130653&jid=230&journal-id=230&FROM=Article|Banner&TO=Publisher|Other|N%2FA&rendering-type=normal&&http://www.annalsofsurgery.com
Ann Surg. 2006 April; 243(4): 439–448.
doi: 10.1097/01.sla.0000205820.57261.76.
PMCID: PMC1448959
Quality and Safety in Surgical Care
Hiram C. Polk, Jr, MD, John Birkmeyer, MD, David R. Hunt, MD, R Scott Jones, MD, Anthony D. Whittemore, MD, and Bruce Barraclough, FRACS
From the Department of Surgery, University of Louisville, Louisville, KY.
 
DR. HIRAM C. POLK, JR. (Moderator): The subject of the panel this morning is obviously a paraphrase and derivative of what I talked to you about yesterday.1 If you listened to any part of the scientific program, you now are aware that there are a dozen or more people in this Association who are directly and personally involved in this quality issue. They are doing all that they can as individuals and as group leaders to make quality and safety center-stage for us.
This morning we have 2 special people who are going to lead our discussion: Dr. John Birkmeyer and Dr. David R. Hunt. More importantly, the body of their discussions will be followed by leaders from within this Association, with some brief discussion by Drs. R. Scott Jones, Anthony D. Whittemore, and Bruce Barraclough, one of our new honorary Fellows.
Each one of these 5 panelists brings something to this lectern that represents personal skill in approaching this discussion in a positive way. The first speaker is Dr. John Birkmeyer, who is Professor of Surgery and Health Policy. He has had a major role in not only leadership at the University of Michigan in Ann Arbor, but also has taken on an interface with the third parties or the payers to begin to see that patients are referred to doctors and hospitals where they are likely to be well cared for, and perhaps with some financial quid pro quo included. Dr. Birkmeyer, we look forward to your comments.
DR. JOHN BIRKMEYER (Panelist): Thank you, Dr. Polk. It is a real honor to be here as well as to witness the evolution of surgical meetings. The separation between church and state has vanished. I find it is interesting to see issues of payment, policy, and politics creep into scientific sessions.
It is my pleasure to share with such an esteemed group what I call the “Michigan plan.” Let me first start with a couple of basic observations. First, purchasers and payers are taking an increasing interest in the quality of surgical care. It is widely recognized that surgical outcomes vary widely across both hospitals and surgeons. Payers also know that surgical complications have a direct effect on the bottom lines. They have a direct effect on the productivity and well-being of their employees. Also, it is clear that surgical complications are very costly, a minimum of $10,000, by one recent estimate.2 One way or the other, these costs are ultimately passed on to payers, and ultimately patients. For these reasons, the growing interest in linking of payment to performance (in surgery as well as other areas of health care) should not be a surprise to anyone here.
Let me give you a couple examples of ongoing projects that illustrate payers' interests in this area. I am sure that you will hear more from Dr. Hunt about some of the efforts of the Center for Medicare and Medicaid Services (CMS). For example, the CMS centers of excellence project in cardiac surgery is paying a reimbursement premium to hospitals scoring in the top 10 percentile on composite measure of performance. At many of your hospitals, you have already seen evidence of the partnership between payers and the Surgical Infection Program, manifested by reimbursement premiums for hospitals achieving certain benchmarks regarding appropriate use of perioperative antibiotics. Of course, most of you are familiar with the activities of the Leapfrog Group, a large bloc of both public and private purchasers interested in leveraging better outcomes with financial and other incentives.3 Quality improvement per se is not their primary goal. Instead, they are trying to steer selected surgical patients to hospitals likely to have the best outcomes, as defined by volume, selected processes of care, and direct outcome measures.
Unfortunately, there are at least 3 reasons to be worried about the ultimate effectiveness of these efforts. First, as they are currently formulated, incentives for surgeons to change their practice are fairly small. At the present time, reimbursement premiums are low, at least in proportion to costs of surgical care, and go to hospitals, not individual surgeons. It is increasingly clear that even the best organized efforts, like those of the Leapfrog Group, are unlikely to affect major changes in the distribution of surgical patients. Simply put, private payers lack the leverage in most markets to implement measures with “teeth.” Patients themselves have shown very little inclination to shop for surgical quality. As a result, current pay-for-performance plans are severely lacking in both carrots and sticks.
A second reason is weak measures and weak data. Although process-of-care measures are the current fashion, most are related to ancillary care and secondary outcomes. For example, there is no doubt that we can do better with administering perioperative antibiotics, but I do not think anybody here believes that antibiotics are a major player in explaining variation in surgical outcomes across hospitals and surgeons. In addition, it is very difficult to identify who is the “best,” a prerequisite for policies aimed at steering patients to higher-quality settings. These difficulties are attributable, in part, to fundamental flaws in the specific performance measures. They also reflect a lack of resource investment in clinical data infrastructures for tracking surgical outcomes with sufficient rigor.
Third, current pay-for-performance plans are creating an unhealthy competition among surgeons and hospitals. One way or another, these plans create winners and losers, without giving losers the tools they need to improve. These plans discourage the type of collaboration necessary for us to understand what works and what does not work, and ultimately improves care everywhere.
I introduce to you the “Michigan plan,” a partnership between Blue Cross and Blue Shield of Michigan (BCBSM), outcomes researchers and surgeons at the University of Michigan, and surgical leaders from hospitals all over the state of Michigan. The crux of the plan is improvement-oriented, payer-funded, clinical outcomes registries. Hospitals are paid (in the form of DRG supplement) for simply gathering their data and participating in quality improvement efforts. They are not paid according to how well they do or do not do.
BCBSM is also underwriting the cost of centralized functions, including data management, analytic functions, and improvement activities in the field. It is worth noting that funding for these programs is coming from its clinical operating budget. This is not a grant or one-time contract.
Effective this year, we are launching 3 clinical outcomes registries, in cardiac surgery, bariatric surgery, and other areas of general and vascular surgery. I would like to give credit to some of the main players, including Dr. David Share, who is spearheading this BCBSM, and surgeon/researchers from the University of Michigan, including Drs. Skip Campbell, Richard Prager, and Nancy Birkmeyer.
For cardiac surgery, the national Society of Thoracic Surgery (STS) registry will serve as the data collection and reporting platform. We have enrolled each of the 28 hospitals performing cardiac surgery in the state of Michigan. We expect to enroll about 10,000 patients a year, at a cost of just over $1 million annually.
For bariatric surgery, there currently are no good off-the-shelf data systems, so we are developing our own outcomes registry, borrowing as much as possible from parallel efforts by the American College of Surgeons (ACS) and the American Society of Bariatric Surgery. We will collect information about outcomes at both 30 days and at 2 years. We have enrolled almost all of the 20 hospitals in the state currently doing at least 40 cases a year. About 5000 patients will be enrolled each year, at cost just under $1 million annually.
And finally, for other areas of general and vascular surgery, we will be building off the ACS National Surgical Quality Improvement (NSQIP) program.4 This program will involve 15 of the largest hospitals in the state and between 20,000 and 30,000 patients annually. The cost is approximately $3 million a year, or, for those of you doing the math, about $100 a patient.
Although we are relying as much as possible on off-the-shelf databases, we hope that additional funding from Blue Cross and Blue Shield will allow for several enhancements to these registries. For STS registry, we will add an external auditing function, essential for ensuring the accuracy and completeness of the data. For our own bariatric surgery registry, we will also conduct external auditing. We plan to gather just those data that are crucial for risk adjustment and for assessing outcomes specific to bariatric surgery, but nothing more. Our ultimate goal is a database that does not hinge on the availability of dedicated personnel for data collection. Thus, our aim is to build most aspects of measurement into daily clinical practice. And finally, for the NSQIP program, we plan to target specific operations for quality improvement. For those operations, we will gather both outcomes data on all patients, not just a random sample. We also hope to add “flex fields” for capturing specific processes of care.
As a profession, we often “drop the ball,” in that we get so wrapped up in measurements that we overlook the process by which data are translated into meaningful quality improvement in the field. As we launch these programs, we are working by a number of basic principles. The first is that quality improvement, like politics, is local. As I have mentioned, we plan to build off national systems, but it is essential that some analytic functions and coordination of quality improvement activities occur locally. Second, simply measuring your outcomes is insufficient. It is essential to identify high leverage processes of care specific to individual operations and ensure that they are broadly implemented. This is a cornerstone of these projects. And third, quality improvement cannot just be an afterthought. It requires an explicit plan, dedicated funding, and active involvement of surgical leaders at each hospital. In this last regard, periodic conference calls are not adequate. Hospitals will only get paid if their representatives show up in person to review their data and coordinate quality improvement activities.
The climate in Michigan is very favorable for this type of work. I can say this because I just moved from Dartmouth, the only place in the country colder than Michigan. With regards to surgical quality improvement, Michigan is dominated by big purchasers, particularly from the auto industry, with longstanding interests in value-based purchasing. Second, we have the luxury of a single dominant private payer, BCBSM. Many would argue that this is a curse, but, in this context, there is no doubt that BCBSM has the leverage to make this happen. They can also invest in quality improvement knowing that benefits will accrue primarily to their patients. Third, we are working off of a successful template from interventional cardiology, also organized at the University of Michigan and underwritten by BCBSM. This improvement-oriented outcomes registry has made demonstrable improvements in outcomes for Michigan patients undergoing percutaneous coronary interventions.5 And finally, we have a critical mass of outcomes researchers and surgeon leaders with the interest and expertise to get this off the ground.
We will have to meet a number of challenges to launch these programs. A primary challenge will be establishing trust: trust between Michigan providers who are more used to fighting over market share than collaborating; trust between the University of Michigan and other state hospitals; and trust between surgeons and BCBSM, whose motives are often questioned. A second challenge is that we need to show a return on investment. BCBSM will be investing about $5 million annually in these 3 surgical programs alone. When you count their parallel efforts in breast cancer and interventional cardiology, they are putting between $8 million to $10 million into this project. Their purchasers want to see a return on this investment, in the form of better outcomes and lower costs for treating of surgical complications. Finally, we will have to fend off pressures from both patients and purchasers who want to see the data. Of course, public reporting is a controversial area. I think that both the harms and the benefits of public reporting are probably overstated. Nonetheless, demand for public accountability is not going to fade away. The best way for us to defuse it will be to demonstrate significant reductions in statewide rates of morbidity and mortality and, more importantly, reductions in current levels of variation of performance across providers.
In summary, we would all agree about the general failure to “incentivize” surgical quality care in the United States. Payers, both public and private, have obvious interests in motivating quality improvement in surgery by linking payment to performance. However, we are hoping that a more collaborative model, the Michigan “pay-for-participation” plan that I have just described, will ultimately have better bang for the buck. Thank you for your time and attention.
DR. HIRAM C. POLK, JR: Our next speaker brings a different dimension to all of this. David Hunt is a protégé of LaSalle Leffall, a board-certified Fellow of the College in full-time government service. In fact, David is one of the highest-ranking surgeons in government service in the United States. He also is a full-time practitioner of surgery. He covers his wife's suburban practice on nights and weekends, so he is the throwback to the old-fashioned surgeon who works day and night. David has been an increasingly important implementer of policy activities at CMS over the last few years, in which he, perhaps more than anyone, has tried in a hands-on way to push these things forward, and to recognize our shortcomings, while recognizing our opportunities when the time is right politically to push these things forward with the kind of support that we know this will need in the long run.
DR. DAVID R. HUNT (Panelist): Dr. Polk, I am honored to be here because I appreciate what the American Surgical Association is and stands for. The ASA is the history of modern surgery, from Halsted's first presentation in 1898,6 to Dr. Charles Drew's in 1940,7 and on to Dr. Birkmeyer's today. The ASA is the gallery to display the highest art form in the science of surgery. I speak for Dr. Mark McClellan, the Administrator at the Centers for Medicare and Medicaid services, when I say that we are humbled and honored to have the opportunity to present at this Forum.
Now, I am here to make a credible case for the Surgical Care Improvement Partnership (SCIP). I know the title of my presentation, “A Grammar of Surgical Quality from Physics to Syphax to Shakespeare,” is not intuitively obvious. I chose it for that very reason, because the essence of this work, surgical quality improvement, is not intuitively obvious. But my hope is that you will be comfortable enough with the fundamental underpinnings to discuss the operational details of a national surgical quality partnership.
So you can see that I am using the term “grammar” in one of its least used forms, namely, as a discourse of fundamental principles. But, before I go too far, I should step back and outline my take-home messages. I am not ashamed to recount the public speaking advice my daughter gave me when she said, “Dad, you have to bottom line any audience very early on. When you talk for more than 5 minutes, you become boring and tedious.” So, I am going to take Dana's advice, and I am going to “bottom line you.”
My first point is that surgical quality measurement and improvement must be led by surgeons. Surgery is a team sport, and surgical quality improvement is no different. Any progress lives or dies solely on the strength of our team. And lastly, I will not say that all we can promise you is blood, sweat, toil, and tears, but the path forward requires a system, tremendous resources, and courage.
What does this grammar of quality look like? I mentioned before that this work is not intuitively obvious, but that is okay. This is hardly a unique situation in the world of science. In the scientific disciplines, we have a very reasonable formula for moving forward in a complex field resistant to simple intuition and analysis. The formula entails an iterative process of pushing the boundaries of new knowledge against known truths. In science, we call our fundamental known truths laws: concepts that defy formal proof yet set standards with unvarying uniformity. We test our progress against the relevant laws. When there is concordance, we place that new knowledge block on top of the others. Discordance means we must either reject the new or knock down the foundation.
Halsted built upon Hunter who built upon Paré. Aristotle's ether was swept aside by Newton's mechanics, which was replaced by Einstein's relativity.
At times, fundamentals provide for some uncomfortable boundaries. Case in point: The 3 laws of thermodynamics (Table 1). Now, these laws which govern all energy and matter, in a very real way, create boundaries for everything we do. As proof, I offer C.P. Snow's interpretation of these laws.8 His Darwinian understanding of these rules provides a wonderful metaphor for our work in quality.
Table thumbnail
TABLE 1. Laws of Thermodynamics
The first is we cannot beat the system. That is, we cannot get something for nothing because matter and energy are always conserved. Everything we take comes at a cost. Second, we cannot break even. Just returning to the same energy state from which we started comes at a cost. Entropy always increases. Just holding on to what you have will cost you. Third, we cannot get out of the game. But this last law is not as bleak as it sounds. It does offer some hope because while perfection, like absolute zero, is unattainable, it is not unapproachable. The key is that any approach toward perfection requires a system of increasing order and organization.
Yes, this grammar can be harsh, but in some cases, the basic principles can embody an elegance that is awe-inspiring. As proof, I offer the work of James Clerk Maxwell.9 As improbable as it sounds, these 4 equations clearly state everything we know, and possibly everything we can ever know, about electromagnetic phenomenon. Can we have any hope of finding the equivalent of Maxwell's equations in healthcare quality? Yes. I learned the principle as a resident and did not realize the overarching value of what I had been taught until I reached CMS.
Now, as Dr. Polk mentioned, I trained at Howard under Dr. LaSalle Leffall. Howard is the home of Charles Drew, LaSalle Leffall, and that singularly brilliant bridge between those 2 men, Dr. Burke Syphax. With a clear, coherent, eloquence I have never seen before or since, Burke Syphax is likely the closest I will ever come to meeting a mind the equivalent of James Clerk Maxwell. For over 60 years, this professor emeritus has clearly articulated the fundamental precept of surgical quality. He made it very clear that the quality of our art rests in our character, and by extension, in our ability to honestly appreciate when we have helped, and when we have harmed.
He would also say that in the operating room, whenever you have lost your way, go back to what you know is always true, and move from there. Follow the taenia and you will find the appendix. To quickly identify proximal and distal loop of bowel, let your hand go to the root of the mesentery. Proximal is north; distal is south. In the operating room, a proper history can guide you better than a CAT scan.
So understanding when you have helped and when you have harmed. Is that it? Well, Dr. Syphax's most famous student, LaSalle Leffall, put it very simply. He would say that every day, every surgeon must ask themselves 2 questions. What did I do that I shouldn't have? And what didn't I do that I should have? Did I help or did I harm?
Is that what we can use as surgeons to find our way back? Make no mistake, we must find our way back. We are in an awful void. It is more than a void; it is a black hole, which is threatening to take us out of the 21st century and put us right back in an era before antibiotics, back to a time when many Americans could not expect to get adequate health care, back to the day when you were just as likely to be properly cared for as not. This is not hyperbole. CMS is banking that we as surgeons know best how to find our way out of this void.
Many of you see that Dr. Syphax is, as all of us should be, a well-trained student of Hippocrates, who in “Epidemics I” wrote: “As to diseases make a habit of 2 things, to help or at least to do know harm.” Even though some of you may not understand all the integrals of Maxwell's equations, I am sure everyone in this room can appreciate the power of this law as well as Dr. Syphax. This firmly establishes upper and lower bounds, to help at the high end and to do no harm at the lower. And when Hippocrates says make a habit, today we would say, make a system. Wonderfully, this rule provides an answer to a question we hear a lot at CMS. What is the relationship of patient safety and quality (Fig. 1) ?
figure 3FF1
FIGURE 1. Relationship of patient safety and quality.
Well here is how we read it. This is the model we use at CMS, a model validated in the 2003 IOM Report on Patient Safety Data Standards when Paul Tang and his team wrote: “Patient safety is indistinguishable from the delivery of quality healthcare.”10
In this model, all elements of safety are within the domain of quality, but there are aspects of quality outside of the domain of safety. This model helped frame our early thinking in the SCIP.
Now that I have discussed our theoretical framework, let us get into some practical applications. The goal of SCIP is to reduce the morbidity and mortality of major surgical procedures in the United States 25% by 2010.
Surgical quality improvement lives or dies on the strength of its team, and as you can see, we have assembled a strong starting lineup with the American College of Surgeons, the Society of Anesthesiologists, and a host of others.
Now, I know that for many of you, CMS appears to be the odd duck in a list of organizations working to improve the quality of surgical care. The history and evolution of the CMS quality improvement program are somewhat irregular, but today we have found our voice and have identified 4 strategies to make our vision a reality. They are: 1) performance measurement and reporting, 2) systems adoption and use, 3) process redesign, and 4) organizational culture change.
The modern era of quality measurement and reporting as a formal discipline has its origins in the work of Walter Shewhart,11 William Deming,12 and Joseph Juran.13 These men are brilliant luminaries, and I don't want my comments ever to suggest anything but the utmost respect for their work. I do take exception with the current trend to frame all healthcare quality improvement in the context of the industrial experience. For all of its merits, industry-based measurement and quality improvement can only apply the concern of an actuarial ledger. The dispassionate cost/benefit analysis of typical industry quality improvement efforts can be obscene when applied to individual patients.
So, while there are many lessons to be learned from Deming, surgical quality measurement and improvement must be led by informed surgeons because the nature of our relationships with our patients changes the dynamic for possible intervention and improvement. And while we lead, we can do so knowing that before Deming,12 before Shewhart,11 before Juran,13 surgeons were leading quality improvement on exactly the same scale as our needs today. The names of Lister, Codman, and Halsted are not revered just because they knew how to tie silk. They are legends because they met the challenges of quality in their day.
In his most famous work, delivered to the graduating class of Yale Medical School in 1904, William Stewart Halsted14 asked: “How can American surgeons achieve the quality seen in the clinics of Germany and Vienna?” Halsted concluded that the etiology of the poor outcomes seen in the surgical wards of the United States was fundamental to the systems of healthcare delivery. Halsted went on to revolutionize surgical training in the United States, and his system is largely credited with moving the epicenter of surgery from Europe to the United States.
In 1904, a void in the quality of care in our surgical wards called for a solution by surgeons; and in 2005, an identical call to meet a remarkably similar need is being made. Only this time, the pressures are external. Consumers, purchasers, and insurance plans, public and private, are demanding measurement, reporting, and improvement throughout health care. Yes, my agency is actively, thoughtfully, and some may even say, feverishly, trying to construct a model of pay-for-performance. We have a lot of work to do. I can tell you that in the world of surgery, the due process of pay-for-performance, will be through SCIP.
To our credit, CMS recognizes that the American College of Surgeons is correct. We recognize that the best system and agent of change is already at work in the Veterans Administration. The VA has led the most comprehensive, innovative national surgical quality improvement effort in the past 50 years.15 It rests at the heart of the SCIP coalition. It is important to understand that this coalition represents a radical departure from business as usual for the 800-pound gorilla known as CMS. In SCIP, CMS acknowledges that surgical quality improvement is one of those things even an 800-pound gorilla cannot lift alone, and more importantly, we recognize that we should not even try.
With our steering committee, we are setting a framework that will be the vehicle for CMS to work synergistically with a broad range of surgical quality improvement efforts, such as the ACS NSQIP.
At the heart of any effort to change must be a system to measure progress. The approach to this subject really represented a defining moment for the SCIP partnership. You see, CMS traditionally works with process measures, such as the percentage of patients receiving antibiotics on time. One elegant feature of process measures is that they require no risk adjustment. Another is that the performance of an institution on process measures provides information on institutional intent. What processes have you put in place to assure the quality of your services? Now, I do believe Deming was correct when he said: “If you can't describe what you are doing as a process, you don't know what you are doing.” Make a habit!
But having said that, we must all acknowledge that surgical outcomes measures were over 3600 years old on the day that William Edwards Deming was born. Surgical quality outcomes measures were written in the code of Hammurabi. So, from Hammurabi to Codman16 to NSQIP, it would be folly to ignore the history and the depth of surgical outcomes measures.
So here was our early dilemma in SCIP. Process or outcomes? We know all too well that outcome is a derivative of process, but we also know that none of our patients consents to surgery just to have a good process. The special nature of surgery warranted a measurement system that balances process and outcomes.
So, now within this balanced approach, what postoperative complications will we address first? Looking again to the work of the VA, 4 broad clinical areas seemed appropriate for a concerted quality improvement focus: 1) surgical site infection, 2) perioperative myocardial infarction, 3) postoperative venous thromboembolic disease, and 4) postoperative respiratory complications.
A fifth domain, end-stage renal disease (ESRD) vascular access, was co-opted from a parallel quality improvement initiative at CMS. This project, known as “Fistula First,” has the goal of increasing the use of primary AV fistulas for our ESRD hemodialysis patients to 66%.17 Time precludes providing any detail, except to highlight 2 points about the “Fistula First” component. First, in this one area in the United States, we provide the worst quality in the industrialized world. Secondly, odds are favorable that improvement in this domain will be one of the first categories for surgical pay-for-performance at CMS.
But looking at our clinical areas, what are the opportunities for improvement? We do not have very good numbers regarding the question of ventilator-associated pneumonia, but in surgical site infection, myocardial infarction, venous thromboembolic, and vascular access, you can see that the overall compliance with recommended guidelines provides a significant opportunity to improve (Table 2). 18–21 Make a habit to help.
Table thumbnail
TABLE 2. Compliance With Recommended Guidelines
I misled you in my presentation title. The message from Cassius to Brutus that “Men at some time are masters of their fates”22 is my penultimate—I'll actually close with some words from A. Phillip Randolph.23 I hope you understand that the rhetorical alliterative of the title “Physics to Syphax to Shakespeare” was a little bit too compelling. But returning to my themes, and framing things within the context of Act 1 Scene 2: yes, Medicare doth bestride our world like a Colossus, but the agency will only dictate our fate if we as surgeons fail to act.
Surgeons have a tradition of leadership that is earned and desperately needed now, but that will not be easy. There are many competing priorities. Do we need to passionately advocate for economic reform, legal reform, and educational reform, to assure that surgeons in the mid-21st century represent the best and brightest of their generation? Absolutely. But, as important as that is, we all know that surgeons speak loudest and are heard best when we stand to protect our patients. I need you to stand. Join us. Commit to making a difference.
In his 2003 Presidential Address to the American College of Surgeons, Dr. Claude Organ convincingly let the Initiates know that it is imperative that they understand that they can make a difference.24 The College is committed to making a difference. One hundred one years after Halsted spoke at Yale, our patients need us to make that difference again. Surgeons do not wear the cloak of complacency well. It does not fit us. It is uncomfortable, unseemly. The current confluence of pressures in American health care demands our leadership and our unique sense of excellence. We can do that because we inherit the professional legacy of Lister, Bilroth, Cushing, and Codman. Any profession with that lineage is up to this task. In launching NSQIP, the College has shown foresight in moving to fill this void.
What is my role? My role is that of an informed facilitator. I am comfortable in this role because I have been fortunate enough to stand on the shoulders of giants. I was taught by Leffall, who was taught by Syphax, who was taught by Drew. So my reason to be where I am is to make sure that when surgical quality improvement is discussed at CMS, there is someone at the table who understands the difference between a colectomy and a colostomy.
Do I agree with all the decisions that have been made in our agency? No. I agree completely with our intent, our focus, and our commitment to work together. Make no mistake, it is only together that we can overcome the systemic inertia that is our real foe. It is so apropos that for, this year, Dr. Laws chose as a theme “Unity in Surgery.”25 Unity and organization in our profession really is the whole reason I am here and is my take home message today.
I will close by citing another reading of the 3 laws of thermodynamics. This time by A. Phillip Randolph23:
“At the banquet table of nature there are no reserved seats. You get what you can take, and you keep what you can hold. If you can't take anything, you won't get anything, and if you can't hold anything, you won't keep anything, and you can't get anything without organization.”23
Again, I am indebted Dr. Polk and the ASA. Thank you very much for this opportunity.
DR. HIRAM C. POLK, JR: You see how privileged we are to have such a person in the philosophical, technical, and political senses at the table. There are many people here who have comments, but I wanted to lead with our leaders. As most of you know, Scott Jones leads the College's program in this area. I have asked him to give us an update on that and to share us his thoughts in general.
DR. R. SCOTT JONES (Panelist): Dr. Polk, first, let me thank you for the privilege and the opportunity of being on this podium today. I stand here with a certain sense of humility, and you can understand why, having just listened to these 2 excellent and outstanding presentations. Both of these leaders keep us energized in this important endeavor.
I would like to say a few words about the things that the American College of Surgeons is doing now, and some new initiatives that we have, and to bring you up to date on a few things. I will do this in a fairly superficial way to keep the program moving.
First of all, I am going to repeat Dr. Polk's invitation for you to read this important book.26 I have read it a couple of times, and I have learned a great deal. I would go on to say, however, that we can learn a lot about design, about human behavior, forcing functions, culture change, and all of these things, but that is simply not going to get the job done. What we have to think about is how to get these matters into practice, day in and day out, and incorporate them into the fabric of our lives. That is why we should listen to Dr. Hunt in his point that it requires organization.
I will say a word or 2 about CMS programs. We have been looking at this very actively for the last 2 or 3 years under Dr. Hunt's leadership. The first thing I would like to tell you is about quality improvement organizations. I had to reeducate myself about that. I would just remind you that when Medicare was signed into law (1965), a portion of the statute includes a mandate for quality improvement. This was initially manifested by the professional review organizations, which later changed into professional service review organizations. In the early 1990s, this took a whole different change and a different philosophy when they morphed into quality improvement organizations.
There are actually 54 quality improvement organizations. These are 501(c)3 corporations that exist to carry out the quality improvement activities and mandates of Medicare. There is one in every state, plus one in D.C., Puerto Rico, and the Virgin Islands. The reason I emphasize this to you is that everyone should know about these organizations. You should make it your business to find out what they are doing and work with them in your hospital, as Dr. Polk has been doing for the last few years, to improve the quality of care. It is a resource there waiting for you to step up and work with them.
The Surgical Care Improvement Project that Dr. Hunt just mentioned uses the College as an initial partner to enter into this arrangement with CMS, to bring to bear its resources to work in partnership with CMS and the other organizations that are in this project. I will tell you that next week we will be having a face-to-face meeting of the Executive Committee of the Surgical Care Improvement Project in Chicago to review many of the things and projects and objectives that Dr. Hunt just mentioned.
I would like to say a word to you about the College. Very quickly, this is our master plan for the College getting into quality improvement, many of which have been in place for many years. We are going from the evidence to the high quality care, that's the process. There are the processes of care that we need to look at, and to monitor, and to quantify, and to standardize. We need to put standards of care into practice. We are looking at outcomes. So, we are not saying either process or outcomes. We are saying that we have to look at refinement of processes and the quantification of outcomes. We are working on that steadfastly today.
Another thing that is a very important driver of quality of surgical care is innovation and the introduction of new knowledge and new technology. We must step up to that in a way we have not done heretofore. We have to identify new knowledge. We have to make sure that it works before we put it into the standard of care, and we must avoid harm.
I have 1 or 2 words about the National Surgical Quality Improvement Program. There have been a couple of papers on this program about it. The College is indeed thankful and appreciative of the Agency for Health Care, Research, and Quality which funded a program, a demonstration project for 18 hospitals, 14 universities, and 4 community hospitals, to demonstrate whether this worked in the private sector.
Data are collected and submitted reliably by dedicated, trained nurses. This permits calculation of the observed versus expected ratios, giving risk-adjusted outcome assessment.
After the VA proved the effectiveness of this program, we put it into practice in private sector hospitals. We did a 3-year demonstration project with 14 university hospitals and 4 community hospitals. About 2 weeks ago, we analyzed the data from this project, and I report to you that we observed during this 3-year trial a significant reduction in surgical morbidity. We also observed a reduction in mortality, which was not yet statistically significant, but we demonstrated that this program works very effectively. The American College of Surgeons is taking responsibility to make this program available to any hospital that wants to enroll.
All of the 18 hospitals have continued and will continue to be partners, except one, and I think that one will come along. Since then, we have added additional hospitals that are currently actively engaged in this program collecting data for analysis. In addition to the hospitals, we have at least 60 hospitals in the queue that will be enrolled in the next few months.
I finish by saying a word or 2 about a renewal of an old initiative that the American College of Surgeons is taking on. For many years, we have supported and approved trauma centers and cancer centers. Now, these trauma centers and cancer centers have stipulated standards. They promote certain processes of care. If you want to find a fine example of how a clinical pathway works, I submit the ATLS course. By doing things in this way, they probably save more lives than most of the things that we can come up with. So, we are going to take those principles forward, in terms of having standards of care and outcomes, analysis, and the review process.
The Board of Regents of the American College of Surgeons recently instructed our division and the other staff of the College to proceed with developing these principles by forming other centers. We will look at hospital leadership, practice standards, the physical resources that are present in hospitals, the human resources, and there will be an obligatory outcome measurement that will be part of this new approvals process.
We have several other programs in the queue, but the one that is front and center at this moment is the development of American College of Surgeons Bariatric Surgery Centers. We have established a document outlining the requirements and the standards. There will be volume requirements for centers and volume requirements for individual surgeons. There will be obligatory outcome measurements, and we will be enrolling hospitals into bariatric surgery centers beginning in approximately 3 weeks.
We developed a partnership with the Society for Vascular Surgeons for Carotid Stenosis Centers. We have a proposal from the American Society of Breast Surgeons for breast centers.
All of these things are coming into place: processes, outcomes, and the implementation of innovation. We will continue to work on this for years to come. Thank you very much for the opportunity to speak.
DR. HIRAM C. POLK, JR: If you have seen a philosophical, national, and now an organizational intent, let me direct your attention now to some comments by Dr. Anthony Whittemore. Dr. Whittemore has salvaged the treasury of this organization over his tour and has presided over our newfound prosperity. What he also does as Chief Medical Officer in one of the most distinguished hospitals in our country has provided the opportunity to do some hands-on, internal, in-house decisions about quality that he will share with you instructively.
DR. ANTHONY D. WHITTEMORE (Panelist): Thank you very much. As you alluded to, about 6 years ago I began the privilege of serving as the Chief Medical Officer at the Brigham and Women's Hospital. This was a decision that is not altogether clear to me as to how I made it or why I made it, but it has been a good fit, and I have thoroughly enjoyed the challenges imposed.
I saw an opportunity to impact about 55,000 admissions every year, and a major part of my charge, of course, is to ensure a safe environment for administering that care. I would like to touch on some nuts-and-bolts aspects of implementing that kind of a program and basically concentrate on 2 areas: one technology, the other trust.
Medication errors occurred in about 6% to 7% of all our admissions. Transcription errors account for about 40% of them; medication administration for the other 60%. Some 25 years ago, we began to pioneer computerized entry of orders. This has reduced the incidence of transcription errors by half, and we do know that bar-code administration will reduce the administration errors by about 80%. So why has it taken us so long to implement bar-code technology? You go to the grocery store and buy Rice Krispies. It is scanned at the cashier. The inventory is updated. Your bank account is debited. Yet, why have we not done this?
Part of the reason is that “big pharma” has not supplied us with bar codes on the packaging. The other reason has to do with dollars. It has taken our hospital a commitment of about $10 million. This is on a $1.2 billion budget with a 3% profit margin, which represents 25% of our annual profit margin. A real challenge has been to convince our Chief Financial Officer that this is worthwhile. It took me about 2 years of steady work to soften him up to the point whereby we had a discussion with our CEO. The discussion went something like this. The CFO was bent on preserving the bottom line: “I am worried about our bottom line.” I carried the day by simply saying: “I am worried about dead patients.” We got the investment, and the $10 million has now basically been spent as we are rolling out bar-code technology throughout the hospital as we speak.
The process has had its ups and downs, and it is not without its glitches. I am firmly convinced that we must make this kind of effort and investment to make our hospitals safer environments.
All the technology in the world, however, will not help unless the entire hospital staff buys into creating this culture change that you have heard a number of speakers talk about and Hiram talk about yesterday during his address. We have to create a blame-free environment, which is not something that comes naturally. When we have all been trained to say that “we do not know,” this is a sign of weakness; to make an error, unpardonable if not punishable. So how can we create this blame-free culture that suddenly converts a kind of covert organization to a far more transparent institution?
Transparency is only achievable through trust, and that is mutual trust. And mutual trust, in fact, is only achievable through transparency!
So the first thing we did was to create an Office of Patient Safety. We have a full-time director and full-time infrastructure with advanced nurse practitioners and support staff. They oversee the activities of a number of different initiatives, one of which has been the institution of a Web-enabled reporting system that takes the cumbersome, tedious aspects of reporting errors out of the equation. That, too, is not without its hiccups, and we are constantly refining the software. We do not know what is broken, unless we can get a wider net cast and a better understanding of where the errors are occurring.
We have also instituted executive walk-arounds. Senior management meets every week on one unit, foxhole by foxhole, unit by unit, and tries to inculcate this blame-free culture and make people aware of the fact that we are listening to them; we need to know about near-misses and about medical errors. Our staff generates a to-do list from each one of those weekly meetings, and importantly enough, we have to feed back to the person who reported what we have done, what action has been taken. We have been very conscientious in realizing that to sustain that trust we have to get back to them with actionable items.
As a matter of fact, we are able to bring closure to 80% of our to-do list within 6 months. It is incredibly important to communicate that success to those whom you have enlisted for support.
So sustained trust depends on that delivery. I think we are on our way. When we initially started the walk-around process, we were greeted with skepticism, as you can imagine, on each of the units. Most of the nursing personnel wanted to show us the latest bell or whistle or how well they have performed. It took us a while to gain their trust to have them understand what we were really after: where the errors occurred. They are much more open now. We have been reasonably successful in establishing that transparency, and we work very hard at trying not to sabotage their trust.
Thank you all very much for letting me share with you all these brief comments, and I hope you find some of them helpful.
DR. HIRAM C. POLK, JR: That's a real “rubber meets the road” kind of discussion to convince your CFO to take $10 million off of what he views as “his” bottom line and invest it in patient well-being.
America is not on the leading edge in many of these areas. We have not led in either and certainly not led it medically. Some of you will have heard yesterday in the introduction I made for Dr. Barraclough, one of our new honorary fellows from Australia, that one of his focal points for more than a decade has been in this, and we would like very much to hear from him about his thoughts and ideas about the quality struggle. Dr. Barraclough is Chair of the Australian Council for Safety and Quality in Health Care.
DR. BRUCE BARRACLOUGH (Panelist): Dr. Polk, ladies and gentlemen, thank you very much for giving me the opportunity of speaking today.
It has been interesting to listen to the speakers that have gone before me in this Forum. They have all opened different windows into what, for most surgeons, is a very dark room. My task is to try to bring some of that together and to work out where some of the apparent conflicts that were in the presentations can actually be brought together.
Living in Australia is a bit like being up on a satellite. You are down at the bottom end of the Pacific 10,000 miles from here, 26 hours by air, a country the same size as the continental United States with 20 million people, 6 states, 2 territories, 9 governments. In the safety and quality agenda, I lead a council that answers to 9 health ministers.
We have heard a lot about process and outcomes and what is right and what is wrong. With the privilege of living a long way away from where everything else happens and having an advanced world health system, we were able to look at colleagues in the United States, the United Kingdom, and Europe, Hong Kong, and everywhere else, where there is an advanced health system, to look for the lessons, to look for the lessons from other industries, and to look to the past.
Our task was to improve the safety of the 9 different health systems in our country. We found out very early that patient safety is about health delivery systems. If we are going to be involved in that as surgeons, we need more control over the systems that we are involved in, or, more particularly, we need to work with the people who have the control over the systems in which we work. Surgeons actually have very little control over the systems in which they work at the moment anywhere in the world. So we need to be collaborative with others, especially those who administer and fund the system, to get those delivery systems right.
So what about Shewhart in the 1920s who actually taught people how to build a better product? He did that by developing process control charts. If you were within 3 standard deviations from the mean, you were within a stable system, and things were going pretty well.
Where did Deming come in? He built on Shewhart's work and in the 1950s he taught the Japanese to make a better motor car, and he told us if we did not measure something, we really could not manage it. In one sense he is right, and it is all about process. Donabedian in the 1960s and 1970s brought these concepts to health care and even indicated that we get the biggest bang for our buck if we look not at structure or outcome but at process.
So where does this argument around outcomes and process come from? One of the issues that need to be addressed in health care is that it is not what we do but how we do it. If we are to move from listing and collecting and talking about what we do, and start to talk about how we do it to improve it, we need to know our outcomes. This is where the audits come in. This is where the program that Dr. Jones talked about comes in. However, the collecting of individual outcomes is expensive. We were told earlier in this Forum that it costs $100 a patient to feedback risk-adjusted information and to actually know what is the adjusted outcome.
In a small number of cases, surgeons can then change their practice to alter that outcome. But most of the time, if we are going to get major change across the system, there needs to be system change, and that means process change. Yes, we need to know our individual outcomes, but if we are going to move from what to how, we need to change process. This is the nexus between process and outcomes.
We also need to learn to talk about different aspects of the quality agenda. We talk about it as if it is only one thing and get confused about process and outcomes and bits and pieces of what we do. Most people would define the quality agenda as having about 6 dimensions, which vary little across the world. Safety is one dimension. Appropriateness is another dimension. The RAND people, particularly Beth McGlynn in recent years,27 have done the most work anywhere in the world on that. Patient centeredness is another dimension; access, efficiency, and effectiveness are others.
Safety just happens to be that dimension of quality that is most valued by patients and their families. Charles Vincent working at St. Mary's Hospital in London has done the research work to show this.28 It is an important issue for everyone to understand because we can use safety as a lead-in for the other dimensions of quality, and everyone can grab hold of safety.
I do not think we need to scare surgeons by telling them how bad they are; they are darned good. We have never before in the history of humankind been able to do so much for so many people. Everybody that I know who works in health care goes to work to do good. We actually demand too much of ourselves, and we recognize too little our dependence on the systems in which we work. This is a mind-set and culture change that we need to start to look at.
One of the speakers in this Forum mentioned a blame-free culture. There is no such thing. Someone runs into the back of my motor car, and the first thing I want to do is get out and punch them in their head. This is a natural human response. What we are looking for is lack of inappropriate blame. Yes, there should be blame if we willfully disregard normal standards. Yes, we should be blamed if we do things under the influence of alcohol or drugs. Yes, we should be blamed if we do things in a criminal way, if we break the law. Probably one of the things we also should get blamed for is not reporting when things go wrong.
So, we need a reporting culture where we can actually, openly, freely, and transparently say when something goes wrong.
All this does have to occur in advanced world health systems like yours, in the United States and ours, in Australia. Even the World Health Organization has a safety agenda as a major plank of its worldwide work. They have set up an international alliance in patient safety under the slogan “safe care is clean care.” So, even in the most simple of medical environments, it can be done.
Let me talk about a few of the issues that we, in Australia, have addressed and get down from those high levels to the particular. We picked up on the Institute for Healthcare Improvement's collaborative program, and ran a 100-hospital collaborative in medication safety. One of the things that has come out of that is we have managed to convince 6 states, 2 territories, and a federal government that in every hospital in my country there should be a single medication chart.
Most of our big hospitals like yours have 20 or 30 different medication charts. This is probably a bigger source of error than the way clinicians enter material on those charts. This is just one particular issue. That's the standardization that Deming was talking about. Standardization is really an important process.
Another thing we have done is to address the consumer centeredness aspect of this agenda. Our consumer reference group told us that the most important part of our agenda was open disclosure, the transparency that Association President Hiram Polk talked about yesterday in his tremendous Address. I think that what he said yesterday does identify major changes in the way surgeons look at their practice, and, in fact, the way anybody working in health care looks at the way they do it.
Open disclosure. We developed a national standard about open disclosure, and we have a similar legal environment to yours. You tell me that it is not possible. I will tell you it is because that standard has 3 parts: 1) The education and support you need for the hearing and the telling, 2) an in-depth analysis of the event, and 3) a commitment to fix the recommendations from that analysis.
We are piloting that across Australia now. On the back of this happening, there has been some increased impetus in tort law reform. We are already seeing a reduction in cases. The nexus between adverse events and the legal system is in reality, pretty small. Probably only a half of one percent of all the adverse events ever gets a payout in the courts.
If you look back at the study by Leape and his colleagues, they thought that about 20% of the adverse events might be legally actionable.29 Of those, 12% made a claim, and of those that made a claim, the courts found for them in less than half the cases. So, this is less than half of one percent of all the adverse events actually getting a payout in the courts. Most of those cases did not have an adverse event. They had an unexpected or unsatisfactory outcome, usually on the back of lack of effective communication. So, the communication agenda as it relates to safety and quality is the key nexus with the medical litigation issue. We are actually putting the open disclosure standard in place.
We have done some things in medication safety, as I mentioned, with a standard medication chart. We have national standards around credentialing and defining the scope of practice of clinicians. We are starting to roll those out.
We are looking at the safe hours exercise, but not from the hours' point of view. Like you, we have limited resident duty hours, more limited than yours. We are looking at hand-over practices. We are looking at fatigue management rather than hours. Even airlines are moving to train people in fatigue management rather than trying to look at just hours of work. Most of the fatigue that you and I feel when we are in the hospital we get because we are looking after a sick child, the dog's run away, or something is happening that is really not quite to do with medicine.
Ladies and gentlemen, across the world, people have made a great start. We can learn from each other. We are picking up the Scottish Audit of Surgical Mortality and putting that in place in every state.30 We picked up our root cause analysis program from the VA.31 We are using it as widely as they are.
Teaching 2.5 thousand people out of the 100,000 healthcare workers in my state's healthcare system about the elements of root cause analysis and a bit about human factors, we increased the reporting rate for severe adverse events 30-fold in 6 months, with the same legal system and the same administrative system. This is an extraordinary amount of information that we did not have before. There are an extraordinary number of vulnerabilities that the system can correct by process change because they actually looked at the outcomes and what caused them. Ladies and gentlemen, I will leave it there. More information about the approach taken by the Australian Council for Safety and Quality in Health Care can be found on www.safetyandquality.org. Thank you very much.
DR. HIRAM C. POLK, JR: I dislike a panel that does not have an opportunity for open discussion, but, by taking advantage of the chance to present both the philosophy, the hands-on accomplishments in Michigan, the CMS posture, what Scott is doing with the College, what Andy has done in leadership in a single institution, and what Bruce sees as a world order issue, I think you can see that this is a real crusade that surgery and the American Surgical with its influence and opportunity must say that this is important. We need to get on board, individually and collectively, for this particular endeavor. Our patient is the beneficiary. Thank you so much for your attendance and interest.
Footnotes
Presented at the Annual Meeting of the American Surgical Association, Friday, April 15, 2005.
Reprints: Hiram C. Polk, Jr, MD, Department of Surgery, University of Louisville, Louisville, KY 40292. E-mail: hcpolk01/at/louisville.edu.
REFERENCES
1.
Polk HC Jr. Presidential address: quality, safety, and transparency. Ann Surg. 2005;242:1–9. [PubMed].
2.
Dimick J, Chen S, Taheri P, et al. Hospital costs associated with surgical complications: a report from the Private-sector National Surgical Quality Improvement Program. J Am Coll Surg. 2004;199:531–537. [PubMed].
3.
The Leapfrog Group. Rewarding higher standards for patient safety. Leapfrog Fact Sheet. 2005. Available at: http://www.leapfroggroup.org. Accessed September 23, 2005.
4.
Khuri SF, Daley J, Henderson WG. The comparative assessment and improvement of quality of surgical care in the Department of Veterans Affairs. Arch Surg. 2002;137:20–27. [PubMed].
5.
Moscucci M, Share D, Kline-Rogers E, et al. The Blue Cross Blue Shield of Michigan Cardiovascular Consortium (BMC2) collaborative quality improvement initiative in percutaneous coronary interventions. J Intervent Card. 2002;15:381–386.
6.
Halsted W. A clinical and histological study of certain adenocarcinoma of the breast; and a brief consideration of the superclavicular operation and of the results of the operations for cancer of the breast from 1889–1898 at the Johns Hopkins Hospital. Trans Am Surg Assoc. 1898;16:114.
7.
Drew CR, Bull DC. The preservation of blood. Ann Surg. 1940;112:489.
8.
Snow CP. The Two Cultures and the Scientific Revolution. New York: Cambridge University Press, 1959.
9.
Maxwell JC. A dynamical theory of the electromagnetic field. Phil Trans R Soc Lond. 1865;155:459–512.
10.
Institute of Medicine. Patient Safety: Achieving a New Standard for Care. Washington, DC: National Academies Press, 2004.
11.
Shewhart WA. Economic Control of Quality of Manufactured Product. New York: Van Nostrand, 1931 (Republished 1980 by the American Society of Quality Control).
12.
Deming WE. Out of Crisis, 1st ed. Cambridge, MA: MIT Press, 2000.
13.
Juran JM, ed. Juran's Quality Control Handbook, 4th ed. Toronto: McGraw-Hill, 1988.
14.
Halsted WS. The training of a surgeon. Bull Johns Hopkins Hospital. 1904;15:266–275.
15.
Khuri SF, Daley J, Henderson W, et al. The Department of Veterans Affairs' NSQIP: the first national, validated, outcome-based, risk-adjusted, and peer-controlled program for the measurement and enhancement of the quality of surgical care. National VA Surgical Quality Improvement Program. Ann Surg. 1998;228:491–507. [PubMed].
16.
Hanlon CR. The American College of Surgeons at 90. Bull Am Coll Surg. 2003;88:19–26.
17.
Fistula first breakthrough initiative for vascular access overview. Available at: http://www.cms.hhs.gov/quality/esrd/FFOverview.pdf. Accessed April 15, 2005.
18.
Bratzler DW, Houck PM, et al. Use of antimicrobial prophylaxis for major surgery: baseline results from the National Surgical Infection Prevention Project. Arch Surg. 2005;140:174–182. [PubMed].
19.
Siddiqui AK, Ahmed S, Delbeau H, et al. Lack of physician concordance with guidelines on the perioperative use of beta-blockers. Arch Intern Med. 2004;164:664–667. [PubMed].
20.
Claggett GP, Anderson FA Jr, Geerts W, et al. Prevention of venous thromboembolism. Chest 1998;114(suppl):531–560.
21.
Eggers PW, Milam RA. Trends in vascular access procedures and expenditures in Medicare's ESRD program. In: Mitchell L, Henry WL, eds. Vascular Access for Hemodialysis, vol. VII. Chicago, IL: Gore & Associates Precept Press, 2001:133–143.
22.
Julius Caesar, Act I, Scene II, line 146.
23.
A. Philip Randolph Institute. Available at: http://www.apri.org. Assessed Sept. 30, 2005.
24.
Organ CH. Presidential Address: You can make a difference. Bull Am Coll Surg. 2003;88:19–20.
25.
Laws ER. Presidential Address: Harvey Cushing and the unity of surgery. Bull Am Coll Surg. 2004;89:8–12.
26.
Manuel BM, Nora PF, eds. American College of Surgeons: Essential Information for Surgeons in Today's Environment. Chicago: American College of Surgeons, 2004.
27.
McGlynn EA, Asch SM, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645. [PubMed].
28.
Vincent C, ed. Clinical Risk Management: Enhancing Patient Safety, 2nd ed. London: BMJ, 2001.
29.
Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients: results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324:377–384. [PubMed].
30.
Thompson AM, Stonebridge PA. Building a framework for trust: critical event analysis of deaths in surgical care. BMJ. 2005;330:1139–1142. [PubMed].
31.
Bagian JP, Lee C. Developing and deploying a patient safety program in a large health care delivery system. Jt Comm J Qual Improv. 2001;27:522–532. [PubMed].