III.

To overcome limitations in measuring program effectiveness on actual organ retrieval, the organ donation community has used three related sets of performance indicators, each with varying

Source: Lewin, 1998

degrees of separation from the actual intended outcome of organ donation. The three types of measures are shown in Exhibit 3 as concentric circles, illustrating their relative proximity to the donation event. In addition, the US population is used as an example to depict the relative sizes of the populations captured in pre-event, post-event, and donation measures.

• Pre-event measures (represented by the outermost circle) are used to gauge effectiveness in increasing organ donation before an actual donation opportunity arises. The particular measures tend to be specific to the type of activity being performed. Compared to the other types of performance indicators, pre-event measures are most varied in nature, and are the farthest removed from the goal of measuring increased organ donation.
• Post-event measures (the middle circle) are commonly used in the organ donation community to measure the five crucial steps in organ retrieval after a potential donor situation has occurred. Chronologically, and with increasingly close ties to donation itself, these steps measure: 1) donor identification rate, 2) referral rate, 3) request rate, 4) consent rate, and 5) retrieval rate.
• Donation rates (the innermost circle) represent the most direct measure of the success of programs to increase the number of organs made available for donation.

Pre-event measures are most commonly used to evaluate activities that precede the trauma leading to brain death. Though frequently used as a proxy for program effectiveness on donation rates, they are process measures that serve as weak predictors of actual donation rates. A media campaign to increase the number of people who become organ donors when they renew their driver's licenses could be deemed successful if significantly more people signed donor cards than in months past. However, the link between the number of new licenses with organ donation approval and the donation rate is uncertain at best. A large follow-up study would be required to determine whether the process of signing a donor card had an impact on a family’s decision to donate organs. The link between pre-event measures and the primary goal of increasing organ donation is weak.

Many researchers are striving to improve pre-event measures by using these measures to assess the five stages of behavioral change that may lead to organ donation, i.e.: precontemplation, contemplation, preparation, action, and maintenance. These five stages of change were developed by Prochaska et al. (1983), originally in relation to smoking cessation programs. Studies in the areas of smoking cessation, mammography use, and weight loss programs have attempted to: 1) design measures for the stages of change, 2) determine if the stage of change correlates to success in the designated program, and 3) determine if activities can be implemented to improve cycling through the stages of change. These activities are evaluated relative to activities that were not specifically designed to promote cycling through stages of change. In the field of organ donation, these stages of change can be thought of as follows (Rohr, manuscript).

• Precontemplation: the person has not thought about donating organs.
• Contemplation: the person has thought about donating organs.
• Preparation: the person has made phone calls or requested information about organ donation.
• Action: the person has taken action to express his/her wishes about organ donation (e.g., signed a donor card, talked to a lawyer, discussed wishes with family).
• Maintenance: the person may occasionally remind or reinforce statement of wishes.

Multiple studies have indicated that familial consent for organ donation is one of the biggest barriers to donation. Other papers have shown that families are more likely to consent to donation if the deceased individuals have made their wishes known. Applying the stages of change methodology to measure progress towards action may be a useful indicator of outcomes. For example, stages of changes measures can be developed to determine whether a particular intervention moved people to sign donor cards and discuss organ donation with their families.

Another group of broadly applicable measures relates to the various steps of the organ retrieval process: identification, referral, request, consent, and retrieval. Because these steps are more immediate to the act of donation, they are more meaningful, though still intermediate, indicators of the effectiveness of an organ donation activity. Also, because they are common to many different kinds of programs, they allow for more meaningful comparison across programs than pre-event measures. Each of the rates is described in detail below, with comments about potential uses.

The first step of the retrieval process is the recognition of a patient as a potential donor of viable organs. The identification rate is the number of potential donors identified divided by the total number of actual potential donors, as determined by medical record review (MRR).

• The identification rate is a potential measure of the effectiveness of provider education programs that seek to improve the ability of medical staff to properly recognize the fulfillment of donor criteria in a patient.

The second step of the retrieval process is the referral of a potential donor to an OPO. The referral rate is the number of potential donors referred to an OPO divided by either total potential donors (determined by MRR) or by total identified donors (determined from hospital data).

• The referral rate can be used to evaluate the effectiveness of Required Referral and Required Request legislation. Although the intent of such legislation is to increase donation by involving expert OPO staff more frequently and earlier in the donation process, it is referral rate, rather than donation rate, that best indicates the impact of the legislation.

The third step of the retrieval process is the request made by hospital or OPO staff of the potential donor’s family. The request rate is the number of potential donors whose families are asked to donate divided either by total potential donors (determined by MRR) or by total identified or referred potential donors (determined from OPO or hospital data).

• The request rate can also be used to evaluate how well doctors and OPO staff respond to new Required Request legislation. While there are few intervening factors to make consent and donation themselves the primary measure of interest in an evaluation, the request rate, cross-referenced with these behaviors, is a useful tool for use in hospital and OPO-level quality review and intervention.

The fourth step of the retrieval process is the consent granted for donation by the family of the potential donor. The consent rate is the number of consenting families of potential donors divided either by total potential donors (determined by MRR) or by total identified or referred potential donors (determined from OPO or hospital data).

• Requests decoupled from explanations of brain death, requests made in a private, quiet setting, and requests made of minority families by minority requestors have been correlated with higher consent and donation rates (e.g., Beasley 1997 and Kappel 1993). Because steps in the retrieval process after consent are affected largely by scientific matters such as tissue preservation and not by factors influenced by most donation activities (predisposition through request), consent rate is considered by many donation professionals to be the de facto outcome measure of programs to increase organ donation.

The final step of the retrieval process is the retrieval of a viable organ from a donor. While regularly collected by OPOs to determine the efficiency of their retrieval processes, the primary factor in the difference between consent rate and donation rate lies in medical procedures to ensure tissue and organ viability.

• Due to scientific concerns such as tissue preservation and autopsy requests, the retrieval rate is not regularly used in place of either consent or donation in the evaluation of programs designed to increase donation.

The organ donation rate is a measure of the eventual goal of all donation-related activities: an increase in successful donations. Any activity that can be shown to independently increase donation is clearly successful.

The most precise definition of donation rate uses the actual number of potential donors as its denominator, whether or not the potential donors had actually been asked or even identified by procurement staff. The size of the potential donor pool is best calculated by retrospective medical record review (MRR).

• This measure is particularly useful for hospital and OPO-level evaluation, yielding measures not only of the donation rate but also of post-event measures, such as consent rate. However, the cost and time-intensive nature of the MRR is prohibitive for national-level donation studies.

A less expensive and faster alternative to estimating the potential donor pool involves using the number of deaths attributed by death certificate codes to causes that suggest potential donors. These data are relatively accessible from national mortality databases or state and local offices of vital statistics.

• The availability of death certificate data on the national level allows national rates to be determined consistently with state and local rates. Some researchers have expressed concern that the estimation is compromised by two problems: inconsistency in death coding patterns in different areas of the country and an insufficient definition of potential donors, particularly since there is no authoritative coding that assures potential for donation.

The crudest outcome measure, and the one most widely used, is donors per million population (DPMP). It requires the assumption that potential donors are more or less equally distributed over a population, so that the population itself can be used as a rough proxy of the potential donor pool. The calculation of this measure is the easiest of the three outcome measures described here, requiring only census data and the number of donations in a specified geographical area.

• While the simplicity of this approach has made it the most recognized of measures, including use internationally and by the US DHHS in OPO certification, the use of unadjusted population size raises significant methodological problems. For example, any state with higher than average proportions of residents older than the allowable age for donation (e.g., Florida) or with a lower than average rate of trauma death will have an artificially high denominator in the DPMP rate, thus deflating its true procurement efficiency with "potential donors" who can’t be realized. Despite the ease of calculating DPMP, its lack of comparability across geographic borders compromises its utility for meaningful evaluation.