Gordon R. Bernard, M.D.
Associate Director, Division of Allergy,
Pulmonary, and Critical Care Medicine
Vanderbilt University Medical Center
Room T1208 MCN
Nashville, TN 37232-2650

Clinical Trials in Sepsis: What Are We Doing Wrong

Joseph V. Bonventre, M.D., Ph.D.
Professor of Medicine
Harvard Medical School
149 13th Street, MGH East
Charlestown, MA 02129-2060

Potential Targets: Strength of Evidence

Josephine Briggs, M.D.
Director, Division of Kidney,
Urologic and Hematologic Diseases
National Institute of Diabetes and Digestive
and Kidney Diseases
National Institutes of Health
Building 31, Room 9A17
31 Center Drive, MSC 2560
Bethesda, MD 20892-2560

Welcome

Robert Califf, M.D.
Director, Clinical Research Institute
Duke University
2400 Pratt Street
Durham, NC 27705

Acute Heart Failure

Glenn M. Chertow, M.D., M.P.H.
Assistant Professor of Medicine
University of California at San Francisco
672 Health Sciences East, Box 0532
513 Parnassus Avenue
San Francisco, CA 94143-0532

Epidemiology of ARF
Traditional Endpoints for ARF Trials

John Conger, M.D.
Professor of Medicine
University of Colorado Health Sciences Center
Department of Medicine
Division of Renal Diseases
University of Colorado School of Medicine
VA Building A, Room 216B
4200 East 9th Avenue
Denver, CO 80262

Strength of Science of Current Animal Models

Stan Finkelstein, M.D.
Senior Research Scientist
Massachusetts Institute of Technology
Sloan School of Management
38 Memorial Drive
Cambridge, MA 02139

Economics and Market Size Analysis of ARF

Charles Fisher, M.D.
Eli Lilly and Company
Lilly Corporate Center
Indianapolis, IN 46285

Alternative Endpoints: Organ Failure Free Days
Statistics in Heterogenous Populations

Joel Gore, M.D.
Chief, Division of Cardiovascular Medicine
University of Massachusetts
Memorial Medical Center
Division of Cardiology
55 Lake Avenue North
Worcester, MA 01655

Critique of Acute Renal Failure Trials
Advancement to Clinical Trials

Tom Greene, Ph.D.
Statistician
Cleveland Clinic Foundation
9500 Euclid Avenue
Cleveland, OH 44195-0002

What Have We Learned from ARF Trials: Statistical Issues
Clinical Trial Design and Implementation: Power Calculations

Michael Kashgarian, M.D.
Professor
Yale University School of Medicine
310 Cedar Street
P.O. Box 208023
New Haven, CT 06520-8023

Pathology of Acute Renal Failure

Bertram L. Kasiske, M.D.
Director, Division of Nephrology
University of Minnesota
Hennepin County Medical Center
701 Park Avenue
Minneapolis, MN 55415-1623

Delayed Graft Function After Renal Transplantation

Joel D. Kopple, M.D.
Professor of Medicine and Public Health
Division of Nephrology
Harbor University of California
at Los Angeles Medical Center
1000 West Carson Street
Torrance, CA 90509

Nutritional Issues in Acute Renal Failure

William Macias, M.D.
Medical Director
APC Product Team
Eli Lilly and Company
Lilly Corporate Center
Indianapolis, IN 46285

Implementation Issues

Ravindra L. Mehta, M.D., F.A.C.P.
Professor of Medicine
University of California at San Diego
200 West Arbor Drive, #8342
San Diego, CA 92103-8781

What Have We Learned from ARF Trials: Dialysis

Steven B. Miller, M.D.
Vice President and Chief Medical Officer
Barnes Jewish Hospital
Administration, Mailstop: 90-71-313
One Barnes-Jewish Hospital Plaza
St. Louis, MO 63110

What Have We Learned from Trials to Prevent ARF

Bruce A. Molitoris, M.D.
Professor of Medicine
Indiana University School of Medicine
Fessler Hall, Room 115
1120 South Drive
Indianapolis, IN 46202-5116

Diagnostic and Prognostic Urinary Markers in ARF

Bryan D. Myers, M.D.
Professor of Medicine
Stanford University
Chief, Division of Nephrology
300 Pasteur Drive, Room S201
Stanford University Medical Center
Stanford, CA 94305-5114

Pathology, Pathophysiology, and Natural History of DGF

Emil P. Paganini, M.D.
Section Head
Cleveland Clinic Foundation
9500 Euclid Avenue
Cleveland, OH 44195

Clinical Trial Design and Implementation Risk Adjustment/Stratification

Seymour Rosen, M.D.
Director of Surgical Pathology
Harvard University
Beth Israel Deaconess Medical Center
330 Brookline Avenue
Boston, MA 02215

Difficulties in Understanding Human ATN: Lessons from Animal Models

Robert W. Schrier, M.D.
Professor and Chair, Department of Medicine
University of Colorado Health Sciences Center
4200 East 9th Avenue, Box B178
Denver, CO 80120

Drug Treatment of Established ARF

Norman J. Siegel, M.D.
Professor of Pediatrics and Medicine
Yale University School of Medicine
Department of Pediatrics
333 Cedar Street
P.O. Box 208064
New Haven, CT 06520-8064

ARF in Children

Robert Star, M.D.
Senior Scientific Advisor
Division of Kidney, Urologic, and
Hematologic Diseases
National Institute of Diabetes and
Digestive and Kidney Diseases
National Institutes of Health
Building 31, Room 9A35
31 Center Drive
Bethesda, MD 20892-2560

Goals and Structure of the Conference

Taylor Thompson, M.D.
Director, Medical Intensive Care Unit
Harvard Medical School
Pulmonary Unit
Massachusetts General Hospital
55 Fruit Street
Boston, MA 02114

Ventilator Settings in Acute Lung Injury and ARDS

Douglas C. Throckmorton, M.D.
Deputy Division Director
Cardio-Renal Drug Division
U.S. Food and Drug Administration
5600 Fishers Lane, HFD-110
Rockville, MD 20857-0001

FDA Requirements for Investigational New Drug Applications and New Drug Applications

Anne Willoughby, M.D., M.P.H.
Branch Chief
Center for Research for Mothers and Children
National Institute of Child Health and
Human Development
Pediatric, Adolescent and Maternal AIDS Branch
National Institutes of Health
Room 4B 11J
6100 Executive Boulevard, MSC 7510
Bethesda, MD 20892-7510

Acute renal failure (ARF) is a life-threatening illness whose mortality has remained high despite hemodialysis and other advances in supportive care. Understanding of the pathophysiology of ARF has advanced because of information gained from animal models. However, translation of these advances to the patient bedside has been slow. Because of this difficulty, the NIH sponsored a workshop on “Design Issues for Clinical Trials in Acute Renal Failure” held in Bethesda, MD on September 10-12, 2000. The meeting brought many issues into the open and enhanced communication between basic and clinical scientists in the field. The following is an initial synthesis of the main conclusions of the conference; however, a detailed conference report will be published elsewhere.

Outside Advice

We asked experts from the areas of ARDS, sepsis, and acute heart failure to tell us why interventions have succeeded or failed in their areas. Drs. Taylor Thompson and Claude Bernard suggested that large-scale interventional trials succeed because information from sound preclinical studies and pilot clinical trials is used to select trial hypotheses and aid in the design of clinical trials. Knowledge about the natural history of the disease allows accurate estimation of control group event rates; information about the intervention allows assessment of the golden window of opportunity and effect size. The third discussant, Dr. Robert Califf, suggested that animal studies of acute myocardial infarction provided the wrong targets, thus impeding the development of effective interventions. Instead, plausibility for subsequent clinical trials was provided by an astute clinical observation that opening the disease artery helps; i.e., the heart likes blood flow. He stressed the importance of large simple trials, where the entry criteria are consistent with clinical practice. All discussants agreed that successful interventional trials require a network of experienced clinical investigators and study coordinators, access to a large number of patients, an adequate network size to achieve statistical power, and careful and continuous attention to implementation issues.

The most common reasons for failed trials include poor design because of low statistical power, inadequate definition of disease, improper selection of endpoints, and failure to control for patient heterogeneity and non-standardized concurrent therapies. Negative trials may occur because of problems with the intervention including inactive drugs, adverse drug effects that overpower a beneficial effect, missing the golden window of opportunity, wrong or redundant targets, and a misunderstanding of the mechanism of disease. This list was long and daunting. However, recent successes with low volume ventilation in ARDS [1] and an unpublished trial using activated protein C in sepsis indicate that patients with severe illnesses can be successfully treated.

Learn from Past Successes and Mistakes

Drs. Steve Miller, Robert Schrier, Ravi Mehta, and Bert Kasiske reviewed past clinical trials in ARF. Ronco et al. [10] recently found that more ultrafiltration during continuous renal replacement therapy improved survival. This is the most direct evidence to date that treatment of ARF can alter mortality in ICU patients with ARF. The recent report that N-acetyleysteine treatment prevents GFR from falling after radiocontrast [12] promotes antioxidant pathways as new targets for other interventions in ARF. The speakers also reviewed trials that failed to show statistically significant effects. Drug trials have had trouble with delayed diagnosis and randomization [2, 6], delayed administration of study drug [2, 5-7], study drug-induced hypotension [2, 9], low statistical power caused by small sample size [5], confounding effects of nonstudy drugs [4], and poorly defined endpoints (e.g., when to start dialysis). Dialysis studies have been hampered by unbalanced randomization caused by small sample size and the heterogeneity of human ARF [8]. Dialysis trials have been difficult to implement because there is a lack of standard criteria for initiating dialysis, multiple methods of delivering dialysis, and difficulties in measuring and comparing dialysis dose across the different methods.

Identify Gaps and Fill Them

The speakers stressed that many critical tools are missing which would greatly enhance the design and implementation of clinical trials in ARF. These missing tools include, for example, an adequate definition of ARF, non-invasive diagnostic and prognostic tests, real-time measurements of renal function, accurate severity of illness scores that measure both the renal and nonrenal components of illness, and an accurate method to measure dialysis dose. There is insufficient epidemiological data for accurate estimation of the event rate in ICU patients with ARF, and inadequate knowledge of the pathophysiology of human ARF, although recent advances in our understanding of delayed graft function are encouraging. Successful tool development will likely require collaboration of basic science and clinical investigators with access to modem tools (genomics, gene arrays, proteomics, etc.), sufficient well-characterized patients, and human biopsy material.

Advice from Clinical Trialist

A clinical trial explores the cause-and-effect relationship between an intervention and an outcome. The ideal trial is a prospective comparison of intervention X to a control group; randomized to reduce confounding third factors, and double blind (masked) to decrease bias. Drs. Ray Bain, Joel Gore, William Macias, and Tom Greene indicated that conduct of successful clinical trials will likely require (1) a testable hypothesis usually based upon a sound understanding of the mechanism of disease, (2) a feasible intervention usually based upon preclinical animal data or pilot human studies, (3) practical entry criteria based upon knowledge of the population at risk, (4) identification of clinically meaningful outcomes, and (5) pilot data to estimate event rate, effect size, and sample size. As discussed by Dr. Califf, the first two criteria are helpful but not necessary; in rare cases, other information can be substituted for mechanism and preclinical animal data.

Tom Greene described the difference between internal and external validity. Studies aiming for internal validity are designed to determine the effect of treatment on the primary outcome. These studies are typically Phase II studies with narrow entry criteria that control for extraneous variation in the patient population, have a tightly controlled intervention, and have a sensitive outcome measure (change in GFR). Because this type of study minimizes variation, it can be quite small. In contrast, studies aiming for external validity attempt to generalize to broad clinical practice patterns. Thus, they have broad entry criteria, a realistic population that matches the clinical population of interest, realistic interventions that are clinically achievable, and clinically relevant outcomes (death or dialysis). This tension between demonstrating a treatment effect and having external validity arises during the design phase of every clinical trial.

A clinical trial may be summarized as follows: “To evaluate the efficacy of [intervention regimen X] vs. control regimen in participants with [population to be studied] as assessed by [primary outcome measurement].” We will not discuss potential interventions, since they have been reviewed in several recent reviews [3, 11, 13]. The study population is defined by inclusion and exclusion criteria. Inclusion criteria define the disease to be studied, whereas exclusion criteria narrow the population to exclude patients who will not make it through the trial because of noncompliance or serious underlying medical conditions. Because the ARF field lacks a standard definition of disease, this critical portion of the clinical trial design process is currently very difficult.

Many outcomes have been used in interventional trials in ARF. The FDA requires clinically meaningful and compelling outcomes, which include living longer and feeling better; i.e., decreased need for dialysis, and decreased dialysis-free days, but does not necessarily include improved renal function (GFR). Small pilot trials commonly employ GFR as a functional surrogate endpoint; however, the primary outcome in larger trials is death or dialysis. For example, an intervention that improves GFR by 20% would not be clinically compelling. However, if the same intervention decreased the need for dialysis from 50 to 40%, that would be clinically compelling. Obviously, the outcome should be amenable to therapy. This sounds simple, but may be a problem in ARF since nonrenal factors may dominate. Binary outcomes (dead/ alive; dialysis/no dialysis), time to event (death, dialysis), or organ failure-free days may be useful as primary outcome variables because they are structured to pick up events that occur in only a portion of the population (death, dialysis), or are likely amenable to therapy (dialysis-free days).

Statistical Issues

Drs. Tom Greene, Bill Macias, and Charles Fisher discussed the statistical issues to consider when designing a clinical trial, including patient heterogeneity, sample size calculations, prespecified data analysis plans, and interim stopping rules. The first two items will be considered here. The extreme patient heterogeneity of human ARF has led to unbalanced randomization in several ARF trials. This problem can be lessened in larger trials because of the law of large numbers. Smaller trials may need to stratify patients during randomization, or adjust the analysis for prespecified covariates measured before randomization. Stratification should more evenly allocate the patients to the different treatment. There was general agreement that stratification will be critical for small and moderately-sized trials; however, stratification may unnecessarily complicate large simple trials.

Calculating sample size is a critical design element of any clinical trial. Many previous ARF trials have been underpowered because of overgenerous hypothesized treatment effects and failure to correct sample size for deviations from protocol. The sample size calculation requires accurate knowledge of the control outcome which is usually based upon epidemiologic information of patients with the same inclusion/exclusion criteria used for the trial. Ideally, this should be a conservative estimate that includes recent improvements in clinical practice. Second, one needs an accurate estimate of the treatment effect size. The outcome of a large trial should be clinically relevant, compelling, and sufficient to change clinical practice. The estimate should also be based upon a realistic expectation of the treatment effect. In ARF trials, the estimate must consider the effect of the intervention on renal and nonrenal pathways to the measured outcome. Thus, a drug may decrease renal events by 50%, but if only 20% of the patient outcome is determined by renal effects, the outcome will only change by approximately 10%. Sample size calculations are traditionally performed using an alpha level of 0.05 and a power of 90%. With these four pieces of information, one can estimate the sample size. For example, consider the case of an intervention that changes the mortality of ICU ARF from 50 to 40%. At an alpha of 0.05 and a power of 90%, 540 patients per group are needed to achieve statistical significance. This example illustrates that large trials will be needed for adequate power. A common response to obtaining a large sample size estimate is to increase the hypothesized effect size, and thus lower the sample size. All the statisticians and clinical trialists emphasized that this should not be done; trials should be planned with realistic effect sizes. A statistician can aid in correcting this initial estimate for reduction in event rate, effect of covariates, crossovers, dropouts, etc. These corrections can nearly double the sample size estimate.

Design of Clinical Trial Networks

Drs. Ann Willoughby and William Macias described the design issues for the formulation of a successful clinical trial network. A clinical trial network and an individual clinical trial have a well-defined “life cycle” that ends with a publication; this end should be kept in mind at all times. The process begins with the formation of a small planning committee that explicitly defines in writing the goals and objectives of the network along with the roles and responsibilities of every member of the project. It is important to define in writing who has the responsibility for defining the scientific agenda, pacing the activities, distributing resources, and establishing sanctions. Strong leadership is absolutely essential; clinical trialists and statisticians should be involved from the start. The planning committee, along with the data coordinating center, may be formed before the rest of the network is developed. The sequencing of studies and trials is defined by discussions of the state of the art, and key informative questions including what is known about the epidemiology, pathogenesis, and natural history of the disease. For example, is the field ready for therapeutics clinical trials? Are pilot studies needed? These discussions may take 1 year, but must yield definable pieces of work including specifications for epidemiologic studies, hypothesis generating pilot studies, and definitive trials. These mechanisms will be used throughout the lifetime of the network to select and prioritize studies and trials. For each trial, the network will develop a detailed written protocol, a prespecified statistical analysis plan, a monitoring plan, and a validation plan. These plans will specify the data collection needs, systems (database) support, and the structure of the final report. Finally, the network must develop a well-designed case report form for capturing data.

The network will include basic scientists, clinical researchers, clinical trialists, and statisticians. A well-crafted external advisory board, liaison to professional societies, high visibility at key scientific meetings, and advocacy groups are also essential. Once the clinical and basic science centers have been added, the network sets up a training structure to train the principal investigators, study coordinators, and other study personnel. Ideally, this training allows all study personnel to understand the issues of the trial and “buy into” the trial. This training creates a cadre of excellent and dedicated study personnel that can properly execute the network protocols. The implementation of the trial protocol, monitoring plan, and validation plan are discussed below. The data should be transmitted from each clinical center via the Internet to a versatile data warehouse. This centralized data collection allows the trial to be continuously monitored; at the end of the trial, the prespecified analysis to be rapidly performed, allowing for timely publication of the clinical trial results.

Implementation Issues

While planning and data analysis phase of a clinical trial is critical, the majority of the effort is placed into implementation of the trial, including recruitment, retention, outcome assessment, and monitoring. The trial team must be ready to face and rapidly solve all problems that occur during this stage. This is commonly aided by training workshops for the study personnel and by computerized methods to monitor the trial in real time. Recruitment is always an issue, since it commonly lags far behind the initial goals.

Clinical trials must follow good clinical practice guidelines as outlined in CFR Title 21, CFR Title 45, and the International Conference on Harmonization [www.fda.gov/cder]. These provide a unified standard for designing, conducting, recording, and reporting trials that involve the participation of human subjects. For example, all trial procedures, policies, and actions should be documented in writing. The data coordinating center should monitor “patient quality” in real time to prevent randomization of patients who do not meet the inclusion and exclusion criteria of the trial. The trial must monitor drop-ins, drop- outs, and noncompliant patients. Standard operating procedures must be designed to maintain the trial intervention (e.g., level of dialysis dose) and standardize other nontrial interventions (e.g., dopamine). The data quality should be monitored by validating a portion of the submitted data. The conference also considered regulatory (e.g., Institutional Review Board, Food and Drug Administration) issues that will not be discussed here.

Conclusions

Management of patients with ARF is challenging because the existing evidence base is weak or lacking. Researchers must aim to obtain conclusive evidence at the level of evidence-based medicine because this type of information is right for patients, is critical for health policy decisions, and is increasingly used for reimbursement decisions. Gathering the necessary tools and clinical evidence will require significant work and resources. The design of a large-scale interventional trial network in ARF will be difficult. It is up to the community and NIH to obtain advice from a broad range of opinions, and adhere to the highest standards when designing the network. However, recent clinical trials indicate that ARF is amenable to treatment.

References

1. The Acute Respiratory Distress Syndrome Network: Ventilation with lower tidal volumes as compared with traditional tidal volumes for acute lung injury and the acute respiratory distress syndrome. N Engl J Med 2000;342:1301-1308.
   
2. Allgren RL, Marbury TC, Rahman SN, Weisberg LS, Fenves AZ, Lafayette RA, Sweet RM, Genter FC, Kurnik BRC, Conger JD, Sayegh MH: Anaritide in acute tubular necrosis: Auriculin Anaritide Acute Renal Failure Study Group. N Engl J Med 1997;336:828-834.
   
3. Bonventre JV: Mechanisms of ischemic acute renal failure. Kidney Int 1993;43:1160-1178.
   
4. Chertow GM, Sayegh MH, Allgren RL, Lazarus JM: Is the administration of dopamine associated with adverse or favorable outcomes in acute renal failure? Am J Med 1996;101:49-53.
   
5. Franklin SC, Moulton M, Sicard GA, Hammerman MR, Miller SB: Insulin-like growth factor I preserves renal function postoperatively. Am J Physiol 1997;272:F257-F259.
   
6. Hirschberg R, Kopple J, Lipsett P, Benjamin E, Minei J, Albertson T, Munger M, Metzler M, Zaloga G, Murray M, Lowry S, Conger J, McKeown W, O'Shea M, Baughman R, Wood K, Haupt M, Kaiser R, Simms H, Warnock D, Summer W, Hintz R, Myers B, Haenftling K: Multicenter clinical trial of recombinant human insulin-like growth factor I in patients with acute renal failure. Kidney Int 1999;55:2423-2432.
   
7. Lewis J, Salem MM, Chertow GM, Weisberg LS, McGrew F, Marbury TC, Allgren RL: Atrial natriuretic factor in oliguric acute renal failure. Am J Kidney Dis 2000;36:767-774.
   
8. Mehta R, McDonald R, Gabbai F, Pahl M, Farkas A, Pascual M, Fowler W, ARF Collaborative Study Group: Continuous versus intermittent dialysis for acute renal failure in the ICU: Results from a randomized multicenter trial (abstract). J Am Soc Nephrol 1996;7:1457.
   
9. Rahman SN, Kim GE, Mathew AS, Goldberg CA, Allgren R, Schrier RW, Conger JD: Effects of atrial natriurelic peptide in clinical acute renal failure. Kidney Int 1994;45:1731-1738.
   
10. Ronco C, Bellomo R, Homel P, Brendolan A, Dan M, Piccinni P, La Greca G: Effects of different doses in continuous veno-venous haemofiltration on outcomes of acute renal failure: A prospective randomized trial. Lancet 2000;356:26-30.
    
11. Star RA: Treatment of acute renal failure. Kidney Int 1998;54:1817-1831.
    
12. Tepel M, van der Griet M, Schwarzfeld C, Laufer U, Liermann D, Zidek W: Prevention of Radiographic-contrast-agent induced reductions in renal function by acetylcysteine. N Engl J Med 2000;343:180-184.
    
13. Thadhani R. Pascual M, Bonventre JV: Medical progress--Acute renal failure. N Engl J Med 1996;334:1448-1460.

Design Issues for Clinical Trials in ARF
Blood Purif 200l;l9:233-237
Accessible online at: www.karger.com/journals/bpu