This is the accessible text file for GAO report number GAO-03-494 entitled 'Dietary Supplements: Review of Health-Related Call Records for Users of Metabolife 356' which was released on April 30, 2003. This text file was formatted by the U.S. General Accounting Office (GAO) to be accessible to users with visual impairments, as part of a longer term project to improve GAO products' accessibility. Every attempt has been made to maintain the structural and data integrity of the original printed product. Accessibility features, such as text descriptions of tables, consecutively numbered footnotes placed at the end of the file, and the text of agency comment letters, are provided but may not exactly duplicate the presentation or format of the printed version. The portable document format (PDF) file is an exact electronic replica of the printed version. We welcome your feedback. Please E-mail your comments regarding the contents or accessibility features of this document to Webmaster@gao.gov. Report to the Chairman, Subcommittee on Wellness and Human Rights, Committee on Government Reform, House of Representatives: United States General Accounting Office: GAO: March 2003: DIETARY SUPPLEMENTS: Review of Health-Related Call Records for Users of Metabolife 356: Metabolife 356 Adverse Event Reports: GAO-03-494: Contents: Letter: Results in Brief: Background: Consumer Information in Call Records Was Limited, Sometimes Difficult to Interpret, and Not Consistently Recorded: Metabolife International Had Thousands of Call Records Reporting Adverse Events Associated with Metabolife 356: Findings of Different Reviews of Metabolife International Call Records Vary: Summary: Agency and Metabolife International Comments and Our Evaluation: Appendix I: Scope and Methodology for Categorizing the Call Records: Call Records And Supplementary Information: Exclusion of Duplicate and Nonhealth-Related Call Records: Classification of Records and Data Entry Procedures: Appendix II: Metabolife 356 Label: Appendix III: Requirements for Reporting Adverse Events to FDA: Appendix IV: Comments from the Food and Drug Administration: Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Acknowledgments: Tables: Table 1: Percentage of Call Records in Which Consumer and Response Details Were Recorded: Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke, Seizure, or Death: Table 3: Metabolife 356 Call Records Reporting Adverse Events Described as Serious or Potentially Serious in FDA's 1997 Proposed Rule: Table 4: Number of Call Records Containing Reports of Heart Attack, Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of Metabolife International Call Records: Table 5: Requirements for Reporting Adverse Events to FDA: Abbreviations: ANADA: Abbreviated New Animal Drug Application: ANDA: Abbreviated New Drug Application: FDA: Food and Drug Administration: HHS: Department of Health and Human Services: NADA: New Animal Drug Application: NDA: New Drug Application: This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO's product. United States General Accounting Office: Washington, DC 20548: March 31, 2003: The Honorable Dan Burton Chairman Subcommittee on Wellness and Human Rights Committee on Government Reform House of Representatives: Dear Mr. Chairman: Medical experts have expressed concerns about the safety of dietary supplements containing ephedra or ephedrine alkaloids, which are used by millions of Americans annually.[Footnote 1] On February 28, 2003, the Food and Drug Administration (FDA) announced several proposed changes to its regulation of dietary supplements containing ephedra, including requiring a product label warning that "Heart attack, stroke, seizure, and death have been reported after consumption of ephedrine alkaloids."[Footnote 2] As of September 27, 2002, FDA had received approximately 1,800 adverse event reports regarding consumers of dietary supplements containing ephedra. Of these, 322 concerned Metabolife 356, a weight loss product first marketed in 1995 by Metabolife International, a large manufacturer of dietary supplements containing ephedra. Adverse event reports can signal potential health problems that deserve additional investigation, but, on their own, generally are not sufficient to establish that the reported problems were caused by use of the product. Metabolife International has also received complaints about adverse health events among users of Metabolife 356.[Footnote 3] Between August and December 2002, Metabolife International made available to the public redacted[Footnote 4] copies of nearly 16,000 pages of documentation that it identified as containing reports of adverse events among consumers of Metabolife 356.[Footnote 5] These complaints, which were received between May 1997 and July 2002, had not been previously released to FDA. Most of them were records of calls received through a consumer health information phone line established by Metabolife International in 1998 and staffed by its nurses.[Footnote 6] Metabolife International officials told us that the phone line was established to provide information to consumers regarding appropriate use of Metabolife 356. In letters to the Texas Department of Health and FDA,[Footnote 7] company officials described the phone line as a "safety monitoring procedure" for the reporting of medical complaints. The call records ranged from handwritten notes to printed versions of records that had been entered into a database developed by Metabolife International. These call records have been the subject of six previous reviews: one by Metabolife International,[Footnote 8] three by consultants commissioned by Metabolife International,[Footnote 9] one by the minority staff of the Committee on Government Reform, House of Representatives,[Footnote 10] and one by the RAND Corporation.[Footnote 11] You asked us to review the content of all health-related call records made public by Metabolife International. Specifically, you asked us to answer the following questions. (1) To what extent was consumer information in the call records comprehensive, interpretable, and consistently recorded? (2) How many call records reported health- related problems, and how many of those were serious? (3) How do our counts of reported serious adverse events compare with those of other reviews for those events counted by Metabolife International? In responding to your request, we reviewed all the pages of documentation voluntarily provided to us by Metabolife International. We did not independently verify that we received all of the call records held by Metabolife International. We excluded from our review call records that duplicated other records.[Footnote 12] To determine the extent to which consumer information was comprehensive, interpretable, and consistently recorded in the call records, we recorded information about the adverse event, demographic information about the individual consumer, and other details in the call record. Specifically, we recorded background information similar to that used by FDA in the reporting of adverse events, including age, sex, weight, height, the amount of Metabolife 356 used, the duration of use, and whether any medical history was noted in the call record. To assess how many call records reported health-related problems and how many of those were serious, we first counted the number of call records that reported at least one adverse event. Within this set of call records, we then counted the number of reports of heart attack, stroke, seizure, and death--the types of serious adverse events identified in FDA's proposed label warning. We also counted the number of reports of the 23 other types of adverse events that were described as serious or potentially serious in FDA's 1997 proposed rule on dietary supplements containing ephedrine alkaloids.[Footnote 13] For call records that did not report any of the above adverse events, we counted the number of records, but did not count the number of other specific types of adverse events reported. We classified events in the call records based solely on the words and phrases therein; we did not attempt to diagnose a consumer's condition or to otherwise interpret the information presented.[Footnote 14] We did not apply medical judgment in the process of classifying events and we did not independently verify the accuracy of the information in the records or determine the validity of the claims made in the call records. We also did not attempt to determine whether Metabolife 356 caused the reported adverse events. Our results may either overestimate or underestimate the number and severity of adverse events because the call records generally do not include medical diagnoses made by qualified professionals.[Footnote 15] To determine how our counts of reported serious adverse events compare with those of other reviews, we examined the six previous reviews of Metabolife International's call records. In addition, we interviewed Metabolife International and FDA officials. Appendix I describes our methodology in more detail. We conducted our work from September 2002 through March 2003 in accordance with generally accepted government auditing standards. Results in Brief: The information in the Metabolife International call records was limited, sometimes difficult to understand and interpret, and consumer information was not consistently recorded. In some cases, the evidence for a report of an adverse event was limited to a single word on a call record. In addition, most call records did not record complete information about the consumer's age, sex, weight, and height. Information about both the amount of product used and duration of use was recorded for 60 percent of the call records. Further, handwritten call records were difficult to read and understand. Different versions of the call records sometimes contained different information about the consumer and the symptoms they reported. Nearly all of the reports of adverse events that contained information about the amount of Metabolife 356 used and duration of use were for consumers who reported following the usage guidelines on the product label, not for consumers who reported that they took too much Metabolife 356 or used it for too long a period. We categorized 14,684 call records from Metabolife International as containing reports of at least one adverse event associated with Metabolife 356. We found that there were 92 reports of the serious adverse events identified in FDA's proposed label warning for dietary supplements containing ephedrine alkaloids: 18 reported heart attacks, 26 reported strokes, 43 reported seizures, and 5 reported deaths. Among the other adverse events reported that were identified as serious or potentially serious in FDA's 1997 proposed rule, we found, for example, 93 reports of significant elevation of blood pressure, 31 reports of abnormal heart rhythm, 47 reports of loss of consciousness, and 181 reports of systemic rash. Because of the inherent limitations of adverse event reports and the incomplete nature of these call records, we cannot establish that the reported adverse events were caused by the use of Metabolife 356. All of the reviews of the Metabolife International call records, including ours, counted reports of serious adverse events, although none of the reviews reported identical results. For those adverse events that Metabolife International counted--heart attacks, strokes, seizures, deaths, and cardiac arrests--our counts of reported events are similar to the counts from the other reviews. We counted 96 such reported events. Metabolife International counted 78, and the counts of the other reviews ranged from 65 to 107. In commenting on a draft of this report, FDA discussed the value of reports of adverse events in helping to understand the causes of such events. Background: Metabolife 356, which claims to raise the body's metabolism and help dieters lose weight while maintaining high energy levels, contains 32 ingredients, including ephedra, guarana (an herbal source of caffeine), bee pollen, and caffeine.[Footnote 16] The product label recommends that adults take one to two caplets two to three times per day or every 4 hours, not to exceed eight caplets per day. Warnings on the product label suggest that a health care professional be consulted by individuals who are using any other dietary supplement, prescription drug, or over-the-counter drug containing ephedrine alkaloids or who have, or have a family history of, any of 11 health conditions, including heart disease, high blood pressure, diabetes, recurrent headaches, and depression. The label also recommends that persons should not use the product for more than 12 weeks and that exceeding the recommended amount may cause serious adverse health effects including heart attack or stroke. Other possible side effects mentioned on the label include rapid heartbeat, dizziness, severe headache, and shortness of breath. The complete product label is in appendix II. The Dietary Supplement Health and Education Act of 1994 created a framework for FDA's regulation of dietary supplements as part of its oversight of food safety. Dietary supplements are generally marketed without prior FDA review of their safety and effectiveness.[Footnote 17] Manufacturers of dietary supplements are responsible for ensuring the safety of the dietary supplements they sell. Therefore, FDA relies on voluntary reports of adverse events from consumers, health professionals, and others in its effort to oversee the safety of marketed dietary supplements. Although there are no adverse event reporting requirements for manufacturers of dietary supplements, there are such requirements for many other products regulated by FDA. Various types of adverse events associated with the use of human drugs and biologics,[Footnote 18] animal drugs, animal feeds containing animal drugs, medical devices, infant formulas, and radiation-emitting devices must be reported to FDA. In addition to dietary supplements, other products regulated by FDA that do not require adverse event reporting are foods, cosmetics, and color additives. (See app. III for details about adverse event reporting requirements.): Voluntary adverse event reporting systems can be valuable tools for identifying potentially serious health issues that may be associated with the use of a product and for maintaining ongoing surveillance. FDA has used adverse event reports to identify issues for further investigation and, as we previously reported, it has used adverse event reports to help identify dietary supplements for which evidence of harm existed, and has issued warnings and alerts for dietary supplements.[Footnote 19] However, by themselves, adverse event reporting systems generally are not sufficient to establish that a product caused the reported health problem. As we noted in 1999, all voluntary surveillance systems, including FDA's adverse event reporting system, have certain weaknesses.[Footnote 20] These include underreporting, reporting biases, difficulties estimating population exposure, and poor report quality. For example, the Department of Health and Human Services (HHS) Inspector General reported that a study commissioned by FDA estimated that FDA receives reports for less than 1 percent of adverse events associated with dietary supplements. [Footnote 21] In addition, it is often difficult to rule out other possible explanations for the event; for example, the event may have been caused by preexisting medical conditions, or by the concurrent use of prescription drugs, over-the-counter drugs, or other supplements. For these reasons, data from adverse event reports alone cannot be used to determine if the occurrence of a symptom among product users is unusually high. Between August and December 2002, Metabolife International released copies of 15,948 pages of documents that it said contained call records that reported adverse events associated with Metabolife 356 that the company had received from May 1997 through July 2002. Some pages of call records contained information about more than one call while others did not contain reports of adverse events. Some pages were photocopies or duplicates of other pages. Consumer Information in Call Records Was Limited, Sometimes Difficult to Interpret, and Not Consistently Recorded: The information about reported adverse events in the 14,684 health- related call records we examined was limited. Most of the call records we reviewed did not completely record demographic or medical history information about the consumer. Information about age, sex, weight, height, the amount of product used, and the duration of use was frequently not recorded. Handwritten call records were difficult to read and interpret. Information was often inconsistent across different versions of the same call record. The call records contained limited information about reported adverse events and consumers. In some cases the evidence for a report of an adverse event was a single health-related word on the call record, such as "seizure" or "stroke." In addition, demographic and medical history information was not consistently recorded in the call records. Most of the call records we reviewed did not record information about the consumers' sex, age, weight, or height. Eighty-eight percent of the call records did not record at least one of these variables. In addition, information about the amount of Metabolife 356 used and the duration of use was not recorded in 27 and 33 percent of the call records, respectively. (See table 1.) The absence of this information makes it difficult to assess whether the call records represent a signal of health concerns related to the consumption of Metabolife 356.[Footnote 22] Table 1: Percentage of Call Records in Which Consumer and Response Details Were Recorded: Type of detail recorded: Age; Percentage of call records: with information: (n=14,684): 42%. Type of detail recorded: Sex[A]; Percentage of call records: with information: (n=14,684): 41. Type of detail recorded: Weight; Percentage of call records: with information: (n=14,684): 62. Type of detail recorded: Height; Percentage of call records: with information: (n=14,684): 34. Type of detail recorded: Amount of Metabolife used; Percentage of call records: with information: (n=14,684): 73. Type of detail recorded: Duration of use; Percentage of call records: with information: (n=14,684): 67. Type of detail recorded: Medical history; Percentage of call records: with information: (n=14,684): 45. Source: GAO. Note: Analysis of 14,684 health-related call records provided by Metabolife International. Where information was not recorded, we do not know if Metabolife International did not record information in the call records or if the caller did not provide the information. [A] Metabolife International likely has more information about consumers' sex than we did because in many cases the company had access to the names of consumers to help make that determination. Consumers' names had been removed from the records Metabolife International provided us to protect consumer privacy. [End of table] Both the amount of product used and duration of use were recorded for 60 percent of the calls reporting adverse events. Relatively few of these records involved consumers who reported taking too much Metabolife 356 or using it for too long a period. Specifically, among call records containing information on the amount of product used or duration of use, 99 and 91 percent of consumers, respectively, reported using the product within the guidelines recommended on the label. The format of the call records varied from brief handwritten notes to typed notations to printed versions of a form used by Metabolife International. In general, less information was recorded for the one- third of call records that were handwritten than all other types of records. For example, calls recorded on a typed form more frequently recorded additional information such as recommendations by Metabolife International to discontinue Metabolife 356 (62 percent) or contact a doctor (54 percent) than did those on handwritten forms (13 percent and 8 percent, respectively). Further, it was often difficult to read handwritten call records. We could not always determine how many calls were reported on a single page since there was rarely a clear delineation of events. Because handwritten call records did not follow a template, we were unable to determine if some information was medical history or symptom information, or if a number was a weight, heart rate, or blood pressure. Information in call records was sometimes inconsistent. Where duplicate call records were available, information about consumers and their usage of Metabolife 356 was sometimes presented differently in the different records of the same consumer call. In addition, Metabolife International officials told us that its nurses sometimes used several different terms to document the same type of adverse event. Metabolife International Had Thousands of Call Records Reporting Adverse Events Associated with Metabolife 356: We found that 14,684 of the Metabolife International call records reported at least one adverse event. Ninety-two of these were for the serious adverse events identified in the proposed label warning for dietary supplements containing ephedra that FDA announced on February 28, 2003. Other adverse events reported included significant elevation of blood pressure, abnormal heart rhythm, loss of consciousness, and systemic rash. We cannot establish that any of the reported adverse events were caused by the use of Metabolife 356. Reports of Adverse Events Identified as Serious in FDA's Proposed Label Warning: We counted 92 reports of heart attack, seizure, stroke, or death--the serious adverse events identified in FDA's proposed label warning for dietary supplements containing ephedra (see table 2).[Footnote 23] Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke, Seizure, or Death: Type of adverse event: Heart attack; Number[A]: 18. Type of adverse event: Stroke; Number[A]: 26. Type of adverse event: Seizure; Number[A]: 43. Type of adverse event: Death; Number[A]: 5. Source: GAO. Note: Analysis of 14,684 health-related call records provided by Metabolife International. [A] The counts do not represent unique consumers because a single call record may have more than one complaint and because some consumers called the Metabolife health information phone line more than once. [End of table] Other Adverse Events: In its 1997 proposed rule on dietary supplements, FDA also identified other types of adverse events as serious or potentially serious. Table 3 shows our counts for almost all such events.[Footnote 24] The serious and potentially serious types of adverse events described in FDA's June 4, 1997, proposed rule were reported to the agency prior to June 7, 1996. FDA officials report that some other types of adverse events not included in the table may be considered serious or potentially serious but had not been reported to FDA during the time period considered by its proposed rule. Table 3: Metabolife 356 Call Records Reporting Adverse Events Described as Serious or Potentially Serious in FDA's 1997 Proposed Rule: Category of event: Cardiovascular. Event reported: Cardiovascular: Category of event: Chest pain; Number[A]: Cardiovascular: Category of event: 433. Event reported: Cardiovascular: Category of event: Significant elevation in blood pressure[B]; Number[A]: Cardiovascular: Category of event: 93. Event reported: Cardiovascular: Category of event: Abnormal heart rhythm (alternative names for this event include dysrhythmia, ventricular tachycardia, ventricular fibrillation, atrial fibrillation, atrial flutter)[C]; Number[A]: Cardiovascular: Category of event: 31. Event reported: Cardiovascular: Category of event: Cardiomyopathy; Number[A]: Cardiovascular: Category of event: 7. Event reported: Cardiovascular: Category of event: Cardiac arrest; Number[A]: Cardiovascular: Category of event: 4. Event reported: Cardiovascular: Category of eventNervous system: Angina; Number[A]: Cardiovascular: Category of eventNervous system: 3. Category of event: Nervous system. Event reported: Cardiovascular: Category of event: Loss of consciousness; Number[A]: Cardiovascular: Category of event: 47. Event reported: Cardiovascular: Category of event: Psychosis; Number[A]: Cardiovascular: Category of event: 7. Event reported: Cardiovascular: Category of event: Altered consciousness (including disorientation or confusion); Number[A]: Cardiovascular: Category of event: 4. Event reported: Cardiovascular: Category of event: Suicidal; Number[A]: Cardiovascular: Category of event: 3. Event reported: Cardiovascular: Category of event: Vestibular (inner ear) disturbance; Number[A]: Cardiovascular: Category of event: 2. Event reported: Cardiovascular: Category of event: Severe depression; Number[A]: Cardiovascular: Category of event: 2. Event reported: Cardiovascular: Category of eventOther: Mania; Number[A]: Cardiovascular: Category of eventOther: 1. Category of event: Other. Event reported: Cardiovascular: Category of event: Systemic rash; Number[A]: Cardiovascular: Category of event: 181. Event reported: Cardiovascular: Category of event: Urinary infection; Number[A]: Cardiovascular: Category of event: 110. Event reported: Cardiovascular: Category of event: Urinary retention; Number[A]: Cardiovascular: Category of event: 72. Event reported: Cardiovascular: Category of event: Elevations of liver function tests; Number[A]: Cardiovascular: Category of event: 54. Event reported: Cardiovascular: Category of event: Prostatitis; Number[A]: Cardiovascular: Category of event: 24. Event reported: Cardiovascular: Category of eventCardiovascular: Hepatitis; Number[A]: Cardiovascular: Category of eventCardiovascular: 1. Source: GAO. Note: Analysis of 14,684 health-related call records provided by Metabolife International. [A] The counts do not represent unique consumers because a single call record may have more than one complaint and because some consumers called the Metabolife health information phone line more than once. [B] We used the MEDLINE Plus Medical Encyclopedia to define significant elevations in blood pressure as a measurement of greater than 160 millimeters of mercury systolic or 100 millimeters of mercury diastolic. This count does not include call records that mentioned "high blood pressure" or "elevated blood pressure" without specifying these levels. [C] Alternative names for abnormal heart rhythm were determined using the MEDLINE Plus Medical Encyclopedia (www.nlm.nih.gov/medlineplus/ encyclopedia.html) (downloaded December 2002 through February 2003). [End of table] In addition, the 14,684 call records with health-related reports presented a broad range of types of adverse events. Many of the call records contained reports of jitters, insomnia, hair loss, bruising, menstrual irregularities, and sexual dysfunction, as well as vague references to events such as "side effect" or "felt sick." Some reported blood in stool, blood in urine, or blood clots. There were also some reports of visits to emergency departments and hospital admissions. Some call records contained reports of diseases such as pulmonary embolus (a blockage of an artery in the lungs), multiple myeloma, and inflammation of heart tissue. Causal Role of Metabolife 356 Cannot Be Established: We cannot establish that any of the adverse events reported in the Metabolife International call records were caused by the use of Metabolife 356. As we noted earlier, adverse event reports by themselves are generally not sufficient to establish that a health problem was caused by the use of a particular product. For example, for many adverse event reports it is difficult to rule out other possible explanations for the event--the event may have been caused by preexisting medical conditions, or by the concurrent use of prescription drugs, over-the-counter drugs, or other dietary supplements. In addition, the limited information available in the Metabolife International call records means that we cannot confirm that a particular adverse event occurred, much less identify a specific cause for it. Findings of Different Reviews of Metabolife International Call Records Vary: All the reviews of the Metabolife International call records, including ours, counted reports of serious adverse events. None of the reviews reported identical tabulations of these events. For the set of adverse events that Metabolife International counted--heart attack, stroke, seizure, death, and cardiac arrest--our counts are similar to those of the other reviews (see table 4). In total, we counted 96 such events, Metabolife International counted 78, and the counts of the other reviews ranged from 65 to 107. Table 4: Number of Call Records Containing Reports of Heart Attack, Stroke, Seizure, Death, or Cardiac Arrest Reported in Reviews of Metabolife International Call Records: Events: Heart attack; GAO: 18; Metabolife: 16[F]; Karch[A]: 17; Mozayani[B]: 13; Molgaard[C]: 13; Minority Staff, Committee on Government Reform, House of Representatives[D]: 20; RAND[E]: 22. Events: Stroke; GAO: 26; Metabolife: 20; Karch[A]: 24; Mozayani[B]: 19; Molgaard[C]: 13; Minority Staff, Committee on Government Reform, House of Representatives[D]: 24; RAND[E]: 31. Events: Seizure; GAO: 43; Metabolife: 35; Karch[A]: 40; Mozayani[B]: 52; Molgaard[C]: 36; Minority Staff, Committee on Government Reform, House of Representatives[D]: 40; RAND[E]: 46. Events: Death; GAO: 5; Metabolife: 3; Karch[A]: 2; Mozayani[B]: 4; Molgaard[C]: 3; Minority Staff, Committee on Government Reform, House of Representatives[D]: 3; RAND[E]: 5. Events: Cardiac arrest; GAO: 4; Metabolife: 4[F]; Karch[A]: 4; Mozayani[B]: 5; Molgaard[C]: NC; Minority Staff, Committee on Government Reform, House of Representatives[D]: NC; RAND[E]: 3. Events: Total; GAO: 96; Metabolife: 78; Karch[A]: 87; Mozayani[B]: 93; Molgaard[C]: 65; Minority Staff, Committee on Government Reform, House of Representatives[D]: 87; RAND[E]: 107. Source: GAO and others. Notes: "NC" indicates that these types of events were not counted by these reviews. The counts do not represent unique consumers because a single call record may have more than one complaint and because some consumers called the Metabolife health information phone line more than once. [A] Steven B. Karch, An Analysis of Metabolife 356 HealthLine Contacts (August 2002), www.ephedrafacts.com/metabolife.html (downloaded Dec. 12, 2002), and An Analysis of an Additional 3268 HealthLine Records (Jan. 17, 2003). [B] Ashraf Mozayani, Analysis of Metabolife 356 Health Line Reports (August 2002), and Supplemental Report of Analysis of Metabolife 356 Health Line Reports (January 2003). [C] Craig A. Molgaard, Epidemiologic Assessment of Health Line Reports about a Dietary Supplement (August 2002), and An Analysis of Additional HealthLine Records (Jan. 17, 2003). [D] Minority Staff Report, Special Investigations Division, Committee on Government Reform, House of Representatives, Adverse Event Reports from Metabolife (October 2002), www.house.gov/reform/min (downloaded Dec. 11, 2002). This review did not include at least 1,480 pages of call records Metabolife International later made available to us and other reviews. [E] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, Ephedra and Ephedrine for Weight Loss and Athletic Performance Enhancement: Clinical Efficacy and Side Effects. Evidence Report/ Technology Assessment No. 76 (Prepared by Southern California Evidence- based Practice Center, RAND, under Contract No. 290-97-0001, Task Order No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for Healthcare Research and Quality, February 2003. [F] Metabolife International provided to us a list of call records it considered to report cardiac events. Because the other reviews counted heart attacks and cardiac arrests separately, we examined the events that Metabolife International classified as cardiac events to categorize them as cardiac arrest or heart attack. [End of table] There are several possible reasons for the slightly different counts of serious adverse events in the different reviews. First, the call records themselves are often difficult to understand and interpret. Second, not all of the reviews included the same set of call records, both because some were completed before all of the Metabolife International call records were released and because the reviews adopted different procedures for identifying and discarding duplicate records. Third, the reviews used different definitions of particular events or established different thresholds for categorizing a particular event. For example, we included reports of "convulsions" in our count of seizures, while some other reviews may not have. Specifically, the counts we report in table 4 for our review and the reviews by Metabolife International and Karch include reports of convulsions, while it is not clear if the other reviewers' counts did. Similarly, we did not count as a report of a heart attack a call record that reported "heart attack?", while at least one other review did. Summary: The information in the Metabolife International call records was limited, sometimes difficult to understand and interpret, and consumer information was not consistently recorded. Most call records contained only limited information about a consumer and the event being reported, and handwritten records were difficult to read and understand. We categorized 14,684 call records from Metabolife International as containing reports of at least one adverse event associated with Metabolife 356. We found that there were 92 reports of the types of serious adverse events identified in FDA's proposed label warning for dietary supplements containing ephedrine alkaloids. All of the reviews of the Metabolife International call records, including ours, counted reports of serious adverse events, although none of the reviews reported identical results. We counted 96 reports of the types of events counted by Metabolife International--heart attack, stroke, seizure, death, and cardiac arrest--and the counts of the other reviews ranged from 65 to 107. Agency and Metabolife International Comments and Our Evaluation: We provided a draft of this report to FDA and Metabolife International for their review. FDA asked us to clarify that it has not conducted its own review of the Metabolife International call records, that we only reviewed reports of adverse events contained in the Metabolife International call records, and that we did not review other reports of adverse events among users of Metabolife 356 that have been received by FDA. In addition, FDA pointed out that, when combined with other information, adverse event reports can help establish that an adverse event was caused by a particular health product. FDA's comments are included as appendix IV. FDA also provided technical comments, which we incorporated as appropriate. In its comments, Metabolife International was primarily concerned about our use of the term "adverse events" to describe the health-related complaints that were reported in the call records we reviewed. We believe that our use of the term is accurate and consistent with its use by FDA and others in the field. Metabolife International also wanted us to clarify that, while it did identify some call records as containing references to types of specific adverse events that have been categorized as serious by others, it has not identified any call records as reporting "serious adverse events." We have made revisions so as not to imply that Metabolife International labeled these events as serious adverse events. Metabolife International also made other comments, which we incorporated as appropriate. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this report. At that time, we will send copies to the Secretary of HHS, the Commissioner of FDA, and others who are interested. We will also provide copies to others upon request. In addition, the report will be available at no charge on GAO's Web site at http://www.gao.gov. If you or your staff have any questions, please contact me at (202) 512-7119. Another contact and major contributors to this report are listed in appendix V. Sincerely yours, Marcia Crosse Acting Director, Health Care--Public Health and Science Issues: Signed by Marcia Crosse: [End of section] Appendix I: Scope and Methodology for Categorizing the Call Records: We reviewed call records and supplementary information voluntarily provided to us by Metabolife International to (1) determine the extent to which information was comprehensive, interpretable, and consistently recorded in the call records, and (2) count the number of call records reporting health-related problems associated with Metabolife 356, and how many of them were serious. During our review we removed duplicate call records and records that did not report health-related events. For each record we recorded demographic information about the individual consumer, other details about the call record and the consumer, and categorized the reported events. Call Records And Supplementary Information: From August 2002 through December 2002, Metabolife International voluntarily provided to us 15,948 pages[Footnote 25] of documentation relating to reports of adverse events among consumers of Metabolife 356. Most of these records were from calls made to the company's consumer health information phone line from May 1997 through July 2, 2002.[Footnote 26] Other records included e-mail messages and letters that had been sent to the company. Nurses on the staff of Metabolife International documented the calls to the consumer HealthLine in a variety of formats. The records included handwritten notes on a page, typed and handwritten letters, forms with handwritten entries, e-mails, and printed versions of records that had been entered into a database developed by Metabolife International. Many kinds of forms were used to record calls, ranging from simple forms with few spaces or check boxes to full-page forms with multiple boxes for consumer and event-related information. Metabolife International officials told us that health complaints that were noted on product return forms that it received were not in the call records provided to us. Metabolife International also provided to us copies of 46 redacted medical records and a list of corresponding call records. After reviewing these records we found 8 that were not associated with other call records. Five of these records contained enough information to determine the nature of the adverse event and were coded in the same way as other call records. The other medical records were used as additional sources of information for documenting the events and consumer information reported in their corresponding records. While most pages of call records contained information about a single call, some included information about multiple calls on the same page, other calls spanned multiple pages, and some did not include any report of adverse events. Records that spanned multiple pages were often letters to the company, some of which were sent with additional information (such as medical bills). Records that did not report an adverse event were either incomplete printouts of other records from the database, product questions, complaints about not losing weight, or reports of consumer satisfaction. As a result, the number of pages of call records that we received from Metabolife International does not correspond to the number of reports of adverse events. The call records and medical records we received were redacted by Metabolife International to remove personal identifying information such as name, phone number, address, fax number, and e-mail address to protect consumer privacy. Metabolife International officials told us that in the process of redacting the records, some relevant adverse event information was also inadvertently removed. Exclusion of Duplicate and Nonhealth-Related Call Records: Metabolife International officials told us that there were duplicate call records in the set of call records they provided to us. Some duplicate reports were photocopies of the same call record. In other cases, there were multiple versions of the same call record in different formats. Metabolife International officials reported these multiple versions were the result of nurses taking handwritten notes and later entering the same information directly into a database established in September 1999. Metabolife International gave us lists of those call records it believed to be duplicates. Over the course of our review, it identified more than 2,200 records for which there were at least one duplicate. Metabolife International officials reported that they identified the duplicates on the basis of the name of the consumer. Duplicates may have included subsequent calls about different events from the same individual. We examined the duplicate call records identified in the lists provided throughout our review by Metabolife International. Because identifying information was removed, we examined the date of the call record, demographic information about the consumer (such as age, height, weight, the amount of the product used, and duration of use), and event details to determine if they were duplicate records. Where this information was the same or similar, we considered the records to be duplicates and excluded the extra records from our review. We did, however, include in our analysis any additional information that appeared on the duplicate records. For example, if one version included height and another weight, we recorded both of these. We agreed with Metabolife International that most of the more than 2,200 records it identified as duplicates were, in fact, duplicates. However, we did not exclude records that represented multiple calls from the same consumer for different events if the dates on the call records differed by more than a few days or the symptoms were clearly different. During the course of our review, we also identified duplicates not previously identified by Metabolife International, including photocopied records and records that used identical language in event descriptions. We do not know if all duplicate call records were identified. We also excluded from our analysis records in which there was no health complaint or the health complaint could not be clearly determined. We also excluded call records that reported third-hand knowledge of adverse events (such as a friend of a friend who experienced an adverse event). In addition, we did not count call records that clearly referred to nutrition bars or other ephedra-free products manufactured by Metabolife International. In total, we determined that the 15,948 pages of documentation provided by Metabolife International contained 14,684 separate health-related call records. Classification of Records and Data Entry Procedures: We classified the adverse events reported in each call record and entered the appropriate codes into a database. We classified the reported adverse events as either one of the events FDA identified as serious in its February 28, 2003, announcement regarding a proposed label warning for dietary supplements containing ephedra (heart attack, stroke, seizure, or death) or as an other adverse event. All serious events reported within a particular call record were counted. Therefore an individual could have reported multiple serious adverse events, though this happened in few records. For other adverse events, we documented whether the call record reported one or more adverse events. We did not count the number of reports for every type of event reported in the record. We did, however, count the number of all but 1 of the 24 other types of adverse events that were described as serious or potentially serious in FDA's June 4, 1997, proposed rule on dietary supplements containing ephedrine alkaloids.[Footnote 27] The set of events identified by FDA in the proposed rule is not an exhaustive list of the adverse events that may be associated with the use of dietary supplements containing ephedrine alkaloids. FDA officials told us that some other types of adverse events may be considered serious or potentially serious but had not yet been reported to FDA during the time period considered by its proposed rule. We did not apply medical judgment in the process of identifying and classifying events. Our classification of events in the call records was based solely on the words and phrases therein; we did not diagnose a consumer's condition or otherwise interpret the information presented. For example, if a report said "poss. heart attack," "heart attack symptoms," or "heart attack?", we did not classify it as a heart attack since it was not clear that a heart attack was reported. Also, while we counted "blood pressure 210/120" as an instance of significantly elevated blood pressure because it reported measurements greater than 160 systolic or 100 diastolic, we did not place in the same category call records that reported only "high blood pressure" because they did not contain the specific measurements needed for that determination. We used MEDLINE Plus Medical Encyclopedia definitions to further clarify individual symptoms related to these categories.[Footnote 28] We also did not attempt to determine whether Metabolife 356 caused the reported adverse events. [End of section] Appendix II: Metabolife 356 Label: [See PDF for image] Source: Metabolife International, February 12, 2003. [End of section] Appendix III: Requirements for Reporting Adverse Events to FDA: Adverse events about many types of products regulated by FDA are required to be reported to the agency. Such products include human drugs, biologics, animal drugs, animal feeds containing animal drugs, medical devices, infant formulas, and radiation-emitting devices. There are, however, no reporting requirements for adverse events associated with other products regulated by FDA, including food and food additives, dietary supplements, cosmetics, or color additives. (See table 5.): Table 5: Requirements for Reporting Adverse Events to FDA: Product: Human drugs (including over-the-counter drugs) with approved New Drug Application (NDA) or Abbreviated New Drug Application (ANDA)[A]; Adverse events that must be reported to FDA: Serious and unexpected adverse drug experiences from all sources (domestic and foreign).[B]; Who reports: NDA and ANDA applicants, and any person whose name is on the label of an approved drug as manufacturer, packer, or distributor (" nonapplicants").; When reported: As soon as possible but within 15 calendar days. Nonapplicants may, instead, submit reports to applicants within 5 calendar days. Adverse events that must be reported to FDA: New information obtained as result of follow-up investigation of earlier reports.; Who reports: Same as above.; When reported: Within 15 calendar days of receipt of new information or as requested by FDA. Nonapplicants may, instead, submit reports to applicants within 5 calendar days. Adverse events that must be reported to FDA: Adverse experiences that occur domestically and that are serious and expected or not serious (expected or unexpected).; Who reports: NDA and ANDA applicants.; When reported: At quarterly intervals for the first 3 years after approval and then annually or at different times upon written notice by FDA. Adverse events that must be reported to FDA: Serious and unexpected adverse drug experiences described in scientific literature as case reports or as the result of a formal clinical trial, or from or during postmarketing studies where the applicant concludes that there is a reasonable possibility that drug caused reaction.[B]; Who reports: NDA and ANDA applicants and nonapplicants.; When reported: Within 15 calendar days. Product: Human drugs without approved NDAs/ANDAs[C]; Adverse events that must be reported to FDA: Serious and unexpected adverse drug experiences from all sources (domestic and foreign).[B]; Who reports: Any person whose name is on the label as a manufacturer, packer, or distributor; and the manufacturer even if its name does not appear on the label, when it receives adverse drug experience reports directly from a packer or distributor.; When reported: As soon as possible but within 15 calendar days; packers and distributors may, instead, submit reports to manufacturers within 5 calendar days. Adverse events that must be reported to FDA: Serious and unexpected adverse drug experiences from a postmarketing study where there is reasonable possibility that drug caused reaction.[B]; Who reports: Same as above.; When reported: Same as above. Adverse events that must be reported to FDA: New information obtained as result of follow-up investigation of 15-day alert reports.; Who reports: Same as above.; When reported: Within 15 calendar days of obtaining the information or as requested by FDA. Product: Biologics[D]; Adverse events that must be reported to FDA: Serious and unexpected adverse experiences from all sources described in scientific literature, or described in postmarketing clinical studies where there is a reasonable possibility product caused reaction.[B]; Who reports: Licensed manufacturerse and manufacturers, packers, distributors, or other manufacturing participants whose name appears on the label.; When reported: As soon as possible but no later than 15 calendar days. Packers, distributors, and other nonlicensees required to report may submit reports to licensed manufacturers within 5 calendar days. Adverse events that must be reported to FDA: New information obtained as a result of follow-up of 15-day alert reports.; Who reports: Same as above.; When reported: Within 15 days of receipt of new information or as requested by FDA. Packers, distributors, and other unlicensed firms required to report may submit reports to licensees within 5 calendar days. Adverse events that must be reported to FDA: Adverse experiences that are expected or nonserious.; Who reports: Licensed manufacturers.; When reported: At quarterly intervals for the first 3 years after license approval and then annually or at different times upon written notice by FDA. Adverse events that must be reported to FDA: Certain reactions associated with administration of vaccines listed in 42 U.S.C. §300aa- 14.; Who reports: Vaccine manufacturers and health care providers.; When reported: Within 7 days of the administration of listed vaccines or as specified.[F]. Adverse events that must be reported to FDA: Fatality resulting from blood collection or transfusion.[G]; Who reports: Collecting facilities in the event of donor reaction; facilities performing compatibility tests in the event of transfusion reaction.; When reported: As soon as possible by telephone, facsimile, express mail, or electronic transmission with a written report to follow within 7 days. Product: Animal drugs[H]; Adverse events that must be reported to FDA: Unexpected side effects, injury, toxicity, sensitivity, reaction; unexpected incidence or severity, or unusual failure to exhibit expected pharmacological activities.; Who reports: Applicants for New Animal Drug Application (NADA) or Abbreviated New Animal Drug Application (ANADA), including those whose name appears on the labeling as a manufacturer, packer, distributor, or who are engaged in manufacturing, processing, packing, or labeling of drug.; When reported: As soon as possible but within 15 working days of receipt by the applicant. Adverse events that must be reported to FDA: Mix-up in new animal drug or its labeling with another article, bacteriological or significant physical or other change or deterioration in the drug, or failure to meet specifications.; Who reports: Same as above.; When reported: Immediately (generally within 3 days). Product: Animal feeds bearing or containing animal drugs[I]; Adverse events that must be reported to FDA: Mix-up with another drug or its labeling with another article; bacteriological or significant chemical, physical, or other change or deterioration in the drug; or failure to meet specifications.; Who reports: NADA and ANADA applicants.; When reported: Immediately (generally within 3 days). Adverse events that must be reported to FDA: Information concerning unexpected side effect, injury, toxicity, sensitivity reaction, any unexpected incidences or severity, or unusual failure to exhibit expected pharmacological activities.; Who reports: Same as above.; When reported: As soon as possible but within 15 working days of receipt by the applicant. Product: Medical devices[J]; Adverse events that must be reported to FDA: Device-related deaths or serious injuries.; Who reports: Device user facilities.[K]; When reported: Within 10 work days of receiving relevant information; annual reports must summarize all reported events. Adverse events that must be reported to FDA: Device-related deaths or serious injuries.; Who reports: Importers.; When reported: Within 30 days of becoming aware of event. Adverse events that must be reported to FDA: Device-related deaths or serious injuries.[L]; Who reports: Device manufacturers.; When reported: Within 30 days of becoming aware of event, or within 5 days if the event requires remedial action to prevent an unreasonable risk of substantial harm to the public health or if FDA has made a written request. Adverse events that must be reported to FDA: Information that would have had to have been reported earlier but was unknown or unavailable.; Who reports: Same as above.; When reported: Within 1 month of receiving information. Product: Infant formula[M]; Adverse events that must be reported to FDA: Possible causal connection between consumption of an infant formula and infant death.; Who reports: Manufacturers.; When reported: Within 15 days, conduct an investigation and notify FDA. Product: Radiation-emitting devices[N]; Adverse events that must be reported to FDA: Injurious or potentially injurious exposure to radiation from nonmedical electronic products.[O]; Who reports: Manufacturers.; When reported: Immediately. Product: Food and food additives; Adverse events that must be reported to FDA: No requirements to report adverse events.; Who reports: Not applicable.; When reported: Not applicable. Product: Dietary supplements; Adverse events that must be reported to FDA: No requirements to report adverse events.; Who reports: Not applicable.; When reported: Not applicable. Product: Cosmetics; Adverse events that must be reported to FDA: No requirements to report adverse events.; Who reports: Not applicable.; When reported: Not applicable. Product: Color additives; Adverse events that must be reported to FDA: No requirements to report adverse events.; Who reports: Not applicable.; When reported: Not applicable. [A] 21 C.F.R. §§ 314.80, 314.98 (2002). Over-the-counter drugs are subject to FDA's adverse event reporting requirements only to the extent they are covered by approved NDAs or ANDAs. On March 14, 2003, FDA published a proposed rule which includes requirements for reporting suspected adverse events associated with drugs and biological products ("Safety Reporting Requirements for Human Drug and Biological Products," 68 Fed. Reg. 12406). [B] FDA refers to these as 15-day alert reports. [C] 21 C.F.R. § 310.305. Adverse events associated with investigational new drugs are required to be reported under sections 312.32 and 312.33 of Title 21 of the Code of Federal Regulations. Also see FDA's proposed rule at 68 Fed. Reg. 12406 (Mar. 14, 2003). [D] 21 C.F.R. § 600.80. There are no reporting requirements for manufacturers of whole blood or components of whole blood. 21 C.F.R. § 600.80(k)(1). Also see FDA's proposed rule at 68 Fed. Reg. 12406 (Mar. 14, 2003). [E] In vitro diagnostic products are subject to the reporting requirements for devices. 21 C.F.R. § 600.80(k)(2). [F] 42 U.S.C. § 300aa-25(b). [G] 21 C.F.R. § 606.170(b). [H] 21 C.F.R. § 510.300. FDA is in the process of redrafting the adverse event reporting rules for approved animal drugs. [I] 21 C.F.R. § 510.301. Certain medicated items incorporated into animal feeds are also subject to the animal drug reporting requirements. See 21 C.F.R. § 514.80(a)(4). [J] 21 C.F.R. pt. 803. Not all medical device adverse events must be reported to FDA; user facilities are required to report serious injuries to FDA only if the manufacturers are not known. 21 C.F.R. § 803.30(a)(2). Adverse events associated with devices under Investigational Device Exemptions must be reported and summaries must be included in applications submitted to FDA for premarket approval. 21 C.F.R. §§ 812.150, 814.20. [K] Device user facilities do not include physician offices, school nurse offices, and employee health units. 21 C.F.R. § 803.3(f). [L] Manufacturers must also report to FDA if a device has malfunctioned and such malfunction, were it to recur, would be likely to cause or contribute to a death or serious injury. 21 C.F.R. § 803.50(a)(2). [M] 21 C.F.R. § 106.100(k)(3). Manufacturers must promptly report to FDA knowledge about an infant formula it has processed and that has left its establishment if the infant formula may be adulterated or misbranded and that may present a risk to human health. 21 C.F.R. § 106.120(b). [N] 21 C.F.R. §1002.20. [O] 21 C.F.R. §§ 1000.3, 1002.20. Nonmedical electronic products include, for example, microwave ovens and infrared alarm systems. If a product is classified as a medical device, the normal medical device reporting requirements apply. [End of table] [End of section] Appendix IV: Comments from the Food and Drug Administration: DEPARTMENT OF HEALTH & HUMAN SERVICES: Food and Drug Administration Rockville MD 20857: March 20, 2003: Marcia Crosse, Ph.D. Acting Director, Health Care-Public Health and Science Issues United States General Accounting Office 441 G Street, NW: Washington, DC 20548: Dear Dr. Crosse: Please find the enclosed comments from the Food and Drug Administration on the GAO draft report entitled, DIETARY SUPPLEMENTS: Review of Reports of Adverse Events Among Users of Metabolite 356 (GAO-03-494). The Agency provided extensive technical comments directly to your staff. We appreciate the opportunity to review and comment on this draft report before its publication as well as the opportunity to work with your staff in developing this report. Sincerely, Mark B. McClellan, M.D., Ph.D Commissioner of Food and Drugs: Signed for Mark B. McClellan Enclosure: General Comments by the Department of Health and Human Service's Food and Drug Administration (FDA) on General Accounting Office's (GAO) Draft Report DIETARY SUPPLEMENTS Review of Reports of Adverse Events among Users of Metabolife 356 (GAO-03-494): FDA appreciates the opportunity to comment on GAO's draft report which focuses additional attention on the area of adverse event reporting associated with dietary supplements. We have a few general comments regarding the overall report, as follows: The draft report implies that FDA conducted its own review and analyses of the adverse event reports submitted by Metabolife. This is not accurate. There are multiple sets of adverse event reporting systems and databases related to dietary supplements containing ephedrine alkaloids and Metabolife and multiple databases. The GAO report references two different reporting systems (FDA's and Metabolife's) and discusses various interpretations of data subsets from these reporting systems (RAND, Minority House staff, etc.). The draft report is not sufficiently clear about which subset of data was used for this review. FDA encourages GAO to make additional clarifications regarding these systems and databases in the final report. We conclude with our concern about authoritative statements made against the use of adverse events to prove, determine, or establish causality. While it may be true that causality can only rarely be definitively established from a reported adverse event, this does not mean that causality can never be established in an individual adverse event. Aggregated adverse events can not be used to establish risks in a population because this requires more complete and accurate information about the size of population exposed to a particular agent, and the number of individuals experiencing a particular type of adverse event (in exposed and non-exposed persons); for these reasons, aggregated adverse events are used to signal a problem that requires further study. Statements to the effect that "adverse event reports are not sufficient on their own to definitively establish causality" while technically true, are not an adequate reflection of current scientific standards for adverse event assessment. With enough supporting evidence, such as supporting medical documents, dechallenge, rechallenge, temporality, biological plausibility, dose response, etc., a causal association may be determined. [End of section] Appendix V: GAO Contact and Staff Acknowledgments: GAO Contact: Martin T. Gahart, (202) 512-3596: Acknowledgments: Carolyn Feis Korman, Chad Davenport, Julian Klazkin, and Roseanne Price also made major contributions to this report. FOOTNOTES [1] It has been estimated that 12 million Americans consumed dietary supplements with ephedra in 1999 (C. A. Haller and N. L. Benowitz, "Adverse Cardiovascular and Central Nervous System Events Associated with Dietary Supplements Containing Ephedra Alkaloids," The New England Journal of Medicine, vol. 343, no. 25 (2000)). [2] See 68 Fed. Reg. 10417 (Mar. 5, 2003). FDA also announced that it is reopening the comment period for its June 4, 1997, proposed rule, "Dietary Supplements Containing Ephedrine Alkaloids" (62 Fed. Reg. 30678). [3] There is no information available about the extent to which reports of particular adverse events may have been reported to both FDA and Metabolife International. [4] The redaction consisted primarily of the removal of personal identifying information (such as names, phone numbers, addresses, and e-mail addresses) to protect consumer privacy. Although data relevant to the adverse event being reported were not supposed to be removed, Metabolife International officials noted that such information was occasionally accidentally removed. [5] The number of adverse event reports does not equal the pages of documentation because some pages contained reports of more than one call reporting an adverse health event, some reports of adverse health events spanned several pages, and some pages included reports not related to negative health consequences. [6] In addition to phone calls, some call records were letters and e- mails sent to Metabolife International. [7] The letter to FDA is available at http://www.fda.gov/ohrms/dockets/ dockets/98n0148/2.htm (letter from Metabolife International received February 10, 1999) (downloaded March 24, 2003). [8] Metabolife International has not issued a report on its review of the call records, but provided to us a list of the calls it believed to report heart attack, stroke, seizure, death, and cardiac arrest. [9] Each of the consultants reviewed the first set of approximately 12,700 pages of Metabolife International records released in August 2002. Steven B. Karch, An Analysis of Metabolife 356 HealthLine Contacts (August 2002) www.ephedrafacts.com/metabolife.html (downloaded Dec. 12, 2002); Craig A. Molgaard, Epidemiologic Assessment of Health Line Reports about a Dietary Supplement (August 2002); Ashraf Mozayani, Analysis of Metabolife 356 Health Line Reports (August 2002). After more pages of call records were made available, each of the consultants completed updated reviews with these additional reports. Steven B. Karch, An Analysis of an Additional 3268 HealthLine Records (Jan. 17, 2003); Craig A. Molgaard, An Analysis of Additional HealthLine Records (Jan. 17, 2003); Ashraf Mozayani, Supplemental Report of Analysis of Metabolife 356 Health Line Reports (January 2003). [10] Minority Staff Report, Special Investigations Division, Committee on Government Reform, House of Representatives, Adverse Event Reports from Metabolife (October 2002). www.house.gov/reform/min (downloaded Dec. 11, 2002). [11] Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, Ephedra and Ephedrine for Weight Loss and Athletic Performance Enhancement: Clinical Efficacy and Side Effects. Evidence Report/ Technology Assessment No. 76 (Prepared by Southern California Evidence- based Practice Center, RAND, under Contract No. 290-97-0001, Task Order No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for Healthcare Research and Quality, February 2003. [12] Metabolife International officials identified call records they believed were duplicates of each other. We reviewed the relevant call records to determine which were duplicates. Call records identified by Metabolife International officials as duplicates were either photocopies of specific call records, multiple entries of the same call (such as handwritten notes that were later also entered into the database, creating two pages of call records for the same call), or multiple calls about the same consumer describing different events. We considered the first two instances, but not the third, to be duplicates. We did not include in our review reports that we considered duplicates. We also identified additional call records that were duplicates and removed them from our review. [13] FDA's June 4, 1997, proposed rule identified serious or potentially serious adverse events associated with the use of ephedra based on a review of the literature and an analysis of 600 adverse event reports that FDA had received by June 7, 1996. See "Dietary Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678. We did not count reports of one of the events FDA identified, "altered serum enzymes," because the proposed rule did not specify threshold values. [14] We required that certain words be in the call record for it to be counted as a specific type of event. For example, for a call record to meet the criteria for a stroke, it needed to specifically include the word "stroke," not related terms like "stroke-like symptoms." [15] Our findings may either overestimate or underestimate the number and severity of adverse events. Our findings may overestimate the number of adverse events because we accepted the events as they were reported on the page. For example, if a call record reported a stroke, we counted it as a stroke even though the consumer may not have actually had a stroke. Conversely, our findings may underestimate the number and severity of adverse events because individual adverse events we categorized as other adverse events may collectively suggest a more serious event. For example, we categorized a call record reporting left-side numbness and tingling and left-side face drooping as an other adverse event where a physician or other health professional might have determined that these symptoms actually represented a stroke. [16] According to Metabolife International officials, the only ingredient change since Metabolife 356 was placed on the market was made in early 2001, when bovine complex was removed from the product. Some other inactive ingredients may vary by manufacturing facility. Metabolife International officials told us that the same labels are used for products sold in all states. [17] FDA officials reported that the agency conducts a premarket review of safety information for certain supplements that contain new dietary ingredients. [18] Biologics are any virus, therapeutic serum, toxin, antitoxin, vaccine, blood, blood component or derivative, allergenic product, or analogous product applicable to the prevention, treatment, or cure of a human disease or condition. Biological products include, but are not limited to, bacterial and viral vaccines, human blood and plasma and their derivatives, and certain products produced by biotechnology, such as interferons and erythropoietins. [19] U.S. General Accounting Office, Health Products for Seniors: "Anti-Aging" Products Pose Potential for Physical and Economic Harm, GAO-01-1129 (Washington, D.C.: Sep. 7, 2001). [20] U.S. General Accounting Office, Dietary Supplements: Uncertainties in Analyses Underlying FDA's Proposed Rule on Ephedrine Alkaloids, GAO/ HEHS/GGD-99-90 (Washington, D.C.: July 2, 1999). [21] HHS Office of Inspector General, Adverse Event Reporting for Dietary Supplements: An Inadequate Safety Valve, OEI-01-00-00180 (Washington, D.C.: April 2001). [22] We previously reported that adverse event reports should optimally include demographic data (GAO/HEHS/GGD-99-90). Such information is useful for determining whether or not the adverse events reported would be unexpected in a specific population of users, for example, heart attacks in young adults. [23] See 68 Fed. Reg. 10417 (Mar. 5, 2003). [24] We counted all reports of 23 of the 24 other types of adverse events FDA identified as serious or potentially serious in its 1997 proposed rule. We did not count reports of "altered serum enzymes" since the proposed rule did not specify threshold values. The other serious or potentially serious adverse events--coma, myopathies, exfoliative dermatitis, and epididymitis--are not reported in the table because we did not find any reports of them in the call records provided by Metabolife International. [25] These 15,948 pages contained 14,684 call records that we categorized as reporting adverse events. The number of adverse event reports does not equal the pages of documentation because some pages contained reports of more than one call reporting an adverse health event, some reports of adverse health events spanned several pages, and some pages included reports not related to negative health consequences. [26] Metabolife International received the call records we reviewed primarily from mid 1998 through July 2002, although 12 call records were from 1997 and some were from early 1998. [27] FDA's June 4, 1997, proposed rule identified serious or potentially serious adverse events associated with the use of ephedra based on a review of the literature and an analysis of 600 adverse event reports that FDA had received by June 7, 1996. See "Dietary Supplements Containing Ephedrine Alkaloids," 62 Fed. Reg. 30678. [28] www.nlm.nih.gov/medlineplus/encyclopedia.html (downloaded December 2002 through February 2003). GAO's Mission: The General Accounting Office, the investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO's commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of GAO Reports and Testimony: The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO's Web site ( www.gao.gov ) contains abstracts and full-text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as "Today's Reports," on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select "Subscribe to daily E-mail alert for newly released products" under the GAO Reports heading. Order by Mail or Phone: The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548: To order by Phone: Voice: (202) 512-6000: TDD: (202) 512-2537: Fax: (202) 512-6061: To Report Fraud, Waste, and Abuse in Federal Programs: Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm E-mail: fraudnet@gao.gov Automated answering system: (800) 424-5454 or (202) 512-7470: Public Affairs: Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548: