I would like to submit the following comments regarding the three draft
concept papers concerning risk assessment and risk management which were
the subject of the CDER/CBER Risk Management Public Workshop held at the
National Safety Board Boardromm and Conference Center April 9-11, 2003.

My name is David Goldsmith, MD, and I am a board certified Pediatric
Nephrologist with more than 20 years of experience in Drug Safety
Surveillance in the Pharmaceutical Industry.  I have held the position
of chairperson for the PhRMA Clinical Safety Surveillance Committee, and
served on the ICH E2B EWG and as rapporteur for the ICH E2BM EWG. I was
the PhRMA representative to the original MedDRA working party and worked
on the task force that wrote the guidance on MedDRA term selection. 

I am also a founding member of the International Society of
Pharmacoepidemiology, and have served the organization as a board
member, Vice President, Vice President Finance, and Chairperson for the
Scientific Program Committee for 3 of their 19 International
Conferences. the I have been elected to fellowship of the Society and
received their distinguished Service Award. Prior to joining the
industry, I was an Associate Professor of Pediatrics at a Major Medical
School, where I conducted bench and clinical research as well as
providing care to children afflicted with kidney disease.

I have also provided input to the FDA as a member of its task force for
the development of the MedWatch Form (3500 and 3500A forms) and as a
member of the industry advisory group for the AERS development group.

I would like to applaud the FDA on their activities to bring these
important topics into open discussion and on their efforts to develop
the drafts.  However there are key points which I suggest could be
improved and others which may be counter productive and detrimental to
the goal of improving public safety.

These comments are organized to provide general considerations which are
pertinent to all the discussions that took place during the meeting and
to the three topics in the concept papers. These are followed by
comments to specific areas in each of the documents.  Reference is made
to the line numbers of the documents.

In general there is little attention paid to the unintended consequence
of decreased benefit that may occur with any of the risk assessment or
management activities included in the concept papers.  Examples can be
found in each of draft, but it is illustrated here using the
Premarketing Risk Assessment Draft, lines 21-23: specify draft indicates
the need to have as good as possible of underlying risks and benefits
prior to approval and lines 24-26 indicate the need to comprehensively
describe a product's safety.  There is no mention of the fact that an
overly large Premarketing program can be associated with a substantial
risk to patients who might have been able to benefit from a more timely
approval process, and a less costly drug resulting from a smaller
investment requirement and a consequently smaller return on investment..
Risk Management activities described in the second concept paper,
especially at levels 3 and 4 are strictly related to limitation of
inappropriate patient exposure, but no consideration is given to the
fact that an unintended consequence of the programs may well be
limitation of appropriate patient exposure thereby reducing the overall
benefit to the population.

This latter point raises an additional question which may be more
fundamentally political rather than medical.  Much of the concept papers
have a goal of risk reduction by influencing medical practice.  While
educational efforts alone are well recognized and well regarded
activities carried out by pharmaceutical manufactures, in particularly
by means of the product labeling, monitoring the outcome of changes in
the education program or the use restrictive distribution plans are
clearly intended to influence the practice of medicine.  Is this the
remit of the FDA, even if it is in the best interests of the public
health? I do not believe this was the intent of the act or PDUFA III
which authorized postmarketing risk assessment.  But if the agency feels
it necessary or wishes to affect the practice of medicine would it not
be best for the FDA to work directly with practitioners?  Is it
appropriate to require pharmaceutical manufacturers to act as an
intermediary and hold them responsible for the outcome desired by the
agency, even if it is in the best interests of the public health. I
would like to suggest that the FDA and others are more highly regarded
by medical practitioners than is industry, and that the FDA, Medical and
Pharmaceutical Societies, as well as the State Boards of Medicine and
Pharmacy are in a better position to influence physician behavior. This
is also supported in the literature concerning academic detailing. The
FDA needs to provide a better rationale, for placing the burden of
influencing the use of medicines on their manufacturers.

The need to define the appropriate role for each segment of the health
care community in the attempt to influence medical practice, leads to a
final general comment.  The concept papers do not consider the factors
which play a role in influencing practitioner, or patient behavior.
There is an extensive body of evidence and scientific papers in the
medical literature which addresses this, and even a greater amount of
information in the social science literature on behavior modification
and risk perception. Any program which does not examine and incorporate
the findings of these studies carries a substantial likelihood of
causing unintended consequences. It is therefore essential that this
important information be included in the final draft and be at the
foundation for all activities that could be used to maximize the
benefits provided to the population as well as to reduce the risks.  As
indicated above, the logic for assigning appropriate roles to each
segment of the health care community in the attempt to achieve the
outcomes desired needs to be clearly addressed in the concept papers.
The agency should also address how the assignment of roles, influences
the both the public health consequences and promotes the desired
outcome.  Finally the agency should carefully consider how it will
examine each action taken to assess or reduce risk in order to determine
the magnitude of unintended consequences, in particular the loss of
benefits. The agency should further specify the potential results
uncovered by monitoring the consequences of the risk assessment or
management that would be sufficient to indicate at what level of loss of
benefit would it be appropriate to cease the program or lessen the
requirements for risk assessment. 

Specific Comments:
Concept Paper: Premarketing Risk Asssessment

Lines 21 - 26: See general comments paragraph 6, which discusses then
need to include an assessment of the unintended consequences of any
program designed to augment our understanding of risks.  It is
conceivable and possible to have premarketing assessments sufficiently
large and prolonged to understand even very rare risks associated with a
new product.  But not everything is reasonable, and both in terms of
denial of benefits and the costs of delivered benefits would be severely
and negatively impacted by such programs

Lines 30 - 31: Although the concept paper does not address preclinical
safety assessments, it would be advisable to indicate that the
elucidation of the safety profile of a medicinal product should be a
continuum beginning even before animal toxicity studies and continuing
after marketing.

Lines 35 - 36: The elucidation of "drug-drug" interactions as part of a
comprehensive preclinical program seems exceedingly broad and perhaps is
best modified to restrict the activity to "appropriate drug-drug"
interactions.

Lines 79 - 83:  The FDA asked for general guidance on appropriate sizes
of databases.  In response, it is worthwhile to suggest that at this
stage it is appropriate to begin to factor in the anticipated benefits
both in terms of frequency and magnitude as well as the size of the
patient population likely to reap the benefits.  These weights of the
benefits could be established in much the same manner as is done with
quality of life, health outcomes, and health economic research.
Similarly, the frequency and magnitude of the potential adverse effects
of concern to be associated with exposure need to be weighted.
Categories of benefit (e.g. high and frequent in a large population)
would be balanced with the nature of the adverse reactions considered
(e.g. serious, and frequent).  The resulting categorization would allow
for various sizes of the population to be studied depending on the
anticipated benefit-risk balance including the duration of exposure.
The ICH recommendations concerning the size of the database are helpful
and should be considered along with the anticipated weighted benefits
and risks.

Lines 134 - 136: The FDA suggests considerations additional to those of
the ICH when a very safe alternative to the investigational product is
already available or when there is the potential for rapid exposure to a
large population.  While in some circumstances raising the bar by
requiring a larger than usual database might be wise, it is also likely
that there will be others that would have a negative impact on the
public health.  Safer or more efficacious alternative therapies should
not be restrained in the approval process.  The consideration of
frequency, magnitude, and population size benefiting from the medicinal
product needs to be factored into the analysis.

Lines 145 - 157: The agency asserts that it would be preferable to have
controlled safety data, including long-term safety data, to allow for
comparisons of event rates and for accurate attribution of adverse
events.  However, the agency does not provide the reasoning used to
arrive at this preference.  An alternative preference would be to
balance the need for controlled studies with special consideration of
the potential unintended consequences of lack of availability of an
effective therapeutic agent and the potential for a higher cost, which
could be prohibitive for some needing the treatment, against the need
for control groups.

Lines 160 - 168: The FDA needs to consider the negative impact that will
result by increasing variability and thereby making it more difficult to
establish statistical significance for the demonstration of benefit.
Prolonging the preapproval study period will deprive patients of needed
therapy and add to the cost of the final approved product.  A balance
needs to be maintained between the desired safety analysis and the
desired benefits of a new medicinal compound.

Lines 178-80:  The concept paper suggests that using a range of doses in
phase 3 trials would better characterize the relationship between
exposure and the resulting clinical benefit and risk, allowing provision
of the best dosing advice.  It should be noted however that undertaking
such a program will likely result in multiple benefit risk
relationships.  It is difficult enough to evaluate a single benefit risk
relationship (see CIOMS V), and making it comparative with the same drug
would only add to the difficulty.  The FDA should reconsider their
suggestions concerning a range of doses until there are more formal,
objective and better means to evaluate benefit risk relationships.

Lines 202 - 205: While product-food interactions may well have an impact
on ADME (absorption, distribution, metabolism, and elimination) these
are all likely to increase or decrease the bioavailability and hence the
known pharmacologic effects of the product.  The FDA should explain why
they suggest that new unusual and unexpected reactions are likely to be
uncovered with this aspect of the suggested risk assessment plan.

Lines 207 - 209: It will be extraordinarily difficult to anticipate the
likelihood of consumer exposure to dietary supplements.  These agents
are not prescribed and are often taken without a sound rationale.

Lines 235 - 236: If there is a need to characterize background rates of
certain adverse events in order to adequately assess the product, a well
designed epidemiologic study can answer this question more rapidly and
at a much lower cost.  The guidance should provide the framework for
determining when a comparative safety study would be preferable to the
epidemiologic study and what are the anticipated costs.

Lines 242 - 244: The recommendation that a comparative safety study
would be useful when there is a well-establish, well-characterized
product with minimal toxicity to treat the condition of interest is
problematic since in effect it raises the bar for a possibly more
effective and safer product.  When the risks are rare with reactions
occurring in 1:1000 or 1:10,000 the number of patients that would be
needed to demonstrate equivalency would make such a study prohibitively
expensive and time consuming.

Lines 260 - 264: The suggestion that for chronically used products, or
for those with a very long half-life, an examination of whether a
maintenance dose lower than the initial dose or decreases in dosing
frequency would be appropriate, overlooks that selection of dose should
be based on solid pharmacodynamics.  If a loading dose is needed to
increase the rapidity of onset this should be part of the
pharmacokinetic and pharmacodynamic evaluation.

Lines 269 - 272: The draft guidance, when issued, should indicate when
it is appropriate to perform an assessment of less obvious adverse
effects that might not be detected or readily reported by patients.

Lines 274 - 277: The FDA should take into consideration that school
entry requirements for immunization are likely to be variable from state
to state.  While the goal is laudable it may be practically impossible
to achieve it.

Lines 279-299: Large, simple safety studies can be conducted prior to
approval, however the endpoints provided as examples in the concept
paper (QT prolongation which would require holter monitoring [indeed for
an unspecified length of time] unless surrogate end points are used, and
hepatotoxicity requiring laboratory examination and possibly the
designation of a central laboratory) are not simple.  The draft guidance
should use simple outcomes as examples.  In addition the use of LSSS to
assess background risks would only be necessary if this could not be
answered by an effective well designed epidemiologic study that could
provide answers in a more timely and cost effective manner.

Lines 300 - 309: The concept paper suggests that sponsors could consider
preserving blood (or other body fluids) samples for retrospective
testing at some later point.  The draft guidance will need to include
the types of preservatives and the conditions of storage so that
sponsors will be able to limit the preservation requirements to a cost
effective number of samples.

Lines 318 - 322: The draft guidance should include the methods
recommended by the FDA to identify known and potential medication error
modalities and the other three modalities suggested in the concept paper
for medication error prevention analysis.

Lines 421 -426: The concept paper suggests that events which decrease in
frequency over time be subjected to time-to-event analysis to identify
possible mechanisms for the observed phenomenon.  Depletion of
susceptibles should be included in the draft guidance as another
potential explanatory phenomenon.

Lines 431 - 440: The recommendation to consider analyses of event rates
and severity by dose should be tempered by providing the statistical
caveats concerning multiple looks at the data, and the robustness of
findings uncovered in multiple regression analyses that were not
predefined in the analytical plan.

Lines 495 - 500: The concept paper suggests that "when the results of a
pooled analysis show a diminished statistical association and/or less
risk compared to the safety signal originally obtained from one or more
of the contribution trials, it could suggest inappropriate use of data
pooling." The FDA needs to make clear in the draft guidance that it
could also be a primary tool used to refute a false positive signal.  It
is to be recalled that signals are precisely that.  They are not facts
but only suggestions that something might underlie an observation not
that the observation establishes a causal association or even a
statistically demonstrated fact.

Concept Paper: Risk Management Programs

Lines 25 - 26: The concept paper acknowledges that no medicinal product
is risk free, and suggests that a product is considered safe if the
benefit risk balance is positive both for populations and individuals.
This concept should be re-examined with attention being paid to the fact
that a product may have an enormously positive benefit risk balance at
the level of a population and be very negative for a specific
individual.  An example might be penicillin which for certain
individuals could provoke a fatal anaphylactic shock.  Indeed the
concept of a contraindication underscores that the benefit risk balance
must be assessed at the population level and that individual patients
need to be considered separately. 

Lines 28 - 32: As indicated in the general comments the agency should
also consider that an risk management concepts include optimizing
benefits which might be achieved by early approvals for a large
population needing therapy.

Lines 64 - 67: While the three examples cited all can be effective
sponsor initiated activities to minimize some risks, the draft guidance
should place these sponsors' efforts into the larger framework of
reducing the risks associated with pharmaceuticals. Studies have clearly
shown that adverse effects of drug are preventable errors and are not
due to lack of knowledge or unknown intrinsic harmful effects.  A good
example is bleeding associated with heparin therapy.
Line 85 - 99: The concept paper at this point clearly specifies that the
goal of an RMP is to alter the practice of medicine.  As indicated in
the general section, the FDA should provide the rationale for putting
this burden on sponsors rather than other players in the health delivery
systems in the US including State Boards of Pharmacy and Medicine.  In
addition some of the recommendations such as having pharmacist confirm
that patients with product B prescription do not have condition A will
be impacted by HIPAA, and do not necessarily preclude that the patient
with condition A will not receive product B.  It should also be
recognized that the benefit risk assessment at an individual level is
probably best made by the patient and the health care provider.


Lines 107 - 122: The agency should be lauded for recognizing that
currently there is no ready formula to determine when risks exceed
benefits.  This concept needs to be further highlighted and expanded
upon to address the unintended consequences of any risk management plan
or program.  In addition the FDA should provide more guidance on how
sponsors and the agency are expected to determine when the number or
severity of a product's risks appears to undermine the magnitude of its
benefits.

Lines 132 - 133: The concept paper asserts that professional product
labeling is an important tool used to communicate risks and benefits.
This statement, if included in the guidance, should be supported by
citations in the literature.  It should also be balanced by the studies
indicating the lack of behavioral changes after changes in the insert
and even "Dear Healthcare Provider" letters.

Lines 136 - 213: While the objectives indicated in this section are
laudable, the guidance should include better consideration of the
methods to achieve them.  For example is it in the purview of the FDA or
sponsors to have physicians tested and certified that they possess
appropriate knowledge?  Have the agencies actions to limit prescribing
ability to certain physicians for specific products been tested legally?
Is the agency using the threat of non approval or removal of approval to
require sponsors restrict distribution and thereby alter the practice of
medicine?  Have the limited supply or refills programs resulted in a
negative impact on appropriate use?  In this regard one should recall
the untoward consequences of the 3 part prescription requirement for
benzodiazepines in New York State.  The agency should also discuss the
roles of other members of the healthcare community in enhancing the
benefit risk balance, and consider the effect of HIPPA on each of the
recommended tools.

Lines 256 - 261: The comments above concerning HIPAA and efforts to
influence the practice of medicine clearly apply to the choice of tools
at Levels 3 and 4.  The guidance should provide sponsors with the legal
basis for their actions and the potential legal consequences.

Lines 306 -308: The agency should be praised for including in the
concept paper the precise ultimate goal to ensure that the risk
management program achieves a positive benefit risk balance in a cost
effective manner.  The agency should also include that the program's
plan should include definition of the positive risk benefit balance
prior to initiating the program so that efforts can be appropriately
monitored without slippage of the endpoints.

Lines 326 - 333: The agency makes clear in the concept paper that the
use of two different evaluation methods is not always feasible and that
suggests other quantitative methods to confirm the evaluation in such
circumstances.  The agency should include some discussion of power
calculation to be used in these instances.

Lines 368 - 372: The concept paper appropriately suggests that
spontaneous reporting of adverse events are influenced by many factors
and that a decrease in reporting rate does not constitute assurance that
a safety issue has been resolved.  However in the same paragraph the
agency takes a very inconsistent approach suggesting that continuing
reports of adverse events may signal a persistent safety problem.  As
phrased in the concept paper both statements can be defended because the
wording includes "may signal a safety problem" and "does not constitute
assurance" of its resolution.  The agency should make clear that changes
in spontaneous reports, or the lack thereof, should not be used either
to signal the persistence or resolution of the safety problem.

Lines 378 - 379: The draft guidance should be balanced with regard to
inclusion and exclusion of high risks groups.  Medicaid data sets
included a higher proportion of the high risk group specified in this
paragraph.

Lines 389 - 394: It would be useful for the agency to include statements
concerning the interpretation of expected results with both best and
worst case scenarios.  The plan should predefine how the results will be
interpreted.

Lines 416 - 424: Sponsors should not only be requested to specify the
findings that would escalate an RMP but also to reduce its level.  The
agency should specify in the guideline if it believes that the level of
the RMP even if effective should never be reduced.
Concept Paper: Risk Assessment of Observational Data: Good
Pharmacovigilance Practices and Pharmacoepidemiologic Assessment.

Lines 55 - 60: The concept paper defines a plan as an ongoing evalution
of identified safety signals through enhanced pharmacovilance practices.
The agency needs to specify what the enhancements might be.  The three
examples currently provided are all currently in effect.  The ICH E2A
guidance and the current as well as the proposed new regulations include
the possiblily of reporting certain serious events in an expedited
manner. There are already in place active surveillance activities and
most sponsors conduct additional observations studies or clinical trials
after approval.

Lines 75 - 77: The draft guidance should specify what is meant by
complete data.  The proposed new regulations specify all the
"appropriate" information on the 3500A form.  However even this
definition does not adequate specify that it should provide clearly and
concisely all the information that would normally be included in a case
presentation for medical rounds.  Obviously the extent of the data
needed depends on the nature of the case, and not necessarily on the
information requested on a form. In addition line 100 bullet 5 "Method
of Diagnosis of the event" needs to be clarified.

Lines 109 - 110: The guidance should include a statement that while each
of the specified item in bullet 1 may be useful, they are not all always
pertinent or necessary.

Lines 115 - 117: The guidance should provide the web site for NCC MERP
(http://www.nccmerp.org/taxo0514.pdf).  In addition the agency should
provide the rationale and cost benefit analysis for using an 18 page
taxonomy for which neither the AERS database nor most homegrown or
commercial databases are designed to handle.

Lines 121 - 130: The guidance needs to be augmented as compared to the
concept paper to include appropriate exclusion of cases which are
irrelevant.  While it is necessary to facilitate retrieval of all
clinically relevant cases from a database it is also necessary to
facilitate final review of only the clinically relevant cases.  The
precision of the query needs to address both the selectivity and the
specificity of the results.  The efforts should be directed towards
maximizing the positive predictive value and the efficiency of the
query. The concept paper also specifies that data mining techniques may
be applied to the database to identify relevant cases but it does not
provide any basis for this assertion.  Data mining as applied to
pharmacovigilance has not been rigorously tested using statistical
techniques to confirm that it is both sensitive and specific.  In the
often cited work of DuMouchel it is clearly pointed out that there was a
collapse of over 7000 drug codes into only 1398 used in the final
analysis ("with no guarantee that every such equivalence has been
identified", and that only 952 event codes were used where as MedDRA
(version 5.1) has 16,102 preferred terms.  In addition information
concerning the sensitivity and specificity of the data mining procedures
is lacking. Thus, it would be appropriate for the agency to provide such
data and the methods of data mining so tested to ensure selection of the
right methodologies.  Does the agency believe that methods described by
DuMouchel, the WHO, and the CSM all have equal validity?  Finally the
agency needs to consider the effect of MedDRA on the ability to conduct
data mining.  For example a some patients with anaphylaxis will be
described as such, some will be described with the term anaphylactoid
reation, others will be described only with their symptoms.  Data mining
using MedDRA preferred terms is likely to miss a signal based on the
inconsistent coding especially given the requirement to code according
to the initial reporter's verbatim.

Lines 140 - 152: The guidance needs to point out that for rare events
controlled clinical trials is not feasible and limit the areas which
pharmacoepidemiologic studies are applicable to just a few examples.
The guidance should also discuss the problem that study designs to
uncover rare events in populations that are rarely exposed are not yet
adequately defined.

Line 180: The guidance should provide information concerning the HIPAA
and its relationship to the ability to use these claims databases.

Lines 269 -286: The points raised in the concept paper should be
expanded in the draft guidance which should clearly indicate that there
is no direct measure of for the size of the population at risk for an
event and that estimates of this important denominator may over or
underestimate it by orders of magnitude.  The agency should also refrain
from using such estimates.  Despite the limitations the concept paper
asserts that there is value of comparisons of reporting rates across
similar product for the same indication.  The agency would be well
advised to review the published materials which clearly demonstrate
significant differences in reporting for the same product in different
countries or by different companies. Comparisons of this type are
fraught with the possibility of arriving at erroneous conclusions.

Lines 452 - 455: The concept paper indicates that the FDA may decide to
take interim regulatory actions to communicate information about safety
signals.  It would be appropriate for the agency to provide information
concerning how it will evaluate the benefit risk balance when the risk
is not quantified and the consequences of actions on benefits and
unintended consequences remain unknown.

Respectfully Submitted
David I. Goldsmith, MD
President, Senior Consultant

Goldsmith Pharmacovigilance & Systems 
139 East 63rd Street
New York, NY 10021
Tel: 917-415-7357 or 212-688-7556
Fax: 212-750-0988
email: david_goldsmith@alum.wustl.edu