Department of Health and Human Services s19

1 1 1 UNITED STATES OF AMERICA 2 DEPARTMENT OF HEALTH AND HUMAN SERVICES 3 FOOD AND DRUG ADMINISTRATION 4 5 + + + + + 6 7 RISK MANAGEMENT PUBLIC WORKSHOP 8 PHARMACOVIGILANCE PRACTICES AND 9 PHARMACOEPIDEIOLOGIC ASSESSMENT 10 11 + + + + + 12 13 FRIDAY, 14 APRIL 11, 2003 15 16 + + + + + 17 18 The Workshop was called to order at 8:07 19 a.m., in the NTSB Board Room and Conference Center, at 20 429 L'Enfant Plaza, S.W., Washington, D.C., by Steven 21 Galson, Deputy Director, CDER, presiding. 22 23 PRESENT: 24 25 DR. STEVEN GALSON Deputy Director, CDER 26 DR. MARK MCCLELLAN Commissioner FDA 27 DR. SUSAN ELLENBERG CBER 28 DR. JULIE BEITZ CDER 29 DR. MILES BRAUN CBER 30 DR. MIN CHEN CDER 31 MS. AILEEN CIAMPA CDER 32 DR. EDWARD COX CDER 33 PATRICIA GUINN CDER 34 DR. VICTOR RACZKOWSKI CDER 35 DR. PAUL SELIGMAN CDER 36 DR. JUDY STAFFA CDER 37 DR. JOYCE WEAVER CDER 38 DR. CAROL HOLOQUIST CDER2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2 1 PUBLIC COMMENT: 2 3 DR. GERALD FAICH Pharmaceutical Safety 4 Assessments 5 DR. JOHN FERGUSON Biotechnology Industry 6 Organization 7 GREGORY GOGATES CRF Box, Inc. 8 DR. STEPHEN GOLDMAN Stephen Goldman Consulting 9 Services, LLC 10 LINDA HOSTELLEY Merck, PhRMA 11 DR. SIDNEY KAHN Pharmacovigilance & Risk 12 Management, Inc. 13 DR. ROBERT C. NELSON RCN Associates 14 DR. ANNETTE STEMHAGEN ISPE Membership Committee 15 DR WENDY STEPHENSON PhRMA 16 DR. ANSHU VASHISHTHA Watson Pharmaceuticals2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 3 1 C-O-N-T-E-N-T-S 2 3 Welcome and Introductions...... 4 4 5 Session I - Overview and Good Pharmacovigilence 6 Practices 7 FDA Presentation by Dr. Julie Beitz...... 8 8 FDA Presentation by Dr. Min Chen...... 14 9 10 Oral Presentations - Public Comment 11 Dr. Robert C. Nelson...... 21 12 Dr. Stephen Goldman...... 36 13 Dr. Gerald Faich...... 53 14 15 FDA Presentation by Dr. Mark McClellan...... 67 16 17 Oral Presentations - Public Comment 18 Dr. Anshu Vashishtha...... 92 19 Mr. Gregory Gogates...... 102 20 Ms. Linda Hostelley...... 111 21 22 Panel Discussion...... 120 23 24 Session II - Pharmacoepidemiologic Assessment 25 FDA Presentation by Dr. Judy Staffa...... 157 26 27 Oral Presentations - Public Comment 28 Dr. Annette Stemhagen...... 172 29 Mr. John Ferguson...... 186 30 Dr. Wendy Stephenson...... 193 31 Dr. Sidney Kahn...... 201 32 33 Panel Discussion and Q&As...... 212 34 35 Closing Remarks by Dr. Julie Beitz...... 256 36 37 Adjournment...... 2582 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 41 P-R-O-C-E-E-D-I-N-G-S2 (8:07 a.m.)3 DR. GALSON: Welcome. I hope to have a4 good group today, and I am sure that we will have more5 people filtering in as time goes on. I wanted to6 welcome you to the third day of our Risk Management7 Public Workshop, and what I am going to do first is8 just ask the FDA staff who are here to quickly9 introduce themselves. 10 And first up at the table here, if you11 folks would just sequentially push your buttons and12 say who you are and what you do.13 DR. MILES BRAUN: My name is Miles Braun,14 and I am the Director of the Division of Epidemiology15 in the Center for Biologics at FDA.16 DR. SELIGMAN: Good morning. I am Paul17 Seligman, and I am the Director of the Office of18 Pharmacoepidemiology and Statistical Science in the19 Center for Drugs at the FDA.20 DR. BEITZ: I am Julie Beitz, and I am the21 Deputy of the Office of Drug Evaluation III.22 DR. COX: And I am Ed Cox, Deputy23 Director, Office of Drug Evaluation IV, CDER, FDA.24 DR. GALSON: Okay. And then we have got25 at the table at my left is the working group that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 51 focused on this concept paper that we are discussing2 today, and if we could just go down the table.3 DR. WEAVER: Joyce Weaver, Office of Drug4 Safety5 DR. HOLQUIST: Carol Holquist, Office of6 Drug Safety.7 DR. CHEN: Min Chen, Office of Drug8 Safety.9 DR. STAFFA: Judy Staffa, Office of Drug10 Safety.11 DR. GUINN: Patrick Guinn, Office of Drug12 Safety.13 DR. GALSON: And that is Lee Lemley, and14 thank you, Lee, for doing all the hard work to15 organize this 3 day meeting. All right. First, I16 want to thank the members of the CDER/CBER Executive17 Oversight Committee, none of whom are sitting here,18 but I think we will have some a little bit later.19 Going quickly over this, particularly for20 those of you who may be here for the first day today,21 we have had -- this is our third day of meetings to22 discuss three concept papers that corresponded to23 three groups working in this area, and today's meeting24 is focused on pharmacovigilance.25 MS. LEMLEY: We will be with you in a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 61 minute.2 (Brief Pause.)3 DR. GALSON: So, let me go over the4 schedule for today. We are in the introductions now,5 and then we are going to be split up into two general6 areas today. The first session this morning is7 overview and good pharmacovigilance practices.8 We are going to have a number of FDA9 presentations, and then oral presentations from the10 folks sitting at the table in front. We will then11 have a break at 9:45, and Dr. Mark McClellan, the12 Commissioner of the FDA, will be here at -- we should13 have some time for him to ask and to respond to14 questions from the audience.15 Then we will continue with oral16 presentations and have a discussion among the people17 who spoke, and questions and answers from the18 audience. At 11:30, we will have lunch, and then we19 will start on the second session,20 pharmacoepidemiologic assessment, and FDA's21 presentation, and then presentations from speakers,22 and then questions and answers from the audience, and23 then some closing remarks at the end.24 Some logistics. We are going to25 distribute cards. If you don't want to go up to the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 71 microphone and you just want to write down your2 questions on the cards, hand them to the people who3 will be in the aisles, and we will bring them up and4 we will open mikes.5 The transcripts from these sessions will6 be available within 30 days on the website you see,7 and as well, we have got a general website for the8 meeting, and we are going to try to post all of the9 slide presentations that you are seeing today and for10 the last few days on that website address. So look11 for that if you are interested in copies of the12 slides.13 No food or drinks are allowed in here.14 The restroom is out in the lobby, and please note the15 fire exits from the room. Thank you very much.16 Before we move on with the program, I just want to17 have the speakers for this morning start, and those of18 you that are sitting at the table, just introduce who19 you are, and we will get around to each of you. Just20 real quickly on the mike.21 DR. NELSON: My name is Bob Nelson, and I22 am a consultant in Drug Safety and Regulatory Affairs.23 DR. FAICH: I am Gerry Faich, and I am a24 second consultant in Safety and Regulatory Affairs.25 DR. GOLDMAN: Steve Goldman, and I am the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 81 third consultant in Safety and Regulatory Affairs.2 DR. GALSON: Okay. No shortage of3 consultants. 4 DR. NELSON: The order means only5 administrative order.6 DR. GALSON: Okay. I am now going to turn7 the mike over to Dr. Julie Beitz, who has done a great8 job managing the work involved in creating this third9 concept paper. Julie.10 DR. BEITZ: Good morning, and welcome to11 day three of the CDER/CBER Risk Management Public12 Workshop. Today, we will be discussing the concept13 paper entitled, "Risk Assessment of Observational14 Data."15 This paper was written in a general way to16 stimulate discussion at this workshop regarding what17 constitutes good pharmacovigilance practices and good18 pharmacoepidemiologic assessment. 19 Comments that we receive today and in the20 docket will be considered as we prepare a longer and21 more comprehensive draft guidance document in the22 coming weeks and months. 23 The CDER/CBER pharmacovigilance working24 group represents a broad spectrum of talented25 individuals at FDA who came together to produce a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 91 concept paper in very short order. Drs. Mark2 Goldberger and Miles Braun, served as group leads for3 CDER's Office of New Drugs, and for CBER.4 Our support staff, notably Aileen Ciampa,5 provided regulatory input; and our project manager,6 Patrick Guinn, helped with scheduling meetings and7 organizing today's workshop. Many group members are8 in attendance today, and you will have an opportunity9 to interact with them. 10 Dr. Ed Cox will be filling in for Dr.11 Goldberger, who could not be with us today. You have12 already seen the agenda presented, and I will just13 point out that we are currently in the overview for14 the morning session. 15 The concept paper under discussion today16 encompasses the following: Important17 pharmacovigilance concepts; safety signal18 identification; pharmacoepidemiologic assessment and19 interpretation of safety signals, and a development of20 pharmacovigilance plans.21 Today's discussion will focus on risk22 assessment based on observational data sources,23 including case reports, case series,24 pharmacoepidemiologic studies, registries, and25 surveys. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 101 Risk assessment, in the context of2 clinical trials, was discussed on the first day of3 this workshop. First, let me begin by defining the4 concept of pharmacovigilance. As we have stated in5 the concept paper, pharmacovigilance includes all6 post-approval, scientific, and data gathering7 activities relating to the detection, assessment,8 understanding, and prevention of adverse effects, or9 other product related problems.10 This includes the use of11 pharmacoepidemiologic studies. Risk assessment occurs12 throughout a product's life cycle. At the time of13 approval, available clinical trial data involve14 limited numbers of patients treated for relatively15 short periods of time.16 In drug approval, large numbers of17 patients may be exposed to a product, including18 patients with co-morbid illnesses, patients using a19 variety of concomitant medications, and patients using20 the product chronically.21 After approval, new safety information may22 become available from a variety of sources through use23 of the product either domestically or in other24 countries, through use of other drugs in the same25 class, from preclinical or pharmacologic studies of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 111 the product, or from controlled clinical trials.2 Many definitions have been put forth to3 characterize a safety signal. For the purpose of4 today's discussion, we have defined a signal as an5 apparent excess of adverse events associated with the6 use of a product.7 Small numbers of well-documented case8 reports, even a single case report, may be viewed as a9 signal. In addition, we believe that pre-clinical10 findings or experience with other products in the same11 class, may be sufficient to generate a safety signal12 even in the absence of case reports on patients.13 Thus, a products risk profile may be14 characterized by several safety signals. Throughout15 the concept paper, we have tried to clarify the16 following activities as they relate to safety signals.17 We say that signals are identified from18 case reports or other sources. That signals are19 evaluated in pharmacoepidemiologic studies,20 registries, or surveys. That signals are interpreted21 in the context of all available safety information,22 and that signals are monitored through enhanced23 pharmacovigilance efforts.24 This paper also introduces the concept of25 a pharmacovigilance plan. This would be a plan2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 121 submitted by a sponsor for the ongoing evaluation of2 safety signals identified by the use of a product.3 The plan could also include monitoring for4 at risk populations which have not been adequately5 studied to date. The plan may be developed at the6 time of product launch, or after a signal is7 identified.8 For a product without safety signals9 identified pre-or-post approval, and for which at-risk10 populations have been adequately studied, routine11 post-marketing reporting may suffice as a12 pharmacovigilance plan.13 For a product with safety signals14 identified pre-or-post approval, or for which at risk15 populations have not been adequately studied, a16 collection of additional safety information beyond17 routine post-marketing reporting may be desired.18 Examples of the kinds of activities that19 could constitute a sponsored pharmacovigilance plan20 include expedited reporting of serious adverse events21 of interest, submission of adverse event report22 summaries at more frequent pre-specified intervals.23 For example, on a quarterly basis rather24 than annually; conduct additional observational25 studies or control trials; implementation of active2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 131 surveillance activities to identify as yet unreported2 adverse events. 3 We recognize that many of these activities4 are already being carried out by sponsors, even though5 the term, pharmacovigilance plan, has not yet been6 applied to them. 7 Active surveillance efforts may involve8 data gathering activities that are focused on specific9 products of interest on specific health care settings,10 such as emergency departments, or on specific events11 that are often considered to be drug related. 12 FDA looks forward to hearing public13 comment on active surveillance strategies using14 electronic health information systems or other15 databases. In particular, we would like to hear your16 thoughts regarding the circumstances for which these17 efforts would prove most useful.18 We recognize that emerging new safety data19 may result in ongoing revisions to the sponsor's20 pharmacovigilance plan for a product. 21 While additional safety information is22 being generated, the FDA will work with sponsors to23 communicate information about safety signals, and24 minimize events occurring in the users of a product25 through risk management programs as appropriate.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 141 This concludes my presentation. We will2 now move on to the next FDA presentation on Good3 Pharmacovigilance Practices.4 DR. GALSON: Thank you, Julie, and our5 next speaker is Min Chen, who is the Associate6 Director of the Division of Drug Risk Evaluation in7 our Office of Drug Safety.8 DR. CHEN: Good morning. I am here to9 talk about Good Pharmacovigilance Practices and our10 current thinking at the FDA. As Dr. Beitz mentioned,11 the pharmacovigilance definition is generally regarded12 as all post-approval scientific and data gathering13 activities relating to the detection, assessment,14 understanding, and prevention of adverse events, or15 any other product related problems.16 This includes the use of17 pharmacoepidemiological studies, too. We think that18 good pharmacovigilance practice starts by acquiring19 complete data from spontaneous adverse events or20 reports. It is critical for sponsors to actively seek21 information on an adverse event by direct contact with22 the initial reporter so that a thorough assessment of23 the event can be made expeditiously.24 Good reporting practices are available in25 several guidances that we have. We are looking for2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 151 the following information in a good case report,2 whether reported spontaneously or published in the3 medical literature.4 The adverse event details, description,5 patient's baseline characteristics, and concomitant6 drug therapy details, time to onset of the signs or7 symptoms, diagnosis of the event which hopefully can8 be supported by the lab data stated here, and the9 clinical course of the event, and the outcome; not10 just regulatory outcomes, but also the patient outcome11 in the end of this event.12 Any other relevant information can help us13 in a good case report. For reports of medication14 errors, a good case report would also include a full15 description of the following; the products involved,16 the sequence of events leading to the medication error17 and the work environment in which the error occurred.18 Also, types of personnel involved with the19 error. We encourage sponsors to include in the case20 narrative all of the data elements outlined in the NCC21 MERP taxonomy. That is National Coordinating Council22 for Medication Error Reporting and Prevention.23 When case reports are identified in24 adverse events where there are bases that associate a25 similar event with a product, a case series could be2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 161 developed. In FDA's experiences, identification of2 all clinical relevant cases depends on thorough search3 strategies based on measured terminology.4 Generally, case definitions would be5 developed to provide consistent characterization of6 the adverse event of interest, and to facilitate the7 retrieval of all clinically relevant cases from the8 database. In addition, data mining techniques may be9 applied to the databases to identify some cases of10 interest. 11 For any individual case report the FDA has12 found that without complete details, it is rarely13 possible to know with absolute certainty whether it14 was product induced.15 However, a number of features as follows16 are generally recognized as supportive of an17 association between a product and an event. If the18 event occurs in the expected time frame -- for19 example, allergic reactions occurred within a few20 hours, or a few days 21 -- most likely cancer will develop after long term22 years of therapy. 23 If there are no symptoms related to the24 event prior to exposure, and if there were no other25 use of a concomitant drug that could contribute to the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 171 event or co-morbid conditions. Of course we are2 looking for the positive challenge, or positive3 rechallenge scenarios if we have them.4 Also, the event is consistent with the5 established mechanism of action of the product.6 However, we always keep that in mind, that7 idiosyncratic reactions that are not necessarily8 consistent with the mechanism would be considered in9 the assessment.10 For medication error related events, the11 FDA would expect the sponsor to evaluate each event to12 identify the root cause of factors that led to the13 actual error, or even possible error, the NCC MERP14 that taxonomy that was mentioned before would be used.15 When appropriate the case reports may be16 grouped into probable, possible, or unlikely for our17 assessment purpose. In general, a safety signal may18 be described as an apparent access of adverse event19 associated with the products used. 20 Occasionally, however, even a single well-21 documented case report, particularly if the report22 describes the positive rechallenge, may be viewed as a23 potential signal. In addition, technical findings24 with a product, or experiences with other similar25 products in the class may be sufficient to generate2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 181 the hypothesis for a safety signal.2 Safety signals may be further assessed in3 terms of their magnitude, population at risk, changes4 in risk over time, biological possibilities, or other5 factors. A product's risk profile may be6 characterized by several safety signals. 7 Data mining can be used as a signaling8 tool by using the Bayesian methodology, and data9 mining uses raw case report data, and all drug adverse10 event combinations. It calculates the expected number11 of events for all drugs, and then compares each drug's12 observed to expected count.13 There are many methods that may be used to14 generate the relative signal scores observed and15 expected count. This may be a useful adjunct to our16 routine safety signaling methods.17 However, our current thinking is that18 further exploration and validation is still needed to19 use as a tool. Safety signals may be able to tell us20 any new unlabeled adverse event or observe an increase21 in the severity or specificity of a labeled event, and22 an increase in the frequency of a labeled event, and23 new interactions, from drug interaction, drug food24 interaction, or drug dietary supplement interactions.25 Of course, where there is a confusion with2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 191 the product's name, package, or use, actual or2 potential, then that is a signal to us. After3 detecting the potential safety signals from the case4 series, this descriptive analysis of the cases are5 very useful in characterizing the adverse event and6 the population at risk. 7 These analyses should include8 demographics, including age, gender, or race9 information, the effect of dose exposure duration, is10 it short or long term duration exposure, and the dose11 effect. If there is any concomitant medication, then12 what is the potential interactions between the drug13 and the suspect, and the rest of the event.14 And if there is a co-morbid condition of15 the patient, such as baseline hepatic or renal16 impairment, in the risk event. We are looking for a17 lot to lot differences in product formulation, and the18 risk of the event, and if there is a potential for an19 access adverse event given the disease being treated,20 such as in an advanced cancer condition or21 immunocompromised patients that we have to consider22 all these conditions in the analysis.23 Finally, we would like to do some24 estimates of the magnitude of risk or differences from25 the known background rates of the events that are2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 201 interesting. All these risk assessments of post-2 marketing data will be considered if further3 pharmacoepidemiologic studies may be recommended to4 further characterize the risk.5 Dr. Judy Staffa will present this6 afternoon the various aspects of pharmacoepidemiologic7 studies. The questions we are asking for public8 comments for these sections of the guidelines for9 pharmacovigilance practices are how can the quality of10 spontaneous reported case reports be improved.11 What are the possible advantages or12 disadvantages of applying data mining techniques to13 spontaneous reports databases for the purpose of14 identifying safety signals.15 Finally, what are possible advantages or16 disadvantages of performing causality assessment at17 the individual case level, or even at case series18 level. Thank you.19 DR. GALSON: Thank you, Min. We are going20 to move on now to oral presentations from members of21 the public who have preregistered, and the first of22 those is Dr. Robert Nelson.23 DR. NELSON: Good morning and thank you24 once again for the opportunity to comment. And it is25 always a pleasure to follow my old colleague, Min2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 211 Chen. I know that she is a cornerstone for all the2 activities that are happening here.3 For the record, my name is Bob Nelson, and4 I am a consultant in drug safety and epidemiology.5 Previously, I was a commissioned officer in the Public6 Health Service, having spent 20 of my 23 years at the7 Food and Drug Administration, both in the Office of8 New Drugs and in the Post-Marketing Surveillance9 Activity, with my last position being the Assistant10 Director in what is now called the Office of OPaSS, I11 guess. It changes so rapidly.12 What I want to begin with is just putting13 up my set of definitions, and not to argue with the14 definitions in the concept documents, but just to15 point out two things.16 In general, the term, pharmacovigilance is17 considered a much more narrow term, usually focused on18 spontaneous reports and not being the umbrella term19 which I would call risk assessment.20 And the second point is that the only term21 that is in the regulations is post-marketing22 surveillance, and that is again an umbrella term that23 encompasses all safety activities, not just the ones24 that we are talking about today, but also advertising25 and quality of the products, field inspections, and so2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 221 on. So that is a term that should not be left out.2 But I will just leave those for you guys3 to look at. But what I will spend most of my time4 talking about is just commenting as you have requested5 to the questions that you have laid out, and then I6 will end by discussing some popular myths.7 The first one, of course, is how can the8 quality of spontaneously reported cases be improved.9 And I think that the most important thing that could10 occur is the philosophy and the position that the11 agency takes on them.12 And I call that the public health focus13 philosophy, meaning that when a report comes from a14 health professional, the agency should consider it15 true for what the health professional has reported it16 as, unless the company does sufficient research to17 show otherwise. 18 I think that is an important position to19 take rather than the Congress, and that is a public20 health position, true unless shown otherwise. That21 encourages to obtain as full data sets as possible,22 and rule out alternate explanations, or rule in23 alternate explanations as the most likely cause.24 And I need to remind everybody that25 confounding, or in other words, multiple indications2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 231 or medical conditions, and drugs, does not rule out a2 causal relationship, which I see so often done.3 Secondly, I think we have to focus on4 those reports that are likely to be important. Those5 reports that are for medically significant outcomes,6 commonly known in the regulatory vain as serious, but7 in addition there are a number of other ones.8 And the most important way to increase the9 quality of those important reports is at the query at10 the point of initial contact, and you will see that11 the new proposed suspected adverse drug reaction12 regulations also says that, and again I have to point13 to my colleague, Min Chen, and I worked on those for14 so many years, and I am sure glad to see them out15 finally.16 And I think that once they are out and the17 comments are received, I would hope that the agency18 moves as rapidly as possible to finalize those19 regulations, and as soon as they are finalized, that20 they revise the client's documents so that the21 compliance officers focus more on the quality of the22 cases rather than the timeliness of the cases.23 I think that is going to be a very24 important thing in turning the tide. So that is my25 comments for question one, and again I want to focus2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 241 on this public health focus philosophy, because2 without it there is a temptation among some companies3 -- very few, but some -- not to acquire the needed4 data, because they know that the FDA will disregard5 the case as not being credible. And that is a bad6 position to be in. Question Number 2, advantages and7 disadvantages of data mining. I think data mining is8 a value for secondary alerting, and what do I mean by9 secondary alerting? 10 I don't think that any computer system or11 any statistical technique of any kind can tell you12 what a signal is. A signal is something that a person13 must judge. But a computer system is useful in14 telling you things that are disproportionate,15 discordant, or whatever.16 So they are useful, but they must be kept17 in context, and I call them secondary because of18 course primary alerting is just recognizing good19 cases, and so on. So it has a role, but it is not a20 primary role.21 And it also has the ability, of course, to22 identify previously unrecognized relationships amongst23 the data that would not be intuitive by looking at an24 individual case. And they are most valid when you do25 the internal proportional analysis. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 251 The third question, is advantages and2 disadvantages of performing causality assessments. I3 really think that it is useful? I think it is4 important, especially when we have high volumes of5 reports as we do in 2003, to sort the good quality6 reports from the vast majority of reports that are7 possible at best, and another large group of reports8 that are just not -- just don't contain any9 information value at all.10 Because I believe that unless you have11 some good quality reports that would lead to probable12 criteria, you really don't have very much evidence, at13 least from the individual cases. Maybe when you get14 enough possible cases you can do a content analysis,15 and a case series mode, and you can extract a little16 more evidence.17 But it is really the quality of the cases18 that are important in the spontaneous reporting19 schemes, and not the quantity. Remember, these are20 qualitative data.21 And then when you do a case series review,22 you can add the possible cases back in, and as my23 first myth is the myth of a definite case. I think24 that is the unicorn of spontaneous reporting. There25 is no such thing as a definite case in a single2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 261 spontaneous report, even with the positive dechallenge2 and rechallenge, because the background disease is3 very common, and it could still be an artifact.4 Four, what circumstances would a registry5 be useful. I think that we need to be careful with6 registries. Just getting cohorts of people and7 calling them a registry is not of much utility in and8 of itself. 9 In order to interpret the data from a10 cohort of individuals, you need to have a valid11 comparative group, and registries I see being formed12 do not have this. 13 Also, the safety data extracted from these14 prospective cohorts need to be collected with the same15 methods and the same data collection instruments that16 are used in the clinical trial environment, and not17 just spontaneous reports rising from this cohort. 18 It does not give you interpretable data.19 And always remember that prospective cohort studies or20 registries, which are the same thing, are not powered21 to find new adverse reactions. 22 Many of you will recall that there was a23 big effort if you have a history of this field in the24 late '70s on prospective cohorts, and there was quite25 a number of them assembled by industry. and actually2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 271 they were defused by a report by an FDA official, Alan2 Rossi, that basically says that they weren't worth3 anything because we didn't find any new reactions.4 Remember that that is not what they are5 there for. They are best in assessing risk factors6 for known reactions so that you can subsequently7 prevent those known reactions, and so it gets to the8 root cause question of why these are occurring.9 And they are also a good source as I10 mentioned yesterday of quantitative data to aid in11 defining the safety profile of a drug in those12 assembled cohorts, but without comparator cohorts, you13 can't interpret them.14 So be careful in just assembling15 registries to say that you have a registry, because it16 is very misleading. What circumstances would active17 surveillance strategies be useful in identifying yet18 unreported adverse advents?19 This is a tough one. Many of you also20 will recall the active case control surveillance21 methodology that Sloan and Shapiro had in effect in22 the 1970s. That was usually a hospital-based nurse-23 extracted dedicated drug safety data, mostly funded by24 the FDA and the drug industry, that has kind of25 disappeared. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 281 I think it should be revisited now that we2 have modern computers and a lot of other things, and I3 think we can better address this issue. I think it is4 a powerful methodology, and the agency should consider5 seeding the reinstitution of this kind of thinking.6 Also, I think the MedSun program in the7 Center for Devices could be extended to the drugs area8 to have representative samples of -- not use the9 facilities, but clinical practice environment that10 could report a more concentrated from of reporting.11 Thirdly, we have a resource in this12 country that is tapped, and that is our Poison Control13 Centers and our Drug Information Centers, and our14 university hospitals, and our regional systems.15 And for those of you who don't know the16 French system of pharmacovigilance, those regional17 pharmacovigilance centers, are only secondarily18 pharmacovigilance centers. Primarily, they are poison19 control and drug information centers.20 So we have this network in the U.S., and21 we are not tapping into it. I see that as an22 incredible resource for getting very high quality23 reports from an institutional setting, and basically24 you can leverage your FDA staff through contracts to25 these groups, and I think that really needs to be2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 291 explored.2 That would be particularly powerful for3 drug-drug interactions, as well as medication errors,4 and things that are going to be very difficult to get5 anywhere else. So I think that those are three6 strategies that need to be given some attention.7 Now, pharmacoepidemiological studies, I8 think they are useful only when they can be9 replicated. I am often stunned by the fact that we10 need replication in clinical trials to get a drug on11 the market, but some observational research that they12 can't be replicated can do great harm to a drug once13 it is on the market.14 I think that it needs to be replicated,15 and I think they need to be validated, which is16 another problem, and they need to provide as I said17 yesterday quantitative data, so that you can measure18 baseline risk, and you can measure change from that19 baseline after an intervention.20 If you can't do those three things, then21 you shouldn't be doing pharmacoepidemiology. Now, let22 me change after looking at those questions, and23 address some things that may also be helpful and may24 not. I have been involved in working with a number of25 companies over the past couple of years and getting2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 301 them supported for advisory committees to the agency2 and so on.3 And in a recent advisory committee, I4 heard this statement, and I will paraphrase. Today's5 FDA error system isn't of value because due to under-6 reporting, incidents cannot be reliably calculated.7 And I heard that statement and no one objected to it8 when it was stated, and I find it amazing that a9 statement like that can go uncontested nowadays.10 First of all, as a -- and again along with11 Min, as a developer and creator of the FDA error12 system, I take personal affront to people banging on13 it like that.14 What they are really talking about, of15 course, is spontaneous reporting schemes and not the16 system itself, which is the computer software. But17 the whole concept of under-reporting, which I will18 address in a minute, is absolutely a fascinating myth19 that continues.20 But before I get to that, let me talk21 about the confounding myth. We all know that22 spontaneous reports seldom contain all the history23 data and concomitant conditions and drug data that we24 would need for assessment.25 But this absence may be due to one of two2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 311 factors; either a true absence -- they didn't occur --2 or poor data acquisition. Conversely, the presence of3 con factors does not automatically clear a drug from4 causality. 5 I have seen in many cases where large6 numbers of confounded cases in complex medical7 conditions nonetheless were dismissed because they8 were confounded, and I think that is a terrible error.9 Because unconfounded cases, especially in10 complex medical conditions, may most likely be due to 11 poor data acquisition, and so you are actually relying12 on the worst data collection techniques in those13 cases, and throwing out the better cases. 14 There is a lot of information to be gotten15 from those confounded cases when you do content16 assessment of case series, and we need to stop17 diagnosing individual pieces of paper and look at them18 more epidemiologically.19 Again, you need to apply this public20 health focus philosophy. In other words, a report21 from a health professional is true unless otherwise22 shown by good data collection techniques, and of23 course the way that you look at case series is by24 Popperian logic of relative likelihood of all25 qualitative explanations, because you cannot prove any2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 321 one of these things. All you can do is rank audit the2 likelihood of the different alternate explanations.3 The second myth that I would like to4 address is the reporting rate myth, and this is one5 that always makes me crazy. The statistical6 assumptions in Finney's 1971, The Class Paper, which7 talks about the random distribution of non-causal8 events in spontaneous data, is no longer valid in the9 U.S. spontaneous reporting scheme.10 It may still be valid in the U.K. data, or11 the French data, because there are much more refined12 and defined population. In the U.S., we have all the13 non-health professional reporters, and we have non-14 series reports, we have solicited reports. We have a15 whole bunch of stuff.16 So all the statistical assumptions in that17 paper don't apply, and so therefore we don't know the18 sampling scheme from which these reports come from,19 and therefore, they are qualitative. And if they are20 qualitative, they cannot be true, and they are not21 enumerators.22 And if you don't have a true enumerator,23 there is no way to calculate any kind of rate. So I24 think that these extrapolations that I hear from the25 agency, as well as from the industry -- oh, we think2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 331 only one percent, or 10 percent of these are reported,2 and therefore, we will extrapolate -- I think that is3 doo-doo, and we have got to stay away from that, and4 we have to stop torturing these data.5 On the other hand, there is an exception.6 Reporting rates can provide some contributory evidence7 for regulatory decision making when the outcome8 reaction is exceedingly rare, and there are a number9 of good cases that show that that has been beneficial.10 Otherwise, a calculation of reporting11 rates, if it is to be done at all, can only serve to12 confirm that the signal may actually be real, and that13 we ought to do some more research, and that is all14 that it can be interpreted as.15 It is much more valid to look at the16 internal proportional analysis across the17 pharmacological class using the internal data as your18 denominator than it is to use external data as a19 denominator given the kinds of reports we have.20 Now, my favorite one of all is the under-21 reporting myth. Spontaneous reporting schemes were22 put into place and were designed to signal new rare,23 unusual, and serious adverse reactions, and there is a24 very powerful tool for doing that.25 They were never designed to collect all2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 341 adverse reactions. We all know that. That is not a2 limitation or a flaw of spontaneous reporting systems3 as it is a characteristic.4 You know, you can't blame a car for it not5 being able to fly. It just is not the kind of data6 that should yield the kinds of things that we try to7 make it yield. I actually think given the initial8 intent of the spontaneous reporting scheme that we9 have gross over-reporting in the U.S., and not under-10 reporting.11 We have much too many reports coming in12 and a high percentage of those have no information13 value at all, and I know that we have taken some steps14 in the new regulations, the new proposed regulations,15 to sort between those that are most likely to be16 important, and those most likely not to be important,17 and I think that is a good step forward.18 The only true under-reporting in19 spontaneous reports is the under-reporting of20 important data in important reports. Otherwise, I21 think we have to get off this thing of saying that22 under-reporting is a problem with spontaneous data,23 because it doesn't make any sense.24 Lastly, I will just briefly talk about the25 secondary data myth, because I did go into this a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 351 little bit yesterday. We have to be very careful when2 we are doing any kind of observational research using3 medical record and billing data, ICD-9-CM codes do not4 denote adverse reactions.5 When you do a study in here, you are6 actually telling the differences in disease or7 diagnostic occurrence between two groups after drug8 exposure in one hand, and without drug exposure in the9 other.10 And the differences may be due to the drug, but11 it may now. And they are not adverse reactions. They12 are subsequent diagnoses. Plus, we have in many of13 these data bases gross misclassification often can't14 be assessed. So these data are not of the caliber or15 the kinds of data that we need for risk management,16 and I urge the FDA to seed new data sources with their17 contract vehicles, with the PDUFA money, let's move18 beyond the current limitations.19 And let's get to some dedicated drug data20 sources, and again I encourage that the FDA seeds21 their creation. We need to stop torturing existing22 data sources to yield answers that they cannot23 possibly provide, and actually that should be a24 violation of the Geneva Convention. Thank you very25 much.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 361 DR. GALSON: Thank you very much, Dr.2 Nelson. Our next speaker is Dr. Stephen Goldman.3 DR. GOLDMAN: Good morning. Unlike Bob, I4 am focusing on one topic in particular, and I enjoyed5 Bob's presentation as usual. I am going to talk about6 the quality of the data, garbage in, garbage out;7 spontaneous reporting systems that handle the quality8 of work that you get in.9 And so that you know my background, I am10 first and foremost a clinician, and it has been my11 privilege to treat thousands of patients over the12 years. I do feel having been in academic medicine, in13 regulatory agencies, I have treated patients in every14 possible health care system, having been in industry15 as a consultant, and I think I have seen almost all16 sides.17 So I feel actually qualified to speak for18 my fellow physicians in saying that I think I know19 what it is like to be in an office trying to decide20 what to do with patients, and trying to decide which21 meds to use, and what to do about adverse events.22 One thing that I did want to point out for23 those of you who don't know, I am also the former24 medical director of the MedWatch Program. So the25 people of the United States paid me to get better2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 371 reports into the agency as one of my purviews.2 I am sort of an amateur historian, and I3 always like to start with a particularly apt quote,4 and I tried to figure what historical figure could I5 use to start this talk, and I came up with the obvious6 one, Bob Marley.7 And Bob Marley sang V. Ford's famous8 words, "And in this bring future, you can't forget9 your past." There are lessons as Bob pointed out, and10 I will point out, and I think Jerry will also, that11 there are things that we know already that we seem to12 have either forgotten or the program seems distant in13 our memory. We know a fair amount, and I am going to14 run through some of those.15 Actually, I am going to go back one. In16 1969, the year that the New York Mets won the World17 Series, there was a very nice study done at Koch-Weser18 and colleagues, and they assessed the factors that19 impacted physician reporting of ADRs at MGH, and they20 were the certainty of the reaction, the mechanism of21 the reaction, the morbidity associated with the22 reaction, prolongation of hospitalization, and the23 onset of reaction.24 Don't these things seem familiar in terms25 of what we look at. They also looked at quality.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 381 Now, they also looked at quantity, which I will talk2 about in a few minutes, but they noted that the3 program that they had not only increased the quantity4 but frankly more importantly the quality of the5 physician ADR reporting since they initiated the6 studies.7 There was an ADR definition they applied8 that was easily applied, and it was felt to be9 meaningful operationally, and what is so important for10 practicing doctors and other health care11 professionals, it excluded what seemed to be trivial12 side effects that did not seem to have true clinical13 significance.14 Secondly, feedback clearly showed that15 those reports were being both carefully monitored and16 assessed. Feedback is crucial and we will talk about17 that further, and they also thought that emphasizing18 the importance of ADRs, in terms of monitoring,19 heightened the visibility of the program throughout20 the institution.21 All right. Now that is going on 34 years22 ago. Let's fast forward to 1988. Did the Dodgers win23 the World Series that year? I forgot. Okay. They24 asked why physicians in Rhode Island were hesitant to25 report suspected ADRs. Notice ADR, not ADE. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 391 Thirty-eight percent didn't have the form.2 At that point that was the famous 1639, and not the3 MedWatch form. Approximately 30 percent were unsure4 the drug caused the reaction. And 24 percent said the5 reaction was expected, and one of my favorites, 216 percent didn't know how to report.7 And in 21 percent, it never occurred to8 them, and close to three-fifth’s were unfamiliar with9 the FDA forms an guidelines for ADR reporting.10 Following hot on that was what I thought and still11 think was the famous Rhode Island ADR reporting12 project, in which an intensive program was put into13 place designed to increase physician reporting of14 suspected ADRs via sustained education utilizing15 several formats.16 They found after two years a greater than17 17-fold increase in direct reports to the FDA by Rhode18 Island docs, while at the same time there was no19 increase in the overall reporting rate in the United20 States. This predates MedWatch by the way. 21 Now, the important part of that, which I22 always include now in a slide, is the trend on serious23 events, and to reiterate what Bob was saying, these24 systems are designed to pick up where unknown,25 generally idiosyncratic events. Not the common2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 401 events.2 And you can see that they went from before3 the program, 0.4 percent of total reports to the FDA4 from Rhode Island; and after the program, 3.6 percent.5 That is a remarkable increase and of that, 31 more6 armed reports on unlabeled reactions.7 I would consider this program a success.8 In addition to that, they did pre-imposed intervention9 surveys, finding significant gains in both knowledge10 and attitude towards the ADR reporting system.11 That is, before the intervention,12 approximately half were familiar with the ADR13 reporting system, and afterwards, 85 percent were14 familiar, and similarly, approximately 40 percent were15 familiar with the FDA forms and guidelines, and 7016 percent, or 69 to 70 percent after the intervention.17 I think that this is important information that I am18 bringing back to memory.19 MedWatch. MedWatch came into existence in20 June of 1993. To give you an idea of a temporal21 association versus causal association, I was at the22 FDA in that time, the temporal association, but I had23 nothing to do with the founding of MedWatch. So no24 causal association. I came later.25 MedWatch was launched in June of 1993, and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 411 Tony Piazza and (inaudible) Kenney looked at both2 trends and reporting of serious adverse events from3 '92 to '94, and the quality of the reports before and4 after launching MedWatch.5 What they found was that a proportion of6 serious adverse event reports went from 34 percent in7 '92 to 49 percent in 1994. By the way that figure8 went up. When I left MedWatch in 1999, we had9 approximately two-thirds of all drug reports to the10 FDA through MedWatch. That was a voluntary reporting11 system, and two-thirds were on serious events.12 And that is what the program was designed13 to do. MedWatch was never designed to increase the14 total number of reports to the agency. It was15 designed to increase the proportion of serious reports16 to the agency.17 Secondly, they found the overall quality18 of the reports went up between '94 and '93, and they19 determined quality as a significantly greater20 proportion, say whether the drug was a newer entity,21 and they talked about the seriousness of the event,22 and they also supplied the crucial data that Min went23 through, lab data, clinical data, supporting your24 diagnosis.25 Pharmacist reports increased both in2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 421 quality and in number, and physician reports, although2 of high quality before and after, the numbers did go3 down. The highest percentage report to the MedWatch4 program were pharmacists and not physicians, and I5 believe that still holds in terms of that.6 That should not be a surprise, because7 that doesn't mean that the pharmacist was the first8 one always to pick up the adverse event, but they are9 often the first one to report it, particularly through10 P&T committees and other systems they have.11 All right. This is from Spain, and I12 thought that this was a nice study that I found when I13 did my literature review for this presentation. This14 is a case control study that looked at docs who had15 reported ADEs and those who hadn't. 16 And they found that the probability of17 reporting an adverse drug event increased with18 increasing prescription volume, and what a shock, it19 decreased with increasing patient load. Not a20 surprise. But look at the things that they found21 would lower the likelihood that you would report.22 The belief that truly serious ADEs were23 well known by the time of marketing. Well, we all24 know how true that is. The belief that the25 determination of whether the drug was responsible was2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 431 nearly impossible.2 Reporting only when they were sure of the3 reaction was to the product in question, and the4 belief that an isolated case seen by one doc couldn't5 make any contribution to medical knowledge. I will6 talk about those myths as I follow up and summarize. 7 In Germany, in 2002, they took a look very8 similarly at docs who reported and docs who hadn't.9 Anywhere between three-quarters and close to 9010 percent never submitted a report tied to the11 government or professional programs, and yet the12 reporting was much better from pharmaceutical13 companies.14 A whopping, almost two-thirds, or more15 than two-thirds, suspected an ADR, but did not report16 it. The major reasons, once again, is that they17 thought it was a well know ADR; they thought it was a18 trivial ADR, and they were uncertain of causality.19 In addition to that, they found that the20 highest probability of reporting an ADR were for21 serious unknown ADRs for new entities, or an older22 drug, or a serious known ADR to a new drug. Along23 with that, however, 20 percent acknowledged no24 awareness of the spontaneous reporting system in25 Germany, and 30 percent didn't know how to report, and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 441 54 percent would report if offered therapeutic advice.2 There are common themes in these studies.3 I didn't have a complete list at the time, and I did a4 literature search like I said. I looked at several5 countries, and I don't know if this is good news or6 bad news. The attitudes tend to be the same. 7 The complaints tend to be the same, and8 the reason why docs don't report tend to be the same9 in Europe and in the United States. There aren't much10 differences there. So what do we do about it?11 Well, once again I went back to history.12 Abraham Lincoln said we must use what tools we have.13 This was an assessment of the intervention used to14 stimulate ADE reporting. This is from the Mississippi15 adverse drug reaction, the ADR program. 16 They noted that docs, particularly health17 professional docs, didn't like the fact that they got18 no feedback. They send in a report, and they don't19 find out what happened to that report.20 So what happened? They put together a21 newsletter that summarized both the number and types22 of the reports, and the drugs involved, and they also23 felt that this enabled docs to get educated about new24 drugs, warnings, and careful observation. There was a25 very good response to that.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 451 Secondly, they felt that maintenance of2 awareness about ADR surveillance, and a concomitant3 effect on quality of care was important, and they4 found that posters strategically placed in the5 hospital, and ongoing in-services in hospitals were6 effective. This is the Mississippi program.7 I might add that this preceded MedWatch.8 One of the things that we did at MedWatch was make the9 attempt to tell people what we were doing with these10 reports, which led to the first conference that we11 ran. 12 This was a conference that we ran at FDA,13 myself and Ron Leiberman, actually working with Bob14 through the staff college when Bob was directing that.15 We put together what we think at the time and still16 think was the first conference designed specifically17 to talk about the recognition and management of drug18 induced disease.19 We used things that we had heard were20 effective. We used multiple formats; straight21 lectures, panels, small group discussions. We used an22 underlying clin-pharm approach, and we found that23 knowledge was improved by pre-imposed testing. And in24 addition it was very well received.25 That led to the first MedWatch continuing2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 461 education article, the "Clinical Therapeutic2 Recognition of Drug Induced Disease." It was part of3 that conference, and we distributed nationwide through4 the MedWatch partners, which is over 150 health5 professional associations who signed on with MedWatch6 to get information.7 And the carrot that we used was free8 continuing education credit; two hours of contact for9 pharmacists, and two hours of CME for docs. The10 results were very positive.11 We got a 2.2 response rate. My12 understanding from the marketing professionals that I13 discussed this with when I published this was that14 that is a very good response rate for a blind mail15 out. 16 We got over 15,000 docs and pharmacists17 asking for the CE credit. You will also notice that18 the majority were docs, in terms of that. Over 9919 percent agreed that it met their objectives and was20 relevant. Obviously a halo effect. 21 If you didn't feel that it was relevant,22 you would have sent it in the first place, but they23 did24 say that. What we didn't expect was spontaneous25 comments. People wrote in the margins of the answer2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 471 sheet. I got e-mails. I got calls.2 This was so successful that led to the3 other MedWatch articles, such as the next one, The4 Clinical Impact of Adverse Event Reporting, which I5 wrote from scratch rather than conference, and others6 that were done on vaccines, devices, and they are all7 still up on the MedWatch website.8 The research program basically took that9 idea. This is up on the FDA website, and this is a10 new learning module that was developed through what11 was the Georgetown CERT, and now the CERT at Arizona.12 And as you can see, it is about13 preventable ADRs, and it was based on a need survey14 that was sent to all third-year medicine clerkships,15 and residency program directors.16 It was sponsored by AHRQ. I have no data17 from this yet, but we are looking forward to seeing18 how effective this is. Other successful19 interventions. Very briefly just as an example of a20 multidisciplinary ADR committee, they have a form that21 they developed, a 24-hour hotline, an ADR newsletter,22 broad range in-service.23 Pharmacists and investigators sit on the24 committee, and increased numbers of reports.25 Switzerland, a clinical pharmacist making rounds,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 481 soliciting information from docs and nurses, doing2 chart review, and what a surprise; improved ADE, ADE3 identification and reporting.4 Two other papers looking at the morning5 report as a way of facilitating detection of AEs, and6 as a form for reporting AEs. Both of them were very7 effective, and the references are there.8 We talked about form completeness. I am a9 tremendous believer in directed questioning and a10 check-off list. The MedWatch instructions for both11 the voluntary and the mandatory go item by item. I12 think they are a good guide frankly, in terms of13 utilizing it. 14 The idea of adverse event-adverse reaction15 specific questions is crucial. If you know that you16 have a problem in the liver, or a perception that you17 do, if you put together a list of questions specific18 to liver, and those are the questions that should be19 asked when somebody calls.20 Homicide detectives will tell you the21 first interview is crucial. It is no different when22 someone calls in to give you an adverse event report.23 You get one shot for the most part, and that first24 shot is the best shot in terms of that, because people25 don't want you to call them 2, 3, and 4 times, and it2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 491 becomes diminishing returns.2 So I really advocate for this, and I3 mention two examples, both liver functions and the4 dermitities. These are the kinds of data that we5 talked about getting, and I will put in a particular6 plug. When you ask somebody what medicines they are7 taking, ask them everything that they put in their8 mouths or on their skin during the day.9 People don't think OTC drugs or dietary10 supplements are drugs. It is like if you ask someone11 if they drink alcohol, and they say no. How many12 beers did you have? Seven. 13 People sometimes don't perceive that as14 being alcohol, and the same thought with taking15 histories. These are the things that you have to ask.16 Temporal information, biopsy, or god forbid, autopsy17 results, and dechallenge, and rechallenge.18 This last slide before I go to my summary19 is to point out that when we look at medication20 errors, take a look at the relative percentages of21 what causes them.22 Eighteen percent were errors, correctable23 errors, in calculations, decimal point placement,24 unirate expression, but 60 percent were deficiencies25 in knowledge. That is not remedial, per se, by2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 501 changing a package label. That is a knowledge deficit,2 and that is a harder issue. 3 Lessons learned. I fully believe that4 when we educate health professionals that it should5 not be product specific. It must be clinically6 oriented. The goal should be to increase the7 awareness of medical product induced disease. 8 The only way a doc will diagnose an9 adverse event is to consider it in a differential10 diagnosis. You must enhance knowledge of the11 application of pharmacotherapy, and the impact of the12 individual factors that patients have on13 pharmacotherapy.14 Thirdly, you just explain clearly not only15 how, but why, busy health professionals should be16 reporting adverse events to the regulatory agency or17 both. Feedback is crucial, and I believe the more18 clinical the feedback should be the better. 19 Education, education, education, all20 levels, in professional schools, training programs,21 post-graduate continuing education, and in the22 clinical care settings; hospitals, clinics, and other23 settings.24 That includes all health professional25 disciplines, and it must be ongoing. One shot2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 511 programs will not work. The ADR program in Rhode2 Island was not a one shot. It was a two-year3 sustained program, and look at the results.4 The programs that I mentioned about having5 a pharmacist on rounds. These were not one-shot6 deals. They were ongoing. You have to demystify the7 process of adverse event reporting and assessment and8 what goes on in the FDA and industry, and I am going9 to do the same thing that Bob did, exploiting some10 myths.11 Obviously all serious adverse event12 reactions are not known at the time of marketing.13 That's why we have post-marketing surveillance.14 Causality not being a requirement for reporting. This15 is crucial. How many of those studies that I just16 showed, people felt that if they couldn't say it was17 due to the product absolutely, they wouldn't report18 it. 19 Wrong. That's completely wrong, and we20 tried so hard with the MedWatch Program to make it21 clear that you didn't have to know that. It was not a22 requirement, and it even said that on the form. 23 A single case can lead to general24 knowledge. Somebody somewhere has to be the first25 health professional or consumer to report that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 521 particular adverse event. So someone somewhere was2 the first one.3 And, fourthly, even if a known adverse4 event is known, accumulating good cases on serious5 known adverse events can definitely increase general6 knowledge, can look at possibly patients at greater7 risk, and certainly specificity and severity does make8 a labeled event unlabeled in terms of that.9 We need ongoing evaluation of methods to10 solicit information. I mentioned directed11 questioning, and specialized questions for known12 adverse events in AERs of interest. The brand new13 proposed rule category of always expedited reports,14 the known bad actors, the reason that we have post-15 marketing surveillance.16 The bad news is that we have no quick fix.17 There must be a commitment of resources, and that18 means financial commitment of resources, personnel,19 involvement of multiple sectors and partnerships. 20 This is not just the FDA's problem. This21 is not just industry's problem. What we wrote in the22 1999 task force to the commissioner made it clear that23 this is s a shared responsibility from all sectors,24 including health professional associations. And that25 includes doctor health professional associations. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 531 I will leave my last slide with a2 historical statement. I am always concerned about3 people making the correct attribution between the4 methods used and the results. We have talked about5 this a lot the last three days.6 I have what I believe is one of the7 greatest single line correct attributions in history.8 When George Pickett was asked why his attack on9 Cemetery Ridge failed at Gettysburg on Day 3, what he10 answered was, "I think the Union Army had something to11 do with it." I will stop here. Thank you.12 DR. GALSON: Thank you very much, Dr.13 Goldman. Our next speaker is Dr. Gerald Faich.14 DR. FAICH: Good morning. I am Gerry15 Faich. It is a pleasure to be here. While I am here16 as yet another consultant, I also will be drawing on17 my background as FDA's office director. I was at the18 agency when the 1985 NDA rewrite was put in place.19 That indeed was or did form the structure20 for adverse drug reaction reporting as we now know it,21 in terms of defining serious, and 15 day, and the22 like. I am also here as a member of the board of23 directors of a safety search company, and so I need to24 say that, and that is Centrix, by way of full-25 disclosure. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 541 My opinions, however, are my own. I found2 the concept paper as outlined this morning to be3 thoughtful and useful. I think it attempts to4 carefully balance concepts involved in spontaneous5 report collection, as well as analytic epidemiology6 studies, which we are going to hear about this7 afternoon. 8 So I salute those who wrote the concept9 paper. I, too, will stick to the questions that were10 put before us, and I would like to comment on11 spontaneous report quality, data mining, causality12 assessments, registries, and then I would like to13 comment on large simple as an aside.14 Even though that was mentioned two days15 ago, I think that is pertinent, and so you will notice16 that I am not going to comment in two areas. Let's17 talk about quality of case reports for a moment. We18 have over many years now concentrated on coding19 issues, standardization issues, and report submission20 issues.21 I strongly believe that the key in the22 goal of going forward ought to be to improve the23 content. So if that is true, the question is how does24 one improve content, and I would submit that the way25 to do it, and Steve Goldman just mentioned this, is to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 551 get as much information on the first encounter with2 the reporter in a structured manner as possible.3 If that is true, the question is how does4 one do that. Some of the guidelines talk about using5 a physician for primary intake, and I think that is6 not a feasible approach. I think that it is not cost7 effective.8 However, I do strongly believe that a9 trained health care individual should be, if not the10 first answer of the phone call, the first referral11 point for in taking information.12 The other thing that I would submit, and13 we have learned this from experience, is that the14 intake point should utilize computer assisted15 technology with built-in logic.16 As Steve Goldman just said, if you know17 the problem is in the chest, then you ought to have an18 algorithm which branches the questioning to get you to19 lungs, and from there, to shortness of breath, in a20 structured way that guides both the questioner, but21 also the intake of the data.22 It is perfectly clear that you will23 improve the quality of the data, the quantity of the24 data, and you will contribute mightily to25 standardization of the data, and the database, even in2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 561 the intake process.2 Moreover, if that system is automated, it3 is clear as well that at the end of the interview you4 will know which data elements are missing, and you can5 make an arrangement or a contract with the reporter at6 that point in time for call back on those specific7 items. 8 I again a hundred percent agree -- and9 anybody who has been in this business knows, that if10 you called back the reporter 2, 3, and 4 times, by the11 fourth time, forget it, that you are not going to get12 further data.13 And by the way, as an aside, the new14 safety tone which came out on March 14th, suggests15 that sponsors will have to document their due16 diligence for the multiple phone calls, and submit17 that due diligence to the agency. 18 I think that is wrong-minded frankly. I19 think that is a regulatory burden that does not have a20 payoff. I do think, however, that establishing right21 at the outset with a reporter what data elements are22 missing, and how those will be obtained, and when the23 call back will happen, is dependent upon using24 automated telephone-based systems in large part.25 What we normally do when we intake case2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 571 reports is that there is a reception point. We log it2 in, and we write it up. We code it. We enter it. We3 validate it. We review it. 4 Then somebody finally gets around to5 submitting a form, and what you hear me saying here is6 a plea for using more direct electronic immediate data7 entry. Again, I would emphasize that I am not talking8 about mere wordprocessing. 9 We are talking about a built-in logic in10 the script that an informed health care recipient will11 be receiving and intaking information. It is sorely12 lacking. This technology does exist. It is being13 used increasingly, and I strongly believe that this is14 the way, the single most important way, to improve15 adverse drug event report quality.16 In January of this year, I and Hugh17 Tilson, and Ralph Edwards, and others, participated in18 an International Society of Pharmacovigilance meeting19 in Paris.20 We, among other things, set out some 10 or21 12 points of interest that we are going to be22 discussing at the International Society of23 Pharmacoepidemiology meeting this August. 24 But I cite it here only to tell you that25 one of the points was relative to signal detection, we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 581 noted that despite report volume issues that the2 reporting rate again must be seen as not a true3 measure of the risk rate. 4 The reporting rate is just that. It is5 the number of reports, spontaneous reports, over some6 denominator. It is a signaling device that needs to7 be used with caution. 8 Similarly, you will see here at the bottom9 that we noted that when observational studies,10 epidemiologic studies are done to investigate a report11 and a signal, those should be seen when well done is12 largely trumping signals that arise from spontaneous13 reports.14 So I would submit that in terms of both15 report quality and in report interpretation, once a16 signal occurs, if it is important, there ought to be17 great urgency in getting a pharmacoepidemiologic study18 mounted.19 And when it comes in, in general, it20 should be seen as outweighing the signal value of21 spontaneous reports. Let me turn to the second area,22 and that is data mining. The FDA database in23 particular has evolved over time as you well know. 24 That database has millions of reports in25 it, and is now receiving 250,000 reports a year, as2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 591 opposed to the 5,000 when we started in 1983, and the2 50,000 in 1990 when I left the agency.3 Because of that the database overall has4 marked variation in terms of the quality of reports,5 the requirements that were present at the time,6 waivers have affected it, and the like. 7 This is simply a way of saying that there8 are numerous non-biologic influences in the database,9 and that is bound to perturbate any data mining10 effort. You can make efforts to control for that, but11 my biggest concern about data mining is the generation12 of false signals, which inevitably will happen.13 And I don't think that is necessarily so14 bad. I think that we can interpret false signals as15 long as we know that they are there. The question is16 what is the overall value.17 Two quick examples of false signals that18 come out of data mining. One of them was a letter19 published by Ralph Edwards, in fact, in Lancet about20 the association of Claritin, a non-arrhythmogenic21 antihistime, and arrhythmias, and in fact it was based22 largely on U.S. ADR reports that flowed through the23 air system.24 And in fact FDA reviewers wrote a letter25 to Lancet saying totally wrong. What we are looking2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 601 at are people who had pre-existent arrhythmias, and2 that this is a clear example of channeling bias.3 Those that were at the highest risk of having the4 outcome were assigned the safest disease, will still5 manifest some of them, that outcome, and that is6 obviously a non-causal selection bias.7 We are bound to see that in data mining,8 and of course one of the other issues is that you9 can't know that if just use the data that are in the10 electronic database. You may well need to have much11 more data that would be available in the appendices12 and the like.13 So I would refer you to that glaring14 example of a false signal achieved by data mining.15 Another one that one will find is if you look at16 prozac and suicide, you are going to find out that17 prozac and suicide has the highest rate certainly in18 the neuropharmacologic agents, clearly confounded by19 indication.20 It should surprise no one that the press21 people commit suicide and that there has been a large22 number of reports that came in after publicity about23 this. Also, not easily controlled in a data mining24 exercise. One way to go at this is that I would25 recommend that data mining be restricted to within a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 611 therapeutic class. I think that comparing antibiotics2 or cardiovascular agents to neuropharmacologic agents3 makes no sense at all to me.4 And I certainly since Steve was quoting5 Abraham Lincoln and the like, I will quote Yogi Berra,6 who has of course said that if I hadn't believed it, I7 wouldn't have seen it. And I think that is a clear8 thing that goes on in data mining.9 Let's move on and talk about causality10 assessment quickly. We all know that there is no11 standardized way to do causality assessment. Navarro,12 or algorthims, or others, doesn't do it. Koch-Weser13 recognized this in 1968.14 In fact, what hasn't been said is that Von15 Koch-Weser showed that if you gave a panel of five16 physicians 30 reports, you would get virtually no17 agreement between the physicians as to whether it was18 possible or probable.19 But more interestingly if you shuffled the20 reports, and gave them back to the same reporter, you21 wouldn't get any agreement the second time aground.22 So you have inter-observer and intra-observer23 variation.24 And one has to say wait in terms of the25 value of doing what is a time and energy consumptive2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 621 process. When I got to the FDA in 1983, causality2 assessment, report by report as reports came in, was3 the routine. One of the first things that we did to4 improve the process was to stop that. And we stopped5 it because it was backlogging the system, and it was6 not all clear what the value of that was. So I am7 going to suggest that we should not do routine8 causality assessment, with two notable exceptions.9 One is in the conduct of clinical trials10 for serious adverse events not in the investigator11 brochure, and the other one is that once a signal12 surfaces, however it surfaces, then I do believe that13 you have to lay out the reports and do this arduous14 process. 15 But I would hate to see us retrogress to16 the point where we expended energy and resources on17 the front end, when it is not clear or useful at all.18 I am going to skip this, because I think that I just19 said it.20 But I would point out again the new safety21 tone calls for submission of serious reports from22 trials that cannot be where causality cannot be ruled23 out. Those reports are to be submitted in an24 expedited fashion. 25 I suspect that that is wrongheaded, and by2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 631 the way, the change is that the current regulations2 say that if you have a serious AE not in the3 investigator's brochure, that is reasonably and4 possibly due to the drug. Notice the language. That5 is the standard, and then it should be submitted. 6 This one says that if you can't be sure7 that it might not be associated with a drug, then you8 do have to submit it. I think that is going to clog9 the system, and I think that we are going to lose our10 ability to discriminate between those reports, and I11 would submit that we should revisit that issue.12 DR. GALSON: Dr. Faich, you have got one13 minute.14 DR. FAICH: Thank you. I will speak15 quickly. In terms of registries, follow large numbers16 of patients. I have three problems with registries,17 both on the demand side from FDA, and also on the18 feasibility side. 19 We are asking registries to be too large20 and to last too long. They can't do it. You can't21 follow patients much beyond 5 years, except for using22 national death index, and I would urge caution in23 doing that.24 I also think that data collection needs to25 be highly focused, and by the way, I would say that we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 641 ought to think more and more about directed patient2 interviewing in these registries.3 Large simple safety trials. I just put4 this out here because I think that is going to be the5 fundamental tool. If we depend on classical6 pharmacoepidemiology, we are going to have to wait two7 years or more until there is enough population8 penetration.9 And instead of doing that, we ought to do10 registries in large simple trials if we need to know11 answers quickly. There is technology, again, that12 facilitates both registries and large simple trials.13 This is taken from a smoking cessation study, and it14 just is using telephone interviewing to accrue the15 data on the front end, and to output the data on the16 back end, something that needs to be given higher17 priority.18 If I can get past this, I will summarize19 my talk under a minute hopefully. No, I won't. Well,20 let me summarize then. I have talked about the need21 to impact improved quality by improving intake of22 spontaneous reports. 23 I question the value of data mining as you24 have heard me say, and I think that registries do have25 a role. I think that we need to be careful about2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 651 their sizing and which data gets collected, and I2 think that we should harness new technologies to get3 our data faster and in ways that make more sense to4 us. Thank you.5 DR. GALSON: Thank you very much. We are6 now going to take a break until 9:45, when we will7 have Dr. McClellan. Thanks a lot.8 (Whereupon, at 9:31 a.m., the workshop was9 recessed and resumed at 9:46 a.m.)10 DR. GALSON: Okay. Thanks, and if folks11 could come in and get seated. It is my great pleasure12 to introduce now our Commissioner of Food and Drugs,13 Dr. Mark McClellan. I am not going to go through his14 whole bio, because I think that his background is very15 well known to most of you.16 But just a couple of notes. From 199817 through 1999, Dr. McClellan was the Deputy Assistant18 Secretary of Treasury for Economic Policy, putting19 him as part of a small club of people who have been20 political appointees in both the Clinton and the21 current administration.22 During 2001 and 2002, Dr. McClellan served23 in the White House. He was a member of the24 President's Council of Economic Advisors, and he25 advised on domestic economic issues. He also served2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 661 during this time as a senior policy director for2 health care and related economic issues.3 Dr. McClellan's research interests and4 studies have addressed measuring and improving the5 quality of health care, the economic and policy6 factors influencing medical treatment decisions and7 health outcomes, estimating the effects of medical8 treatments, technological change in health care, and9 its consequences for health.10 And medical expenditures and the11 relationship between health and economic well-being.12 During the few months that I have had the privilege of13 working with Dr. McClellan. I am, and the other senior14 managers at the agency have been incredibly impressed15 with his energy, his interests in detail, and the16 complexity and challenges of the issues that we are17 facing at the agency.18 And as well, his clear focus on the public19 health parts of his task. So now I am really happy to20 introduce Dr. Mark McClellan.21 COMMISSIONER MCCLELLAN: Thanks, Steve,22 for that introduction. That actually made my day. It23 is very much appreciated. I am very glad to be here24 with you all today. This is a great conference room25 and I hope that you will appreciate what I have to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 671 say.2 I am a little nervous with these drug3 experts behind me, and so if they start making faces,4 or groaning, or something like that, I hope that you5 all will let me know.6 But again I am very glad to be here. This7 is an important meeting that is going to be of great8 value in helping us do our job more effectively, and9 that is very important today with the tremendous and10 growing responsibilities that this agency has, the11 growing volume and complexity of our responsibilities,12 both for medical products, and for food, and even13 national security issues, with limited resources.14 We have got to prioritize, and we have got15 to focus on using the most effective methods possible16 for carrying out our mission, and so that means that17 we have to think critically and carefully about how we18 can do the most good for the public health with the19 resources and authority that we have.20 In some cases this is requiring a21 reexamination and an updating of the way that the FDA22 is doing its job. To bring all of this together, the23 agency has undertaken a major strategic action plan24 that is moving forward right now within FDA, and we25 are going to have much more to say about soon.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 681 One of the central themes in our strategic2 planning is the concept of efficient risk management,3 and I know that risk management is the topic for much4 of what you all are discussing here, but I want to5 emphasize that efficient risk management goes way6 beyond the risk management programs for certain7 pharmaceuticals that are the immediate topic of this8 in related meetings.9 The broad principles, however, are very10 much applicable to your efforts here, and that is one11 reason we are so interested in hearing from you on12 this topic. 13 What we mean by efficient risk management14 is first, using the best scientific evidence to assess15 and manage the risks facing the public. This includes16 both the safety risks of the medical products that we17 regulate and the risks from the diseases that they are18 intended to treat.19 This is one of the many areas of20 regulatory science where the FDA has long led the way21 in assessing risks to the public health and22 identifying ways to manage these risks. 23 Second, now more than ever we need to do24 this efficiently and in the least costly way from the25 standpoint of the agency, and especially from the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 691 standpoint of society. 2 Lower costs of risk management means lower3 cost of medical products, and that translates into a4 greater input to do on the public health. Efficient5 risk management in all of our regulatory activities is6 more important than ever given the expanding7 complexity of our challenges, and the need to reduce8 the health risk facing the public at the lowest9 possible cost.10 As drug costs continue to rise, and as11 health care affordability is a front page issue right12 up there with the war and national security concerns,13 steps to reduce the cost of drug development and to14 increase the value of prescription drugs in use are15 more critical than ever.16 A major new technology development17 initiative has been included as part of our risk18 efficient risk management program. This has focused19 on the pre-market side of drug development, and many20 of you I am sure are familiar with it already.21 It includes an analysis of the so-called22 multiple cycle product approvals that have occurred in23 the agency in recent years to identify whether at24 least in some cases those multiple cycles might be25 avoided, making it possible to make a safe and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 701 effective new drug treatment available more quickly2 and at a lower cost.3 Second, the implementation of quality4 systems in our review procedures to make sure that we5 are using the best possible review practices and6 rating ourselves on how we are doing. 7 And third, new guidance that we intend to8 develop collaboratively with outside experts in key9 areas of disease treatment and emerging technologies.10 Another part of our efficient risk management11 initiatives include an overhaul of the pharmaceutical12 good manufacturing practices based on the latest13 science of risk management and quality assurance in14 order to encourage innovation in the production, and15 manufacturing, and distribution of pharmaceuticals.16 From what we have seen and in talking to17 outside experts, there are many opportunities for18 producing more efficiently, with fewer errors, with19 fewer defects in production, and that, too, can20 translate into lower costs and perhaps even safer21 medical treatments.22 Finding ways to help ensure that drugs we23 review are used safely once they are approved, and to24 reduce preventable adverse events through better25 information, better systems for monitoring, assessing2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 711 and reducing adverse events, is also part of this2 broad initiative. And that is why we have asked you3 to participate in this meeting, to help the agency4 consider new concept papers that stem from risk5 management activities growing out of last year's6 important legislation reauthorizing the Prescription7 Drug User Fee Act.8 In this area of mitigating adverse events9 the agency is taking many steps to reduce the too10 common occurrence of adverse events involving the11 products that we regulate. 12 Our new regulation requiring bar codes on13 pharmaceuticals will be important in reducing adverse14 events due to dispensing and administering15 pharmaceuticals improperly. It will help make sure16 that the right patient gets the right drug and the17 right dose, and in the right form at the right time.18 We are also working on developing19 information technology tools to identify adverse20 events automatically in a timely way, and to provide a21 two-way street so that we can provide feedback quickly22 and effectively on ways to mitigate the risks that we23 identify.24 We have entered into partnerships with25 hospitals, such as Columbia Presbyterian in New York,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 721 through the Marconi Project, and in our devices2 center, we are expanding our so-called MedSun3 initiative, which relies on the web for two-way4 communication to identify and respond to adverse event5 problems with devices, and hopefully that can be6 expanded to drugs and other treatment soon as well. 7 But adverse events that can be prevented8 aren't just caused by medical errors. Some are caused9 by rare problems, such as liver toxicities, or adverse10 interactions in certain groups of patients that may11 only become apparent after a product has been on the12 market in use with real world patients for some time,13 rather than in the somewhat idealized conditions of a14 clinical trial. 15 Finding better ways to monitor the safety16 of new drugs could improve the drug approval process,17 as well as give patients and doctors greater18 confidence that the drugs that they use will work as19 expected. And if they don't, will give them greater20 confidence that the problems will be identified21 quickly and addressed quickly and effectively. 22 We have an unprecedented opportunity to23 implement these mechanisms now, thanks to the24 important new post-market authorities in the PDUFA 325 legislation, and thanks to the agency's and the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 731 department's strategic focus on improving patient2 safety and improving the use of modern health3 information systems.4 We are looking for better information in5 this process, and not more information. We are6 looking for more useful information for guiding7 decisions about the benefits and risks throughout the8 process of developing and using new technologies, not9 burdensome additions to the requirements for approving10 new products, and monitoring their use after approval.11 But this kind of creative thinking isn't12 easy, and we need your help. That's why workshops13 like this are absolutely critical, and as we consider14 use of the drug development process, we need15 constructive suggestions from all sources. 16 This is the best way to reduce the time17 and the cost of the complex process of developing18 medical technology to fulfill our shared goal of19 giving patients more value in the medications that20 they use.21 We hope that industry will work with us in22 this effort, and I am very encouraged by the way this23 discussion has progressed so far in today's and24 yesterday's workshops, and we hope that the product25 developers will work closely with us to plan and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 741 implement these risk management programs.2 We also hope that the result, better Phase3 IV studies, can potentially reduce the amount of4 information we need to collect during the preapproval5 process, and I believe that this will be possible. 6 But to achieve this goal, we need to7 identify specific opportunities where it is possible.8 We can also use guidance on the broader framework from9 improved Phase IV activities. That is, on how we can10 through our approach to pre-approval and post-approval11 studies, achieve the greatest reductions in the health12 risks facing the public.13 This risk management includes both risks14 of poor outcomes from adverse events caused by15 treatments, and risks of poor outcomes that can be16 prevented by the use of new treatments.17 In addition, I believe, and I think that18 many from yesterday's discussion, many of whom were19 here for yesterday's discussion, also agree that the20 FDA can encourage safer and more effective, and high21 value use of prescription drugs by helping patients22 and health professionals get better information.23 That includes better package inserts.24 They need to be more effective benefit risk25 communication tools, and we think that there is room2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 751 for improvement. For example, package inserts may not2 be designed so that health professionals can get the3 information they need, the most up to date information4 they need to treat a particular patient.5 And many package inserts today have become6 so laden with legal considerations that they are often7 hard for the average consumer to understand, and they8 are often ignored by many physicians.9 Finally, we aren't yet taking advantage of10 the possibilities provided by modern electronic health11 information systems for accurate and easy12 communication of the relevant information contained in13 a product label. 14 In all these areas, the FDA intends to15 make progress in the months ahead. As we approach the16 FDA's century mark, it is important to remember that17 the FDA's hallmark has long been its ability to adapt18 to keep up with the best new science and to use new19 opportunities to fulfill its mission more effectively.20 This meeting is a very important addition21 to that tradition, and I am sure that it will help us22 find better ways to fulfill our mission in light of23 new technologies and new statutory authorities. 24 I want to thank all of you here today for25 your commitment to help us find the best way to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 761 protect and promote the public health. Thank you for2 coming, and thank you for participating, and thank you3 for engaging in this very important constructive4 dialogue.5 I am looking forward to seeing the results6 of this meeting translated into improvements in our7 guidance, and improvements in our efforts on8 pharmacovigilance, risk management, and other critical9 responsibilities for the agency. 10 Again, thanks very much, and I would be11 happy to take a few questions if you have got any.12 Thank you. 13 DR. GALSON: Thank you very much, Dr.14 McClellan. If folks have questions, please come up to15 one of the mikes in the audience there. And while16 people are thinking about it, Dr. Sullivan, did you17 have a question?18 DR. SELIGMAN: Since I am being put on the19 spot. This particular workshop was often focused on20 sort of the risk side of the risk benefit balance, and21 I wondered if you would like to say a little bit more22 about what the agency can do sort of on the benefit23 sides of medications, and ways and thoughts you have24 about promoting both the beneficial use of medications25 that may be under-utilized, or are not fully utilized2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 771 in our health care system.2 COMMISSIONER MCCLELLAN: I think even3 though as you said the focus here is on the risk side,4 obviously in the decisions that we are going to make5 based on the results of a meeting like this one, in6 particular cases of pharmaceuticals the benefit side7 is very important, too. I have often said that there8 is no such thing as a completely safe medication.9 It depends on the conditions under which10 they are used, and the benefits and the risks facing11 an individual patient, and we need to do more. We12 need to do all we can to try to make sure that13 medications are used where the benefits outweigh the14 risks by the greatest margin.15 So while we are focusing on the risk side16 today, obviously in any particular decisions about17 pharmaceuticals, we will consider those risks against18 the potentially important benefits of the medication19 as well, and that is what I meant by considering all20 the risks.21 Diseases that medications are intended to22 treat, as well as risks of the medications themselves,23 in our approach to efficient risk management24 throughout the agency. And the things that I think25 can help us work on risk can also help us work on2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 781 benefits. 2 Better information for patients and for3 doctors on labeling to help them use the most up-to-4 date evidence in their treatment decisions. Programs5 to help prevent medication errors and adverse events6 may also prove handy in supporting higher quality care7 if we have got these kinds of information systems and8 mechanisms in place.9 So I think that there are a lot of10 opportunities to work on both together, and while the11 focus here may be on the risk side, you can rest12 assured that we are thinking about the same kinds of13 insights and opportunities to improve the benefit side14 as well.15 DR. GALSON: Great. Please identify16 yourself. 17 DR. NELSON: Dr. McClellan, thank you for18 your comments. My name is Bob Nelson, and I am a19 consultant, but a former FDAer. I apologize if this20 puts you on the spot, but since you are here I will21 let it go.22 COMMISSIONER MCCLELLAN: That's all right.23 It comes with the territory.24 DR. NELSON: Dr. Galson mentioned the25 DailyMed program. From what I know of it, I think it2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 791 is very exciting and a collaborative effort with VA2 and the National Library of Medicine to put bar-coded3 labels.4 But I understand that that has not really5 been funded. If that is true, can you verify that,6 and if it has not been funded by the agency, can you7 comment on why?8 COMMISSIONER MCCLELLAN: Well, let me take9 a step back and talk about the agency's overall10 funding levels. The FDA is bigger than it has ever11 been, both in terms of personnel and in terms of12 resources available, and I might add to that in terms13 of statutory authorities with things like the new14 post-market authorities in PDUFA-3.15 We had a big legislative year last year.16 The problem is that although our resources are17 expanding, so are our responsibilities, and so are the18 complexities and difficulties of the public health19 issues that we are charged with addressing, and so we20 do have to think very carefully about prioritizing. 21 We can't do everything, and we can't22 devote unlimited amounts of funds to each individual23 product, and Congress, and the rest of the24 Administration, also have a say in how these25 priorities are set, and we try to work with them to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 801 make sure that we are focusing our efforts and2 resources on where they can do the most good for the3 public health.4 I don't know the exact details of the5 current funding status of the DailyMed program. I can6 tell you that a lot of work is proceeding on the7 development of that program. That is a good example8 of what I mean by how we can improve the medical9 labels, the pharmaceutical product labels in a way10 that should lead to better information being available11 to physicians.12 And as a result, better care for patients,13 and so that effort is proceeding. I think that there14 are a lot of opportunities to implement steps in the15 DailyMed program without a major new expansion of16 funding.17 For example, these collaborations with the18 VA are ongoing, and if we get established some ties19 with health care organizations like the VA, whereby20 they can send us information on potentially important21 adverse events that we may need to take steps to help22 address, we can make it a two-way street by giving23 them access to updated information that is relevant24 for the product labels.25 And that is moving forward, and so we will2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 811 keep working as hard as we can to identify how we can2 use our limited funds as effectively as possible, but3 I don't think we are in a situation where DailyMed is4 not moving forward at all. 5 In fact, it is one of the more important6 and I think promising efforts in our whole set of7 patient safety initiatives. 8 DR. GALSON: Great. Thank you. 9 DR. SZARFMAN: My name is Anna Szarfman,10 and I am a medical officer at the FDA, and have11 implemented both the patient profile viewer that is12 now being incorporated into that routine, and I have13 worked together with Dr. Demichele to implement the14 information on data mining tools.15 COMMISSIONER MCCLELLAN: Yes.16 DR. SZARFMAN: And the questions that I17 have is how -- you know, with the patient profile18 viewer, in the beginning it was difficult, because19 people were accustomed to things such as lots of paper20 and equivalency in other ways until we started these21 studies, that we were gaining speed tremendously and22 in a tremendous way.23 And then there were no questionable24 difficulties in implementing that piece. We think25 that data mining is more difficult because people2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 821 think that we are torturing the data, and like you2 have heard this morning, and we don't systematical3 approaches in place. 4 How can we set up like in the clinical5 pathology lab in the clinical pathologies. How can we6 move from one place to the next one that is7 (inaudible). You know, we have systems in place where8 we can test things objectively, and I think that we9 have to do it.10 And that helps everybody to have objective11 methods and recently in a (inaudible) committee, data12 mining provided some information that was critical to13 understand where are the specific drug, and if it was14 hepatoxic, or if it was more hepatoxic, or less15 hepatoxic, and so I would say it helps with the16 answers.17 COMMISSIONER MCCLELLAN: I think that18 those are all good points. Obviously in any19 activities that we undertake in this whole risk20 management strategy, we need to think carefully about21 the most efficient ways to move forward. You know,22 where are the greatest needs, and where are the23 greatest opportunities for improving the way that we24 do what we do.25 And absolutely we need to test whether the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 831 changes that we implement have an objective impact on2 important performance measures, such as review times,3 and review costs, and whether we are identifying risks4 more efficiently.5 That really is the core goal here. There6 is a tremendous amount of information out there, and7 there is getting to be more and more every day, and8 the question is can we use that information9 efficiently and effectively in guiding our review10 decisions.11 I think the work that Dr. Demichele and12 others are doing on data mining and related innovative13 statistical techniques is very important, but in some14 ways it is harder than the patient profiler.15 You know, it is not just a matter of16 implementing a software program that presents the17 information that is there more efficiently and easily18 for people to use. It is a matter of drawing19 inferences about and sort of extracting signals from20 very complex data that is not completely21 straightforward.22 There is not a yes/no answer. There are23 different degrees of competence about the conclusions24 that can be reached from data analyzed using any25 statistical method. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 841 And so that is going to be a more complex2 technical process to improve upon. But I think that3 the data mining holds a lot of promise, and I think4 that there are a lot of other techniques that have5 been suggested to the agency by other outside6 statistical experts like Dr. Demichele that hold7 tremendous promise for helping us make more or do more8 with the information that we get.9 And again as the information that we get10 continues to grow in complexity, both in terms of the11 information coming in and what it means, we need to12 make sure that we are up to date on our statistical13 methods.14 And we need to have methods in place to15 evaluate whether those statistical methods are having16 an impact in a positive way on what we do. So I think17 that these are the right kinds of issues that we need18 to think about, and we need to couple it with making19 sure that as we move forward on some of these ideas20 that we are seeing an impact on our agency's21 performance.22 DR. GALSON: We have time for one last23 question.24 DR. YODRIN: Sam Yodrin, head of drug25 safety, Millennium Pharmaceuticals. Thank you, Dr.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 851 McClellan, for your talk, which really highlighted two2 significant issues that we have to confront in this3 new era of risk management during productive life4 cycles.5 The first is efficiency, and it is very6 good to hear a regulator talk increasing of cost7 effectiveness, and also the early identification of8 significant signals.9 And then also in terms of effectiveness,10 and the issue of data quality. Increasingly we are11 being asked to have an evidence base for our decision12 making with regards to our risk management. 13 And the current situation is characterized14 by increasing partnerships that are being established15 primarily by the FDA with health maintenance16 organizations.17 There is the CERTS program, but the18 challenge there for industry and companies, whether19 they are a big pharma or a small pharma, alike, we all20 cannot establish these types of relationships to21 ensure that we have the available sources of data to22 enable the appropriate risk assessment that we are now23 faced with. 24 And I know that we have made this25 representation to you in the past through a different2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 861 medium, but we strongly propose here that there is a2 collaborative effort between the agency and industry3 towards looking to establish an integrated national4 pharmacoepidemiology database that would address all5 the disadvantages that are currently inherent to the6 fragmented approach that is being taken right now.7 And we believe that this integrated8 national pharmacoepidemiology database would align9 with the current risk management framework, in the10 sense that it would involve all relevant stakeholder11 parties in collaboration, and all the issues that12 relates to data quality certainly we could address and13 maximize the benefits of information technology.14 And so I just wanted to use this15 opportunity again to make this proposal to the16 Commissioner. Thank you.17 COMMISSIONER MCCLELLAN: I think that18 idea, such a national high quality system, is a very19 interesting one, and a very nice vision for where we20 would like to end up.21 And as you pointed out, we are exploring22 expanded partnerships with health care organizations23 and the like, and basically we are going where the24 data are, and we don't have such an integrated25 national system now, and I don't want us to wait until2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 871 we finally get one before we move forward on some of2 these initiatives.3 I think that this is going to be an4 incremental step-by-step process, but certainly the5 product manufacturers and product developers have an6 important role to play in both helping to develop this7 kind of information and helping to interpret the data8 that are there.9 So the idea of moving towards more of a10 partnership as we move towards better data systems11 sounds like a good one, and I think it is something12 that we should keep in mind.13 DR. YODRIN: Thank you. 14 DR. GALSON: Dr. Sachs.15 DR. SACHS: Thank you for letting me ask16 my question. Susan Sachs from Roche, and since Bob17 said since you are here, we have a chance to ask the18 question, and since this is about risk, and it is not19 about drugs, but I am interested in your thoughts the20 fact that the FDA is not able to regulate supplements21 and things like that, which actually have been in the22 news in terms of risk, and I would be really23 interested in your thoughts on that.24 COMMISSIONER MCCLELLAN: Well, we actually25 do regulate supplements. We just don't regulate them2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 881 as drugs. We have a different statute, like our food2 regulatory processes, has a presumption that they are3 safe, and therefore, they can go to market without4 having to demonstrate safety and effectiveness up5 front, and we have to prove a safety risk in order to6 get them off the market, or in order to restrict their7 use.8 And that is something that we highlighted9 back in February with respect to Ephedra that we are10 undertaken and we have asked for comments, and in a11 public comment period that just closed, for a review12 of all of the evidence on Ephedra, which by far13 accounts for the largest share of adverse event14 reports that we get about dietary supplements.15 And we are evaluating that evidence from16 the comments right now on whether the evidence is such17 that Ephedra as currently marketed presents an18 unreasonable risk to the public health.19 That is the standard under the statute and20 we have to demonstrate that. So we will have more to21 say about that soon. So it is not that we don't22 regulate it. It is a different regulatory structure,23 and it is one that presents some different challenges24 than being able to ask product developers to25 demonstrate up front that their treatments are safe2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 891 and effect. 2 The other important way in which we are3 moving forward on some additional regulation4 enforcement action on dietary supplements is that we5 published a new proposed rule last month that would6 impose some national standards for manufacturing,7 national good manufacturing practices for all dietary8 supplements sold in this country.9 And I think that will give people a higher10 degree of confidence that the products that they are11 buying were actually produced using good methods12 without impurities or problems in storage, and so13 forth.14 And it also includes new labeling15 requirements that the active ingredients in a dietary16 supplement need to be listed on the label and clearly17 identified. And this is also a good way to help give18 people confidence that what they are taking is what is19 represented.20 We would like and are trying to work on21 developing better evidence on both the safety and22 effectiveness of dietary supplements. We have got a23 partnership going with the National Center for24 Complimentary and Alternative Medicine to help us with25 that, but I keep an eye and hand on that.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 901 And one other area of concern is to get2 them all out there, and that many dietary supplements3 are making claims about their impact on patient health4 that are misleading, and that are not based on good5 science.6 And back in December outlined in a white7 paper a more comprehensive strategy for enforcing the8 law against those claims. It is not legal to make9 claims about the health effects of a product, and even10 the structure function effects of a product, that are11 not based on good science.12 And you have seen a lot of enforcement13 actions where we have actually seized products, and in14 some cases companies have shut down as a result, where15 they were making misleading claims.16 And I think you will see more of that. I17 do view it as a public health threat that as we try to18 encourage people to do more, and to take control of19 their own health, because that is the best way to20 improve the public health in this country, through21 people's own actions and diet lifestyle choices and22 the like, have a bigger impact than most of the23 medication that we regulate.24 It is a public health threat if they end25 up spending their money and wasting their effort on2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 911 products that are making claims about effects that are2 just not right. So we will be doing more on that,3 too.4 It is a different regulatory standard, and5 it is one that is more challenging in some ways, but I6 don't agree that we can't do anything there, and we7 are going to be trying very hard to improve things on8 the dietary supplement side.9 Thank you all very much for taking this10 much time in hearing some answers that are a little11 bit afield of the topic. The topic is a very12 important one, and we truly appreciate your input and13 your willingness to work constructively with the14 agency both in this workshop, and I hope going forward15 as we actually move into implementing some of these I16 think quite promising new efficient risk management17 activities. Thank you again.18 (Applause.)19 DR. GALSON: Thank you very, very much,20 Dr. McClellan. We appreciate you coming. We are21 going to go on with our program now, and it may seem22 like we are behind, but we are actually not, because23 we have in contrast to what is in the printed program,24 we have three speakers in the next section, rather25 than four. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 921 So we are just fine and on time, and the2 first oral presentation in this next session is Dr.3 Anshu Vashishtha, from Watson Pharmaceuticals, and Dr.4 Vashishtha, if you could give us your talk now, and5 then when you are done, go ahead and sit at the table,6 and the other speakers can speak up at the table, too,7 whenever you want.8 DR. VASHISHTHA: I thought Dr. McClellan9 wanted to listen on some views to causality, but we10 will do that another time. So today I just wanted to11 give some comments on causality assessment, and I12 think some of them have been presented. 13 Some of the comments will be along the14 lines made earlier by other speakers this morning.15 But as it was one of the questions that was addressed16 to give the benefits, considerations, and causation17 assessments, I think I will explain some of the pros18 and cons as I see them for performing causality19 evaluations, and what criteria to consider.20 And beyond causality assessment, if we21 look at the data mining as a way of computing22 causality, or causal relationship, some of the23 limitations of that.24 And then some proposals and thoughts on going forward25 in the area of assessing risk, and relationships of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 931 risk to pharmaceutical drugs.2 So I think we can break it up into as has3 been mentioned the causality evaluation at the4 individual level, and at the case collection level,5 and then clearly either way there is a very important6 rule of the case definition.7 That when we start looking at causality,8 the first thing we do is that we confirm that what we9 are talking about is a report of the claimed event.10 As an internist in drug safety, I have seen reports11 that claim to be aplastic anemia, and which has turned12 out to be only anemia, and not meet the case13 definition as my standard of criteria.14 And again I think as some of the speakers15 earlier had mentioned, the importance of case quality,16 beginning with good data capture, and defining the17 case is really what the event being reported is, and18 what it is stated to be.19 I do see it as a necessity that we have to20 look at causality at some level or the other. I think21 the question is whether to do it for every case or to22 wait until there is a case series, or a signal, a23 potential signal is detected.24 And as has been outlined I think very well25 in the concept paper, it is because -- and in Dr.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 941 McClellan's remarks, the reported adverse (inaudible)2 concomitant medication that are known to have that3 same safety risk or are not known to have that same4 safety risk (inaudible) in disease.5 So I think we cannot validate from it6 because the current system is to cast a wide net, and7 the consumer can call the company and say (inaudible)8 and by the way does your title now cause it, and then9 it becomes an adverse event according to current10 requirements.11 So since the letter is some broad, just12 data mining or just counting adverse events will give13 us a lot of false positives. I think that some of the14 pros of doing a case assessment are that it can be to15 better case quality and follow-up, because as we start16 trying to do a causality assessment, we can think of17 the confounding factors, and we can think of the18 follow-up needs, and get those questions addressed19 while the case is still fresh and the data is20 available.21 Also, it will enhance ongoing signal22 assessment by breaking down the reports from just23 counting them as reports that may be incidental,24 rather than to the categories, and ranking the25 categories, and we can discuss the ones outlined in2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 951 the concept paper work, as well as any others.2 But I think that those are some of the3 advantages of doing it. I also think that, and again4 as Dr. McClellan also mentioned, that if you want to5 make the package inserts more useful to prescribers6 and patients that the assessment of causal7 relationship can provide a tool to do that in keeping8 events which are incident reported out, and not9 considering them as adverse reactions merely related10 to the drug.11 And on the other hand, events which are12 assessed as adverse reactions truly belong in the risk13 information on the label, and perhaps linked to the14 management programs and could get in there.15 The points of course have been pointed out16 that it indicates that it does take time, and it does17 take resources, and again if we do it on the industry18 side that it takes them on both ends. 19 It may be that there are some approaches20 which are not all the way down the road of distinction21 between a probable and a possible, but at least22 outlined in a case whether it is a substantial case or23 not, and the WHO criteria which I will go into at24 least indicate as an example.25 It is limited of course by a lack of all2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 961 relevant data in several instances, and it can be2 misused as was mentioned again in the morning, that3 just because something is confounded does not mean4 that it is not related. 5 I think that one has to take the whole6 picture into account. There is also the issue from7 the industry side of what is the legal liability of8 the sponsor who has assessed certain cases as possibly9 related, but not made the decision to put them in the10 package insert.11 The criteria that can be considered I12 think are well outlined in the concept paper, and I do13 not need to review them all. We can just go to the14 slides, and I think the CIOMS working group has15 indicated them. 16 Again, the most important being that if17 there is a rechallenge after the drug, and as was18 mentioned in the morning even that is not a certain19 one, and so as for most of these criteria, whether it20 is rechallenge, dechallenge, class effect, biological21 plausibility, a lack of a alternate explanation, or a22 typical ADR type event, like a Stephens-Johnson, with23 a low background rate not related to drugs.24 All those response factors lack further25 concomitant factors in a clean subject patient, or a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 971 consistency of time to onset, or a high frequency of2 reported cases of similar findings in doctors case3 studies.4 Or perhaps positive individual or5 individual tests, like IG antibodies, which are only6 available in a small minority of adverse events or7 adverse reactions; or if there is a clear8 identification of an identified subset, where there is9 a clear risk identified, and where the reaction is10 related to the drug.11 And again I think the protopathic bias has12 been mentioned earlier, that a drug which is13 administered to treat a symptom may be temporarily be14 associated to the illness which manifests after the15 drug is started, even though the drug was started to16 treat the symptom of that same illness.17 And also it has been mentioned the issue18 of stimulated reporting, and in particular I think for19 causality assessment when there is stimulated20 reporting, there may be a recall bias of not recalling21 perhaps and a bias towards false attribution towards22 the suspected drug.23 One thing that I think that I would24 recommend to the working group is that we consider the25 spectrum of causality assessment methods available,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 981 and evaluate them in a way to go forward, and I think2 to some extent that there are some literature reports3 which given the limitations of these, as well as4 (inaudible) products ability back I think from '68 I5 don't think should be our guidance going forward.6 I think we need to look at what the7 current status of these is. There has been research8 on them and extensive use of them in Europe which has9 continued after the FDA discontinued causality10 assessments from '83.11 And I think that it would be helpful to12 draw on this using criteria like applicability,13 explicitness, explanatory, culpability, completeness,14 biological balancing, and lack of constraints, and15 these are reviewed, of course, in several writings on16 the topic.17 So one of the methods that can be not all18 the way to categorizing it into probable or possible19 is the literature of Dr. Edwards criterion of using20 three index cases.21 And where cases can be divided, I think,22 with relatively less time and effort into substantial23 index case or feasible case, and a substantial index24 case has information on the first six -- on all the 1125 items, and there is a feasible case as on the first2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 991 items.2 So just having the items on each of these3 could be one way of categorizing an adverse event4 report into a feasible or substantial report which5 then can make the basis for a more detailed causality6 evaluation.7 And these sort of rely on the fact that8 for rare events the probability of picking up three9 cases is very low, or in a short period of time it is10 very low, and so that is the strength of that. 11 And just to mention some of the risk data12 mining approaches, where the approaches are based on13 the proportion of the reporting rates, and some14 published opinions that this remains as only one15 aspect of causality assessment, subject of course to16 the usual biases.17 And I don't think we can get away from18 global clinical assessment or more case detail based19 approaches, as opposed to again counting and doing20 status tests on events as reported. 21 And so in terms of the categorization into22 broad categories, I think the one in the concept paper23 is one, and there is also this proposal from the --24 based on the review of causality classification at 25 pharmacovigilance centers in the European community,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1001 which Meyboom, et al., had published a couple of years2 ago, almost 10 years ago, into Category A, B, or O,3 where there is either good reason and good4 documentation to presume a causal relationship in5 Category A.6 Or in Category B, where the connection is7 uncertain, and even doubtful, perhaps because of8 missing data, and maybe because a case does not meet9 substantial or case, or category, which is not10 assessable because of missing or conflicting data.11 I think that some sort of categorization12 into such broad categories would at least make causal13 evaluation more efficient in terms of assessing risk.14 So at the individual level, then I would recommend15 that the working group consider the researching16 experience of the European agency.17 The French agency, for example, does18 require causality assessment by sponsors for a long19 time, and so that there might be some data there to20 base our decision going forward on as we try to get21 evidence based on this, and not just go by our22 impression of the utility or lack thereof.23 For the case collections, I do think that24 it remains necessary for signal evaluation. Again,25 the working group may consider really looking at the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1011 different alternative approaches and perhaps testing2 them on the FDA database. 3 It remains important to define a case for4 the cases confirmed and it is of good quality, and5 algorithms can be evaluated I think in different or on6 a worldwide basis, perhaps categorizing into these7 broad zones, either A, B, or O, or something that is8 more amenable to perhaps less label intensive, or9 medical review intensive methods.10 But nevertheless lays a foundation for a11 fully detained risk assessment, and perhaps these can12 be compared by really doing a retrospective evaluation13 for the utility of these methods using the existing14 data from the FDA database. 15 So just seeing if the two have been16 applied what would we have found, and comparing maybe17 some of the tools that are used in Europe, and see if18 that would have been helpful in identifying the risk19 which we now know in an earlier way.20 One may consider a pilot on certain drugs21 and in certain instances, and one can then determine22 the overall guidance based on the results of the23 assessments. Thank you very much again to the whole24 working group and the FDA for this portion.25 DR. GALSON: Thank you very much. Our2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1021 next speaker is Gregory Gogates, or Gogates.2 MR. GOGATES: Good morning everyone. My3 name is Greg Gogates, and I represent CRF Box. We are4 a clinical data collection company using remote data5 collection tools, namely PDAs and cell phones. So I6 thought that this would be a good venue to sort of7 tell people what is going on in the world as far as8 the status of collecting this type of information9 using portable data collectors.10 Okay. The overall overview I think, and11 everybody has said that a million different times, but12 I guess the thing that I am trying to state here is13 that some of this data that we are talking about14 collection with AE data could certainly be done using15 PDAs and mobile phones. 16 One of the things that we are talking17 about doing is getting this information through the18 doctor-nurse and all the different organizations that19 exist, but one thing that I think we are missing is20 direct collection of data from the patients directly21 into databases and this is happening today.22 So I am going to discuss some of the pros23 and cons of doing this, and show you a case in which24 and where this is actually happening right now in25 Europe. So the typical compliance that we have out2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1031 there, and this is from some ad hoc interviews that I2 have done with some physicians and a few people in the3 industry, and we have relatively good compliance with4 the clinical trials, hospitals, and pharmacies.5 And I think the answer to that is6 typically because these people are salaried people,7 and they are being paid to be in their office all day8 long. So if they have to answer a few forms, that is9 just part of their daily jobs. 10 So you have relative good compliance.11 Now, the quality of the data, and other people are12 talking about the quality of data, but I think we have13 good compliance there, because it is part of their14 job.15 Now you move into the citizens, or you16 move into the private physician area, and citizens17 don't necessarily know that this avenue exists, or18 know and don't want to bother. I think that is what19 you are finding there.20 And you have private physicians who say,21 well, I am being squeezed by all means. I am a broke,22 fixed type of a person, and a patient comes in, and I23 fix them, and they leave the office, and what is my24 incentive to fill out all these forms.25 And I think you find that a lot of times2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1041 they say I am not going to bother. I fixed my2 problem, and I am going to move on to the next3 patient. So by its nature, it is voluntary on that4 avenue, and I think that we can improve some of that5 by making it a little easier.6 So you have your traditional MedWatch7 methods using the 3500A and the 3500 forms, but on the8 3500 forms, it is all not mandatory. So there are9 some custom on-line tools which I believe have been10 discussed a little bit earlier that do exist, but they11 are not or they don't travel home with the patients.12 So what I am saying is that maybe we13 should consider getting some of this information and14 making it available for the patients to input this15 information from their homes.16 So the current challenges of course is to17 get this collected and report this spontaneous patient18 level safety data per the CFR as referenced. And of19 course we want to get the timeliness of the data that20 has to come in there, and so we want to make this data21 more timely.22 And of course we have to reside within the23 HIPAA regulations, and I understand that AES are24 pretty much -- that they don't have to worry about25 HIPAA, but at the same time, we still have to worry2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1051 about some of the fringes, because it really is not2 that well defined.3 So we have to get increased public4 awareness of the public safety concerns, and make sure5 that they understand that there are concerns with6 these products, and as we said before, these products7 aren't perfect. We understand that there are risks in8 taking them, and that people have to understand that.9 So how do we respond to these things?10 Well, I think the key to this whole thing is11 spontaneity. If you don't get it up front, you don't12 get it at all. So let's use some electronic tools.13 Some of these tools as you see up there on14 the screen, you have PDAs, and the PDAs right now, you15 can buy wireless ones that are actually part also cell16 phones, and you can buy them with cradled modems so17 they can come up with data. 18 You can also use normal cell phones, and19 there are of these more expensive cells phones, but20 the bottom line is that this information could be21 brought directly into data bases, and viewed on-line22 by anybody who has an access, and it can be stored23 centrally for review by anyone who wants to look at24 it. It is very simple.25 Implementation issues. Patients can use2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1061 the proprietary electronic devices; i.e., the mobile2 phone. So we don't have to give them anything. They3 already own a mobile phone.4 Mobile phones not only can talk voice, but5 they can also transmit data, and the SMS messaging6 system, which is on most of everybody's cell phones7 here in the audience, is usually not that well used in8 the States, but it is very pervasive in Europe and the9 rest of the world.10 But if you have a budget, if this happens11 to be a trial, and there is some budgets, you could12 give them a proprietary device, say a PDA, or13 something else to collect data with. But, in reality,14 I think the mobile phone is probably a better venue15 for this. 16 So the only thing that is recorded is that17 you could record the patient's cell phone number at18 the point of prescription. So you could say, okay,19 Mr. Patient, I am giving you this drug, and would you20 like to volunteer to provide some information on your21 experience with taking this product. 22 If the man answers yes or if the person23 answers yes, you collect their cell phone, and then a24 week later, or two weeks later, or a day later, or25 whatever frequency you want, the system would call up2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1071 the person's cell phone and say, hello, this is the2 drug company, this is the pharmacy, how do you feel.3 Have you had any experiences with this4 drug. They could ask 3 or 4 questions and you answer5 the questions, and there is no voice going on, and6 they don't have to worry about doing it. It is not7 intrusive at all. It's just that your phone beeps,8 and you look at it, and if you don't want to answer9 the question now, you answer the questions later.10 You just push the buttons, and you say11 have you had any diarrhea, yes/no, enter. Have you12 had any vomiting, yes/no, enter. Headaches, whatever13 your questions want to be, you enter the information,14 and it goes right back to the database, and it is15 immediately available for all people to view.16 So these periodic safety questions can be17 sent and answered and sent just as I mentioned. There18 is text space, and there is no voice involved, though19 there is one side benefit of having the voice. 20 If the right question is answered yes,21 then there could be a follow-up. Someone could call22 this person back and say hello, you answered yes on23 this question. We need some more information. So it24 would be nice and straightforward, and very simple to25 manage.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1081 Pros and cons. They don't necessarily2 need to visit the physician. They don't need to get3 involved. It just shows up on their cell phone. And4 the sponsor and anyone else who wants to, has the5 ability to see this information in real time, and they6 can combine it with any other safety data that they7 need.8 So the timeliness is improved, and the9 questions can be changed on the fly. So if you decide10 that the questions being asked aren't correct, you can11 tell the system to ask new questions. 12 So the cost, which is the cost of this13 technology, it does add to the budget, which is a14 deficit. But if you are using the patient's15 proprietary cell-phone, you don't have to buy their16 equipment. You just have to deal with the server17 side.18 So don't get me wrong. I am not saying19 this is an end-all, but it certainly could be used in20 some situations. You may have the disabled, or the21 elderly, or someone who doesn't have a cell phone, or22 maybe something that just wouldn't make sense, and of23 course you wouldn't do it.24 But this is a system that could be used in25 a lot of areas. So there is detail as I said earlier,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1091 and there would be detailed analysis that still would2 need to be conducted with the physician or the nurse,3 but in this case at least you would get that up front4 spontaneous information, which everybody understands5 is the best way to get the information.6 There are some examples out there that I7 can state that our company is doing at this point.8 There is a Phase III trial going on in Europe at this9 time, and there is 12,000 children across Europe, and10 at 75 different sites, and they have been given a11 vaccine, and this is a situation where they need to12 collect data for 2 years every 4 weeks, from 12,00013 parents.14 And this would be a phenomenal expense,15 and so we came up with this idea of collecting the16 parents' cell phone number, and then every 4 weeks the17 system calls the parents' cell phone.18 They get a beep, and it says they have19 received a message that says, hello, this is the study20 and how does your child feel. And the responses,21 there are maybe 4 or 5 questions, and they answer22 yes/no, and the system answers back, thank you, and I23 will call you back in four weeks.24 Now, we are not getting a hundred percent25 response out of this, but at least we are taking at2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1101 least a 90 percent response on this information, and2 now the study nurses only have to call up 10 percent3 of those 12,000 people.4 So it is a significant cost savings in5 doing all of this follow-up work, and there is no6 reason why this technology, or these concepts,7 couldn't be moved into the AE realm of people taking8 medicines, and just taking it from their physician or9 their pharmacy, and not necessarily in a clinical10 trial.11 So in conclusion, basically I think that I12 would like the organization to consider using mobile13 phone infrastructure. It is in this country, and it14 is global, and it is very simple to use, and it would15 be a very good venue to collect patient data.16 I would say that in Europe, and in the17 rest of the world, you have very good -- just about18 everybody and their brother, and children, all have19 cell phones, and it is becoming just as pervasive here20 in this country.21 As I said, the EU system are mature, with22 good inoperability, and one issue that we do have in23 the States is the inoperability between cell phone24 providers is not perfect, but it is increasingly25 getting better.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1111 So considering that this is going to be a2 long term permanent effect, that this is something3 that we should consider. And that is really my simple4 message that I have to say. Thank you very much. 5 DR. GALSON: Thank you very much. Our6 last speaker for this morning is Linda Hostelley.7 Linda. And she is going to tell you who she is8 speaking for.9 MS. HOSTELLEY: Good morning, ladies and10 gentlemen. I am Linda Hostelley, the executive11 director of worldwide product safety in epidemiology12 at Merck Research Laboratories. 13 I would like to first thank the FDA panel14 and the entire working group for the opportunity to15 provide public comment today on behalf of the16 Pharmaceutical Research and Manufacturers of America,17 concerning concept paper number three, risk assessment18 of observational data, good pharmacovigilance19 practices in pharmacoepidemiologic assessment.20 As mentioned previously over the past few21 days, PhRMA supports FDA in the following concepts.22 We support FDA's efforts to provide guidance for23 developing pharmacovigilance in risk management plans.24 We support the concept that risk25 management is an ongoing process throughout a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1121 product's life cycle to optimize the benefit risk2 balance. We agree with the agency that the ultimate3 goal of any plan in its evaluation is to ensure that4 the efforts and costs involved in an RMP are expended5 on effective processes that achieve a positive benefit6 risk balance.7 PhRMA believes that risk management8 programs and interventions should balance access to9 drug with level of concern. We agree with the FDA10 that decisions which are made concerning individual11 drug benefit and risk, and RMPs, must be based on12 scientific evidence.13 We believe that any risk management14 program should rely on systems based interventions,15 and specific tools should be used consistently across16 RMPs. We recognize that efforts to identify all risks17 prior to drug approval may not be feasible without18 delays in drug development.19 And we recognize that collaboration among20 all the stakeholders, including FDA, the academic21 institutions, help care providers, pharmacists,22 professional organizations, and patients, is23 absolutely critical to achieve significant24 improvements in the benefit risk balance. 25 My comments today will focus on the first2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1131 three questions posed by the FDA in concept paper2 number three concerning good pharmacovigilance3 practices. Dr. Stephenson will address the questions4 on pharmacoepidemiologic assessment during the5 afternoon session. 6 The first question. How can the quality7 of spontaneously reported case reports be improved.8 PhRMA supports FDA's efforts to improve the quality of9 individual spontaneously reported cases which serve as10 the foundation for identification of safety signals,11 and are the critical building blocks of a post-12 marketing safety surveillance system?13 We recognize the importance of obtaining14 complete information for these reports, but recognize15 that a balance must exist between the significance of16 the clinical event reported, and the intrusiveness of17 follow-up with treating health care professionals.18 The more onerous, intrusiveness, or19 threatening the follow-up, the more likely the number20 of reports will diminish. Such an outcome is exactly21 the opposite of what is needed. 22 It would be useful to all stakeholders if23 the agency provided a prospective on what aspects of24 case reports, what sources of reports, and what sort25 of follow-up has been most helpful in signal detection2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1141 and regulatory actions.2 It would also be useful to know that where3 there is waste in a reporting system, and without a4 formal study of these questions, it is speculation as5 to how to improve the quality. 6 The provision of adequate clinical7 information is a shared responsibility with all8 stakeholders. PhRMA supports the education of all9 partners involved in post-marketing safety10 surveillance to improve the understanding of the need11 for reporting and the quality of reporting.12 To that end there should be education of13 practitioners to communicate high quality data with14 specific clinical details, and education of patients15 to recognize signals and report specific details.16 There is also a need to increase awareness17 of the importance of pharmacovigilance in clinical18 pharmacology for health care professionals during19 their medical, nursing, or pharmacy school curricula.20 Education of these health care providers21 on a continuum throughout their careers will improve22 the understanding of the need for reporting and the23 quality of reporting. 24 PhRMA and FDA should continue to partner25 with medical, nursing, and pharmacy schools to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1151 increase the awareness of the importance of2 pharmacovigilance in clinical pharmacology.3 Partnership with health care organizations4 and academia is also needed to reinforce these5 principles. This effort to improve report quality6 will also benefit from continued collaboration with7 the National Patient Safety Foundation. 8 Question Number 2. What are the possible9 advantages or disadvantages of applying data mining10 techniques to spontaneous report databases for the11 purpose of identifying safety signals? 12 One of the greatest challenges in post-13 marketing pharmacovigilance is determining what14 constitutes a safety signal. Post-marketing adverse15 event databases, which are based on voluntary adverse16 event reporting, lack formal controlled data, and17 exposure data, and until recently have been difficult18 to use objectively and efficiently for signal19 detection.20 Traditional pharmacovigilance methods have21 consisted primarily of individual case review and22 empiric analysis of crude frequency data. During the23 past 5 years, tremendous progress has been made in the24 development and application of statistical methods for25 systematically screening post-marketing adverse event2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1161 databases to efficiently identify adverse events that2 occur at a higher than expected frequencies in the3 absence of exposure data. 4 Examples of these data mining or5 disproportional analysis methods include the6 proportional reporting ratios developed by the U.K.7 Medicines Control Agency, the multi-item gamma poison8 shrinker, developed by William Demichele, of AT&T, in9 collaboration with Anna Szarfman, of the FDA; and the10 Beijiang (phonetic) Neural Networks, developed by the11 WHO. 12 Recently these sophisticated tools for13 data mining have become more available to regulators14 in industry through packaging and deployment of modern15 easy to use web-based interfaces. 16 Data mining may therefore become a useful17 adjunct to traditional pharmacovigilance methods, and18 that it provides a systematic, efficient, and19 objective approach to screening large adverse event20 databases for safety signals.21 It may be especially useful for detecting22 safety signals involving rare events that are unlikely23 to be observed in clinical trials or observational24 studies. These rare events may also allude detection25 by traditional pharmacovigilance methods. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1171 Other potential applications include2 screening for signals associated with polypharmacy.3 For example, drug interactions, and for demographic4 factors associated with adverse events. 5 Data mining analyses compliment data from6 well-designed, well-monitored clinical trials, and7 observational pharmacoepidemiologic studies, and can8 be used to generate hypotheses that can be further9 tested in such studies. 10 Furthermore, in some instances, the11 information gleaned from data mining can be used to12 enhance risk management and clinical development13 planning. The limits, however, of the underlying data14 and the data mining techniques must be fully15 appreciated to avoid false positive causality16 conclusions with costly interventions and17 inappropriate product restrictions.18 Research efforts should be targeted to19 reduce false positives. To this end a collaborative20 working group of statistical and medical experts from21 both the industry and FDA was recently formed. 22 The goal of the working group is to define23 best practices for the application of data mining24 technology that will optimize use of these tools in25 the areas of pharmacovigilance and risk management.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1181 The group will seek to better understand the2 comparative performance characteristics of various3 data mining methods.4 The optimal specifications and5 configurations for various uses. Data quality issues6 and the limitations of these methods, particularly as7 they affect the interpretation of the results.8 It is also hoped that this joint effort9 will provide an opportunity for FDA and industry to10 develop a common language, and share systematic11 approaches to the detection and assessment of signals12 from post-marketing adverse event data.13 Question Number 3. What are the possible14 advantages or disadvantages of performing causality15 assessments at the individual case level? Given the16 inadequacy of spontaneous report data, the application17 of causality algorithms to a single case is fraught18 with misinterpretation.19 The ability to rule out the likelihood20 that the suspect drug may have contributed to the21 adverse experience in most instances becomes22 impossible. Therefore, most adverse experiences at23 the individual case report level end up with a24 possible association.25 With the exception of cases involving a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1191 positive rechallenge, there is little or no advantage2 in performing causality assessment on individual case3 reports. 4 In addition, a lack of consistency among5 experts in the evaluation of causality assessment for6 individual cases strains the already limited7 resources.8 Although a series of cases may be used to9 generate hypotheses concerning the association between10 an adverse experience and drug exposure, there is no11 methodology determined to date that is reliable and12 reproducible for individual causality assessment.13 Thus, causality assessment at the14 individual case level is open to a high likelihood of15 misinterpretation. In summary, PhRMA supports the16 development of best practices for improving the17 quality of spontaneous reports and for developing the18 methodologies for application of data mining19 techniques. 20 PhRMA also supports education of all21 partners involved in post-marketing surveillance. In22 closing, PhRMA would like to pose an additional23 question for the FDA. What criteria should be used to24 determine whether a pharmacovigilance plan describing25 pharmacovigilance efforts above and beyond post-2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1201 marketing spontaneous reporting is warranted. 2 Thank you for this opportunity to provide3 comment on behalf of PhRMA. 4 DR. GALSON: Thank you very much. We now5 have about a half-an-hour for a discussion session and6 question and answers, and if all the six speakers from7 this morning, the six presenters, want to sit at the8 table, that would facilitate this.9 I don't know if everyone is still here,10 but we have got six seats, and there were six of us.11 And so we will take questions from cards and people12 are walking around to pick up any other questions on13 cards. 14 People can come up to the microphones and15 we will just have a free exchange between those of us16 up at the front of the room and people who want to17 come up and talk. I think I will just start with a18 question while people get themselves going.19 And this is one that came in on a card and20 it says please address the effects of MedRA on label21 and safety signals, and I think that is one of our22 working group members, Min Chen, is going to address23 that.24 DR. CHEN: It is the ADR proposed rule25 that requires all adverse events in the case reports2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1211 coded in the most recent version of MedRA. Currently,2 however, the ADR individual events, or the organ class3 in the drug labeling has no specific guidance as to4 what to use. 5 But in a safety signals detection in post-6 marketing, we use the measure terminology to store and7 to retrieve clinically irrelevant cases in the8 databases.9 Once risk is identified, currently it is10 the clinical or medical concept of the identified risk11 information that will be considered for incorporation12 into the labeling.13 In the future, I think there may be some14 considerations in the agency to have a consistent15 labeling on the specific ADR information in the16 labeling using MedRA terminology. 17 But I think that still has some way to go,18 because the clinical trial data should be coded in19 measure first before they use the information in the20 proposed labeling. So that is the current situation21 right now.22 DR. GOLDMAN: Are we allowed to comment on23 that?24 DR. GALSON: Yes.25 DR. GOLDMAN: Okay. A couple of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1221 principles if I may, and I say this to the other2 people who basically helped develop the measure3 sections. The tail should not be wagging the dog.4 Labeling is designed to convey clinical concepts. 5 And there have been several studies done,6 such as work Ali Brown (phonetic) has done, and been7 published, and others. That you could actually have8 terms that when the MedRA version changed, and went9 from labeled to unlabeled, and I have -- I am putting10 out a cautionary note in public, and as we have done11 in print, that MedRA was not designed for that in12 particular.13 And that that has to be borne in mind when14 we talk about labeling, particularly when we would be15 coding measure in terms of clinical trial data, and I16 just wanted to have that point.17 DR. GALSON: Okay. Let's move to the18 mikes now on this side. Please identify yourself.19 MR. STANG: Paul Stang, Gold Associates.20 A couple of general comments. One is that when you21 read through the document, although there is22 separation between signal detection and evaluation, I23 think it is very important to make that absolutely24 crisp and clear, because I think that a lot of people25 confuse signal detection with signal evaluation.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1231 I think that there are just many2 opportunities to clarify that. A second comment has3 to do with data mining, and that is whether or not we4 have progressed to the point where we have5 complimentary rules for when we don't find a signal,6 inasmuch as when we do.7 And the point here is that the fear would8 be that there would be multiple looks at the data many9 times over time, and without proper statistical10 adjustment, or other ways of accounting for the fact11 that you keep going over time looking for something,12 and eventually you will find it, is my sense if you13 look enough times.14 So I think that this is just a plea to, as15 others have said, to develop some methodology around16 it. The third point is that I work with a non-profit17 group of physicians, a grass roots organization, and18 we developed just on a whim a self-training module in19 safety surveillance, just because we had heard from a20 few of the 10,000 members that they had an interest.21 And we developed a little CD-based22 training system, and a thousand of them were handed23 out -- well, actually, people had to request them, and24 all thousand of them went in two weeks.25 So there is great thirst for this2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1241 information at the primary care level. It is a2 primary care based organization, and I actually shared3 an early version of this with members of the agency.4 So I do want to point out that primary5 care is screaming for this, and that as someone has6 pointed out, they don't get this kind of training in7 medical school, and they certainly don't get it8 subsequent to that. They themselves are looking for9 more information. 10 And finally along those lines is the11 training for the actual consumers of this information,12 and I specifically have been struck in a couple of13 meetings that I have attended with the agency with14 their external clinical advisory boards for various15 products that are coming up under review post-16 marketing for certain issues.17 And I think that there is a great18 opportunity to educate some of the external clinical19 experts that are brought into these meetings along20 safety issues, and the resources that are available,21 and with what their strengths and what their22 limitations are. Thank you. 23 DR. GALSON: Thank you and I think I will24 turn to Dr. Beitz now, who had a question.25 DR. BEITZ: I had a question actually that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1251 I would like to direct to some of our industry2 representatives here today. And it has to do with the3 application of an FDA newsletter of adverse event drug4 combinations.5 And there is really some sensitivities6 around publishing such a newsletter, and how would we7 choose which events and which drugs might be included8 in a particular issue.9 And I would just like to get a sense of10 how people would think about publishing on particular11 adverse events or combinations even though it has not12 been totally worked out as to causality. Labeling may13 not be fully worked out or discussed, or negotiated.14 Do people feel that a newsletter approach15 could or could not occur in parallel with the more16 traditional labeling negotiations, or does something17 have to be in labeling first before folks in industry18 would feel comfortable about FDA putting it in a19 newsletter?20 MS. HOSTELLEY: Linda Hostelley,21 representing PhRMA, and perhaps others from the22 organization will also comment, but I would say that23 this is an interesting concept, and I think PhRMA24 would certainly welcome an opportunity to have further25 dialogue on this topic.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1261 Again, you know, the concerns from an2 industry perspective would be, you know, how much can3 be said before an actual label is constructed, and of4 course there are the legal implications for us as a5 regulated industry.6 But certainly it is an interesting7 concept, which I think would warrant further8 discussion. 9 MS. STEPHENSON: Can I add to that?10 DR. GALSON: Sure.11 MS. STEPHENSON: Wendy Stephenson from12 Wyeth. I actually -- my first impression is that a13 newsletter of that sort would be very problematic. I14 think it would be much better to have fully vetted the15 issue, and evaluated data, and come to a conclusion,16 which then would usually result in a labeling change,17 or result in the conclusion that there is not an18 issue.19 And then perhaps it might be reasonable to20 publish. We have had some very sort of strange21 experiences with a similar type of newsletter that the22 Upsalla monitoring center of WHO has issued.23 They generally issue these reports of24 signals -- they are called signals -- and then after25 the publication, they will ask the company for a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1271 comment. And in every case where we have had one of2 those signals referred to us, it was a completely3 spurious association, and there was generally a pretty4 good explanation for what they were seeing.5 And there was no causal association. So I6 would just caution against publishing preliminary7 information before it has been fully vetted. 8 DR. VASHISHTHA: Anshu Vashishtha from9 Watson. I think another issue that of course comes up10 is the issue of the importance of sequence reporting,11 the stimulated reporting, and the recall by issues12 that have come up with such a publication.13 I think that is where we have to clear the14 package inserts and they have professional letters and15 other ways to have established lists (inaudible)16 risks.17 And anything done prior to that in my mind18 to the general public would distort the evaluation of19 that risk. 20 DR. GALSON: Alan.21 DR. GOLDHAMMER: Yes, Alan Goldhammer,22 PhRMA. I would echo Dr. Stephenson's comments on23 this. I think that you can go back and look at the24 literature being rife with examples of preliminary25 assessments of risks, which later are disproved. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1281 Unfortunately, companies in many cases2 have incurred significant legal bills defending3 themselves in court. I mean, we would be more than4 happy to look at the proposal, but I am not terribly5 optimistic that the legal side of PhRMA would be6 encouraged by it.7 DR. GALSON: Okay. We are done with8 follow-up on that question. Oh, one more. Okay. 9 DR. NELSON: I was going to comment that10 what could possibly be more valuable is after the11 labeling change has been agreed to, have a publication12 that explains the evidence base for that labeling13 change.14 DR. GALSON: Good. Okay. I think we will15 move to the next mike question.16 MS. OMINOFF: Okay. I am June Ominoff17 from the directorate at GlaxoSmithKline, and I am also18 the chair of the PhRMA/FDA working group on safety19 evaluation.20 And I am actually up here to respond to21 Dr. Faich's comments, and concerns about data mining.22 We thank him for his comments, because they do23 highlight some important caveats about the tool.24 However, I want to make two points.25 The first is that if we adapt Min Chen-2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1291 Julie Beitz's, the working group's definition, of a2 signal which is, quote, apparent excess of adverse3 events associated with the use of a product, then what4 could be more valuable then a disproportionate5 analysis or data mining tool that objectively applies6 to statistical methods to identify these excesses.7 My second point is that he addresses8 examples of false positives, or what he calls false9 positives, and I wanted to say that no one who uses10 this took responsibly would ever, ever draw11 conclusions without reviewing the medical case12 information using standard pharmacovigilance13 practices.14 So in my view, the examples that Dr. Faich15 has presented as false positives really seem to16 reflect an intermediate stage in the evaluation of a17 signal. Thank you. 18 DR. GALSON: Thanks a lot. 19 DR. GOLDSMITH: I would like to address20 the same issue with regard to data mining. My name is21 David Goldsmith, and I am a consultant, but I am22 representing myself. And the issues that I would like23 to raise are actually on both sides, and then follow24 it up with a question. 25 First is the issue of the false positives,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1301 and I would like to present a rather simple scenario2 that gives an example of a false positive. You have3 Drug A, which is a recently developed drug, and it is4 effective. It is so effective that it works in a5 hundred percent of the patients who are treated.6 It has a very clean profile and indeed7 after being marketed for 6 months there are a hundred-8 thousand patients exposed, and there are 10 serious9 adverse events that are reported, nine of which10 represent migraine headache.11 The other drugs in the class that are used12 to treat the disease are not terribly effective. They13 are only about effective in 50 percent of the cases,14 and they are very dirty drugs. And they actually have15 lots and lots of serious adverse events, and they are,16 for example, in that same 6 month period with that17 same sales ratio 500 cases of migraine headache, and18 500 cases of other serious adverse events.19 And using a proportional reporting ratio,20 we come up with a proportional reporting ratio of 17,21 which far exceeds the three that one would normally be22 seeing, all right?23 And therefore would potentially represent24 the signal. Given that people would very likely look25 at the denominator effect, which is clearly evident in2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1311 this, and one would assume that the signal is probably2 an erroneous signal.3 But at the same time when does one turn4 off the system in order to stop looking for a signal5 because this was a false signal. That is my question.6 The second point is that there are false7 negatives associated with data mining, especially as8 it relates to the utilization of MedDRA. I presented9 a paper at the annual DIA meeting last year which10 looked at two specific conditions. 11 One was member proliferative glomera12 nephritis, a rather rare occurrence, but is associated13 with heroin necropsy. The other was TTP, again a14 rather rare entity, but is associated with some of the15 anti-platelet agents.16 In neither of those cases could you use17 either the MedDRA term itself at the low level,18 preferred term, high level, high level group term, or19 at the SOC level, to make sure that you had all of the20 cases and eliminated appropriate cases that did not21 belong there when you had appropriate case22 definitions.23 So one really needs to be very, very24 careful about the data mining considering, one, how25 has the terminology been put in, and two, what do you2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1321 do with false positives, and how do you turn off your2 risk management program when a false signal is3 identified.4 DR. GALSON: Who would like to address5 that?6 DR. SZARFMAN: Can I say something? May I7 -- I brought a computer, a laptop, and we can look at8 the results that data mining show for the specific9 examples, and then let's see how data mining10 performed, because we need to look at the data.11 DR. GALSON: So you are going to do that12 in the back room?13 DR. SZARFMAN: Yes.14 DR. GALSON: Wow. Full-service FDA here. 15 DR. GOLDMAN: I would like to make a16 comment. Steve Goldman.17 DR. GALSON: Yes.18 DR. GOLDMAN: We have been hearing very19 good points of view obviously from Dr. Goldsmith, and20 Dr. Szarfman, and also Dr. Faich. I am going to do21 the same cautionary note that I am going to say about22 MedRA. 23 This is a technique. This is a tool. It24 does not replace the greatest tool of all, the human25 brain. It does not take away what we have learned in2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1331 50 years of pharmacovigilance and how we look at each2 individual case.3 And I think if I may point out that if we4 can take down the level of the heat and show a little5 light that this is another tool that we are using to6 look for possible signals to use the dynamic database,7 such as the AERS database, as to look for8 possibilities.9 And I think that in that sense I think we10 have more of a consensus than we have disagreement11 about how the actual application should be. 12 DR. GALSON: Dr. Faich, did you want to13 contribute something to that?14 DR. FAICH: Yes.15 DR. GOLDSMITH: Can I follow up?16 DR. GALSON: Wait just one second. 17 DR. FAICH: Can I just say that I don't18 want to be misunderstood. I am not opposed to data19 mining, and I think it is an interesting technique and20 a research tool that is in development. So I think21 that it does have utilitarian interest. 22 The second point that I would simply make,23 and I was trying to make it by giving that Yogi Berra24 comment earlier, was that most signals I believe will25 come from alert safety monitors either at the FDA or2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1341 at companies who are at the intake end of important2 reports.3 And that the main triage tool is to look4 for a serious unlabeled. It doesn't help very much5 for a serious labeled increased frequency, and I think6 we all know that, and I believe that is one of the7 main utilities that data mining can help us with.8 I also would like to point out that Lou9 Lisana (phonetic) many, many years ago said that the10 most important source of new information about safety11 for drugs is the alert observant clinician who sees12 something out of kilter.13 And I think the same thing is true for a14 safety monitor inside the FDA or perhaps maybe more15 importantly inside a manufacturer who has launched a16 new drug and now receives reports that he didn't17 expect to receive.18 The issue of whether false signals will19 occur or will be lessened because those who use this20 steel-edged axe are indeed sophisticated and will go21 back and review the data is dependent upon whether22 they have the data in-hand or not. 23 My fear is that decisions or alerts will24 be made based on only the electronic data, and not a25 more complete clinical review of individual cases once2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1351 a preliminary finding has been found.2 That is exactly what WHO has done, and3 that is exactly what happened in the case of Claritin4 that I cited, and that is exactly what happens when5 attorneys troll through the database and do crude case6 count, such as prozac and suicide.7 DR. GOLDSMITH: Just a follow-up to8 Steve's comment and also to Gerry's. I think that it9 is very important that we keep our eye on the ball and10 watch what is happening to the signal to noise ratio,11 and we are increasing to a large extent the amount of12 noise.13 Yes, we are increasing the signals, but we14 are increasing the noise to signal ratio so much that15 we won't be able to find those signals terribly16 effectively.17 And the same thing is true, for example,18 of the data that Steve presented. Yes, there was a19 17-fold increase in reporting following the Rhode20 Island program, and there was a 9-fold increase in21 serious reporting, but 17 versus 9, that seems that22 there was a lot more of a bigger increase in the non-23 serious.24 DR. GOLDMAN: I will respond to that.25 There was also 31 previously unsuspected unlabeled2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1361 serious events that were reported. I will stand with2 having to make the choice between getting more3 possibility of drops into a system if it were to4 enable me to get things that I didn't previously know5 about, and that is my philosophy in terms of that.6 DR. GALSON: Okay. Let's move on. Next,7 Dr. Szarfman.8 DR. SZARFMAN: Well, I will try to answer9 some of the things that were said here. First of all,10 data mining is a tool. It is not the only tool.11 There are different tools. Data mining is a tool, but12 we have 13 -- I don't think that there is a chance again that we14 are having this meeting at the National Transportation15 and Safety Board. 16 The human brain is very important, but we17 have and we not always remember to fasten our seatbelt18 and we need a light to let us remember that we need to19 do so; and to airplanes that are maybe too close, the20 human brain, of course, you know, of course -- you21 know, we might compare that with other things that we22 need to be alerted.23 And this is essentially what we are trying24 to do. Data mining is not data dredging. We are25 applying one statistical method to the whole database,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1371 and there is not a problem of multiple logs, because2 using this method, you can do multiple comparisons.3 And to include all of the contents that4 you need to make the assessment. We don't have5 backgrounds, background information for specific6 diseases. And it will take a long time to get that7 part down. 8 If we can analyze every single drug used9 to treat that indication, and then it will keep us10 informed as to which level of signals you have for a11 certain indication.12 We have also the human brain, and we have13 in the database over 50 quadrillion combinations for14 drugs. This means a 5 with 16 trailing zeros. I15 don't think that the human brain can efficiently keep16 up with these.17 You know, if you don't have the gold18 standard for these, and what the profile of every19 document, and we don't. You know, that's why we need20 systematic approaches. 21 DR. GALSON: Okay. Can we move on to the22 next. Thanks, Dr. Szarfman. I want to just use one23 of the questions that came in on a card. We have24 heard the call to capture safety data electronically.25 How can this happen with the industry2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1381 pushing back on Part 2 of HIPAA, and other security2 issues are implied in this question as well. Do one3 of the members of the working group want to address4 that?5 MS. CIAMPA: Well, first of all, I guess--6 DR. GALSON: If you could identify7 yourself, please.8 MS. CIAMPA: I'm sorry. I am Aileen9 Ciampa and I am with the Office of Regulatory Policy10 in CDER. First of all, I would just note that with11 regard to Part 11, the commentor might be aware of12 this, but FDA did just recently issue a guidance13 document regarding our approach to Part 11.14 And in trying to make sure that it is15 consistent with the adoption of new technologies, and16 that is an ongoing process. And more broad based, I17 guess I would just note that with regard -- and I am18 not a HIPAA expert, and so there might be people here19 who can speak better to that.20 But I would just note that both patient21 privacy concerns and concerns for data integrity and22 record integrity have always been present in the23 collection of safety data and the transmission of24 safety data.25 And so although I think the movement to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1391 the electronic environment might have implications for2 new regulatory schemes, and might pose additional3 concerns, that the fact that those concerns, and4 particularly the patient privacy and data record5 integrity, should not prevent us from moving and6 adopting new technologies.7 DR. GALSON: Okay. Anybody else want to8 address that issue?9 DR. VASHISHTHA: A new issue.10 DR. GALSON: Any other questions on the11 security and those related issues before we move on to12 something else. Okay. 13 DR. VASHISHTHA: Just a question for some14 of the members of the working group possibly. One is15 that there might have been already done some16 evaluation of the algorithms and causality assessments17 used in Europe or globally, and secondly, whether if18 data mining is only one of the tools, then defining a19 signal as an apparent access of adverse events in20 association with a product sort of biases us towards21 more of a data mining approach.22 What about redefining a signal as an23 apparent risk associated with the products? Any24 comments on that from the working group members?25 DR. GALSON: Dr. Chen. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1401 DR. CHEN: We are aware of many2 publications and many individual's proposals for the3 criteria of causality assessment, and that we have not4 found a consistent way that can help us as a standard5 or a guideline to use currently.6 So we are exploring all these7 possibilities and getting inputs from you all. I8 think that your suggestion was wonderful, to look at9 the European ways, and if that is something that most10 people think is applicable to us, we certainly will11 consider it.12 And the second question was about the13 signal difference?14 DR. VASHISHTHA: (Off microphone) on the15 first question, also about using the existing database16 to perhaps evaluate the appropriateness of some17 algorithms with respect to that.18 DR. CHEN: I think that we are all19 exploring all possible tools helping us out to20 identify the signals, and human brains are limited in21 some ways, but I think that in the process that we22 like to prioritize different kinds of signals that we23 like to see as a work-up first.24 So I think that is something that is only25 probably reasonable for such a big volume of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1411 information coming into the agency every day. So our2 priorities usually are the more serious ones, like on3 liver failure, and severe skin reactions, and such as4 all of the events mentioned in the medically5 significant events, and we pay more attention to that.6 And data mining, as we said before, is an7 adjunct to that to help us out to identify signals8 that human eyes cannot detect right away. As far as9 the definition, I think we are open to suggestions on10 access to a certain degree, and nobody knows when for11 a new drug that just came on the market with new12 populations, and that we have never seen before, maybe13 one case of a serious event is worth looking for and14 looking into a further investigation.15 But for other drugs, you probably need a16 lot of events to support the hypothesis that the drug17 really caused that. So I think this all needs to be18 considered in here on how much is access and what is19 the threshold for a signal detection.20 DR. GOLDMAN: There is a very famous law21 called Sutton's Law; you go where the money is, and22 with the always expedited reports under the new23 proposed rule, with the designated events which24 triages errors, we are talking about scarce resources,25 scarce time, and yet public safety.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1421 And the idea that we look for the most2 serious events -- and again the concept of risk, well,3 that entails benefit risk. When you are looking at4 signals, I think we should be looking at -- my5 philosophy is are the actual events themselves are the6 ones that are the bad actors, the ones that we know7 about.8 The ones that tend to be lethal, in9 particular, and then tie that on in terms of that. I10 gather, Min, that is the philosophy?11 DR. CHEN: Yes, I agree.12 DR. GALSON: Okay. Let's move to the13 question over here at the mike. 14 MR. HARVEY: Yes, in a complex area where15 we are trying to move the ball forward in a very, very16 unknown way, I think it is often -- I'm sorry, Bill17 Harvey from Life Free Technology. 18 It is very often worthwhile to keep things19 as simple as possible. And the confusion between20 large simple safety studies and Phase IV studies has21 been very difficult for me to comprehend, and22 especially in the context of a statement that was made23 on Monday from Dr. Paul Stolley, who said that 1324 percent of Phase IV studies have been completed. 25 And implying that fully 87 percent of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1431 those studies have not been completed, and given the2 expertise in this room either from PhRMA, from the3 FDA, or from others, I think that is worth discussion.4 DR. GALSON: Do you want to form it into a5 question?6 MR. HARVEY: What is going on?7 DR. GALSON: Who are you asking?8 MR. FENISHELL: May I speak to that? This9 is Bob Fenishell, now a consultant, but at FDA for a10 long time. We looked into this in the division when11 those data were being gathered, and so we went back12 and looked at all of the Phase IV commitments that we13 had gathered from sponsors, and then looked at indeed14 what fraction of them had been completed, and15 certainly the figure of 13 percent is consistent with16 our experience.17 It was not a large fraction, and then we18 said, gee, how do we feel about this, and just looking19 at the raw numbers one must indeed ask as the20 questioner asked what is going on.21 But in fact you start looking at one of22 them after another, and you know, that was a pretty23 stupid thing to ask for. And that other thing, you24 know, that was a pretty good thing to ask for. That25 would be really interesting to have.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1441 But we now realize that it is just2 impossible to collect that data, or you know, that was3 probably a pretty good idea at the time, but it has4 been superseded by events. 5 We no longer believe that that is a6 significant problem with that class of drugs, and so7 it perhaps was appropriate, but it is no longer to8 push the sponsor to find out whether it is happening.9 And so at the end of it, and of course10 there was self-serving and one may have some11 skepticism about the freedom from bias of our process,12 but we thought that he had not done badly. 13 DR. GALSON: Victor. 14 DR. RACZAKOWSKI: I am Victor Raczkowski15 from the Office of Drug Safety in the Center for16 Drugs. The only thing I will add is that Phase IV17 studies are not always safety studies. Many of them18 are pharmacokinetics studies, and there might be19 chemistry manufacturing types of studies, and so just20 keep that in mind as you talk about Phase IV studies.21 DR. NELSON: I might also be able to shed22 some light on that 13 percent figure, because that23 comes from an internal regulatory research report that24 Dr. James Grumlish and I did in the early '90s. I25 have two things to say about it.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1451 One is that it was on a time frame -- I2 think the data was pulled from the late 1980s, and3 into early 1990, and so that data are real, and it did4 look just at the safety Phase IV studies, and not all5 the other kinds of Phase IV studies.6 And that is a very valid point, Victor,7 because actually most of the Phase IV studies are8 biopharmaceutical, and long term cancer studies, and9 all kinds of other things. 10 But that study was submitted to the agency11 and was actually submitted down to Congress prior to12 the FDAMA Act being developed. It also served as the13 initiator for the Phase IV tracking system that you14 have at the agency, as well as the stipulations in15 FDAMA about putting the Phase IV requirements for16 approval in the approvable letter and monitoring those17 after.18 And so it did have an impact. I think19 there has been some remedy at the agency for that20 deficiency, and I would love to see that study redone21 to see if the rates have gone up in the last decade.22 DR. GALSON: Just one note on Phase IV23 studies, is that we are about to activate in the24 agency a new website which is going to allow people to25 search those studies directly; and without spending a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1461 lot of detail on this, I think all the comments that I2 have heard are fairly or are valid and a lot more as3 well.4 The 13 percent number I think is really5 sort of deceptive, a deceptively simple description of6 what is actually going on there, which is quite7 complex. But we will be making more information8 public about what is actually going on with those, and9 the follow-up on those studies, soon.10 Dr. Stephenson, I think we will take this11 last question and then if we could hold all additional12 questions, we have got a lot more time in the13 afternoon than the morning for this type of14 discussion. We will have time for all the questions.15 DR. STEPHENSON: Okay. My comment, and I16 guess perhaps a question for FDA, has actually been17 touched on slightly two questions ago. And that18 relates to signals, and the description of what a19 signal is in the concept paper, I think it needs to be20 flushed out more, especially in just listening to this21 discussion, everyone seemed to use the word signal in22 very different ways.23 And I have heard possible signal,24 potential signal, and signal, used sort of25 synonymously sometimes. I think the description of a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1471 signal is good, but it is a broad description that I2 think encompasses several degrees of strength of a3 signal. 4 Many of us are used to having a very low5 threshold for calling something a signal, and which6 would then trigger you to take further action in terms7 of pulling data together and better assessing the8 situation.9 But more times than not those sort of10 signals end up being non-issues. And so I raise this11 because when you read further on in the document,12 there is a section that refers to reporting the13 signals to FDA, and the expectation that if there is a14 signal that the FDA would expect the company to submit15 a fairly comprehensive report, which would include16 pre-clinical findings, and a discussion of potential17 risk management plans, if that is appropriate.18 And even further in the document there is19 some language to suggest that risk, management plans20 may be a way to mitigate a signal, which sort of21 brings you to the point of a signal being used almost22 synonymously with risk. I wonder if Dr. Chen, if you23 can comment on that.24 DR. CHEN: I think the points are well25 taken. We have not addressed the difference between2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1481 signals and risk and certainly we are still at a level2 with the signals on what does that mean when we3 identify from the database.4 And then what do we do with the signals,5 and what we are hoping is that the sponsor can help us6 out to fill the gap of when do you find that the7 signals are useful to become a risk, a real risk, and8 that something has to be done with that risk.9 And how big is the risk, and again it10 depends on how big is the signal. So hopefully this11 afternoon some of the studies and some of the analyses12 of the information will help to bring the gap a little13 bit closer. 14 However, I think we need a lot of help15 from you all experts out there to help us out to do16 that. So putting in writing what you think about it17 and how we do that, I am really open.18 DR. GALSON: Dr. Braun.19 DR. BRAUN: Miles Braun, CBER, FDA. That20 last point was well taken. I think that to take a21 step back from the discussion today, this is a concept22 paper and the idea was to lay out different ideas and23 get discussion. 24 You have homed in on what is a good point,25 and I think that having reviewed the literature2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1491 somewhat in this area, I can say that there is not an2 agreed upon definition outside this room.3 And that the one that was laid out of an4 excess of beyond the expected in the reports certainly5 raises the issue of what one would expect. Some have6 taken exception to the expected values that are7 generated through automated data mining approaches.8 And there is certainly or that is9 certainly a well-taken point. So this needs to be10 flushed out further. It is in my view, it is a key11 issue, and I will throw out an additional thought, is12 that the actual actions, or actually two thoughts.13 One is that actually because we have14 different views of signals, the signal can be15 conceived of to some extent, and I am not trying to be16 subjective about this; but in the eye of the beholder.17 Let me give you an example. The physician18 who takes the trouble to fill out a MedWatch form19 because a patient that was being taken care of had a20 terrible adverse event to that physician, they are21 signaling to the FDA.22 And to somebody that sits at the FDA and23 sees 300,000 reports a year, this particular event may24 not seem to be a signal. And there are very many25 other people involved in the process. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1501 So I think that we need to think about who2 actually is the beholder, and the reason that I think3 that is important is that each person has a different4 action that he or she can take.5 That physician's action is to notify the6 FDA. As an FDA person, we have certain actions, and7 our agency can take certain actions, and similarly at8 a company, you have different repertoire of actions9 that are available to you.10 And I think that it will be helpful if we11 can blend in some of this, and as you said, if we can12 go into this in a little more depth, and we are only13 laying out the concepts.14 And to try to construct this signal idea15 in a way that will be helpful in terms of actions that16 we all can take.17 DR. STEPHENSON: Perhaps just one other18 point, and I appreciate the responses. I agree with19 what you are saying. I think one of the things that I20 was reacting to in the concept paper is that there was21 a discussion of reporting, which is really not a22 concept.23 And I think that if you start talking24 about the expectation that companies should report25 signals to the FDA, you are going to get barraged with2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1511 things that you probably would not consider signals,2 just because companies are going to want to be in3 compliance. So that was one of the things that I was4 reacting to.5 DR. GALSON: At the table, Dr. Faich.6 DR. FAICH: Yes, I was just going to make7 a comment and an observation, and it links to some of8 the discussion yesterday of Level 1, Level 2, and9 Level 3, and that the levels analogy takes on a life10 of their own. 11 And I would like Anne Trontell's comments12 of let's call them Georgia, Betty, and Sam, or13 something. We had a similar experience at one point14 long ago that is worthwhile sharing, and that is that15 we started talking about alerts driven out of the16 adverse drug reaction reporting system.17 And the term alert is a potentially18 evocative one, and that was the problem, and we19 learned that, and we changed the terminology to20 monitor to adverse reactions. And then of course we21 were then accused of bringing these from the outer22 reaches of the solar system and Mars. 23 But nonetheless, I mean, the point of that24 was to get away from a term that could be25 misinterpreted, and to go to a more neutral term, and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1521 I wondered whether thinking about not using the term2 signal, but using some other term, might be3 considered.4 DR. BRAUN: But not Georgia, or Betty.5 Those are taken.6 DR. FAICH: Miles, to follow up on your7 comment asking about causality, I have not got the8 exact reference about the authors, but I know that9 there was a look at the French causality assessment10 method, versus physicians reporting. 11 There was a tremendous rush to connect12 between the relative weight that the reporting13 physician placed on particularly dechallenge, as14 opposed to the algorithm system, and very much15 pointing to what you said, Miles, as to what the16 treating doc, or the health professional would place,17 as opposed to a straightforward algorithm, and that is18 one of the reasons again that we don't advocate for19 that, because the weighting is different.20 DR. GALSON: Is this a follow-up?21 MR. FENISHELL: Yes, it is. Yes, it is.22 DR. GALSON: Okay.23 MR. FENISHELL: I think what is the24 problem here --25 DR. GALSON: Please identify yourself.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1531 MR. FENISHELL: Oh, I am still Bob2 Fenishell. I think the problem here is what in3 computer terms would be called a missing level of4 indirection, and what I mean by that is that what you5 have defined as the notion of signal is that signal is6 an apparent excess of adverse events.7 And the focus has been continually on this8 idea of apparent. Well, is it real, and can we verify9 it by data mining, and can we verify it by this or10 that. But if we verify it, well, then it is adverse11 events.12 Now, the fact is that is not what in my13 experience most of the Phase IV stuff of the post-14 marketing stuff that I used to see coming across my15 desk at the agency consisted of.16 It consisted of things that were not even17 adverse events. Prolonged QT is not an adverse event.18 A doubling of hepatocellular enzyme levels is not an19 adverse event. The patient doesn't suffer from any or20 either of those things.21 And here are indeed drugs that cause22 colossal changes in the hepatocellular enzymes, or of23 QT -- not very many, but there are some -- without24 ever causing adverse events.25 The thing is that you are looking for an2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1541 apparent excess in one phenomenon, and then entirely2 independent -- you know, once you decide, yes, there3 really is an excess amount, and it is not apparent.4 Then you have to decide, well, is that a5 phenomenon that in fact is associated with a6 phenomenon that we care about, or is that just an7 anomaly.8 If you look at most big trials, you will9 see an excess of something. You will see that, oh,10 yes, the people getting active treatment had slightly11 higher bilirubin than the people who didn't, let's12 say.13 And you say, oh, well, that is something14 that makes you think about some liver disaster. Well,15 in the CAPRI trial, the Clopidogrel patients had16 slightly higher bilirubin, and it was something in the17 third decimal place.18 But with 19,000 patients, you can show19 that. And that was just absolute garbage. I mean, it20 didn't mean anything. I am sure that it is true, but21 it just doesn't mean anything. If you say that a22 signal is -- and I am not sure what the word is, but a23 signal is something that catches your eye, and it may24 be real.25 And now, first of all, it is appropriate2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1551 to see if it is real, and secondly, it is appropriate2 to see if there was any reason to worry about it.3 Those are two separate processes, and I think that a4 lot of the discussion has been concerned about the5 onus of, well, yes, it really is real, and there is a6 second step here.7 It really is real, but perhaps the answer8 is so what. 9 DR. GALSON: Thank you. Does anybody want10 to comment on that? That is sort of an open-ended11 point. All right. We are going to break for lunch12 now, but before we break for lunch, I want to let13 folks know that if you didn't have a chance to ask the14 question that you wanted to ask, we will have plenty15 of time this afternoon.16 I wanted to leave a question on the table17 particularly for the PhRMA folks so that they have18 something to do during lunch. They asked us what19 criteria should be used to determine whether a20 pharmacovigilance plan describing firm efforts above21 and beyond post-marketing spontaneous reporting is22 warranted.23 And my lunchtime assignment for them is to24 provide us an answer to that question, and then we25 will try to comment on that. And that is where we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1561 will start the discussion. Thank you.2 (Whereupon, at 11:40 a.m., the public3 workshop was recessed.)4567891011121314152 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1571 A-F-T-E-R-N-O-O-N S-E-S-S-I-O-N2 (1:12 p.m.)3 DR. GALSON: We are going to start out4 this afternoon with a number of presentations, first5 from the FDA, and then members of the public, and then6 we will move into the discussion, and question and7 answer periods. 8 So our first presentation today is from9 Dr. Judy Staffa from the Food and Drug Administration.10 She is an epidemiology team leader in our Office of11 Drug Safety. Thanks, Judy.12 DR. STAFFA: Good afternoon. I have the13 privilege of getting up here on the third day of a14 three day meeting on Friday afternoon, at one o'clock,15 right after lunch, and giving FDA's last presentation16 on this concept paper.17 So I feel very privileged to do that, but18 I also have been told that I have the authority if I19 need to, to burst into song at any time, and that way20 if I see everybody kind of nodding off, I may do that.21 And so since I am a really bad singer, I22 will try to keep my remarks brief so that we can get23 to the input that we are really here looking for24 today. But if you can try and stay with me, that25 would be great, too.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1581 Okay. What I am going to try to do today2 is give you an overview of the section of the concept3 paper that really tries to focus on what we are4 looking for from sponsors about observational data5 that we receive, whether it is in the form of study6 results or protocols, that go beyond the case reports7 that we talked about this morning. 8 Pharmacoepidemiologic studies obviously9 can be done at any time, but typically there are two10 times when these studies are usually recommended or11 most appropriate. 12 The first time is typically right after a13 product is marketed, or at the time of marketing, and14 the purpose of those studies is to further15 characterize potential safety signals that may have16 been identified during the controlled trials. 17 This is typically important, particularly18 of interest in patients that may be chronically19 exposed once the drug is marketed, or product is20 marketed I should say, or in patients with co-morbid21 conditions or concomitant drug therapies.22 Another time when pharmacoepidemiologic23 studies might be recommended would be after a safety24 signal is identified once the drug has been on the25 market. And as we talked about this morning, many2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1591 times a signal is identified first from case reports,2 and then that signal, we try to put that signal into3 an appropriate context using drug use data and4 sometimes calculating reporting rates as you have5 heard about this morning.6 We all recognize that reporting rates are7 very preliminary, crude tools, with which to describe8 and try to characterize the risk, but we really need9 to move to the next step to pharmacoepidemiologic10 studies in order to calculate an incidence rate.11 Just very briefly, and I know that this is12 overkill for this crowd, but very briefly there are13 several pharmacoepidemiologic study designs that are14 commonly used in pharmacoepidemiology. 15 The first is a cohort study, in which16 patients are identified based on exposure to a medical17 product, and then they are followed over time to18 estimate the incidents of various events.19 These studies, or the main advantage of20 this kind of study is that an incidence rate can be21 generated for various events. 22 A case control study is a design in which23 patients are identified based on the presence or24 absence of a particular disease outcome, and then25 their history is examined for the use of different2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1601 medical products.2 Case control studies are very useful3 because they can help us identify risk factors, in4 addition to the relative risk for an event occurring.5 Another design, called nested case control, is a6 hybrid of the two designs, in which a case control7 study is nested within a larger cohort of medical8 product users. 9 One of the main advantages of this design10 is that it allows you to both calculate an incidence11 rate, but also to then examine and explore risk12 factors. 13 And then there are other hybrid designs,14 and as the field progresses more and more designs are15 tried out, and some of them end up being very16 appropriate for use in pharmacoepidemiology. We17 encourage sponsors or researchers doing these kinds of18 studies to consider what design might be best for19 answering the questions at hand.20 In 1996, the International Society for21 Pharmacoepidemiology published a document called22 "Guidelines for Good Epidemiology Practices for Drug,23 Device, and Vaccine Research." And this document can24 be found or can be linked to through the ISP website.25 We find this document very helpful and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1611 very useful, and would encourage people to consult2 this document when undertaking a pharmacoepidemiologic3 study. 4 For the purposes of our concept paper, we5 selected just the minimum requirements that we would6 like to see when protocols are submitted, or studies7 are submitted that are just worth going over today8 very briefly.9 First, we would like to see a clearly10 specified study objectives, and these objectives11 should be tied in with the analysis so that it can be12 easily seen how the study can achieve and answer the13 study questions defined.14 There should be a critical review of the15 literature that can take advantage of earlier16 experience with either the drugs or products of17 interest, and as well the outcomes of interest.18 And we would like to see detailed research19 methods. That includes a detailed description of the20 study population, including comparator groups that21 have been identified as being relevant, as well as22 operational definitions of both exposure and outcome23 that will be used.24 We would like to see a full description of25 data sources that are going to be used. We try very2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1621 hard in the Office of Drug Safety to get out to2 meetings and to see what kinds of data sources are3 available, because that is always a very evolving4 field. 5 But we often run into protocols where6 there are -- into data sources that are unfamiliar to7 us, and we welcome detailed descriptions of those data8 and the streams that generate them.9 We would like to see projected sample size10 calculations, and the associated statistical power.11 As you know, in the world of rare events, power12 becomes a very important issue when determining13 whether to go forward with the study. 14 And finally we would like to see detailed15 methods for how data will be collected, and how it16 will be managed, and how it will be analyzed. In17 pharmacoepidemiology, we use a lot of automated data,18 whether it is administrative claims data or19 computerized patient record data, but these present20 interesting challenges for some of the studies that we21 do.22 And what we have listed here are just some23 factors as sponsors or researchers are choosing the24 appropriate database to study the question of25 interest, and we encourage you to openly discuss these2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1631 in the protocols, in terms of how you came to the2 choice of the database where you have decided to3 conduct your study.4 Some of these factors include the5 demographics of the patient population. Clearly, if6 you are studying an issue in children, or in women,7 you would want to select a data resource that captures8 a lot of information on that subpopulation.9 The turnover rate or mobility of the10 population in and out of the database is critical,11 particularly for studies that are looking at12 cumulative risk. As you know, many American health13 care is very fragmented. 14 People have a great deal of mobility, and15 since many of our data sources are based on health16 insurance plans, we are limited to the length of17 follow-up that we can actually capture people, which18 is as long as they stay in the system.19 Plan coverage of study medications is20 another crucial element to look at. You need to21 understand what both the uptake of a product is, and I22 know that was mentioned this morning, but also whether23 or not a plan will cover particular products when you24 are looking to study them.25 There are many plans that will not cover2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1641 products they consider to be lifestyle drugs, such as2 weight loss products, or products used for erectile3 dysfunction, and it is an important consideration to4 understand.5 We would like to have detailed information6 on the size of the population available for study, and7 in automated data sources that is a particular issue,8 because rather than simply get a first take which says9 that, gee, I have 10,000 patients that had at least10 one prescription for this drug in the database, we all11 know that by the time that we get done apply different12 enrollment and eligibility criteria, that 10,00013 number may be quite substantially lower.14 And any estimation that you can give of15 that quantity would be very helpful in trying to get a16 realistic sense of how large the population will be.17 It is important to understand whether the outcomes of18 interest are actually available in the database, and19 one very good example of this is death.20 Death would seem to be a very21 straightforward outcome to study, but in an22 administrative claims database, unless death happens23 in a health care facility where care is being24 provided, it may not be captured. 25 A death may just be the absence of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1651 continued claims, and so it is important to consider2 whether the outcomes that you want to capture are3 actually available in the data.4 It is also important to understand whether5 the outcome of interest is actually coded, or there is6 an specific ISD-9 code since many systems, at least7 administrative claims systems, use this coding system,8 and it is important to know if the code that you are9 looking for actually does exist, whether it exists in10 a code with about nine other diseases as well or to11 understand whether it is even captured in a specific12 code. 13 And finally we feel that it is very14 important to understand whether or not you can access15 medical records through the data source that you are16 using. Unlike some of our colleagues in17 pharmacoeconomics and health services research, we18 often find the need to validate the diagnostic data in19 the claims with medical record data to truly20 understand whether the event that we think has21 happened has actually happened, and has happened at22 the time at which we believe it has occurred.23 Now, with new regulations, it can be quite24 challenging to gain access to medical records, but25 there is now about 20 to 30 years of research using2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1661 these kinds of databases, and many authors have been2 publishing when they are able to validate claims.3 And it is important to go to the4 literature and understand if the outcome that you are5 looking at has already been studied and validated in a6 system like the one that you are planning to use, and7 that would be useful information for us all to know.8 Moving on to another source of9 observational data, I am going to talk a little bit10 about registries. Now, we have defined a registry for11 the purposes of the concept paper as a systematic12 collection of defined events or product exposures in a13 defined patient population for a defined period of14 time.15 Registries are typically prospective in16 nature, collecting data on patients that are17 identified either as having a particular disease, or18 as receiving a particular treatment. 19 And typically they are used to collect20 information that may not be able to be collected21 within standard systems. Like good22 pharmacoepidemiologic study protocols, good registry23 protocols should include clear objectives of what is24 to be gained, as well as a literature review.25 Now, the detailed research methods really2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1671 should focus quite a bit on patient recruitment and2 follow-up, because that is clearly of interest to3 understand whether the patients in the registry will4 truly be representative of patients out there.5 And if they are not, to understand in6 which ways they are not. Projected sample size really7 is an important issue, particularly with regard to8 attribution and loss to follow-up, and any estimations9 to that effect can be very crucial to how useful the10 registry might be.11 Methods for data collection management and12 analysis is important also. As was pointed out this13 morning, there are often not comparator groups for14 registries, and it is important to understand if there15 is a way to develop or to anticipate a comparator16 group, or what background rates may be relevant for17 that particular registry.18 If possible, we would love to see the data19 collection forms that have been designed to collect20 data on the registry, and help to be able to21 contribute to those. 22 And also validation of findings,23 particularly if the findings are patient reported, and24 whether there is a plan to validate those with health25 care providers.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1681 Another strategy that is used to collect2 observational data are surveys. Surveys are not3 really -- surveys are not studies that could actually4 determine an incidence rate, or a cause and effect,5 but a survey can be very helpful to provide6 supplemental information with regard to a drug safety7 issue.8 You can use surveys to assess knowledge of9 both patients and providers about labeled events to10 better understand medical product use; to assess11 compliance with risk management programs as was12 discussed yesterday; or in the world of medication13 errors, surveys are often used to try to survey14 pharmacists to understand more about the potential for15 proprietary trade name confusion before a product is16 launched.17 And again good survey protocols are very18 much like other study protocols. Detailed objectives19 and methods are very helpful to have in the protocol20 or in the finished report, with particular focus on21 where the patient or provider sample came from.22 And to understand the pharmacist, or23 patients, or providers that were sampled, what was the24 sampling universe, so that we can better understand25 whose views they represent within the profession.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1691 Projected sample size and methods again,2 and again if there are survey instruments that can be3 provided, then the reviewers at FDA could do a much4 better job of determining the validity of the5 findings. 6 So what do we do once we have all this7 observational data? Well, what we would like to do is8 rather than have things come in kind of piecemeal,9 because we all know that there is no one piece of10 observational data that is really going to solve an11 issue, or really give us an idea of exactly the12 magnitude of a risk.13 What we would like to see is a synthesis14 of the information that is available; the case15 reports, the drug use data, the pharmacoepi studies,16 the registries, the publications, all of the data that17 are available about that signal that can help us to18 understand what in a total picture might be going on.19 We would encourage sponsors to provide20 their own assessment of the risk benefit profile based21 upon that synthesis. And that synthesis should be22 providing the limits and the strengths and limitations23 of the different study designs and data sources used.24 We would also encourage sponsors to25 propose steps to further investigate if they feel that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1701 further study is needed. And finally this is an2 opportunity to propose risk management programs if3 they appear to be appropriate given the signal.4 And I would also in response to one of the5 questions this morning, to me if a signal turns out to6 not be a signal because of other observational data, I7 would say that could be part of the synthesis as well,8 and that that was really what the sponsor believes is9 the take home message.10 Then at the FDA, we can do our own review11 of the synthesis, and we will be looking at probably12 -- I am hoping the same things that you are looking13 at, which is that we will be looking at the magnitude14 of the signal and its precision; trying to see what15 are the consistency of these findings across the16 available data sources given all of their limitations17 and more of the things that we know about them.18 To be looking at the biological19 plausibility of the event, and the seriousness of the20 event, but also the degree of benefit that the product21 offers; the availability of other therapies, and in22 the event that things aren't to the point where a risk23 management program is proposed, the ability of that24 program to realistically be able to mitigate risk.25 And now this is just a summary of the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1711 three questions that I guess we have felt were2 relevant, and I know that some folks have already3 presented their thoughts on those, and we are very4 appreciative.5 But for the afternoon conversation, we are6 very interested in hearing more about what folks think7 about registries as surveillance tools, and when they8 would be useful, and when they might not be.9 Secondly, we are very interested in10 hearing your thoughts about active surveillance11 strategies, and when you feel that they might be12 useful to identify events that are not reported to our13 passive surveillance system. 14 And finally what circumstances do you feel15 that additional pharmacoepidemiologic studies might be16 useful, in addition to the circumstances that we have17 described. Thank you.18 DR. GALSON: Thank you, Dr. Staffa, and19 now it is time for our presentations from the public,20 and I am going to shift the order of the presentations21 a little bit. 22 First will be Dr. Stemhagen, and then John23 Ferguson, then Dr. Stephenson, and then Dr. Kahn. 24 So if Dr. Stemhagen could come up, please.25 DR. STEMHAGEN: Thank you very much. I am2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1721 Annette Stemhagen, and I am Vice President of2 Strategic Development for Covance Perry Approval3 Services, and today I am speaking on behalf of the4 International Society of Pharmacoepidemiology or ISPE.5 As you have heard, if you have been here6 the last few days, ISPE is a non-profit, international7 professional membership organization decided to8 promoting the science of applying epidemiologic9 approaches to the study and value of therapeutics.10 The society provides an international11 forum for sharing knowledge and scientific approaches12 to foster the science of pharmacoepidemiology. We13 have over 700 members representing more than 4514 countries.15 Our members work in academic institutions,16 in government agencies, and in industry, from for17 profit and for not for profit organizations. The18 specific backgrounds of our membership is quite19 varied, with epidemiologists, biostatisticians, those20 involved in medicine, nursing, pharmacology, pharmacy,21 law, health economics, and journalism.22 And I would like to thank the FDA for23 allowing us the opportunity to provide comments on24 this concept paper number three. The comments that I25 am going to provide are based on feedback from senior2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1731 members of ISPE, and we will also be providing written2 comments with a broader representation of our3 membership as those comments are collected and4 collated over the next week.5 I would first like to say that we are very6 pleased in this concept paper with the recognition of7 the role of epidemiology and the value of the use of8 observational data and tools for the understanding of9 risk assessment and risk evaluation, and assessments,10 excuse me.11 It is not often that epidemiology and its12 value is quite as recognized as in this document. So13 thank you very much, and we are very pleased, and we14 look forward to working with you on that.15 I would like to begin with several areas16 of the concept paper, where we either have comments or17 some questions for the agency. I'm sorry, I am behind18 in my slides. This is our website. 19 The first is a qualitative definition of20 risk. From the epidemiologic standpoint, we recommend21 providing a quantitative definition of risk in the22 guidance document. 23 Risk is a measure of the frequency based24 on an enumerator and dominator, and if you have ever25 been around epidemiologists, we certainly live and die2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1741 by enumerators and dominators. So we are always going2 to have a comment like this.3 The enumerator in risk consists of the4 number of people who develop an event, and the5 denominator consists of the number of people exposed6 per unit of time.7 This can also be thought of as an8 incidence rate over a particular time period. The9 next thing that we would like to comment on is10 expansion of the type of appropriate studies for11 pharmacoepidemiology assessments.12 We would suggest that the epidemiologic13 assessment of safety end points is important and14 should be further emphasized in this concept paper.15 Also very important are studies of the natural history16 of disease, as well as evaluation of drug use patterns17 and patient characterization studies. 18 And that these studies should be done19 proactively, and they will certainly assist the20 sponsor in terms of gathering data should a risk21 management program be required. So I think as Dr.22 Jones talked about on speaking of concept paper number23 one, really pharmacoepidemiology needs to extend back24 as you have acknowledged into the development phase,25 as well as the post-marketing.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1751 And we would like to urge that sponsors2 consider looking at natural history studies and3 patient characterization studies so that you really4 can gather background information that is going to be5 critical as we go forward with risk management.6 This serves to expand our understanding of7 safety end points, and also helps put the spontaneous8 reports in context as well. Natural history of9 disease and safety study among users of available10 therapeutics before your own product is on the market11 should be initiated during early development. 12 Drug use in patient characterization13 studies should be initiated during the development14 stage of a new drug based on the therapeutic class,15 and then continue after launch to further understand16 drug use in diverse populations and across various17 countries.18 Similarly, evaluation of end points among19 products in the therapeutic class and within diverse20 populations are important, and I think we have already21 acknowledged that as well during the discussions the22 past few days.23 Targeted safety studies should be24 initiated shortly post-launch based on the25 specifications of the development program and the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1761 pharmacovigilance plan. The next comment, I am2 actually going to deviate from my prepared script.3 This has caused quite a controversy, in4 terms of the definition of registry, and as the three5 days have evolved, there are a number of ISPE members6 here, and we have been discussing a lot the discussion7 of registries.8 I think we have here, because we couldn't9 even agree among ourselves exactly what a registry10 was, with a lot of contention between the definition11 as presented by the FDA, a systematic collection of12 defined events or product exposure, in a defined13 population for a defined time period. 14 That is somewhat self-limiting. It does15 describe, for instance, a birth defects registry,16 which would be a collection of events, but it doesn't17 necessarily describe all of the kinds of registries18 which must be considered more like an observational19 cohort.20 So I think here, again just in evidence21 from the discussions that the audience has been having22 among themselves, that we need a better nomenclature I23 think for registries. 24 It is not totally clear exactly what they25 are, and based on my experience when we are talking2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1771 about registries in other countries, many countries2 really shy away from actually even using the term3 registry.4 And it makes it even more muddied as to5 what they are talking about. So I think that we need6 to work together to get a better definition on that.7 I am also going to talk a little bit about definitions8 for pharmacoepidemiologic studies. 9 We recommend that in Section 4(a) about10 pharmacoepidemiologic studies that they be classified11 as pharmacoepidemiologic safety studies. Just for12 clarification purposes, pharmacoepidemiologic studies13 are not limited to safety studies as I am sure that14 everyone knows, but include all observational studies15 which describe the use and effects of pharmaceuticals16 in a real world setting. 17 We also recommend that in Section 4(a),18 where it is a definition of pharmacoepidemiologic19 studies, that this be expanded to include post-20 marketed data collection, in addition to the already21 mentioned existing automated databases.22 In several sections of the concept paper,23 pharmacoepidemiologic studies seem to refer only to24 those studies conducted in automated databases, and25 that is probably not the intent, but it is certainly2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1781 could be construed that way. 2 Things like the large simple studies which3 were discussed on day one in terms of premarketing,4 can also be pharmacoepidemiologic or observational5 studies.6 So really the definition needs to be7 expanded to include prospective data collection as8 well. In terms of Section 5 regarding safety signals,9 there is a list on lines 254 to 258 of five examples10 of signals. It is very useful, and it probably covers11 most of the options or types of safety signals.12 However, we recommend expanding that list13 to include a category of other risk factors, or risk14 factors associated with the development of the adverse15 event of interest. 16 Safety signals regarding specific17 subgroups of patients, and that might be either age,18 race, genetics, specific co-morbidities, or ways in19 which the drug is used -- for example, dose escalation20 may also be important.21 So there are really other categories that22 could be included in there. We would also like to ask23 for a clarification of line number 319, point number24 8, which says that analyses of the potential for an25 excess of adverse events given the disease being2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1791 treated, such as might be observed in advanced cancer2 or immunocompromised patients.3 Our question is whether that is intended4 to address analyses to identify the background risk of5 adverse events in the disease being treated, which is6 actually then point number 9.7 We weren't really clear what the8 distinction between those two are. And then looking9 at all of those points together, we would like to make10 the point that many of those are probably not11 appropriate for analyses using spontaneous adverse12 event reporting data. 13 But much more apply to analyses based on14 other types of pharmacoepidemiologic data, either15 prospectively collected or through a large automated16 database. 17 The next point that we would like to talk18 about is Line Number 352 actually, the relative risk19 or odds ratio. The point mentions that relative risks20 or odds ratios from pharmacoepidemiologic studies can21 be used to evaluate causality.22 Although risk ratios or odds ratios are23 very frequently reported as an outcome frequency in24 pharmacoepidemiologic studies, we would like to be on25 record as noting that risk differences, and not2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1801 ratios, are best for evaluating the frequency with2 which a drug causes or contributes to an event. 3 So in addition to an estimate of relative4 risk, we would suggest that risk differences or5 discussions of incidents rates in particular be6 included in that section.7 In the end the frequency with which an8 event occurs because of a drug, along with other9 characteristics of the event, must be weighed against10 the benefit of the drug.11 And I think this point again was discussed12 several times over the last few days, but in addition13 a risk management program needs to consider how the14 risk associated with the intervention, which is15 designed to reduce the risk of a particular event --16 for example, the potential reduction of benefit17 associated with the use of the therapeutic as a result18 of the risk management intervention.19 So we have to remember again that balance20 of risk and benefit, and that if we are putting a risk21 intervention in place, and we have to be sure what the22 impact on the benefit is.23 If we restrict things as an example too24 much, then the benefit may be reduced with patients25 not able to get access to the drug. The next is a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1811 discussion of reporting of safety signals. In Line2 374, Paragraph (d), and I think this is what Dr.3 Stephenson had pointed out earlier as well, in the4 reporting of safety signals, there really needs to be5 a definition of when is a signal really a signal.6 A lot of what is discussed may be our own7 analysis to evaluate whether it is a signal or a false8 positive. And the way that this paragraph is9 described, it looks like hypothesis generating,10 testing, prospectives of risk and benefit, and risk11 management, all need to be done in a package and sent12 to the FDA.13 When in fact many of these things, or the14 first part of this, will be done many times by a15 pharmaceutical sponsor before the FDA needs to be16 notified, and we are trying to rule out some things.17 And in addition there are sequential --18 there is a sequential order to this. You may not need19 to get to all of these steps again if you have ruled20 something out. So I think we are asking for a little21 bit more clarification on that point as well. 22 In terms of surveys, I thought that this23 was very interesting that this was included. You24 don't often see again some discussion of surveys. We25 would like to comment on the inclusion of surveys as a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1821 risk assessment or risk management evaluation tool.2 Many of the examples provided in the3 document are actually cross-sectional designs. We4 would just like to make note that longitudinal cohort5 studies to evaluate changes in risk over time can also6 be surveys, and so they don't need to be limited to7 cross-sectional surveys.8 And in talking about these kinds of9 surveys, or some of the types of surveys that are10 listed as examples, in addition to epidemiologic11 expertise, which of course we believe is very12 important in many of these analyses, it is going to be13 important to include members of other academic14 disciplines, such as social scientists, who bring15 expertise in areas such as knowledge, attitudes, and16 behavior surveys.17 I think that this is an area where we as18 epidemiologists probably are not as strong, or many19 people that are not psychosocial epidemiologists are20 not as strong.21 And within pharmaceutical companies, or22 within the FDA, I think we are building this23 expertise. So, just an acknowledgement that we really24 have a number of disciplines that will be important in25 trying to evaluate risk intervention programs. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1831 And I guess just one comment on what Dr.2 Staffa had said, that surveys are really not used to3 assess cause and effect, but actually I think there4 may be instances, especially if we are talking about a5 longitudinal survey, where you could look at cause and6 effect. 7 You might use some of this survey8 methodology if you have a risk intervention, and you9 want to evaluate its effect, you may do surveys in the10 beginning, and you may do surveys again, and you11 really can look at cause and effect.12 So there is really a broader application I13 think to surveys than the document might suggest. One14 other comment really before I talk about analysis,15 which is really our last point, is that many people16 already even at this concept stage are taking this17 document quite to heart, in terms of very literally18 assuming that the study designs discussed in here are19 the study designs that you must do, I think it is not20 always as straightforward as I think everybody knows,21 in terms of the circumstances in trying to decide what22 type of study design, a pharmacoepidemiologic study23 design, is important.24 That the guidance document is not25 prescriptive, and so many people that are not2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1841 pharmacoepidemiologists are taking this very literally2 as, okay, this is the issue and this is the design we3 are going to do.4 If there can be some discussion that there5 are a multitude of designs that might be appropriate6 for a situation, I think that would be very helpful.7 Again, it is a guidance document and should be8 interpreted that way, but I am not sure that it always9 is.10 Our final point in terms of ISPE is that11 we believe that there is still a great deal of work to12 be done in the development and refinement of analytic13 methods for evaluating safety signals, as well as14 methods for risk assessment and risk management15 evaluation.16 ISPE is committed to working with the FDA17 and to others, such as the CERTS, to further the18 knowledge of the methodological and statistical19 techniques required. ISPE is firmly committed to20 providing an unbiased scientific forum for the views21 of all parties with an interest in the science of risk22 management. 23 And we are deeply committed to advancing24 this science. We would like to welcome the25 opportunity to work with the agency in this area as we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1851 did in actually developing the guidelines for good2 epidemiology practices in 1996, and the data privacy3 document in 1997, both of which are referenced in this4 concept paper.5 And as I said, we will engage our full6 membership in the feedback process on this concept7 paper as well. Our next annual conference will be8 focused on risk management, and anyone who has been9 here for the last couple of days has already heard10 this.11 There is several workshops and sessions12 being planned jointly with the FDA. So as my13 colleagues over the past two days have done as well, I14 would also like to extend an invitation to everyone to15 join us for the first International Conference on16 Therapeutic Risk Management, which is held in17 conjunction with the 19th International Conference on18 Pharmacoepidemiology.19 It will be held in Philadelphia, August20 21st through the 24th of this year, and so we would21 like you all to come. I would also just like to add22 one more thing. 23 I am the membership chair for ISPE, and so24 hopefully I have tantalized you with all the wonderful25 things and great thoughts that ISPE members have. And2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1861 if anybody would like to become a member, please see2 me.3 DR. GALSON: Thank you very much, Dr.4 Stemhagen. Our next speaker is John Ferguson. I5 should have said Dr. Ferguson, excuse me.6 DR. FERGUSON: My name is John Ferguson,7 and I am the Vice President of Drug Safety at Biogen,8 and on behalf of the Biotechnology Industry9 Organization, I thank the FDA for the opportunity to10 comment on concept paper number three, risk assessment11 of observational data, good pharmacovigilance12 practices in pharmacoepidemiologic assessment.13 I will begin my presentation with general14 comments, followed by comments in selected elements of15 the concept paper, and in so doing, I will address16 questions posed by the agency, and at the end of the17 document, I will close by reiterating key points.18 BIO congratulates the FDA on concept paper19 number three. This important initiative continues to20 move toward the increasing use of evidence-based21 approaches to evaluating selected important safety22 questions. 23 BIO agrees that observational data can be24 very useful for confirming signals. However, work is25 needed to maximize that utility. The level of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1871 evidence derived from this data varied significantly2 with the type of data collection, control of3 confounding, and the magnitude of the observed effect4 size among other things.5 Clarification of the agency's view of how6 level of evidence for causal association varies with7 the design would be most helpful. Similarly, we would8 welcome specification of the way in which regulatory9 decision making might be conditioned on the level of10 evidence and the strength of evidence. 11 While I believe that the need for12 pharmacovigilance plans will vary, and should be13 determined on a case-by-case basis, explicit criteria14 that could serve as the basis for decisions about15 pharmacovigilance plans would help to ensure their16 consistent and effective use.17 Small companies will require time to18 effectively implement the full range of proposed19 activities, since some techniques require special20 expertise that is not widely available or easily21 accessible to them. 22 Moving now to specific comments on23 selected elements of the concept paper, the paper24 refers to the agency's expectations when a signal is25 identified, criteria that delineate the basis for2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1881 deciding that a signal has been identified, contrasted2 with those that confirm a signal would be very3 helpful.4 And in this we reiterate the comments of5 colleagues in other organizations made earlier. While6 it is not the intent of this paper to explain the7 nature and strength of safety findings to regulatory8 decision making, it seems logical to do so.9 For example, what impacts to signal10 identification, as opposed to confirmation have on11 labeling. High quality case reports are the12 cornerstone of good pharmacovigilance practice. 13 The major limitations to high quality case14 reports are the lack of reporter incentive to complete15 the considerable amount of work that is required to16 obtain complete information on safety issues and17 confidentiality, which I will address in a moment. 18 I want to stress that incomplete19 information will remain a problem, and so discussions20 concerning the rational use of incomplete information21 for purposes of signaling in regulatory decision22 making would be very useful. 23 There is no doubt that confidentiality24 issues impact our ability to obtain complete25 information on safety issues. It is noteworthy that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1891 confidentiality initiatives are progressing in2 parallel to this risk assessment initiative.3 Ideally over time the developers of the4 risk assessment initiative can provide guidance to5 those working on confidentiality initiatives to ensure6 that confidentiality regulations support risk7 assessment needs.8 In particular, guidance which is clear9 regarding which activities, which of the proposed10 activities do not constitute surveillance activities11 would be most useful. 12 The utility of registries and surveys for13 evaluating signals depends on design, and I want to14 reiterate a point that was made earlier that there15 seems to be a lot of confusion in terms of the16 definition of registry.17 It ranges in our experience from designs18 that are purely exploratory and that may be able to19 identify signals, but can do little to confirm them.20 And on the other extreme to studies that almost21 approximate large simple trials.22 The FDA can do a lot to reduce confusion23 in this regard by providing more detailed definitions24 of registries or registry variance in surveys; and25 delineating a clear approach to the consistent2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1901 implementation of this design.2 In this regard the FDA has considerable3 experience with registries and could increase the4 utility of the guidance document by way of examples of5 situations where registries have proven to be very6 useful for confirming signals and where they have7 failed in this regard.8 The use of automated databases is a9 complex subject as has already been recognized. BIO10 suggests that the concept paper include discussions11 with relative merits and limitations of automated12 databases for purposes of evaluating signals.13 We would also welcome specific guidance14 regarding the validation of data derived from15 automated databases. Background rates are useful16 despite limitations. 17 But since there are limitations,18 clarifications for guidelines for use in19 interpretation would be most helpful, and in20 particular with regard to the need for and value of21 standardization.22 And the basis for requiring alternatives23 when background rates may not be optimal are24 available, but may be useful for addressing safety25 issues in a given context.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1911 There are clearly pros and cons that have2 been discussed regarding reporting rates. BIO feels3 that reporting rates are often useful as crude tools4 for signal evaluation. However, reporting rates can5 be very difficult to interpret.6 As such, we would welcome a discussion of7 the assumptions and basic requirements for generating8 the reporting rates, the limits to their utility, and9 the approach to their consistent interpretation10 conditioned on under-reporting.11 In contrast, product comparisons based on12 reporting rates are very limited. They require strong13 assumptions that are often questionable. Regulatory14 decisions based on comparisons of product reporting15 rates should not be made without a clear understanding16 of the surveillance models used for each compound, as17 well as the evaluation of the procedures used to18 estimate treatment and exposure.19 Such comparisons should be avoided when20 products are used for different indications. In21 general, causality assessments are unreliable, but22 there are clearly situations where causality can be23 attributed in individual cases and excluded in24 individual cases.25 Therefore, there is a need to define the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1921 role of causality assessment in signaling. For2 example, how should causality be used in constructing3 reporting rates and sensitivity analyses that go to4 those reporting rates. 5 The FDA's concept of linking the degree of6 causality to the quality of evidence is a very useful7 concept that warrants further development, and goes to8 the point that I made at the opening of my9 presentation related to the level of evidence and how10 it should impact regulatory decision making.11 The strengths and limitations of data12 mining are not well understood at this point in time.13 There are strong underlying assumptions, and there is14 a need for additional developmental work to define the15 exact role of data mining in regulatory decision16 making.17 In particular at this point in time, BIO18 feels that data mining may be capable of identifying19 signals, and maybe more appropriately identifying20 alerts as described by Gerry Faich, but it is not21 clear how useful data mining will be in terms of22 confirming signals on safety information.23 In summary, observational data can be24 useful for signal confirmation, and represents an25 important move towards evidence-based assessment of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1931 safety issues. Additional work is needed to2 understand the contribution of different types of3 observational data to levels of evidence for4 causality.5 We recommend efforts to link the level of6 evidence to specific regulatory actions, and explicit7 criteria for pharmacovigilance plan requirements will8 increase their effectiveness and consistency of their9 use, as small companies need time to acquire the10 expertise needed for effective implementation of the11 full range of the proposals.12 In closing, we thank the FDA for the13 opportunity to comment on concept paper number three,14 and we again applaud the agency's efforts to date in15 this important endeavor, and we look forward to16 working with the various partners to continue the17 development of these key concepts. Thank you.18 DR. GALSON: Thank you very much, Dr.19 Ferguson. Our next speaker is Dr. Wendy Stephenson.20 DR. STEPHENSON: Good afternoon. I am21 Wendy Stephenson, and I am a senior vice president for22 Global Safety Surveillance and Epidemiology Labeling23 and Health Outcomes for Wyeth.24 I have worked within the industry in the25 area of safety for about 15 years, and I have been a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1941 member of the CIOMs working group on adverse drug2 reaction reporting for just about as long.3 This afternoon, I am here representing4 PhRMA. I would first like to commend the FDA and the5 various committees for their excellent work drafting6 the three concept papers that have been the subject of7 these very interesting three days.8 And to thank the FDA for allowing PhRMA to9 participate and to publicly comment at this meeting.10 I will be addressing the FDA's questions 4, 5, and 6,11 following Linda having addressed the first three12 questions.13 But will end with some additional14 questions and points to consider for the FDA that15 PhRMA hopes will generate some additional discussion.16 Under what circumstances would a registry be useful as17 a surveillance tool and when would it cease to be18 useful. While there are clear circumstances where19 registries represent an important surveillance tool --20 for example, for the use of pregnancy registries -- in21 most situations there are likely to be better22 alternatives. 23 For most products and in most24 circumstances, it is probably not going to be useful25 to establish a registry. Registries may be useful2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1951 when the information you need to ensure safe use of a2 product cannot be obtained from another mechanism.3 And the need for the information justifies4 the extensive resource commitment necessary to set up,5 operate, and comply with the registry. Ideally there6 would be a concomitant comparison group, and if not,7 then at the very least a comparable historical8 control, or well established background incidence of9 the event in question.10 With these criteria in mind, PhRMA11 supports the concept of pregnancy registries under12 appropriate circumstances, and agrees that there may13 be other similarly special circumstances where a14 registry may be appropriate. 15 One example that comes to my mind is the16 Myelotarg VOD registry, which was established after a17 signal suggested the possibility of an unusual liver18 toxicity in patients with AML.19 VOD is veno-occlusive disease and it is an20 unusual liver toxicity. That registry, with a well21 defined outcome of interests, in a well defined22 population, is expected to provide useful information23 on this potential risk that would not otherwise be24 available through other sources.25 Where a registry is deemed appropriate to2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1961 achieve a safety surveillance goal, there should be2 specific objectives and a well defined outcome of3 interests. The size of the patient population, as4 well as the duration of follow-up, should also be well5 defined, and not open-ended.6 Industry responsive registries tend to be7 drug specific registries. However, it may be8 advantageous to establish a disease specific registry9 with collaboration across companies and other parties10 as appropriate.11 If separate registries are established for12 similar compounds, there should at least be some13 standardization so that comparable levels of reporting14 and evaluation may be accomplished.15 A well tested alternative to registries is16 the use of large automated databases to serve as a17 source of appropriate cohorts. Databases such as18 Kaiser Permanente, Saskatchewan in Canada, GPRD in the19 U.K., are powerful tools used by industry to sponsor20 research without assuming the cause of initial data21 collection, which registries entail.22 In addition, large automated databases23 capture all patients in a defined population; whereas,24 registries usually only capture those patients who are25 voluntarily enrolled in a study.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1971 Therefore, selection bias of an unknown2 magnitude and origin may be a particular concern for3 registries. PhRMA encourages collaborative review by4 industry and FDA of large linked databases for the5 evaluation of safety signals. 6 Under what circumstances would active7 surveillance strategies prove useful to identify as8 yet unreported events.9 Active surveillance strategies, which10 often include the concept of registries, may be useful11 when information cannot be obtained from more readily12 available tools that are less resource intensive and13 less obtrusive.14 One approach to active surveillance15 involves the use of specially trained sentinel sites16 generally affiliated with academic centers to17 stimulate the reporting of events that might not18 otherwise have been spontaneously reported.19 Such strategies may be subject to20 selection bias, and are likely to involve relatively21 small populations, and so it may not be useful for22 detecting rare events.23 When successfully employed, however, they24 may provide more in-depth information and a more25 accurate estimate of the reporting rate, and may be2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1981 more likely to provide information that will help with2 making a causality assessment.3 Another approach to active surveillance is4 prescription event monitoring, such as the green card5 system that has been in place in the U.K. for many6 years. This approach differs from most others, in7 that events are collected, regardless of the suspicion8 of drug relationship.9 If conducted with a systematic consistent10 approach across a wide variety of products, the11 resulting database can be used to make comparisons12 across products. 13 And because the denominator is known14 actual incidence rates can be calculated. Care must15 be taken, however, in the interpretation of16 comparisons between drugs that may be prescribed under17 different circumstances that lead to confounding and18 bias.19 It is not clear that prescription event20 monitoring has ever lead to the identification of21 previously undetected signals. It can, however,22 sometimes be useful for confirming signals that have23 been previously detected through spontaneous reports.24 Other initiatives which probably should be25 considered to encourage the identification and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 1991 reporting of adverse events were mentioned earlier by2 Dr. Goldman, and that is the implementation of3 educational programs for health care professionals.4 Perhaps providing CME credits for5 reporting complete and high quality reports to either6 the agency or the company might be considered as an7 incentive. 8 And the last question, under what9 circumstances would additional pharmacoepidemiological10 studies be useful. Pharmacoepidemiological studies11 are useful to test hypotheses generated by a signal12 from the spontaneous reporting system, and to further13 evaluate the safety profile of a product for which a14 potential risk has been identified prior to approval.15 However, as with all data sources we have16 been discussing today, no study will ever be17 definitive. Rather, it will represent just one more18 piece of the puzzle. Pharmacoepidemiology studies may19 have false positive, as well as false negative,20 results due to bias and confounding.21 Fortunately, utilization of some of the22 better automated databases allows for control of23 confounding to a large extent. Perhaps the greatest24 value of pharmacoepidemiologic studies is their25 usefulness both pre-and-post approval in identifying2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2001 background incident rates for adverse events of2 interest.3 In addition, they may be useful to4 identify high risk groups or risk factors for a5 particular adverse event. The same databases may be6 used to conduct drug utilization studies which can7 provide useful information on physician prescribing8 and monitoring behavior, characteristics of patients9 who are treated with the drug of interest, as well as10 drug usage patterns, and the impact of those usage11 patterns on outcome related to both benefit and risk.12 The same types of studies can thus be used13 as a mechanism for measuring and evaluating the14 effectiveness of risk management interventions, at15 least in the population covered by the database.16 In summary, PhRMA supports the development17 and use of observational studies of information beyond18 spontaneous reports. PhRMA agrees with FDA that19 pharmacoepidemiologic studies are useful to further20 evaluate the safety profile of a product for which a21 potential risk has been identified through the22 spontaneous reporting system or other source.23 All of the modalities available have24 limitations, and no one approach can be expected to25 provide a definitive answer. Conclusions regarding2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2011 actual risk will largely depend upon the consistency2 of findings, and the strength of evidence after taking3 into account information from as many sources as4 possible and within reason.5 And I end with some additional -- and I am6 afraid to ask these questions, but hopefully these7 will be more difficult to pass back to us. 8 DR. GALSON: Try me.9 DR. STEPHENSON: Since both the FDA and10 industry conduct pharmacoepidemiologic studies, it11 would be helpful to hear FDA's thoughts on when a12 study should be conducted by the company, versus FDA,13 or both.14 And, secondly, what does FDA view as its15 obligations to a company when the FDA conducts a16 pharmacoepidemiologic study on that company's drug.17 Thank you for your time.18 DR. GALSON: Thank you very much. Our19 next speaker is Dr. Sidney Kahn.20 DR. KAHN: Thank you. Good afternoon. I21 am Sidney Kahn, President of Pharmacovigilance and22 Risk Management, Incorporated, an independent23 organization providing consulting and support services24 to the pharmaceutical industry, and its supporting25 organizations.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2021 And I am very appreciative of being2 allowed to present my views in front of this august3 gathering today, and I would also like to thank the4 FDA for providing such provocative and insightful5 documents, as well as allowing the public to comment6 on their content and hopefully their eventual7 improvement into the eventual guidance as the product8 is developed.9 I would like to start with what I consider10 to be an omission from the current documents, which is11 the role of therapeutic or Phase IV studies in risk12 assessment. I started talking about this yesterday13 afternoon, and so this is the right slide and the14 right presentation at this point.15 The May 1999 reports of the task force on16 risk management had a fairly extensive discussion on17 the use of utility or the potential utility of Phase18 IV studies in risk assessments. 19 And I was quite surprised then to find20 that the current concept paper makes almost -- I think21 it makes almost no mention of it at all, or if it22 does, it is purely in passing. 23 And this is odd to me because it strikes24 me that this is a wonderful mechanism and tool for25 identifying or characterizing, or both, either new2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2031 risks or established risks that have been identified2 during clinical trials.3 And this is a mechanism that is already in4 place, and this is not something that would have to be5 added on as a new or large burden. The additional6 burden of uniform data collection and data aggregation7 is real, but it is nowhere near as large as some of8 the other interventions that have been proposed.9 So using it in the vernacular, I see a10 substantial bang for the buck in doing this type of11 approach, and it would also I believe help us to move12 forward with some additional characterization of new13 signals that could then be evaluated further in other14 subsequent larger databases.15 And I would like to suggest that that may16 be included in the final guidance as an area that17 could be used as a significant risk assessment tool in18 the marketed product that we know. 19 I am going to address several specific20 points in the document, and some I will and some I21 won't touch, as several have been dealt with much22 better than I could by others. 23 But if I could talk about an issue of case24 follow-up and to a lesser extent expedited reporting,25 which is not part of the guidance. In relation to a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2041 different regulation that the FDA has proposed, which2 are related to format and content of the adverse3 reaction sections of the drug safety labeling, under4 the new proposal only adverse reactions, whatever they5 may be, will be included in the label.6 And events that occur in the study7 population at a comparable frequency or without drug8 therapy, will not be included in that label proposal.9 Now, for those of us who have worked for any length of10 time in spontaneous post-marketing reports, I think we11 all know that the profile of the events that get12 reported post-marketing tends to be quite smaller to13 that which was observed in the clinical trials in the14 population.15 And if we omit from the labeling any16 description or listing of those common events which17 may be serious in many populations, such as diabetics18 or cancer patients, or people with congestive heart19 failure, or a variety of other conditions, if those20 are omitted from the labeling, they will become under21 current regulations unexpected or unlisted if they are22 in the company core data sheet.23 And unexpected events have specific24 regulatory requirements which are quite burdensome,25 especially if there are very many of them. The issue2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2051 of aggressive follow-up, we have heard talked about.2 And in addition to the burden on health3 care providers and the difficulties that the new HIPAA4 legislation creates for them, all this would do would5 require the sponsors or the manufacturers of drugs6 that have high background rates of significant adverse7 events to follow up the natural history of the8 disease, which I don't believe was the intent.9 So I see this as an area where there is10 some unintended consequence of two intrinsically11 logical approaches that when taken together give you12 something that you really had not intended at the end13 of the day.14 So I would ask the agency please to take15 that into consideration, and just to help them along.16 I am putting up here some issues or methods that I17 consider could be used to obviate that problem,18 because this would be a real problem.19 One is that we could continue to include20 the totality of the adverse events, including those21 that occur with comparable frequencies in the active22 control groups in the label, as we do now in the table23 of adverse reactions.24 We could create an additional section of25 the label that contained adverse events that the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2061 agency and the sponsor would consider expected for the2 purposes of regulatory reporting and follow-up.3 Or there is a possibility, and I don't4 know how this would drive with the lawyers, but that5 is something that we could explore, of omitting these6 comments from the prescribing information because it7 is not clear that the physicians necessarily need to8 know how many patients with congestive cardiac failure9 have been hospitalized with pulmonary edema.10 But rather to agree between the sponsor11 and the agency at approval, at the time of ongoing12 periodic reporting, for example, what common events13 could be considered expected, and therefore not14 subject to aggressive follow-up, or expedited15 reporting.16 The FDA asked a question in this concept17 paper on population background rates. We have heard a18 lot about this and clearly knowledge of this is very19 important, and is very valuable. 20 What I would submit though is that the FDA21 has a unique opportunity to find such data in great22 detail from its own databases, especially for those of23 controlled populations in their archives in NDAs and24 BLAs that the pharmaceutical and biologic industries25 have presented to the agency.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2071 And there is an initiative that I just2 heard about recently from Randy Levin, called JANUS3 which is under way in CDER, which is apparently4 allowing electronic integration of such data, and5 perhaps our FDA colleagues could comment on that in6 due course. 7 And I would certainly urge sponsors to8 allow their proprietary data to be used in an9 appropriately automized way to help the FDA develop10 these assessments of background rates, because these11 data are probably even more robust than the12 pharmacoepidemiologic data for background rate13 calculations.14 We have heard a lot about under-reporting15 over the years, and I would like to make some caveats16 and points about the myth as we heard this morning on17 under-reporting.18 First of all, high reporting rates for19 events that have a high background rate to the20 population I think have to be considered21 uninterpretable. So you can't say anything about them22 in my opinion, and here we have an issue where again23 the labeling proposals might also have an effective,24 because whether you include or exclude an event from25 the label, will have some impact -- I don't know how2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2081 big or in what direction -- on the reporting by health2 care professionals.3 And that would be something that should be4 evaluated. It would be very helpful here to have some5 input from the medical professional organizations as6 we heard about in relation to, say, the Rhode Island7 study results, to determine how physicians would see8 such a change.9 The issue of the much publicized 1010 percent reporting rate which has been touted as11 reflecting the tip of the iceberg, I am not sure that12 many people believe that, and we have heard examples13 of why that may or may not be so earlier.14 But I want to quote you an actual example15 from personal experience. The FDA sent my company at16 the time a letter in which they assessed an incidence17 rate of a very severe, life-threatening, or fatal18 adverse event at approximately 20 times the population19 background rate for the population exposed to the20 company product, and based entirely on spontaneous21 reports.22 The company went out and did two extensive23 studies. We also went to the PM database as Dr.24 Stephenson mentioned, and a variety of other places,25 and we have been unable to confer anything like the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2091 number of cases in the epidemiology studies. 2 We are paying an independent expert3 opinion from a clinician with great expertise in the4 area who assured us that in his experience at least 805 percent of such cases would be reported by practicing6 physicians.7 And rather than under-reporting and8 getting to the over-reporting concept, we received9 multiple reports of the same case from various10 reporters, and so further supporting the idea that the11 threshold for reporting such cases is relatively low.12 And to cap this, and I know that there are13 studies that have suggested that publicity can alter14 the reporting weight following publications in the15 medical literature in the U.S. and other countries,16 the addition of a black box warning to the label, and17 a dear doctor letter, and there was no increase in the18 reporting rate, which remained constant throughout19 (inaudible).20 And so I realize that this is an anecdotal21 experience, but I think it is illustrative, and as a22 result, I would like to propose the following23 hypothesis. That the more severe the event and the24 outcome, and the more obviously it appears to be25 related to the drug, that is to say that the less2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2101 commonly it occurs in the treated population, and the2 less commonly it has in alternative explanation, the3 more closely the reporting will approach the true4 incidence rate.5 That hypothesis is potentially invaluable,6 and if the FDA can access in its various databases for7 a variety of products which sponsors do not8 necessarily have access to, and compare instances of9 known adverse reactions of varying degrees of severity10 -- cough with ACE inhibitors, versus liver failure11 with troglitazone, for example, with the corresponding12 numbers of reports and time adjusted in errors, and it13 would be very easy to get true estimates of the14 reporting rates.15 The FDA has asked for comments on how to16 improve spontaneous report quality. Not on the slide,17 but a thought that has occurred to me on numerous18 occasions is that in my estimation probably the19 majority of supposed SADRs received by pharmaceutical20 manufacturers come into medical information in the21 form of I had a patient taking your drug among 5, 7,22 or 10 other drugs, who has developed Event X. Is this23 known for your drug.24 And sometimes it is and sometimes it25 isn't. Every one of those is captured and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2111 characterized as a suspected adverse drug reaction,2 and is given precisely the same weights as a report3 from a health professional saying that I had a4 perfectly healthy patient who had a minor condition.5 I put her on your drug and so this6 terrible thing happened, and your drug did it. I see7 those two scenarios as being somewhat different, and I8 believe it could be helpful to alter the categories of9 reporting in such a way as to differentiate between10 these sort of incidental reports where a physician is11 looking to confirm or refute a differential diagnosis12 in relation to looking for a known drug explanation,13 as opposed to reporting a previously unknown adverse14 action.15 DR. GALSON: Excuse me, Dr. Kahn. You16 have got a little bit less than a minute left.17 DR. KAHN: Okay. Well, I will complete18 the rest of my presentation publicly in terms of19 written comments to the document. The last thing that20 I want to switch to is to add a few comments or a21 couple of comments to data mining.22 We have heard a great deal about it, and23 the biggest problem that I see with it is the absence24 of any pre-specified hypothesis. And therefore25 because data mining can pick up signals that no one2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2121 knows what to do with, it is my opinion that this2 should probably be reserved for regulators who do not3 have any necessary regulatory obligation to act on4 this, as opposed to pharmaceutical manufacturers who5 could be placed in a very awkward position, both from6 a regulatory and a legal perspective if they undergo7 data mining when they do not have access to the8 complete data set that the regulators have.9 Pretty much everything else I had to say10 has in fact been said by other speakers, and so I will11 be happy to conclude there, and I thank you very much.12 DR. GALSON: I thank you very much. Let's13 just have a quick show of hands whether folks want to14 take a break or just go on through conclusion, and if15 you want to just go on until we finish, raise your16 hand.17 (A show of hands.)18 DR. GALSON: Okay. Good. Let's move now19 to discussion and Q&A section, and let's go back to20 the question that we left hanging that I mentioned21 right before lunch. 22 I don't know who from PhRMA wants to23 address that. Is it possible to put the question back24 up, or you can just restate it, whichever is easier.25 DR. STEPHENSON: I am going to give you2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2131 the short answer, and Linda is going to give you the2 longer answer. The shorter answer is -- well, read3 the question first. 4 MS. HOSTELLEY: Okay. The question is5 what criteria should be used to determine whether a6 pharmacovigilance plan describing pharmacovigilance7 efforts above and beyond post-marketing spontaneous8 reporting is warranted. 9 Okay. And the short answer is that on a10 case by case basis. And the more elaborate answer is11 that essentially we would encourage the agency to12 consider the criteria that have been put forward in13 CIOMS-III actually, specifically strength of evidence,14 significance of medical saliency, the seriousness of15 the event.16 That is, you know, does it fulfill the17 serious criteria, or is it a non-serious type of18 situation; the predictability, and the reversibility,19 as well as the validity of methods available to be20 able to answer the question within reason.21 We would encourage also open dialogue22 between the FDA, and that would be the Office of Drug23 Safety Review Division, and the sponsor, to really24 reach a consensus as far as the best approach in25 involving this.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2141 DR. GALSON: Okay. Thanks very much.2 Let's just stick on that for a minute and see whether3 anyone else wants to make a comment on those4 responses. Anybody from the agency want to say5 anything?6 We will certainly consider that question7 and I think in general we agree that there has to be a8 case-by-case assessment. It is very difficult and it9 may be possible in certain classes of drugs to set out10 specific criteria if you have a lot of background11 information.12 But we can't or we won't be able to leave13 at least some aspect of a case-by-case analysis. Any14 other comments on that? FDA folks? Go ahead. I want15 to remind folks as well that if you want to fill out16 cards and send them up, we are happy to take more17 cards.18 MS. PEREZ-GUTTHANN: Susana Perez-19 Gutthann, Pharmaceia. It is regarding this question20 of the pharmacovigilance plan, and I think one of the21 useful parts of the discussions that we have had these22 last couple of days is that there has been a lot of23 clarification in terms of what we mean when we say a24 word, and the same word doesn't mean the same to25 everybody.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2151 And I think that something like this is2 happening to the pharmacovigilance plan, where we are3 discussing when to initiate a pharmacovigilance plan4 because in the definition it is implied that it is a5 reaction to something happening to the drug. 6 And in fact I might add that probably most7 companies or most sponsors at the time of launch have8 a pretty clear idea of what the background is in terms9 of risk profile, and have formalized some kind of10 document in which they say this is the area that we11 should monitor more closer or not.12 And whether the activities are going to be13 monitoring some kind of reports, or going beyond and14 doing some pharmacoepidemiologic, et cetera. And as I15 mentioned, if we compare it a little bit with the16 perspective that the European agencies are taking, and17 this is referring to the document that I was18 mentioning yesterday, the summary of the heads of19 agency reports, they are taking a little bit of a20 broader pharmacovigilance plan, and that these are21 general activities that one would take.22 And the criteria that they use for23 initiating this or for recommending it, the24 pharmacovigilance plan, rather than based on an event25 or safety issue related to the drug, is based on the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2161 type of drug or the medications.2 And if you allow me since we were talking3 about criteria, they mentioned that they would be4 particularly relevant to new chemical entities on5 biotech derived products, or from additional products,6 and significant changes in established products, such7 as new forms introduced to a population, establish8 products introduced to a new population, of9 significant new indications, and finally establish10 products when reclassified from prescription only to11 non-prescription availability.12 I am not saying that we should be adopting13 this criteria, but this is kind of a different take on14 when do we initiate the pharmacovigilance plan.15 DR. KAHN: May I continue the discussion16 on pharmacovigilance plans?17 DR. GALSON: Sure.18 DR. KHAN: This is Sidney Khan. In my19 last position in industry, I had the opportunity to20 both develop and review a variety of what we called21 pharmacovigilance plans, but which were primarily22 focusing on the Phase III stage of development,23 because the goal was to try and focus attention on24 events that could possibly be indicators of problems25 down the road.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2171 And there was a carryover as Linda2 Hostelley I think said earlier, and we all understand3 that drug safety is a continuum, and therefore, you4 can't start pharmacovigilance plans at the time of5 launch. You really have to know up front where you6 are going to look, and you have to have high7 sensitivity to what I would call harbingers of future8 events.9 Bob Temple yesterday mentioned several10 times the elevation of the minor transaminase and11 bilirubin as potential indicator of a pathotoxicity,12 which would be very nice to see established by a13 formal means.14 But pharmacokinetics should be focusing on15 safety professionals in the companies and in the16 agencies on what to look for in drugs, and which can17 be identified from the preclinical data, and from18 pharmacokinetics, and pharmacodynamics, et cetera,19 throughout the development and life cycle of the20 product.21 DR. BEITZ: I think what I would just like22 to say is that by giving this activity a name, like23 pharmacovigilance plan, and it gives us the24 opportunity to actually talk about what we all want to25 be doing to monitor the product. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2181 And I am not so sure that the folks at the2 FDA always know exactly what industry might be doing,3 and vice versa, and I think if we had some clarify on4 what is appropriate or what is feasible, then we are5 way ahead of where we are now.6 DR. GALSON: Any other comments from7 folks, from the FDA, or any questions from the8 microphone, folks at the microphone?9 DR. BEITZ: This is not about10 pharmacovigilance plan, but this was actually related11 to our active surveillance question. I didn't hear a12 whole lot of comment about that, and it seemed as13 though what I was hearing was somewhat of a lukewarm14 endorsement of this kind of activity.15 And I was hoping that there might be some16 more discussion if possible. One of the things that17 we sometimes toy with is perhaps mining information18 from liver transplant centers, for example, to look19 for potential cases of liver failure, or acute liver20 injury.21 Does that sort of activity make sense to22 you all, and do you have any other possible examples;23 surveying emergency departments for adverse events24 coming in to such places, such settings? Any comment25 at all?2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2191 DR. STEPHENSON: I think that example2 makes a lot of sense. In fact, I know that when we3 were dealing with the issue with Bromfenach, that was4 one of the places that we went to, to see if there5 were any additional cases of liver failure. 6 It was a new transplant registry, I7 believe, or it was a liver failure registry, and that8 was very helpful. 9 DR. KAHN: In relation to the liver10 transplant registry, it doesn't cover all the11 transplant centers. It is only a subset, and it is12 very useful. But I think if one were to take the data13 from that database as published and presented by Dr.14 Lee out of the University of Texas at the Hepatoxicity15 Workshop, I think we would certainly have to take a16 certain amount of it off the market.17 DR. VASHISHTHA: This is a situation in18 which I think an active survey might make sense.19 DR. GALSON: If you would introduce20 yourself, please.21 DR. VASHISHTHA: I am Anshu Vashishtha22 from Watson. Probably in Stevens-Johnson23 situations, because again they are typically drug24 induced reactions and it might be fruitful to an25 active surveillance there. That is one situation that2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2201 I think comes to mind.2 We might for overdose consider poison3 control centers. I think it was mentioned earlier4 about not integrating the poison control and drug5 information centers, and they might be helpful to get6 (inaudible) in those situations.7 DR. GALSON: Let me just -- well, any8 comments on that comment? Yes.9 DR. STEPHENSON: Just one comment. I10 think that maybe some of us misunderstood what was11 being discussed, in terms of registries, as part of a12 pharmacovigilance plan.13 My comments were sort of focused on should14 you develop a registry for a particular product. I15 think that there are certain already existing types of16 registries, like liver transplants, or Stevens-17 Johnson, that obviously are very useful, but cut18 across all products.19 So you are basically starting with the20 reaction and looking to see whether there are any21 signals for any particular product. So I don't know22 if that was also what was intended in the document,23 and maybe that could be clarified a bit.24 DR. GALSON: Let me read one of the25 questions that came in on a card this morning. Dr.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2211 Hostelley stated that efforts should be directed2 towards limiting false positives of data mining. 3 The question is are similar efforts under4 way to limit false negatives, and next, further, how5 would one decide that a particular signal was a false6 positive. Is a controlled trial necessary in that7 case. Did you have any response to that?8 DR. STEPHENSON: I looked to members of9 the collaborative group, either Dr. Szarfman, or Dr.10 Almanoff, to perhaps respond.11 DR. SZARFMAN: Dr. Almanoff left,12 unfortunately, and I think that -- oh, I thought you13 left. First of all, we need to start the program a14 little bit. I think that the (inaudible) that we can15 do multiple comparisons and we can have multiple16 controls, and (inaudible) that having controls of what17 is going on with other drugs used to treat the same18 disease, so that you understand where you are.19 And it is a way of finding a (inaudible)20 for the event across other drugs that are being used21 (inaudible), and move away from the (inaudible)22 comparisons, and we need to look at the big picture23 and it is not two dimensions. It is multi-24 dimensional. But we are going to do the testing25 together.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2221 DR. ALMANOFF: The working group actually2 has a number of initiatives, and we will be studying a3 number of factors that affect performance, and what is4 the best database, and what are the best5 stratification schemes, and what are best practices6 for this tool.7 A couple of points. I wanted to reiterate8 what I said this morning about the fact that when I9 hear the term false positive, I get a little bit10 nutty, because I just view this as a positive signal11 as -- you know, as a -- I'm sorry, a positive score in12 a data mining run.13 I just view that as an intermediate step,14 and I view it as an alert that I should pull the cases15 and use my clinical judgment, and actually pull cases16 of related events, and maybe even look at related17 drugs.18 And then use all of that information and19 bring it together as an intelligent clinician to make20 a decision. So again I think that the risk of pulling21 these things out is very low. 22 And in terms of false negatives, maybe23 Anna has some thoughts on that, but -- well, go ahead.24 DR. SZARFMAN: The false positives. There25 were examples, for example, this morning about drugs,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2231 prozac, related to suicide. Well, suicide events that2 are suicidal ideation, et cetera, it is an indication3 where prozac is being used. 4 The identification of adverse events, the5 data mining finds them because they are called adverse6 events. If we are looking at frequency counts, we7 will find them also. 8 And this is not bad, and the data miming9 is behaving, and if we call them adverse events, or10 are defined as adverse events, then the interpretation11 is important. You drill down -- for example, with12 pemoline, I have a new signal for multiple sclerosis,13 and I drill down and this is a new thing now.14 The multiple sclerosis is an indication,15 an off-label indication, and where it is being used16 (inaudible) patients with multiple sclerosis. You17 drill down in the media until you realize what is18 going on. 19 Then maybe we need a protocol on20 indications or indications in a different field is21 solvable. The negative signals. Okay. This is very22 interesting. We are working with the same numerator.23 When people are analyzing reporting rates,24 we are working from the same database, then25 (inaudible) would agree that something somebody can2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2241 call a signal from data mining after looking and doing2 multiple comparisons, and we need to move and when we3 are comparing drugs, we need to look at the4 overlapping (inaudible) limits, then we are now using5 the tools to be able to do it systematically. 6 But when you know that when you are7 looking at the ones that are bad, the drug in8 question, and you are looking at the ones that are the9 same as the drug in question, and you look at the ones10 that are below the drug in question, and you see where11 -- because you can do multiple comparisons, that when12 we are looking at the (inaudible) output, and you have13 multiple comparisons, you know (inaudible) to do14 multiple comparisons when they are trying to find a15 (inaudible), then you can do these sorts of things.16 And if we are working with the same17 evidence, and we are working with the same numerator18 data, and essentially with the same denominator, then19 these things are not at issue.20 You know, what is the background that we21 need to use, and if we are using external sources of22 data that the location of the event in question was23 found very differently -- and maybe it is a paper, a24 published paper that assumes the expected value of25 acute liver failure across the board is one in a2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2251 million.2 But maybe for a specific disease it is3 different, and this is where data mining can help you4 within the same database, because if I know something5 is about, I touch the data, and I know that things can6 go wrong, and if you are using data from different7 sources, and even the management of the data is8 different, and it can make a difference.9 That's why we stratify by time, because it10 is different, like in 1968, and this is now, and this11 was the problem with the false positives and data12 mining thought that they were not stratified at all.13 And then they came to me and they showed14 me something else, and I knew beforehand that you are15 stratifying or having false positives. 16 DR. ALMANOFF: Just three more quick17 comments. First of all, in terms of false negatives,18 another thought that occurred to me was that I think19 that we cannot rely always on this tool to find every20 single signal of importance, and as somebody said21 earlier, one or two cases that are very compelling22 might be critical.23 So I think that for the way that we think24 it is good to look for signals is to use this to25 increase frequency, but if you see a case or two of2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2261 Stevens-Johnson Syndrome, or interstitial nephritis,2 or something that is clearly drug related, you know,3 that is the way that you deal with the question of4 false negatives.5 If it is medically important and likely to6 be drug related, then you can cover that by simply7 using your medical knowledge. No method is perfect.8 Two quick things about false positives. 9 One thing Anna has alluded to is the fact10 that there can be confounding by age and gender, and11 so internal stratification of the data really does12 help eliminate false positives, and we can talk about13 that off-line if folks are interested.14 The other thing that we have discovered as15 a source of "false positives: is the phenomena that we16 described as an innocent bystander phenomenon, and the17 way that you deal with that is that you might have two18 drugs that are used a lot together, and you can19 actually subset patients who are on one drug, or just20 the other drug, and on the combination.21 And as you actually go through those22 exercises, you can often comb out what is going on.23 So I hope that is helpful.24 DR. YONDRIN: Sam Yondrin, Millennium.25 This is just a different slant on data mining. I know2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2271 that when we talk about data mining currently the2 focus is on market put out data, but data mining has3 also been applied to clinical trial data, and4 primarily here, and which is where I have an interest,5 is how do we apply data mining methods, or at least6 develop these methods, such that we could probably7 identify predictive algorithms using statistical8 packages.9 And I think that this is where I think10 identifying or developing clinical -- you know,11 predictors, probably is within reach compared to12 pharmacogenomics, which is where my colleagues talk a13 lot about.14 But I think also in parallel that there15 may be a utility for clinical predictive algorithms.16 What do I mean? So if you were able to identify a17 patient, say, that was a male, and of a certain age18 group, and received the same concomitant medication,19 and had the lab parameter value, then they would have20 a certain probability for an adverse outcome.21 So I would just like to understand if22 there is any efforts in this direction, because in23 terms of data quality, we have a better change of24 achieving data quality in pre-marketing. 25 DR. ALMANOFF: Yes, this is an important2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2281 area, especially because of risk management and2 actually the working group has a subteam that Christy3 Chaung-Stein is going to head up to explore ways that4 one can look at NDA data. 5 There obviously have been a lot of6 challenges in sort of pooling lots of NDAs, and7 getting access to that type of information, but the8 view has been that any sort of -- and as Sidney says,9 harbingers that can be seen as smoking guns in10 clinical trials could be useful certainly for setting11 up pharmacovigilance plans.12 I know to date that Christy has done a13 little bit of work with this, and I know in one large14 NDA that she looked at actually didn't find anything,15 which I guess is a good thing. I don't remember what16 drug it was and how that turned out.17 But the other question though is that when18 we start digging into these NDAs, and how does that19 impact on the sort of more frequented safety profile,20 and how do you sort of reconcile the two, and will21 that become maybe a concern as companies move forward.22 Do they maybe feel that a more exploratory23 analysis is not something that they want to get24 involved with. So we will have to see how that goes.25 Thanks.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2291 MS. HOSTELLEY: I wanted to address2 something that has to do with quality, and also has to3 do with the nature of some of the events that go into4 adverse or post-marketing adverse event databases, and5 in which we make the assumption that they are6 unsolicited and that they are therefore potentially7 related.8 The example of the MedWatch form sent in9 by the physician is something that we should always10 assume a presumption of a relationship. In certain11 areas, particularly in oncology transplant medicine,12 and certain enzyme replacement therapies, the sponsor13 has extensive long term contact with groups or14 representatives in medical information, and has15 extensive long term contact with the treating16 physicians.17 And under those circumstances you collect18 information. An example, for example, in a transplant19 situation, everything went fine. You are looking20 good, and there was a wound dehiscence, and he had to21 go back to the OR.22 So that wound dehiscence goes in under23 those circumstances as a spontaneous report. They24 really can't be viewed as unsolicited reports. There25 is really no way of distinguishing that in the data2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2301 that are submitted.2 In fact, often the situation, and I think3 Sidney sort of alluded to, where a physician will4 adamantly deny that they submitted an adverse event5 report, and you place on them report forms, and send6 it to them and so on, when they don't believe that7 this is what happened.8 But because of compliance concerns, that9 is what you do. Now, as far as I can see, in each of10 these there is a field that you can use to reflect11 that, and in our database we do preserve it. But I12 don't see how that comes in to the information that13 the FDA has, except perhaps as a comment in the14 narrative, which is essentially lost. 15 So in thinking about data quality issues,16 I wonder if there is any way that we can deal with17 that kind of information. Thank you.18 DR. GALSON: Did anybody want to comment19 on that? 20 DR. BEITZ: That is certainly an issue21 that we have dealt with at CBER, and I don't know22 whether there is any kind of a field or a statement in23 the narrative. We are aware of certain companies that24 do have certain programs, where there is a lot of25 active calling and talking to centers and hospitals2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2311 where certain treatment is delivered.2 And we understand that under those3 circumstances that they are going to call and say --4 and this is sort of maybe a combination of finding out5 really what is going on and also to connect it to6 marketing to some extent.7 But if there is all this discussion, you8 know, they will say how are things going, and somebody9 will mention something that they might not have10 written in their report, and then it has to be put11 into the database.12 And so we are aware when we look at13 certain products that we understand that there is14 going to be an elevated rate of reporting because of15 these programs.16 But I don't know that there is any17 systematic way that we have of knowing across the18 board for which products there are such programs.19 Miles, do you want to --20 DR. BRAUN: Well, I think there is a box21 on the MedWatch form that relates to consumer reports,22 and so that is one potential way that those could be23 subsetted out, but that is definitely an issue, and24 sometimes companies actually have at least in25 biologics had -- they had a policy of submitting such2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2321 reports, and then they realized that for patient2 support programs that according to the guidance they3 are not necessarily obligated.4 I am talking about patient support5 programs now to support reports to the FDA, unless6 they believe that there is a reasonable association,7 causal association with a product. 8 And so you can have even within one9 product over time a change in the reporting for that10 product, and you could certainly imagine that there is11 a whole bunch, a large number of reports, that would12 have come into the agency in the early period, where13 everything was sent in.14 And then when they realized that there was15 a guidance that could be applied, and they did not16 have to send those in, that there would be a different17 -- that many of those earlier reports would no longer18 be sent in.19 And then if you stepped back and said,20 well, if you are using one of these data mining tools21 that really look at proportional rates, you basically22 could change the proportions radically even though it23 is the same product, and really with the same risk24 profile, artifactually change those proportions,25 because a big segment of the reports is no longer sent2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2331 in as part of the database.2 So I like to look at data mining as kind3 of a chainsaw. It is a good tool that can do4 important work, but it can be dangerous if not used by5 a trained operator.6 DR. GALSON: Is there follow-up to that?7 DR. GOLDSMITH: If I can add a point to8 that question, and Wendy, I am sure that you probably9 remember during our discussions of V2A, when we were10 actually together. There was a discussion of what you11 do with a clinical trial patient who has completed his12 clinical trial data.13 And indeed after the patient has completed14 his clinical trial data, you go back and you touch15 base with the investigator, and the investigator says,16 oh, by the way, this patient had 2 years after the17 trial was over, and that was considered not to be a18 spontaneous report in E2A, and it would be subject to19 analysis causality according to the rules for clinical20 trial data.21 DR. SZARFMAN: Of course, you can do a22 specific report for a (inaudible) coming from a single23 source. Then you can separate the other options that24 at a grade level you can look at the reports and25 photographs submitted by the applicant, and the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2341 reports for the same drugs omitted by other sources.2 There are ways of understanding what is going on.3 You know, we have many possibilities, and4 there are many tools, and a computerized environment5 can let you do things that before was impossible.6 DR. GALSON: Min, did you have a comment7 or anything that you wanted to add to that; that or8 the previous point?9 DR. CHEN: On the previous point, yes. I10 think that without going into any specifics about that11 report, I just want to say, yes, almost or a lot of12 reports from the spontaneous resource. 13 We know that they are solicited, and they14 are solicited either from studies or for disease15 management purposes. But I think that spontaneous16 reports is really -- that is the beauty of it. That17 it really accommodates all kinds of reports. 18 A lot of information flow in, but not19 necessarily that you can use every report for every20 purpose that you are looking into investigating. So21 you have to be careful how you use the information.22 Although you should allow a specific field23 to capture some information, we know that will help us24 to identify. These reports may not be directly25 related to the drug. But we know also that there is2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2351 always a kind of safety net somewhere, either within2 a company or in the FDA.3 When we look at certain issues with a4 certain drug, we will look at the reports, the case5 reports, and then look at the narrative. If it is not6 in the E2B field, we will look at the narrative and7 then decide whether there is a reasonable possibility8 that the drug has something to do with the event and9 make your best judgment to use that report as part of10 your case series analysis.11 So I am not that worried about the source12 of the information, and really the quality of the13 reports is the key here, and we are not going to get14 all kinds of reports with all kinds of efforts to15 gather the information to help us for every instance16 of the suspicious.17 But I think that the SADR rule, the18 proposed rule, dictates that really for those serious19 outcome events that the company should make more of an20 effort to append additional information to help us out21 on whether there are recognized events so that we have22 the information to analyze them, or for those23 recognized events with serious outcomes and increased24 frequency situations.25 And that we can use the quality2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2361 information that you obtain from the initial reporter2 to help to understand more about this recognized3 event. So there is a priority issue when you look at4 the whole database. 5 It is not that we don't encourage6 everybody to gather all the information for every7 case, but we have got to put our priorities there on8 how to use the information, and then we know how to9 gather the information for our own use and for10 everybody's use.11 So that is probably the most important12 message that I feel about from the reporting system.13 And data mining has all its uses, and we have14 evaluated so many situations in the past, and it is15 very useful.16 But the FDA database is so big, and it has17 30,000 -- I'm sorry, almost 3 million records, and18 from day one, 1969, all the way to 1997, it was coded19 in COSARS, and there were only four terms to be coded20 for each report.21 And we know that each case report can go22 over 40 or 50 events. So the limits are already there23 to capture quality of events on those older reports.24 And from 1997 on, they are unlimited events coded25 MedRA in those reports and with E2B element2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2371 specifications.2 So there is a whole horizon of new things3 going on since 1997, and we have this whole data4 mining technique applying to the whole thing. And we5 have to be careful what we are getting into the6 system, and what we are getting out of the system.7 And when we validate it, we have to think8 about all these caveats, all these validation, too, on9 whether it is there to help us out to understand the10 signals. 11 So I am really happy to see today that we12 had so much discussion about data mining, because we13 are sometimes in a dilemma saying how do we use these14 scores in our data lives, but we can't really ignore15 them, and we are trying our best to understand it, and16 see how we can best use it.17 DR. SZARFMAN: We stratify by year, and by18 age group, and by gender, and by report source. Then19 we are taking care of that problem, and (inaudible)20 device, the whole scheme, and I think that we need to21 look at the big picture, and I am willing to help as22 much as I can with interpretation and implementation.23 DR. GALSON: Victor.24 DR. RACZKOWSKI: The discussion about25 causality assessment makes me wonder whether this2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2381 might be a time to think about a shift in paradigm.2 Someone had suggested a possibility that instead of3 classifying a drug as an adverse events as possibly,4 probably, or unlikely related to drug, rather5 modifying the MedWatch form along the lines of do you6 think this drug -- what do you think this adverse7 event is due to.8 Do you think it might be due to the9 underlying disease, drug A, drug B, drug C. Might it10 be related to background incidence, and perhaps11 capturing that sort of information on a MedWatch form12 would be more informative than these attempts to13 causal associations for individual adverse events.14 DR. NELSON: I have two comments regarding15 spontaneous reports.16 DR. GALSON: Could you identify yourself.17 DR. NELSON: Oh, Bob Nelson, consultant.18 I have two comments. There was discussion this19 morning about how to best collect quality data for20 specific organ cells, and I really need to put a21 reference on the record. 22 There is an excellent book that most of23 you know about that Christian Beneshou (phonetic)24 wrote about a dozen years ago, and I would have to25 give you the exact title, but it is something like the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2391 detection and diagnosis of adverse reactions.2 In there, he has good logic strings of how3 to think through liver toxicity, bone marrow4 dyscrasia, et cetera. And there are data collection5 instruments that have been validated, and these are6 very, very strong tools to help you guide collection7 of the correct kind of data elements to help you8 determine whether or not the drug was likely to cause9 that reaction.10 It was a very powerful tool, and that if11 used correctly, will trump the idea of using an12 algorithm later on to determine whether it is possible13 or probable, because you will have such strong data.14 The second point about spontaneous reports15 regards the -- well, I just wanted to put that16 reference on record, because it is a very great text,17 and I think that most people know it. 18 The other one about reports that are not19 really spontaneous. There is of course a need to be a20 company comment field, and the CIOMS comment. It is21 good pharmacovigilance practice for a company to state22 their opinion on that case. 23 This was a non-solicited or a solicited24 report that we don't believe had anything to do with25 the drug, period on the bottom of the report. You may2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2401 in fact submit it because you feel that you need to2 submit it because of regulatory compliance, but you3 are also -- it is incumbent on you with good practices4 to state your opinion on the value of that case.5 And as Min Chen just said, when they do6 case series analysis, they just throw those out. So7 those are my comments. Thank you.8 DR. DANA: It is on a different topic, and9 it is when you get ready to talk about registries. So10 go ahead.11 DR. GALSON: Okay. 12 DR. KAHN: I was going to come back to Dr.13 Raczkowski's comments on the issue of causality14 assessment and also just Bob Nelson's. I am not15 absolutely sure that I know what value causality16 assessment on individual cases actually adds at this17 point, because causality is like pornography. You18 know it when you see it, but it is very hard to19 define.20 And typically the active pharmacovigilance21 professional will have (inaudible) and say I don't22 like the look of that, and they will go and explore23 for similar cases. 24 And at the case series is a far more25 useful tool for traveling to assess causality than2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2411 individual cases, and of course in the United States2 currently we do have the FDA disclaimer for legal3 protection, and if we were being asked as companies to4 add causality assessments to individual cases, there5 would probably have to be some type of legal relief6 that goes with that. So if I could just add that to7 the causality discussion.8 DR. YODRIN: Sam Yodrin, Millennium. Just9 a follow-up on the issue of causality. I think to10 really have clarity as to the role of causality11 assessments with ICSRs, we need to step back and ask12 ourselves what is the question that we are trying to13 answer. 14 Is it firstly does the drug cause this15 ADR, or does the drug cause the ADR in this patient.16 And they are two different questions. And the initial17 question that we are faced with is the broader18 question as to whether the drug causes the ADR.19 And the answer to that question is not20 from individual case causality assessments, and that's21 why when you have regulators like Germany or Japan22 asking for company causalities, it is misguided,23 because in the process of signaling, the first24 question is does the drug cause the ADR, and that25 requires a population approach.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2421 And actually Judy Staffa at the FDA has2 communicated this very well in several DIA sessions,3 and even on the FDA website. There are nice slides4 there that clearly communicate this concept for anyone5 who doesn't understand it.6 So individual case causality really has no7 place in terms of signaling. Now the case here is8 that one actually develops and that is where9 ultimately we need to address that question as to10 whether the drug causes the ADR.11 Now, what I haven't heard so far today is12 that a lot of the time we actually achieve that13 concept of threshold, and not based on some very14 specific or direct causality assessment.15 But from things like the clinical pattern.16 You know, either from time dependency, or time to17 onset, or clinical features of the case. And a lot of18 the time I find that is where the signal really lies,19 and this is where the case disposition is very20 important, how you achieve that case series that you21 now say that this is the analysis data set.22 We have heard during this workshop how23 several companies exclude relevant individual cases24 from that analysis data set simply because they look25 at risk factors involving patients, and they interpret2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2431 that as alternative etiological factors.2 And then there is also the misuse of risk3 factors which are confounders, but all these patients4 who have risk factors, there is a pattern there. The5 question is there is any sub-population or subgroup at6 risk for the event due to the drug.7 And so it is primarily -- I think the8 issue is one of pattern recognition, rather than some9 sort of algorithm that tells you that, yes, there is a10 causality threshold there.11 I am not dismissing that, you know, in12 totality, but a lot of the time we recognize signals13 simply based on pattern recognition.14 DR. FENISHELL: Bob Fenishell, Washington.15 We have heard that this has to be determined on a16 case-by-case basis, and I think that most people have17 come to roll their eyes at that, because it seems to18 be non-communicative.19 Nevertheless, in this case I think that is20 quite applicable and I guess the analogy to make is do21 we ask for causality on the efficacy side. When we22 were receiving hypertensive trials, did we ask the23 sponsors, well, let's look at this patient. 24 Do you think that the blood pressure25 change was attributable to the drug in this patient.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2441 What about this other patient. So we never ask that.2 It seems senseless, in part because the variance -- or3 what we were trying to derive from a large population4 of patients was a result which could not be derived5 from one in the period of observation that we had.6 And in a different trial design it could7 have been. There are trials to show the efficacy of8 antihypertensives, for example, in single patients.9 It can be done.10 But that is not the way that we did it,11 and that is not the way that most people do it in12 hypertension. Now my understanding, and now I am13 going to areas where my clinical experience is much14 less.15 But my understanding was that in the anti-16 infective area, even within the agency, even in areas17 of efficacy, individual patients were being followed18 in a sense that we never followed individual patients19 in anti-hypertensive trials.20 That an application for an anti-infective21 product is really a mass of little micro applications,22 one of which might be does this drug really work for23 serratia infections of the central nervous system24 where there is no localized abscess.25 Well, there are probably about five people2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2451 who have ever had that disease in the history of the2 world, and the sponsors managed to get three of them3 into the trial. 4 And so the real question is for each of5 those patients, gee, did that patient really have6 that. Is this confounded by other things that were7 happening with the patient at the same time because he8 was very sick, and he was getting six other treatments9 and so forth and so on.10 And so there was patient by patient11 causality consideration, which was necessary. I think12 it is going to depend on what the ADR is, and what the13 population is, and what the intrinsic variance is,14 whether one is able to approach the question as a15 variance reduction through sample size operation, or16 whether one is groping as one often is in safety17 matters with individual cases that must be18 individually massaged. 19 So there is no simple answer and I hate to20 say it, but we are back to case by case.21 DR. SZARFMAN: Except for a famous case of22 interaction between (inaudible) that was a single23 patient, and there was a blood level that was too24 high, and the patient had no risk factors. It was a25 young woman that received the two medications at once.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2461 And this brings up that I was a little2 disappointed that being a clinical pathologist that3 the comment of (inaudible) that we couldn't use blood4 level information, because I thought that this could5 be a very objective way of -- we have done it for6 toxicity for certain drugs.7 And I don't think that developing blood8 tests where you can measure blood levels will be9 really expensive.10 DR. GALSON: Thank you. Are there any11 other comments from the audience?12 DR. DANA: Can we switch gears to13 registries?14 DR. GALSON: Absolutely.15 DR. DANA: I am Joanna Dana in Regulatory16 Affairs for Amlin Pharmaceuticals. Having been in17 this industry for a number of years, probably more18 than most of you around here, I kept seeing patient19 registries, patient registries, meetings on patient20 registries.21 I was involved with Accutane and some of22 the early issues with that, and as I started to go to23 meetings to get a better appreciation of what was24 meant by this patient registry comment, two statements25 came out that were somewhat problematic. 2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2471 One, 65 percent of all recently approved2 drugs now have registries; and the second one is that3 35 percent are FDA mandated. Now, I don't know if4 this is a true statement or not, and it came from a5 meeting that was entitled, "Patient Registries."6 And it may have its own sort of internal7 self-fulfilling purposes for those that spoke at that8 meeting. But the issue in listening to the various9 presentations there was that the definition of patient10 registry there was quite different than what I see in11 this document.12 And to take the spectra of definitions13 that are out there for what compiles anything from14 what used to be called in a seating study to one that15 is a very specific pregnancy registry, and for one16 that is therapeutic utilization and call it all the17 same thing, is problematic.18 So I think clarity of that definition19 within this document would be extremely useful.20 DR. GALSON: Thank you for that comment.21 Any other follow-ups on that registry issue or other22 registry issues? Okay. If not, I am going to turn23 the mike over to Julie Beitz for some closing24 comments.25 DR. STEPHENSON: Before you do that, we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2481 had a question for the FDA.2 DR. GALSON: Okay. Oh, the second3 question, your second question.4 DR. STEPHENSON: Yes. 5 DR. GALSON: Why don't you read it again,6 unless you want to put it up on the screen.7 DR. STEPHENSON: It had to do with --8 since both FDA and industry tend to use9 pharmacoepidemiology data studies, the question was,10 it would be helpful to hear FDA's thoughts on when a11 study should be conducted by the company, versus the12 FDA, or both.13 And then the more specific question is14 that if the FDA is going to be conducting a15 pharmacoepidemiology study on a company's drug, are16 there any obligations to include the company or17 communication, or collaborate, with the company. That18 was the question.19 DR. GALSON: Who wants to go after that20 one?21 DR. RACZKOWSKI: I will go ahead and22 start. Victor Raczkowski. The paradigm in general at23 the Food and Drug Administration is that typically24 sponsors perform studies on their products.25 And my sense is that what we often times2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2491 require in those cases, however, is that we have some2 sort of -- and particularly on the pre-approval side,3 is access to the data to be able to validate the4 results of studies, and efficacy studies in5 particular.6 And my initial reaction to seeing that7 question is that it seems likely that in the post-8 marketing arena as well that generally these Phase IV9 type studies on safety issues would be conducted by10 sponsors.11 But I do think that raises the issue of12 then how do we go and look at the data, and how do we13 validate the patient population, and what are the14 tools that we need at the FDA if we are not doing the15 studies ourselves in order to be able to give us some16 level of comfort that the result is as it is stated.17 I would also say, however, that this is --18 you know, we are trying to be somewhat innovative in19 this area, and we are making great attempts to see20 what types of resources and databases that we might be21 able to utilize, such as claims databases, in order to22 answer particular public health questions.23 And Judy can talk about that more24 specifically, but I think that there has not been a25 lot of attention to this necessarily in the past, and2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2501 so we are trying to be very proactive in that respect,2 in terms of trying to find out how we might be able to3 use databases that were not particularly designed to4 answer public health questions.5 But that also requires a certain level of6 validation, such as access to medical records, and7 that sort of thing.8 MS. HOSTELLEY: I think the question9 behind the question was sort of the awareness that the10 agency is moving into today's world into consideration11 of doing pharmacoepidemiologic studies like through12 various academic centers, such as through the CERTS13 organizations, et cetera.14 And so I think that the driver for the15 question was that apart from perhaps seeing a listing16 on the web that this is a study being undertaken17 through a grant provided by the agency, PhRMA was18 asking the question is it routine to sponsor that at19 least that you are doing that since it does involve20 the sponsor's drug.21 So that -- I mean, it is helpful to have a22 dialogue with the sponsor in order to ensure that the23 sponsor is not sitting there thinking about doing the24 same type of study. So that we can figure out a way25 to better collaborate and more effectively utilize2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2511 resources.2 DR. GALSON: Did you want to add something3 to that?4 DR. RACZKOWSKI: I just wanted to say that5 I think that is a fair comment. 6 DR. BEITZ: What I actually was going to7 say was that at least some of our resources that we8 have through cooperative agreements, for example, are9 fairly limited in scope, and so I think the kinds of10 studies that we might design to do in using the funds11 there, versus the kinds of studies you might be able12 to conduct are quite different.13 But there is no doubt that some dialogue14 might be useful.15 DR. BRAUN: I just wanted to mention from16 the observational studies, that if you look certainly17 in the cancer arena and to some extent also in18 cardiovascular disease, it is well known that the19 studies don't agree all the time.20 In fact, there is a large amount of21 disagreement among findings of independent studies,22 and I think that is an important consideration when we23 talk about the numbers of studies in this area.24 So if there is an important question one25 could look at it as actually a strength that there are2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2521 two separate groups looking at it independently. And2 so I am not sure -- you know, there is a concept of --3 some might say that it is not cost effective to4 duplicate efforts. 5 In my view, it is not duplication, and in6 safety systems redundancy sometimes is actually a good7 thing to have. So I see some merit in actually doing8 that. So independently and actually what might seem9 in a duplicative way. So that was my point. 10 DR. STAFFA: I think that Miles makes a11 good point about the -- there is not -- it is not12 always bad to have two independent studies going on of13 the same issue at the same time, because the14 differences in those findings can be instructive.15 But I also think that it is an intriguing16 idea to have more dialogue and collaboration. I know17 as a reviewer of protocols that come in that we would18 like to see some of our comments and issues be19 incorporated into the protocols, and I am sure that20 the same is true, vice-versa. 21 And I think that we would welcome22 insights. One of our frustrations and one of the23 reasons that we are trying to build more capability to24 do studies on our own is that we sometimes find that25 in collaborative efforts, when we can't touch the2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2531 data, we can't always do the designs or ask the2 questions in a way that we would really like to see3 them asked. And so sometimes that is really what is4 driving us to do that.5 DR. GALSON: Let me just make one comment6 on that myself, which is that clearly we don't have an7 agency policy on this question right now, and I do8 think that it would be very valid for us to cover this9 in the guidance document, because it clearly is10 something that deserves clarity, in terms of know11 where we stand on this question. 12 So we will definitely take that into13 consideration. Are there any other follow-ups on that14 issue? Any additional issues?15 MS. BORSTU: Yes, maybe one. I will say16 that I think --17 DR. GALSON: Just identify yourself.18 MS. BORSTU: Maria Borstu (phonetic),19 Organom International USA. My question pertains to20 the very far end of the third concept paper, I think,21 but it links into this. We were talking about22 multiple parties studying the same issue, and23 consistency of results.24 But what we know from history usually25 there isn't consistency in results, especially if we2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2541 have more than just the FDA or the government, and the2 PhRMA industry is studying the issue as soon as you3 have a -- how do you call that -- academia,4 government, industry, studying and sometimes even5 duplicate partners.6 Are there any thoughts or have there been7 any thoughts during the discussions about guidance on8 interpretation of the findings from the different9 sources from the different studies done by the10 different parties using different sources.11 I was thinking that there would be a need12 somewhere along the line for norms and criteria as to13 put some weighting on the various studies that we all14 are going to do. I was just curious whether that has15 been on the table already.16 DR. GALSON: Judy.17 DR. STAFFA: Actually, I think that is an18 excellent point and I think that is something that we19 are planning to tackle when we get to the guidance20 stage. 21 Clearly we are going to have to at least22 lay out some criteria that tries to speak to that23 issue, because it happens routinely.24 DR. UNGER: Ellis Unger from CBER. This25 is a miscellaneous comment. At the end of the day,2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2551 irrespective of how a safety signal is generated,2 whether it is data mining or what have you, we have to3 apply judgment.4 And the quality of the judgment is5 inextricably tied to the quality of the data, and at6 the end of the day the FDA reviewer has maybe 15 forms7 on his desk or her desk.8 And the quality is often marginal, and9 sitting here I keep thinking about the potential of10 the internet for collecting the ADRs. If there were11 an interactive system developed, whereby what is the12 patient's problem, and check box, liver problem, and13 then it went through an algorithm and took it all the14 way out to did the patient have a biopsy, yes, no,15 don't know, et cetera, obviously it would take time to16 develop.17 But eventually we could get some very good18 data I would think from a system like that, and I will19 just put that idea out there.20 DR. KAHN: Can I answer that to some21 degree? Without naming names, which is not22 appropriate here, in fact such a system has been23 developed by an independent private vendor and I am24 not sure to what extent it has been tested and25 validated.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2561 But I have actually seen a demonstration2 of exactly such a system. 3 DR. GALSON: Okay. Thank you. Anything4 else? All right. I would like to turn the mike over5 then to Dr. Julie Beitz.6 DR. BEITZ: As you have heard today, there7 are many challenges to be faced in carrying out good8 risk assessment based on observational data sources.9 Some of the more thornier questions that came up today10 revolved around how can the quality of adverse event11 reports be improved, and what conclusions can be drawn12 from observational data sources given the inherent13 limitations of these data.14 And what more can be done to enhance our15 understanding of safety signals once they are16 identified. On behalf of the members of Group III,17 the CDER/CBER Pharmacovigilance Working Group, I would18 like to thank all of the attendees today for their19 well considered comments and questions.20 We plan to consider all the comments that21 we received today, as well as those submitted to the22 docket when our prepare our draft guidance document.23 I also would like to thank all the members of the24 steering committee and of the working group itself for25 their scientific input on the concept paper thus far.2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2571 But I would also like to thank Lee Lemley2 and Diane Erlich for organizing a very successful3 public workshop. And now I turn it over to Dr. Galson4 to close.5 DR. GALSON: Thank you very much, Julie,6 and I want to just very quickly thank a number of7 people in particular today, the working group on8 pharmacoepidemiologic. You have done a wonderful job.9 The document was excellent, and we had good discussion10 today.11 Also the members of the executive12 oversight committee who participated today and13 throughout the week. Specifically, I want to thank14 the project managers for each of the three working15 groups. These products and this meeting wouldn't have16 gone so well if it was not for your hard work.17 And to all the people sitting up here18 today, and all the audience for your participation,19 thanks very much. Please submit written comments if20 there are things that you feel haven't been covered.21 The docket is going to be open and we need22 those comments for us to produce high quality draft23 guidance documents over the next few months. Thanks24 again.25 (Whereupon, at 3:26 p.m., the meeting was2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2581 concluded.)234567891011121314151617182 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com

Department of Health and Human Services s19

Details

Download

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

Support