Department of Health and Human Services s19

1 1 1 UNITED STATES OF AMERICA 2 DEPARTMENT OF HEALTH AND HUMAN SERVICES 3 FOOD AND DRUG ADMINISTRATION 4 5 + + + + + 6 7 RISK MANAGEMENT PUBLIC WORKSHOP 8 PHARMACOVIGILANCE PRACTICES AND 9 PHARMACOEPIDEIOLOGIC ASSESSMENT 10 11 + + + + + 12 13 FRIDAY, 14 APRIL 11, 2003 15 16 + + + + + 17 18 The Workshop was called to order at 8:07 19 a.m., in the NTSB Board Room and Conference Center, at 20 429 L'Enfant Plaza, S.W., Washington, D.C., by Steven 21 Galson, Deputy Director, CDER, presiding. 22 23 PRESENT: 24 25 DR. STEVEN GALSON Deputy Director, CDER 26 DR. MARK MCCLELLAN Commissioner FDA 27 DR. SUSAN ELLENBERG CBER 28 DR. JULIE BEITZ CDER 29 DR. MILES BRAUN CBER 30 DR. MIN CHEN CDER 31 MS. AILEEN CIAMPA CDER 32 DR. EDWARD COX CDER 33 PATRICIA GUINN CDER 34 DR. VICTOR RACZKOWSKI CDER 35 DR. PAUL SELIGMAN CDER 36 DR. JUDY STAFFA CDER 37 DR. JOYCE WEAVER CDER 38 DR. CAROL HOLOQUIST CDER

2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 2 1 PUBLIC COMMENT: 2 3 DR. GERALD FAICH Pharmaceutical Safety 4 Assessments 5 DR. JOHN FERGUSON Biotechnology Industry 6 Organization 7 GREGORY GOGATES CRF Box, Inc. 8 DR. STEPHEN GOLDMAN Stephen Goldman Consulting 9 Services, LLC 10 LINDA HOSTELLEY Merck, PhRMA 11 DR. SIDNEY KAHN Pharmacovigilance & Risk 12 Management, Inc. 13 DR. ROBERT C. NELSON RCN Associates 14 DR. ANNETTE STEMHAGEN ISPE Membership Committee 15 DR WENDY STEPHENSON PhRMA 16 DR. ANSHU VASHISHTHA Watson Pharmaceuticals

2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 3 1 C-O-N-T-E-N-T-S 2 3 Welcome and Introductions...... 4 4 5 Session I - Overview and Good Pharmacovigilence 6 Practices 7 FDA Presentation by Dr. Julie Beitz...... 8 8 FDA Presentation by Dr. Min Chen...... 14 9 10 Oral Presentations - Public Comment 11 Dr. Robert C. Nelson...... 21 12 Dr. Stephen Goldman...... 36 13 Dr. Gerald Faich...... 53 14 15 FDA Presentation by Dr. Mark McClellan...... 67 16 17 Oral Presentations - Public Comment 18 Dr. Anshu Vashishtha...... 92 19 Mr. Gregory Gogates...... 102 20 Ms. Linda Hostelley...... 111 21 22 Panel Discussion...... 120 23 24 Session II - Pharmacoepidemiologic Assessment 25 FDA Presentation by Dr. Judy Staffa...... 157 26 27 Oral Presentations - Public Comment 28 Dr. Annette Stemhagen...... 172 29 Mr. John Ferguson...... 186 30 Dr. Wendy Stephenson...... 193 31 Dr. Sidney Kahn...... 201 32 33 Panel Discussion and Q&As...... 212 34 35 Closing Remarks by Dr. Julie Beitz...... 256 36 37 Adjournment...... 258

2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com 1 4

1 P-R-O-C-E-E-D-I-N-G-S

2 (8:07 a.m.)

3 DR. GALSON: Welcome. I hope to have a

4 good group today, and I am sure that we will have more

5 people filtering in as time goes on. I wanted to

6 welcome you to the third day of our Risk Management

7 Public Workshop, and what I am going to do first is

8 just ask the FDA staff who are here to quickly

9 introduce themselves.

10 And first up at the table here, if you

11 folks would just sequentially push your buttons and

12 say who you are and what you do.

13 DR. MILES BRAUN: My name is Miles Braun,

14 and I am the Director of the Division of Epidemiology

15 in the Center for Biologics at FDA.

16 DR. SELIGMAN: Good morning. I am Paul

17 Seligman, and I am the Director of the Office of

18 Pharmacoepidemiology and Statistical Science in the

19 Center for Drugs at the FDA.

20 DR. BEITZ: I am Julie Beitz, and I am the

21 Deputy of the Office of Drug Evaluation III.

22 DR. COX: And I am Ed Cox, Deputy

23 Director, Office of Drug Evaluation IV, CDER, FDA.

24 DR. GALSON: Okay. And then we have got

25 at the table at my left is the working group that

1 focused on this concept paper that we are discussing

2 today, and if we could just go down the table.

3 DR. WEAVER: Joyce Weaver, Office of Drug

4 Safety

5 DR. HOLQUIST: Carol Holquist, Office of

6 Drug Safety.

7 DR. CHEN: Min Chen, Office of Drug

8 Safety.

9 DR. STAFFA: Judy Staffa, Office of Drug

10 Safety.

11 DR. GUINN: Patrick Guinn, Office of Drug

12 Safety.

13 DR. GALSON: And that is Lee Lemley, and

14 thank you, Lee, for doing all the hard work to

15 organize this 3 day meeting. All right. First, I

16 want to thank the members of the CDER/CBER Executive

17 Oversight Committee, none of whom are sitting here,

18 but I think we will have some a little bit later.

19 Going quickly over this, particularly for

20 those of you who may be here for the first day today,

21 we have had -- this is our third day of meetings to

22 discuss three concept papers that corresponded to

23 three groups working in this area, and today's meeting

24 is focused on pharmacovigilance.

25 MS. LEMLEY: We will be with you in a

1 minute.

2 (Brief Pause.)

3 DR. GALSON: So, let me go over the

4 schedule for today. We are in the introductions now,

5 and then we are going to be split up into two general

6 areas today. The first session this morning is

7 overview and good pharmacovigilance practices.

8 We are going to have a number of FDA

9 presentations, and then oral presentations from the

10 folks sitting at the table in front. We will then

11 have a break at 9:45, and Dr. Mark McClellan, the

12 Commissioner of the FDA, will be here at -- we should

13 have some time for him to ask and to respond to

14 questions from the audience.

15 Then we will continue with oral

16 presentations and have a discussion among the people

17 who spoke, and questions and answers from the

18 audience. At 11:30, we will have lunch, and then we

19 will start on the second session,

20 pharmacoepidemiologic assessment, and FDA's

21 presentation, and then presentations from speakers,

22 and then questions and answers from the audience, and

23 then some closing remarks at the end.

24 Some logistics. We are going to

25 distribute cards. If you don't want to go up to the

1 microphone and you just want to write down your

2 questions on the cards, hand them to the people who

3 will be in the aisles, and we will bring them up and

4 we will open mikes.

5 The transcripts from these sessions will

6 be available within 30 days on the website you see,

7 and as well, we have got a general website for the

8 meeting, and we are going to try to post all of the

9 slide presentations that you are seeing today and for

10 the last few days on that website address. So look

11 for that if you are interested in copies of the

12 slides.

13 No food or drinks are allowed in here.

14 The restroom is out in the lobby, and please note the

15 fire exits from the room. Thank you very much.

16 Before we move on with the program, I just want to

17 have the speakers for this morning start, and those of

18 you that are sitting at the table, just introduce who

19 you are, and we will get around to each of you. Just

20 real quickly on the mike.

21 DR. NELSON: My name is Bob Nelson, and I

22 am a consultant in Drug Safety and Regulatory Affairs.

23 DR. FAICH: I am Gerry Faich, and I am a

24 second consultant in Safety and Regulatory Affairs.

25 DR. GOLDMAN: Steve Goldman, and I am the

1 third consultant in Safety and Regulatory Affairs.

2 DR. GALSON: Okay. No shortage of

3 consultants.

4 DR. NELSON: The order means only

5 administrative order.

6 DR. GALSON: Okay. I am now going to turn

7 the mike over to Dr. Julie Beitz, who has done a great

8 job managing the work involved in creating this third

9 concept paper. Julie.

10 DR. BEITZ: Good morning, and welcome to

11 day three of the CDER/CBER Risk Management Public

12 Workshop. Today, we will be discussing the concept

13 paper entitled, "Risk Assessment of Observational

14 Data."

15 This paper was written in a general way to

16 stimulate discussion at this workshop regarding what

17 constitutes good pharmacovigilance practices and good

18 pharmacoepidemiologic assessment.

19 Comments that we receive today and in the

20 docket will be considered as we prepare a longer and

21 more comprehensive draft guidance document in the

22 coming weeks and months.

23 The CDER/CBER pharmacovigilance working

24 group represents a broad spectrum of talented

25 individuals at FDA who came together to produce a

1 concept paper in very short order. Drs. Mark

2 Goldberger and Miles Braun, served as group leads for

3 CDER's Office of New Drugs, and for CBER.

4 Our support staff, notably Aileen Ciampa,

5 provided regulatory input; and our project manager,

6 Patrick Guinn, helped with scheduling meetings and

7 organizing today's workshop. Many group members are

8 in attendance today, and you will have an opportunity

9 to interact with them.

10 Dr. Ed Cox will be filling in for Dr.

11 Goldberger, who could not be with us today. You have

12 already seen the agenda presented, and I will just

13 point out that we are currently in the overview for

14 the morning session.

15 The concept paper under discussion today

16 encompasses the following: Important

17 pharmacovigilance concepts; safety signal

18 identification; pharmacoepidemiologic assessment and

19 interpretation of safety signals, and a development of

20 pharmacovigilance plans.

21 Today's discussion will focus on risk

22 assessment based on observational data sources,

23 including case reports, case series,

24 pharmacoepidemiologic studies, registries, and

25 surveys.

1 Risk assessment, in the context of

2 clinical trials, was discussed on the first day of

3 this workshop. First, let me begin by defining the

4 concept of pharmacovigilance. As we have stated in

5 the concept paper, pharmacovigilance includes all

6 post-approval, scientific, and data gathering

7 activities relating to the detection, assessment,

8 understanding, and prevention of adverse effects, or

9 other product related problems.

10 This includes the use of

11 pharmacoepidemiologic studies. Risk assessment occurs

12 throughout a product's life cycle. At the time of

13 approval, available clinical trial data involve

14 limited numbers of patients treated for relatively

15 short periods of time.

16 In drug approval, large numbers of

17 patients may be exposed to a product, including

18 patients with co-morbid illnesses, patients using a

19 variety of concomitant medications, and patients using

20 the product chronically.

21 After approval, new safety information may

22 become available from a variety of sources through use

23 of the product either domestically or in other

24 countries, through use of other drugs in the same

25 class, from preclinical or pharmacologic studies of

1 the product, or from controlled clinical trials.

2 Many definitions have been put forth to

3 characterize a safety signal. For the purpose of

4 today's discussion, we have defined a signal as an

5 apparent excess of adverse events associated with the

6 use of a product.

7 Small numbers of well-documented case

8 reports, even a single case report, may be viewed as a

9 signal. In addition, we believe that pre-clinical

10 findings or experience with other products in the same

11 class, may be sufficient to generate a safety signal

12 even in the absence of case reports on patients.

13 Thus, a products risk profile may be

14 characterized by several safety signals. Throughout

15 the concept paper, we have tried to clarify the

16 following activities as they relate to safety signals.

17 We say that signals are identified from

18 case reports or other sources. That signals are

19 evaluated in pharmacoepidemiologic studies,

20 registries, or surveys. That signals are interpreted

21 in the context of all available safety information,

22 and that signals are monitored through enhanced

23 pharmacovigilance efforts.

24 This paper also introduces the concept of

25 a pharmacovigilance plan. This would be a plan

1 submitted by a sponsor for the ongoing evaluation of

2 safety signals identified by the use of a product.

3 The plan could also include monitoring for

4 at risk populations which have not been adequately

5 studied to date. The plan may be developed at the

6 time of product launch, or after a signal is

7 identified.

8 For a product without safety signals

9 identified pre-or-post approval, and for which at-risk

10 populations have been adequately studied, routine

11 post-marketing reporting may suffice as a

12 pharmacovigilance plan.

13 For a product with safety signals

14 identified pre-or-post approval, or for which at risk

15 populations have not been adequately studied, a

16 collection of additional safety information beyond

17 routine post-marketing reporting may be desired.

18 Examples of the kinds of activities that

19 could constitute a sponsored pharmacovigilance plan

20 include expedited reporting of serious adverse events

21 of interest, submission of adverse event report

22 summaries at more frequent pre-specified intervals.

23 For example, on a quarterly basis rather

24 than annually; conduct additional observational

25 studies or control trials; implementation of active

1 surveillance activities to identify as yet unreported

2 adverse events.

3 We recognize that many of these activities

4 are already being carried out by sponsors, even though

5 the term, pharmacovigilance plan, has not yet been

6 applied to them.

7 Active surveillance efforts may involve

8 data gathering activities that are focused on specific

9 products of interest on specific health care settings,

10 such as emergency departments, or on specific events

11 that are often considered to be drug related.

12 FDA looks forward to hearing public

13 comment on active surveillance strategies using

14 electronic health information systems or other

15 databases. In particular, we would like to hear your

16 thoughts regarding the circumstances for which these

17 efforts would prove most useful.

18 We recognize that emerging new safety data

19 may result in ongoing revisions to the sponsor's

20 pharmacovigilance plan for a product.

21 While additional safety information is

22 being generated, the FDA will work with sponsors to

23 communicate information about safety signals, and

24 minimize events occurring in the users of a product

25 through risk management programs as appropriate.

1 This concludes my presentation. We will

2 now move on to the next FDA presentation on Good

3 Pharmacovigilance Practices.

4 DR. GALSON: Thank you, Julie, and our

5 next speaker is Min Chen, who is the Associate

6 Director of the Division of Drug Risk Evaluation in

7 our Office of Drug Safety.

8 DR. CHEN: Good morning. I am here to

9 talk about Good Pharmacovigilance Practices and our

10 current thinking at the FDA. As Dr. Beitz mentioned,

11 the pharmacovigilance definition is generally regarded

12 as all post-approval scientific and data gathering

13 activities relating to the detection, assessment,

14 understanding, and prevention of adverse events, or

15 any other product related problems.

16 This includes the use of

17 pharmacoepidemiological studies, too. We think that

18 good pharmacovigilance practice starts by acquiring

19 complete data from spontaneous adverse events or

20 reports. It is critical for sponsors to actively seek

21 information on an adverse event by direct contact with

22 the initial reporter so that a thorough assessment of

23 the event can be made expeditiously.

24 Good reporting practices are available in

25 several guidances that we have. We are looking for

1 the following information in a good case report,

2 whether reported spontaneously or published in the

3 medical literature.

4 The adverse event details, description,

5 patient's baseline characteristics, and concomitant

6 drug therapy details, time to onset of the signs or

7 symptoms, diagnosis of the event which hopefully can

8 be supported by the lab data stated here, and the

9 clinical course of the event, and the outcome; not

10 just regulatory outcomes, but also the patient outcome

11 in the end of this event.

12 Any other relevant information can help us

13 in a good case report. For reports of medication

14 errors, a good case report would also include a full

15 description of the following; the products involved,

16 the sequence of events leading to the medication error

17 and the work environment in which the error occurred.

18 Also, types of personnel involved with the

19 error. We encourage sponsors to include in the case

20 narrative all of the data elements outlined in the NCC

21 MERP taxonomy. That is National Coordinating Council

22 for Medication Error Reporting and Prevention.

23 When case reports are identified in

24 adverse events where there are bases that associate a

25 similar event with a product, a case series could be

1 developed. In FDA's experiences, identification of

2 all clinical relevant cases depends on thorough search

3 strategies based on measured terminology.

4 Generally, case definitions would be

5 developed to provide consistent characterization of

6 the adverse event of interest, and to facilitate the

7 retrieval of all clinically relevant cases from the

8 database. In addition, data mining techniques may be

9 applied to the databases to identify some cases of

10 interest.

11 For any individual case report the FDA has

12 found that without complete details, it is rarely

13 possible to know with absolute certainty whether it

14 was product induced.

15 However, a number of features as follows

16 are generally recognized as supportive of an

17 association between a product and an event. If the

18 event occurs in the expected time frame -- for

19 example, allergic reactions occurred within a few

20 hours, or a few days

21 -- most likely cancer will develop after long term

22 years of therapy.

23 If there are no symptoms related to the

24 event prior to exposure, and if there were no other

25 use of a concomitant drug that could contribute to the

1 event or co-morbid conditions. Of course we are

2 looking for the positive challenge, or positive

3 rechallenge scenarios if we have them.

4 Also, the event is consistent with the

5 established mechanism of action of the product.

6 However, we always keep that in mind, that

7 idiosyncratic reactions that are not necessarily

8 consistent with the mechanism would be considered in

9 the assessment.

10 For medication error related events, the

11 FDA would expect the sponsor to evaluate each event to

12 identify the root cause of factors that led to the

13 actual error, or even possible error, the NCC MERP

14 that taxonomy that was mentioned before would be used.

15 When appropriate the case reports may be

16 grouped into probable, possible, or unlikely for our

17 assessment purpose. In general, a safety signal may

18 be described as an apparent access of adverse event

19 associated with the products used.

20 Occasionally, however, even a single well-

21 documented case report, particularly if the report

22 describes the positive rechallenge, may be viewed as a

23 potential signal. In addition, technical findings

24 with a product, or experiences with other similar

25 products in the class may be sufficient to generate

1 the hypothesis for a safety signal.

2 Safety signals may be further assessed in

3 terms of their magnitude, population at risk, changes

4 in risk over time, biological possibilities, or other

5 factors. A product's risk profile may be

6 characterized by several safety signals.

7 Data mining can be used as a signaling

8 tool by using the Bayesian methodology, and data

9 mining uses raw case report data, and all drug adverse

10 event combinations. It calculates the expected number

11 of events for all drugs, and then compares each drug's

12 observed to expected count.

13 There are many methods that may be used to

14 generate the relative signal scores observed and

15 expected count. This may be a useful adjunct to our

16 routine safety signaling methods.

17 However, our current thinking is that

18 further exploration and validation is still needed to

19 use as a tool. Safety signals may be able to tell us

20 any new unlabeled adverse event or observe an increase

21 in the severity or specificity of a labeled event, and

22 an increase in the frequency of a labeled event, and

23 new interactions, from drug interaction, drug food

24 interaction, or drug dietary supplement interactions.

25 Of course, where there is a confusion with

1 the product's name, package, or use, actual or

2 potential, then that is a signal to us. After

3 detecting the potential safety signals from the case

4 series, this descriptive analysis of the cases are

5 very useful in characterizing the adverse event and

6 the population at risk.

7 These analyses should include

8 demographics, including age, gender, or race

9 information, the effect of dose exposure duration, is

10 it short or long term duration exposure, and the dose

11 effect. If there is any concomitant medication, then

12 what is the potential interactions between the drug

13 and the suspect, and the rest of the event.

14 And if there is a co-morbid condition of

15 the patient, such as baseline hepatic or renal

16 impairment, in the risk event. We are looking for a

17 lot to lot differences in product formulation, and the

18 risk of the event, and if there is a potential for an

19 access adverse event given the disease being treated,

20 such as in an advanced cancer condition or

21 immunocompromised patients that we have to consider

22 all these conditions in the analysis.

23 Finally, we would like to do some

24 estimates of the magnitude of risk or differences from

25 the known background rates of the events that are

1 interesting. All these risk assessments of post-

2 marketing data will be considered if further

3 pharmacoepidemiologic studies may be recommended to

4 further characterize the risk.

5 Dr. Judy Staffa will present this

6 afternoon the various aspects of pharmacoepidemiologic

7 studies. The questions we are asking for public

8 comments for these sections of the guidelines for

9 pharmacovigilance practices are how can the quality of

10 spontaneous reported case reports be improved.

11 What are the possible advantages or

12 disadvantages of applying data mining techniques to

13 spontaneous reports databases for the purpose of

14 identifying safety signals.

15 Finally, what are possible advantages or

16 disadvantages of performing causality assessment at

17 the individual case level, or even at case series

18 level. Thank you.

19 DR. GALSON: Thank you, Min. We are going

20 to move on now to oral presentations from members of

21 the public who have preregistered, and the first of

22 those is Dr. Robert Nelson.

23 DR. NELSON: Good morning and thank you

24 once again for the opportunity to comment. And it is

25 always a pleasure to follow my old colleague, Min

1 Chen. I know that she is a cornerstone for all the

2 activities that are happening here.

3 For the record, my name is Bob Nelson, and

4 I am a consultant in drug safety and epidemiology.

5 Previously, I was a commissioned officer in the Public

6 Health Service, having spent 20 of my 23 years at the

7 Food and Drug Administration, both in the Office of

8 New Drugs and in the Post-Marketing Surveillance

9 Activity, with my last position being the Assistant

10 Director in what is now called the Office of OPaSS, I

11 guess. It changes so rapidly.

12 What I want to begin with is just putting

13 up my set of definitions, and not to argue with the

14 definitions in the concept documents, but just to

15 point out two things.

16 In general, the term, pharmacovigilance is

17 considered a much more narrow term, usually focused on

18 spontaneous reports and not being the umbrella term

19 which I would call risk assessment.

20 And the second point is that the only term

21 that is in the regulations is post-marketing

22 surveillance, and that is again an umbrella term that

23 encompasses all safety activities, not just the ones

24 that we are talking about today, but also advertising

25 and quality of the products, field inspections, and so

1 on. So that is a term that should not be left out.

2 But I will just leave those for you guys

3 to look at. But what I will spend most of my time

4 talking about is just commenting as you have requested

5 to the questions that you have laid out, and then I

6 will end by discussing some popular myths.

7 The first one, of course, is how can the

8 quality of spontaneously reported cases be improved.

9 And I think that the most important thing that could

10 occur is the philosophy and the position that the

11 agency takes on them.

12 And I call that the public health focus

13 philosophy, meaning that when a report comes from a

14 health professional, the agency should consider it

15 true for what the health professional has reported it

16 as, unless the company does sufficient research to

17 show otherwise.

18 I think that is an important position to

19 take rather than the Congress, and that is a public

20 health position, true unless shown otherwise. That

21 encourages to obtain as full data sets as possible,

22 and rule out alternate explanations, or rule in

23 alternate explanations as the most likely cause.

24 And I need to remind everybody that

25 confounding, or in other words, multiple indications

1 or medical conditions, and drugs, does not rule out a

2 causal relationship, which I see so often done.

3 Secondly, I think we have to focus on

4 those reports that are likely to be important. Those

5 reports that are for medically significant outcomes,

6 commonly known in the regulatory vain as serious, but

7 in addition there are a number of other ones.

8 And the most important way to increase the

9 quality of those important reports is at the query at

10 the point of initial contact, and you will see that

11 the new proposed suspected adverse drug reaction

12 regulations also says that, and again I have to point

13 to my colleague, Min Chen, and I worked on those for

14 so many years, and I am sure glad to see them out

15 finally.

16 And I think that once they are out and the

17 comments are received, I would hope that the agency

18 moves as rapidly as possible to finalize those

19 regulations, and as soon as they are finalized, that

20 they revise the client's documents so that the

21 compliance officers focus more on the quality of the

22 cases rather than the timeliness of the cases.

23 I think that is going to be a very

24 important thing in turning the tide. So that is my

25 comments for question one, and again I want to focus

1 on this public health focus philosophy, because

2 without it there is a temptation among some companies

3 -- very few, but some -- not to acquire the needed

4 data, because they know that the FDA will disregard

5 the case as not being credible. And that is a bad

6 position to be in. Question Number 2, advantages and

7 disadvantages of data mining. I think data mining is

8 a value for secondary alerting, and what do I mean by

9 secondary alerting?

10 I don't think that any computer system or

11 any statistical technique of any kind can tell you

12 what a signal is. A signal is something that a person

13 must judge. But a computer system is useful in

14 telling you things that are disproportionate,

15 discordant, or whatever.

16 So they are useful, but they must be kept

17 in context, and I call them secondary because of

18 course primary alerting is just recognizing good

19 cases, and so on. So it has a role, but it is not a

20 primary role.

21 And it also has the ability, of course, to

22 identify previously unrecognized relationships amongst

23 the data that would not be intuitive by looking at an

24 individual case. And they are most valid when you do

25 the internal proportional analysis.

1 The third question, is advantages and

2 disadvantages of performing causality assessments. I

3 really think that it is useful? I think it is

4 important, especially when we have high volumes of

5 reports as we do in 2003, to sort the good quality

6 reports from the vast majority of reports that are

7 possible at best, and another large group of reports

8 that are just not -- just don't contain any

9 information value at all.

10 Because I believe that unless you have

11 some good quality reports that would lead to probable

12 criteria, you really don't have very much evidence, at

13 least from the individual cases. Maybe when you get

14 enough possible cases you can do a content analysis,

15 and a case series mode, and you can extract a little

16 more evidence.

17 But it is really the quality of the cases

18 that are important in the spontaneous reporting

19 schemes, and not the quantity. Remember, these are

20 qualitative data.

21 And then when you do a case series review,

22 you can add the possible cases back in, and as my

23 first myth is the myth of a definite case. I think

24 that is the unicorn of spontaneous reporting. There

25 is no such thing as a definite case in a single

1 spontaneous report, even with the positive dechallenge

2 and rechallenge, because the background disease is

3 very common, and it could still be an artifact.

4 Four, what circumstances would a registry

5 be useful. I think that we need to be careful with

6 registries. Just getting cohorts of people and

7 calling them a registry is not of much utility in and

8 of itself.

9 In order to interpret the data from a

10 cohort of individuals, you need to have a valid

11 comparative group, and registries I see being formed

12 do not have this.

13 Also, the safety data extracted from these

14 prospective cohorts need to be collected with the same

15 methods and the same data collection instruments that

16 are used in the clinical trial environment, and not

17 just spontaneous reports rising from this cohort.

18 It does not give you interpretable data.

19 And always remember that prospective cohort studies or

20 registries, which are the same thing, are not powered

21 to find new adverse reactions.

22 Many of you will recall that there was a

23 big effort if you have a history of this field in the

24 late '70s on prospective cohorts, and there was quite

25 a number of them assembled by industry. and actually

1 they were defused by a report by an FDA official, Alan

2 Rossi, that basically says that they weren't worth

3 anything because we didn't find any new reactions.

4 Remember that that is not what they are

5 there for. They are best in assessing risk factors

6 for known reactions so that you can subsequently

7 prevent those known reactions, and so it gets to the

8 root cause question of why these are occurring.

9 And they are also a good source as I

10 mentioned yesterday of quantitative data to aid in

11 defining the safety profile of a drug in those

12 assembled cohorts, but without comparator cohorts, you

13 can't interpret them.

14 So be careful in just assembling

15 registries to say that you have a registry, because it

16 is very misleading. What circumstances would active

17 surveillance strategies be useful in identifying yet

18 unreported adverse advents?

19 This is a tough one. Many of you also

20 will recall the active case control surveillance

21 methodology that Sloan and Shapiro had in effect in

22 the 1970s. That was usually a hospital-based nurse-

23 extracted dedicated drug safety data, mostly funded by

24 the FDA and the drug industry, that has kind of

25 disappeared.

1 I think it should be revisited now that we

2 have modern computers and a lot of other things, and I

3 think we can better address this issue. I think it is

4 a powerful methodology, and the agency should consider

5 seeding the reinstitution of this kind of thinking.

6 Also, I think the MedSun program in the

7 Center for Devices could be extended to the drugs area

8 to have representative samples of -- not use the

9 facilities, but clinical practice environment that

10 could report a more concentrated from of reporting.

11 Thirdly, we have a resource in this

12 country that is tapped, and that is our Poison Control

13 Centers and our Drug Information Centers, and our

14 university hospitals, and our regional systems.

15 And for those of you who don't know the

16 French system of pharmacovigilance, those regional

17 pharmacovigilance centers, are only secondarily

18 pharmacovigilance centers. Primarily, they are poison

19 control and drug information centers.

20 So we have this network in the U.S., and

21 we are not tapping into it. I see that as an

22 incredible resource for getting very high quality

23 reports from an institutional setting, and basically

24 you can leverage your FDA staff through contracts to

25 these groups, and I think that really needs to be

1 explored.

2 That would be particularly powerful for

3 drug-drug interactions, as well as medication errors,

4 and things that are going to be very difficult to get

5 anywhere else. So I think that those are three

6 strategies that need to be given some attention.

7 Now, pharmacoepidemiological studies, I

8 think they are useful only when they can be

9 replicated. I am often stunned by the fact that we

10 need replication in clinical trials to get a drug on

11 the market, but some observational research that they

12 can't be replicated can do great harm to a drug once

13 it is on the market.

14 I think that it needs to be replicated,

15 and I think they need to be validated, which is

16 another problem, and they need to provide as I said

17 yesterday quantitative data, so that you can measure

18 baseline risk, and you can measure change from that

19 baseline after an intervention.

20 If you can't do those three things, then

21 you shouldn't be doing pharmacoepidemiology. Now, let

22 me change after looking at those questions, and

23 address some things that may also be helpful and may

24 not. I have been involved in working with a number of

25 companies over the past couple of years and getting

1 them supported for advisory committees to the agency

2 and so on.

3 And in a recent advisory committee, I

4 heard this statement, and I will paraphrase. Today's

5 FDA error system isn't of value because due to under-

6 reporting, incidents cannot be reliably calculated.

7 And I heard that statement and no one objected to it

8 when it was stated, and I find it amazing that a

9 statement like that can go uncontested nowadays.

10 First of all, as a -- and again along with

11 Min, as a developer and creator of the FDA error

12 system, I take personal affront to people banging on

13 it like that.

14 What they are really talking about, of

15 course, is spontaneous reporting schemes and not the

16 system itself, which is the computer software. But

17 the whole concept of under-reporting, which I will

18 address in a minute, is absolutely a fascinating myth

19 that continues.

20 But before I get to that, let me talk

21 about the confounding myth. We all know that

22 spontaneous reports seldom contain all the history

23 data and concomitant conditions and drug data that we

24 would need for assessment.

25 But this absence may be due to one of two

1 factors; either a true absence -- they didn't occur --

2 or poor data acquisition. Conversely, the presence of

3 con factors does not automatically clear a drug from

4 causality.

5 I have seen in many cases where large

6 numbers of confounded cases in complex medical

7 conditions nonetheless were dismissed because they

8 were confounded, and I think that is a terrible error.

9 Because unconfounded cases, especially in

10 complex medical conditions, may most likely be due to

11 poor data acquisition, and so you are actually relying

12 on the worst data collection techniques in those

13 cases, and throwing out the better cases.

14 There is a lot of information to be gotten

15 from those confounded cases when you do content

16 assessment of case series, and we need to stop

17 diagnosing individual pieces of paper and look at them

18 more epidemiologically.

19 Again, you need to apply this public

20 health focus philosophy. In other words, a report

21 from a health professional is true unless otherwise

22 shown by good data collection techniques, and of

23 course the way that you look at case series is by

24 Popperian logic of relative likelihood of all

25 qualitative explanations, because you cannot prove any

1 one of these things. All you can do is rank audit the

2 likelihood of the different alternate explanations.

3 The second myth that I would like to

4 address is the reporting rate myth, and this is one

5 that always makes me crazy. The statistical

6 assumptions in Finney's 1971, The Class Paper, which

7 talks about the random distribution of non-causal

8 events in spontaneous data, is no longer valid in the

9 U.S. spontaneous reporting scheme.

10 It may still be valid in the U.K. data, or

11 the French data, because there are much more refined

12 and defined population. In the U.S., we have all the

13 non-health professional reporters, and we have non-

14 series reports, we have solicited reports. We have a

15 whole bunch of stuff.

16 So all the statistical assumptions in that

17 paper don't apply, and so therefore we don't know the

18 sampling scheme from which these reports come from,

19 and therefore, they are qualitative. And if they are

20 qualitative, they cannot be true, and they are not

21 enumerators.

22 And if you don't have a true enumerator,

23 there is no way to calculate any kind of rate. So I

24 think that these extrapolations that I hear from the

25 agency, as well as from the industry -- oh, we think

1 only one percent, or 10 percent of these are reported,

2 and therefore, we will extrapolate -- I think that is

3 doo-doo, and we have got to stay away from that, and

4 we have to stop torturing these data.

5 On the other hand, there is an exception.

6 Reporting rates can provide some contributory evidence

7 for regulatory decision making when the outcome

8 reaction is exceedingly rare, and there are a number

9 of good cases that show that that has been beneficial.

10 Otherwise, a calculation of reporting

11 rates, if it is to be done at all, can only serve to

12 confirm that the signal may actually be real, and that

13 we ought to do some more research, and that is all

14 that it can be interpreted as.

15 It is much more valid to look at the

16 internal proportional analysis across the

17 pharmacological class using the internal data as your

18 denominator than it is to use external data as a

19 denominator given the kinds of reports we have.

20 Now, my favorite one of all is the under-

21 reporting myth. Spontaneous reporting schemes were

22 put into place and were designed to signal new rare,

23 unusual, and serious adverse reactions, and there is a

24 very powerful tool for doing that.

25 They were never designed to collect all

1 adverse reactions. We all know that. That is not a

2 limitation or a flaw of spontaneous reporting systems

3 as it is a characteristic.

4 You know, you can't blame a car for it not

5 being able to fly. It just is not the kind of data

6 that should yield the kinds of things that we try to

7 make it yield. I actually think given the initial

8 intent of the spontaneous reporting scheme that we

9 have gross over-reporting in the U.S., and not under-

10 reporting.

11 We have much too many reports coming in

12 and a high percentage of those have no information

13 value at all, and I know that we have taken some steps

14 in the new regulations, the new proposed regulations,

15 to sort between those that are most likely to be

16 important, and those most likely not to be important,

17 and I think that is a good step forward.

18 The only true under-reporting in

19 spontaneous reports is the under-reporting of

20 important data in important reports. Otherwise, I

21 think we have to get off this thing of saying that

22 under-reporting is a problem with spontaneous data,

23 because it doesn't make any sense.

24 Lastly, I will just briefly talk about the

25 secondary data myth, because I did go into this a

1 little bit yesterday. We have to be very careful when

2 we are doing any kind of observational research using

3 medical record and billing data, ICD-9-CM codes do not

4 denote adverse reactions.

5 When you do a study in here, you are

6 actually telling the differences in disease or

7 diagnostic occurrence between two groups after drug

8 exposure in one hand, and without drug exposure in the

9 other.

10 And the differences may be due to the drug, but

11 it may now. And they are not adverse reactions. They

12 are subsequent diagnoses. Plus, we have in many of

13 these data bases gross misclassification often can't

14 be assessed. So these data are not of the caliber or

15 the kinds of data that we need for risk management,

16 and I urge the FDA to seed new data sources with their

17 contract vehicles, with the PDUFA money, let's move

18 beyond the current limitations.

19 And let's get to some dedicated drug data

20 sources, and again I encourage that the FDA seeds

21 their creation. We need to stop torturing existing

22 data sources to yield answers that they cannot

23 possibly provide, and actually that should be a

24 violation of the Geneva Convention. Thank you very

25 much.

1 DR. GALSON: Thank you very much, Dr.

2 Nelson. Our next speaker is Dr. Stephen Goldman.

3 DR. GOLDMAN: Good morning. Unlike Bob, I

4 am focusing on one topic in particular, and I enjoyed

5 Bob's presentation as usual. I am going to talk about

6 the quality of the data, garbage in, garbage out;

7 spontaneous reporting systems that handle the quality

8 of work that you get in.

9 And so that you know my background, I am

10 first and foremost a clinician, and it has been my

11 privilege to treat thousands of patients over the

12 years. I do feel having been in academic medicine, in

13 regulatory agencies, I have treated patients in every

14 possible health care system, having been in industry

15 as a consultant, and I think I have seen almost all

16 sides.

17 So I feel actually qualified to speak for

18 my fellow physicians in saying that I think I know

19 what it is like to be in an office trying to decide

20 what to do with patients, and trying to decide which

21 meds to use, and what to do about adverse events.

22 One thing that I did want to point out for

23 those of you who don't know, I am also the former

24 medical director of the MedWatch Program. So the

25 people of the United States paid me to get better

1 reports into the agency as one of my purviews.

2 I am sort of an amateur historian, and I

3 always like to start with a particularly apt quote,

4 and I tried to figure what historical figure could I

5 use to start this talk, and I came up with the obvious

6 one, Bob Marley.

7 And Bob Marley sang V. Ford's famous

8 words, "And in this bring future, you can't forget

9 your past." There are lessons as Bob pointed out, and

10 I will point out, and I think Jerry will also, that

11 there are things that we know already that we seem to

12 have either forgotten or the program seems distant in

13 our memory. We know a fair amount, and I am going to

14 run through some of those.

15 Actually, I am going to go back one. In

16 1969, the year that the New York Mets won the World

17 Series, there was a very nice study done at Koch-Weser

18 and colleagues, and they assessed the factors that

19 impacted physician reporting of ADRs at MGH, and they

20 were the certainty of the reaction, the mechanism of

21 the reaction, the morbidity associated with the

22 reaction, prolongation of hospitalization, and the

23 onset of reaction.

24 Don't these things seem familiar in terms

25 of what we look at. They also looked at quality.

1 Now, they also looked at quantity, which I will talk

2 about in a few minutes, but they noted that the

3 program that they had not only increased the quantity

4 but frankly more importantly the quality of the

5 physician ADR reporting since they initiated the

6 studies.

7 There was an ADR definition they applied

8 that was easily applied, and it was felt to be

9 meaningful operationally, and what is so important for

10 practicing doctors and other health care

11 professionals, it excluded what seemed to be trivial

12 side effects that did not seem to have true clinical

13 significance.

14 Secondly, feedback clearly showed that

15 those reports were being both carefully monitored and

16 assessed. Feedback is crucial and we will talk about

17 that further, and they also thought that emphasizing

18 the importance of ADRs, in terms of monitoring,

19 heightened the visibility of the program throughout

20 the institution.

21 All right. Now that is going on 34 years

22 ago. Let's fast forward to 1988. Did the Dodgers win

23 the World Series that year? I forgot. Okay. They

24 asked why physicians in Rhode Island were hesitant to

25 report suspected ADRs. Notice ADR, not ADE.

1 Thirty-eight percent didn't have the form.

2 At that point that was the famous 1639, and not the

3 MedWatch form. Approximately 30 percent were unsure

4 the drug caused the reaction. And 24 percent said the

5 reaction was expected, and one of my favorites, 21

6 percent didn't know how to report.

7 And in 21 percent, it never occurred to

8 them, and close to three-fifth’s were unfamiliar with

9 the FDA forms an guidelines for ADR reporting.

10 Following hot on that was what I thought and still

11 think was the famous Rhode Island ADR reporting

12 project, in which an intensive program was put into

13 place designed to increase physician reporting of

14 suspected ADRs via sustained education utilizing

15 several formats.

16 They found after two years a greater than

17 17-fold increase in direct reports to the FDA by Rhode

18 Island docs, while at the same time there was no

19 increase in the overall reporting rate in the United

20 States. This predates MedWatch by the way.

21 Now, the important part of that, which I

22 always include now in a slide, is the trend on serious

23 events, and to reiterate what Bob was saying, these

24 systems are designed to pick up where unknown,

25 generally idiosyncratic events. Not the common

1 events.

2 And you can see that they went from before

3 the program, 0.4 percent of total reports to the FDA

4 from Rhode Island; and after the program, 3.6 percent.

5 That is a remarkable increase and of that, 31 more

6 armed reports on unlabeled reactions.

7 I would consider this program a success.

8 In addition to that, they did pre-imposed intervention

9 surveys, finding significant gains in both knowledge

10 and attitude towards the ADR reporting system.

11 That is, before the intervention,

12 approximately half were familiar with the ADR

13 reporting system, and afterwards, 85 percent were

14 familiar, and similarly, approximately 40 percent were

15 familiar with the FDA forms and guidelines, and 70

16 percent, or 69 to 70 percent after the intervention.

17 I think that this is important information that I am

18 bringing back to memory.

19 MedWatch. MedWatch came into existence in

20 June of 1993. To give you an idea of a temporal

21 association versus causal association, I was at the

22 FDA in that time, the temporal association, but I had

23 nothing to do with the founding of MedWatch. So no

24 causal association. I came later.

25 MedWatch was launched in June of 1993, and

1 Tony Piazza and (inaudible) Kenney looked at both

2 trends and reporting of serious adverse events from

3 '92 to '94, and the quality of the reports before and

4 after launching MedWatch.

5 What they found was that a proportion of

6 serious adverse event reports went from 34 percent in

7 '92 to 49 percent in 1994. By the way that figure

8 went up. When I left MedWatch in 1999, we had

9 approximately two-thirds of all drug reports to the

10 FDA through MedWatch. That was a voluntary reporting

11 system, and two-thirds were on serious events.

12 And that is what the program was designed

13 to do. MedWatch was never designed to increase the

14 total number of reports to the agency. It was

15 designed to increase the proportion of serious reports

16 to the agency.

17 Secondly, they found the overall quality

18 of the reports went up between '94 and '93, and they

19 determined quality as a significantly greater

20 proportion, say whether the drug was a newer entity,

21 and they talked about the seriousness of the event,

22 and they also supplied the crucial data that Min went

23 through, lab data, clinical data, supporting your

24 diagnosis.

25 Pharmacist reports increased both in

1 quality and in number, and physician reports, although

2 of high quality before and after, the numbers did go

3 down. The highest percentage report to the MedWatch

4 program were pharmacists and not physicians, and I

5 believe that still holds in terms of that.

6 That should not be a surprise, because

7 that doesn't mean that the pharmacist was the first

8 one always to pick up the adverse event, but they are

9 often the first one to report it, particularly through

10 P&T committees and other systems they have.

11 All right. This is from Spain, and I

12 thought that this was a nice study that I found when I

13 did my literature review for this presentation. This

14 is a case control study that looked at docs who had

15 reported ADEs and those who hadn't.

16 And they found that the probability of

17 reporting an adverse drug event increased with

18 increasing prescription volume, and what a shock, it

19 decreased with increasing patient load. Not a

20 surprise. But look at the things that they found

21 would lower the likelihood that you would report.

22 The belief that truly serious ADEs were

23 well known by the time of marketing. Well, we all

24 know how true that is. The belief that the

25 determination of whether the drug was responsible was

1 nearly impossible.

2 Reporting only when they were sure of the

3 reaction was to the product in question, and the

4 belief that an isolated case seen by one doc couldn't

5 make any contribution to medical knowledge. I will

6 talk about those myths as I follow up and summarize.

7 In Germany, in 2002, they took a look very

8 similarly at docs who reported and docs who hadn't.

9 Anywhere between three-quarters and close to 90

10 percent never submitted a report tied to the

11 government or professional programs, and yet the

12 reporting was much better from pharmaceutical

13 companies.

14 A whopping, almost two-thirds, or more

15 than two-thirds, suspected an ADR, but did not report

16 it. The major reasons, once again, is that they

17 thought it was a well know ADR; they thought it was a

18 trivial ADR, and they were uncertain of causality.

19 In addition to that, they found that the

20 highest probability of reporting an ADR were for

21 serious unknown ADRs for new entities, or an older

22 drug, or a serious known ADR to a new drug. Along

23 with that, however, 20 percent acknowledged no

24 awareness of the spontaneous reporting system in

25 Germany, and 30 percent didn't know how to report, and

1 54 percent would report if offered therapeutic advice.

2 There are common themes in these studies.

3 I didn't have a complete list at the time, and I did a

4 literature search like I said. I looked at several

5 countries, and I don't know if this is good news or

6 bad news. The attitudes tend to be the same.

7 The complaints tend to be the same, and

8 the reason why docs don't report tend to be the same

9 in Europe and in the United States. There aren't much

10 differences there. So what do we do about it?

11 Well, once again I went back to history.

12 Abraham Lincoln said we must use what tools we have.

13 This was an assessment of the intervention used to

14 stimulate ADE reporting. This is from the Mississippi

15 adverse drug reaction, the ADR program.

16 They noted that docs, particularly health

17 professional docs, didn't like the fact that they got

18 no feedback. They send in a report, and they don't

19 find out what happened to that report.

20 So what happened? They put together a

21 newsletter that summarized both the number and types

22 of the reports, and the drugs involved, and they also

23 felt that this enabled docs to get educated about new

24 drugs, warnings, and careful observation. There was a

25 very good response to that.

1 Secondly, they felt that maintenance of

2 awareness about ADR surveillance, and a concomitant

3 effect on quality of care was important, and they

4 found that posters strategically placed in the

5 hospital, and ongoing in-services in hospitals were

6 effective. This is the Mississippi program.

7 I might add that this preceded MedWatch.

8 One of the things that we did at MedWatch was make the

9 attempt to tell people what we were doing with these

10 reports, which led to the first conference that we

11 ran.

12 This was a conference that we ran at FDA,

13 myself and Ron Leiberman, actually working with Bob

14 through the staff college when Bob was directing that.

15 We put together what we think at the time and still

16 think was the first conference designed specifically

17 to talk about the recognition and management of drug

18 induced disease.

19 We used things that we had heard were

20 effective. We used multiple formats; straight

21 lectures, panels, small group discussions. We used an

22 underlying clin-pharm approach, and we found that

23 knowledge was improved by pre-imposed testing. And in

24 addition it was very well received.

25 That led to the first MedWatch continuing

1 education article, the "Clinical Therapeutic

2 Recognition of Drug Induced Disease." It was part of

3 that conference, and we distributed nationwide through

4 the MedWatch partners, which is over 150 health

5 professional associations who signed on with MedWatch

6 to get information.

7 And the carrot that we used was free

8 continuing education credit; two hours of contact for

9 pharmacists, and two hours of CME for docs. The

10 results were very positive.

11 We got a 2.2 response rate. My

12 understanding from the marketing professionals that I

13 discussed this with when I published this was that

14 that is a very good response rate for a blind mail

15 out.

16 We got over 15,000 docs and pharmacists

17 asking for the CE credit. You will also notice that

18 the majority were docs, in terms of that. Over 99

19 percent agreed that it met their objectives and was

20 relevant. Obviously a halo effect.

21 If you didn't feel that it was relevant,

22 you would have sent it in the first place, but they

23 did

24 say that. What we didn't expect was spontaneous

25 comments. People wrote in the margins of the answer

1 sheet. I got e-mails. I got calls.

2 This was so successful that led to the

3 other MedWatch articles, such as the next one, The

4 Clinical Impact of Adverse Event Reporting, which I

5 wrote from scratch rather than conference, and others

6 that were done on vaccines, devices, and they are all

7 still up on the MedWatch website.

8 The research program basically took that

9 idea. This is up on the FDA website, and this is a

10 new learning module that was developed through what

11 was the Georgetown CERT, and now the CERT at Arizona.

12 And as you can see, it is about

13 preventable ADRs, and it was based on a need survey

14 that was sent to all third-year medicine clerkships,

15 and residency program directors.

16 It was sponsored by AHRQ. I have no data

17 from this yet, but we are looking forward to seeing

18 how effective this is. Other successful

19 interventions. Very briefly just as an example of a

20 multidisciplinary ADR committee, they have a form that

21 they developed, a 24-hour hotline, an ADR newsletter,

22 broad range in-service.

23 Pharmacists and investigators sit on the

24 committee, and increased numbers of reports.

25 Switzerland, a clinical pharmacist making rounds,

1 soliciting information from docs and nurses, doing

2 chart review, and what a surprise; improved ADE, ADE

3 identification and reporting.

4 Two other papers looking at the morning

5 report as a way of facilitating detection of AEs, and

6 as a form for reporting AEs. Both of them were very

7 effective, and the references are there.

8 We talked about form completeness. I am a

9 tremendous believer in directed questioning and a

10 check-off list. The MedWatch instructions for both

11 the voluntary and the mandatory go item by item. I

12 think they are a good guide frankly, in terms of

13 utilizing it.

14 The idea of adverse event-adverse reaction

15 specific questions is crucial. If you know that you

16 have a problem in the liver, or a perception that you

17 do, if you put together a list of questions specific

18 to liver, and those are the questions that should be

19 asked when somebody calls.

20 Homicide detectives will tell you the

21 first interview is crucial. It is no different when

22 someone calls in to give you an adverse event report.

23 You get one shot for the most part, and that first

24 shot is the best shot in terms of that, because people

25 don't want you to call them 2, 3, and 4 times, and it

1 becomes diminishing returns.

2 So I really advocate for this, and I

3 mention two examples, both liver functions and the

4 dermitities. These are the kinds of data that we

5 talked about getting, and I will put in a particular

6 plug. When you ask somebody what medicines they are

7 taking, ask them everything that they put in their

8 mouths or on their skin during the day.

9 People don't think OTC drugs or dietary

10 supplements are drugs. It is like if you ask someone

11 if they drink alcohol, and they say no. How many

12 beers did you have? Seven.

13 People sometimes don't perceive that as

14 being alcohol, and the same thought with taking

15 histories. These are the things that you have to ask.

16 Temporal information, biopsy, or god forbid, autopsy

17 results, and dechallenge, and rechallenge.

18 This last slide before I go to my summary

19 is to point out that when we look at medication

20 errors, take a look at the relative percentages of

21 what causes them.

22 Eighteen percent were errors, correctable

23 errors, in calculations, decimal point placement,

24 unirate expression, but 60 percent were deficiencies

25 in knowledge. That is not remedial, per se, by

1 changing a package label. That is a knowledge deficit,

2 and that is a harder issue.

3 Lessons learned. I fully believe that

4 when we educate health professionals that it should

5 not be product specific. It must be clinically

6 oriented. The goal should be to increase the

7 awareness of medical product induced disease.

8 The only way a doc will diagnose an

9 adverse event is to consider it in a differential

10 diagnosis. You must enhance knowledge of the

11 application of pharmacotherapy, and the impact of the

12 individual factors that patients have on

13 pharmacotherapy.

14 Thirdly, you just explain clearly not only

15 how, but why, busy health professionals should be

16 reporting adverse events to the regulatory agency or

17 both. Feedback is crucial, and I believe the more

18 clinical the feedback should be the better.

19 Education, education, education, all

20 levels, in professional schools, training programs,

21 post-graduate continuing education, and in the

22 clinical care settings; hospitals, clinics, and other

23 settings.

24 That includes all health professional

25 disciplines, and it must be ongoing. One shot

1 programs will not work. The ADR program in Rhode

2 Island was not a one shot. It was a two-year

3 sustained program, and look at the results.

4 The programs that I mentioned about having

5 a pharmacist on rounds. These were not one-shot

6 deals. They were ongoing. You have to demystify the

7 process of adverse event reporting and assessment and

8 what goes on in the FDA and industry, and I am going

9 to do the same thing that Bob did, exploiting some

10 myths.

11 Obviously all serious adverse event

12 reactions are not known at the time of marketing.

13 That's why we have post-marketing surveillance.

14 Causality not being a requirement for reporting. This

15 is crucial. How many of those studies that I just

16 showed, people felt that if they couldn't say it was

17 due to the product absolutely, they wouldn't report

18 it.

19 Wrong. That's completely wrong, and we

20 tried so hard with the MedWatch Program to make it

21 clear that you didn't have to know that. It was not a

22 requirement, and it even said that on the form.

23 A single case can lead to general

24 knowledge. Somebody somewhere has to be the first

25 health professional or consumer to report that

1 particular adverse event. So someone somewhere was

2 the first one.

3 And, fourthly, even if a known adverse

4 event is known, accumulating good cases on serious

5 known adverse events can definitely increase general

6 knowledge, can look at possibly patients at greater

7 risk, and certainly specificity and severity does make

8 a labeled event unlabeled in terms of that.

9 We need ongoing evaluation of methods to

10 solicit information. I mentioned directed

11 questioning, and specialized questions for known

12 adverse events in AERs of interest. The brand new

13 proposed rule category of always expedited reports,

14 the known bad actors, the reason that we have post-

15 marketing surveillance.

16 The bad news is that we have no quick fix.

17 There must be a commitment of resources, and that

18 means financial commitment of resources, personnel,

19 involvement of multiple sectors and partnerships.

20 This is not just the FDA's problem. This

21 is not just industry's problem. What we wrote in the

22 1999 task force to the commissioner made it clear that

23 this is s a shared responsibility from all sectors,

24 including health professional associations. And that

25 includes doctor health professional associations.

1 I will leave my last slide with a

2 historical statement. I am always concerned about

3 people making the correct attribution between the

4 methods used and the results. We have talked about

5 this a lot the last three days.

6 I have what I believe is one of the

7 greatest single line correct attributions in history.

8 When George Pickett was asked why his attack on

9 Cemetery Ridge failed at Gettysburg on Day 3, what he

10 answered was, "I think the Union Army had something to

11 do with it." I will stop here. Thank you.

13 Goldman. Our next speaker is Dr. Gerald Faich.

14 DR. FAICH: Good morning. I am Gerry

15 Faich. It is a pleasure to be here. While I am here

16 as yet another consultant, I also will be drawing on

17 my background as FDA's office director. I was at the

18 agency when the 1985 NDA rewrite was put in place.

19 That indeed was or did form the structure

20 for adverse drug reaction reporting as we now know it,

21 in terms of defining serious, and 15 day, and the

22 like. I am also here as a member of the board of

23 directors of a safety search company, and so I need to

24 say that, and that is Centrix, by way of full-

25 disclosure.

1 My opinions, however, are my own. I found

2 the concept paper as outlined this morning to be

3 thoughtful and useful. I think it attempts to

4 carefully balance concepts involved in spontaneous

5 report collection, as well as analytic epidemiology

6 studies, which we are going to hear about this

7 afternoon.

8 So I salute those who wrote the concept

9 paper. I, too, will stick to the questions that were

10 put before us, and I would like to comment on

11 spontaneous report quality, data mining, causality

12 assessments, registries, and then I would like to

13 comment on large simple as an aside.

14 Even though that was mentioned two days

15 ago, I think that is pertinent, and so you will notice

16 that I am not going to comment in two areas. Let's

17 talk about quality of case reports for a moment. We

18 have over many years now concentrated on coding

19 issues, standardization issues, and report submission

20 issues.

21 I strongly believe that the key in the

22 goal of going forward ought to be to improve the

23 content. So if that is true, the question is how does

24 one improve content, and I would submit that the way

25 to do it, and Steve Goldman just mentioned this, is to

1 get as much information on the first encounter with

2 the reporter in a structured manner as possible.

3 If that is true, the question is how does

4 one do that. Some of the guidelines talk about using

5 a physician for primary intake, and I think that is

6 not a feasible approach. I think that it is not cost

7 effective.

8 However, I do strongly believe that a

9 trained health care individual should be, if not the

10 first answer of the phone call, the first referral

11 point for in taking information.

12 The other thing that I would submit, and

13 we have learned this from experience, is that the

14 intake point should utilize computer assisted

15 technology with built-in logic.

16 As Steve Goldman just said, if you know

17 the problem is in the chest, then you ought to have an

18 algorithm which branches the questioning to get you to

19 lungs, and from there, to shortness of breath, in a

20 structured way that guides both the questioner, but

21 also the intake of the data.

22 It is perfectly clear that you will

23 improve the quality of the data, the quantity of the

24 data, and you will contribute mightily to

25 standardization of the data, and the database, even in

1 the intake process.

2 Moreover, if that system is automated, it

3 is clear as well that at the end of the interview you

4 will know which data elements are missing, and you can

5 make an arrangement or a contract with the reporter at

6 that point in time for call back on those specific

7 items.

8 I again a hundred percent agree -- and

9 anybody who has been in this business knows, that if

10 you called back the reporter 2, 3, and 4 times, by the

11 fourth time, forget it, that you are not going to get

12 further data.

13 And by the way, as an aside, the new

14 safety tone which came out on March 14th, suggests

15 that sponsors will have to document their due

16 diligence for the multiple phone calls, and submit

17 that due diligence to the agency.

18 I think that is wrong-minded frankly. I

19 think that is a regulatory burden that does not have a

20 payoff. I do think, however, that establishing right

21 at the outset with a reporter what data elements are

22 missing, and how those will be obtained, and when the

23 call back will happen, is dependent upon using

24 automated telephone-based systems in large part.

25 What we normally do when we intake case

1 reports is that there is a reception point. We log it

2 in, and we write it up. We code it. We enter it. We

3 validate it. We review it.

4 Then somebody finally gets around to

5 submitting a form, and what you hear me saying here is

6 a plea for using more direct electronic immediate data

7 entry. Again, I would emphasize that I am not talking

8 about mere wordprocessing.

9 We are talking about a built-in logic in

10 the script that an informed health care recipient will

11 be receiving and intaking information. It is sorely

12 lacking. This technology does exist. It is being

13 used increasingly, and I strongly believe that this is

14 the way, the single most important way, to improve

15 adverse drug event report quality.

16 In January of this year, I and Hugh

17 Tilson, and Ralph Edwards, and others, participated in

18 an International Society of Pharmacovigilance meeting

19 in Paris.

20 We, among other things, set out some 10 or

21 12 points of interest that we are going to be

22 discussing at the International Society of

23 Pharmacoepidemiology meeting this August.

24 But I cite it here only to tell you that

25 one of the points was relative to signal detection, we

1 noted that despite report volume issues that the

2 reporting rate again must be seen as not a true

3 measure of the risk rate.

4 The reporting rate is just that. It is

5 the number of reports, spontaneous reports, over some

6 denominator. It is a signaling device that needs to

7 be used with caution.

8 Similarly, you will see here at the bottom

9 that we noted that when observational studies,

10 epidemiologic studies are done to investigate a report

11 and a signal, those should be seen when well done is

12 largely trumping signals that arise from spontaneous

13 reports.

14 So I would submit that in terms of both

15 report quality and in report interpretation, once a

16 signal occurs, if it is important, there ought to be

17 great urgency in getting a pharmacoepidemiologic study

18 mounted.

19 And when it comes in, in general, it

20 should be seen as outweighing the signal value of

21 spontaneous reports. Let me turn to the second area,

22 and that is data mining. The FDA database in

23 particular has evolved over time as you well know.

24 That database has millions of reports in

25 it, and is now receiving 250,000 reports a year, as

1 opposed to the 5,000 when we started in 1983, and the

2 50,000 in 1990 when I left the agency.

3 Because of that the database overall has

4 marked variation in terms of the quality of reports,

5 the requirements that were present at the time,

6 waivers have affected it, and the like.

7 This is simply a way of saying that there

8 are numerous non-biologic influences in the database,

9 and that is bound to perturbate any data mining

10 effort. You can make efforts to control for that, but

11 my biggest concern about data mining is the generation

12 of false signals, which inevitably will happen.

13 And I don't think that is necessarily so

14 bad. I think that we can interpret false signals as

15 long as we know that they are there. The question is

16 what is the overall value.

17 Two quick examples of false signals that

18 come out of data mining. One of them was a letter

19 published by Ralph Edwards, in fact, in Lancet about

20 the association of Claritin, a non-arrhythmogenic

21 antihistime, and arrhythmias, and in fact it was based

22 largely on U.S. ADR reports that flowed through the

23 air system.

24 And in fact FDA reviewers wrote a letter

25 to Lancet saying totally wrong. What we are looking

1 at are people who had pre-existent arrhythmias, and

2 that this is a clear example of channeling bias.

3 Those that were at the highest risk of having the

4 outcome were assigned the safest disease, will still

5 manifest some of them, that outcome, and that is

6 obviously a non-causal selection bias.

7 We are bound to see that in data mining,

8 and of course one of the other issues is that you

9 can't know that if just use the data that are in the

10 electronic database. You may well need to have much

11 more data that would be available in the appendices

12 and the like.

13 So I would refer you to that glaring

14 example of a false signal achieved by data mining.

15 Another one that one will find is if you look at

16 prozac and suicide, you are going to find out that

17 prozac and suicide has the highest rate certainly in

18 the neuropharmacologic agents, clearly confounded by

19 indication.

20 It should surprise no one that the press

21 people commit suicide and that there has been a large

22 number of reports that came in after publicity about

23 this. Also, not easily controlled in a data mining

24 exercise. One way to go at this is that I would

25 recommend that data mining be restricted to within a

1 therapeutic class. I think that comparing antibiotics

2 or cardiovascular agents to neuropharmacologic agents

3 makes no sense at all to me.

4 And I certainly since Steve was quoting

5 Abraham Lincoln and the like, I will quote Yogi Berra,

6 who has of course said that if I hadn't believed it, I

7 wouldn't have seen it. And I think that is a clear

8 thing that goes on in data mining.

9 Let's move on and talk about causality

10 assessment quickly. We all know that there is no

11 standardized way to do causality assessment. Navarro,

12 or algorthims, or others, doesn't do it. Koch-Weser

13 recognized this in 1968.

14 In fact, what hasn't been said is that Von

15 Koch-Weser showed that if you gave a panel of five

16 physicians 30 reports, you would get virtually no

17 agreement between the physicians as to whether it was

18 possible or probable.

19 But more interestingly if you shuffled the

20 reports, and gave them back to the same reporter, you

21 wouldn't get any agreement the second time aground.

22 So you have inter-observer and intra-observer

23 variation.

24 And one has to say wait in terms of the

25 value of doing what is a time and energy consumptive

1 process. When I got to the FDA in 1983, causality

2 assessment, report by report as reports came in, was

3 the routine. One of the first things that we did to

4 improve the process was to stop that. And we stopped

5 it because it was backlogging the system, and it was

6 not all clear what the value of that was. So I am

7 going to suggest that we should not do routine

8 causality assessment, with two notable exceptions.

9 One is in the conduct of clinical trials

10 for serious adverse events not in the investigator

11 brochure, and the other one is that once a signal

12 surfaces, however it surfaces, then I do believe that

13 you have to lay out the reports and do this arduous

14 process.

15 But I would hate to see us retrogress to

16 the point where we expended energy and resources on

17 the front end, when it is not clear or useful at all.

18 I am going to skip this, because I think that I just

19 said it.

20 But I would point out again the new safety

21 tone calls for submission of serious reports from

22 trials that cannot be where causality cannot be ruled

23 out. Those reports are to be submitted in an

24 expedited fashion.

25 I suspect that that is wrongheaded, and by

1 the way, the change is that the current regulations

2 say that if you have a serious AE not in the

3 investigator's brochure, that is reasonably and

4 possibly due to the drug. Notice the language. That

5 is the standard, and then it should be submitted.

6 This one says that if you can't be sure

7 that it might not be associated with a drug, then you

8 do have to submit it. I think that is going to clog

9 the system, and I think that we are going to lose our

10 ability to discriminate between those reports, and I

11 would submit that we should revisit that issue.

12 DR. GALSON: Dr. Faich, you have got one

13 minute.

14 DR. FAICH: Thank you. I will speak

15 quickly. In terms of registries, follow large numbers

16 of patients. I have three problems with registries,

17 both on the demand side from FDA, and also on the

18 feasibility side.

19 We are asking registries to be too large

20 and to last too long. They can't do it. You can't

21 follow patients much beyond 5 years, except for using

22 national death index, and I would urge caution in

23 doing that.

24 I also think that data collection needs to

25 be highly focused, and by the way, I would say that we

1 ought to think more and more about directed patient

2 interviewing in these registries.

3 Large simple safety trials. I just put

4 this out here because I think that is going to be the

5 fundamental tool. If we depend on classical

6 pharmacoepidemiology, we are going to have to wait two

7 years or more until there is enough population

8 penetration.

9 And instead of doing that, we ought to do

10 registries in large simple trials if we need to know

11 answers quickly. There is technology, again, that

12 facilitates both registries and large simple trials.

13 This is taken from a smoking cessation study, and it

14 just is using telephone interviewing to accrue the

15 data on the front end, and to output the data on the

16 back end, something that needs to be given higher

17 priority.

18 If I can get past this, I will summarize

19 my talk under a minute hopefully. No, I won't. Well,

20 let me summarize then. I have talked about the need

21 to impact improved quality by improving intake of

22 spontaneous reports.

23 I question the value of data mining as you

24 have heard me say, and I think that registries do have

25 a role. I think that we need to be careful about

1 their sizing and which data gets collected, and I

2 think that we should harness new technologies to get

3 our data faster and in ways that make more sense to

4 us. Thank you.

5 DR. GALSON: Thank you very much. We are

6 now going to take a break until 9:45, when we will

7 have Dr. McClellan. Thanks a lot.

8 (Whereupon, at 9:31 a.m., the workshop was

9 recessed and resumed at 9:46 a.m.)

10 DR. GALSON: Okay. Thanks, and if folks

11 could come in and get seated. It is my great pleasure

12 to introduce now our Commissioner of Food and Drugs,

13 Dr. Mark McClellan. I am not going to go through his

14 whole bio, because I think that his background is very

15 well known to most of you.

16 But just a couple of notes. From 1998

17 through 1999, Dr. McClellan was the Deputy Assistant

18 Secretary of Treasury for Economic Policy, putting

19 him as part of a small club of people who have been

20 political appointees in both the Clinton and the

21 current administration.

22 During 2001 and 2002, Dr. McClellan served

23 in the White House. He was a member of the

24 President's Council of Economic Advisors, and he

25 advised on domestic economic issues. He also served

1 during this time as a senior policy director for

2 health care and related economic issues.

3 Dr. McClellan's research interests and

4 studies have addressed measuring and improving the

5 quality of health care, the economic and policy

6 factors influencing medical treatment decisions and

7 health outcomes, estimating the effects of medical

8 treatments, technological change in health care, and

9 its consequences for health.

10 And medical expenditures and the

11 relationship between health and economic well-being.

12 During the few months that I have had the privilege of

13 working with Dr. McClellan. I am, and the other senior

14 managers at the agency have been incredibly impressed

15 with his energy, his interests in detail, and the

16 complexity and challenges of the issues that we are

17 facing at the agency.

18 And as well, his clear focus on the public

19 health parts of his task. So now I am really happy to

20 introduce Dr. Mark McClellan.

21 COMMISSIONER MCCLELLAN: Thanks, Steve,

22 for that introduction. That actually made my day. It

23 is very much appreciated. I am very glad to be here

24 with you all today. This is a great conference room

25 and I hope that you will appreciate what I have to

1 say.

2 I am a little nervous with these drug

3 experts behind me, and so if they start making faces,

4 or groaning, or something like that, I hope that you

5 all will let me know.

6 But again I am very glad to be here. This

7 is an important meeting that is going to be of great

8 value in helping us do our job more effectively, and

9 that is very important today with the tremendous and

10 growing responsibilities that this agency has, the

11 growing volume and complexity of our responsibilities,

12 both for medical products, and for food, and even

13 national security issues, with limited resources.

14 We have got to prioritize, and we have got

15 to focus on using the most effective methods possible

16 for carrying out our mission, and so that means that

17 we have to think critically and carefully about how we

18 can do the most good for the public health with the

19 resources and authority that we have.

20 In some cases this is requiring a

21 reexamination and an updating of the way that the FDA

22 is doing its job. To bring all of this together, the

23 agency has undertaken a major strategic action plan

24 that is moving forward right now within FDA, and we

25 are going to have much more to say about soon.

1 One of the central themes in our strategic

2 planning is the concept of efficient risk management,

3 and I know that risk management is the topic for much

4 of what you all are discussing here, but I want to

5 emphasize that efficient risk management goes way

6 beyond the risk management programs for certain

7 pharmaceuticals that are the immediate topic of this

8 in related meetings.

9 The broad principles, however, are very

10 much applicable to your efforts here, and that is one

11 reason we are so interested in hearing from you on

12 this topic.

13 What we mean by efficient risk management

14 is first, using the best scientific evidence to assess

15 and manage the risks facing the public. This includes

16 both the safety risks of the medical products that we

17 regulate and the risks from the diseases that they are

18 intended to treat.

19 This is one of the many areas of

20 regulatory science where the FDA has long led the way

21 in assessing risks to the public health and

22 identifying ways to manage these risks.

23 Second, now more than ever we need to do

24 this efficiently and in the least costly way from the

25 standpoint of the agency, and especially from the

1 standpoint of society.

2 Lower costs of risk management means lower

3 cost of medical products, and that translates into a

4 greater input to do on the public health. Efficient

5 risk management in all of our regulatory activities is

6 more important than ever given the expanding

7 complexity of our challenges, and the need to reduce

8 the health risk facing the public at the lowest

9 possible cost.

10 As drug costs continue to rise, and as

11 health care affordability is a front page issue right

12 up there with the war and national security concerns,

13 steps to reduce the cost of drug development and to

14 increase the value of prescription drugs in use are

15 more critical than ever.

16 A major new technology development

17 initiative has been included as part of our risk

18 efficient risk management program. This has focused

19 on the pre-market side of drug development, and many

20 of you I am sure are familiar with it already.

21 It includes an analysis of the so-called

22 multiple cycle product approvals that have occurred in

23 the agency in recent years to identify whether at

24 least in some cases those multiple cycles might be

25 avoided, making it possible to make a safe and

1 effective new drug treatment available more quickly

2 and at a lower cost.

3 Second, the implementation of quality

4 systems in our review procedures to make sure that we

5 are using the best possible review practices and

6 rating ourselves on how we are doing.

7 And third, new guidance that we intend to

8 develop collaboratively with outside experts in key

9 areas of disease treatment and emerging technologies.

10 Another part of our efficient risk management

11 initiatives include an overhaul of the pharmaceutical

12 good manufacturing practices based on the latest

13 science of risk management and quality assurance in

14 order to encourage innovation in the production, and

15 manufacturing, and distribution of pharmaceuticals.

16 From what we have seen and in talking to

17 outside experts, there are many opportunities for

18 producing more efficiently, with fewer errors, with

19 fewer defects in production, and that, too, can

20 translate into lower costs and perhaps even safer

21 medical treatments.

22 Finding ways to help ensure that drugs we

23 review are used safely once they are approved, and to

24 reduce preventable adverse events through better

25 information, better systems for monitoring, assessing

1 and reducing adverse events, is also part of this

2 broad initiative. And that is why we have asked you

3 to participate in this meeting, to help the agency

4 consider new concept papers that stem from risk

5 management activities growing out of last year's

6 important legislation reauthorizing the Prescription

7 Drug User Fee Act.

8 In this area of mitigating adverse events

9 the agency is taking many steps to reduce the too

10 common occurrence of adverse events involving the

11 products that we regulate.

12 Our new regulation requiring bar codes on

13 pharmaceuticals will be important in reducing adverse

14 events due to dispensing and administering

15 pharmaceuticals improperly. It will help make sure

16 that the right patient gets the right drug and the

17 right dose, and in the right form at the right time.

18 We are also working on developing

19 information technology tools to identify adverse

20 events automatically in a timely way, and to provide a

21 two-way street so that we can provide feedback quickly

22 and effectively on ways to mitigate the risks that we

23 identify.

24 We have entered into partnerships with

25 hospitals, such as Columbia Presbyterian in New York,

1 through the Marconi Project, and in our devices

2 center, we are expanding our so-called MedSun

3 initiative, which relies on the web for two-way

4 communication to identify and respond to adverse event

5 problems with devices, and hopefully that can be

6 expanded to drugs and other treatment soon as well.

7 But adverse events that can be prevented

8 aren't just caused by medical errors. Some are caused

9 by rare problems, such as liver toxicities, or adverse

10 interactions in certain groups of patients that may

11 only become apparent after a product has been on the

12 market in use with real world patients for some time,

13 rather than in the somewhat idealized conditions of a

14 clinical trial.

15 Finding better ways to monitor the safety

16 of new drugs could improve the drug approval process,

17 as well as give patients and doctors greater

18 confidence that the drugs that they use will work as

19 expected. And if they don't, will give them greater

20 confidence that the problems will be identified

21 quickly and addressed quickly and effectively.

22 We have an unprecedented opportunity to

23 implement these mechanisms now, thanks to the

24 important new post-market authorities in the PDUFA 3

25 legislation, and thanks to the agency's and the

1 department's strategic focus on improving patient

2 safety and improving the use of modern health

3 information systems.

4 We are looking for better information in

5 this process, and not more information. We are

6 looking for more useful information for guiding

7 decisions about the benefits and risks throughout the

8 process of developing and using new technologies, not

9 burdensome additions to the requirements for approving

10 new products, and monitoring their use after approval.

11 But this kind of creative thinking isn't

12 easy, and we need your help. That's why workshops

13 like this are absolutely critical, and as we consider

14 use of the drug development process, we need

15 constructive suggestions from all sources.

16 This is the best way to reduce the time

17 and the cost of the complex process of developing

18 medical technology to fulfill our shared goal of

19 giving patients more value in the medications that

20 they use.

21 We hope that industry will work with us in

22 this effort, and I am very encouraged by the way this

23 discussion has progressed so far in today's and

24 yesterday's workshops, and we hope that the product

25 developers will work closely with us to plan and

1 implement these risk management programs.

2 We also hope that the result, better Phase

3 IV studies, can potentially reduce the amount of

4 information we need to collect during the preapproval

5 process, and I believe that this will be possible.

6 But to achieve this goal, we need to

7 identify specific opportunities where it is possible.

8 We can also use guidance on the broader framework from

9 improved Phase IV activities. That is, on how we can

10 through our approach to pre-approval and post-approval

11 studies, achieve the greatest reductions in the health

12 risks facing the public.

13 This risk management includes both risks

14 of poor outcomes from adverse events caused by

15 treatments, and risks of poor outcomes that can be

16 prevented by the use of new treatments.

17 In addition, I believe, and I think that

18 many from yesterday's discussion, many of whom were

19 here for yesterday's discussion, also agree that the

20 FDA can encourage safer and more effective, and high

21 value use of prescription drugs by helping patients

22 and health professionals get better information.

23 That includes better package inserts.

24 They need to be more effective benefit risk

25 communication tools, and we think that there is room

1 for improvement. For example, package inserts may not

2 be designed so that health professionals can get the

3 information they need, the most up to date information

4 they need to treat a particular patient.

5 And many package inserts today have become

6 so laden with legal considerations that they are often

7 hard for the average consumer to understand, and they

8 are often ignored by many physicians.

9 Finally, we aren't yet taking advantage of

10 the possibilities provided by modern electronic health

11 information systems for accurate and easy

12 communication of the relevant information contained in

13 a product label.

14 In all these areas, the FDA intends to

15 make progress in the months ahead. As we approach the

16 FDA's century mark, it is important to remember that

17 the FDA's hallmark has long been its ability to adapt

18 to keep up with the best new science and to use new

19 opportunities to fulfill its mission more effectively.

20 This meeting is a very important addition

21 to that tradition, and I am sure that it will help us

22 find better ways to fulfill our mission in light of

23 new technologies and new statutory authorities.

24 I want to thank all of you here today for

25 your commitment to help us find the best way to

1 protect and promote the public health. Thank you for

2 coming, and thank you for participating, and thank you

3 for engaging in this very important constructive

4 dialogue.

5 I am looking forward to seeing the results

6 of this meeting translated into improvements in our

7 guidance, and improvements in our efforts on

8 pharmacovigilance, risk management, and other critical

9 responsibilities for the agency.

10 Again, thanks very much, and I would be

11 happy to take a few questions if you have got any.

12 Thank you.

14 McClellan. If folks have questions, please come up to

15 one of the mikes in the audience there. And while

16 people are thinking about it, Dr. Sullivan, did you

17 have a question?

18 DR. SELIGMAN: Since I am being put on the

19 spot. This particular workshop was often focused on

20 sort of the risk side of the risk benefit balance, and

21 I wondered if you would like to say a little bit more

22 about what the agency can do sort of on the benefit

23 sides of medications, and ways and thoughts you have

24 about promoting both the beneficial use of medications

25 that may be under-utilized, or are not fully utilized

1 in our health care system.

2 COMMISSIONER MCCLELLAN: I think even

3 though as you said the focus here is on the risk side,

4 obviously in the decisions that we are going to make

5 based on the results of a meeting like this one, in

6 particular cases of pharmaceuticals the benefit side

7 is very important, too. I have often said that there

8 is no such thing as a completely safe medication.

9 It depends on the conditions under which

10 they are used, and the benefits and the risks facing

11 an individual patient, and we need to do more. We

12 need to do all we can to try to make sure that

13 medications are used where the benefits outweigh the

14 risks by the greatest margin.

15 So while we are focusing on the risk side

16 today, obviously in any particular decisions about

17 pharmaceuticals, we will consider those risks against

18 the potentially important benefits of the medication

19 as well, and that is what I meant by considering all

20 the risks.

21 Diseases that medications are intended to

22 treat, as well as risks of the medications themselves,

23 in our approach to efficient risk management

24 throughout the agency. And the things that I think

25 can help us work on risk can also help us work on

1 benefits.

2 Better information for patients and for

3 doctors on labeling to help them use the most up-to-

4 date evidence in their treatment decisions. Programs

5 to help prevent medication errors and adverse events

6 may also prove handy in supporting higher quality care

7 if we have got these kinds of information systems and

8 mechanisms in place.

9 So I think that there are a lot of

10 opportunities to work on both together, and while the

11 focus here may be on the risk side, you can rest

12 assured that we are thinking about the same kinds of

13 insights and opportunities to improve the benefit side

14 as well.

15 DR. GALSON: Great. Please identify

16 yourself.

17 DR. NELSON: Dr. McClellan, thank you for

18 your comments. My name is Bob Nelson, and I am a

19 consultant, but a former FDAer. I apologize if this

20 puts you on the spot, but since you are here I will

21 let it go.

22 COMMISSIONER MCCLELLAN: That's all right.

23 It comes with the territory.

24 DR. NELSON: Dr. Galson mentioned the

25 DailyMed program. From what I know of it, I think it

1 is very exciting and a collaborative effort with VA

2 and the National Library of Medicine to put bar-coded

3 labels.

4 But I understand that that has not really

5 been funded. If that is true, can you verify that,

6 and if it has not been funded by the agency, can you

7 comment on why?

8 COMMISSIONER MCCLELLAN: Well, let me take

9 a step back and talk about the agency's overall

10 funding levels. The FDA is bigger than it has ever

11 been, both in terms of personnel and in terms of

12 resources available, and I might add to that in terms

13 of statutory authorities with things like the new

14 post-market authorities in PDUFA-3.

15 We had a big legislative year last year.

16 The problem is that although our resources are

17 expanding, so are our responsibilities, and so are the

18 complexities and difficulties of the public health

19 issues that we are charged with addressing, and so we

20 do have to think very carefully about prioritizing.

21 We can't do everything, and we can't

22 devote unlimited amounts of funds to each individual

23 product, and Congress, and the rest of the

24 Administration, also have a say in how these

25 priorities are set, and we try to work with them to

1 make sure that we are focusing our efforts and

2 resources on where they can do the most good for the

3 public health.

4 I don't know the exact details of the

5 current funding status of the DailyMed program. I can

6 tell you that a lot of work is proceeding on the

7 development of that program. That is a good example

8 of what I mean by how we can improve the medical

9 labels, the pharmaceutical product labels in a way

10 that should lead to better information being available

11 to physicians.

12 And as a result, better care for patients,

13 and so that effort is proceeding. I think that there

14 are a lot of opportunities to implement steps in the

15 DailyMed program without a major new expansion of

16 funding.

17 For example, these collaborations with the

18 VA are ongoing, and if we get established some ties

19 with health care organizations like the VA, whereby

20 they can send us information on potentially important

21 adverse events that we may need to take steps to help

22 address, we can make it a two-way street by giving

23 them access to updated information that is relevant

24 for the product labels.

25 And that is moving forward, and so we will

1 keep working as hard as we can to identify how we can

2 use our limited funds as effectively as possible, but

3 I don't think we are in a situation where DailyMed is

4 not moving forward at all.

5 In fact, it is one of the more important

6 and I think promising efforts in our whole set of

7 patient safety initiatives.

8 DR. GALSON: Great. Thank you.

9 DR. SZARFMAN: My name is Anna Szarfman,

10 and I am a medical officer at the FDA, and have

11 implemented both the patient profile viewer that is

12 now being incorporated into that routine, and I have

13 worked together with Dr. Demichele to implement the

14 information on data mining tools.

15 COMMISSIONER MCCLELLAN: Yes.

16 DR. SZARFMAN: And the questions that I

17 have is how -- you know, with the patient profile

18 viewer, in the beginning it was difficult, because

19 people were accustomed to things such as lots of paper

20 and equivalency in other ways until we started these

21 studies, that we were gaining speed tremendously and

22 in a tremendous way.

23 And then there were no questionable

24 difficulties in implementing that piece. We think

25 that data mining is more difficult because people

1 think that we are torturing the data, and like you

2 have heard this morning, and we don't systematical

3 approaches in place.

4 How can we set up like in the clinical

5 pathology lab in the clinical pathologies. How can we

6 move from one place to the next one that is

7 (inaudible). You know, we have systems in place where

8 we can test things objectively, and I think that we

9 have to do it.

10 And that helps everybody to have objective

11 methods and recently in a (inaudible) committee, data

12 mining provided some information that was critical to

13 understand where are the specific drug, and if it was

14 hepatoxic, or if it was more hepatoxic, or less

15 hepatoxic, and so I would say it helps with the

16 answers.

17 COMMISSIONER MCCLELLAN: I think that

18 those are all good points. Obviously in any

19 activities that we undertake in this whole risk

20 management strategy, we need to think carefully about

21 the most efficient ways to move forward. You know,

22 where are the greatest needs, and where are the

23 greatest opportunities for improving the way that we

24 do what we do.

25 And absolutely we need to test whether the

1 changes that we implement have an objective impact on

2 important performance measures, such as review times,

3 and review costs, and whether we are identifying risks

4 more efficiently.

5 That really is the core goal here. There

6 is a tremendous amount of information out there, and

7 there is getting to be more and more every day, and

8 the question is can we use that information

9 efficiently and effectively in guiding our review

10 decisions.

11 I think the work that Dr. Demichele and

12 others are doing on data mining and related innovative

13 statistical techniques is very important, but in some

14 ways it is harder than the patient profiler.

15 You know, it is not just a matter of

16 implementing a software program that presents the

17 information that is there more efficiently and easily

18 for people to use. It is a matter of drawing

19 inferences about and sort of extracting signals from

20 very complex data that is not completely

21 straightforward.

22 There is not a yes/no answer. There are

23 different degrees of competence about the conclusions

24 that can be reached from data analyzed using any

25 statistical method.

1 And so that is going to be a more complex

2 technical process to improve upon. But I think that

3 the data mining holds a lot of promise, and I think

4 that there are a lot of other techniques that have

5 been suggested to the agency by other outside

6 statistical experts like Dr. Demichele that hold

7 tremendous promise for helping us make more or do more

8 with the information that we get.

9 And again as the information that we get

10 continues to grow in complexity, both in terms of the

11 information coming in and what it means, we need to

12 make sure that we are up to date on our statistical

13 methods.

14 And we need to have methods in place to

15 evaluate whether those statistical methods are having

16 an impact in a positive way on what we do. So I think

17 that these are the right kinds of issues that we need

18 to think about, and we need to couple it with making

19 sure that as we move forward on some of these ideas

20 that we are seeing an impact on our agency's

21 performance.

22 DR. GALSON: We have time for one last

23 question.

24 DR. YODRIN: Sam Yodrin, head of drug

25 safety, Millennium Pharmaceuticals. Thank you, Dr.

1 McClellan, for your talk, which really highlighted two

2 significant issues that we have to confront in this

3 new era of risk management during productive life

4 cycles.

5 The first is efficiency, and it is very

6 good to hear a regulator talk increasing of cost

7 effectiveness, and also the early identification of

8 significant signals.

9 And then also in terms of effectiveness,

10 and the issue of data quality. Increasingly we are

11 being asked to have an evidence base for our decision

12 making with regards to our risk management.

13 And the current situation is characterized

14 by increasing partnerships that are being established

15 primarily by the FDA with health maintenance

16 organizations.

17 There is the CERTS program, but the

18 challenge there for industry and companies, whether

19 they are a big pharma or a small pharma, alike, we all

20 cannot establish these types of relationships to

21 ensure that we have the available sources of data to

22 enable the appropriate risk assessment that we are now

23 faced with.

24 And I know that we have made this

25 representation to you in the past through a different

1 medium, but we strongly propose here that there is a

2 collaborative effort between the agency and industry

3 towards looking to establish an integrated national

4 pharmacoepidemiology database that would address all

5 the disadvantages that are currently inherent to the

6 fragmented approach that is being taken right now.

7 And we believe that this integrated

8 national pharmacoepidemiology database would align

9 with the current risk management framework, in the

10 sense that it would involve all relevant stakeholder

11 parties in collaboration, and all the issues that

12 relates to data quality certainly we could address and

13 maximize the benefits of information technology.

14 And so I just wanted to use this

15 opportunity again to make this proposal to the

16 Commissioner. Thank you.

17 COMMISSIONER MCCLELLAN: I think that

18 idea, such a national high quality system, is a very

19 interesting one, and a very nice vision for where we

20 would like to end up.

21 And as you pointed out, we are exploring

22 expanded partnerships with health care organizations

23 and the like, and basically we are going where the

24 data are, and we don't have such an integrated

25 national system now, and I don't want us to wait until

1 we finally get one before we move forward on some of

2 these initiatives.

3 I think that this is going to be an

4 incremental step-by-step process, but certainly the

5 product manufacturers and product developers have an

6 important role to play in both helping to develop this

7 kind of information and helping to interpret the data

8 that are there.

9 So the idea of moving towards more of a

10 partnership as we move towards better data systems

11 sounds like a good one, and I think it is something

12 that we should keep in mind.

13 DR. YODRIN: Thank you.

14 DR. GALSON: Dr. Sachs.

15 DR. SACHS: Thank you for letting me ask

16 my question. Susan Sachs from Roche, and since Bob

17 said since you are here, we have a chance to ask the

18 question, and since this is about risk, and it is not

19 about drugs, but I am interested in your thoughts the

20 fact that the FDA is not able to regulate supplements

21 and things like that, which actually have been in the

22 news in terms of risk, and I would be really

23 interested in your thoughts on that.

24 COMMISSIONER MCCLELLAN: Well, we actually

25 do regulate supplements. We just don't regulate them

1 as drugs. We have a different statute, like our food

2 regulatory processes, has a presumption that they are

3 safe, and therefore, they can go to market without

4 having to demonstrate safety and effectiveness up

5 front, and we have to prove a safety risk in order to

6 get them off the market, or in order to restrict their

7 use.

8 And that is something that we highlighted

9 back in February with respect to Ephedra that we are

10 undertaken and we have asked for comments, and in a

11 public comment period that just closed, for a review

12 of all of the evidence on Ephedra, which by far

13 accounts for the largest share of adverse event

14 reports that we get about dietary supplements.

15 And we are evaluating that evidence from

16 the comments right now on whether the evidence is such

17 that Ephedra as currently marketed presents an

18 unreasonable risk to the public health.

19 That is the standard under the statute and

20 we have to demonstrate that. So we will have more to

21 say about that soon. So it is not that we don't

22 regulate it. It is a different regulatory structure,

23 and it is one that presents some different challenges

24 than being able to ask product developers to

25 demonstrate up front that their treatments are safe

1 and effect.

2 The other important way in which we are

3 moving forward on some additional regulation

4 enforcement action on dietary supplements is that we

5 published a new proposed rule last month that would

6 impose some national standards for manufacturing,

7 national good manufacturing practices for all dietary

8 supplements sold in this country.

9 And I think that will give people a higher

10 degree of confidence that the products that they are

11 buying were actually produced using good methods

12 without impurities or problems in storage, and so

13 forth.

14 And it also includes new labeling

15 requirements that the active ingredients in a dietary

16 supplement need to be listed on the label and clearly

17 identified. And this is also a good way to help give

18 people confidence that what they are taking is what is

19 represented.

20 We would like and are trying to work on

21 developing better evidence on both the safety and

22 effectiveness of dietary supplements. We have got a

23 partnership going with the National Center for

24 Complimentary and Alternative Medicine to help us with

25 that, but I keep an eye and hand on that.

1 And one other area of concern is to get

2 them all out there, and that many dietary supplements

3 are making claims about their impact on patient health

4 that are misleading, and that are not based on good

5 science.

6 And back in December outlined in a white

7 paper a more comprehensive strategy for enforcing the

8 law against those claims. It is not legal to make

9 claims about the health effects of a product, and even

10 the structure function effects of a product, that are

11 not based on good science.

12 And you have seen a lot of enforcement

13 actions where we have actually seized products, and in

14 some cases companies have shut down as a result, where

15 they were making misleading claims.

16 And I think you will see more of that. I

17 do view it as a public health threat that as we try to

18 encourage people to do more, and to take control of

19 their own health, because that is the best way to

20 improve the public health in this country, through

21 people's own actions and diet lifestyle choices and

22 the like, have a bigger impact than most of the

23 medication that we regulate.

24 It is a public health threat if they end

25 up spending their money and wasting their effort on

1 products that are making claims about effects that are

2 just not right. So we will be doing more on that,

3 too.

4 It is a different regulatory standard, and

5 it is one that is more challenging in some ways, but I

6 don't agree that we can't do anything there, and we

7 are going to be trying very hard to improve things on

8 the dietary supplement side.

9 Thank you all very much for taking this

10 much time in hearing some answers that are a little

11 bit afield of the topic. The topic is a very

12 important one, and we truly appreciate your input and

13 your willingness to work constructively with the

14 agency both in this workshop, and I hope going forward

15 as we actually move into implementing some of these I

16 think quite promising new efficient risk management

17 activities. Thank you again.

18 (Applause.)

19 DR. GALSON: Thank you very, very much,

20 Dr. McClellan. We appreciate you coming. We are

21 going to go on with our program now, and it may seem

22 like we are behind, but we are actually not, because

23 we have in contrast to what is in the printed program,

24 we have three speakers in the next section, rather

25 than four.

1 So we are just fine and on time, and the

2 first oral presentation in this next session is Dr.

3 Anshu Vashishtha, from Watson Pharmaceuticals, and Dr.

4 Vashishtha, if you could give us your talk now, and

5 then when you are done, go ahead and sit at the table,

6 and the other speakers can speak up at the table, too,

7 whenever you want.

8 DR. VASHISHTHA: I thought Dr. McClellan

9 wanted to listen on some views to causality, but we

10 will do that another time. So today I just wanted to

11 give some comments on causality assessment, and I

12 think some of them have been presented.

13 Some of the comments will be along the

14 lines made earlier by other speakers this morning.

15 But as it was one of the questions that was addressed

16 to give the benefits, considerations, and causation

17 assessments, I think I will explain some of the pros

18 and cons as I see them for performing causality

19 evaluations, and what criteria to consider.

20 And beyond causality assessment, if we

21 look at the data mining as a way of computing

22 causality, or causal relationship, some of the

23 limitations of that.

24 And then some proposals and thoughts on going forward

25 in the area of assessing risk, and relationships of

1 risk to pharmaceutical drugs.

2 So I think we can break it up into as has

3 been mentioned the causality evaluation at the

4 individual level, and at the case collection level,

5 and then clearly either way there is a very important

6 rule of the case definition.

7 That when we start looking at causality,

8 the first thing we do is that we confirm that what we

9 are talking about is a report of the claimed event.

10 As an internist in drug safety, I have seen reports

11 that claim to be aplastic anemia, and which has turned

12 out to be only anemia, and not meet the case

13 definition as my standard of criteria.

14 And again I think as some of the speakers

15 earlier had mentioned, the importance of case quality,

16 beginning with good data capture, and defining the

17 case is really what the event being reported is, and

18 what it is stated to be.

19 I do see it as a necessity that we have to

20 look at causality at some level or the other. I think

21 the question is whether to do it for every case or to

22 wait until there is a case series, or a signal, a

23 potential signal is detected.

24 And as has been outlined I think very well

25 in the concept paper, it is because -- and in Dr.

1 McClellan's remarks, the reported adverse (inaudible)

2 concomitant medication that are known to have that

3 same safety risk or are not known to have that same

4 safety risk (inaudible) in disease.

5 So I think we cannot validate from it

6 because the current system is to cast a wide net, and

7 the consumer can call the company and say (inaudible)

8 and by the way does your title now cause it, and then

9 it becomes an adverse event according to current

10 requirements.

11 So since the letter is some broad, just

12 data mining or just counting adverse events will give

13 us a lot of false positives. I think that some of the

14 pros of doing a case assessment are that it can be to

15 better case quality and follow-up, because as we start

16 trying to do a causality assessment, we can think of

17 the confounding factors, and we can think of the

18 follow-up needs, and get those questions addressed

19 while the case is still fresh and the data is

20 available.

21 Also, it will enhance ongoing signal

22 assessment by breaking down the reports from just

23 counting them as reports that may be incidental,

24 rather than to the categories, and ranking the

25 categories, and we can discuss the ones outlined in

1 the concept paper work, as well as any others.

2 But I think that those are some of the

3 advantages of doing it. I also think that, and again

4 as Dr. McClellan also mentioned, that if you want to

5 make the package inserts more useful to prescribers

6 and patients that the assessment of causal

7 relationship can provide a tool to do that in keeping

8 events which are incident reported out, and not

9 considering them as adverse reactions merely related

10 to the drug.

11 And on the other hand, events which are

12 assessed as adverse reactions truly belong in the risk

13 information on the label, and perhaps linked to the

14 management programs and could get in there.

15 The points of course have been pointed out

16 that it indicates that it does take time, and it does

17 take resources, and again if we do it on the industry

18 side that it takes them on both ends.

19 It may be that there are some approaches

20 which are not all the way down the road of distinction

21 between a probable and a possible, but at least

22 outlined in a case whether it is a substantial case or

23 not, and the WHO criteria which I will go into at

24 least indicate as an example.

25 It is limited of course by a lack of all

1 relevant data in several instances, and it can be

2 misused as was mentioned again in the morning, that

3 just because something is confounded does not mean

4 that it is not related.

5 I think that one has to take the whole

6 picture into account. There is also the issue from

7 the industry side of what is the legal liability of

8 the sponsor who has assessed certain cases as possibly

9 related, but not made the decision to put them in the

10 package insert.

11 The criteria that can be considered I

12 think are well outlined in the concept paper, and I do

13 not need to review them all. We can just go to the

14 slides, and I think the CIOMS working group has

15 indicated them.

16 Again, the most important being that if

17 there is a rechallenge after the drug, and as was

18 mentioned in the morning even that is not a certain

19 one, and so as for most of these criteria, whether it

20 is rechallenge, dechallenge, class effect, biological

21 plausibility, a lack of a alternate explanation, or a

22 typical ADR type event, like a Stephens-Johnson, with

23 a low background rate not related to drugs.

24 All those response factors lack further

25 concomitant factors in a clean subject patient, or a

1 consistency of time to onset, or a high frequency of

2 reported cases of similar findings in doctors case

3 studies.

4 Or perhaps positive individual or

5 individual tests, like IG antibodies, which are only

6 available in a small minority of adverse events or

7 adverse reactions; or if there is a clear

8 identification of an identified subset, where there is

9 a clear risk identified, and where the reaction is

10 related to the drug.

11 And again I think the protopathic bias has

12 been mentioned earlier, that a drug which is

13 administered to treat a symptom may be temporarily be

14 associated to the illness which manifests after the

15 drug is started, even though the drug was started to

16 treat the symptom of that same illness.

17 And also it has been mentioned the issue

18 of stimulated reporting, and in particular I think for

19 causality assessment when there is stimulated

20 reporting, there may be a recall bias of not recalling

21 perhaps and a bias towards false attribution towards

22 the suspected drug.

23 One thing that I think that I would

24 recommend to the working group is that we consider the

25 spectrum of causality assessment methods available,

1 and evaluate them in a way to go forward, and I think

2 to some extent that there are some literature reports

3 which given the limitations of these, as well as

4 (inaudible) products ability back I think from '68 I

5 don't think should be our guidance going forward.

6 I think we need to look at what the

7 current status of these is. There has been research

8 on them and extensive use of them in Europe which has

9 continued after the FDA discontinued causality

10 assessments from '83.

11 And I think that it would be helpful to

12 draw on this using criteria like applicability,

13 explicitness, explanatory, culpability, completeness,

14 biological balancing, and lack of constraints, and

15 these are reviewed, of course, in several writings on

16 the topic.

17 So one of the methods that can be not all

18 the way to categorizing it into probable or possible

19 is the literature of Dr. Edwards criterion of using

20 three index cases.

21 And where cases can be divided, I think,

22 with relatively less time and effort into substantial

23 index case or feasible case, and a substantial index

24 case has information on the first six -- on all the 11

25 items, and there is a feasible case as on the first

1 items.

2 So just having the items on each of these

3 could be one way of categorizing an adverse event

4 report into a feasible or substantial report which

5 then can make the basis for a more detailed causality

6 evaluation.

7 And these sort of rely on the fact that

8 for rare events the probability of picking up three

9 cases is very low, or in a short period of time it is

10 very low, and so that is the strength of that.

11 And just to mention some of the risk data

12 mining approaches, where the approaches are based on

13 the proportion of the reporting rates, and some

14 published opinions that this remains as only one

15 aspect of causality assessment, subject of course to

16 the usual biases.

17 And I don't think we can get away from

18 global clinical assessment or more case detail based

19 approaches, as opposed to again counting and doing

20 status tests on events as reported.

21 And so in terms of the categorization into

22 broad categories, I think the one in the concept paper

23 is one, and there is also this proposal from the --

24 based on the review of causality classification at

25 pharmacovigilance centers in the European community,

1 which Meyboom, et al., had published a couple of years

2 ago, almost 10 years ago, into Category A, B, or O,

3 where there is either good reason and good

4 documentation to presume a causal relationship in

5 Category A.

6 Or in Category B, where the connection is

7 uncertain, and even doubtful, perhaps because of

8 missing data, and maybe because a case does not meet

9 substantial or case, or category, which is not

10 assessable because of missing or conflicting data.

11 I think that some sort of categorization

12 into such broad categories would at least make causal

13 evaluation more efficient in terms of assessing risk.

14 So at the individual level, then I would recommend

15 that the working group consider the researching

16 experience of the European agency.

17 The French agency, for example, does

18 require causality assessment by sponsors for a long

19 time, and so that there might be some data there to

20 base our decision going forward on as we try to get

21 evidence based on this, and not just go by our

22 impression of the utility or lack thereof.

23 For the case collections, I do think that

24 it remains necessary for signal evaluation. Again,

25 the working group may consider really looking at the

1 different alternative approaches and perhaps testing

2 them on the FDA database.

3 It remains important to define a case for

4 the cases confirmed and it is of good quality, and

5 algorithms can be evaluated I think in different or on

6 a worldwide basis, perhaps categorizing into these

7 broad zones, either A, B, or O, or something that is

8 more amenable to perhaps less label intensive, or

9 medical review intensive methods.

10 But nevertheless lays a foundation for a

11 fully detained risk assessment, and perhaps these can

12 be compared by really doing a retrospective evaluation

13 for the utility of these methods using the existing

14 data from the FDA database.

15 So just seeing if the two have been

16 applied what would we have found, and comparing maybe

17 some of the tools that are used in Europe, and see if

18 that would have been helpful in identifying the risk

19 which we now know in an earlier way.

20 One may consider a pilot on certain drugs

21 and in certain instances, and one can then determine

22 the overall guidance based on the results of the

23 assessments. Thank you very much again to the whole

24 working group and the FDA for this portion.

25 DR. GALSON: Thank you very much. Our

1 next speaker is Gregory Gogates, or Gogates.

2 MR. GOGATES: Good morning everyone. My

3 name is Greg Gogates, and I represent CRF Box. We are

4 a clinical data collection company using remote data

5 collection tools, namely PDAs and cell phones. So I

6 thought that this would be a good venue to sort of

7 tell people what is going on in the world as far as

8 the status of collecting this type of information

9 using portable data collectors.

10 Okay. The overall overview I think, and

11 everybody has said that a million different times, but

12 I guess the thing that I am trying to state here is

13 that some of this data that we are talking about

14 collection with AE data could certainly be done using

15 PDAs and mobile phones.

16 One of the things that we are talking

17 about doing is getting this information through the

18 doctor-nurse and all the different organizations that

19 exist, but one thing that I think we are missing is

20 direct collection of data from the patients directly

21 into databases and this is happening today.

22 So I am going to discuss some of the pros

23 and cons of doing this, and show you a case in which

24 and where this is actually happening right now in

25 Europe. So the typical compliance that we have out

1 there, and this is from some ad hoc interviews that I

2 have done with some physicians and a few people in the

3 industry, and we have relatively good compliance with

4 the clinical trials, hospitals, and pharmacies.

5 And I think the answer to that is

6 typically because these people are salaried people,

7 and they are being paid to be in their office all day

8 long. So if they have to answer a few forms, that is

9 just part of their daily jobs.

10 So you have relative good compliance.

11 Now, the quality of the data, and other people are

12 talking about the quality of data, but I think we have

13 good compliance there, because it is part of their

14 job.

15 Now you move into the citizens, or you

16 move into the private physician area, and citizens

17 don't necessarily know that this avenue exists, or

18 know and don't want to bother. I think that is what

19 you are finding there.

20 And you have private physicians who say,

21 well, I am being squeezed by all means. I am a broke,

22 fixed type of a person, and a patient comes in, and I

23 fix them, and they leave the office, and what is my

24 incentive to fill out all these forms.

25 And I think you find that a lot of times

1 they say I am not going to bother. I fixed my

2 problem, and I am going to move on to the next

3 patient. So by its nature, it is voluntary on that

4 avenue, and I think that we can improve some of that

5 by making it a little easier.

6 So you have your traditional MedWatch

7 methods using the 3500A and the 3500 forms, but on the

8 3500 forms, it is all not mandatory. So there are

9 some custom on-line tools which I believe have been

10 discussed a little bit earlier that do exist, but they

11 are not or they don't travel home with the patients.

12 So what I am saying is that maybe we

13 should consider getting some of this information and

14 making it available for the patients to input this

15 information from their homes.

16 So the current challenges of course is to

17 get this collected and report this spontaneous patient

18 level safety data per the CFR as referenced. And of

19 course we want to get the timeliness of the data that

20 has to come in there, and so we want to make this data

21 more timely.

22 And of course we have to reside within the

23 HIPAA regulations, and I understand that AES are

24 pretty much -- that they don't have to worry about

25 HIPAA, but at the same time, we still have to worry

1 about some of the fringes, because it really is not

2 that well defined.

3 So we have to get increased public

4 awareness of the public safety concerns, and make sure

5 that they understand that there are concerns with

6 these products, and as we said before, these products

7 aren't perfect. We understand that there are risks in

8 taking them, and that people have to understand that.

9 So how do we respond to these things?

10 Well, I think the key to this whole thing is

11 spontaneity. If you don't get it up front, you don't

12 get it at all. So let's use some electronic tools.

13 Some of these tools as you see up there on

14 the screen, you have PDAs, and the PDAs right now, you

15 can buy wireless ones that are actually part also cell

16 phones, and you can buy them with cradled modems so

17 they can come up with data.

18 You can also use normal cell phones, and

19 there are of these more expensive cells phones, but

20 the bottom line is that this information could be

21 brought directly into data bases, and viewed on-line

22 by anybody who has an access, and it can be stored

23 centrally for review by anyone who wants to look at

24 it. It is very simple.

25 Implementation issues. Patients can use

1 the proprietary electronic devices; i.e., the mobile

2 phone. So we don't have to give them anything. They

3 already own a mobile phone.

4 Mobile phones not only can talk voice, but

5 they can also transmit data, and the SMS messaging

6 system, which is on most of everybody's cell phones

7 here in the audience, is usually not that well used in

8 the States, but it is very pervasive in Europe and the

9 rest of the world.

10 But if you have a budget, if this happens

11 to be a trial, and there is some budgets, you could

12 give them a proprietary device, say a PDA, or

13 something else to collect data with. But, in reality,

14 I think the mobile phone is probably a better venue

15 for this.

16 So the only thing that is recorded is that

17 you could record the patient's cell phone number at

18 the point of prescription. So you could say, okay,

19 Mr. Patient, I am giving you this drug, and would you

20 like to volunteer to provide some information on your

21 experience with taking this product.

22 If the man answers yes or if the person

23 answers yes, you collect their cell phone, and then a

24 week later, or two weeks later, or a day later, or

25 whatever frequency you want, the system would call up

1 the person's cell phone and say, hello, this is the

2 drug company, this is the pharmacy, how do you feel.

3 Have you had any experiences with this

4 drug. They could ask 3 or 4 questions and you answer

5 the questions, and there is no voice going on, and

6 they don't have to worry about doing it. It is not

7 intrusive at all. It's just that your phone beeps,

8 and you look at it, and if you don't want to answer

9 the question now, you answer the questions later.

10 You just push the buttons, and you say

11 have you had any diarrhea, yes/no, enter. Have you

12 had any vomiting, yes/no, enter. Headaches, whatever

13 your questions want to be, you enter the information,

14 and it goes right back to the database, and it is

15 immediately available for all people to view.

16 So these periodic safety questions can be

17 sent and answered and sent just as I mentioned. There

18 is text space, and there is no voice involved, though

19 there is one side benefit of having the voice.

20 If the right question is answered yes,

21 then there could be a follow-up. Someone could call

22 this person back and say hello, you answered yes on

23 this question. We need some more information. So it

24 would be nice and straightforward, and very simple to

25 manage.

1 Pros and cons. They don't necessarily

2 need to visit the physician. They don't need to get

3 involved. It just shows up on their cell phone. And

4 the sponsor and anyone else who wants to, has the

5 ability to see this information in real time, and they

6 can combine it with any other safety data that they

7 need.

8 So the timeliness is improved, and the

9 questions can be changed on the fly. So if you decide

10 that the questions being asked aren't correct, you can

11 tell the system to ask new questions.

12 So the cost, which is the cost of this

13 technology, it does add to the budget, which is a

14 deficit. But if you are using the patient's

15 proprietary cell-phone, you don't have to buy their

16 equipment. You just have to deal with the server

17 side.

18 So don't get me wrong. I am not saying

19 this is an end-all, but it certainly could be used in

20 some situations. You may have the disabled, or the

21 elderly, or someone who doesn't have a cell phone, or

22 maybe something that just wouldn't make sense, and of

23 course you wouldn't do it.

24 But this is a system that could be used in

25 a lot of areas. So there is detail as I said earlier,

1 and there would be detailed analysis that still would

2 need to be conducted with the physician or the nurse,

3 but in this case at least you would get that up front

4 spontaneous information, which everybody understands

5 is the best way to get the information.

6 There are some examples out there that I

7 can state that our company is doing at this point.

8 There is a Phase III trial going on in Europe at this

9 time, and there is 12,000 children across Europe, and

10 at 75 different sites, and they have been given a

11 vaccine, and this is a situation where they need to

12 collect data for 2 years every 4 weeks, from 12,000

13 parents.

14 And this would be a phenomenal expense,

15 and so we came up with this idea of collecting the

16 parents' cell phone number, and then every 4 weeks the

17 system calls the parents' cell phone.

18 They get a beep, and it says they have

19 received a message that says, hello, this is the study

20 and how does your child feel. And the responses,

21 there are maybe 4 or 5 questions, and they answer

22 yes/no, and the system answers back, thank you, and I

23 will call you back in four weeks.

24 Now, we are not getting a hundred percent

25 response out of this, but at least we are taking at

1 least a 90 percent response on this information, and

2 now the study nurses only have to call up 10 percent

3 of those 12,000 people.

4 So it is a significant cost savings in

5 doing all of this follow-up work, and there is no

6 reason why this technology, or these concepts,

7 couldn't be moved into the AE realm of people taking

8 medicines, and just taking it from their physician or

9 their pharmacy, and not necessarily in a clinical

10 trial.

11 So in conclusion, basically I think that I

12 would like the organization to consider using mobile

13 phone infrastructure. It is in this country, and it

14 is global, and it is very simple to use, and it would

15 be a very good venue to collect patient data.

16 I would say that in Europe, and in the

17 rest of the world, you have very good -- just about

18 everybody and their brother, and children, all have

19 cell phones, and it is becoming just as pervasive here

20 in this country.

21 As I said, the EU system are mature, with

22 good inoperability, and one issue that we do have in

23 the States is the inoperability between cell phone

24 providers is not perfect, but it is increasingly

25 getting better.

1 So considering that this is going to be a

2 long term permanent effect, that this is something

3 that we should consider. And that is really my simple

4 message that I have to say. Thank you very much.

6 last speaker for this morning is Linda Hostelley.

7 Linda. And she is going to tell you who she is

8 speaking for.

9 MS. HOSTELLEY: Good morning, ladies and

10 gentlemen. I am Linda Hostelley, the executive

11 director of worldwide product safety in epidemiology

12 at Merck Research Laboratories.

13 I would like to first thank the FDA panel

14 and the entire working group for the opportunity to

15 provide public comment today on behalf of the

16 Pharmaceutical Research and Manufacturers of America,

17 concerning concept paper number three, risk assessment

18 of observational data, good pharmacovigilance

19 practices in pharmacoepidemiologic assessment.

20 As mentioned previously over the past few

21 days, PhRMA supports FDA in the following concepts.

22 We support FDA's efforts to provide guidance for

23 developing pharmacovigilance in risk management plans.

24 We support the concept that risk

25 management is an ongoing process throughout a

1 product's life cycle to optimize the benefit risk

2 balance. We agree with the agency that the ultimate

3 goal of any plan in its evaluation is to ensure that

4 the efforts and costs involved in an RMP are expended

5 on effective processes that achieve a positive benefit

6 risk balance.

7 PhRMA believes that risk management

8 programs and interventions should balance access to

9 drug with level of concern. We agree with the FDA

10 that decisions which are made concerning individual

11 drug benefit and risk, and RMPs, must be based on

12 scientific evidence.

13 We believe that any risk management

14 program should rely on systems based interventions,

15 and specific tools should be used consistently across

16 RMPs. We recognize that efforts to identify all risks

17 prior to drug approval may not be feasible without

18 delays in drug development.

19 And we recognize that collaboration among

20 all the stakeholders, including FDA, the academic

21 institutions, help care providers, pharmacists,

22 professional organizations, and patients, is

23 absolutely critical to achieve significant

24 improvements in the benefit risk balance.

25 My comments today will focus on the first

1 three questions posed by the FDA in concept paper

2 number three concerning good pharmacovigilance

3 practices. Dr. Stephenson will address the questions

4 on pharmacoepidemiologic assessment during the

5 afternoon session.

6 The first question. How can the quality

7 of spontaneously reported case reports be improved.

8 PhRMA supports FDA's efforts to improve the quality of

9 individual spontaneously reported cases which serve as

10 the foundation for identification of safety signals,

11 and are the critical building blocks of a post-

12 marketing safety surveillance system?

13 We recognize the importance of obtaining

14 complete information for these reports, but recognize

15 that a balance must exist between the significance of

16 the clinical event reported, and the intrusiveness of

17 follow-up with treating health care professionals.

18 The more onerous, intrusiveness, or

19 threatening the follow-up, the more likely the number

20 of reports will diminish. Such an outcome is exactly

21 the opposite of what is needed.

22 It would be useful to all stakeholders if

23 the agency provided a prospective on what aspects of

24 case reports, what sources of reports, and what sort

25 of follow-up has been most helpful in signal detection

1 and regulatory actions.

2 It would also be useful to know that where

3 there is waste in a reporting system, and without a

4 formal study of these questions, it is speculation as

5 to how to improve the quality.

6 The provision of adequate clinical

7 information is a shared responsibility with all

8 stakeholders. PhRMA supports the education of all

9 partners involved in post-marketing safety

10 surveillance to improve the understanding of the need

11 for reporting and the quality of reporting.

12 To that end there should be education of

13 practitioners to communicate high quality data with

14 specific clinical details, and education of patients

15 to recognize signals and report specific details.

16 There is also a need to increase awareness

17 of the importance of pharmacovigilance in clinical

18 pharmacology for health care professionals during

19 their medical, nursing, or pharmacy school curricula.

20 Education of these health care providers

21 on a continuum throughout their careers will improve

22 the understanding of the need for reporting and the

23 quality of reporting.

24 PhRMA and FDA should continue to partner

25 with medical, nursing, and pharmacy schools to

1 increase the awareness of the importance of

2 pharmacovigilance in clinical pharmacology.

3 Partnership with health care organizations

4 and academia is also needed to reinforce these

5 principles. This effort to improve report quality

6 will also benefit from continued collaboration with

7 the National Patient Safety Foundation.

8 Question Number 2. What are the possible

9 advantages or disadvantages of applying data mining

10 techniques to spontaneous report databases for the

11 purpose of identifying safety signals?

12 One of the greatest challenges in post-

13 marketing pharmacovigilance is determining what

14 constitutes a safety signal. Post-marketing adverse

15 event databases, which are based on voluntary adverse

16 event reporting, lack formal controlled data, and

17 exposure data, and until recently have been difficult

18 to use objectively and efficiently for signal

19 detection.

20 Traditional pharmacovigilance methods have

21 consisted primarily of individual case review and

22 empiric analysis of crude frequency data. During the

23 past 5 years, tremendous progress has been made in the

24 development and application of statistical methods for

25 systematically screening post-marketing adverse event

1 databases to efficiently identify adverse events that

2 occur at a higher than expected frequencies in the

3 absence of exposure data.

4 Examples of these data mining or

5 disproportional analysis methods include the

6 proportional reporting ratios developed by the U.K.

7 Medicines Control Agency, the multi-item gamma poison

8 shrinker, developed by William Demichele, of AT&T, in

9 collaboration with Anna Szarfman, of the FDA; and the

10 Beijiang (phonetic) Neural Networks, developed by the

11 WHO.

12 Recently these sophisticated tools for

13 data mining have become more available to regulators

14 in industry through packaging and deployment of modern

15 easy to use web-based interfaces.

16 Data mining may therefore become a useful

17 adjunct to traditional pharmacovigilance methods, and

18 that it provides a systematic, efficient, and

19 objective approach to screening large adverse event

20 databases for safety signals.

21 It may be especially useful for detecting

22 safety signals involving rare events that are unlikely

23 to be observed in clinical trials or observational

24 studies. These rare events may also allude detection

25 by traditional pharmacovigilance methods.

1 Other potential applications include

2 screening for signals associated with polypharmacy.

3 For example, drug interactions, and for demographic

4 factors associated with adverse events.

5 Data mining analyses compliment data from

6 well-designed, well-monitored clinical trials, and

7 observational pharmacoepidemiologic studies, and can

8 be used to generate hypotheses that can be further

9 tested in such studies.

10 Furthermore, in some instances, the

11 information gleaned from data mining can be used to

12 enhance risk management and clinical development

13 planning. The limits, however, of the underlying data

14 and the data mining techniques must be fully

15 appreciated to avoid false positive causality

16 conclusions with costly interventions and

17 inappropriate product restrictions.

18 Research efforts should be targeted to

19 reduce false positives. To this end a collaborative

20 working group of statistical and medical experts from

21 both the industry and FDA was recently formed.

22 The goal of the working group is to define

23 best practices for the application of data mining

24 technology that will optimize use of these tools in

25 the areas of pharmacovigilance and risk management.

1 The group will seek to better understand the

2 comparative performance characteristics of various

3 data mining methods.

4 The optimal specifications and

5 configurations for various uses. Data quality issues

6 and the limitations of these methods, particularly as

7 they affect the interpretation of the results.

8 It is also hoped that this joint effort

9 will provide an opportunity for FDA and industry to

10 develop a common language, and share systematic

11 approaches to the detection and assessment of signals

12 from post-marketing adverse event data.

13 Question Number 3. What are the possible

14 advantages or disadvantages of performing causality

15 assessments at the individual case level? Given the

16 inadequacy of spontaneous report data, the application

17 of causality algorithms to a single case is fraught

18 with misinterpretation.

19 The ability to rule out the likelihood

20 that the suspect drug may have contributed to the

21 adverse experience in most instances becomes

22 impossible. Therefore, most adverse experiences at

23 the individual case report level end up with a

24 possible association.

25 With the exception of cases involving a

1 positive rechallenge, there is little or no advantage

2 in performing causality assessment on individual case

3 reports.

4 In addition, a lack of consistency among

5 experts in the evaluation of causality assessment for

6 individual cases strains the already limited

7 resources.

8 Although a series of cases may be used to

9 generate hypotheses concerning the association between

10 an adverse experience and drug exposure, there is no

11 methodology determined to date that is reliable and

12 reproducible for individual causality assessment.

13 Thus, causality assessment at the

14 individual case level is open to a high likelihood of

15 misinterpretation. In summary, PhRMA supports the

16 development of best practices for improving the

17 quality of spontaneous reports and for developing the

18 methodologies for application of data mining

19 techniques.

20 PhRMA also supports education of all

21 partners involved in post-marketing surveillance. In

22 closing, PhRMA would like to pose an additional

23 question for the FDA. What criteria should be used to

24 determine whether a pharmacovigilance plan describing

25 pharmacovigilance efforts above and beyond post-

1 marketing spontaneous reporting is warranted.

2 Thank you for this opportunity to provide

3 comment on behalf of PhRMA.

4 DR. GALSON: Thank you very much. We now

5 have about a half-an-hour for a discussion session and

6 question and answers, and if all the six speakers from

7 this morning, the six presenters, want to sit at the

8 table, that would facilitate this.

9 I don't know if everyone is still here,

10 but we have got six seats, and there were six of us.

11 And so we will take questions from cards and people

12 are walking around to pick up any other questions on

13 cards.

14 People can come up to the microphones and

15 we will just have a free exchange between those of us

16 up at the front of the room and people who want to

17 come up and talk. I think I will just start with a

18 question while people get themselves going.

19 And this is one that came in on a card and

20 it says please address the effects of MedRA on label

21 and safety signals, and I think that is one of our

22 working group members, Min Chen, is going to address

23 that.

24 DR. CHEN: It is the ADR proposed rule

25 that requires all adverse events in the case reports

1 coded in the most recent version of MedRA. Currently,

2 however, the ADR individual events, or the organ class

3 in the drug labeling has no specific guidance as to

4 what to use.

5 But in a safety signals detection in post-

6 marketing, we use the measure terminology to store and

7 to retrieve clinically irrelevant cases in the

8 databases.

9 Once risk is identified, currently it is

10 the clinical or medical concept of the identified risk

11 information that will be considered for incorporation

12 into the labeling.

13 In the future, I think there may be some

14 considerations in the agency to have a consistent

15 labeling on the specific ADR information in the

16 labeling using MedRA terminology.

17 But I think that still has some way to go,

18 because the clinical trial data should be coded in

19 measure first before they use the information in the

20 proposed labeling. So that is the current situation

21 right now.

22 DR. GOLDMAN: Are we allowed to comment on

23 that?

24 DR. GALSON: Yes.

25 DR. GOLDMAN: Okay. A couple of

1 principles if I may, and I say this to the other

2 people who basically helped develop the measure

3 sections. The tail should not be wagging the dog.

4 Labeling is designed to convey clinical concepts.

5 And there have been several studies done,

6 such as work Ali Brown (phonetic) has done, and been

7 published, and others. That you could actually have

8 terms that when the MedRA version changed, and went

9 from labeled to unlabeled, and I have -- I am putting

10 out a cautionary note in public, and as we have done

11 in print, that MedRA was not designed for that in

12 particular.

13 And that that has to be borne in mind when

14 we talk about labeling, particularly when we would be

15 coding measure in terms of clinical trial data, and I

16 just wanted to have that point.

17 DR. GALSON: Okay. Let's move to the

18 mikes now on this side. Please identify yourself.

19 MR. STANG: Paul Stang, Gold Associates.

20 A couple of general comments. One is that when you

21 read through the document, although there is

22 separation between signal detection and evaluation, I

23 think it is very important to make that absolutely

24 crisp and clear, because I think that a lot of people

25 confuse signal detection with signal evaluation.

1 I think that there are just many

2 opportunities to clarify that. A second comment has

3 to do with data mining, and that is whether or not we

4 have progressed to the point where we have

5 complimentary rules for when we don't find a signal,

6 inasmuch as when we do.

7 And the point here is that the fear would

8 be that there would be multiple looks at the data many

9 times over time, and without proper statistical

10 adjustment, or other ways of accounting for the fact

11 that you keep going over time looking for something,

12 and eventually you will find it, is my sense if you

13 look enough times.

14 So I think that this is just a plea to, as

15 others have said, to develop some methodology around

16 it. The third point is that I work with a non-profit

17 group of physicians, a grass roots organization, and

18 we developed just on a whim a self-training module in

19 safety surveillance, just because we had heard from a

20 few of the 10,000 members that they had an interest.

21 And we developed a little CD-based

22 training system, and a thousand of them were handed

23 out -- well, actually, people had to request them, and

24 all thousand of them went in two weeks.

25 So there is great thirst for this

1 information at the primary care level. It is a

2 primary care based organization, and I actually shared

3 an early version of this with members of the agency.

4 So I do want to point out that primary

5 care is screaming for this, and that as someone has

6 pointed out, they don't get this kind of training in

7 medical school, and they certainly don't get it

8 subsequent to that. They themselves are looking for

9 more information.

10 And finally along those lines is the

11 training for the actual consumers of this information,

12 and I specifically have been struck in a couple of

13 meetings that I have attended with the agency with

14 their external clinical advisory boards for various

15 products that are coming up under review post-

16 marketing for certain issues.

17 And I think that there is a great

18 opportunity to educate some of the external clinical

19 experts that are brought into these meetings along

20 safety issues, and the resources that are available,

21 and with what their strengths and what their

22 limitations are. Thank you.

23 DR. GALSON: Thank you and I think I will

24 turn to Dr. Beitz now, who had a question.

25 DR. BEITZ: I had a question actually that

1 I would like to direct to some of our industry

2 representatives here today. And it has to do with the

3 application of an FDA newsletter of adverse event drug

4 combinations.

5 And there is really some sensitivities

6 around publishing such a newsletter, and how would we

7 choose which events and which drugs might be included

8 in a particular issue.

9 And I would just like to get a sense of

10 how people would think about publishing on particular

11 adverse events or combinations even though it has not

12 been totally worked out as to causality. Labeling may

13 not be fully worked out or discussed, or negotiated.

14 Do people feel that a newsletter approach

15 could or could not occur in parallel with the more

16 traditional labeling negotiations, or does something

17 have to be in labeling first before folks in industry

18 would feel comfortable about FDA putting it in a

19 newsletter?

20 MS. HOSTELLEY: Linda Hostelley,

21 representing PhRMA, and perhaps others from the

22 organization will also comment, but I would say that

23 this is an interesting concept, and I think PhRMA

24 would certainly welcome an opportunity to have further

25 dialogue on this topic.

1 Again, you know, the concerns from an

2 industry perspective would be, you know, how much can

3 be said before an actual label is constructed, and of

4 course there are the legal implications for us as a

5 regulated industry.

6 But certainly it is an interesting

7 concept, which I think would warrant further

8 discussion.

9 MS. STEPHENSON: Can I add to that?

10 DR. GALSON: Sure.

11 MS. STEPHENSON: Wendy Stephenson from

12 Wyeth. I actually -- my first impression is that a

13 newsletter of that sort would be very problematic. I

14 think it would be much better to have fully vetted the

15 issue, and evaluated data, and come to a conclusion,

16 which then would usually result in a labeling change,

17 or result in the conclusion that there is not an

18 issue.

19 And then perhaps it might be reasonable to

20 publish. We have had some very sort of strange

21 experiences with a similar type of newsletter that the

22 Upsalla monitoring center of WHO has issued.

23 They generally issue these reports of

24 signals -- they are called signals -- and then after

25 the publication, they will ask the company for a

1 comment. And in every case where we have had one of

2 those signals referred to us, it was a completely

3 spurious association, and there was generally a pretty

4 good explanation for what they were seeing.

5 And there was no causal association. So I

6 would just caution against publishing preliminary

7 information before it has been fully vetted.

8 DR. VASHISHTHA: Anshu Vashishtha from

9 Watson. I think another issue that of course comes up

10 is the issue of the importance of sequence reporting,

11 the stimulated reporting, and the recall by issues

12 that have come up with such a publication.

13 I think that is where we have to clear the

14 package inserts and they have professional letters and

15 other ways to have established lists (inaudible)

16 risks.

17 And anything done prior to that in my mind

18 to the general public would distort the evaluation of

19 that risk.

20 DR. GALSON: Alan.

21 DR. GOLDHAMMER: Yes, Alan Goldhammer,

22 PhRMA. I would echo Dr. Stephenson's comments on

23 this. I think that you can go back and look at the

24 literature being rife with examples of preliminary

25 assessments of risks, which later are disproved.

1 Unfortunately, companies in many cases

2 have incurred significant legal bills defending

3 themselves in court. I mean, we would be more than

4 happy to look at the proposal, but I am not terribly

5 optimistic that the legal side of PhRMA would be

6 encouraged by it.

7 DR. GALSON: Okay. We are done with

8 follow-up on that question. Oh, one more. Okay.

9 DR. NELSON: I was going to comment that

10 what could possibly be more valuable is after the

11 labeling change has been agreed to, have a publication

12 that explains the evidence base for that labeling

13 change.

14 DR. GALSON: Good. Okay. I think we will

15 move to the next mike question.

16 MS. OMINOFF: Okay. I am June Ominoff

17 from the directorate at GlaxoSmithKline, and I am also

18 the chair of the PhRMA/FDA working group on safety

19 evaluation.

20 And I am actually up here to respond to

21 Dr. Faich's comments, and concerns about data mining.

22 We thank him for his comments, because they do

23 highlight some important caveats about the tool.

24 However, I want to make two points.

25 The first is that if we adapt Min Chen-

1 Julie Beitz's, the working group's definition, of a

2 signal which is, quote, apparent excess of adverse

3 events associated with the use of a product, then what

4 could be more valuable then a disproportionate

5 analysis or data mining tool that objectively applies

6 to statistical methods to identify these excesses.

7 My second point is that he addresses

8 examples of false positives, or what he calls false

9 positives, and I wanted to say that no one who uses

10 this took responsibly would ever, ever draw

11 conclusions without reviewing the medical case

12 information using standard pharmacovigilance

13 practices.

14 So in my view, the examples that Dr. Faich

15 has presented as false positives really seem to

16 reflect an intermediate stage in the evaluation of a

17 signal. Thank you.

18 DR. GALSON: Thanks a lot.

19 DR. GOLDSMITH: I would like to address

20 the same issue with regard to data mining. My name is

21 David Goldsmith, and I am a consultant, but I am

22 representing myself. And the issues that I would like

23 to raise are actually on both sides, and then follow

24 it up with a question.

25 First is the issue of the false positives,

1 and I would like to present a rather simple scenario

2 that gives an example of a false positive. You have

3 Drug A, which is a recently developed drug, and it is

4 effective. It is so effective that it works in a

5 hundred percent of the patients who are treated.

6 It has a very clean profile and indeed

7 after being marketed for 6 months there are a hundred-

8 thousand patients exposed, and there are 10 serious

9 adverse events that are reported, nine of which

10 represent migraine headache.

11 The other drugs in the class that are used

12 to treat the disease are not terribly effective. They

13 are only about effective in 50 percent of the cases,

14 and they are very dirty drugs. And they actually have

15 lots and lots of serious adverse events, and they are,

16 for example, in that same 6 month period with that

17 same sales ratio 500 cases of migraine headache, and

18 500 cases of other serious adverse events.

19 And using a proportional reporting ratio,

20 we come up with a proportional reporting ratio of 17,

21 which far exceeds the three that one would normally be

22 seeing, all right?

23 And therefore would potentially represent

24 the signal. Given that people would very likely look

25 at the denominator effect, which is clearly evident in

1 this, and one would assume that the signal is probably

2 an erroneous signal.

3 But at the same time when does one turn

4 off the system in order to stop looking for a signal

5 because this was a false signal. That is my question.

6 The second point is that there are false

7 negatives associated with data mining, especially as

8 it relates to the utilization of MedDRA. I presented

9 a paper at the annual DIA meeting last year which

10 looked at two specific conditions.

11 One was member proliferative glomera

12 nephritis, a rather rare occurrence, but is associated

13 with heroin necropsy. The other was TTP, again a

14 rather rare entity, but is associated with some of the

15 anti-platelet agents.

16 In neither of those cases could you use

17 either the MedDRA term itself at the low level,

18 preferred term, high level, high level group term, or

19 at the SOC level, to make sure that you had all of the

20 cases and eliminated appropriate cases that did not

21 belong there when you had appropriate case

22 definitions.

23 So one really needs to be very, very

24 careful about the data mining considering, one, how

25 has the terminology been put in, and two, what do you

1 do with false positives, and how do you turn off your

2 risk management program when a false signal is

3 identified.

4 DR. GALSON: Who would like to address

5 that?

6 DR. SZARFMAN: Can I say something? May I

7 -- I brought a computer, a laptop, and we can look at

8 the results that data mining show for the specific

9 examples, and then let's see how data mining

10 performed, because we need to look at the data.

11 DR. GALSON: So you are going to do that

12 in the back room?

13 DR. SZARFMAN: Yes.

14 DR. GALSON: Wow. Full-service FDA here.

15 DR. GOLDMAN: I would like to make a

16 comment. Steve Goldman.

17 DR. GALSON: Yes.

18 DR. GOLDMAN: We have been hearing very

19 good points of view obviously from Dr. Goldsmith, and

20 Dr. Szarfman, and also Dr. Faich. I am going to do

21 the same cautionary note that I am going to say about

22 MedRA.

23 This is a technique. This is a tool. It

24 does not replace the greatest tool of all, the human

25 brain. It does not take away what we have learned in

1 50 years of pharmacovigilance and how we look at each

2 individual case.

3 And I think if I may point out that if we

4 can take down the level of the heat and show a little

5 light that this is another tool that we are using to

6 look for possible signals to use the dynamic database,

7 such as the AERS database, as to look for

8 possibilities.

9 And I think that in that sense I think we

10 have more of a consensus than we have disagreement

11 about how the actual application should be.

12 DR. GALSON: Dr. Faich, did you want to

13 contribute something to that?

14 DR. FAICH: Yes.

15 DR. GOLDSMITH: Can I follow up?

16 DR. GALSON: Wait just one second.

17 DR. FAICH: Can I just say that I don't

18 want to be misunderstood. I am not opposed to data

19 mining, and I think it is an interesting technique and

20 a research tool that is in development. So I think

21 that it does have utilitarian interest.

22 The second point that I would simply make,

23 and I was trying to make it by giving that Yogi Berra

24 comment earlier, was that most signals I believe will

25 come from alert safety monitors either at the FDA or

1 at companies who are at the intake end of important

2 reports.

3 And that the main triage tool is to look

4 for a serious unlabeled. It doesn't help very much

5 for a serious labeled increased frequency, and I think

6 we all know that, and I believe that is one of the

7 main utilities that data mining can help us with.

8 I also would like to point out that Lou

9 Lisana (phonetic) many, many years ago said that the

10 most important source of new information about safety

11 for drugs is the alert observant clinician who sees

12 something out of kilter.

13 And I think the same thing is true for a

14 safety monitor inside the FDA or perhaps maybe more

15 importantly inside a manufacturer who has launched a

16 new drug and now receives reports that he didn't

17 expect to receive.

18 The issue of whether false signals will

19 occur or will be lessened because those who use this

20 steel-edged axe are indeed sophisticated and will go

21 back and review the data is dependent upon whether

22 they have the data in-hand or not.

23 My fear is that decisions or alerts will

24 be made based on only the electronic data, and not a

25 more complete clinical review of individual cases once

1 a preliminary finding has been found.

2 That is exactly what WHO has done, and

3 that is exactly what happened in the case of Claritin

4 that I cited, and that is exactly what happens when

5 attorneys troll through the database and do crude case

6 count, such as prozac and suicide.

7 DR. GOLDSMITH: Just a follow-up to

8 Steve's comment and also to Gerry's. I think that it

9 is very important that we keep our eye on the ball and

10 watch what is happening to the signal to noise ratio,

11 and we are increasing to a large extent the amount of

12 noise.

13 Yes, we are increasing the signals, but we

14 are increasing the noise to signal ratio so much that

15 we won't be able to find those signals terribly

16 effectively.

17 And the same thing is true, for example,

18 of the data that Steve presented. Yes, there was a

19 17-fold increase in reporting following the Rhode

20 Island program, and there was a 9-fold increase in

21 serious reporting, but 17 versus 9, that seems that

22 there was a lot more of a bigger increase in the non-

23 serious.

24 DR. GOLDMAN: I will respond to that.

25 There was also 31 previously unsuspected unlabeled

1 serious events that were reported. I will stand with

2 having to make the choice between getting more

3 possibility of drops into a system if it were to

4 enable me to get things that I didn't previously know

5 about, and that is my philosophy in terms of that.

6 DR. GALSON: Okay. Let's move on. Next,

7 Dr. Szarfman.

8 DR. SZARFMAN: Well, I will try to answer

9 some of the things that were said here. First of all,

10 data mining is a tool. It is not the only tool.

11 There are different tools. Data mining is a tool, but

12 we have

13 -- I don't think that there is a chance again that we

14 are having this meeting at the National Transportation

15 and Safety Board.

16 The human brain is very important, but we

17 have and we not always remember to fasten our seatbelt

18 and we need a light to let us remember that we need to

19 do so; and to airplanes that are maybe too close, the

20 human brain, of course, you know, of course -- you

21 know, we might compare that with other things that we

22 need to be alerted.

23 And this is essentially what we are trying

24 to do. Data mining is not data dredging. We are

25 applying one statistical method to the whole database,

1 and there is not a problem of multiple logs, because

2 using this method, you can do multiple comparisons.

3 And to include all of the contents that

4 you need to make the assessment. We don't have

5 backgrounds, background information for specific

6 diseases. And it will take a long time to get that

7 part down.

8 If we can analyze every single drug used

9 to treat that indication, and then it will keep us

10 informed as to which level of signals you have for a

11 certain indication.

12 We have also the human brain, and we have

13 in the database over 50 quadrillion combinations for

14 drugs. This means a 5 with 16 trailing zeros. I

15 don't think that the human brain can efficiently keep

16 up with these.

17 You know, if you don't have the gold

18 standard for these, and what the profile of every

19 document, and we don't. You know, that's why we need

20 systematic approaches.

21 DR. GALSON: Okay. Can we move on to the

22 next. Thanks, Dr. Szarfman. I want to just use one

23 of the questions that came in on a card. We have

24 heard the call to capture safety data electronically.

25 How can this happen with the industry

1 pushing back on Part 2 of HIPAA, and other security

2 issues are implied in this question as well. Do one

3 of the members of the working group want to address

4 that?

5 MS. CIAMPA: Well, first of all, I guess--

6 DR. GALSON: If you could identify

7 yourself, please.

8 MS. CIAMPA: I'm sorry. I am Aileen

9 Ciampa and I am with the Office of Regulatory Policy

10 in CDER. First of all, I would just note that with

11 regard to Part 11, the commentor might be aware of

12 this, but FDA did just recently issue a guidance

13 document regarding our approach to Part 11.

14 And in trying to make sure that it is

15 consistent with the adoption of new technologies, and

16 that is an ongoing process. And more broad based, I

17 guess I would just note that with regard -- and I am

18 not a HIPAA expert, and so there might be people here

19 who can speak better to that.

20 But I would just note that both patient

21 privacy concerns and concerns for data integrity and

22 record integrity have always been present in the

23 collection of safety data and the transmission of

24 safety data.

25 And so although I think the movement to

1 the electronic environment might have implications for

2 new regulatory schemes, and might pose additional

3 concerns, that the fact that those concerns, and

4 particularly the patient privacy and data record

5 integrity, should not prevent us from moving and

6 adopting new technologies.

7 DR. GALSON: Okay. Anybody else want to

8 address that issue?

9 DR. VASHISHTHA: A new issue.

10 DR. GALSON: Any other questions on the

11 security and those related issues before we move on to

12 something else. Okay.

13 DR. VASHISHTHA: Just a question for some

14 of the members of the working group possibly. One is

15 that there might have been already done some

16 evaluation of the algorithms and causality assessments

17 used in Europe or globally, and secondly, whether if

18 data mining is only one of the tools, then defining a

19 signal as an apparent access of adverse events in

20 association with a product sort of biases us towards

21 more of a data mining approach.

22 What about redefining a signal as an

23 apparent risk associated with the products? Any

24 comments on that from the working group members?

25 DR. GALSON: Dr. Chen.

1 DR. CHEN: We are aware of many

2 publications and many individual's proposals for the

3 criteria of causality assessment, and that we have not

4 found a consistent way that can help us as a standard

5 or a guideline to use currently.

6 So we are exploring all these

7 possibilities and getting inputs from you all. I

8 think that your suggestion was wonderful, to look at

9 the European ways, and if that is something that most

10 people think is applicable to us, we certainly will

11 consider it.

12 And the second question was about the

13 signal difference?

14 DR. VASHISHTHA: (Off microphone) on the

15 first question, also about using the existing database

16 to perhaps evaluate the appropriateness of some

17 algorithms with respect to that.

18 DR. CHEN: I think that we are all

19 exploring all possible tools helping us out to

20 identify the signals, and human brains are limited in

21 some ways, but I think that in the process that we

22 like to prioritize different kinds of signals that we

23 like to see as a work-up first.

24 So I think that is something that is only

25 probably reasonable for such a big volume of

1 information coming into the agency every day. So our

2 priorities usually are the more serious ones, like on

3 liver failure, and severe skin reactions, and such as

4 all of the events mentioned in the medically

5 significant events, and we pay more attention to that.

6 And data mining, as we said before, is an

7 adjunct to that to help us out to identify signals

8 that human eyes cannot detect right away. As far as

9 the definition, I think we are open to suggestions on

10 access to a certain degree, and nobody knows when for

11 a new drug that just came on the market with new

12 populations, and that we have never seen before, maybe

13 one case of a serious event is worth looking for and

14 looking into a further investigation.

15 But for other drugs, you probably need a

16 lot of events to support the hypothesis that the drug

17 really caused that. So I think this all needs to be

18 considered in here on how much is access and what is

19 the threshold for a signal detection.

20 DR. GOLDMAN: There is a very famous law

21 called Sutton's Law; you go where the money is, and

22 with the always expedited reports under the new

23 proposed rule, with the designated events which

24 triages errors, we are talking about scarce resources,

25 scarce time, and yet public safety.

1 And the idea that we look for the most

2 serious events -- and again the concept of risk, well,

3 that entails benefit risk. When you are looking at

4 signals, I think we should be looking at -- my

5 philosophy is are the actual events themselves are the

6 ones that are the bad actors, the ones that we know

7 about.

8 The ones that tend to be lethal, in

9 particular, and then tie that on in terms of that. I

10 gather, Min, that is the philosophy?

11 DR. CHEN: Yes, I agree.

12 DR. GALSON: Okay. Let's move to the

13 question over here at the mike.

14 MR. HARVEY: Yes, in a complex area where

15 we are trying to move the ball forward in a very, very

16 unknown way, I think it is often -- I'm sorry, Bill

17 Harvey from Life Free Technology.

18 It is very often worthwhile to keep things

19 as simple as possible. And the confusion between

20 large simple safety studies and Phase IV studies has

21 been very difficult for me to comprehend, and

22 especially in the context of a statement that was made

23 on Monday from Dr. Paul Stolley, who said that 13

24 percent of Phase IV studies have been completed.

25 And implying that fully 87 percent of

1 those studies have not been completed, and given the

2 expertise in this room either from PhRMA, from the

3 FDA, or from others, I think that is worth discussion.

4 DR. GALSON: Do you want to form it into a

5 question?

6 MR. HARVEY: What is going on?

7 DR. GALSON: Who are you asking?

8 MR. FENISHELL: May I speak to that? This

9 is Bob Fenishell, now a consultant, but at FDA for a

10 long time. We looked into this in the division when

11 those data were being gathered, and so we went back

12 and looked at all of the Phase IV commitments that we

13 had gathered from sponsors, and then looked at indeed

14 what fraction of them had been completed, and

15 certainly the figure of 13 percent is consistent with

16 our experience.

17 It was not a large fraction, and then we

18 said, gee, how do we feel about this, and just looking

19 at the raw numbers one must indeed ask as the

20 questioner asked what is going on.

21 But in fact you start looking at one of

22 them after another, and you know, that was a pretty

23 stupid thing to ask for. And that other thing, you

24 know, that was a pretty good thing to ask for. That

25 would be really interesting to have.

1 But we now realize that it is just

2 impossible to collect that data, or you know, that was

3 probably a pretty good idea at the time, but it has

4 been superseded by events.

5 We no longer believe that that is a

6 significant problem with that class of drugs, and so

7 it perhaps was appropriate, but it is no longer to

8 push the sponsor to find out whether it is happening.

9 And so at the end of it, and of course

10 there was self-serving and one may have some

11 skepticism about the freedom from bias of our process,

12 but we thought that he had not done badly.

13 DR. GALSON: Victor.

14 DR. RACZAKOWSKI: I am Victor Raczkowski

15 from the Office of Drug Safety in the Center for

16 Drugs. The only thing I will add is that Phase IV

17 studies are not always safety studies. Many of them

18 are pharmacokinetics studies, and there might be

19 chemistry manufacturing types of studies, and so just

20 keep that in mind as you talk about Phase IV studies.

21 DR. NELSON: I might also be able to shed

22 some light on that 13 percent figure, because that

23 comes from an internal regulatory research report that

24 Dr. James Grumlish and I did in the early '90s. I

25 have two things to say about it.

1 One is that it was on a time frame -- I

2 think the data was pulled from the late 1980s, and

3 into early 1990, and so that data are real, and it did

4 look just at the safety Phase IV studies, and not all

5 the other kinds of Phase IV studies.

6 And that is a very valid point, Victor,

7 because actually most of the Phase IV studies are

8 biopharmaceutical, and long term cancer studies, and

9 all kinds of other things.

10 But that study was submitted to the agency

11 and was actually submitted down to Congress prior to

12 the FDAMA Act being developed. It also served as the

13 initiator for the Phase IV tracking system that you

14 have at the agency, as well as the stipulations in

15 FDAMA about putting the Phase IV requirements for

16 approval in the approvable letter and monitoring those

17 after.

18 And so it did have an impact. I think

19 there has been some remedy at the agency for that

20 deficiency, and I would love to see that study redone

21 to see if the rates have gone up in the last decade.

22 DR. GALSON: Just one note on Phase IV

23 studies, is that we are about to activate in the

24 agency a new website which is going to allow people to

25 search those studies directly; and without spending a

1 lot of detail on this, I think all the comments that I

2 have heard are fairly or are valid and a lot more as

3 well.

4 The 13 percent number I think is really

5 sort of deceptive, a deceptively simple description of

6 what is actually going on there, which is quite

7 complex. But we will be making more information

8 public about what is actually going on with those, and

9 the follow-up on those studies, soon.

10 Dr. Stephenson, I think we will take this

11 last question and then if we could hold all additional

12 questions, we have got a lot more time in the

13 afternoon than the morning for this type of

14 discussion. We will have time for all the questions.

15 DR. STEPHENSON: Okay. My comment, and I

16 guess perhaps a question for FDA, has actually been

17 touched on slightly two questions ago. And that

18 relates to signals, and the description of what a

19 signal is in the concept paper, I think it needs to be

20 flushed out more, especially in just listening to this

21 discussion, everyone seemed to use the word signal in

22 very different ways.

23 And I have heard possible signal,

24 potential signal, and signal, used sort of

25 synonymously sometimes. I think the description of a

1 signal is good, but it is a broad description that I

2 think encompasses several degrees of strength of a

3 signal.

4 Many of us are used to having a very low

5 threshold for calling something a signal, and which

6 would then trigger you to take further action in terms

7 of pulling data together and better assessing the

8 situation.

9 But more times than not those sort of

10 signals end up being non-issues. And so I raise this

11 because when you read further on in the document,

12 there is a section that refers to reporting the

13 signals to FDA, and the expectation that if there is a

14 signal that the FDA would expect the company to submit

15 a fairly comprehensive report, which would include

16 pre-clinical findings, and a discussion of potential

17 risk management plans, if that is appropriate.

18 And even further in the document there is

19 some language to suggest that risk, management plans

20 may be a way to mitigate a signal, which sort of

21 brings you to the point of a signal being used almost

22 synonymously with risk. I wonder if Dr. Chen, if you

23 can comment on that.

24 DR. CHEN: I think the points are well

25 taken. We have not addressed the difference between

1 signals and risk and certainly we are still at a level

2 with the signals on what does that mean when we

3 identify from the database.

4 And then what do we do with the signals,

5 and what we are hoping is that the sponsor can help us

6 out to fill the gap of when do you find that the

7 signals are useful to become a risk, a real risk, and

8 that something has to be done with that risk.

9 And how big is the risk, and again it

10 depends on how big is the signal. So hopefully this

11 afternoon some of the studies and some of the analyses

12 of the information will help to bring the gap a little

13 bit closer.

14 However, I think we need a lot of help

15 from you all experts out there to help us out to do

16 that. So putting in writing what you think about it

17 and how we do that, I am really open.

18 DR. GALSON: Dr. Braun.

19 DR. BRAUN: Miles Braun, CBER, FDA. That

20 last point was well taken. I think that to take a

21 step back from the discussion today, this is a concept

22 paper and the idea was to lay out different ideas and

23 get discussion.

24 You have homed in on what is a good point,

25 and I think that having reviewed the literature

1 somewhat in this area, I can say that there is not an

2 agreed upon definition outside this room.

3 And that the one that was laid out of an

4 excess of beyond the expected in the reports certainly

5 raises the issue of what one would expect. Some have

6 taken exception to the expected values that are

7 generated through automated data mining approaches.

8 And there is certainly or that is

9 certainly a well-taken point. So this needs to be

10 flushed out further. It is in my view, it is a key

11 issue, and I will throw out an additional thought, is

12 that the actual actions, or actually two thoughts.

13 One is that actually because we have

14 different views of signals, the signal can be

15 conceived of to some extent, and I am not trying to be

16 subjective about this; but in the eye of the beholder.

17 Let me give you an example. The physician

18 who takes the trouble to fill out a MedWatch form

19 because a patient that was being taken care of had a

20 terrible adverse event to that physician, they are

21 signaling to the FDA.

22 And to somebody that sits at the FDA and

23 sees 300,000 reports a year, this particular event may

24 not seem to be a signal. And there are very many

25 other people involved in the process.

1 So I think that we need to think about who

2 actually is the beholder, and the reason that I think

3 that is important is that each person has a different

4 action that he or she can take.

5 That physician's action is to notify the

6 FDA. As an FDA person, we have certain actions, and

7 our agency can take certain actions, and similarly at

8 a company, you have different repertoire of actions

9 that are available to you.

10 And I think that it will be helpful if we

11 can blend in some of this, and as you said, if we can

12 go into this in a little more depth, and we are only

13 laying out the concepts.

14 And to try to construct this signal idea

15 in a way that will be helpful in terms of actions that

16 we all can take.

17 DR. STEPHENSON: Perhaps just one other

18 point, and I appreciate the responses. I agree with

19 what you are saying. I think one of the things that I

20 was reacting to in the concept paper is that there was

21 a discussion of reporting, which is really not a

22 concept.

23 And I think that if you start talking

24 about the expectation that companies should report

25 signals to the FDA, you are going to get barraged with

1 things that you probably would not consider signals,

2 just because companies are going to want to be in

3 compliance. So that was one of the things that I was

4 reacting to.

5 DR. GALSON: At the table, Dr. Faich.

6 DR. FAICH: Yes, I was just going to make

7 a comment and an observation, and it links to some of

8 the discussion yesterday of Level 1, Level 2, and

9 Level 3, and that the levels analogy takes on a life

10 of their own.

11 And I would like Anne Trontell's comments

12 of let's call them Georgia, Betty, and Sam, or

13 something. We had a similar experience at one point

14 long ago that is worthwhile sharing, and that is that

15 we started talking about alerts driven out of the

16 adverse drug reaction reporting system.

17 And the term alert is a potentially

18 evocative one, and that was the problem, and we

19 learned that, and we changed the terminology to

20 monitor to adverse reactions. And then of course we

21 were then accused of bringing these from the outer

22 reaches of the solar system and Mars.

23 But nonetheless, I mean, the point of that

24 was to get away from a term that could be

25 misinterpreted, and to go to a more neutral term, and

1 I wondered whether thinking about not using the term

2 signal, but using some other term, might be

3 considered.

4 DR. BRAUN: But not Georgia, or Betty.

5 Those are taken.

6 DR. FAICH: Miles, to follow up on your

7 comment asking about causality, I have not got the

8 exact reference about the authors, but I know that

9 there was a look at the French causality assessment

10 method, versus physicians reporting.

11 There was a tremendous rush to connect

12 between the relative weight that the reporting

13 physician placed on particularly dechallenge, as

14 opposed to the algorithm system, and very much

15 pointing to what you said, Miles, as to what the

16 treating doc, or the health professional would place,

17 as opposed to a straightforward algorithm, and that is

18 one of the reasons again that we don't advocate for

19 that, because the weighting is different.

20 DR. GALSON: Is this a follow-up?

21 MR. FENISHELL: Yes, it is. Yes, it is.

22 DR. GALSON: Okay.

23 MR. FENISHELL: I think what is the

24 problem here --

25 DR. GALSON: Please identify yourself.

1 MR. FENISHELL: Oh, I am still Bob

2 Fenishell. I think the problem here is what in

3 computer terms would be called a missing level of

4 indirection, and what I mean by that is that what you

5 have defined as the notion of signal is that signal is

6 an apparent excess of adverse events.

7 And the focus has been continually on this

8 idea of apparent. Well, is it real, and can we verify

9 it by data mining, and can we verify it by this or

10 that. But if we verify it, well, then it is adverse

11 events.

12 Now, the fact is that is not what in my

13 experience most of the Phase IV stuff of the post-

14 marketing stuff that I used to see coming across my

15 desk at the agency consisted of.

16 It consisted of things that were not even

17 adverse events. Prolonged QT is not an adverse event.

18 A doubling of hepatocellular enzyme levels is not an

19 adverse event. The patient doesn't suffer from any or

20 either of those things.

21 And here are indeed drugs that cause

22 colossal changes in the hepatocellular enzymes, or of

23 QT -- not very many, but there are some -- without

24 ever causing adverse events.

25 The thing is that you are looking for an

1 apparent excess in one phenomenon, and then entirely

2 independent -- you know, once you decide, yes, there

3 really is an excess amount, and it is not apparent.

4 Then you have to decide, well, is that a

5 phenomenon that in fact is associated with a

6 phenomenon that we care about, or is that just an

7 anomaly.

8 If you look at most big trials, you will

9 see an excess of something. You will see that, oh,

10 yes, the people getting active treatment had slightly

11 higher bilirubin than the people who didn't, let's

12 say.

13 And you say, oh, well, that is something

14 that makes you think about some liver disaster. Well,

15 in the CAPRI trial, the Clopidogrel patients had

16 slightly higher bilirubin, and it was something in the

17 third decimal place.

18 But with 19,000 patients, you can show

19 that. And that was just absolute garbage. I mean, it

20 didn't mean anything. I am sure that it is true, but

21 it just doesn't mean anything. If you say that a

22 signal is -- and I am not sure what the word is, but a

23 signal is something that catches your eye, and it may

24 be real.

25 And now, first of all, it is appropriate

1 to see if it is real, and secondly, it is appropriate

2 to see if there was any reason to worry about it.

3 Those are two separate processes, and I think that a

4 lot of the discussion has been concerned about the

5 onus of, well, yes, it really is real, and there is a

6 second step here.

7 It really is real, but perhaps the answer

8 is so what.

9 DR. GALSON: Thank you. Does anybody want

10 to comment on that? That is sort of an open-ended

11 point. All right. We are going to break for lunch

12 now, but before we break for lunch, I want to let

13 folks know that if you didn't have a chance to ask the

14 question that you wanted to ask, we will have plenty

15 of time this afternoon.

16 I wanted to leave a question on the table

17 particularly for the PhRMA folks so that they have

18 something to do during lunch. They asked us what

19 criteria should be used to determine whether a

20 pharmacovigilance plan describing firm efforts above

21 and beyond post-marketing spontaneous reporting is

22 warranted.

23 And my lunchtime assignment for them is to

24 provide us an answer to that question, and then we

25 will try to comment on that. And that is where we

1 will start the discussion. Thank you.

2 (Whereupon, at 11:40 a.m., the public

3 workshop was recessed.)

1 A-F-T-E-R-N-O-O-N S-E-S-S-I-O-N

2 (1:12 p.m.)

3 DR. GALSON: We are going to start out

4 this afternoon with a number of presentations, first

5 from the FDA, and then members of the public, and then

6 we will move into the discussion, and question and

7 answer periods.

8 So our first presentation today is from

9 Dr. Judy Staffa from the Food and Drug Administration.

10 She is an epidemiology team leader in our Office of

11 Drug Safety. Thanks, Judy.

12 DR. STAFFA: Good afternoon. I have the

13 privilege of getting up here on the third day of a

14 three day meeting on Friday afternoon, at one o'clock,

15 right after lunch, and giving FDA's last presentation

16 on this concept paper.

17 So I feel very privileged to do that, but

18 I also have been told that I have the authority if I

19 need to, to burst into song at any time, and that way

20 if I see everybody kind of nodding off, I may do that.

21 And so since I am a really bad singer, I

22 will try to keep my remarks brief so that we can get

23 to the input that we are really here looking for

24 today. But if you can try and stay with me, that

25 would be great, too.

1 Okay. What I am going to try to do today

2 is give you an overview of the section of the concept

3 paper that really tries to focus on what we are

4 looking for from sponsors about observational data

5 that we receive, whether it is in the form of study

6 results or protocols, that go beyond the case reports

7 that we talked about this morning.

8 Pharmacoepidemiologic studies obviously

9 can be done at any time, but typically there are two

10 times when these studies are usually recommended or

11 most appropriate.

12 The first time is typically right after a

13 product is marketed, or at the time of marketing, and

14 the purpose of those studies is to further

15 characterize potential safety signals that may have

16 been identified during the controlled trials.

17 This is typically important, particularly

18 of interest in patients that may be chronically

19 exposed once the drug is marketed, or product is

20 marketed I should say, or in patients with co-morbid

21 conditions or concomitant drug therapies.

22 Another time when pharmacoepidemiologic

23 studies might be recommended would be after a safety

24 signal is identified once the drug has been on the

25 market. And as we talked about this morning, many

1 times a signal is identified first from case reports,

2 and then that signal, we try to put that signal into

3 an appropriate context using drug use data and

4 sometimes calculating reporting rates as you have

5 heard about this morning.

6 We all recognize that reporting rates are

7 very preliminary, crude tools, with which to describe

8 and try to characterize the risk, but we really need

9 to move to the next step to pharmacoepidemiologic

10 studies in order to calculate an incidence rate.

11 Just very briefly, and I know that this is

12 overkill for this crowd, but very briefly there are

13 several pharmacoepidemiologic study designs that are

14 commonly used in pharmacoepidemiology.

15 The first is a cohort study, in which

16 patients are identified based on exposure to a medical

17 product, and then they are followed over time to

18 estimate the incidents of various events.

19 These studies, or the main advantage of

20 this kind of study is that an incidence rate can be

21 generated for various events.

22 A case control study is a design in which

23 patients are identified based on the presence or

24 absence of a particular disease outcome, and then

25 their history is examined for the use of different

1 medical products.

2 Case control studies are very useful

3 because they can help us identify risk factors, in

4 addition to the relative risk for an event occurring.

5 Another design, called nested case control, is a

6 hybrid of the two designs, in which a case control

7 study is nested within a larger cohort of medical

8 product users.

9 One of the main advantages of this design

10 is that it allows you to both calculate an incidence

11 rate, but also to then examine and explore risk

12 factors.

13 And then there are other hybrid designs,

14 and as the field progresses more and more designs are

15 tried out, and some of them end up being very

16 appropriate for use in pharmacoepidemiology. We

17 encourage sponsors or researchers doing these kinds of

18 studies to consider what design might be best for

19 answering the questions at hand.

20 In 1996, the International Society for

21 Pharmacoepidemiology published a document called

22 "Guidelines for Good Epidemiology Practices for Drug,

23 Device, and Vaccine Research." And this document can

24 be found or can be linked to through the ISP website.

25 We find this document very helpful and

1 very useful, and would encourage people to consult

2 this document when undertaking a pharmacoepidemiologic

3 study.

4 For the purposes of our concept paper, we

5 selected just the minimum requirements that we would

6 like to see when protocols are submitted, or studies

7 are submitted that are just worth going over today

8 very briefly.

9 First, we would like to see a clearly

10 specified study objectives, and these objectives

11 should be tied in with the analysis so that it can be

12 easily seen how the study can achieve and answer the

13 study questions defined.

14 There should be a critical review of the

15 literature that can take advantage of earlier

16 experience with either the drugs or products of

17 interest, and as well the outcomes of interest.

18 And we would like to see detailed research

19 methods. That includes a detailed description of the

20 study population, including comparator groups that

21 have been identified as being relevant, as well as

22 operational definitions of both exposure and outcome

23 that will be used.

24 We would like to see a full description of

25 data sources that are going to be used. We try very

1 hard in the Office of Drug Safety to get out to

2 meetings and to see what kinds of data sources are

3 available, because that is always a very evolving

4 field.

5 But we often run into protocols where

6 there are -- into data sources that are unfamiliar to

7 us, and we welcome detailed descriptions of those data

8 and the streams that generate them.

9 We would like to see projected sample size

10 calculations, and the associated statistical power.

11 As you know, in the world of rare events, power

12 becomes a very important issue when determining

13 whether to go forward with the study.

14 And finally we would like to see detailed

15 methods for how data will be collected, and how it

16 will be managed, and how it will be analyzed. In

17 pharmacoepidemiology, we use a lot of automated data,

18 whether it is administrative claims data or

19 computerized patient record data, but these present

20 interesting challenges for some of the studies that we

21 do.

22 And what we have listed here are just some

23 factors as sponsors or researchers are choosing the

24 appropriate database to study the question of

25 interest, and we encourage you to openly discuss these

1 in the protocols, in terms of how you came to the

2 choice of the database where you have decided to

3 conduct your study.

4 Some of these factors include the

5 demographics of the patient population. Clearly, if

6 you are studying an issue in children, or in women,

7 you would want to select a data resource that captures

8 a lot of information on that subpopulation.

9 The turnover rate or mobility of the

10 population in and out of the database is critical,

11 particularly for studies that are looking at

12 cumulative risk. As you know, many American health

13 care is very fragmented.

14 People have a great deal of mobility, and

15 since many of our data sources are based on health

16 insurance plans, we are limited to the length of

17 follow-up that we can actually capture people, which

18 is as long as they stay in the system.

19 Plan coverage of study medications is

20 another crucial element to look at. You need to

21 understand what both the uptake of a product is, and I

22 know that was mentioned this morning, but also whether

23 or not a plan will cover particular products when you

24 are looking to study them.

25 There are many plans that will not cover

1 products they consider to be lifestyle drugs, such as

2 weight loss products, or products used for erectile

3 dysfunction, and it is an important consideration to

4 understand.

5 We would like to have detailed information

6 on the size of the population available for study, and

7 in automated data sources that is a particular issue,

8 because rather than simply get a first take which says

9 that, gee, I have 10,000 patients that had at least

10 one prescription for this drug in the database, we all

11 know that by the time that we get done apply different

12 enrollment and eligibility criteria, that 10,000

13 number may be quite substantially lower.

14 And any estimation that you can give of

15 that quantity would be very helpful in trying to get a

16 realistic sense of how large the population will be.

17 It is important to understand whether the outcomes of

18 interest are actually available in the database, and

19 one very good example of this is death.

20 Death would seem to be a very

21 straightforward outcome to study, but in an

22 administrative claims database, unless death happens

23 in a health care facility where care is being

24 provided, it may not be captured.

25 A death may just be the absence of

1 continued claims, and so it is important to consider

2 whether the outcomes that you want to capture are

3 actually available in the data.

4 It is also important to understand whether

5 the outcome of interest is actually coded, or there is

6 an specific ISD-9 code since many systems, at least

7 administrative claims systems, use this coding system,

8 and it is important to know if the code that you are

9 looking for actually does exist, whether it exists in

10 a code with about nine other diseases as well or to

11 understand whether it is even captured in a specific

12 code.

13 And finally we feel that it is very

14 important to understand whether or not you can access

15 medical records through the data source that you are

16 using. Unlike some of our colleagues in

17 pharmacoeconomics and health services research, we

18 often find the need to validate the diagnostic data in

19 the claims with medical record data to truly

20 understand whether the event that we think has

21 happened has actually happened, and has happened at

22 the time at which we believe it has occurred.

23 Now, with new regulations, it can be quite

24 challenging to gain access to medical records, but

25 there is now about 20 to 30 years of research using

1 these kinds of databases, and many authors have been

2 publishing when they are able to validate claims.

3 And it is important to go to the

4 literature and understand if the outcome that you are

5 looking at has already been studied and validated in a

6 system like the one that you are planning to use, and

7 that would be useful information for us all to know.

8 Moving on to another source of

9 observational data, I am going to talk a little bit

10 about registries. Now, we have defined a registry for

11 the purposes of the concept paper as a systematic

12 collection of defined events or product exposures in a

13 defined patient population for a defined period of

14 time.

15 Registries are typically prospective in

16 nature, collecting data on patients that are

17 identified either as having a particular disease, or

18 as receiving a particular treatment.

19 And typically they are used to collect

20 information that may not be able to be collected

21 within standard systems. Like good

22 pharmacoepidemiologic study protocols, good registry

23 protocols should include clear objectives of what is

24 to be gained, as well as a literature review.

25 Now, the detailed research methods really

1 should focus quite a bit on patient recruitment and

2 follow-up, because that is clearly of interest to

3 understand whether the patients in the registry will

4 truly be representative of patients out there.

5 And if they are not, to understand in

6 which ways they are not. Projected sample size really

7 is an important issue, particularly with regard to

8 attribution and loss to follow-up, and any estimations

9 to that effect can be very crucial to how useful the

10 registry might be.

11 Methods for data collection management and

12 analysis is important also. As was pointed out this

13 morning, there are often not comparator groups for

14 registries, and it is important to understand if there

15 is a way to develop or to anticipate a comparator

16 group, or what background rates may be relevant for

17 that particular registry.

18 If possible, we would love to see the data

19 collection forms that have been designed to collect

20 data on the registry, and help to be able to

21 contribute to those.

22 And also validation of findings,

23 particularly if the findings are patient reported, and

24 whether there is a plan to validate those with health

25 care providers.

1 Another strategy that is used to collect

2 observational data are surveys. Surveys are not

3 really -- surveys are not studies that could actually

4 determine an incidence rate, or a cause and effect,

5 but a survey can be very helpful to provide

6 supplemental information with regard to a drug safety

7 issue.

8 You can use surveys to assess knowledge of

9 both patients and providers about labeled events to

10 better understand medical product use; to assess

11 compliance with risk management programs as was

12 discussed yesterday; or in the world of medication

13 errors, surveys are often used to try to survey

14 pharmacists to understand more about the potential for

15 proprietary trade name confusion before a product is

16 launched.

17 And again good survey protocols are very

18 much like other study protocols. Detailed objectives

19 and methods are very helpful to have in the protocol

20 or in the finished report, with particular focus on

21 where the patient or provider sample came from.

22 And to understand the pharmacist, or

23 patients, or providers that were sampled, what was the

24 sampling universe, so that we can better understand

25 whose views they represent within the profession.

1 Projected sample size and methods again,

2 and again if there are survey instruments that can be

3 provided, then the reviewers at FDA could do a much

4 better job of determining the validity of the

5 findings.

6 So what do we do once we have all this

7 observational data? Well, what we would like to do is

8 rather than have things come in kind of piecemeal,

9 because we all know that there is no one piece of

10 observational data that is really going to solve an

11 issue, or really give us an idea of exactly the

12 magnitude of a risk.

13 What we would like to see is a synthesis

14 of the information that is available; the case

15 reports, the drug use data, the pharmacoepi studies,

16 the registries, the publications, all of the data that

17 are available about that signal that can help us to

18 understand what in a total picture might be going on.

19 We would encourage sponsors to provide

20 their own assessment of the risk benefit profile based

21 upon that synthesis. And that synthesis should be

22 providing the limits and the strengths and limitations

23 of the different study designs and data sources used.

24 We would also encourage sponsors to

25 propose steps to further investigate if they feel that

1 further study is needed. And finally this is an

2 opportunity to propose risk management programs if

3 they appear to be appropriate given the signal.

4 And I would also in response to one of the

5 questions this morning, to me if a signal turns out to

6 not be a signal because of other observational data, I

7 would say that could be part of the synthesis as well,

8 and that that was really what the sponsor believes is

9 the take home message.

10 Then at the FDA, we can do our own review

11 of the synthesis, and we will be looking at probably

12 -- I am hoping the same things that you are looking

13 at, which is that we will be looking at the magnitude

14 of the signal and its precision; trying to see what

15 are the consistency of these findings across the

16 available data sources given all of their limitations

17 and more of the things that we know about them.

18 To be looking at the biological

19 plausibility of the event, and the seriousness of the

20 event, but also the degree of benefit that the product

21 offers; the availability of other therapies, and in

22 the event that things aren't to the point where a risk

23 management program is proposed, the ability of that

24 program to realistically be able to mitigate risk.

25 And now this is just a summary of the

1 three questions that I guess we have felt were

2 relevant, and I know that some folks have already

3 presented their thoughts on those, and we are very

4 appreciative.

5 But for the afternoon conversation, we are

6 very interested in hearing more about what folks think

7 about registries as surveillance tools, and when they

8 would be useful, and when they might not be.

9 Secondly, we are very interested in

10 hearing your thoughts about active surveillance

11 strategies, and when you feel that they might be

12 useful to identify events that are not reported to our

13 passive surveillance system.

14 And finally what circumstances do you feel

15 that additional pharmacoepidemiologic studies might be

16 useful, in addition to the circumstances that we have

17 described. Thank you.

18 DR. GALSON: Thank you, Dr. Staffa, and

19 now it is time for our presentations from the public,

20 and I am going to shift the order of the presentations

21 a little bit.

22 First will be Dr. Stemhagen, and then John

23 Ferguson, then Dr. Stephenson, and then Dr. Kahn.

24 So if Dr. Stemhagen could come up, please.

25 DR. STEMHAGEN: Thank you very much. I am

1 Annette Stemhagen, and I am Vice President of

2 Strategic Development for Covance Perry Approval

3 Services, and today I am speaking on behalf of the

4 International Society of Pharmacoepidemiology or ISPE.

5 As you have heard, if you have been here

6 the last few days, ISPE is a non-profit, international

7 professional membership organization decided to

8 promoting the science of applying epidemiologic

9 approaches to the study and value of therapeutics.

10 The society provides an international

11 forum for sharing knowledge and scientific approaches

12 to foster the science of pharmacoepidemiology. We

13 have over 700 members representing more than 45

14 countries.

15 Our members work in academic institutions,

16 in government agencies, and in industry, from for

17 profit and for not for profit organizations. The

18 specific backgrounds of our membership is quite

19 varied, with epidemiologists, biostatisticians, those

20 involved in medicine, nursing, pharmacology, pharmacy,

21 law, health economics, and journalism.

22 And I would like to thank the FDA for

23 allowing us the opportunity to provide comments on

24 this concept paper number three. The comments that I

25 am going to provide are based on feedback from senior

1 members of ISPE, and we will also be providing written

2 comments with a broader representation of our

3 membership as those comments are collected and

4 collated over the next week.

5 I would first like to say that we are very

6 pleased in this concept paper with the recognition of

7 the role of epidemiology and the value of the use of

8 observational data and tools for the understanding of

9 risk assessment and risk evaluation, and assessments,

10 excuse me.

11 It is not often that epidemiology and its

12 value is quite as recognized as in this document. So

13 thank you very much, and we are very pleased, and we

14 look forward to working with you on that.

15 I would like to begin with several areas

16 of the concept paper, where we either have comments or

17 some questions for the agency. I'm sorry, I am behind

18 in my slides. This is our website.

19 The first is a qualitative definition of

20 risk. From the epidemiologic standpoint, we recommend

21 providing a quantitative definition of risk in the

22 guidance document.

23 Risk is a measure of the frequency based

24 on an enumerator and dominator, and if you have ever

25 been around epidemiologists, we certainly live and die

1 by enumerators and dominators. So we are always going

2 to have a comment like this.

3 The enumerator in risk consists of the

4 number of people who develop an event, and the

5 denominator consists of the number of people exposed

6 per unit of time.

7 This can also be thought of as an

8 incidence rate over a particular time period. The

9 next thing that we would like to comment on is

10 expansion of the type of appropriate studies for

11 pharmacoepidemiology assessments.

12 We would suggest that the epidemiologic

13 assessment of safety end points is important and

14 should be further emphasized in this concept paper.

15 Also very important are studies of the natural history

16 of disease, as well as evaluation of drug use patterns

17 and patient characterization studies.

18 And that these studies should be done

19 proactively, and they will certainly assist the

20 sponsor in terms of gathering data should a risk

21 management program be required. So I think as Dr.

22 Jones talked about on speaking of concept paper number

23 one, really pharmacoepidemiology needs to extend back

24 as you have acknowledged into the development phase,

25 as well as the post-marketing.

1 And we would like to urge that sponsors

2 consider looking at natural history studies and

3 patient characterization studies so that you really

4 can gather background information that is going to be

5 critical as we go forward with risk management.

6 This serves to expand our understanding of

7 safety end points, and also helps put the spontaneous

8 reports in context as well. Natural history of

9 disease and safety study among users of available

10 therapeutics before your own product is on the market

11 should be initiated during early development.

12 Drug use in patient characterization

13 studies should be initiated during the development

14 stage of a new drug based on the therapeutic class,

15 and then continue after launch to further understand

16 drug use in diverse populations and across various

17 countries.

18 Similarly, evaluation of end points among

19 products in the therapeutic class and within diverse

20 populations are important, and I think we have already

21 acknowledged that as well during the discussions the

22 past few days.

23 Targeted safety studies should be

24 initiated shortly post-launch based on the

25 specifications of the development program and the

1 pharmacovigilance plan. The next comment, I am

2 actually going to deviate from my prepared script.

3 This has caused quite a controversy, in

4 terms of the definition of registry, and as the three

5 days have evolved, there are a number of ISPE members

6 here, and we have been discussing a lot the discussion

7 of registries.

8 I think we have here, because we couldn't

9 even agree among ourselves exactly what a registry

10 was, with a lot of contention between the definition

11 as presented by the FDA, a systematic collection of

12 defined events or product exposure, in a defined

13 population for a defined time period.

14 That is somewhat self-limiting. It does

15 describe, for instance, a birth defects registry,

16 which would be a collection of events, but it doesn't

17 necessarily describe all of the kinds of registries

18 which must be considered more like an observational

19 cohort.

20 So I think here, again just in evidence

21 from the discussions that the audience has been having

22 among themselves, that we need a better nomenclature I

23 think for registries.

24 It is not totally clear exactly what they

25 are, and based on my experience when we are talking

1 about registries in other countries, many countries

2 really shy away from actually even using the term

3 registry.

4 And it makes it even more muddied as to

5 what they are talking about. So I think that we need

6 to work together to get a better definition on that.

7 I am also going to talk a little bit about definitions

8 for pharmacoepidemiologic studies.

9 We recommend that in Section 4(a) about

10 pharmacoepidemiologic studies that they be classified

11 as pharmacoepidemiologic safety studies. Just for

12 clarification purposes, pharmacoepidemiologic studies

13 are not limited to safety studies as I am sure that

14 everyone knows, but include all observational studies

15 which describe the use and effects of pharmaceuticals

16 in a real world setting.

17 We also recommend that in Section 4(a),

18 where it is a definition of pharmacoepidemiologic

19 studies, that this be expanded to include post-

20 marketed data collection, in addition to the already

21 mentioned existing automated databases.

22 In several sections of the concept paper,

23 pharmacoepidemiologic studies seem to refer only to

24 those studies conducted in automated databases, and

25 that is probably not the intent, but it is certainly

1 could be construed that way.

2 Things like the large simple studies which

3 were discussed on day one in terms of premarketing,

4 can also be pharmacoepidemiologic or observational

5 studies.

6 So really the definition needs to be

7 expanded to include prospective data collection as

8 well. In terms of Section 5 regarding safety signals,

9 there is a list on lines 254 to 258 of five examples

10 of signals. It is very useful, and it probably covers

11 most of the options or types of safety signals.

12 However, we recommend expanding that list

13 to include a category of other risk factors, or risk

14 factors associated with the development of the adverse

15 event of interest.

16 Safety signals regarding specific

17 subgroups of patients, and that might be either age,

18 race, genetics, specific co-morbidities, or ways in

19 which the drug is used -- for example, dose escalation

20 may also be important.

21 So there are really other categories that

22 could be included in there. We would also like to ask

23 for a clarification of line number 319, point number

24 8, which says that analyses of the potential for an

25 excess of adverse events given the disease being

1 treated, such as might be observed in advanced cancer

2 or immunocompromised patients.

3 Our question is whether that is intended

4 to address analyses to identify the background risk of

5 adverse events in the disease being treated, which is

6 actually then point number 9.

7 We weren't really clear what the

8 distinction between those two are. And then looking

9 at all of those points together, we would like to make

10 the point that many of those are probably not

11 appropriate for analyses using spontaneous adverse

12 event reporting data.

13 But much more apply to analyses based on

14 other types of pharmacoepidemiologic data, either

15 prospectively collected or through a large automated

16 database.

17 The next point that we would like to talk

18 about is Line Number 352 actually, the relative risk

19 or odds ratio. The point mentions that relative risks

20 or odds ratios from pharmacoepidemiologic studies can

21 be used to evaluate causality.

22 Although risk ratios or odds ratios are

23 very frequently reported as an outcome frequency in

24 pharmacoepidemiologic studies, we would like to be on

25 record as noting that risk differences, and not

1 ratios, are best for evaluating the frequency with

2 which a drug causes or contributes to an event.

3 So in addition to an estimate of relative

4 risk, we would suggest that risk differences or

5 discussions of incidents rates in particular be

6 included in that section.

7 In the end the frequency with which an

8 event occurs because of a drug, along with other

9 characteristics of the event, must be weighed against

10 the benefit of the drug.

11 And I think this point again was discussed

12 several times over the last few days, but in addition

13 a risk management program needs to consider how the

14 risk associated with the intervention, which is

15 designed to reduce the risk of a particular event --

16 for example, the potential reduction of benefit

17 associated with the use of the therapeutic as a result

18 of the risk management intervention.

19 So we have to remember again that balance

20 of risk and benefit, and that if we are putting a risk

21 intervention in place, and we have to be sure what the

22 impact on the benefit is.

23 If we restrict things as an example too

24 much, then the benefit may be reduced with patients

25 not able to get access to the drug. The next is a

1 discussion of reporting of safety signals. In Line

2 374, Paragraph (d), and I think this is what Dr.

3 Stephenson had pointed out earlier as well, in the

4 reporting of safety signals, there really needs to be

5 a definition of when is a signal really a signal.

6 A lot of what is discussed may be our own

7 analysis to evaluate whether it is a signal or a false

8 positive. And the way that this paragraph is

9 described, it looks like hypothesis generating,

10 testing, prospectives of risk and benefit, and risk

11 management, all need to be done in a package and sent

12 to the FDA.

13 When in fact many of these things, or the

14 first part of this, will be done many times by a

15 pharmaceutical sponsor before the FDA needs to be

16 notified, and we are trying to rule out some things.

17 And in addition there are sequential --

18 there is a sequential order to this. You may not need

19 to get to all of these steps again if you have ruled

20 something out. So I think we are asking for a little

21 bit more clarification on that point as well.

22 In terms of surveys, I thought that this

23 was very interesting that this was included. You

24 don't often see again some discussion of surveys. We

25 would like to comment on the inclusion of surveys as a

1 risk assessment or risk management evaluation tool.

2 Many of the examples provided in the

3 document are actually cross-sectional designs. We

4 would just like to make note that longitudinal cohort

5 studies to evaluate changes in risk over time can also

6 be surveys, and so they don't need to be limited to

7 cross-sectional surveys.

8 And in talking about these kinds of

9 surveys, or some of the types of surveys that are

10 listed as examples, in addition to epidemiologic

11 expertise, which of course we believe is very

12 important in many of these analyses, it is going to be

13 important to include members of other academic

14 disciplines, such as social scientists, who bring

15 expertise in areas such as knowledge, attitudes, and

16 behavior surveys.

17 I think that this is an area where we as

18 epidemiologists probably are not as strong, or many

19 people that are not psychosocial epidemiologists are

20 not as strong.

21 And within pharmaceutical companies, or

22 within the FDA, I think we are building this

23 expertise. So, just an acknowledgement that we really

24 have a number of disciplines that will be important in

25 trying to evaluate risk intervention programs.

1 And I guess just one comment on what Dr.

2 Staffa had said, that surveys are really not used to

3 assess cause and effect, but actually I think there

4 may be instances, especially if we are talking about a

5 longitudinal survey, where you could look at cause and

6 effect.

7 You might use some of this survey

8 methodology if you have a risk intervention, and you

9 want to evaluate its effect, you may do surveys in the

10 beginning, and you may do surveys again, and you

11 really can look at cause and effect.

12 So there is really a broader application I

13 think to surveys than the document might suggest. One

14 other comment really before I talk about analysis,

15 which is really our last point, is that many people

16 already even at this concept stage are taking this

17 document quite to heart, in terms of very literally

18 assuming that the study designs discussed in here are

19 the study designs that you must do, I think it is not

20 always as straightforward as I think everybody knows,

21 in terms of the circumstances in trying to decide what

22 type of study design, a pharmacoepidemiologic study

23 design, is important.

24 That the guidance document is not

25 prescriptive, and so many people that are not

1 pharmacoepidemiologists are taking this very literally

2 as, okay, this is the issue and this is the design we

3 are going to do.

4 If there can be some discussion that there

5 are a multitude of designs that might be appropriate

6 for a situation, I think that would be very helpful.

7 Again, it is a guidance document and should be

8 interpreted that way, but I am not sure that it always

10 Our final point in terms of ISPE is that

11 we believe that there is still a great deal of work to

12 be done in the development and refinement of analytic

13 methods for evaluating safety signals, as well as

14 methods for risk assessment and risk management

15 evaluation.

16 ISPE is committed to working with the FDA

17 and to others, such as the CERTS, to further the

18 knowledge of the methodological and statistical

19 techniques required. ISPE is firmly committed to

20 providing an unbiased scientific forum for the views

21 of all parties with an interest in the science of risk

22 management.

23 And we are deeply committed to advancing

24 this science. We would like to welcome the

25 opportunity to work with the agency in this area as we

1 did in actually developing the guidelines for good

2 epidemiology practices in 1996, and the data privacy

3 document in 1997, both of which are referenced in this

4 concept paper.

5 And as I said, we will engage our full

6 membership in the feedback process on this concept

7 paper as well. Our next annual conference will be

8 focused on risk management, and anyone who has been

9 here for the last couple of days has already heard

10 this.

11 There is several workshops and sessions

12 being planned jointly with the FDA. So as my

13 colleagues over the past two days have done as well, I

14 would also like to extend an invitation to everyone to

15 join us for the first International Conference on

16 Therapeutic Risk Management, which is held in

17 conjunction with the 19th International Conference on

18 Pharmacoepidemiology.

19 It will be held in Philadelphia, August

20 21st through the 24th of this year, and so we would

21 like you all to come. I would also just like to add

22 one more thing.

23 I am the membership chair for ISPE, and so

24 hopefully I have tantalized you with all the wonderful

25 things and great thoughts that ISPE members have. And

1 if anybody would like to become a member, please see

4 Stemhagen. Our next speaker is John Ferguson. I

5 should have said Dr. Ferguson, excuse me.

6 DR. FERGUSON: My name is John Ferguson,

7 and I am the Vice President of Drug Safety at Biogen,

8 and on behalf of the Biotechnology Industry

9 Organization, I thank the FDA for the opportunity to

10 comment on concept paper number three, risk assessment

11 of observational data, good pharmacovigilance

12 practices in pharmacoepidemiologic assessment.

13 I will begin my presentation with general

14 comments, followed by comments in selected elements of

15 the concept paper, and in so doing, I will address

16 questions posed by the agency, and at the end of the

17 document, I will close by reiterating key points.

18 BIO congratulates the FDA on concept paper

19 number three. This important initiative continues to

20 move toward the increasing use of evidence-based

21 approaches to evaluating selected important safety

22 questions.

23 BIO agrees that observational data can be

24 very useful for confirming signals. However, work is

25 needed to maximize that utility. The level of

1 evidence derived from this data varied significantly

2 with the type of data collection, control of

3 confounding, and the magnitude of the observed effect

4 size among other things.

5 Clarification of the agency's view of how

6 level of evidence for causal association varies with

7 the design would be most helpful. Similarly, we would

8 welcome specification of the way in which regulatory

9 decision making might be conditioned on the level of

10 evidence and the strength of evidence.

11 While I believe that the need for

12 pharmacovigilance plans will vary, and should be

13 determined on a case-by-case basis, explicit criteria

14 that could serve as the basis for decisions about

15 pharmacovigilance plans would help to ensure their

16 consistent and effective use.

17 Small companies will require time to

18 effectively implement the full range of proposed

19 activities, since some techniques require special

20 expertise that is not widely available or easily

21 accessible to them.

22 Moving now to specific comments on

23 selected elements of the concept paper, the paper

24 refers to the agency's expectations when a signal is

25 identified, criteria that delineate the basis for

1 deciding that a signal has been identified, contrasted

2 with those that confirm a signal would be very

3 helpful.

4 And in this we reiterate the comments of

5 colleagues in other organizations made earlier. While

6 it is not the intent of this paper to explain the

7 nature and strength of safety findings to regulatory

8 decision making, it seems logical to do so.

9 For example, what impacts to signal

10 identification, as opposed to confirmation have on

11 labeling. High quality case reports are the

12 cornerstone of good pharmacovigilance practice.

13 The major limitations to high quality case

14 reports are the lack of reporter incentive to complete

15 the considerable amount of work that is required to

16 obtain complete information on safety issues and

17 confidentiality, which I will address in a moment.

18 I want to stress that incomplete

19 information will remain a problem, and so discussions

20 concerning the rational use of incomplete information

21 for purposes of signaling in regulatory decision

22 making would be very useful.

23 There is no doubt that confidentiality

24 issues impact our ability to obtain complete

25 information on safety issues. It is noteworthy that

1 confidentiality initiatives are progressing in

2 parallel to this risk assessment initiative.

3 Ideally over time the developers of the

4 risk assessment initiative can provide guidance to

5 those working on confidentiality initiatives to ensure

6 that confidentiality regulations support risk

7 assessment needs.

8 In particular, guidance which is clear

9 regarding which activities, which of the proposed

10 activities do not constitute surveillance activities

11 would be most useful.

12 The utility of registries and surveys for

13 evaluating signals depends on design, and I want to

14 reiterate a point that was made earlier that there

15 seems to be a lot of confusion in terms of the

16 definition of registry.

17 It ranges in our experience from designs

18 that are purely exploratory and that may be able to

19 identify signals, but can do little to confirm them.

20 And on the other extreme to studies that almost

21 approximate large simple trials.

22 The FDA can do a lot to reduce confusion

23 in this regard by providing more detailed definitions

24 of registries or registry variance in surveys; and

25 delineating a clear approach to the consistent

1 implementation of this design.

2 In this regard the FDA has considerable

3 experience with registries and could increase the

4 utility of the guidance document by way of examples of

5 situations where registries have proven to be very

6 useful for confirming signals and where they have

7 failed in this regard.

8 The use of automated databases is a

9 complex subject as has already been recognized. BIO

10 suggests that the concept paper include discussions

11 with relative merits and limitations of automated

12 databases for purposes of evaluating signals.

13 We would also welcome specific guidance

14 regarding the validation of data derived from

15 automated databases. Background rates are useful

16 despite limitations.

17 But since there are limitations,

18 clarifications for guidelines for use in

19 interpretation would be most helpful, and in

20 particular with regard to the need for and value of

21 standardization.

22 And the basis for requiring alternatives

23 when background rates may not be optimal are

24 available, but may be useful for addressing safety

25 issues in a given context.

1 There are clearly pros and cons that have

2 been discussed regarding reporting rates. BIO feels

3 that reporting rates are often useful as crude tools

4 for signal evaluation. However, reporting rates can

5 be very difficult to interpret.

6 As such, we would welcome a discussion of

7 the assumptions and basic requirements for generating

8 the reporting rates, the limits to their utility, and

9 the approach to their consistent interpretation

10 conditioned on under-reporting.

11 In contrast, product comparisons based on

12 reporting rates are very limited. They require strong

13 assumptions that are often questionable. Regulatory

14 decisions based on comparisons of product reporting

15 rates should not be made without a clear understanding

16 of the surveillance models used for each compound, as

17 well as the evaluation of the procedures used to

18 estimate treatment and exposure.

19 Such comparisons should be avoided when

20 products are used for different indications. In

21 general, causality assessments are unreliable, but

22 there are clearly situations where causality can be

23 attributed in individual cases and excluded in

24 individual cases.

25 Therefore, there is a need to define the

1 role of causality assessment in signaling. For

2 example, how should causality be used in constructing

3 reporting rates and sensitivity analyses that go to

4 those reporting rates.

5 The FDA's concept of linking the degree of

6 causality to the quality of evidence is a very useful

7 concept that warrants further development, and goes to

8 the point that I made at the opening of my

9 presentation related to the level of evidence and how

10 it should impact regulatory decision making.

11 The strengths and limitations of data

12 mining are not well understood at this point in time.

13 There are strong underlying assumptions, and there is

14 a need for additional developmental work to define the

15 exact role of data mining in regulatory decision

16 making.

17 In particular at this point in time, BIO

18 feels that data mining may be capable of identifying

19 signals, and maybe more appropriately identifying

20 alerts as described by Gerry Faich, but it is not

21 clear how useful data mining will be in terms of

22 confirming signals on safety information.

23 In summary, observational data can be

24 useful for signal confirmation, and represents an

25 important move towards evidence-based assessment of

1 safety issues. Additional work is needed to

2 understand the contribution of different types of

3 observational data to levels of evidence for

4 causality.

5 We recommend efforts to link the level of

6 evidence to specific regulatory actions, and explicit

7 criteria for pharmacovigilance plan requirements will

8 increase their effectiveness and consistency of their

9 use, as small companies need time to acquire the

10 expertise needed for effective implementation of the

11 full range of the proposals.

12 In closing, we thank the FDA for the

13 opportunity to comment on concept paper number three,

14 and we again applaud the agency's efforts to date in

15 this important endeavor, and we look forward to

16 working with the various partners to continue the

17 development of these key concepts. Thank you.

19 Ferguson. Our next speaker is Dr. Wendy Stephenson.

20 DR. STEPHENSON: Good afternoon. I am

21 Wendy Stephenson, and I am a senior vice president for

22 Global Safety Surveillance and Epidemiology Labeling

23 and Health Outcomes for Wyeth.

24 I have worked within the industry in the

25 area of safety for about 15 years, and I have been a

1 member of the CIOMs working group on adverse drug

2 reaction reporting for just about as long.

3 This afternoon, I am here representing

4 PhRMA. I would first like to commend the FDA and the

5 various committees for their excellent work drafting

6 the three concept papers that have been the subject of

7 these very interesting three days.

8 And to thank the FDA for allowing PhRMA to

9 participate and to publicly comment at this meeting.

10 I will be addressing the FDA's questions 4, 5, and 6,

11 following Linda having addressed the first three

12 questions.

13 But will end with some additional

14 questions and points to consider for the FDA that

15 PhRMA hopes will generate some additional discussion.

16 Under what circumstances would a registry be useful as

17 a surveillance tool and when would it cease to be

18 useful. While there are clear circumstances where

19 registries represent an important surveillance tool --

20 for example, for the use of pregnancy registries -- in

21 most situations there are likely to be better

22 alternatives.

23 For most products and in most

24 circumstances, it is probably not going to be useful

25 to establish a registry. Registries may be useful

1 when the information you need to ensure safe use of a

2 product cannot be obtained from another mechanism.

3 And the need for the information justifies

4 the extensive resource commitment necessary to set up,

5 operate, and comply with the registry. Ideally there

6 would be a concomitant comparison group, and if not,

7 then at the very least a comparable historical

8 control, or well established background incidence of

9 the event in question.

10 With these criteria in mind, PhRMA

11 supports the concept of pregnancy registries under

12 appropriate circumstances, and agrees that there may

13 be other similarly special circumstances where a

14 registry may be appropriate.

15 One example that comes to my mind is the

16 Myelotarg VOD registry, which was established after a

17 signal suggested the possibility of an unusual liver

18 toxicity in patients with AML.

19 VOD is veno-occlusive disease and it is an

20 unusual liver toxicity. That registry, with a well

21 defined outcome of interests, in a well defined

22 population, is expected to provide useful information

23 on this potential risk that would not otherwise be

24 available through other sources.

25 Where a registry is deemed appropriate to

1 achieve a safety surveillance goal, there should be

2 specific objectives and a well defined outcome of

3 interests. The size of the patient population, as

4 well as the duration of follow-up, should also be well

5 defined, and not open-ended.

6 Industry responsive registries tend to be

7 drug specific registries. However, it may be

8 advantageous to establish a disease specific registry

9 with collaboration across companies and other parties

10 as appropriate.

11 If separate registries are established for

12 similar compounds, there should at least be some

13 standardization so that comparable levels of reporting

14 and evaluation may be accomplished.

15 A well tested alternative to registries is

16 the use of large automated databases to serve as a

17 source of appropriate cohorts. Databases such as

18 Kaiser Permanente, Saskatchewan in Canada, GPRD in the

19 U.K., are powerful tools used by industry to sponsor

20 research without assuming the cause of initial data

21 collection, which registries entail.

22 In addition, large automated databases

23 capture all patients in a defined population; whereas,

24 registries usually only capture those patients who are

25 voluntarily enrolled in a study.

1 Therefore, selection bias of an unknown

2 magnitude and origin may be a particular concern for

3 registries. PhRMA encourages collaborative review by

4 industry and FDA of large linked databases for the

5 evaluation of safety signals.

6 Under what circumstances would active

7 surveillance strategies prove useful to identify as

8 yet unreported events.

9 Active surveillance strategies, which

10 often include the concept of registries, may be useful

11 when information cannot be obtained from more readily

12 available tools that are less resource intensive and

13 less obtrusive.

14 One approach to active surveillance

15 involves the use of specially trained sentinel sites

16 generally affiliated with academic centers to

17 stimulate the reporting of events that might not

18 otherwise have been spontaneously reported.

19 Such strategies may be subject to

20 selection bias, and are likely to involve relatively

21 small populations, and so it may not be useful for

22 detecting rare events.

23 When successfully employed, however, they

24 may provide more in-depth information and a more

25 accurate estimate of the reporting rate, and may be

1 more likely to provide information that will help with

2 making a causality assessment.

3 Another approach to active surveillance is

4 prescription event monitoring, such as the green card

5 system that has been in place in the U.K. for many

6 years. This approach differs from most others, in

7 that events are collected, regardless of the suspicion

8 of drug relationship.

9 If conducted with a systematic consistent

10 approach across a wide variety of products, the

11 resulting database can be used to make comparisons

12 across products.

13 And because the denominator is known

14 actual incidence rates can be calculated. Care must

15 be taken, however, in the interpretation of

16 comparisons between drugs that may be prescribed under

17 different circumstances that lead to confounding and

18 bias.

19 It is not clear that prescription event

20 monitoring has ever lead to the identification of

21 previously undetected signals. It can, however,

22 sometimes be useful for confirming signals that have

23 been previously detected through spontaneous reports.

24 Other initiatives which probably should be

25 considered to encourage the identification and

1 reporting of adverse events were mentioned earlier by

2 Dr. Goldman, and that is the implementation of

3 educational programs for health care professionals.

4 Perhaps providing CME credits for

5 reporting complete and high quality reports to either

6 the agency or the company might be considered as an

7 incentive.

8 And the last question, under what

9 circumstances would additional pharmacoepidemiological

10 studies be useful. Pharmacoepidemiological studies

11 are useful to test hypotheses generated by a signal

12 from the spontaneous reporting system, and to further

13 evaluate the safety profile of a product for which a

14 potential risk has been identified prior to approval.

15 However, as with all data sources we have

16 been discussing today, no study will ever be

17 definitive. Rather, it will represent just one more

18 piece of the puzzle. Pharmacoepidemiology studies may

19 have false positive, as well as false negative,

20 results due to bias and confounding.

21 Fortunately, utilization of some of the

22 better automated databases allows for control of

23 confounding to a large extent. Perhaps the greatest

24 value of pharmacoepidemiologic studies is their

25 usefulness both pre-and-post approval in identifying

1 background incident rates for adverse events of

2 interest.

3 In addition, they may be useful to

4 identify high risk groups or risk factors for a

5 particular adverse event. The same databases may be

6 used to conduct drug utilization studies which can

7 provide useful information on physician prescribing

8 and monitoring behavior, characteristics of patients

9 who are treated with the drug of interest, as well as

10 drug usage patterns, and the impact of those usage

11 patterns on outcome related to both benefit and risk.

12 The same types of studies can thus be used

13 as a mechanism for measuring and evaluating the

14 effectiveness of risk management interventions, at

15 least in the population covered by the database.

16 In summary, PhRMA supports the development

17 and use of observational studies of information beyond

18 spontaneous reports. PhRMA agrees with FDA that

19 pharmacoepidemiologic studies are useful to further

20 evaluate the safety profile of a product for which a

21 potential risk has been identified through the

22 spontaneous reporting system or other source.

23 All of the modalities available have

24 limitations, and no one approach can be expected to

25 provide a definitive answer. Conclusions regarding

1 actual risk will largely depend upon the consistency

2 of findings, and the strength of evidence after taking

3 into account information from as many sources as

4 possible and within reason.

5 And I end with some additional -- and I am

6 afraid to ask these questions, but hopefully these

7 will be more difficult to pass back to us.

8 DR. GALSON: Try me.

9 DR. STEPHENSON: Since both the FDA and

10 industry conduct pharmacoepidemiologic studies, it

11 would be helpful to hear FDA's thoughts on when a

12 study should be conducted by the company, versus FDA,

13 or both.

14 And, secondly, what does FDA view as its

15 obligations to a company when the FDA conducts a

16 pharmacoepidemiologic study on that company's drug.

17 Thank you for your time.

19 next speaker is Dr. Sidney Kahn.

20 DR. KAHN: Thank you. Good afternoon. I

21 am Sidney Kahn, President of Pharmacovigilance and

22 Risk Management, Incorporated, an independent

23 organization providing consulting and support services

24 to the pharmaceutical industry, and its supporting

25 organizations.

1 And I am very appreciative of being

2 allowed to present my views in front of this august

3 gathering today, and I would also like to thank the

4 FDA for providing such provocative and insightful

5 documents, as well as allowing the public to comment

6 on their content and hopefully their eventual

7 improvement into the eventual guidance as the product

8 is developed.

9 I would like to start with what I consider

10 to be an omission from the current documents, which is

11 the role of therapeutic or Phase IV studies in risk

12 assessment. I started talking about this yesterday

13 afternoon, and so this is the right slide and the

14 right presentation at this point.

15 The May 1999 reports of the task force on

16 risk management had a fairly extensive discussion on

17 the use of utility or the potential utility of Phase

18 IV studies in risk assessments.

19 And I was quite surprised then to find

20 that the current concept paper makes almost -- I think

21 it makes almost no mention of it at all, or if it

22 does, it is purely in passing.

23 And this is odd to me because it strikes

24 me that this is a wonderful mechanism and tool for

25 identifying or characterizing, or both, either new

1 risks or established risks that have been identified

2 during clinical trials.

3 And this is a mechanism that is already in

4 place, and this is not something that would have to be

5 added on as a new or large burden. The additional

6 burden of uniform data collection and data aggregation

7 is real, but it is nowhere near as large as some of

8 the other interventions that have been proposed.

9 So using it in the vernacular, I see a

10 substantial bang for the buck in doing this type of

11 approach, and it would also I believe help us to move

12 forward with some additional characterization of new

13 signals that could then be evaluated further in other

14 subsequent larger databases.

15 And I would like to suggest that that may

16 be included in the final guidance as an area that

17 could be used as a significant risk assessment tool in

18 the marketed product that we know.

19 I am going to address several specific

20 points in the document, and some I will and some I

21 won't touch, as several have been dealt with much

22 better than I could by others.

23 But if I could talk about an issue of case

24 follow-up and to a lesser extent expedited reporting,

25 which is not part of the guidance. In relation to a

1 different regulation that the FDA has proposed, which

2 are related to format and content of the adverse

3 reaction sections of the drug safety labeling, under

4 the new proposal only adverse reactions, whatever they

5 may be, will be included in the label.

6 And events that occur in the study

7 population at a comparable frequency or without drug

8 therapy, will not be included in that label proposal.

9 Now, for those of us who have worked for any length of

10 time in spontaneous post-marketing reports, I think we

11 all know that the profile of the events that get

12 reported post-marketing tends to be quite smaller to

13 that which was observed in the clinical trials in the

14 population.

15 And if we omit from the labeling any

16 description or listing of those common events which

17 may be serious in many populations, such as diabetics

18 or cancer patients, or people with congestive heart

19 failure, or a variety of other conditions, if those

20 are omitted from the labeling, they will become under

21 current regulations unexpected or unlisted if they are

22 in the company core data sheet.

23 And unexpected events have specific

24 regulatory requirements which are quite burdensome,

25 especially if there are very many of them. The issue

1 of aggressive follow-up, we have heard talked about.

2 And in addition to the burden on health

3 care providers and the difficulties that the new HIPAA

4 legislation creates for them, all this would do would

5 require the sponsors or the manufacturers of drugs

6 that have high background rates of significant adverse

7 events to follow up the natural history of the

8 disease, which I don't believe was the intent.

9 So I see this as an area where there is

10 some unintended consequence of two intrinsically

11 logical approaches that when taken together give you

12 something that you really had not intended at the end

13 of the day.

14 So I would ask the agency please to take

15 that into consideration, and just to help them along.

16 I am putting up here some issues or methods that I

17 consider could be used to obviate that problem,

18 because this would be a real problem.

19 One is that we could continue to include

20 the totality of the adverse events, including those

21 that occur with comparable frequencies in the active

22 control groups in the label, as we do now in the table

23 of adverse reactions.

24 We could create an additional section of

25 the label that contained adverse events that the

1 agency and the sponsor would consider expected for the

2 purposes of regulatory reporting and follow-up.

3 Or there is a possibility, and I don't

4 know how this would drive with the lawyers, but that

5 is something that we could explore, of omitting these

6 comments from the prescribing information because it

7 is not clear that the physicians necessarily need to

8 know how many patients with congestive cardiac failure

9 have been hospitalized with pulmonary edema.

10 But rather to agree between the sponsor

11 and the agency at approval, at the time of ongoing

12 periodic reporting, for example, what common events

13 could be considered expected, and therefore not

14 subject to aggressive follow-up, or expedited

15 reporting.

16 The FDA asked a question in this concept

17 paper on population background rates. We have heard a

18 lot about this and clearly knowledge of this is very

19 important, and is very valuable.

20 What I would submit though is that the FDA

21 has a unique opportunity to find such data in great

22 detail from its own databases, especially for those of

23 controlled populations in their archives in NDAs and

24 BLAs that the pharmaceutical and biologic industries

25 have presented to the agency.

1 And there is an initiative that I just

2 heard about recently from Randy Levin, called JANUS

3 which is under way in CDER, which is apparently

4 allowing electronic integration of such data, and

5 perhaps our FDA colleagues could comment on that in

6 due course.

7 And I would certainly urge sponsors to

8 allow their proprietary data to be used in an

9 appropriately automized way to help the FDA develop

10 these assessments of background rates, because these

11 data are probably even more robust than the

12 pharmacoepidemiologic data for background rate

13 calculations.

14 We have heard a lot about under-reporting

15 over the years, and I would like to make some caveats

16 and points about the myth as we heard this morning on

17 under-reporting.

18 First of all, high reporting rates for

19 events that have a high background rate to the

20 population I think have to be considered

21 uninterpretable. So you can't say anything about them

22 in my opinion, and here we have an issue where again

23 the labeling proposals might also have an effective,

24 because whether you include or exclude an event from

25 the label, will have some impact -- I don't know how

1 big or in what direction -- on the reporting by health

2 care professionals.

3 And that would be something that should be

4 evaluated. It would be very helpful here to have some

5 input from the medical professional organizations as

6 we heard about in relation to, say, the Rhode Island

7 study results, to determine how physicians would see

8 such a change.

9 The issue of the much publicized 10

10 percent reporting rate which has been touted as

11 reflecting the tip of the iceberg, I am not sure that

12 many people believe that, and we have heard examples

13 of why that may or may not be so earlier.

14 But I want to quote you an actual example

15 from personal experience. The FDA sent my company at

16 the time a letter in which they assessed an incidence

17 rate of a very severe, life-threatening, or fatal

18 adverse event at approximately 20 times the population

19 background rate for the population exposed to the

20 company product, and based entirely on spontaneous

21 reports.

22 The company went out and did two extensive

23 studies. We also went to the PM database as Dr.

24 Stephenson mentioned, and a variety of other places,

25 and we have been unable to confer anything like the

1 number of cases in the epidemiology studies.

2 We are paying an independent expert

3 opinion from a clinician with great expertise in the

4 area who assured us that in his experience at least 80

5 percent of such cases would be reported by practicing

6 physicians.

7 And rather than under-reporting and

8 getting to the over-reporting concept, we received

9 multiple reports of the same case from various

10 reporters, and so further supporting the idea that the

11 threshold for reporting such cases is relatively low.

12 And to cap this, and I know that there are

13 studies that have suggested that publicity can alter

14 the reporting weight following publications in the

15 medical literature in the U.S. and other countries,

16 the addition of a black box warning to the label, and

17 a dear doctor letter, and there was no increase in the

18 reporting rate, which remained constant throughout

19 (inaudible).

20 And so I realize that this is an anecdotal

21 experience, but I think it is illustrative, and as a

22 result, I would like to propose the following

23 hypothesis. That the more severe the event and the

24 outcome, and the more obviously it appears to be

25 related to the drug, that is to say that the less

1 commonly it occurs in the treated population, and the

2 less commonly it has in alternative explanation, the

3 more closely the reporting will approach the true

4 incidence rate.

5 That hypothesis is potentially invaluable,

6 and if the FDA can access in its various databases for

7 a variety of products which sponsors do not

8 necessarily have access to, and compare instances of

9 known adverse reactions of varying degrees of severity

10 -- cough with ACE inhibitors, versus liver failure

11 with troglitazone, for example, with the corresponding

12 numbers of reports and time adjusted in errors, and it

13 would be very easy to get true estimates of the

14 reporting rates.

15 The FDA has asked for comments on how to

16 improve spontaneous report quality. Not on the slide,

17 but a thought that has occurred to me on numerous

18 occasions is that in my estimation probably the

19 majority of supposed SADRs received by pharmaceutical

20 manufacturers come into medical information in the

21 form of I had a patient taking your drug among 5, 7,

22 or 10 other drugs, who has developed Event X. Is this

23 known for your drug.

24 And sometimes it is and sometimes it

25 isn't. Every one of those is captured and

1 characterized as a suspected adverse drug reaction,

2 and is given precisely the same weights as a report

3 from a health professional saying that I had a

4 perfectly healthy patient who had a minor condition.

5 I put her on your drug and so this

6 terrible thing happened, and your drug did it. I see

7 those two scenarios as being somewhat different, and I

8 believe it could be helpful to alter the categories of

9 reporting in such a way as to differentiate between

10 these sort of incidental reports where a physician is

11 looking to confirm or refute a differential diagnosis

12 in relation to looking for a known drug explanation,

13 as opposed to reporting a previously unknown adverse

14 action.

15 DR. GALSON: Excuse me, Dr. Kahn. You

16 have got a little bit less than a minute left.

17 DR. KAHN: Okay. Well, I will complete

18 the rest of my presentation publicly in terms of

19 written comments to the document. The last thing that

20 I want to switch to is to add a few comments or a

21 couple of comments to data mining.

22 We have heard a great deal about it, and

23 the biggest problem that I see with it is the absence

24 of any pre-specified hypothesis. And therefore

25 because data mining can pick up signals that no one

1 knows what to do with, it is my opinion that this

2 should probably be reserved for regulators who do not

3 have any necessary regulatory obligation to act on

4 this, as opposed to pharmaceutical manufacturers who

5 could be placed in a very awkward position, both from

6 a regulatory and a legal perspective if they undergo

7 data mining when they do not have access to the

8 complete data set that the regulators have.

9 Pretty much everything else I had to say

10 has in fact been said by other speakers, and so I will

11 be happy to conclude there, and I thank you very much.

12 DR. GALSON: I thank you very much. Let's

13 just have a quick show of hands whether folks want to

14 take a break or just go on through conclusion, and if

15 you want to just go on until we finish, raise your

16 hand.

17 (A show of hands.)

18 DR. GALSON: Okay. Good. Let's move now

19 to discussion and Q&A section, and let's go back to

20 the question that we left hanging that I mentioned

21 right before lunch.

22 I don't know who from PhRMA wants to

23 address that. Is it possible to put the question back

24 up, or you can just restate it, whichever is easier.

25 DR. STEPHENSON: I am going to give you

1 the short answer, and Linda is going to give you the

2 longer answer. The shorter answer is -- well, read

3 the question first.

4 MS. HOSTELLEY: Okay. The question is

5 what criteria should be used to determine whether a

6 pharmacovigilance plan describing pharmacovigilance

7 efforts above and beyond post-marketing spontaneous

8 reporting is warranted.

9 Okay. And the short answer is that on a

10 case by case basis. And the more elaborate answer is

11 that essentially we would encourage the agency to

12 consider the criteria that have been put forward in

13 CIOMS-III actually, specifically strength of evidence,

14 significance of medical saliency, the seriousness of

15 the event.

16 That is, you know, does it fulfill the

17 serious criteria, or is it a non-serious type of

18 situation; the predictability, and the reversibility,

19 as well as the validity of methods available to be

20 able to answer the question within reason.

21 We would encourage also open dialogue

22 between the FDA, and that would be the Office of Drug

23 Safety Review Division, and the sponsor, to really

24 reach a consensus as far as the best approach in

25 involving this.

1 DR. GALSON: Okay. Thanks very much.

2 Let's just stick on that for a minute and see whether

3 anyone else wants to make a comment on those

4 responses. Anybody from the agency want to say

5 anything?

6 We will certainly consider that question

7 and I think in general we agree that there has to be a

8 case-by-case assessment. It is very difficult and it

9 may be possible in certain classes of drugs to set out

10 specific criteria if you have a lot of background

11 information.

12 But we can't or we won't be able to leave

13 at least some aspect of a case-by-case analysis. Any

14 other comments on that? FDA folks? Go ahead. I want

15 to remind folks as well that if you want to fill out

16 cards and send them up, we are happy to take more

17 cards.

18 MS. PEREZ-GUTTHANN: Susana Perez-

19 Gutthann, Pharmaceia. It is regarding this question

20 of the pharmacovigilance plan, and I think one of the

21 useful parts of the discussions that we have had these

22 last couple of days is that there has been a lot of

23 clarification in terms of what we mean when we say a

24 word, and the same word doesn't mean the same to

25 everybody.

1 And I think that something like this is

2 happening to the pharmacovigilance plan, where we are

3 discussing when to initiate a pharmacovigilance plan

4 because in the definition it is implied that it is a

5 reaction to something happening to the drug.

6 And in fact I might add that probably most

7 companies or most sponsors at the time of launch have

8 a pretty clear idea of what the background is in terms

9 of risk profile, and have formalized some kind of

10 document in which they say this is the area that we

11 should monitor more closer or not.

12 And whether the activities are going to be

13 monitoring some kind of reports, or going beyond and

14 doing some pharmacoepidemiologic, et cetera. And as I

15 mentioned, if we compare it a little bit with the

16 perspective that the European agencies are taking, and

17 this is referring to the document that I was

18 mentioning yesterday, the summary of the heads of

19 agency reports, they are taking a little bit of a

20 broader pharmacovigilance plan, and that these are

21 general activities that one would take.

22 And the criteria that they use for

23 initiating this or for recommending it, the

24 pharmacovigilance plan, rather than based on an event

25 or safety issue related to the drug, is based on the

1 type of drug or the medications.

2 And if you allow me since we were talking

3 about criteria, they mentioned that they would be

4 particularly relevant to new chemical entities on

5 biotech derived products, or from additional products,

6 and significant changes in established products, such

7 as new forms introduced to a population, establish

8 products introduced to a new population, of

9 significant new indications, and finally establish

10 products when reclassified from prescription only to

11 non-prescription availability.

12 I am not saying that we should be adopting

13 this criteria, but this is kind of a different take on

14 when do we initiate the pharmacovigilance plan.

15 DR. KAHN: May I continue the discussion

16 on pharmacovigilance plans?

17 DR. GALSON: Sure.

18 DR. KHAN: This is Sidney Khan. In my

19 last position in industry, I had the opportunity to

20 both develop and review a variety of what we called

21 pharmacovigilance plans, but which were primarily

22 focusing on the Phase III stage of development,

23 because the goal was to try and focus attention on

24 events that could possibly be indicators of problems

25 down the road.

1 And there was a carryover as Linda

2 Hostelley I think said earlier, and we all understand

3 that drug safety is a continuum, and therefore, you

4 can't start pharmacovigilance plans at the time of

5 launch. You really have to know up front where you

6 are going to look, and you have to have high

7 sensitivity to what I would call harbingers of future

8 events.

9 Bob Temple yesterday mentioned several

10 times the elevation of the minor transaminase and

11 bilirubin as potential indicator of a pathotoxicity,

12 which would be very nice to see established by a

13 formal means.

14 But pharmacokinetics should be focusing on

15 safety professionals in the companies and in the

16 agencies on what to look for in drugs, and which can

17 be identified from the preclinical data, and from

18 pharmacokinetics, and pharmacodynamics, et cetera,

19 throughout the development and life cycle of the

20 product.

21 DR. BEITZ: I think what I would just like

22 to say is that by giving this activity a name, like

23 pharmacovigilance plan, and it gives us the

24 opportunity to actually talk about what we all want to

25 be doing to monitor the product.

1 And I am not so sure that the folks at the

2 FDA always know exactly what industry might be doing,

3 and vice versa, and I think if we had some clarify on

4 what is appropriate or what is feasible, then we are

5 way ahead of where we are now.

6 DR. GALSON: Any other comments from

7 folks, from the FDA, or any questions from the

8 microphone, folks at the microphone?

9 DR. BEITZ: This is not about

10 pharmacovigilance plan, but this was actually related

11 to our active surveillance question. I didn't hear a

12 whole lot of comment about that, and it seemed as

13 though what I was hearing was somewhat of a lukewarm

14 endorsement of this kind of activity.

15 And I was hoping that there might be some

16 more discussion if possible. One of the things that

17 we sometimes toy with is perhaps mining information

18 from liver transplant centers, for example, to look

19 for potential cases of liver failure, or acute liver

20 injury.

21 Does that sort of activity make sense to

22 you all, and do you have any other possible examples;

23 surveying emergency departments for adverse events

24 coming in to such places, such settings? Any comment

25 at all?

1 DR. STEPHENSON: I think that example

2 makes a lot of sense. In fact, I know that when we

3 were dealing with the issue with Bromfenach, that was

4 one of the places that we went to, to see if there

5 were any additional cases of liver failure.

6 It was a new transplant registry, I

7 believe, or it was a liver failure registry, and that

8 was very helpful.

9 DR. KAHN: In relation to the liver

10 transplant registry, it doesn't cover all the

11 transplant centers. It is only a subset, and it is

12 very useful. But I think if one were to take the data

13 from that database as published and presented by Dr.

14 Lee out of the University of Texas at the Hepatoxicity

15 Workshop, I think we would certainly have to take a

16 certain amount of it off the market.

17 DR. VASHISHTHA: This is a situation in

18 which I think an active survey might make sense.

19 DR. GALSON: If you would introduce

20 yourself, please.

21 DR. VASHISHTHA: I am Anshu Vashishtha

22 from Watson. Probably in Stevens-Johnson

23 situations, because again they are typically drug

24 induced reactions and it might be fruitful to an

25 active surveillance there. That is one situation that

1 I think comes to mind.

2 We might for overdose consider poison

3 control centers. I think it was mentioned earlier

4 about not integrating the poison control and drug

5 information centers, and they might be helpful to get

6 (inaudible) in those situations.

7 DR. GALSON: Let me just -- well, any

8 comments on that comment? Yes.

9 DR. STEPHENSON: Just one comment. I

10 think that maybe some of us misunderstood what was

11 being discussed, in terms of registries, as part of a

12 pharmacovigilance plan.

13 My comments were sort of focused on should

14 you develop a registry for a particular product. I

15 think that there are certain already existing types of

16 registries, like liver transplants, or Stevens-

17 Johnson, that obviously are very useful, but cut

18 across all products.

19 So you are basically starting with the

20 reaction and looking to see whether there are any

21 signals for any particular product. So I don't know

22 if that was also what was intended in the document,

23 and maybe that could be clarified a bit.

24 DR. GALSON: Let me read one of the

25 questions that came in on a card this morning. Dr.

1 Hostelley stated that efforts should be directed

2 towards limiting false positives of data mining.

3 The question is are similar efforts under

4 way to limit false negatives, and next, further, how

5 would one decide that a particular signal was a false

6 positive. Is a controlled trial necessary in that

7 case. Did you have any response to that?

8 DR. STEPHENSON: I looked to members of

9 the collaborative group, either Dr. Szarfman, or Dr.

10 Almanoff, to perhaps respond.

11 DR. SZARFMAN: Dr. Almanoff left,

12 unfortunately, and I think that -- oh, I thought you

13 left. First of all, we need to start the program a

14 little bit. I think that the (inaudible) that we can

15 do multiple comparisons and we can have multiple

16 controls, and (inaudible) that having controls of what

17 is going on with other drugs used to treat the same

18 disease, so that you understand where you are.

19 And it is a way of finding a (inaudible)

20 for the event across other drugs that are being used

21 (inaudible), and move away from the (inaudible)

22 comparisons, and we need to look at the big picture

23 and it is not two dimensions. It is multi-

24 dimensional. But we are going to do the testing

25 together.

1 DR. ALMANOFF: The working group actually

2 has a number of initiatives, and we will be studying a

3 number of factors that affect performance, and what is

4 the best database, and what are the best

5 stratification schemes, and what are best practices

6 for this tool.

7 A couple of points. I wanted to reiterate

8 what I said this morning about the fact that when I

9 hear the term false positive, I get a little bit

10 nutty, because I just view this as a positive signal

11 as -- you know, as a -- I'm sorry, a positive score in

12 a data mining run.

13 I just view that as an intermediate step,

14 and I view it as an alert that I should pull the cases

15 and use my clinical judgment, and actually pull cases

16 of related events, and maybe even look at related

17 drugs.

18 And then use all of that information and

19 bring it together as an intelligent clinician to make

20 a decision. So again I think that the risk of pulling

21 these things out is very low.

22 And in terms of false negatives, maybe

23 Anna has some thoughts on that, but -- well, go ahead.

24 DR. SZARFMAN: The false positives. There

25 were examples, for example, this morning about drugs,

1 prozac, related to suicide. Well, suicide events that

2 are suicidal ideation, et cetera, it is an indication

3 where prozac is being used.

4 The identification of adverse events, the

5 data mining finds them because they are called adverse

6 events. If we are looking at frequency counts, we

7 will find them also.

8 And this is not bad, and the data miming

9 is behaving, and if we call them adverse events, or

10 are defined as adverse events, then the interpretation

11 is important. You drill down -- for example, with

12 pemoline, I have a new signal for multiple sclerosis,

13 and I drill down and this is a new thing now.

14 The multiple sclerosis is an indication,

15 an off-label indication, and where it is being used

16 (inaudible) patients with multiple sclerosis. You

17 drill down in the media until you realize what is

18 going on.

19 Then maybe we need a protocol on

20 indications or indications in a different field is

21 solvable. The negative signals. Okay. This is very

22 interesting. We are working with the same numerator.

23 When people are analyzing reporting rates,

24 we are working from the same database, then

25 (inaudible) would agree that something somebody can

1 call a signal from data mining after looking and doing

2 multiple comparisons, and we need to move and when we

3 are comparing drugs, we need to look at the

4 overlapping (inaudible) limits, then we are now using

5 the tools to be able to do it systematically.

6 But when you know that when you are

7 looking at the ones that are bad, the drug in

8 question, and you are looking at the ones that are the

9 same as the drug in question, and you look at the ones

10 that are below the drug in question, and you see where

11 -- because you can do multiple comparisons, that when

12 we are looking at the (inaudible) output, and you have

13 multiple comparisons, you know (inaudible) to do

14 multiple comparisons when they are trying to find a

15 (inaudible), then you can do these sorts of things.

16 And if we are working with the same

17 evidence, and we are working with the same numerator

18 data, and essentially with the same denominator, then

19 these things are not at issue.

20 You know, what is the background that we

21 need to use, and if we are using external sources of

22 data that the location of the event in question was

23 found very differently -- and maybe it is a paper, a

24 published paper that assumes the expected value of

25 acute liver failure across the board is one in a

1 million.

2 But maybe for a specific disease it is

3 different, and this is where data mining can help you

4 within the same database, because if I know something

5 is about, I touch the data, and I know that things can

6 go wrong, and if you are using data from different

7 sources, and even the management of the data is

8 different, and it can make a difference.

9 That's why we stratify by time, because it

10 is different, like in 1968, and this is now, and this

11 was the problem with the false positives and data

12 mining thought that they were not stratified at all.

13 And then they came to me and they showed

14 me something else, and I knew beforehand that you are

15 stratifying or having false positives.

16 DR. ALMANOFF: Just three more quick

17 comments. First of all, in terms of false negatives,

18 another thought that occurred to me was that I think

19 that we cannot rely always on this tool to find every

20 single signal of importance, and as somebody said

21 earlier, one or two cases that are very compelling

22 might be critical.

23 So I think that for the way that we think

24 it is good to look for signals is to use this to

25 increase frequency, but if you see a case or two of

1 Stevens-Johnson Syndrome, or interstitial nephritis,

2 or something that is clearly drug related, you know,

3 that is the way that you deal with the question of

4 false negatives.

5 If it is medically important and likely to

6 be drug related, then you can cover that by simply

7 using your medical knowledge. No method is perfect.

8 Two quick things about false positives.

9 One thing Anna has alluded to is the fact

10 that there can be confounding by age and gender, and

11 so internal stratification of the data really does

12 help eliminate false positives, and we can talk about

13 that off-line if folks are interested.

14 The other thing that we have discovered as

15 a source of "false positives: is the phenomena that we

16 described as an innocent bystander phenomenon, and the

17 way that you deal with that is that you might have two

18 drugs that are used a lot together, and you can

19 actually subset patients who are on one drug, or just

20 the other drug, and on the combination.

21 And as you actually go through those

22 exercises, you can often comb out what is going on.

23 So I hope that is helpful.

24 DR. YONDRIN: Sam Yondrin, Millennium.

25 This is just a different slant on data mining. I know

1 that when we talk about data mining currently the

2 focus is on market put out data, but data mining has

3 also been applied to clinical trial data, and

4 primarily here, and which is where I have an interest,

5 is how do we apply data mining methods, or at least

6 develop these methods, such that we could probably

7 identify predictive algorithms using statistical

8 packages.

9 And I think that this is where I think

10 identifying or developing clinical -- you know,

11 predictors, probably is within reach compared to

12 pharmacogenomics, which is where my colleagues talk a

13 lot about.

14 But I think also in parallel that there

15 may be a utility for clinical predictive algorithms.

16 What do I mean? So if you were able to identify a

17 patient, say, that was a male, and of a certain age

18 group, and received the same concomitant medication,

19 and had the lab parameter value, then they would have

20 a certain probability for an adverse outcome.

21 So I would just like to understand if

22 there is any efforts in this direction, because in

23 terms of data quality, we have a better change of

24 achieving data quality in pre-marketing.

25 DR. ALMANOFF: Yes, this is an important

1 area, especially because of risk management and

2 actually the working group has a subteam that Christy

3 Chaung-Stein is going to head up to explore ways that

4 one can look at NDA data.

5 There obviously have been a lot of

6 challenges in sort of pooling lots of NDAs, and

7 getting access to that type of information, but the

8 view has been that any sort of -- and as Sidney says,

9 harbingers that can be seen as smoking guns in

10 clinical trials could be useful certainly for setting

11 up pharmacovigilance plans.

12 I know to date that Christy has done a

13 little bit of work with this, and I know in one large

14 NDA that she looked at actually didn't find anything,

15 which I guess is a good thing. I don't remember what

16 drug it was and how that turned out.

17 But the other question though is that when

18 we start digging into these NDAs, and how does that

19 impact on the sort of more frequented safety profile,

20 and how do you sort of reconcile the two, and will

21 that become maybe a concern as companies move forward.

22 Do they maybe feel that a more exploratory

23 analysis is not something that they want to get

24 involved with. So we will have to see how that goes.

25 Thanks.

1 MS. HOSTELLEY: I wanted to address

2 something that has to do with quality, and also has to

3 do with the nature of some of the events that go into

4 adverse or post-marketing adverse event databases, and

5 in which we make the assumption that they are

6 unsolicited and that they are therefore potentially

7 related.

8 The example of the MedWatch form sent in

9 by the physician is something that we should always

10 assume a presumption of a relationship. In certain

11 areas, particularly in oncology transplant medicine,

12 and certain enzyme replacement therapies, the sponsor

13 has extensive long term contact with groups or

14 representatives in medical information, and has

15 extensive long term contact with the treating

16 physicians.

17 And under those circumstances you collect

18 information. An example, for example, in a transplant

19 situation, everything went fine. You are looking

20 good, and there was a wound dehiscence, and he had to

21 go back to the OR.

22 So that wound dehiscence goes in under

23 those circumstances as a spontaneous report. They

24 really can't be viewed as unsolicited reports. There

25 is really no way of distinguishing that in the data

1 that are submitted.

2 In fact, often the situation, and I think

3 Sidney sort of alluded to, where a physician will

4 adamantly deny that they submitted an adverse event

5 report, and you place on them report forms, and send

6 it to them and so on, when they don't believe that

7 this is what happened.

8 But because of compliance concerns, that

9 is what you do. Now, as far as I can see, in each of

10 these there is a field that you can use to reflect

11 that, and in our database we do preserve it. But I

12 don't see how that comes in to the information that

13 the FDA has, except perhaps as a comment in the

14 narrative, which is essentially lost.

15 So in thinking about data quality issues,

16 I wonder if there is any way that we can deal with

17 that kind of information. Thank you.

18 DR. GALSON: Did anybody want to comment

19 on that?

20 DR. BEITZ: That is certainly an issue

21 that we have dealt with at CBER, and I don't know

22 whether there is any kind of a field or a statement in

23 the narrative. We are aware of certain companies that

24 do have certain programs, where there is a lot of

25 active calling and talking to centers and hospitals

1 where certain treatment is delivered.

2 And we understand that under those

3 circumstances that they are going to call and say --

4 and this is sort of maybe a combination of finding out

5 really what is going on and also to connect it to

6 marketing to some extent.

7 But if there is all this discussion, you

8 know, they will say how are things going, and somebody

9 will mention something that they might not have

10 written in their report, and then it has to be put

11 into the database.

12 And so we are aware when we look at

13 certain products that we understand that there is

14 going to be an elevated rate of reporting because of

15 these programs.

16 But I don't know that there is any

17 systematic way that we have of knowing across the

18 board for which products there are such programs.

19 Miles, do you want to --

20 DR. BRAUN: Well, I think there is a box

21 on the MedWatch form that relates to consumer reports,

22 and so that is one potential way that those could be

23 subsetted out, but that is definitely an issue, and

24 sometimes companies actually have at least in

25 biologics had -- they had a policy of submitting such

1 reports, and then they realized that for patient

2 support programs that according to the guidance they

3 are not necessarily obligated.

4 I am talking about patient support

5 programs now to support reports to the FDA, unless

6 they believe that there is a reasonable association,

7 causal association with a product.

8 And so you can have even within one

9 product over time a change in the reporting for that

10 product, and you could certainly imagine that there is

11 a whole bunch, a large number of reports, that would

12 have come into the agency in the early period, where

13 everything was sent in.

14 And then when they realized that there was

15 a guidance that could be applied, and they did not

16 have to send those in, that there would be a different

17 -- that many of those earlier reports would no longer

18 be sent in.

19 And then if you stepped back and said,

20 well, if you are using one of these data mining tools

21 that really look at proportional rates, you basically

22 could change the proportions radically even though it

23 is the same product, and really with the same risk

24 profile, artifactually change those proportions,

25 because a big segment of the reports is no longer sent

1 in as part of the database.

2 So I like to look at data mining as kind

3 of a chainsaw. It is a good tool that can do

4 important work, but it can be dangerous if not used by

5 a trained operator.

6 DR. GALSON: Is there follow-up to that?

7 DR. GOLDSMITH: If I can add a point to

8 that question, and Wendy, I am sure that you probably

9 remember during our discussions of V2A, when we were

10 actually together. There was a discussion of what you

11 do with a clinical trial patient who has completed his

12 clinical trial data.

13 And indeed after the patient has completed

14 his clinical trial data, you go back and you touch

15 base with the investigator, and the investigator says,

16 oh, by the way, this patient had 2 years after the

17 trial was over, and that was considered not to be a

18 spontaneous report in E2A, and it would be subject to

19 analysis causality according to the rules for clinical

20 trial data.

21 DR. SZARFMAN: Of course, you can do a

22 specific report for a (inaudible) coming from a single

23 source. Then you can separate the other options that

24 at a grade level you can look at the reports and

25 photographs submitted by the applicant, and the

1 reports for the same drugs omitted by other sources.

2 There are ways of understanding what is going on.

3 You know, we have many possibilities, and

4 there are many tools, and a computerized environment

5 can let you do things that before was impossible.

6 DR. GALSON: Min, did you have a comment

7 or anything that you wanted to add to that; that or

8 the previous point?

9 DR. CHEN: On the previous point, yes. I

10 think that without going into any specifics about that

11 report, I just want to say, yes, almost or a lot of

12 reports from the spontaneous resource.

13 We know that they are solicited, and they

14 are solicited either from studies or for disease

15 management purposes. But I think that spontaneous

16 reports is really -- that is the beauty of it. That

17 it really accommodates all kinds of reports.

18 A lot of information flow in, but not

19 necessarily that you can use every report for every

20 purpose that you are looking into investigating. So

21 you have to be careful how you use the information.

22 Although you should allow a specific field

23 to capture some information, we know that will help us

24 to identify. These reports may not be directly

25 related to the drug. But we know also that there is

1 always a kind of safety net somewhere, either within

2 a company or in the FDA.

3 When we look at certain issues with a

4 certain drug, we will look at the reports, the case

5 reports, and then look at the narrative. If it is not

6 in the E2B field, we will look at the narrative and

7 then decide whether there is a reasonable possibility

8 that the drug has something to do with the event and

9 make your best judgment to use that report as part of

10 your case series analysis.

11 So I am not that worried about the source

12 of the information, and really the quality of the

13 reports is the key here, and we are not going to get

14 all kinds of reports with all kinds of efforts to

15 gather the information to help us for every instance

16 of the suspicious.

17 But I think that the SADR rule, the

18 proposed rule, dictates that really for those serious

19 outcome events that the company should make more of an

20 effort to append additional information to help us out

21 on whether there are recognized events so that we have

22 the information to analyze them, or for those

23 recognized events with serious outcomes and increased

24 frequency situations.

25 And that we can use the quality

1 information that you obtain from the initial reporter

2 to help to understand more about this recognized

3 event. So there is a priority issue when you look at

4 the whole database.

5 It is not that we don't encourage

6 everybody to gather all the information for every

7 case, but we have got to put our priorities there on

8 how to use the information, and then we know how to

9 gather the information for our own use and for

10 everybody's use.

11 So that is probably the most important

12 message that I feel about from the reporting system.

13 And data mining has all its uses, and we have

14 evaluated so many situations in the past, and it is

15 very useful.

16 But the FDA database is so big, and it has

17 30,000 -- I'm sorry, almost 3 million records, and

18 from day one, 1969, all the way to 1997, it was coded

19 in COSARS, and there were only four terms to be coded

20 for each report.

21 And we know that each case report can go

22 over 40 or 50 events. So the limits are already there

23 to capture quality of events on those older reports.

24 And from 1997 on, they are unlimited events coded

25 MedRA in those reports and with E2B element

1 specifications.

2 So there is a whole horizon of new things

3 going on since 1997, and we have this whole data

4 mining technique applying to the whole thing. And we

5 have to be careful what we are getting into the

6 system, and what we are getting out of the system.

7 And when we validate it, we have to think

8 about all these caveats, all these validation, too, on

9 whether it is there to help us out to understand the

10 signals.

11 So I am really happy to see today that we

12 had so much discussion about data mining, because we

13 are sometimes in a dilemma saying how do we use these

14 scores in our data lives, but we can't really ignore

15 them, and we are trying our best to understand it, and

16 see how we can best use it.

17 DR. SZARFMAN: We stratify by year, and by

18 age group, and by gender, and by report source. Then

19 we are taking care of that problem, and (inaudible)

20 device, the whole scheme, and I think that we need to

21 look at the big picture, and I am willing to help as

22 much as I can with interpretation and implementation.

23 DR. GALSON: Victor.

24 DR. RACZKOWSKI: The discussion about

25 causality assessment makes me wonder whether this

1 might be a time to think about a shift in paradigm.

2 Someone had suggested a possibility that instead of

3 classifying a drug as an adverse events as possibly,

4 probably, or unlikely related to drug, rather

5 modifying the MedWatch form along the lines of do you

6 think this drug -- what do you think this adverse

7 event is due to.

8 Do you think it might be due to the

9 underlying disease, drug A, drug B, drug C. Might it

10 be related to background incidence, and perhaps

11 capturing that sort of information on a MedWatch form

12 would be more informative than these attempts to

13 causal associations for individual adverse events.

14 DR. NELSON: I have two comments regarding

15 spontaneous reports.

16 DR. GALSON: Could you identify yourself.

17 DR. NELSON: Oh, Bob Nelson, consultant.

18 I have two comments. There was discussion this

19 morning about how to best collect quality data for

20 specific organ cells, and I really need to put a

21 reference on the record.

22 There is an excellent book that most of

23 you know about that Christian Beneshou (phonetic)

24 wrote about a dozen years ago, and I would have to

25 give you the exact title, but it is something like the

1 detection and diagnosis of adverse reactions.

2 In there, he has good logic strings of how

3 to think through liver toxicity, bone marrow

4 dyscrasia, et cetera. And there are data collection

5 instruments that have been validated, and these are

6 very, very strong tools to help you guide collection

7 of the correct kind of data elements to help you

8 determine whether or not the drug was likely to cause

9 that reaction.

10 It was a very powerful tool, and that if

11 used correctly, will trump the idea of using an

12 algorithm later on to determine whether it is possible

13 or probable, because you will have such strong data.

14 The second point about spontaneous reports

15 regards the -- well, I just wanted to put that

16 reference on record, because it is a very great text,

17 and I think that most people know it.

18 The other one about reports that are not

19 really spontaneous. There is of course a need to be a

20 company comment field, and the CIOMS comment. It is

21 good pharmacovigilance practice for a company to state

22 their opinion on that case.

23 This was a non-solicited or a solicited

24 report that we don't believe had anything to do with

25 the drug, period on the bottom of the report. You may

1 in fact submit it because you feel that you need to

2 submit it because of regulatory compliance, but you

3 are also -- it is incumbent on you with good practices

4 to state your opinion on the value of that case.

5 And as Min Chen just said, when they do

6 case series analysis, they just throw those out. So

7 those are my comments. Thank you.

8 DR. DANA: It is on a different topic, and

9 it is when you get ready to talk about registries. So

10 go ahead.

11 DR. GALSON: Okay.

12 DR. KAHN: I was going to come back to Dr.

13 Raczkowski's comments on the issue of causality

14 assessment and also just Bob Nelson's. I am not

15 absolutely sure that I know what value causality

16 assessment on individual cases actually adds at this

17 point, because causality is like pornography. You

18 know it when you see it, but it is very hard to

19 define.

20 And typically the active pharmacovigilance

21 professional will have (inaudible) and say I don't

22 like the look of that, and they will go and explore

23 for similar cases.

24 And at the case series is a far more

25 useful tool for traveling to assess causality than

1 individual cases, and of course in the United States

2 currently we do have the FDA disclaimer for legal

3 protection, and if we were being asked as companies to

4 add causality assessments to individual cases, there

5 would probably have to be some type of legal relief

6 that goes with that. So if I could just add that to

7 the causality discussion.

8 DR. YODRIN: Sam Yodrin, Millennium. Just

9 a follow-up on the issue of causality. I think to

10 really have clarity as to the role of causality

11 assessments with ICSRs, we need to step back and ask

12 ourselves what is the question that we are trying to

13 answer.

14 Is it firstly does the drug cause this

15 ADR, or does the drug cause the ADR in this patient.

16 And they are two different questions. And the initial

17 question that we are faced with is the broader

18 question as to whether the drug causes the ADR.

19 And the answer to that question is not

20 from individual case causality assessments, and that's

21 why when you have regulators like Germany or Japan

22 asking for company causalities, it is misguided,

23 because in the process of signaling, the first

24 question is does the drug cause the ADR, and that

25 requires a population approach.

1 And actually Judy Staffa at the FDA has

2 communicated this very well in several DIA sessions,

3 and even on the FDA website. There are nice slides

4 there that clearly communicate this concept for anyone

5 who doesn't understand it.

6 So individual case causality really has no

7 place in terms of signaling. Now the case here is

8 that one actually develops and that is where

9 ultimately we need to address that question as to

10 whether the drug causes the ADR.

11 Now, what I haven't heard so far today is

12 that a lot of the time we actually achieve that

13 concept of threshold, and not based on some very

14 specific or direct causality assessment.

15 But from things like the clinical pattern.

16 You know, either from time dependency, or time to

17 onset, or clinical features of the case. And a lot of

18 the time I find that is where the signal really lies,

19 and this is where the case disposition is very

20 important, how you achieve that case series that you

21 now say that this is the analysis data set.

22 We have heard during this workshop how

23 several companies exclude relevant individual cases

24 from that analysis data set simply because they look

25 at risk factors involving patients, and they interpret

1 that as alternative etiological factors.

2 And then there is also the misuse of risk

3 factors which are confounders, but all these patients

4 who have risk factors, there is a pattern there. The

5 question is there is any sub-population or subgroup at

6 risk for the event due to the drug.

7 And so it is primarily -- I think the

8 issue is one of pattern recognition, rather than some

9 sort of algorithm that tells you that, yes, there is a

10 causality threshold there.

11 I am not dismissing that, you know, in

12 totality, but a lot of the time we recognize signals

13 simply based on pattern recognition.

14 DR. FENISHELL: Bob Fenishell, Washington.

15 We have heard that this has to be determined on a

16 case-by-case basis, and I think that most people have

17 come to roll their eyes at that, because it seems to

18 be non-communicative.

19 Nevertheless, in this case I think that is

20 quite applicable and I guess the analogy to make is do

21 we ask for causality on the efficacy side. When we

22 were receiving hypertensive trials, did we ask the

23 sponsors, well, let's look at this patient.

24 Do you think that the blood pressure

25 change was attributable to the drug in this patient.

1 What about this other patient. So we never ask that.

2 It seems senseless, in part because the variance -- or

3 what we were trying to derive from a large population

4 of patients was a result which could not be derived

5 from one in the period of observation that we had.

6 And in a different trial design it could

7 have been. There are trials to show the efficacy of

8 antihypertensives, for example, in single patients.

9 It can be done.

10 But that is not the way that we did it,

11 and that is not the way that most people do it in

12 hypertension. Now my understanding, and now I am

13 going to areas where my clinical experience is much

14 less.

15 But my understanding was that in the anti-

16 infective area, even within the agency, even in areas

17 of efficacy, individual patients were being followed

18 in a sense that we never followed individual patients

19 in anti-hypertensive trials.

20 That an application for an anti-infective

21 product is really a mass of little micro applications,

22 one of which might be does this drug really work for

23 serratia infections of the central nervous system

24 where there is no localized abscess.

25 Well, there are probably about five people

1 who have ever had that disease in the history of the

2 world, and the sponsors managed to get three of them

3 into the trial.

4 And so the real question is for each of

5 those patients, gee, did that patient really have

6 that. Is this confounded by other things that were

7 happening with the patient at the same time because he

8 was very sick, and he was getting six other treatments

9 and so forth and so on.

10 And so there was patient by patient

11 causality consideration, which was necessary. I think

12 it is going to depend on what the ADR is, and what the

13 population is, and what the intrinsic variance is,

14 whether one is able to approach the question as a

15 variance reduction through sample size operation, or

16 whether one is groping as one often is in safety

17 matters with individual cases that must be

18 individually massaged.

19 So there is no simple answer and I hate to

20 say it, but we are back to case by case.

21 DR. SZARFMAN: Except for a famous case of

22 interaction between (inaudible) that was a single

23 patient, and there was a blood level that was too

24 high, and the patient had no risk factors. It was a

25 young woman that received the two medications at once.

1 And this brings up that I was a little

2 disappointed that being a clinical pathologist that

3 the comment of (inaudible) that we couldn't use blood

4 level information, because I thought that this could

5 be a very objective way of -- we have done it for

6 toxicity for certain drugs.

7 And I don't think that developing blood

8 tests where you can measure blood levels will be

9 really expensive.

10 DR. GALSON: Thank you. Are there any

11 other comments from the audience?

12 DR. DANA: Can we switch gears to

13 registries?

14 DR. GALSON: Absolutely.

15 DR. DANA: I am Joanna Dana in Regulatory

16 Affairs for Amlin Pharmaceuticals. Having been in

17 this industry for a number of years, probably more

18 than most of you around here, I kept seeing patient

19 registries, patient registries, meetings on patient

20 registries.

21 I was involved with Accutane and some of

22 the early issues with that, and as I started to go to

23 meetings to get a better appreciation of what was

24 meant by this patient registry comment, two statements

25 came out that were somewhat problematic.

1 One, 65 percent of all recently approved

2 drugs now have registries; and the second one is that

3 35 percent are FDA mandated. Now, I don't know if

4 this is a true statement or not, and it came from a

5 meeting that was entitled, "Patient Registries."

6 And it may have its own sort of internal

7 self-fulfilling purposes for those that spoke at that

8 meeting. But the issue in listening to the various

9 presentations there was that the definition of patient

10 registry there was quite different than what I see in

11 this document.

12 And to take the spectra of definitions

13 that are out there for what compiles anything from

14 what used to be called in a seating study to one that

15 is a very specific pregnancy registry, and for one

16 that is therapeutic utilization and call it all the

17 same thing, is problematic.

18 So I think clarity of that definition

19 within this document would be extremely useful.

20 DR. GALSON: Thank you for that comment.

21 Any other follow-ups on that registry issue or other

22 registry issues? Okay. If not, I am going to turn

23 the mike over to Julie Beitz for some closing

24 comments.

25 DR. STEPHENSON: Before you do that, we

1 had a question for the FDA.

2 DR. GALSON: Okay. Oh, the second

3 question, your second question.

4 DR. STEPHENSON: Yes.

5 DR. GALSON: Why don't you read it again,

6 unless you want to put it up on the screen.

7 DR. STEPHENSON: It had to do with --

8 since both FDA and industry tend to use

9 pharmacoepidemiology data studies, the question was,

10 it would be helpful to hear FDA's thoughts on when a

11 study should be conducted by the company, versus the

12 FDA, or both.

13 And then the more specific question is

14 that if the FDA is going to be conducting a

15 pharmacoepidemiology study on a company's drug, are

16 there any obligations to include the company or

17 communication, or collaborate, with the company. That

18 was the question.

19 DR. GALSON: Who wants to go after that

20 one?

21 DR. RACZKOWSKI: I will go ahead and

22 start. Victor Raczkowski. The paradigm in general at

23 the Food and Drug Administration is that typically

24 sponsors perform studies on their products.

25 And my sense is that what we often times

1 require in those cases, however, is that we have some

2 sort of -- and particularly on the pre-approval side,

3 is access to the data to be able to validate the

4 results of studies, and efficacy studies in

5 particular.

6 And my initial reaction to seeing that

7 question is that it seems likely that in the post-

8 marketing arena as well that generally these Phase IV

9 type studies on safety issues would be conducted by

10 sponsors.

11 But I do think that raises the issue of

12 then how do we go and look at the data, and how do we

13 validate the patient population, and what are the

14 tools that we need at the FDA if we are not doing the

15 studies ourselves in order to be able to give us some

16 level of comfort that the result is as it is stated.

17 I would also say, however, that this is --

18 you know, we are trying to be somewhat innovative in

19 this area, and we are making great attempts to see

20 what types of resources and databases that we might be

21 able to utilize, such as claims databases, in order to

22 answer particular public health questions.

23 And Judy can talk about that more

24 specifically, but I think that there has not been a

25 lot of attention to this necessarily in the past, and

1 so we are trying to be very proactive in that respect,

2 in terms of trying to find out how we might be able to

3 use databases that were not particularly designed to

4 answer public health questions.

5 But that also requires a certain level of

6 validation, such as access to medical records, and

7 that sort of thing.

8 MS. HOSTELLEY: I think the question

9 behind the question was sort of the awareness that the

10 agency is moving into today's world into consideration

11 of doing pharmacoepidemiologic studies like through

12 various academic centers, such as through the CERTS

13 organizations, et cetera.

14 And so I think that the driver for the

15 question was that apart from perhaps seeing a listing

16 on the web that this is a study being undertaken

17 through a grant provided by the agency, PhRMA was

18 asking the question is it routine to sponsor that at

19 least that you are doing that since it does involve

20 the sponsor's drug.

21 So that -- I mean, it is helpful to have a

22 dialogue with the sponsor in order to ensure that the

23 sponsor is not sitting there thinking about doing the

24 same type of study. So that we can figure out a way

25 to better collaborate and more effectively utilize

1 resources.

2 DR. GALSON: Did you want to add something

3 to that?

4 DR. RACZKOWSKI: I just wanted to say that

5 I think that is a fair comment.

6 DR. BEITZ: What I actually was going to

7 say was that at least some of our resources that we

8 have through cooperative agreements, for example, are

9 fairly limited in scope, and so I think the kinds of

10 studies that we might design to do in using the funds

11 there, versus the kinds of studies you might be able

12 to conduct are quite different.

13 But there is no doubt that some dialogue

14 might be useful.

15 DR. BRAUN: I just wanted to mention from

16 the observational studies, that if you look certainly

17 in the cancer arena and to some extent also in

18 cardiovascular disease, it is well known that the

19 studies don't agree all the time.

20 In fact, there is a large amount of

21 disagreement among findings of independent studies,

22 and I think that is an important consideration when we

23 talk about the numbers of studies in this area.

24 So if there is an important question one

25 could look at it as actually a strength that there are

1 two separate groups looking at it independently. And

2 so I am not sure -- you know, there is a concept of --

3 some might say that it is not cost effective to

4 duplicate efforts.

5 In my view, it is not duplication, and in

6 safety systems redundancy sometimes is actually a good

7 thing to have. So I see some merit in actually doing

8 that. So independently and actually what might seem

9 in a duplicative way. So that was my point.

10 DR. STAFFA: I think that Miles makes a

11 good point about the -- there is not -- it is not

12 always bad to have two independent studies going on of

13 the same issue at the same time, because the

14 differences in those findings can be instructive.

15 But I also think that it is an intriguing

16 idea to have more dialogue and collaboration. I know

17 as a reviewer of protocols that come in that we would

18 like to see some of our comments and issues be

19 incorporated into the protocols, and I am sure that

20 the same is true, vice-versa.

21 And I think that we would welcome

22 insights. One of our frustrations and one of the

23 reasons that we are trying to build more capability to

24 do studies on our own is that we sometimes find that

25 in collaborative efforts, when we can't touch the

1 data, we can't always do the designs or ask the

2 questions in a way that we would really like to see

3 them asked. And so sometimes that is really what is

4 driving us to do that.

5 DR. GALSON: Let me just make one comment

6 on that myself, which is that clearly we don't have an

7 agency policy on this question right now, and I do

8 think that it would be very valid for us to cover this

9 in the guidance document, because it clearly is

10 something that deserves clarity, in terms of know

11 where we stand on this question.

12 So we will definitely take that into

13 consideration. Are there any other follow-ups on that

14 issue? Any additional issues?

15 MS. BORSTU: Yes, maybe one. I will say

16 that I think --

17 DR. GALSON: Just identify yourself.

18 MS. BORSTU: Maria Borstu (phonetic),

19 Organom International USA. My question pertains to

20 the very far end of the third concept paper, I think,

21 but it links into this. We were talking about

22 multiple parties studying the same issue, and

23 consistency of results.

24 But what we know from history usually

25 there isn't consistency in results, especially if we

1 have more than just the FDA or the government, and the

2 PhRMA industry is studying the issue as soon as you

3 have a -- how do you call that -- academia,

4 government, industry, studying and sometimes even

5 duplicate partners.

6 Are there any thoughts or have there been

7 any thoughts during the discussions about guidance on

8 interpretation of the findings from the different

9 sources from the different studies done by the

10 different parties using different sources.

11 I was thinking that there would be a need

12 somewhere along the line for norms and criteria as to

13 put some weighting on the various studies that we all

14 are going to do. I was just curious whether that has

15 been on the table already.

16 DR. GALSON: Judy.

17 DR. STAFFA: Actually, I think that is an

18 excellent point and I think that is something that we

19 are planning to tackle when we get to the guidance

20 stage.

21 Clearly we are going to have to at least

22 lay out some criteria that tries to speak to that

23 issue, because it happens routinely.

24 DR. UNGER: Ellis Unger from CBER. This

25 is a miscellaneous comment. At the end of the day,

1 irrespective of how a safety signal is generated,

2 whether it is data mining or what have you, we have to

3 apply judgment.

4 And the quality of the judgment is

5 inextricably tied to the quality of the data, and at

6 the end of the day the FDA reviewer has maybe 15 forms

7 on his desk or her desk.

8 And the quality is often marginal, and

9 sitting here I keep thinking about the potential of

10 the internet for collecting the ADRs. If there were

11 an interactive system developed, whereby what is the

12 patient's problem, and check box, liver problem, and

13 then it went through an algorithm and took it all the

14 way out to did the patient have a biopsy, yes, no,

15 don't know, et cetera, obviously it would take time to

16 develop.

17 But eventually we could get some very good

18 data I would think from a system like that, and I will

19 just put that idea out there.

20 DR. KAHN: Can I answer that to some

21 degree? Without naming names, which is not

22 appropriate here, in fact such a system has been

23 developed by an independent private vendor and I am

24 not sure to what extent it has been tested and

25 validated.

1 But I have actually seen a demonstration

2 of exactly such a system.

3 DR. GALSON: Okay. Thank you. Anything

4 else? All right. I would like to turn the mike over

5 then to Dr. Julie Beitz.

6 DR. BEITZ: As you have heard today, there

7 are many challenges to be faced in carrying out good

8 risk assessment based on observational data sources.

9 Some of the more thornier questions that came up today

10 revolved around how can the quality of adverse event

11 reports be improved, and what conclusions can be drawn

12 from observational data sources given the inherent

13 limitations of these data.

14 And what more can be done to enhance our

15 understanding of safety signals once they are

16 identified. On behalf of the members of Group III,

17 the CDER/CBER Pharmacovigilance Working Group, I would

18 like to thank all of the attendees today for their

19 well considered comments and questions.

20 We plan to consider all the comments that

21 we received today, as well as those submitted to the

22 docket when our prepare our draft guidance document.

23 I also would like to thank all the members of the

24 steering committee and of the working group itself for

25 their scientific input on the concept paper thus far.

1 But I would also like to thank Lee Lemley

2 and Diane Erlich for organizing a very successful

3 public workshop. And now I turn it over to Dr. Galson

4 to close.

5 DR. GALSON: Thank you very much, Julie,

6 and I want to just very quickly thank a number of

7 people in particular today, the working group on

8 pharmacoepidemiologic. You have done a wonderful job.

9 The document was excellent, and we had good discussion

10 today.

11 Also the members of the executive

12 oversight committee who participated today and

13 throughout the week. Specifically, I want to thank

14 the project managers for each of the three working

15 groups. These products and this meeting wouldn't have

16 gone so well if it was not for your hard work.

17 And to all the people sitting up here

18 today, and all the audience for your participation,

19 thanks very much. Please submit written comments if

20 there are things that you feel haven't been covered.

21 The docket is going to be open and we need

22 those comments for us to produce high quality draft

23 guidance documents over the next few months. Thanks

24 again.

25 (Whereupon, at 3:26 p.m., the meeting was

1 concluded.)

2 3 NEAL R. GROSS 4 COURT REPORTERS AND TRANSCRIBERS 5 1323 RHODE ISLAND AVE., N.W. 6 (202) 234-4433 WASHINGTON, D.C. 20005-3701 www.nealrgross.com