What Is Junk Science?

Second Edition

SCIENTIFIC CONSULTANTS to INDUSTRY and COUNSEL www.innovativescience.net ● 973.889.1600 The Litigator’s Guide to Combating Junk Science

Table of Contents Table of Contents ...... 2

Introduction ...... 3

What Is Junk Science? ...... 6

Limitations of the Peer-Review Process ...... 12

Teaching Science To Jurors ...... 15

An Example: Explaining Complex Science/Statistics Using Trial Graphics ...... 18

Anti-Junk Science Websites ...... 25

How to Evaluate Scientific Sources Online ...... 29

Anti-Junk Science Books ...... 31

Resources for Regulatory Guidance ...... 37

Resources for Scientific Standards ...... 43

Placebo and Nocebo Effects ...... 46

Preparing An Expert To Testify ...... 48

More Preparation For Your Expert ...... 56

It’s Not Just Science That Can be “Junk”: Trial Graphics Can Be, Too ...... 58

About Us ...... 63

The Litigator’s Guide to Combating Junk Science

Introduction

In many legal cases, science plays a critical role in the courtroom. Access to scientific research and an understanding of scientific principles, as well as the ability to convey this information to jurors and the court in an effective way, enable the litigator to build a powerful case if these tools are properly used. The team’s expert witnesses must effectively communicate to the judge and jury technical concepts that support the central arguments of the case and show how they fit within the relevant law, but first they must be able to sort the sound science from the junk science.

Examples of legal cases that often rely heavily on scientific information and testimony:

■ Personal Injury: Scientific evidence can be important to any personal injury case, but it is almost always critical in mass torts litigation involving allegations of personal injury from a product, an environmental exposure, or a chemical exposure. In these cases, plaintiffs typically rely on scientific studies to demonstrate a link between the exposure and the injury, while the defense typically presents scientific research to cast doubt on the purported causal relationship between the exposure and the injury.

■ Consumer Fraud: In these cases, a plaintiff’s claim that a product failed to achieve the results promised, the defense typically argues that its marketing claims were supported by reliable scientific evidence. The plaintiff typically then reply with evidence that questions or counters the defense’s data.

■ Medical Malpractice: To counter a plaintiff’s claim that a healthcare provider’s error resulted in harm, the defense will typically present scientific studies to demonstrate that the provider adhered to the generally accepted standard-of-care.

■ Securities and Shareholders Lawsuits: In securities and shareholder lawsuits involving medical devices or pharmaceutical products, the defense typically presents scientific evidence to demonstrate that the information relayed to shareholders was reasonable and reliable.

■ Patent Challenges: These types of case generally focus on science in the form of technology. In such cases, often the plaintiff’s challenge is to establish that the

The Litigator’s Guide to Combating Junk Science

defendant’s actions infringe the plaintiff’s patent and in doing so may rely on scientific evidence relating to how that technology is implemented in the real world. Moreover, the defendant will have their own scientific evidence to counter the plaintiff’s, as well as potentially adding scientific evidence relating to the state of the prior art with respect to the asserted patent in an effort to invalidate its claims as a defense to infringement.

There are certainly many other examples of the importance of science to litigation, but those set forth above are common. The parties involved in any legal dispute will typically engage in a debate about the validity of the scientific evidence presented. Even the experts often disagree when interpreting sound scientific data.

However, dubious or biased scientific information is occasionally presented in court cases and unfairly influences outcomes. Judges and juries who lack the background and knowledge to properly evaluate the validity of scientific data tend to accept the information placed before them, for better or worse.

When a case relies on misinformation, unsubstantiated claims, and misleading data, opposing counsel can successfully counterattack by using and providing access to the right resources.

In the following pages, we provide you with tools and resources you need to combat junk science in the courtroom. The remainder of this Guide is structured as follows:

■ What Is Junk Science identifies examples of junk science. How can you combat junk science if you cannot identify it. This section provides information and resources to do just that.

■ How to Evaluate Scientific Sources Online. Further to the previous section, here we provide you with a checklist for identifying credible scientific sources. If you can identify the “good” then you can spot the “junk.”

■ Limitations on the Peer-Review Process. Peer review is one of the foundations of good science, but the concept is also abused to push junk science. How do you tell the difference? This section sheds some light on the issue.

■ Anti-Junk Science Websites provides access to resources dedicated to exposing junk science. You are not alone in the battle against junk science. In this section we provide a plethora of resources developed by your allies in this endeavor.

The Litigator’s Guide to Combating Junk Science

■ Anti-Junk Science Books lists publications focused on illuminating the issue of junk science. The fight against junk science in the courtroom has raged for many years. This section identifies some terrific resources for continuing this fight.

■ Resources for Regulatory Guidance provides access to government resources that will allow you to counter misinformation with scientifically sound principles.

■ Resources for Scientific Standards provides access to reputable online resources that present generally accepted scientific methods.

■ Placebo and Nocebo Effects provides the latest research on this medical phenomenon, which is central to many cases, and pharmaceutical-related cases in particular.

The Litigator’s Guide to Combating Junk Science

What Is Junk Science?

Legitimate controversies erupt daily in political arenas, the news media, the courtroom, and the court of public opinion. In these debates, valid evidence usually emerges to support both sides. However, in some controversies, one or both sides rely on faulty data—or junk science—rather than data generated using rigorous scientific methodologies.

The term “Junk Science” refers to scientific research that fails to meet the basic standards of a rigorous scientific investigation, such as falsifiable hypotheses, replication, peer-review and publication, general acceptance, neutrality, objectivity, and adherence to the scientific method. Another layer of complexity has also emerged with the revolution in electronic scholarly publishing, which has spawned predatory open-access publishers and journal.

Purveyors of junk science typically have an agenda, and their research and results are driven by non-scientific motives that may be political, financial or ideological.

Junk science is different from science that is merely imperfect or incomplete. Junk science studies do not adhere to the core scientific principles described above, although they present themselves as doing so. However, the scientific process is not perfect and our understanding of

The Litigator’s Guide to Combating Junk Science certain topics may change as new data emerge. The following are some qualities of solid scientific research:

■ Peer reviewed

■ Large sample sizes, when possible

■ Control for independent variables

■ Use blinding, randomization, and appropriate control groups

■ Objective and unbiased

Junk science fails one or more (or all) of these tests.

For instance, a case report or case series may present stories of several people who experienced unpleasant symptoms when working in an office with Wi-Fi radiation. This does not support a causative link between Wi-Fi and adverse health effects. The sample size is too small, these types of reports are not designed to test for associations, and because the information is anecdotal and observational in nature, there is no way to rule out the many other possible alternative causes for the subjects' symptoms. In addition, if the subjects knew they were being exposed to Wi-Fi, their symptoms could have been caused by their belief in its harmfulness— the nocebo effect.1

1 The nocebo effect, related to the better-known placebo effect, is a psychological and physiological phenomena relating to patients expression of symptoms because of their own anticipation of those symptoms. This topic is dealt with in greater detail below.

The Litigator’s Guide to Combating Junk Science

Of course, the scientific process is far from perfect. Even properly conducted scientific studies can produce misleading or conflicting results. For example, in 2009 the L.A. Times reported on a pair of studies on antidepressant effectiveness. An Italian meta-analysis of recent research ranked Zoloft and Lexapro as the most effective antidepressants, while a U.S. meta-analysis found that all the drugs on the market were about equally effective. The authors of the news article noted that “[b]oth teams took pains to avoid influence or bias,” so what caused the disparity? At a basic level, it came down to methodology. The two research teams used different statistical methods to compare the medications. These conflicting data do not suggest that either

The Litigator’s Guide to Combating Junk Science team had a prescribed agenda or failed to adhere to strict scientific principles. Rather, the conflicting data merely suggest that more research is needed on antidepressant effectiveness.2

Sometimes sensational claims can masquerade as legitimate science. In 2012, a paper on toxicity from genetically modified (GMO) corn was published in the respected journal Food and Chemical Toxicology. The paper reported that when rats were fed corn resistant to the pesticide Roundup, they developed tumors at a higher rate than normal. The study was criticized by many scientists because of its small sample size, questionable treatment of animals, lack of statistical analysis, unclear results, and use of rats that were unusually predisposed to tumors. Overall, the questionable methodologies coupled with the strong conclusions from this study was an initial signal that this could be a junk study. The journal editors ultimately retracted the paper on the grounds that the data were not conclusive.

This GMO study example shows that junk science can initially deceive even leading experts in the field and squeaking its way into a major scientific journal. As a result of this deception, anti- GMO activists and some media outlets continue to cite to this study to bolster their view that GMOs are toxic and can cause cancer. This example also demonstrates that junk science can be successfully challenged through appropriate channels. However, oftentimes the damage is done and the study continues to be cited as legitimate across various venues.

To illuminate the problem, below we provide examples of controversial claims based on junk science, and links to resources refuting their questionable claims. The claims below are based on junk science and they were each initially accepted by some segment of the general and scientific community. Luckily for us, we now know better thanks to the work of real scientists.

Controversial Claim #1: Electromagnetic radiation causes adverse health effects. Junkscience.com - EMFs and Cell Phones National Institute of Environmental Health Sciences - Electric and Magnetic Fields

2 These studies are distinguished from junk science since both used sound scientific principles, and made appropriate conclusions based on their methodologies. There was no indication that the science was being used to prove some preconceived opinion by the authors.

The Litigator’s Guide to Combating Junk Science

Controversial Claim #2: Endocrine disruptors disturb human hormone balances. SafeChemicalPolicy.org - Endocrine Disruptors Sciencebasedmedicine.org - Endocrine disruptors—the one true cause?

Controversial Claim # 3: Hydraulic fracturing leads to adverse health effects for nearby residents. Junkscience.com - Penn pilot study: Group of Bradford Co, Pa. residents concerned about health effects of hydrofracking K&L Gates Law Firm - Junk Science Meets Hydraulic Fracturing: Unfounded Environmental Scare Threatens Energy Development

Controversial Claim #4: Multiple chemical sensitivity causes a host of adverse health effects. Junkscience.com - Multiple Chemical Sensitivity Testimony Inadmissible Under Daubert, Court Says Skepdic.com - multiple chemical sensitivity (MCS) from Skeptic’s dictionary

Controversial Claim #5: Vaccines lead to autism. Junkscience.com - The risk of autism is not increased by ‘too many vaccines too soon’ Nature - A case of junk science, conflict and hype

Controversial Claim #6: Ex-gay conversion therapy is effective at changing sexual orientation. LGBTScience.com: Empirical evidence (ex-gay) Psychology.ucdavis.edu: Attempts to change sexual orientation

Of course, these examples are not junk merely because they were controversial. The fact that an idea is controversial does not, by itself, mean it is junk. Scientists may disagree on a topic because there is legitimate evidence on both sides. An example of this type of healthy scientific debate is whether red wine is good for your health or not. There are plenty of well-conducted

The Litigator’s Guide to Combating Junk Science studies supporting both sides of the argument. Conversely, purveyors of junk science may present their widely rejected ideas as simply “controversial.” For this reason, it is crucial to seek out an unbiased spectrum of scientific resources to obtain the big picture of the debate rather than relying on the results of any single study.

The Litigator’s Guide to Combating Junk Science

Limitations of the Peer-Review Process

The practice of peer-review is one of the scientific establishment's strongest weapons against human error and bias. Unfortunately, the peer-review process is not perfect and it certainly cannot always ensure the integrity or quality of the science under review. Problems with the peer review process in today's scientific publications have increasingly caught the attention (The Economist article on the subject) of the press and research community, and some of these issues are discussed in more detail below.

One problem is a widespread lack of replication. In theory, if an experiment's conclusions are valid, they can be repeated by any researcher who performs the same experiment. But in reality, most experiments that are written up in journals are never repeated by anyone, so their reproducibility is never tested.

However, in today's hyper-competitive academic world, there is often little professional benefit for scientists in testing an old experiment to see if the results can be repeated. Indeed, the most prestigious journals actually turn down the vast majority of the submissions they receive, and the articles that are accepted are more likely to be those that make a new or startling claim.

When attempts are made to replicate findings, they often fail. The Economist reported that "researchers at one biotech firm, Amgen, found they could reproduce just six of 53 'landmark' studies in cancer research." The reasons for the discrepant results can range from poor study design to mere statistical chance. According to widely accepted standards, a set of data is statistically significant if there is a 5% chance or less that it is

The Litigator’s Guide to Combating Junk Science the product of random chance. But this means that of all published data sets, 1 in 20 could be misleading. With little attention to reproducing studies, it is difficult to determine which accepted theories deserve to be rejected.

An even more damaging problem is the proliferation of journals offering fake "peer review." Science magazine and other publications have reported on the growing threat of open access publications that will publish almost any manuscript in exchange for a "publication fee" from the author. These publications are often based in the developing world (although they may claim to be American or European), and offer no substantive review process for submitted work.

In 2005, three MIT students set out to prove the existence of fraudulent journals by creating software, SCIgen, that could generate a gibberish "scientific paper" on any topic. They made the software available for free and encouraged others to use it. Since then, Science reports, "85 SCIgen papers were discovered in the published proceedings of 24 different computer science conferences between 2008 and 2011"—some submitted as hoaxes, and others by unscrupulous scholars hoping to boost their publication record. Another scientist, John Bohannon, intentionally created a paper filled with glaring scientific flaws, and found that it was accepted by 197 of the 305 open-access journals to which it was submitted.

Distrust any single Distrust any paper paper that makes a that has rarely or spectacular claim, never been cited by unless it is backed up other researchers, or by a body of work that fails to describe that points in the its research methods same direction. in detail.

Most importantly, If you have doubts about a journal, look don't assume someone has science it up on Beall's List, a on their side simply regularly updated list of questionable or because they can point to a "peer- predatory journals reviewed" article that and publishers. supports their ideas.

The Litigator’s Guide to Combating Junk Science

All this makes it much more difficult to distinguish fact from junk. But there are some rules of thumb you can follow:

The Litigator’s Guide to Combating Junk Science

Teaching Science To Jurors

Very often, trial attorneys in complex cases need to explain extremely difficult and elusive scientific concepts to jurors who are not well versed in science. The lawyer’s job is to convey the science correctly to the jury so that they can make a rational decision – yet not to bury the jury under a blizzard of scientific terms and concepts that they will never understand.

The answer is to use visuals in the form of photographs, schematic diagrams, animation, timelines, demonstrative evidence, document call outs or whatever is suited to the situation, and to explain them in terms that jurors who are not specialists in the scientific subject can understand.

Analogies (in other words, what is something like?), contrasts (how is something different from something else?), and simple definitions (what are the components of an object? how is it used?) are very useful tools for the trial lawyer.

As Jan D’Arcy wrote in 1998 in Technically Speaking, “Many scientific subjects are hard to describe; they can be difficult to see, touch, measure or imagine. A presenter should find ways to illuminate a concept in known terms with the least amount of distortion. . . . Comparisons and contrasts are two of the best ways to translate your information clearly to your audience. Similes, metaphors, and analogies are comparisons that can often lead to amazing insights.”

The brief movie linked-to here shows how restenosis (the formation of new blockages at the site of an angioplasty or stent placement) can form in blood vessels when a non- drug-eluting stent (one that does not contain an anti- stenosis drug) is used by a heart surgeon. This is a highly technical medical subject, yet after seeing the presentation, jurors will understand how stents work and why such drugs are used.

The Litigator’s Guide to Combating Junk Science

Below, we created a very straightforward, highly memorable patent litigation graphic that shows one person walking his own path, away from conventional wisdom, to show that an inventor’s idea was unique and non-obvious.

Similarly, we have devised a 78-second video presentation that details the challenges of inbred reproduction, and the advantages of hybrid reproduction, in the corn plant. This is easily understandable to a juror, even one who does not have a background in biology or food science.

(link to video)

Finally, the schematic diagram below uses the excellent analogy to the letters and numbers in a license plate – an object familiar to jurors – to indicate how many possible structures of a chemical compound can exist and thus how the one structure designed by a client was not obvious and therefore was deserving of patent protection.

The Litigator’s Guide to Combating Junk Science

We believe that no scientific concept is too difficult to teach to a jury. In our history, we have found a way to successfully teach and persuade about everything from the genetic development of cancer, genetically modified corn, stem cells, physical separation in patented pharmaceuticals, metal fatigue, the transportation of air, water and ground pollution, DNA, bioequivalence, how allergies work, epidemiology, physics, chemistry and countless applied science medical principles.

With the right combination of trial team, trial consulting firm and expert consulting firm, any concept can be made understandable by combining a good explanation and a good visual.

The Litigator’s Guide to Combating Junk Science

An Example: Explaining Complex Science/Statistics Using Trial Graphics

Many of us have been there in the course of a trial or hearing. An expert or opposing counsel starts spouting obscure statistical jargon. Terms like "variance," "correlation," "statistical significance," "probability" or the "null hypothesis." For most, especially jurors, such talk can cause a mental shutdown as the information seems obscure and unfamiliar.

It’s no surprise that talk of statistics causes confusion in a courtroom setting. Sometimes, a number can be much higher than another number and yet the finding will not be statistically significant. In other instances, a number can be nearly the same as its comparison value and this difference can be highly statistically significant.

Helping judge and jury develop a clear and accurate understanding of statistical principles is critical – and using the right type of trial graphics can be invaluable.

Let’s demonstrate this by way of example.

Suppose we want to know whether a petroleum refinery increases the level of benzene in fish that inhabit the coastal

The Litigator’s Guide to Combating Junk Science waters near the refinery.

The hypothesis is that the benzene level in the coastal fish near the refinery (the Refinery Fish) is higher than the benzene level in off-shore fish that live in waters far from the refinery (the Control Fish).

Because we can never collect every single fish and measure benzene levels in all of them, we will never know the precise answer to the hypothesis (not to mention the fact that if we did, the study would be irrelevant because there would be no more fish). But we can sample some of the fish near the refinery and then compare the benzene levels in these fish to a sample of fish collected from the middle of the sea. Statistical techniques are a clever tool that we use to answer the research question, even though we haven't measured all the fish in each location.

Unless one is trained in statistics, the evaluating might appear easy and straightforward. Simply compare benzene levels in the Refinery Fish sample to the benzene levels in the Control Fish sample and see which is higher. But what if our sample only reveals a very small difference between the benzene levels in the Refinery Fish sample compared to the Control Fish sample? How do we know if that difference we observed in our samples is a real difference (i.e., potentially due to a causal relationship with the refinery) or whether it was simply due to our sampling techniques (i.e., due to chance)? Statistical techniques provide us with a way to properly interpret our findings.

An overview of well-established statistical techniques surrounding hypothesis testing is in the trial graphic at left:

While this graphic is somewhat oversimplified, it does provide the basic steps that are taken in the hypothesis testing decision tree.

The Litigator’s Guide to Combating Junk Science

Although imperfect, a criminal case serves as a useful analogy to help understand how statistics work. In a criminal case, the defendant is assumed to be innocent unless proven guilty beyond a reasonable doubt. In statistical terms, the overall trial can be likened to statistical testing of a hypothesis (i.e. did he do it?), and the presumption of innocence can be likened to the "null hypothesis." Like the null hypothesis, the starting point in a criminal trial is that defendant is not guilty, and in statistical terms, that the connection you've set out to establish is just not there. The trial graphics below provide an overview of this concept. Again, this is an imperfect metaphor and is subject to criticism from a pure statistical vantage point. Neverteless, it provides some assistance to the novice in clarifying the fundamental tenets of hypothesis testing.

Returning to our refinery hypothetical, we form our null hypothesis.

In this case, the null hypothesis is that the Refinery Fish are exactly the same as all the other fish in the ocean in terms of benzene levels — specifically, that they come from the same population. Succinctly, the null hypothesis is as follows:

Null Hypothesis

There is no difference in benzene levels between the Refinery Fish and the Control Fish.

In our study, as in all scientific studies, we will be testing how likely it is that we would obtain dataat least as extreme as our data if the null hypothesis were true. In other words, we will be evaluating the conditional probability of obtaining

The Litigator’s Guide to Combating Junk Science the data that we observe.

In plain English, proper statistical testing means assuming your hypothesis is wrong and then evaluating the likelihood that you would come up with the findings that you did. Statistical testing is not about proving things true. Rather, it is about proving that the alternative — i.e. your null hypotheses — is likely not true. Only then can we reject the null hypothesis and conclude that our research hypothesis is plausible.

Determining whether or not it is reasonable to reject the null hypothesis is done by collecting data in a scientific study. Here, we start by measuring benzene levels in two samples of fish: (1) a group of fish near the refinery (Refinery Fish); and . . .

(2) a group of fish in the middle of the ocean, nowhere near the refinery (Control Fish).

We will then calculate an average benzene level in each group of fish, which will serve as a reasonable estimate of the benzene level in each population of fish (i.e., all fish living near the refinery and all fish not living near the refinery). Of course, how we take our samples is a critical component of the study design, but we will assume for this example that we have used appropriate sampling techniques.

Let's examine 3 possible outcomes in the trial graphics below. The first possibility will deal with an obvious result.

In this example, let's assume that every fish in the Refinery Fish sample had a benzene level of 10, and every fish in the Control Fish sample had a benzene level of 1. Thus, the average Refinery Fish benzene level is 10 and the average Control Fish benzene level is 1. When we do our statistical test, we calculate the conditional probability – i.e., the probability that we would have obtained this dramatic difference (10 vs. 1) given that the null hypothesis is true. This probability is called a "p value."

The Litigator’s Guide to Combating Junk Science

In this case, the p value is so low (let's say: p = 0.00000001) that we reject the null hypothesis. Stated another way: The probability of obtaining such extreme data if the null hypothesis were true is 0.0000001. Based on this analysis, it doesn’t make sense to believe that we would have obtained these results if the null hypothesis were true. So we reject the null hypothesis.

Our study was a success. We reject the null hypothesis, and we draw a clear-cut conclusion -- i.e., the Refinery Fish come from a different population of fish with respect to benzene levels. So we conclude that the refinery, absent other factors, may have something to do with the benzene levels in these fish. Because this difference was so clear-cut (every single fish in the Refinery Fish sample had extremely high benzene levels and every single fish in the Control Fish sample had extremely low values), we didn’t even need statistics to get our answer.

Now let's look at another, more realistic, possibility. This time the difference between the two samples is a little less clear cut.

In this example, the average benzene level in the Refinery Fish sample is 8 and the average benzene level in the Control Fish sample is 3. When we do our statistical test, we learn that the p value is 0.02. Said another way, the probability that we would have obtained these findings, given that the null hypothesis is true, is about 2%.

Thus, as with the extreme example above, the probability of obtaining

The Litigator’s Guide to Combating Junk Science these findings, given that the null hypothesis is true is very low (not quite as low as in the prior example, but still pretty low). This raises the question: how low a probability is low enough?

Traditionally, statisticians have used a “cut-off” probability level of 5%. If the probability of obtaining a certain set of results is less than 5% (given the null hypothesis), then scientists and statisticians have agreed that it is reasonable to reject the null hypothesis. In this case, we reject the null hypothesis and conclude that the Refinery Fish must come from a different population than the Control Fish. Again, as with the earlier example, we conclude that the refinery must have something to do, absent other factors, with the benzene levels.

So far, so good. Now, let's do one more. This time let's assume that the difference between the Refinery Fish sample and the Control Fish sample has gotten much smaller.

In this example, the average benzene level in the Refinery Fish sample is 5 and the average benzene level in the Control fish sample is 4. The benzene levels, on average, are numerically higher in the Refinery Fish compared to the Control Fish. But are they statistically higher? In statistical terms, how likely would it be to obtain these findings if all the fish were the same with respect to their benzene levels? In other words, is it reasonable to conclude we would have obtained findings this extreme if the refinery had nothing to do with the benzene levels?

When we do our statistical test, we learn that the p value is 0.25. Thus, the probability that we would have obtained findings this extreme, given that the null hypothesis is true, is about 25%. One in four

The Litigator’s Guide to Combating Junk Science times that we take these samples, we will get findings like this if the null hypothesis is true.

A twenty-five percent chance is not so unlikely. It certainly doesn't meet the 5% cut-off rule (i.e., less than 5%). Therefore, statistical best practices tell us that we cannot reject the null hypothesis.

But what does it mean when we cannot reject the null hypothesis? Can we conclude that the null is true? This is actually a critical question, and it represents an area where statistics often get misused in court, in trial graphics, in the media and elsewhere. And what about other intervening factors like bias and confounding?

Our next posts on using trial graphics and statistics to win or defend your case will grapple with these important questions. Please do leave a comment below (your email address is not displayed or shared).

The Litigator’s Guide to Combating Junk Science

Anti-Junk Science Websites

The following online resources are dedicated to exposing and combating junk science. They offer a wealth of information that counters assertions based on unreliable data. If you suspect you are facing junk science in your legal case, these resources provide a great place to begin preliminary research on the issue. These sites tend to do a great job at debunking junk science and provide straight-forward explanations of why some studies can be considered junk science.

Bad Science

Dr. Ben Goldacre, author of the bestseller Bad Science, continues his campaign against junk science in his witty and insightful blog of the same name. In his book and blog, he exposes the tenuous scientific claims of newspapers, doctors, government reports and drug companies.

Bad Science Debunked

A non-scientist-friendly guide to junk science on the internet, with a focus on nutrition and alternative medicine.

Debunkosaurus

Maintained by JunkScience.com, Debunkosaurus™ is a Wiki-based tool for researching health scares and scams.

Debunking Denialism

This site is dedicated to the refutation of pseudoscience and denialism by applying scientific skepticism and defending evidence-based science. It has taken on a wide range of topics including creationism, HIV/AIDS denialism, anti-psychiatry, crank claims about physics, vaccine rejectionism, alleged psychics, opposition to genetically modified crops, climate change denialism, misuses of statistics and many others.

The Litigator’s Guide to Combating Junk Science

Gary Taubes

Gary Taubes is a science writer and the author of Nobel Dreams (1987), Bad Science: The Short Life and Weird Times of Cold Fusion (1993), and Good Calories, Bad Calories (2007). He has won the Science in Society Award of the National Association of Science Writers three times and was awarded an MIT Knight Science Journalism Fellowship for 1996-97. Taubes’ books deal with controversies in the field of science. Nobel Dreams takes a critical look at the politics and experimental techniques behind the Nobel Prize-winning work of physicist Carlo Rubbia. Bad Science chronicles the short-lived media frenzy surrounding the Pons-Fleischmann cold fusion experiments of 1989.

JunkScience

“All the Junk That’s Fit to Debunk.” As its motto indicates, JunkScience.com’s mission is to reveal the facts behind junk science in the media and the courtroom. JunkScience takes on Global Warming theorists, the EPA, the American Heart Association and others.

LGBT Science

This site was created by the activist group Truth Wins Out to expose the ways anti-gay activists regularly distort science for political gain. By rebutting deliberate scientific distortions made by unaccomplished and unpublished faux experts, LGBT Science hopes to limit the harm caused by their misinformation.

Not Even Wrong

William H. Kinney, associate professor in the University at Buffalo Physics Department, named his website after the Wolfgang Pauli quote “It is not even wrong.” The website is described as “A page dedicated to the ample evidence that we need better science education.” Topics include vaccines and autism, EMFs, plastic softeners, silicone breast implants, and more.

The Litigator’s Guide to Combating Junk Science

QuackWatch

QuackWatch is an international network of advocates who are concerned about health-related frauds, myths and misconduct. Activities include investigating questionable claims, debunking pseudoscientific claims, and distributing reliable publications, among others. Visitors can search the website’s vast database of questionable products, services and theories by topic.

Science-Based Medicine

Science-based medicine evaluates medical treatments and products in a scientific context. “Good science is the best and only way to determine which treatments and products are truly safe and effective,” according to the website. Recent articles have analyzed placebo prescribing, cancer screening tests, and SPECT scans.

Science-ish

Science-ish, a blog on the Maclean’s magazine website, checks health-related news against the evidence to hold politicians, opinion leaders, and journalists accountable for the information they disseminate.

Sense About Science

Sense About Science USA are a non-profit, non-partisan American branch of the British charitable trust, Sense About Science, which was founded in 2003 and which grew to play a pivotal role in promoting scientific understanding and defending scientific integrity in the UK and Europe. They aim to create and curate a national conversation about the value of scientific research, the importance of evidence and transparency, and the need to better communicate the process and progress of science.

Skeptic Blog

SkepticBlog is a collaboration among some of the most recognized names in promoting science, critical thinking, and skepticism. Regular bloggers include Brian Dunning, Daniel Loxton, Donald Prothero, Mark Edward, Michael Shermer, and Steven Novella.

The Litigator’s Guide to Combating Junk Science

This site is hosted by the Skeptics Society. While the site stopped updating in September 2014, archived content addresses topics including vaccines, homeopathy and the power of prayer.

Skeptical Raptor

The author of Skeptical Raptor has over 25 years of experience in marketing, business development, and product development in the medical products industry, and aims to debunk magical thinking and pseudoscientific claims in this field. The blog focuses on identifying logical fallacies, false equivalency and other misleading tactics. It includes guest posts by law professor Dorit Rubenstein Reiss.

Skeptoid

Skeptoid is a weekly podcast dedicated to debunking the widespread “pseudosciences.” According to the website, “Skeptoid attempts to expose the folly of belief in non-evidence based phenomena, and more importantly, explains the factual scientific reality.” Topics include hydraulic fracturing, high-fructose corn syrup, DDT, vaccine ingredients, cell phone radiation, and more.

STATS

Are you interested in the latest research questioning the purported connection between cell phone use and brain cancer? Do you want to know why a leading expert condemned the crusade against BPA? Do you question the supposed link between sugary beverages and obesity? You can find information on these topics and more on the STATS website, an affiliate of George Mason University that aims to correct scientific misinformation in the media and public policy. STATS offers analyses of the latest research on key issues and links to revealing articles.

The Litigator’s Guide to Combating Junk Science

How to Evaluate Scientific Sources Online

Often, lawyers need to try on another hat, that of a scientific researcher, to zealously advocate for their clients. To effectively research a science topic, there are a few rules of thumb you can use to find reliable sources of information.

A credible scientific source should:

 Cite a representative collection of peer-reviewed scientific studies, and not just those that are supportive of their position (i.e. confirmation bias). For example, in a published review article, look for indications that the authors clearly outline their inclusion criteria, their strategy for identifying relevant data, what standards were used to evaluate evidence, and methodologies used to synthesize results across studies. Beware of reviews which rely solely on solely on narrative analysis of a cherry-picked sample of studies.

 Avoid drawing conclusions about humans based solely on animal research. Studies conducted on animals or human cells cannot be straightforwardly applied to living people, and writers should not suggest that they can. Rather, animal and in vitro data should be evaluated as part of a weight-of-the-evidence analysis.

 Avoid the “appeal to nature.” This is a logical fallacy based on the assumption that whatever is “natural” is good. Examples would be the belief that Wi-Fi radiation is unhealthy because it is a product of technology, or that some supplement must be beneficial because it is “all-natural.”

 Give an accurate picture of the existing scientific debate. A biased writer may claim, for example that there is fierce debate among scientists about the safety of vaccines, when in fact scientists overwhelmingly agree that they are safe. You can check for this form of bias by seeking out multiple reputable sources rather than relying on a single source of information. For example, on the vaccine issue, it would be prudent to consult multiple state-of-the-art reviews on the topic, as well as some authoritative sources (e.g. CDC, WHO), rather than relying on a single study conducted by a potentially biased researcher (e.g. Dr. Andrew Wakefield’s debunked studies over the safety of vaccines).

The Litigator’s Guide to Combating Junk Science

 Fully acknowledge funding sources. A credible source should be written by someone who does not have to conceal their source of funding.

 Be published in a journal that has not been implicated as a potential predatory scholarly open-access publisher or journal, including some that conduct fake peer-reviews. Jeffrey Beall maintains a website dedicated to the identification of such potential predatory journals and publishers.

With these guidelines, you will have some tools to help you sort fact from junk, no matter what topic you are researching.

The Litigator’s Guide to Combating Junk Science

Anti-Junk Science Books

Junk science thrives because some element of a claim appeals to people's deeply held beliefs or emotions. The following books offer insight into the insidious nature of junk science and how we can guard against it. If you suspect you are facing junk science in your legal case, these resources provide a great place to begin preliminary research on the issue. These books tend to provide a great background on general and specific scientific topics and are written for a non-technical audience. For example, if you need to get up to speed on sound epidemiological principles Gordis’ Reference Guide to Epidemiology will help you understand the basic concepts.

Snake Oil Science: The Truth About Complementary and Alternative Medicine R. Barker Bausell Oxford University Press, 2009

A biostatistician, author and Senior Research Methodologist at the University of Maryland, Bausell looks at alternative methods by posing the question, "Is any complementary and alternative medical therapy more effective than a placebo?" In short, his answer is no. Step by step, Bausell builds a rigorous case against alternative therapies like acupuncture and megavitamins, showing why the studies that support them are flawed.

Galileo's Revenge: Junk Science In The Courtroom Peter Huber Basic Books, 1993 288 pages

A scathing indictment of the growing role of junk science in our courtrooms. Peter W. Huber shows how time and again lawyers have used—and the courts have accepted—spurious claims by so-called expert witnesses to win astronomical judgments that have bankrupted companies, driven doctors out of practice, and deprived us all of superior technologies and effective, life- saving therapies.

The Litigator’s Guide to Combating Junk Science

Bad Science: Quacks, Hacks, and Big Pharma Flacks Ben Goldacre Faber & Faber, 2010

This book exposes quack doctors and nutritionists, bogus credentialing programs, and biased scientific studies, while taking the media to task for their willingness to throw facts and proof out the window. Goldacre shows readers how to evaluate placebo effects, double-blind studies, and sample sizes, so that you can recognize bad science when you see it.

Hyping Health Risks: Environmental Hazards in Daily Life and the Science of Epidemiology Geoffrey C. Kabat Columbia University Press, 2011

In this book, epidemiologist Geoffrey C. Kabat explains which environmental hazards are actually hazardous, and which are mere media scares. Kabat shows that many factors can contribute to the inflating of a hazard, including skewed reporting by the media as well as wishful thinking by researchers, regulatory agencies and politicians. The book includes in- depth case studies on pollutants such as DDT as a cause of breast cancer, electromagnetic fields from power lines, radon within residences, and secondhand tobacco smoke.

Inheritance: How Our Genes Change Our Lives—and Our Lives Change Our Genes Sharon Moalem, M.D., Ph.D. Grand Central Publishing, 2014

Conventional wisdom dictates that our genetic destiny is fixed at conception. Dr. Moalem's groundbreaking book shows us that the human genome is far more fluid and fascinating than your ninth grade biology teacher ever imagined. By bringing us to the bedside of his unique and complex patients, Maolem masterfully demonstrates what rare genetic conditions can teach us all about our own health and well-being.

Junk Science Judo: Self-Defense against Health Scares and Scams Steven J. Milloy Cato Institute, 2001 218 pages

The Litigator’s Guide to Combating Junk Science

This book is a simple, easy-to-read guide to debunking health scares and scams before you get hurt.

The Panic Virus: A True Story of Medicine, Science and Fear Seth Mnookin Simon & Schuster, 2012

In this book Mnookin draws on interviews with parents, public-health advocates, scientists, and anti-vaccine activists to tackle a fundamental question: How do we decide what the truth is? The fascinating answer helps explain everything from the persistence of conspiracy theories about 9/11 to the appeal of talk-show hosts who demand that President Obama “prove” he was born in America.

The Emperor of All Maladies: A Biography of Cancer Siddhartha Mukherjee Scribner, 2011

Physician and researcher Siddhartha Mukherjee offers a comprehensive and eloquent "biography" of one of the most virulent diseases of our time. The Emperor of All Maladies illustrates how modern treatments came into being thanks to a century's worth of research, trials, and breakthroughs around the globe, while shedding light on how far we have yet to go in the fight against cancer.

Voodoo Science: The Road from Foolishness to Fraud Robert Park Oxford University Press, 2001 240 pages

In a time of dazzling scientific progress, how can we separate genuine breakthroughs from the noisy gaggle of false claims? From Deepak Chopra's "quantum alternative to growing old" to unwarranted hype surrounding the International Space Station, Robert Park leads us down the back alleys of fringe science, through the gleaming corridors of Washington power and even into our evolutionary past to search out the origins of voodoo science. Along the way, he offers simple and engaging science lessons, proving that you don't have to be a scientist to spot the fraudulent science that swirls around us. Park argues that the public does not need a specific

The Litigator’s Guide to Combating Junk Science knowledge of science so much as a scientific worldview—an understanding that we live in an orderly universe governed by natural laws that cannot be circumvented.

Why People Believe Weird Things: Pseudoscience, Superstitions, and Other Confusions of Our Times Michael Shermer Holt Paperbacks, 2002

A no-holds-barred assault on popular superstitions and prejudices, this book debunks nonsensical claims and explores the very human reasons people find otherworldly phenomena so appealing. In the chapter "Why Smart People Believe in Weird Things," Michael Shermer takes on science luminaries like physicist Frank Tippler and others, who hide their spiritual beliefs behind the trappings of science.

Willful Ignorance: The Mismeasure of Uncertainty Herb Weisberg, Ph.D. Wiley 2014

This book explains how statistical methodology, though enormously productive and influential over the past century, is approaching a crisis. The field's problems are exemplified by the deep and troubling divides between qualitative and quantitative modes of research, and between research and practice. The author outlines a path toward the re-engineering of data analysis to help close these gaps and accelerate scientific discovery.

Reference Manual on Scientific Evidence: Third Edition (2011)

The Reference Manual on Scientific Evidence is intended to assist judges in managing cases involving complex scientific and technical evidence by describing the basic tenets of key scientific fields from which legal evidence is typically derived, and by providing examples of cases in which that evidence has been used. Includes chapters on statistics, forensics, epidemiology, medical testimony and other topics.

Reference Guide on Epidemiology Michael D. Green, D. Michal Freedman, and Leon Gordis

The Litigator’s Guide to Combating Junk Science

This book chapter provides insight into epidemiology, the study of the distribution and causes of diseases among populations. The authors address reasons that epidemiological research may produce misleading conclusions about the dangers of environmental factors. These include false positives and negatives, confounding factors, and biases such as selection bias. The authors stress that assessing whether an association is causal requires an understanding of the strengths and weaknesses of the study’s design and implementation, as well as a judgment about how the study findings fit with other scientific knowledge.

Science for Lawyers Ed. Eric York Drogan

Science for Lawyers clearly explains and discusses 13 applied scientific disciplines, in jargon-free language specifically geared toward lawyers. The book explores the definitions (what is science), the practice (what scientists do) and the professional roles of 13 professional disciplines including ballistics, medicine, statistics, linguistics and chemistry. With dozens of photos, figures, graphics and artwork, the book covers these subjects in terms that are easy to understand.

Scientific Evidence Review: Admissibility and the Use of Expert Evidence in the Courtroom, Monograph #9 Ed. Cynthia H. Cwik, Jules Epstein and Carol Henderson

Selecting the right expert to provide scientific testimony is extremely important, and should include investigating the expert's credentials as well as assessing how how their qualifications will project to a judge or juror. This book presents an introductory overview on United States Supreme Court decisions regarding expert admissibility issues, with chapters by experienced attorneys with in-depth knowledge of rules regarding expert evidence.

The Future of Evidence: How Science and Technology will Change the Practice of Law Ed. Carol Henderson and Jules Epstein

New information in areas such as digital and multimedia sciences, canine scent detection, touch DNA and neuroscience is emerging daily. Each subject area requires new statutes or rules of

The Litigator’s Guide to Combating Junk Science evidence. The fields of law, science and technology intersect in both the civil and criminal areas and are developing at a rapid pace. This book will help keep you on top of the multitude of new theories, policies, laws and rules that will affect scientific evidence.

The Litigator’s Guide to Combating Junk Science

Resources for Regulatory Guidance

The following authoritative websites contain a plethora of valuable resources that can be tapped when confronting junk science. Available information includes research studies, databases, publications, news, regulatory guidelines, and a host of other materials. If you suspect you are facing junk science in your legal case, these regulatory resources provide a great place to obtain authoritative information. For example, if you need to know about current regulations on GMOs, the USDA website is a great place to start. Or check out IARC if you are wondering what the latest science says about the association between mobile phones and brain cancer.

Food and Drug Administration (FDA)

The FDA is responsible for protecting public health by ensuring that foods are safe, wholesome, sanitary and properly labeled (except for meat from livestock, poultry and some egg products, which are regulated by the U.S. Department of Agriculture). The FDA is also charged with ensuring that human and veterinary drugs, vaccines and other biological products and medical devices intended for human use are safe and effective. In addition, the FDA is responsible for protecting the public from electronic product radiation, ensuring that cosmetics and dietary supplements are safe and properly labeled, regulating tobacco products, and advancing public health by promoting product innovations.

The following links to FDA guidances provide authoritative information related to dietary supplements, food health claims, and radiation-emitting products:

Dietary Supplements - Draft Guidance for Industry

This guidance is intended to assist industry in deciding when a premarket safety notification for a dietary supplement containing a new dietary ingredient (NDI) is necessary, and in preparing such notifications. The guidance addresses what qualifies as a NDI, when a NDI notification is necessary, the procedures for submitting a NDI notification, the types of data and information that FDA recommends manufacturers and distributors consider, and what should be included in a NDI notification. This guidance is not legally enforceable, but describes the Agency's current thinking on a topic and should be viewed only as recommendations on the topic.

The Litigator’s Guide to Combating Junk Science

Food Health Claims - Guidance for Industry

This guidance document describes the evidence-based review system that FDA intends to use to evaluate the publicly available scientific evidence for food health claims. The specific topics addressed in this guidance document include identifying studies that evaluate the substance/disease relationship, evaluating the human studies to determine whether scientific conclusions can be drawn from them about the substance/disease relationship, assessing the methodological quality of each human study from which scientific conclusions about the substance/disease relationship can be drawn, and evaluating the totality of scientific evidence.

Radiation-Emitting Products

This reference contains radiation safety regulations for manufacturers of radiation-emitting electronic products. Manufacturers are responsible for producing products that do not emit hazardous and unnecessary radiation. Generally speaking, if your firm is subject to these regulations, there are four steps you must follow prior to marketing your product in the United States. Addresses design standards, quality control and certification of compliance.

FDA All Guidances

This searchable table lists all official FDA Guidance Documents and other regulatory guidance. You can search by keyword for any drug, substance or other health-related topic.

National Institute for Occupational Safety and Health (NIOSH)

NIOSH is responsible for conducting research and making recommendations to prevent work-related injury and illness. The mission of NIOSH is to generate new knowledge in the field of occupational safety and health and to put that knowledge into practice to benefit workers. To accomplish this mission, NIOSH conducts scientific research, develops guidance and authoritative recommendations,

The Litigator’s Guide to Combating Junk Science disseminates information, and responds to requests for workplace health hazard evaluations. NIOSH is part of the Centers for Disease Control and Prevention (CDC) within the U.S. Department of Health and Human Services.

NIOSH Pocket Guide to Chemical Hazards

The NIOSH Pocket Guide to Chemical Hazards is intended as a source of general industrial hygiene information on several hundred chemicals/classes for workers, employers, and occupational health professionals. The Guide presents key information and data in abbreviated or tabular form for chemicals or substance groupings (e.g. cyanides, fluorides, manganese compounds) that are found in the work environment. The information found in the NIOSH Pocket Guide should help users recognize and control occupational chemical hazards.

NIOSH Manual of Analytical Methods

This manual is a collection of methods for sampling and analysis of contaminants in workplace air, and in the blood and urine of workers who are occupationally exposed. These methods have been developed or adapted by NIOSH or its partners and have been evaluated according to established experimental protocols and performance criteria. NMAM also includes chapters on quality assurance, sampling and portable instrumentation.

United States Environmental Protection Agency (EPA)

The EPA has primary responsibility for enforcing many of the environmental statutes and regulations of the United States. As such, the Agency is granted explicit enforcement authority in environmental statutes, and sometimes issues policy or guidance to encourage compliance with environmental requirements. Policy documents represent EPA's official interpretation or view of specific issues. Guidance documents are published to further clarify regulations and to assist in implementation of environmental regulations.

The Litigator’s Guide to Combating Junk Science

Significant Guidance Documents

The EPA maintains a list of significant guidance documents, as called for by the Office of Management and Budget's. These documents address a variety of environmental topics including mercury, pesticides and hazardous waste.

United States Geological Survey (USGS)

The USGS provides impartial information on the health of the Earth’s ecosystems and environment, the natural hazards that threaten humans, the natural resources we rely on, the impacts of climate and land-use change, and the core science systems that help us provide timely, relevant, and usable information. The USGS provides reliable scientific information to describe and understand the Earth; minimize loss of life and property from natural disasters; manage water, biological, energy, and mineral resources; and enhance and protect our quality of life.

United States Department of Agriculture (USDA)

The USDA is responsible for developing and executing U.S. federal government policy on farming, agriculture, forestry and food. The agency provides leadership on food, agriculture, natural resources, rural development, nutrition, and related issues based on sound public policy, the best available science, and efficient management.

How the Federal Government Regulates Biotech Plants

The Federal government has a coordinated, risk-based system to ensure that new biotechnology products are safe for the environment and human and animal health. Established as a formal policy in 1986, the Coordinated Framework for Regulation of Biotechnology describes the Federal system for evaluating products developed using modern biotechnology. The framework is based upon existing laws designed to protect public health and the environment.

The Litigator’s Guide to Combating Junk Science

Centers for Disease Control and Prevention (CDC)

CDC’s mission is to protect America from threats to health, safety and security, both foreign and in the U.S. As the nation’s health protection agency, CDC conducts critical science research and provides health information that protects the nation against expensive and dangerous health threats, as well as responding when they arise. CDC tracks disease, finding out what is making people sick, and identifying the most effective ways to prevent it.

World Health Organization (WHO)

WHO monitors regional and global health situations and trends. WHO is considered a reliable and up-to-date health information and evidence source. WHO works with countries to improve the generation, sharing and use of high-quality knowledge resources.

The International Agency for Research on Cancer (IARC)

The IARC is the specialized cancer agency of the World Health Organization. The objective of the IARC is to promote international collaboration in cancer research. This interdisciplinary agency brings together skills in epidemiology, laboratory sciences and biostatistics to identify the causes of cancer so that preventive measures may be adopted and the burden of disease and associated suffering reduced.

International Classification of Diseases (ICD)

The International Classification of Diseases (ICD) is the standard diagnostic tool for epidemiology, health management and clinical purposes. This includes the analysis of the general health situation of population groups and is used to monitor the incidence and prevalence of diseases and other health problems, proving a picture of the general health

The Litigator’s Guide to Combating Junk Science situation of countries and populations. ICD is used by physicians, nurses, researchers, policy-makers, insurers and others to classify diseases and other health problems. These records also provide the basis for the compilation of national mortality and morbidity statistics by WHO Member States.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH)

The ICH is unique in bringing together regulatory authorities and pharmaceutical industry to discuss scientific and technical aspects of drug registration. Since its inception in 1990, ICH has gradually evolved, to respond to the increasingly global face of drug development. ICH’s mission is to make recommendations towards achieving greater harmonization in the interpretation and application of technical guidelines for pharmaceutical product registration, thereby reducing the need to carry out duplicate testing during the research and development of new medicines.

The Litigator’s Guide to Combating Junk Science

Resources for Scientific Standards

The following independent organizations provide information on generally accepted scientific standards and practices. Specific knowledge of scientific principles will enable you to formulate an argument discrediting junk science. For example, the Cochrane Collaboration resource can help you better understand how to conduct a proper systematic review of the literature. If you suspect you are facing an expert that has cherry-picked data, you might want to confront them on the methodology they used, and compare it to the methodology proffered by the Cochrane Collaboration.

The Cochrane Collaboration

From the Cochrane website:

“Cochrane Reviews are systematic reviews of primary research in human health care and health policy. Cochrane Reviews are internationally recognized as the highest standard in evidence- based health care. They investigate the effects of interventions for prevention, treatment and rehabilitation. They also assess the accuracy of a diagnostic test for a given condition in a specific patient group and setting. They are published online in The Cochrane Library.

Each systematic review addresses a clearly formulated question; for example: Can antibiotics help in alleviating the symptoms of a sore throat? All the existing primary research on a topic that meets certain criteria is searched for and collated, and then assessed using stringent guidelines, to establish whether or not there is conclusive evidence about a specific treatment. The reviews are updated regularly, ensuring that treatment decisions can be based on the most up-to-date and reliable evidence.

Cochrane Reviews are designed to facilitate the choices that practitioners, consumers, policy- makers and others face in health care.”

Consolidated Standards of Reporting Trials (CONSORT)

From the CONSORT website:

“The main product of CONSORT is the CONSORT Statement, which is an evidence-based, minimum set of recommendations for reporting RCTs. It offers a standard way for authors to

The Litigator’s Guide to Combating Junk Science prepare reports of trial findings, facilitating their complete and transparent reporting, and aiding their critical appraisal and interpretation.

The CONSORT Statement comprises a 25-item checklist and a flow diagram, along with some brief descriptive text. The checklist items focus on reporting how the trial was designed, analyzed, and interpreted; the flow diagram displays the progress of all participants through the trial.”

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)

From the PRISMA website:

PRISMA “is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.

The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses. We have focused on randomized trials, but PRISMA can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions. PRISMA may also be useful for critical appraisal of published systematic reviews, although it is not a quality assessment instrument to gauge the quality of a systematic review.

The PRISMA Statement consists of a 27-item checklist and a four-phase flow diagram. It is an evolving document that is subject to change periodically as new evidence emerges. In fact, the PRISMA Statement is an update and expansion of the now-out dated QUOROM Statement. This website contains the current definitive version of the PRISMA Statement.”

Grading of Recommendations Assessment, Development and Evaluation (GRADE)

From the GRADE website:

“The Grading of Recommendations Assessment, Development and Evaluation (short GRADE) Working Group began in the year 2000 as an informal collaboration of people with an interest in addressing the shortcomings of present grading systems in health care. The working group has developed a common, sensible and transparent approach to grading quality of evidence and strength of recommendations. Many international organizations have provided input into the development of the approach and have started using it.”

The Litigator’s Guide to Combating Junk Science

U.S. Preventive Services Task Force (USPSTF)

From the USPSTF website:

“Created in 1984, the U.S. Preventive Services Task Force (USPSTF or Task Force) is an independent group of national experts in prevention and evidence-based medicine that works to improve the health of all Americans by making evidence-based recommendations about clinical preventive services such as screenings, counseling services, or preventive medications. The USPSTF is made up of 16 volunteer members who come from the fields of preventive medicine and primary care, including internal medicine, family medicine, pediatrics, behavioral health, obstetrics/gynecology, and nursing. All members volunteer their time to serve on the USPSTF, and most are practicing clinicians.”

The National Research Council (NRC) and National Academy of Sciences (NAS)

From the NRC website:

“The National Research Council, created under the NAS charter in 1916 by executive order of President Woodrow Wilson, extended the scope of the NAS in its advisory role. The National Academy of Engineering (NAE) and the Institute of Medicine (IOM) were founded under the NAS charter in 1964 and 1970, respectively. Together, the NAS, NRC, NAE and IOM enlist the aid of the nation’s most knowledgeable scientists, engineers, health professionals, and other experts who volunteer their time to produce reports that have led to some of the most significant and lasting improvements in the health, education, and welfare of all the world’s citizens. The Academy's service to government has become so essential that Congress and the White House have issued legislation and executive orders over the years that reaffirm its unique role.”

The Litigator’s Guide to Combating Junk Science

Placebo and Nocebo Effects

The placebo effect refers to a positive health outcome resulting from a patient’s belief that a treatment will be effective. The nocebo effect is the antithesis—it refers to the symptoms patients experience due to their anticipation of those symptoms. Both are real, physiological phenomena that can be measured using standard scientific techniques. The placebo effect plays a significant role in lawsuits against drug companies.

Clinical drug trials typically include a placebo control group, allowing researchers and the Food and Drug Administration to effectively evaluate the efficacy of any new medication. The nocebo effect is also a factor, as participants taking the placebo occasionally experience negative side effects. The nocebo effect can also apply to non-drug aspects of the environment that a subject believes may harm their well-being.

The following resources provide essential background information on the placebo and nocebo effects. If you are potentially thinking of a nocebo or placebo defense, these articles will arm you with all of the information you will need:

Pain Physician Journal - Placebo and Nocebo in Interventional Pain Management: A Friend or a Foe - Or Simply Foes?

Journal of Bio-behavioral Medicine - Drug-Related Information Generates Placebo and Nocebo Responses That Modify the Drug Response

Badscience.net - Ben Goldacre on the Placebo and Nocebo Effect

Philosophical Transactions B (Biological Sciences) - Harnessing the placebo effect: the need for translational research

The Psychiatrist - The placebo effect

The New Yorker - The Nocebo Effect: How We Worry Ourselves Sick

Discover - Are Warnings About Drug Side Effects Actually Making Us Sick?

The Litigator’s Guide to Combating Junk Science

Nocebo Effect - Think Sick and You’ll Be Sick

Program in Placebo Studies & Therapeutic Encounter (PiPS) - Seminar Series on the Power of the Placebo Effect

Placebo and nocebo effects are clinically real and can play a significant role in personal injury, and consumer fraud lawsuits since plaintiffs are often inclined to ignore the relevance of the placebo and nocebo effects. For example, they may claim a particular drug caused a negative health outcome, but fail to produce comparative evidence demonstrating that a placebo control group did not have the same outcome.

The Litigator’s Guide to Combating Junk Science

Preparing An Expert To Testify

Even if you know all the traps to watch out for relating to junk science, if your expert witness cannot do your knowledge justice on the stand, your knowledge doesn’t matter much. Litigators and their witnesses are confronted with difficult situations during testimony, and it’s nice to have reliable ways out of those sticky situations.

Expert witnesses are engaged to provide their expert insight and opinions supporting their client’s case during testimony and are there to tell the truth to the best of their knowledge when questioned at trial or deposition. Litigators get paid to ask good and, at times, tough questions to get desired answers from the opposition’s witnesses and to help their own witnesses do their best.

During both courtroom testimony and in depositions there are common situations where an attorney tries to make things difficult for the witness. Below, we identify 14 of these common situations and provide some good strategies, both from my own experience as a litigator and from tips collected from attorneys and expert witnesses. Consider the points below when advising and preparing your witnesses for trial and depositions. The main and reoccurring principles are:

1. The “Yes or No” Question

If you’re a witness (an expert) you are going to be asked “yes or no” questions (where the forced response appears to be a “yes” or a “no”) on cross-examination or during a deposition. This

The Litigator’s Guide to Combating Junk Science type of questioning will put you in a tough spot because whatever you’re asked to respond “yes” to is most likely something you’d rather say “no” to, and vice versa. But, to be truthful, you’ll feel that you must answer in a way that seems counter to your beliefs or the foundations of your case.

There are many easy ways to get yourself out of this predicament. First, you need to identify that you’re in it. Then, in response to the question you say this: “I understand that you’re asking me for a ‘yes or no’ answer here, and I could answer you in that way, but doing so would be an incomplete answer and I don’t want to mislead you or the court.” Now, what have you done?

You’ve instantly made yourself look very reasonable in front of the jury/court and like someone interested in getting the “truth” out rather than an unreasonable (paid) witness who won’t answer questions. If the attorney asking the “yes or no” question insists that you go ahead and answer simply “yes or no” he looks like a jerk pushing his own agenda and uninterested in the truth – neither of which will help him in the jury’s or court’s eyes. It’s unlikely he’ll do this, but if he does, you go ahead and answer as he’s asked, but you’ve made him look bad and also have clearly identified the issue for re-direct from your own counsel.

2. The “Yes or No” Question – Take Two

As mentioned above, there are a variety of ways to get yourself out of the sticky “yes or no” question problem. So, in addition to the solution above, here are some additional tip/tricks to consider.

One expert witness has suggested that a response she uses to combat this situation is to go ahead and answer the question with the “yes” or “no” sought by the examining attorney, and then add, “under certain conditions,” with nothing further.

This presents the examining attorney with a dilemma. Should she let that answer stand? What circumstances is the expert referring to? Should she follow up and inquire about the

The Litigator’s Guide to Combating Junk Science circumstances the expert has in mind? Doing this surely exposes the attorney to a strong counter point by the expert. Responding in this way allows the expert to take the advantage.

3. The “Yes or No” Question – Take Three

Another expert surveyed for this article suggested replying to the “yes or no” question with, “as I understand your question the answer is [insert ‘yes’ or ‘no’].” As this expert explains it, this is a non-answer; it means nothing because there is no way for the lawyer to know how the expert understood his question and the answer can be either yes or no based on whatever is going on in the expert’s mind.

So, again, this begs the question: will the attorney follow up and allow the expert to express what’s on his/her mind? Again, advantage: expert witness.

As mentioned, experts will be asked “yes or no” questions during their deposition as they will at trial – the purpose being, once the examining attorney has probed the depths of the expert’s knowledge and bases for opinions, he or she will want to lock the expert into some position for trial. Just as in the trial testimony scenario, experts can use the same, and even more, techniques to wiggle out of this sticky situation during a deposition (I say “more” because you’re not responding in front of a judge and will have more flexibility).

There are other types of “sticky situations” expert witnesses will be confronted with during their examination by an attorney. Several are explored below.

4. “I Don’t Understand”

As an expert witness, you’ll be subjected to some pretty tough, sometimes technical questions. Often the questioning attorney will offer a lot of hypothetical facts and complexity within a question. If confronted by such a question, when in doubt, respond that you just don’t understand the question and request that the attorney rephrase it.

At worst, this buys you a moment of time to consider the question. At best, you’ll throw off the questioning attorney, who may have carefully scripted his question because he or she simply had to in order to address the complexity necessary to the issue being investigated.

The Litigator’s Guide to Combating Junk Science

5. “I Don’t Understand” – Take Two

When you express lack of understanding and ask the attorney to rephrase a confusing question, sometimes the attorney will ask what was confusing to you. Don’t play this game. Don’t parse the question for what was clear and what was not.

The entire question was confusing and it’s his job to figure out a way to make it clear. Just make sure that, before you go this route, the question is at least too confusing for the jury to easily understand, otherwise, they’ll perceive you as playing games and being deceptive.

As mentioned, often, the examining attorney will have been asking his questions from a script that he or an associate prepared or that he obtained from a book. If the expert being examined is in a dense or very high tech field, the attorney may not understand the topic well enough to craftily rephrase his question.

6. “I Don’t Understand” – Take Three

Also, make opposing counsel define words if something could be ambiguous. Here’s an example based on the examination of a fact witness in a child custody battle:

Opposing counsel began asking leading questions to the mother in the case designed to try to paint her as a promiscuous parent who paraded men in front of her kids night and day. If you knew the mom, you would know how utterly laughable this tactic was. So, the examining attorney began the questioning by asking if the mom had “dated” anyone. The mom-witness responded to each of the attorney’s questions with her own, e.g., what do the terms “date,” “relationship,” “intimate,” “boyfriend,” etc., mean? The attorney finally gave up in frustration and the mom-witness's attorney got a good laugh out of it – the examining attorney got nowhere.

Don't assume you know what examining counsel means by the words he/she uses. Make them explain it (assuming doing so isn’t ridiculous enough to make you look stupid or difficult in front of the jury).

The Litigator’s Guide to Combating Junk Science

7. Think Before You Answer

The next common technique of examining-counsel is the use of rapid fire questioning. This is an easy technique to defuse since the witness can control the rate of questioning by taking the time to consider each question before answering. When the expert witness takes his time to answer, he also gives his counsel time to object.

Our [A2L’s] CEO, Ken Lopez, was once questioned about an animation in a plane crash case and the question was something like: “the clouds in this animation are really like a video game aren’t they?” Ken explained, “I felt defensive, but choose to take my time answering. After a long pause, I replied, ‘I can't think of a video game like that works like that.’” He was surprised that the examining-attorney dropped the questioning at that point.

Remember, whatever you say is going permanently on the record – so make it accurate, make it useful, and make it count.

8. Don’t “Help” Them

Most expert witnesses are, on some level, teachers. They want to instruct, inform, and educate. Often, the greatest and most sought-after experts are well-regarded university professors. This presents a problem when they’re under questioning at trial or (especially) in depositions. It’s often difficult for these witnesses to refrain from offering additional information, filling-in the pauses with education, and generally responding to questions that weren’t asked.

If an expert finds that their questioning attorney is at a loss for words, don't offer any. Let the uncomfortable silences sit there. Not an easy thing to do, but necessary.

If an examining-attorney asks a question that doesn’t get the science right, or misses the point somehow, don’t educate them. Let them stay ignorant and let the record stay ignorant until the right time to inform it, which is when the witness is on direct.

The Litigator’s Guide to Combating Junk Science

9. Don’t Guess

Remember, the expert’s testimony is forever on the record and will be held against him and his client if possible. If you can't answer a question, or don't know the answer to a question, say so. If your answer is an estimate or only an approximation, say so. If you think you might have the answer in the future, say, “I don't recall at this time.” If you do not remember, say so.

Never think that you must have 100% total recall or something even close. Do what you can before a deposition to refresh your recollection if it’s appropriate, but don’t refresh yourself on irrelevant or unhelpful things.

10. Don’t Guess – Take Two (or Stick to What You’re There For)

Another expert recognized that a standard trick is to get an expert to answer a question that is outside her experience because of the natural tendency to try and help by giving an answer. But, doing so can trap the expert because it then calls into question everything she has previously written and all her opinions expressed in court.

It is much better to simply say you cannot answer the question because it is outside your experience. So the cross examining counsel's armory is even further reduced. In addition, the image that the jury (or Judge) then has of you will further be improved. Knowing your business very well and the specific limits of your experience and expertise should garner your more respect.

11. Don’t Guess – Take Three (or Stick to What You’re There For – Take Two)

Following the previous note, what if the line of questioning moves to a subject for which your expert IS knowledgeable, but not there to talk about? He can’t say he doesn’t know how to respond.

The Litigator’s Guide to Combating Junk Science

Another expert suggests that if the subject matter of cross exam is not outside the expert’s experience, but is outside the scope work conducted in the matter, consider answering, at least in the U.S. – “I am sorry, but that work was outside of the scope of my retention in this matter, and so was not considered.” This expert gives the following example: “I have a specialty of deciphering Traffic Signal Timing plans to try and determine who REALLY had the green, as opposed who THOUGHT they had the green. In many of these cases, a separate Accident Reconstructionist is hired [as another expert]. If an Accident Reconstruction question is asked, it will most probably be within the scope of my EXPERIENCE and TRAINING, but is outside of the scope of my RETENTION in that matter.”

The danger of this scenario is that opposing counsel will try to drive a wedge between your multiple experts’ testimony, make them contradict one another, and diminish one or more of your experts and, thereby, your case. To combat this possibility, have your experts well prepared on what they are there to testify about. Have them stick to their expert reports, if they were required. Have them well prepared on what other experts on your team are testifying about and well prepared not to step on their teammate’s toes.

12. Only Answer One Question At A Time.

Compound questions are objectionable, whether in deposition or at trial. Nonetheless, have your experts prepared for this possibility. When asked multiple questions at one time, they should ask for clarification to be clear which part they are responding to. For example:

Q: Do you drink alcohol or take illegal drugs?

A: Yes to the alcohol; no to the illegal drugs.

There would often be an objection here. If there is no objection, and it is too complicated to easily respond to both parts, then do not be afraid to ask for the question to be restated.

The Litigator’s Guide to Combating Junk Science

13. Don't Let Yourself Get Cut Off

Another expert recommends: “If there is more that you need to say, then say it. If that means adding it to the next answer or simply saying, ‘I'm sorry counselor, I wasn't finished answering your question,’ and then continuing,” then do it.

Also, be careful if asked a question that attempts to cut off your response, such as: “Is that everything?” Leave the door open in case you might have forgotten something. Respond to such a question with, “that’s what I can recall at this time” or something to that effect. Your attorney can try to fix any problems or misrepresentations on your redirect and it will be easier for the attorney to remind you what you have forgotten if you do not testify under oath that you have already covered everything.

14. Make Your Own Hypotheticals

Cross examination involving hypotheticals is common for experts. Another surveyed expert suggested that, when asked a hypothetical question, they are also very seldom complete – engineered that way to be more helpful to the opposing side and damaging to yours. This expert suggests responding with “I am sorry, that is an incomplete hypothetical, which I cannot answer as phrased. Would you like me to fill in the missing pieces and then give you an answer?” How can the examining-attorney possibly refuse and still appear reasonable to the jury?

The Litigator’s Guide to Combating Junk Science

More Preparation For Your Expert

Expert witnesses, if they are well prepared and know your case well, can go a long way to helping you win your case at trial. Often, a case will center on an engineering, scientific, environmental, or similar issue, and having the right expert can make all the difference. However, the flip side is that a poorly prepared expert witness, or one who does not testify effectively, can help you lose your case.

Here are seven things that your expert witness should never say.

“That’s not my field of expertise, but …” The classic mistake an expert can make is to wander outside his or her area of knowledge and expertise. An expert should never sound evasive or ill-informed. If the answer to a question on cross-examination is truly outside his or her field, it’s not relevant to his or her direct testimony, or the question should draw an objection, the best way for an expert to be believed about what they do know is to admit what they don’t know when it isn’t in their domain. If it’s relevant, the expert should be prepared and should answer.

“I have no idea.” Again, don’t sound evasive or ill-informed. A better answer is, “Under the assumptions that I am making, which are …, here is what I’d expect to happen.” In addition, the expert should explain why it is not relevant.

“I said that in my report, but …” Do not back down from the report and create uncertainty. The report should be carefully crafted to embody the expert’s conclusions. A significant weakness for any witness is to reverse positions. If for some reason such as new information that was not available when the report was prepared became known to the expert, then it should be made clear that the report was based on what was known at the time. Otherwise, there are better ways to explain apparent inconsistencies. Cross examination is likely to exaggerate such points and it is the expert’s job to neutralize them and put them into better perspective.

“I changed my mind.” Again, this creates a dangerous amount of uncertainty for the jury and leads them not to rely on an expert as an expert. If the expert really needs to modify some aspect of his or her testimony, tackle that directly by explaining in open court what slight change is needed and why.

The Litigator’s Guide to Combating Junk Science

“I could be wrong, but …” The expert should never make this concession. The expert’s job is to be forceful and help the jury. The jurors may discount part of the expert’s testimony, but his or her job is not to help them do this. Such type of humility does not serve an expert well.

“I’m not really an expert.” Then why are you on the stand? Under the law, expert testimony is admissible only if the expert is qualified, if his or her testimony will help the jury decide issues in the case or understand the evidence, and if the expert’s testimony is based on sufficient facts or data, is the product of reliable methods and principles, and if the expert has reliably applied the methods and principles to the facts of the case. Otherwise, the expert shouldn’t be on the stand. If an expert is unwilling to make a firm commitment to an opinion and to their area of expertise, do not risk putting them on the stand. This is especially relevant when using an expert without experience testifying.

“The lawyers told me to say that.” No. Although the expert is on your side, he or she is not a mouthpiece for the lawyers. He or she has objective expertise based on science and technology and has composed an independent opinion. It is up to the expert to own it.

The Litigator’s Guide to Combating Junk Science

It’s Not Just Science That Can be “Junk”: Trial Graphics Can Be, Too

Charts don't lie, people do.

As demonstrative evidence consultants, we see a lot of charts and graphs that are designed to mislead or that end up misleading the viewer, and ultimately the jury. I don't think it is always intentional on the part of the trial team. Sometimes, a demonstrative evidence consultant is to blame for introducing a misleading tactic. This article will help you spot those misleading charts before they do damage.

Remember that each piece of demonstrative evidence is subject to the balancing test under Rule 403 of the Federal Rules of Evidence, among other evidentiary standards. Under Rule 403, an otherwise relevant demonstrative will be excluded when its probative value is substantially outweighed by unfair prejudice, its cumulative nature or if confusing or misleading.

For example, I believe that a chart using any of these five techniques described below runs the risk of not passing muster under Rule 403; however, objections to demonstrative evidence are relatively rare. Successfully make the objection during trial and you might just call the credibility of your opponent into question.

1) The Slippery Scale: This is the most common trick I see, and

The Litigator’s Guide to Combating Junk Science once you know about it, you'll see it everywhere too. By setting your y-axis (the vertical one) to a narrow range not including zero (e.g. below, 94M to 108M), it is easy to make relatively small changes look enormous. For example, the Simply Statistics blog recently highlighted this technique used by Fox News. Here, this trick makes changes in the welfare rolls that are relatively small seem enormous.

2) Compared to what? If you want to show a small change on a percentage basis, all you need to do is vary your x-axis (the horizontal one) so that time is literally on your side. The Obama campaign truncated its timeframe in its spending chart, claiming that in 2010, President Obama presided over the smallest increase in spending in 50 years. While technically true, 2010 was being compared to 2009, the year that the one-time stimulus spending (championed by the Obama/Clinton/Biden Congress) ballooned government spending by 18%.

3) The Percentage Increase Trick? How many times have you heard someone talk about a 200% increase and really wonder exactly what they are trying to say? Do they mean it doubled, it quadrupled or something else? If they are using the percentages correctly, a 100% increase is a doubling and a 200% increase in something is a tripling – three times as much as at the outset. However, the trickery comes in where one might say in a chart that the same 200% increase is 300% of the first figure or a threefold change. To help stay accurate and monitor your opponents, use A2L's Percentage Calculator for Lawyers (the image here is a link to the online resource).

The Litigator’s Guide to Combating Junk Science

4) Tricking the Eye with 3D Charts: Flat charts with no depth or 3D aspect are harder to trick the viewer with, so always scrutinize your opponent's charts when the third dimension introduced. For example, have a look at the two pie charts at right. Both red areas are the same percentage of the pie, but if you are like most people, when the slice is closer to you, it looks bigger. A similar trick can be used with bar charts.

5) Misleading emotional imagery: Putting an image of a homeless person in the background of a chart about increasing homelessness is designed to evoke emotion. It might be admissible since it is clearly tied to the underlying issue. Showing an oil-covered bird in the background in an explanation of how much oil was spilled would not add to one's understanding of the amount of oil spilled. Some examples of emotional imagery in charts that add little probative value but add undue prejudice are below.

This one is used to sell water filters, but if used in court in a fracking lawsuit, the poison symbol would (if objected to) rule the chart inadmissible.

The chart here shows the surprisingly small size of the Deepwater Horizon spill when compared to historical spills. The photograph adds nothing to the viewers understanding, so it might be objectionable if used as a demonstrative.

Since it requires the viewer to decode several different riddles before understanding the message, the chart below is a model of poor chart design worthy of its own article. This

The Litigator’s Guide to Combating Junk Science

riddle-making mistake is commonly made by those without training in preparing demonstrative evidence and non-demonstrative evidence consultants. Here a legend is used (generally speaking, this is always a bad idea), so you have to first find that. Then you have to read sideways - twice - in two different directions. Then you have to figure out from the subtle color coding of the legend that blue is left and red is right. THEN, you have to determine that left is 2005 and right is 2010. It's a mess, however, it provides a good example of some imagery that would potentially be objectionable. The cigarettes being snuffed out add little to the message and are there only for emotional impact.

The Litigator’s Guide to Combating Junk Science

We believe this book can be a valuable resource for you when preparing yourself, your clients, and your experts for trial. Remember to question the science your opposition chooses to use against you. Armed with the arsenal provided by this book, you will be well apportioned to combat junk science and persuasively present your affirmative and defensive case to judges and juries.

Above all else, we hope that you will count your greatest resource provided here as ISS and A2L Consulting. Our teams collaborated on this book and likewise collaborate for our clients to enable them to craft winning case strategies using the best practices in scientific expertise, courtroom technology, and demonstrative evidence. In the following pages we briefly describe our services and provide direct contact information. Please use it when you need it.

The Litigator’s Guide to Combating Junk Science

About Us

Innovative Science Solutions (ISS) is a leading scientific consulting firm for the worldwide pharmaceutical, biotechnology, and medical device industries. Our expert team of scientists, regulatory strategists, and consultants provide a wide range of fully integrated services to industry and counsel.

From strategic insight for FDA Advisory Committee meetings to scientific expertise for complex litigation, ISS delivers specialized and effective solutions for your legal and regulatory challenges. Let our proven combination of experience and innovation work for you.

David H. Schwartz, Ph.D. Head of Scientific Support to Counsel

A “scientific detective,” Dr. Schwartz excels at analyzing how science and law interact. As head of ISS’s Support to Counsel practice, he helps clients defend and support pharmaceuticals, industrial chemicals, medical devices, foods, and dietary supplements in the courts, the regulatory arena, and the market place. With a talent for communicating complex scientific information, Dr. Schwartz has become a go-to for lawyers, business professionals, and financial and investment firms. Dr. Schwartz specializes in providing strategic and tactical support to counsel in legal cases involving complex scientific issues. He has played a lead role in winning multi-billion dollar legal cases for clients. With the sheer weight of the scientific data he’s able to collect, evaluate, and present as evidence, he’s helped counsel win cases before going to trial, saving ISS clients time and money.

The Litigator’s Guide to Combating Junk Science

Dr. Schwartz counts Daubert hearings among his biggest successes. His expertise guides the ISS team across all disciplines related to the health sciences: toxicology, epidemiology, environmental health and safety, biostatistics, pharmacology, and regulatory affairs.

He has pioneered programs to critically review the scientific foundation of an adversary's case and provided persuasive, evidence-based responses. For 20 years, he has worked with the legal community evaluating product safety and defending a host of products and exposures, including the following: welding rods, cellular phones, ionizing and non-ionizing radiation, hydraulic fracturing and other drilling technologies, breast implants, wound care products, dietary supplements, general healthcare products, and a host of over-the-counter and prescription pharmaceutical agents (including: antidepressants, dermatologics, anti-malarials, anxiolytics, antipsychotics, and diet drugs).

Prior to founding ISS, Dr. Schwartz served as the Director of Scientific Litigation Support for the law firm McCarter and English, where he played a principal role in the safety evaluation of Prozac. Since then, he has guided clients using scientific principles to defend products before, during and after launch.

Dr. Schwartz received a Ph.D. in Neuroscience from Princeton University and postdoctoral training in Neuropharmacology and Neurophysiology from the Center for Molecular and Behavioral Neuroscience at Rutgers University.

Steven M. Weisman, Ph.D. Head of Clinical and Regulatory Support

Dr. Weisman knows how to bridge the gap between science and marketing. As head of ISS's Clinical and Regulatory Support practice, he focuses on the development of scientific and regulatory approaches that increase a product's market potential. He's an invaluable resource for scientific litigation support for products in crisis and, under his guidance, ISS has encouraged firms to proactively monitor the safety and effectiveness of their products and develop systems that reduce liability claims.

Dr. Weisman has over 20 years of experience in pharmacology, toxicology, pharmaceutical product development, clinical and regulatory affairs, and marketing evaluation and communication.

The Litigator’s Guide to Combating Junk Science

He played a leading role in the pharmaceutical industry's response to the removal of PPA (phenylpropanolamine) from over-the-counter products. He's worked extensively on management initiatives for opiate safety, antibiotic drug resistance, and emergency contraceptives. He's even credited with establishing the safety and continued marketing of artificial fingernail products.

Dr. Weisman is a leading authority on the switch of prescription-only products to over-the- counter status and is widely published in these areas. His development and support work includes shepherding the analgesic, Aleve, from a prescription-only product to its popular over- the-counter form, and promoting the cardiovascular uses of aspirin for heart attack and stroke prevention. He is also recognized for effective regulatory strategies that maximize the global market potential of regulated products. His expertise in claim substantiation has helped clients successfully defend against challenges to their marketing initiatives and saved them millions of dollars.

Prior to founding ISS he ran the Pharmaceutical and Food practices at a major scientific consulting firm in Washington, DC. Before that, he served as Global Director of Medical and Clinical Affairs at Bayer, Director of Strategic Research at Sterling Winthrop, and held similar positions at Hoffman La Roche and Procter & Gamble. He has organized and presented at numerous symposia, FDA advisory committee meetings, and other regulatory venues, worldwide. He manages the FDA advisory committee process for many of the largest pharmaceutical concerns and represents companies before regulatory authorities in major markets around the world.

Dr. Weisman received his Ph.D. in Pharmacology from Cornell University Medical College and completed his postdoctoral training in Immunopharmacology at the Roche Institute of Molecular Biology.

For more information, please contact us at 973.889.1600 or [email protected].

The Litigator’s Guide to Combating Junk Science

A2L CONSULTING: LITIGATION CONSULTANTS NATIONWIDE

We believe in Winning, by Design.™ We believe that carefully crafted messages and effectively designed presentations change people's decision-making behavior. We are in the business of making belief.

A2L Consulting is the nation's leading attorney owned and operated litigation consulting, jury consulting, litigation graphics, trial technology and visual persuasion firm. Since 1995, A2L has supported litigators and executives from all of the nation’s largest law firms, hundreds of corporate clients, as well as government agencies worldwide on more than 10,000 court cases and a myriad of visual persuasion consulting engagements.

Litigation Graphics

. Nationwide strategic trial presentation consulting and development

. Trial boards, physical models and all forms of trial presentation graphics

. Advocacy presentations for lobbying or regulatory presentations

Trial Technicians/Hot-Seat Operators

. Pre-trial document/video database preparation

. Flawless courtroom presentations with experienced operators

. Electronic courtroom and warroom set-up

Jury Consulting

. Mock trials

. Focus groups

The Litigator’s Guide to Combating Junk Science

. Venue analysis

E-Briefs

. DVD, CD or Flash Drive Based

Have Questions or Need Help?

Contact A2L Consulting 24/7 to run a conflicts check, ask a last-minute question or talk to us for any reason.

Alex Brown Director, Operations 800.337.7697 [email protected] www.a2lc.com