Ben Goldacre: Battling Bad Science
Total Page:16
File Type:pdf, Size:1020Kb
L11 gap-filling test for 1A5, 1L1, and 2R at KU (2012/01/05) prepared by Kow Kuroda The script below was taken from TED (http://www.ted.com) and modified by the tester to make it more faithful to the actual speech. of my discipline— 4. evidence-based medicine. And I Ben Goldacre: Battling Bad Science will talk you through all of these and demonstrate how So I’m a uh doctor, but I kind of slipped sideways into they work, exclusively using examples of people getting research, and now I’m an epidemiologist. And nobody re- stuff wrong. ally knows what epidemiology is. 1. Epidemiology is the So we’ll start with the absolute weakest form of evi- science of how we know in the real world if something dence known to man, and that is 5. authority . In science, is good for you or bad for you. And it’s best understood we don’t care how many letters you have after your name. uh through example as the science of those crazy, wacky In science, we want to know what your reasons are for be- newspaper headlines. And these are just some of the ex- lieving something. How do you know that something is amples. good for us or bad for us? But we’re also unimpressed by These are from the Daily Mail. Every country in the authority, because it’s so easy to contrive. This is some- world has a newspaper like this. It has this kind of bizarre, body called Dr. Gillian McKeith, Ph.D, or, to give her ongoing philosophical project of dividing all the inanimate full medical title, Gillian McKeith. (Laughter) Again, ev- objects in the world into the ones that either cause or pre- ery country has somebody like this. She is our TV diet vent 2. cancer . So here are some of the things they said 6. guru . She has massive kind of five series of prime- cause cancer recently: divorce, Wi-Fi, toiletries and cof- time television, giving out very lavish and exotic health fee. Here are some of the things they say prevents cancer: advice. She, ah, it turns out, has uh a non-accredited cor- crusts, red pepper, licorice and coffee. So already you can respondence course Ph.D. from somewhere in America. see there are contradictions. Coffee both causes and pre- She also boasts that she’s a certified professional member vents cancer. And as you start to read on, you can see that of the American Association of Nutritional Consultants, maybe there’s some kind of political veilance behind some which sounds very glamorous and exciting. You get a cer- of this. So for women, housework prevents breast cancer, tificate and everything. This one belongs to my dead cat but for men, shopping could make you impotent. Hetti. She was a horrible cat. You just go to the website, So we know that we need to start 3. unpicking the sci- fill out the form, give them $60, and it arrives in the post. ence behind this. And what I hope to show is that unpick- Now that’s not the only reason that we think this per- ing dodgy claims, unpicking the evidence behind dodgy son is an idiot. She also goes on ah, ah, ah and says claims, isn’t uh a kind of nasty carping activity; it’s so- things like, you should eat lots of dark green leaves, be- cially useful, but it’s also a kind of an– an extremely valu- cause they contain lots of chlorophyll, and that will really able explanatory tool. Because real science is all about oxygenate your blood. And anybody who’s done school critically appraising the evidence for somebody else’s po- biology 7. remembers that chlorophyll and chloroplasts sition. That’s what happens in academic journals. That’s only make oxygen in sunlight, uh and it’s quite dark in what happens uh at academic conferences. The Q&A ses- your bowels after you’ve eaten spinach. sion after a post-op presents data is often uh a blood bath. Next, we need proper science, proper evidence. So, uh And nobody minds that. We actively welcome it. It’s like a “Red wine can help prevent breast cancer.” This is a head- consenting intellectual S&M activity. So what I’m gonna line from the Daily Telegraph in the U.K. “A glass of red show you is all of the main things, all of the main features wine a day could help prevent breast cancer.” So you go 1 and find this paper, and what you find is it is a real piece over the past decade. And this is the trial of fish oil pills. of science. It is a description of the changes in one en- And the claim was 12. fish oil pills improve school per- zyme when you drip a chemical extracted from some red formance and behavior in mainstream children. And they grape skin onto some cancer cells in a dish on a bench in a said, “We’ve done a trial. All the previous trials were pos- 8. laboratory somewhere. And that’s a really useful thing itive, and we know this one’s gonna be too.” That should to describe in a scientific paper, but on the question of your always ring alarm bells. Because if you already know the own personal risk of getting breast cancer if you drink red answer to your trial, you shouldn’t be doing one. Either wine, it tells you absolutely bugger all. Okay? Actually, it you’ve rigged it by design, or ah you’ve got enough data turns out that your risk of breast cancer actually increases so there’s no need to 13. randomize people anymore. slightly with every amount of alcohol that you drink. So this is what they were gonna do in their trial. They So, all we want is studies in real human, people. And were taking 3,000 children, they were gonna give them here’s another example. This is from uh Britain’s lead- all these huge fish oil pills, six of them a day, and then ing diet and nutritionist in the Daily Mirror, which is our a year later, they were gonna measure their school exam second biggest selling newspaper. “An Australian study performance and compare their school exam performance in 2001 found that olive oil in combination with fruits, against what they predicted their exam performance would vegetables and pulses offers 9. measurable protection have been if they hadn’t had the pills. Now can any- against skin wrinklings.” And then they give you advice: body spot a 14. flaw in this design? And no professors “If you eat olive oil and vegetables, you’ll have fewer skin of clinical trial methodology are allowed to answer this wrinkles.” And they very helpfully tell you how to go and question. So there’s no control; Okay, there’s no control find the paper. So you go and find the paper, and what you group, but that sounds really techie, right? That’s a techni- find is an observational study, right? Obviously nobody cal term. The kids got the pills, and then their performance has been able to go back to 1930, get all the people born improved. What else could it possibly be if it wasn’t the in one maternity unit, and half of them eat lots of fruit and pills? veg and olive oil, and then half of them eat McDonald’s, They got older, okay? We all develop over time. And and then we see how many wrinkles you’ve got later. You of course, also there’s the placebo effect. The placebo have to take a 10. snapshot of how people are now. And effect is one of the most fascinating things in the whole what you find is, of course, people who eat veg and olive of 15. medicine . It’s not just about taking a pill, and oil have fewer skin wrinkles. But that’s because people your performance and your pain getting better. It’s about who eat fruit and veg and olive oil, they’re freaks, okay? our beliefs and expectations. It’s about the cultural mean- They’re not normal, they’re like you; they come to events ing of a treatment. And this has been demonstrated in a like this, right? They are posh, they’re wealthy, they’re less whole raft of fascinating studies comparing one kind of uh likely to have outdoor jobs, they’re less likely to do manual placebo against another. So we know, for example, that labor, they have better social support, they’re less likely to two sugar pills a day are a more effective treatment for smoke —so for a whole host of fascinating, interlocking getting rid of gastric ulcers than one sugar pill. Two sugar social, political and cultural reasons, they are less likely to pills a day 16. beats one sugar pill a day. And that’s an have 11. skin wrinkles . That doesn’t mean that it’s the outrageous and ridiculous finding, but it’s true. We know vegetables or the olive oil. (Laughter) from three different studies on three different types of pain So ideally what you want to do is a trial. And everybody that a saltwater injection is a more effective treatment for thinks they’re very familiar with the idea of a trial. Tri- pain than taking a sugar pill, taking a dummy pill that has als are very old.