Faces Its Limits The search for subtle links between diet, lifestyle, or environmental factors and disease is an unending source of fear-but often yields little certainty

The news about health on the press for its report- Rothman, editor of the journal Epidemiology: risks comes thick and fast ing of epidemiology, and “We’repushing the edge of what can be done these days, and it seems al- even on the public “for its with epidemiology.” most constitutionally con- unrealistic expectations” of With epidemiology stretched to its limits tradictory. In January of what modern medical re- or beyond, says Dimitrios Trichopoulos, last year, for instance, a search can do for their head of the epidemiology department at the Swedish study found a sig- health. But many epidemi- Harvard School of Public Health, studies nificant association be- ologists interviewed by Sci- will inevitably generate false positive and tween residential radon ence say the problem also false negative results “with disturbing fre- exposure and lung cancer. lies with the very nature of quency.” Most epidemiologists are aware of A Canadian study did not. epidemiologic studies-in the problem, he adds, “and tend to avoid Three months later, it was particular those that try to causal inferences on the basis of isolated pesticide residues. The isolate causes of noninfec- studies or even groups of studies in the ab- Journal of the National Can- tious disease, known vari- sence of compelling biomedical evidence. cer Institute published a ously as “observational” or However, exceptions do occur, and their fre- study in April reporting- “risk-factor” or “environ- quency appears to be increasing.” As contrary to previous, less mental” epidemiology. Trichopoulos explains, “Objectively the powerful studies-that the The predicament of problems are not more than they used to be, presence of DDT metabo- these studies is a simple one: but the pressure is greater on the profession, lites in the bloodstream Anxiety epidemic. Protesting risks Over the past 50 years, epi- and thenumber who practice it is greater.” seemed to have no effect that may-or may not-be real. demiologists have succeeded As a result, journals today are full of stud- on the risk of breast can- in identifying the more con- ies suggesting that a little risk is not nothing cer. In October. it was soicuous determinants of at all. The findines are often touted in mess abortions and breast cancer. Maybe yes. noninfectious diskases-smoking, for in- releases by the journals that publish them Maybe no. In January of this year it was stance, which can increase the risk ofdevelop- or by the researchers’ institutions, and news- electromagnetic fields (EMF) from power ing lung cancer by as much as 3000%. Now papers and other media often report the lines. This time a study of electric utility they are left to search for subtler links be- claims uncritically (see box on p. 166).And workers in the United States suggested a pos- tween diseases and environmental causes or so the anxiety pendulum swings at an ever sible link between EMF and brain cancer lifestyles. And that leads to the Catch-22 of more dizzying rate. “We are fast becoming a but-contrary to a study a year ago in modern epidemiology. nuisance to society,” says Trichopoulos. Canada and France-no link between EMF On the one hand, these subtle risks-say, “People don’t take usseriously anymore, and and leukemia. the 30% increase in the risk when they do take us seri- These are not isolated examples of the of breast cancer from alco- iT ously, we may unintention- conflicting nature of epidemiologic studies; hol consumption that some ally do more harm than good.” they’re just the latest to hit the newspapers. studies suggest-may affect 5 As a solution, epidemiolo- Over the years, such studies have come up such a large segment of the 8 gists interviewed by with a mind-numbing array of potential dis- population that they have $ could suggest only that the ease-causing agents, from hair dyes (lym- potentially huge impacts on press become more skeptical phomas, myelomas, and leukemia) to coffee public health. On the other, of epidemiologic findings, (pancreatic cancer and heart disease) to oral many epidemiologists con- that epidemiologists become contraceptives and other hormone treat- cede that their studies are so more skeptical about their ments (virtually every disorder known to plagued with , uncer- own findings-or both. woman). The pendulum swings back and tainties, and methodologi- forth, subjecting the public to an “epidemic cal weaknesses that they An observational science of anxiety,” as Lewis Thomas put it over a may be inherently incapable What drives the epidem- decade ago. Indeed, last July, the New England of accurately discerning siPeople don’t take iologic quest for risk factors is Journal of Medicine (NEJM) published an edi- such weak associations. As the strong circumstantial torial by editors Marcia Angell and Jerome Michael Thun, the director us seriously 9.. and evidence that what we eat, Kassirer asking the pithy question, “What of analytic epidemiology for when they do ... drink, breathe, and so on are Should the Public Believe?” Health-con- the American Cancer Soci- we may &,inten- major factors in many devas- scious Americans, wrote Angell and Kas- ety, puts it, “With epidemi- tating illnesses. Rates of tionally do more sirer, “increasingly find themselves beset by oloeva, vou, can tell a little heart disease, for examde.., contradictory advice. No sooner do they thing from a big thing. harm than good.” have changed much faster learn the results of one research study than What’s very hard to do is to --Dimitrios over recent decades than can they hear of one with the opposite message.” tell a little thing from noth- be explained by genetic Kassirer and Angell place responsibility ing at all.” Agrees Ken Trichopou’os changes, implicating dietary

164 SCIENCE VOL. 269 14 JULY 1995 ___ __-_ Sizing Up the Cancer Risks

Inthe history of epidemiology, only a dozen or so environmental Use of phenoxy herbicides on lawns-rr 1.3 for malignant agents have ever heen repeatedly and strongly linked to human lymphoma in dogs (September 1991) cancer, says University of Alabama epidemiologist Philip Cole. Weighing 3.6 kilograms or more at birth-rr 1.3 for breast Among them are cigarette smoke, :ilcohol, ionizing radi:ition, ii cancer (October 1992) few drugs, a handful of tkccupational carcinogcns, such as nshestos, and perhaps three viruses-hepatitis-R virus, hum;inTceIl leuke- Vasectomy-rr 1.6 for prostate cancer (February 1993) mia virus, and human papillomavirus. But every yex, epicfern- iologic papers are puhlished by the journal-load, many of them Pesticide exposure, indicated by high residues in blood-rr 4 reporting new potential causes of cancer in the environment. for breast cancer (April 1993); contradicted 1 year later in a Most are the prtdtlct of nhscnmional epidemiology, in which larger study with one of the same authors. researchers try to compare the lives of people suffering from a Drinking more than 3.3 liters of fluid (particularly chlorinated disease with those of healthy controls. Even its prxtiticiners tap water) a day-rr 2-4 for bladder cancer (July 1993) admit this cffort is plagued by biases and confounding factors (see main text). As a result, most epidemiologists intervirwed by Experiencing psychological stress in the workplace-rr 5.5 for Science said they would not take seriously :I sinfile study reporting colorectal cancer (September 1993) a new potential cause oicancer unless it reported that exposure to Diet high in saturated fat-rr 6 for lung cancer in nonsmoking the agent in question increased person’s risk :tt least a fxtor a hy women (December 1993) of %-which is to say it carries a risk ratinof3. Even then, they say, skepticism is in ortfer unless the study was very large and ex- Eating more than 20 grams of processed meats (i.e., bologna) tremely well done and biological data supporr the hypothesized a day-rr 1.72 for colon cancer (February 1994) link. Sander Greenland, a university of Californi:i, Lo.; Angeles, epidemiologist. says a study reporting a twofold increasd risk Eating red meat five or more times a week-rr 2.5 for colon might then he worth taking seriously-“but nor that seriously.” cancer (February 1994) Few of the entries in the following list ofpotcnrial cancer risks, Occupational exposure to electromagnetic fields-rr 138 for reported in the jcturnals and picked tip in the popular press over breast cancer (June 1994) the past 8 years, have come close to fulfilling those criteria. Are these dangers real? As the saying goes, you be the judge. Smoking two packs of cigarettes a day-rr 1.74 for fatal breast 4.T. cancer (July 1994) Eating red meat twice a day-rr 2 for breast cancer (July 1994) High-cholesterol diet-risk ratio (rr) 1.65 for rectal cancer in men (January 1987) Reguiar cigarette smoking-rr 1.7 for pancreatic cancer (Octo- ber 1994) Eating yogurt at least once a month-rr 2 for ovarian cancer (July 1989) Ever having used a sun lamp-rr 1.3 for melanoma (November Smoking more than i00 cigarettes in a lifetime-rr 1.2 for 1994) breast cancer (February 1990) Abortion-rr 1.5 for breast cancer (November 1994) High-fat diet-rr 2 for breast cancer (August 1990) Having shorter or longer than average menstrual cycles-rr 2 Lengthy occupational exposure to dioxin-rr 1.5 for all cancers for breast cancer (December 1994) (January 1991) Obesity in men (the heaviest 25% of those in the studykrr 3 Douching once a week-rr 4 for cervical cancer (March 1991) for esophageal cancer (January 1995) Regular use of high-alcohol mouthwash-rr 1.5 for mouth Consuming olive oil only once a day or less-rr 1.25 for breast cancer (June 1991) cancer (January 1995) and environmental causes. And the fact that search: Assign subjects at random to test and Because the experimental approach is off- no single cancer affects every population at control groups, alter the exposure of the test limits for much of epidemiology, researchers the same rate suggests that factors external to group to the suspected risk factor, and follow resort to observational approaches. In case- the human body cause 70% to 90% of all both groups to learn the outcome. Often, control studies, for example, they select a cancers. In other words, says Richard Peto, both the experimenters and the subjects are group of individuals afflicted with a particu- an Oxford University epidemiologist, “there “blinded”-unaware who is in the test group lar disorder, then identify a control group are ways in which human beings can live and who is a control. But randomized trials free of the disorder and compare the two, whereby those cancers would not arise.” would be prohibitively slow and expensive looking for differences in lifestyle, diet, or Only a few of these environmental factors for most risk factors, because they can take some environmental factor. Potentially are known-cigarette smoke for lung cancer, years or decades to show an effect and hun- more reliable, but also much more costly, are for example, or sunlight for skin cancer- dreds of thousands of individuals may need to cohort studies, in which researchers take a and epidemiology seems to provide the best be followed to detect enough cases of the large population-as many as 100,000-and shot at identifying the others. disease for the results to be significant. And question the subjects in detail about their The most powerful tool for doing so is the randomly subjecting thousands of healthy habits and environment. They then follow randomized trial, which is the standard for people to pollutants or other possible car- the entire population for years or decades to studies of new drugs and other medical re- cinogens raises obvious ethical problems. see who gets sick and who doesn’t, what dis- SCIENCE VOL. 269 14 JULY 1995 165 eases they suffer from, and what factors might dialing: Researchers take the phone numbers sure can be measured reliably, a subtle asso- be different between them. Either way, risk- of their cases and randomly change the last ciation may be credible-as it is in the case of factor epidemiology is “a much duller scal- four digits until they find a suitable control. early childbirth and a lower risk of breast pel” than randomized trials, says Scott Zeger, Random digit dialing, however, seems to cre- cancer. The reason is that both cause and a biostatistician at the Johns Hopkins School ate “a pronounced toward the control effect can be measured with some certainty, of Mental and Public Health. group being deficient in persons of very low says Harvard epidemiologist Jamie Robins. What blunts its edge are systematic errors, socioeconomic status,” says Poole. Poor ‘‘It’s easy to know which people got breast known in the lingo as biases and confound- people, it seems, are either less likely to be cancer, and it’s easy to know at what age they ing factors. “Bias and confounders are the home during the day to answer the phone, had kids,” he says, adding that virtually every plague upon the house of epidemiology,” says less likely to want to take part in a study, or study on the subject comes to the same con- Philip Cole, chair of epidemiology at the less likely to have an answering machine and clusion: Early childbirth reduces the risk by University of Alabama. They represent any- call the researchers back. about 30%. thing that might lead an epidemiologic study Indeed, the North Carolina researchers But epidemiologists are quick to list risk to come up with the wrong answer, to postu- reported that their data showed that the risk factors for which accurate exuosure measure- late the existence of a causal association that ofleukemia and brain cancer ments are virtually impossible. does not exist or vice versa. rises not just with exposure Joe Fraumeni, director of the Confounding factors are the hidden vari- to EMF but also with higher epidemiology and biostatis- ables in the populations being studied, which levels of breast-feeding, ma- tics program at the National can easily generate an association that may ternal smoking, and traffic Cancer Institute (NCI), be real but is not what the epidemiologist density, all of which are mar- points to radon: “When you’re thinks it is. A ubiquitous example is cigarette kers for poverty. This sug- studying smoking,” he says, smoking, which can confound any study gests, says Poole, that the study “that’s easy. Just count the looking, for instance, at the effects of alcohol group was poorer than the number of cigarettes and du- on cancer. “It just so happens,” explains controls, and that some pov- ration and packs per day. But Trichopoulos, “that people who drink also erty-associated factor other something like radon, how tend to smoke,”boosting their risk of cancer. than EMF could have re- do you measure exposure, As a result, epidemiologists face the possibil- sulted in the apparent in- particularly biologically rel- ity that any apparent cancer-alcohol link crease in cancer risk. None- evant exposure that has may be spurious. Smoking may also have theless, the study is still cited “We’re pushing taken place in the past?’’ confounded a study Trichopoulos himself as supporting the hypothesis Equally uncertain are those co-authored linking coffee-drinking and that EMF causes childhood the edge of what risk factors recorded only in pancreatic cancer-a finding that has not cancer, although even Savitz can be done with human memory, such as been replicated. The study, published over a concedes that the random dig- consumption of coffee or di- epidemiology. ” decade ago, corrected for smoking, which it dialing problem is “a legiti- etary fat. Ross Prentice of often accompanies heavy coffee drinking- mate source of uncertainty.’’ -Ken Rothman the University of Wash- but only for smoking during the 5 years be- Even when such biases ington notes, for example, fore the cancer was diagnosed. Trichopoulos can be identified, their mag- that underweight individu- I now says that he and his colleagues might nitude-and sometimes even their direc- als tend to overreport fat intake on question- have done better to ask about smoking habits tion-can be nearly impossible to assess. naires or in interviews and obese subjects a full 20 years before diagnosis. David Thomas, for example, an epidemiolo- tend to underreport it. Biases are problems within study designs gist at the Fred Hutchinson Cancer Research Such recall bias is known to be especially themselves. The process of choosing an ap- Center in Seattle, points to studies analyzing strong, as Willett points out, among patients propriate population of controls in a case- the effect of Breast Self-Examination (BSE) diagnosed with the disease in question or control study, for instance, can easily lead to on breast cancer mortality rates, which, he among their next of kin. In studies of a pos- an apparent difference between cases and says, have yielded some “modest suggestion sible relationship between fat intake and controls that has nothing to do with what that there might be a beneficial effect” from breast cancer, for instance, says Willett, caused the disease. “It’s often not even theo- BSE. “You have to ask what motivates a “people may recall their past intake of fat retically clear who the right comparison woman to practice BSE,” says Thomas. differently if they have just been diagnosed group is,” says Harvard epidemiologist “Maybe she has a strong family history of with breast cancer than if you pluck them out Walter Willett. “And sometimes, even if you breast cancer. If so, she’s more likely to get of a random sample, call them up out of the can design the study so that you have the breast cancer. That would be an obvious blue over the phone, and ask them what their theoretically correct comparison group, you bias,” which could make BSE look less useful past diet was.” usually don’t get everybody willing to partici- than it is. “Or maybe a woman with a strong Recall bias, for instance, apparently ac- pate, and the people who do participate in family history of breast cancer would be counts for the conflicting findings about oral your study will be different from the people afraid to practice BSE. You have no way of contraceptive use and breast cancer. Many who don’t, often in health-related ways.” predicting the direction of the bias. So it studies have looked for this association over For example, Charles Poole of Boston would be very difficult to interpret your re- the years, both case-control studies and co- University has spent several years analyzing sults. You have to go to a randomized study to hort studies. Trichopoulos notes that case- the results and methodology of a 1988 study get a reliable answer.” control studies have tended to show an asso- of EMF and cancer, which found that expo- ciation between oral contraceptives and sure to relatively high EMF from power lines Tricks of memory breast cancer, while cohort studies have not. appeared to increase the risk of leukemia and Of all the biases that plague the epidemio- Epidemiologists who have done cohort stud- brain cancer in children. David Savitz of the logic study of risk factors, the most pernicious ies say the problem is in case-control studies, University of North Carolina, the study’s is the difficulty of assessing exposure to a which are thrown off by recall bias-women author, selected controls for that study with a particular risk factor. Rothman, for instance, who are diagnosed with breast cancer are common technique known as random digit calls it “a towering obstacle.” When expo- more likely to give complete information

SCIENCE * VOL. 269 * 14 JULY 1995 167 about contraceptive use than women who mathematical techniques is co-authored a study errone- don’t. Those who did case-control studies say people will think they have ously suggesting that women the bias is in the cohort studies. Cohort stud- been able to control for who took the anti-hyperten- ies have to rely on impersonal questionnaires things that are inherently sion medication reserpine because they are so much larger than case- not controllable.” had up to a fourfold increase control studies, and women are less likely to Breslow adds that epide- in their risk of breast cancer, give complete and honest information than miologists will commonly re- suggests that no single epi- they are in the more intimate interviews pos- port that they have unveiled demiologic study is persua- sible in case-control studies. “The point,” a possible causal association sive by itself unless the lower says Trichopoulos, “is which do we believe.” between a risk factor and a limit of its 95% confidence It’s not just the subjects of studies who are disease because the associa- level falls above a threefold prone to bias; epidemiologic studies can be tion is “statistically signifi- increased risk. Other re- plagued by interviewer bias as well. The in- cant,”meaning that the error searchers, such as Harvard’s terviewers are rarely blinded to cases and bars-the limits of a 95% Trichopoulos, opt for a four- controls, after all, and questionnaires, the confidence interval40 not “Authors and fold risk increase as the traditional measuring instrument of epide- include the null result, which investigators are lower limit. Trichopoulos’s miology, are neither peer-reviewed nor pub- is the absence of an effect. ill-fated paper on coffee con- lished with the eventual papers. “In the labo- But, as Breslow explains, such worried that sumption and pancreatic ratory,” as Yale University clinical epidemi- statistical “confidence”means there’s a bias cancer had reported a 2.5- ologist Alvin Feinstein puts it, “you have all considerably less than It seems against negative fold increased risk. kinds of procedures for calibrating equip- to. The calculation of confi- “As a general rule of ment and standardizing measurement proce- dence limits only takes into studies.” thumb,” says Angell of the dures. In epidemiology . . . it’s all immensely consideration random varia- -Marcia Angell New EnglandJournal, “we are prey to both the vicissitudes of human tion in the data. It ignores looking for a of memory and the biases of the interview.” the systematic errors, the bi- three or more [before accept- ases and confounders, that will almost in- ing a paper for publication], particularly if it Salvation from statistics? variably overwhelm the statistical variation. is biologically implausible or if it’s a brand- With confounders, biases, and measurement University of California, Los Angeles new finding.” Robert Temple, director of drug errors virtually inevitable, many epidemiolo- (UCLA) epidemiologist Sander Greenland evaluation at the Food and Drug Adminis- gists interviewed by Science say that risk-fac- says most of his colleagues fail to understand tration, puts it bluntly: “My basic rule is if the tor epidemiology is increasingly straying be- this simple point. “What people want to do relative risk isn’t at least three or four, forget yond the limits of the possible no matter how when they see a 95% confidence [interval],” it.” But as John Bailar, an epidemiologist at carefully the studies are done. “I have trouble he says, “is say ‘I bet there’s a 95% chance the McGill University and former statistical imagining a system involving a human habit true value is in there.’ Even if they deny it, consultant for the NEJM, points out, there is over a prolonged period of time that could you see them behaving and discussing their no reliable way of identifying the dividing give you reliable estimates of [risk] increases study result as though that’s exactly what it line. “If you see a 10-fold relative risk and it’s that are of the order of tens of percent,” says means. There are certain conditions under replicated and it’s a good study with biologi- Harvard epidemiologist Alex Walker. Even which it’s not far from the truth, but those cal backup, like we have with cigarettes and the sophisticated statistical techniques that conditions are generally not satisfied in an lung cancer, you can draw a strong infer- have entered epidemiologic research over epidemiologic studv.” ence.” he savs. “If it’s a 1.5 relative risk, and the past 20 years-tools for teasing out subtle it’s only one’study and even a very good’one, effects, calculating the theo- What to believe? you scratch your chin and say maybe.” retical effect of biases, cor- So what does it take to Some epidemiologists say that an asso- recting for possible con- make a study worth taking ciation with an increased risk of tens of per- founders, and so on-can’t seriously? Over the years, cent might be believed if it shows up consis- compensate for the limita- epidemiologists have of- tently in many different studies. That’s the tions of the data, says biostat- fered up a variety of crite- rationale for meta-analysis-a technique for istician Norman Breslow of ria, the most important of combining many ambiguous studies to see the University of Washing- which are a very strong asso- whether they tend in the same direction ton, Seattle. ciation between disease and (Science, 3 August 1990, p. 476). But when “In the past 30 years,” risk factor and a highly Science asked epidemiologists to identify he says, “the methodology plausible biological mecha- weak associations that are now considered has changed a lot. Today nism. The epidemiologists convincing because they show up repeatedly, people are doing much more interviewed by Science say opinions were divided-consistently. in the way of mathematical they prefer to see both be- Take the question of alcohol and breast modeling of the results of “People [may] fore believing the latest cancer. More than 50 studies have been their study, fitting of regres- study, or even the latest done, and more than 30 have reported that sion equations, regression think they have group of studies. Many re- women who drink alcohol have a 50% in- analysis. But the question re- been able to spected epidemiologistshave creased risk of breast cancer. Willett, whose mains: What is the funda- control for things published erroneous results Nurse’s Health Study was among those that mental quality of the data, in the past and say it is so showed a positive association, calls it “highly and to what extent are there that are easy to be fooled that it is probable” that alcohol increases the risk of biases in the data that cannot inherently not almost impossible to believe breast cancer. Among other compelling fac- be controlled by statistical controllable.” less-than-stunning results. tors, he says, the finding has been “repro- analysis? One of the dangers Sir Richard Doll of Ox- duced in many countries with many investi- of having all these fancy -Norman Breslow ford University, who once gators controlling for lots of confounding

168 SCIENCE VOL. 269 14 JULY 1995 variables, and the association keeps coming effect may be so dramatic that it would be bladder cancer-one of the few cases in up.” But Greenland isn’t so sure. “I’d bet irresponsible not to publish it. The Univer- which epidemiology had managed to put an right now there isn’t a consensus. I do know sity of North Carolina’s Savitz, for instance, end to a suspected association. Yet 14 years just from talking to people that some hold it’s who recently claimed a possible link between later, television advertisements for Nutra- a risk factor and others deny it.” Another EMF exposure and a tens of percent increase Sweet, which contains the artificial sweet- Boston-based epidemiologist, who prefers to in the risk of breast cancer, says: “This is ener aspartame, still tout it as the sweetener remain anonymous, says nobody is con- minute. . . . But you could make an argument I that does not have saccharine. vinced of the breast cancer-alcohol connec- that even if this evidence is 1000-fold less Epidemiologists themselves are at a loss as tion “except Walt Willett.” than for [an EMF-leukemia link], it is still to how to curb the “anxiety of the week” Another example is long-term oral con- more important, because the disease is 1000- syndrome. Many, like Rothman, simply ar- traceptive use and breast cancer, a link that fold more prevalent.” gue that risk factor epidemiology is a young has been studied for a quarter of a century. One of the more pervasive arguments for science that will take time to mature. Others, Thomas of the Fred Hutchinson Cancer Re- publishing weak effects, Rothman adds, is like Robins, suggest that barring a major search Center says he did a meta-analysis in that any real effect may be stronger than the breakthrough in the methodological tools of 1991 and found a dozen studies showing a reported one. Any mismeasurement of expo- epidemiology, maturity will be hard to come believable association in younger women sure, so the argument goes, will only serve to by. The pressures to publish inconclusive re- who were long-time users of oral contracep- reduce the observed size of the association. sults and the eagerness of the press to publi- tives. “The bottom line,” he says, “is it’s Once researchers learn how to measure ex- cize them, he and others say, mean that the taken us over 20 years of studies before some posure correctly, in other words, the actual anxiety pendulum. like Foucault’s. will con- consistency starts to emerge. Now it’s fairly association will turnout to be tinue to swing indefinitely clear there’s a modest risk.” But Noel Weiss bigger-and thus more criti- (see box on p. 165). of the University of Washington says he did cal to public health. That was The FDA’s Temple does a similar review of the data that left him the case in studies of steel- make one positive sugges- unconvinced. “We don’t know yet,” he says. workers and lung cancer de- tion: Although risk-factor “There is a small increased risk associated cades ago, says Robins. Early epidemiology will never be [with oral contraceptive use], but what that studies saw only a weak asso- as sharp a tool as random- represents is unclear.” Mary Charleson, a ciation, but once researchers ized clinical trials, epidemi- Cornell Medical Center epidemiologist, homed in on coke-oven ologists could still benefit by calls the association “questionable.” Marcia workers, the group most ex- adopting some of the scien- Angell calls it “still controversial.” posed to the carcinogens, the tific practices of those stud- Consistency has a catch, after all, explains relative risk shot up. None ies. “The great thing about a David Sackett of Oxford University: It is per- of the epidemiologists who clinical control trial,” he suasive only if the studies use different archi- spoke to Science could recall says, “is that, within limits, tectures, methodologies, and subject groups any more recent parallels, The sin comes in you don’t have to believe and still come up with the same results. If the however. anybody or trust anybody. studies have the same design and “if there’s believing a causal The planning for a clinical an inherent bias,” he explains, “it wouldn’t An unholy alliance hypothesis is control trial is prospective; make any difference how many times it’s rep- There would be few draw- true because they’ve written the licated. Bias times 12 is still bias.” What’s backs to publishing weak, before they’ve done the more, the epidemiologists interviewed by uncertain associations if epi- your study came study, and any deviation Science point out that an apparently consis- demiologists operated in a up with a positive that you introduce later is tent body of published reports showing a vacuum, wrote Brian Mac- result. completely visible.” While positive association between a risk factor and Mahon, professor emeritus of agencies like the NCI do in- a disease may leave out other, negative find- epidemiology at Harvard, in -Sander Greenland sist on seeing study proto- ings that never saw the light of day. an Ad1994editorial in the cols in risk-factor epidemi- “Authors and investigators are worried Jou&l of the National Cancer Institute. But ology prospectively, this is still not standard that there’s a bias against negative studies,” they do not, he said. “And, however cau- procedure throughout the field. Without it, and that they will not be able to get them tiously the investigator may report his con- says Temple, “you always wonder how published in the better journals, if at all, says clusions and stress the need for further evalu- many ways they cut the data. It’s very hard Angell of the NEjM. “And so they’ll try very ation,” he added, “much of the press will pay to be reassured, because there are no rules hard to convert what is essentially a negative little heed to such cautions. ... By the time for doing it.” study into a positive study by hanging on to the information reaches the public mind, via In the meantime, UCLA’s Greenland has very, very small risks or seizing on one posi- print or screen, the tentative suggestion is one piece of advice to offer what he calls his tive aspect of a study that is by and large likely to be interpreted as a fact.” “most sensible, level-headed, estimable col- negative.” Or, as one National Institute of This is what one epidemiologist calls the leagues.” Remember, he says, “there is noth- Environmental Health researcher “unholy alliance” between epidemiology, the ing sinful about going out and getting evi- puts it, asking for anonymity, “Investigators journals, and the lay press. The first one or dence, like asking people how much do you who find an effect get support, and investiga- two papers about a suspected association drink and checking breast cancer records. tors who don’t find an effect don’t get support. “spring into the general public consciousness There’s nothing sinful about seeing if that When times are tough it becomes extremely in way that does not happen in any other evidence correlates. There’s nothing sinful difficult for investigators to be objective.” field of scientific endeavor,” says Harvard’s about checking for confounding variables. When asked why they so willingly publish Walker. And once a possible link is in the The sin comes in believing a causal hypoth- inconclusive research, epidemiologists say public eye, it can be virtually impossible to esis is true because your study came up with a they have an obligation to make the data discredit. As far as scientists were concerned, positive result, or believing the opposite be- public and justify the years of work. They also for instance, a 1981 epidemiologic study put cause your study was negative.” argue that if the link is real, the public health to rest a suggestion that saccharine can cause -Gary Taubes

SCIENCE VOL. 269 14 JULY 1995 169 Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission.