Opinion VIEWPOINT The Challenge of Reforming Nutritional Epidemiologic Research John P.A. Ioannidis, Some nutrition scientists and much of the public often advocatesdictatewhatprimarystudiesshouldreport.Not MD, DSc consider epidemiologic associations of nutritional factors surprisingly, an independent assessment by the National Stanford Prevention to represent causal effects that can inform public health Academies of Sciences, Engineering, and Medicine of the Research Center and policy and guidelines. However, the emerging picture of nationaldietaryguidelinessuggestedmajorredesignofthe Meta-Research Innovation Center at nutritionalepidemiologyisdifficulttoreconcilewithgood developmentprocessfortheseguidelines:improvingtrans- Stanford (METRICS), scientific principles. The field needs radical reform. parency,promotingdiversityofexpertiseandexperience, Stanford University, In recent updated meta-analyses of prospective supporting a more deliberative process, managing biases Stanford, California. cohort studies, almost all foods revealed statistically and conflicts, and adopting state-of-the-art processes.5 significant associations with mortality risk.1 Substantial Proponents of the status quo may maintain that the deficienciesofkeynutrients(eg,vitamins),extremeover- true associations are even larger than what are reported consumption of food, and obesity from excessive calo- because of attenuation from nondifferential misclassifi- riesmayindeedincreasemortalityrisk.However,cansmall cation. Indeed, self-reported data have error,6 but there intake differences of specific nutrients, foods, or diet pat- is no guarantee it is nondifferential. Nevertheless, if er- terns with similar calories causally, markedly, and almost ror is nondifferential and estimated effects are attenu- ubiquitously affect survival? ated,reportedresultsbecomeevenmoreimplausible:eat- Assuming the meta-analyzed evidence from cohort ing 12 hazelnuts daily would increase life expectancy by studies represents life span–long causal associations, for 20 to 30 years, not just 12 years. abaselinelifeexpectancyof80years,eating12hazelnuts Individuals consume thousands of chemicals in mil- daily (1 oz) would prolong life by 12 years (ie, 1 year per lions of possible daily combinations. For instance, there hazelnut),1 drinking 3 cups of coffee daily would achieve are more than 250 000 different foods and even more a similar gain of 12 extra years,2 and eating a single man- potentially edible items, with 300 000 edible plants darin orange daily (80 g) would add 5 years of life.1 Con- alone. Seemingly similar foods vary in exact chemical sig- versely, consuming 1 egg daily would reduce life expec- natures(eg,morethan500differentpolyphenols).Much tancy by 6 years, and eating 2 slices of bacon (30 g) daily oftheliteraturesilentlyassumesdiseaseriskismodulated would shorten life by a decade, an effect worse than by the most abundant substances; for example, carbo- smoking.1 Could these results possibly be true? Authors hydrates or fats. However, relatively uncommon chemi- often use causal language when reporting the findings cals within food, circumstantial contaminants, serendipi- from these studies (eg, “optimal consumption of risk- tous toxicants, or components that appear only under decreasing foods results in a 56% reduction of all-cause specific conditions or food preparation methods (eg, red mortality”).1 Burden-of-diseasestudiesandguidelinesen- meat cooking) may be influential. Risk-conferring nutri- dorse these estimates. Even when authors add caveats, tional combinations may vary by an individual’s genetic results are still often presented by the media as causal. background, metabolic profile, age, or environmental ex- These implausible estimates of benefits or risks as- posures. Disentangling the potential influence on health sociated with diet probably reflect almost exclusively the outcomesofasingledietarycomponentfromtheseother magnitude of the cumulative biases in this type of re- variables is challenging, if not impossible. search, with extensive residual confounding and selec- Touseananalogyfromgenetics,studyingassociations tive reporting.3 Almost all nutritional variables are cor- ofspecificfoodsislikestudyingwhetherlargechromosomal related with one another; thus, if one variable is causally regions increase mortality risk. For decades, genome link- relatedtohealthoutcomes,manyothervariableswillalso agescansstruggledtolinklargechromosomalareastodis- yield significant associations in large enough data sets. easerisk.Accordingtocurrentknowledge,theseprevious With more research involving big data, almost all nutri- efforts were doomed: each chromosomal area contains tional variables will be associated with almost all out- thousandsofgeneticvariants.Linkagescansresultedinnu- comes. Moreover, given the complicated associations of merousarticles,butlimitedusefulinformation.Retrospec- eating behaviors and patterns with many time-varying tively,usingafewhundredmicrosatellitemarkerstostudy social and behavioral factors that also affect health, no anentiregenomewithmanymillionpolymorphismsseems Corresponding currently available cohort includes sufficient informa- naive. Similarly,limited self-reported nutrition data ascer- Author: John P. A. tion to address confounding in nutritional associations. tainedwithahandfulofquestionsandself-reporteditems Ioannidis, MD, DSc, Stanford Prevention Furthermore,theliteratureisshapedbyinvestigators fail to acknowledge or accurately measure a system that Research Center, whoreportnonprespecifiedresultsthatarepossibletoana- matches or exceeds the genome in complexity. Stanford University, lyzeinverydifferentways.4 Consequently,meta-analyses Beyond food studies, results of single-nutrient stud- 1265 Welch Rd, M/C becomeweightedaveragesofexpertopinions.Inaninverse ies have largely failed to be corroborated in randomized 5411, Stanford, CA 94305 (jioannid sequence, instead of carefully conducted primary studies trials. False-positive associations are common in the lit- @stanford.edu). informing guidelines, expert-driven guidelines shaped by erature.Forexample,updatedmeta-analysesofpublished jama.com (Reprinted) JAMA Published online August 23, 2018 E1 © 2018 American Medical Association. All rights reserved. Downloaded From: by Mexico | Access Provided by JAMA , Mark Tuttle on 08/23/2018 Opinion Viewpoint datafromprospectivecohortstudieshavedemonstratedthatasingle a composite end point but was recently retracted and republished9 antioxidant, beta carotene, has a stronger protective effect on mor- after it was realized that there were multiple subversions of ran- tality than all the foods mentioned above.7 The relative risk of death domization. Findings from the reanalysis showed results similar to for the highest vs lowest group of beta carotene levels in serum or those of the initially reported findings; however, the study should plasma was 0.69 (95% CI, 0.59-0.80).7 Even when measurement er- no longer be considered a randomized trial. Regardless, the trial ror is mitigated with biochemical assays (as in this example), nutri- showed no survival benefit. Large pragmatic trials for more com- tionalepidemiologyremainsintrinsicallyunreliable.Theseresultscan- plex diet patterns also may yield largely negative results. Neverthe- not be considered causal, especially after multiple large trials have less,theiroutcomesmayhelpinformnutritionalguidelineswithsome yielded CIs excluding even a small benefit. pragmatic “intention-to-eat” data. Proponents of the status quo of nutritional epidemiology point Reform has long been due. Data from existing cohorts should be- to occasional small trials with surrogate or metabolic outcomes come available for reanalysis by independent investigators. Their re- (eg, lipids, diabetes, composite end points) whose results agree with sultsshouldbepresentedintheirtotalityforallnutritionalfactorsmea- epidemiologic findings. However, these small trials often have selec- sured,withstandardizedmethodsandstandardizedexplorationofthe tive reporting bias similar to that of nutritional epidemiology. sensitivity of conclusions to model and analysis choices. Readers and Nutritional research may have adversely affected the public per- guidelinedevelopersmayignorehastystatementsofcausalinference ceptionofscience.Resourcesforsomeofthesestudiescouldhavebeen and advocacy to public policy made by past nutritional epidemiology better spent on unambiguous, directly manageable threats to health articles.10 Such statements should be avoided in the future. suchassmoking,lackofexercise,airpollution,orclimatechange.More- The nutritional epidemiology community includes superb scien- over, the perpetuated nutritional epidemiologic model probably also tists. The best of them should take ownership of this reform process. harms public health nutrition. Unfounded beliefs that justify eating They can further lead by example (eg, by correcting their own ar- morefood,provided“qualityfood”isconsumed,confusethepublicand ticles that have misleading claims). Such corrections would herald detract from the agenda of preventing and treating obesity. high scientific standards and public responsibility. A flawed method- Confusion is further enhanced by some approaches to publica- ologicalapproachhasdominatedresearchquestionsthathaveproved tion in this field. Slices of data are often published from a cohort with- particularly difficult to answer, more difficult than those of other epi- out accounting for other findings from the same cohort. A single demiologic