The Challenge of Reforming Nutritional Epidemiologic Research

Opinion

VIEWPOINT The Challenge of Reforming Nutritional Epidemiologic Research

John P.A. Ioannidis, Some nutrition scientists and much of the public often advocatesdictatewhatprimarystudiesshouldreport.Not MD, DSc consider epidemiologic associations of nutritional factors surprisingly, an independent assessment by the National Stanford Prevention to represent causal effects that can inform public health Academies of Sciences, Engineering, and Medicine of the Research Center and policy and guidelines. However, the emerging picture of nationaldietaryguidelinessuggestedmajorredesignofthe Meta-Research Innovation Center at nutritionalepidemiologyisdifficulttoreconcilewithgood developmentprocessfortheseguidelines:improvingtrans- Stanford (METRICS), scientific principles. The field needs radical reform. parency,promotingdiversityofexpertiseandexperience, Stanford University, In recent updated meta-analyses of prospective supporting a more deliberative process, managing biases Stanford, California. cohort studies, almost all foods revealed statistically and conflicts, and adopting state-of-the-art processes.5 significant associations with mortality risk.1 Substantial Proponents of the status quo may maintain that the deficienciesofkeynutrients(eg,vitamins),extremeover- true associations are even larger than what are reported consumption of food, and obesity from excessive calo- because of attenuation from nondifferential misclassifi- riesmayindeedincreasemortalityrisk.However,cansmall cation. Indeed, self-reported data have error,6 but there intake differences of specific nutrients, foods, or diet pat- is no guarantee it is nondifferential. Nevertheless, if er- terns with similar calories causally, markedly, and almost ror is nondifferential and estimated effects are attenu- ubiquitously affect survival? ated,reportedresultsbecomeevenmoreimplausible:eat- Assuming the meta-analyzed evidence from cohort ing 12 hazelnuts daily would increase life expectancy by studies represents life span–long causal associations, for 20 to 30 years, not just 12 years. abaselinelifeexpectancyof80years,eating12hazelnuts Individuals consume thousands of chemicals in mil- daily (1 oz) would prolong life by 12 years (ie, 1 year per lions of possible daily combinations. For instance, there hazelnut),1 drinking 3 cups of coffee daily would achieve are more than 250 000 different foods and even more a similar gain of 12 extra years,2 and eating a single man- potentially edible items, with 300 000 edible plants darin orange daily (80 g) would add 5 years of life.1 Con- alone. Seemingly similar foods vary in exact chemical sig- versely, consuming 1 egg daily would reduce life expec- natures(eg,morethan500differentpolyphenols).Much tancy by 6 years, and eating 2 slices of bacon (30 g) daily oftheliteraturesilentlyassumesdiseaseriskismodulated would shorten life by a decade, an effect worse than by the most abundant substances; for example, carbo- smoking.1 Could these results possibly be true? Authors hydrates or fats. However, relatively uncommon chemi- often use causal language when reporting the findings cals within food, circumstantial contaminants, serendipi- from these studies (eg, “optimal consumption of risk- tous toxicants, or components that appear only under decreasing foods results in a 56% reduction of all-cause specific conditions or food preparation methods (eg, red mortality”).1 Burden-of-diseasestudiesandguidelinesen- meat cooking) may be influential. Risk-conferring nutri- dorse these estimates. Even when authors add caveats, tional combinations may vary by an individual’s genetic results are still often presented by the media as causal. background, metabolic profile, age, or environmental ex- These implausible estimates of benefits or risks as- posures. Disentangling the potential influence on health sociated with diet probably reflect almost exclusively the outcomesofasingledietarycomponentfromtheseother magnitude of the cumulative biases in this type of re- variables is challenging, if not impossible. search, with extensive residual confounding and selec- Touseananalogyfromgenetics,studyingassociations tive reporting.3 Almost all nutritional variables are cor- ofspecificfoodsislikestudyingwhetherlargechromosomal related with one another; thus, if one variable is causally regions increase mortality risk. For decades, genome link- relatedtohealthoutcomes,manyothervariableswillalso agescansstruggledtolinklargechromosomalareastodis- yield significant associations in large enough data sets. easerisk.Accordingtocurrentknowledge,theseprevious With more research involving big data, almost all nutri- efforts were doomed: each chromosomal area contains tional variables will be associated with almost all out- thousandsofgeneticvariants.Linkagescansresultedinnu- comes. Moreover, given the complicated associations of merousarticles,butlimitedusefulinformation.Retrospec- eating behaviors and patterns with many time-varying tively,usingafewhundredmicrosatellitemarkerstostudy social and behavioral factors that also affect health, no anentiregenomewithmanymillionpolymorphismsseems Corresponding currently available cohort includes sufficient informa- naive. Similarly,limited self-reported nutrition data ascer- Author: John P. A. tion to address confounding in nutritional associations. tainedwithahandfulofquestionsandself-reporteditems Ioannidis, MD, DSc, Stanford Prevention Furthermore,theliteratureisshapedbyinvestigators fail to acknowledge or accurately measure a system that Research Center, whoreportnonprespecifiedresultsthatarepossibletoana- matches or exceeds the genome in complexity. Stanford University, lyzeinverydifferentways.4 Consequently,meta-analyses Beyond food studies, results of single-nutrient stud- 1265 Welch Rd, M/C becomeweightedaveragesofexpertopinions.Inaninverse ies have largely failed to be corroborated in randomized 5411, Stanford, CA 94305 (jioannid sequence, instead of carefully conducted primary studies trials. False-positive associations are common in the lit- @stanford.edu). informing guidelines, expert-driven guidelines shaped by erature.Forexample,updatedmeta-analysesofpublished

jama.com (Reprinted) JAMA Published online August 23, 2018 E1

Downloaded From: by Mexico | Access Provided by JAMA , Mark Tuttle on 08/23/2018 Opinion Viewpoint

datafromprospectivecohortstudieshavedemonstratedthatasingle a composite end point but was recently retracted and republished9 antioxidant, beta carotene, has a stronger protective effect on mor- after it was realized that there were multiple subversions of ran- tality than all the foods mentioned above.7 The relative risk of death domization. Findings from the reanalysis showed results similar to for the highest vs lowest group of beta carotene levels in serum or those of the initially reported findings; however, the study should plasma was 0.69 (95% CI, 0.59-0.80).7 Even when measurement er- no longer be considered a randomized trial. Regardless, the trial ror is mitigated with biochemical assays (as in this example), nutri- showed no survival benefit. Large pragmatic trials for more com- tionalepidemiologyremainsintrinsicallyunreliable.Theseresultscan- plex diet patterns also may yield largely negative results. Neverthe- not be considered causal, especially after multiple large trials have less,theiroutcomesmayhelpinformnutritionalguidelineswithsome yielded CIs excluding even a small benefit. pragmatic “intention-to-eat” data. Proponents of the status quo of nutritional epidemiology point Reform has long been due. Data from existing cohorts should be- to occasional small trials with surrogate or metabolic outcomes come available for reanalysis by independent investigators. Their re- (eg, lipids, diabetes, composite end points) whose results agree with sultsshouldbepresentedintheirtotalityforallnutritionalfactorsmea- epidemiologic findings. However, these small trials often have selec- sured,withstandardizedmethodsandstandardizedexplorationofthe tive reporting bias similar to that of nutritional epidemiology. sensitivity of conclusions to model and analysis choices. Readers and Nutritional research may have adversely affected the public per- guidelinedevelopersmayignorehastystatementsofcausalinference ceptionofscience.Resourcesforsomeofthesestudiescouldhavebeen and advocacy to public policy made by past nutritional epidemiology better spent on unambiguous, directly manageable threats to health articles.10 Such statements should be avoided in the future. suchassmoking,lackofexercise,airpollution,orclimatechange.More- The nutritional epidemiology community includes superb scien- over, the perpetuated nutritional epidemiologic model probably also tists. The best of them should take ownership of this reform process. harms public health nutrition. Unfounded beliefs that justify eating They can further lead by example (eg, by correcting their own ar- morefood,provided“qualityfood”isconsumed,confusethepublicand ticles that have misleading claims). Such corrections would herald detract from the agenda of preventing and treating obesity. high scientific standards and public responsibility. A flawed method- Confusion is further enhanced by some approaches to publica- ologicalapproachhasdominatedresearchquestionsthathaveproved tion in this field. Slices of data are often published from a cohort with- particularly difficult to answer, more difficult than those of other epi- out accounting for other findings from the same cohort. A single demiologic disciplines. articlereportingasignificanteffectofadietarycomponentmayseem Acounterargumentmaybe,byanalogy,thatgenomelinkagescan plausible in isolation but would be untenable if all results were avail- publications have not been corrected, so why correct nutritional epi- able. Given the vast space of analyzable associations, some prolific demiology? The difference is that genomic scans performed with a cohorts (eg, European Prospective Investigation Into Cancer and handful of microsatellite markers have been replaced by better meth- Nutrition, Nurses’ Health Study) have yielded more than 1000 ar- ods and generally did not affect public policy and people’s lives. Con- ticles each. Nutritional epidemiology articles also attract attention versely,studies of nutritional epidemiology continue to be published because the public is very interested in (and perpetually misin- regularly,spuriously affect guidelines, and confuse the public through formedabout)nutrition.Forexample,oneofthe20highestAltmetric heated advocacy by experts and nonexperts. scores in 2017 was for a study reporting major survival benefits from Nutritional epidemiologists who espouse reform in past and coffee.8 Despite important limitations and shortcomings, such stud- future work should be rewarded, for example, with continued fund- ies also accrue substantial numbers of citations. ing to conduct pivotal trials, widely share their cohort data, con- Someadditional,large-scale,long-term,randomizedtrialsonnu- duct transparent all-encompassing analyses, and explore entirely tritionmaybeuseful,especiallyforassessingdietpatterns.3 Themost new avenues of nutrition research. Funding agencies should sup- promising large trial to date, Prevención con Dieta Mediterránea port the reform agenda and thereby rejuvenate the field of nutri- (PREDIMED), a trial of Mediterranean diet, had shown a benefit on tional research.

ARTICLE INFORMATION 2. Poole R, Kennedy OJ, Roderick P, et al. Coffee 7. Zhao LG, Zhang QL, Zheng JL, et al. Dietary, Published Online: August 23, 2018. consumption and health: umbrella review of circulating beta-carotene and risk of all-cause doi:10.1001/jama.2018.11025 meta-analyses of multiple health outcomes. BMJ. mortality: a meta-analysis from prospective studies. 2017;359:j5024. Sci Rep. 2016;6:26983. doi:10.1038/srep26983 Conflict of Interest Disclosures: The author has completed and submitted the ICMJE Form for 3. Trepanowski JF, Ioannidis JPA. Limiting 8. Gunter MJ, Murphy N, Cross AJ, et al. Coffee Disclosure of Potential Conflicts of Interest and dependence on non-randomized studies and drinking and mortality in 10 European countries. none were reported. improving randomized trials in human nutrition Ann Intern Med. 2017;167(4):236-247. research: why and how. Adv Nutr. 2018;9(4):367-377. Funding/Support: The Meta-Research Innovation 9. Estruch R, Ros E, Salas-Salvadó J, et al; Center at Stanford (METRICS) has been funded by 4. Ioannidis JP. Implausible results in human PREDIMED Study Investigators. Primary prevention the Laura and John Arnold Foundation. The work of nutrition research. BMJ. 2013;347:f6698. of cardiovascular disease with a Mediterranean diet Dr Ioannidis is funded by an unrestricted gift from 5. National Academies of Sciences, Engineering, supplemented with extra-virgin olive oil or nuts. Sue and Bob O’Donnell. and Medicine. Redesigning the Process for N Engl J Med. 2018;378(25):e34. Establishing the Dietary Guidelines for Americans. 10. Prasad V, Jorgenson J, Ioannidis JPA, Cifu A. REFERENCES Washington, DC: National Academies Press; 2017. Observational studies often make clinical practice 1. Schwingshackl L, Schwedhelm C, Hoffmann G, 6. Archer E, Hand GA, Blair SN. Validity of US recommendations: an empirical evaluation of et al. Food groups and risk of all-cause mortality: nutritional surveillance: National Health and authors’ attitudes. J Clin Epidemiol. 2013;66(4):361- a systematic review and meta-analysis of Nutrition Examination Survey caloric energy intake 366.e4. doi:10.1016/j.jclinepi.2012.11.005 prospective studies. Am J Clin Nutr. 2017;105(6): data, 1971-2010. PLoS One. 2013;8(10):e76632. 1462-1473.

E2 JAMA Published online August 23, 2018 (Reprinted) jama.com

Downloaded From: by Mexico | Access Provided by JAMA , Mark Tuttle on 08/23/2018