Soskolne et al. Environ Health (2021) 20:90 https://doi.org/10.1186/s12940-021-00771-6

COMMENTARY Open Access Toolkit for detecting misused epidemiological methods Colin L. Soskolne1* , Shira Kramer2 , Juan Pablo Ramos‑Bonilla3 , Daniele Mandrioli4 , Jennifer Sass5,6 , Michael Gochfeld7 , Carl F. Cranor8, Shailesh Advani9,10 and Lisa A. Bero11

Abstract Background: Critical knowledge of what we know about health and disease, risk factors, causation, prevention, and treatment, derives from epidemiology. Unfortunately, its methods and language can be misused and improperly applied. A repertoire of methods, techniques, arguments, and tactics are used by some people to manipulate science, usually in the service of powerful interests, and particularly those with a fnancial stake related to toxic agents. Such interests work to foment uncertainty, cast doubt, and mislead decision makers by seeding confusion about cause- and-efect relating to population health. We have compiled a toolkit of the methods used by those whose interests are not aligned with the public health sciences. Professional epidemiologists, as well as those who rely on their work, will thereby be more readily equipped to detect bias and faws resulting from fnancial confict-of-interest, improper study design, data collection, analysis, or interpretation, bringing greater clarity—not only to the advancement of knowledge, but, more immediately, to policy debates. Methods: The summary of techniques used to manipulate epidemiological fndings, compiled as part of the 2020 Position Statement of the International Network for Epidemiology in Policy (INEP) entitled Confict-of-Interest and Disclo- sure in Epidemiology, has been expanded and further elucidated in this commentary. Results: Some level of uncertainty is inherent in science. However, corrupted and incomplete literature contributes to confuse, foment further uncertainty, and cast doubt about the evidence under consideration. Confusion delays scientifc advancement and leads to the inability of policymakers to make changes that, if enacted, would—sup‑ ported by the body of valid evidence—protect, maintain, and improve public health. An accessible toolkit is provided that brings attention to the misuse of the methods of epidemiology. Its usefulness is as a compendium of what those trained in epidemiology, as well as those reviewing epidemiological studies, should identify methodologically when assessing the transparency and validity of any epidemiological inquiry, evaluation, or argument. The problems resulting from fnancial conficting interests and the misuse of scientifc methods, in conjunction with the strategies that can be used to safeguard public health against them, apply not only to epidemiologists, but also to other public health professionals. Conclusions: This novel toolkit is for use in protecting the public. It is provided to assist public health professionals as gatekeepers of their respective specialty and subspecialty disciplines whose mission includes protecting, maintaining, and improving the public’s health. It is intended to serve our roles as educators, reviewers, and researchers. Keywords: Disinformation, Ethics, Flawed science, Manufactured scientifc controversy, Obfuscation, Partiality, Public policy, Research integrity, Scientifc misconduct, Undue infuence

*Correspondence: [email protected] 1 School of Public Health, University of Alberta, Edmonton, AB, Canada Full list of author information is available at the end of the article

© The Author(s) 2021. Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creat​ iveco​ mmons.​ org/​ licen​ ses/​ by/4.​ 0/​ . The Creative Commons Public Domain Dedication waiver (http://creat​ iveco​ ​ mmons.org/​ publi​ cdoma​ in/​ zero/1.​ 0/​ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data. Soskolne et al. Environ Health (2021) 20:90 Page 2 of 16

Background public health disciplines, including risk assessment, toxi- Educated in the application of epidemiological meth- cology, and exposure assessment. ods, epidemiologists study where diseases occur, what causes them, and how to prevent them. According to A The role of undue infuence in increasing Dictionary of Epidemiology [1], the knowledge derived uncertainty from epidemiological inquiry is not used solely for dis- Policy decisions are infuenced by factors and inputs covery purposes. It is also applied to control and prevent related not only to health risk assessments based on epi- health problems and is used to restore, promote, and pro- demiological data; they are also infuenced by economic, tect population health across all levels of society. Hence, political, social values, and special interest stakeholder by virtue of their focus on protecting the public’s health, considerations [10]. When policies informed by epide- epidemiologists, as a profession, are expected to serve the miological evidence are debated in government, the pref- public, with the public interest trumping all others [2]. erence is to make policy decisions in the presence of the As an applied interventionist science, epidemiology is greatest possible certainty. However, absolute certainty used not only to study health problems, but also to pro- is not possible in science, given the inherent uncertainty vide evidence to inform rational policy debate among that accompanies scientifc inquiry. Consequently, epide- interested stakeholders [3]. Tis evidence provides the miologists are usually cautious and provide caveats for scientifc basis for correcting and, ideally, preventing their fndings. Tis creates an entry point for those bent health problems through government-driven health and on manipulating policy to promote confusion and engage social policy. Aside from informing policy, epidemio- in disinformation [11]. logical data also provide the basis for individuals’ choices Poorly or inappropriately designed and executed epide- about lifestyle, diet, and other critical factors that infu- miological research that makes its way into the scientifc ence health. Whether working as scholars, researchers, literature serves to increase uncertainty. Tis renders the public health, or non-government agency profession- policy maker less likely to vote in favor of a policy change als, as consultants, or even as expert witnesses in legal in support of public health. If the science can be muddied proceedings, the work-product and ultimate goal of the to foment uncertainty, or perhaps to mislead, a policy epidemiologist should be to promote and protect the could ensue that leads to even more adverse population public’s health, both at the population as well as the indi- health risks. vidual level. Aware of this, a well-developed strategy among those Yet, in a world of conficting interests, some parties with a vested self-interest in infuencing and undermin- may use the methods and language of epidemiology for ing policy, in a manner that is not consistent with the personal gain or for corporate proft. Tey do so by man- health of the public, is to fnd ways to increase scientifc ufacturing and casting doubt [4, 5] to confuse both poli- uncertainty, or to outright mislead. Science can be mis- cymakers and the public to the detriment of the public’s used, either intentionally, through error, or from bias. In health. Goldberg and Vandenberg [6] have most recently epidemiology, bias is defned as “an error in the concep- identifed commonly applied tactics used to misrepresent tion and design of a study—or in the collection, analysis, scientifc discovery: spinning the facts to manufacture interpretation, reporting, publication, or review of data— doubt, generating or perpetuating falsehoods. Tey point leading to results or conclusions that are systematically out that deceit can result in confusion that delays action (as opposed to randomly) diferent from truth [1] .” by calling into question the scientifc basis for concern. Financial confict-of-interest (COI), including author Documents presenting best practices and ethics guide- fnancial ties, review sponsorship, and journal funding, lines have been developed and adopted by the major introduces a bias at all levels of the research and publica- epidemiology professional organizations to support the tion process [12]. Contrary to what many scholars might discipline and protect its integrity [7–9]. Tese provide believe, this bias is not prevented by the peer review pro- the moral basis for epidemiology’s mission; they guide cess [12]. Distortion and disinformation practices regard- the normative practices of the discipline. While profes- ing scientifc methods and evidence were intentionally sionals who are not adherent to the guidelines can be employed by the lead industry in the early 1900s [13] and, called to account, there is no mechanism to ensure their since the 1950s, by the tobacco industry [14], and have implementation; moral suasion through peer pressure is since been honed by the asbestos industry [15–18]. Te the only enforcement mechanism. methods have become more sophisticated over time as In this commentary, our focus is on the discipline of played out from one industry to the next [19]. Te goal epidemiology. Te problems resulting from conficting is to pollute the scientifc literature with studies designed interests, and the strategies that can be used to protect to serve the interests of powerful sponsors and special public health from them, however, also apply to other interests. While scientists routinely disagree, the most Soskolne et al. Environ Health (2021) 20:90 Page 3 of 16

intransigent disagreements arise when fnancial [20, 21], and transparent. Its byline states: “Integrity, Equity, and as well as political [22] interests are at play. When people Evidence in Policies Impacting Health.” become ill, die prematurely, and the health of future gen- Tanks to investigative journalism, exposés of corpo- erations is placed in jeopardy, then transparency about rate and political infuence in the United States (U.S.) the stakes involved becomes even more pressing. in the period 2017–2021, reveal how the Environmen- Most recently, COVID-19 has taught us the impor- tal Protection Agency (EPA) under the Trump admin- tance of following epidemiological evidence in policy and istration, the American Chemistry Council (ACC), and health decision-making, especially in a global pandemic industry law frms colluded to weaken the EPA’s new [3]. It has starkly revealed science’s politicization, cor- chemical safety reviews [32]. Te exposés also reveal ruption, and suppression [22]. Indeed, the pandemic has how the fossil fuel industry has persisted over decades in exposed relationships that need to be confronted if prof- infuencing policy by obfuscating and denying negative teering is to be contained, calling on values that support impacts on human and planetary health [33–36]. Two the public interest rather than self-serving relationships seminal volumes, rich in well-established examples, were with industry [23]. Health harms are likely when the produced by the European Environment Agency [37, public is exposed to misinformation. Confusion ensues, 38]. Furthermore, the harmful impacts of powerful infu- which in turn creates a space for the mistrust of science, ence through research sponsorship have been recently and the amplifcation of conspiracy theories through recognized, such that research and related professional social media—resulting in aberrant behaviors that ham- sponsorship by Big Oil and Tobacco are being strongly per vital public health measures [24]. discouraged [39]. Recognizing the range of factors afecting the policy On June 10, 2013, a few years prior to the aforemen- process, and how they compete with one another, would tioned exposés, Dr. Margaret Chan, World Health Organ- help public health scientists appreciate the vulnerability ization (WHO) Director General, made the following of their discipline to being perverted for manipulating statement to the 8th Global Conference on Health Pro- science, misguiding policy development, and supporting motion, held in Helsinki, Finland: “… In the view of WHO, special interests. By following the money, one can iden- the formulation of health policies must be protected from tify the role that infuence has played, and how this has distortion by commercial or vested interests [40].” INEP encouraged the misuse of epidemiology [25]. Te con- began to develop its Position Statement in 2014, soon duct of invalid science for generating “evidence” involv- after the WHO Director General’s pronouncement. What ing researchers fnancially supported by special interests Dr. Chan noted indeed is an ongoing phenomenon. (e.g., [26–29]), is a common and worrisome practice. With INEP working at the interface of research and policy, its mission includes recognizing and highlight- Methods ing the misuse of data and potential corruption of the Forces having direct or indirect fnancial stakes in policy science practiced by epidemiologists. INEP comprises interventions, especially those with a short-term focus on 24 national and international volunteer member asso- reports refecting profts or personal gain to stakehold- ciations and societies of epidemiology across fve conti- ers, have been shown to be the most active in efectively nents. It is registered as a 501(c)(3) public charity in the working against the public’s health. Particularly, there has U.S. It is thus well positioned internationally to develop been a precipitous increase in the corporate funding of strategies to combat the misuse of epidemiological sci- epidemiological research and an ever-growing reliance ence. Te INEP Position Statement [31] addresses two of academic institutions on such sources of funding. Tis questions: has resulted in increasing instances of conficting inter- ests [30] which were brought to attention in the 2020 a) How is it that public health policy remains under International Network for Epidemiology in Policy (INEP) siege? Position Statement on Confict-of-Interest and Disclosure b) Could public health be better protected through the in Epidemiology [31]. improved management of Confict-of-Interest and INEP is the major global network of epidemiologists Disclosure in Epidemiology? with a focus on providing a bridge between epidemiolog- ical research and evidence-based, rational, government- To address these questions, the INEP Position State- formulated health policy that serves the public interest. ment [31] equips scientists with a set of tools to expose It thus provides a unique forum to protect and promote and root out so-called science that is designed to mislead public health, and works to ensure scientifc integrity, and deceive. Hopefully, the actions of those drawing from promote ethical conduct in research, and support evi- the methods exposed in the Toolkit Table 1 (presented dence-based research fndings that are both independent in the Results section below) to distort science should Soskolne et al. Environ Health (2021) 20:90 Page 4 of 16

become less infuential. Teir infuence will diminish epidemiologists on corporate funding, as well as the because reviewers of epidemiological studies, be they infuence of politics on public health research, further peer reviewers or otherwise, should, by virtue of the exacerbate harms resulting from misusing the methods of toolkit, be more efective in identifying invalid science the discipline and/or misinterpreting research fndings. introduced to delay policy actions for protecting public To counter the types of forces noted above, a listing health. of key methods/techniques, arguments, and tactics has Te toolkit’s role in the litigation process, from depo- been assembled in the Table 1. It is provided to help iden- sition to cross-examination in court proceedings, should tify how epidemiologists, usually fnancially supported also be helpful in both the pursuit of truth and for ensur- by or under the infuence of vested interests, manipulate, ing social justice. With the potential of the toolkit for misuse, or inappropriately apply the methods of epidemi- bolstering the integrity of the discipline, we recognize ology, or misinterpret fndings, to skew results and pro- that there are many journals with no or inefective peer duce invalid science. Te Table 1 is a toolkit that can be review; and, that industries have bought their own jour- used as follows: nals, limiting the extent to which the literature could be freed of corrupted science. Te once-revered peer-review • By peer reviewers as a checklist of, or guide to key process is at risk, especially in journals controlled by methodological parameters; vested interests. • To train epidemiologists and other healthcare pro- Consolidation of the toolkit was thus included in the fessionals on the ways in which epidemiology can be INEP Position Statement [31]. It is now made accessible distorted; as a standalone and expanded commentary. Te com- • To review the literature for invalid science or unin- pendium of tools provided in this commentary brings formative studies (e.g., underpowered studies, or together work initially identifed by Cranor [41, 42], and misleading samples); and subsequently expanded upon by Soskolne [43–45] who • To identify who is misusing epidemiology. saw the importance of expanding and consolidating this work to better arm epidemiologists, policymakers, and Te benchmark against which the toolkit can be com- the scientifc community with a greater appreciation for pared is assembled from a selection of 12 foundational how epidemiological methods can be misused, abused, epidemiological textbooks, developed since the 1970s, and perverted, counter to the advancement of knowledge with more recent editions cited here and used in epi- and the public’s health. demiology training programs [50–61]. Tis selection Tis commentary has a role to play in not only bring- is somewhat arbitrary; any well-established textbook ing attention to, but also shining a light on, mechanisms should sufce to gain understanding about the correct of demonstrated infuence and their harmful impacts on use of epidemiological methods. the advancement of science and the protection of pub- Biostatistical methods are relied on for the design lic health. It therefore should be used as a teaching and of specifc epidemiological studies. As such, statisti- training resource in graduate programs in epidemiology cal methods are a critical component of the epidemi- and other related public health disciplines. Every student ologist’s toolkit. Statistics is a discipline that has been in emerging from any such program should be prepared to play for a longer period than what epidemiology has. It is confront the world of malfeasance. Ideally, reviewers of not surprising, therefore, to fnd in the statistical litera- manuscripts will be better positioned to separate public ture articles extending over a longer timespan that bring interest science from inappropriately designed studies attention to statistical mistakes that both researchers and that infltrate the literature and hence the policy debate practitioners can make in their work [62, 63]. specifcally to mislead science in the service of special Inappropriate techniques applied in epidemiology, interests. including those that manipulate fndings in ways that bias them toward the null, are assembled in the Table 1. Results Tese techniques may apply to the full realm of epide- As noted above, application of the epidemiological miologic inquiry, including descriptive and analytical method can be infuenced by interests that manipu- study designs. Tey include the use of unbalanced discus- late it in ways to produce fndings that cast doubt, sion that emphasizes fndings not supported by the data, foment uncertainty, and seek to mislead decision mak- selective disclosure of competing interests, and publica- ers. Unfortunately, some epidemiologists are suscepti- tion in ‘pay-to-play’ journals without appropriate peer ble to incentives that induce unprofessional conduct, review, and with issues involving undisclosed conficting thereby undermining the integrity of science [46–49]. interests. Te increasing reliance of public health institutions and Soskolne et al. Environ Health (2021) 20:90 Page 5 of 16 Reference(s) [ 64 , 65 ] [ 50 , 62 66 67 ] [ 62 ] [ 68 ] [ 69 – 71 ] [ 72 ] [ 73 ] ‑ ‑ ‑ 1) > years of exposure, which of exposure, years 2 is a “null” result; concluding that lack of result; “null” 2 is a < 1 is “null” if it is not statistically signifcant at if it is not statistically signifcant “null” 1 is > - II error; and sta ‑ highly dependent upon sample size term exposure among a large number of young workers number of young in among a large - term exposure powered studies can arise through underfunding, studies can arise in a through which results - powered no use. In such circumstances, what is actually In being undertaken- no use. such circumstances, is a tistical power; common strategy for dismissing study results that are indeterminate indeterminate that are dismissing study results for tistical power; common strategy an arbitrary risk or yield elevated ratios but do not reach power, because of low of statistical signifcance level be detected by a statistical study; asserting that a “negative” study (even if RR study (even “negative” a statistical study; by be detected asserting that a is proof of no efect. This can be addressed transparently by providing a power a power providing by transparently can be addressed This of no efect. is proof calculation an appearance of concern. give not know a question, but want to to the answer under Thus, detect a to the statistical power have be able to shortfall needed to in resources In can in truth, exists. addition, if not all needed information when one, diference the study will be of confounding, for control properly instance, for to, be gathered - to limited number any For not one that is capable of demonstrating an efect. study, “token” Token undertaking fnd themselves may epidemiologists of reasons, such a study. be dem ‑ will be able to studies can serve (a) no efect in two ways: special interests thus ensuring that the status quo is maintained; and (b) special inter onstrated, has the This of scientists is exploring that a team a concern.est will be armed say to a health concern when, be doing social good in addressing of being seen to efect in fact, it will be a null study an elevated risk ratio is proof of no elevated risk (i.e., proof of the null hypothesis) proof risk of no elevated risk (i.e., an elevated ratio is proof the 0.05 level; concluding that a riskthe 0.05 level; ratio so that adverse efects can be identifed (i.e., incomplete accrual problem) incomplete can be identifed (i.e., efects so that adverse between in utero exposure to diethylstilbestrol (DES) and appearance of cervical diethylstilbestrol to exposure between in utero and appear asbestos or the latency to between exposure cancer of about 20 years, 45 years ance of cancers up to often represents shortoften represents short and latency too whom duration of exposure are observe to an efect ifestation (case–controlifestation studies). It the exposed into also includes placing controls studies example, For and exposed subjects/participants groups. control into group, which includes unexposed ofce workers. And as exposed, use all employees may as unexposed when this includes work line) residents then they use nearby (fence line exposed residents ers and fence Efects Increases the probability of Type Increases of the probability Sample size too small to detect an adverse efect, or adverse efect is too rare to to rare is too efect or adverse efect, detect an adverse small to too Sample size Token studies are undertaken studies are tactic as a delay because decision makers rather would Token Concluding that a study with riskConcluding ratio Not measuring appropriate endpoints through the pathogenesis of a disease process the pathogenesis of a disease process endpoints through Not measuring appropriate Not allowing sufcient time in a study for disease to manifest as with the latency manifest disease to sufcient time in a study for Not allowing Analyses based on a seemingly large number of person - Analyses based on a seemingly large Control groups that include exposed persons (cohort groups studies) and early disease man ‑ Control - II errors years of exposure, seen of exposure, years powered studies; ignoring Type studies; ignoring - powered of probability as a strictof probability of statisti ‑ decision criterion determine the interpretation to conclusions and drawing cal results especially in occupational health studies Toolkit of inappropriate applications of the epidemiological method applications of the epidemiological of inappropriate Toolkit Method/technique Relying on statistical hypothesis testing; Using “statistical signifcance” at the 0.05 level at the 0.05 level signifcance” “statistical Using testing; Relying on statistical hypothesis Conducting statistically under Interpreting the statistical analysis or results inappropriately (see B8 below) inappropriately Interpreting the statistical analysis or results - up methods follow use adequate to Failing - up time follow adequate for allow to Failing Introducing inappropriate representation of total person - of total representation Introducing inappropriate Contaminating controls Contaminating science] 1 Table --- Part A --- uncertainty and cast doubt about cause-and-efect foment used to methods/techniques invalid Epidemiology-specifc producing and measurements biased study designs [through Item # A1 A2 A3 A4 A5 A6 A7 Soskolne et al. Environ Health (2021) 20:90 Page 6 of 16 [ 74 ] [ 75 ] [ 76 ] [ 77 , 78 ] [ 79 ] [ 80 , 81 ] [ 71 , 82 ] ‑ ‑ never’ comparison be may - never’ modifying variables, may lead to biased or lead to may - modifying variables, related risks combining individuals with by - related established healthy worker efect. Thus, a general Thus, worker established healthy efect. never’ categorization to hide efects, when, in fact, it is the “peak “peak when, in fact, it is the hide efects, categorization to - never’ a broad range of exposure characteristics statistical range of exposure or histories without proper a broad adjustment or stratifcation sentative of the population from which the “exposed” group, or the case group, or the case group, group, “exposed” which the of the population from sentative A good example is one comparing exposed workers in an industrialemerged. the general population. Exposed workers setting to in occupational studies are not repre are They perform able to and physically generally young heavy jobs. general population. Such comparisons diverse biologically of the more sentative - the well bias due to from sufer control A proper in an occupational study. is not appropriate population control the same industry from be comprised of unexposed employees would with group characteristicssimilar demographic more appropriate) more leukemia in for formaldehyde example, the cancers (for workers that have exposed” the NCI studies) which the signal of efect will be demonstrated (that is akin in which will be demonstrated of efect toxicology which the signal to demonstrating efects) is needed for the correct strain of rodent ‘ever an (whereas statistical signifcance reach fails to risk of the whole, in which all components may not only interact to cause efects, cause efects, not only interact to in which all components may risk of the whole, but also work synergistically; indi ‑ assess the risk to of pesticide active ingredients ‑ contain multiple active pesticide products ingredi commercial whereas vidually, species the target to enhance toxicity ents plus adjuvants to are complex mixtures. Analysis and representation of risk with one associated Analysis and representation complex mixtures. are chemical or agent without taking lead of the mixture may account the efects into distortedto risk and erroneous estimates reduced (high specifcity), but false negatives will be increased (low sensitivity). (low (high specifcity), will be increased but false negatives reduced conditions, Laboratory storage include timing of sampling, methods (which may sensitivity adequate of or levels and/or instrumentation that do not have etc.) use of vary or efects; levels exposure ‑ underestimate detection will systematically laboratory methods diferent of detectioning levels in analyzing blood samples by adjust for confounding and/or efect confounding adjust for case–control Examples also include analyzing matched results. inaccurate designs the matching using methods that do not retain importantis it shift null the always will these of to some that note to However, tives. etc.) common cancer endpoints, small sample sizes, (like from Potential dilution of efect of exposure dilution of efect Potential Invalidates comparisons between study groups. A control group should be repre group A control comparisonsInvalidates between study groups. Also, using an ‘ever using an Also, Combining all risk groups when it is in only a relatively small susceptible group in small susceptible group when it is in only a relatively all riskCombining groups that each one groups exposure so many dividing the exposed population into Also, To study only a portion of a mixture to which people are exposed so as to dilute the study only a portion dilute exposed so as to which people are of a mixture to To Virtually all exposures—be they ambient, occupational, or through other vectors— or through Virtually they ambient, occupational, all exposures—be If the criteria for a positive test or detection are stringent, then false positives will be stringent, or detection then false positives are test If a positive the criteria for Failure to utilize appropriate statistical analytical techniques, as well as failure to to as failure as well statistical analytical techniques, appropriate utilize to Failure or false posi ‑ in false negatives can shift resulting in either direction This the outcome among the exposed group (cohortamong the exposed group studies) should be representative of the population from which the exposed group (cohort which the exposed group of the population from should be representative studies) or the cases (case–control studies) emerged instrumentation (continued) Failing to statistically analyze or account for a broad range of exposure characteristics range of exposure a broad or account for statistically analyze to Failing Selecting inappropriate controls; failing to adhere to the requirement that controls that controls the requirement to adhere failing to Selecting controls; inappropriate Diluting / washing out / averaging efects in descriptive population comparisons in descriptive efects Diluting / washing out averaging Ignoring known synergies among components of the mixture knownIgnoring of chemicals synergies Failing to account for the efects of exposure to complex mixtures in risk to assessments of exposure the efects account for to Failing Using inadequate or insensitive laboratory or practices, or insensitive measurement Using inadequate methods, Inappropriate analytical methods used in the statistical analysis 1 Table A8 A9 A10 A11 A12 A13 A14 Soskolne et al. Environ Health (2021) 20:90 Page 7 of 16 [ 1 , 20 83 84 ] [ 85 ] [ 79 ] [ 51 , 86 ] Reference(s) [ 62 , 68 87 ] [ 10 , 11 ] [ 88 , 89 ] [ 82 , 90 ] [ 91 , 92 ] ‑ report or publish the fndings. Deliberate omission of fndings or inappropriate report omission of fndings or inappropriate Deliberate or publish the fndings. from events their impact. hide or dilute Omission of rare to of outcomes groupings peak exposures outliers could include removing or removing statistical analysis, be found to all the cancers are where vide both a context for the variables included in the quantitative analysis as well as analysis as well the variables included in the quantitative vide both a context for fndings of the quantitative the interpretation a context for studies that should have been included studies that should have with ethylene oxide reduces the risk estimates reduces oxide with ethylene Hill, have been erroneously interpreted as required criteria, as required interpreted been erroneously Hill, have including Bradford of evidence that should properly the dismissal of weight leading to thereby outright Despite in the Brad ‑ errors policies. in health - protective be considered about his proposed caveats expressed and his own Hill guidelines, suggested ford regulatory in legal by Hill still cited agencies, guidelines are the Bradford guidelines, as a requirement professionals and healthcare epidemiologists and by proceedings, causation for design or analysis, or selective interpretation of the fndings or selective interpretation or analysis, design increased risk so they can be studied rather than only studying the whole popula ‑ increased with a very RR associated elevated on demonstrating a consistently tion; insistence mecha ‑ classifcations that are broader risk elevated for showing vs. outcome rare virtue by activitynistically related of similar biological on structurally related compounds (e.g., the perfuorinated compounds (e.g., alkylateon structurally related substances [PFASs]) Suppression bias from: Suppression to or failure key in the statistical analysis, fndings in subgroups, include, to Failure The exclusive reliance on quantitative methods when qualitative research can pro research methods when qualitative on quantitative reliance exclusive The analysis includes studies with diferent study designs, or it selectively excludes or it selectively excludes study designs, Meta - analysis includes studies with diferent For example, using mortality example, cancer risk associated of morbidity breast instead For for Efects Guidelines in the form of “viewpoints” or “aspects” profered for interpreting causation, interpreting for profered “aspects” or “viewpoints” of Guidelines in the form The absence of objectivityThe / impartiality in the application of a biased resulting Ignoring or dismissing information pertaining or dismissing information Ignoring susceptible populations having to Failure to synthesize knowledge from all disciplines that relate to a disease process to knowledge all disciplines that relate synthesize to from Failure Insisting on the need for more research by ignoring prior ignoring by knowledge or information research more Insisting on the need for of - - driven funders, political infuences, or vested or vested political infuences, driven funders, interest in the presence of a fnancial confict in the presence of - interest - evidence summary result of - - weight rate Hill of evidence viewpoints in a or aspects) the weight Bradford in interpreting causation infer causation analysis to interest goals (see C6 below) interest interest, fnancial control of agenda - fnancial control interest, potential human health hazard potential hazards (continued) Suppressing data Suppressing Failing to recognize the validity of evidence from qualitative methods the validity qualitative of evidence from recognize to Failing analyses and reporting them as representing a or biased meta - analyses and reporting erroneous them as representing Producing Using mortality of morbidity a cancer endpoint with high survival instead data for Argument Insisting on the erroneous application of “criteria” for causation proposals (e.g., (e.g., causation proposals for “criteria” Insisting application of on the erroneous Failing to disclose a confict to Failing Ignoring mechanistic information suggestive of adverse efects of adverse suggestive mechanistic information Ignoring Exaggerating diferences, or dismissing them, when toxicological studies suggest a or dismissing them, when toxicological Exaggerating diferences, Ignoring related or families of molecular structures that predict potential health potential that predict or families of molecular structures related Ignoring 1 Table A15 A16 A17 A18 --- Part B --- and methods of suppression] standards [imposing inappropriate divisions among scientists action, the status quo , and create delay maintain used to Arguments Item # B1 B2 B3 B4 B5 Soskolne et al. Environ Health (2021) 20:90 Page 8 of 16 [ 75 , 83 ] [ 13 , 14 18 ] [ 93 – 95 ] Reference(s) [ 7 , 64 84 96 97 ] ‑ populations and failure to recognize heightened heightened recognize to - populations and failure populations or about rare conditions indicates no risk conditions indicates of - populations or about rare compromised persons; erroneously assuming that a lack of data persons; erroneously compromised - susceptibilities; e.g., of the developing brain to neurotoxicants, or from heightened heightened or from neurotoxicants, brain to of the developing susceptibilities; e.g., risks immune to in the literature about sub in the literature with exposure disease associated regulatory but not by scientists, by as recognized development, and early life fetal agencies ments, although risk of a health hazard may vary due to diferential susceptibility vary may although risk of a health hazard ments, diferential due to In an individual. U.S. doubt for not be discernable a reasonable and may beyond or of the evidence” “preponderance tort is of proof litigation, the typical standard than not.” probable “more a determination of that require “balance of probabilities” level and public health scientists endorse a lower health advocates Environmental of proof. industry higher standards whereas evidence, for of probabilistic argues public err protect on the side of caution to groups In public interest summary, of proof an unattainable standard for polluting industries press health, whereas application of science and the appropriate Thus, that science cannot often meet. of evidence support public health policies weight as to not used in judgments are or litigation outcomes in a large population—as in pesticide exposure and autism—leading to a larger a larger and autism—leading to population—as in pesticide exposure in a large the public health importance recognize demand fails to This public health impact. while risk may estimates exposures; prevalent of population attributable risk for In the example people can be large. number of afected the absolute be low, get an OR higher than about 3 because of all the practical it is difcult to provided, (seasonal) intermittent issues of conducting a study such as multiple exposures, workers undocumented not wanting due to exposures undocumented exposures, report,to etc equate to “no risk.” However, the absence of data (because of the failure to conduct to the absence of data (because failure However, risk.” “no to equate absence of The as evidence of no risk. or misinterpreted studies) is often invoked does not research, including the absence of epidemiological scientifc research, data can be sufcient evidence to Mechanistic and toxicological risk.” “no to equate human riskindicate Failure to protect vulnerable sub protect to Failure Neurotoxic chemicals tend to be more harmful when exposures take harmful place during when exposures be more to chemicals tend Neurotoxic Demanding proof “beyond a reasonable doubt,” typical require of criminal proof law doubt,” “beyond a reasonable Demanding proof The odds ratio for a population may be 1.5 and represent millions of people at risk be 1.5 and represent a population may odds ratio for The Efects Lack of research about a public health issue—andLack of research a paucity of data—does not of identifying and protecting from adverse health impacts adverse the most vulnerable, from of identifying and protecting and including children and genetically susceptible in society, chemically sensitive, women pregnant ‑ risks; of elevated rejec “proof” needed for data are claims that more be addressed; Principle tion of the Precautionary disease must be 2 or greater before the study can be admitted to support to the study can be admitted expert before disease must be 2 or greater [See A3 above] testimony (continued) Focusing on studying and reporting the detriment to only general population efects Focusing Demanding an unusually high degree of certainty to Demanding an unusually high degree the public health problems for Demanding that any observedDemanding and risk that any between exposure odds ratio / relative Tactic Assuming that “no data” equates to “no risk”“no to equates data” “no that Assuming 1 Table B6 B7 B8 --- Part C --- special interests] by the positions taken from misdirect [imposing undisclosed values policy to infuence priorities through invoked Tactics Item # C1 Soskolne et al. Environ Health (2021) 20:90 Page 9 of 16 [ 1 , 11 15 84 98 ] [ 14 , 15 ] [ 15 , 35 36 ] [ 7 , 8 28 ] [ 99 , 100 ] [ 101 ] ‑ This is importantThis for includes a number of examples: for review The a policy. to - base relevant and others have not been properly addressed due to the improper infuence of the improper due to addressed not been properly and others have Bias arises Repression in situations which a line of inquiryspecial interests. is not that pur aware consciously or subconsciously, is, pursued because the researcher or the upset the dominant culture/paradigm, question would suing such a research because funding be investigated never question may research The funding agency. In practice, students the funding agency its study. from is not made available for such questions if the funding support from com ‑ away needed to be directed may In the cannot be secured. program component of their graduate the research plete no actionabsence of new‑ information, is demanded of those bearing responsibil the ofend that which may persisting in researching Students and researchers ity. respectively or unemployed, unemployable establishment could fnd themselves to operate in much the same way in a similarly exposed population in a diferent in a similarly in much the same way exposed population in a diferent operate to location morbidity or mortality action is taken of the although the weight before rates) evidence exposure supports from efects risk of adverse excess exposure and ensure a voting majority a voting support to and ensure a particular interest vested stakeholder’s with that of the public interest that is not consistent industry’s harm position OR detracts of their product. from because it is not about infuencing individual studies, but rather the entire but rather the entire because it is not about infuencing individual studies, evidence industry tobacco on contaminants of indoor air OTHER instance, funding research industry beverage on sweetened than second - hand smoke; and, funding research rather than on the harms of sugar the benefts of exercise Critical public health threats, including climate change, frearm violence, obesity/diet, violence, frearm change, including climate Critical public health threats, Refusing to accept that health efects observed accept that health efects Refusing to likely in one exposed population are Insistence on occurrence of dire public health impacts (e.g., signifcantly increased increased signifcantly public health impactsInsistence (e.g., of dire on occurrence Failing to discern to acceptable risks as a policy the actual determination risk vs. of Failing By gaining a presence, the ability infuence decisions to of impartialBy gaining a presence, representatives Infuencing the research agenda by funding research that supports funding research a stakeholder agenda by Infuencing the research interests, or infuence of special interest groups resulting in a Repression Bias. Bias. in a Repression resulting groups or infuence of special interest interests, done because never should not lose sight of the fact that some studies are We is Sometimes the reason not granted. some reason, for them was, for approval is repressed because the topic lations of exposed people despite demonstrated efects in humans elsewhere efects demonstrated lations of exposed people despite to justify interventions to reduce/eliminate exposures justify interventionsto reduce/eliminate to sions about selecting appropriate standards of evidence to draw policy draw of evidence to sions about selecting- relevant standards appropriate and values) dominant interests in suppressing conclusions (i.e., kinds (see B2 above) (continued) Failing to study a critical public health issue because of political infuence, fnancial study a critical to public health issue because of political infuence, Failing Failing to generalize health risks, and restricting the assignment of risk local popu ‑ and restricting health risks, to the assignment generalize to Failing Neglecting to apply or dismissing the Precautionary Principle when there is evidence when there Neglecting Principle apply or dismissing the Precautionary to Failing to be transparent in making be transparent that underlie to deci ‑ explicit those value judgments Failing Infltrating editorial boards, scientifc review panels, and decision - making panels, scientifc review Infltrating bodies of all editorial boards, Misdirecting policy priorities infuence through 1 Table C2 C3 C4 C5 C6 C7 Soskolne et al. Environ Health (2021) 20:90 Page 10 of 16

In brief, the above Table 1, constituting the toolkit, is be able to detect them in evaluating studies. However, organized in three parts: “generally not included in current risk of bias tools is Part A of the Table 1 refects on how the fndings from potential bias because of problems in statistical methods. epidemiological inquiry are afected by the design of Concerns include choice of an inappropriate and badly studies, as well as on the how and what is being meas- ftting model, failure to model exposure–response or to ured. We have compiled epidemiology-specifc methods/ evaluate diferent exposure–response models, incorrect techniques used to foment uncertainty and cast doubt use of mixed models, incorrect use of Bayesian tech- about cause-and-efect through biased study designs and niques, violation of statistical assumptions (e.g., normal measurements producing invalid science. residuals in linear regression), overadjustment for covari- Part B of the Table 1 reveals arguments that impose ates related to exposure but not to outcome, adjusting for inappropriate standards and methods of suppression causal intermediates, etc.” counter to the principle of openness and transparency. We note that statistical models and methods are We have compiled arguments used to delay action, main- quite complex. As such, many epidemiologists and peer tain the status quo, and create divisions among scientists reviewers, as well as the general reader, may not be able by imposing inappropriate standards and methods of to evaluate their appropriateness. Yet, bias due to COI suppression. has been increasingly considered and assessed in system- Part C of the Table 1 identifes tactics imposed by those atic review methodologies and RoB tools of epidemio- serving special interests to upset the very foundation logical studies, including the Navigation Guide [103], and of reason as it pertains to the core values and methods the WHO/ILO Joint Estimates of the Work-related Bur- of the discipline. We have compiled tactics invoked to den of Disease and Injury [104]. misdirect policy priorities through infuence imposing Another domain of evidence synthesis that does not undisclosed values from the positions taken by special entail bias per se is “informativeness.” Consideration interests. in this domain includes whether the study has a large enough sample size, whether the study has sufcient Discussion latency, whether results have been reported selectively, Since the compilation of this toolkit, the literature has, and whether the study has sufcient exposure contrast over the past year, seen many more examples of confict- to see an efect of exposure on outcomes. Tis domain is ing interests and failures to disclose them. Each example sometimes called sensitivity in some evidence syntheses. exposes the inappropriate role of infuence-wielding at all Tere is considerable overlap between the strategies levels of scientifc inquiry and knowledge advancement. identifed in the toolkit of Goldberg and Vandenberg [6] In this commentary, we focus on the toolkit aspect of and those independently identifed in our Table 1 (above). the INEP Position Statement [31], and thus limit our- Tis lends credence to our respective approaches for selves in this discussion to one recent contribution to the addressing the challenge of manufactured doubt. It adds topic of bias assessment because of its focus on methods. a degree of validation to each of our respective Tables It appears in a 2020 commentary by Steenland et al. [102] revealing strategies, arguments, and tactics used in doubt in which they consider risk of bias (RoB) assessments mongering. In the clinical realm, regarding disclosure as and evidence syntheses for observational epidemiologi- a mechanism for mitigating the efects of COI, Rimmer cal studies of environmental and occupational exposures. [105] notes that, until the introduction of a mandatory RoB tools are used to evaluate epidemiological studies as register of doctors’ interests, patients would have no idea part of evidence synthesis, the latter requiring a broader who was funding their doctor’s voice, or who might be approach than simply evaluating RoB in individual stud- biased towards certain treatments. Related health profes- ies. Tose authors recognize the need to include classical sional bodies are thus calling out the biases to health and considerations for judging causality in human studies, “as science induced by commercial interests. well as triangulation and integration of animal and mech- In practice, broad opportunity exists to publish inva- anistic data.” lid science owing to: (1) the existence of predatory pay- As with the INEP Position Statement [31], Steenland to-play journals; (2) open access journals with little peer et al. [102] recognize confict-of-interest, which can cre- review; and (3) editors/peer reviewers who themselves ate the potential for bias, a bias that is not always assessed have a COI and/or little-to-no knowledge of the topic in RoB tools. Tey point to strong evidence that “stud- under review. Given this, those who rely on the pub- ies authored by those with vested interest are generally lished literature, in both government and among the pub- favorable to those interests, hence the need to disclose lic, including the media, should be aware that research potential confict of interests.” In the view of Steenland strategies exist that can be misleading. Above all, since et al. [102], if specifc biases are present, reviewers should professional epidemiologists are the gatekeepers of the Soskolne et al. Environ Health (2021) 20:90 Page 11 of 16

discipline, they have the moral responsibility to execute • Recognizing erroneous and misleading interpreta- its mission. It therefore behooves them, along with other tions of underpowered studies healthcare professionals, to be familiar with this toolkit • Acknowledging the scientifc assessment of uncer- as but one mechanism for better ensuring the mainte- tainty nance of professional standards of integrity [43] through- out the public health sciences. ◦ Bias; statistical (aleatoric) uncertainty; epistemic uncertainty Recommendations ◦ Model uncertainty; parameter uncertainty Common practices to distort and misapply epidemiologi- ◦ Expected value = (value of outcome) X (prob- cal science should be recognized and called out profes- ability of outcome) sionally when they occur. INEP member organizations, ◦ Uncertainty intervals academic institutions, and other public health profes- sionals can adopt INEP recommendations and strategies • Highlighting when the logic of an argument is inva- for COI management that include identifcation, avoid- lid ance, disclosure, and recusal [31]. It would be of added beneft to incorporate this commentary into the curricu- ◦ False premises lum of graduate training programs in the health sciences ◦ Invalid argument and in medical schools to equip entry-level professionals ◦ Misapply conclusions to better serve as gatekeepers of the discipline. Te toolkit can be used as a guide in what to look for, • Exposing the motives of researchers, journal editors, to train epidemiologists and others on how epidemiol- peer reviewers, decision makers and other stakehold- ogy can be distorted, to evaluate the literature for inva- ers in the policy process lid science or uninformative studies (e.g., underpowered • Critically appraising the evidence as presented studies), and to identify who it is that is misusing epide- • Publishing standards for good practice, e.g., the INEP miology along with their motivations. It can be used as Position Statement a checklist for critically appraising descriptive or analyti- • Calling out malpractice. cal studies pre- and post-publication, policies, and argu- ments in legal proceedings. While the Council on Publication Ethics (COPE) has In summary, techniques to manufacture and cast doubt guidelines designed to keep the literature free of cor- (i.e., irrational skepticism), targeted at policymakers and rupted or poor science, they are known to be inad- consumers through the misapplication of the epidemio- equately enforced and are insufcient to stop the logical method, claim that: manipulation of the literature [16, 106, 107]. Actions on the part of the epidemiology community, as well as the • Te science is unclear broader health sciences, could help to change this as the • Tere is dissent (where the evidence is clear) problems are recognized and addressed. Te scientifc • Te data are inconclusive community should engage by recognizing and profes- • Scientists are biased—“You can’t trust scientists” sionally calling out common practices used to distort • Regulation is unjustifed—“It’s a slippery slope.” and misapply epidemiological and other health-related sciences. Tis is achieved through: To demonstrate the seriousness of serving as gatekeep- ers with the moral responsibility to uphold professional • Delaying action standards, epidemiologists could expand upon the INEP • Infuencing policy decisions—risk factors for bias Position Statement, using it as a launching pad to write other documents (e.g., other position statements, policy briefs, commentaries, letters, case studies, and editorials) ◦ Pulls: Vested interest (stand to gain personally) to extend the reach of INEP’s Position Statement. Ulti- ◦ Pushes: Lobbying. mately, exposing the public and policymakers to the INEP Defenses that work against epidemiology being misap- Position Statement will provide reassurance about the plied include: seriousness that professionals hold in protecting the pub- lic’s health. It is possible that, in return, with enhanced • Correctly applying and clarifying the methods of sta- credibility in the profession, funding could be made avail- tistical inference able to support organizations like INEP as valued coun- • Exposing undisclosed COI terweights to the manipulation of this key public health Soskolne et al. Environ Health (2021) 20:90 Page 12 of 16

science whose mission it is to serve the public interest the literature, a domain on which the advancement of sci- above any other. ence and public policy rely. Epidemiologists and other health professionals must not be naïve. Tey need to remain vigilant to the variety Abbreviations of forces at play that infuence both science and policy. ACC​: American Chemistry Council; INEP: International Network for Epidemiol‑ In addition to vigilance, personal integrity is required ogy in Policy; COI: Confict-of-Interest/conficting interests; COPE: Council on Publication Ethics; EPA: United States’ Environmental Protection Agency; HREB: to counter the infuence of economically powerful enti- Human Research Ethics Board; IARC​: International Agency for Research on ties and corrupt and/or morally bankrupt governments Cancer; IRB: Institutional Review Board; OR: Odds Ratio; RoB: Risk of bias; RR: whose focus is not on protecting public health, but rather Relative Risk; U.S.: United States; WHO: World Health Organization. on protecting narrow, special interests. Acknowledgements Tis said, there are frailties in both human beings as This commentary draws heavily on the INEP Position Statement Confict- well as in governmental structures. Sensitive to this real- of-Interest and Disclosure in Epidemiology, approved by the INEP Board on 16 September 2020, and which exceeded its endorsement threshold for ity, we provide specifc short-term objectives that each public release by 24 December 2020 [31]. Mark J.J. McCormack reviewed the epidemiologist could immediately implement: Recognize manuscript; he advanced some concepts and the idea of developing an app. our professional obligation to be vigilant and especially Michael Power made the connection, through clinical epidemiology, with critical appraisal methods in evidence-based medicine and provided the careful in peer review to avoid contaminating the litera- summary/checklist of the Toolkit Table. Dany Gagnon provided technical and ture with invalid or poor science; and, support added editing support throughout. Lastly, independent constructive reviews, as well oversight, as in Human Research Ethics Boards (HREBs) as editor-suggested improvements and changes, helped to both refne and focus the manuscript. or Institutional Review Boards (IRBs), on the need to keep ourselves on track with the moral responsibility for Authors’ contributions being aware of and compliant with our profession’s ethics Under the leadership of CLS, all authors contributed to the compilation of the International Network for Epidemiology in Policy (INEP) Position Statement guidelines. on Confict-of-Interest and Disclosure in Epidemiology (see https://​epide​miolo​ We recommend accepting that uncertainty is inher- gyinp​olicy.​org/​coi-d-​posit​ion-​state​ment). Thereafter, those included here ent in science. In our role as scientists, we strive to be have expanded in this commentary on the “Summary of techniques used to manipulate epidemiological fndings” appearing on pages 34–37 of the INEP value-neutral or value-free, but the human instrument is, Position Statement at the above link. The author(s) read and approved the in fact, incapable of achieving this point of neutrality or fnal manuscript. impartiality. Consequently, we need to look frst to our- Funding selves, because causal inference is a function of who it is Not applicable. that is making the inference which, in turn, is a function of how we apply our scientifc methods. Anything that we Availability of data and materials DOIs and hyperlinks are included throughout the literature cited. can do to build protections into the system of self-gov- ernance that is expected of professions like epidemiology, Declarations we ought to engage with and embrace. Ethics approval and consent to participate Conclusions Not applicable. Tis novel toolkit exposes the negative impacts of the Consent for publication misuses of epidemiology. As such, it provides an essen- Not applicable. tial foundation for expanding the science and methods of Competing interests argumentation (i.e., disagreement) through formal logic The authors declare that they have no competing interests. and dialectics. While beyond the scope of this commen- Author details tary, the challenge posed to develop an application (i.e., 1 School of Public Health, University of Alberta, Edmonton, AB, Canada. an app) based on the Table 1—to more efciently review 2 Epidemiology International, Hunt Valley, MD, USA. 3 Department of Civil the literature and for rooting out invalid science and mis- and Environmental Engineering, Universidad de Los Andes, Bogotá, Colombia. 4 Cesare Maltoni Cancer Research Centre, Ramazzini Institute, Bologna, Italy. leading conclusions—warrants further exploration in this 5 Natural Resources Defense Council, Washington, DC, USA. 6 George Wash‑ philosophical context. ington University, Washington, DC, USA. 7 Environmental and Occupational Te toolkit, consistent with INEP’s mission, is made Health Sciences Institute, Rutgers Biomedical and Health Sciences, Newark, NJ, USA. 8 Departments of Philosophy and Environmental Toxicology, University available to protect the public. It is provided to assist of California, Riverside, CA, USA. 9 Terasaki Institute of Biomedical Innovation, public health professionals whose mission includes pro- Los Angeles, CA, USA. 10 Georgetown University School of Medicine, Washing‑ 11 tecting, maintaining, and improving the public’s health. ton, DC, USA. Center for Bioethics and Humanities, University of Colorado Anschutz Medical Campus, Aurora, CO, USA. Its utility lies in our more specifc roles as educators, reviewers, and researchers. It is to be used to detect and Received: 22 March 2021 Accepted: 9 July 2021 professionally expose the misuse and distortions of epide- miology that result in misinformation that contaminates Soskolne et al. Environ Health (2021) 20:90 Page 13 of 16

References 21. Grundy Q, Mayes C, Holloway K, Mazzarello S, Thombs BD, Bero L. 1. Porta M, editor. A dictionary of epidemiology. 6th ed. Oxford: Oxford Confict of interest as ethical shorthand: understanding the range and University Press; 2014. nature of ‘“non-fnancial confict of interest”’ in biomedicine. J Clinical 2. Olsen J, Bertollini R, Victora C, Saracci R. Global response to non- Epidemiol. 2020;120:1–7. https://​doi.​org/​10.​1016/j.​jclin​epi.​2019.​12.​014. communicable diseases—the role of epidemiologists. Int J Epidemiol. 22. Abbasi K. Covid-19: politicisation, “corruption,” and suppression of 2012;41:1219–20. https://​doi.​org/​10.​1093/​ije/​dys145. https://​acade​mic.​ science. When good science is suppressed by the medical-political oup.​com/​ije/​artic​le/​41/5/​1219/​713559?​login ​true. complex, people die. (Editorial). BMJ. 2020;371:m4425. https://​doi.​org/​ 3. Oremus M, Taylor-Wilson R, Aldrich M, Bell K, Gaudino= J, Palevsky S, 10.​1136/​bmj.​m4425. Payne J, Raynes-Greenow C, Sim F, Smith M, Weiss S, Zhang Y. The role 23. Becker C. Relationships between academic medicine leaders and of epidemiologists in SARS-CoV-2 and COVID-19 research (Letter). Pub‑ industry – time for another look? JAMA. 2020;324(18):1833–4. https://​ lic Health. 2021;190(2021):e3–4. https://​doi.​org/​10.​1016/j.​puhe.​2020.​10.​ doi.​org/​10.​1001/​jama.​2020.​21021. PMID: 33170245. 006. https://​www.​ncbi.​nlm.​nih.​gov/​pmc/​artic​les/​PMC75​68048/. 24. Miller BL. Science denial and COVID conspiracy theories: poten‑ 4. Michaels D. Doubt is their product: how industry’s assault on science tial neurological mechanisms and possible responses. JAMA. threatens your health. New York: Oxford University Press; 2008. 2020;324(22):2255–6. https://​doi.​org/​10.​1001/​jama.​2020.​21332. 5. Michaels D. The triumph of doubt: dark money and the science of 25. McHenry LB. The Monsanto Papers: Poisoning the scientifc well. Int deception. New York: Oxford University Press; 2020. J Risk Saf Med. 2018;29(3–4):193–205. https://​doi.​org/​10.​3233/​JRS-​ 6. Goldberg RF, Vandenberg LN. The science of spin: targeted strategies 180028. PMID: 29843257. to manufacture doubt with detrimental efects on environmental and 26. Bofetta P, Adami HO, Cole P, Trichopoulos D, Mandel JS. Epidemio‑ public health. Environ Health. 2021;20:33. https://​doi.​org/​10.​1186/​ logic studies of styrene and cancer: a review of the literature. J Occup s12940-​021-​00723-0. Environ Med. 2009;51(11):1275–87. https://​doi.​org/​10.​1097/​JOM.​0b013​ 7. Soskolne CL, Light A. Towards ethics guidelines for environmental e3181​ad49b2. epidemiologists. Sci Total Environ. 1996;184(1,2):137–47. Adopted by 27. La Vecchia C, Bofetta P. Role of stopping exposure and recent exposure the International Society for Environmental Epidemiology in 1999. to asbestos in the risk of mesothelioma. Eur J Cancer Prev. 2012;21:227– Accessible at: http://​colin​sosko​lne.​com/​docum​ents/​025.​pdf. 30. https://​doi.​org/​10.​1097/​CEJ.​0b013​e3283​4dbc56. 8. American College of Epidemiology (ACE) ethics guidelines, January 24, 28. Acquavella J, Garabrant D, Marsh G, Sorahan T, Weed DL. Glyphosate 2000. https://​www.​acepi​demio​logy.​org/​ethics-​guide​lines. Accessed 26 epidemiology expert panel review: a weight of evidence system‑ July 2021. atic review of the relationship between glyphosate exposure and 9. International Society for Environmental Epidemiology (ISEE). Ethics non-Hodgkin’s lymphoma or multiple myeloma. Crit Rev Toxicol. Guidelines for Environmental Epidemiologists, April 25, 2012. https://​ 2016;46(sup1):28–43. https://​doi.​org/​10.​1080/​10408​444.​2016.​12146​81. www.​iseepi.​org/​docs/​ISEE_​Ethics_​Guide​lines_​adopt​ed_​april_​25_​2012-​ Erratum in: Crit Rev Toxicol. 2018;Sep 26:1. Engli​sh.​pdf. Accessed 26 July 2021. 29. Ciocan C, Franco N, Pira E, Mansour I, Godono A, Bofetta P. Methodo‑ 10. Silbergeld EK, Mandrioli D, Cranor CF. Regulating chemicals: law, logical issues in descriptive environmental epidemiology. The example science, and the unbearable burdens of regulation. Annu Rev Public of study Sentieri. Med Lav. 2021;112(1):15–33. https://​doi.​org/​10.​23749/​ Health. 2015;18(36):175–91. https://​doi.​org/​10.​1146/​annur​ev-​publh​ mdl.​v112i1.​10099. ealth-​031914-​122654. PMID: 25785889. 30. Kramer S, Soskolne CL. Ethics guidelines in environmental epidemiol‑ 11. Pearce N. Corporate infuences on epidemiology. Int J Epidemiol. ogy: their development and challenges we face. Curr Environ Health 2008;37:46–53. https://​doi.​org/​10.​1093/​ije/​dym270. https://​acade​mic.​ Rep. 2017;4:142. https://​doi.​org/​10.​1007/​s40572-​017-​0138-z. See oup.​com/​ije/​artic​le/​37/1/​46/​771539?​login ​true. https://​rdcu.​be/​L4cj. 12. Mandrioli D, Kearns CE, Bero LA. Relationship= between research out‑ 31. Soskolne CL, Caldwell JC, London L, Bero L, Gochfeld M, Cranor CF, comes and risk of bias, study sponsorship, and author fnancial conficts Ramos-Bonilla JP, Mandrioli D, Sass, J, Advani S. International Network of interest in reviews of the efects of artifcially sweetened bever‑ for Epidemiology in Policy (INEP) position statement series: confict-of- ages on weight outcomes: a systematic review of reviews. PLoS One. interest and disclosure in epidemiology. 2020. https://​epide​miolo​gyinp​ 2016;11(9):e0162198. olicy.​org/​coi-d-​posit​ion-​state​ment (95 pages). Released to the public 13. Rosner D, Markowitz G. The politics of lead toxicology and the devastat‑ on January 5, 2021. Accessed 26 July 2021. ing consequences for children. Am J Industrial Med. 2007;50:740–56. 32. Denison R. Trump EPA, ACC and industry law frms colluded to weaken https://​doi.​org/​10.​1002/​ajim.​20435. EPA new chemical safety reviews. Environmental Defense Fund (EDF); 14. Ong EK, Glantz SA. Constructing “sound science” and “good epidemiol‑ 2021. http://​blogs.​edf.​org/​health/​2021/​03/​11/​trump-​epa-​acc-​and-​ ogy”: tobacco, lawyers, and public relations frms. Am J Public Health. indus​try-​law-​frms-​collu​ded-​to-​weaken-​epa-​new-​chemi​cal-​safety-​revie​ 2001;91(11):1749–57. https://​ajph.​aphap​ublic​ations.​org/​doi/​full/​10.​ ws/?​utm_​source ​exper​t&​utm_​campa​ign ​edf-​health_​none_​upd_​ 2105/​AJPH.​91.​11.​1749. hlth&​utm_​medium= ​email​&​utm_​id ​16154=​84423. Accessed 26 July 15. Marsili D, Terracini B, Santana VS, Ramos-Bonilla JP, Pasetto R, Mazzeo A, 2021. = = Loomis D, Comba P, Algranti E. Prevention of asbestos-related disease 33. Cook J, Supran G, Lewandowsky S, Oreskes N, Maibach E. How fossil in countries currently using asbestos. Int J Environ Res Public Health. fuel industry misled Americans deliberately about climate change. The 2016;13(5):E494. https://​doi.​org/​10.​3390/​ijerp​h1305​0494. Print; 2019. https://​thepr​int.​in/​opini​on/​how-​fossil-​fuel-​indus​try-​misled-​ 16. Algranti E, Ramos-Bonilla JP, Terracini B, Santana VS, Comba P, Pasetto R, ameri​cans-​delib​erate​ly-​about-​clima​te-​change/​315179/. Accessed 26 Mazzeo A, Cavariani F, Trotta A, Marsili D. Prevention of asbestos expo‑ July 2021. sure in Latin America within a global public health perspective. Ann 34. The Climate Reality Project. The climate denial machine: how the fossil Glob Health. 2019;85(1):49, 1–15. https://​doi.​org/​10.​5334/​aogh.​2341. fuel industry blocks climate action. 2019. https://​www.​clima​terea​lityp​ 17. Baur X, Frank AL. Ongoing downplaying of the carcinogenicity roject.​org/​blog/​clima​te-​denial-​machi​ne-​how-​fossil-​fuel-​indus​try-​ of chrysotile asbestos by vested interests. J Occup Med Toxicol. blocks-​clima​te-​action. 2021;16(1):6. https://​doi-​org.​ezpro​xy.​unian​des.​edu.​co:​8443/​10.​1186/​ 35. Noor D. Trump’s EPA now says oil and gas production are ‘insignif‑ s12995-​021-​00295-2. cant’ sources of pollution. January 13, 2021. https://​gizmo​do.​com/​ 18. Ruf K, Mirabelli D. Confict of interest, tailored science, and responsibil‑ trumps-​epa-​now-​says-​oil-​and-​gas-​produ​ction-​are-​insig​nif-​18460​49568. ity of scientifc institutions and journals. New Solut. 2014;24(3):259–66. Accessed 26 July 2021. https://​doi.​org/​10.​2190/​NS.​EOV. 36. Milman O. Oil frms knew decades ago fossil fuels posed grave health 19. Van der Eijk Y, Bero LA, Malone RE. Philip Morris International-funded risks, fles reveal. The Guardian; 2021. https://www.​ thegu​ ardian.​ com/​ ​ ‘Foundation for a Smoke-Free World’: analysing its claims of independ‑ environment/​ 2021/​ mar/​ 18/​ oil​ -​industry​ -​fossil-​fuels-​air-​pollution​ -​ ence. Tob Control. 2019;28(6):712–8. documents​ . Accessed 26 July 2021. 20. Bero L, Grundy Q. Why having a (non-fnancial) interest is not a confict 37. European Environment Agency (EEA). Late lessons from early warnings: of interest (perspective). PLoS Biol. 2017;14(12): e2001221. https://​doi.​ the precautionary principle 1896–2000. Environmental issue report org/​10.​1371/​journ​al.​pbio.​20012​21. no 22. Luxembourg: Ofce for Ofcial Publications of the European Soskolne et al. Environ Health (2021) 20:90 Page 14 of 16

Communities; 2001. https://​www.​eea.​europa.​eu/​publi​catio​ns/​envir​ 59. Mausner JS, Kramer S. Mausner & Bahn epidemiology: an introductory onmen​tal_​issue_​report_​2001_​22/​Issue_​Report_​No_​22.​pdf/​view. text. Philadelphia: W.B. Saunders; 1985. 38. European Environment Agency (EEA). Late lessons from early 60. Szklo M, Nieto FJ. Epidemiology: beyond the basics. 4th ed. Burlington: warnings: science, precaution, innovation. Summary. EEA report no Jones & Bartlett Learning; 2019. 1/2013. Denmark: Rosendahls‑Schultz Grafsk; 2013. https://​www.​ 61. Haynes RB, Sackett DL, Guyatt GH, Tugwell P. Clinical epidemiology: eea.​europa.​eu/​publi​catio​ns/​late-​lesso​ns-2. Accessed 26 July 2021. how to do clinical practice research. 3rd ed. Philadelphia: Lippincott 39. Nieuwenhuijsen M, Fletcher T, de Nazelle A, Etzel RA. Re: Sponsorship Williams & Wilkins; 2005. by Big Oil, like the tobacco industry, should be banned by the research 62. Smith RA, Levine TR, Lachlan KA, Fediuk TA. The high cost of complexity community. Epidemiology. 2021; Volume Publish Ahead of Print - Issue in experimental design and data analysis. Type I and type II error rates in - https://​doi.​org/​10.​1097/​EDE.​00000​00000​001325. https://​journ​als.​lww.​ multiway ANOVA. Hum Commun Res. 2002;28(4):515–30. https://​acade​ com/​epidem/​Citat​ion/​9000/​Re__​Re__​Spons​orship_​by_​Big_​Oil,_​Like_​ mic.​oup.​com/​hcr/​artic​le-​abstr​act/​28/4/​515/​43311​32?​redir​ected​From ​ the_​Tobac​co.​98302.​aspx. Accessed 26 July 2021. fullt​ext. = 40. Global Policy Forum, New York, N.Y. June 12, 2013. UN NEWS CENTRE: 63. Makin TR, Orban de Xivry J-J. Ten common statistical mistakes to watch Head of WHO criticizes “big business” and its role in public health. out for when writing or reviewing a manuscript. eLife. 2019;8:1–13. https://​archi​ve.​globa​lpoli​cy.​org/​compo​nent/​conte​nt/​artic​le/​221-​trans​ https://​doi.​org/​10.​7554/​eLife.​48175. natio​nal-​corpo​ratio​ns/​52420-​who-​criti​cizes-​qbig-​busin​essq-​and-​its-​ 64. Neutra RR, Cranor CF, Gee D. The use and misuse of Bradford Hill in U.S. role-​in-​public-​health.​html. Accessed 26 July 2021. Tort Law. Jurimetrics J. 2018;58:127–62. 41. Cranor CF. Toxic torts: science, law and the possibility of justice. New 65. Lieber RL. Statistical signifcance and statistical power in hypothesis York: Cambridge University Press; 2017. testing. J Orthop Res. 1990;8(2):304–9. https://​onlin​elibr​ary.​wiley.​com/​ 42. Cranor CF. Legally poisoned: how the law puts us at risk from toxicants. doi/​abs/​10.​1002/​jor.​11000​80221. Boston: Harvard University Press; 2013. 66. Altman DG, Bland JM. Absence of evidence is not evidence of absence. 43. Soskolne CL. Global, regional and local ecological change: ethical BMJ. 1995;311:485. https://​www.​ncbi.​nlm.​nih.​gov/​pmc/​artic​les/​PMC25​ aspects of public health research and practice. Part1, Chapter 1. In: 50545/. Zölzer F, Meskens G, editors. Ethics of environmental health. Routledge 67. Alderson P. Absence of evidence is not evidence of absence. BMJ (Clini‑ studies in environment and health. London and New York: Routledge, cal research ed). 2004;328(7438):476–7. https://​doi.​org/​10.​1136/​bmj.​ Taylor & Francis; 2017. p. 3–16. 328.​7438.​476. 44. Soskolne CL. The role of vested interests and dominant narratives in 68. Gee D. Establishing evidence for early action: the prevention of science, risk management and risk communication. Chapter 8. In: reproductive and developmental harm. Basic Clin Pharmacol Toxicol. Zölzer F, Meskens G, editors. Environmental health risks: ethical aspects. 2008;102(2):257–66. https://​doi.​org/​10.​1111/j.​1742-​7843.​2008.​00207.x. Routledge studies in environment and health. London and New York: PMID: 18226081. Routledge, Taylor & Francis; 2019. p. 123–34. 69. Huo D, Anderson D, Palmer JR, Herbst AL. Incidence rates and 45. Soskolne CL. Public health and environmental health risk assessment: risks of diethylstilbestrol-related clear-cell adenocarcinoma of the which paradigm and in whose best interests? In: Westra L, Gray J, Kara‑ vagina and cervix: Update after 40-year follow-up. Gynecol Oncol. georgou V, editors. Ecological systems integrity: governance, law and 2017;146(3):566–71. https://doi.​ org/​ 10.​ 1016/j.​ ygyno.​ 2017.​ 06.​ 028​ . human rights. Chapter 16. London: Earthscan; 2015. p. 191–200. Epub 2017 Jul 6. PMID: 28689666. 46. Congressional Research Service. Federal scientifc integrity policies: a 70. Alpert N, van Gerwen M, Taioli E. Epidemiology of mesothelioma in the primer. 2020. p. R46614. https://​crsre​ports.​congr​ess.​gov. 21st century in Europe and the United States, 40 years after restricted/ 47. Baur X, Budnik LT, Ruf K, Egilman DS, Lemen RA, Soskolne CL. Ethics, banned asbestos use. Transl Lung Cancer Res. 2020;9(Suppl 1):S28–38. morality, and conficting interests: how questionable professional integ‑ https://​doi.​org/​10.​21037/​tlcr.​2019.​11.​11. PMID: 32206568; PMCID: rity in some scientists supports global corporate infuence in public PMC7082259. health. Int J Occup Environ Health. 2015;21:172–5. 71. International Agency for Research on Cancer. World Health Organiza‑ 48. Soskolne C, Baur X. How corporate infuence continues to undermine tion. IARC monographs on the identifcation of carcinogenic hazards to the public’s health. Commentary on the Collegium Ramazzini sympo‑ humans. Preamble. Lyon; 2019. https://​monog​raphs.​iarc.​who.​int/​wp-​ sium held in Carpi, Italy. J Sci Pract Integr. 2018;1(1). https://​www.​jospi.​ conte​nt/​uploa​ds/​2019/​07/​Pream​ble-​2019.​pdf. Accessed 26 July 2021. org/​artic​le/​9747-​how-​corpo​rate-​infu​ence-​conti​nues-​to-​under​mine-​ 72. Sass J. MacLennan et al. report on an elevated incidence of prostate the-​public-​s-​health. Accessed 26 July 2021. cancer among workers in a triazine manufacturing plant. J Occup 49. Baur X, Soskolne CL, Bero LA. Commentary. How can the integrity of Environ Med. 2003;45(4):343–4. https://​doi.​org/​10.​1097/​01.​jom.​00000​ occupational and environmental health research be maintained in the 63624.​37065.​7a. author reply 344. PMID: 12708135. presence of conficting interests? Environ Health. 2019;18:93. https://​ 73. Torgerson DJ. Contamination in trials: is cluster randomisation the ehjou​rnal.​biome​dcent​ral.​com/​artic​les/​10.​1186/​s12940-​019-​0527-x. answer? BMJ. 2001;322(February):355–7. https://​pubmed.​ncbi.​nlm.​nih.​ 50. Schlesselman JJ. Case-control studies: design, conduct, analysis. New gov/​11159​665/. York: Oxford University Press; 1982. 74. Velentgas P, Dreyer NA, Nourjah P, Smith SR, Torchia MM, eds. Develop‑ 51. Rothman KJ, Greenland S, Lash TL. Modern epidemiology. 3rd ed. ing a protocol for observational comparative efectiveness research: a Philadelphia: Lippincott Williams & Wilkins; 2012. user’s guide. AHRQ publication no. 12(13)-EHC099. Rockville: Agency 52. Breslow NE, Day NE. Statistical methods in cancer research. Volume for Healthcare Research and Quality; 2013. www.efec​ tiveh​ ealth​ care.​ ​ I: the analysis of case-control studies. Lyon: International Agency for ahrq.gov/​ Metho​ ds​ -​OCER.cfm​ . https://www.​ ncbi.​ nlm.​ nih.​ gov/​ books/​ ​ Research on Cancer (IARC); 1980. NBK126191/​ . Accessed 26 July 2021. 53. Breslow NE, Day NE. Statistical methods in cancer research. Volume II: 75. Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, the design and analysis of cohort studies. Lyon: International Agency Pocock SJ, Poole C, Schlesselman JJ, Egger M. Strengthening the report‑ for Research on Cancer (IARC); 1987. ing of observational studies in epidemiology (STROBE): explanation and 54. Checkoway H, Pearce N, Kriebel D. Research methods in occupational elaboration. PLoS Med. 2007;4(10): e297. https://​doi.​org/​10.​1371/​journ​ epidemiology. 2nd ed. New York: Oxford University Press; 2004. al.​pmed.​00402​97. 55. Gordis L. Epidemiology. 2nd ed. Philadelphia: Saunders; 2000. 76. Hauptmann M, Lubin JH, Stewart PA, Hayes RB, Blair A. Mortality from 56. Hennekens CH, Buring JE. Epidemiology in medicine. Boston: Little lymphohematopoietic malignancies among workers in formaldehyde Brown; 1987. industries. J Natl Cancer Inst. 2003;95(21):1615–23. 57. Kleinbaum DG, Kupper LL, Morgenstern H. Epidemiologic research: 77. Mao Q, Manservisi F, Panzacchi S, Mandrioli D, Menghetti I, Vornoli A, principles and quantitative methods. Belmont: Lifetime Learning Publi‑ Bua L, Falcioni L, Lesseur C, Chen J, Belpoggi F, Hu J. The Ramazzini cations; 1982. Institute 13-week pilot study on glyphosate and Roundup administered 58. MacMahon B, Trichopolous D. Epidemiology: principles & methods. at human-equivalent dose to Sprague Dawley rats: efects on the 2nd ed. Boston: Little, Brown; 1996. Soskolne et al. Environ Health (2021) 20:90 Page 15 of 16

microbiome. Environ Health. 2018;17(1):50. https://​doi.​org/​10.​1186/​ 89. Smith MT, Guyton KZ, Gibbons CF, Fritz JM, Portier CJ, Rusyn I, DeMarini s12940-​018-​0394-x. PMID: 29843725; PMCID: PMC5972442. DM, Caldwell JC, Kavlock RJ, Lambert PF, Hecht SS, Bucher JR, Stewart 78. Manservisi F, Lesseur C, Panzacchi S, Mandrioli D, Falcioni L, Bua L, BW, Baan RA, Cogliano VJ, Straif K. Key characteristics of carcinogens as Manservigi M, Spinaci M, Galeati G, Mantovani A, Lorenzetti S, Miglio R, a basis for organizing data on mechanisms of carcinogenesis. Environ Andrade AM, Kristensen DM, Perry MJ, Swan SH, Chen J, Belpoggi F. The Health Perspect. 2016;124(6):713–21. https://​doi.​org/​10.​1289/​ehp.​ Ramazzini Institute 13-week pilot study glyphosate-based herbicides 15099​12. Epub 2015 Nov 24. PMID: 26600562; PMCID: PMC4892922. administered at human-equivalent dose to Sprague Dawley rats: efects 90. National Academies of Sciences, Engineering, and Medicine. The use of on development and endocrine system. Environ Health. 2019;18(1):15. systematic review in EPA’s toxic substances control act risk evaluations. https://​doi.​org/​10.​1186/​s12940-​019-​0453-y. PMID: 30857531; PMCID: Washington, DC: The National Academies Press; 2021. https://​doi.​org/​ PMC6413565. 10.​17226/​25952. 79. Kavlock RJ, Daston GR, DeRosa, Fenner-Crisp P, Gray LE, Kaattari S, Lucier 91. Grandjean P. Delayed discovery, dissemination, and decisions on inter‑ G, Luster M, Mac MJ, Maczka C, Miller R, Moore J, Rolland R, Scott G, vention in environmental health: a case study on immunotoxicity of Sheehan DM, Sinks T, Tilson HA. Research needs for the risk assess‑ perfuorinated alkylate substances. Environ Health. 2018;17:62. https://​ ment of health and environmental efects of endocrine disruptors: a doi.​org/​10.​1186/​s12940-​018-​0405-y. report of the U.S. EPA-sponsored workshop. Environ Health Perspect. 92. Temkin AM, Hocevar BA, Andrews DQ, Naidenko OV, Kamendulis LM. 1996;104(Suppl 4):715–40. https://​doi.​org/​10.​1289/​ehp.​96104​s4715. Application of the key characteristics of carcinogens to per and poly‑ https://​www.​ncbi.​nlm.​nih.​gov/​pmc/​artic​les/​PMC14​69675/. fuoroalkyl substances. Int J Environ Res Public Health. 2020;17(5):1668. 80. Rose G, Barker DJ. Epidemiology for the uninitiated. Chapter 4: meas‑ https://​doi.​org/​10.​3390/​ijerp​h1705​1668. PMID: 32143379; PMCID: urement bias and error. Br Med J. 1978;2(6149). Available at https://​ PMC7084585. www.​bmj.​com/​about-​bmj/​resou​rces-​reade​rs/​publi​catio​ns/​epide​miolo​ 93. Amrhein V, Greenland S, McShane B. Scientists rise up against statistical gy-​unini​tiated. Accessed 26 July 2021. signifcance. Nature. 2019;567(7748):305–7. https://​doi.​org/​10.​1038/​ 81. Eick SM, Goin DE, Chartres N, Lam J, Woodruf TJ. Assessing risk of bias d41586-​019-​00857-9. PMID: 30894741. in human environmental epidemiology studies using three tools: difer‑ 94. Francis JA, Shea AK, Samet JM. Challenging the epidemiologic evidence ent conclusions from diferent tools. Syst Rev. 2020;9(1):249. https://​doi.​ on passive smoking: tactics of tobacco industry expert witnesses. Tob org/​10.​1186/​s13643-​020-​01490-8. https://​pubmed.​ncbi.​nlm.​nih.​gov/​ Control. 2006;15(Suppl 4):iv68–76. https://​doi.​org/​10.​1136/​tc.​2005.​ 33121​530/. 014241. https://​www.​ncbi.​nlm.​nih.​gov/​pmc/​artic​les/​PMC25​63583/. 82. Portier CJ, Armstrong BK, Baguley BC, Baur X, Belyaev I, Bellé R, Belpoggi 95. Greenland S, Senn SJ, Rothman KJ, Carlin JB, Poole C, Goodman SN, F, Biggeri A, Bosland MC, Bruzzi P, Budnik LT, Bugge MD, Burns K, Calaf Altman DG. Statistical tests, P values, confdence intervals, and power: GM, Carpenter DO, Carpenter HM, López-Carrillo L, Clapp R, Cocco a guide to misinterpretations. Eur J Epidemiol. 2016;31(4):337–50. P, Consonni D, Comba P, Craft E, Dalvie MA, Davis D, Demers PA, De https://​doi.​org/​10.​1007/​s10654-​016-​0149-3. Epub 2016 May 21. PMID: RoosDeWitt AJJ, Forastiere F, Freedman JH, Fritschi L, Gaus C, Gohlke 27209009; PMCID: PMC4877414. JM, Goldberg M, Greiser E, Hansen J, Hardell L, Hauptmann M, Huang 96. Cranor CF. Milward v. Acuity Specialty Products: advances in general W, Huf J, James MO, Jameson CW, Kortenkamp A, Kopp-Schneider causation testimony in toxic tort litigation . Wake Forest J Law Policy. A, Kromhout H, Larramendy ML, Landrigan PJ, Lash LH, Leszczynski 2013;3:105–37. D, Lynch CF, Magnani C, Mandrioli D, Martin FL, Merler E, Michelozzi 97. Cogliano VJ, Baan RA, Straif K, Grosse Y, Secretan B, El Ghissassi F. P, Miligi L, Miller AB, Mirabelli D, Mirer FE, Naidoo S, Perry MJ, Petronio Use of mechanistic data in IARC evaluations. Environ Mol Mutagen. MG, Pirastu R, Portier RJ, Ramos KS, Robertson LW, Rodriguez T, Röösli 2008;49:100–9. M, Ross MK, Roy D, Rusyn I, Saldiva P, Sass J, Savolainen K, Scheepers PT, 98. Elliott KC, Resnik DB. Science, policy, and the transparency of values. Sergi C, Silbergeld EK, Smith MT, Stewart BW, Sutton P, Tateo F, Terracini Environ Health Perspect. 2014;122:647–50. https://​doi.​org/​10.​1289/​ehp.​ B, Thielmann HW, Thomas DB, Vainio H, Vena JE, Vineis P, Weiderpass E, 14081​07. Weisenburger DD, Woodruf TJ, Yorifuji T, Yu IJ, Zambon P, Zeeb H, Zhou 99. Traversy G, Barnieh L, Akl EA, Allan GM, Brouwers M, Ganache I, Grundy SF. Diferences in the carcinogenic evaluation of glyphosate between Q, Guyatt GH, Kelsall D, Leng G, Moore A, Persaud N, Schünemann HJ, the International Agency for Research on Cancer (IARC) and the Straus S, Thombs BD, Rodin R, Tonelli M. Managing conficts of interest European Food Safety Authority (EFSA). J Epidemiol Community Health. in the development of health guidelines. CMAJ. 2021;193:E49-54. 2016;70(8):741–5. https://​doi.​org/​10.​1136/​jech-​2015-​207005. Epub https://​doi.​org/​10.​1503/​cmaj.​200651. 2016 Mar 3. PMID: 26941213; PMCID: PMC4975799. 100. Neltner TG, Alger HM, O’Reilly JT, Krimsky S, Bero LA, Mafni MV. 83. Frakt AB, Bagley N. Protection or harm? Suppressing substance-use Conficts of interest in approvals of additives to food determined to data. N Engl J Med. 2015;372(20):1879–81. https://​doi.​org/​10.​1056/​ be generally recognized as safe: out of balance. JAMA Intern Med. NEJMp​15013​62. https://​pubmed.​ncbi.​nlm.​nih.​gov/​25875​196/. 2013;173(22):2032–6. https://​doi.​org/​10.​1001/​jamai​ntern​med.​2013.​ 84. Tran TH, Stefen JE, Clancy KM, Bird T, Egilman DS. Talc, asbestos, and 10559. PMID: 23925593. epidemiology: corporate infuence and scientifc incognizance. Epide‑ 101. Fabbri A, Lai A, Grundy Q, Bero LA. The infuence of industry sponsor‑ miology. 2019;30(6):783–8. https://​doi.​org/​10.​1097/​EDE.​00000​00000​ ship on the research agenda: a scoping review. Am J Public Health. 001091. 2018;108(11):e9–16. https://​doi.​org/​10.​2105/​AJPH.​2018.​304677. 85. Suter G, Nichols J, Lavoie E, Cormier S. Systematic review and weight 102. Steenland K, Schubauer-Berigan MK, Vermeulen R, Lunn RM, Straif of evidence are integral to ecological and human health assessments: K, Zahm S, Stewart P, Arroyave WD, Mehta SS, Pearce N. Risk of bias they need an integrated framework. Integr Environ Assess Manag. assessments and evidence syntheses for observational epidemiologic 2020;16:718–28. https://​doi.​org/​10.​1002/​ieam.​4271. studies of environmental and occupational exposures: strengths and 86. Jinot J, Fritz JM, Vulimiri SV, Keshava N. Carcinogenicity of ethylene limitations. Environ Health Perspect. 2020;128(9):10. https://​doi.​org/​10.​ oxide: key fndings and scientifc issues. Toxicol Mech Methods. 1289/​EHP69​80. 2018;28(5):386–96. https://​doi.​org/​10.​1080/​15376​516.​2017.​14143​43. 103. Woodruf TJ, Sutton P. The Navigation Guide systematic review method‑ 87. Risk Assessment Forum. Guidelines for carcinogen risk assessment. ology: a rigorous and transparent method for translating environmen‑ Washington, DC: U.S. Environmental Protection Agency; 2005. 166 tal health science into better health outcomes. Environ Health Perspect. pages. EPA/630/P-03/001B. https://​www3.​epa.​gov/​airto​xics/​cancer_​ 2014;122(10):1007–14. https://​doi.​org/​10.​1289/​ehp.​13071​75. Epub guide​lines_​fnal_3-​25-​05.​pdf. 2014 Jun 25. PMID: 24968373; PMCID: PMC4181919. 88. Pearce N, Vandenbroucke JP, Lawlor DA. Causal inference in envi‑ 104. Pega F, Norris SL, Backes C, Bero LA, Descatha A, Gagliardi D, Godderis ronmental epidemiology: old and new approaches. Epidemiology. L, Loney T, Modenese A, Morgan RL, Pachito D, Paulo MBS, Scheepers 2019;30(3):311–6. https://​doi.​org/​10.​1097/​EDE.​00000​00000​000987. PTJ, Schlünssen V, Sgargi D, Silbergeld EK, Sørensen K, Sutton P, Tenkate https://​journ​als.​lww.​com/​epidem/​Fullt​ext/​2019/​05000/​Causal_​Infer​ T, Torreão Corrêa da Silva D, Ujita Y, van Deventer E, Woodruf TJ, Man‑ ence_​in_​Envir​onmen​tal_​Epide​miolo​gy_.1.​aspx. drioli D. RoB-SPEO: a tool for assessing risk of bias in studies estimating Soskolne et al. Environ Health (2021) 20:90 Page 16 of 16

the prevalence of exposure to occupational risk factors from the WHO/ 107. Ruf K. Serving industry, promoting skepticism, discrediting epide‑ ILO Joint Estimates of the Work-related Burden of Disease and Injury. miology. Chapter 7. In: Walker MJ, editor. Corporate ties that bind: an Environ Int. 2020;135:105039. https://doi.​ org/​ 10.​ 1016/j.​ envint.​ 2019.​ ​ examination of corporate manipulation and vested interest in public 105039. health. New York: Skyhorse Publishing; 2017. p. 119–35; 482–5. 105. Rimmer A. Nine in 10 professional organisations say doctors should have to register their fnancial interests. BMJ. 2021;373: n933. https://​ doi.​org/​10.​1136/​bmj.​n933. Publisher’s Note 106. Ruf K. Commentary: scientifc journals and confict of interest dis‑ Springer Nature remains neutral with regard to jurisdictional claims in pub‑ closure: what progress has been made? Environ Health. 2015;14:45. lished maps and institutional afliations. https://​doi.​org/​10.​1186/​s12940-​015-​0035-6 (8pages).

Ready to submit your research ? Choose BMC and benefit from:

• fast, convenient online submission • thorough peer review by experienced researchers in your field • rapid publication on acceptance • support for research data, including large and complex data types • gold Open Access which fosters wider collaboration and increased citations • maximum visibility for your research: over 100M website views per year

At BMC, research is always in progress.

Learn more biomedcentral.com/submissions