Interpreting Epidemiologic Evidence: Strategies for Study Design and Analysis
Total Page:16
File Type:pdf, Size:1020Kb
Datu Dr Andrew Kiyu 24/6/2013 Importance of topic INTERPRETATION OF EPIDEMIOLOGIC EVIDENCE: • Epidemiology success stories have led to improved policy and Truth, Chance and Biases practice. • However, observational epidemiology has: – methodologic limitations that Dr Andrew Kiyu – affect the ability to infer causation. (MBBS, MPH, DrPH, AM, FACE) Consultant Epidemiologist • The major challenge of : Sarawak Health Department • dealing with the data deluge and Email address: [email protected] – uncovering true causal relationships from – the millions and millions of observations that are Sibu Clinical Research Seminar background noise. RH Hotel, Sibu • Date: 24 June 2013 Increased consumer awareness and education • Source: NCI's Epidemiology and Genomics Research Program sponsored a workshop entitled” T rends in 21st Century Epidemiology- From Scientific Discoveries to Population Health Impact “ on 12-13 December 2012 at 1 http://epi.grants.cancer.gov/workshops/century-trends/ 2 “Interpretation of Epidemiologic Evidence” Osteoporosis as a Health Issue Producer of • The US Surgeon General estimates that one out of every two Epidemiologic Evidence women over the age of 50 will have an osteoporosis-related fracture in their lifetime. • In addition, 20% of those affected by osteoporosis are men with 6% of white males over the age of 50 suffering a hip fracture. Personal level Population level • It is estimated that the national direct care costs for osteoporotic fractures is US$12.2 to 17.9 billion per year in 2002 dollars, with costs rising. • This cost is comparable to the Medicare expense for coronary heart disease ($11.6 billion) (Thom et al 2006) Consumer of • John A Sunyecz. The use of calcium and vitamin D in the management of osteoporosis. Therapeutics and Clinical Risk Management 2008:4(4) 827–836 at http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2621390/pdf/TCRM-4-827.pdf Epidemiologic Evidence 3 4 The Use of Calcium and vitamin D in the Anlene advertisement Management of Osteoporosis (2004, 2008) Therapeutics and Clinical Risk Management 2008:4(4) 827–836 at http://www.surgeongeneral.gov/library/rep http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2621390/pdf/TCRM-4-827.pdf orts/bonehealth/OsteoBrochure1mar05.pd f 5 6 http://adsoftheworld.com/files/images/anlene_spine.jpg Sibu Research Seminar 2013 1 Datu Dr Andrew Kiyu 24/6/2013 Calcium Supplements Increase Heart Attack Risk by 86 Percent (Heart 2012) • New research published in the journal Heart has confirmed the findings of two controversial studies on calcium supplementation and heart attack risk published in the British Medical Journal last year, and which found a 24-27% increased risk of heart attack for those who took 500 mg of elemental calcium a day.[1] [2] • The results of this newest review, involving 24,000 people between the ages of 35 and 64, were even more alarming. Those participants who took a regular calcium supplement increased their risk of Flow Galindez. Fight Osteoporosis with two glass of Anlene everyday. Sunday, April 3, 2011. having a heart attack by 86% versus those who took no calcium supplements at all. http://angsawariko.blogspot.com/2011/04/fight-osteoporosis-with-two-glass-of.html • Conclusions: Increasing calcium intake from diet might not confer significant cardiovascular benefits, while calcium supplements, which might raise MI risk, should be taken with caution. 7 • Heart 2012;9 8:920e925. doi:10.1136/heartjnl-2011-301345 at http://heart.bmj.com/content/98/12/920.full.pdf+html 8 How to read a paper Greenhalgh T, Taylor R. BMJ 1997 1. The Medline database. BMJ 1997;315:180-183 (19 July). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2127107 2. Getting your bearings (deciding what the paper is about). BMJ 1997;315:243-246 (26 July). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2127173 3. Assessing the methodological quality of published papers. BMJ 1997;315:305-308 (2 August). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2127212 4. Statistics for the non-statistician. I: Different types of data need different statistical tests. BMJ 1997;315:364- 366 (9 August). http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2127256&blobtype=pdf 5. How to read a paper: Statistics for the non-statistician. II: "Significant" relations and their pitfalls. BMJ 1997;315:422-425 (16 August). http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2127270&blobtype=pdf 6. Papers that report drug trials. BMJ 1997;315:480-483 (23 August). http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2127321 7. Papers that report diagnostic or screening tests. BMJ 1997;315:540-543 (30 August). NOT ACADEMIC EXERCISE OR A http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2127365 8. Papers that tell you what things cost (economic analyses). BMJ 1997;315:596-599 (6 September). ‘HOW TO’ SESSION http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2127419&blobtype=pdf 9. Papers that summarise other papers (systematic reviews and meta-analyses). BMJ 1997;315:672-675 (13 September). http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2127461&blobtype=pdf 10. Papers that go beyond numbers (qualitative research). BMJ 1997;315:740-743 (20 September). http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=2127518&blobtype=pdf 9 10 http://eprints.mdx.ac.uk/2981/1/Developing_a_framework_for_critiquing_health_research.pdf Natalie Wolchover, (Life's Little Mysteries Staff Writer). What If Humans Had Eagle Vision? 11 Date: 24 February 2012. at http://www.lifeslittlemysteries.com/2184-humans-eagle-vision.html 12 Sibu Research Seminar 2013 2 Datu Dr Andrew Kiyu 24/6/2013 Unpublished evidence Publication bias Published evidence Producers Of Epidemiologic Policymakers Legal Evidence Scientist Truth Health & medicine Public Journal decision to Pseudoscience accept Researcher decision whether to submit Which parts to publish for publication Junk science Study design analysis EVIDENCE Epidemiologic research/ Epidemiologic Interpretation methods Scientific method paradigm How scientific decisions are arrived at 14 Evidence Evidence • Evidence is and includes everything that is used to • Research findings are defined (for the purpose of reveal and determine the truth, and therefore is this paper) as presumed to be true and related to a case – any relationship reaching formal statistical • Scientific evidence is evidence which serves to either significance. support or counter a scientific theory or hypothesis. – Such evidence is expected to be empirical evidence and in accordance with scientific method • However, “Negative” research is also very useful (Ioannidis. 2005) • Empirical evidence is a source of knowledge • John P.A Ioannidis. Why most published research findings are false. PLoS Medicine. August 2005; acquired by means of observation or vol2;Issue 8. p0696-0701 experimentation • Ref: http://en.wikipedia.org/wiki/Evidence • http://en.wikipedia.org/wiki/Scientific_evidence 15 16 • http://en.wikipedia.org/wiki/Empirical_evidence Scientific evidence Absence of evidence is not evidence of absence • By convention a P value greater than 5% (P>0.05) is called “not • As Bradford Hill (1965) said nearly 50 years ago: significant.” – “All scientific work is incomplete—whether it be • Randomised controlled clinical trials that do not show a observational or experimental. significant difference between the treatments being compared are often called “negative.” • All scientific work is liable to be upset or modified by advancing knowledge. • This term wrongly implies that the study has shown that there is no difference, whereas • That does not confer upon us a freedom to – usually all that has been shown is an absence of evidence of – ignore the knowledge we already have, or a difference. – to postpone the action that it appears to demand at a • These are quite different statements. given time.” • Source: Douglas G Altman, J Martin Bland. Statistics notes: Absence of evidence is not evidence of absence. BMJ 1995;311:485 at http://www.bmj.com/content/311/7003/485 • Bradford Hill A. 1965. The environment and disease: association or causation? Proc R Soc Med 58:295–300. 17 19 Sibu Research Seminar 2013 3 Datu Dr Andrew Kiyu 24/6/2013 PPV of Research Findings for Various Combinations of Power (1 − β), Ratio of Why Most Published Research Findings True to Not-True Relationships (R), and Bias (u) Are False by John P. A. Ioannidis (2005) 1 − β R u Practical Example PPV 0.80 1:1 0.10 Adequately powered RCT with little bias and 1:1 pre- 0.85 • The probability that a research claim is true may study odds depend on: 0.95 2:1 0.30 Confirmatory meta-analysis of good quality RCTs 0.85 1. study power and bias, 0.80 1:3 0.40 Meta-analysis of small inconclusive studies 0.41 0.20 1:5 0.20 Underpowered, but well-performed phase I/II RCT 0.23 2. the number of other studies on the same 0.20 1:5 0.80 Underpowered, poorly performed phase I/II RCT 0.17 question, 0.80 1:10 0.30 Adequately powered exploratory epidemiological study 0.20 • and, most importantly, 0.20 1:10 0.30 Underpowered exploratory epidemiological study 0.12 0.20 1:1,000 0.80 Discovery-oriented exploratory research with massive 0.0010 3. the ratio of true-to-no-relationships among the testing relationships probed in each scientific field. 0.20 1:1,000 0.20 As in previous example, but with more limited bias (more 0.0015 standardized) – Ioannidis JPA (2005) Why most published research findings are false. PLoS Med 2(8): e124. R = the ratio of the number of “true relationships” to “no relationships” among those tested in the field U = the proportion of probed analyses that would not have been “research findings,” but nevertheless end up presented and reported as such, because of bias The estimated PPVs (positive predictive values) are derived assuming α = 0.05 for a single study.