9Th EVIDENCE–BASED CLINICAL PRACTICE WORKSHOP for Clinical Decision Making and Health Management

9th EVIDENCE–BASED CLINICAL PRACTICE WORKSHOP For Clinical Decision Making and Health Management

IDENTIFY – APPRAISE – INTERPRET– ELABORATE – IMPLEMENT

AMIL LIFESCIENCE

Suzana Alves da Silva Peter Wyer

Rio de Janeiro, 2015

Section III INTERPRETATION$OF$STUDY$RESULTS$

18 INTERPRETATION OF STUDY RESULTS

18.1 OBJECTIVES

! To introduce concepts for the interpretation of study results.

18.2 SUPPLEMENTS

Supplemental material is available at the workshop SIMPLE. Website under Teaching Tips, Videos and Users Guides.

Supplemental material to the understanding of topics related to this theme is also available under Supplements such combined endpoints and subgroup analysis.

18.3 MEASURES OF FREQUENCY, EFFECT, IMPACT AND PRECISION

In general, results can be categorized as measures of frequency, effect, impact and precision. Such measures are found in different types of studies and they are the most frequently used to display the results of clinical researches. The same study design can be used to answer questions under the domains of Therapy, Diagnosis, Prognosis and Harm, sometimes with subtle variations among the domains (Figure 3).

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 69 Janeiro. Sylabbus; 2015: 1-104

Figure 3. Measures of frequency, effect, impact and precision according to the action domains (therapy, diagnosis, prognosis and harm) and to the main types of individual studies.

RR: Relative risk HR: Harzard Ratio OR: Odds Ratio RD: Risk Difference LR: Likelihood ratio CI: Confidence Interval

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 70 Janeiro. Sylabbus; 2015: 1-104

18.4 INTERPRETATION OF FREQUENCY MEASURES

18.4.1 Absolute Risk

Absolute risk of an outcome means the percentage of patients that had the outcome during reseach follow up, in a predetermined period of time.

For example, if we followed 100 patients with severe aortic stenosis for over a year, and during this period 30 out of 100 experienced syncope, then the absolute risk of syncope in this population would be 30% in 1 year. If we followed another population of 100 patients similar to these, with similar severity of aortic stenosis, similar clinical treatment, but submitted to a surgical intervention and we observed that 20 of 100 experienced syncope, then the absolute risk of the population surgically treated would be 20%. This means that surgical intervention may have reduced the risk of syncope from 30% to 20%, ie, may have caused an impact of 10% in the absolute risk reduction of syncope.

The difference in absolute risk is an Measure widely used to calculate the number needed to tartar (NNT) for "screening" (NNS) or to cause harm (NNH). The major limitation of using the absolute risk difference is that it can not extrapolate to other populations indiscriminately since they vary in severity and characteristic. Such extrapolations are usually made based on the relative risk, as discussed below.

18.4.2 Pre and Post Test Probability and Decision Threshold

The pretest probability of a clinical condition was defined according to Bayes’ theorem.<> This theorem expresses how a subjective degree of belief should rationally change the probability of a fact. Based on this theorem we may say that pretest probability is the estimate of how likely is a condition of interest given the knowledge of the condition and its clinical manifestations and it is highly dependent on professional intuition upon request for testing.

None of todays available tests is able to give us 100% certainty regarding the presence or absence of a clinical condition. To deal with some degree of uncertainty is inherent to clinical practice and the clinician needs to know when to stop an investigation and when to start treatment regardless of any additional testing. This requires the explicit knowledge of the performance of the available tests and of the disease and the tacit knowledge that will enable the health professional to recognize and prioritize the tacit information that are being held by the patient and his/her relatives.

The diagnostic and therapeutic decision thresholds are represented in Figure 4 and they demarcates the two certainty zones. The zone to the left of the diagnostic threshold is the zone of the certainty of the absence of the condition. When the clinician is below this threshold means

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 71 Janeiro. Sylabbus; 2015: 1-104

that he believes so much that the patient does not have the condition of interest that a further test would be unable to further lower the pretest probability to a level more clinically significant. Likewise when the pre-test likelihood is above the therapeutic threshold, it means that the clinician believes so much that the patient in fact has the condition that a further test would be unable to further enlarge the pretest probability to a level more clinically significant.48

Patients with chest pain, for example, who have risk factors for cardiovascular events such as diabetes, dyslipidemia and smoking, are above the sixth decade of life, are male and have as symptomatic manifestation the presence of typical chest pain, have a high pre-test probability of coronary artery disease. Patients with these characteristcs do not need further investigation to confirm the presence of the disease and can be put under medical treatment and sent directly to coronary angiography if an invasive treatment is indicated.49 This is true because the tests that are available to investigate the presence of coronary artery disease are unable to change the pretest probability to a level that would change decision. According to Diamond and Forrester criteria, the pretest probability of coronary artery disease in patients with these characteristics is above 95%. In situations like this further testing becomes unnecessary and may become even harmful when the aim is solely and exclusively to confirm the presence of the condition.49

Several clinical decision rules have been studied and validated around the world to support the clinician in estimating the pretest probabilities of different conditions more accurately. However there are many clinical situations that can not count on any prediction rule and in these situations the health professional must rely on her/his knowledge of the disease, her/his professional experience and knowledge of the prevalence of that condition in her/his locality, when available.

Working with quartiles of 25%, 50% and 75% is a way to estimate low, moderate and high pretest probability, respectively, when there is no valid prediction rule for it. You may then ask whether the test will be able to modify this pretest probability into a posttest probability out of the uncertainty zone, if the result of the test comes positive or negative. This ability of the test to change the pretest probability into a post test probability that will be usefull for decision making, wheather the result of test is positive or negative, depends on its likelihood ratio, as will be discussed below.

The sustainability of the health system depends on the proper use of available resources in an efficient and safe manner to the patients. The overutilization of the huge arsenal of resources that are available is besides unsustainable, dangerous to patients and society.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 72 Janeiro. Sylabbus; 2015: 1-104

Figure 4. Diagnostic and thearpeutic thresholds

Pretest! Post!test! Odds! LR! Odds!

Absence! Zone!of!uncertainty! Presence!

! !

Diagnostic Threshold Therapeutic Threshold

18.5 INTERPRETATION OF EFFECT MEASURES

18.5.1 Likelihood Ratio

The assessment of the accuracy of a test or prediction rule for detecting a particular health condition depends on the understanding of the meaning of sensitivity, specificity and likelihood ratios and how they correlate with the pre- and post-test probability of the condition.

As shown in Table 6, the sensitivity of a test refers to the ability of the test to detect the patients who trully have the condition of interest (sickers) when the result of the test is positive, ie, the proportion of true positives when the disease is present.48 The specificity of a test refers to the ability of the test to detect the patients who truly do not have the clinical condition of interest (healthiers) when the test result is negative, ie, the proportion of true negatives when the disease is not presente.48

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 73 Janeiro. Sylabbus; 2015: 1-104

Table 6. Likelihood ratio calcs based on the proportion of true and false positives or negatives

% with the clinical % without the clinical condition* condition*

Test + %T(+) %F(+) LR(+) = %T+ / %F+

Test - %F(-) %T(-) LR(-) = %F- / %T-

Sens = %T(+) Spec = %T(-)

T+ = True positives; T- = True negatives; F+ = False positivs; F- = False negatives; Sens = Sensitivity; Spec = Specificity

(*) Definitive diagnosis is based on the reference criteria that was used in the cross-sectional study design. Some clinical conditions lack an specific reference criteria or “gold standard”. In these cases the follow up for a period of time long enough may be used to define the diagnosis.

A positive likelihood ratio [LR(+)] is a relation between the proportion of positive results in patients 48 with and without the clinical condition [%T(+) / %F(+)], when the result of the test is positive.

A negative likelihood ratio [LR(–)] is a relation between the proportion of negative results in patiets with and without the clinical condition of interest [%F(-) / %T(-)], when the result of the test is negative.48

Thus, if you look at Table 6, you may conclude that you only need to know the sensitivity and specificity of the test to get the LR calcs easily done.

LR(+) = Sensitivity / (100 – Specificity)

LR(–) = (100 – Sensitivity) / Specificity

The likelihood ratio of a test allow you to transform the pretest probability of the disease into a post test probability, and by doing so moving the observer out of the uncertainty zone of the disease to a certainty zone of the absence or presence of the clinical condition of main interest, once the result of the test is released. Therefore, the likelihood ratio of a test is important to find the post test probability of the disease based on its pretest probability (Figure 5).

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 74 Janeiro. Sylabbus; 2015: 1-104

Figure 5. Fagan nomogram to estimate the post test probability of a disease based on its pretest probability and on test likelihood ratios.

18.5.2 Relative risk

Relative Risk is a ratio of Absolute Risks, ie, the absolute risk of the exposure group over the control group (Table 7). For example, suppose that in patients with Acute Myocardial Infarction the Risk of a new infarction is around 5% if treated with primary angioplasty and it is around 8% if treated with thrombolytics. We may say then that the Relative Risk of Angioplasty over Trombolysis is 5%/8% = 0.625. We could say that angioplasty may reduce the risk of a new infarction by 37,5% (1 – 0.625 = 0.375) in relation to thrombolysis.

Relative Risks are usually used to demonstrate the effect of different interventios and they usually give the impression that the impact of the intervention is higher than it is in fact. Relative risks alone can not inform the impact of an intervention or exposure. Impact refers to absolute risk difference and to calculate it we need to know the baseline risk of the outcome in the population without the intervention or exposure. By applying the Relative Risk to the known baseline risk of the population of interest we may estimate the impact of the intervention or exposure in that specific population.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 75 Janeiro. Sylabbus; 2015: 1-104

Table 7. Relative risk calculation using the 2x2 table

N with the N without the outcome outcome

Exposure a b ARE = a/(a+b)

Control c d ARC = c/(c+d)

RR = ARI / ARC

ARI = Absolute risk in the exposure group; ARC = Absolute risk in the control group; RR = Relative risk

Thus, relative risk is always related to a baseline risk, which is extremely relevant to clinical practice. For example, if we take 3 population with different baseline risks, and submit these 3 population to the same type of intervention, we will observe that the impact of the intervention on the absolute risk reduction will vary from a population to another, but the relative risks will remain the same (Figure 6). This is a very important measure of effect and can be applied accross different population receiving the same intervention or submitted to the same exposure. This is the principal of meta-analysis, which pools together the results of different study populations (Figure 7).

Figure 6. Relative risk reduction versus absolute risk reduction in difference population receiving the same intervention.

RR: Relative risk; ARR: Absolute risk reduction; RRR: Relative risk reduction

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 76 Janeiro. Sylabbus; 2015: 1-104

Figure 7. Meta-analysis of the studies All-Sirius, Pache and Ravel, which assessed the effect of the pharmacological stents compared to the non pharmacological to reduce the risk of myocardial

Estudos RR % Peso (IC 95%)

All - Sirius 1,32(0,63-2,78) 48 Pache 1,40(0,45-4,35) 19,9 Ravel 1,72(0,75-3,95) 32,1

IC 95% 1,47 (0,89-241)

0,23 1 4,35 infarction in patients with obstructive coronary artery disease.

CI: Confidence Interval; RR: Relative risk

18.5.3 Harzard Ratio

Hazard Ratio has a meaning similar to Relative Risk, but includes the relative rate of outcomes over time instead of the relative rate of outcomes in one specific point in time. For didactic purposes we may say that Hazard Ratio is the average of the Relative Risks in each unit of time over the follow up (Figure 8).

Figure 8. Demonstration of Hazard Ratio calcultation based on the survival curves of two hypothetical groups of patients

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 77 Janeiro. Sylabbus; 2015: 1-104

18.5.4 Odds Ratio

The meanings of Odds and Risks are different. Odds refers to how many times a clinical event is more likely to occur than not occur, i.e., it is the ratio between the likelihood of occurance (P), over the likelihood of no occurance of the outcome (1 – L). May be mathematically represented as L / (1 – L).

If we throw a die, for example, the likelihood of obtaining the number 3 is 1/6 (likelihood of event happening) and the likelihood of not obtaining the number 3 is 5/6 (likelihood of event not happening). Therefore, the odds is (1/6) / (5/6) = 1/5. It means that it is 5 times more likely of not having the number 3 than having it as a result of throwing a die.

Likewise, when we are studying the likelihood of an outcome as a result of an intervention or exposure or presence of a clinical condition, we can also calculate the odds of that outcome (likelihood of happening / likelihood of not happening), as shown in Table 8 below.

Table 8. Odds ratio calculation using the 2x2 table

N with the N without the outcome outcome

Exposure a b OddsE = a/b

Control c d OddsC = c/d

OR = OddsE / OddsC

C = Control; E = Exposure, intervention or presence of a clinical condition; OR = Odds Ratio

Odds usually offers bigger numbers than risks and may overestimate the effect of an intervention in the eyes of an inattentive reader. Sometimes this type of measure is preferred by researchers who are conflicted by some interest and need to give more value to the results of their research.

However, there are situations when odds are preferred over risks and there are situations when odds and risks are similar and can be used interchangeably. In general terms, when the event rate in each group is lower than 10%, odds and risks look the same, thus, odds ratio in situations like this can be read and interpreted as risk ratio (relative risk). It can be easily illustrated by using the conversion of different risks into odds as described below:

Risk = 80% Odds = 80/20 = 4 Risk = 60% Odds = 60/40 = 1.5

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 78 Janeiro. Sylabbus; 2015: 1-104

Risk = 40% Odds = 40/60 = 0.67 Risk = 20% Odds = 20/80 = 0.25 Risk = 10% Odds = 10/90 = 0.11 Risk = 5% Odds = 05/95 = 0.05

18.6 INTERPRETATION OF IMPACT MEASURES

18.6.1 Risk difference or atributable risk

The absolute risk difference, also called attributable risk, is the difference between the risk of occurrence of the outcome in the exposure (intervention) and the non exposure (Control) groups. For example, if the risk of a new infarction in patients treated with thrombolytic agents is 8% and the risk of a new infarction in patients treated with primary angioplasty is 5% in the first month after the infarction, then the risk difference is 3% in 1 month.

These 3% means that if a patient similar to those studied is treated by angioplasty, her/his likelihood of a new event at 1 month will decrease by 3% in absolute terms. For public health this means that for every 100 pacientes treated by primary angioplasty instead of thrombolysis, 3 re- infarctions will be avoided in the time horizon of 1 month; 30 re-infarctions will be avoided for each 1 000 patients treated; 300 re-infarction avoided for each 10 000 treated and so on.

The difference in absolute risk is considered a measure of impact as it says the magnitude by which the health system may be impacted if that intervention is applied or if that exposure happens. For an individual patient it is important to note that despite the risk decrease with the angioplasty treatment, the patient continues to have a 5% likelihood of having a re-infarction at 30 days. Thus the intervention decreases the risk, but does not eliminate the risk and patients must be aware of that, aware of the gain with one intervention or another and instructed of what to do if new symptoms appear.

It is important to note that the risk difference observed in one population can not be generalized to other populations. To circumvent this problem typically we use a measure of effect that can be transported across different populations and applied on their baseline risks.

18.6.2 Number Need to Treat (NNT) and Number Need to Harm (NNH)

Number Need to Treat (NNT) means the number of patients that need to be treated so one single outcome can be avoided, in a prespecified period of time. Number Need to Harm (NND) means the number of patients that need to be exposed so one single adverse event related to the exposure can be caused, in a prespecified period of time.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 79 Janeiro. Sylabbus; 2015: 1-104

Both NNT and NNH are calculated from the Absolute Risk Difference. For example, if the absolute risk decreases by 3%, this means that for every 100 Patients treated, three events will be avoided. Therefore, by rule of three, we conclude that it is necessary to treat approximately 33 pacientes for 1 event to be avoided. The NNT is therefore 33. If the absolute risk of a harm increases by 2% with the intervention, so for every 100 pacientes treated, two will have an adverse event because of the intervention. Soon also using simple rule of three, for every 50 patients treated 1 will have a harm caused by the intervention. The NND is therefore 50.

How can we use NNT to estimate risk and benefit ratio?

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 80 Janeiro. Sylabbus; 2015: 1-104

19 APPLICABILITY OF STUDY RESULTS

19.1 OBJECTIVES

! To introduce the concepts of applicability of scientific evidence to clinical decision making ! To introduce the concept of directness of evidence taking into account the PICO answered by the scientific information compared to the PICO built to solve a clinical problem

19.2 INDIRECT EVIDENCE

Applicability of evidence is one of the most important topic in the decision making process and it starts with the interpretation of scientific information, integration of this information with the background knowledge of the disease and assessment of the exequibility of the application of that information in the surrounding circunstances. In this intial step, evidence is assessed according to the question that is being answered by the research against the question that was built based on the actual problem (Figure 9). The Applicability of the results based on critical appraisal of different studies will be discussed at the end of each chapter in the Section II of this book.

Figure 9. Assessment of evidence directness to the problem under question.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 81 Janeiro. Sylabbus; 2015: 1-104

Another chapter within applicability refers to the various aspects considered in the field of knowledge translation, such as delineating the problem, identifying needs, mapping barriers, developing strategies, and monitoring implementation refer to the field of implementation science. Section III of this book shows an example of some processes involved in the implementation of a clinical practice guideline.

Several initiatives arising from medical societies, government and non government led the development of guidelines in order to encourage and facilitate the use of evidence in decision making. However, what we often see in such guidelines are heterogeneous or non systematized approach to finding information and developing recommendations, frequently with enormous influence of funding sources. The elaboration of recommendations will alo be explored in the Section III of this book.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 82 Janeiro. Sylabbus; 2015: 1-104

References

1. Tan SY, Uyehara P. William Osler (1849-1919): medical educator and humanist. Singapore medical journal. 2009;50(11):1048-1049. 2. Sackett DL. Teaching critical appraisal. Journal of general internal medicine. 1990;5(3):272. 3. Sackett DL, Wennberg JE. Choosing the best research design for each question. BMJ. 1997;315(7123):1636. 4. Evidence-based medicine. A new approach to teaching the practice of medicine. JAMA : the journal of the American Medical Association. 1992;268(17):2420-2425. 5. Guyatt G. Evidence-based medicine (Editorial). ACP Journal Club. Annals of Internal Medicine. 1991;114(Suppl. 2):A16. 6. Haynes RB. What kind of evidence is it that Evidence-Based Medicine advocates want health care providers and consumers to pay attention to? BMC health services research. 2002;2:3. 7. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. Bmj. 1996;312(7023):71-72. 8. Tanenbaum SJ. What physicians know. The New England journal of medicine. 1993;329(17):1268-1271. 9. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. 1996. Clinical orthopaedics and related research. 2007;455:3-5. 10. Onate-Ocana LF, Ochoa-Carrillo FJ. The GRADE system for classification of the level of evidence and grade of recommendations in clinical guideline reports. Cirugia y cirujanos. 2009;77(5):417-419. 11. Schunemann HJ, Best D, Vist G, Oxman AD. Letters, numbers, symbols and words: how to communicate grades of evidence and recommendations. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 2003;169(7):677-680. 12. Glasziou P, Vandenbroucke JP, Chalmers I. Assessing the quality of research. BMJ. 2004;328(7430):39-41. 13. Barbui C, Dua T, van Ommeren M, et al. Challenges in developing evidence-based recommendations using the GRADE approach: the case of mental, neurological, and substance use disorders. PLoS medicine. 2010;7(8). 14. Owens DK, Lohr KN, Atkins D, et al. AHRQ series paper 5: grading the strength of a body of evidence when comparing medical interventions--agency for healthcare research and quality and the effective health-care program. Journal of clinical epidemiology. 2010;63(5):513-523. 15. Kunz R, Djulbegovic B, Schunemann HJ, Stanulla M, Muti P, Guyatt G. Misconceptions, challenges, uncertainty, and progress in guideline recommendations. Seminars in hematology. 2008;45(3):167-175. 16. Hitt J. Evidence-Based Medicine. THE YEAR IN IDEAS: A TO Z. 2001; http://www.nytimes.com/2001/12/09/magazine/the-year-in-ideas-a-to-z-evidence-based- medicine.html. Accessed Mar 29, 2012. 17. Godlee F. Milestones on the long road to knowledge. BMJ. 2007;334 Suppl 1:s2-3. 18. Sehon SR, Stanley DE. A philosophical analysis of the evidence-based medicine debate. BMC health services research. 2003;3(1):14. 19. Silva SA, Charon R, Wyer PC. The marriage of evidence and narrative: scientific nurturance within clinical practice. Journal of evaluation in clinical practice. 2011;17(4):585-593. 20. Silva SA, Wyer P. The Roadmap: a blueprint for evidence literacy within a Scientifically Informed Medical Practice and LEarning model. Journal of Person Centred Medicine. 2012:In press. 21. Fryback DG, Thornbury JR. The efficacy of diagnostic imaging. Medical decision making : an international journal of the Society for Medical Decision Making. 1991;11(2):88-94. 22. Silva SA, Wyer PC. The Roadmap: a blueprint for evidence literacy within a Scientifically Informed Medical Practice and Learning Model. European Journal of Person Centered Healthcare. 2013;3(1):53-68. 23. Hlatky MA, Greenland P, Arnett DK, et al. Criteria for evaluation of novel markers of cardiovascular risk: a scientific statement from the American Heart Association. Circulation. 2009;119(17):2408-2416.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 102 Janeiro. Sylabbus; 2015: 1-104

24. Methods Guide for Medical Test Reviews. 2010; http://www.effectivehealthcare.ahrq.gov/tasks/sites/ehc/assets/File/methods_guide_for_ medical_tests.pdf, 2014. 25. Guidance by type. NICE Guidance 2014; http://www.nice.org.uk/guidance/index.jsp?action=byType, 2014. 26. Schwartz MD, Valdimarsdottir HB, DeMarco TA, et al. Randomized trial of a decision aid for BRCA1/BRCA2 mutation carriers: impact on measures of decision making and satisfaction. Health psychology : official journal of the Division of Health Psychology, American Psychological Association. 2009;28(1):11-19. 27. Saadatmand S, Rutgers EJ, Tollenaar RA, et al. Breast density as indicator for the use of mammography or MRI to screen women with familial risk for breast cancer (FaMRIsc): a multicentre randomized controlled trial. BMC cancer. 2012;12:440. 28. Anderson DR, Kahn SR, Rodger MA, et al. Computed tomographic pulmonary angiography vs ventilation- perfusion lung scanning in patients with suspected pulmonary embolism: a randomized controlled trial. JAMA : the journal of the American Medical Association. 2007;298(23):2743-2753. 29. Hachamovitch R, Hayes SW, Friedman JD, Cohen I, Berman DS. Comparison of the short-term survival benefit associated with revascularization compared with medical therapy in patients with no prior coronary artery disease undergoing stress myocardial perfusion single photon emission computed tomography. Circulation. 2003;107(23):2900-2907. 30. Guyatt GH, Oxman AD, Vist GE, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. 31. Brozek JL, Akl EA, Compalati E, et al. Grading quality of evidence and strength of recommendations in clinical practice guidelines part 3 of 3. The GRADE approach to developing recommendations. Allergy. 2011;66(5):588-595. 32. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schunemann HJ. What is "quality of evidence" and why is it important to clinicians? BMJ. 2008;336(7651):995-998. 33. Conti R, Veenstra DL, Armstrong K, Lesko LJ, Grosse SD. Personalized medicine and genomics: challenges and opportunities in assessing effectiveness, cost-effectiveness, and future research priorities. Medical decision making : an international journal of the Society for Medical Decision Making. 2010;30(3):328-340. 34. Green RC, Roberts JS, Cupples LA, et al. Disclosure of APOE genotype for risk of Alzheimer's disease. The New England journal of medicine. 2009;361(3):245-254. 35. Briel M, Ferreira-Gonzalez I, You JJ, et al. Association between change in high density lipoprotein cholesterol and cardiovascular disease morbidity and mortality: systematic review and meta-regression analysis. BMJ. 2009;338:b92. 36. Charon R. Narrative Medicine: Attention, Representation, Affiliation. Narrative. 2005;13:261-269. 37. Charon R, Wyer P. Narrative evidence based medicine. Lancet. 2008;371(9609):296-297. 38. Epstein RM, Peters E. Beyond Information: Exploring Patient's Preferences. JAMA : the journal of the American Medical Association. 2009;302:195-197. 39. Freire P. Education for Critical Consciousness. New York: Continuum; 1974. 40. Richardson WS, Wilson MC, Nishikawa J, Hayward RS. The well-built clinical question: a key to evidence- based decisions. ACP journal club. 1995;123(3):A12-13. 41. Pfisterer M, Buser P, Rickli H, et al. BNP-guided vs symptom-guided heart failure therapy: the Trial of Intensified vs Standard Medical Therapy in Elderly Patients With Congestive Heart Failure (TIME-CHF) randomized trial. JAMA : the journal of the American Medical Association. 2009;301(4):383-392. 42. Ellis P. Research on the Comparative Effectiveness of Medical Treatments: Issues and Options for an Expanded Federal Role. 2007; http://www.cbo.gov/publication/41655. Accessed April 9, 2012. 43. Haynes B. Clinical Study Categories. 2011; http://www.ncbi.nlm.nih.gov/pubmed/clinical. Accessed April 9, 2012. 44. Pasternak T, Movshon JA, Merigan WH. Creation of direction selectivity in adult strobe-reared cats. Nature. 1981;292(5826):834-836. 45. Pasternak T, Merigan WH, Movshon JA. Motion mechanisms in strobe-reared cats: psychophysical and electrophysical measures. Acta Psychol (Amst). 1981;48(1-3):321-332.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 103 Janeiro. Sylabbus; 2015: 1-104

46. Melvill Jones G, Mandl G, Cynader M, Outerbridge JS. Eye oscillations in strobe reared cats. Brain Res. 1981;209(1):47-60. 47. Mandl G, Melvill Jones G, Cynader M. Adaptability of the vestibulo-ocular reflex to vision reversal in strobe reared cats. Brain Res. 1981;209(1):35-45. 48. Kopans DB. A strobe-sequenced device to facilitate the three-dimensional viewing of cross-sectional images. Radiology. 1980;135(3):780-781. 49. Skinner JS, Smeeth L, Kendall JM, Adams PC, Timmis A. NICE guidance. Chest pain of recent onset: assessment and diagnosis of recent onset chest pain or discomfort of suspected cardiac origin. Heart. 2010;96(12):974-978.

Silva, S. A. and Wyer, P. 9th Workshop on Evidence-Based Clinical Practice for Clinical Decision Making and Health Management. Rio de 104 Janeiro. Sylabbus; 2015: 1-104