Any Threat to External Validity (The Extent to Which the Result Will Apply Widely)?

CRITICAL APPRAISAL WORKSHEET

The 8 key questions In detail Does this affect validity? 1a. What question did the researchers About the magnitude of health problem? Any differences between the two want to answer? questions? About the impact of a therapy of some sort?

N About whether a specific agent causes a O I health problem? T S E U

Q About the accuracy of a diagnostic test?

R About the prognosis of a disease? A E S E R

1b. What is the research question E

H actually posed in the paper? T

CA_Worksheet_2013_UNSW Page 1 The 8 key questions In detail Does this affect validity? 2a. What is the best study type to Why? Any problem with the study type the answer this question? researchers used?

RCT? Cohort? Case-control? Descriptive? Other? Y D U T S

N 2b. What study type was used? Why? G I How useful are the results likely to be? S E

D RCT? Cohort? Case-control? Descriptive? Other?

CA_Worksheet_2013_UNSW Page 2 The 8 key questions In detail Does this affect validity? 3a. Who are the authors interested in, Are the people in the study good Any threat to external validity (the extent and who actually were enrolled in the representatives of the population of patients to which the result will apply study? the authors were interested in? widely)? N O I T A C O L L A

T 3b. If the subjects were assigned to Was allocation random? Was allocation made N

E groups, how was this done? after the decision to enter the trial? M I Any threat to internal validity (the T

U validity or correctness of the study R C

E methods)? R

CA_Worksheet_2013_UNSW Page 3 The 8 key questions In detail Does this affect validity? 4a. Who are the authors / researchers? Check out the authors’ names, qualifications, Any threat to internal validity? where they work and what organisations they are connected to.

4b. Who is funding the research? Where did the research funding come from? G N

I Health Care Funding/? Direct Government D

N Grant? Pharmaceutical company? Private? U F

Or a mixture of these? D N A

S C I H T E

. 4 4c. Who is publishing this? Which journal is this study published in? Is it peer reviewed?

CA_Worksheet_2013_UNSW Page 4 The 8 key questions In detail Does this affect validity? 5a. What exactly is the factor being Is this new drug / test / therapy available to Any threat to usefulness? studied? E.g. the new drug (generic your patient? name, dose and route of administration), or the risk factor for a disease.

Could there be any problem with how it was Any threat to making sure your patient measured? gets the correct dose? S E

M 5b. What exactly is the main outcome Are these the best outcomes to use? Any Any threat to external validity? O

C being used? important ones missing? T U O

A Could there be any problem with how they

T were measured? A D

. 5 5c. What are the main confounders the How did they deal with them and were they Any threat to internal validity? authors have identified? dealt with adequately?

Any important confounders missing? Any threat to external validity?

CA_Worksheet_2013_UNSW Page 5 The 8 key questions In detail Does this affect validity? 6a. What is the main result given? Does Is it reasonable to accept these results are Is the study conclusive or inconclusive in it make sense when you think of what it not due to chance? answering the question in 1 above? means to your patients?

Is the difference reported likely to be important to your patients? S T L U S E R

. 6 6b. Are confidence intervals given? If the Did the study have enough people in it to get study does not give a ‘significant’ result, a reliable answer? is the power given?

CA_Worksheet_2013_UNSW Page 6 The 8 key questions In detail Does this affect validity? 7. What is the conclusion of the authors? Have the authors correctly interpreted the Was the study type used the best results? possible to answer this question and if not what would have been better?

Are the conclusions reasonable and justified by the data? S N O I S U

L Have they considered the limitations of the C

N study (including possible confounders)? O C

CA_Worksheet_2013_UNSW Page 7 The 8 key questions In detail Does this affect validity? 8a. Is the answer useful to answer your What is the ARR (absolute risk reduction) and What is your overall judgement of this question? hence the NNT value (Number needed to study re validity and its usefulness to you treat)? http://www.cebm.net/index.aspx? and to the patient or population you o=1044 have in mind? O C I P

B 8b. How will you apply it back to your

G patient / population? N I Y L P P A

A/Prof Mike Bennett_Dr Rachel Thompson_UNSW Medicine_2006 / edited 27.09.11, 14.05.12, 15.07.13

CA_Worksheet_2013_UNSW Page 8