StudyStudy DesignDesign forfor ChemopreventionChemoprevention
CancerCancer Epidemiology,Epidemiology, PreventionPrevention andand ControlControl WorkshopWorkshop Shanghai,Shanghai, MarchMarch 12,12, 20082008
I.I. INTRODUCTIONINTRODUCTION
ExperimentalExperimental studiesstudies areare conductedconducted toto assessassess thethe effecteffect ofof aa treatmenttreatment usingusing aa drug,drug, oror interventionintervention usingusing aa preventivepreventive agent,agent, etc.etc. •• TreatmentTreatment •• PreventionPrevention •• EarlyEarly detection/screeningdetection/screening •• DiagnosticDiagnostic •• QualityQuality ofof life/supportivelife/supportive carecare A.A. Defined:Defined:
•• AA studystudy designdesign inin whichwhich thethe investigatorinvestigator activelyactively controlscontrols whowho isis exposedexposed andand whowho isis not.not. SubjectsSubjects areare randomlyrandomly assignedassigned toto variousvarious treatmenttreatment groupsgroups andand followedfollowed toto observeobserve outcomes.outcomes.
ComparisonComparison ofof TwoTwo StudyStudy DesignsDesigns
StratifiedStratified RandomizationRandomization B.B. ExperimentalExperimental studiesstudies comparedcompared toto cohortcohort studiesstudies 1.1. SimilaritiesSimilarities betweenbetween cohortcohort andand experimentalexperimental studies.studies. Both:Both: a.a. SubjectsSubjects mustmust bebe freefree ofof thethe outcomeoutcome atat thethe startstart ofof thethe study.study. b.b. PeoplePeople areare groupedgrouped intointo “exposed”/“exposed”/ “not“not exposed”exposed” categories.categories. c.c. GroupsGroups areare followedfollowed forfor aa periodperiod ofof timetime toto determinedetermine outcome.outcome. d.d. YieldYield incidenceincidence datadata soso allowallow thethe calculationcalculation ofof riskrisk andand relatedrelated measures.measures. e.e. SusceptibleSusceptible toto lostlost--toto--followfollow--upup bias.bias. B.B. ExperimentalExperimental studiesstudies comparedcompared toto cohortcohort studiesstudies 2.2. DifferencesDifferences betweenbetween cohortcohort andand experimentalexperimental studies:studies: a. Experimental studies involve active manipulation of exposure (treatment/alternative treatment), whereas in cohort studies, the investigator must merely observe the effect of exposure. b. Random allocation (or randomization) is an essential part of a good experimental study. Not possible in a cohort. c. Ethical issues often a major issue in experimental epidemiological studies. d. Compliance with study protocol is an important concern in experimental studies. C.C. RandomRandom allocation/allocation/ randomizationrandomization 1.1. Defined:Defined: AA procedureprocedure forfor assigningassigning patientspatients toto experimentalexperimental treatmenttreatment andand otherother treatmenttreatment groupsgroups soso thatthat chancechance alonealone isis responsibleresponsible forfor thethe groupgroup assignment…eachassignment…each subjectsubject hashas anan equalequal chancechance ofof beingbeing inin anyany ofof thethe treatmenttreatment groups.groups. 2.2. PurposePurpose ofof randomization:randomization: ToTo (attempt(attempt to)to) assureassure comparabilitycomparability ofof thethe studystudy groupsgroups withwith respectrespect toto factorsfactors whichwhich maymay bebe relatedrelated toto outcome.outcome. C.C. RandomRandom allocation/allocation/ randomizationrandomization 3.3. IMPORTANT:IMPORTANT: RandomizationRandomization isis donedone afterafter informedinformed consentconsent isis obtained!!!obtained!!! C.C. RandomRandom allocation/allocation/ randomizationrandomization 4.4. DoDo NotNot ConfuseConfuse RandomRandom AllocationAllocation withwith RandomRandom Selection!!!Selection!!!
RandomRandom selectionselection ofof subjects:subjects: A procedure for selecting subjects so that each has the same chance of being included in the study. When we can’t afford to use all possible subjects in the source population. Purpose:Purpose: To assure representativeness of subjects (of source pop.) WhenWhen used?used? In any type of study design where a sample of the population is being selected.
C.C. RandomRandom allocation/allocation/ randomizationrandomization 5.5. RandomizationRandomization doesdoes notnot guaranteeguarantee similaritysimilarity ofof groupsgroups D.D. UsesUses ofof experimentalexperimental studystudy design:design: 1.1. EvaluateEvaluate benefitsbenefits ofof anan intervention:intervention: •• TherapeuticTherapeutic •• PreventivePreventive
2.2. ConfirmConfirm etiologicetiologic relationshiprelationship II.II. TWOTWO MAJORMAJOR TYPESTYPES OFOF EXPERIEMENTALEXPERIEMENTAL STUDIES:STUDIES: ClinicalClinical andand communitycommunity trialstrials A.A. RandomizedRandomized clinicalclinical trialtrial (RCT):(RCT):
1. Defined (phase III): An experimental study where the effectiveness of the intervention is being tested on individuals. Phase I trials • How does the agent affect the human body? • What dosage is safe? Phase II trials • Does the agent or intervention have an effect on the disease? Phase III trials • Is the new agent or intervention (or new use of a treatment) better than the standard? • Participants have an equal chance to be assigned to one of two or more groups A.A. RandomizedRandomized clinicalclinical trialtrial (RCT):(RCT):
2.2. ClinicalClinical trialstrials areare conductedconducted forfor thethe treatment/preventiontreatment/prevention ofof bothboth infectiousinfectious diseasesdiseases andand chronicchronic diseases:diseases: a.a. InfectiousInfectious diseases:diseases: fieldfield trialstrials (often(often refersrefers toto vaccinevaccine trials)trials) b.b. ChronicChronic diseases:diseases: e.g.e.g. Women’sWomen’s HealthHealth InitiativeInitiative (WHI):(WHI): B.4.B.4. RecruitRecruit studystudy Population:Population: EffectivenessEffectiveness ofof InterventionIntervention From: Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of the estrogen plus progestin in healthy postmenopausal women. Principal results from the Women’s Health Initiative Randomized Controlled Trial.JAMA.2002;288:321-33 B.B. CommunityCommunity trialtrial
1.1. Defined:Defined: AnAn experimentalexperimental studystudy wherewhere thethe effectivenesseffectiveness ofof anan interventionintervention isis testedtested onon aa community.community. 2.2. ExampleExample ofof aa communitycommunity trial:trial: fluoridationfluoridation ofof water.water. FluoridationFluoridation ofof WaterWater
B.B. CommunityCommunity trialtrial
3.3. ProblemsProblems ofof conductingconducting communitycommunity trials:trials: a.a. ObtainingObtaining anan appropriateappropriate controlcontrol group:group: 1) Same community before and after intervention 2) A control community: similar to experimental community with respect to possible confounders. b.b. OtherOther problems:problems: 1) It is hard to get individual’s informed concent 2) Intervention not at individuals level 3) Collaboration of communities III.III. STEPSSTEPS ININ CONDUCTINGCONDUCTING RANDOMIZEDRANDOMIZED CLINICALCLINICAL TRIALSTRIALS DiagramDiagram ofof aa designdesign flow:flow: B.1.B.1. StepsSteps
•• 1.1. SpecifySpecify HypothesisHypothesis B.2.B.2. StepsSteps
•• 2.2. SpecifySpecify targettarget andand sourcesource ofof populations:populations: B.3B.3 DefineDefine endpoints/possibleendpoints/possible sideside effectseffects ofof intervention:intervention: B.5.B.5. ObtainObtain informedinformed consentconsent fromfrom thosethose willingwilling toto participate:participate: 1.1. InformedInformed consent:consent: An agreement (signed) that the subject understands the benefits and risks of the study. 2.2. HumanHuman SubjectsSubjects ProtectionProtection Committees:Committees: act as watchdogs…review research applications the be sure they comply with issues pertaining to protection of human subjects. To receive research money from NIH (and other agencies), these committees must approve grand proposals. Require that all subjects must read and sign “Informed Consent” form.
B.6B.6 RandomizeRandomize toto treatmenttreatment
1.1. ExperimentalExperimental treatment.treatment. 2.2. AlternativeAlternative treatmenttreatment (“controls”):(“controls”): a.a. TheThe currentcurrent standardstandard treatment:treatment: b.b. Placebo:Placebo: 1) Defined: An inert substance prepared to look as similar as possible to the experimental treatment. 2) Placebo effect: Am improvement on health, symptoms, due to fact of being treated…and not due to the treatment! RandomizationRandomization ThreeThree advantagesadvantages ofof thethe randomizedrandomized design:design: •• RandomizationRandomization removesremoves thethe potentialpotential ofof biasbias inin thethe allocationallocation ofof subjectssubjects toto thethe interventionintervention groupgroup oror toto thethe controlcontrol group;group;
•• RandomizationRandomization tendstends toto produceproduce comparablecomparable groups;groups; thatthat is,is, thethe measuredmeasured oror unknownunknown prognosticprognostic factorsfactors andand otherother characteristicscharacteristics ofof thethe subjectssubjects atat thethe timetime ofof randomizationrandomization willwill be,be, onon thethe average,average, evevenlyenly balancedbalanced betweenbetween thethe interventionintervention andand controlcontrol groups;groups;
•• TheThe internalinternal validityvalidity ofof statisticalstatistical teststests ofof significancesignificance isis guaranteed.guaranteed. B.7.B.7. BlindnessBlindness
•• FundamentalFundamental PointPoint:: ToTo avoidavoid potentialpotential problemsproblems ofof biasbias duringduring datadata collectioncollection andand assessmentassessment a clinical trial Ideally wewe shouldshould havehave aa doubledouble--blindblind design.design. InIn studiesstudies wherewhere suchsuch aa designdesign isis impossible,impossible, aa singlesingle--blindblind approachapproach andand otherother measuresmeasures toto reducereduce potentialpotential biasbias areare favored.favored.
•• UnblindedUnblinded:: InIn anan unblindedunblinded oror openopen trial,trial, bothboth thethe subjectsubject andand thethe investigatorinvestigator knowknow toto whichwhich interventionintervention thethe subjectsubject hashas beenbeen assigned.assigned. TheThe studiesstudies involvinginvolving mostmost surgicalsurgical procedures,procedures, changeschanges inin lifelife stylestyle (eating(eating habits,habits, exercise,exercise, smoking)smoking) oror learninglearning techniquestechniques cancan bebe conductedconducted onlyonly inin thisthis manner.manner. TheThe advantageadvantage isis thatthat itit isis usuallyusually simplersimpler toto carrycarry outout andand thethe disadvantagedisadvantage isis thethe possibilitypossibility ofof bias.bias. B.7.B.7. BlindnessBlindness
•• SingleSingle--BlindBlind:: Patient does not know which treatment group he/she is in. OnlyOnly thethe investigatorinvestigator isis awareaware ofof whichwhich interventionintervention eacheach subjectsubject isis receiving.receiving. TheThe advantageadvantage andand disadvantagedisadvantage areare similarsimilar toto thosethose ofof unblindedunblinded trials.trials.
•• DoubleDouble--BlindBlind:: NeitherNeither thethe subjectssubjects nornor thethe investigatorsinvestigators responsibleresponsible forfor followingfollowing thethe subjectssubjects knowknow whichwhich interventionintervention thethe subjectsubject hashas beenbeen assigned.assigned. SuchSuch designsdesigns areare usuallyusually restrictedrestricted toto trialstrials ofof drugdrug efficacy.efficacy. TheThe advantageadvantage isis thatthat thethe riskrisk ofof biasbias isis reduced.reduced. TheThe disadvantagedisadvantage isis thatthat certaincertain responsibilities,responsibilities, whichwhich inin openopen oror singlesingle--blindblind studiesstudies couldcould bebe accomplishedaccomplished byby thethe investigators,investigators, mustmust bebe takentaken overover byby othersothers inin ordeorderr toto maintainmaintain thethe blindness.blindness. B.7.B.7. BlindnessBlindness
•• TripleTriple--BlindBlind:: NeitherNeither thethe subjects,subjects, nornor thethe investigators,investigators, nornor thethe committeecommittee monitoringmonitoring responseresponse variablesvariables isis toldtold thethe identityidentity ofof thethe groups.groups. TheThe theoreticaltheoretical advantageadvantage isis toto allowallow thethe monitoringmonitoring committeecommittee toto evaluateevaluate thethe responseresponse variablevariable resultsresults moremore objectively.objectively. TheThe disadvantagedisadvantage isis thatthat inin aa trialtrial wherewhere thethe monitoringmonitoring committeecommittee hashas anan ethicalethical responsibilityresponsibility toto ensureensure subjectsubject safety,safety, suchsuch aa designdesign maymay bebe counterproductive.counterproductive. BlindnessBlindness B.7.B.7. BlindingBlinding (cont.)(cont.) b.b. PurposePurpose ofof blinding:blinding: To prevent biases in assessing outcome, which may be influenced by knowledge of treatment group. c.c. BlindingBlinding isis mostmost importantimportant whenwhen d.d. BlindingBlinding isis lessless importantimportant whenwhen e.e. NotNot alwaysalways possiblepossible toto blindblind subjectssubjects and/orand/or investigatorsinvestigators B.8B.8 FollowFollow--upup studystudy groupsgroups forfor outcomesoutcomes B.9.B.9. AnalyzeAnalyze results:results: UseUse “Intention“Intention toto treat”treat” analysesanalyses 1.1. Defined:Defined: DataData areare analyzedanalyzed soso thatthat subjectssubjects remainremain inin groupsgroups asas originallyoriginally assigned…evenassigned…even ifif subjectssubjects dodo notnot complycomply oror changechange treatmentstreatments onon theirtheir own.own. ThisThis meansmeans that,that, eveneven ifif thethe dayday afterafter aa subjectsubject waswas assignedassigned toto thethe controlcontrol group,group, hehe startsstarts thethe experimentalexperimental treatmenttreatment onon hishis own…theown…the outcomeoutcome forfor thatthat patientpatient isis analyzedanalyzed asas ifif hehe werewere stillstill inin thethe controlcontrol group!group! …..In…..In short:short: “Once“Once randomized,randomized, alwaysalways analyzed.”analyzed.” B.9.B.9. AnalyzeAnalyze results:results: UseUse “Intention“Intention toto treat”treat” analysesanalyses 2.2. WhyWhy wewe mustmust useuse intentionintention toto treattreat analyses:analyses: a. The results reflect what happens in the real world when the treatment is offered. b. The trial concerns whether the offering of a new treatment is more effective. c. Non-compliers are often different with respect to outcome, so likely to be less bias in keeping them in original groups than analyzing data in other ways. d. Though final results must reflect intention-to-treat analysis, investigators should look at compliers and non-compliers and their possible effect on the results. B.10.B.10. EstablishEstablish proceduresprocedures forfor terminatingterminating thethe trialtrial andand informinginforming subjectssubjects ofof results.results. ToTo ProtectProtect thethe welfarewelfare ofof subjects,subjects, datadata mustmust bebe monitoredmonitored regularlyregularly andand analyzedanalyzed forfor anyany obviousobvious benefitsbenefits oror clearclear risksrisks toto subjects.subjects. IfIf eithereither occurs,occurs, thethe trialtrial mustmust bebe stoppedstopped andand appropriateappropriate actionaction takentaken (if(if therethere isis aa benefit,benefit, thenthen controlscontrols mustmust bebe offeredoffered newnew treatment.treatment. IfIf aa risk,risk, experimentalsexperimentals mustmust bebe takentaken ofof experexper.. treatment.treatment. B.10.B.10. EstablishEstablish proceduresprocedures forfor terminatingterminating thethe trialtrial andand informinginforming subjectssubjects ofof results.results.
Caveat:Caveat: Small,Small, transienttransient differencesdifferences betweenbetween thethe groupsgroups maymay bebe observedobserved earlyearly inin thethe trial.trial. SoSo therethere mustmust bebe aa balancebalance betweenbetween aa longlong--enoughenough followfollow--upup toto seesee whetherwhether truetrue benefitsbenefits occuroccur vs.vs. deprivingdepriving thethe controlscontrols ofof aa betterbetter treatment.treatment. IV.IV. SOMESOME MEASURESMEASURES OFOF EFFECTEFFECT ININ EXPERIMENTALEXPERIMENTAL STUDIESSTUDIES A.A. RiskRisk ratio:ratio:
IICC // IITT B.B. RiskRisk difference:difference:
IICC -- IITT C.C. Efficacy:Efficacy:
IcIc –– ItIt ------xx 100100 IcIc D.D. Example:Example:
849849 workingworking adultsadults betweenbetween thethe agesages ofof 1818--6464 tooktook partpart inin aa doubledouble--blind,blind, placeboplacebo--controlled,controlled, randomizedrandomized trialtrial ofof thethe effecteffect ofof influenzainfluenza vaccinationvaccination onon threethree outcomes:outcomes: upperupper respiratoryrespiratory illnessillness (URI),(URI), absenteeismabsenteeism fromfrom workwork duedue toto URI,URI, andand visitsvisits toto doctorsdoctors forfor URI.URI. BaselineBaseline characteristicscharacteristics werewere comparedcompared toto assureassure comparabilitycomparability ofof thethe twotwo groups.groups. TheThe subjectssubjects werewere followedfollowed fromfrom Dec.Dec. 1,1, 1994,1994, toto MarchMarch 31,31, 1995.1995. (Nichol,(Nichol, K.L.K.L. etet al.al. NN EnglEngl JJ MedMed 1995;1995; 333:333: 889889--93.)93.) D.D. ExamplesExamples
•• a.a. TheThe followingfollowing tablestables showsshows thethe baselinebaseline characteristicscharacteristics ofof thethe twotwo groups:groups: doesdoes thethe randomizationrandomization appearappear toto havehave beenbeen successful,successful, thatthat is,is, dodo thethe twotwo groupsgroups appearappear toto bebe similarsimilar forfor thosethose variables?variables? BaselineBaseline AssessmentAssessment
FundamentalFundamental PointPoint:: RelevantRelevant baselinebaseline datadata shouldshould bebe measuredmeasured inin allall studystudy subjectssubjects beforebefore thethe startstart ofof thethe study.study.
•• BaselineBaseline datadata includeinclude riskrisk factors,factors, prognosticprognostic factors,factors, demographicdemographic andand socioeconomicsocioeconomic factors,factors, andand medicalmedical history.history. BaselineBaseline assessmentassessment cancan bebe usedused toto assessassess comparabilitycomparability ofof thethe studystudy groups.groups. AlthoughAlthough thethe randomizationrandomization onon thethe averageaverage producesproduces balancebalance betweenbetween comparisoncomparison groups,groups, itit doesdoes notnot guaranteeguarantee balancebalance inin anyany specificspecific trial.trial. IfIf baselinebaseline factorsfactors areare notnot balanced,balanced, statisticastatisticall adjustmentadjustment methodsmethods suchsuch asas stratifiedstratified analysisanalysis andand multivariatemultivariate analysisanalysis cancan bebe used.used. BaselineBaseline AssessmentAssessment D.D. ExamplesExamples
•• b.b. OneOne concernconcern inin anyany experimentalexperimental study,study, eveneven ifif subjectssubjects areare blinded,blinded, isis whetherwhether subjectssubjects cancan telltell whetherwhether theythey areare inin thethe treatmenttreatment oror controlcontrol group.group. OneOne wayway toto determinedetermine isis toto askask thethe subjectsubject whichwhich groupgroup theythey thinkthink theythey areare in.in. InIn thethe aboveabove study,study, 60%60% ofof thethe placeboplacebo andand 54%54% ofof thethe vaccinevaccine recipientsrecipients correctlycorrectly identifiedidentified theirtheir groupgroup (slightly(slightly betterbetter thanthan chancechance along).along). D.D. ExamplesExamples
•• c.c. AnotherAnother wayway toto looklook atat thisthis isis toto seesee ifif thethe twotwo groupsgroups differdiffer inin reportingreporting sideside effects.effects. TableTable 22 showsshows thethe sideside effectseffects reportedreported byby eacheach ofof thethe groups.groups. DoesDoes itit looklook asas ifif anyany ofof thethe sideside effectseffects nightnight givegive awayaway aa subject’ssubject’s status?status?
D.D. Example:Example:
d.d. TableTable 33 summarizessummarizes thethe datadata regardingregarding thethe threethree outcomes,outcomes, usingusing ratesrates perper 100100 subjects.subjects. ForFor “Episodes“Episodes ofof UPI”,UPI”, calculatecalculate thethe threethree measuresmeasures ofof effect:effect:
•• RiskRisk ratio:ratio: •• RiskRisk differencedifference •• EfficacyEfficacy (or(or “Vaccine“Vaccine Effectiveness”)Effectiveness”) D.D. Example:Example: From: Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. Principal results from the Women’s Health Initiative Randomized Controlled Trial.JAMA.2002;288:321-33. MajorMajor potentialpotential biasesbiases inin experimentalexperimental studies:studies: •• LostLost--toto--followfollow--upup
•• Compliance/adherenceCompliance/adherence toto study’sstudy’s protocol,protocol, e.g.,e.g., WHI:WHI: ReasonsReasons forfor nonnon--compliance:compliance:
1.1. MisunderstandingMisunderstanding ofof instructions.instructions. 2.2. InconvenienceInconvenience ofof participation.participation. 3.3. SideSide effectseffects ofof treatment.treatment. 4.4. CostCost ofof participation.participation. 5.5. Forgetfulness.Forgetfulness. 6.6. DisappointmentDisappointment withwith results.results. 7.7. PreferencePreference forfor anotheranother treatment.treatment. WaysWays toto improveimprove compliance:compliance:
1. Select motivated persons. 2. Pretest ability and willingness of participants to comply. 3. Provide simple and lucid instructions to subjects. 4. Offer incentives to comply (e.g., no charge for therapeutic intervention and associated examinations). 5. Provide positive reinforcements to subjects for adherence to treatment regimen. 6. Maintain frequent contract with participants and remind them about importance of adherence to the regimen. 7. Measure adherence through pill counts or sampling of biologic specimens. 8. Limit duration of intervention. SampleSample Size:Size:
FundamentalFundamental PointPoint:: ClinicalClinical trialstrials shouldshould havehave sufficientsufficient statisticalstatistical powerpower toto detectdetect differencesdifferences betweenbetween groupsgroups consideredconsidered toto bebe ofof clinicalclinical interest.interest. Therefore,Therefore, calculationcalculation ofof samplesample sizesize withwith provisionprovision forfor adequateadequate levelslevels ofof significancesignificance andand powerpower isis anan essentialessential partpart ofof planning.planning.
ClinicalClinical trialstrials tendtend toto bebe small.small. EvenEven ifif therethere isis aa realreal differencedifference inin twotwo treatments,treatments, thethe differencedifference maymay notnot bebe statisticallystatistically significantsignificant becausebecause ofof thethe smallsmall numbers.numbers. ThatThat is,is, thethe studystudy diddid notnot havehave thethe powerpower toto detectdetect (statistically)(statistically) aa realreal difference.difference. SampleSample sizesize •• FalseFalse positivepositive (alpha(alpha--level,level, oror TypeType II error).error). TheThe alphaalpha-- levellevel usedused andand acceptedaccepted traditionallytraditionally areare 0.010.01 oror 0.05.0.05. TheThe smallersmaller thethe levellevel ofof alpha,alpha, thethe largerlarger thethe samplesample size.size. •• FalseFalse negativenegative (beta(beta--level,level, oror TypeType IIII error).error). (1(1--beta)beta) isis calledcalled thethe powerpower ofof thethe study.study. InvestigatorInvestigator likelike toto havehave aa powerpower ofof aroundaround 0.900.90 oror 0.950.95 whenwhen planningplanning aa study,study, whichwhich meansmeans thatthat therethere havehave aa 90%90% oror 95%95% chancechance ofof findingfinding aa statisticallystatistically significantsignificant differencedifference betweenbetween studystudy andand controlcontrol groups.groups. •• TheThe differencedifference betweenbetween studystudy andand controlcontrol groupsgroups (delta).(delta). TwoTwo factorsfactors needneed toto bebe consideredconsidered here:here: oneone isis whatwhat differencedifference isis clinicallyclinically important,important, andand thethe anotheranother isis whatwhat isis thethe differencedifference reportedreported byby previousprevious studies.studies. V.V. TheThe ValidityValidity ofof ResultsResults ofof EpidemiologicEpidemiologic StudiesStudies A.A. InternalInternal ValidityValidity
1.1. Defined:Defined:
TheThe truthtruth oror accuracyaccuracy ofof resultsresults fromfrom aa studystudy withwith respectrespect toto itsits defineddefined sourcesource populationpopulation 2.2. FactorsFactors affectingaffecting internalinternal validity:validity:
•• InformationInformation bias:bias: distortiondistortion ofof truetrue effectseffects duedue toto systematicsystematic errorserrors inin collectioncollection ofof datadata onon exposureexposure OROR outcomeoutcome
•• SelectionSelection bias:bias: distortiondistortion ofof truetrue effectseffects duedue toto errorerror inin selectionselection ofof subjectssubjects
•• Confounding:Confounding: distortiondistortion ofof truetrue effectseffects fromfrom otherother riskrisk factorsfactors oror exposuresexposures extraneousextraneous toto thethe studystudy 3.3. AlgorithmAlgorithm forfor assessingassessing flawsflaws inin anan experimentalexperimental study:study: B.B. ExternalExternal ValidityValidity
1.1. Defined:Defined:
ApplicabilityApplicability ofof studystudy resultsresults toto otherother populationspopulations ((ieie,, toto specifiedspecified targettarget oror referencereference population)population)
GeneralizabilityGeneralizability 2.2. FactorsFactors affectingaffecting externalexternal validity:validity:
a.a. DoesDoes thethe studystudy havehave internalinternal validity?validity? externalexternal validityvalidity isis notnot possiblepossible withoutwithout internalinternal ((ieie,, aa studystudy mustmust bebe internallyinternally validvalid firstfirst beforebefore itit cancan alsoalso bebe externallyexternally valid)valid)
a.a. IsIs thethe studystudy groupgroup representativerepresentative ofof thethe targettarget population?population? ifif thethe studystudy groupgroup isis nonnon--representative,representative, thenthen thethe studystudy isis notnot externallyexternally validvalid 3.3. AlgorithmAlgorithm forfor assessingassessing externalexternal validity:validity:
Is experimental population representative of the reference population?
YES NO No external validity problem Would reference population have same response to treatment You CAN extrapolate as experimental population?
YES NO No external validity problem External validity problem
You CAN extrapolate You CANNOT extrapolate