sign in sign up
<<
Home , Random assignment

Chapter 10 – Lecture 10

Internal – Control through Experimental Design

1) Test the effects of IV on DV

2) Protects against threats to

Causation Experimental Design

 Highest Constraint

 Comparisons btw grps

 Random

 Random assignment

Infer Causality Experimental Design (5 characteristics)

1) One or more hypothesis

2) Includes at least 2 “levels” of IV

3) Random assignment

4) Procedures for testing hypothesis

5) Controls for major threats to internal validity Experimental Design

 Develop the problem statement  Define IV & DV  Develop research hypothesis  Identify a population of interest  Random sampling & Random assignment  Specify procedures (methods)  Anticipate threats to validity  Create controls  Specify Statistical tests • Ethical considerations

Clear Experimental Design… Experimental Design 2 sources of 1. between groups variance (systematic)

no drug drug

2. Within groups variance (nonsystematic) (error variance) Remember… Sampling error

Significant differences…variability btw is larger than expected on the basis of sampling error alone (due to chance alone) Variance Need it! Without it… No go VARIANCE “Partitioning of the variance”

Between Group Within Group Experimental Variance (Due to your treatment) + Error Variance Extraneous Variance (not due to treatment – chance) (confounds etc.)

CON TX Subs Variance: Important for the statistical analysis

F = between groups variance Within groups variance

F = Systematic effects + error variance error variance

F = 1.00 No differences btw groups Variance

Your should be designed to

• Maximize experimental variance

•Control extraneous variance

•Minimize error variance

Maximize “Experimental” Variance

• At least 2 levels of IV (IVs really vary?)

•Manipulation check: make sure the levels (exp. conditions) differ each other

Ex: anxiety levels (low anxiety/hi anxiety)  performance on math task

anxiety scale Control “Extraneous” Variance 1. Ex. & Con grps are similar to begin with

2. Within subjects design (carryover effects??)

3. If need be, limit population of interest (o vs o )

4. Make the extraneous variable an IV (age, sex, socioeconomic) = factorial design

M F

Lo Anxiety M-low F-low Factorial design (2 IV’s)

Hi Anxiety M-hi F-hi YOUR Proposals Control through Design – Don’ts

1. Ex Post Facto 2. Single-group, posttest only 3. Single-group pretest-posttest 4. Pretest-Posttest natural control group

1. Ex Post Facto – “after the fact”

Group A Naturally Occurring Event Measurement

No manipulation Control through Design – Don’ts

Single group posttest only

Group A TX Posttest

Single group Pretest-posttest

Pretest Group A TX Posttest

Compare Control through Design – Don’ts

Pretest-Posttest Naturalistic Control Group

Group A Pretest TX Posttest

Compare Group B Pretest no TX Posttest

Natural Occurring Control through Design – Do’s – Experimental Design

• Manipulate IV • Control Group •

4 Basic Designs Testing One IV

1. Randomized Posttest only, Control Group 2. Randomized Pretest-Posttest, Control Group 3. Multilevel Completely Randomized Between Groups 4. Solomon’s Four- Group Randomized Posttest Only – Control Group (most basic experimental design)

R Group A TX Posttest (Ex) Compare R Group B no TX Posttest (Con) Randomized, Pretest-Posttest, Control Group Design

R Group A Pretest TX Posttest (Ex) Compare R Group B Pretest no TX Posttest (Con) Multilevel, Completely Randomized Between Subjects Design (more than 2 levels of IV)

R Group A Pretest TX1 Posttest

R Group B Pretest TX 2 Posttest Compare

R Group C Pretest TX3 Posttest

R Group D Pretest TX4 Posttest Solomon’s Four Group Design (extension Multilevel Btw Subs)

R Group A Pretest TX Posttest

R Group B Pretest ---- Posttest Compare

R Group C ------TX Posttest

R Group D ------Posttest

Powerful Design! What stats do you use to analyze experimental designs?

Depends the

Test difference between groups

Nominal  chi square (/categorical)

Ordered data  Mann-Whitney U test

Interval or ratio  t-test / ANOVA (F test) t-Test Compare 2 groups

Independent One sample (Within) Samples (between Subs) Evaluate differences bwt two Evaluate differences bwt conditions in a single groups 2 independent groups Assumptions to use t-Test

1. The test variable (DV) is normally distributed in each of the 2 groups

2. The of the normally distributed test variable are equal – Homogeniety of Variance

3. Random assignment to groups t-distribution

Represents the distribution of t that would be obtained if a value of t were calculated for each sample for all possible random samples of a given size from some population Degrees of freedom (df)

When we use samples we approximate means & SD to represent the true population

Sample variability (SS = squared deviations) tends to underestimate population variability

Restriction is placed = making up for this mathematically by using n-1 in denominator S2 = variance ss (sum of squares) df (degrees of freedom)

(x - x )2 n-1

Degrees of freedom (df): n-1

The number of values (scores) that are free to vary given mathematical restrictions on a sample of observed values used to estimate some unknown population = price we pay for sampling Degrees of freedom (df): n-1

Number of scores free to vary

Data Set  you know the mean (use mean to compute variance)

n=2 with a mean of 6 X In order to get a mean of 6

8 with an n of 2…need a sum

of 12…second score must be x = ? 6 4… second score is restricted by sample mean (this score is not free to vary)

Group Statistics

Std. Error DRUG N Mean Std. Deviation Mean ENDURANC doped 10 7.9000 1.1972 .3786 no dope 10 2.6000 1.2649 .4000

Independent Samples Test

Levene's Test for Equality of Variances t-test for Equality of Means 95% of the Mean Std. Error Difference F Sig. t df Sig. (2-tailed) Difference Difference Low er Upper ENDURANC Equal variances .065 .801 9.623 18 .000 5.3000 .5508 4.1429 6.4571 assumed Equal variances 9.623 17.946 .000 5.3000 .5508 4.1427 6.4573 not assumed ANOVA

ENDURANC Sum of Squares df Mean Square F Sig. Betw een Groups 140.450 1 140.450 92.604 .000 Within Groups 27.300 18 1.517 Total 167.750 19 (ANOVA)

Two or more groups ….can use on two groups… t2 = F

Variance is calculated more than once because of varying levels (combo of differences)

Several Sources of Variance SS – between Partitioning SS – Within the variance SS – Total Sum of Squares: sum of squared deviations from the mean Assumptions to use ANOVA

1. The test variable (DV) is normally distributed

2. The variances of the normally distributed test variable is equal – Homogeniety of Variance

3. Random assignment to groups F = between groups variance Within groups variance

F = Systematic effects + error variance error variance

F = 1.00 No differences btw groups

F = 21.50 22 times as much variance between the groups than we would expect by chance After Omnibus F… Planned comparisons & Post Hoc tests

A Priori (spss: ) A Posteriori

part of your hypothesis…before Not quite sure where data are collected…prediction is made differences will occur Why not just do t-tests!

2 types of errors that you must consider when doing Post Hoc Analysis

Alpha

1. Per-comparison error (PC) 2. Family wise error (FW)

Inflate Alpha!!!! FW = c()

c = # of comparisons made  = your PC

Ex: IV ( 5 conditions)

1 vs 2 1 vs 3 1 vs 4 3 vs 4 FW = c() 1 vs 5 3 vs 5 2 vs 3 4 vs 5 10 (0.05) = .50 2 vs 4 2 vs 5 HSD