Policy Information Report Exploring What Works in Science Instruction: A Look at the Eighth-Grade Science Classroom ® This.report.was.written.by: Table of Contents Henry Braun Boston College Richard Coley Preface....................................................................................................................2 Yue Jia Acknowledgments..................................................................................................2 Catherine Trapani Educational Testing Service Executive.Summary..............................................................................................3 Science.Achievement......................................................................................3 The.reviews.expressed.in.this.report. are.those.of.the.authors.and.do.not. Teacher.Characteristics...................................................................................3 necessarily.reflect.the.views.of.the. officers.and.trustees.of.Educational. Instructional.Strategies..................................................................................3 Testing.Service. Understanding.Science.Achievement............................................................3 Additional.copies.of.this.report.can. Introduction...........................................................................................................6 be.ordered.for.$15.(prepaid).from: Science.Achievement......................................................................................7 Policy.Information.Center. The.Eighth-Grade.Science.Classroom...........................................................7 Mail.Stop.19-R. Methodology........................................................................................................11 Educational.Testing.Service. Rosedale.Road. Cautions.in.Interpretation............................................................................13 Princeton,.NJ.08541-0001. 609-734-5212. Exploring.What.Works.in.Science.Instruction..................................................15 [email protected] Phase.1...........................................................................................................15 Copies.can.be.downloaded.free. Phase.2...........................................................................................................20 from:.www.ets.org/research/pic Phase.3...........................................................................................................23 Conclusions.and.Implications............................................................................32 Copyright.©.2009.by.Educational. Testing.Service..All.rights.reserved.. Appendix.A:.Data.for.HLM.................................................................................34 ETS,.the.ETS.logo,.LISTENING... Appendix.B:.HLM.Methodology.........................................................................36 LEARNING..LEADING.,.GRE,. TOEFL.and.TOEIC.are.registered. Appendix.C:.Standard.Errors.and.Significance.Tests.for.Figures.3.to.13........39 trademarks.of.Educational.Testing. Service.(ETS)..THE.PRAXIS.. SERIES.is.a.trademark.of.ETS. May.2009. Policy.Evaluation.and. ....Research.Center. Policy.Information.Center. Educational.Testing.Service Preface In.a.recent.opinion.piece.in.The.New York Times,.. these.concerns.from.both.policy.and.pedagogical.. Brian.Green,.professor.of.physics.at.Columbia.. perspectives..In.support.of.these.efforts,.Henry.. University.and.author.of.The Elegant Universe.. Braun,.Richard.Coley,.Yue.Jia,.and.Catherine.. and.The Fabric of the Cosmos,.writes:. Trapani.have.mined.the.data.of.the.National.. Assessment.of.Educational.Progress.(NAEP).with.. Science is the greatest of all adventure stories, sophisticated.statistical.models.in.an.attempt.. one that’s been unfolding for thousands of years to.identify.aspects.of.U.S..eighth-grade.science.. as we have sought to understand ourselves and classrooms.that.are.associated.with.science.. our surroundings. Science needs to be taught to achievement..Their.report,.Exploring What Works the young and communicated to the mature in a in Science Instruction: A Look at the Eighth-Grade manner that captures this drama. We must embark Science Classroom,.identifies.instructional.strategies. on a cultural shift that places science in its rightful and.teacher.characteristics.that.appear.to.make.a.. place alongside music, art and literature as an difference.in.NAEP.science.scores..And.because. indispensable part of what makes life worth living. teacher.practices.that.make.a.difference.must.make.. a.difference.for.all.Americans,.these.practices.are.. In.reality,.however,.the.science.achievement.of.. examined.for.the.population.generally.and.for.. U.S..students.and.the.science.literacy.of.the.general.. students.grouped.by.their.various.racial/ethnic.. population.are.mediocre..Such.mediocrity.does.not. characteristics.and.income.level..The.results.offer. bode.well.for.our.nation’s.ability.to.compete.with.. much.food.for.thought.for.anyone.concerned.with. the.growing.science.and.engineering.talent.that.is.. science.education. emerging.among.the.many.nations.with.which.we.. will.be.both.competing.and.cooperating.in.the.future. Michael.T..Nettles. Senior.Vice.President.. Broad.recognition.of.the.challenges.that.our.nation. Policy.Evaluation.and.Research.Center faces.has.led.to.the.creation.of.several.special.. commissions.and.resulting.reports.that.address.. Acknowledgments The.authors.wish.to.acknowledge.the.helpful.. Scott.Davis.provided.data.analyses..Richard.Pliskin. comments.and.feedback.provided.by.the.following.. was.the.editor,.Marita.Gray.designed.the.cover,.. reviewers:.Courtney.Bell,.ETS;.Richard.Duschl,.. and.Sally.Acquaviva.provided.desktop.publishing.. Pennsylvania.State.University;.Kate.McNeil,.. services..Errors.of.fact.or.interpretation.are.those.. Boston.College;.and.Andreas.Oranje,.ETS... of.the.authors. Waverly.Van.Winkle,.Darlene.Rosero,.and.. 2 Executive Summary The.science.achievement.of.U.S..students.has.been. About.one-third.of.eighth-grade.students.have.. flat.for.a.decade;.in.fact,.in.a.recent.international. teachers.with.four.years.or.less.experience.teaching. assessment,.U.S..students.ranked.lower,.on.aver- science..Many.students.have.science.teachers.with. age,.than.their.peers.in.16.of.30.developed.nations.1. more.overall.teaching.experience.than.science.. The.National.Assessment.of.Educational.Progress. teaching.experience,.raising.the.possibility.that.. (NAEP),.in.addition.to.serving.as.the.nation’s.report. science.teaching.was.not.the.field.of.choice.for.those. card,.collects.information.from.students,.teach- teachers.or.the.subject.for.which.they.trained. ers,.and.schools.on.instructional.practices,.teacher. characteristics,.and.school.factors,.allowing.us.to. Instructional Strategies peer.into.U.S..schools.and.view.students.and.teachers. As.reported.by.students,.several.instructional.activi- as.they.interact.in.the.context.of.schools.and.class- ties.stand.out.as.more.commonly.practiced.than. rooms..This.report.takes.advantage.of.these.data.. others..Reading.a.science.textbook,.taking.a.science. and.provides.a.view.of.eighth-grade.science.class- test.or.quiz,.and.having.a.teacher.conduct.a.science. rooms.in.2005,.using.statistical.models.particularly. demonstration.were.activities.that.more.than.half. suited.to.this.type.of.data.—.hierarchical.linear.mod- of.eighth-grade.students.indicated.occurred.almost. els.(HLMs)..Our.intent.is.to.identify.teacher.charac- every.day.or.once.or.twice.a.week..Other.popular. teristics.and.instructional.strategies.that.are.associ- activities.(40.percent.or.more.of.students.report- ated.with.student.science.achievement..After.briefly. ing.frequent.activity).were.hands-on.exercises.and. describing.the.state.of.science.achievement,.teacher. investigations,.talking.about.results.from.hands-on. characteristics,.and.instructional.strategies.across. exercises,.and.working.with.other.students.on.a.. U.S..eighth-grade.classrooms,.the.report.presents.. science.activity.or.project..Activities.occurring.much. the.results.of.a.three-phase.analysis.that.is.designed. less.often.included.presenting.an.oral.science.report,. to.illuminate.the.statistical.relationships.among.. preparing.a.written.science.report,.using.library.. these.variables. resources.for.science,.and.reading.a.book.or.. magazine.about.science..In.addition,.70.percent.. Science Achievement of.students.indicated.that.they.did.science.projects. Since.1996,.the.overall.science.scores.of.eighth.grad- that.took.a.week.or.more. ers.have.remained.unchanged.at.149.(on.a.scale.of.0. to.300)..Black.students,.however,.showed.an.increase. Understanding Science Achievement of.3.scale.points.since.the.1996.assessment..Scores. Using.multilevel.analyses,.we.found.the.following. of.no.other.racial/ethnic.group.improved..Although. student.and.teacher.characteristics.to.be.associated. reduced,.the.White-Black.achievement.gap.remained. with.student.science.achievement: substantial..In.2005,.White.eighth.graders.scored. 36.points.higher.than.Black.eighth.graders.and.31. • Student.demographic.characteristics.had.statisti- points.higher.than.Hispanic.eighth.graders..Male. cally.significant.associations.with.achievement.. students.continued.to.score.higher.than.females. Black.and.Hispanic.students.scored.considerably. lower.than.White.students,.and.males.scored. With.respect.to.the.achievement.levels.set.by.the.. higher.than.females..English-language.learners. National.Assessment.Governing.Board.(NAGB),.in. and.students.with.disabilities.scored.much.lower.