Journal of Economic Perspectives—Volume 24, Number 2—Spring 2010—Pages 31–46 Tantalus on the Road to Asymptopia Edward E. Leamer y fi rrstst rreactioneaction toto “The“The CredibilityCredibility RevolutionRevolution inin EmpiricalEmpirical Economics,”Economics,” aauthoreduthored byby JoshuaJoshua DD.. AAngristngrist aandnd JJörn-Steffenörn-Steffen Pischke,Pischke, wwas:as: Wow!Wow! ThisThis M ppaperaper makesmakes a stunninglystunningly ggoodood ccasease fforor rrelyingelying oonn ppurposefullyurposefully random-random- iizedzed oror accidentallyaccidentally randomizedrandomized experimentsexperiments toto relieverelieve thethe doubtsdoubts thatthat afflaffl iictct iinferencesnferences ffromrom nnonexperimentalonexperimental ddata.ata. OOnn ffurtherurther rreflefl eection,ction, I realizedrealized thatthat I maymay hhaveave beenbeen oovercomevercome wwithith iirrationalrrational eexuberance.xuberance. MMoreover,oreover, wwithith tthishis ggreatreat hhonoronor bbestowedestowed oonn mmyy ““con”con” aarticle,rticle, I couldn’tcouldn’t easilyeasily throwthrow thisthis childchild ofof minemine overboard.overboard. WWee eeconomistsconomists ttrudgerudge rrelentlesslyelentlessly ttowardoward AAsymptopia,symptopia, wwherehere ddataata aarere uunlimitednlimited aandnd eestimatesstimates areare consistent,consistent, wherewhere thethe lawslaws ofof largelarge numbersnumbers applyapply perfectlyperfectly andand wwherehere thethe fullfull intricaciesintricacies ofof tthehe eeconomyconomy aarere ccompletelyompletely rrevealed.evealed. BButut it’sit’s a frus-frus- ttratingrating jjourney,ourney, ssince,ince, nono mmatteratter howhow farfar wewe travel,travel, AAsymptopiasymptopia rremainsemains iinfinfi nnitelyitely ffarar aaway.way. WWorstorst ooff aall,ll, wwhenhen wwee ffeeleel ppumpedumped upup withwith ourour progress,progress, a tectonictectonic shiftshift cancan ooccur,ccur, llikeike thethe PanicPanic ofof 2008,2008, makingmaking itit seemseem asas thoughthough ourour longlong journeyjourney hhasas leftleft usus ddisappointinglyisappointingly ccloselose toto thethe SStatetate ofof CompleteComplete IgnoranceIgnorance whencewhence wewe began.began. TThehe ppointlessnessointlessness ooff mmuchuch ooff oourur ddailyaily aactivityctivity mmakesakes uuss rreceptiveeceptive wwhenhen tthehe PPriestsriests ooff oourur ttriberibe ringring tthehe bellsbells andand announceannounce a shortenedshortened pathpath toto Asymptopia.Asymptopia. ((RememberRemember tthehe CCowlesowles FFoundationoundation offeringoffering asymptoticasymptotic propertiesproperties ofof simultaneoussimultaneous eequationsquations estimatesestimates andand structuralstructural parameters?)parameters?) WeWe maymay listen,listen, bbutut wwee ddon’ton’t hhear,ear, wwhenhen thethe PriestsPriests warnwarn thatthat tthehe nnewew directiondirection isis onlyonly forfor thosethose withwith Faith,Faith, thosethose wwithith ccompleteomplete beliefbelief inin thethe AssumptionsAssumptions ofof thethe Path.Path. ItIt oftenoften takestakes yearsyears downdown thethe PPath,ath, butbut soonersooner oror later,later, someonesomeone articulatesarticulates thethe concernsconcerns thatthat gnawgnaw awayaway inin eacheach ofof ■ EEdwarddward EE.. LLeamereamer iiss PProfessorrofessor ooff EEconomics,conomics, MManagementanagement andand Statistics,Statistics, UniversityUniversity ofof CCaliforniaalifornia atat LosLos Angeles,Angeles, LosLos Angeles,Angeles, California.California. HisHis e-maile-mail addressaddress isis 〈[email protected]@ aanderson.ucla.edunderson.ucla.edu〉. doi=10.1257/jep.24.2.31 32 Journal of Economic Perspectives us and asks if the Assumptions are valid. (T. C. Liu (1960) and Christopher Sims (1980) were the ones who proclaimed that the Cowles Emperor had no clothes.) Small seeds of doubt in each of us inevitably turn to despair and we abandon that direction and seek another. TTwowo ooff tthehe llatestatest pproducts-to-end-all-sufferingroducts-to-end-all-suffering aarere nnonparametriconparametric eestimationstimation aandnd cconsistentonsistent standardstandard eerrors,rrors, wwhichhich promisepromise rresultsesults wwithoutithout aassumptions,ssumptions, asas ifif wwee wwereere aalreadylready iinn AAsymptopiasymptopia wwherehere datadata areare soso pplentifullentiful tthathat nnoo aassumptionsssumptions aarere nneeded.eeded. ButBut likelike proceduresprocedures thatthat relyrely explicitlyexplicitly onon assumptions,assumptions, thesethese newnew mmethodsethods workwork wellwell inin thethe circumstancescircumstances iinn wwhichhich explicitexplicit oorr hhiddenidden aassumptionsssumptions hholdold ttolerablyolerably wwellell aandnd ppoorlyoorly ootherwise.therwise. BByy ddisguisingisguising tthehe aassumptionsssumptions oonn wwhichhich nnonparametriconparametric mmethodsethods aandnd cconsistentonsistent sstandardtandard eerrorsrrors rrely,ely, thethe ppurveyorsurveyors ooff tthesehese mmethodsethods hhaveave mmadeade iitt iimpossiblempossible ttoo hhaveave aann iintelligiblentelligible cconversationonversation aaboutbout tthehe ccircumstancesircumstances iinn wwhichhich ttheirheir ggimmicksimmicks ddoo nnotot wworkork wwellell aandnd ooughtught nnotot ttoo bbee uused.sed. AAss fforor mme,e, I ppreferrefer toto carrycarry parametersparameters onon mymy journeyjourney soso I knowknow wherewhere I amam andand wwherehere I amam going,going, nnotot ttravelravel sstonedtoned oonn tthehe llatestatest eeuphoriauphoria ddrug.rug. TThishis iiss a sstorytory ooff TTantalus,antalus, ggraspingrasping fforor kknowledgenowledge tthathat rremainsemains alwaysalways beyondbeyond rreach.each. IInn GGreekreek mythologymythology TantalusTantalus waswas favoredfavored amongamong allall mortalsmortals byby beingbeing askedasked ttoo ddineine wwithith tthehe ggods.ods. BButut hehe misbehaved—somemisbehaved—some ssayay bbyy tryingtrying ttoo ttakeake ddivineivine ffoodood bbackack toto thethe mortals,mortals, somesome saysay byby invitinginviting thethe godsgods toto a dinnerdinner forfor whichwhich TantalusTantalus bboiledoiled hishis sonson andand servedserved himhim asas thethe mainmain dish.dish. WhateverWhatever thethe etiquetteetiquette fauxfaux pas,pas, TTantalusantalus waswas punishedpunished byby beingbeing immersedimmersed upup toto hishis neckneck inin water.water. WhenWhen hehe bowedbowed hhisis headhead toto drink,drink, thethe waterwater draineddrained away,away, aandnd wwhenhen hehe stretchedstretched uupp ttoo eeatat thethe fruitfruit hhanginganging aabovebove him,him, wwindind wwouldould blowblow itit outout ofof reach.reach. ItIt wouldwould bebe muchmuch healthierhealthier fforor aallll ooff uuss iiff wwee ccouldould aacceptccept oourur ffate,ate, rrecognizeecognize tthathat pperfecterfect kknowledgenowledge wwillill bebe fforeverorever bbeyondeyond ourour reachreach andand fi nndd hhappinessappiness wwithith wwhathat wwee hhave.ave. IIff wwee sstoppedtopped ggraspingrasping fforor tthehe aapplepple ooff AAsymptopia,symptopia, wwee wwouldould ddiscoveriscover tthathat oourur ppoolool ooff TTantalusantalus iiss fullfull ofof smallsmall butbut enjoyableenjoyable iinsightsnsights andand wisdom.wisdom. CCanan wwee eeconomistsconomists agreeagree thatthat itit isis extremelyextremely hardhard workwork toto squeezesqueeze truthstruths ffromrom ourour ddataata ssetsets aandnd wwhathat wwee ggenuinelyenuinely uunderstandnderstand wwillill remainremain uuncomfort-ncomfort- aablybly llimited?imited? WWee needneed wordswords inin ourour methodologicalmethodological vocabularyvocabulary toto expressexpress thethe llimits.imits. WWee nneedeed ssensitivityensitivity analysesanalyses toto makemake thosethose limitslimits transparent.transparent. ThoseThose whowho tthinkhink ootherwisetherwise shouldshould bebe requiredrequired toto wearwear a scarlet-letterscarlet-letter O aaroundround theirtheir necks,necks, fforor ““overconfioverconfi ddence.”ence.” AngristAngrist aandnd PischkePischke obviouslyobviously knowknow this.this. TheirTheir ppaperaper isis ppepperedeppered withwith concernsconcerns aboutabout quasi-experimentsquasi-experiments andand withwith criticismscriticisms ofof instru-instru- mmentalental vvariablesariables thoughtlesslythoughtlessly chosen.chosen. I tthinkhink wwee wwouldould mmakeake pprogressrogress ifif wwee sstoppedtopped uusingsing tthehe wwordsords ““instrumentalinstrumental vvariables”ariables” aandnd uusedsed iinsteadnstead ““surrogates”—surrogates”— mmeaningeaning ssurrogatesurrogates forfor thethe experimentexperiment thatthat wewe wishwish wwee ccouldould hhaveave cconducted.onducted. TThehe ppsychologicalsychological ppowerower ooff tthehe vvocabularyocabulary rrequiresequires a ““surrogate”surrogate” toto bebe chosenchosen wwithith mmuchuch ggreaterreater carecare tthanhan aann ““instrument.”instrument.” AAss AngristAngrist aandnd PPischkeischke ppersuasivelyersuasively argue,argue, eithereither ppurposefullyurposefully randomizedrandomized eexperimentsxperiments oorr aaccidentallyccidentally rrandomizedandomized ““natural”natural” experimentsexperiments ccanan bbee eextremelyxtremely hhelpful,elpful, bbutut AAngristngrist aandnd PischkePischke sseemeem toto meme toto overstateoverstate tthehe ppotentialotential benefibenefi ttss ooff tthehe aapproach.pproach. SSinceince hhardard aandnd iinconclusivenconclusive tthoughthought iiss nneededeeded toto transfertransfer thethe Edward E. Leamer 33 rresultsesults learnedlearned fromfrom randomizedrandomized experimentsexperiments intointo otherother domains,domains, therethere mustmust tthereforeherefore