<<

CAUSAL PARAMETERS ANDPOLICY ANALYSIS IN ECONOMICS: ATWENTIETHCENTURY RETROSPECTIVE*

JAMES J. HECKMAN

Themajor contributions of twentiethcentury econometricsto knowledge were thedeŽ nition of causal parameters within well-deŽ ned economic models in which agentsare constrained by resourcesand markets and causes are interrelated, the analysisof what isrequired to recover causal parameters from data (the identiŽcation problem), and clariŽ cation of the role of causal parameters in policy evaluationand in forecasting the effects of policies never previously experienced. Thispaper summarizes the development of these ideas by theCowles Commission, theresponse to their work bystructural econometriciansand V AReconometri- cians,and the response to structural andV AReconometricsby calibrators, advocatesof natural and social experiments, and by nonparametric econometri- ciansand statisticians.

I. INTRODUCTION Thispaper considersthe deŽ nition and identiŽcation of causal parametersin economicsand theirrole in econometric policyanalysis. It assessesdifferent researchprograms designed torecovercausal parametersfrom data. Atthebeginning of this century,economictheory was mainly intuitive,and empiricalsupport forit was largelyanecdotal. At theend of thecentury ,economicshas aricharray offormalmodels and ahigh-quality database. Empirical regularitiesmotivate theoryin manyareas of economics,and data areroutinely used to testtheory .Many economictheories have been developed as measurementframeworks to suggest what data should becol- lectedand howthey should beinterpreted. Econometrictheory was developedto analyze and interpret economicdata. Mosteconometric theory adapts methodsorigi- nally developedin statistics. Themajor exception to this ruleis theeconometric analysis ofthe identiŽ cation problem and the companionanalyses ofstructural equations, causality ,and eco- nomicpolicy evaluation. Although an economistdid notinvent the phrase,‘ ‘correlationdoes not imply causation,’’1 economistsclari-

*Thisresearch was supported by NSF 97-09-873to the University of .I havebeneŽ ted from comments received in discussions with Jeff Biddle, JenniferBoobar ,Ray Fair,Edward Glaeser,ClaudiaGoldin, Lars Hansen, LawrenceKatz, StevenLevitt, ,JudeaPearl, Jose ´Scheinkman, ChristopherSims, and Edward Vytlacil.For historical background on the Cowles Commission,I haverelied on Christ{1952}, Epstein {1987}, and Morgan {1990}. 1.The phrase is generallyattributed to Karl Pearson. r 2000by thePresident and Fellowsof Harvard Collegeand theMassachusetts Institute of Technology. The Quarterly Journal ofEconomics, February2000

45 46 QUARTERLY JOURNALOF ECONOMICS

Žedthe meaning of causation within well-speciŽed models, the requirementsfor a causal interpretationof an empiricalrelation- ship, and thereasons why acausal frameworkis necessaryfor evaluating economicpolicies. 2 Thefundamental workwas doneby economistsassociated with theCowles Commission. 3 Thelasting legacyof this research programincludes the concepts of exogenous(external) and endoge- nous(internal) variables,and thenotions of ‘ ‘policyinvariant parameters’’ and ‘‘structuralparameters’ ’ whichhave entered everydayparlance inside and outsideof economics. Just as theancient Hebrews were ‘ ‘thepeople of the book,’ ’ economistsare ‘ ‘thepeople of themodel.’ ’ Formaleconomic models arelogically consistent systems within whichhypothetical ‘ ‘thought experiments’’ canbe conducted to examine the effects of changes inparametersand constraintson outcomes. Within a model,the effectson outcomes of variationin constraintsfacing agentsin a marketsetting are well deŽ ned. Comparative statics exercises formalizeMarshall’ s notionof a ceterisparibus changewhich is what economistsmean by acausal effect.In his ownwords,

Itis sometimessaid that the laws of economicsare ‘ ‘hypothetical.’’ Of course,like every other science, it undertakes to study theeffects which will beproduced by certain causes, not absolutely ,butsubject to the condition thatother things are equal and that the causes are able to work outtheir effectsundisturbed. Almost every scientiŽ c doctrine,when carefully and formallystated, will be found to contain some proviso to the effect that other thingsare equal; the action of the causes in question is supposed to be isolated;certain effects are attributed to them, but only on the hypothesis thatno cause is permitted to enter except those distinctly allowed for {Marshall,1961, p. 36}.

The‘ ‘otherthings areequal’ ’ orceteris paribus clauseis a cornerstoneof economicanalysis. DeŽning causality within amodelis relativelystraightfor-

2.For example, the artiŽ cial intelligence community has just begun to appreciatethe contributions of to the deŽ nition and identiŽ cation of causalrelationships. See the papers in Glymourand Cooper {1999} and the paper byPearl{1998}. 3.The Cowles Commission was founded by Alfred Cowles to promote the synthesisof mathematics and economics. Cowles and the Cowles Commission playeda leadingrole in creating the .Itwas originally based in ColoradoSprings andhad a looseorganizational arrangement with Colorado College.It relocated to the from 1939 to 1955. See Christ {1952,reprinted 1995}, Epstein {1987}, and Morgan {1990} for valuable histories of econometricsand the role of the Cowles Commission in deŽ ning modern econometrics. CAUSALP ARAMETERSAND POLICY ANALYSIS 47 ward whenthe causes can be independently varied. 4 DeŽning causality whenthe causes are interrelated is lessstraightforward and is amajorachievement of econometrics. Recovering causal parametersfrom data is notstraightforward. An important contributionof econometric thought was theformalization of the notiondeveloped in philosophy that manydifferent theoretical modelsand hencemany different causal interpretationsmay be consistentwith thesame data. In economics,this is called the problemof identiŽcation. The econometric analysis ofthe identiŽ - cationproblem clariŽ es thelimits of purelyempirical knowledge. It makesprecise the idea that correlationis notcausation by using fully speciŽed economic models as devicesfor measuring and interpretingcausal parameters.It presentsconditions under whichthe hypothetical variations mentioned in thequotation fromMarshall, orthe structural parameters of well-speciŽ ed economicmodels, can in principle beidentiŽ ed fromdata. Differ- enta prioriassumptions can identify thesame causal parameter oridentify different causal parameters.The key insight in the literatureof twentieth century econometrics was thediscovery of theconditional nature of empirical knowledge. The justiŽ cation forinterpreting an empiricalassociation causally hingeson the assumptionsrequired to identify thecausal parametersfrom the data. Thispaper proceedsin thefollowing way .(1) Theconcept of a causal parameterwithin awell-posed economicmodel is deŽned in an economicsetting that respectsthe constraints imposed by preferences,endowments, and socialinteractions through mar- kets.By awell-posed economicmodel, I meana modelthat speciŽes all ofthe input processes,observed and unobservedby theanalyst, and theirrelationship to outputs. My deŽnition of causal parametersformalizes the quotation from Marshall. This formalizationis amoreappropriate frameworkfor economic causal analysis than otherframeworks developed in statistics that donot recognize constraints, preferences, and socialinterac- tions(i.e., are not based onformal behavioral models). The conceptof identiŽ cation of acausal parameteris discussed using themarket demand-supply examplethat motivatedthinking aboutthe identiŽ cation problem through the Ž rsthalf ofthe twentiethcentury .Thisexample emphasizes the consequences of

4.Marini and Singer {1988} present a valuablesummary of therancorous and confusingdebates about the nature of causallaws developed in model-freeŽ elds. 48 QUARTERLY JOURNALOF ECONOMICS interdependenceamong economic agents, but has somespecial featuresthat arenot essential for understanding thefundamental natureof theidentiŽ cation problem. A moregeneral statement of theidentiŽ cation problem is giventhan appears in thepublished literature.The role of causal parametersin policyanalysis is clariŽed. (2) Thepaper thenassesses the response in thelarger economicscommunity to the Cowles Commission research pro- gram.The Cowles group developed the linear equation simulta- neousequations model (SEM) that is still presentedin most econometricstextbooks. It extensivelyanalyzed oneform of the identiŽcation problem that mosteconomists still thinkof as the identiŽcation problem. It focusedattention on estimation of Keynesianmacro models and onthe parameters of market-level supply and demand curves.By themid-1960s theCowles research programwas widely perceivedto be an intellectualsuccess but an empiricalfailure. Thisled totwo radically different responses.The Ž rstwas the VARor‘ ‘innovationaccounting’ ’ programmost often associated with thework of Sims{1972,1977,1980,1986} that objectedto the ‘‘incredible’’ natureof the identifying assumptionsused in the CowlesCommission models and advocated application ofmore looselyspeciŽ ed economicmodels based ondevelopments in the multivariatetime series literature. This research program system- atically incorporatedtime series methods into macroeconometrics and producedmore accurate descriptions of macrodata than did its Cowlespredecessors. Its useof economic theory was less explicit,but it drewon the dynamic economicmodels developed in theseventies and eightiesto motivate its statistical decompositions. At aboutthe same time, and moreexplicitly motivatedby the developmentof a macroeconomicsbased ondynamic general equilibriumtheory under uncertainty ,structuralequations meth- ods based onexplicit parameterizationof preferencesand technol- ogyreplaced the Cowles paradigm formarket aggregates and Keynesiangeneral equilibrium systems. The notion of a struc- tural orcausal parametersurvived, but it was deŽned more preciselyin termsof preferenceand technologyparameters, and newmethods for recovering them were proposed. Nonlinear dynamic econometricmodels were developed to incorporate the insights ofnewlydeveloped economic theory into frameworks for economicmeasurement and toincorporate rational expectations CAUSALP ARAMETERSAND POLICY ANALYSIS 49 intothe formulation and estimationof models. This approach emphasizesthe clarity with whichidentifying assumptionsare postulated and advocatesan approach toestimation that tests and rejectswell-posed models.It is ambitiousin its attemptto identify and estimateeconomically interpretable ‘ ‘policyinvari- ant’’ structuralparameters that canbe used to ascertain the impacts ofavarietyof policies. Theempirical track record of the structural approach is,at best,mixed. Economic data, bothmicro and macro,have not yielded manystable structuralparameters. Parameter estimates fromthe structural research program are widely held notto be credible.The empirical research program of estimating policy invariant structuralparameters in thepresence of policy shifts remainsto be implemented. The perceived empirical failures of well-posed structuralmodels have often led tocalls forabandon- mentof thestructural approach in manyapplied Želds,and notto thedevelopment of betterstructural models in thoseŽ elds. Partof the continuing popularity oftheV ARprogramis that it sticksmore closely to the data and in that senseis more empiricallysuccessful than structuralistapproaches. At thesame time,its criticsargue that it is difficult tointerpret the estimates obtained fromapplication ofthis programwithin thecontext of well-speciŽed economicmodels and that theCowles vision of usingeconomics to evaluate economic policy and interpretphenom- enahas beenabandoned by adherentsof this researchprogram. In addition, thedata summariesreported by VAReconometricians areoften not transparent, and thechoice of an appropriate data summaryrequires knowledge of multivariate time series meth- ods.Hence, the time series data summariesproduced by this approach oftenhave a black-box quality aboutthem, and judg- mentsabout Ž tareoften mysterious to outsiders. Thetension between the goal of producing accuratedescrip- tionsof thedata and thegoal of producing counterfactualcausal analyses forinterpretation and policyprediction is alasting legacyof the research of the Cowles Commission, and amajor themeof this essay.It mightbe said that thetheoretical reach of theCowles analysts exceededtheir empirical grasp. Theydevel- oped avisionof empiricaleconomics that has beenhard torealize in practice. Threevery different responsesto the perceived lack of empiricalsuccess of the structural research program and thelack ofeconomic interpretability and apparent arbitrarinessin the 50 QUARTERLY JOURNALOF ECONOMICS choiceof VARmodelsemerged in the1980s. All stressthe need for greatertransparency in generatingestimates, although there is disagreementover what transparencymeans. At therisk of gross oversimpliŽcation, these responses can be classiŽ ed in thefollow- ing way.TheŽ rstresponse is thecalibration movement, which respondsto theperceived inability offormalstructural economet- ricmethods to recover the parameters of economic models from time-seriesdata and theperceived overemphasis on statistics to theexclusion of economics in theapplication ofV ARmodels.This economic-theory-drivenmovement stresses the role of simple generalequilibrium models with parametersdetermined by intro- spection,simple dynamic time-seriesaverages, or by appeal to microestimates. Calibrators emphasizethe fragility ofmacro data and willingly embracethe conditional nature of causal knowledge.They explicitly reject‘ ‘Žt’’ as aprimarygoal of empiricaleconomic models and emphasizeinterpretability over Ž t. Thecalibrators have been accused of beingtoo casual in their useof evidence. Sample averagesfrom trended time series are usedto determine parameters; and whentested, the time series Žts ofthe calibrated modelsare often poor .Themicroestimates that aresometimes used in this literatureare often taken out of thecontexts that justify them. Thesecond response is thenonparametric research program ineconometricsand theearlier ‘ ‘sensitivityanalysis’ ’ researchin statistics that viewsthe functional forms and distributional assumptionsmaintained in conventionalstructural (and nonstruc- tural) approachesas amajorsource of their lack of credibility and seeksto identify theparameters of economicmodels nonparametri- cally orto examine the sensitivity of estimates to different identifying assumptions.The nonparametric identiŽ cation analy- sesconducted within this researchprogram clarify therole of functionalforms and distributional assumptionsin identifying causal parameters.Using hypothetical inŽ nite samples, it sepa- ratesout what canin principle beidentiŽ ed without functional formand distributional assumptionsfrom what cannot.Many questionthe practical empiricalrelevance of nonparametric theory inthelimited sample sizes available tomost . Others questionthe novelty of the approach. Someform of bounding or sensitivityanalysis has always beenpracticed by mostcareful empiricaleconomists. Sensitivity analysis is acornerstoneof calibration econometrics. Athird,more empirical, approach tocausal analysis has also CAUSALP ARAMETERSAND POLICY ANALYSIS 51 emergedunder the general rubric of the ‘ ‘natural experiment’’ movement.This popular movementsearches for credible sources ofidentifying informationfor causal parameters,using ideal randomexperiments as abenchmark.It rejectsthe use of structuraleconometric models because, according to its adher- ents,such models do notproduce credible estimates and impose arbitrary structureonto the data. In addition, thesevere computa- tionalcosts of estimating most structural models make the simplerestimation methods advocated by this groupmore appeal- ing becauseŽ ndings canbe easilyreplicated. The economic theory usedto interpretdata is typically keptat an intuitivelevel. In manyrespects, this grouphas muchin commonwith advocatesof the V ARapproach. Bothapproaches are strongly empiricallygrounded. However ,natural experimentersprefer simplerdata summariesthan thoseproduced from modern time- seriesmodels. One goal, shared in commonwith thenonparamet- riceconometricians and thestatisticians whoadvocate sensitivity analysis, is tocarefully locate what is ‘‘in thedata’ ’ beforeany elaboratemodels are built oreconometric identiŽ cation assump- tionsare invoked. In this literaturethe ‘ ‘causal parameters’’ areoften deŽ ned relativeto an instrumentalvariable deŽned by some‘ ‘natural experiment’’ or,in thebest case scenario, by asocialexperiment. Thedistinction betweenvariables that determinecauses and variables that entercausal relationshipsis sometimesblurred. Accordingly,in this literaturethe deŽ nition of a causal parameter is notalways clearlystated, and formalstatements of identifying conditionsin termsof well-speciŽ ed economicmodels are rarely presented.Moreover, the absence of explicit structuralframe- worksmakes it difficult tocumulate knowledge across studies conductedwithin this framework.Many studies producedby this researchprogram have a ‘‘stand alone’’ featureand neitherinform norare in uenced by thegeneral body ofempiricalknowledge in economics.This literature emphasizes the role of causal models forinterpreting data and analyzing existingpolicies, not for makingthe counterfactual policy predictions that motivatedthe researchprogram of theCowles Commission. That goal is viewed as impossible. In orderto makethis paper accessibleto a generalaudience, I discuss onlythe simplest models and deliberatelyavoid elaborate formalarguments. This strategy risks the danger of grossoversim- pliŽcation of some very subtle points. It is hopedthat thepoints 52 QUARTERLY JOURNALOF ECONOMICS madeusing simplemodels capture the essential features of the importantcontribution of econometrics to the understanding of causality,identiŽcation, and policyanalysis.

II. CAUSAL PARAMETERS, IDENTIFICATION , AND ECONOMETRIC POLICY EVALUATION Amajorcontribution of twentieth century econometrics was therecognition that causality and causal parametersare most fruitfully deŽned within formaleconomic models and that compara- tivestatics variationswithin thesemodels formalize the intuition in Marshall’s quotationand mostclearly deŽ ne causal parame- ters.A secondmajor contribution was theformalization of the insight developedin philosophy that manymodels are consistent with thesame data and that restrictionsmust be then placed on modelsto use the data torecover causal parameters.A third major contributionwas theclariŽ cation of the role of causal modelsin policyevaluation.

II.1. Causal Parameters Withinthe context of a well-speciŽed economic model, the conceptof a causal parameteris welldeŽ ned. For example, in a modelof production of output Y based oninputs X that can be independently varied,we write the function F: RN R1 as

(1) Y 5 F(X1, . . . , XN), where X 5 (X1, . . . , XN)is avectorof inputs deŽned over domain D (X [ D).Theyplay theroles of the causes, i.e., factors that produce Y. 5 Thesecauses are the primitives of the relevant

5.Philosophers would no doubt claim that I ambegging the question of deŽning a causalparameter by assumingthe existence of economic models like (1). Mypointis that given such models, discussions of causality become trivial. The wholegoal of economic theory is to produce models like (1), and I takethese as primitives.The multiplicity of possible models for the same phenomenon is the reasonwhy amultiplicityof possible causal relationships may exist for the same phenomenon. Amoreabstract approach to the deŽ nition of acausalrelation that does not requirespeciŽ cation of afunction F orawell-speciŽed economic model builds on thework ofSimon {1952} and Sims {1977} and speciŽ es properties of the input space (X)andthe output space ( Y)andtheir relationship. The crucial idea is that inputscan be manipulated in ways that do not affect the structure ofthe causal relationbut that affect the realized outputs. Thus,consider an abstractspace S ofpossiblefeatures of models,both inputs andoutputs. Consider two setsof restrictions: A , S restrictsinputs, and B , S restrictsoutputs. Suppose that S ismapped into two spaces: PX: A X; PY: B Y. Then (A,B) deŽnes a causal ordering fromX to Y if A restricts X (ifat all)but not Y, and B restricts Y (ifat all) without further restricting X.Moreformally ( A,B), CAUSALP ARAMETERSAND POLICY ANALYSIS 53 economictheory .Assuming that eachinput canbe freely varied, so thereare no functionalrestrictions connecting the components of X,thechange in Y produced fromthe variation in Xj holding all otherinputs constantis thecausal effectof Xj. If F is differentiable in Xj,themarginal causal effectof Xj is

­ Y (2) 5 Fj(X1, . . . , Xj, . . . , XN) X5 x. ­ Xj If F is notdifferentiable, Ž nitechanges replace derivatives. Nothingin this deŽnition requires that any orall ofthe Xj be observed.Moreover ,the Xj maybe stochastic. Agents may make decisionsabout subsets of the X based onlyon expectationsabout theremaining X.In this case,realized X componentsenter (1), and wedeŽne thecausal parameterin an expost sense. 6 A variety

restrictionson S,determinea causalordering from X to Y iff PY(A) 5 Y and PX(A > B) 5 PX(A).Geweke{1984} and Sims {1977} provide examples. The leading example is S 5 (x,y) [ R2 , x 5 a (correspondsto A), y 1 bx 5 c (correspondsto B). (A,B)isa causalordering from X to Y because A determines x withoutaffecting y. B along with A,determines y withoutfurther restricting x.Theremay be manypairs ofrestrictions on S thatproduce the same causal ordering. A versionof this examplewith uncorrelated error termsacross the two equationsproduces the causalchain model. TheSimon-Sims deŽ nition of acausalorder is fora givenpair of restrictions (A,B).Thenotion of causality is intimately involved with the idea of a stable relationship;i.e., that if A ischanged, the outcome will still be A > B with B (the input-outputrelation) unchanged. Otherwise, when A ischanged, a different causalordering may result. T oguaranteethat this does not occur, werequire the 2 1 followingcondition: for any A , S whichconstrains only X (i.e., PX (PX(A )) 5 A), (A,B)determinesa causalordering from X to Y.(Thisis sometimescalled ‘ ‘B # S’’ accepts X as‘ ‘input’’.) Thus,a fullspeciŽ cation of a causalmodel entails a descriptionof admissibleinput processes and the notion that B isunchangedwhen A ismanipulated(and hence the X ischanged). This deŽ nition can be modiŽ ed to applyonly to certain subsets, and not all A.Forthe model to be ‘ ‘correct,’’ theset B # S mustbe such that if B accepts X asaninput,and when any set C # X is 2 1 implemented( A ismanipulated),then PY(PX (C) > B)is‘ ‘true,’’ i.e.,in some sense depictsreality .Thismore general deŽ nition does not require that functions connectingcauses to effects be speciŽed. 6.From Billingsley {1986} we know thatif Y isa randomvariable, and X is a vector ofrandomvariables, then Y ismeasureablewith respect to X ifand only if Y 5 F(X).Thus,if we claim that an outcomeis ‘‘explained’’ by X inthissense, then arelationshiplike (1) is automatically produced. Saying that Y ismeasurablewith respect to X isnot enough to deŽ ne acausalfunction, however .If Y 5 X 1 Z, then X 5 Y 2 Z. Y ismeasurablewith respect to X and Z; X ismeasureable with respect to Y and Z.Economictheory produces causal functions in which the inputs (or externallyspeciŽ ed X variables)affect outputs. Different conceptual experiments deŽne different causal relations. Thus, consider a microeconomicdemand curve, where Y isthe quantity of agooddemanded and X isavector ofprice, income, and preferenceparameters. In the conceptual experiment where the agent is a price taker,andpreferences and incomes are externally speciŽ ed, Y 5 g(X) is the Marshalliandemand curve, andthe X arethe causal variables. In a different conceptualexperiment, the roles of these variables may be reversed. Thus, in a choiceexperiment examining ‘ ‘willingnessto accept’ ’ functions,quantity Y may be speciŽed externally,andthe minimum price the consumer would be willing accept togive up a unitof Y (acomponent of X)isthe outcome of interest. V ariationsin Y 54 QUARTERLY JOURNALOF ECONOMICS ofcausal mechanismscan be contemplated even in this simple setting,because variations in theprices of inputs and outputs can cause X tovary .All oftheparametric variations entertained in the microeconomictheory of the Ž rmare possible sources of causal variation. Theassumption that thecomponents of X canbe varied independently is strongbut essentialto the deŽ nition of acausal parameter.Theadmissible variationmay be local or global. 7 Restrictionson theadmissible variationof thevariables affect the interpretationof causal effects.For example, in aLeontief,or Žxed-coefficientproduction model, it is necessaryto vary all inputs toget an effectfrom any .Thus,an increasein Xj is necessaryto increase Y but is notsufficient. 8 Moregenerally , socialand economicconstraints operating on a Žrmmay restrict therange of admissible variationsso that aceterisparibus change inonecoordinate of X is notpossible. Entire classes of variations fordifferent restrictionson domain D canbe deŽ ned but in generalthese are distinct fromthe ceteris paribus variationsused todeŽne acausal law. 9 The domain D is sometimesjust one point as aconsequenceof the properties of a model,as Idemonstrate below.

cause acomponentof X to vary, say X1,thereservation price. Depending on the exactquestion, the answer to the second problem may ,ormaynot, be derivedby invertingthe g(X)functionspeciŽ ed in theŽ rst probleminterchanging the roles of Y andthe Ž rst componentof X, X1.Thus,if income effects are small, g(X) is the utilityconstant demand function. V arying quantitiesto produce associated marginalwillingness to accept valueswould entail inverting g to obtain X1 5 ˜ ˜ w (Y,X), where X 5 (X2, . . . , XJ),assumingthat a localimplicit function theorem is 2 1 satisŽed (so in particular ­ w /­ Y 5 (­ g/­ X1) , where ­ g/­ X1 Þ 0).However, if incomeeffects are nonzero, the causal function required to answer the willingness topay questioncannot be obtainedsimply by inverting g.Onewould have to derive theHicksian demand from the Marshallian demand and derive w from the Hicksiandemand. Ina productionfunction example, Y 5 F(X). If inputs X areexternally speciŽed, F isacausalfunction. T odeterminethe amount of X1 requiredto produce at output Y holding (X2, . . . , XJ)atprespeciŽ ed values, one would invert F to obtain X1 5 M(Y, X2, . . . , XJ),assumingthat ­ F/­ X1 Þ 0. M isacausalfunction associatedwith the conceptual thought experiment that Y, X2, . . . , XJ are externallyspeciŽ ed while X1 isdetermined. Thecrucial idea is that causal functions are derived from a conceptual experimentwhere externally speciŽ ed causes are varied. There are as manycausal functionsas thereare conceptual experiments. 7.A formaldeŽ nition of global variation independence is that the domain of D ¯ ¯ ¯ ¯ ¯ istheCartesian product X1 3 X2 3 X3, . . . , 3 XN , where Xi isthe domain of Xi and thereis no restriction across the values of the Xi. When the X termssatisfy this restriction,they are termed ‘ ‘variationfree.’ ’ Thelocal version imposes this requirementonly in neighborhoods. 8.This corresponds to the concept of the‘ ‘conjunctsof causality .’’SeeMarini andSinger {1988}. 9.One can deŽ ne many different restricted ‘ ‘effects’’ dependingon the restrictionsimposed on D. CAUSALP ARAMETERSAND POLICY ANALYSIS 55

Model(1) with norestrictionsamong the X deŽnes a modelof potentialoutcomes. This can be linked to models of causal effects based onpotential outcomes presented in the‘ ‘treatmenteffect’ ’ literatureby choosingthe X valuesto correspond to different treatments. 10 When(1) isseparable in X,wecan writeit as

N

Y 5 w j(Xj), j5 1 and thecausal effectof Xj canbe deŽ ned independently ofthelevel oftheother values of X.Suchseparability is especiallyconvenient if someof the Xj arenot observed, because it avoids theneed to deŽne causal parametersin termsof unobservedlevels of factors. Forthis reason,separable econometricmodels are widely used, and werethe exclusive focus of theCowles Commission analysts. Amajoradvance in thinkingabout causal parameterscame whenearly econometric analysts recognizedthe possibility that Y and someor all ofthecomponents of X couldbe jointly determined orinterrelated. This imposed severe restrictions on the causal parametersthat canbe deŽ ned in suchmodels because it restricts thepossibilities ofvariationin thecauses. The paradigm forthis analysis was amodelof marketdemand and supply:

(3) QD 5 QD(PD,Z D,U D) Demand

(4) QS 5 QS(PS,Z S,U S) Supply, where QD and QS arevectors of goodsdemanded and supplied at prices PD and PS,respectively.(Throughoutmuch of this paper, littleis lostexpositionally in thinkingof the Q and P as scalars.) Z D, Z S, U D, and U S areshifters of market demand and supply equations(i.e., determinants of demand and supply). Theyare determinedoutside of the markets where the P and Q are

10.The most direct way isto deŽ ne X1 asa treatmentindicator and to deŽ ne 5 5 Yx1 Fx1 (X2, . . . , XN)asthepotential outcome for treatment X1 x1. Thus, the modelsof potential outcomes of Neyman (1935), Fisher {1935}, Cox {1959}, and Rubin{1978} are versions of the econometric causal model. Galles and Pearl {1998} establishthe formal equivalence of thesetwo frameworks.Pearl {1998} presents a synthesisof thesetwo approachesusing directed acyclic graph theory.Thus,the contrastsometimes made between ‘ ‘structural’’ and‘ ‘causal’’ modelsformulated at theindividual level is a falseone. Imbens and Angrist {1994}present a precise formulationof the Rubin model. The statistical models ignore the constraints acrosspotential outcomes induced by social interactions and by resource con- straints,i.e., the potential restrictions on D.Heckmanand V ytlacil{2000} discuss therelationships among population treatment effect parameters, structural equationsmodels, and causal models. 56 QUARTERLY JOURNALOF ECONOMICS determinedand arecalled externalvariables. 11 The P and Q are called internalvariables. They may include distributions ofthe characteristicsof consumers and producers.The U arecauses not observedby theanalyst; the Z areobserved. In this sectionof the paper thereis nodistinction between Z and U.Thisdistinction is traditional and usefulin latersections, so I makeit here. In Marshall’s modelof industry equilibrium,(3) is thede- mand fora goodby arepresentativeconsumer while (4) is the supply functionof the representative price-taking Žrmthat maximizesproŽ t givenproduction technology (1) and factor prices.Assume that QD and QS aresingle-valued functions.If an equilibriumexists, Q 5 QD 5 QS, and P 5 PD 5 PS. If (P,Q) is uniquelydetermined as afunctionof the Z and U,themodel is said tobe‘ ‘complete’’ {Koopmansand Hood1953}. Themeaning of acausal parameterin (3) and (4) is thesame asin theanalysis ofequation(1). If pricesare Ž xedoutside of the market,say byagovernmentpricing program,we can hypotheti- cally vary PD and PS toobtain causal effectsfor (3) and (4) as partial derivativesor as Žnitedifferences of pricesholding other factorsconstant. 12 As in theanalysis ofthe production function, thedeŽ nition of a causal parameterdoes not require any state- mentabout what is actually observedor what canbe identiŽ ed fromdata. As before,the deŽ nition of a causal parameteronly requiresa hypotheticalmodel and theassumption that pricescan bevaried within therules speciŽ ed by themodel. A statistical justiŽcation of (3) and (4) interprets(3) as theconditional expecta- tion of QD given PD, Z D, and U D,and interprets(4) as the conditionalexpectation of QS given PS, Z S, and U S.13 Since we conditionon all causes,these conditional expectations are just deterministicfunctional relationships. The effect of PD on QD holding Z D and U D constantis different fromthe effect of PD on QD not holding U D constant;that is, E(QD PD,Z D,U D) Þ E(QD PD,Z D). In theearly investigations of causal models,and mostmodels in currentuse, linear equation versions of (3) and (4) wereused, so

11.The term external variable appears to originate in Wright {1934}. Frisch {1933}wrote about autonomous relationships. Given the numerous con icting deŽnitions of ‘ ‘exogenous’’ and‘ ‘endogenous’’ variablesdocumented by Leamer {1985},the ‘ ‘internal-external’’ distinctionis a usefulone for focusing on what is determinedin amodeland what is speciŽed outsideof it. 12.Both (3) and (4) have well-deŽ ned interpretations for their inverse functions.Thus, in (3), PD isthecompetitive price that would emerge if quantity QD weredumped on themarket. In (4), PS istheminimum price that competitive Žrmswould accept toproduce an externallyspeciŽ ed QS. 13.The justiŽ cation for this is givenin footnote 6. CAUSALP ARAMETERSAND POLICY ANALYSIS 57 causal parameterscould be deŽ ned independently ofthe levels of thecausal variables. As amatterof model speciŽ cation, we might require that candidate causal functionsobey certain restrictions. We might requirethat (3) and (4) haveat leastone solution P 5 PD 5 PS and Q 5 QD 5 QS,sothereis at leastone market equilibrium. Other restrictionslike positivity of(the diagonals of) ­ QS/­ PS (supply increasingin price) ornegativity of (the diagonals of) ­ QD/­ PD (downward sloping demand) mightbe imposed. In theanalysis ofequations(3) and (4), onespecial caseplays acentralrole. It is themodel that equatesdemand and supply.In theimportant special casewhen prices and quantities areas- sumedto obey an equilibriumrelationship, there is nomeaning attached toa ‘‘causal’’ effectof apricechange because the model restrictsthe domain ( D) of P and Q toa singlevalue if equilibrium is unique.Price and quantity areinternal (or endogenous) vari- ables jointlydetermined by the ZD, Z S, U S, and U D.External (or exogenous)variables ( Z D, Z S,U D,U S)determine( P,Q) but are not determinedby them. Holdingeverything else Ž xedin equilibrium(all otherdeter- minantsof demand and supply), theprices and quantities are Žxed.Thus, in equilibriumthe price of good j cannotbe changed unlessthe exogenous or forcing variables, Z D, Z S,U S, U D, are changed.More formally ,undercompleteness, we can obtain the reducedforms:

(5a) P 5 P(Z D,Z S,U D,U S)

(5b) Q 5 Q(Z D, Z S,U D,U S). Theconcept of an externallyspeciŽ ed variable is amodel- speciŽc notion.It entailsspeciŽ cation of (5a) and (5b) as causal relationshipsin thesense of (1) toreplace (3) and (4) when QD 5 QS and PD 5 PS.In afully nonparametricsetting, this requires that thevariables onthe right-hand sides haveno functional restrictionsconnecting them. 14 It alsoentails the notion that within themodel, Z D and Z S can beindependently varied foreach givenvalue of U D and U S (i.e.,it is possibleto vary Z D and Z S within themodel holding U D and U S Žxed).15

14.If functional forms (e.g., linearity) are maintained, some forms of depen- dencecan be tolerated (e.g., nonlinear relationships among the variables in a linearmodel). 15.Formally ,thesupport of ( Z D,Z S)isassumed to be the same for all values of (U D,U S).Inthis section the Z D and Z S entersymmetrically with U D and U S so 58 QUARTERLY JOURNALOF ECONOMICS

Assuming that somecomponents of Z D do notappear in Z S, that somecomponents of Z S donot appear in Z D,and that those componentshave a nonzeroimpact onprice, one can use the variationin theexcluded variables tovary the PD or PS in equations(3) and (4) whileholding theother arguments of those equationsŽ xed.With this variationone can deŽ ne the causal parametersof the effect of PD on QD and theeffect of PS on QS. S Assuming differentiable functions,and letting Z e bea variable excludedfrom Z D and fornotational simplicity assumingonly a singlemarket,

­ QD ­ Q ­ P 5 D S S , ­ P ­ Z e ­ Z e wherethe right-hand side expressionscome from (5a) and (5b). 16 D DeŽning Z e comparably,

­ QS ­ Q ­ P 5 S D D . ­ P ­ Z e ­ Z e Underthese conditions, we can recover the price derivatives of (3) and (4) eventhough an equilibriumrestriction connects PD 5 PS. Thecrucial notion in deŽning thecausal parameterfor price variation,when the market outcomes are characterized by an equilibriumrelationship, is variationin externalvariables that affect causes(the PD and PS,respectively,in theseexamples) but that do notaffect causal relationships(i.e., that areexcluded from weshould also require that the support of ( U D,U S)isassumed to be thesame for allvalues of ( Z D,Z S)or,moregenerally ,wemight require that all variables be variationfree in thesense of footnote 7. S 16.Proof: Differentiate (3) with respect to Z e to obtain

­ QD ­ QD ­ PD 5 S D S . ­ Z e ­ P ­ Z e

Usingequilibrium values ( PD 5 PS 5 P),substitutefrom (5a) to obtain D S S D S S ­ P /­ Z e 5 ­ P/­ Z e andfrom (5b) to obtain ­ Q /­ Z e 5 ­ Q/­ Z e .Assumingthat S ­ P/­ Z e Þ 0, we obtain

­ QD ­ Q ­ P 5 D S S . ­ P ­ Z e ­ Z e

S Ifthereare several Z e variablesthat satisfy the stated conditions, each deŽ nes the samecausal parameter . CAUSALP ARAMETERSAND POLICY ANALYSIS 59 therelationship in question). 17 If an externalvariable is excluded fromthe causal relationshipso it doesnot directly affect the causal relationship,the causal law is said tobe invariant with respectto variations in that externalvariable. If thevariable in questionis apolicyvariable, the causal relationshipis said tobe ‘‘policyinvariant.’ ’ Variationsin theincluded Z variables havedirect effects (holding all othervariables in (3) or(4) constant)and indirect effects(through their effects on theinternal variables via (5a) and (5b)). Thedirect effects of Z canbe computed by compensatingfor changesin the P induced by thechanges in theincluded compo- nents of Z byvarying theexcluded components of Z.18 Thesedirect and indirecteffects play acrucialrole in path analysis developed by Wright{1934} and widely usedin sociology(see Blalock {1964}).19 Thedirect causal effectsare called structural.Both directand indirecteffects are causal, and aredeŽ ned by well- speciŽed variations. Asaconsequenceof thepotentially interdependentnature of somecauses, a newterminology was created.Structural causal effectsare deŽ ned as thedirect effects of the variables in the behavioralequations. Thus, the partial derivativesof (3) and (4) aretermed structural (or causal) effects.When these equations arelinear, the coefficients on the causal variables arecalled structuralparameters, and theyfully characterizethe structural effects.In moregeneral nonlinear models, the derivatives of a structural(or behavioral) equation no longer fully characterize thestructural relationship. The parameters required to fully characterizethe structural relationship are termed structural parameters.A majordifference between the Cowles group, which workedexclusively with linearequations, and lateranalysts workingwith explicitly parameterizedeconomic models, is in the

17.The deŽ nition of a causalparameter crucially depends on independent variation.In the equilibrium setting under consideration, without an exclusion restriction,the equilibrium quantities cannot be independently varied. Thus, no independentvariation is possible. However ,ifwe consider a disequilibrium setting,where prices (or quantities) are set externally ,saythrough government policyor asocialexperiment, then the causal parameter can be deŽned, as before. 18.The required compensation for the excluded variables may not be achievableand depends on the curvature ofthe functions, the magnitude of the changein theincluded Z,andthe support of theexcluded Z. 19.Path analysis estimates the direct effectof structural variablesand the direct effectsof external variables as well as their indirect effects operating throughstructural variables.The ‘ ‘totaleffect’ ’ ofan external variable is the sum of thedirect effectand the indirect effects operating through all of theendogenous variablesin arelationship. 60 QUARTERLY JOURNALOF ECONOMICS deŽnition of a structuralparameter and theseparation between theconcept of a structural(or causal) effectfrom the concept of a structuralparameter. 20 Bothstructural equations and reduced-formequations can be usedto generate causal parameters.They differ in what is held constantin thesense of Marshall. Reduced-formrelationships can bedeŽ ned without the exclusion restrictions required to deŽ ne structuralrelationships. Functionalrelationships among variables that areinvariant tovariations in excludedexternal variables arecentral to the deŽnition and identiŽcation of causal laws in thecase when some variables ofasystemof equationsare interdependent. The notion ofinvariant relationshipsmotivated the Cowles Commission deŽnition of a structuralequation. It alsomotivated the economet- ricestimation method of instrumental variables using empirical counterpartsto thehypothetical relationships. Thesenotions all havecounterparts in dynamic settings, wherethe variables aretime-dated. Time-series notions of causal- ity as developedby Granger{1969} and Sims{1972}, areconceptu- ally distinct and sometimesat odds with thenotion of causality based oncontrolledvariation that is presentedin this paper and at theheart of the quotation from Marshall presentedin the introduction.The time-series literature on causality usestime dating ofvariables (temporalprecedence relationships) todeter- mineempirical causes and doesnot deŽ ne or establish ceteris paribus relationships.Thus letting t denotetime, past Yt is said to causefuture Xt if past Yt helps predict future Xt given past Xt using somestatistical goodness-of-Žt criterion.Such causality can ariseif future Xt determinespast Yt as oftenarises in dynamic economicmodels. The ‘ ‘causality’’ determinedfrom such testing proceduresdoes not correspond to causality as deŽned in this paper,and in this instanceis in directcon ict with it. 21 Thelimited role of the time-series tests for causality within articulated causal dynamic modelsis discussed by Hansenand

20.The term ‘ ‘deepstructural parameter’’ wasintroduced in the 1970s to distinguishbetween the derivatives of a behavioralrelationship used to deŽ ne causaleffects and the parameters that generate the behavioral relationship. 21.In a perfectforesight model like that of Auerbach and Kotlikoff {1987}, futureprices determine current investment.Time-series causality tests would revealthat investment ‘ ‘causes’’ futureprices which is precisely the wrong conclusionfor the concept of causality used in this paper .Hamilton{1994, pp. 306–309} presents an example in which Granger causalityis in oppositionto the causalinterpretation in the sense of this paper and another example in which Granger causalityis in accord withthe deŽ nition of causalityused in this paper. CAUSALP ARAMETERSAND POLICY ANALYSIS 61

Sargent{1980}. Thedynamic structuralmodels derived from economictheory of the sort analyzed by Hansenand Sargent providethe framework for deŽ ning causality as usedin this paper and forconducting counterfactual policy analysis. Idonot exposit thesemodels only because of my self-imposed limitation on the technicallevel of this paper.

II.2. IdentiŽcation: Determining Causal Modelsfrom Data Theformalization of models, the deŽ nition of causal and structurallaws, and thenotion of structural laws that are invariant with respectto variation in excludedexternal variables wereimportant contributions of economics.Even more important was theclariŽ cation of the limits of empirical knowledge. 22 An identiŽcation problem arises because many alternative structural modelsare consistent with thesame data, unlessrestrictions are imposed.Empirical knowledgeabout structural models is contin- genton theserestrictions. TheŽ rststudies ofthis problemwere in thecontext of the supply-demand modelof equations(3) and (4), assumingequilib- rium (PS 5 PD and QS 5 QD).Thiscase is still featuredin econometricstextbooks. The identiŽ cation problem is particularly starkin this settingif thereare no Z D or Z S variables,and if the U D and U S areset to zero, so there is noproblem of the unobservables U D or U S beingcorrelated with P or Q.23 In this case,two equations, (3) and (4), relating Q to P coexist(the demand curveand thesupply curve,respectively). They contain thesame variables. Empirically ,thereis noway todetermine eitherrelationship from the joint distribution of Q and P unless extrainformation (restrictions on models) is available. 24 Althoughthe identiŽ cation problem was Žrstsystematically exploredin this context,it is amuchmore general problem, and it is usefulto consider it moregenerally . 25 In its essence,it considers what particular modelswithin abroaderclass ofmodels are consistentwith agivenset of data orfacts. MorespeciŽ cally , considera modelspace M whichis theclass ofall modelsthat are consideredas worthyof consideration. Elements m [ M are

22.Other Želdsindependently developed their own analyses of theidentiŽ ca- tionproblem in more specialized settings {Koopmans and Reirsol 1950}. 23.IdentiŽ cation problems can arise even if there are no error termsin the model. 24.Morgan {1990} discusses early attempts to solve this problem using ad hoc statisticalconventions. 25.This framework is based on myinterpretationof Barros{1988}. 62 QUARTERLY JOURNALOF ECONOMICS possibletheoretical models. There are two attributes of amodel, correspondingto what onecan observe about the model in agiven setof data and what onewould liketo know about the model. DeŽne functions g: M T and h: M S, where T and S are spaces chosenso that g(M) 5 T and h(M) 5 S. S is thesource or data space, and T is thetarget space. For any speciŽc model m [ M, h(m) 5 s [ S representsthose characteristics of the model that canbe observed in theavailable data. Themap g(m) 5 t [ T applied tothe model gives the characteristics of amodelthat we would liketo identify .Someparameters may be of interestwhile othersare not, and modelsmay only be partially identiŽed. Only if oneis interestedin determiningthe entire model would T 5 M and then g would bean identity map. Many models m maybe consistent with thesame source space, so h is notone to one. In this abstract setting,the identiŽ cation problemis todetermine whether elements in T canbe uniquely determinedfrom elements in S.Elementsin T and S arerelated by the correspondence f g + h2 1.TheidentiŽ cation problem arises because for some s [ S, f (s)mayhave more than oneelement in T.In that case morethan oneinterpretati onof the same evidence is available. If we limitattention to T via g,ratherthan M, f is morelikely to map points of S intopoints of M.Thegoal of identiŽ cation analysis is toŽ nd restrictions R to modify f to f R so that f R(s)has at mostone element in T;i.e.,so that onlyone story about T is possiblegiven the data. In theusual formof the identiŽ cation problem, restrictions areimposed on theadmissible modelsin M so R # M. For each s [ S deŽne f R(s) 5 g(h 2 1(s) > R). If R is chosenso that forall s [ S, f R(s)has at mostone element in T, R formsa setof identifying restrictions.Thus, R # M identiŽes g from h when for any s [ S, f R(s) 5 g(h2 1(s) > R)containsat mostone element in T.26 If R is toorestrictive, for some values of s, f R(s) 5 B,theempty set, so R maybe incompatible with some s;i.e.,some features of themodel augmentedby R areinconsistent with somedata. In this case, R formsa setof overidentifying restrictions. Otherwise, if f R(s) containsexactly one element in T for each s [ S, R formsa setof just-identifying restrictions. 27

26.In principle, restrictions can also be placedon T butthese restrictions are oftenless easy to interpret. In the context of a demandfunction, T couldinclude boththe price and income elasticities, and we might restrict theincome elasticity to be known. 27.Note that f R(s) 5 B iff h2 1(s) > R 5 B sincewe assume that g(m) Þ B for all m [ M.Thus,a setof identifying restrictions forms a setof just-identifying restrictionsiff h(R) 5 S. CAUSALP ARAMETERSAND POLICY ANALYSIS 63

Fora givenset of data, it will usually benecessary to restrict themodel via R toproduce unique identiŽ cation. Alternatively , enlargingthe data (expanding thesource space S) may also produceidentiŽ cation. Enlarging S mayentail larger samples of thesame distributions ofcharacteristicsor expanding thedata to includemore variables. 28 In theclassical linear-in-parameters, simultaneous-equationsmodel, exclusion restrictions are used to deŽne R.Thesource space is thejoint distribution of( P,Q) given (Z D,Z S).Undernormality ,all ofthe information in thesource space is containedin themeans and covariancesof thedata. In thesupply-demand example,the model space M consists of all admissible modelsfor supply and demand,and thetarget space T couldbe the price elasticities of supply and demand.Even when U D 5 U S 5 0,the model of equations (3) and (4) is not identiŽed if Z D 5 Z S.As Žrstnoted by Tinbergen{1930, reprinted 1995}, if Z D and Z S eachcontain variables notin theother ,but that determine P and Q via (5a) and (5b), themodel is identiŽed. In this deterministicsetting, the variation in P induced by the externalvariables in Z that arenot in theequation being analyzed substitutesfor variation in P that would occurif marketequilib- riumconditions did notgovern the data. If thereare unobserv- ables in themodel, their effects on Z mustbe controlledto use this variation.Conventional statistical assumptionsmade to effect suchcontrol are that ( U D,U S)arestatistically independentor meanindependent of ( Z D,Z S).29 Onesample counterpart to this identiŽcation argument is the methodof instrumental variables. 30 Variationin theexcluded Z inducesvariation in the P.Thisvariation is essentialwhether or notthere are error terms in themodel. Many different estimation methodscan be used to induce the required variation in thedata. An instrument—deŽ ned as asourceof variation—need not neces- sarily beused as an instrumentalvariable toempiricallyidentify causal parameters.Social experiments that donot alter the

28.Most modern analyses of identiŽ cation assume that sample sizes are inŽnite so that enlarging the sample size is not informative. However ,inany appliedproblem this distinction is not helpful. Having a smallsample (e.g., fewer observationsthan regressors) can produce an identiŽcation problem. 29.Mean independenc eisthe condition E(U D Z D, Z S ) 5 0, and E(U S Z D,Z S) 5 0,or ,moregenerally ,thatthese means do notdepend on Z D and Z S. 30.The method was independently developed by Wright {1928} and Reirsol {1945}.Epstein {1987} speculates that Sewall Wright, the famous geneticist, inventorof path analysis and son of Philip Wright, was the likely inventor of this method.See also Morgan {1990} and the intellectual history reported in Gold- berger{1972}. 64 QUARTERLY JOURNALOF ECONOMICS structuralrelationship being studied caninduce the required variation.31 Themethod of control functions that explicitly models thedependence of theerror term on the right-hand side variables cando so as well. 32 Amuchricher class ofrestrictions R can also produceidentiŽ cation including restrictionson covariances, restric- tionson coefficients,and restrictionson functional forms {Fisher 1966}. In dynamic models,cross-equation restrictions produced fromtheory and restrictionson the time-series processes of the unobservablesprovide additional sourcesof information{Hansen and Sargent1980}. Restrictionson thecovariances of theinnova- tionsof vector autoregressive models play acrucialidentifying rolein VARframeworks{Sims 1980}. Amajorlesson of theeconometric literature on identiŽcation that is still notwell understood by manyempirical economists is that just-identifying assumptionsare untestable. Different restric- tions Ri, i 5 1, . . . , I that securejust-identiŽ cation produce I different storiesabout the parameters in thetarget space T for the sameelement s in data sourcespace S.Differentadmissible models m producedby different Ri that arejust-identifying are compatiblewith thesame elements of thesource space S, and all empiricalknowledge is in S.Accordingly,noempirical test of just-identifying restrictionsis possible.They must be justiŽ ed by anappeal tointuition or priorinformation, and thejustiŽ cation is notstatistical innature.The extra restrictions on the source space are testable.T estsof identiŽ cation reported in theempirical literatureare always testsof overidentifying assumptions and theyare based onmaintained identifying assumptionsalthough theyare frequently and confusinglyreferred to as testsof identifying conditions. 33

31.Heckman, LaLonde, and Smith {1999} show that randomization is an instrumentalvariable. 32.Instead of purging theendogenous variables of the errors, this method modelsthe errors in terms of theright-hand-side endogenous variables and all of theregressors. In a two-equation,simultaneous-equations system: (*) Y1 5 a Y2 1 b Z1 1 U1 and Y2 5 g Y1 1 d Z2 1 U2, where E(U1,U2 Z1,Z2) 5 0,the method of controlfunctions forms E(U1 Y2,Z1,Z2,w ), where w issome unknown parameter vector andenters it as a conditioningfunction in (*): Y1 5 a Y2 1 b Z1 1 E(U1 Y2, Z1, Z2,w ) 1 (U1 2 E(U1 Y2, Z1, Z2,w )).Withsufficient variation in E(U1 Y2,Z1,Z2,w ),whichin the absence of parametric restrictions clearly requires variationin Z2, when Z1 isheld Ž xed,it is possible to identify a , b , and w . See Heckmanand Robb {1986}. Nonparametric versions of the method exist. The methodunderlies an entire class of nonparametricselection bias estimators. See Heckman,LaLonde, and Smith {1999} and Heckman and V ytlacil{2000}. 33.T estsfor endogeneity in simultaneous-equations models are always predicatedon the existence of at least one just-identifying exclusion restriction thatis used as an instrument. CAUSALP ARAMETERSAND POLICY ANALYSIS 65

Modelsthat arenot fully identiŽed may contain subsets of parametersthat canbe identiŽ ed. As Žrstnoted by Wright{1934} and Marschak and Andrews {1944}, modelsthat arenot identiŽ ed maystill containinformation on ranges of structural parameter values.This insight is afocusof recent research which I discuss below.

II.3. EconometricPolicy Evaluation: Marschak and theLucas Critique In thecontext of structural economic models, identiŽ cation necessarilyprecedes testing of speciŽc causal hypotheses.Thus, if an effectcannot, in principle,be identiŽ ed, it is notpossibleto test whetherthe effect is presentin thedata. 34 Structuralparameters havea cleareconomic interpretation. Estimates of them can be usedto test theories if identifying assumptionsare maintained, to interpretempirical relationships, to perform welfare analysis (e.g.,compute consumer surplus), and toimprove the efficiency of estimatesand forecastsif modelsare overidentiŽ ed. They can also beused as frameworksfor empirical analyses that facilitate the accumulationof evidence across different studies.These beneŽ ts areto be set against thecosts of makingidentifying assumptions. OnebeneŽ t that motivatedmuch of the early econometric literature,and onethat still motivatesmuch econometric re- search,is theability touse structural models to predict the consequencesand evaluatethe effects of alternative economic policies.Formal econometrics was perfectedby economistsgreatly inuenced by theGreat Depression. Early pioneerslike Tinbergen and Frischwere policy activists whowere optimistic about the ability ofenlightenedsocial planners tocontrol the business cycle throughthe introduction of new policies based onempirically justiŽed econometricmodels. 35 Theclearest statement of the beneŽ t ofstructural economet- ricmodels for social planning is by Marschak{1953}. Laterwork by Lucas{1976} builds onMarschak’ s analysis by updating it to incorporateexplicit modelsof intertemporal decision making underuncertainty .

34.If the causal effect is identiŽ ed within a rangeof valuesthat excludes the no-effectvalue, exact identiŽ cation of the causal effect is not required to test the nullhypothesis of no-effect. One simply asks whether the identiŽ ed rangeincludes thevalue implied by nullhypothesis. 35.For an illuminating discussion of thework ofFrischand his commitment tosocial planning, see Bjerkholt {1998}, Chipman {1998}, and the other papers in Strom{1998}. 66 QUARTERLY JOURNALOF ECONOMICS

Thegoals of econometricpolicy evaluation are to consider the impact ofpolicy interventions on the economy ,tocompute their consequencesfor economic welfare, and toforecast the effects of newpolicies never previously experienced. If apolicyhas previ- ouslybeen in place,and it is possibleto adjust outcomevariables forchanges in externalcircumstances unrelated to thepolicy that alsoaffect outcomes,econometric policy evaluation is justa table look-upexercise. Structural parameters are not required to forecastthe effects of the policy on outcomes, although causal effectsmay still bedesired for interpretive purposes. It isenough toknow the reduced forms (5a) and (5b). What happened before will happen again. If thesame basic policyis proposedbut levelsof policyparameters are different fromwhat has beenexperienced in thepast, someinterpolation or extrapolationof past relationships is required.A disciplined way todo this is toimpose functional formson estimatingequations. Thetheory of econometric policy evaluation typically pro- ceedsby assumingthat policyparameters are external vari- ables.36 Underthis assumption,one can use reduced forms like (5a) and (5b) toforecast the effects of policies on outcomes, using historicalrelationships to predict thefuture. Marschak’s casefor knowing structural parameters was for theiruse in predicting theeffects of anewpolicy ,neverpreviously experienced.Such a problemis intrinsically difficult and ofthe samecharacter as theproblem of forecasting the demand fora newgood, never previously consumed. Without the ability to projectpast experienceinto knowledge of thefuture, the problem is intractable.The essence of this problemis succinctlysumma- rized in aquotationfrom Knight: ‘ ‘Theexistence of aproblemin knowledgedepends onthe future being different fromthe past, whilethe possibility ofa solutionof a problemof knowledge depends onthefuture being like the past’ ’ {Knight, 1921, p. 313}. Marschak analyzed aclass ofpolicies that canbe evaluated using structuralequations. The prototype was theanalysis ofthe effectof a commoditytax neverpreviously experienced on equilib- riumprices and quantities.Since there is noprevious experience with thepolicy ,thetable look-upmethod is notavailable tosolve this problem. Marschak operationalizesKnight’ s quotationby notingthat

36.Marschak noted, but did not develop, the notion that policies themselves mightbe internalor endogenous. CAUSALP ARAMETERSAND POLICY ANALYSIS 67 in generala commoditytax changesthe price of a good.For a proportionaltax t ,theafter-tax pricepaid by aconsumeris P* 5 P(1 1 t ), where P is theprice received by producers.With structural equationsin hand, onecan in principle accuratelyforecast the effectof a newtax onmarket aggregates using structuralequa- tions(3) and (4) modiŽed to incorporate the tax wedge.Simply insertthe tax wedgeand solvethe structural equations for the newequilibrium prices and quantities. Underthe conditions sketched in subsectionII.2, itis possible touse external variation in excludedvariables in theprepolicy periodto identify theseequations from equilibrium observations. Apotentially seriousproblem is that thefunctions (3) and (4) may benonlinear ,and therange of historicalprice variation may not besufficient totrace out those functions over the range of values that arerelevant in thenew policy regime. Marschak avoids this problemby assuminglinear structural equations. In his case,a Žniteset of data points determinesthese equations over their entiredomain of deŽ nition. A fully nonparametricapproach would requiresufficient variationin P in thepretax sampleused to Ž t thestructural equations to encompass the relevant support of P in theperiod of thepolicy . By estimatingthe structural parameters in theprepolicy periodand by linkingthe policy variation to price variation, Marschaksolves the problem of forecastingthe effect of a newtax policyon market aggregates. It is importantto note that bothof his steps arenecessary .Knowledgeof structuralparameters is not enough.It is alsonecessary to somehow link the variation that will beinduced by thenew policy to some variation within the modelin theprepolicy sample period so ‘ ‘thefuture will belike the past.’’ In Marschak’s example,tax and pricevariation are on the samefooting. If this linkcannot be made, the problem of forecasting the impact ofa newpolicy cannot be solved using an econometric model.Consider a policythat injectsrandom taxation intothe deterministicenvironment considered by Marschak.If theuncer- tainty induced by thepolicy is ofa fundamentally different characterthan anything previouslyexperienced, an empirically based solutionto the policy forecasting problem is intractable. 37

37.The parallel problem of estimating the demand for a newgood entails the samedifficulties. Lancaster {1971} solves the problem by assumingthat new goods arerebundlings of the same characteristics as in the old goods and proposes estimationof the demand for characteristics and distributions of population 68 QUARTERLY JOURNALOF ECONOMICS

In his seminalarticle Marschak madeanother important point.Different criteria for evaluating apolicyrequire different parameters.(Different policiesrequire knowledge of different targetspaces T.)Givenan assumptionthat describesa proposed policyin termsof variationexperienced in aprepolicyperiod, the answersto different economicdecision problems require different structuralparameters or combinationsof structuralparameters. It maynot be necessary to determinethe full structureor to know any particular structuralparameter to answer a particular policy evaluationquestion. Thus, if weseek to determine the effect of an externallydetermined price change on consumer welfare, it is onlynecessary to identify thedemand curve,a weakerrequire- mentthan full systemidentiŽ cation. Marschak presentsan extensivediscussion of decision problems requiring different amountsof structural information including somewhere only ratiosof someof thestructural parameters are required. Aquarterof a centurylater, Lucas {1976} returnedto the policyevaluation problem addressed by Marschak.Among other contributions,Lucas updated Marschak’s analysis toincorporate explicit modelsof decision making under uncertainty and to incorporateendogenous expectations. Lucas criticized the prac- tice ofeconometric policy evaluation in thelate 1960s and early 1970s becauseit was carelessin modelingthe expectations of agentsand did notaccount for the effects of policychanges on the expectationsof the agents about future shocks. He noted that different stochasticprocesses for external forcing variables pro- duced by different economicpolicies would in generalinduce different behavioralresponses and that an adequate modelof policyforecasting should accountfor this. Econometricmodels that ignoreexpectations are misspeciŽ ed or‘ ‘incomplete’’ in theprecise sense of the Cowles economists. A missinginternal or endogenous variable would makean esti- matedmisspeciŽ ed ‘‘structure’’ appear tobeempirically unstable whenthe distribution oftheforcing variables was changed,say by changesinduced byneweconomic policies. Lucasclaimed that this type ofmodel misspeciŽ cation ac- countedfor the parameter drift observedin manyempirical macro preferenceparameters to generate forecasts. Quandt {1970} and Domencich and McFadden{1975} use similar schemes in theanalysis of discrete choice to forecast thedemand for new goods. In Domencichand McFadden, unobserved attributes of newgoods are assumed to be independentrealizations of unobservedattributes for old goods. CAUSALP ARAMETERSAND POLICY ANALYSIS 69 models,although the empirical evidence for his claimis controver- sial.38 Thesource of theparameter instability is modelmisspeciŽ - cationdue to omitted endogenous expectations variables and any omittedexternal forcing variables associatedwith theequation determiningexpectations. 39 Thus,estimates of misspeciŽed struc- tural equationswill appear tobe noninvariant in responseto changesin thedistributions oftheforcing variables. Such changes maycome from changes in policies,but this is onlyone possible sourceof changein theforcing variables. 40 Torecastthe Lucas critique in thelanguage of a classical linearsimultaneous equations model, write

(6) G Y 1 BX 5 U, where Y is avectorof endogenous variables, X is avectorof exogenousor external forcing variables, and U is avectorof unobservedforcing variables. If G has an inverse,the system is said tobe complete, and areduced-form Y as afunctionof X is welldeŽ ned. As notedby Hansenand Sargent{1980}, modern dynamic theoryin generalrequires nonlinear models with cross- equationrestrictions and lagged (and sometimesfuture) valuesof Y, X, and U,so(6) is toosimple to capture the class ofdynamic structuralmodels discussed by Lucas. However,it is usefulto make Lucas’ point in this frameworkif onlyto make an analogy.Todoso, partition Y 5 (Y1,Y2) into two componentscorresponding to the included and omittedendoge- nousvariables. Think of Y2 as theexpectations variables deter- minedby themodel under rational expectations. Then partition

G 11 G 12 B1 G 5 , B 5 , G 21 G 22 B2

X1 U1 X 5 , and U 5 X2 U2

38.See the evidence in Fair{1994}. The evidence that policy shifts affect the parametersof Ž ttedmacro models is atbest mixed. However ,thesupply shocks of the1970s deŽ nitely affected the stability of Ž ttedmacro relationships. 39.As demonstrated, e.g., by White {1987}, estimated misspeciŽ ed models willappear to exhibit parameter instability if the distributions of the external forcingvariables change over the sample period. 40.Lucas compares two steady states— one before the policy change and one after—and does not analyze the short-run responses to the regime shift that are requiredfor explaining the time-series data accompanying a regimeshift. 70 QUARTERLY JOURNALOF ECONOMICS conformably.Assumingthat G 11 has an inverse,

2 1 2 1 2 1 Y1 5 2 G 11 G 12Y2 2 G 11 B1X1 1 G 11 U1.

By omitting Y2 and any determinantsof Y2,themisspeciŽ ed model for Y1 will exhibitwithin-sample instability if thedistribu- tionof theforcing variables changesin different sampleperiods. Thesechanges could arise because of policy interventions or becauseof external shocks to the economy unrelated to policy interventions.If changesin theforcing variables aremodeled alongwith theunobserved expectations of themodel, there should benodrift in estimatedparameters. 41 Marschakestablished that structuralmodels are only a necessaryingredient for evaluating anewpolicy .Evenif correctly speciŽed structural equations are estimated, econometric policy evaluationwill beineffective in evaluating newpolicies that involvevariation in variables that previouslydid notvary and that cannotbe linked to variables inthemodel that havevaried. Theproblems of forecasting the effects of a newpolicy are profound,and manywould arguethat theyare impossibly hard. Inadvertently,theCowles group fashioned apowerfulargument against thepossibilities ofempirically based socialplanning— a majorgoal of theearly econometricians.

III. THE COWLES RESEARCH PROGRAM AS A GUIDE TO EMPIRICAL RESEARCH AND RESPONSES TO ITS PERCEIVED LIMITATIONS By themid-1950s theCowles research program deŽ ned mainstreamtheoretical econometrics. It was widely recognized that it neededto be augmented to include a richerarray of dynamic modelsand toaccountfor serial correlation in aggregate timeseries. Granger and Newbold{1977}, Zellnerand Palm {1974}, and Nerlove,Grether ,and Carvalho {1979}, amongothers, madeimportant extensions of theCowles Commission framework totime series settings integrating the work of Frisch{1933} and Slutsky{1937} onpropagation mechanismsand thedynamic

41.Wallis {1980} presents a systematicanalysis of the formulation and estimationof the rational expectations models within a linearin-parameters Cowles-likemodel augmented to account for serial correlation in the equation errors.He does not, however ,developthe consequences of modelmisspeciŽ cation thatare discussed in this paper. Readers of his paper will recognize that the ‘‘reection problem’ ’ thatarises in identifying models of social interactions is closelyrelated to his analysis of equilibrium systems with self-fulŽ lling expecta- tions. CAUSALP ARAMETERSAND POLICY ANALYSIS 71 stochastictheory of Mann and Wald {1943} and Phillips (1966) intothe Cowles Commission framework. Thesources of identiŽ cation for the macro models were controversial.In an inuential paper Liu{1960} arguedthat the exclusionrestrictions used in theCowles models were artiŽ cial, that economictheory would requirethe inclusion of many more variables than thosefound in theexisting models of theday ,and that structuralmodels were fundamentally underidentiŽed. He carefullyscrutinized thead hocnature of identiŽ cation in the inuential Klein-Goldberger model {1955}. In his view,thequest forstructural estimation was quixotic,and hencethe goal of estimatingthe impact ofa newpolicy was an impossibledream. Headvocated theuse of reduced-form equations for making economicforecasts. As subsequentgenerations of macro models based ontheCowles program became larger ,thecriticism of them becamemore intense. Their computational complexity and fragile identiŽcation attracted unfavorablediscussion that elaboratedon thenegative commentary of Liu (see the discussion in Brunner {1972}). At thesame time, as documentedby Epstein {1987} and Morgan{1990}, theCowles models were perceived to be empirical failures.Estimated parameterinstability and thepractice of reŽtting empiricalmacro models over short time periods indicated that theofficial rhetoricof econometrics was notfollowed in practice.Naive forecasting models often did betterthan fully articulated structuralmodels estimated under the Cowles para- digm.This evidence motivated the ‘ ‘Lucascritique’ ’ as onepossible explanation forthe observed parameter instability . Althoughthe Cowles Commission was successfulin pressing thepotential importance of endogeneity bias (thefeedback be- tween the U and the Y in equation(6)), theempirical importance ofthis problemcompared with theother problems in estimating macromodels was neverclearly established. Indeed Basmann {1974}, oneinventor of the method of two-stage least squares, claimedthat endogeneitybias was ofsecond-order importance comparedwith thebasic measurementerror in themacrodata. In his invited lectureat the1970 WorldCongress of theEconometric Society,heplotted measurementerror boxes around the least squaresand endogeneity-correctedestimates of theconsumption functionfrom an inuential article by Haavelmo{1947}. The slight changein theOLS Žtted linethat arosefrom correcting for simultaneitybias was dwarfed by theintrinsic variation in the 72 QUARTERLY JOURNALOF ECONOMICS data dueto measurementerror . 42 Simultaneitybias was asecond- orderproblem in Haavelmo’s data, althoughit was thefocal point ofCowleseconometrics. 43 Tworadically different responsesto the perceived failure of theCowles research program emerged. One response was the developmentof more explicitly parameterizedstructural eco- nomicmodels. The other response, developed by Granger{1969} and Sims{1972,1977,1980,1986}, was based onsystematicappli- cationof vectorautoregression time-series methods that had been fully developedafter the basic Cowlesprogram had beencom- pleted but whichoriginated in thework of Mann and Wald {1943}. Thevector autoregression approach (VAR) econometricallyimple- mentedFrisch’ s vision{1933} ofdynamic propagation mecha- nismsfor business cycle research. Aroughcharacterization of thetwo responses is that structur- alists adheredto and reŽned the theory often at theexpense of obtaining goodŽ ts tothedata. VAReconometriciansstuck to the data as summarizedby vectorautoregressions and usedeconomic theoryas an informalguide forinterpreting statistical decomposi- tions.Covariance restrictions on time-series processes replaced thea priorirestrictions on behavioral equations invoked by the CowlesCommission. Innovation accounting replaced causal analysis. Structuralists adopted adeductive,theory-orie nted ap- proach,that estimatedstructural models with testableimplica- tionsthat werefrequently rejected in thedata. Therejections wereto be used as abasis forimprovements in themodels, althougha precisemechanism for learning from the rejections was neverspeciŽ ed. Interpretability ,counterfactualpolicy evalua- tion,and welfareanalyses were featured in this approach. Advo- catesof VARapproachesstarted with modelsthat Žtthedata and imposedminimal ground-up restrictionson timeseries processes that wereonly loosely motivated by economictheory .Bothgroups

42.Appreciation of the importance of measurement error ineconomic data goesback tothe work ofMorgenstern{1950}. The importance and consequences of measurementerror inmicroeconomicdata is a runningtheme of the work ofZvi Griliches.For a recentcomprehensive analysis of measurement error insurvey data,see Bound, Brown, andMathiowetz {2000}. 43.A quotationfrom Klein {1960}, as presented in Epstein {1987, p. 119}, reinforcesthis point: ‘ ‘Thebuilding of institutional reality into a prioriformula- tionsof economic relationships and the reŽ nement of basic data collection have contributedmuch more to the improvement of empirical econometric results than havemore elaborate methods of statisticalinference.’ ’ CAUSALP ARAMETERSAND POLICY ANALYSIS 73 retainedthe original Cowles emphasis of using state-of-the-art formalstatistical methodsto describe the data. TheV ARapproach starts with avectorautoregression for a k-dimensionalvector Zt:

(7) Zt 5 C1Zt2 1 1 C2Zt2 2 1 · · · 1 CqZt2 q 1 e t, where E(e te 8t) 5 V, E(e t) 5 0 and e t is uncorrelatedwith all variables dated t 2 1and earlier.Understandard conditions,

C1, . . . , Cq and V canbe estimated by leastsquares. Estimates of(7) summarizethe time-series data by matricesof regression parametersand thevariance V.Thisframework extends the Cowlesmodel of equation(6) toa time-seriessetting. The ‘ ‘struc- tural’’ versionof (7) is

(8) A0Zt 5 A1Zt2 1 1 · · · 1 AqZt2 q 1 Ut, where A0 is assumedto be nonsingular and the Ai are k 3 k matricesof constantsand E(UtU 8t) 5 D and A0e t 5 Ut.Theoriginal Cowlesmodel assumed that Ai 5 0, i . 1,

G B Z 5 {Y ,X }, and A 5 . t t t 0 0 I

The‘ ‘0’’ in thelower left block made Xt exogenousor external—it determined Yt but was notdetermined by it. Equations (7) and (8) extendthe Cowles framework to accountfor serial correlation and generaldynamic responses acrossequations. Equation (7) and (8) areconnected by the 2 1 2 1 2 1 relationships Ci 5 A0 Ai and V 5 A0 D(A0 )8.Restrictionson the Ai and D serveto identify thestructural model (8). Thelink to an explicit dynamic economictheory is at bestweak {Hansen and Sargent1991}. FullyspeciŽ ed dynamic economicmodels can rarelybe cast in theform of equations (7) and (8). Innovationaccounting— estimating the effects of innovations in Ut onthetime paths ofthe Zt—is prominentlyfeatured in this literature.Such accounting takes the place ofsimple comparative statics exercisesin theoriginal Cowles models and tracesout the dynamic responseof exogenousshocks on the outcome variables using (8). In orderto recover the Ut fromthe estimated e t to performsuch accounting exercises, it is necessaryto impose some structureon themodel (8). WhileSims {1980} and otherscriticize thepractice of ‘ ‘incredible’’ identiŽcation as practiced by Cowles econometricians,to outsiders the substitute sources of identiŽca- 74 QUARTERLY JOURNALOF ECONOMICS tionadvocated in this literaturelook no more credible and often appear tobe of the same character . Many different identifying conventionsonly loosely moti- vated by economictheory have been assumed. The leading approach is toadopt acausal chain approach and toassume that D is diagonal and A0 is triangular.As notedby manyanalysts, suchrestrictions are intrinsically arbitrary.Otherconventions, as summarizedin Christiano,Eichenbaum, and Evans {1999}, in- cludeimposing other restrictions on the Ai and the D to accord with featuresof the model viewed to be intuitively satisfactory suchas long-runneutrality of certain variables. 44 (See also Shapiro and Watson{1988}.) Alternativeidentifying assumptions producedifferent estimatesof theimportance of theinnovations. Thegoals of estimating the impacts ofnew policies never previ- ouslyexperienced or of assessing the welfare consequences of existingpolicies are abandoned. 45 Vectorautoregression data summariesare often not transparent tooutsiders. An air of mysteryand controversysurrounds the generation of the ‘ ‘facts’’ assummarizedby (7). Theappeal tomodern time-series methods appears tosubstitute the black boxof CowlesidentiŽ cation with theblack boxof time-seriesidentiŽ cation. Asecondresponse to the perceived empirical failure of the Cowlesresearch program was todevelop more explicit structural modelsand toexploitnew sources of microdata. About thesame timethat disillusionmentwas settingin with theCowles meth- ods,a vast newarray ofmicrodata in theform of panels and cross sectionsbecame available toeconomists. Orcutt {1952} had force- fully arguedthat time-seriesvariation was toolimited to empiri- cally recoverthe parameters of structuralmodels and that useof microdata crosssections and panels would greatlyaid in this regard.New sources of microdata became available, in part throughthe pioneering efforts of theInstitute forSocial Research at theUniversity of Michigan, and in part becausegovernment agenciesdisseminated microŽ les from Census and CurrentPopu- lationsurvey data. Microeconometricsbegan to  ourishas aseparateŽ eld. Regressionanalysis as synthesizedby Goldberger{1964} was the toolof choice for analyzing thesedata. Awealthof empirical

44.Neutrality restricts certain sums of coefficients to be zero. 45.Sims {1986} proposes a formof timeseries interpolation/ extrapolationto assessthe impacts of new policies, but does not discuss the construction of welfare measures. CAUSALP ARAMETERSAND POLICY ANALYSIS 75 regularitieswas presented,and neweconometric methods were developedto solve problems of censoring, self-selection, and limiteddependent variables that arosein theanalysis ofmicro data, and todevelop new methods to exploit cross-section and panel data.46 Withthe advent ofcomputers, the pace of empirical work increaseddramatically in macro-and .This for- everchanged the scale of the empirical enterprise in economics and createdthe phenomenon of asoftware-drivenapplied econo- metrics.Long tables ofregression coefficients with different conditioningvariables becamecommonplace in researchpapers and interpretationof estimateswas difficult. 47 ‘‘Holdingvariables constantin thesense of linearregression’ ’ becamethe operational empiricalcounterpart of Marshall’ s ceterisparibus clause.Endo- geneitybias was always aconcern,and manyof the careful empiricalstudies oftheday accountedfor such bias, althoughthe precisionof the estimates was usually greatlyreduced when correctionsfor endogeneity were made, especially in themicro studies whereinstruments were usually weak. The oodof oftenuninterpretable estimates that arosefrom this researchactivity produced anewdemand forstructural modelsto organize the data and facilitate comparisonsof esti- matesacross studies. Jorgenson’ s {1963} powerfullysimple model ofthedemand forcapital showedthe value of structural economet- ricmodels for interpreting data and focusingempirical research onessential economic ideas. In onevariable— the user cost of capital—Jorgenson was able tosummarize the essential features ofan importanttheory of investment. The simplicity and the eleganceof his empiricalanalysis contrastedsharply with the longand uninterpretabletables ofregressioncoefficients reported in studies ofinvestment prior to Jorgenson’ s. Although serious empiricalobjections were raised against his theory,its interpret- ability was neverquestioned. Mincer’sstudy offemale labor supply {1962} was equally inuential in demonstratingthe power of a simplestructural modelto organize evidence and predict phenomena.By introduc- ing wageand incomeeffects into the empirical analysis oflabor

46.Amemiya {1985} and Hsiao {1986} provide comprehensive discussions of thesenew methods. 47. in a privateconversation recalled the days at the NBER inthe 1930s when extensive formal discussions were required to justify the substantialcost of addingan additional regressor to an equation. 76 QUARTERLY JOURNALOF ECONOMICS supply,hewas able toreconcile empirical anomalies between time-seriesand cross-sectionestimates of female labor supply , and was able toexplain thetime series of female labor supply using aparsimonious,economically interpretable framework. Theinterpretability of structuralestimates and thevalue of structuralmodels in constructingcounterfactuals spurred the structuralestimation movement in the1970s and early1980s. 48 The‘ ‘LucasCritique’ ’ suggestedthat properlyspeciŽ ed structural modelswould avoid theproblem of parameter drift in estimated macromodels. This instability was especiallypronounced in the estimationof the ‘ ‘Phillip’s Curve’’ in thelate sixties and early seventies.The quest for policy invariant structuralequations in macroeconomicsstimulated aoodof theoretical papers and a trickleof empirical ones. 49 Aparallel movementemerged in microeconomicswhere research on labor supply ,selectionbias, thereturns to schooling, the causal effectof unionson wages,and thedeterminants of unemployment  ourished. Thetheoretical distinctions introducedin thepost-Cowles Commissionstructural estimation literatures are of fundamental lasting importance.They demonstrate what canbe extracted from thedata provided that sufficient priorknowledge is assumed. Thesemodels extend the Cowles paradigm by specifying prefer- ences,technology ,and expectationsand by addressing theidenti- Žcationof structural parameters in avarietyof empiricalsettings. At thesame time, the empirical track record from the structuralresearch program is notimpressive. Computational limitationshave hindered progress. Typically onlythe simplest of structuralmodels can be estimated, and theseare often rejected in thedata, apoint vigorouslyemphasized by advocatesof the VARprogram.Few structural models have been estimated that systematicallyaccount for the vast array ofeconomicpolicies that confrontthe agents being analyzed. In practice,analysts seeking policyinvariant structuralparameters to conduct policy evalua- tiontypically haveignored the policies in place whenestimating thestructural parameters. Moreover, the functional forms that facilitate structuralestimation are often inconsistent with the data. Eulerequation estimation methods developed in the1980s

48.This dating is onlyrough. In industrialorganization, structural modeling beganto be practiced inthe late 1980s and is anactivefrontier area of researchin that Želd. 49.See the collection edited by Lucas and Sargent {1981}. CAUSALP ARAMETERSAND POLICY ANALYSIS 77 and 1990s becamepopular becausethey avoid thecomputational costof full structuralestimation and enableanalysts toestimate onestructural parameter under weaker assumptions than are requiredto estimate full structuralequation systems. In this sense,the Euler equation methods are more robust than full systemstructural estimates. 50 By estimatinga fewparameters of empiricalmodels that arenot capable ofgenerating forecasts of thelevels of the variables, it is possibleto avoid embarrassing confrontationswith thedata. At thesame time, pursuit ofthis researchprogram was tantamountto abandonment of theCowles programfor macro policy evaluation. 51 In manyquarters of economics,the evidence from structural econometricmodels is held tobe unreliable.An inuential book by aleading empiricallabor , Lewis {1986}, is typical ofthe responseto structural econometrics by manyempirical econo- mists.Reviewing estimates of the structural (causal) effectof unionismon wages that correctfor self-selection into union status—the average effect of unionismon thosewho were union- ized holding theirobserved and unobservedpersonal characteris- ticsŽ xed—Lewis found awide rangeof estimates from different econometricmethods, and different conditioningvariables. This variability led himto dismiss structuralequation methods as unreliable,and toadvocate a returnto more familiar leastsquares methodsthat apparently generatemore robust estimates. Similar Žndings werereported for the estimates of structurallabor supply parameters{Killingsworth 1983, Table4.3}. In fairnessto the structural approach, someof the variability reportedin theempirical literature arises from the imposition of false overidentifyingrestrictions about distributions and unobserv- ables that couldhave been tested and relaxed,but typically were not.Part of thevariability in theestimates emphasized by Lewis {1986, Table4.2} arisesfrom the variation in theconditioning variables and choicesof combinations of functional forms and identifying assumptions.Different conditioning sets and different estimatorsdeŽ ne different structuralparameters. 52 This is an intrinsicfeature of structuralmodels, not of econometricmethods. Lewiswas frustrated bythefailure ofa purelyempirical approach

50.However ,therange of estimatesproduced in the Euler equation literature itselfvaries widely .See,e.g., Browning and Lusardi {1996}. 51.Summers {1991} presents a vigorouscritique of the Euler equation researchprogram and questions its contributions to economic knowledge. 52.This problem is discussed in Heckman {1990} and Heckman, LaLonde, andSmith {1999}. 78 QUARTERLY JOURNALOF ECONOMICS tosolve a causal problem,and thedifficult taskof working backward fromthe assumptions embedded in any particular empiricalstudy toexplain why its resultsare different fromany other. Lewis’evidence on the sensitivity of estimates of causal parametersto alternative identifying assumptionsdoes not dem- onstratethe impossibility ofestimatinga causal effectof unions onwages.Assuming that thefunctional form of population wage equationsis known,the only safe conclusionsthat canbe drawn fromhis study are(a) selectionbias is an empiricallyimportant problemin estimatingthe causal effectof unionson wages,and (b) different identifying assumptionsproduce different estimatesof thecausal parameters. 53 Only if selectionbias is not a problem, will different methodsfor solving the selection problem estimate thesame causal effect.The only way to‘ ‘solve’’ theproblem of parametervariability reportedin theempirical literature in labor economicsis todevelop different economicmodels to evaluate the plausibility ofthe different assumptionsused to generate the estimates.No purely empirical method is available. Agreeingto reportonly least squares estimates establishes a conventionthat iseasyto follow but that evadesthe stated causal question. Lewis’reaction to the variability ofstructural estimates underdifferent identifying assumptionsis typical ofthe reaction tostructural models by manyeconomists. In application, struc- tural econometriciansoften impose onto the data manyassump- tionsnot intrinsic to the economics of the problem for the sake of computationalconvenience. In manyapplications ofthemethod, and ofV ARmethods,there is an appeal toformal statistical methodsthat have‘ ‘black-box’’ features,and thenumbers pro- duced using themare often not perceived to be transparent or easilyreplicable. The quest for transparency underlies all ofthe recentresearch programs in econometrics,although there is no agreementover what constitutestransparency . Methodsdeveloped in nonparametriceconometrics and sensi- tivity analysis in statistics in principle reducesome of this arbitrariness.Nonparame tricidentiŽ cation analyses reveal whethercausal distinctions aremade solely on the basis of distributional assumptionsor on the basis offunctional form

53.These are my conclusions and not his. Note that the assumption of correct speciŽcation of populationwage functions was not questioned by Lewis. In labor economicsit ispresumed that the ‘ ‘Mincerequation’ ’ isthecorrect functionalform foran earnings equation. See Mincer {1974}. CAUSALP ARAMETERSAND POLICY ANALYSIS 79 assumptions.They open up theblack boxof parametriceconomet- ricsto establish thesources of identiŽ cation of economic models. 54 Thosepromoting calibration asamethodfor determining the parametersof structuralmodels emphasize the value of securing transparent estimatesfor structural equations and furtherempha- sizethe conditional— model-depende nt—nature of empirical knowledge.Like Lewis, they reject the black-box featuresof structuralestimation. Unlike Lewis, they seek to resolve empiri- cal ambiguitiesby developingthe economic theory .Unlikethe VAReconometricians,the calibrators focus on afew maincorrela- tionsand meansand build explicit modelsto account for them. Unlikethe structural econometricians, and theV AReconometri- cians,formal statistical modelsare not used by this groupand the goodness-of-Žt ofmodels is deemphasized as amodelevaluation criterion. Athird responseto the structural equation program is that of themore empirically oriented natural experimentmovement, whichis looselyallied with,and takesits inspiration fromthe socialexperiment movement. Like the calibrators, practitioners of this approach seektransparent estimatesobtained fromsimple econometricmethods because transparency and simplicity pro- motereplicability and understanding. Likethe V AReconometri- cians,this groupattempts to ground its activitiesin thedata, althoughit doesnot use time series methods, and tends to deemphasizeformal discussions of econometric methods. A hall- markof this approach is thequest for plausible sourcesof external variationto solve identiŽ cation problems, with theideal beinga randomizationthat doesnot alter the causal relationshipbeing studied. Likethe nonparametric econometricians, advocates of natural experimentsdistinguish what is in thedata fromwhat has tobe added toit toobtain estimatesof causal parameters.The emphasisis onrecovering causal parametersand noton evaluat- ing theeffects of new policies or the welfare consequences of policiesalready in place.I discuss eachof these research pro- gramsin turn.

54.For example, Flinn and Heckman {1982} establish the nonparametric nonidentiŽability of thestationary job search model of unemployment relative to dataon accepted wages and unemployment durations and demonstrate the extremesensitivity of structural estimatesderived from this model to alternative distributionalassumptions. 80 QUARTERLY JOURNALOF ECONOMICS

III.1. Understandingthe Limitations of theData: Bounding and SensitivityAnalysis In responseto the variation of the estimates produced from alternativespeciŽ cations of parametric econometric models ap- plied tothesame data, econometricianshave increasingly empha- sized separating aspects ofan estimationthat requireout-of- sampleextrapolation or impositionof difficult-to-justify functional forms,from aspects ofan estimationthat arebased onsample informationand arenonparametrically identiŽ ed. Exploring thelimits of identiŽ cation, this lineof work also considersthe restrictions placed onranges of parameter values whenmodels are not formally identiŽ ed. The paradigm forthis lineof work goes back to the research of Marschak and Andrews {1944} whopresent ranges of estimatesfor the structural parame- tersof Žrmproduction functions that areconsistent with biased leastsquares estimates. 55 Marschakand Andrews workwithin thecontext of anunderidentiŽed parametricmodel. More recent workemphasizes more nonparametric approaches to perform this type ofbounding and sensitivityanalysis. Recentprototypes for this separationof conjectural from factual informationare studies by Smithand Welch{1986}, Glynn,Laird, and Rubin {1986}, Holland {1986}, and Rosenbaum {1995} whoanalyze thestandard selectionproblem that is at the heartof Lewis’ problem of recovering the union-nonunion wage differential. Smithand Welchconsider identiŽ cation of means. Glynn,Laird, and Rubin, Holland,and Rosenbaumconsider identiŽcation of entiredistributions. Toillustratethese ideas in thesimplest possible setting, let f (W D 5 1) bethe density ofoutcomes (e.g., wages) forpersons who work (D 5 1correspondsto work). Suppose that weknow Pr (D 5 1 Z), where Z is avectorof determinantsof work.Hence weknowthat Pr( D 5 0 Z ).Missing is f (W D 5 0), e.g.,wages of nonworkers. 56 In orderto estimate E(W Z),Smithand Welch {1986} usethe law ofiterated expectations to obtain

E(W Z) 5 E(W D 5 1,Z ) Pr (D 5 1 Z)

1 E(W D 5 0,Z ) Pr (D 5 0 Z).

55.Wright {1934} presents ranges of estimates for underidentiŽ ed path coefficientmodels. 56.In Lewis’ unionism problem, there are two setsof missing data: the wages thatnonunion workers wouldearn if they were union workers andthe wages that unionworkers wouldearn if theywere nonunion workers. Ianalyzethe selection problemfor one case only to simplify the exposition. CAUSALP ARAMETERSAND POLICY ANALYSIS 81

Toestimatethe left-hand side ofthis expression,it is necessaryto obtain informationon the missing component E(W D 5 0,Z). Smithand Welchpropose and implementbounds on E(W D 5 0,Z), e.g.,

U WL # E(W D 5 0,Z ) # W ,

U 57 where W is an upper bound and WL is alowerbound. Using this information,they construct the bounds

E(W D 5 1,Z ) Pr (D 5 1 Z ) 1 WL Pr (D 5 0 Z) # E(W Z)

# E(W D 5 1,Z) Pr (D 5 1 Z) 1 W U Pr (D 5 0 Z). By doing asensitivityanalysis, theyproduce a rangeof valuesfor E(W Z)that areexplicitly dependenton the range of values assumed for E(W D 5 0,Z).58 Glynn,Laird, and Rubin {1986} presenta sensitivityanalysis fordistributions using Bayesian methodsunder a varietyof different assumptions.Letting F denotea distribution,by thelaw ofiteratedexpectations,

F(W Z ) 5 F(W D 5 1 ,Z ) Pr (D 5 1 Z )

1 F(W D 5 0,Z) Pr (D 5 0 Z).

Sensitivity analysis using alternativevalues of F(W D 5 0,Z) is performedto determine a rangeof values of F(W Z). Holland {1986} and Rosenbaum(in aseriesof papers starting in theearly 1980s and summarizedin {1995}) considermore classical sensitiv- ity analyses that vary theranges of parametersof models. Theobjective of the Smith-Welch bounds and theBayesian and classical sensitivityanalyses is toclearly separate what is knownfrom what is conjecturedabout the data, and toexplore the sensitivityof reported estimates to the assumptions used to securethem. Work on nonparametric identiŽ cation by Heckman {1990}, Heckmanand Honore´{1990}, and Heckmanand Smith {1998} examinesnonparametric assumptions required to identify selectionmodels from inŽ nite samples. This work establishes what is in thedata and what has tobe imposed on it tomake different causal distinctions.

57.In their problem there are plausible ranges of wages which dropouts can earn. 58.Later work byManski {1995} and Robins {1989} develops this type of analysismore systematically .Balkeand Pearl {1997} present an elegantapproach tothe problem of incorporatingmultiple sources of priorinformation using linear programmingmethods. 82 QUARTERLY JOURNALOF ECONOMICS

Someof the analysis presentedin therecent literature is nonparametricalthough in practice,high-dimensional nonpara- metricestimation is notfeasible. However ,feasibleparametric versionsof these methods run the risk of imposing the false parametricstructure used in estimatingthe structural models that is sodeeply criticized by advocatesof this approach. These methodsalso depend critically onthe choice of conditioning variables Z.In principle,all possiblechoices of the conditioning variables should beexplored, especially in computingbounds for all possiblemodels, although in practicethis is neverdone. If it were,the range of estimates produced by thebounds would be substantial. 59 Afully nonparametricapproach in derivingestimates or bounds forcausal parametersis unlikelyto produce useful results,although semiparametric models to compute bounds and estimatecausal modelsare beginning to be used on awide scale. Explorations ofbounds with simplefunctional forms of the sort advocated and implementedby Marschak and Andrews {1944} are likelyto be more informative. 60 Unlesssome assumptions about functionalforms are maintained, as in theparametric or semipa- rametricliteratures, economic data areunlikely to be very informativeabout causal parameters,especially when there are manypossible conditioning variables.

III.2. TheCalibration Movement Thedevelopment of dynamic economictheory and comput- able generalequilibrium theory in themid-1960s poseda serious challengeto structural econometrics. Unlike the simple static

59.The term ‘ ‘nonparametric’’ isactuallysomething of a misnomer.InŽnite samples,all nonparametric methods are parametric. The choices of the parameter- izationare dictated by propertiespossessed by thesefunctions as samplesbecome inŽnite, and not necessarily by theireconomic interpretability in smallsamples of thekind that are available— the criterion adopted in the older parameteric structural literature.(See, e.g., Fuss, McFadden, and Mundlak {1978}.) The semiparametriceconometric literature relaxes some of the parametric assump- tionsof parametricmodels, while retaining the remaining parametric structure. Semiparametricestimation methods are more suited for samples of the size availableto economists and for that reason are more widely used. In many empiricalapplications, simple parametric methods applied to analyze the data are oftenmore convincing, and accurate, than nonparametric or semiparametric methodsbased on arbitrary bandwidthschosen by mechanicalmean square error criteria.The variety of nonparametric approximation schemes currently inuse producesa varietyof smallsample approximations all of which converge in large samplesto the same functions but which often exhibit very differentproperties in thesmall samples to which they are applied. 60.Bounds for the regression coefficients in errors in variables models are developedby Klepperand Leamer {1984}. CAUSALP ARAMETERSAND POLICY ANALYSIS 83 theoryof demand, or the simple Keynesian models that deŽned thestructural econometrics of the 1940s and 1950s, recursive dynamic theorydoes not typically producesimple functional formsfor estimating equations. V ectorautoregression models like (7) and (8) rarelycapture the dynamic economictheory in an explicit way {Hansenand Sargent1980, 1991}. Problemsof estimatingthe parameters of large-scale static generalequilib- riummodels are equally formidable. Many ofthese new dynamic modelsproduce a richdynamics with qualitative propertiesthat depend critically onvalues as- sumedby parameters.Thus, while dynamic theoryneeds good empiricalinput todetermine which qualitative propertiesare empiricallyrelevant, at thesame time there are computational problemsthat makeestimation of the required parameters a formidabletask. This computational challenge is sogreat that a newŽ eld ofcomputational economics, concerned solely with simulationof these models, has openedup (see,e.g., Amman, Kendrick,and Rust {1996} orJudd {1998}). In an attemptto anchor the theory in data, and touse the theoryto produce counterfactual policy simulations, calibration has comeinto use on a wide scale.In static generalequilibrium models,the practice is topick simplefunctional forms (typically Cobb-Douglas) and Žtonecross section exactly using sharedata (seeDawkins, Srinivasan, and Whalley{2000}). In thereal businesscycle models, parameters are picked fromtime-series averagesto match the parameters of simplemodels that produce growthsteady states(see, e.g., Cooley and Prescott{1995}). In otherbranches of this literature,calibrators pick parametersfrom microstudies. This practice has beencriticized becausethe source micromodels are often based onare assumptions about the economicenvironments that areincompatible with theassump- tionsof thecalibrated macromodel {Hansen and Heckman1996}. Thisresearch program emphasizes interpretability of the estimatesin termsof economicmodels and subjectsthe calibrated modelsto rigorous internal consistency checks. Certain features ofdata (elementsof the source space S)becomethe focus of attentionwhile others are ignored. Overall Ž tofthe model is deemphasized,and formaltesting programs are explicitly re- jected.There is noattemptto account for the time-series correla- tionsin thefashion ofthe V AReconometricians.Current ap- proachesto producing theestimates for this class ofcausal models arecasual, although in its defense,the econometrics is transpar- 84 QUARTERLY JOURNALOF ECONOMICS entand oftenavoids theblack-box featuresof standard structural econometricsand theproblems of determining the appropriate representationof economic time series that plague theV AR approach. Somehave argued that thecalibrators are transpar- entlywrong. When real business cycle models Ž tby calibration havebeen subject to rigorous empirical testing of thesort used in VAReconometrics,they have performed poorly in termsof good- ness-of-Žt criteria{Watson 1993; Fair1994}. Giventhe weak empirical foundations forthese models, it is notsurprising that thepolicy counterfactuals based onthemare controversial,and fewoutside the subŽ eld takethe estimates of thewelfare consequences of policies produced by this lineof researchvery seriously .At thesame time, the models are intellec- tually interestingframeworks and demonstratewhat is logically possible.Anchoring them in data, howeverloosely ,givesthem someplausibility .Routinelyperformed sensitivity analyses reveal whichparameters are crucial and whichare not important {Auerbach and Kotlikoff1987}. TheseŽ ndings serveto directthe attentionof empirical analysts toward estimatingeconomically importantparameters. At thetime of this writing,it is unclearwhether the calibra- tionmovement is atransitional stagethat will bereplaced by a moreformal structural econometric research program, or aperma- nentŽ xtureof the economics landscape. In thelife cycleof any class ofgeneral equilibrium models, it is likelythat calibration will bethe Ž rststage of empiricalimplementation and that formal structuralestimation will followfor the subclass ofcalibrated modelsthat attract themost professional attention.

III.3. TheNatural ExperimentMovement Athird responseto theperceived failure of structuralecono- metricsis thenatural experimentmovement. Somewhat controver- sially,Iincludein this groupadvocates of socialexperiments. This group,like the nonparametric econometricians, the statisticians whoadvocate sensitivity analysis, and thecalibrators, seeks transparencyin its useof empirical evidence. The movement is organizedaround the principle ofŽnding goodinstruments (in the senseof subsection II.2) as inputs intoa standard instrumental variable estimatorof causal parametersthat is simpleto estimate CAUSALP ARAMETERSAND POLICY ANALYSIS 85 and easyto replicate. Randomization is oftenheld tobe the ideal instrument. 61 Unlikethe calibrators, members of this movementare much lessconcerned about estimating structural parameters derived fromeconomic theory .Theemphasis is onidentifying causal parameters,intuitively deŽ ned. By nottying theempirical work toany speciŽc economicmodel, the evidence produced fromthis approach appears tobe relevant to a widerclass ofmodels althoughthe link to any speciŽc modelis weaker.Simultaneity and self-selectionare recognized as potentially importantprob- lems,but simplesolutions to them are sought using transparent, replicable,instrumental variables methods,or difference-in- differencesmethods. 62 Giventhe emphasis on intuitivelydeŽ ned causal parametersas opposedto structural parameters, this groupeschews formal presentations of economic theory to moti- vateempirical models, as is favoredby thecalibrators, or explicit statementsof identifying assumptionswhich characterize the analyses ofnonparametriceconometricians. 63 Applications ofthis approach oftenrun the risk of producing estimatesof causal parametersthat aredifficult tointerpret.Like theevidence produced in VARaccountingexercises, the evidence produced bythis schoolis difficult torelateto the body ofevidence aboutthe basic behavioralelasticities of economics.The lack of a theoreticalframework makes it difficult tocumulate Ž ndings acrossstudies, or to compare the Ž ndings ofone study with another.Many applications ofthis approach produceestimates verysimilar to biostatistical ‘‘treatmenteffects’ ’ withoutany clear economicinterpretation. 64 Theless explicit discussionabout economicmodels and sourcesof identiŽ cation that characterizes

61.Social experiments are sometimes alleged to be the ‘ ‘goldstandard’ ’ for causalinference. For one discussion of theproblems arising in implementing social experimentsand for the interpretation of social experiments as instruments,see Heckman,LaLonde, and Smith {1999}. 62.Differences-in-differences are a formof instrumental variables method. SeeBlundell, Duncan, and Meghir {1998} and Heckman, LaLonde, and Smith {1999}.Both papers question the validity of themethod for evaluating many social programs. 63.However ,thechoice of instrumentsis highlycontroversial. In an impor- tantpaper ,Boundand Jaeger {1999} question the validity of exclusionrestrictions usedto justify the instruments employed in the recent literature on estimating the return toschooling in the assumed presence of ability bias. 64.However ,somestructural approachesare subject to the same criticism. Thus,in the structural unionwage literature, the ‘ ‘treatment’’ ofunionismis often estimatedrather than the direct economiceffects of unionismon technology,labor supply,andfactor substitution. 86 QUARTERLY JOURNALOF ECONOMICS muchof the research conducted in this stylesometimes creates theimpression that thereported empirical evidence is more robustthan theempirical evidence produced fromresearch pro- gramsthat adopt amoreexplicitly qualiŽed approach in interpret- ing evidence. Thisgroup, like the V AReconometricians,stresses empirical credibility,intuitiveplausibility ,and replicability.Analysts work- ing within this paradigm haveproduced an importantbody of factual knowledge.Like the nonparametric econometricians, this groupemphasizes the limits of empiricalknowledge. Counterfac- tual questionsabout the effects of new policies, never previously implemented,are viewed as empiricallyimpossible to answer . Explanation is emphasized overprediction. When prediction is performed,it is doneusing interpolationor extrapolation of existingestimates rather than theformal methods advocated in theeconometric policy evaluation literature.

IV. THE ADEQUACYOF THE COWLES RESEARCH PROGRAM AS A GUIDE TO LEARNINGFROM DATA At thetime the Cowles analysts weredeveloping their body of theory,classical statistical inferenceas embodiedin thework of Neymanand Pearson{1933} had avirtual intellectualmonopoly in statistics. Thatapproach emphasized theformal testing of modelsarrived at througha priorimeans. Haavelmo {1944} synthesizedthe Cowles approach with theNeyman-Pearson approach in his Nobel-Prize-winningfoundational essay. Fewempiricists now embrace the Cowles research program advanced by Haavelmothat remainsthe credo of moststructural econometriciansand is implicitly advocated inmosteconometrics textbooks.The Haavelmo program is avisionof induction on causal parametersproduced fromwell-deŽ ned structural eco- nomicmodels derived from explicitly articulated axioms.This approach toempirical analysis and modeltesting seems foreign to manyempirically oriented economists who favor more interaction betweenmodels and data than is envisionedin theHaavelmo paradigm. Thisdifference in approach toempirical analysis is a majordividing linebetween structural econometricians and most otherempirical economists. Classical statistics separatesthe act ofconstructing a model fromthe act ofverifyingit. Two fundamental assumptionsunder- liethis approach. TheŽ rstassumption is that asetof consequen- CAUSALP ARAMETERSAND POLICY ANALYSIS 87 tial facts (thesource space S)is Žxed inadvance ofconducting an empiricalanalysis orin advance ofwriting down formaleconomic modelsto describe the available data. Thesecond assumption is that theset of models that livein modelspace M is Žxed in advance oflookingat thedata. Thetask ofconstructingtheoreti- cal modelsis assumedto be divorced from the task of using data to test them. Separationbetween model formation and data analysis does notdescribe much empirical research activity in economicsor in any otherscientiŽ c Želd.Y etit is acornerstoneof classical statistics and thePopper {1959} falsiŽcationist program which is theofficial paradigm ofeconometrics and classical statistics. The Haavelmoprogram offers a veryrigid visionof empiricalanalysis. It ignoresthe interactive nature of theempirical learning process that movesbetween theory and data, aprocessthat is centralto theact ofcreating empirical knowledge. It is notinformative aboutwhat steps totake in responseto the rejection of amodel, norabout the implications of any responsefor the interpretation ofteststatistics usedin subsequenttests. 65 Theset of available modelsused to analyze agivenbody of data is neverŽ xed in advance oflooking at it.Inspection of the data usually suggestsnew models, and thenew models may in turnsuggest collecting new data— or morecareful examination of old data including addition of‘ ‘extra’’ variables tothe source space S—in thelight ofnew theoretical insights. The Haavelmo pro- gramdoes not describe this empiricallearning process. Nor does a Bayesian versionof it. Neither captures the act ofdiscoveryof new modelsnot previously contemplated that aresuggested by analy- sis ofthedata ortheinsight that amodelmay produce about new sourcesof data totest it. 66 Whilethere are serious problems in using thedata that suggesta theoryto test that theory,evenmore

65.Friedman {1953, p. 12, footnote 11} was an early critic ofthe Haavelmo program.He claimed that the choice of a classof models M oftenforced the conclusionsof an empirical study .Morgan{1990} presents an illuminating discussionof the failure of theCowles group toproduce a modelfor learning from the data. 66.The program of modeltesting and ‘ ‘encompassing’’ advocatedby Hendry andRichard {1982}appeals to the Haavelmo program and suffers from the same criticism.It proposes a priorispeciŽ cation of a largeclass of models and use of classicalsigniŽ cance tests to test down from a generalmodel. This in uential researchprogram does not account for nonnested hypotheses and does not account forlearning about new models not previously contemplated (and placed in thea prioriencompassing speciŽ cation). T estsconducted within the Hendry and Rich- ard programare sensitive to the order in whichthey are performed. 88 QUARTERLY JOURNALOF ECONOMICS importantproblems arise from refusing to learnfrom the data in revisingeconomic models. Sample reusevitiates standard testingprocedures, and re- ported t valuesdo notconvey the information traditionally assigned tothem. Bayesian methodsare more  exiblein this regard,but likethe classical methodsthey do not allow for surprise(i.e., new discoveries previously thought to be impossible ornotthought about prior to seeingthe data). 67 Only testson fresh data with different distributions offorcing variables provide convincingveriŽ cation of a model. 68 In manyareas of economics there are few precisely formu- lated modelsthat areknown in advance oflooking at thedata. Usuallya lotof vaguea priorinotions seem equally plausible as theoreticalpoints ofdeparture. In suchsettings, identiŽ cation analyses arenecessarily informal because the theory is nottightly speciŽed and theV ARand natural experimentresearch programs maybe goodstarting points forinitial data explorations. Theeconometric paradigm oftheidentiŽ cation problem and thenotion of aprecisetheory constructed in advance oflookingat theevidence do not apply toareas of social science where knowledgeis notsettled.It maydescribe a maturescience built on numerousprevious empirical studies conductedunder other researchparadigms whereempirical regularities about a phenom- enonhave accumulated. In this regardthe vision of empirical researchin economicsconducted within theparadigm ofthe Haavelmoprogram is averylimited one. Econometricians operat- ing within theHaavelmo paradigm tooeasily forget that apriori theoriesspeciŽ ed inadvance oflookingat thedata areoften just condensationsof accumulated empirical knowledge acquired us- ing crudeempirical methods. Leamer’santhropologyof econometricpractice {1978} and the recentstudy ofthe use of ‘ ‘tacit knowledge’’ in econometric practiceby Magnus and Morgan{1999} demonstratethat in practicethe Haavelmo program of specifying modelsin advance of thedata is rarelyused, although test statistics arereported as if it

67.However ,Bayesianmethods applied to events with positive prior probabili- tiesallow for surprise in the sense that posteriors can differ greatly from priors on speciŽc outcomes.In this sense Bayesians have a formalmethod for measuring surprise. 68.Replications of thesame model on thesame type ofdata(i.e., data with thesame distributions of variables) are unlikely to be informative. One can replicatethe same misspeciŽ ed results in each instance. Only ifanalysts use modelson data sets with different distributions of external forcing variables will differencesin models due to misspeciŽ cation be detected. CAUSALP ARAMETERSAND POLICY ANALYSIS 89 had beenused. While interest in causal modelsand causal questionsmotivates empirical studies, the rules for induction to generateempirical causal modelsthat wereprescribed by Haavelmoand theCowles group are typically ignored. 69 Nosuccessfulmechanical algorithm for discovering causal or structuralmodels has yetbeen produced, and it is unlikelythat onewill everbe found. 70 At thesame time, it isunlikelythat the questfor a mechanicalalgorithm for determining causality from data will everbe abandoned. 71 Thetension between the use of tacit knowledgeand formalalgorithmic methods is likelyto be a permanentfeature of empirical research in economics.It arises becausein mostempirical studies thereis always moreknowledge abouta problembeing studied than appears in thesampling distributions ofthe measured variables beinganalyzed orin well-speciŽed Bayesian priors.The best empirical work in econom- icsuses economic theory as aframeworkfor integrating all ofthe available evidence,tacit and algorithmic,to tell a convincingstory . Asdocumentedby Leamer{1978}, creativeempirical work is oftenpresented as if it has beenconducted within theHaavelmo paradigm, whilein fact, it is not.The format of the Haavelmo programis merelyused as areportingstyle to accord with the official rhetoricof econometrics. 72

V. CONCLUSIONS ANDA VISION OF THE FUTURE In thesmoke of battle overthe ‘ ‘correct’’ way todo empirical researchin economics,it is easyto lose sight oftheimportant and enduringcontributions that twentiethcentury econometrics has madeto knowledge. Its fundamental contributionsinclude estab- lishing formallythat causality is apropertyof amodel,that many modelsmay explain thesame data and that assumptionsmust be madeto identify causal orstructural models. It extendednine- teenthcentury notions of causality by recognizingthe possibility ofinterrelationships among causes. Econometrics clariŽ ed the conditionalnature of causal knowledgeand theimpossibility ofa

69.One contributing factor wasthe emergence of rival schools of statistical inferenceand the breakdown in the consensus about the appropriate way todo empiricalresearch. 70.This debate has a counterpartin thedebate in the artiŽ cial intelligence communityover whether machines can think (see Dreyfus andDreyfus {1986}). 71.See Glymour and Cooper {1999} for a varietyof algorithmsfor mechani- callyproducing causal parameters from data. 72.See McCloskey {1985} on the important role of rhetoric in economic discourse. 90 QUARTERLY JOURNALOF ECONOMICS purelyempirical approach toanalyzing causal questions.The informationrequired to evaluate economic policies and toforecast theeffects of newpolicies was carefullydelineated. Economistsrespond differently tothese universally agreed- uponprinciples. Some embrace a stylethat emphasizesthe conditionalnature of causal knowledgewhile others embrace a stylethat deemphasizesit. All agreethat identiŽcation of struc- tural orcausal relationshipsis adifficult task.The apparent empiricalfailure ofmanystructural research efforts, developed in the1970s and 1980s, toproduce credible estimates of economic parametersand causal relationshipsmotivates current research programs.A desirefor clearer ,moretransparent, sources of identiŽcation of causal effectsis acommongoal. Disagreementsarise over the interpretation of what consti- tutestransparency ,reportingstyles, the identifying assumptions that different groupsare willing tomake, and theemphasis placed onlinking empirical work to explicit economicmodels. Some of thesedifferences arise from differences in theintended audiences forthe empirical work. Evidence that is convincingin public policy forumsmay not be convincing to professional economists trained in theuse of modernstatistical and economicmethods. In many public policysettings, precise statements of identifying conditions and qualiŽed presentationsof evidenceare unwelcome. Thedifferent approachesto empirical research in economics havemuch to learn from each other .Structuralmethods will be morewidely acceptedif sensitivityanalyses areconducted, the consequencesof functional form and distributional assumptions areinvestigated, and nonparametricor semiparametric methods areused. With the continuing decline in computercosts, such sensitivitystudies will becomefeasible. The natural experiment movementwill gain awiderfollowing if it becomesmore inte- grated with economics.It will producea cumulativebody of knowledgeif economictheory is usedto guide thechoice of estimatingequations and toreport estimates. 73 Similarly,social

73.The contrasting analyses of Card {1999}and Heckman and V ytlacil{1998} demonstratehow simple economic models of the returns to schooling clarify the beneŽts and limitations of instrumentalvariables methods. In particular ,Heck- manand V ytlacilshow that Card’ s implicitassumption that tuition costs together withforgone earnings are not a costof schooling is critical to the successful applicationof the standard form of the method of instrumental variables to estimatingthe rate of return toschooling. Bound and Jaeger {1999} raise serious questionsabout the exclusion restrictions endorsed by Card. Blundell,Duncan, andMeghir {1998} demonstrate the value of economic models in interpreting policyparameters estimated by difference-in-differences methods. Rosenzweig and CAUSALP ARAMETERSAND POLICY ANALYSIS 91 experimentsthat useexperimental variation to identify economic parametersrather than program-speciŽc treatmenteffects are morelikely to produce cumulative knowledge. 74 Thecalibration movementwill producemore credible empiri- cal estimatesif it draws onthe estimates produced froma more data-sensitive structuraleconometrics movement and amore economics-sensitivenatural experimentmovement, if it uses microestimatesobtained fromempirical models consistent with themacro-general equilibrium frameworks and if it subjects calibrated modelsto the rigorous time-series tests advocated by VAReconometricians.The bounding and sensitivityanalysis movementis likelyto be more in uential if it relieson explicit economicmodels and useseconomically interpretable models to conductsemiparametric bounding and sensitivityanalyses. 75 It is far fromobvious that thesedivergent empirical practices will everconverge to a commonpractice for extracting causal parametersand conductingpolicy analysis. TheCowles Commis- sionpresented a bold programof causal analysis and policy evaluationthat has proveddifficult tooperationalize in practice. Laterdevelopments in dynamic generalequilibrium theory and gametheory have not made structural estimation any easier. Responding tothis difficulty is amajorsource of anxiety and tensionin empiricaleconomics. Those most strongly motivated by its theoreticalvision are less easily discouraged by theempirical failings oftheCowles program. Those most strongly motivated to describethe economic world, rather than toexplain it,or predict theeffects of new policies, are acutely aware of the empirical limitationsof highly structuredmodels. Someof thedisagreement that arisesin interpretinga given body ofdata is intrinsicto the Ž eld ofeconomics because of the conditionalnature of causal knowledge.The information in any body ofdata is usually tooweak to eliminate competing causal explanations ofthe same phenomenon. There is nomechanical algorithmfor producing asetof ‘ ‘assumptionfree’ ’facts orcausal estimatesbased onthose facts. In reachingthis understanding,

Wolpin{1980} demonstrate the power of natural-experimenttwins data to test an otherwiseunidentiŽ ed economic model of fertility . 74.The original goal of thesocial experimentation movement was to recover structural parametersto perform econometric policy evaluation. See Orcutt and Orcutt {1968}and Cain and Watts {1973}. See the discussion in Heckman, LaLonde,and Smith {1999}. 75.The analysis of Marschak and Andrews {1944} is a goodparametric paradigm.A morerecent analysis is thatof Viverberg{1993}. 92 QUARTERLY JOURNALOF ECONOMICS economistspart companywith statisticians who,as agroup,still fail tounderstand this importantlesson of twentieth century econometrics,and advocatepurely empirical approaches for deter- miningcausal relationships. 76 Theonly certain routes for eliminat- ing someof the disagreement among economists who maintain different identifying assumptionsare through collecting better data and stating differencesin identifying assumptionsmore clearly.77 Economictheory as aframeworkfor interpretation and synthesisis an inseparable part ofgoodempirical research in econ- omics.These are major lessons of twentieth century econometrics.

UNIVERSITYOF CHICAGO ANDTHE

REFERENCES Amemiya,Takeshi, AdvancedEconometrics (Cambridge,MA: Harvard University Press,1985). Amman,Hans, David Kendrick, andJohn Rust, Handbookof Computational Economics:V olume1 (Amsterdam:North-Holland, 1996). Auerbach,Alan, and Laurence Kotlikoff, DynamicFiscal Policy (Cambridge: CambridgeUniversity Press, 1987). Balke,Alexander ,andJudea Pearl, ‘ ‘Boundson Treatment Effects from Studies withImperfect Compliance,’ ’ Journalof the American Statistical Association, XCII(1997), 1171– 1176. Barros,Ricardo Paes, ‘ ‘MinimalJust-Identifying Assumptions in Nonparametric SelectionModels,’ ’ unpublishedmanuscript, Rio de Janeiro, IPEA, 1988. Basmann,Robert, ‘ ‘Exact FiniteSample Distributions for Some Econometric Estimatorsand Test Statistics: A Survey andAppraisal,’ ’ inMichaelIntriliga- tor andDavid Kendrick, eds., Frontiers ofQuantitative Economics (New York: North-Holland,1974), Chapter 4. Billingsley,Patrick, Probabilityand Measure, secondedition (New Y ork: John Wiley,1986). Bjerkholt,Olav ,‘‘RagnarFrisch and the Foundation of the Econometric Society and Econometrica, ’’pp.26– 57 in Steinar Strom, ed., Econometrics and EconomicTheory in the 20thCentury (Cambridge:Cambridge University Press,1998). Blalock,Hubert, Causal Inferences inNonexperimental Research (ChapelHill: Universityof North Carolina Press, 1964). Blundell,Richard, Alan Duncan, and Costas Meghir, ‘ ‘EstimatingLabor Supply ResponseResponses Using Tax Reforms,’ ’ Econometrica, LXVI(1998), 827– 862.

76.There is considerable irony in theobservation that some econometricians intherecent ‘ ‘treatmenteffect’ ’ literaturehave turned for guidance to statistics to obtainframeworks for interpreting causal laws while statisticians and experts in artiŽcial intelligence such as many writing inGlymour and Cooper {1999} have turnedto Cowles econometrics for clear deŽ nitions of causalitywithin a model. 77.Throughout this essay I havetaken it asself-evident that the introduction ofnewdata sources has greatly enriched economic knowledge. T ociteonly a few majorcontributors, the research of Kuznets on national income {1937} and economicgrowth {1973},Summers and Heston {1991} on international compari- sons,and Griliches {1995} on research and development have had a major inuence on ourprofession. CAUSALP ARAMETERSAND POLICY ANALYSIS 93

Bound,John, Charles Brown, andNancy Mathiowetz,‘ ‘MeasurementError in Survey Data,’’ inJames Heckman and Edward Leamer,eds., Handbook of Econometrics: Volume5 (Amsterdam:North-Holland, 2000). Bound,John, and David Jaeger ,‘‘DoCompulsory Schooling Attendance Laws AloneExplain the Association between Quarter ofBirth andEarnings,’ ’ S. Polachek,ed., Research inLabor Economics (Greenwich,CT: Jai Press, 1999). Browning,Martin, and Annamaria Lusardi, ‘ ‘HouseholdSaving: Micro Theories andMicro Facts,’ ’ Journalof EconomicLiterature, XXXIV(1996), 1797– 1855. Brunner,Karl, Problemsand Issues inCurrent Econometric Practice (Columbus, OH:College of AdministrativeScience Monograph AA-6, 1972). Cain,Glenn, and Harold Watts, ‘ ‘Summaryand Overview ,’’inGlenn Cain and HaroldWatts, eds., IncomeMaintenance andLabor Supply (Chicago: Markham,1973). Card, David,‘ ‘TheCausal Effect ofEducation on Earnings,’’ Orley Ashenfelterand DavidCard, eds., Handbookof Labor Economics: V olume3 (Amsterdam: North-Holland,1999). Chipman,John, ‘ ‘TheContributions of RagnarFrisch to Economics and Economet- rics,’’ inSteinar Strom, ed., Econometrics andEconomic Theory in the 20th Century (Cambridge:Cambridge University Press, 1998). Christ,Carl, ‘ ‘Historyof the Cowles Commission 1932– 1952,’ ’ EconomicTheory andMeasurement (Chicago:Cowles Commission For Research in Economics, 1952),reprinted in Econometrics, Macroeconomics andEconomic Policy: Selected Papers ofCarl Christ (Oxford: Elgar,1995). Christiano,Larry ,MartinEichenbaum, and George Evans, ‘ ‘MonetaryPolicy Shocks:What Have We Learned and to What End?’ ’ inJohn B. Taylorand MichaelWoodford, eds., Handbookof Macroeconomics (Amsterdam:North- Holland,1999). Cooley,Thomas,and Edward Prescott,Economic Growth andBusiness Cycles,’ ’ in ThomasCooley ,ed., Frontiers ofBusiness Cycle Research (Princeton:Prince- tonUniversity Press, 1995). Cox,David Roxbee, The Design ofExperiments (NewY ork: JohnWiley ,1959). Dawkins,Christina, T .N.Srinivasan,and John Whalley ,‘‘Calibration,’’ inJames Heckmanand Edward Leamer,eds., Handbookof Econometrics: Volume5 (Amsterdam:North-Holland, 1999). Domencich,Thomas, and Daniel McFadden, Urban Travel Demand (Amsterdam: North-Holland,1975). Dreyfus,Hubert, and Stuart Dreyfus, The Power ofHuman Intuition and Expertise inthe Era ofthe Computer (NewY ork: FreePress, 1986). Epstein,Roy , AHistoryof Econometrics (Amsterdam:North-Holland, 1987). Fair, Ray, Testing Macroeconometric Models (Cambridge:Harvard University Press,1994). Fisher,Franklin, The IdentiŽcation Problemin Economics (NewY ork: McGraw- Hill,1966). Fisher,Ronald A ylmar, Design ofExperiments (NewY ork: Hafner,1935). Flinn,Christopher ,andJames Heckman, ‘ ‘NewMethods for Analyzing Structural Modelsof Labor Force Dynamics,’ ’ Journalof Econometrics, XVIII(1982), 115–168. Friedman,Milton, Essays inPositive Economics (Chicago:University of Chicago Press,1953). Frisch,Ragnar ,‘‘PropagationProblems and Impulse Problems in Dynamic Econom- ics,’’ in Essays inHonor of Gustav Cassell (London:Allen and Unwin, 1933). Fuss,Melvyn, Daniel McFadden, and Y airMundlak, ‘ ‘ASurvey ofFunctional Formsin the Economic Analysis of Production,’ ’ inMelvyn Fuss and Daniel McFadden,eds., ProductionEconomics: A Dual Approachto Theory and Applications, Volume1 (Amsterdam:North-Holland, 1978). Galles,David, and Judea Pearl, ‘ ‘An AxiomaticCharacterization of Counterfactu- als,’’ Foundationsof Science, III(1998),151– 182. Geweke,John, ‘ ‘Inferenceand Causality in Economic Time Series Models,’ ’ inZvi Grilichesand Michael Intriligator, eds., Handbookof Econometrics: VolumeII (Amsterdam:North-Holland, 1984). Glymour,Clark, andGregory Cooper, Computation,Causation andDiscovery (Cambridge,MA: MITPress,1999). 94 QUARTERLY JOURNALOF ECONOMICS

Glynn,Robert, Nan Laird, and Donald Rubin, ‘ ‘SelectionModeling vs. Mixture Modelingwith Nonignorable Response,’ ’ inHoward Wainer, ed., Drawing Inferences fromSelf-Selected Samples (NewY ork: Springer-Verlag,1986). Goldberger,Arthur, Econometric Theory (NewY ork: JohnWiley ,1964). ,‘‘Structural EquationMethods in the Social Sciences,’ ’ Econometrica, XL (1972),979– 1001. Granger, Clive,‘ ‘InvestigatingCausal Relations by Econometric Models and Cross-SpectralMethods,’ ’ Econometrica, XXXVII(1969), 424– 438. Granger, Clive,and Paul Newbold, Forecasting EconomicTime Series (New York: AcademicPress, 1977). Griliches,Zvi, ‘ ‘R&D andProductivity: EconometricResults and Measurement Issues,’’ inPaul Stoneman, ed., Handbookof the Economicsof Innovation and Technological Change (Oxford: BasilBlackwell, 1995), pp. 52– 89. Haavelmo,Trygve, ‘‘TheProbability Approach inEconometrics,’ ’ Econometrica, XII,Supplement (1944), 1– 186. ,‘‘Methodsof Measuringthe Marginal Propensity to Consume,’ ’ Journalof the American Statistical Association, XLII(1947), 105– 122. Hamilton,James, Time Series Analysis (Princeton: Press, 1994). Hansen,Lars, and James Heckman, ‘ ‘TheEmpirical Foundations of Calibration,’ ’ Journalof EconomicPerspectives, X(1996),87– 104. Hansen,Lars, and Thomas Sargent, ‘ ‘Formulatingand Estimating Dynamic LinearRational Expectations Models,’ ’ Journalof Economic Dynamics and Control, II(1980),7– 46. Hansen,Lars, and Thomas Sargent, ‘ ‘TwoDifficulties in Interpreting V ector Autoregressions,’’ in RationalExpections Econometrics (Boulder,CO: West- viewPress, 1991). Heckman,James, ‘ ‘Varietiesof Selection Bias,’ ’ American EconomicReview, LXXX (1990),313– 318. Heckman,James, and Bo Honore ´,‘‘TheEmpirical Content of the Roy Model,’ ’ Econometrica, LVIII(1990), 1121– 1149. Heckman,James, Robert LaLonde, and Jeffrey Smith, ‘ ‘TheEconomics and Econometricsof Active LaborMarket Programs,’ ’ inOrley Ashenfelterand DavidCard, eds., Handbookof Labor Economics: V olume3 (Amsterdam: North-Holland,1999), Chapter 31. Heckman,James, and Richard Robb,‘ ‘AlternativeMethods for Solving the Problemof Selection Bias in Evaluating the Impact of Treatments on Outcomes,’’ inHoward Wainer, ed., Drawing Inference FromSelf-Selected Samples (NewY ork: Springer-Verlag,1986), pp. 63– 107. Heckman,James, and Jeffrey Smith, ‘ ‘Evaluatingthe Welfare State,’ ’ inSteinar Strom, ed., Econometric andEconomic Theory in the 20thCentury (Cam- bridge:Cambridge University Press, 1998). Heckman,James, and Edward Vytlacil,‘ ‘InstrumentalV ariablesMethods for the CorrelatedRandom Coefficient Model: Estimating the A verageRate of Return toSchooling when the Return is Correlated with Schooling,’ ’ Journal of HumanResources, XXXIII(1998), 974– 987. Heckman,James, and Edward Vytlacil,‘ ‘EconometricEvaluation of Social Pro- grams,’’ inJamesHeckman and Edward Leamer,eds., Handbookof Economet- rics, VolumeV (Amsterdam:North-Holland, 2000). Hendry,David,and Jean-Francis Richard, ‘ ‘On theFormulation of Empirical Modelsin Dynamic Econometrics,’ ’ Annalsof Applied Econometrics, XX (1982),3– 34. Holland,Paul, ‘ ‘ACommenton Remarks by Rubin and Hartigan,’ ’ inHoward Wainer,ed., Drawing Inferences FromSelf-Selected Samples (New York: Springer-Verlag,1986), pp. 149– 152. Hsiao,Cheng, Analysisof Panel Data (Cambridge:Cambridge University Press, 1986). Imbens,Guido, and Joshua Angrist, ‘ ‘IdentiŽcation and Estimation of Local AverageTreatment Effects,’ ’ Econometrica, LXII(1994),467– 476. Jorgenson,Dale, ‘ ‘CapitalTheory and Investment Behavior ,’’ American Economic Review, LIII(1963), 247– 259. Judd,Kenneth L., Numerical Methods inEconomics (Cambridge,MA: MITPress, 1998). CAUSALP ARAMETERSAND POLICY ANALYSIS 95

Killingsworth,Mark, LaborSupply (Cambridge:Cambridge University Press, 1983). Klein,Lawrence, ‘ ‘SingleEquation vs. Equation System Methods of Estimationin Econometrics,’’ Econometrica, XXVIII(1960), 866– 871. Klein,Lawrence, and Arthur Goldberger, AnEconometric Modelof the United States, 1929–1952 (Amsterdam:North-Holland, 1955). Klepper,Stevenand Edward Leamer,‘‘ConsistentSets of Estimates for Regres- sionswith Errors inAllV ariables,’’ Econometrica, LII(1984),163– 183. Knight,Frank, Risk,Uncertainty andProŽ t (NewY ork: HoughtonMifflin Co, 1921). Koopmans,Tjalling, and William Hood, ‘ ‘TheEstimation of SimultaneousLinear EconomicRelationships,’ ’ inWilliam Hood and , eds., Studies inEconometric Method (NewY ork: JohnWiley ,1953). Koopmans,Tjalling, and Olav Reirsol, ‘ ‘TheIdentiŽ cation of Structural Character- istics,’’ Annalsof Mathematical Statistics, XVIII(1950), 165– 181. Kuznets,Simon, NationalIncome and Capital Formation (NewY ork: National Bureauof EconomicResearch, 1937). ,‘‘ModernEconomic Growth: Findingsand Re ections,’ ’ American Economic Review, LXIII(1973), 247– 258. Lancaster,Kelvin, Consumer Demand:A New Approach (NewY ork: Columbia UniversityPress, 1971). Leamer,Edward, SpeciŽcation Searches (NewY ork: JohnWiley ,1978). ,‘‘VectorAutoregression for Causal Inference?’ ’ Carnegie-Rochester Confer- ence Series onPublicPolicy, XXII(1985),255– 303. Lewis,H. Gregg, Unionismand Relative Wages Effects: ASurvey (Chicago: Universityof ChicagoPress, 1986). Liu,Ta-Chung, ‘ ‘UnderidentiŽcation, Structural Estimationand Forecasting,’ ’ Econometrica, XXVIII(1960), 855– 865. Lucas,Robert, ‘ ‘EconometricPolicy Evaluation: A Critique,’’ The Phillips Curve andLabor Markets, I(CarnegieRochester Conference Series on PublicPolicy: 1976),pp. 19– 46. Lucas,Robert, and Thomas Sargent, Essays onRational Expectations and Econometric Practice (Minneapolis:University of Minnesota Press, 1981). Magnus,Jan, and Mary Morgan, Methodologyand Tacit Knowledge (New York: JohnWiley ,1999). Mann,Henry ,andAbraham Wald, ‘ ‘On theStatistical Treatment of Linear StochasticDifference Equations,’ ’ Econometrica, XI(1943),173– 220. Manski,Charles, The IdentiŽcation Problemin the SocialSciences (Cambridge: HarvardUniversity Press, 1995). Marini,Margaret, and Burton Singer, ‘ ‘Causalityin theSocial Sciences,’ ’ Sociologi- cal Methodology, XVIII(1988), 347– 410. Marschak,Jacob, ‘ ‘EconomicMeasurements for Policy and Prediction,’ ’ inWilliam Hoodand Tjalling Koopmans, eds., Studies inEconometric Method (New York: JohnWiley ,1953),pp. 1– 26. Marschak,Jacob, and William Andrews, ‘ ‘RandomSimultaneous Equations and theTheory of Production,’ ’ Econometrica XII(1944),143– 205. Marshall,Alfred, Principles ofEconomics, NinthV ariorumedition (London: Macmillan,1961). McCloskey,Donald, The Rhetoric ofEconomics (Madison:University of Wisconsin Press,1985). Mincer,Jacob,‘ ‘LaborForce Participation of MarriedWomen,’ ’ inH. Gregg Lewis, ed., Aspects ofLaborEconomics (Princeton:Princeton University Press, 1962). , Schooling,Experience andEarnings (NewY ork: ColumbiaUniversity Press, 1974). Morgan,Mary , The Historyof Econometric Ideas (Cambridge:Cambridge Univer- sityPress, 1990). Morgenstern,Oskar, On the Accuracy ofEconomic Observations (Princeton: PrincetonUniversity Press, 1950). Nerlove,Marc, DavidGrether ,andDomingo Carvalho, Analysisof EconomicT ime Series: ASynthesis (NewY ork: AcademicPress, 1979). Neyman,Jerzy ,‘‘StatisticalProblems in Agricultural Experiments,’ ’ Supplement to the Journalof the RoyalStatistical Society, II(1935),107– 180. 96 QUARTERLY JOURNALOF ECONOMICS

Neyman,Jerzy ,andEgon Pearson, ‘ ‘On theT estingof Statistical Hypotheses in Relationto Probability a Priori,’’ Proceedings ofthe Cambridge Philosophical Society, XXIX(1933), 492– 510. Orcutt, Alice,and Guy Orcutt, ‘‘Experimentsfor Income Maintenance Policies,’ ’ American EconomicReview, LVIII(1968), 754– 772. Orcutt, Guy,‘‘TowardsPartial Redirection of Econometrics,’ ’ Review ofEconomics andStatistics, XXXIV(1952), 195– 200. Pearl,Judea, ‘ ‘Graphs,Causality and Structural EquationsModels,’ ’ Sociological Methods andResearch, XXVII(1998), 226– 249. Phillips,Allan W .,‘‘Estimationof Systems of Difference Equations with Moving AverageDisturbances,’ ’ inAlanRay Bergstrom,ed., Stability andIn ation (NewY ork; Wiley,1966),pp. 181– 199. Popper,Karl, The Logic ofScientiŽ c Discovery (London:Hutchinson, 1959). Quandt,Richard, The Demand ForT ravel (Lexington,MA: D. C.Heath,1970). Reirsol,Olav ,‘‘Conuence Analysis by Meansof InstrumentalSets of Variables,’’ Arkivfor Mathematik Astronomioch Fysik XXXII(A)(1945), 1– 119. Robins,Jamie, ‘ ‘TheAnalysis of Randomized and Non-randomized AIDS Treat- mentTrials Using a NewApproach toCausal Inference in Longitudinal Studies,’’ inLee Sechrest, Howard Freeman, and Albert Mulley ,eds., Health Service Research Methodology:A Focus onAIDS (Washington,DC: U.S. PublicHealth Service, 1989), pp. 113– 150. Rosenbaum,Paul, Observational Studies (NewY ork: Springer-Verlag,1995), Chapter 4. Rosenzweig,Mark, andKenneth Wolpin, ‘ ‘Testingthe Quantity-Quality Fertility Model:The Use of Twins as a NaturalExperiment,’ ’ Econometrica, XLVIII (1980),227– 240. Rubin,Donald, ‘ ‘BayesianInference for Causal Effects: The Role of Randomiza- tion,’’ Annalsof Statistics, VI(1978),34– 58. Shapiro,Matthew ,andMark Watson,‘ ‘Sourcesof Business Cycle Fluctuations,’ ’ in StanleyFischer ,ed., NBER Macroeconomics Annual (Cambridge,MA: MIT Press,1988). Simon,Herbert, A., ‘ ‘On theDeŽ nition of the Causal Relation,’ ’ Journal of Philosophy, XLIX(1952), 517– 527. Sims,Christopher, ‘ ‘Money,Incomeand Causality ,’’ American EconomicReview, LXII(1972), 540– 552. ,‘‘Exogeneityand Causal Ordering inMacroeconomicModels,’ ’ inC.A.Sims, ed., New Methods InBusiness Cycle Research: Proceedings FromA Conference (Minneapolis:Federal Reserve Bank ofMinneapolis, 1977). ,‘‘Macroeconomicsand Reality ,’’ Econometrica, XLVIII(1980), 1– 48. ,‘‘Are ForecastingModels Usable for Policy Analysis?’ ’ Quarterly Review ofthe Federal Reserve ofMinneapolis, Winter(1986), 2– 16. Slutsky,Eugene,‘ ‘TheSummation of Random Causes as the Source of Cyclic Processes,’’ Econometrica, V(1937),105– 146. Smith,James, and Finis Welch, ‘ ‘Closingthe Gap: Forty Yearsof Economic Progressfor Blacks,’ ’ SantaMonica, CA: RandCorporation, 1986. Strom,Steinar, Econometrics andEconomic Theory in the 20thCentury (Cam- bridge:Cambridge University Press, 1998). Summers,Lawrence, ‘ ‘TheScientiŽ c Illusionin Empirical Macroeconomics,’ ’ ScandanavianJournal of Economics, XCII(1991), 129– 148. Summers,Robert, and Alan Heston, ‘ ‘ThePenn World Table (Mark V): An ExpandedSet of InternationalComparisons,’ ’ Quarterly Journalof Econom- ics, CVI(1991), 327– 368. Tinbergen,Jan, ‘ ‘Bestimmungund Deutung V onAngesbotscurven,’ ’ Zeitshrift Fu¨r Nationalo¨konomies, I(1930),669– 679, reprinted and translated in David Hendryand Mary Morgan,eds., The Foundationsof Econometric Analysis (Cambridge:Cambridge University Press, 1995). Viverberg,William, ‘ ‘Measuringthe UnidentiŽ ed Parameter of the Roy Model of Selectivity,’’Journalof Econometrics, LVII(1993),69– 90. Wallis,Kenneth, ‘ ‘EconometricImplications of the Rational Expectations Hypothe- sis,’’ Econometrica, XLVIII(1980), 49– 74. Watson,Mark, ‘‘Measuresof Fit for Calibrated Models,’ ’ Journalof Political Economy, CVI(1993), 1011– 1041. CAUSALP ARAMETERSAND POLICY ANALYSIS 97

White, Hal, Estimation, Inference andSpeciŽ cation Analysisfor Econometricians (NewY ork: AcademicPress, 1987). Wright,Philip Green, The Tariff onAnimal and V egetable Oils (New York: Macmillan,1928). Wright,Sewall, ‘ ‘TheMethod of Path Coefficients,’ ’ Annalsof Mathematical Statistics, V(1934),161– 215. Zellner,Arnold, and Franz Palm, ‘ ‘TimeSeries Analysis and Simultaneous EquationEconometric Models,’ ’ Journalof Econometrics, II(1974),17– 54.