Evaluating the Atheoretical Turn in Economics 1

Form and Function: Evaluating the Atheoretical Turn in Economics Meredith M. Paker Nuffield College University of Oxford meredith.paker@nuffield.ox.ac.uk Prepared for Philosophy and Methodology of Economics Economic and Social History June 2017 1 Introduction Since the late 1990s, research in economics has increasingly employed “atheoretical” experimental and quasi-experimental methods.1 As laboratory and field experiments became more accepted in the discipline, paralleling the rise of behavioral economics, statistical quasi-experimental methods were developed and standardized. These tools, including difference-in-difference, regression discontinuity, and instrumental variable approaches, have made quasi-experimental analyses of natural or institutional variation more accessible, popular, and persuasive. Experimentally-based approaches are now common throughout the discipline and are especially prominent in empirical microeconomic research, where they have displaced structural and theory-driven work in fields such as labor, education, and public economics.2 Two main reasons are often cited for the growth of work in experimental economics. First, experimentally-based methods are seen as “atheoretic” when compared with tradi- 1. “Economists are prone to fads, and the latest is machine learning,” The Economist, November 2016, http://www.economist.com/news/finance-and-economics/21710800-big-data-have-led- latest-craze-economic-research-economists-are-prone, In addition to an increase in machine learning methods, the figure shows a remarkable upward trend in laboratory, randomized control trial, difference-in-difference, and regression discontinuity approaches. 2. John Rust, “Comments on ‘Structural vs. Atheoretic Approaches to Econometrics’ by Michael Keane,” Journal of Econometrics 156, no. 1 (2010): p. 21. 1 tional structural methods that make a priori parametric assumptions.3 Second, structural modeling is difficult, time consuming, and less likely to be appreciated by academic jour- nals looking for intuitive and easily-explained work. This skews professional incentives toward the more accessible and time-efficient experimental methods.4 Regardless of the motive, the move from broadly structural to broadly experimental research requires some methodological consideration. Experimental work abandons the traditional deductive model of reasoning in favor of simpler “atheoretic” methods, osten- sibly avoiding the sticky issues of theory validation and realism of assumptions. However, critics have argued that these experimental methods may still involve strong assumptions5 and may rely on unsound inductive reasoning for external validation.6 Thus the shift in economic methods from structural to experimental reflects a deeper shift in the “form” of economic reasoning. This might also imply a shift in the “function” of economics as a discipline. Though there are still disagreements about the exact goals of the field, both realists and instrumentalists would agree that some level of policy- relevance is desirable.7 A shift toward experimental methods, however, could threaten the ability of economics to be policy-relevant. In this essay, I will begin by considering the methodological developments of the twen- tieth century, where issues of deduction and empirics became central debates. With this essential grounding, I will then compare and contrast the experimental and structural approaches to economics on a general level. I will argue that experimentally-based methods require unsound inductive reasoning to achieve external validity. Then, I will question whether this makes experimental economics a weaker tool for policy than analyses rely- ing on deductive methods, which are developed to be more universal and which explicitly state their premises. Finally, to conclude, I will briefly consider whether the experimental turn in economics also reflects a shift in the intended purpose of economics. Is policy relevance becoming less important to the discipline? 3. Michael P. Keane, “Structural vs. Atheoretic Approaches to Econometrics,” Journal of Econometrics 156, no. 1 (2010): pp. 3-4. 4. Rust, “Comments on ‘Structural vs. Atheoretic Approaches to Econometrics’ by Michael Keane,” p. 21. 5. James Heckman, “Instrumental Variables: A Study of Implicit Behavioral Assumptions Used in Making Program Evaluations,” The Journal of Human Resources 32, no. 3 (1997): 441–462; Mark R. Rosenzweig and Kenneth I. Wolpin, “Natural ‘Natural Experiments’ in Economics,” Journal of Economic Literature 38, no. 4 (2000): 827–874. 6. Volker Gadenne, “External validity and the new inductivism in experimental economics,” Rational- ity, markets, and morals 4, no. 1957 (2013): 1–19. 7. Daniel M. Hausman, The Inexact and Separate Science of Economics (Cambridge: Cambridge University Press, 1992), pp. 285-288. 2 2 The tradition of deduction in economics Economics has been portrayed and conducted as a deductive science since the classical economists. Though Adam Smith is a notable exception, Ricardo, Malthus, and J.S. Mill all relied on deductive methods in their analyses. The neoclassical tradition and its further developments have continued in this vein, and the deductive foundations of economics have been the subject of much consideration and criticism in philosophy of economics. In this section, I will present Mill’s 1836 deductive methodology for political economy in detail and then will consider modern variants of this deductive method. I will also present the problem of induction as an insoluable challenge to purely empirical work. Counter-methodologies have also been influential, though not dominant, since the late nineteenth century. The German Historical School, early Institutional economics, and the Wisconsin School all emphasized the historical dependency of economic theory, which is an important threat to the validity of global deductions. While space does not permit a detailed discussion of these schools of thought, I will present the problem of induction as an insoluble challenge to purely empirical work in this section and will briefly consider the major empiricist and instrumentalist alternatives to deduction in the next section. Ultimately, this conversation will serve as important backdrop to our consideration of the methodologies of empirical microeconomics dominating the field today. J.S. Mill’s a priori method for political economy John Stuart Mill famously makes the case for a priori deductive reasoning in his 1836 essay “On the Definition of Political Economy and the Method of Investigation Proper to It.” Though Mill was an inductivist and empiricist, supporting observational and experimental foundations for knowledge in the natural sciences, he considered political economy a separate field posing special challenges of two types. First, Mill’s political economy offers only a limited ability to conduct experiments due to its consideration of entire systems of governments and broad policies, which are hard to vary for experiment. Second, in political economy and in other “moral sciences” there are many factors that can contribute to an effect, making it difficult to control for all influential conditions. These two characteristics prevent political economy from obtaining Bacon’s experimen- tum crucis, which proves a causal effect by negating all other possible explanations for some occurrence. Thus political economy needs to rely on a priori reasoning, deducing predictions from an assumed hypothesis. While Mill’s a priori method of political economy is deductive, it relies on an es- 3 tablished and unquestionable hypothesis as a beginning premise. To Mill, this assumed hypothesis should be a universal principle that is achieved inductively from observation about human nature. He believes it is possible to induce principles of human nature from observation, writing, “Although sufficiently ample grounds are not afforded in the field of politics, for a satisfactory induction by a comparison of the effects, the causes may, in all cases, be made the subject of specific experiment...The desires of man, and the nature of the conduct to which they prompt him, are within reach of our observation.”8 Thus, to Mill, we can use the laws of nature and our observations about the actions of man as universal principles that provide unshakable premises for valid deduction. These laws can sometimes be psychological in nature, determined introspectively, or can relate to the natural sciences, determined through experiment. In both cases, the laws should be proven empirically, through observation or experiment, prior to the deduction. From these universal laws, the political economist can deduce specific laws about the operation of the economy. Further predictions can then be deduced from these more specific laws. Mill offers a famous description of a theorist deriving laws inductively and then making deductions: “...those who are called theorists aim at embracing a wider field of experience, and, having argued upwards from particular facts to a general principle including a much wider range than that of the question under discussion, then argue downwards from that general principle to a variety of specific conclusions.”9 The final conclusions are always justified in this deductive-nomological model if the premises hold. To Mill, this condition is always met because at the top of this chain of logic is an infallible universal law induced from observation. Thus regardless of how many levels of deduction there are, the final predictions must be true because the premises hold. However, Mill understood

Load more