Econometric Methodology and the Scientific Status of Economics
Total Page:16
File Type:pdf, Size:1020Kb
ECONOMETRIC METHODOLOGY AND THE SCIENTIFIC STATUS OF ECONOMICS MICHAEL SHERLOCK Senior Sophister To many economists, econometrics is a method of exploring many of the heated debates in a clinical, scientific way. However in this essay, Michael Sherlock argues that, despite the myriad rules and rigidity in the models, econometrics can be seen as a deeply flawed attempt by economists to legitimise their subject in the eyes of broader scientific disciplines. He discusses the weaknesses inherent in the field and explains how they clash with the standard ideas of what constitutes a science. Introduction This essay seeks to engage in the debate on the scientific status of economics by considering whether econometric methodology constitutes a scientific process sufficiently similar to that of other sciences in order that the epithet of ‘science’ can be conferred on the discipline of economics. The essay will first attempt to clarify what is meant by the term ‘science’ and situate it within the context of economic history. It will then proceed to discuss the crux of the intellectual debate, considering in turn the various problems that have been identified within econometric methodology. The conclusion will reflect upon the issues raised and take a position as to whether economics can be classified as a science. Economics as a science: a false hypothesis? ‘Science is a public process. It uses systems of concepts called theories to help interpret and unify observation statements called data, in turn data are used to check or test the theories’ (Hendry, 1980: 388). Although various definitions of the term ‘science’ exist, what is interesting about this one, written by an economist, is that it highlights the importance of empirical testability within economic models. Traditionally, classical economics (economics without econometrics) largely consisted of deductive theories, wholly devoid of any real data, which relied on quite complicated mathematics for their existence (consider general competitive equilibrium, a fundamental axiom of microeconomics whose existence was eventually proved using fixed point theorems). Accordingly, classical economics could not be considered a true science in any meaningful sense of the word. This led to an identity crisis in the economics profession which has resulted in the birth of econometrics, a branch of economics which provides a series of methods necessary for the analysis of data. Pearson (1938) championed the ‘unity of science’ principle which conceived that the essence of any science consists of a scientific method. Ritchie (1923) concurred, arguing that the only constant in science was this scientific method and that while scientific theories are in a constant state of flux, the process used to generate these theories has remained static. This stimulated debate among economists as to whether econometrics provided economics with this much needed ‘scientific process’, thereby providing the discipline with the intellectual legitimacy which it sought. The essence of econometric methodology is the development of a framework which seeks an adequate ‘conjunction of economic theory and actual measurement, using the theory and technique of statistical inference as a bridge pier’ (Haavelmo, cited in Pesaran & Smith, 1992: 9). Ever since the Cowles commission, regression analysis has become the empirical workhorse of econometrics, apparently providing the methodology of the scientific process at last. ‘It must be possible for an empirical scientific system to be refuted by experience’ (Popper, 1959: 41). This statement encapsulates the principle of falsifiability - the fact that in order for a theory to be considered scientific it must be capable of being disproved. Much of the controversy surrounding econometric methodology is whether it is capable of testing theories. Ostensibly, it seemed to do so. Nash (2007: 56) highlighted the fact that, from the outset, econometric methodology appeared to graft a scientific method onto mainstream economics ‘as now, apparently, hypotheses can be tested empirically and also falsified, thereby satisfying the scientific method’. In the early days, many commentators were less sanguine and even displayed open scepticism about the ability of econometrics to achieve this objective. Spanos (1986: 660) best articulated this position when he said ‘No economic theory was ever abandoned because it was rejected by some econometric test nor was a clear-cut decision between competing theories made in lieu of such a test’. To disambiguate the position we need to examine in depth the econometric process itself. A ‘failure to accept’ the econometric methodology Koutsoyiannis (1973) has identified the following steps as the core of econometric methodology: formulation of maintained hypothesis, testing of maintained hypothesis, evaluation of estimates and evaluation of the model’s forecasting validity. A cursory glance suggests the pre-eminence of hypothesis testing within the overall framework of regression analysis. Koutsoyiannis extols the benefits of such; and he adds that it confers scientific status on classical economics by virtue of the very fact that it is capable of sustaining rigorous testing. Many authors are critical of such claims. Hypothesis testing essentially involves what Koop (2005: 80) has referred to as ‘knocking down the straw man’, i.e. rejecting the null hypothesis and thereby establishing statistical significance. However, such a process is riven with a variety of interrelated problems. Firstly, a finding of statistical significance does not necessarily denote scientific significance. Popper (1959: 23) defines a scientifically significant effect ‘as that which can be regularly reproduced by anyone who carries out the appropriate experiment in the way prescribed’. Much research has highlighted the remarkably high incidence of inability to replicate empirical studies in economics (Dewald et al., 1986). Hence, an econometric finding of statistical significance cannot be considered scientifically significant in any meaningful way. The problem is that the nature of data in a non-experimental discipline such as economics makes reproducibility impossible. This, in turn makes testability and falsifiability impractical, thereby rendering the whole process de facto unscientific. Kennedy (2003: 8) describes economic data as being ‘weak’ which refers to the fact that many of the forces governing economic behaviour are unquantifiable, being neither numerical nor measurable. O’Dea (2005: 40) even contends that they cannot be truly considered ‘economic’ and argues that the unwillingness of economists to consider such forces ‘flies in the face of its claim to be scientific’. Some of the desirable features of any science are those of objectivity and precision. Regarding the latter, the fact that ‘outcomes are only probable to a given level of confidence, places econometrics and hence economics into a realm which is too imprecise to be deemed science’ (Nash, 2007: 57). As it is very often human behaviour that is being modelled, exact or deterministic relationships are impossible. Researchers compensate for this implicit uncertainty through the use of inferential statistics based on probability distributions. Consequently, levels of significance are assigned to outcomes. When one carries out a hypothesis test it is always at a given level of significance. It should also be noted that this imprecision is captured in the linguistic register of the terminology employed - it is best practice never to say that one rejects a null hypothesis, instead one employs the term ‘fails to accept’. This highlights the fact that econometrics is ‘a language for communicating results as well as a set of methods of analysis’ (Krueger, 2001: 10). At an alternative level of significance, a previously statistically insignificant regression coefficient may become statistically significant. This arbitrary use of significance levels raises the interrelated question of objectivity. Scientific credibility demands objectivity. Keuzenkamp and Magnus (1995) took issue with such an arbitrary use of significance levels whilst Berkson (1938) noted that for asymptotic samples, any null hypothesis was likely to be rejected and suggested that the choice of level should be decided by such pragmatic considerations. Unfortunately, in practice, choice is usually determined by the subjective needs of the econometrician; Keuzenkamp and Magnus (1995: 16) note that ‘the choice of significance level seems arbitrary and depends more on convention and, occasionally, on the desire of an investigator to reject or accept a hypothesis rather than on a well-defined evaluation of conceivable losses that might result from incorrect decisions’. That being the case, the objectivity of the econometric process is severely compromised. This leads to a related problem extensively observed in econometrics, that of data-mining. Leontief (1971: 390) once presciently commented on the state of econometrics describing it as ‘an attempt to compensate for the glaring weakness of the data base available to us by the widest possible use of more and more sophisticated statistical techniques’. This emphasis on statistical analysis has lead to the problem of data-mining which has been frequently cited as a major source of evidence against econometrics’ claim to scientific status. Data-mining consists of ‘moulding or selecting models based only on an ability to pass desired statistical tests rather than underlying theory’ (Hansen, 1996: 1408). The