sign in sign up
<<
Home , Agricultural economics, Marc Nerlove, Sherwin Rosen

Field Experiments in and Resource Research (Brian Roe, Ohio State University, Organizer)

CAN FIELD EXPERIMENTS RETURN TO THE GLORY DAYS?

DAVID H. HERBERICH,STEVEN D. LEVITT, AND JOHN A. LIST

As we contemplate the state of academic re- the next twenty years, before declining again search in the agricultural economics commu- to less than 1% of all papers by the end of the nity, there are two facts that are difficult to sample. overlook. First, over roughly the first half of the In this study, we hypothesize on why agri- twentieth century, scholars working on agri- played such a prominent cultural issues were the toast of the academy, role in economics for a long time, and why setting forth an empirical research agenda that its importance has diminished more recently. served to influence social and natural scientists We then provide one solution as to how agri- profoundly. The seminal contributions made in cultural can reverse this troubling these early years continue to shape important trend and potentially return to a central place academic and policy debates. A second fact re- on the frontier of economic science: more ex- mains that this dominance has waned in the tensive use of field experiments. We view our past several decades, with major empirical ad- solution as stemming from the roots of farm- vances now more likely in sister fields, such as ing, harkening back to Ralph Waldo Emerson labor economics or . who wrote: “The glory of the farmer is that, in One piece of ocular evidence showing this the division of labors, it is his part to create.” decline is shown in figure 1, which reports the share of papers devoted to agricultural top- ics in four top economic journals over the period 1911–2000, quinquennially.1 In the ear- Empirical Contributions liest years more than 10% of papers in these of the Twentieth Century four top journals were devoted to , with that share steadily falling through World In this section, we briefly detail a select set of War II. The share stayed roughly constant for contributions that we view as seminal within the area of agriculture. From the outset, we David H. Herberich is a graduate student and research professional concede that we cannot do this field justice at University of Maryland and University of , Becker Cen- due to space constraints so we limit our focus ter. Steven D. Levitt is the William B. Ogden Distinguished to briefly summarizing contributions of agri- Professor, and John A. List is a professor, Department of Eco- nomics, , and NBER. cultural economists over the twentieth cen- Discussions with Bruce Gardner, Richard Just, and Marc tury. A useful starting point is the work of Nerlove considerably enhanced the content of this paper. As usual, the authors are responsible for any errors, however. Fisher and Neyman, who were the first to This article was presented in an invited paper session at the conceptualize randomization as a key ele- ASSA annual meeting in San Francisco, CA, January 2009. The ment of the experimental method in the 1920s articles in these sessions are not subjected to the journal’s standard refereeing process. and 1930s in answering important economic 1 The journals are the , Economet- questions regarding agricultural productiv- rica, Journal of Political ,andQuarterly Journal of Eco- ity (e.g., Splawa-Neyman 1990[1923]; Fisher nomics. For a more detailed description of our methodology for constructing figure 1, see the online appendix Herberich, Levitt, 1926). Since Levitt and List (2008) discuss and List (2009). We understand that many approaches could be these contributions, we focus our study on sub- used to measure success of a field, and indeed, even under our approach one might argue that we include the wrong journals sequent decades, which witnessed agricultural (Economic Journal, for example, played a significant role in the economists making fundamental advances in twentieth century and remains important today). We suspect that regression analysis, including problems in- most, if not all, reasonable strategies will yield the general spatial pattern observed in figure 1. volving errors-in-variables and identification Amer. J. Agr. Econ. 91 (Number 5, 2009): 1259–1265 Copyright 2009 Agricultural and Association DOI: 10.1111/j.1467-8276.2009.01294.x 1260 Number 5, 2009 Amer. J. Agr. Econ.

Note: The four Journals used are The Quarterly Journal of Economics, The American Economic Review, The Journal of and Econometrica (begins 1933).Article by article search was done to determine if articles were related to Agriculture. Rejoinders, replies, restatements, comments orcorrections are omitted. The data was aggregated to 5-year periods begining in 1911 and running through 2000.

Figure 1. Share of agricultural articles in four top economic journals problems concerning estimating elasticities of of multiple regression by deriving matrix alge- (Fox 1986). The avail- bra computations. Working (1925) and Schultz ability of agricultural data during these years (1925) recognized the attenuation bias that re- and the relevance of the estimation issues to sults from the measurement error as early as agricultural questions are two possible expla- 1925. nations for why agricultural economists were Agricultural economists remained among early in addressing these issues. the most prominent in the profession during the 1950s and 1960s. A pair of University of Chicago economists, and D. Other Early Empirical Contributions Gale Johnson, made lasting contributions to of the Twentieth Century the field during this time period. Schultz’s early work dealt with agriculture in developed coun- Fox (1986) provides an important account tries, including issues of farm management. of contributions that agricultural economists The work for which he was awarded the Nobel made to the areas of statistics and econo- Prize in 1979, however, dealt with agriculture metrics in the early half of the twentieth in developing countries. In particular, his re- century. He mentions eight main agricultural search challenged the conventional view that economists during this time period: Louis H. traditional agriculture was not amenable to Bean, Mordecai Ezekiel, Bradford B. Smith, economic analysis because it was driven by tra- Howard R. Tolley, Henry A. Wallace, Fred- dition and culture rather than rational choices erick V. Waugh, Holbrook Working, and E.J. (Schultz 1964).2 Johnson’s contributions to Working as well as one working agricultural economics spanned decades, be- with agricultural commodities, Henry Schultz. ginning with his landmark analysis of the A majority of these economists contributed to economics of share-cropping contracts (John- the field by playing a role in correctly address- son 1950). His influential 1973 book World ing identification issues resulting from sup- Agriculture in Disarray argued that govern- ply and demand analysis of agricultural data. ment product and trademark interventions on However, more general work within multi- ple regression estimation was also being done by these scholars. For example, Tolley and 2 On arriving at the University of Chicago, one of the authors Ezekiel (1923) initiated work toward expand- of this paper (Levitt) inherited the office that Ted Schultz had oc- ing least squares estimation to multiple regres- cupied for decades. Upon hearing that Levitt was using Schultz’s sion. Ezekiel (1930) and Bean (1929) utilized office, the economist Sherwin Rosen lamented, “It’s too bad they gave you Ted’soffice—all the good ideas in that office are already graphical analysis to consider nonlinear im- used up.” For a more detailed discussion of Schultz’s life and eco- pacts. Waugh (1935) increased the applicability nomic contributions, see Johnson (1999). Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1261 behalf of farmers had provided little real ben- sis” (Leontief 1971, p. 5). Regardless of the efit to farmers. Even as an octagenarian, John- underlying explanation, due to the cutting son continued to publish in leading economics edge nature of the contributions in this field journals (e.g., Johnson 2000). during this time period, major journals were Other major players in this era include common publishing outlets for agricultural and . The collection work as noted earlier in figure 1. of Nerlove’s papers in economics covers an extraordinarily wide range of topics, from Later Empirical Contributions to spectral analysis to efficient methods of pooling time series of In aggregate, figure 1 suggests that agricul- cross sections, but much of his work had agri- tural economists of more recent generations cultural applications or considerable implica- have struggled to achieve the publishing suc- tions for agricultural analysis (e.g., Nerlove cess of their forebearers. One partial explana- 1958; Balestra and Nerlove 1966). Although tion, among many others, for this decline is Zvi Griliches, who won the Clark medal just that there has been an increased emphasis in four years before Nerlove, is not principally top journals on empirical work that features thought of as an agricultural economist today, plausibly exogenous sources of identification virtually all of his early work was on agri- (i.e., “natural experiments”); this research ap- cultural topics. This work included analyses proach has been less frequently used by agri- of the demand for fertilizer and other inputs cultural economists. There are, of course, many and most notably the diffusion of hybrid corn notable exceptions. For instance, Knoeber and (Griliches 1957), which is the single most cited Thurman (1994) use a plausibly exogenous of his more than 200 papers. shift in the contractual form of the incentive On the economics front, scheme facing producers of broiler chickens. Marschak and Andrews (1944) was seminal A firm that purchased broilers switched from in combining first-order conditions with the rewards based on the ordinal rank across pro- production problem for estimation. Mundlak ducers to one that rewarded producers based made important contributions along this line on performance relative to the mean. Using considering whether such systems involved this shift in compensation structures, the au- simultaneous equation bias (Mundlak and thors test the predictions of tournament the- Hoch 1965). This work provided a starting ory, finding the data consistent with all of the point for incredibly insightful refinements, testable predictions. particularly relevant to later agriculture work (e.g., Just and Zilberman 1983; Chambers and A Path Forward Just 1989). Another related area of study that has been influential more carefully considers Clearly, over the last third of the twenti- implications of risk, such as Just and Pope eth century, agricultural economists have not (1978) and Chambers and Quiggin (2002). achieved the same level of publication success As mentioned, the availability of agricul- in these top journals as they had in earlier tural data as well as the relevance of the years. While our empirical analysis is limited issues mentioned above to the types of ques- to top general journal publications, we tions approached in agricultural economics are conjecture that this decline would be true un- two possible explanations for why agricultural der any of a broad number of metrics—faculty economists were early in addressing these is- positions, quality and quantity of PhD stu- sues. Further, during these formidable years, dents, share of federal grant monies, share of one aspect that separated the work of agri- overall citations, etc. Our working hypothesis cultural economists from the work of their is that the combination of richer data sets com- economic counterparts may have been the ing on-line in sister fields—such as the large- former’s willingness to combine a strong theo- scale data sets used today in labor economics— retical component with empirical analysis. As and issues in agricultural economics losing pointed out by Fox, Keynes lamented in 1917 their relative cachet set forth the wheels of mo- that “to some economists the very idea of a tion to send senior scholars packing for more mathematical treatment of economic problems fertile fields and limited the attractiveness of is not only repugnant, but seems even absurd;” the field of agricultural economics to the best while Leontief assessed agricultural economics young minds. as “an exceptional example of a healthy bal- There are several ways to proceed as a field. ance between theoretical and empirical analy- In one case, we can simply allow the to 1262 Number 5, 2009 Amer. J. Agr. Econ. run its course, and perhaps in time our area will experiments—have shed insights on diverse again become en vogue, perhaps in part driven areas through tests that pit neoclassical the- by policy relevance, media relevance, or richer ory against prospect theory, that explore issues data sets of some sort being introduced. In in cost/benefit analysis and elicita- some sense, we are observing this phenomenon tion, that explore competitive market theory now with the recent surge of research in global in the field of alternative incentive schemes in climate change amongst economists. A more developing nations, as well as tests of auction active approach is to consider past successes theory,the theory of private provision of public and build on what seemed to work decades and of information assimilation among ago. As Levitt and List (2008) note and is professional financial traders.3 recalled above, what might well be consid- So how does this growth in field experiments ered the dawn of “field” experimentation was lend itself to agricultural economists? We the work of Neyman and Fisher. Unfortu- speculate that the future of field experimen- nately, researchers in farm management in the tal methods can be importantly engineered 1950s and 1960s never connected their knowl- by agricultural and resource economists. As edge of the experimental method with labora- Levitt and List (2008) discuss, one lesson tory experiments using human subjects, which learned from this most recent surge of field was at the very same time being pioneered experimental research is that the researcher by Vernon Smith at (e.g., carrying out field experiments faces a set Smith 1962; this might be one reason for the of challenges different from those that arise subsequent demise of the farm management either in conducting laboratory experiments movement). or relying on naturally occurring variation. Yet,there now might be a new train at the de- The field experimenter does not exert the pot, perhaps providing agricultural economists same degree of control over real markets as with a unique second chance. While laboratory the scientist does in the lab. Yet, unlike an experiments have dominated the experimen- empiricist who collects existing data, the field tal landscape in economics, the past decade experimenter is in the data generating business has witnessed a significant surge in studies that rather than data collection. Consequently,con- gather data via field experiments. Field experi- ducting successful field experiments demands ments in economics occupy an important mid- a different set of skills from the researcher: the dling ground between laboratory experiments ability to recognize opportunities for experi- and studies making use of naturally occurring mentation hidden amidst everyday phenom- field data (see List 2006; Levitt and List 2008). ena, an understanding of experimental design, This is beneficial because, on the one hand, knowledge of economic theory to motivate re- economic theory is inspired by behavior in the search and interpret empirical results, and pos- field, so we would like to know if results from session of interpersonal skills to manage what the laboratory domain are transferable to field are often complex sets of relationships involv- environments. Further, field experiments al- ing parties to an experiment. low a search for similar causal effects using In this manner, agricultural economists have different identification assumptions than the a comparative advantage in several factors in- strict assumptions necessary to achieve identi- strumental in conducting cutting-edge field ex- fication using naturally occurring data. perimental research. First, existing extension Harrison and List (2004) propose six fac- programs provide specialists with a unique tors that can be used to determine the field view of important everyday phenomena such context of an experiment: the nature of the as proposals for auctions to preserve conserva- subject pool, the information held by subjects, tion land, new policies, insurance mar- the commodity, the task or trading rules ap- kets, markets to hedge risk, and demand for plied, the stakes, and the environment in which new products. Many times, such ideas present the subjects operate. Using these factors, they themselves on the doorstep of extension spe- discuss a broad classification scheme for field cialists long before they reach the “Ivory experiments, shown in figure 2, which helps or- Tower.” Second, extension managers have the ganize one’s thoughts about the factors that requisite interpersonal skills and knowledge of might be important when moving from the existing markets and market players to man- lab to the field. To date, field experiments age the complex set of relationships involved in across each of the three field experimental classifications—artifactual field experiments, 3 See List’s website www.fieldexperiments.com, which contains framed field experiments, and natural field a catalog of field experiments in these and several related areas. Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1263

Controlled Data Naturally Occurring Data ______

Lab AFE FFE NFE NE, PSM, IV, STR

Lab: Lab experiment

AFE: Artefactual field experiment

FFE: Framed field experiment

NFE: Natural field experiment

NE: Natural experiment

PSM: Propensity score estimation

IV: Instrumental variables estimation

STR: Structural modeling

Note: This figure takes the field experimental classification scheme of Harrison and List (2004) and places it within other more traditional empirical approaches. Figure 2. Field experiments carrying out a clean experimental test. Finally, example, Schechter (2007b) evaluates risk and such an approach might better link extension trust behavior using artifactual field experi- specialists with theorists and statisticians who ments, finding that explains a sub- are trained in complementary research areas, stantial amount of decisions made by her group yielding a much greater than the sum of of Paraguayan farmers. Schechter (2007a) uti- any of the inputs could yield in isolation. As in lizes surveys that were run in conjunction with all such related cases, there is certainly avail- the experiments in Schechter (2007b) to pro- able that will flock to such joint vide an interesting discovery of theft deter- endeavors once the of field experimen- rence: gift-giving, resulting in the formulation tation is better recognized in this area. of a dynamic limited-commitment model on Such a research agenda, besides fostering theft. important research links, has the capacity to Framed field experiments with farmers have lend important empirical insights into the types also been used to inform policymakers, such of theoretical models that best fit the data. as the auction work done by Cummings, Holt, Equally as important will be those cases in and Laury (2003). The aforementioned au- which the models are refuted by the data—in thors began their research in response to The such instances, because the field experiments Flint River Drought Protection Act passed are able to collect enough facts to help con- in Georgia in 2000, which mandated that an struct new economic theories, there will be a auction-like process for foregoing irrigation be healthy give and take between empiricists and implemented in years of drought. By running theorists. Additionally,the field experiments in experimental auctions with farmers prior to this area can measure key parameters that pro- the actual irrigation auctions, they were able vide economists with useful information and to make suggestions on the auction design to might lend insights into policymaking discus- make it more competitive and at the same time sions. better understood by the participants. The underpinnings of such an approach are Examples of natural field experiments in this now being formed, as recent work across all area also exist. For example in a partnership three field experimental classifications in fig- with the management of a leading fruit farm in ure 2 has begun in a fruitful direction.4 For the United Kingdom, Bandiera, Barankay, and Rasul (2005, 2006) use a natural field exper- iment to explore interesting economic ques- 4 We only discuss a few here, but we would be remiss not to mention the work of Binswanger (1981), and the more recent work tions. Their subjects are farm workers, whose of Jayson Lusk (see, e.g., Lusk, Norwood, and Pruitt 2006). main task is to pick fruit. In one experiment, 1264 Number 5, 2009 Amer. J. Agr. Econ. workers were paid according to a relative in- References centive scheme that provided a rationale for cooperation as the of the group was Balestra, P., and M. Nerlove. 1966. “Pooling Cross maximized when workers fully internalize the Section and Time Series Data in the Estimation negative that their effort placed on of a Dynamic Model: The Demand for Natural others. Bandiera, Barankay, and Rasul (2005) Gas.” Econometrica 34(3):585–612. find that behavior is consistent with a model of Bandiera, O., I. Barankay, and I. Rasul. 2005. social preferences when workers can be moni- “Social Preferences and the Response to In- tored, but when workers cannot be monitored, centives: Evidence from Personnel Data.” pro-social behaviors disappear. In their 2006 Quarterly Journal of Economics 120(3):917– study, they find that individuals learn to coop- 62. erate over time, both from their experience and ——. 2006. “The Evolution of Cooperative Norms: from the experience of others. Together, these Evidence from a Natural Field Experiment.” advances help us to understand workplace B.E. Journal of Economic Analysis & Policy incentives. 6(2):Article 4. Bean, L.H. 1929. “A Simplified Method of Graphic Curvilinear Correlation.” Journal of the Amer- Conclusion ican Statistical Association 24(168):386–97. Binswanger, H. 1981. “Attitudes toward Risk: Theo- Agricultural economics stands today at a cross- retical Implications of an Experiment in Rural roads. Scholars working on agricultural issues India.” Economic Journal 91:867–91. once held a prominent role in conceptualiz- Chambers, R.G., and R.E. Just. 1989. “Estimating ing key empirical approaches that impacted Multi-output Technologies.” American Journal the economics profession more generally. Yet of Agricultural Economics 7(4):980–95. the past several decades have witnessed other Chambers, R.G., and J. Quiggin. 2002. “Optimal fields taking over central roles in the profes- Producer Behavior in the Presence of Area- sion. We hypothesize that this relative decline Yield Crop Insurance.” American Journal of is due to the combination of richer data sets Agricultural Economics 84(2):320–34. coming on-line in sister fields and the issues of Cummings, R.G., C.A. Holt, and S.K. Laury. 2003. agricultural economics losing their appeal. Yet, “Using Laboratory Experiments for Policy we do not view the situation as dire; rather, we Making: An Example from the Georgia Irriga- advocate an active approach to returning to the tion Reduction Auction.” Working paper No. roots of agricultural economics to help regain 06-14, Andrew Young School of Policy Studies our relative footing. Research Paper Series, September. In this manner, we view agricultural Ezekiel, M. 1930. “The Sampling Variability of economists possessing an enviable position Linear Curvilinear Regressions: A First Ap- to reestablish their place at the frontier of proximation to the Reliability of the Results economic science. Though several years have Secured by the Graphic ‘Successive Approx- passed since the Farm Management Move- imation’ Method.” Annals of Mathematical ment was crushed, the time is ripe for theo- Statistics 1(4):275–315. rists to combine with extension specialists to Fisher, R.A. 1926. “The Arrangement of Field Tri- lend insights into both normative and posi- als.” Journal of the Ministry of Agriculture of tive issues of the day. Years ago, there was Great Britain 33:503–13. not the methodological coherence and direc- Fox, K. 1986. “Agricultural Economists as World tion that could be used by experimentalists Leaders in Applied , 1917–1933.” on the farm; those days have passed. Agricul- American Journal of Agricultural Economics tural economists can more easily combine the- 68(2):381–6. ory and empirical work, taking advantage of Griliches, Z. 1957. “Hybrid Corn: An Exploration their unique opportunities to execute field ex- in the Economics of Technological Change.” periments. Whether such experiments revolve Econometrica 2(4):501–22. around exploring optimal incentive schemes Harrison, G.W., and J.A. List. 2004. “Field Ex- of farm laborers in Europe, measuring de- periments.” Journal of Economic Literature mand elasticities for Bt cotton in the South, 42(4):1009–55. or examining farmers’ risk preferences in the Herberich, D.H., S.D. Levitt, and J.A. List. 2009. Midwest, there is much to be done. In this way, “AJAE Appendix: Can Field Experiments Re- agricultural economists have a clear route back turn Agricultural Economics to the Glory to share the status of their subjects: substantial Days?” Unpublished Manuscript. Available at: creation. http://ageconsearch.umn.edu. Herberich, Levitt, and List Field Experiments in Agricultural and Resource Economics 1265

Johnson, D.G. 1950. “Resource Allocation under Cobb-Douglas Production Functions.” Econo- Share Contracts.” Journal of Political Economy metrica 33(4):814–28. 58(2):111–23. Nerlove, M. 1958. “Adaptive Expectations and Cob- ——. 1973. World Agriculture in Disarray. London: web Phenomena” Quarterly Journal of Eco- Macmillan. nomics 72(2):227–40. ——. 1999. “Theodore William Schultz.” Biograph- Schechter, L. 2007a. “Theft, Gift-giving, and Trust- ical Memoirs. 77:303–13. worthiness: Honesty Is its Own Reward in ——. 2000. “Population, Food, and Knowledge.” Rural Paraguay” American Economic Review American Economic Review 90(1):1–14. 97(5):1560–82. Just, R.E., and R.D. Pope. 1978. “Stochastic Spec- ——. 2007b. “Traditional Trust Measurement and ification of Production Functions and Eco- the Risk Confound: An Experiment in Rural nomic Implications.” Journal of Econometrics Paraguay.” Journal of Economic Behavior & 7(1):67–86. Organization 62:272–92. Just, R.E., and D. Zilberman. 1983. “Stochastic Schultz, H. 1925. “The Statistical Law of Demand as Structure, Farm Size and TechnologyAdoption Illustrated by the Demand for Sugar.” Journal in Developing Agriculture.” Oxford Economic of Political Economy 33(5):481–504. Papers 35(2):307–28. ——. 1928. The Statistical Laws of Demand and Sup- Keynes, J.N. 1917. The Scope and Method of ply, with Special Application to Sugar. Chicago: Political Economy. London: Macmillan. University of Chicago Press. Knoeber, C.R., and W.N. Thurman. 1994. “Test- Schultz, T. 1964. Transforming Traditional Agricul- ing the Theory of Tournaments: An Empirical ture. New Haven: Press. Analysis of Broiler Production.” Journal of La- Smith, V. 1962 “An Experimental Study of Com- bor Economics 12(2):155–79. petitive Market Behavior.” Journal of Political Leontief, W. 1971. “Theoretical Assumptions and Economy 70(2):111–37. Nonobserved Facts.” The American Economic Splawa-Neyman, J., 1990[1923]. “On the Appli- Review 61(1):1–7. cation of Probability Theory to Agricultural Levitt, S., and J. List. 2008. “Field Experiments in Experiments. Essay on Principles. Section 9.” Economics: The Past, The Present, and The Statistical Science 5(4):465–472. Translated and Future.” NBER Working paper 14356, NBER, edited bv D.M. Dabrowska & T.P. Speed from Cambridge, MA. the Polish original, which appeared in Roczniki List, J.A. 2006. “Field Experiments: A Bridge be- Nauk Rolniczyc, Tom X (1923): 1–51 (Annals tween Lab and Naturally Occurring Data.” ofAgricultura1 Sciences). Advances in Economic Analysis and Policy Tolley, H.R., and M.J.B. Ezekiel. 1923. “A Method 6(2):1747. of Handling Multiple Correlation Problems.” Lusk, J.L., B. Norwood, and R. Pruitt. 2006. “Con- Journal of the American Statistical Association sumer Demand for a Ban on Subtherapeutic 18(144):993–1003. Antibiotic Use in Pork Production.” American Waugh, F.V. 1935. “A Simplified Method of Deter- Journal of Agricultural Economics 88:1015–33. mining Multiple Regression Constants.” Jour- Marschak, J., and W.H. Andrews. 1944. “Random nal of the American Statistical Association Simultaneous Equations and the Theory of 30(192):694–700. Production.” Econometrica 12(3,4):143–205. Working, H. 1925. “The Statistical Determination Mundlak, Y., and I. Hoch. 1965. “Consequences of Demand Curves.” The Quarterly Journal of of Alternative Specifications in Estimation of Economics 39(4):503–43.