Using "InsiderEconometrics" to Study Productivity By ANN BARTEL, CASEY ICHNIOWSKI, AND KATHRYN SHAW* Great advances have been made in theory At the micro level, the firms or plants that and in econometric techniques, but these we analyze differ a greatdeal, even within will be wasted unless they are applied to what one might think of as a well-defined the right data. "industry".... They differ in the parti- -Zvi Griliches (1994 p. 2) cular assortment of products they may produce .... [and] in the inputs and tech- Griliches' 1994 presidential address consid- nologies that they use to producethem .... ers the limited success economists had in trying Unfortunately,standard census type data to account for the slowdown of do not provide enough additional infor- productivity relevant and char- the 1970's and 1980's, and us towardthe mation or product plant "urges acteristics to allow one to a task of observation and measurement."In the pursue substantive .... To make further 1990's, the rates of analysis high productivity growth we need to infuse [production the need for progress, emphasized new models of pro- functions] with new data and appropriate ductivity, this time turning to estimating theoretical and econometric models for organization-leveldeterminants of productivity dealing with the real heterogeneitythat is focusing on businesses' use of new computer- the hallmarkof the world we live in. based information technologies (IT), and new methods of work organization (Timothy Ichniowski and Shaw (2003) use the term "in- Bresnahanet al., 2002). In this paper, we take sider econometrics" to describe productivity up the charge to develop new data and new studies that combine extensive field work to methods for modeling the productivityof orga- assemble useful organization-level data sets nizations. We summarizethree methods for as- with rigorous econometrichypothesis testing of sembling data for an "insider econometrics" the effects of organization-specificdeterminants study of the productivityof organizations,and of productivity.This section summarizes three we illustrate one method that we refer to as approachesto "insidereconometrics" studies. "informedsurvey analysis." 1. Cross-Organization Studies Based on I. Three Methodsfor ConductingInsider Plant Visits.-Insider econometrics is defined Studiesof OrganizationalProductivity by two broad principles. First, it uses field work to generate a detailed understandingof a Griliches and Jacques Mairesse (1995 pp. specific production process, its technology, 22-24) describe why it is so challenging to and the nature of the work in a particular assemble the "rightdata" to investigate produc- industry. This field work in turn provides tivity determinantsof real organizations: valuable insights about how to model produc- tion in that industry and what data to collect to estimate those models. Second, detailed operating data from the industry are used to estimate econometric productivity models * Bartel: Columbia Business School, Columbia Univer- that permit convincing tests of hypotheses sity, 3022 Broadway, New York, NY 10027, and NBER; about the determinants of productivity. Ichniowski: Columbia Business School and NBER; Shaw, One method of implementing insider econo- StanfordBusiness School, StanfordUniversity, 518 Memo- metrics is to gather data from firms on the very rial Way, Stanford, CA 94305, and NBER. We thank that use in monitor- Ricardo Correa and Yoonsoo Lee for their excellent performancemeasures they research assistance and Lawrence Katz for his valuable ing production. Ichniowski, Shaw, and coau- comments. thors implementthis approachin their studies of 217 218 AEA PAPERSAND PROCEEDINGS MAY2004 the effects of human-resource management 3. Insider Productivity Research with "In- practices on productivity in the steel industry, formed Surveys."-A third approach for col- visiting about 85 plants in the steel industryto lecting "the right data" for organization-level conduct interviews and obtain data. The advan- productivitystudies is to obtain data from "in- tages of this approachare that researcherscan formed surveys."Plant visits and interviews are model very sensible cross-firmproduction func- conducted at a small sample of plants in an tions, and can model why some firms adopt new industryand then used to understandthe indus- human-resources practices and some do not. try's productionprocess and technology and to This approachis, unfortunately,also very costly develop a narrow industry-specificsurvey. We and time-consuming. illustrate this approach using our results from the valve industry below. Note however, that 2. Single-FirmStudies.-A second and more others have utilized "informed surveys" that common way to conduct insider productivity Census researchers with expertise in specific researchis to focus on the operationsof a single industrieshave tailored to specific industriesor firm. Insider insights about key productionpro- occupations.2This third approachis quite sim- cesses in the firm identify situations where in- ilar to the first above and is considerably dividual employees, teams of workers, or cheaper to undertake,but it suffers from poten- separate establishments inside the same com- tial recall bias or measurementerror. pany comprisethe productionunits. These with- in-firmstudies then provide convincing analysis II. Insider Insights into the U.S. Valve-Making of the effects of changing personnel practices Industry across these units. Examples include Edward Lazear's (2000) study of piece rates in wind- To pursuethis thirdapproach for plants in the shield installation, Barton Hamilton et al.'s U.S. valve-making industry (SICs 3491, 3492, (2003) study of team methods in apparel man- 3494, and 3593), we conducted site visits and ufacturing,Martin Gaynor et al.'s (2004) study interviews at five valve-making plants during of incentives in an HMO, Rosemary Batt's 1999-2000 and in 2002 (during survey devel- (1999) study of teams in a telecommunications opment). A valve is typically a metal device company, and studies by Bartel (2004) and Bar- attachedto pipes that regulates the flow of liq- tel et al. (2003) of employee satisfactionin bank uids or gases, such as the flow of naturalgas in branches of one Canadian company and one a heating system, or the control of liquids in a U.S. company, respectively. The advantage of chemical factory. The central production pro- this approach is that the research can often cess in valve-makingis the machiningphase. A model the sources of productivity change, in- simple valve would be made by taking a steel cluding changes in the selection of workers. Of block or pipe and completing several processes course, single-firm studies cannot model the on one or more machines, such as etching causes of the adoption of practices. grooves at each end for screwing the valve to pipes, boring holes at different spots to attach control devices, and then making and attach- ing the various devices that control the flow. Based on our visits and interviews at these sites, 1 Specifically, Ichniowski, Shaw, and coauthors visited we developed an industry-specific survey to 45 productionlines of 20 companies in the U.S. integrated steel industry(Ichniowski et al., 1997), five integratedsteel mills at two Japanese companies (Ichniowski and Shaw, 2 1999), and 34 productionlines operatedby 19 U.S. minimill Examples of the use of Census surveys are Thomas companies (Brent Boning et al., 2001). Otherinsider studies Hubbard (2004) and George Baker and Hubbard (2003) noteworthy for visits and data from many companies and for studies of the effects of information technologies in work sites include John Paul MacDuffie's (1995) analysis of trucking; Luis Garicano and Hubbard (2003) for their productivityeffects of human-resourcesmanagement prac- study of lawyers; and Chad Syverson (2003) for his study tices in automobile assembly plants and Kim B. Clark's of the cement industry. Maryellen Kelley (1994) con- (1984) study of unions and productivity in the cement ducts her own survey of machine operations in 21 indus- industry. tries to study the effects of work organization and IT. VOL. 94 NO. 2 NEW DATAAND NEW QUESTIONSIN PERSONNELECONOMICS 219 measure productivity, technologies, and work TABLE 1-SUMMARY STATISTICS ON PRODUCTION TIMES practices.3 IN VALVE MACHINING, NEW COMPUTER-BASED PRODUCTION TECHNOLOGIES, AND HUMAN-RESOURCE MANAGEMENT PRACTICES A. Measuring Efficiency in the Machining Process A Mean valuea itself involves time to Machining setup pro- Component 1997 2002 Log change gram machines so they will perform the right combination of tasks for the valve's Setup time 0.49 0.28 -0.681 specifica- Run time 0.45 0.39 -0.371 the actual run time to the ma- tion, complete Inspection time 0.05 0.03 -0.334 chining, and inspection time to verify the of the valves. We measure these three Total time 1.03 0.72 -0.481 quality Number of 5.63 4.97 -0.189 components by asking survey respondents to machines provide setup time, run time, and inspection time in 1997 and 2002 for the product they B the most over those Our sur- Fraction of observations produced years. Technology or vey results show that the production times for practiceb Using, 2002C Adopting, 1997-2002d these declined over the last five products years FMS 0.337 0.151 (Table 1). Auto sensors 0.283 0.137 3-D CADe 0.738 0.387 B. Technologies and Valve-MakingEfficiency Basic training 0.333 0.119 Technical training 0.726 0.211 Teams 0.647 0.298 Today, the central piece of equipment in the valve-making