-1- Validating Simulation Models: AGeneral Framework and Four Applied Examples Robert E. Marks Australian Graduate School of Management Australian School of Business University of NewSouth Wales SydneyNSW 2052 Australia
[email protected] June 18, 2007 ABSTRACT:This paper provides a framework for discussing the empirical validation of simulation models of market phenomena, in particular of agent-based computational economics models. Such validation is difficult, perhaps because of their complexity; moreover, simulations can prove existence, but not in general necessity.The paper highlights the Energy Modeling Forum’sbenchmarking studies as an exemplar for simulators. A market of competing coffee brands is used to discuss the purposes and practices of simulation, including explanation. The paper discusses measures of complexity,and derivesthe functional complexity of an implementation of Schelling’s segregation model. Finally,the paper discusses howcourts might be convinced to trust simulation results, especially in merger policy. Acknowledgments: We thank Rich Burton, Yaneer Bar-Yam, Ian Wilkinson, Shayne Gary, Peter McBurney, Hill Huntington, Leigh Tesfatsion, the participants at the Fourth UCLA LakeArrowhead Conference on Human ComplexSystems, and twoanonymous referees for their comments and assistance. JEL codes: C63, C15 Ke ywords: simulation; validation; agent-based computational economics; antitrust; functional complexity; Schelling; sufficiency; Daubert criteria -2- 1. Introduction The apparent past reluctance of some in the discipline to accept computer simulation models of economic phenomena might stem from their lack of confidence in the behaviour and results exhibited by such models. Even if there are other reasons, better validation of such models would reduce anyskepticism about their results and their usefulness. Leombruni et al.