Limitations on the Use of Mathematical Models in Transportation Policy Analysis

Limitations on the Use of Mathematical Models in Transportation Policy Analysis ABSTRACT: Government agencies are using many kinds of mathematical models to forecast the effects of proposed government policies. Some models are useful; others are not; all have limitations. While modeling can contribute to effective policyma king, it can con tribute to poor decision-ma king if policymakers cannot assess the quality of a given application. This paper describes models designed for use in policy analyses relating to the automotive transportation system, discusses limitations of these models, and poses questions policymakers should ask about a model to be sure its use is appropriate. Introduction Mathematical modeling of real-world use in analyzing the medium and long-range ef- systems has increased significantly in the past fects of federal policy decisions. A few of them two decades. Computerized simulations of have been applied in federal deliberations con- physical and socioeconomic systems have cerning policies relating to energy conserva- proliferated as federal agencies have funded tion, environmental pollution, automotive the development and use of such models. A safety, and other complex issues. The use of National Science Foundation study established mathematical models in policy analyses re- that between 1966 and 1973 federal agencies quires that policymakers obtain sufficient infor- other than the Department of Defense sup- mation on the models (e-g., their structure, ported or used more than 650 models limitations, relative reliability of output) to make developed at a cost estimated at $100 million informed judgments concerning the value of (Fromm, Hamilton, and Hamilton 1974). Many the forecasts the models produce. The ap- more models have been developed since 1973. propriate use of models and their output can contribute to effective policymaking, but misuse of models or misinterpretation of their out- The appropriate use of madets and put can mislead decision-making. their output can contribute to efferctive policymaking, but misuse of rn~ddsor misEnterpretatSan sf What is a Model? their output can mislead decision- making. A model is a representation of reality. Neces- sarily it is a simplification or abstraction. A model may be a physical representation, for ex- The success of models that simulate physical ample, a globe. A mathematical model differs systems has often been dramatic. A widely from the more tangible physical model, in that known example is the modeling that the "reality" is represented by an equation or series National Aeronautics and Space Administra- of equations. There are many kinds of models. tion conducted in its lunar exploration pro- This paper is concerned with mathematical gram. Computer-simulated "landings" and models, in particular, econometric models. "retrievals" were conducted hundreds of times Econometric models have their basis in eco- before the first manned landing was attempted. nomic theory, are derived using statistical tech- Successful simulation of physical systems niques, and are used in studying relationships has encouraged development of mathematical among economic variables. models of social/economic/political "systems." Two important elements of equations are For example, mathematical models are used to variables and parameters. Variables represent forecast such economic indicators as the gross the elements of the system being modeled (e.g., national product, capital investment rates, the number of automobiles in the United employment rates, federal tax revenues, and States). In a mathematical model the values of other measures of the national economy. These some variables are specified outside the model. socioeconomic models are designed to project These variables are called exogenous or forecast the future behavior of real-world variables. The values of other variables are systems under scrutiny. While physical prin- calculated within a model. These variables are ciples are well understood and stable enough to called endogenous. Knowing which variables be predictable, social, economic, and political are exogenous and which are endogenous can behavior is not well understood, not stable, and be important in understanding the results of a not very predictable (with some exceptions) ex- model. This is discussed later. cept within broad limits. Parameters of an equation are factors that Some relatively new forms of mathematical qualify the variables. For example, one might models have been developed in recent years for calculate the number of large-size cars sold in a year as a function of the annual income of car buyers and their tendency to buy large cars. of and assumptions about the real world. These People with higher incomes might purchase observations and assumptions support the more large-size cars than people with lower in- modeler's selection of variables, parameters, comes. Thus, a simple form of an equation to functions, and the basic logic of a model. Some calculate large-size car sales might equate models represent systems whose behavior is sales with some number "a" times the number well understood. An example is an electrical cir- of people with high incomes, plus some number cuit. Observations and assumptions con- "b" times the number of people with low in- cerning the behavior of such systems are ex- comes. In this example, the numbers chosen for plicit and objective. "a" and "b" are the parameters of the equation. The interpretation of the parameters depends on the specification of the equation structure, that is, on the mathematical form of the equa- An inherent limitation of models is tion. The specification of the equations and the that judgments are necessary in derivation of the values of the parameters are building them. important tasks in creating a model. Parameters remain constant for a particular analysis, while the values assigned to variables change. As information about real-world systems The example above involves only one equa- becomes less precise or harder to measure, tion. Usually, a modeler needs to address more more assumptions must be made. Modeling than one question at the same time. For exam- becomes a less precise endeavor as it moves ple, the modeler may wish to predict both the away from physical systems and toward social demand for automobiles and fuel consump- systems. Modeling an electrical circuit is a tion. Since these variables are related to each straightforward task, compared to modeling other, a more complex model with more than human decision-making. Also, the nature of the one equation may be required. A modeler might information about physical and human systems create a model in which automobile demand is is different. Good historical information about a a function of several variables (e.g., income) physical system is quite valuable in modeling and in which fuel consumption is a function of future performance, because the system usual- auto demand, plus several other variables (e.g., ly does not change. Good information about a the fuel economy of the automobiles sold). The social system is equally desired but may be of specification of the equations and the relation- less value in forming assumptions, because ships among the various equations that social systems often change and in ways that describe how the variables are linked in the real were not part of the past. Thus, to understand a world represent the overall logic of the model. model's limitations it is important to under- An analyst needs to understand the logic of a stand the assumptions that were used to create model in order to use it intelligently. it. When constructing models, model builders Mathematical models have been used pre- usually experiment with a wide range of alter- dominantly in two ways in studies of the auto- native forms to find the closest fit of the equa- mobile transportation system-forecasting and tions to the sample data. Undue emphasis on policy analysis. close fit, however, sometimes leads to mis- When a model is used primarily for fore- specification of the model structure, for exam- casting, the user exercises the model to pro- ple, by use of spurious variables in an equation duce a forecast based on the general assump- to improve its fit. When used for forecasting or tion that past relationships among variables will policy analysis, such a misspecified model is continue. Often this is done to identify future poorer than a proper specification that fits less problems that may occur if past relationships closely to sample data. continue. A model is obviously based on observations Policy analysis applications also produce forecasts, but the concern is with the different futures associated with different policy assumptions. Policy changes are imposed on the model and forecasts made to assess the effects of the An important inh changes. a model is creat For example, a forecasting use of an out. automobile demand model would involve specification of the exogenous variables and in- put using the best information available to the user. The primary interest is to estimate future effectiveness in reproducing values over some automobile demand as it is likely to occur under historical period where the output values are future conditions. A policy analysis application known. After this capability of the model is might examine the expected effects on future tested, it is placed into use. automobile demand of increases in gasoline The steps clearly are complicated but, as prices caused by an increased federal tax. In stated above, each step may seem relatively both cases, the model output-a forecast-is of straightforward. In reality this is not the case. interest. Each step requires judgment. It is difficult to Models that produce forecasts are usually represent real-world systems in terms of designed to produce either short-term or long- mathematical relationships. Data are often term forecasts. The assumptions, structure, and unavailable or inaccurate. Combining the sub- associated factors required for the one purpose system models to create the model is seldom often make the model less suitable for the simple.

Load more