Developments in Business Simulation & Experiential Learning, Volume 26, 1999 THE NEED TO MEASURE VARIANCE IN EXPERIENTIAL LEARNING AND A NEW STATISTIC TO DO SO John R. Dickinson, University of Windsor James W. Gentry, University of Nebraska-Lincoln ABSTRACT concern should be given to the learning process rather than performance outcomes. The measurement of variation of qualitative or categorical variables serves a variety of essential We contend that insight into process can be gained purposes in simulation gaming: de- via examination of within-student and across-student scribing/summarizing participants’ decisions for variance in decision-making. For example, if the evaluation and feedback, describing/summarizing role of an experiential exercise is to foster trial-and- simulation results toward the same ends, and error learning, the variance in a student’s decisions explaining variation for basic research purposes. will be greatest when the trial-and-error stage is at Extant measures of dispersion are not well known. its peak. To the extent that the exercise works as Too, simulation gaming situations are peculiar in intended, greater variances would be expected early that the total number of observations available-- in the game play as opposed to later, barring the usually the number of decision periods or the game world encountering a discontinuity. number of participants--is often small. Each extant measure of dispersion has its own limitations. On the other hand, it is possible that decisions may Common limitations are that, under ordinary be more consistent early and then more variable later circumstances and especially with small numbers if the student is trying to recover from early of observations, a measure may not be defined and missteps. Similar to what Ross (1991) found with the maximum theoretical value may not be salespeople who are in danger of not making quota, attainable. This paper reviews existing measures students desperate to finish in the black may make of dispersion and describes a new measure which is radical decisions (conceivably outside company always defined and for which a maximum value is policy). A measure of variance could be used in the always attainable. game’s diagnostic software to signal the instructor that the nature of decision-making has changed CONSIDERATION OF VARIANCE dramatically. Roles of Variance in Experiential Learning Another central issue in the administration of simulation games is comparability across worlds or Measurement in the domain of experiential industries. Due to a number of factors, the learning does not have a rich history (Gentry et al. worlds/industries may not be directly comparable: 1998). Recent Association for Business Simulation and Experiential Learning (ABSEL) conferences, ¾ Defining parameters may be varied by the for example, have seen considerable discussion instructor to make the simulated environment about the measurement of learning, without much different across worlds/industries. consensus being reached in terms of the types of criterion variables to be measured. To the extent that educators have even attempted to measure learning, analyses have generally been limited to the investigation of mean differences of outcome variables, despite frequent admonitions that greater 153 Developments in Business Simulation & Experiential Learning, Volume 26, 1999 ¾ Worlds/industries may comprise different central location (e.g., mean, median, mode) and numbers of competing companies. measures of dispersion (e.g., range, percentiles, variance, standard deviation, coefficient of variation) ¾ The dynamics of competition may vary, (cf. Keller and Warrack 1997, Chapter 4). with some worlds/industries collectively employing more aggressive or conservative Other variables related to simulation games are strategies or the mix of aggressive- qualitative or categorical. That is, they are decisions conservative competitors varying across or results of kind or type rather than degree worlds/industries. (Dickinson 1995). Examples include decisions relating to advertising messages, sales promotion ¾ Companies may “go missing” during the types, product mix, types of research and competition differentially across development, types of marketing research studies, worlds/industries. Is it fair to reward more menus of ethical alternatives, and so on. More highly the second-place finisher who makes specifically, for instance, Micromatic (Scott et al. large profits in a world where several 1992) makes available a list of nine types of competitors have defaults than the highest competitive information participants may purchase. performer in a more conservative, more The Marketing Game! (Mason and Perreault 1987) competitive world, even though that features a menu of five advertising types. highest performer has generated less profit? For qualitative variables, too, it is often useful to Measures of variation across a number of input and summarize their distributions. Percentage output variables could help make company distributions, either univariate or in some performances more comparable for evaluation crosstabular form, are the most common type of purposes. descriptive statistic for qualitative variables. Otherwise, description of such variables most often Types of Input and Output Variables takes the literal form of bar and pie charts (cf. Anderson, Sweeney, and Williams 1996, Chapter 2; Strategic decisions in business simulation games Siegel and Morgan 1996, pp. 23-26). are predominantly quantitative. Many comprise dollar amounts, either in the form of expenditures Measures of dispersion of qualitative variables, or prices. Other “amounts” include numbers of although they do exist, are not at all well known. employees, numbers of distributors, units of None of eight contemporary business statistics products manufactured or ordered, and so on. textbooks (list available on request from the senior Many output variables are also quantitative, author) presents any such measures.1 None of the including basic sales, profit, and other income SPSS DESCRIPTIVES, FREQUENCIES, or NPAR statement results, inventory levels, perceptual map TESTS procedures includes any such measures distances, indices of morale or product quality, and (SPSS 1997, pp. 256, 382, 569). so on. For purposes of monitoring participant performance, providing feedback to participants, and the conducting of basic research, it is often necessary to summarize the distributions of both input and output variables. For quantitative 1 variables, appropriate descriptive statistics are well An exception is the variance of a binomial known and are usually classified into measures of random variable. 154 Developments in Business Simulation & Experiential Learning, Volume 26, 1999 Following is a brief review of existing measures of EXISTING MEASURES OF DISPERSION dispersion for qualitative variables. A new statistic which rectifies two common limitations of extant At least 10 measures of dispersion of qualitative measures, and is particularly suited for application variables have been developed, beginning as early as to simulation games, is then described. 1912. The most familiar of these are presented below, accompanied by a brief description of their THE CONCEPT OF VARIANCE OF A characteristics. Let J indicate the number of QUALITATIVE VARIABLE categories of the variable, n indicate the total number of observations, n j indicate the number of The intuitive notion of variance or dispersion of a observations in category j (j=1 to J), and pj indicate qualitative variable is easily illustrated. A the proportion of observations in category j (pj=nj/n, population comprising all women and no men j=1 to J). would have zero gender dispersion. A population comprising 80% women would be more Using this notation coupled with the concept of concentrated in gender than a population dispersion as described above, minimum dispersion comprising 60% women. A population comprising exists when nj=n for some single j and ni=0 for all 50% women and 50% men would exhibit maximal i≠j. Maximum dispersion exists when dispersion. A population of 80% women would be n1=n2=...=nJ=n/J (i.e., p1=p2=...=pJ). of equal gender dispersion as a population comprising 80% men. Index of Diversity (Gini’s Concentration Measure) and Index of Qualitative Variation A simulation game participant selecting an informative advertising message in each of, say, 10 Apparently the first measure of qualitative decision periods, would exhibit less variability than dispersion was a measure of concentration proposed a participant selecting an informative message in by Corrado Gini in 1912 (Agresti and Agresti 1977, each of five periods, a persuasive message in each p. 206). of three periods, and a reminder message in each of two periods. J 2 J 2 D = C = 1− ()/ n = 1− p (1) ∑ n j ∑ j Finally, a simulation game providing a menu of, j j say, eight types of sales promotions has a greater Gini’s measure currently is incorporated into the potential variability than does a game providing a Logit Loglinear Analysis procedure of SPSS menu of five types of sales promotions. This (Norusis 1994, p. 216). parameter of measures of dispersion reflects the total number of categories comprising the variable, The interpretation of D or C is straightforward: “The not the number of categories containing nonzero index of diversity gives the probability that a pair of observed frequencies. “...the variation in a nominal randomly selected observations will be in different variable depends in part on