Mixed Methods and Simulation Research Designs

MIXED METHODS AND SIMULATION RESEARCH DESIGNS

INTRODUCTION

Following my plan of presenting themes for possible discussion and inquiry to the audience of the Journal of the International Journal of Transport Economics, I thought whether I could have elaborated in greater details my reflections on the related issue of integration of literature reviews previously published1.

As an introduction of the following considerations, I recall to have noticed quite a synthetic but effective message on our task of research designers:

“ The point at which the data analysis begins and ends depends on the type of data collected, which in turn depends on the sample size, which in turn depends on the research design, which in turn depends on the purpose” 2.

In fact, this message should be considered as a guide in an effort to underline what its authors presumably have implied in their wording and that I wish to give here a more explicit indication than I may have done before: appropriate importance should be given to the methodological options at our disposal estimated to be more suitable in the different circumstances dependent on the nature of the study. Hence, we should make choices of methods that are both philosophically defensible and, at the same time, practicable and responsive. For instance, taking the case of researchers becoming aware that the purposes of their study often involve both quantitative and qualitative aspects, it follows that it will be more appropriate to explore the opportunity of developing mixed method research designs that may be better associated with their investigation purposes.

1 See the Journal, Vol. XXXIII, 2, 2006 2 William E. Hanson, John W. Creswell, Vicki L. Plano Clark and Kelly S. Petska J, David Creswell, “Mixed Methods Research Designs in Counseling Psychology”, Journal of Counseling Psychology 2005, Vol. 52, No. 2. Then, following the ideal thread of earlier observations, I choose to concentrate here on the mixed methods methodological approach employed as a research configuration in social sciences at large - and in our discipline in particular - especially in cases of integration or connections of quantitative and qualitative data. Moreover, I wish to comment briefly on another methodological issue prompted by the fact that when using adaptive rather than optimizing strategies, inferring possible consequences is often impossible. Then – according to Axelrod – simulation becomes the necessary option3.

MIXED METHODS RESEARCH SCHEME

Considering the first issue, I observe that an increasing development of mixed methods research is commonly accepted as a typical feature of contemporary research designs to profit from the inclusion of both quantitative and qualitative sources of information, mostly when generalization of results and feedback evaluations are the purposes pursued. In fact, as it has been remarked, the insertion of qualitative data can help researchers to enlighten relationships emerging from quantitative data. Similarly, the inclusion of quantitative data can help in compensating for the fact that qualitative data normally cannot be generalized 4.

Undeniably, if I consider the experience of editorship of this Journal, I should come to the conclusion that the quantitative paradigm has been and still is the dominant one, although there has been some within-discipline dissimilarity from one research ‘community’ to another. Progressively, a call for a socio-psychological based approach to transportation analysis emerged with emphasis, particularly in the study of social determinants of mobility and in welfare-economic investigation of the implications of unpaid consequences of conducts.

Major qualitative research has been undertaken increasingly in the transportation sector to assess – for instance - users’ thought-processes in decision-making

3 See R. Axelrod, Advancing the Art of Simulation in the Social Sciences - Obtaining, analyzing, and sharing results of computer mode”, Complexity. 1997, Vol. 3, No. 2, 4 See A. J., Onwuegbuzie, R. B. Johnson “Mixed research”, in R. B. Johnson, L. B. Christensen (Eds.), Educational research: Quantitative, qualitative, and mixed approaches, 2004 (2nd ed., pp. 408-431), Allyn & Bacon, Needham Heights. 2 involving issues such as “willingness-to-pay” or – say – “hedonic pricing” and generally when a tentative measurement of contribution to the generation of negative externalities is the kernel of possible trade-offs.

Consistent with the prevailing literature on mixed research design 5, combining quantitative and qualitative analyses has been advocated when the process presents evident complementary strengths as in what Denzin (1978)6 dubbed triangulation of different data source, i.e. the process of testing the consistency of findings obtained through different instruments for the study of the same occurrence. Specifically, the combination of the two approaches seems useful when:”

results from qualitative interviews can help to identify unobserved heterogeneity in quantitative data as well previously unknown explaining variables and misspecified models; results from the qualitative part of mixed methods design can help to understand previously incomprehensible statistical findings; qualitative research can help to discover quality problems of quantitative measurement instruments; and quantitative research can be used to examine the scope of results from a qualitative study and support the transfer of such findings to other domains” (Kelle, 2005) 7.

As Tashakkori and Teddlie argue 8, mixed methods are properly employed when quantitative-confirmatory/deductive and qualitative-exploratory/inductive questions are posed in the same study. It is a situation frequently faced by researchers in our discipline when they need complementary approaches in answering simultaneously inductive and deductive queries, as I had the opportunity to argue in the previous notes when I discussed the illustrative issue of the inter-disciplinary procedure of conceptualization of land use and the related inference processes. Particularly, beyond the logic of inquiry which includes the use of induction and deduction, I made the case for the employment of abductive reasoning which relies on the best

5 See, for instance, A. Tashakkori, C Teddlie. Mixed Methodology: Combining Qualitative and Quantitative Approaches, 1998, Sage Publications, Thousand Oaks, Calif..; Creswell J., “Mixed-method research: introduction and application. in: Cizek GJ, ed. Handbook of Educational Policy. San Diego, Calif: Academic Press; 1999, 455-472; Tashakkori A, Teddlie C, eds. Handbook on Mixed Methods in the Behavioral and Social Sciences. Sage Publications, Thousand Oaks, Calif., 2003. 6 Denzin, N. K., “Triangulation”. In N. K. Denzin (ed.), The research act: An introduction to sociological methods, 1978, McGraw-Hill, New York: 7 U. Kelle, Mixed Methods as a Means to Overcome Methodological Limitations of Qualitative and Quantitative Research, Workshop on mixed-methods held on October 26-27th 2005 at the University of Manchester. 8 See A. Tashakkori, C. Teddlie, “Issues and dilemmas in teaching research methods courses in social and behavioural sciences: US perspective”, International Journal of Social Research Methodology, 2003, vol. 6, 1. 61-77 3 of a set of explanations for understanding one’s results, taking us a substantial step further than pure deduction or induction because it helps us to meet theory and data in a creative way.

Regardless of the simultaneity of queries, designing a research methodology in which both the quantitative and qualitative approaches are involved still entails matters of priority, concurrence or sequentiality besides that of integration whenever the mixing or connecting of different sources of information occurs. If, for example, the purpose is to triangulate or converge the results, then data may be collected concurrently. Conversely, if elaboration of the results is the issue at stake, then a sequential design may be more appropriate (Hanson, Creswell, et. al., 2005, op.cit.).

Above all, several articulated types of major mixed methods research designs are envisaged and classified as: concurrent triangulation, concurrent nested, concurrent transformative, and sequential explanatory, sequential exploratory, sequential transformative.

It is quite obvious that the technical level for planning and implementing mixed- method researches seems to have been a main focus of the most recent investigation by scholars in the social and behavioral sciences whenever the mixed method approach was reckoned to have a proper application.

The extant surge of debate on technical taxonomy seems to have somehow reduced the preoccupation with explicit assumptive differences among strategies and the relevant philosophical traditions that have characterized the early stage of the discussion on mixed methods paradigms: positivism and interpretevism. On its side, the philosophical discussion has been occasionally so intense and even disruptive, to press researchers towards pedantic issues of identification or classification as

4 members of a given school of thought instead of another 9. Moreover, as Greene expressed 10, there were:

“ different stances on the sensibility of mixing paradigms while mixing methods in evaluative inquiry: (1) the purist stance, whose adherents argue against the sensibility of mixing paradigms; (2) the pragmatic stance, in which paradigms are viewed as useful conceptual constructions but of little value in guiding practice, and in which methodological decisions should be made to maximize contextual responsiveness; and (3) the dialectic stance, in which paradigms are viewed as important frameworks for inquiry practice, and the inevitable tensions invoked by juxtaposing different paradigms are viewed as potentially generating more complete, more insightful, even more revisioned or transformed evaluative understandings”.

Philosophical differences - progressively becoming more severe - led to profound disagreement and to a search for a dominant position by the different schools of thought, sometimes setting aside the ultimate purpose of inquiry: the attainment of knowledge. To stand out against such a diversion, a wiser research strategy pursuing the search for a sort of ‘equilibrium’ between the different traditions seems to be the proper plan of action.

In a mixed-method scheme a reconciliation of those traditions may be achieved. For instance, following Greene (1997), efficiency and utilitarianism which are the essential characteristics of post-positivism, and diversity and community that are the values considered by interpretevism can both constitute the core of a mixed-method approach through the consideration of the action-oriented and value-based dimensions of each of the different inquiry traditions. Thus, “mixed methods have the potential of enabling us to understand more fully, to generate insights that are deeper and broader, and to develop important knowledge claims that respect a wider range of interests and perspectives” (Greene, op.cit.).

Due to its features, mixed method research has been labeled: the third major research paradigm and gained the legitimacy of being a stand-alone research design. However, besides its strengths, controversial issues still hinder its potential and some of the details remain to be worked out by research methodologists (e.g.,

9 As it has been reported: “qualitative researchers have spent more time defining quantitative methods than quantitative researchers have themselves” .See J. Brennan, “Mixed method research: a discussion paper”, Economic & Social Research Council, National Centre for Research Methods, Discussion paper NCRM/005 10 J. C. Greene, Harvard Family Research Project, Harvard Graduate School of Education, 1997, III, 1. 5 problems of paradigm mixing, how to qualitatively analyze quantitative data, how to interpret conflicting results). Then, a word of caution seems appropriate to be addressed to researchers (and better to research teams) who plan to design inclusive, complementary methods that are capable to embrace diverse perspectives, data and values within and across studies.

SIMULATION

As I mentioned previously, mixed methods research has been labeled: the ‘third’ major research paradigm. Robert Axelrod, instead, classified simulation as the ‘third’ research methodology. Beyond issues of classification of methods depending on whether either mixed methods or simulation are compared with the two standard methods of induction and deduction, certainly simulation is a different way of conducting scientific research, a fairly "young" methodology for economics relatively to the traditional approach based on theorems and proofs and on econometric analysis. It finds its place mainly when adaptive rather than optimizing strategies - implied by the rational choice assumption - are involved.

For sure, rigorous analyses, based on the axioms of ‘objective’ rationality, have produced valuable insights, being the expressions of rational choice as a normative idea. However, the usual rational-choice assumption, built on a unitary concept of individual utility, appears at odds according to psychological research that explores the functioning of the human brain as a parallel system of elaboration that accommodates multiple utilities11.

Furthermore, the unique perspective of a rational individual acting at a point in time seems inappropriate to represent an adequate knowledge basis, as I had the opportunity to mention while dealing with sustainable transport infrastructure

11Parentthetically, by saying so, I clearly do not intend to drive the reader towards possible systematic misunderstandings about rationality, as has been often the case when economists were somehow responsible for having created them. Certainly, individuals remain subjectively rational even if we postulate a motive or a goal that they might pursue. As Castelfranchi efficaciously expressed: “the point is that even by adopting the rational decision framework as it is, we can postulate any kind of motive/goal we want or need in our agents: benevolence, group concern, altruism, and so on. This does not make them less rational, since rationality is defined subjectively! This might make them less efficient, less adaptive, less competitive, but not less subjectively rational“ [emphasis added]. See C. Castelfranchi, “Through the minds of the agents”, Journal of Artificial Societies and Social Simulation, 1998, vol. 1, no. 1 6 configurations and particularly (but not exclusively) when equity issues of groups of individuals forming a community are involved. As the reactions of citizens to the infrastructure investments projects or their implementation affect the decision to be taken by public authorities, I considered that such human reply might be in conflict with the axioms of perfect rationality. Moreover, single agent-based models cannot encapsulate properly the complexities of human behaviour, mainly for the known man’s ‘bounded rationality’ and inadequate cognitive and computational capabilities. I wish to remark here that even the expedient of a ‘representative or identical agent’, whose conduct is described by a tractable utility function, as assumed by neoclassical economic theories and policies, shows clearly its lack of realism and failure in representing a base for accurate predictions about human behaviour.

Additionally, it should be registered the uneasiness of researchers for the inability to fully estimate the informative content of data due to the limits of the techniques available for evaluation. Analytically, I believe that a point of impasse can be found where the breaking up of the economic model is bound to be stopped for the impossibility to be fully aware of the interdependences that the disaggregating process would have shown if carried ahead.

Then, the complexity of the socio-economic system, which presents situations highly interrelated and subject to uncertainty and external disturbances, has rendered the search of equilibrium solutions analytically complicated due – essentially - to non-linear relations Against this background a way-out was detected in the use of alternative ‘quasi-analytical’ models, to be applied especially when trying to explain the real economic behavior conditioned by the presence of the emotive component of human actions, in an effort to go beyond the assumptions of strictly 'personal interest' and 'substantive rationality'. In fact, in relation to the concept of rationality of homo economicus and to its basis of a maximizing and optimizing behavior that requires possession of complete and complex information, expensive to acquire and not completely utilizable for the human limits in is

7 elaboration, it has been put into evidence how ‘economizing’ agents choose even to remain ‘ignorant’ in respect to this kind of rationality and rather follow the proclivity to give greater weight to the emotive component of their actions. I’d add that if some social phenomena, which are typically assumed to occur through rational behavior, arise instead due to complex dynamics that are little influenced by conscious intent, then we have to allow for this in shaping the model design 12.

As a result, within the context of alternative models and in the perspective I have just mentioned, the simulation method appears as a proper research approach and technical instrument to be adopted. Actually, as Axelrod pointed out:

“ The main alternative to the assumption of rational choice is some form of adaptive behavior. The adaptation may be at the individual level through learning, or it may be at the population level through differential survival and reproduction of the more successful individuals. Either way, the consequences of adaptive processes are often very difficult to deduce when there are many interacting agents following rules that have nonlinear effects. Thus, simulation is often the only viable way to study populations of agents who are adaptive rather than fully rational” [emphasis added]13.

An organism of agents and the relationships between them represent what may be termed a multi agent-based modelling or multi-agent simulation. Its common definition refers to a method for studying systems exhibiting the following two properties: (a) the system is composed of interacting agents; and (b) it exhibits emergent properties, that is, characteristic attributes arising from the interactions of the agents that cannot be deduced simply by aggregating the properties of the agents themselves. In all, it constitutes an important instrument to symbolize complex behavioural patterns that provide precious information about the dynamics of the real world system it emulates 14. By way of an application, multi agent-based models have been used in our discipline to investigate – for instance - the effects of land-use planning constraints on populations. But, as the recurrent theme of my reflections tries to show, I advocate the concomitant participation of a plurality of discipline to perform

12 On this issue see C. Goldspink, “Modelling social systems as complex: Towards a social simulation meta-model”, Journal of Artificial Societies and Social Simulation, 2000, vol. 3, no. 2 13 R. Axelrod, 1997, op.cit. 14 The use of the term ‘complex’ is made in a technical sense to mean that the behaviour of the system as a whole cannot be determined by partitioning it and understanding the non-linear behaviour of each of the parts separately. 8 simulation of adaptive systems with many agents. Particularly, because of their complexity, deriving also from the operation of feedback inter-relations, I believe it necessary to perform a study of their structure and evolutionary development based on a ‘cultural’ dimension. The evolutionary perspective appears appropriate in an economic setting because – in relation to its characteristics - it is possible to analyze phenomena deeply entangled with cognitive, institutional, organizational and political dimensions, i.e. a world full of "informational fuzziness”. This efficacious expression used by Pyka and Ahrweiler (2004) fits well their shareable conception of evolutionary economics as it:

“focuses on the historicity or path-dependence of economic life: processes on the micro-level are characterised by their discontinuity (mutation), their direction and irreversibility (selection), their sometimes un-expected areas of stability (retention) and multiple non-linear interactions between heterogeneous actors, i.e. individuals or firms and other organisations, and between actors and the environment (adaptation). The features of micro level processes create emergent structures on the system level. These characteristics of evolutionary economic processes with high dynamics and uncertainty can be easily wedded to agent-based simulation techniques which are typically directed towards action/interaction and emergent structure allowing for multiple feedbacks in system creation to analyse and visualise complex processes “ 15. Concentrating on the investigation of the latter issue, I wish to consider in a more specific way what I summarize as the constituent elements of a cultural evolutionary process within a context of a simulation process. Briefly, I indicate those constitutive features by mentioning the initial trait of the evolutionary course which is marked by a state of competition, then by the establishment of a communication mechanism and the formation of stable interaction models; subsequently, models of behaviour are presented based on ‘replicators’ (memes) 16 commonly accepted and that promote cooperation, to arrive, finally, at the integration between memes as elements of coordination. The results that come out of the entire process clearly entail a problem of elaboration and sharing of information and a consequent communication issue.

15 A. Pyka, P. Ahrweiler, “Applied Evolutionary Economics and Social Simulation”, Journal of Artificial Societies and Social Simulation, 2004, vol. 7, no. 2 16 A meme may be defined as a ‘constituent brick’ of mind and culture as a gene is the ‘basic element’ of the biological life (See R. Dawkins, The selfish gene, 1976). 9 A bottom-up strategy of information processing is usually performed in connection with the operation of the simulation tool to understand and somehow ‘predict’ the evolution of such adaptive systems in the sense that possible scenarios are conceived instead of specific forecasts. Specifically, information about a lower level of the systems (individual agents) is acquired to formulate hypotheses about their behaviour and then a simulation is run to observe the emergence of system-level properties in relation to particular topics 17.

I notice, here, that a meta-system process is involved, shaped by the integration of a number of initially independent entities, such as individuals, and the emergence of a system controlling their interaction. Such a system is represented by culture as a form of social control. Clearly, a meta-system transition (in Turchin’s terminology 18) is involved consisting in the evolutionary integration of lower level systems, linked to the emergence of higher level ones that control them, thus explaining the formation of social systems. In other words, meta-system transition evolves from a selection process at the ‘individual’ level towards a ‘group’ selection one and such a transition is made possible by culture as a type of social control. Thus, a simulation process can be envisaged under the idea that social systems may be studied in a ‘hierarchical’ way, according to different levels of organization, which are attained following an evolutionary process that goes from lower to higher levels of complexity.

Then, as I had the occasion to indicate in a previous note 19, we are in the domain of cybernetics that tends to substitute a linear causation with a circular one and that had a crucial influence on the birth of simulation ‘science’, particularly when it tries to explain systems ‘organized in a composite mode’. According to the cybernetic approach, the system is modelled as a whole and its dynamics is expressed through levels having a teleological structure able to contrast any ‘up-settings’ through a

17 Quite recently, pattern-oriented attempts have been proposed to make bottom-up modelling more rigorous and inclusive. 18 See V. Turchin, The phenomenon of science. A cybernetic approach to human evolution, Cambridge University Press, 1977 and V. Turchin, A dialogue on metasystem transition, July 12, 1999. 19 This Journal, 2005, vol. XXXII, 2 10 control function. On the whole, due to this peculiar feature, I believe that the traditional cybernetic analysis, integrated by components of the evolution of complex systems, may contribute to the comprehension of a cultural evolutionary process. Given such a background, the role of agent-based simulation appears to be that of providing the proper tool to enrich our understanding of fundamental processes and to investigate self-organizing systems and emergent phenomena, even beyond the apparently independent activities of individual agents.

Even though the agent-based simulation approach offers the advantage of extending the ability of mathematical modelling or statistical analysis and is progressively recognized by economists as a tool that meets the complexity view of social systems, I take from my experience that it has still a restricted part in the research contributions submitted to this Journal, obscuring in some way the potentialities of the instrument in gaining insights on the interactions and structural presuppositions that may be good candidates for explaining observed patterns of behaviour.

I propose an explicatory hypothesis for this phenomenon: probably a ‘psychological’ resistance is at the base of the limited use of such simulation models, even in an era of extremely vast calculation capabilities that are now available. Such an attitude that – as I understand – has been a source of difficulties in gaining an adequate place also in other related journals 20, may be ascribed to the argument that:

“Traditional analytical modelling practices rely on very well established, although implicit, methodological standards, both with respect to the way the models are presented and to the kind of analyses that are performed. These standards are useful because (1) they contribute to the creation of a common language among scientists, (2) they can be referred to without detailed discussion, (3) they force model homogeneity and hence comparability, (4) they increase methodological awareness and guide individual scientists towards better quality research. Unfortunately, computer-simulated models often lack such a reference to accepted methodological standards [emphasis added]” 21.

20 See M. Fontana, “Simulation in Economics: Evidence on Diffusion and Communication”, Journal of Artificial Societies and Social Simulation, 2006 vol. 9, no. 2 21 M. Richiardi, R. Leombruni, N. Saam, M. Sonnessa, “ A Common Protocol for Agent-Based Social Simulation”, Journal of Artificial Societies and Social Simulation, 2006, vol. 9, no. 1 11 Likewise, problems of different nature still confront the implementation of the simulation tool; I shall refer - as an example - to some of them such as: a) how to reduce the variance of an output variable without disturbing its expectation.or b) how to deal with the difficulties associated with the validation and verification of composite applications; c) the way of selecting input probability distribution and d) how to generate ‘variates’ according to these distribution, that is random variables with numerical value defined on a given sample space. Additional complex conditions of non-stationarity and autocorrelation may be present and obstacles may show up linked to the replicability of results that jeopardize the soundness of the causal links proposed.

Notwithstanding, the identification of persisting methodological drawbacks and the difficulties in formulating interpretations and generalizations still encountered, I envisage that simulation may become a flourishing area of research and a possible source of more frequent inspiration for research designs in analysing the intricacies of the social interactions and their relevant emergent patterns within the domain of transport economics investigations. They will be taken as well into careful consideration upon submission to the Journal.