ELUCIDATING STAKEHOLDER PERSPECTIVES WITH REGARDS TO ASSESSING AGENT-BASED MODEL OUTPUTS: METHODOLOGIES & RESULTS
Deliverable no. D2.4.1
Marian Raley, Guy Garrod; Myriam Gomez, Olivier Barreteau, Nadine Turpin, Olivier Aznar; Diana Kopeva; Franziska Schaft, Omar Baqueiro Espinosa; Marta Konecná, Ladislav Jelinek, Milos Delin; Branka Šakic Bobic, Mario Njavro
Partners: UNEW, Cemagref, UNWE, IAMO,UZEI, AFSK
Submission date: 20-Jun-11
Seventh Framework Programme Theme 6 (ENV-2007-1)
Environment (including climate change)
Collaborative project (Small or medium-scale focused research project) Grant agreement no. : 212345 Project duration: November 2008 - - November 2011 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
PRIMA aims to develop a method for scaling down the analysis of policy im- pacts on multifunctional land uses and on the economic activities. The scoped policies will include the cohesion policy (ERDF, ESF, CF), the enlargement process (IPA) & the rural development policy (EAFRD) of the European Commission, with a special focus on agriculture, forestry, tourism, and eco- system services. The approach will: rely on micro-simulation and multi- agents models, designed and validated at municipality level, using input from stakeholders; address the structural evolution of the populations (appear- ance, disappearance and change of agents) depending on the local conditions for applying the structural policies on a set of municipality case studies. In- volving eleven partners, the project is coordinated by Cemagref.
Email: [email protected] & [email protected] Internet: https://prima.cemagref.fr
Authors of this report and contact details
Name: Marian Raley Partner acronym: UNEW Address: Centre for Rural Economy, School of Agriculture Food and Rural Development, University of Newcastle upon Tyne, Newcastle upon Tyne, NE1 7RU, UK E-mail: [email protected]
Name: Nadine Turpin Partner acronym: Cemagref Address: 24 avenue des Landais, BP 50085 F-63172 AUBIERE Cédex, France E-mail: [email protected]
Name: Diana Kopeva Partner acronym: UNWE Address: Studentski Grad "Hristo Botev" 1700, Sofia, BULGARIA E-mail: [email protected]
Name: Franziska Schaft Partner acronym: IAMO Address: Theodor-Lieser Str. 2, 06120 Halle (Saale), Germany E-mail: [email protected]
Name: Ladislav Jelinek Partner acronym: UZEI Address: 75, Mánesova. 120 58, Prague, CZECH REPUBLIC E-mail: [email protected]
Name: Mario Njavro Partner acronym: AFSZ Address: University of Zagreb Faculty of Agriculture, Department of Agricultural Economics and Rural Sociology. Svetosimunska c. 25. Zagreb, Croatia. E-mail: [email protected]
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 2/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
If you want to cite a Public Deliverable that originally was meant for use within the project only, please make sure you are allowed to disseminate or cite this report. If so, please cite as follows:
Marian Raley, Guy Garrod; Myriam Gomez, Olivier Barreteau, Nadine Turpin, Olivier Aznar; Diana Kopeva; Franziska Schaft, Omar Baqueiro Espinosa; Marta Konecná, Ladislav Jelinek, Milos Delin; Branca Šakic Bobic, Mario Njavro. Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies and results PD no. D2.4.1 PRIMA collaborative project, EU 7th Framework Programme, contract no. 212345, https://prima.cemagref.fr, 57 p.
DISCLAIMER
“This publication has been funded under the PRIMA collaborative pro- ject, EU 7th Framework Programme, Theme 6 (ENV 2007-1) Environment (including climate change) European Commission, DG Research, contract no. 212345. Its content does not represent the official position of the European Commission and is entirely under the responsibility of the authors.”
“The information in this document is provided as is and no guarantee or war- ranty is given that the information is fit for any particular purpose. The user thereof uses the information at its sole risk and liability.”
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 3/57
Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
TABLE OF CONTENTS
Executive summary ...... 9
1 Introduction ...... 11
2 Outline of type 1 workshops (Germany, France and the UK) ...... 11
2.1 Workshop objectives ...... 11
2.2 Method ...... 12
2.3 The UK Workshop ...... 12
2.4 The French workshop ...... 13
2.5 The German Workshop ...... 14
3 Findings of type 1 workshops ...... 15
3.1 Plausibility of the conceptual model...... 15
3.2 Examination of output values ...... 18
3.3 Scenarios ...... 26
3.4 Germany scenario discussion ...... 31
4 Discussion of the Type 1 workshops ...... 34
4.1 Stakeholder selection ...... 34
4.2 Conducting the workshop ...... 34
4.3 Level of detail ...... 35
4.4 Model relevance ...... 35
4.5 Conclusions, Type 1 workshops ...... 36
5 Outline of type 2 workshops (Croatia, Czech republic, Bulgaria) .... 38
5.1 Introduction to Type 2 workshops ...... 38
5.2 The generic method ...... 38
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 5/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
5.3 Outline workshop programme ...... 39
6 Conduct & findings of type 2 workshops...... 40
6.1 Organisation of the Workshop in Croatia ...... 40
6.2 Organisation of the Czech Republic workshop ...... 40
6.3 Organisation of the Bulgarian workshop ...... 41
6.4 Discussion of the non-model workshops ...... 43
6.5 Conclusions for Type 2 workshops...... 47
7 Discussion and conclusions ...... 48
7.1 Main outcomes of stakeholder on-model engagement in PRIMA in the field of participatory modelling ...... 48
7.2 Methodological suggestions from stakeholder on-model engagement in PRIMA ...... 49
7.3 Model Brokering: a set of activities to be made explicit ...... 51
7.4 Scenario development as a key tool for stakeholder involvement ...... 51
7.5 Conclusion ...... 53
References ...... 55
Glossary ...... 57
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 6/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
LIST OF TABLES
Tab. 2.1 | The UK Workshop: participants ...... 12 Tab. 2.2 | The French Workshop: participants ...... 13 Tab. 2.3 | The German workshop: participants ...... 14 Tab. 3.1 | Unemployed individuals by age group and type of job (SPC) looked for, Hohenberg-Krusemark July 2011...... 22 Tab. 3.2 | Employed individuals by age group and type of job (SPC) looked for, Hohenberg-Krusemark ...... 23 Tab. 3.3 | Role of self-employment in Hohenberg-Krusemark and/or district for 2011 ...... 26 Tab. 3.4 | French workshop: suggested outline scenarios ...... 27 Tab. 3.5 | UK workshop: linkage of stakeholder narrative to model parame- ters ...... 30 Tab. 3.6 | German workshop: Future trends ...... 32 Tab. 6.1 | The Croatian Workshop: organisation and participants ...... 40 Tab. 6.2 | The Czech Republic Workshop: organisation and participants ...... 41 Tab. 6.3 | The Bulgarian Workshop: organisation and participants ...... 41 Tab. 6.4 | Currently, what policies show a good appreciation of the local geo- graphical level? ...... 43 Tab. 6.5 | Are the model’s components appropriate: databases and driving forces? ...... 44 Tab. 6.6 | Model plausibility: does the model accurately represent reality? ..45 Tab. 6.7 | Is the model accessible? Is it easy to understand?...... 45 Tab. 6.8 | Is the model acceptable to policymakers? ...... 46 Tab. 6.9 | If available, how would the PRIMA model be used? ...... 46 Tab. 6.10 | Who would you trust to deliver the model? ...... 47
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 7/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
GENERAL INFORMATION
Task(s) code(s): D2.4.1
Input from (Task codes): T2.3.1, T2.3.2
Output to (Task codes): T3.6
Related milestones: M2.2.3, M2.3.1, M2.3.2
CONTACTS
Marian Raley UNEW Centre for Rural Economy, School of Agriculture, Food and Rural Develop- ment, Agriculture Building, University of Newcastle upon Tyne, Newcastle upon Tyne, NE1 7RU, United Kingdom. [email protected]
...
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 8/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
EXECUTIVE SUMMARY
Towards the end of the project, a single workshop was conducted in each case study area involving local stakeholders including residents, elected represen- tatives and policy officers. Due to the agent-based model’s incomplete state, and uncertainty over the timing of its eventual delivery, the options for the workshop content were heavily compromised. In the event, two types of workshop were conducted:
Type 1 workshop (France, Germany and UK) was centred round technical development of an early version of the micro-simulation model adapted for each case study area. It focused on discussion of assumptions and data, as well as scenarios which it might be possible to explore.
Type 2 workshop (Bulgaria, Croatia, and Czech Republic) was conducted in case study areas for which an adapted model was not available. This focused on improving understanding of the needs of model users.
Considerable effort was made to promote stakeholder learning about model- ling, which appeared to generate feedbacks which created pathways for model revision. On the whole, participants recognised models as simplifications of reality. They found explanations about the model (components, driving forces, dynamics) comprehensible, and reacted fairly favourably to the idea of having such a decision-support tool available. A tension exists between the need to obtain locally-specific insights to support adaption of the model to the case study area and yet without generating from stakeholders a level of detail that the model cannot accommodate, or which can’t be supported by data. The model concept was considered to be broadly plausible. However, the lack of good data at local scales in most case study areas was seen as a severe limitation. Due to the heterogeneity of each area, the alternative of using data derived from larger geographical units was unsatisfactory.
The Type 2 workshop explored policymakers’ reactions to models. While being receptive to a decision-support tool which could improve the impact of policy locally, the current problem was not so much that the policy goals were unsympathetic to their area, but that their impact was low because the rules and structures for their implementation were poorly designed. Policy-makers would be able to understand such a model although a model of greater com- plexity might be avoided. However in Croatia and Bulgaria there was mistrust about any possible future model outputs due to the lack of low level input data. For pragmatic reasons (availability of datasets and local knowledge) it was preferred that the model should be administered in the region.
PRIMA’s workshops identified some possible methodological advances in the field of participatory modelling. By encouraging stakeholders to handle the model, the development of scenarios proved to be a useful tool to deepen
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 9/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
stakeholders’ understanding of the model and to challenge it, despite the model being apparently far in advance of their technical skills.
We propose that an explicit intervention by a model broker, who is able to interpret stakeholders’ responses and communicate them in modellers’ lan- guage (and vice versa), would improve the various interpretations of partici- pants and the usefulness of these outcomes in model development. This will be further enhanced if a specific stage within the workshop is dedicated to interpretation.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 10/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
1 INTRODUCTION
This deliverable focuses on Step 3 of WP2’s objectives, namely
‘On-model engagement with stakeholders mirroring agent-based models’ (DoW, page 34).
Critically this engagement, in the form of a workshop, aims to provide infor- mation to assist the development of area-specific models which have been adapted from the generic micro-simulation model. However, by the time the workshops were finally conducted in the last 6 months of the project, it had become clear that an adapted model could not be produced for Croatia or Bulgaria due to a lack of suitable data, and that the Czech Republic’s would not be ready in time.
Consequently two different types of workshop were conducted:
Type 1 workshop: supporting model development was conducted in Ger- many, France and the UK and is described in Sections 2 to 4. This had the benefit of having an early version of the adapted model available.
Type 2 workshop: stakeholder reactions to agent-based models was con- ducted in Croatia, Czech Republic and Bulgaria and is described in Sections 5 to 6.
Part 7 concludes with a discussion concerning the linking of modellers’ activi- ties and stakeholders.
2 OUTLINE OF TYPE 1 WORKSHOPS (GERMANY, FRANCE AND THE UK)
2.1 Workshop objectives
The workshop objectives were derived from D3.3. They are:
1. To explain the context of the study and the structure and dynamics of the simulation model so that participants could understand and contribute to the subsequent discussion.
2. To gain feedback from participants on the plausibility of the concep- tual model.
3. Following their critical examination, to gain feedback from partici- pants on the plausibility of some preliminary model outputs
4. To elaborate scenarios of interest to participants to check the model’s relevance.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 11/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
2.2 Method
Workshop participants were selected from the population of stakeholders who were identified directly or indirectly during earlier interviews. Essen- tially participants all had considerable knowledge of the case study area, and had a stake in the area on account of living there and/or holding a represen- tative role. Some would be potential model users due to their involvement in policy implementation. Tables 2.1 - 2.3 list the participants. Most stakeholder participants are involved in local policy making.
The setting for each workshop was a neutral venue such as a hotel or village- hall. Proceedings were recorded using audio equipment and also by a notetaker. The duration of the workshops was from 2 ¾ to 3 ¾ hours. Prepa- rati0ns for each workshop was made by WP2 staff in co-operation with their asssociated model developer(s) from WP3, who prepared a detailed technical presentation and preliminary model outputs for use at the event, as well as a basic introduction explaining what a model is. At least one member of WP3 staff attended each Type 1 workshop to give a technical presentation and participate in the dialogue.
After a general introduction to the PRIMA project and the objectives of the workshop, a detailed scientific presentation of the PRIMA micro-simulation model was made by WP3 colleagues using material derived from D3.3 and D3.4. This was meant as an initiation to modelling as well as a presentation of the main structural and dynamic components of the model, and any past scenarios which has been used so far. In one case (France), the initiation to modelling and main model assumptions was performed by WP2 staff, while simulation outcomes and data issues were handled by a WP3 colleague. The model was presented as a machine to produce stories.
Following the technical presentation, a series of facilitated discussions of key topics took place. The content varied between the 3 workshops depending on the state of model development and the information wanted from the interac- tion.
2.3 The UK Workshop
As described in D2.2, the UK case study area consists of the two adjacent wards (LAU2 areas) of Bradwell and Hope. These lie in the Peak District Na- tional Park. An overview of the UK workshop is shown in Table 2.1.
Tab. 2.1 | The UK Workshop: participants
Location Bakewell, Derbyshire Date May 2011 Duration 2 hours 45 minutes Stakeholder participants:
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 12/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
1. Local resident, Bradwell (Parish Council; formerly parish member of PDNPA) 2. Local resident, Hope (Parish Council) 3. Officer, CAP Pillar 2,LEADER Policy implementation 4. Local authority (LAU1), Economic development officer 5. Local authority (LAU1), Economic development officer; formerly rural policy development
6. Officer, Peak District National Park Authority (PDNPA) Project staff participants Moderator (Marian Raley, UNEW) Note-taker (UNEW) WP3 technical expert (Omar Baqueiro Espinosa)
2.4 The French workshop
The French case study area consists of the Condat cluster of municipalities. The Condat cluster of municipalities is located in the Pays de St-Flour Haute- Auvergne, and included in the Regional Natural Park of the Volcans d'Au- vergne. The Regional Park promotes sustainable development strategies for the municipalities and behaves as a governance and facilitator structure (mi- cro-institution), not as a subsidy supplier. Table 2.2 provides an overview of the French workshop.
Tab. 2.2 | The French Workshop: participants
Location Condat, department of Cantal Date April 2011 Duration 3 hours Stakeholder participants: 1 to 7. Local authority (LAU2), mayors of Chanterelle, Condat, Lugarde, Mar- cenat, Montboudif, Montgreleix and St-Bonnet-de-Condat 8. Policy official, director of Chambre d'Agriculture du Cantal with collabora- tor 9. Policy official, director of land development in Conseil Général du Cantal with collaborator 10. Local official, coordinator of Pays de St-Flour Haute-Auvergne 11. Policy official, head of division FEADER – Développement Rural in DRAAF Auvergne
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 13/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
12. Local official, head of division Agriculture et Pastoralisme in Parc Natu- rel Régional des Volcans d'Auvergne 13. Local official, development officer and president of Communauté de communes du Cézallier 14. Policy official, service assistant Connaissance, Aménagement et Dévelop- pement and head of division Développement des Territoires in DDT du Can- tal Project staff participants Moderator (Olivier Aznar) Project presentation (Nadine Turpin) Note taker and logistic support (Myriam Gomes) Presentation of modelling and main model assumptions (Olivier Barreteau) WP3 technical expert (Sylvie Huet) WP3 PhD (Maxime Lenormand) WP3 technical expert (Guillaume Deffuant) External view (Dominique Vollet)
2.5 The German Workshop
The German case study area consists of the municipalities (LAU 2) Hohen- berg-Krusemark, Osterburg, Gardelegen, Arneburg and Stendal City. Apart from Gardelegen, all other municipalities are located in Stendal district. The Network consists of rather heterogeneous settlement sizes. Hohenberg- Krusemark is the smallest municipality with 637 inhabitants in 2006, fol- lowed by Arneburg with 1,674 inhabitants. Stendal with more than 36,000 inhabitants is the district capital and provides supraregional service functions and employment opportunities, not only to the network municipalities but for the whole Stendal district.
It was decided that the simulation runs presented to the stakeholders during the workshop should focus on results for Hohenberg-Krusemark municipality rather than on results for the whole municipal network. This decision was derived from the findings of the UK workshop, conducted in May 2011, prior to the Altmark workshop. UK Workshop stakeholders noted that because the included municipalities are very heterogeneous, grouping the simulation results at network level would be too aggregated and therefore would not sufficiently reflect the different features of included villages.
Tab. 2.3 | The German workshop: participants
Location Hohenberg-Krusemark (Stendal district) Date 16th September 2011 Duration 3 hours, 54 minutes
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 14/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Stakeholder participants:
1. Mayor Hohenberg-Krusemark, Member of Parlament Saxony-Anhalt
2. Mayor of the association of municipalities Arneburg-Goldbeck
3. Head of Regional Planning Agency Altmark
4. Officer Regional management Initiative Altmark
5. Head of the Agency for Agriculture, Land consolidation and Forestry Altmark
6. Officer Employment Agency Stendal (Agentur für Arbeit Stendal)
Project staff participants
Moderator (Franziska Schaft)
Translator for WP3 Expert (Amanda Sahrbacher)
Note taker (Arlette Ostermeyer)
PhD (IlkayUnay Gailhard)
WP3 technical expert (Omar Baqueiro Espinosa)
3 FINDINGS OF TYPE 1 WORKSHOPS
3.1 Plausibility of the conceptual model
This activity aimed at answering three key questions: ‘Does the model provide a reasonable representation of the socio-economic system of the case-study area?’ ‘Are its underlying theories and assumptions correct?’ ‘Is it able to capture the main potential evolutions of the case study area?’
The technical presentation and associated discussions systematically exam- ined each sub-system of the model, and followed (traced) the behaviour of each entity to determine if the model’s logic is correct (Sargent, 2009). This was a time-consuming process as many explanations were demanded of whether, and how, various elements and processes were included and opera- tionalised in the model. Differences were drawn between dynamics in that particular locality and others. Overall stakeholders agreed that most compo- nents and dynamics of the model appeared plausible, and that the model
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 15/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
appeared to be a reasonable though simplified representation of reality, and of the main driving forces/issues in the surveyed rural areas. However, some deficiencies were identified. i. Access to work and services: definition of the ward network (UK)
The ward network consists of other areas which are important for commuting into or away from the case study area. Stakeholders criticised the ward net- work as being restricted to wards in the wider study area of Nottinghamshire and Derbyshire. Consequently significant urban areas lying outside the boundary were lumped together with the rest of the UK into a single ‘outside’. This results in a loss of detail in important commuter flows, the most signifi- cant of which were reported as (relatively low paid) public sector workers into the Bradwell network, and out-commuting of well-paid professionals to Shef- field and Manchester.
An additional finding was that, if Sheffield and Manchester were included, the ward network, defined by commuting, included the most important ser- vice centres for the case study. Moreover, access to services was not perceived as a strong driver of residential decisions since, so long as individuals had access to a car, all necessary services are available in the network. It is access to housing and a job which are the critical determinants of whether people can live in the area. ii. Income and SPC (FR, UK, DE)
In the UK, the absence of income as a model parameter was seen as an im- portant deficit as it is salary that influences decisions to change job and will- ingness to commute. The use of socio-professional class (SPC) as a proxy for income was considered by stakeholders expeditious in that accurate income data at ward-level is unobtainable. However, the implied assumption that job change is motivated by improvement of socio-professional class (SPC) was felt untenable. Several examples were given of low skilled work which yielded a better salary than higher SPC occupations, illustrating that SPC and income do not have a clear direct relationship.
In France, the SPC proxy was more widely accepted but there was criticism that the model did not explicitly include the economy. The discussion ex- tended to agricultural land use, and the inability of the model to capture the differing levels of income from different arable cropping and breeding types (dairy, suckling, fattening, intensive or not) were noted.
The model determines the SPC of new entrants to the labour market by their parents’ SPC. Stakeholders, however, were not convinced of the over-riding influence of parental status and considered gender was an important deter- minant of SPC.
In Germany, stakeholders recommended correlating the type of job (SPC) being sought by unemployed individuals in the age group 15<25 or job new entrants to the labour market with their educational level.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 16/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
iii. Housing as a critical driver (FR, UK, DE)
French workshop participants recognised that the housing market is a major driver in the model but believed that in reality it is more nuanced than the model suggests. Although in theory there are enough houses, some may re- main unoccupied (e.g. due to the unemployment of the occupant or because they are below modern standards) and some are used as holiday homes rather than principal residences. The PRIMA modelling team explained that some adjustments can be made to reflect such situations. For example it is possible to have an unbalanced relationship between housing supply and demand, which would encompass unoccupied properties.
In the UK there are the severe restrictions on built development in the Peak District National Park resulting in a near-fixed housing stock, whereas the model dynamics allow for increases in stock (‘residences becoming created’). The resulting high prices and lack of affordable (below market price) housing were seen as absolutely critical in decisions about residential decision, and drove a trend towards an ageing and professional resident population. People with modest incomes, notably young people and families with children, were increasingly unable to afford to live in the area. This constraint operates in all the National Parks of England and Wales.
In Germany it was emphasized that the ownership structure in the housing market is an important feature which should be included in the model. Most of the houses in Hohenberg-Krusemark and surrounding villages are owned by their residents, while the rental structures seem to be of minor impor- tance. This ownership structure on the housing market has, according to the stakeholders, a decisive impact on the people’s willingness to commute. Since the prices on the regional housing market are comparably low and houses cannot be sold easily without making losses, people tend to stay in the village whilst accepting high commuting distances. iv. Technological changes (FR)
The effect of technological changes as well as impact of new infrastructures was discussed in France. Stakeholders insist that technological change im- pacts on farm productivity. Improved tourist infrastructure can drive restora- tion of old houses for rental to tourists, and modernization of enterprises by creating new activity zones can modify the employment demand. However no such big changes occurred in the area during the past years, and none are in these plans either. Participants missed a representation of local specificities on impact of public policies which were fed into the model on a regional ba- sis. v. The role of the second labour market (DE)
The role of job creation measures initiated by the state was brought up in Germany. Stakeholders emphasized that especially at the beginning of 2000 and before, the impact of job creation measures (= jobs, initiated and subven- tioned by the state) had a strong impact on the employment situation in the pilot municipality. Many jobs have been created through these measures.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 17/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Stakeholders indicated that the second labour market has different mecha- nisms and dynamics from the first labour market (specifically time-limited contracts, state driven). Stakeholders recommend that this situation has to be taken into account by the model.
3.2 Examination of output values
This activity explored whether the output values produced by the model were plausible. Unfortunately only preliminary outputs were available in the UK and Germany
3.2.1 UK
Prior to the workshop, a simulation was run for the period 2001 to 2007. The base year, 2001, is the date of the last published Census, which provided a comprehensive database for use in model building. Co-efficients derived from trends for 2000 to 2001 (taken mainly from the Labour Force Survey) were used to drive a year-by-year simulation, stopping at 2007 so as to pre-date the 2008 shock to the economic system.
A discussion was motivated by a comparison of simulated values for 2007 with ‘real’ values. The unit of comparison was the Bradwell/Hope ward net- work. The objectives were: i. to check whether the assumption that annual rates of change for 2001 to 2007 are the same as those between 2000 to 2001 is realistic. ii. to check the appropriateness of the data used for comparison purposes. iii. to comment on the plausibility of the trends identified in the real and simulated data. i. Are rates of change constant throughout 2001 to 2007?
It would be expected that model and real values would be dissimilar if large changes in real values had occurred, as a result of big events (step changes) affecting the case study area. However, for the period 2001 to 2007 no such big shifts were identified by stakeholders.
However, two more gradual but nevertheless important changes were identi- fied. First, due to productivity increases at the biggest employers, there had been a reduction in total local jobs, and it was postulated that the model would overestimate the stock of jobs in the area. Second, there had been large increases in house prices such that owner characteristics of houses bought and sold during this period would probably have changed.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 18/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
ii. Data appropriateness
Stakeholder deliberations highlighted the difficulties of modelling the socio- economic sphere at a local level which arise from a lack of empirical data. While a large number of variables are available at LAU2 level for 2001 (drawn mainly from the Census), the comparative ‘real’ data for 2007 had to be drawn from other sources. In some cases they were scaled down from a higher spatial scale, and stakeholders believed that trends at LAU2 level would be different from those at the higher spatial scale, for example due to urban-rural differences. However, these are the best available. This situation can be remedied when 2011 Census data become available in late 2012.
The second issue arose from the use of aggregated data for the ward network for comparison and, therefore, the combining of rural and ‘more urban’ areas. Stakeholders expected that some trends in Bradwell/Hope would be different from those of the wider ward network but would not be clearly shown in the data. The local insights which stakeholders provided would consequently be of limited use in considering the plausibility of the data. iii. Subjective comparisons of real and simulated values
Stakeholders were shown selected graphs of real and simulated data for 2001 to 2007 relating to population age. On inspection, the two series appeared to be fairly similar. A series of questions were discussed and illustrate three areas where stakeholder inputs are helpful. a. Explaining trends shown in real and simulation data, as supporting evidence of their plausibility.
Both real and simulated data show a diminishing number of 30 to 44 year olds and increasing numbers of 45 to 49 year olds and 60 to 64 years olds increased. These trends were considered to be plausible, and could be ex- plained by family life cycle, such as the ability to in-migrate to the area once children no longer needed schooling and the out-migration of 30 to 44 year olds due to the need to find family-sized housing. However there was surprise that the decrease in 5 to 9 year olds wasn’t greater as local schools were ex- periencing falling rolls. This might be explained by the camouflaging of se- vere local problems by the use of aggregated ward data. Stakeholders sug- gested that the over-estimation of 20 to 24 year olds may be because the model underestimates the outmigration of young people who seek work, housing and new experiences. b. Commenting on the suitability of imputed ‘real’ data
Ward level Census data show the number of retired people in 2001. A compa- rable variable is not available for 2007, so ‘retirement population’ was im- puted from age statistics. Stakeholders compared two simulations: a. retire- ment population equals all people aged 65+; and b. retirement population equals all people aged 60+. A better fit was observed if people aged 60+ were included than when all over 64s were shown. Participants thought the sup-
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 19/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
porting assumption that many men retire (early) at 60 was reasonable. In part, these would be retirement in-migrants. c. Using stakeholder judgement to overcome the problem of missing data
Although commuting flows in aggregate are known, necessary data relating to the employment characteristics of commuters were not available from pub- lished sources. Stakeholder information would help derive coefficients to describe commuters. For example, modestly paid public sector workers such as teachers and health workers commuted into the area because they couldn’t afford to live there. Professional commuters could be commuting either into the area or out of it, particularly to the large urban areas.
3.2.2 French workshop and calibration
The French workshop followed a different procedure which relied more on a technical presentation and theoretical discussion rather than data compari- sons, because the French model relies on some parameters that are not data driven. It presented to participants the path followed to produce an output value (indicator) for comparison with real data. This entailed a detailed ex- amination of the model processes, the parameters controlling them and the assumptions invoked with participants being invited to discuss their plausi- bility.
The discussion was motivated by two points, the presentation and explana- tion of those parameters that are not determined by real data, and some comparisons of modelled versus monitored data on population at the munici- pality level. The key here is that good fitness of modelled versus measured data on population can be obtained with different sets of calibrated parame- ters, which was a difficult point to grasp for the stakeholders. i. What changes in the rate of change during the simulation period?
During the modelled period, no large modification of real values occurred, no threshold was passed, stakeholders and modellers agree on smooth variations on the case study area. The gradual changes that were identified include a productivity increase both in industry and farming, which decrease the total local jobs (as for the UK case study). Second, there has been a modification of the housing acceptable standards, making difficult for some owners to rent out old houses. The third smooth change is a bit tricky: the modellers use the same birth ratio before and after the year 2000, and based it on a national value. Stakeholders did not notice a difference in the birth ratio from their area and the national value; they just insist that this ratio evolves along years. ii. data and parameters appropriateness
A large number of variables are available at LAU2 level in France, but the model relies on non measured parameters too. These include the probability of looking for a spouse, the number of time people seek for a partner, the acceptable difference in age between wife and husband. As for UK, the stake- holders' deliberations highlight the difficulty of modelling the socio-economic
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 20/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
sphere at this level of details. First, some stakeholders stressed that it may be extremely difficult to expand the results from the cluster of municipalities to the whole Auvergne, because of local specificities, and because some regional features may not be present in the cluster. Second, some stakeholders came back on the lack of economic information in the model, related to the high level of details about the demographic part. iii. subjective comparison of real and simulated values
Stakeholders were presented selected graphs of real and simulated data for the 1999-2006 period. Some of the series appear to be very similar, some others did not match much, and the stakeholders were invited to help the modellers to get clues why the differences occur. a. explaining trends
For the number of people living in each municipality, the measured and mod- elled series are very similar. Indeed, some misinterpretation could have oc- curred because the variation between municipalities is higher on the graph than between the modelled and series, but this was not a difficulty for the stakeholders. This first step ensured that the graphs are comprehensive.
Both real and simulated series depict accurately, according to the stake- holders, the age distribution evolution, for people older than 25 years. Dis- crepancies are noted in the evolution of the number of children number. The number and composition of the families evolves in measured data and not exactly the same way in the model outputs, but the stakeholders could grasp easily that they were presented only one simulation among many existing ones. At this stage arose the question why the modellers need to model the evolution of data they could just pick in statistics, and that was a good oppor- tunity for us to introduce the drivers of evolution. b. suitability of input data
Using a fixed birth ratio obviously introduced a discrepancy between mod- elled and measured number of people by age class, because the model could not capture the increase in birth ratio after 2000. The stakeholders did not focus on this point that was important for the modellers, but rather on the economic drivers. The net migration, which is included in the model as as- sumptions so far, was found as a key driver for the stakeholders, who deal daily with the attractiveness of their area. c. overcoming the problem of missing data and parameters
The missing parameters are currently calibrated in the French model. The assumptions of the probability of choosing a SPC, on the location of the work- ing place, and on the various parameters depicting the type of family (see above) did not raise major comments from the stakeholders. Stakeholders suggested that individuals and single large businesses could have a very strong influence on model outputs which might make it unstable.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 21/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
3.2.3 Germany
The German workshop added an additional component to the stakeholder interaction. Parallel to the French and UK set-up, the model was introduced and discussed with the stakeholders in the beginning. The additional second part consisted of a “stakeholder interaction”, where stakeholders were invited to validate selected probability tables and to add missing data according to their local expertise. A third part focused on the discussion and plausibility- testing of the scenario runs. The results are summarized below: i. What changes the rate of change during the simulation period?
Probability tables in the German model refer in most cases to data sets from the Labour Force Survey (national level, 2002). The probability functions do not change over the yearly simulation runs. Stakeholders remarked that probability functions should not be static but include more actual reference data over the simulation runs in order to incorporate changes occurring in the reality: Stakeholders indicated that after 2002 the labour market condi- tions in Hohenberg-Krusemark have changed considerably. Job creation measures have been cut back; in addition two new factories opened and gen- erated some job opportunities in SPC 7 (= crafts & trade). As stakeholders assume that these events affected the job search mechanisms of individuals, it was recommended to use more actual and localized data for the “looking for a job probability tables” in order to reflect realistic local labour market condi- tions. ii. Data and parameter appropriateness
Two probability tables were presented to the stakeholders for validation. The discussion was motivated by the question: “Do these values realistically pic- ture the real job seeking activities of unemployed/employed inhabitants of Hohenberg-Krusemark?” Stakeholders indicated that the dataset which was derived from national level did not fully reflect the local situation for Hohen- berg-Krusemark and modified both tables accordingly (See Tab. 3.1 and 3.2).
Tab. 3.1 | Unemployed individuals by age group and type of job (SPC) looked for, Hohenberg-Krusemark July 2011 (Stakeholder modifications in bold)
Socio-professional class of target jobs for an unemployed individual (%) Age SPC 1 SPC 2 SPC 3 SPC 4 SPC 5 SPC 6 SPC 7 SPC 8
15 < 1% 3% 13% 9 % 21 % 5 % 33 % 15 % 25 less 11% 1-2% more
25 < 3% 6% 14% 9 % 15 % 3 % 31 % 19 % 45 less 11% more
45 < 2% 7% 15% 8 % 14 % 4 % 24 % 25 % 55 less 11% 25%
55 < 1% 11% 15% 7 % 12 % 5 % 19 % 30 % 65 less 5% 20%
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 22/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Tab. 3.2 | Employed individuals by age group and type of job (SPC) looked for, Hohenberg-Krusemark (Stakeholder modifications in bold)
Socio-professional class of target jobs for an employed individual (%)
AGE SPC 1 SPC 2 SPC 3 SPC 4 SPC 5 SPC 6 SPC 7 SPC 8 Total*
15 < 3% 3% 2% 3% 4% 4% 5% 7% 31% 25 more -2% ok
25 < 3% 3% 2% 3% 3% 2% 6% 6% 28% 45 more -2% 30%
45 < 1% 1% 1% 2% 3% 3% 4% 5% 20% 55 more -2% ok
55 < 0,2% 0,3% 3% 0,5% 2% 1% 1% 2% 10% 65 more more less
* Since not every employed person is searching for a new job, the totals do not sum to 100%
The line of arguments for the proposed modifications can be summarized as follows: a. The type of job (SPC) that an unemployed individual will look for
It was recommended to include the level of education as an additional parameter for the age group 15< 25.
According to the stakeholders the presented data table underestimated the share of unemployed looking for a job in SPC 4 (= clerks), since public administration was assessed as a major employer in the region. Therefore it was recommended to increase the percentages for the first three age groups. Since the public service sector tends to employ a preferably younger workforce, stakeholders assed it as more difficult for unemployed (55<65 years) to find an occupation in this sector. In order to reflect realistic labour market conditions in the model, stake- holders therefore recommended to decrease the amount of unem- ployed looking for a job in SPC 4 in the age group 55<65.
Stakeholders recommended to increase the percentages for SPC 7 (= crafts & trade) to take the effects of some recent industrial allocations (a paper mill, pulp factory) nearby Hohenberg-Krusemark into consid- eration.
Stakeholders emphasized that highly qualified jobs opportunities (e.g. engineering) are rare and that therefore the values in SPC 2 (= profes- sionals) should be further reduced.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 23/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Finally it was recommended to further reduce the number of young employees (15<25 years) who seek for an occupation in SPC 6 (agricul- tural & fishery workers). It was argued that this sector has shrinking employment capacities and the interest of the young generation in this sector is decreasing as well. b. The type of job (SPC) that an employed individual will look for
According to the stakeholders, high shares of commuters and assembly workers are a characteristic feature of the Altmark region and Hohen- berg-Krusemark municipality. Accordingly, it was recommended to slightly increase the total values in the age group 25<45. Moreover, it was assessed that job mobility decreases over age.
Since wages and career perspectives were in general not positively as- sessed for SPC 8 (= elementary occupations) stakeholders recom- mended decreasing the values of employees seeking a job in SPC 8.
Stakeholders recommended to increase the percentages for SPC 7 (= crafts & trade) to take the effects of some recent industrial allocations (paper mill, pulp factory) nearby Hohenberg-Krusemark into consid- eration. iii. Subjective comparison of real and simulated values
Prior to the workshop, model simulation outputs were prepared for Hohen- berg-Krusemark municipality. The simulation focused on employment and demography parameters. The initialisation year was 2000; simulation runs have been generated on a yearly base for the period 2000-2020. The average of 100 simulation runs was reported, including a confidence interval to im- prove transparency and presentational clarity. Validation years vary for each simulated parameter but cover in most cases (exception: population by age groups, birth rate) the years 2000-2010 at LAU 2 level. a. explaining trends
The discussion was motivated by a comparison of simulated values with ‘real’ values for the period 2000-2010 and focused especially on those parameters where real and simulated values differentiated.
With regard to the confidence intervals, stakeholders could grasp easily that they were presented with only one simulation among many existing ones. Moreover, they caught without any problem the idea of comparing the statis- tical data with the simulated data. This might serve as an indication that the graphs and the intention of the exercise were comprehensible. Stakeholders pointed out that the model is at one hand underrating the flexibility of the population in the age group 18-24 to change their residence in order to find an employment, training or further education opportunity. In the beginning of 2000 many young people left the Altmark due to the lack of job and train- ing opportunities. Members of the age group 18-24 who stay in the region have to demonstrate a high mobility and do therefore not necessarily remain
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 24/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Hohenberg-Krusemark residents but move their households closer to their place of work/training. One the other hand, stakeholders emphasized that the model is overrating the flexibility of the elder generation (45-65, 65+) to change their place of residence. Most of the active workforce in these age groups tends to accept rather high commuting distances but still remains located in Hohenberg-Krusemark. Drivers of this situation are: i) the high share of privately owned houses among Hohenberg-Krusemark residents; and ii) the low prices on the regional property market, which make it difficult to sell property without losses. b. plausibility of future baseline scenarios
Stakeholders were invited to comment on the future baseline scenarios (2011- 2020). In general, these were perceived as plausible by the stakeholders. However, some critical comments were raised as well:
Stakeholders noted that in the period 2011-2020 the population de- cline in the age group 10-17 is far too strong. This intuitive comment was supported by the argument that the age group 0-9 is slightly in- creasing over the same time period. This increase should have over the years some visible effects on the successor age group and by that „soften” its declining curve. In addition stakeholders noted that the prediction of employed resi- dents in the age group 15-25 for the period of 2010-2020 is not plau- sible; especially when it is cross-tabulated with the demographic de- velopment of the corresponding age group. The interpretation of these two graphs would somehow suggest that more than 2/3 of the Hohenberg-Krusemark residents between 15-25 years would be un- employed in 2020 - which is according to the stakeholders far too high. (Clarification remark: the population simulation by age groups includes students and inactive individuals as well) The strong population increase in the age group 18-24 for the year 2019 was perceived as unrealistic. Stakeholders remarked that due to the aging population an increase in the medical services could be expected. c. suitability of input data
Since the demographic development for the population under 25 in the future baseline scenario did not comply with stakeholders’ expectations, it was rec- ommended to consult the regional demography prognosis for Saxony-Anhalt and to consult its underlying assumption. Moreover, it was noted by the stakeholders for the observation year 2011 that the real employment data all should incorporate the merger of Hohenberg-Krusemark with Schwarzholz municipality. d. Overcoming the problem of missing data and parameters
The labour data in the German model covers employees and apprenticeships from all sectors liable to social insurance. Accordingly, self employed persons are not included. This fact has some implications for the data interpretation
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 25/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
especially in the agricultural sector. The majority of farms in the Altmark are established as family farms, managed in self-employment. Since this work- force group is not included in the model, one target of the workshop was to receive some numbers on self employed individuals living in Hohenberg- Krusemark. Forms, as shown in Tab. 3.3 were handed out to the two mayors. They agreed to update the table and to send it back to IAMO team.
Tab. 3.3 | Role of self-employment in Hohenberg-Krusemark and/or district for 2011 (shortened & translated)
Self-employed persons per sector
Total or percent (%) of the total workforce In Hohenberg-Krusemark In Stendal district Agriculture, hunting, forestry, and fishing
Construction
Production Services Other
3.3 Scenarios
Scenarios are used in ex-ante policy assessment and allow comparisons to be made of possible alternative futures and the possible effects of different policy measures. These alternative futures might arise from the continuation of current trends, from changes in trends, or from the occurrence of particular events including new policies.
Had a refined model been ready, qualitative scenarios (narratives) could have been combined with quantitative scenarios, resulting in the production of detailed narratives supported by quantitative outputs from a running model (as proposed in D2.2, Section 1.4). The capacity of the model as a tool for comparing policy scenarios would then have been demonstrated.
Nevertheless, it is necessary to establish that narratives which are relevant to model users can be expressed within the system described by the model (pre- determined causal relationships and their supporting assumptions, key vari- ables, and spatial and temporal scales, as outlined in D2.2 S1.2.1). Outline scenarios were also be developed for later elaboration.
It therefore follows that the process of scenario development can be a useful tool in model development in that:
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 26/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
i. it supports conceptual model validation in that ‘real world’ causal relation- ships are considered, and compared to those in the model and their underly- ing assumptions. ii. it provides an opportunity to identify user needs, whether the variables and driving forces included in the model are appropriate and useful, and whether the model is relevant to actual policy concerns.
3.3.1 French scenario discussion
The discussion of scenarios during the French workshop concerned only the officials coming from various administrations of the area. All the mayors but one left, pretending some other duties, which they had not mentioned at the beginning of the workshop. They may also have left because they did not want to participate in the same arena as the officials, who would be framing their action. Stakeholders proposed many topics and examples of events which would form a foundation for scenarios. They focused first on desired goals and then finding a policy by which they could be achieved. Table 3.4 summa- rizes the type of events and directions which stakeholders wished to explore, and how these events would be captured within the model parameters. These require further validation with stakeholders and an exploration of their feasi- bility with modellers.
The main assumption in this interpretation process is that policies are effi- cient in reaching their objectives. If a policy aims at attracting young people to the farming sector through specific incentives, we consider that it is suc- cessful; hence the scenario includes a rise in attractiveness of the farming sector for young people. Outcomes for stakeholders are not an indication of which policy to implement but rather whether it is important to find a policy efficient at reaching a given goal.
Certain variables were missing and would have to be represented by proxies. For example, an increase in agricultural revenue cannot be shown directly, but rather would be represented as an increase in the attractiveness of em- ployment in agriculture to young people. Changes to indirect employment are not explicitly included. Certain indicators, for example local wealth (GVA), although not included in the model outputs, might be calculated (by WP6) from parameters such as employment in various sectors.
The modelling team (WP3) has indicated that it would be difficult to intro- duce economics dynamics into the model because it would need a lot of fur- ther data and parameters, and the introduction of other processes in order to represent the local economy. Therefore the modelling team chose not to rep- resent the economy.
Tab. 3.4 | French workshop: suggested outline scenarios
Issue Interpretation in the model Energy cost evolution: impact on Decrease commuting distance: ie. Probability to unemployment accept a job far from residence without changing
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 27/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
residence (expected rise in unemployment) Financial support from CAP Attractiveness of farming sector: probability to accept/quit a job in farming sector. This can be modulated according to the age of the individual (because support can be age dependent) Evolution of direct employment in Evolution of number of jobs opened in farming farming sector and indirect em- sector in the various possible SPC and changes ployment in other industries from family to employed labour.
Evolution of number of jobs in other industries in the various possible SPC, and at various points in time. Several possible scenarios are possible here according to strategic choice of type of agri- culture (eg. production-oriented or land- scape/heritage-oriented)
Need for model to characterize impact on indi- rect jobs, which is not computed in current ver- sion because of the absence of economy. The only relation included in the model is between popu- lation and local services. However a minimal way to take in account this request is to add, to sce- narios, changes in job supply for sectors other than agriculture but with a delay. Participants considered possible a decrease of indirect jobs and a stabilisation of the number of farmers Reduction in provision of local ser- Decrease of in-migrants with children, and in- vices (school) crease probability of out-migration for house- holds with children Local services, such as schools, have been described as a potentially cru- cial element in the dynamics of the area. Some agreement reached with the following belief: a lower attractivity for families with chil- dren. Changes in level of other local ser- Expected impacts on migration and housing vices market, but couldn’t agree in which direction. Residential dynamics: increasing Increase of residential supply of various sizes. supply of small houses or flats for rent (permanent home) as a key Increase probability of acceptance of a residence driver of dynamics (Increased at- (to take into account increased comfort level). tractiveness of Condat area) This can be selected according to age if policy aimed at a specific population and tries to meet its demand e.g. small, high specification homes to attract retired people. Indirect effect on de- mand for services. Residential dynamics through active Increase probability of acceptance of a residence promotion (by mayor) Evolution of ICT. Adoption linked Increase of potential pool of in-migrants. In- to energy price (commuting cost) crease in distance from employer if able to home-
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 28/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
work.
Decrease of local jobs (in service sector) due to e- shopping Impact of norms on employment Decrease of local job supply in craftsmanship (disproportionate cost of compli- sector. This is the baseline scenario. Another ance with regulation for small busi- scenario could assume a policy to mitigate this nesses) impact of evolution of norms.
3.3.2 UK scenario discussion
Based on a format for environmental scenario analysis provided by Alcamo and Henrichs (2008), the discussion explored chains of causality and impacts associated with a possible future local event. Participants proposed a scenario arising from the closure of the largest employer in the case study area (a quarry and associated cement works). Remediation by means of EU rural development policy was also discussed.
Initial state
- Lafarge limestone and shale quarries and cement works, located in Hope village, are the largest employer in the case study area (220 jobs). - The Hope works provides direct jobs and indirect jobs in well- developed local supply chains. - Employment at the Hope works drives commuting patterns of resi- dents of the ward network and in-commuters from beyond it. - Employment provides incomes. Incomes are mostly spent outside Bradwell and Hope, but partly within the ward network. Therefore it contributes modestly to induced employment. - The availability of work may retain residents of working age and their children in Hope/Bradwell. This creates demand for local pri- mary schools. - Although environmental effects are not in the model, the plant causes visual and atmospheric pollution including greenhouse gases.
Driving forces
- Increasing house prices due to a fixed housing stock due to National Park location. - Fairly static population size, but ageing with consequent changes to birth and death rates. (Exit of young people and increasing numbers of retirees.) Local primary schools have fewer pupils. - Productivity improvements causing gradual decreases in the Lafarge workforce.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 29/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Event
- Closure. Due in 2034 when permission to quarry runs out. Might happen earlier due to a proposed merger.
Consequences
- Temporary jobs will be created by restoration and possible re- development of the site. Site re-use (might be for housing, business premises, leisure/tourism activities) and possible creation of perma- nent jobs.
Quantification
It appeared to be relatively easy to identify the nature of the main impacts of the Hope closure. However, quantification of these impacts would be more difficult, and consequently modelling will require a large number of assump- tions to be made. Consideration must also be given to the timing of when these impacts occur.
Tab. 3.5 | UK workshop: linkage of stakeholder narrative to model parame- ters
Target model variable Determinants of target variable value (assumptions required) Economic activity of local Outcomes for redundant workers. Rates of: residents (total) ▪ re-employment into local/distant jobs
▪ early retirement (economic inactivity)
▪ out-migration
Depends on age and occupational type Employment by job type Net change determined by direct, indirect and of local residents (defined induced job losses (Hope works, local supply by SPC and SoA*) chain, local consumer spending) minus re- employment of those who remain resident lo- cally and availability of newly created jobs Employment in local re- Level of change in consumer spending by resi- tail and personal services dents and in-commuters
Increased expenditure by tourists (see next row) Tourism employment Increase in number of overnight stays. (Closure of works makes area more attractive for holi- days.) Local population charac- Rates of out-migration and out-migrant charac- teristics. Characteristics of new occupants of
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 30/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
teristics existing housing stock. (House prices likely to increase in value after the works have closed; (defined by age and job lack of local work deters working age people in type) lower ranked occupations from moving in. Ex- pect older, professional people to move in.)
Impact on birth/death rates of change in age structure. Housing Stock fixed in short term, but more attractive following closure.
In long term, possibility of limited development of new housing at restored site. Affects local population characteristics (Dependent upon who occupies new housing.)
*SPC= socio-professional category; SoA=sector of activity (industry)
Examples of insights from stakeholders (local knowledge)
Professional workers more likely to commute, and also less likely to find new jobs locally.
Currently there is a concentration of quarry workers in the 45 to 65 age group rather than young workers. Therefore an expectation of high levels of early retirement in redundant quarry workers rather than unemployment/re- employment. They will remain resident in their existing house.
Spending power (ability to buy a house; other expenditure) depends less on SPC than on income level. Process workers may be paid more than well- educated white collar workers.
New housing for local need is allowable on special sites. If desirable full mar- ket price housing is also permitted, this may draw in entrepreneurs and job creation.
Indirect (supply chain) jobs include those in transport, technical services and hospitality.
3.4 Germany scenario discussion
Initially the IAMO team planned that the scenario discussion with the stake- holders should first focus on desired development goals for the region. It was envisaged that, after ranking identified goals, the three most important goals would be broken down into policy measures supporting the goal achievement. However, when stakeholders were requested to brainstorm desired develop- ment goals, a quite intense and controversial debate on the Altmark’s future development directions started. It was argued by the stakeholders that the
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 31/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
assessment of the development direction would be a precondition for them to formulate meaningful development goals for the region. The IAMO team assessed this discussion as a fundamental base for a participatory scenario development. Therefore this discussion on the general development direc- tions was spontaneously given more space. In the end two main development directions of the Altmark could be identified (Table 3.6). Due to time- constraints, the break down into policy measures was only roughly sketched and needs further clarification.
The discussion was driven on one hand by the perception that regional growth would be a feasible development direction of the Altmark. Stake- holders outlined that the development direction of a “growing region” would be supported by policy measures which aim to stimulate new private sector investments and entrepreneurship on all levels. The municipalities should receive additional financial support to facilitate regional growth potentials. Stakeholders referred herein to the project “Regional Budget” (part of the German GRW Program) which should be further expanded. Investments in the tourism sector should be further expanded as well.
On the other hand, another group of stakeholders proposed that a “growing Altmark” - despite all potential policy efforts – might not be a realistic future trend when taking the decreasing birth rate, the continuing outmigration, the increasingly ageing population, the peripheral location and the insufficient economic power into consideration. They perceived the future development trend of a “shrinking region” as more likely. It was argued that this trend would imply different policy measures. Stakeholders noted that this percep- tion is not easy to communicate in the political and public sphere but might be a more consequent reaction to the anticipated future regional develop- ment.
Tab. 3.6 | German workshop: Future trends
Development trend Potential related measures “Growing region” Industrial location policy Expansion of industrial &areas Microfinance schemes for small projects & entrepreneurs Strengthening of municipalities: Continua- tion of the Program “Regional Budget” Tourism development “Shrinking region” Conserve nature/expand nature conservation areas Support “vital villages” & local identity (sup- port cultural live, civil engagement, village clubs) Downsized/customized infrastructure (e.g. schools, sewage)
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 32/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Given that the focus of the workshop was on the analysis of the model and the data, there was not enough time left to formulate the scenario in more detail. For this reason, the identified development trends can only serve as a basis to develop more concrete scenarios that are realistic for the region (by desk analysis or further consultation with expert stakeholders). The definition of those concrete scenarios would include a set of measures which are applicable as policy programs for the region and that can be defined in terms of model parameters. Such a collection of programs will then form part of a narrative that develops in line with the proposed development trend (shown in Table 3.6), the impacts of which on the model dynamics can be analyzed with the simulations.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 33/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
4 DISCUSSION OF THE TYPE 1 WORKSHOPS
4.1 Stakeholder selection
Desirable characteristics for workshop participants were a detailed knowl- edge of both the socio-economic processes in the case study municipality and the operation of EU policies (enabling them to check the credibility of the model and its outputs at the local scale, and also generate realistic scenarios that might result from EU policy), and the ability to understand a conceptual model and use it as a focus for discussion. Ideally suitability would be as- sessed before a firm invitation is issued. Preparatory interviews, as reported in D2.2, were a useful tool for identifying and screening stakeholders.
4.2 Conducting the workshop
The approach chosen for conducting the on-model engagement was deter- mined by the information sought from the workshops, and the lack of famili- arity of the participating population with modelling. Thus the method im- plemented was characterized by a combination of framing in terms of model structure, and also openness to the variety in content and format of represen- tations by stakeholders. This may be conceptualized as being positioned be- tween knowledge elicitation techniques, by which ethnological observations (verbatims) are collected and interpreted into the model, and model parame- terisation that forces participants completely into the model. The setting with open debate facilitated the expression of various viewpoints. However, the counterpart is that the work of interpretation of workshop findings into the model remains to be done by the research team, limiting the level of influence by participants.
The French workshop highlighted the issue of how best to conduct a work- shop when there is a diversity of power and agendas between participants (policymakers) who operate at a more strategic level and local officials. In the event, all the elected people but one (mayors i.e. LAU2 level) left before the scenario discussion took place taking the opportunity of a coffee break and pretending overcommitment, while they did not announce any specific time constraint earlier. There remained only more strategic level officials, in charge of higher levels (from county to regional representation of French Agriculture Department). This may be due to the mayors feeling constrained by the presence of strategic level officials, or them feeling disenfranchised by then on-going reforms of political organization. The German workshop was conducted in the special situation that the major of Hohenberg-Krusemark, who was also hosting the workshop, had resigned from his position only the night before. Most of the workshop participants (including IAMO’s team) were informed of this fact shortly before the workshop started. It is assumed by IAMO team that this special context influenced to a certain degree the discussion atmosphere and workshops results.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 34/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
4.3 Level of detail
In both UK and French workshops, the discussion, especially at first, was at a level of detail which was considerably beyond what the model is capable of delivering. To some stakeholders, the message that the model is a simplifica- tion of reality seemed to be contrary to the notion of asking them to provide local level information. Consequently there is a clear need for effective facili- tation to focus such discussions, which can otherwise be very time- consuming. In the French workshop, one of the PRIMA research team mem- bers took somehow the role of a “model broker”, even though it was not ex- plicitly framed as such. We derive this new concept role from the existing “knowledge broker” (Brown and Duguid, 1998; Pawlowski and Roley, 2004). Implementation of such a role here was rather in ‘translating’ the model’s objectives and the model’s main assumptions, thus addressing one communi- cation direction only, that from model to stakeholders. This at least helped in making clear the divergence between stakeholders’expectations and the model’s potential. However this role of model broker could be enlarged to encompass both directions more extensively.
In the German case, the IAMO team could afford a larger presence, hence endorsing some of the activities of a “model broker”. They observed that stakeholders picked up the idea of the model and its functions quite quickly and accepted that the model is a simplification of reality. The level of detail of stakeholders’ inputs was sufficiently focused, which was maybe helped by the fact that the workshop and discussions concentrated predominantly on one municipality.
4.4 Model relevance
There was clearly interest among stakeholders in having information from a model to supplement existing methods of decision-making which were reliant on common-sense, judgement and instinct.
All workshops gave broad approval to the conceptual model. Nevertheless criticism was made of it, especially the lack of economic variables, with in- come considered by stakeholders as the key driver of decisions relating to job transitions and commuting. In many respects, stakeholders found the simula- tion model was realistic and contained many of the key features of the socio- economic system which it aimed to represent. Scenario development activi- ties at both the French and UK workshops demonstrated that the model components allowed a wide variety of rural development issues to be repre- sented. Stakeholders were able to suggest issues for examination and to iden- tify the model parameters which would be affected, and the logical chains which connect them. Explanations from modellers were necessary to explain how some parameters could be adapted (by changing their underlying as- sumptions) to make them relevant.
It became clear that many scenarios would affect a complicated set of inter- related variables for which the expected direction of change might be hy- pothesised. Future work would add to this complication by trying to quantify the impacts, and because in many cases there will be a lack of data (for exam-
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 35/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
ple the absence of local input-output tables for quantifying indirect jobs), reliance will be placed on stakeholder deliberations. This might include as- sumptions about levels of change in response to an event. For example, if the number of jobs was to fall, how great is the loss, and which people (as defined by age, SPC and Sector of Activity) would be affected. In practice, a range of assumptions would be made, each creating a slightly different scenario, and contributing to a mass of information which might be regarded as specula- tive.
Scenario development was also a useful way of consolidating stakeholders’ understanding of the model. Ambiguity of scenario meanings, as it is well explained in Voinov and Bousquet (2010), is somehow useful for reaching this output. Scenarios can be considered either as a result of a process aiming at designing possible futures (Alcamo and Henrichs, 2008), or as “a plausible description of a management plan” (Voinov and Bousquet, 2010) to be tested through simulation. The first point of view does not require any technical modelling skill, which legitimizes the involvement of all participants in exer- cises framed to design scenarios according to the second point of view. The workshop moderator was important here to handle this ambiguity. However this could be better articulated as an explicit activity for a model broker. With this framing, participants try to get accustomed to the model (a story- generator) through designing what it can work out. Through this activity the model becomes a less abstract device, and its strengths and limits become clearer. Our objectives here are to control the possibilities to stretch the model (how far can it be used, i.e. restating its domain of validity) and iden- tify the needs for adaptation. In our cases, the scenario development activity generated discussion of model dynamics, and identification of relevant pa- rameters. As mentioned in Voinov & Bousquet (2010), scenario development activity is definitely an efficient way to mobilize stakeholders in participatory modelling.
Finally, despite reservations about the appropriateness of the ‘real’ data used for comparative purposes (e.g. inappropriate geographical scale, not captur- ing productivity increases), the comparisons of real data and model outputs did not dispute the general plausibility of the values produced by the simula- tion. In Germany though, some future runs were seen as non-plausible. This apparent acceptance of simulation outputs has still to be tested against the assumption of an an ex-post rationalisation for any findings.
4.5 Conclusions, Type 1 workshops
Stakeholder reactions showed they were receptive to both the concept of modelling and the use of models as decision-support tools. When explained step-by-step, stakeholders found the approach transparent, and they under- stood the logical basis for the initialisation and the dynamics of the model. Moreover the PRIMA model was considered to provide a framework which is relevant for examining a range of rural development policy issues by scenario analysis.
The transparency provided by a forum in which stakeholders are allowed to interrogate experts is likely to improve stakeholder confidence in the model.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 36/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Some important points were clarified in such discussions, for example by explaining how some of the model’s apparent limitations (such as ‘missing’ variables) are overcome by means such as adjusting coefficients, operation of constraints, and using proxy variables.
The benefits of the workshop forum, as opposed to individual interviews, arise partly from efficient use of resources. Clearly, the model is complicated and explaining it takes a long time. Furthermore, experts are sometimes re- quired to answer any highly technical questions and their capacity to answer them can improve the confidence of all participants. It would be less efficient if not impossible to conduct such an interaction on an individual basis. A more comprehensive, more nuanced range of responses is obtained if a range of stakeholders are addressed.
Finally these experiences highlighted the need to specify further a set of ac- tivities characterizing a model broker or the role of model translator. This is examined further in section 7.3 including some techniques available to per- form these activities. Attention has to be paid to the interpretation of model outputs by stakeholders as well as to the interpretation of feedbacks from participants by both other participants and modellers.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 37/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
5 OUTLINE OF TYPE 2 WORKSHOPS (CROATIA, CZECH REPUBLIC, BUL- GARIA)
5.1 Introduction to Type 2 workshops
As explained in Section 1, difficulties were encountered in adapting the mi- cro-simulation model for 3 of the case study areas. Consequently stakeholder interactions were conducted with different intentions from those of France, Germany and the UK, and instead aimed to obtain stakeholder assessments of the acceptability and potential usefulness of the micro-simulation ap- proach. The objectives were:
- To investigate how stakeholders react to the concept of the micro- simulation model. - To explore the extent to which stakeholders trust such approaches to deliver realistic and useful information. - To explore where they feel such models to be more or less likely to deliver reliable information - To explore stakeholders’ reactions to other potential approaches to inform policy development. - To identify past situations where a simulation approach could have led to better or more appropriate policies and interventions being developed. - To explore possible future scenarios where participants feel that the micro-simulation approach could make the most useful contribution.
5.2 The generic method
Stakeholders were selected as being knowledgeable about the area being studied, with most also experienced in the implementation or development of socio-economic policy. Moreover it was important that they would be able to contribute to a discussion of a conceptual and technical nature, an attribute which was ascertained during earlier contacts.
To elicit this information, a focus group method was adopted with an indica- tive group size of 6 to 8 participants. The research team would consist of a facilitator and note-taker, and also possibly a second speaker to deliver the teaching material. An audio recording would be made, subject to participants’ agreement. The suggested duration of the workshop was 2.5 hours.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 38/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
5.3 Outline workshop programme
1. A brief introduction to explain the workshop’s context and objectives.
2. Identify policies in the case study area which show a good /poor apprecia- tion of local level factors, and the process by which these policies were devel- oped.
3. Stakeholder familiarisation with the MS model:
a. Technical presentation (PowerPoint): key elements and advan- tages of the conceptual model
b. Discussion of simple scenarios and relating them to model ele- ments (variables, transitions between variables, driving forces, causal relationships)
4. Feedback after familiarisation with the model:
Is the conceptual model plausible?
Can storylines be told (‘tracked’) using the model as a framework?
5. Evaluation of the MS approach (plausibility, accessibility, acceptability)
6. Potential uses of in policy design and selection.
7. Perceptions of the trustworthiness of different model developers.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 39/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
6 CONDUCT & FINDINGS OF TYPE 2 WORK- SHOPS
6.1 Organisation of the Workshop in Croatia
Equipment: laptop, data projector, extension cable. Sound recording equip- ment were available but not used.
Table 6.1 summarises the organisation of the Croatian workshop. The invited stakeholders were regional decision makers possessing an excellent under- standing of the case study area and experienced in developing and imple- menting policy.
Tab. 6.1 | The Croatian Workshop: organisation and participants
Location Pazin (Istria), Premises of AZZRI (Agency for rural development of Istria). Date 19/10/2011 Duration 3 hours Stakeholder participants: 1. Head of AZZRI (Agency for Rural Development of Istria)
2. Former head of Pazin municipality and member of AZZRI's supervision board 3. Head of Fund for Agriculture and Rural Tourism Development of Istria 4. Head of Department for Agriculture, Forestry, Hunting and Water Man- agement of Istria County Project staff participants (Faculty of Agric., Zagreb University)
Moderator and technical presentation –Mario Nvaro
Note taker – Branka Sakic Bobic
Equipment: laptop, data projector, extension cable. Sound recording equip- ment were available but not used.
6.2 Organisation of the Czech Republic workshop
The workshop was conducted in June 2011 in Brno in the central part of the case study area of South Moravia Region. The workshop was attended by 5 stakeholders and 3 research staff (See Table 6.2). All participants agreed to
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 40/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
audio recording which is available on the following web site :http://www.ulozto.cz/9402462/prima-fg-brno-wma.
Tab. 6.2 | The Czech Republic Workshop: organisation and participants
Location Brno Date June 17, 2011 Duration 3 hours Stakeholder participants Type Description 1. Policy maker Mayor of the municipality Černá Hora 2. Policy maker Mayor of the municipality Rájec-Jestřebí 3. Policy Official Regional development Officer Municipality with extended competency1 Blansko 4. Policy Official Regional development Officer NUTS 3 (re- sponsible for responsible for the agriculture and rural area) 5. Policy Official Regional development Officer NUTS 2 (re- sponsible for strategic plans and metodol- ogy) Project staff participants Facilitator Second speaker Notetaker
6.3 Organisation of the Bulgarian workshop
Tab. 6.3 | The Bulgarian Workshop: organisation and participants
Location Svishtov, centre of Svishtov munici- pality Date September 2011 Duration 4 hours Stakeholder participants Type Description
1 Municipality with Extended Competency is administrative and statistic spatial unit that covered the case study area. The Unit NUTS 3 includes 21 Municipalities with Extended Competency.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 41/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
1. Local resident Resident, village Ovcha mogila, farmer 2. Local Resident Resident, village Morava, Chairman of the Production Co-operative, member of the initiative Committee for establishment of the Local Initia- tive Group 3. Local Resident Resident, village Delyanovtzi, Chair- man of the Production Co-operative, member of the initiative Committee for establishment of the Local Initia- tive Group 4. Local Resident Resident, village Hadji Dimitrovo, Chairman of the Production Co- operative, member of the initiative Committee for establishment of the Local Initiative Group 5. Local Resident Resident, village Sovata, Chairman of the Production Co-operative, mem- ber of the initiative Committee for establishment of the Local Initiative Group 6. Local Resident Resident, Svishtov, Business Center 7. Local Resident Resident, Svishtov, Council of Tour- ism 8. Local Resident Resident, Svishtov, Chairman of the NGO “Earth for everyone” 9. Local Resident Resident, Svishtov, Chairman of the Local Office of the Association of agricultural producers 10. Local resident Resident, Svishtov; Member of the Council Committee 11. Policy Official Economic development Officer 12. Policy Official Agriculture and Forestry Officer 13. Policy Official LEADER – Local Initiative Group, project officer 14. Policy Official Director of Persina National Park 15. Policy Official Deputy Mayor of Svishtov, Economic Development Project participants (UNWE) Moderator Diana Kopeva Note-taker Maria Peneva
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 42/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
6.4 Discussion of the non-model workshops
The Type 2 (non-model) workshops all employed a similar schedule, and it has therefore been possible, in Tables T6.4 to T6.9, to group their responses by theme. The workshop started by gaining some understanding of how suc- cessful local policy-making is currently. This highlighted the often poor im- plementation of policy programmes.
Tab. 6.4 | Currently, what policies show a good appreciation of the local geo- graphical level?
Member state Findings
CZ (South Moravia) NUTS3 RD Plan for S. Moravia (developed from the national RDP by Dept of Regional Development, Brno) perceived as appropriate and able to address local needs. It had been necessary to alter eligibility and thresholds of allocations to make appropriate for local needs.
Local perspective included by drawing on offi- cers‘ experience and local knowledge. Sensitiv- ity to local factors is most apparent in social policy, followed by economic and finally envi- ronmental. Nevertheless, some decisions would have been better if ex-ante information on impacts of proposed policies had been avail- able.
From the local perspective, national policies, although having acceptable goals, are badly implemented and organised, and often poorly co-ordinated with local and regional policies Bulgaria (Svishtov Munici- The municipality is the main administrative pality) level for policy implementation and develop- ment planning (=LAU1 level). Municipality Development Plans are developed according to Article 14 of the Regional Development Act and comply with all national strategic planning and development documents as well as regulations of the EU‘s Structural and Cohesion Funds.
During plan development the municipality authorities made consultations with various stakeholders and organized public fora to clar- ify stakeholders’ vision for future development and local priorities, to take into account the
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 43/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
public interest and to ensure public support for its implementation. The following methods were used in the process of plan development: system analysis and synthesis; SWOT analysis; statistic methods; index and the coefficient methods.
The design of EU policy measures (eligibility criteria, time lags, documentation required) is such that implemention levels are poor. Croatia Poor vertical co-ordination. Centralised deci- sion-making so state has no overview of local problems and conditions. e.g. lack of consulta- tion with local level policy makers for IPARD (pre-accession fund) and consequent develop- ment of inapproriate eligibility criteria. Poor administrative structures. County rural devel- opment dept is ignored when national RD poli- cies are created.
Nevertheless an agricultural capital investment (started 2004) has been fairly successful, though having unintended consequences (ma- chinery dealers etc increasing price)
The lack of good comprehensive data is a fun- damental problem.
After the model demonstration, stakeholders commented on the model com- ponents.
Tab. 6.5 | Are the model’s components appropriate: databases and driving forces?
Member state CZ (South Moravia) Demographic change, socio-professional structure and housing dynamics are included. Change of land use and infrastructure were not included but con- sidered important. Friction in the housing market caused by multiple ownership (following inheri- tance) resulted in non-occupancy and inability to sell houses was important in dynamics BUL Demographic change (age, migration) and econ- omy are key driving forces. Economy along with business development, land use change and new infrastructure are important drivers but missing. Commuting is complicated (weekly commuting is common; commuting beyond the municipality, district, region (especially to the big cities of Sofia
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 44/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
and Varnia) and even country. CRO Population out-migratrion from rural Istria to coastal area and tourism. Loss of agricultural em- ployment. In-migrants (often foreigners) trans- forming farms into residences. Specialised high value farming
The following sections deal with the key issue of whether the stakeholders find such models plausible, whether they are comprehensible, and whether they are acceptable to policymakers.
Tab. 6.6 | Model plausibility: does the model accurately represent reality?
Member state CZ (South Moravia) Perception of plausibility varies between SHs, de- pending on their own professional background or specialisation. There appears to be a gap between how local politi- cians formulate a story and the kind of story the model ‘needs’. Datasets are robust but lack municipality level data.
BUL Evaluation of plausibility varied between SHs, de- pending on their age, educational background and current position. Only modest amounts of data available at LAU1 (mu- nicipality) level. Almost none at LAU2 level (villages and towns),although coverage at NUTS3 is quite good. Therefore cynical about accuracy of any outputs at low geographical scale
CRO Model concept is valid, though possibly appropriate for more developed regions than Istria. Model would only be useful if adapted for the local area and built upon reliable data. Lack of trust if the majority of local data are estimated. Data availability at LAU1 and LAU 2 level is a problem.
Tab. 6.7 | Is the model accessible? Is it easy to understand?
Member state CZ (South Moravia) Need to distinguish between using the outputs, and administration of the model. BUL Distinguished between model experts and policy mak- ers who use its outputs. Existing policy officers with high skills would be able to use the model. Would need a detailed manual and training
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 45/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
CRO Policy makers at the local level have knowledge and skills to understand use such a model (as clients). De- velopers' capacity at the local level exists.
Tab. 6.8 | Is the model acceptable to policymakers?
Member state CZ (South Mora- Rating related to individual’s understanding of the via) model. Thought they would like to use it. A more com- plex model would be too difficult to interpret and would be avoided. The need to understand the com- plexity of the model, and the data requirement to run the model limit the acceptability of the model. BUL Questioned the accuracy of the output for use in deci- sion-making given the limited data. Also ahHigh level of dependency on the model operator.
Visualisation tool for displaying results is necessary CRO Enthusiastic to assess its applicability. However lack of input data. All found the model to be comprehensible. A working system would attract more users.
Tab. 6.9 | If available, how would the PRIMA model be used?
Member state If available, how would the PRIMA model be used? CZ (South Moravia) There appears to be a gap between how local politi- cians formulate a story and the kind of story the model ‘needs’. BUL Model might support plan development at various levels, and compatibility between Rural Dev Policy and regional policy design. Would be able to judge better after seeing model outputs. CRO Quicker decision-making in both determining and operating strategic programmes. Could advance deci- sion-making in pre-accession phase and later in EU.
Decisions were not made without knowing local situa- tion, but model could be a tool for detailed analysis of decisions and what-if analysis.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 46/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Tab. 6.10 | Who would you trust to deliver the model?
Member state Who would you trust to deliver the model? CZ (South Moravia) The municipality level model should be run by the regional office of the Czech Statistical Office as they hold the available data. The model could capitalise on data which municipalities with extended competen- cies in the district are obliged to collect for analysis of development. Not so much trust as effectiveness. BUL Either by the Municiaplity authority (with external experts to advise) or by a Business Centre mandated to deliver a model CRO In the case study area, the County Administration (NUTS3 level) could employ or hire competent staff to run the model with support from external advisers for data management and scenario development. Based on current experiences international consultants would be inappropriate as they lack sufficient under- standing of the area.
6.5 Conclusions for Type 2 workshops.
Bodies do exist for policy development at lower levels. However there were marked problems in integrating EU, national and regional policy pro- grammes with local needs. On the whole, the intentions of higher level poli- cies were considered to be appropriate for the case study areas. Discussions highlighted the sensitivity of the outcomes of the policy process to lowest level decisions. Inappropriate implementation criteria for measures and bu- reaucratic difficulty resulted in much less impact than might be obtained had they been better designed.
Stakeholders followed the logic of the model as it was explained to them. They were able to identify driving forces, many of which would be included in a finished model. However, they considered some driving forces which are important in the case study areas were missing e.g. long-distance commuting (Bulgaria) and friction in the housing market (Czech Republic). Nevertheless, it is necessary to spend a long time explaining the model before any informed discussion can occur.
The plausibility of the model was limited by 2 factors: the omission of key dynamics such as infrastructure development and land use change; and the lack of suitable datasets at the level of the case study area. If the obstacle of poor data were overcome, then the model would be acceptable because it was found to be comprehensible. Further complexity would render it too complex to interpret. It raises the issue of the presence of a broker: someone who can conduct and interpret analysis if it is too difficult for the institution’s own
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 47/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
staff, adding some distance between the analysis and its inclusion in model updated version.
The question of who would be preferred to run such a model drew interesting responses. All three groups considered that they would trust it most if it was maintained and run locally and they seemed confident that local expertise existed or could be contracted in. There was a need to distinguish between those who use the outputs, and those who administer the model, running scenarios and maintaining data.
7 DISCUSSION AND CONCLUSIONS
Despite the previously mentioned difficulties in performing these on-model workshops, which arose from the difficulty of tuning three schedules (model- lers, field scientists and stakeholders) within the constrained timeframe of a research project, experience acquired throughout PRIMA project neverthe- less confirms some recent propositions in the domain of participatory model- ling, and brings some new pieces of knowledge to that field, as observed in previous sections.
7.1 Main outcomes of stakeholder on-model engagement in PRIMA in the field of participatory modelling
The specification of the PRIMA population model allowed the exploration of a new range of models in participatory modelling (Voinov and Bousquet, 2010; Barreteau et al. 2001), with stakeholders facing pre-identified con- straints. Specifically, the PRIMA models used in the French, UK and German workshops are based on large data sets and probabilities of transitions be- tween the states of individuals in respect to private life, education and job, and place of living. This format is far from the common cognitive frameworks with which stakeholders are familiar with in describing their environment. These models are also rather complex in terms of the number of processes represented. They focus on basic dynamics such as moving, getting married, and retiring. These dynamics are known about as much from their own per- sonal experience as from objective expert knowledge, leading to potential gaps between heartfelt representations and detached expert observations. Finally the models used in the workshops aimed at being realistic and at a very local scale, generating sensibility to the outcomes among the affected population, and reaching deep emotional levels. The local scale is also per- ceived as the place of singularities which have a rather high impact on the path of evolution.
All these characteristics raise issues for participatory modelling because they generate at the same time both wide gaps in the language being spoken, but a feeling of being very close with regard to the stakes dealt with. What is the potential capacity of stakeholders to get into this type of model? How can their knowledge be incorporated into formal models that need substantiated data?
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 48/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Despite this difficult context, we had encouraging outcomes:
- enhanced knowledge on case studies thanks to on-model (type 1) and ‘no-model’ (type 2) discussions; - discussion on the limits of the ‘system’ at stake: what is the suitable area to work at when dealing with policy impacts on activities in ru- ral municipalities? - An increased concern for data quality; - Some learning on model use; - And, in the three ‘on-model’ case studies, pathways for possible model evolution have been proposed (as described in previous sec- tions.
7.2 Methodological suggestions from stakeholder on-model engagement in PRIMA
From a methodological point of view, we also learned about the possible fu- ture operationalisation of on-model stakeholder engagement with respect to the kind of model used in our workshops. We can consider two broad catego- ries of models: probabilistic model on one hand and behavior-based models on the other. However when presenting a model, the meta-information about the type of model is never given and anyway would be difficult to understand. This induces in the interactions taking place in the participatory modelling process some risk of confusion between the types of model, i.e. places for misinterpretations. The second type of model is easier to communicate, but the model presented in PRIMA is of the first type. Therefore despite the rela- tively good understanding reported from the workshops, due to carefully translating the model’s content, feedback from stakeholders did not fit com- pletely the model’s type, implying a need for interpretation. Control of this interpretation by emitters of feedbacks was not ensured in our setting, and should be enabled in further experience, depsite being problematic.
A second methodological lesson concerns the issue of scale or granularity. With a local scale model, stakeholders are able to identify gaps between the model and their perception of reality. The feedbacks from them are then to- wards a higher model precision in order to reach a better fit with their appre- ciation of local nuances. While modellers aim at generating contrasted situa- tions so that discussions on policies could emerge, stakeholders aim at cap- turing local nuances which are important to them and which they don’t see in the model. Even if we paid attention to this issue in the presentation of the model’s rationale and objective, we need to be even clearer on what can be expected from the model and in determining stakeholders’ requirements from the modelling activity. With this kind of model, the interaction process should implement a kind of ‘ interpretation buffer stage ‘, in which interpreta- tions of the propositions of all participants could be discussed and validated. This stage could be useful as well for modellers to better qualify the validity and generality of stakeholders’ perceptions, and provide an opportunity to question them.
A tension, generated by the gap between basic dynamics and formal represen- tation, leads to a third methodological outcome related to handling stake-
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 49/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
holders’ beliefs and perceptions in a participatory modelling process. This issue is not specific to PRIMA’s situation, but it is exacerbated, because stakeholders come with time-specific and place-specific perceptions they consider are appropriate for the model’s granularity. For example they ask for new dimensions they believe are important in the dynamics to be added to the model, such as ‘level of comfort in housing’. However data do not always exist to characterize these additional elements properly in the model. To cope with the type of model, some new categories and transitions have to be identi- fied and calibrated. To go on with the example of level of comfort: housing is currently described only by its size, whereas a 2 or 3 level attribute (low, standard, premium) for comfort level could be added. This implies it is also necessary to elaborate transitions for one housing unit between these levels if we want to cope with a housing redevelopment policy and set the probabili- ties for a household to settle in a specific level of comfort housing, for which no data are available. The perception of the system’s boundaries is another source of misunderstanding, with stakeholders considering functional rela- tions while researchers consider data reporting units. Although difficult to solve, these tensions could enlarge participants’ knowledge, and have already proved useful in adjusting the dynamics of the participatory modelling pro- cess. It has led modellers to revise the model’s spatial extent to be able to cope with the functional ties identified by stakeholders.
Participatory modelling, and participation in general, generates better mutual understanding among participants and even empathy. When evaluating the outcomes of a participatory process, biases due to this empathy make unsuit- able any assessments based solely on questionnaires administered at the end of the process. Further, due to time constraints in PRIMA, there has been only a one-shot interaction, providing little possibility for stakeholders to step back, and formulate a sound critique of the model. Participatory modellers have a common concern of getting true and relevant critiques. Further, being part of a research project, stakeholders have low incentives to point out weaknesses in any models presented. This leads then to the following ques- tion: ‘how wrong should the model be before stakeholders comment that it does not match their broad expectations?’ To overcome this limitation, we suggest that in future projects, more repeated interactions should be imple- mented and a metrics generated to assess changes in the model. This could not be done in PRIMA due to time constraints.
A fifth methodological outcome concerns the level of complexity of the mod- el’s presentation. We aimed in the workshop to provide an exhaustive presen- tation, which was time-consuming and not necessarily required by stakehold- ers. Strengthening trust in the model (e.g. through data validation) and focus- ing on interfaces (inputs, outputs) rather than on the underlying mechanics was more aligned to stakeholders’ expectations. Participatory modelling in this case aims at involving stakeholders in the modelling activity, which they consider should be the sole responsibility of the modellers and field scientists. Several steps during modelling, as implemented in companion modelling (Bousquet et al., 2002), could provide ways to improve the process: an initial more conceptual round where stakeholders can frame what could be in the model and also ‘play’ with it, and a second stage, more technical, where the modellers are alone in charge of producing the model, interacting with stake- holders on interfaces.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 50/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
7.3 Model Brokering: a set of activities to be made explicit
We crafted earlier a concept-role of the ‘model broker’, being a “knowledge broker” (Brown and Duguid, 1998; Pawlowski and Robey, 2004) who focuses on knowledge about a model, knowledge generated by a model or knowledge to be incorporated into a model. As a subcategory of knowledge broker, a model broker facilitates communication between multiple communities who have their own experience in modelling, and in the system being represented in the model. In an interaction, the model broker helps make explicit all the interpretations generated in relation to a model. The broker then gathers crucial activities in a participatory modelling process: explaining the model content, putting words on simulation outcomes to make stories from them, formalizing reactions to the model’s contents and outcomes to enable updates in the model to be generated, etc. In PRIMA, the communities involved fit into four groups: modellers, stakeholders or model users, field scientists, and model administrators. Modellers consist of people who formalized the model design and implemented it. Field scientists are people whose purpose is to better understand the system dynamics. They can provide knowledge to the modellers or help them in structuring the main assumptions and are interest- ed in simulation outcomes. Stakeholders are the potential target population: those who are involved in the system management and who are very knowl- edgeable about this system. This group is very diverse and, as we observed in the French case , this diversity should be cautiously handled as it may conceal important power relations. Model administration gathers people who can provide technical support to stakeholders in using the model.
Making explicit the role of model broker would ensure that attention is paid to a suitable interpretation process at all interaction levels, for example in implementing an interpretation stage as suggested above. The question is then who should enact this role of model broker. In two workshops in PRIMA a team member did it implicitly, which is not ideal as we could see, although it does have the advantage of a rather short pathway between stakeholders and the model. This needs to be further elaborated.
7.4 Scenario development as a key tool for stakeholder in- volvement
Among the tools and methods available to implement model brokering, sce- nario development is an efficient way to involve stakeholders in participatory modelling. Scenario development pushes participants to handle the model when they formulate the questions they expect it should be possible to frame within the model. Our observations in PRIMA are in line with the conclusions of the meta-analysis made by Voinov & Bousquet (2010). A workshop format would still be efficient, provided moderation is driven towards that objective, but an explicit intervention by a model broker would probably improve the usefulness of outcomes. Other means found in the literature to lead stake- holders to ‘play’ with a model and generate scenarios include the implemen- tation of scenarios within a role playing game (RPG) (Campo et al. 2010; Etienne et al. 2003; Barreteau et al.; 2001). However, this technique was
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 51/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
considered unsuitable for use with the kind of stakeholders we wanted to involve. Debriefing of simulations or game sessions is a complementary ap- proach of the latter, as used by Worrapimphong et al (2010)
Surprisingly, the feedback from scenario design into model design is hardly mentioned in papers reporting participatory modelling case studies. Fur- thermore, scenarios are often mentioned as ‘emerging from discussions’ (eg. in papers in the special issue of Environmental Modelling & Software). How- ever, these model adaptations are a practical answer to feedbacks collected from stakeholders, and their subsequent translation into model code should be explicit. Martinez-Santos & colleagues (2010) mention a joint design for both model and scenario. Further they report implicit expectations of scenar- ios that might be simulated in modelling choices (here implementation of a ground water flow model). Another notable exception is found in Simon & Etienne (2010), who mention explicitly that they adapted a model to be able to cope with scenarios designed using participative techniques, but are cau- tious in tracking these adaptations back. In contrast to the PRIMA case, translation is fully performed by modellers who also act as workshop facilita- tors. Their explicit (written) transcription ensures the possibility to track back from model to scenario. In the PRIMA case studies, this collection of con- straints to the model was instead performed by workshop facilitators and field scientists who then had to negotiate the possibilities for model adapta- tion with modellers. As a consequence this translation came after the interac- tion process in the workshop, inducing an increased distance between stake- holders and the modelling process. Another common outcome from PRIMA with other participatory modelling techniques addresses the type of model. A numeric model such as PRIMA or the groundwater flow (GFM) model of Martinez-Santos and colleagues (2010) requires more structured protocols for interactions because they are further from the common representations of stakeholders than other techniques such as Bayesian Belief network (BBN) or RPG.
Scenario development for the kind of model developed in PRIMA also pro- vides the opportunity to bring in and test the singularities suitable for the scale and granularity of the representation chosen. The model provides a substratum, as a workbench to test these singularities in the same way as it does policy choices.
Technically, two limits appear in the use of scenario development as a tool for participatory modelling. First, it is difficult to push stakeholders towards thinking of possible radical changes by which all possible pathways in simula- tions might be tested. Second, setting boundary conditions requires separa- tion of feedbacks from the simulated process. For example, the creation of jobs in the PRIMA model can be endogenous (i.e. generated through simula- tion) or an external input (i.e. embedded in scenarios as a boundary condi- tion). This distinction needs a deep understanding of the model, which is difficult to reach in a comparatively short workshop.
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 52/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
7.5 Conclusion
Finally, despite difficulties in implementing this part of the project, we could see that dedicated work to explain the model to stakeholders has the benefits of apparently increasing understanding of the model by stakeholders and generating feed backs from them. From these we could generate pathways for model revision.
We could also formalize better what takes place in this kind of participatory modelling process. This stakeholder involvement process constitutes a dia- logue between a trust in observations and a trust in perceptions of specific events. Someone is required to adopt the role of model broker in order to make more explicit the interpretations of perceptions and observations into the model.
Last but not least, implementation of this process is time consuming for all participants. Hence we suggest ‘keeping the participative component to the minimum essential’ in order to protect the integrity of participation as a common good!
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 53/57
Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
REFERENCES
Alcamo J. and Henrichs T. (2008). Towards guidelines for environmental scenario analysis. Chapter 2 in ed. Alcamo J. Environmental Futures: the practice of environmental scenario analysis. Amsterdam, Elsevier
Barreteau, O. ; Bousquet, F. ; Attonaty, J.-M. (2001). Role-playing games for opening the black box of multi-agent systems: method and teachings of its application to Senegal River Valley irrigated systems. Journal of artificial societies and social simulations vol. 4, n°2, http://jasss.soc.surrey.ac.uk/4/2/5.html
Bousquet, F., O. Barreteau, P. D'Aquino, M. Etienne, S. Boissau, S. Aubert, C. Le Page, D. Babin, and J.-C. Castella. (2002). Multi-agent systems and role games: an approach for ecosystem co-management. Pp. 248-285 in Complex- ity and Ecosystem Management: The Theory and Practice of Multi-agent Approaches, edited by M. Janssen. Cheltenham, U.K.: Edward Elgar Publish- ing
Brown, J.S. ; Duguid, P. (1998). Organizing Knowledge. California Manage- ment Review vol.40, n°3, pp.90-111.
Campo, P. C. ; Bousquet, F. ; Villanueva, T. R. (2010). Modelling with stake- holders within a development project. Environmental Modelling & Software vol.25, pp. 1302-1321
Etienne, M. ; Le Page, C. ; Cohen, M (2003). A step by step approach to build up land management scenarios based on multiple viewpoints on multi-agent systems simulations. Journal of Artificial Societies and Social Simulations vol. 6, n°2, http://jasss.soc.surrey.ac.uk/6/2/2.html
Martinez-Santos, P. ; Henriksen, H. J. ; Zorilla, P. ; Martinez-Alfaro, P. E. (2010). Comparative reflections on the use of modelling tools in conflictive water management settings: The Mancha Occidental aquifer, Spain. Envi- ronmental Modelling & Software vol.25, pp. 1439-1449
Pawlowski, S D.; Robey D. (2004). Bridging User Organizations: Knowledge Brokering and the Work of Information Technology Professionals. MIS Quaterly vol. 28, n°4, pp. 645-672.
Sargent R.G. (2009). Verification and validation of simulation models. ed. Rossetti M.D., Hill R.R., Johansson B., Dunkin A., Ingalls R.G.,Proceedings of the 2009 Winter Simulation Conference, Association for Computing Ma- chinery
Simon, C. ; Etienne, M. (2010). A companion modelling approach applied to forest management planning. Environmental Modelling & Software vol. 25, pp. 1371-1384
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 55/57 Contract no. 212345 | Deliverable no. D2.4.1 | 20/12/2011
Voinov, A.; Bousquet F. (2010). Modelling with Stakeholders. Environmental Modelling & Software vol. 25, pp. 1268-1281
Worrapimphong, K. ; Gajaseni, N. ; Le Page, C. ; Bousquet, F. (2010). A com- panion modeling approach applied to fishery management. Environmental Modelling & Software vol. 25, pp.1334-1344
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 56/57 Contract no 212345 | Deliverable no. DX.x | 20-Dec-11
GLOSSARY
Agent-based model An agent-based model (ABM) is a computational model for simulating the actions and interactions of autonomous individu- als in a network, with a view to assessing their effects on the sys- tem as a whole.
Conceptual model validation Determining that the theories and assumptions underlying the conceptual model are correct and that the model representation of the problem entity is ‘reasonable’ for the intended purpose of the model (Sargent, 2009).
Data validity Ensuring that the data necessary for model building, model evaluation and testing, and conducting model experiments to solve the problem are adequate and correct (Sargent, 2009).
Elucidating stakeholder perspectives with regard to assessing agent-based model outputs: methodologies & results 57/57