Certification Test: Simulated Interaction

Forecasting Decisions in Conflicts using Simulated Interaction A Self-Certification Course

Last modified on 18 January, 2007 Designed by: J. Scott Armstrong and Kesten C. Green Reviewed by: Paul Goodwin

Simulated interaction was developed to forecast the decisions that people will make in conflict situations such as buyer-seller negotiations, employer-union disputes, commercial competition, hostile takeover bids, civil unrest, international trade negotiations, counter-terrorism, and warfare. These situations can be characterized as conflicts involving a small number of parties that are interacting with each other. There is often a great deal of money at stake in such situations and, in the case of counter-terrorism and warfare, lives. And yet predictions are typically made using unaided judgment. Simulated interaction provides forecasts that are more accurate than the methods that are currently used.

The Course includes the following materials: Course Checklist Pre-Course Test Certification Examination Certificate of Completion Course Guide Method Guide Examination Answers

Course Checklist Self-learning courses have been found to be more effective than traditional courses taught in classrooms. For effective learning, you must follow the steps described in the Course. You can do so when and where you like. This is also a self-certification course. In order to obtain your Certificate, you will be asked the six questions in the Checklist shown below. Your answers to the questions are your testimony as to your diligence, effort, and success in fulfilling the Course’s self-learning requirements. In other words, you will be asked to sign that you have followed the Course as designed or, if not, to indicate how you modified the Course.

Please keep track of your progress using this Course Checklist: 1. I completed all tasks prescribed in the Course. 2. I kept a diary of my learning in which I described what I did and what I learned from each task. 3. I completed the Pre-Course Test and the Certification Examination before looking at the Model Examination Answers. 4. I fully understand the answers that I gave and could explain them to another person. 5. The time I spent on the tasks described in the Course Guide was… minutes 6. The number of times I applied the principles covered in this Course to a real problem times was… One of my applications was…

Note that the Certification Examination is only one aspect of the Course. Completing the Examination will not in itself develop your skills in the use of simulated interaction. The Certificate of Completion (on the last page of this document) allows you to provide a record of your efforts in completing the Course. Course Guide Forecasting Decisions in Conflicts using Simulated Interaction

The numbers on the right of this Guide indicate the recommended minimum times to allow for the tasks and for diary entries. As you work through the Course, keep a diary of what you did, what you learned, and how you intend to apply what you learned. As you can see, we recommend at least seventeen hours to become certified in this area. Record the actual time you spent on each task as you complete it.

Minimum Actual (minutes) 1. Answer the questions in the Pre-Course Test and print a copy. 20 2. Familiarize yourself with conflictforecasting.com. 60 3. Study the following and, as you do, make plans for small applications. Armstrong, J. S. (2001). Role Playing: A method to forecast decisions. In Armstrong, J. S. (Ed.), Principles of Forecasting: A handbook for researchers and practitioners. Norwell, MA: Kluwer Academic Publishers, 15–30. Green, K. C. (2002). Forecasting decisions in conflict situations: A comparison of game theory, role-playing, and unaided judgement. International Journal of Forecasting, 18, 321–344. 150 For additional information, see Green, K. C. (2005). Game theory, simulated interaction, and unaided judgment for forecasting decisions in conflicts. International Journal of Forecasting, 21, 463–472. Green, K. C. & Armstrong, J. S. (2005). The war in Iraq: Should we have expected better forecasts? Foresight, 2, 50-52. Green, K. C. & Armstrong, J. S. (2007). The Ombudsman: The value of expertise for forecasting decisions in conflicts. Interfaces. 4. Study the conflict descriptions provided on conflictforecasting.com under 60 Resources for Researchers so that you can write similar descriptions. 5. Use the Method Guide to help you prepare materials to use in applying the 480 simulated interaction method to at least one real forecasting problem. 6. Conduct a simulated interaction session to forecast the decision that will be 120 made in at least one real situation. 7. Take the Certification Examination. 30 8. Use your Certification Examination answers to grade your Pre-Course Test 10 answers using grades A through F. 9. Open the Examination Answers file and use the answers in that document to 10 grade your Certification Examination answers using grades A through F. 10. Develop and record the action steps you will take in order to obtain the 20 knowledge to improve on your weak answers. 11. Complete your actions steps. 60

Total minutes 1020 Pre-Course Test Forecasting Decisions in Conflicts using Simulated Interaction

Answer the following questions as best you can before looking at the course material. You will be able to refer back to your answers to this pre-course test after completing the course to assess how much you have learned.

1. Under what conditions is unaided expert judgment unlikely to provide accurate forecasts?

2. Explain the differences between role playing and simulated interaction.

3. Under what conditions (e.g. type of situation) would simulated interaction improve accuracy?

4. What can be done to reduce forecast errors when using the simulated interaction method?

5. “In order for simulated interaction to be effective, it is necessary to closely match role players with the actual decision makers in the target situation.” True False

6. Describe how to use simulated interaction to forecast decisions in conflict situations.

7. What are the desirable characteristics of a conflict description?

8. “The primary reason simulated interaction works is that it gets a person to think like the other party. That is, it puts them in the other person’s shoes.” True False

9. Describe real problems where simulated interaction could help to improve forecast accuracy.

10. How many simulated interaction forecasts are needed to get a reliable forecast? (Discuss)

Method Guide Forecasting Decisions in Conflicts using Simulated Interaction

Forecasting the decisions people will make in a conflict situation with the simulated interaction method involves four main steps: (1) describe the roles of the key people in the conflict, (2) describe the target situation, (3) simulate the situation, and (4) derive forecasts.

(1) Describe the roles of the people in the conflict Describe the roles of the main decision-makers involved in the target conflict, or describe the roles of two people to represent each party. Using at least two people to represent a party can increase realism by making interactions within parties possible. Role descriptions need only be brief: a short paragraph should be sufficient. If the cost is not great, allocate roles to people who are somewhat similar to the actual people or type of people involved, otherwise assign roles at random. Ask the role players to read the description of their allocated role and to adopt it for the duration of the simulation.

(2) Describe the target situation Before you start writing your description, determine what must be forecast in order to help the client make the best decision. When the forecast requirement is clear, write a description of the situation. Descriptions of about one page in length are sufficiently long to include information about the parties to the conflict and their goals, relevant history, current positions and expectations, the nature of the parties’ interaction, and the issue to be forecast. Longer descriptions and unnecessary detail are likely to overburden role players. Provide an objective description of the situation. Avoid speculation about the motives and objectives of people in the conflict and, instead, let the facts, including public statements, speak for themselves. A comprehensive and accurate description will help role players to conduct realistic simulations. A good understanding of the situation and considerable care is advisable at this stage. To this end, obtain information from independent experts, have the description reviewed, and test the material. This process requires many rewrites and may consume much calendar time. Because we are concerned here with forecasting decisions, the situation description should leave role players expecting that they will have to reach a decision. In order to obtain a forecast, it may be necessary to invent a meeting of the parties or a deadline that will occur during the simulation. If the relevant options can be identified, provide a list of possible decisions for role-players to choose from with the situation description. Decision options should be relevant to the client’s own decision-making requirements, so check with the client to learn how the forecasts will relate to decisions. (By keeping the client involved, you also increase the likelihood that the forecasts will aid in decision making.) A short list of decisions provides clarity for role players who are unfamiliar with the type of situation they are simulating, ensures forecasts are useful, and makes coding easier. Based on research to date, we recommend that the list should include no more than four decision options. Three decision options—in essence Party-A’s favored decision, Party-B’s favored decision, and a compromise decision—often works well. In cases where a more specific forecast is required, break the forecasting task into parts and forecast each part. For example, having forecast that the decision favored by Party-B is made, use simulated interaction to forecast what if any concessions are made to Party-A. Because the decisions might be sensitive to how the situation is described, consider conducting the simulation with alternative descriptions. To ensure independence, the alternative description should be made by another team. If it is feasible, test the materials—role and situation descriptions—by conducting and observing simulations, and interviewing the role players after each simulation. Change the materials in order to eliminate any problems that you identify.

(3) Simulate the situation Provide realistic settings, if this is affordable. In many cases, a meeting room and breakaway rooms for each party will be suitable. Ask the role players to improvise and interact with others in ways that are consistent with the target situation. In many cases, interactions involve face-to-face meetings. Exchanging information using notes or networked computers, or conducting telephone conversations might be more realistic for other situations. Allow one hour for a simulation or parallel simulations. Use fresh role players in order to avoid the risk that tired role players will harm forecast accuracy.

(4) Derive forecasts For each simulation, the forecast is the final decision made by the role-players. Role players should each describe the decision that was made in their simulation or, preferably, chose from a list of potential decisions that was provided to them the one that was made in their simulation or which was closest to the decision they made.

Conduct as many as ten simulations, in parallel if this is feasible. Combine the decisions recorded by the role players from the independent simulations. For example, if seven out of ten simulations end in a decision to go to war, one would predict war. Certification Examination Forecasting Decisions in Conflicts using Simulated Interaction