One World, Two Classrooms, Thirteen Days: Film As an Active Teaching and Learning Tool in Cross-National Perspective

ONE WORLD, TWO CLASSROOMS, THIRTEEN DAYS: FILM AS AN ACTIVE TEACHING AND LEARNING TOOL IN CROSS-NATIONAL PERSPECTIVE

Cristina Yumie Aoki Inoue Instituto de Relações Internacionais Campus Universitário Darcy Ribeiro Universidade de Brasilia – Asa Norte Caixa Postal 04306 – CEP 70919 – Brasilia, DF (Brazil) Email: [email protected] Phone: 55 61 3107 3644 or 31073637

and

Matthew Krain Department of Political Science The College of Wooster Wooster, OH 44691, USA Email: [email protected] Phone: 1 330 263 2469

ABSTRACT This study assesses the pedagogical value of film as case material, and whether that value is affected by the different national and institutional contexts of the students engaging that text. We test whether students in two different Theories of International Relations classrooms – one in Brazil and one in the United States – demonstrated a statistically significant, different level of increased knowledge, understanding, or ability to apply theoretical frameworks to real world cases. Students in each class cover the same theoretical approaches using the same focused readings on theories of foreign policy decision-making, and then view the same film – Thirteen Days – to which they apply these theoretical frameworks. Using a quasi-experimental design, we use pre- and post-test surveys or “quizzes” to directly measure students’ knowledge about the case, and ability to apply IR theories to it. We also measure film’s utility indirectly by asking questions about students’ perceptions of their own understanding of the cases, and their perceptions of the utility of film as a tool in the IR classroom on those surveys. The film selected is a U.S. film about a U.S. case, making the comparative effect of the film on Brazilian and U.S.-based students particularly interesting.

AUTHORS’ NOTES: The authors would like to thank the ISA for its Catalytic Workshop Grant to fund the 2012 ISA Workshop “Think Globally, Teach Locally? Active Teaching and Learning in Cross-National Perspective,” at which this project was first devised. We would also like to thank Stephanie T. Sugars for her research assistance on this project. This research was performed under College of Wooster Human Subjects Research Committee Approval HSRC 03-1213. Replication data for this article can be found at http://discover.wooster.edu/mkrain/research

Forthcoming: Journal of Political Science Education 2

Active teaching and learning (ATL) approaches have been applied regularly in international studies classrooms in the United States since the 1990s. In Lantis, Kille, and Krain’s (2010) survey of the literature on ATL in international relations (IR) that draws on work primarily done in the United

States, the authors identify a wide range of benefits, including: enriching student understanding of key concepts in international affairs; creating powerful and effective learning environments by encouraging students to take risks and to express their views on complex and controversial issues; and increasing the retention of knowledge.

In other countries, such approaches have evolved more recently as IR has emerged as a distinct field of inquiry and study. For instance, in Brazil a more comprehensive reflection on active teaching and learning was initiated with a workshop for Brazilian instructors and advanced students, hosted at the University of Brasilia’s Instituto de Relações Internacionais in October 2006. The workshop gave a broad overview about many active teaching and learning techniques, and provided a forum for critical reflection on instructional approaches in international studies and work as teachers and educators (Kille, Krain and Lantis 2008). It has also led to a wider and more systematic usage of these techniques in Brazilian classrooms (Inoue 2012). However, there are no published studies yet about either the extent of the evolution of active learning in Brazil, or whether the students have benefited from a more student-centered approach.

Attempts to compare the effects of active teaching and learning cross-nationally are also scarce. A recent one was the Think Globally, Teach Locally? Active Teaching and Learning in Cross-National

Perspective ISA Catalytic Workshop 2012. During the workshop there was a very lively and rich discussion on active teaching and learning within several national and institutional contexts, the benefits of active teaching and learning, and challenges faced by each instructor. Also, the need for in-depth comparative analysis was identified in order to assess whether the same active teaching and learning approaches have similar effects cross-nationally. This paper is a response to these identified 3 needs. It aims at comparing active teaching and learning in two different national contexts: the

United States and Brazil. We chose to compare the use of film in IR theory classes in both countries.

The film that we chose addresses an oft-used case in the study of IR more generally, and IR theories more specifically – the Cuban Missile Crisis. It has been fifty years since the Cuban Missile

Crisis, and yet for scholars of IR the case remains indispensible (Cyr 2013). Key theoretical arguments about conflict, disputes, deterrence, great power rivalry, and of course foreign policy decision-making have been developed or tested using the events of those dramatic thirteen days in

October 1962. And the case remains a staple pedagogical tool in the IR classroom. The drama of the case, its high profile actors, and the level of detail available about the case make it particularly useful for demonstrating how to apply different theoretical lenses to real world cases, and how to use those cases to test different theories’ hypothetical expectations.

Indeed, if anything the popularity of the case as a pedagogical tool has only risen over time, most notably after the release of a popular film about the case – Thirteen Days – that lays out the events of the crisis (from the United States’ perspective) in tense, dramatic fashion. Recent research has demonstrated the utility of case studies in general, and film in particular, in the IR classroom.

What we want to know is whether an American film focused on U.S. decisionmaking during a U.S.-

Soviet crisis has different pedagogical effects in both U.S. and non-U.S. classrooms. In short, we ask whether the pedagogical value of the film as text is affected by the different national and institutional contexts of the students engaging that text.

To address this question, we test whether students in two different Theories of IR classrooms – in Brazil and the United States – demonstrated a statistically significant, different level of increased knowledge, understanding, or ability to apply theoretical frameworks to real world cases. Students in each class covered the same theoretical approaches using the same focused readings on theories of foreign policy decision-making, and then viewed the same film, to which 4 they applied these theoretical frameworks. Using a quasi-experimental design, we used pre- and post-test surveys or “quizzes” to directly measure students’ knowledge about the case, and ability to apply IR theories to it. We also assess film’s utility indirectly by asking questions about students’ perceptions of their own understanding of the cases, and their perceptions of the utility of film as a tool in the IR classroom on those surveys. After seeing the film and answering the post-test, we facilitated discussions in each class in order to obtain a more in depth assessment of students’ views and understandings. The discussions also served for clarifications on theoretical concepts, and reflections on the applications of theories to real world case and on the use of film as a pedagogical tool

FILMS AS ACTIVE TEACHING AND LEARNING TOOLS

Active Teaching and Learning

The approach employed in this study, active teaching and learning, is a student-centered approach designed to produce learning, develop critical thinking skills, and elicit discovery and the construction of knowledge. This process of discovery situates students as critical thinkers in a collaborative and experiential learning environment. It also tasks them with being both consumers and generators of knowledge (Dewey 1938; Barr and Tagg 1995).

While the evidence is mixed as to whether active teaching and learning leads to greater short- term knowledge acquisition than more traditional approaches (Albanese and Mitchell 1993; Lieux

1996; Powner and Allendorfer 2008), this kind of collaborative learning involving real-world applications has been shown to promote a deeper understanding of key concepts, arguments, or cases

(Dewey 1938; Kolb 1984; Krain and Lantis 2006; Lantis 2012). When students actively engage with the material they are more likely to learn, understand, and retain that knowledge (Schachter, 1996; 5

Jensen 1998; Brown and King 2000; Kuzma and Haney 2001; Krain and Nurse 2004). Research in cognitive psychology and education has found that material learned experientially and through a combination of approaches (visual and verbal; heard, expressed and acted upon) tap into multiple senses and emotions and create “memorable events.” These have been found to yield greater knowledge retention across a range of students with different learning styles (Paivio 1975; Schacter

1996; Jensen, 1998; Prince 2004).

Studies also consistently show that active learning approaches enhance student critical thinking and problem-solving skills, and the ability to transfer their knowledge to new, complex and uncertain, situations (Bonwell and Eison 1991; Lieux 1996). And active learning has been shown to help students develop a sense of personal efficacy and the willingness to take risks in expressing and acting on their ideas (Krain and Nurse 2004). For a more extensive discussion of the wide-ranging pedagogical effects of active teaching and learning in the IR classroom, see Lantis, Kille and Krain

(2010).

Case Learning and the Use of Film

One of the most frequently employed approaches in IR classrooms is what is sometimes referred to as the case learning approach (Angelo and Boehrer 2002; Krain 2010). This approach allows students to engage abstract concepts while exploring specific real-life cases. Case learning is particularly useful for demonstrating the connection between theory and practice, and for building critical thinking and problem-solving skills (Lamy 2007; Prince 2004).

Case learning exercises are usually designed and employed in order to enhance, rather than supplant, lectures. These class sessions are opportunities for students to apply understanding gained from articles or lectures to real world cases. They can include exercises in the analysis of decision- making (as is the case in our study), but can also include critical thinking exercises played out in 6 student debate and discussion (Lamy 2007; Marks 2008). In discussions of theories and texts, the instructor plays the role of discussion facilitator during the in-class exploration of a particular case.

She or he uses questions and prompts to get students themselves to outline the parameters of the case or problem, explore its relevant theoretical and policy issues, and evaluate the (potential) outcomes (Golich et al. 2000; Lantis, Kille and Krain 2010). Student-centered reflection, in which students have the opportunity to discuss their understanding of the case, allows both students and instructors to connect active learning experiences back to a larger theoretical context. Such reflection is crucial to the learning process, and is the key place where vital connections are made between cases examined or student experiences and the theories or arguments that they have explored

(Lantis, Kille, and Krain 2010).

Films are powerful case learning tools because they take advantage of the reasons why active learning techniques tend to be most successful. Film and video, if used effectively, may be particularly useful in creating memorable multi-sensory experiences that enable students to

“experience” problems or events half a world away, or in the past. Indeed, recent studies have found that film has real, measurable impacts on student learning in large part because of its ability to create such “memorable moments” for students, and to dramatize abstract theoretical concepts in visual representations of real world situations (Kuzma and Haney 2001; 2002; Krain 2010; Lantis 2012;

Swimelar 2013). Films can also be better at eliciting empathy in students by humanizing participants and/or making seemingly distant events or issues seem more “authentic” or “real” (Kuzma and

Haney 2001; 2002; Swimelar 2013). And they tend to be particularly successful case study tools when couched within an analytical framework, and/or as a way to explain and criticize theories (Engert and Spencer 2009).

What is less clear is whether the pedagogical context affects film’s effectiveness as an active teaching and learning tool. Efforts to evaluate these approaches’ appropriateness and effectiveness 7 in diverse national and institutional contexts remain rare. In addition, much of the published work to date is focused on assessing innovative pedagogical approaches within the U.S. context, ignoring the potential for systematically examining cross-national variation of pedagogical innovations in IR.

Cross-National Education

Although there is growing attention to cross-national learning, the scholarship of teaching and learning (SoTL) remains divided over whether there are significant impediments to the exchange of different educational activities across borders or to direct engagement of classes and students across borders. A number of studies suggest that differences in national or cultural orientations can have profound effects on the production and consumption of knowledge. Content, teaching styles, student expectations, learning cues, and learning styles may all be nationally or culturally specific

(Campbell and Li 2008; Holtbrügge and Mohr 2010; Pettenger, West, and Young 2013). As a result, some scholars argue that cross-national education cannot be universalized, and that approaches to teaching and learning need to be adapted to the unique contexts of national environments (Bond and Park 1991; Pimentel 1995; Morgan 2000).

Some teacher-scholars have even cited instances specifically relative to our endeavor

(cooperation between instructors in a U.S. and a Brazilian institution) where obstacles exist to cross- national education. Crabtree and Sapp (2004) maintain that students have very different expectations in the Brazilian educational environment from what U.S. professors and students do. They characterize potential barriers as asynchrony in cultural environments that translate into different learning styles and produce different outcomes (see also, Ginsburg 2010).

Moreover, there is always the concern that student culture may not value active teaching and learning techniques because they are used to a particular style of education. For instance, in some national contexts, students may perceive innovative teaching techniques in a negative light—reacting 8 nervously to new approaches that press for greater critical and analytical thinking or even as an indication of faculty laziness instead of providing information via the traditional lecture format

(Inoue 2012). The impact of different levels of student motivation across different national contexts might suggest that active learning techniques might operate differently and yield different educational outcomes in different contexts.

Conversely, some maintain that national differences do not inhibit cross-national education, or that they might even provide opportunities for exploration of difference and the promotion of critical thinking. Kille, Krain, and Lantis (2008) find, contrary to Crabtree and Sapp (2004), that instructors and students in Brazil were highly receptive to a range of innovative approaches in the discipline, and discussions of how the approaches would work best within new cultural contexts proved quite enriching. Storrs (2009) and McMahon and Zou (2011) relate similar experiences regarding cross-national teaching and learning in Japan and China, respectively. And relevant to our study of the role of film, some teacher-scholars contend that advances in pedagogy, including more instrumental uses of technology, can help overcome national or cultural barriers to learning.

Given recent interest in more rigorous research on the effect of film as a pedagogical tool, and disagreement as to the effects of national context on the success of active teaching and learning techniques, a study examining the pedagogical value of film as case material, and whether that value is affected by the different national contexts of the students engaging that text, seems warranted. We now turn to the procedures that we employ to conduct our study.

METHODOLOGY

Students in two different Theories of IR classrooms – one in Brazil and one in the United States – covered the same theoretical approaches using the same focused readings on theories of foreign 9 policy decision-making, and then view the same film – Thirteen Days (2000) – to which they apply these theoretical frameworks. We employ a quasi-experimental design with pre- and post-test surveys or “quizzes” to directly assess students’ knowledge about the case, and ability to apply IR theories to it. We supplement this analysis with closed- and open-ended questions about students’ perceptions of their own understanding of the cases, and their perceptions of the utility of film as a tool in the IR classroom on those surveys, as a way to measure the film’s utility indirectly.

The classes were surprisingly similar despite being a continent apart. Both were undergraduate courses required as part of an IR bachelor’s degree, but open to other interested students. The US class at a private small liberal arts college in Wooster, Ohio, contained 25 students, ranging from second to fourth year students who were, on average, 20 years old. The class in Brazil, at a public research university in Brasilia, contained 20 students drawn primarily from their second year at college, on average 19 years of age. Despite being at different types of institutions, the degree programs, course backgrounds, previous course content, student ages, student/faculty ratio in the classroom, and seminar style of the class were similar across both classrooms.

Students read and discussed Putnam’s (1998) article on analyzing foreign policymaking as a series of two-level games. Both classes then read Allison's (1969) classic article about the Cuban

Missile Crisis just before the film viewing, although each class did not discuss it before viewing the film. Students took our pre-test in the class session for which the Allison reading was due, but just before viewing the film. That way the pre-test captured their understanding of the case in light of their reading of Allison, but not influenced by the film. The classes then viewed the film Thirteen

Days (2000). In the next class session immediately following the film showing, students took the post-test at the start of class, before the discussion in class about the texts. That way the post-test captured the effects of the film, but not the discussion. Table 1 lists descriptive statistics for the pre- and post-tests for classes. The actual pre- and post-tests are available in Appendix A. 10

<< Table 1 about here >>

After the post-test we discussed the three texts (the articles and the film) in the context of a larger discussion about how we might apply theory to understanding the case. During that discussion we analyzed student comments for trends in their use of the theoretical arguments to understand the case. Each instructor acted as discussion facilitator during the class, using questions and prompts to get students themselves to outline the parameters of the case and how and why it played out as it did, explore its relevant theoretical and policy issues, and evaluate the (potential) outcomes. Debates over differing interpretations about the issues at stake, choices available to decision-makers, or optimal solutions were actively encouraged.

Data from the pre-tests and post-tests were compared for each class, using paired-samples difference of means t-tests. We used paired samples t-tests in this study in all instances where the samples being compared are scores for the same group before and after the introduction of the stimulus (the learning experience). In these instances, the pre- and post-test scores are ‘‘paired’’ for each person surveyed. Observations and groups being compared are not independent of each other, making an independent samples t-test inappropriate. We looked to see whether there was a statistically significant difference between student scores before and after the viewing of the film.

We also analyzed the differences of the means of the two classes – one in the United States and one in Brazil. For this analysis we used employed independent samples t-tests because the observations and groups being compared are independent of each other. Here we looked to see whether the student knowledge gains or insights were indistinguishable from each other regardless of national, cultural or institutional context, or whether these factors affected students’ learning outcomes.

RESULTS

Initial Comparison of Both Classes Prior to Introduction of the Stimulus (Film)

Prior to the introduction of the stimulus (the film), difference of means t-tests for independent samples show that there was no statistically significant difference between the Brazilian Theories of

IR class and its counterpart in the United States on most questions that we asked, with a few minor exceptions (see Table 2). Among the quiz-like multiple choice questions in the first part of the survey, which attempt to directly assess understanding of the theories and their application to the

Cuban Missile Crisis, the students in the United States were better able to correctly identify the correct sequence of foreign policy decision-making according to the Rational Actor Model, and as described in the Allison article (Q1) than their Brazilian counterparts [t(31.029) = -2.602, p < 0.01].

On the other questions testing understanding, (Q’s 3, 5 & 7) there was no significant difference between the two classes prior to the introduction of the film.

We also asked students (Q2) to rank on a scale of 1 to 5 (1 = most important, 5 = least important) which factors they felt played an important role in foreign policy decision-making. The

U.S. students ranked leaders’ unique characteristics as more important than did the Brazilian students [Q2a – t(37.876) = -2.455, p < 0.01] while the Brazilian students ranked organizational processes and routines as more important than did the students in the U.S. classroom [Q2c – t(32.186) = 1.984, p < 0.05]. There was no significant difference between the two classes’ pre-film rankings of leaders’ decision-making processes, internal politics and the bureaucracy, and the interrelationship between domestic and international politics.

<< Table 2 about here >>

Part II of the survey was an attempt to indirectly assess student opinions and attitudes. On all of these questions examining their understanding of the case and the theories, and their opinions of the theories and their applicability to real world instances and to instances beyond the U.S. case examined here, there was no significant difference between the answers given by students in the two classes (Q’s 9-16). There was also no difference between the Brazilian and U.S. classes in terms of their prior exposure to the Allison (1969) article (Q17), and regarding their assessment of the utility of the Allison article in helping to understand how the Cuban Missile Crisis played out (Q18).

However, the students in the class in Brazil found the Allison article slightly more helpful as a tool for understanding multiple theoretical models (Q19) than did the students in the U.S. classroom

[t(33.753) = 1.859, p < 0.05].

Finally, in the post-test given just after the film, we asked those final three questions (Q’s 17-

19) again, but this time about the film Thirteen Days. There was no significant difference between classes in terms of their prior exposure to the film (Q17). There was also no significant difference between the Brazilian and US classes regarding their assessment of the utility of the film in helping to understand how the Cuban Missile Crisis played out (Q18), or as a case study that could be analyzed using multiple theoretical models (Q19).

In sum, these two classes were remarkably similar despite being continents apart. With a few exceptions, they answered questions regarding their knowledge, understanding and opinions, and their experience with the materials used in a very similar fashion, yielding what we believe to be highly comparable groups. Therefore, we now move confidently to examine the effects of the film

Thirteen Days on the students and the learning outcomes in both the Brazilian and the American classroom.

Effects of the Stimulus (Film)

Effects on Learning in the U.S. Classroom: Analysis of Closed-Ended Questions

In the U.S. classroom, direct assessment revealed no change in student knowledge from simply having viewed the film (Table 3). Paired samples difference of means tests revealed that there was no significant difference between the answers given before (n=24) and after (n=22) the showing of the film Thirteen Days on the questions directly assessing understanding of the theories and their application to the Cuban Missile Crisis (Q’s 1, 3, 5 & 7). There was also no significant change in the students’ rankings of which factors they felt played an important role in foreign policy decision- making as a result of having viewed the film.

On the questions examining their understanding of the theories, and their opinions of the theories and their applicability to real world instances, there was no significant difference between the answers given by students pre- and post-film on Q’s 11-16. The U.S. students did report an increase (from a mean of 3.2 [“3 - some”] to 3.9 [“4 - high”] on Q9) in their overall understanding of the relevant details of the Cuban Missile Crisis after having viewed the film [t(45) = -4.7308, p <

0.01], and a similar increase (from a mean of 3.2 [“3 - some”] to 3.9 [“4 - high”] on Q10) in their understanding of the Kennedy Administration’s decision-making during the Cuban Missile Crisis

[t(45) = -3.615, p < 0.01].

Overall, the U.S. students ranked the film (mean of 4.5 out of 5.0) as more helpful than the article (3.5 out of 5.0) in understanding the case itself [Q18 – t(44)= -5.46, p < 0.01]. In sum, the film Thirteen Days appears to have had more of an effect on the U.S. students’ understanding of how the case played out, and on specifics of the case (as depicted in the film) than on their understanding of IR theories, and their opinions of these theories and their applicability to real world cases.

<< Tables 3 and 4 about here >> 14

Effects on Learning in the Brazilian Classroom: Analysis of Closed-Ended Questions

Direct assessment in the Brazilian classroom (Table 4) also revealed no change in student knowledge from simply having viewed the film. Paired samples difference of means tests revealed that there was no significant difference between the answers given before (n=15) and after (n=16) the showing of the film Thirteen Days on the questions directly assessing understanding of the theories and their application to the Cuban Missile Crisis (Q’s 1, 3, 5 & 7). And as with the U.S. students, there was also no significant change in the students’ rankings of which factors they felt played an important role in foreign policy decision-making as a result of having viewed the film.

On the questions examining their understanding of the theories, and their opinions of the theories and their applicability to real world instances, there was no significant difference between the answers given by students pre- and post-film on Q’s 11, and 13-16. As with the U.S. classroom, the Brazilian students reported an increase (from a mean of 3.1 [“3 - some”] to 4.0 [“4 - high”] on

Q9) in their overall understanding of the relevant details of the Cuban Missile Crisis after having viewed the film [t(29)= -4.230, p < 0.01], and, a similar increase (from a mean of 3.1 [“3 - some”] to

3.7 [“4 - high”] on Q10) in their understanding of the Kennedy Administration’s decision-making during the Cuban Missile Crisis [t(29)= -2.061, p < 0.05]. The lone difference was in responses to

Q12, in which the students in Brazil reported an increase in support (from a mean of 2.9 [“3 - neutral”] to 3.6 [“4 - support”]) for the following statement: “The Rational Actor Model accurately characterizes foreign policy decision-making during crises” [t(27)= -1.823, p < 0.05].

Overall, the students in Brazil ranked the film (mean of 4.5 out of 5.0) as more helpful than the article (3.8 out of 5.0) in understanding the case itself [Q18 – t(28)= -3.209, p < 0.01]. The film appears to have had a greater effect on the Brazilian students’ understanding of how the case played out, and on specifics of the case (as depicted in the film) than on their understanding of IR theories, and their opinions of these theories and their applicability to real world cases. 15

Comparative Analysis of Open-Ended Questions

In general, the data from the pre-and post- tests show more similarities than differences between the two classes. As mentioned, for both sets of students the film seemed to have had a greater effect in their understanding of the case itself than on their understanding of theories. From the open ended questions, the initial impression about whether IR theories of foreign policy-decision making are appropriate for understanding cases of foreign policy decision-making outside the U.S., show that both the Brazilian and the U.S. based-students are roughly evenly split in their opinions. Many in both classes believe that IR theories in general, and Allison’s models in particular, can be applied to other circumstances and other countries, while many others think that because the majority of the theories were developed in the global North and that the authors we read were primarily North-

Americans, that theories are more appropriately applied to explaining U.S. foreign policy and IR for

(North) Americans.

Allison’s and Putnam’s articles were important to the exercise as ways to introduce the following ideas: that the state can be seen as something different than a black box, or an unitary actor; that the domestic level can be take into consideration in IR analysis; and that multiple theoretical models can be applied to the same case. Throughout the open-ended responses students appropriately employed the language of both articles (i.e. - “win-sets”, “Level I” vs. “Level II”,

“standard operating procedures”), suggesting that they had absorbed some of the more complex concepts embedded in the works. In the last open-ended question on the pre-test, asking what lessons students took away from Allison’s article, students in both classes highlighted that the article was interesting for demonstrating the importance of a particular model. Some (in particular in the

Brazilian class) emphasized that they now better understood the role of organizations, and how significant they can be in decision-making. Others noted that the article did a good job of 16 demonstrating how foreign policy decisions are not always rational or based solely on assessment of national interest. Many in both classes pointed to the importance of considering more than one model, or of “the ability to analyze the same fact by different lenses”.

In the post-test we asked the same open-ended question, this time about the film. The answers indicate that it was important to watch the film after reading Allison’s article, as students could then analyze the film having Allison’s concepts in mind. The film was also important in helping students consider how other actors and organizations influence decision-making, as well as the role and importance of individual leaders. It helped students get a sense of the urgency, speed and seriousness of a crisis situation and of how difficult it is to arrive to a decision. Almost all of the comments suggested, just as the previous responses had indicated, that students learned more about the details of the case, and were able to better understand the details and intricacies of the foreign policymaking process, as a result of having seen the film. It brought these details to life for them in a way that the article could not. The comments about the utility of the film were distributed roughly evenly across the two classes such that it is impossible to detect differences in patterns in the responses based on national context.

Comparative Analysis of Classroom Discussions

In the class session following the viewing of the film, students first took the post-test, and then we engaged in a class discussion. Recall that the literature on active teaching and learning suggests that experiential learning is incomplete without discussion and debriefing afterwards, as much of the learning that occurs happens as students reflect on integrating knowledge from texts, class, and their experiences (Lantis, Kille, and Krain 2010). Class discussions in both the U.S. and Brazil focused on applying theories of IR to understanding foreign policy decision-making more generally, and the

Cuban Missile Crisis case specifically. 17

Based on their statements during the in-class discussion, the students in the U.S. classroom enjoyed the use of the film to bring the case to life, and to allow them to deploy the theoretical arguments from the Putnam and Allison articles in a real case. Most students in the U.S. class knew of the case, but had not realized the level of complexity of the decision-making process, which they highlighted as a particularly useful contribution of the film. They also identified other ideas that had not been emphasized in the articles as having been introduced by the film account, including the constraining role of international norms, and the possible threats to the rational actor model at the individual level of analysis, such as Kennedy’s family ties and history, and the individual personalities and operational codes of the participants.

In reviewing the different types of ways of explaining foreign policy decision-making, the

U.S. students constantly referenced the film version of the case explicitly (for instance: “in the scene where…”, or “Kennedy looked tired”, or “when the Navy commander and MacNamara are arguing…”) as they supported their preferred argument. And for every thirteen explicit and appropriate and correct references to the film in class discussion students referenced the articles just once. Students in the U.S. classroom were also able to see how these theories could be applied beyond this case, and beyond U.S. foreign policy decision-making in general. The discussion also helped them to make more explicit connections between the articles and the case, and allowed them to test out alternative theoretical explanations for events or decisions in our classroom conversation.

During the discussions in Brazil the students noted that they liked the experience of having seen a film as a part of the course, and liked the film itself. They mentioned that Thirteen Days help them learn more about the Cuban Missile Crisis case, and the decision-making process in general.

Many students noted that before the film they did not know many details about the case, nor did they understand its complexity. One of the students pointed out that he also now understood better the difficulties in the relations between the U.S. and Cuba. In Brazil, most of the opinions tend to be 18 biased towards seeing Cuba as a “victim” of the U.S. The film helped him see the situation from the

U.S. perspective, and understand the larger set of issues from another angle.

Some Brazilian students also noted during the discussion that they did not fully understand

Allison’s Models II and III after reading his article. The discussion helped them clarify the differences and applications of the models in this case and others. One student was impressed about how well he could see Models II and III operating when he viewed the film, and noted that it was very important to first read the article and then watch the film. This same student argued that

Allison’s models and Putnam’s two-level game could also be applied to the Brazilian context, and that it was important to have read them because they broadened his perspectives of theories, actors and levels of analysis. The facilitator of the discussion in the Brazilian classroom made sure to point out the applications of Models II and III in the film, and in a few cases involving Brazilian foreign policy. In the end, the students evaluated that it was a good experience.

In Brazil, the class discussion was important to clarify some concepts that were not clear for the all the students, and to call their attention to some aspects to which they did not pay attention in first place like the role of organizations and their standard procedures, or how individuals in different bureaucracies compete and have their own objectives. Another important theme was whether theories that originate in one context could be used and applied in other contexts. In the beginning, most of the students seemed to have a more restrictive view, and focused more on the difficulties of transcending contextual differences. As the discussions unfolded, they soon realized that, despite the differences, there are many concepts that make sense and can be applied in either the U.S. or Brazilian context.

The discussions also brought to light some students’ difficulties, especially with language and time constraints. As most of the readings are in English, some of the Brazilian students mentioned that they take longer to read and understand a reading, and the period between one class and the 19 other is not enough to assimilate as much as they would like. In Brazil, the students received the pre and post-tests in English. The decision to not translate was made to avoid ambiguities generated by translation of the questions. Perhaps the translation would have helped the students, as it seems that a few of them did not understand the questions as well as they might have otherwise. They did answer in Portuguese, and the translation of their answers to English was not an easy task as well.

We cannot be 100% sure that we got the full essence of their words, although we are certain that we captured their main points and most nuances.

CONCLUSION

This study aimed to explore the effects of active teaching and learning cross-nationally. More specifically, this study assessed the pedagogical value of film as case material, and whether that value is affected by the different national and institutional contexts of the students engaging that text. We tested whether students in two different Theories of IR classrooms – one in Brazil and one in the

United States – demonstrated differences in knowledge acquisition, understanding, or ability to apply theoretical frameworks to real world cases. We used a pre- and post-test design to assess the effects of the film Thirteen Days (2000) on student understanding of theories of foreign policy decision-making and their application to the Cuban Missile Crisis case.

Among our findings is the surprising result that national context did not appear to affect the utility of film as an active teaching and learning tool, even when the film was about a U.S.-produced film about a U.S.-specific case. Students in the classroom in Brazil and in the United States started out the experiment, after reading relevant texts but before the film, as very similar in their initial level of understanding of the case. Moreover, they appear to have experienced the same educational outcomes in similar ways. In sum, national context had little effect on the educational outcomes 20 resulting from using this film as an active learning exercise to apply theories of IR to understand foreign policy decision-making.

We also found that what was learned via the use of film differed from what was learned using more traditional and less “engaging” texts. Consistent with the literature’s recent findings, active teaching and learning techniques do not yield more, or less, knowledge or understanding, but rather lead to different and often deeper or case-specific understanding (Krain and Lantis 2006;

Lantis 2012). Students in neither classroom experienced overall gains in their understanding of the theoretical concepts and how they could be applied after having watched the film. However, students in both classrooms (across two continents) reported that the film helped them to better understand the complexity of foreign policy decision-making, the difficulty of crisis decision-making, and the facts and complexity of the specific case. These results were evident in closed-ended responses, open-ended responses, and in the in-class discussion following the post-test.

The way in which the active learning exercise was employed also mattered. In particular, class discussion was an essential part of the utility of using the film as case learning exercise, confirming much of what the literature suggests. Without this crucial step, students would not have reflected on and processed the material as effectively. While students may not have gained in their knowledge of the theories and their application after seeing the film, the subsequent debriefing and discussion enabled them to clarify the ideas, and helped them more effectively apply the range of theoretical concepts and arguments to a real world case. Another benefit of the class discussion was in our ability to assess the exercise’s effectiveness, and their overall learning as a result of it. More specifically, we would not have understood as well what they learned, when they had learned it, what they thought of the exercise, and what impediments to success existed.

While national context had little effect, institutional constraints, some of which are national in origin, did affect the ability of instructors in different settings to engage in effective teaching using 21 the active learning approach. The most obvious institutional constraint that arose during our experiment was a major strike lasting 100 days that occurred in Brazil. The strike affected the majority of federal public universities in Brazil. Due to the strike, the semester that was scheduled to start in August began at the end of October. So, the exercise for the IR Theories class was delayed three months, making conversation between the US and Brazilian classes impossible. In addition, the students were already very tired and not motivated in classes, as they had to spend their whole

“summer vacation” in the classrooms. The general mood and motivation of the students were low.

Thus, during the whole IR Theories course, the instructor had to prepare exercises and games to get them more involved and motivated.

It is also important for us to emphasize that films are useful, but must be chosen, framed, and used with care and intentionality. Students do not react similarly to all case materials, even of the same type, such as films (Swimelar 2013). Also, because students claimed that they learned details of the case from this particular film, it is important to be sure to clearly frame films used in classes as

“accounts” of events that may have been (slightly) fictionalized, or to use documentary films when possible. It is also important to remember that films are not made as teaching tools, and may distort facts or narratives for the sake of the “product” (Engert and Spencer 2009). Scholars, even those intimately familiar with the history of the crisis, have noted that the film Thirteen Days does a very good job overall of depicting many historical details, the general thrust of events, and sense of crisis.

Yet many also note that the film depicts only one country’s perspective on the events, and gets some key details wrong (Allison 2001; May 2001; Engert and Spencer 2009; Stern 2012). If students see films as accurate depictions of historical events, their understanding of the facts of the case will be affected adversely (Giglio 2002). We tried to guard against this by reiterating Allison’s (2001) comment, “The film is not a documentary. Rather, it is a dramatization.”. Moreover, we do not believe that this affected the outcome of our study, which relied more on student understanding of 22 theoretical models used in the context of the case. Yet the recent debate about the fictionalized and inaccurate depiction of the use of torture to extract useful information in Zero Dark Thirty (Coll

2013) and the less than accurate portrayal of the hostage rescue in Argo (Dargis and Scott 2013) highlight that this concern is one that instructors who use films must continue to consider as they employ this tool.

Finally, it is important to emphasize that active learning is an “attitude” towards teaching, so it cannot be defined only by the “techniques used” but by the way we approach teaching and learning and how we involve the students in our courses. In Brasilia, during the final course evaluation discussions, students could point their difficulties, what they learned, what was important for them. They underlined that the film Thirteen Days, and the other games and exercises used during the course, were highlights of the class, and helped them learn. One student made a point about the importance of students feeling as if they are being heard, and are part of the construction of their own knowledge. These sentiments were echoed in the evaluations of the Theories of IR class in the

U.S. as well, with a particular emphasis on the importance of both applying theory to practice via analysis of real world cases, and the utility of dialogue and discussion about texts, theories, and cases in helping students to better understand these complex ideas and their applicability. Perhaps, the more dialogical way of teaching is the very nature of active learning and that is what makes a difference in the students learning process.

APPENDIX A. ASSESSMENT TOOLS FOR PRE- AND POST-TESTS

From time to time, I will conduct an assessment of teaching and learning in this class. In order to assist me in determining how well readings and class sessions meet our educational objectives, please complete the following survey. Please note that these surveys are anonymous.

PART I

_____ 1. Which best represents the ideal sequence of foreign policy decision-making according to the Rational Actor Model (Model I), as discussed by Graham Allison in his article?

A. goal selection → problem recognition → identification of alternatives → choice → implementation B. goal selection → choice → implementation → problem recognition → identification of alternatives C. problem recognition → goal selection → identification of alternatives → choice → implementation D. problem recognition → identification of alternatives → goal selection → choice → implementation

2. Which of these factors you think play an important role in foreign policy decision-making? Rank order the five factors listed below (1 = most important; 5 = least important).

_____ Leaders’ unique characteristics (identities, operational codes, personalities) _____ Leaders’ decision-making processes _____ Organizational processes and routines _____ Internal politics of the government and its bureaucracy _____ The inter-relationship between domestic and international political choices

_____ 3. Which of the following theoretical frameworks seems to best explain the first few meetings of the U.S. President’s “ExComm” (Executive Committee), during which alternatives for action were discussed?

A. Graham Allison’s Rational Actor Model (Model I) B. Graham Allison’s Organizational Process Model (Model II) C. Graham Allison’s Bureaucratic Politics Model (Model III) D. Robert Putnam’s Two-Level Game Model

4. Please briefly explain your answer to question 3 (above)

_____ 5. Which of the following theoretical frameworks seems to best explain the decision-making surrounding the U.S. naval blockade, or “quarantine” against Soviet ships bound for Cuba?

6. Please briefly explain your answer to question 5 (above)

_____ 7. Which of the following theoretical frameworks seems to best explain President Kennedy’s considerations of both his and Nikita Khrushchev’s domestic constraints and how they might have affected the possible choices available to them both?

8. Please briefly explain your answer to question 7 (above)

PART II: Please answer the following questions by circling the most accurate answer

9. How would you rate your overall understanding of the relevant details of the Cuban Missile Crisis?

1—very low 2—low 3—some 4—high 5—very high

10. How would you rate your understanding of the Kennedy Administration’s decision-making during the Cuban Missile Crisis?

1—very low 2—low 3—some 4—high 5—very high

11. How would you rate your understanding of IR theories that attempt to explain foreign policy?

1—very low 2—low 3—some 4—high 5—very high

12. How do you feel about the statement?: “The Rational Actor Model accurately characterizes foreign policy decision-making during crises.”

1—very opposed 2—opposed 3—neutral 4—support 5—strong support

13. How do you feel about the statement?: “Theories of international relations can be useful tools in helping us understand how and why foreign policy decisionmaking occurs.”

1—very opposed 2—opposed 3—neutral 4—support 5—strong support

14. How do you feel about the statement: “IR theories of foreign policy decision-making are appropriate for understanding cases of foreign policy decision-making that occur outside the US context”?

1—very opposed 2—opposed 3—neutral 4—support 5—strong support

15. Please briefly explain your answer to question 14 (above)

16. How would you rate your level of personal interest in the use of theories of international relations to try to understand real-world cases or phenomena?

1—very low 2—low 3—some 4—high 5—very high

17. Had you read Graham Allison’s article “Conceptual Models and the Cuban Missile Crisis” before taking this class?

0—never 1—more than a year ago 2—within the last year

18. How helpful was the article in assisting you in understanding the way in which the Cuban Missile Crisis played out?

1—useless 2—not very helpful 3—somewhat helpful 4—helpful 5—very helpful

19. How helpful was the article as a case study that could be analyzed using multiple theoretical models?

1—useless 2—not very helpful 3—somewhat helpful 4—helpful 5—very helpful

20. What was the most compelling thing that you learned from Graham Allison’s article?

NOTE: AFTER THE FILM, SUBSTITUTE Q’s 17-20 BELOW FOR THE ONES ABOVE!!!

17. Had you seen the film Thirteen Days before taking this class?

0—never 1—more than a year ago 2—within the last year

18. How helpful was the film in assisting you in understanding the way in which the Cuban Missile Crisis played out?

1—useless 2—not very helpful 3—somewhat helpful 4—helpful 5—very helpful

19. How helpful was the film as a case study that could be analyzed using multiple theoretical models?

1—useless 2—not very helpful 3—somewhat helpful 4—helpful 5—very helpful

20. What was the most compelling thing that you learned from the film Thirteen Days?

Table 1. Descriptive Statistics – Responses to Questions on Pre- and Post-Tests

Pre-Test: Theories of IR Class, Brazil Pre-Test: Theories of IR Class, U.S.A.

Mean Std. Err. N Mean Std. Err. N Q1 0.267 0.118 15 Q1 0.667 0.098 24 Q2a 4.600 0.190 15 Q2a 3.800 0.265 25 Q2b 3.333 0.303 15 Q2b 2.960 0.261 25 Q2c 2.733 0.300 15 Q2c 3.520 0.259 25 Q2d 2.333 0.303 15 Q2d 2.360 0.190 25 Q2e 2.000 0.324 15 Q2e 2.360 0.321 25 Q3 0.533 0.133 15 Q3 0.696 0.098 23 Q5 0.267 0.118 15 Q5 0.136 0.075 22 Q7 0.429 0.137 14 Q7 0.524 0.112 21 Q9 3.067 0.182 15 Q9 3.240 0.105 25 Q10 3.067 0.228 15 Q10 3.160 0.160 25 Q11 3.357 0.169 14 Q11 3.520 0.143 25 Q12 2.929 0.305 14 Q12 3.280 0.178 25 Q13 4.267 0.182 15 Q13 4.000 0.153 25 Q14 3.600 0.273 15 Q14 3.960 0.147 25 Q16 3.733 0.248 15 Q16 3.833 0.155 24 Q17 0.533 0.133 15 Q17 0.435 0.138 23 Q18 3.786 0.187 14 Q18 3.500 0.135 24 Q19 4.308 0.133 13 Q19 3.913 0.165 23

Post-Test: Theories of IR Class, Brazil Post-Test: Theories of IR Class, U.S.A.

Mean Std. Err. N Mean Std. Err. N Q1 0.438 0.128 16 Q1 0.773 0.091 22 Q2a 4.438 0.273 16 Q2a 3.591 0.320 22 Q2b 3.813 0.164 16 Q2b 3.045 0.283 22 Q2c 2.625 0.328 16 Q2c 3.318 0.311 22 Q2d 2.000 0.224 16 Q2d 2.409 0.234 22 Q2e 2.125 0.315 16 Q2e 2.636 0.312 22 Q3 0.625 0.125 16 Q3 0.682 0.102 22 Q5 0.133 0.091 15 Q5 0.227 0.091 22 Q7 0.267 0.118 15 Q7 0.714 0.101 21 Q9 4.000 0.129 16 Q9 3.864 0.749 22 Q10 3.688 0.198 16 Q10 3.864 0.100 22 Q11 3.313 0.176 16 Q11 3.727 0.972 22 Q12 3.600 0.214 15 Q12 3.273 0.164 22 Q13 4.313 0.151 16 Q13 3.909 0.112 22 Q14 3.438 0.128 16 Q14 3.682 0.121 22 Q16 4.133 0.165 15 Q16 3.864 0.178 22 Q17 0.375 0.125 16 Q17 0.727 0.150 22 Q18 4.500 0.129 16 Q18 4.455 0.109 22 Q19 4.000 0.158 16 Q19 3.955 0.139 22

Table 2. Independent Samples Difference of Means t-Tests: Both Classes, Pre-Test Mean Difference SE Difference Q1 -0.400 ** 0.154 Q2a 0.800 ** 0.326 Q2b 0.373 0.400 Q2c -0.787 * 0.397 Q2d -0.027 0.358 Q2e -0.360 0.456 Q3 -0.162 0.166 Q5 0.130 0.140 Q7 -0.095 0.177 Q9 -0.173 0.210 Q10 -0.933 0.279 Q11 -0.163 0.221 Q12 -0.351 0.353 Q13 0.267 0.237 Q14 -0.360 0.310 Q16 -0.100 0.293 Q17 0.099 0.192 Q18 0.286 0.230 Q19 0.395 * 0.212 * p < 0.05, ** p < 0.01; See Appendix A for Question wording

Table 3. Paired Samples Difference of Means Table 4. Paired Samples Difference of Means t-Tests – Class in U.S.A., Pre- vs. Post-Test t-Tests – Class in Brazil, Pre- vs. Post-Test Mean Difference SE Difference Mean Difference SE Difference Q1 0.106 0.135 Q1 0.171 0.175 Q2a -0.209 0.412 Q2a -0.163 0.337 Q2b 0.085 0.385 Q2b 0.479 0.339 Q2c -0.202 0.402 Q2c -0.108 0.446 Q2d 0.049 0.299 Q2d -0.333 0.374 Q2e 0.276 0.450 Q2e 0.125 0.451 Q3 -0.014 0.141 Q3 0.092 0.183 Q5 0.091 0.118 Q5 -0.133 0.149 Q7 0.190 0.151 Q7 -0.162 0.180 Q9 0.624 ** 0.132 Q9 0.933 ** 0.221 Q10 0.704 ** 0.195 Q10 0.621 * 0.301 Q11 0.207 0.178 Q11 -0.045 0.246 Q12 -0.007 0.244 Q12 0.671 * 0.368 Q13 -0.091 0.194 Q13 0.046 0.235 Q14 -0.278 0.194 Q14 -0.163 0.295 Q16 0.030 0.235 Q16 0.400 0.298 Q17 0.292 0.203 Q17 -0.158 0.183 Q18 0.955 ** 0.175 Q18 0.714 ** 0.222 Q19 0.042 0.217 Q19 -0.307 0.213 Notes: Mean Difference = Post-Test – Pre-Test Notes: Mean Difference = Post-Test – Pre-Test * p < 0.05, ** p < 0.01 * p < 0.05, ** p < 0.01 See Appendix A for Question wording See Appendix A for Question wording

REFERENCES

Albanese, Mark A., and Susan Mitchell (1993). Problem-Based Learning: A Review of Literature on its Outcomes and Implementation Issues. Academic Medicine 68: 52-81.

Allison, Graham T. (2001). “Cuban Missile Crisis: Thirteen Days Analyses” Belfer Center for

Science and International Affairs, Harvard University. < http://www.cubanmissilecrisis.org/for- educators/media-depictions/thirteen-days-analyses/> (Accessed November 19, 2013).

Allison, Graham T. (1969). Conceptual Models and the Cuban Missile Crisis. American Political Science

Review 63, 3: 689-718.

Angelo, Thomas and John Boehrer (2002). “Case Learning: How Does It Work? Why Is It

Effective?” Adapted from “Questions and Answers About Case Learning,” by Thomas V. Bonoma,

1990. (Accessed March 22, 2010).

Barr, Robert B. and John Tagg (1995). From Teaching to Learning: A New Paradigm for

Undergraduate Education. Change 27: 13-25.

Bond, D., and J. Park. (1991). An Empirical Test of Rawls’ Theory of Justice: A Second Approach in

Korea and the United States. Simulation and Gaming 22(4): 443-462.

Bonwell, Charles C., and James A. Eison (1991). Active Learning: Creating Excitement In The Classroom.

ASHE-ERIC Higher Education Report No. 1. Washington, DC: George Washington University. 30

Brown, Scott W. and Frederick B. King (2000). Constructivist Pedagogy and How We Learn:

Educational Psychology Meets International Studies. International Studies Perspectives 1: 245–254.

Campbell, J., and M. Li (2008). Asian Students’ Voices: An Empirical Study of Asian Students’

Learning Experiences at a New Zealand University. Journal of Studies in International Education 12 (4):

375-39.

Coll, Steve. (2013). ‘Disturbing’ and ‘Misleading’: Zero Dark Thirty. The New York Review of Books.

February 7, 2013. Accessed March 21, 2013.

Crabtree, Robbin D., and David Alan Sapp (2004). Your Culture, My Classroom, Whose Pedagogy?

Negotiating Effective Teaching and Learning in Brazil. Journal of Studies in International Education 8 (1):

105-132.

Cyr, Arthur I. (2013). The Cuban Missile Crisis after Fifty Years. Orbis 57, 1: 5-19

Dargis, Manohla and A. O. Scott. (2013). Critics’ Notebook: Confronting the Fact of Fiction and the

Fiction of Fact. New York Times. February 23, 2013, C1.

Dewey, John (1938). Experience and Education. New York: Collier Books.

Engert, Stefan and Alexander Spencer (2009). International Relations at the Movies: Teaching and

Learning about International Politics through Film. Perspectives. 17, 1: 83-103.

Giglio, Ernest (2002). Using Film to Teach Political Concepts. European Political Science 1, 2: 53-58.

Ginsburg, Mark B. (2010). Improving Educational Quality Through Active-Learning Pedagogies: A

Comparison of Five Case Studies. Educational Research 1(3): 62-74.

Holtbrügge, Dirk and Alexander T. Mohr (2010). Cultural Determinants of Learning Style

Preferences. Academy of Management Learning & Education 9(4): 622-637.

Inoue, Cristina Yumi Aoki (2012). “Active Learning in Brazil – The University of Brasília Case.”

Paper presented at the Catalytic Workshop on Active Teaching and Learning in Cross-National Perspective,

Annual Convention of the International Studies Association, San Diego, California, March 31, 2012.

Jensen, Eric (1998). Teaching With the Brain in Mind. Alexandria, VA: Association for Supervision and

Curricular Development.

Kille, Kent J., Matthew Krain and Jeffrey S. Lantis (2008). Active Learning Across Borders: Lessons

From An Interactive Workshop in Brazil. International Studies Perspectives 9(4): 411-429.

Krain, Matthew (2010). “The Effects of Different Types of Case Learning on Student Engagement.”

International Studies Perspectives 11(3): 291-308.

Krain, Matthew and Jeffrey S. Lantis (2006). Building Knowledge? Evaluating the Effectiveness of the Global Problems Summit. International Studies Perspectives 7: 395-407.

Krain, Matthew and Anne Nurse (2004). Teaching Human Rights through Service Learning. Human

Rights Quarterly 26: 189-207.

Kolb, David A. (1984). Experiential Learning: Experience as the Source of Learning and Development.

Englewood Cliffs, NJ: Prentice Hall.

Kuzma, Lynn M. and Patrick J. Haney (2001). And…Action! Using Film to Learn About Foreign

Policy. International Studies Perspectives 2: 33-50.

Kuzma, Lynn M. and Patrick J. Haney (2002). Comments by Kuzma and Haney: Using Film in the

Classroom. International Studies Perspectives 3: 92-94.

Lamy, Steven L. (2007). Challenging Hegemonic Paradigms and Practices: Critical Thinking and

Active Learning Strategies for International Relations. PS: Political Science and Politics 40: 112-116.

Lantis, Jeffrey S. (2012). “War and Peace on Film.” In Teaching Politics Beyond the Book: Film, Texts and

New Media in the Classroom. Edited by Robert W. Glover and Daniel Tagliarina, pp. 233-250. New

York: Bloomsbury.

Lantis, Jeffrey S., Kent J. Kille and Matthew Krain (2010). “The State of the Active Teaching and

Learning Literature.” In The International Studies Encyclopedia, Volume X, General Editor, Robert A.

Denemark, pp. 6574-6592. Malden, MA: Wiley-Blackwell.

Lieux, Elizabeth M. (1996). A Comparative Study of Learning in Lecture Versus Problem-Based

Learning. About Teaching 50: 25-27.

Marks, Michael P. (2008). Fostering Scholarly Discussion and Critical Thinking in the Political

Science Classroom. Journal of Political Science Education 4: 205-224.

Masoner, Michael. (1988). An Audit of the Case Study Method. New York: Praeger.

May, Ernest R. (2001). “Thirteen Days in 145 Minutes” The American Prospect. 12, 1 (Jan. 1-15): p.34.

McMahon, Patrice C. and Yue Zou (2011). Thirty Years of Reform and Opening Up: Teaching

International Relations in China. PS: Political Science & Politics 44(1): 115-121.

Morgan, Konrad (2000). Cross-Cultural Considerations for Simulation-Based Learning

Environments. Simulation & Gaming 31(4): 491-508.

Paivio, Allan (1975) “Coding Distinctions and Repetition Effects in Memory.” In The Psychology of

Learning and Motivation, Volume 9. edited by G.H. Bower, pp. 178-214. New York: Academic Press.

Pettenger, Mary, Douglas West, and Niki Young (2013). Assessing the Impact of Role Play

Simulations on Learning in Canadian and US Classrooms. International Studies Perspectives, (published online 11 November 2013) doi: 10.1111/insp.12063.

Pimentel, Florosito Q. (1995). Gaming and Simulation: A Third World Experience. Simulation &

Gaming 26(4): 480-488.

Powner, Leanne C. and Michelle G. Allendoerfer (2008). Evaluating Hypotheses about Active

Learning. International Studies Perspectives 9: 75–89.

Prince, Michael (2004). Does Active Learning Work? A Review of the Research. Journal of Engineering

Education 93: 223–31.

Putnam, Robert (1988). Diplomacy and Domestic Politics: The Logic of Two-Level Games,

International Organization 42(3): 427-460.

Schacter, Daniel (1996). Searching for Memory: The Brain, the Mind, and the Past. New York: Basic Books.

Stern, Sheldon (2012). The Cuban Missile Crisis in American Memory: Myths versus Reality. Stanford

University Press.

Storrs, Debbie (2009). Teaching Mills in Tokyo: Developing a Sociological Imagination Through

Storytelling. Teaching Sociology 37(1): 31-46.

Swimelar, Safia (2013). Visualizing International Relations: Assessing Student Learning Through

Film. International Studies Perspectives 14, 1: 14-38.

Thirteen Days (2000). New Line Cinema. Director: Roger Donaldson.