How Do Experts and Novices Think About Climate Change? Thinking Routines as Learning and Assessment Tools
The Harvard community has made this article openly available. Please share how this access benefits you. Your story matters
Citation Chua, Flossie. 2016. How Do Experts and Novices Think About Climate Change? Thinking Routines as Learning and Assessment Tools. Doctoral dissertation, Harvard Graduate School of Education.
Citable link http://nrs.harvard.edu/urn-3:HUL.InstRepos:27112690
Terms of Use This article was downloaded from Harvard University’s DASH repository, and is made available under the terms and conditions applicable to Other Posted Material, as set forth at http:// nrs.harvard.edu/urn-3:HUL.InstRepos:dash.current.terms-of- use#LAA
How do experts and novices think about climate change?
Thinking routines as learning and assessment tools
Siew Geok Flossie Chua
Dissertation Advisor: David Perkins Committee Member: Paul Harris Committee Member: Veronica Boix Mansilla
A Thesis Presented to the Faculty of the Graduate School of Education of Harvard University in Partial Fulfillment of the Requirements for the Degree of Doctor of Education
2016
© 2016 Siew Geok Flossie Chua All Rights Reserved
Dedicated to my mother and my sisters Florence and Freda, whose unswerving belief in me, and unstinting love and support, have made this dissertation possible
i Acknowledgements
"We are like dwarfs sitting on the shoulders of giants. We see more, and things that are more distant, than they did, not because our sight is superior or because we are taller than they, but because they raise us up, and by their great stature add to ours." - John of Salisbury
The journey to completing my dissertation has been a confirmation of how many intellectual giants I have had the privilege of knowing and working with, and how I stand tall today only because they have raised me up.
The first group of giants I thank is my dissertation committee: David Perkins, Paul Harris, and Veronica Boix Mansilla. All of them have been incredible models of what it means to be intellectually rigorous, deeply thoughtful, and profoundly ethical. Because of them, I understand why educational research matters, and how it should matter.
As my adviser, Dave always took my work seriously, and I never felt alone in trying to navigate the sometimes forbidding and often bewildering waters that doctoral students have to swim in. He always made ample time to think with me about my research, and encouraged me to tinker with ideas, however far out they might have seemed. Above all, his feedback was always insightful, and he provided countless productive provocations for me to think innovatively about my research.
Without Paul, my research design would be less coherent and interesting. Paul brought to my research a level of meticulousness, cogency, and credibility that made it possible to gather the kind of data necessary for me to meet my research goals. At all times, Paul was incisive in his feedback, judicious in his recommendations, and highly supportive whenever I encountered roadblocks. Above all, his wry good humor was invaluable in my moments of stress and distress!
Veronica is the reason why I even thought the doctoral path was possible for me. She saw potential in me way before anyone did, and throughout my years working with her at Project Zero, she has encouraged my ideas, given me opportunities to develop as a researcher, and been a wonderful mentor and friend. Her advice, guidance, and mentorship have shaped the way I think and work as a researcher, and I am forever indebted to her. I will not be the researcher that I am today without her.
The other group of giants I thank is my friends and colleagues at Project Zero. Whenever my confidence in my work flagged, and the light at the end of the tunnel dimmed, Margaret Rundle, Jordy Oakland, Liz Duraisingh, Mara Krechevsky, Sarah Alvord, Faith Harvey, Paromita De, and Melissa Rivard showed incredible faith in me, and encouraged me constantly with laughter and love. I could also depend on Daniel Wilson, Carrie James, and Shari Tishman for their unfailing support and sound advice; their anecdotes of their own dissertation journey were hugely instrumental in shoring up my defenses against self-doubt and anxiety.
ii I also owe my dissertation to the Harvard scientists and high school students who generously gave of their time to my study, and shared unreservedly their ideas, insights, and work with me. I have been moved and humbled by how deeply they care about leaving the room better than they found it, and greatly appreciate how they model what it means to be a contributing citizen of a highly complex world.
Finally, I thank my family – my mother, sisters Florence and Freda, and boyfriend Aaron Ellison – who inspired me to dare to dream and to soar, secure and confident that they would always be there as the wind beneath my wings. They have been my North Star in a world that I don’t always understand, during a time when I could have easily lost my way.
I am deeply grateful to have been surrounded by these extraordinary giants, who have allowed me to see more, do more, and be more.
iii Table of Contents
Abstract vi
Chapter 1: Research Rationale & Overview 1 Organization of the Thesis 2 Education & the Demands of the 21st Century 4 The Global Competence Matrix 7 Assessing Global Competence 10 Thinking Routines as Teaching & Assessment Instruments 16
Chapter 2: Review of Relevant Literature 22 Defining Global Competence 23 Thinking about Climate Change 27 Characterizing Expert & Novice Thinking 32
Chapter 3: Methods 38 Research Question 38 Research Design 39 Site & Sample 39 Expert Groups 40 Novice Groups 45 Data Collection 47 Tasks 47 TED Talk on Geoengineering 47 Task 1: Encoding 48 Task 2: Scenario 48 Task 3: Sourcing 49 Instrument Development 49 Expert Testing Review 50 Pilot Test 51 Data Analysis 55 Limits & Threats to Validity 57 Researcher Bias 57 Reactivity 57
Chapter 4: How Experts & Novices Think about Climate Change 59 How Experts & Novices Encode Information 59 Linear versus Nonlinear or Strategic Thinking 62 Descriptive versus Critical Syntheses 64 Concrete versus Abstract Concepts 65 How Experts & Novices Respond to a Scenario Using Thinking Routines 72 Prior to Connect-Extend-Challenge 72 Time Scales & Historical Lenses 73 Complexity of Systems & Weighing Consequences 74 Epistemic Framing 78
iv After Connect-Extend-Challenge 82 Connections 82 Extensions 84 Challenges 86 Prior to the 3Ys 89 Broader versus Narrower Conceptualization of Impact 89 Expansive versus Narrow Framing 91 Environmental Justice 93 After the 3Ys 94 Identity & Responsibility 94 Action & Personal Impact 96 Comparing Across Contexts 98 The Value of Thinking Routines in Supporting Thinking 101 Using Thinking Routines in General 102 Using Connect-Extend-Challenge 102 Taking a Wider & Longer Perspective 103 Complexity of Systems & Weighing Consequences 104 Epistemic Framing 104 Using the 3Ys 105 Seeking Other Sources of Information 108 Global/Local Orientation 111 Conclusion 114
Chapter 5: Towards More Expert Thinking Using Thinking Routines 115 How Experts Think About Climate Change 116 Geological Framing 117 Why Does This Matter? 119 Identity & Worldview 122 Why Does This Matter? 126 Epistemic Orientation 130 Why Does This Matter? 133 Thinking Routines & Globally Competent Investigations of the World 135 Connect-Extend-Challenge 136 3Ys 138 Assessment Potential of Thinking Routines 138 Implications of the Study 140 Limitations & Further Study 142
Appendix A Study Protocol 145
Appendix B Transcript of Edited TED Talk 148
Appendix C Coding Scheme (Initial & Revised Versions) 151
Bibliography 153
v Abstract
Developments in the last century – the global economy, unprecedented migration patterns, and the digital revolution – have forced a challenging shift in the way we think about what matters most to learn. As traditional systems of learning are substantially challenged and reshaped, consensus is building around the importance of educating for global competence. While much investment has gone into creating frameworks, curricular materials and activities for educating young people for global competence, there are few, if any, assessment tools for assessing students’ global competence that are viable for everyday classroom use.
My qualitative study addressed this gap by testing the extent to which thinking routines
– micro-interventions that focus attention on specific thinking moves that build global competence – might function as a viable instrument for supporting young people in developing more expert patterns of thinking when engaging with complex issues like climate change. Using a structured protocol with a group of 6 experts and 6 novices, I explored their patterns of thinking as they engaged with a scenario on climate change using two thinking routines.
My study revealed that the experts differed from the novices in three principal ways:
(1) the experts characteristically viewed climatic events and phenomena through geological time scales, which has important implications for supporting students to understand the shifting baselines for measuring change that tend to be at the heart of controversial and often bitterly contentious issues; (2) the experts reasoned from their identity and worldview as scientists with a moral responsibility to not only provide scientifically accurate information to the public, but also to do so in a responsible way; and (3) they recognized the
vi provisional nature of knowledge, and engaged in cognitively effortful processing of information that relied less on heuristics and more on culturally specific knowledge. My study also found the thinking routines to be effective as a teaching tool: they extended the novices’ substantive attention to the issues, and scaffolded them towards more critical reasoning and complexity in their thinking. The thinking routines also worked well as assessment tools, revealing trends in the novices’ thinking that a general prompt did not.
vii Chapter 1: Research Rationale & Overview
Our ability to survive and indeed thrive in the 21st century is contingent on our capacity to meet the demands that contemporary society makes on us. As schools and countries address the challenge of preparing young people to understand and participate in today’s highly interdependent and complex world, they find ample support from organizations and researchers who have advanced frameworks explicating 21st century competencies (i.e., what matters most to teach for today’s world), as well as curricular materials for nurturing those competencies in the classroom (i.e., what students will do to learn). Still, the answer to another equally pressing question remains elusive: how do we know that students are making progress1?
My study set out to address this last question by testing the extent to which thinking routines – micro-interventions that focus attention on specific thinking moves that build global competence – might function as a viable instrument for supporting young people in developing more expert patterns of thinking when engaging with complex issues like climate change. By drawing on the tradition of expert-novice research studies and a definition of global competence as the “capacity and disposition to understand and act on issues of global significance” (Boix Mansilla & Jackson, 2011, p. xiii), my study investigated the patterns of thinking that experts and novices demonstrate when they reflect on geoengineering, a controversial proposal to mitigate climate change, and then tested thinking routines as a tool for supporting and evaluating how they unpack the issue in globally competent ways.
1 These questions – what matters most to teach; what students will do to learn; and how will we know they are making progress – are from a set of 4 questions (What topics matter most to teachers? What exactly will students learn? What will students do to learn? How will we know that the students are making progress?) that Perkins (1992) calls “Pandora Questions.”
1 Specifically, I sought to understand the ways that thinking routines prompted experts and novices to develop and/or extend their capacity to investigate the world by: (i) generating questions about, and uncovering the significance of, a global issue; (ii) using sources to identify and weigh relevant evidence; (iii) analyzing, integrating, and synthesizing evidence; and (iv) developing a compelling argument and drawing defensible conclusions about climate change. Although thinking routines have been designed specifically to scaffold good thinking in learning contexts and not for assessment purposes, I believe their potential for making thinking visible can help teachers to assess how students think about a global issue, prior to and following instruction, and consequently allow them to make course corrections in their teaching where necessary. In my study, I define an assessment tool as a strategy for providing information about how learners are thinking, and how they might be supported towards more expert thinking. This study is a first step in uncovering the extent to which thinking routines may support and reveal students’ globally competent thinking.
Organization of the Thesis
In this chapter, I describe the demands of contemporary society on young people to be globally competent, and the educational responses from schools, educators and researchers. I also review current strategies and tools for assessing global competence, and explain why a more flexible tool such as thinking routines is needed to support teachers in uncovering where their students are at in terms of globally competent thinking, so that they might better design learning experiences to nurture global competence in their classrooms.
In Chapter 2, I examine the literature on the kinds of competencies and skills that have been proposed as key to thriving in our complex contemporary society. More specifically, I look at current definitions of global competence, and the tensions surrounding
2 the concept, and clarify an important distinction between 21st century skills and global capacities. I also summarize proposals by educators on how science education should prepare learners for grappling with the complexity of climate change and the kinds of competencies they have foregrounded in their recommendations. Finally, I review the rich tradition of research into expert-novice differences in various cognitive tasks and contexts.
In Chapter 3, I describe the design of my study: instrument validation using an expert panel; a pilot study to test the extent to which the interview protocol captured the dimension of the global competence framework under study; and the tasks in the interview protocol. I explain the sites selected for my study, as well as the selection criteria for the expert and novice samples. I identify the analytic strategies employed in answering my research questions. Finally, I acknowledge several potential threats to the validity of my study’s findings and conclusions, and explain the steps taken to minimize them.
In Chapter 4, I present the key findings from my study. I show how the experts and novices in my study performed across the different tasks in the protocol, highlighting the extent to which the use of thinking routines revealed and supported their thinking about geoengineering and climate change. My findings corroborate existing knowledge of expert- novice differences (e.g., experts think in more global and abstract ways while novices are more likely to focus on concrete details and to think in linear ways; experts reason from a solid knowledge base to explain complex causal relationships among variables and events, while novices create causal chains that are unidirectional and less complex), and offer insight into where and how thinking routines are more and less successful in supporting and assessing globally competent thinking in novices.
In Chapter 5, I describe three key features of the way that the experts in my study engage with the issues of climate change and geoengineering that are specifically related to
3 globally competent thinking: geological (versus ecological) framing of issues; reasoning from a clearly articulated identity and worldview; and epistemic orientation towards knowledge. I show how these three patterns of thinking compare with those of novices, and then propose why they matter as we prepare learners for our contemporary world. I also examine the extent to which the two thinking routines used in the study supported learners toward the patterns of thinking exhibited by the experts when engaging with complex issues like climate change, and explore how thinking routines may function as assessment tools. Then, I consider some implications that my findings have for preparing learners for more globally competent investigations of the world, acknowledge the limitations of my study, and propose some avenues for future research.
Education & the Demands of the 21st Century
The call to effectively prepare young people for the contemporary world that they are going to live, work and play in has dominated educational discourse for the past decade, if not longer. Scholars and organizations, schools and countries alike, have emphasized the importance of educating young people to meet the demands that current global trends make on individuals and societies. Whether in educational frameworks advanced to promote teaching and learning for the 21st century (Partnership for 21st Century Skills, 2008; Jerald,
2009; Lemke, 2002; Boix Mansilla & Jackson, 2011; Ananiadou & Claro, 2009), or in school models designed to establish a new educational paradigm to prepare students for the challenges of our contemporary world (e.g., Big Picture Schools; High Tech High; Studio
Schools; Quest to Learn), a common theme across these efforts has been the urgency to ensure that students are able to engage productively with real world issues, collaborate
4 broadly across cultural and language differences, and develop solutions to complex problems that impact the world we live in.
Frameworks for 21st century learning, such as those created by the Partnership for
21st Century Skills (www.p21.org), Center for Public Education
(www.centerforpubliceducation.org), and the Asia Society (www.asiasociety.org), emphasize equipping students to address complex global issues and participate in rapidly changing job markets. In the U.S., educational initiatives such as the Common Core State Standards
(www.corestandards.org) and the Next Generation Science Standards
(www.nextgenscience.org) anticipate that students’ ability to pose questions, think critically and creatively, communicate effectively, and apply their learning to address real-world problems will prepare them for a life and career in contemporary society. Countries like
Australia (www.acara.edu.au), China (Zhang, Lin, Wang & Wang, 2000), Singapore (Ministry of Education Singapore, 2010), Scotland (http://www.scotland.gov.uk/Publications/
2008/06/06104407/0), and Finland (Jääskeläinen & Repo, 2011), have restructured their educational systems and reframed curricular priorities to prepare their young to thrive in an era marked by globalization, economic interdependence, and rapid technological advancement, by focusing on inculcating respect for differences, nurturing an international perspective, and developing a sense of civic responsibility. Closer to home, the United States
Department of Education (2012) unveiled its international strategy for 2012-2016: strengthening U.S. education and advancing the nation’s international priorities, and most notably ensuring that all students develop global competencies.
Current trends in contemporary society demand new educational responses, and many have responded to the call by designing curricular frameworks, instructional strategies, and resources for teaching and learning. Organizations like the Asia Society
5 (http://asiasociety.org/education), Longview Foundation (2008), Global Kids
(http://www.globalkids.org/), and World Savvy (http://worldsavvy.org) not only offer frameworks for supporting students’ 21st century competencies, but also rich resources and professional development opportunities for teachers. Some educators have designed targeted and thematic lessons, assessments, and activities for teachers to use in classrooms intended to support students in becoming globally competent (Saavedra & Opfer, 2012; Falk, Moss, &
Shapiro, 2010; Asia Society, 2011; Appleyard & McLean, 2011; Merryfield, 1994; Roberts,
2007; Zhao, 2010). Still others have turned their attention to reviewing the proliferation of
21st century frameworks, and proposed that content in the various disciplines be carefully updated and reframed to make learning relevant, compelling and life-worthy for the lives that students will lead (Perkins, 2014), and to counter the lack of specificity of generic 21st century skills for classroom applications (Jacobs, 2010; Stewart, 2010; West, 2012). Indeed, the wheels have been furiously turning to better prepare young people for a globalized world.
Yet, the plethora of curricular frameworks and resources for nurturing 21st century competencies has not resulted in a commensurate quantity of assessments that might support us in quickly and efficiently tracking the extent to which students are developing the desired 21st century competencies. This is inherently problematic because an effective assessment instrument is a valuable resource for teachers and students alike: they provide information on whether learning has taken place, as well as productive insights into how students are thinking as they develop 21st century competencies. An effective assessment instrument supports teachers in designing instruction that targets learning challenges, and reveals important information to students about what global competence looks like and where they are vis-à-vis public criteria for 21st century competencies.
6 The Global Competence Matrix
For my study, I adopt the view of global competence advanced by Boix Mansilla and
Jackson (2011), who define educating for global competence as developing students’
“capacity and disposition to understand and act on issues of global significance” (p. xiii).
This definition, with its four dimensions of investigate the world, recognize perspectives, communicate ideas to a diverse audience, and take informed action, most fully captures the notion of global competence described by many different scholars and organizations. It is also the definition adopted by the U.S. Department of Education to institutionalize international education
(U.S. Department of Education, 2012), as well as by many organizations and scholars in their work on 21st century competencies. Finally, while most published frameworks emphasize skills, Boix Mansilla and Jackson’s definition highlights the dispositions necessary if students are to become globally competent, i.e., if students are to exhibit behavior and thinking that contribute productively to the wellbeing of the world, they need not only the ability to do so, but also the inclination (i.e., to care about and be motivated to do so) and the sensitivity (i.e., attentiveness to occasions to do so) to investigate the world, take perspective, communicate ideas and take informed action (Perkins, Tishman, Ritchhart, Donis, & Andrade, 2000;
Ritchhart & Perkins, 2000; Ritchhart & Perkins, 2008). This dispositional focus calls attention to the role of affect – preferences, motivations, intentions, attitudes and habits of mind – in determining whether people with the abilities associated with global competence actually realize them in productive, respectful, and creative ways (Norris, 1994; Tishman &
Andrade, 1995). The disposition to understand and act on issues of global competence, I believe, is core to a person’s capacity to deeply understand and participate in our contemporary world. While my study, which assessed participants’ responses at one moment in time rather than longitudinally, is not designed to discriminate between dispositional
7 qualities and capabilities, it does provide the foundation for future inquiry into dispositions associated with global competence.
The view of global competence that I adopted for my study – the capacity and disposition to understand and act on issues of global significance (Boix Mansilla & Jackson, 2011) – is explicated in the following matrix of four integrated and dynamically interacting dimensions
(see Figure 1).
Figure 1 Matrix of Global Competence advanced by the Asia Society and the Council of Chief State School Officers
In this conceptualization of global competence, globally competent students are described as being curious about the world, and developing an understanding of the
8 complexities of how the world works that is informed by both disciplinary and interdisciplinary insights. They “investigate the world beyond their immediate environment, framing significant problems and conducting well-crafted and age-appropriate research; recognize perspectives, others’ and their own, articulating and explaining such perspectives thoughtfully and respectfully; communicate ideas effectively with diverse audiences, bridging geographic, linguistic, ideological, and cultural barriers; and take action to improve conditions, viewing themselves as players in the world and participating actively” (p.11).
Although Boix Mansilla and Jackson (2011) propose that globally competent students demonstrate four dimensions of global competence, I focus only on the dimension of investigating the world using disciplinary and interdisciplinary approaches in this exploratory study. Demonstrating the capacity to investigate the world involves understanding the dominant issues and trends shaping the world that we live in, using disciplinary lenses to make sense of the world, and integrating key insights and ways of thinking from different disciplines to arrive at novel questions, explanations, and solutions. It also includes being able to frame questions that explore and investigate issues of global importance, leverage resources from multiple sources, analyze, weigh, and synthesize evidence, and develop coherent and compelling responses (Boix Mansilla & Jackson, 2011).
Two reasons motivate this decision to focus on only one dimension of the framework. Firstly, a deep understanding of the world – e.g., history of nations; international dynamics; economic relationships; knowledge of cultural norms and influences; global issues and trend – is referenced in almost all the 21st century frameworks and definitions of 21st century competencies available to date. Secondly, it is the dimension that is likely to be of greatest relevance to classroom teachers, since most disciplines or subject areas require students to embark on research or inquiry. It is therefore more likely that teachers will
9 naturally gravitate to this dimension first if they are to teach for global competence, compared to taking action or recognizing multiple perspectives, both of which could meet with some resistance from teachers who see them as extraneous to their classroom work.
Assessing Global Competence
The construct of global competence has traveled under multiple names, often because of a difference in emphasis. For instance, definitions of global competence, as cultural intelligence (Peterson, 2004), intercultural competence (Deardorff, 2006), intercultural sensitivity (Bennett, 2004), and “having an open mind while actively seeking to understand cultural norms and expectations of others, leveraging this gained knowledge to interact, communicate and work effectively outside one’s environment” (Hunter, 2004), respond to the contemporary reality of a flattened world where encountering difference is the new normal. Other definitions highlight our common humanity and global responsibility by invoking the notion of “citizenship”: global citizenship (Shultz, 2007); 21st century citizenship
(Partnership for 21st Century Skills, 2014); and the “active engagement of well-educated, cognitively flexible, and culturally sophisticated individuals able to work in groups” (Suarez-
Orozco & Sattin, 2007, p. 2). Still others have defined global competence as both the capacity to understand global events and interrelationships, and the disposition to apply that understanding in informed ways to impact the world, e.g., global competency (Reimers, 2008;
Boix Mansilla & Jackson, 2011; West, 2012); international mindedness (International
Baccalaureate, 2013); global mindedness (Hett, 1993); the “capacity and disposition to understand and act on issues of global significance” (Boix Mansilla & Jackson, 2011); and content knowledge in a domain and the procedural knowledge of how, when and why to apply that content knowledge (Pellegrino & Hilton, 2012).
10 To date, there is no standard instrument to assess how students are developing global competence, and the reasons for that are not surprising. Firstly, the diverse characterizations of global competence outlined earlier generate puzzles over what aspects should be assessed. It stands to reason that a standard assessment instrument will target a standard set of capabilities, but agreement on that standard set does not currently exist in the field. Furthermore, traditional modes of testing, such as multiple-choice and short answer questions, are not able to fully capture the complexity of the competencies. In addition, competencies like cultural perspective taking and cross-cultural sensitivity may be more accurately measured over time. A scan of the assessment instruments available today reveals that they either measure only one specific competency, or they comprehensively assess all competencies associated with the construct.
Many instruments in the field – more than eighty, in fact – assess a single competency – intercultural competence. For instance, the International Civic and Citizenship
Survey assesses students’ acceptance of diversity, while the Intercultural Development Inventory
(http://www.idiinventory.com/) generates an in-depth graphic profile of an individual’s or a group’s predominant level of intercultural competence along with a detailed textual interpretation of that level of intercultural development. The Intercultural Knowledge and
Competence Value Rubric, developed by AAC&U (2010), offers a systematic way to measure our capacity to identify cultural patterns as well as respond to difference with empathy. The
AAC&U’s assessment rubric is informed by Bennett’s Developmental Model of Intercultural
Sensitivity (1993) and Deardorff’s Intercultural Competence Model (2006), the latter being the first research-based consensus model of intercultural competence. Even though it measures only intercultural adaptability, the Global Competencies Inventory (www.intercultural.org) comprises
17 competencies in three categories of intercultural adaptability.
11 More ambitious assessment instruments have also been advanced. The Global
Competence Aptitude Assessment (http://www.globalcompetence.org/) claims to be “the world’s most comprehensive measure of global readiness,” because it measures every single competency that research findings have identified. Using questions based on regions around the world, with particular emphasis on countries with significant contributions to the global population and economy, the test comprises eight aspects of global competence that are each evaluated from a variety of angles. Asia Society’s Graduation Performance System
(http://asiasociety.org/education/resources-schools/professional-learning/gps-future- success) is informed by best-practice research on performance-based assessment around the world. The instrument comprises a set of frameworks and instruments for each subject that support teachers in grading and designing lessons to nurture global competence. The Global-
Mindedness Scale (Hett, 1993) measures affective behaviors, attitudes and values related to the development of global-mindedness and has since been used in various empirical studies to measure levels of global-mindedness among undergraduate students, university faculty and administrators, and agricultural extension agents (Duckworth, Walker-Levy, & Levy, 2005;
Gillian, 1995; Kehl & Morris, 2005; Kirkwood-Tucker, Morris, & Lieberman, 2011; Smith,
2008). The Global Citizenship Scale (Morais & Ogden, 2011) assesses three competencies of global citizenship: social responsibility, global competence, and global civic engagement.
Finally, World Savvy’s assessment instrument employs a variety of ways to evaluate an individual's willingness to explore how the world works, interest in other cultures, and desire to take a stand on global issues, among other competencies (http://worldsavvy.org/).
Some assessments of academic abilities have also addressed global competence, although they are not specifically designed to test for them. For instance, the progress of students towards the goal of international mindedness in the International Baccalaureate
12 programme is evaluated through their performance in coursework, portfolios and examinations (http://www.ibo.org/diploma/assessment/). The Programme for International
Student Assessment (PISA) measures students’ ability to apply their understanding to real-life problems calling for cross-curricular competencies and collaborative problem solving, both considered key to success in the 21st century (http://www.oecd.org/pisa/).
Despite the number of assessment instruments available in the field, their suitability for use by classroom teachers remains limited. Firstly, assessment instruments that measure only one competency, such as intercultural competence, are unable to support teachers in nurturing the different and complex capacities associated with global competence.
Furthermore, even though they only assess one global competency, these instruments do not always lend themselves to flexible and easy use in the classroom. For example, both
Bennett’s Developmental Model of Intercultural Sensitivity (six stages of increasing sensitivity to cultural difference) and Deardorff’s Intercultural Competence Model (comprising several components and multiple sub-components of attitudes, knowledge, skills, and outcomes) require teachers to spend time reviewing and deciding where their students are on the scale of intercultural competence. A more effective instrument is one that quickly diagnoses where students are in their thinking, and that is easily integrated into existing lessons.
Furthermore, most of the available assessment instruments are problematic in terms of the expertise required. Most assessment instruments require specialized training by coaches because of the complexity involved in using them accurately and successfully. If training is offered by the testing agency, teachers need to invest time in being trained to implement and score the tests. Thus, the complexity involved in the use of such instruments restricts their quick and frequent application in the classroom. For example, Asia Society’s
Graduation Performance System (GPS) provides frameworks and assessment instruments
13 that include a graduate profile, a series of discipline-based and interdisciplinary performance targets and rubrics, as well as sample curricula and examples of student work
(http://asiasociety.org/education/partnership-global-learning/events-services/graduation- performance-system). To use these frameworks and instruments as they are intended, teachers are required to undergo training in the core principles of global competence as articulated by the Asia Society and the GPS, including lesson planning, curriculum writing, instruction, evaluation of student work, and the calibration of scoring. The GPS also requires a whole-school approach; on-site leadership training is another component of GPS implementation. Hence, adopting the GPS is a complex process, and makes the instrument ill suited for quick and targeted use in the classroom.
Other assessment instruments such as the Global Competence Aptitude Assessment are implemented and scored by the testing agency, not the teachers. Instead, teachers receive a detailed report of their students’ performance on eight aspects of global competence, including a detailed document with suggestions for further development and improvement.
The extent to which teachers will be able to incorporate such massive amounts of feedback into their instruction is not promising. Furthermore, tests that require implementation and scoring by testing agencies cost a fair amount of money, especially if large numbers of students are to be tested. Schools are unlikely to invest monetary resources in students being frequently evaluated on something that is not tested in state examinations.
Perhaps the most worrisome feature of many of these assessment instruments is their dependence on self-reports. For instance, the Global Perspectives Inventory, the Cross
Cultural Adaptability Inventory, and the Intercultural Sensitivity Inventory comprise a battery of questions that students respond to, and those responses are scored to arrive at a judgment of the students’ level of global competence. Using self-reports as the basis for assessment is
14 problematic on several counts. Firstly, such tests assume honesty on the part of participants, ignoring the possibility that participants may be motivated to manage how they appear to the tester and teacher, and hence may be untruthful on certain sensitive questions. Secondly, students may lack the reflective capacity to assess themselves and provide accurate responses to the questions. Finally, students may not understand the demands of the questions posed, especially with regard to more abstract concepts. This could lead to their responses being more random rather than carefully considered.
In short, existing instruments that assess global competence fall short of what classroom teachers need in order to prepare their students for the 21st century. Instead of being simple and flexible enough for frequent classroom use, most assessment instruments impose huge demands on teachers. Teachers either have to be trained to administer and score the tests, or they have to plough through massive amounts of feedback before being able to use it effectively.
An effective instrument for teachers should provide rich information on how students are becoming globally competent. However, most assessment instruments administer self-reports such as questionnaires and surveys, and rely too much on students’ honesty, understanding, and reflective capacity for accurate information. The conclusions they reach run the risk of being inaccurate or at best half-truthful, and the recommendations they provide to teachers are therefore less than helpful.
Teachers are also more likely to appreciate an assessment instrument that is easily integrated into existing lessons and content, since that will mean more seamless adoption of the instrument without major disruption to curricular plans. The drawback of existing assessment instruments is that they do not lend themselves easily to such integration.
Existing instruments either require time siphoned from normal lessons for students to take
15 the tests, or they provide feedback that requires time and effort to meaningfully incorporate into existing lessons. In cases where teachers could administer and score the tests, teachers still need to be trained by external coaches to do so.
Finally, even if an existing assessment instrument meets all the previous criteria, it still requires one other component to be an accessible and effective assessment instrument for teachers: an empirically based set of descriptors that provide clear criteria for understanding where students are at, and how they might progress to more sophisticated levels of global competence. Deardorff’s (2006, 2009) consensus-based model for intercultural competence lists elements that are agreed-upon by a panel of experts, rather than descriptions of what different levels of global competence actually look like. Similarly,
Bennett’s scale for intercultural sensitivity (2004), the result of direct observations and research, offers general descriptors for the six stages of developing competence.
Classroom teachers are the frontline when it comes to preparing young people for the 21st century. If they are to successfully nurture global competence in their students, they need a dynamic instrument for both formative and summative evaluation, one that can become an integral part of a teacher’s everyday practice in the classroom. Such an instrument should offer a flexible and easy way to diagnose where the students are in their progress towards those competencies, and provide a clear sense of how they might further support their students instructionally.
Thinking Routines as Teaching and Assessment Instruments
Thinking routines specifically designed to nurture global competence (i.e., global thinking routines) are developed by the Interdisciplinary and Global Studies initiative at
Project Zero, which has been studying the nature of global consciousness and global
16 competence among youth, teachers and experts in multiple contexts of learning. Similarly defining global competence as the capacity and disposition to understand and act on issues of global significance, the Interdisciplinary and Global Studies initiative proposes that being globally competent goes beyond having information or skill; it involves developing dispositions that position students to successfully understand and thrive in the world they live in (Boix Mansilla & IDGlobal Team, 2013).
To support students in developing these dispositions, and classrooms in building a culture that nurtures global competence and consciousness, the Interdisciplinary and Global
Studies initiative drew on a long-standing line of work on thinking dispositions and enculturation at Project Zero, which investigated qualities of higher order thinking and the creation of classroom cultures that supported thinking dispositions among learners of all ages. A key outcome of this research has been the design and testing of thinking routines, which are patterned ways of thinking that, when used repeatedly over time, build a culture of thinking in the classroom (http://www.pz.harvard.edu/resources/thinking-routines-core- routines). Thinking routines are designed to support students in key cognitive tasks such as introducing and exploring ideas; synthesizing and organizing ideas; and digging deeper into ideas. An example of a thinking routine is See-Think-Wonder, designed to support students’ capacity to observe closely and craft thoughtful interpretations. A typical way to employ See-
Think-Wonder is to invite students to look closely and carefully at something, and then respond to the three prompts at the same time, i.e.,“I see…, I think…, I wonder….” When used strategically, this process builds the foundation for further inquiry. Another popular thinking routine is Connect-Extend-Challenge, which is intended to support students to connect what they have learned to prior knowledge, as well as to be reflective about what they are learning. When students use this thinking routine, they respond to three questions: How do
17 the ideas and information presented connect to what you already know? How do the ideas and information presented extend or push your thinking in new directions? What is still challenging or confusing for you?
(Boix Mansilla & IDGlobal Team, 2013).
While general-purpose thinking routines like See-Think-Wonder and Connect-Extend-
Challenge are used extensively by teachers (Ritchhart, Church, & Morrison, 2011), and many of them can be successfully applied to content of global significance, they are not specifically designed to nurture globally oriented dispositions. Nor do they target learning challenges that students might encounter when confronted by difference. Global thinking routines build on the work done on thinking routines, and are intended to be micro-interventions that focus attention on specific thinking moves that build global competence. They support and assess students’ competencies to engage with difference, uncover the significance of a target global issue, understand local-global connections, challenge stereotypes, etc. For instance, the 3Ys thinking routine supports students to uncover the significance of global issues across multiple spheres using three prompts: Why does it matter to me, to my community, and to the world? Another global thinking routine is the How else…Why? encourages students to take multiple aspects of language into consideration (e.g., register, nonverbal cues, cultural nuances) when communicating in complex cultural exchanges using the following questions:
How might I say this…Why? How else might I say this…Why? How else might I say this…Why?
Whether they are general in purpose or whether they specifically target globally competent thinking, thinking routines work well as micro-interventions for supporting student development because they make learners’ thinking visible not only to teachers but to the students themselves, and support them in understanding their own thinking processes, and consequently in becoming more reflective about the way they learn. Teachers can also assess how students are thinking and provide information on how they might be supported
18 towards more sophisticated understanding. Because thinking routines are simple structures comprising either a set of questions or a short sequence of steps, and do not require lengthy preparation or implementation time, they can be easily incorporated into existing classroom practice as well as employed for a variety of content. Additionally, they target specific types of thinking, so teachers can select exactly which competency they want to nurture in a particular lesson (Ritchhart, Church, & Morrison, 2011).
Thinking routines have been introduced and used extensively in classrooms in the
U.S. and many other countries, as well as for a variety of disciplines and contexts. Anecdotal reports from teachers have testified to the facility of using thinking routines in their regular lessons, as well as the success of the global thinking routines in supporting students’ developing global competence. In particular, teachers have reported that thinking routines have been richly evocative in making visible their students’ thinking.
When thinking routines are used in a single session, as in my study, they can uncover participants’ assumptions about the significance of climate change across multiple circles of significance, possible local-global connections, their own position in local and global contexts, and what is most worthy of investigation. When used repeatedly over time, they have the potential to nurture dispositions that determine whether people can actually realize those capacities in productive, respectful, and creative ways (Norris, 1994; Salomon, 1994;
Tishman & Andrade, 1995). However, understanding how participants in my sample demonstrate those dispositional traits is beyond the scope of my study; to fully capture disposition will require a different study design.
My study tests two thinking routines as a performance-based assessment instrument for teachers’ easy and flexible use in the classroom, which is an innovative use of thinking routines since they are primarily designed to support learning. Thinking routines support
19 specific thinking moves like observing carefully, making connections, building explanations, uncovering significance, reasoning with evidence, and forming conclusions – moves that teachers of all disciplines engage their students in. Therefore, thinking routines can easily be integrated into existing lessons and content. Furthermore, thinking routines do not pose the same problems to teachers that other existing assessment tools do, e.g., they do not impose huge demands on teachers or students as they consist of a few simple steps that are easy to learn and remember, and so impose no strain on teachers or students who use them frequently. Thinking routines also make students’ thinking visible, and so provide important information for teachers as they design instruction to support students’ developing global competence.
In my study, the general-purpose thinking routine – Connect-Extend-Challenge – and the global thinking routine – 3Ys – were used to engage participants in thinking about climate change and geoengineering. As the participants addressed each question in the thinking routines, they revealed important patterns in the way they connected ideas and perspectives, framed possibilities and dilemmas, considered multiple forms and circles of significance, situated themselves within the problem space, and revealed aspects of the topic or issue they considered most worthy of investigation. As perspectives and significance are not fixed qualities but rather attributions that the participants framed vis-à-vis the topic or issue, the thinking routines offered authentic windows into their thinking.
Testing the thinking routines on expert and novice participants, and paying close attention to how they differ systemically in their investigation of climate change, was an important first step towards creating a developmental framework that captures the nuances involved in more and less globally competent investigations of the world. Such a framework will provide descriptions of the kind and quality of moves that experts and students actually
20 make as they engage with global issues, and ground teachers’ expectations of their students’ developing global competence in actual data. While the scope of this study did not allow for the creation of this framework, it nonetheless provided key insights into the patterns of thinking that the thinking routines did and did not uncover, as well as suggesting productive next steps toward such a framework.
21 Chapter 2: Review of Relevant Literature
In this chapter, I review three sets of literature that conceptually framed the purpose and design of my study, and addressed my research question: What similar or contrasting patterns of thinking do experts and high school students, studying climate change occurring either globally or within a country/community, demonstrate when they engage with a scenario on climate change using a series of thinking routines?
First, I establish the definition of global competence used in my study by unpacking the various frameworks for 21st century skills and competencies that have gained traction in the discourse on how best to prepare young people for our contemporary world. I distinguish between competencies that serve global interests in a general way but that do not specifically reach toward understanding pressing global issues or understanding the other
(e.g., critical and creative thinking; personalization of learning), and those that have a clear global turn, such as intercultural competence.
I next examine the literature on science education to elucidate the recommendations for contemporary scientific inquiry in the classroom, and the extent to which what counts as globally competent scientific inquiry has been the focus in the science classroom. Doing so allowed me to situate my study within a gap in the field: while there are emphatic recommendations for enculturating students in the kind of scientific inquiry and epistemologies important in contemporary society, less attention has been focused on the kinds of globally competent inquiry that young people should bring to their investigations of pressing global issues and problems.
22 Finally, I review the scholarship on expert-novice research and paradigms, examining the relevant work that has been done to elucidate the kinds of gaps that exist between novices and experts when thinking about domain-specific problems. Findings from the research using the expert-novice methodology provided insight into the patterns of thinking that my participants demonstrated, as well as revealed interesting new differences that extend our understanding of expert thinking about complex problems and issues.
Defining Global Competence
The rapid pace at which frameworks have been articulated and resources developed for the purpose of preparing young people for the 21st century belies the fact that to date, there is no unanimous term or label used to refer to the set of 21st century skills and competencies that are necessary to deeply understand and productively engage our contemporary world. Neither is there agreement on what exactly constitutes 21st century skills and competencies. Broadly speaking, 21st century skills and competencies run the gamut from cross-disciplinary knowledge to work habits, character traits to specific skills, from soft skills to cognitive processes. For the purpose of my study, however, I focused on a narrower set of competencies that reach specifically toward broad understanding of our complex globalized world, such as understanding the other in an era marked by mass and rapid human migration, or engaging intractable global issues like climate change and terrorism in informed ways. These competencies, which I refer to as “global capacities”, have an unmistakably global orientation, and specifically position the individual to live and thrive in a world increasingly defined by difference and complexity.
Even though terms that describe these global capacities are widely used in education, they are not consistently defined. In fact, the only thing consistent about them is that they
23 have traveled under different names: global competence or competency (Reimers, 2008; Boix
Mansilla & Jackson, 2011; West, 2012); intercultural sensitivity (Bennett, 2004); international mindedness (International Baccalaureate, 2013); global citizenship (Shultz, 2007); cultural intelligence (Peterson, 2004); intercultural competence (Deardorff, 2006); 21st century citizenship (Partnership for 21st Century Skills, 2014); global mindedness (Hett, 1993). Still, these terms are distinguished more by their commonality than divergence; important overlaps among these terms lend the construct of global capacities far more unity than the variety of names suggests.
To explicate the construct of global capacities, some scholars, educators, and organizations have proposed broad statements. For instance, following extensive discussions with an international panel comprising representatives from global businesses, international education, the United Nations, embassies, and intercultural experts, Hunter (2004) arrived at a working definition that describes a globally competent person as “having an open mind while actively seeking to understand cultural norms and expectations of others, leveraging this gained knowledge to interact, communicate and work effectively outside one’s environment”. Suarez-Orozco and Sattin (2007) described global citizenship as the “active engagement of well-educated, cognitively flexible, and culturally sophisticated individuals able to work in groups” (p. 2). The Asia Society and the Council of Chief State School
Officers proposed that global competence comprises the “capacity and disposition to understand and act on issues of global significance” (Boix Mansilla & Jackson, 2011), while
Curran (2003) defines global competence as an “appreciation of other cultures and the ability to interact with people from foreign lands… the ability to become familiar with an environment, not causing a rift while experiencing something new, and reflection upon the experience at its completion” (p. 10). Pellegrino and Hilton (2012) view content knowledge
24 in a domain and the procedural knowledge of how, when and why to apply that content knowledge to real world problems as 21st century competencies. Hett (1993) proposed that global mindedness is a “worldview in which one sees oneself as interconnected to the world community and feels a sense of responsibility for its members which is reflected in attitudes, beliefs, and behaviors” (p. 143).
Others have advanced lists of skills or competencies to describe global capacities.
For instance, Lambert (1994) listed five components of intercultural competence: world knowledge, foreign language proficiency, cultural empathy, approval of foreign peoples and cultures, and the ability to practice one’s profession in an international setting, while Koester and Olebe (1989) proposed eight components of respect, orientation to knowledge, empathy, interaction management, task role behavior, relational role behavior, tolerance for ambiguity, and interaction posture. Deardorff’s (2004) work with leading scholars on defining the elements of intercultural competence culminated in a model comprising attitudes (respect, openness, curiosity, and discovery), knowledge (cultural self awareness, culture specific knowledge, deep cultural knowledge including understanding other world views, and sociolinguistic awareness), skills (acquisition and processing of knowledge), and internal outcomes (flexibility, adaptability, an ethnorelative perspective, and empathy), that together lead to external outcomes or intercultural competence, defined in the model as
“behaving and communicating appropriately and effectively in intercultural situations” (p.
194).
Despite the lack of a universal definition, the global capacities needed for successful living and working in the 21st century are often viewed as comprising three dimensions: knowledge of world issues, histories, etc.; attitudes such as sensitivity to and respect for difference; and communicative, critical and synthesizing skills (Green & Olson, 2008). Many
25 definitions of global capacities have foregrounded the importance of effective cross-cultural understanding, empathy, and interaction, particularly when encountering difference or people with hybrid identities (Suarez-Orozco & Sattin, 2007; Reimers, 2008, 2013; IBO,
2012; AT21CS, 2012; Boix Mansilla & Jackson, 2011; Deardorff, 2009, Peterson, 2004;
Hunter, White, & Godbery, 2006; Chen & Starosta, 1996; Byram, 1997; Tye, 1991; Levin,
2003; Reardon, 1999). These definitions emphasize empathy, cultural awareness and sophistication, and respect for and appreciation of diverse cultures and peoples, in order to understand cultural norms and influences and consequently communicate and interact effectively across difference.
Definitions of global capacities frequently emphasize a deep cognitive understanding of, and active engagement with, global issues and trends (Shultz, 2007; Suarez-Orozco &
Sattin, 2007; IBO, 2012; Boix Mansilla & Jackson, 2011; Reimers, 2008, 2013; Hunter, 2006;
Hovland, 2009). In this conceptualization, people who succeed in contemporary society demonstrate the capacity to investigate the world, leverage knowledge of global cultures and issues to build their capacity for civic engagement, and integrate disciplinary perspectives in order to arrive at a deepened understanding of global issues and relationships.
More recent definitions of global capacities have included the capacity to take informed action beyond one’s local context, hence demonstrating a sense of responsibility for improving communities (Shultz, 2007; Reimers, 2008, 2013; Boix Mansilla & Jackson,
2011; Hovland, 2009). This inclusion highlights a heightened emphasis on global citizenship, a humanistic stance that recognizes our belonging to a common community regardless of cultural and national affiliations, and our ethical responsibility for proposing and working on solutions to address common global problems.
26 The dispositions that enable respectful interaction and productive collaborations with people different from us have also been emphasized in recent conceptualizations of global capacities (Reimers, 2008, 2013; Boix Mansilla & Jackson, 2011). These definitions highlight how global citizens act from a deep, personal desire to build a better world, with confidence in their ability to act, and with a strong attentiveness to opportunities where action might be needed and are possible.
In sum, while not all definitions agree on the specific combination of capacities that define global competence, they do concur that at least one of four capacities must be present: the capacity to learn about the complexity and interrelatedness of the contemporary world; the capacity to recognize, tolerate and respect difference; the capacity to communicate and collaborate effectively with people across cultures; and the capacity to take action on behalf of pressing global issues. In my study, I adopt Boix Mansilla and Jackson’s definition of global competence as the “capacity and disposition to understand and act on issues of global significance” (2011, p. xiii). In this conceptualization, a globally competent person is deeply curious about the world, and seeks to develop an understanding of complex issues that is informed by both disciplinary and interdisciplinary insights.
Thinking about Climate Change
On November 28, 2015, an article in The Economist called for the creation of new mindsets and tools for dealing with global warming, because same-old, same-old, would no longer serve to counter what it described as “a crisis that will pose its gravest risks long after they [current world leaders] have left office” (http://www.economist.com/news/leaders/
21679193-global-warming-cannot-be-dealt-using-todays-tools-and-mindsets-so-create-some- new). A month earlier, the Yale School of Forestry and Environmental Studies published the
27 first report from its latest national survey, Climate Change in the American Mind: October 2015, revealing that two in three Americans now think global warming is happening (67%); about one in ten Americans understands that nearly all climate scientists are convinced that human-caused global warming is happening; and that about half of Americans think that global warming, if it is happening, is mostly human caused (http://environment.yale.edu/ climate-communication/article/more-americans-perceive-harm-from-global-warming-survey
-finds/). According to the same survey, concern about global warming is geographically variable: a high 74% polled in Washington D.C. see it as worrying, while about 38% in
Pickett County, Tennessee, consider it a problem. Internationally, climate change was identified by publics in 19 of 40 nations surveyed as their biggest worry in a Pew Research
Center survey measuring perceptions of international challenges
(http://www.pewglobal.org/2015/07/14/climate-change-seen-as-top-global-threat/), making it the most widespread concern of any issue in the survey.
Given that climate change is a recognized global threat, how are we supporting young people to think about it in globally competent ways? How might our next generations be prepared to tackle problems such as climate change that impact not just their community, but the world at large? The question becomes even more pressing when we consider how that concern is unevenly distributed across countries: countries that are most worried – Latin
America, Sub-Saharan Africa, and Asia – are the most vulnerable to global climate change.
They are also the most economically disadvantaged nations, often at the mercy of wealthier nations like China, Russia, U.S. and Europe, where publics did not report global climate change among the top threats facing the world.
A survey of science education in schools, where climate change should logically be a topic of study, reveals that learning in the science classroom continues to function in
28 traditional modes: students are taught scientific inquiry as a simplified set of steps known as the `inquiry cycle’ stripped of the sturm und drang of building scientific theories and revising models (Collins & Stevens, 1993; Tabak, Smith, Sandoval, & Reiser, 1996). They mechanically practice defining research questions, formulating research hypotheses, planning experiments, observing outcomes and collecting data, analyzing data, summarizing and communicating findings, revising research questions and hypotheses, and beginning the cycle again, almost as if that were a formula that all scientists follow when in fact the process is likely to be far more dynamic (Ben-David & Zohar, 2009; White & Frederiksen, 1998).
Duschl (1990) describes typical science instruction as “final form science,” where ideas are presented as facts that students merely adopted as “truth,” rather than recognize as theoretical ideas complete with a history of development and multiple, often convoluted, steps that make up the process.
Others like Kuhn, Iordanou, Pease, and Wirkala (2008) argue that scientific thinking involves a multifaceted understanding of scientific phenomena, which often involves multiple variables interacting with and influencing one another. In their work, they found that preadolescents and even some adults performed poorly in multivariate prediction tasks.
Because leveraging multiple sources of disciplinary knowledge and frameworks looks to be the currency for scientists working at the edge of their field and competence, it becomes imperative that such expert strategic thinking be made explicit in scientific instruction, which is unfortunately not always the case in classrooms (Collins, Brown & Newman, 1989).
The role of metacognition as key to developing scientific reasoning has also been proposed as foundational to students’ authentic understanding of natural phenomena, as well as for developing scientific thinking strategies (Bell, Blair, Crawford, & Lederman, 2003;
Gott, Duggan, & Johnson, 1999; Tamir, Nussinovitz, & Friedler, 1982). Ben-David and
29 Zohar (2009) define this as meta-strategic knowledge, i.e., “general, explicit knowledge about scientific thinking strategies, which means an awareness of the type of thinking strategies being used in specific instances” (p. 1658). Such knowledge includes cognitive procedures ranging from generalizing and drawing rules regarding a thinking strategy to naming the thinking strategy, from explaining the choice for a particular thinking strategy to describing the task characteristics that call for the use of the strategy. Similarly, Amsel, Klaczynski,
Johnston, Bench, Close, Sadler and Walker (2008) argue for the development of metacognitive skills to regulate the interactions between our dual-processing experiential and analytic systems. By acknowledging the co-development of both processing systems, one therefore recognizes that “default” experientially-based responses can be reflected on and evaluated for their adequacy and utility (De Neys & Glumicic, 2008; Evans, 2007;
Klaczynski, 2004; Stanovich & West, 2000), and such intercession can potentially inhibit
“default” experientially based responses, distinguish between responses that are more systematic and cognitively effortful versus those that are more automatic, emotional and cognitively economical, and compare responses from the dual processes to decide on the most appropriate response for the given situation (Evans, 2003, 2008; Stanovich & West,
2000; Klaczynski, 2004).
Increasingly, researchers have called for science education to be considered
“epistemic practice” (Sandoval & Reiser, 2004), especially learning that involves students in conducting authentic inquiry that is modeled on what scientists actually do in the field.
Students’ more sophisticated epistemological beliefs have been shown to tip the scales towards successful learning when it comes to science learning: studies have found that those who saw scientific knowledge as provisional and humanly constructed were more successful in scientific inquiry (Sandoval, 2005; Linn & Songer, 1993; Tobin, Tippins, & Hook, 1995;
30 Windschitl & Andre, 1998). Similarly, instruction that places primacy on argumentation as a central practice led to improved inquiry abilities in students (Driver, Newton, & Osbourne,
2000; Kuhn, 1993; Duschl & Osborne, 2002; Erduran & Jimenez-Aleixandre, 2008; Kuhn,
1993; Kelly, Druker & Chen, 1998; Lehrer, Schauble & Petrosino, 2001). Sandoval and
Reiser (2004) propose that because judgments of science are often grounded in epistemological terms – “researchable”, “informative”, “persuasive”, etc. – it becomes imperative that students come to understand that what counts as scientific knowledge and scientific method is necessarily a matter of disciplinary values. Because values change within and across disciplines, and over time, our judgments of them correspondingly shift.
Understanding that essential quality of science positions learners to see themselves as contributors to a dynamic field, rather than the inheritors of a fixed, unchanging body of information or truths.
Linking epistemological development to progress in students’ thinking about science continues to be a clarion call by educational researchers. By consciously moving students away from regarding science as absolute truth and science learning as the accretion of facts, to understanding scientific knowledge as constructed by humans, not simply discovered in the world (Leach, Driver, Millar, & Scott, 1997; Smith, Maclin, Houghton, &
Hennessey, 2000; Sandoval, 2005), researchers predict multiple advantages for student learning: academic advancement in science (Leach et al, 1997; Metz, 2004; Sandoval, 2005); and the likelihood of transferring a more sophisticated epistemological stance to the learning of other academic subjects (Buehl & Alexander, 2005; Mason & Boscolo, 2004; Mason &
Scirica, 2006) as well as to their life outside of and beyond school (Kuhn, 2005; Kuhn &
Park, 2005).
31 Despite the wealth of research into what counts as authentic scientific inquiry, there has been a paucity of understanding about how scientists working at the frontiers of complex and intractable global issues such as climate change engage with complexity and ambiguity. What happens when scientists deal with phenomena that are multilevel, multifaceted, and multi- causal, and often with emergent properties that defy prediction? How do they engage with such problems in ways that are globally competent? In their inquiry into the contemporary scientific practices of three young scientists working in the cutting-edge fields of genetics, nanobiotechnology, and synthetic biology, Liu and Grotzer (2009) identified five key shifts in the way these scientists think about their inquiry: systems thinking, mechanistic (engineering) thinking, interdisciplinary thinking, quantitative thinking, and distributed thinking. The authors propose that these five forms of thinking are relevant to 21st century science, and are worth paying close attention to in instruction if we are to prepare young people adequately for the challenges that lay ahead of them.
In sum, there is a pressing need for nurturing the qualities of expert scientific thinking about climate issues in the classroom, qualities such as a more sophisticated epistemic stance towards knowledge and the nature of knowing, the capacity to take a more complex view of events, the ability to intercede between our experiential and analytic systems, and an appreciation of the multifaceted nature of global events and problems.
Characterizing Expert and Novice Thinking
The tradition of expert-novice research studies has been instrumental in helping us understand what expert performances in thinking and problem solving look like in specific domains, and motivating educators to consider the kind of support learners might need in order to achieve the levels of quality thinking that experts demonstrate. More specifically,
32 such studies have uncovered the patterned differences in thinking between exemplary performances and relatively novice ones, and especially illuminated “critical features of proficiency that should be the targets for assessment” (NRC, 2001, p. 4). Studies using this paradigm have uncovered rich and often surprising insights about expert-novice differences in memory ability (Chase and Simon, 1973), mental representations (Chi, Feltovich, and
Glaser, 1981), and strategies for problem solving (Novick, 1988). By encouraging the view of the expert-novice distinction as continuity rather than a gap (Garfield, Le, Zieffler, & Ben-
Zvi, 2015), such studies position novice understanding as a necessary basis from which more expert-like thinking could be developed.
In their influential synthesis of scientific findings on how experts perform in their domains, Bransford, Brown, Cocking, and National Research Council (U.S.) (1999) highlighted several key principles of expert knowledge vis-à-vis novices: experts are able to quickly pick out key concepts, ideas and chunk information into meaningful patterns; draw from a wealth of disciplinary knowledge that is organized around key ideas and concepts in the domain; move flexibly between top-level concepts and specific examples; recognize and retrieve knowledge swiftly for specific contexts and purposes, cutting through the minutiae of often unnecessary detail; and may be less inflexible in transferring knowledge to novel contexts. By clearly characterizing the content and processes of expert thinking, Bransford et al. (1999) provided a kind of blueprint for educators to rethink instruction for successful learning.
Much of the novice-expert research has focused on how people manage a complex problem, select parameters within it, and organize the material. Perhaps the most well known study was DeGroot’s (1978) research on world-class chess masters, which uncovered how chess masters bring to their game a finely honed strategic perception of moves that allow
33 them to quickly recognize superior configurations and outplay novices. Later studies built on this to elaborate how expert understanding of a domain pivots powerfully on fundamental procedural principles and knowledge meta-structures, while novices were more likely to focus on surface-level concepts and make superficial connections across ideas (Chi et al,
1981). Experts also encode information using associations stemming from their extensive understanding of the domain, and quickly grasp meaningful relations among seemingly disparate bits of information (Charness, 1989; Chase & Simon, 1973; Ericsson & Kintsch,
1995). Posner (1988) describes this extensive understanding of the domain as an elaborate semantic memory, and postulated that it is and can be acquired given the right training.
Hence, expertise, he argues, is not an innate quality. The work of Ericsson and Charness
(1994) supports his contention that expert understanding can be developed with optimal training and the right context.
Research has also demonstrated how experts are able to swiftly navigate the complexity of a given problem and arrive at an effective solution because they can readily access rich and well structured bodies of domain-specific knowledge, allowing them to flexibly retrieve relevant principles and key concepts in the field to solve a problem, rather than become mired in irrelevant and superficial information (Resnick, 1989; Chi, Feltovich,
& Glaser, 1981; Schenk, Vitalari, & Davis, 1998; Bransford et al., 2000; Novick, 2000). For instance, in a series of studies on children’s understanding of chance and distribution, Pratt and Noss (2010) found that children’s emergent understanding of randomness shifted rapidly among four choices of explanation, often triggered by statistically superficial aspects of the data, while experts were attentive to key features of the data in their explanations.
Another key feature of expertise is that it reasons from a solid foundation of domain-specific knowledge and prior contextual experience that is brought to bear on
34 current problems. Experts are more efficient at problem solving because they are able to quickly recognize important features of a novel problem and recruit relevant deep structures and core concepts into a solution, and at the same time employ strategies used in previous experiences and contexts to innovatively design a solution (Bransford et al, 2000; Schwartz,
Bransford, & Sears, 2005). This is in contrast to novices who tend to follow fixed recipes to the letter, and are leery of skipping or combining steps in a problem-solving situation
(Blessing & Anderson, 1996; Larkin, McDermott, Simon, & Simon, 1980).
How experts and novices unpack complex systems has also been a productive area of inquiry, particularly at a time when global and often intractable phenomena such as immigration, climate change, and terrorism are gaining public attention. Because the interactions among the different levels of a complex system are not always intuitive, research has found that novices tend to pay attention to what they can actually see rather than consider invisible agents or actors in the system (Hmelo, Holton, & Kolodner, 2000; Wood-
Robinson, 1995), ostensibly because the cognitive demand for processing the various interacting parts and structures of the system far outstrips their capacity to mentally construct and model the complete system (Graesser, 1999; Narayanan & Hegarty, 1998). The work of Perkins and Grotzer (2000) and Wilensky and Resnick (1999) also reveals how complex systems often place an inordinate load on working memory because their causal interactions are complex with emergent properties that make prediction challenging. In multiple studies, novices were confounded by the interaction between the micro- and macro- level phenomena, often paying close attention to the former and failing to even name the latter (Penner, 2001). They tended to craft explanations in which there was a single cause from which everything else transpired (Resnick & Wilensky, 1998), focused heavily on superficial structures of systems rather than underlying functions (Perkins & Grotzer, 2000),
35 and generally thought of systems as operating with unidirectional causality in feedback loops
(White, 2008; Green, 1997). In contrast, experts zero in on deep underlying principles that structure their knowledge of the system, and identify processes that are often interacting, invisible and difficult to represent (Hmelo-Silver & Pfeffer, 2004; Pimm, 1982; Ricklefs,
1993). They also tend to take a longer, more historical view of events and causes, relying often on causal reasoning to create compelling narrative and explanatory models of phenomena (Jones & Read, 2005; Hmelo-Silver, Marathe, & Liu, 2007).
More recently, Collins (2011) proposed a three-dimensional model of expertise: specialist tacit knowledge (contributory, or knowledge acquired only through practical immersion in the field), interactional expertise (knowledge acquired through deep immersion in the linguistic discourse of the field), and ubiquitous tacit knowledge (knowledge acquired without any contact with specialists, and may range from getting information from a beer coaster to reading original research papers). In this rethinking of the nature of expertise, the distinction between what counts as “expert” and “novice” is complexified, and given the ubiquity of information available to the layman in today’s media-saturated world, the novice-expert paradigm may well be less a dichotomous distinction than a multi-pronged source of variation.
This review of the expert-novice research richly informed my hypotheses of the kind of patterned differences in thinking between the experts and novices that I might uncover in my study. The analysis of my participants’ responses to climate change and geoengineering revealed trends that corroborated current understanding of expert-novices differences. For instance, the experts were able to frame the significance of geoengineering within a broader network of relationships, and asked questions that considered geographic, temporal, and cultural factors (Roan, Strong, Gehlbach, & Metcalf, 2009). They also interpreted sources
36 and propositions in ways that acknowledged paradoxes and shifting frames of reference, while the novices held more dichotomous and absolute views (Osland & Bird, 2000).
37 Chapter 3: Methods
In this chapter, I provide a detailed description of the research design of my study, beginning with a restatement of the research question and a description of the two expert groups and the two novice groups that I recruited, followed by an explanation of the rationale for their selection. Next, I describe the expert testing review that I conducted to establish whether my protocol measured the dimension of global competence as intended, as well as the pilot test that was conducted to reveal whether the instrument yielded rich data with the target groups. I then outline how the data were collected, describing in detail how the pilot test determined the protocol that was eventually used in the study, and my process for analyzing the data collected. Finally, I list two potential threats to the validity of my research findings and explain how I have consciously taken steps to counter them.
Research Question
My study set out to address the following question: What similar or contrasting patterns of thinking do experts and high school students, studying climate change occurring either globally or within a country/community, demonstrate when they engage with a scenario on climate change using a series of thinking routines? A sub-theme of my study is an exploration into the potential of thinking routines as assessment tools that would allow teachers to gauge how students are developing more globally competent thinking about climate change. However, as the methodology employed in my study does not specifically evaluate thinking routines in an assessment capacity, I will not explicitly study them as such. Instead, I will return to a discussion of this potential in Chapter 5.
38
Research Design
As there is currently very little that is known in the field of global education about how people actually engage with complex contemporary issues in globally competent ways, a qualitative research design is most apt for uncovering the “quality and texture” of how participants in my study think about climate change and geoengineering (Willig, 2008, pp. 8).
By conducting intensive, in-person interviews using a structured protocol, I have been able to understand and identify the orientations and patterns of thinking that the experts and novices in my study bring to an exploration of a controversial solution to climate change - geoengineering. My exploratory study allowed me to probe deeply their perspectives on climate change and how they frame their relationship to it, as well as identify unanticipated ideas and patterns of thinking and generate new “grounded” theories or hypotheses about globally competent investigations of the world by novices and experts that will contribute to the field (Maxwell, 2005).
Site and Sample
I identified and invited three participants for each of the four groups in my study: global-expert (GE); local-expert (LE); global-novice (GN); and local-novice (LN). In total, I interviewed twelve participants for the study. The local/global distinction among the novices and experts was important to the study because I had hypothesized that the context that experts and novices worked in could orient them to different patterns of thinking about the issues and problems under inquiry, and if that were indeed the case, I hoped to uncover those differences in my study.
39 My small sample size of twelve participants was deemed sufficient for the purpose of my dissertation, which is designed to be a small-scale, exploratory study. Furthermore, the intensive nature of the interviews – each lasting more than an hour, with a protocol that focused on how the participants framed their ideas and perspectives, how they made connections, what attitudes they held and why, etc., using a series of tasks – elicited rich information for addressing my research question. Most importantly, this sample size was determined to be satisfactory for the purpose of my study after several discussions with my dissertation committee.
Expert Groups
I selected my two groups of experts from within one umbrella organization –Harvard
University’s Center for the Environment – to ensure that institutional influence remained consistent across the sample. Although the two research groups – Harvard Forest and
Atmosphere, Ocean and Climate Dynamics – function as separate entities within the center, all six experts likely align with its mission of encouraging research and education about the environment and its many interactions with human society (http://environment. harvard.edu/about/mission), and are part of a diverse community of faculty members and students from the Harvard schools/departments of chemistry, earth and planetary sciences, engineering and applied sciences, history, biology, public health and medicine, government, business, economics, religion, literature and the law.
Both groups also belong to the same initiative within the center – Future of Energy – whose mission is to provide opportunities for interdisciplinary interactions and collaborations among faculty and students working on major energy-related problems such as global climate change, urban air pollution, technologies for pollution abatement and
40 renewable energy, energy security, and economic growth in developing countries. The two groups are also part of the same program within the initiative – Climate & Global Change.
My sample of local experts is part of the Harvard Forest research department. The
Harvard Forest is one of the oldest and most intensively studied forests in North America, comprising 3000 acres of land, research facilities, and the Fisher Museum
(http://harvardforest.fas.harvard.edu/). These researchers study conservation and environmental change, land-use history, and the ways in which physical, biological and human systems interact to change our earth.
The global experts in my study come from the Atmosphere, Ocean, and Climate Dynamics research department, whose goal is a better understanding of Earth's weather and climate on time scales from a few days to millions of years (http://climate.fas.harvard.edu/). The researchers study fundamental phenomena controlling the global atmospheric and oceanic circulation, particularly sources of variability that result from interactions among the atmosphere, oceans, and the biosphere, which are complex and often nonlinear and even chaotic.
I anticipated that my two expert groups were likely to differ mostly in the local/global context of their research focus (i.e., experts at the Harvard Forest work primarily on issues and phenomena relating to the New England region, while the
Atmosphere, Ocean and Climate Dynamics group was more likely to work on cross-case analyses of issues and phenomena) and that within their research department, they were likely to hold complementary disciplinary perspectives on the topic of climate change, and experience similar contextual forces.
Below is a graphic representation of how the research activities of the two samples are housed within the same research center:
41 Harvard University Center for the Environment
Future of Energy IniBaBve
Climate & Global Change program
Atmosphere, Ocean, & Harvard Forest Climate Dynamics
LE01 LE04 GE01 GE02
LE05 GE04
The global/local research orientation of the experts was crosschecked using two
broad criteria: current research focus, and context of research published in the last decade.
To ensure comparability across the two groups of experts, I also looked at their professional
standing in their respective fields and institutions. Below is a detailed description of each
candidate participant vis-à-vis the three criteria I used:
Current Research Focus Context of Publications Professional Standing LE01 Studies ecological and Focuses on Northeast North Directs a project group at historical processes as well as America/New England: the Harvard University, won management challenges and environmental management multiple awards for his work, issues of ecosystems in specific and holds at least two places, e.g., Martha’s fellowships with prestigious Vineyard, New England associations hemlock forests
42
LE02 Works on identifying and Publishes on conservation in Directs a program on implementing new initiatives the Americas and Western conservation innovation, and in landmark conservation hemisphere holds multiple research innovation fellowships at prestigious university and advisory boards
LE03 Studies long-term and broad- Focuses on Northeastern U.S. Plays a major role in a long- scale changes in forest forests term ecological research ecosystems, with an emphasis program on quantifying how land use affects forest services and processes
GE01 Uses dynamic meteorology to Publishes theoretical Holds an endowed faculty explain the impact of human explanations of the motions position and at least two activities on Earth’s climate of Earth’s atmosphere using fellowships at prestigious physics, mathematics and associations numerical simulation, in collaboration with scientists who work on global climate change
GE02 Studies atmospheric and Focuses on North and South Directs an undergraduate climate measurements and America, as well as theoretical program in earth and modeling earth system modeling planetary sciences, is a multiple award winner in his field, and holds fellowships from at least two professional associations
GE03 Studies the climate system and Publishes findings on global Won multiple awards for his its implications for society, increase in CO2, the annual work, as well as several specifically in relation to cycle of global temperature, fellowships from prestigious glacial cycles, ocean global volcanic activity, wheat institutions circulation, and Earth's yield trends, northern latitude surface temperature explored temperatures, etc. through observational analysis and mathematical models
A further effort was made to check the local/global orientations of the experts in
their work: following each interview, I asked the experts to mark on a continuum where they
would locate their work in terms of the way they framed their inquiry, the kinds of
43 phenomena they studied, and the way they studied phenomena (the pattern of responses from the experts will be discussed in Chapter 4):
Please put “X” on the scales below to show where you would most likely identify yourself in the work you do:
Frame a local issue Frame a local issue within within a global context its specific context
Systematically analyze or Systematically analyze or model global phenomena model local phenomena
Compare local phenomena Study local phenomena and its across different cases impact in its specific context
To the extent that it was possible, I endeavored to ensure comparability between the two expert groups. However, it was challenging to find precisely matched pairs of experts in terms of disciplinary training, professional standing in the field and home institution, and their local/global orientations in their research. The broad pattern of ascertaining a local/global orientation in my expert sample was an effort to ensure that my sample was a reasonably fair one, especially given the challenges of finding matched pairs of experts.
Furthermore, the sample was also subject to the identified experts agreeing to be part of the study, which turned out to be far more difficult than initially anticipated as most of them were constantly traveling out of state or were doing fieldwork for long stretches of time. Hence, the final sample of experts comprises experts who firstly consented to the interview, and secondly were best matches in terms of the stated criteria.
44 Novice Groups
The two novice groups, comprising high school students who have been actively involved in projects relating to climate change, were selected in terms of one key criterion – a demonstrated high level of scientific knowledge in climate science. Selecting novices for my study rather than those entirely naïve to the domain, ensured that they had demonstrated knowledge of climate change issues as well as general scientific knowledge. Naïve participants would be uninteresting from the standpoint of looking at the development of expert thinking, since they would have no prior disciplinary knowledge to work with.
Comparing naïve participants with experts also runs the risk of any differences being simply the gross effects of domain-specific knowledge, which would offer no insight either pedagogically or cognitively. Furthermore, standard expert-novice studies employ novice rather than naive participants. My sample of award-winning novices was specifically selected for their very high level of accomplishment in climate science with the purpose of making the expert-novice contrast more interesting and instructive.
Participants in both novice groups came from relatively similar socio-economic backgrounds (all of them had parents who held professional degrees and were involved in academic and/or professional work), attended well resourced and academically high achieving schools, and have been successful in elite science competitions. As with the expert groups, they were selected to represent a local/global orientation in their science involvement.
The local novice group comprised two male students and a female student, all of whom are currently the captains of their school’s Envirothon teams that competed at the state and national levels. Envirothon is an annual academic competition for high school students that tests school teams on aquatic ecology, forestry, soils and land use, and wildlife
45 (https://en.wikipedia.org/wiki/Envirothon). It is organized by the NCF-Envirothon, a program of the National Conservation Foundation, and is held by the United
States and Canada on a regional, state, and bi-national level. Teams compete in problem solving as well as written field tests. It is estimated that about 500,000 students from forty- five U.S. states and nine Canadian provinces/territories participate in the competition. The high school attended by the local novices has a strong reputation in the Envirothon, having been placed in the top 5 places in the last three years. The local novices reported a strong interest in environmental issues from an early age, and active participation in their school’s environmental science activities, clubs and projects. These novices were determined by me to be more “local” in their orientation because the Envirothon focused largely on North
America – “from the deserts of the southwestern United States to the frozen tundra near the
Arctic Circle in Canada; from the Everglades to the Olympic peninsula”
(http://www.envirothon.org/the-competition/curriculum-guidelines).
The global novices form the winning team in the Environmental Engineering category of the 2015 Intel International Science and Engineering Fair (Intel ISEF), a program of Society for Science & the Public (SSP), and the world’s largest international pre- college science competition (https://student.societyforscience.org/intel-international- science-and-engineering-fair-2015). The competition attracts more than 1,700 high school students from over 70 countries, regions, and territories to showcase their independent research and compete for more than $5 million in prizes. The global novices in my study were awarded the top prize in several categories for their work on breaking down polystyrene waste naturally by applying pseudomonas putida to produce usable by-products.
In their research, they studied the extent to which their proposed solution could be applied to various countries, hence conducting multiple case studies in their work.
46
Data Collection
Tasks
I interviewed the 12 participants using a structured protocol that included an encoding task, a scenario task using two thinking routines, and a sourcing task (see Appendix
A for the complete protocol). Thinking routines are micro-interventions that focus attention on specific thinking moves; the Connect-Extend-Challenge and the 3Ys thinking routines are selected for this study because of their potential to uncover as well as scaffold complex thinking and global competence. See Chapter 1 for a more comprehensive review of thinking routines.
All twelve interviews were either audio or video recorded. The three tasks required participants to engage with the topic of climate change, more specifically with a proposed solution – geoengineering.
As this study is intended to be exploratory work on developing and testing thinking routines as evaluative instruments, I chose to focus on only one dimension of the Global
Competence Framework (Boix Mansilla & Jackson, 2011) – investigating the world. Such a targeted focus not only makes the study practically feasible, it also addresses a substantively core capacity – i.e., a deep understanding of the world in terms of the history of nations; international dynamics; economic relationships; cultural norms and influences; global issues and trends – that has been referenced in almost every 21st century framework and definition of global competence.
TED Talk on geoengineering
A TED talk on geoengineering, a proposed but controversial solution to global warming, by Harvard professor and environmental scientist David Keith
47 (https://www.ted.com/talks/david_keith_s_surprising_ideas_on_climate_change?language
=en), was used as the content for participants to respond to in all three tasks of the study. In the talk, Keith explains geoengineering the earth’s climate by injecting a huge cloud of particles into the atmosphere to deflect sunlight and heat. This idea, while shocking, is a cheap, quick, and effective solution to climate change. Keith discusses why geo-engineering like this is a good idea, how it might work, and what consequences such a mitigation strategy might have. He concludes his talk with three questions: Should we do serious research on geoengineering? Should we have a national research program that looks at geoengineering? Should we have a treaty among nations that decides who should be in charge of geoengineering the world’s climate?
The original talk of 15:58 minutes was edited down to 12 minutes for the purposes of the study. A transcript of the edited version of the talk is included in Appendix B.
Task 1: Encoding
This task was designed to uncover how participants encoded information about climate change and geoengineering after watching the TED talk on geoengineering. After watching the video, participants were asked to give a short oral summary for a colleague or classmate who had missed the screening. Their summaries were then coded for the connections they made among concepts and ideas, the key ideas that stood out in their recall, etc.
Task 2: Scenario
Following the encoding task, participants were asked what the ideas and information from the video made them think about. When they seemed to be slowing down, or pausing for longer than 10 consecutive seconds, the thinking routine, Connect-Extend-Challenge, was used to prompt their thinking. Next, participants were asked why they thought it mattered that people understood climate change and geoengineering as a proposed solution. Again,
48 when they seemed to be slowing down or pausing for longer than 10 consecutive seconds, a second thinking routine – 3Ys – was used to scaffold their thinking. The final question that participants were asked in this task was how they thought someone from another country such as China or Brazil might think about climate change and geoengineering.
Participants’ responses were coded for how they connected the ideas and information presented in the video to what they already knew, the way they took stock of ongoing questions, puzzles and challenges, reflected on what they were learning from the video, how they framed the issues of climate change and geoengineering, and the way they gauged their significance both locally and globally.
Task 3: Sourcing
In the final task of the protocol, participants were asked how they might gather information for an informed response to a challenge raised by David Keith in the video for an expert and non-expert audience: Should we have a treaty among nations that decides who should be in charge of geoengineering the world’s climate? Their responses were then coded for how they used sources to make sense of a complex issue, e.g., what they paid attention to; how they constructed accounts; what kinds of sources they turned to; what ideas and/or disciplines they thought should be recruited for their response; what connections they made across the concepts and issues; etc.
Instrument Development
To establish whether the protocol measured what it was intended to, I conducted an expert testing review with three experts in the field of global competence, followed by a pilot test with 7 adult participants and 6 high school participants.
49 Expert Testing Review
A panel of three experts recognized for their work in global competence was asked to evaluate the protocol for how effectively it invited responses that represented the dimension – investigating the world – under study. In separate interviews, they were asked whether they thought each of the three tasks would elicit responses that fully represented the four sub-dimensions in Figure 2, and whether each was clearly worded.
Figure 2: Sub-dimensions of Investigate the World
Next, they were invited to give feedback on how well all three tasks together would elicit responses that represented the entire dimension. Finally, they assessed the representativeness and clarity of the instrument, and offered recommendations for improving it (Rubio, Berg-Weger, Tebb, Lee, & Rauch, 2003).
50 The expert panel found that the protocol adequately elicited responses that demonstrated the capacity to investigate the world in globally competent ways, and did not propose any substantive changes to the protocol.
Pilot Test
To ascertain whether the protocol would yield rich data, a pilot test was conducted with 7 adult participants and 6 high school participants. Over a period of four weeks, I recruited and interviewed three adults who were involved in scientific research at universities, four adults who had been enrolled in advanced science courses as undergraduates, and six high school students who were taking AP classes in science.
The pilot test yielded interesting feedback that prompted revisions to the protocol.
For instance, the sourcing task originally offered participants the choice of putting together a response to any one of three challenges raised by David Keith in the video. Four of the high school participants chose to work on the first challenge (Should we do serious research on geoengineering?) while the other two chose the second challenge (Should we have a national research program that looks at geoengineering?). One high school participant initially chose the last challenge, but changed his mind quickly after concluding that he would not have much to say. When asked how they chose the challenges, the high school participants provided reasons ranging from how it was “a primary concern” or “the basic question to ask”, to it being something they could speak to (e.g., “I’m not sure I know much about treaties”; “I’m not into politics”). Two high school participants spoke about how they were not sure
“what’s the difference between national programs and serious research”. In contrast, all the adult participants chose to respond to the third challenge: should we have a treaty among nations that decides who should be in charge of geoengineering the world’s climate? As the different choices selected would pose challenges to comparisons across the sample, I decided that all
51 participants in the actual study would be asked to respond to the third challenge for the sourcing task.
I had also originally planned for the participants to type their responses rather than write, because I had thought that most people would be more comfortable using a keyboard.
However, all but two participants (one adult, one high school) preferred to write on paper because they could make connections among ideas easily, were more likely to think about the ideas (e.g., if it were typed, it would end up being more mechanical), and less likely to just take notes on everything. In the final protocol, participants were given paper and pen/pencil to write their notes.
Another revision to the protocol was the introduction of David Keith prior to the screening of the video. I had begun the pilot study with two adult participants and referred to David Keith in the video as “a Harvard professor”. Both participants wondered whether that would affect the way the high school participants received his arguments in the video.
To test their concern, I used three different descriptions with the high school participants: “a
Harvard professor”, “an environmental scientist”, and “a professor in the field of environmental science”. The two high school participants who were introduced to the speaker as “a Harvard professor” asked to watch the video again, and paused frequently to take notes; consequently, they both took two hours to complete the protocol. One of the two high school participants who were told that David Keith was “an environmental scientist’ asked to watch the video again but did not pause as often as the other students.
The remaining students did not ask to watch the video again. During the debrief at the end of the protocol, the students reported that they would feel pressured to take accurate notes if they were told that the speaker was a Harvard professor, and that they were more skeptical if they were told that he was “a professor in the field of environmental science” because they
52 were not sure if he practiced what he taught and kept up with the field. When pressed to explain their response, they said that professors were usually not scientists who conducted experiments, while scientists did. The protocol was subsequently revised to introduce David
Keith as “an environmental scientist” to avoid skewing responses among the novices.
The pilot study was also a good indicator of what the full study might yield. For instance, with the general interview prompt in the scenario task, the adult participants tended to talk about what was “true” and what they found to be less credible. They also tended to give overview statements, and frequently brought themselves into the conversation, e.g., “as pro-Obama as I am…”, “I’m just like him, I don’t really know”. There were also constant references to the information in the video as David Keith’s narrative about climate change, e.g., “he presented this argument”, “the scientist saw it as”. On the other hand, the high school participants responded with information and many details about climate change that were from the video.
With the Connect-Extend-Challenge thinking routine, some of the adult participants referenced information that was not presented in the video, especially information from other countries and regions in the world, while the high school participants stayed mainly with specific pieces of information from the video and discussed them at length. More of the adult participants spoke about the process of geoengineering as a “fix” and put that in relationship with other mitigation ideas, while more of the high school participants focused on how cheap geoengineering was as a solution. The adult participants also seemed more uncertain about geoengineering as a solution because of the complexity of the problem, compared to the high school participants who focused on trying to figure out how geoengineering actually works (e.g., “so this volcano explodes and then sulfur that’s bad comes out?”).
53 With the 3Ys thinking routine, most of the high school participants interpreted
“community” as their school and family, and also focused on the importance of people starting to discuss and take action about a problem that was “dangerous” and “bad” but where there were “ways to fix it”. They also brought up issues of fairness (e.g., “animals have done nothing but will die”, “developing countries are suffering because of us”) and blame
(e.g., “we need to make countries that are responsible pay more”). The adult participants tended to talk about maintaining balance as an important consideration (e.g., between consumption and mitigation, population growth rate versus amount of resources), used a range of examples not mentioned in the video (e.g., the planet of Mercury as an example of what would happen if we lost our atmosphere), and referenced research studies related to the issue. They also took a somewhat historical view when it came to impact, as well as elaborated on other dimensions like health, politics, and history. A few also raised the point about the U.S. as a global leader with responsibilities. In general, the adult participants interpreted “community” as the U.S. itself, rather than just where they lived or worked.
With the invitation to think about other parts of the world, many adult participants took into consideration the relative economic growth of different countries and regions when thinking about how they would view geoengineering as a solution, and considered the extent to which countries could still save the environment in terms of political will. The high school participants tended to see it as a question of responsibility, and recognized that the problem of too many players in the problem might cause some countries to shirk their culpability. They also spoke at length about the cost of mitigation, and the issue of who should pay more for the solution.
These findings from the pilot study were not only instructive in preparing me for the actual study, but also informed the initial coding scheme for analyzing the transcripts.
54
Data Analysis
I conducted all the interviews and had them transcribed verbatim. Following each interview, I wrote up summaries of the participants’ responses, noting interesting trends and patterns that came up in the individual interviews. Next, I recruited an experienced qualitative researcher, who worked in a different research area, to code with me using the qualitative research web application tool, Dedoose 6.2.21. We independently read all the interview transcripts, and then met to discuss the emerging themes we were seeing. This allowed me to begin sketching a scheme for thematically coding the transcripts (Boyatzis,
1998). I also used grounded theory to analyze and code each transcript using emic codes to surface emerging themes. Then, I organized and synthesized the most significant initial codes that emerged (Charmaz, 2006), and mapped them onto a list of preliminary coding categories drawn from my literature review of expert-novice differences and the Global
Competence Framework (see Appendix C). This preliminary coding scheme produced a total of 14 codes (10 principal codes and 4 sub codes). The final coding scheme features the dominant themes or trends in the data, and contains a total of 6 principal codes and 2 sub codes.
Next, my co-coder and I independently coded four transcripts, one from each sub- sample, which were randomly selected. We then met to discuss areas of alignment and misalignment until we reached consensus for each coded excerpt. I clarified the definition of each code, and revised the descriptions of the codes that were not fully articulated or added examples of the codes to illustrate their meaning. For instance, I had originally described the code “emotion” as “refers or articulates expressions of feelings”, but revised it to “When words or phrases (1) relate to emotions (e.g., `concerned’, `horrified’, `tantalized’); (2)
55 connote an emotional relationship (e.g., `old friends’); (3) are intensifiers such as `incredibly’ and `deeply’" when my co-coder found the description ambiguous. Several codes were also combined when they were felt to be repetitive, or when they were felt to be more accurate as an elaborated code. For instance, we discussed whether the codes multiple perspectives and systems view were qualitatively discrete in the context of the data, and combined them when we were satisfied that it made sense to do so. Using the revised coding scheme, my co-coder and I coded two additional transcripts independently, repeating the process until the kappa statistic for each code was above .70. In total, we co-coded 6 transcripts or 50% of the data.
After all the transcripts were coded, I used Dedoose 6.2.21 to track principal themes across the sample and sub-samples. In order to track the extent to which the thinking routines revealed and supported thinking, I divided the transcripts according to responses given before and following the use of the thinking routines. This allowed me to visibly see how the experts and novices were thinking before the thinking routines were used, and what the thinking routines made them think about.
I then analyzed the relationships among the codes that emerged within each group’s data, noting thematic patterns and discrepancies within each group. Then, I developed cross- group analyses of the expert groups with the novice groups to uncover trends in their thinking patterns, followed by cross-group analyses of the global groups with the local groups. Finally, I examined how the themes compared to the literature on global competence and expert-novice differences, and then used matrices to uncover patterns, themes, overlaps and distinctions among the thinking patterns of the four groups (Maxwell,
2005; Miles & Huberman, 1994).
This in-depth exploration and identification of trends and patterns across my sample enabled me to sketch a beginning picture of how experts and novices engaged with a
56 controversial climate change solution, and the extent to which the use of thinking routines was able to move the novices’ thinking closer to the experts.
Limits and Threats to Validity
Researcher Bias
One threat to the validity of my findings is my familiarity with the field of global competence. I have been a researcher on the Interdisciplinary and Global Studies initiative at
Project Zero since 2007, and much of my work has focused on the nature of interdisciplinarity, global consciousness, and global competence among youth, teachers, and experts in multiple contexts of learning. As such, I may be subject to interpreting the data in ways that confirmed my own understanding of global competence. Therefore, during the data analysis stage, I was highly conscious of how I might be interpreting the data in ways that confirmed my understanding of global competence. Enlisting an experienced researcher to code and analyze a subset of the data with me was especially helpful. By first separately coding and analyzing a subset of responses to yield a preliminary coding system, followed by discussions on disagreements in order to reach consensus, and finally more independent coding to ensure that the coding was consistent (Maxwell, 2005), I was able to keep in check any researcher bias on my part.
Reactivity
Another threat to validity in my study is the potential for unintended distortion of data during the data collection process, which Maxwell calls reactivity (1996). Such distortion could be in the form of phrasing questions differently to different participants, asking leading questions, or unconsciously influencing participants’ responses. This is especially
57 relevant when dealing with the novices, who might try to anticipate what I want to hear rather than give voice to their own beliefs and views.
To minimize the potential for such bias, I standardized the procedure for the scenario task by reading out the same instructions for all participants, rather than raise the interview questions conversationally. This meant that the instructions were phrased in such a way that the youngest participant understood them. I also minimized my interactions with the participants during the tasks so that they responded to the tasks rather than to me.
Finally, I made it clear right from the start that there were no right or wrong ideas, and that participants should feel free to express their views. In these ways, I was able to remain vigilant about my potential influence on the participants.
58 Chapter 4: How Experts and Novices Think about Climate Change
In this chapter, I address the research question that motivated my study: what similar or contrasting patterns of thinking do experts and high school students, studying climate change occurring either globally or within a country/community, demonstrate when they engage with a scenario on climate change using a series of thinking routines?
First, I show how the experts and novices in my study summarized information from the TED talk on geoengineering by scientist David Keith in an encoding task, and then use a diagram to show where and how they converged and differed from one another in the way they represented the ideas from the TED talk.
Next, I present some key findings from the scenario task, where participants were invited to consider Keith’s proposal for geoengineering as a mitigation strategy for climate change. I review their responses both before and after I introduced two thinking routines into the conversation – Extend-Connect-Challenge and the 3Ys – and highlight some trends in the ways the two groups responded to the prompts.
Finally, I show how the experts and novices report they would seek out information in order to respond to Keith’s proposal for a treaty on the use of geoengineering. Using graphs, I describe which particular areas the four groups reported they would look into to gather information for an informed response.
How Experts & Novices Encode Information
In the first task of the study – encoding – the experts and novices were asked to summarize the main ideas from a 12-minute TED talk on geoengineering, a proposed but controversial solution to global warming, by scientist David Keith. This neutral recall task
59 was designed to find out how the participants encoded information from the talk, e.g., what connections did they make among the concepts and ideas presented; what key ideas featured prominently in their recall, etc. Participants were first asked to watch a video of the TED talk, and then provide a short oral summary of the information for a colleague or classmate who had not watched the talk. Prior to performing the encoding task, participants were told that they could take notes on the paper provided, as well as pause the video at any point or rewind it to re-watch any part. All the experts elected not to take notes, while all the novices took detailed notes of the talk, such as the one reproduced in Figure 3.
60
Figure 3 Notes for the encoding task taken by LN01
61 Linear versus Nonlinear or Strategic Thinking
Analyses of the responses revealed that the experts tended to provide summaries organized around big ideas or central concepts in the field of climate science, moving flexibly between those concepts and relevant examples that provided more detail. For instance, GE04 summarized the content of the talk by providing an overview: the speaker
“discussed the pros and cons of geoengineering, laying out a framework to think about it, in which it is rational to understand more about what is possible in terms of offsetting climate change, and also what the risks associated with geoengineering are.” Similarly, another expert
LE01 organized his summary in terms of the disciplines that needed to be involved in considering geoengineering, the contexts within which it should be understood, and the issues that required further discussion:
It was an interesting and well-delivered talk that gave a broad perspective on geoengineering, both in terms of some of the physics involved, the actual mechanisms involved, and the major questions that it raises for scientists and policymakers... The potential for CO2 to influence climate had been recognized for at least 50 years. Unfortunately, that has not been paralleled by a similar ability to act on climate change. The ability of engineers and scientists to effectively combat climate change through geoengineering is something that's also been recognized for at least 50 years. He gave a great example of Pinatubo and its influence distributing sulfur into the atmosphere and having an effect on albedo which helped cool the earth. (LE01)
In comparison, all the novices gave blow-by-blow summaries of the talk, organizing their narratives according to how the ideas were presented in Keith’s talk. Many of them
(N=4) provided drawings of the graphs from the talk as part of their summary, having paused the video at appropriate points to write down the details. They also recited, almost verbatim, parts of the talk. A case in point is LN03, whose summary included almost all the examples and details in the talk:
So the TED talk was all about climate change and something called geoengineering. So climate change, it's been known to be harmful for 50 plus years, but nothing's been done about it. So emissions are rising fast,
62 really, really fast, faster than the worst-case scenario that was projected about 20 years ago. Geoengineering was something mentioned. The idea has been around for a while. It was mentioned in a report to the President of the United States in 1965, but after that it was kind of buried underground until Paul Crutzen, who won a Nobel Prize for work with atmosphere... I don't remember quite what... He wrote a paper on the feasibility of geoengineering. And the basic premise is that if climate change becomes catastrophic; if sea levels are rising, cities going underwater, ecosystems being destroyed, then humans can intervene to mitigate the effects and lessen the effects and try to save what we can. One example of doing this would be to deliver particles to the upper atmosphere, increase the albedo of the Earth, reflect more light, and that would cause the Earth to rapidly cool down, just in case of emergency. And this is actually a very, very powerful method. I wrote down a statistic here. For less than 0.001% of the GDP, that's not a lot of money, you can start an ice age. Because it's so powerful and it has so much potential, there are also many, many issues. For example, something this powerful, some people will see that as a weapon. Some people will see it as a way to extort other nations or gain influence, so they'll fight over it. The speaker used an analogy of a box that could control the weather; people would go to war over it. Also, the knowledge that geoengineering is something that we can do and it can fix the climate or at least halt warming quickly, tells people that they shouldn't be too worried about cutting emissions, especially when it's expensive to cut emissions.
One explanation for this difference in how the experts and the novices encode information could very well be the level of expertise they have about the subject matter.
Another plausible explanation could be their assumptions about the audience for their summary. For instance, the novices might have thought that their classmates would want to have more detailed information from the TED talk, while the experts might have assumed that their colleagues already possessed a certain amount of knowledge about geoengineering and hence would not be interested in details. An interesting paradox here is that while the expertise of the experts is clear in this contrast, the novices’ summaries were considerably more informative for people who have no knowledge of geoengineering.
63 Descriptive versus Critical Syntheses
Another difference between the experts and novices in the way they synthesized the main ideas from the talk was the level of critique they brought to the task. Although it was not required in the prompt, the experts brought a critical lens to their summary, often evaluating the purpose of the talk (e.g., “so it looks like a business plan, a business model to me” – GE02), naming what they thought was missing from the talk (e.g., “He then alludes to another point, again where he doesn't really get into as much as he might, and that is the effect internationally on the community of nations.... Again he didn't have time to really delve into this deeply, he sort of began to touch on it and then shied away.” – GE01), or extending Keith’s ideas (e.g., “what he calls a moral hazard, is equivalent to the fact that we're less inclined to clean up the Mississippi River because we know how to extract water from it and treat it so that we can drink it. The same could be true for geoengineering…” –
LE01).
The novices, on the other hand, provided much more descriptive summaries, focusing on producing an accurate synthesis rather than critiquing the content of the talk:
…geoengineering has the capacity to reduce heat throughout the world, but not without possibly severe side effects. So what the researcher is proposing is that we look more to geoengineering as a way to very quickly actually reduce the heat throughout the world. Yet he stresses the fact that it, one, must not be used as an alternative to reducing greenhouse gas emissions, which can be measured as a sum of the emissions over time. So because of the sum of the emissions over time, we really do need to keep reducing it, because otherwise you're going to have to keep geoengineering further. (GN01)
… it also talked about moral dilemma, in using geoengineering as a means to combat climate change, because that can make the issue of climate change look less serious because it seems more solvable with that, and that can hurt efforts to cut down on carbon emissions, and that can also increase many of the issues that geoengineering doesn't solve. (LN02)
64 The descriptive nature of the summaries that the novices provided, however, belied how thoughtful they were about the content of the talk. For instance, when probed about where and why he had stopped the video and rewound it at several points during the task,
GN01 explained:
I stopped it really first at the part where he was talking about the volcano analogy, because with the volcano analogy, he's showing how it's been done before. And I think it's always really important to understand the original hypothesis and the original ideas that were going through his mind as he was coming up with this, so I stopped it there. And then, I stopped it at the point where he had a bunch of different graphs showing, just trying to understand the idea of that, geoengineering can be used instead of this and it doesn't combat CO2 emission; it combats a different issue, and that is actually something that surprised me because I assumed, I think, maybe a little bit too early on, that it was going to be revolving around CO2, because he was talking so much about it.
A plausible explanation for the experts’ focus on critique is the clear expertise of the experts, and the confidence they have in their standing in the field. The experts consistently identified themselves as “scientists” or being a member of a community of scientists. It stands to reason that they would see themselves as being in a position to summarize broadly, since their knowledge and expertise were not in question, and to critique the ideas presented instead of simply providing a descriptive summary. The novices, on the other hand, likely attended closely to details because geoengineering was somewhat a novel idea to them, making the retrieval and processing of information from the video more effortful.
Concrete versus Abstract Concepts
To analyze the kind of thinking that was demonstrated by the participants in their summaries, I analyzed the concepts and ideas that they used in their representation of the talk using the knowledge dimension of understanding explicated by Boix Mansilla and Gardner
(1998) in their Dimensions of Understanding framework that explicates the qualities of
65 understanding first described in the Teaching for Understanding framework (Wiske, 1998) that was created by a team of teachers and researchers at Project Zero-Harvard University. Boix
Mansilla and Gardner’s Dimensions of Understanding framework offers a systematic way for teachers to assess student work for qualities of understanding such as “disciplinary accuracy, social relevance and critical spirit” (p. 162). This framework was conceptualized using findings from a review of how various disciplines (e.g., history, biology, philosophy, and developmental psychology) build, organize, apply and evaluate knowledge, and empirically tested and enriched using actual student work in English, math, science, and history. As such, the dimensions provide an empirically founded set of descriptors for analyzing the cognitive moves evident in the summaries. The table below presents the four dimensions of understanding – knowledge, methods, purposes, and forms – that comprise the framework.
66
Within each dimension in the framework are four levels of understanding – naïve, novice, apprentice, and master – that provides a clear continuum of the features that demarcated less to more expert dimensions of understanding, which was helpful for analyzing the summaries:
67
The continuum of features in the levels of understanding in the knowledge dimension was used to analyze participants’ summaries because they offered a clear way of parsing out the content of the TED talk into different levels of knowledge. I had hypothesized that the encoding task would differentiate the experts from the novices in terms of the kind of information they focused their attention on. Using the knowledge dimension to analyze the summaries helped me assess the way the participants in my study processed the information provided in the TED talk: what seemed particularly relevant to them; how did they associate or connect the different information provided; and how was the information organized so that it could be retrieved quickly.
68 Recalls using structurally Examples: complex, coherent and rich conceptual webs Climate engineering as human action on the climate
Notion of moral hazard when a solution like geoengineering is used
The importance of and need for environmental policy & governance
Recalls by flexibly moving Examples: between details and overviews, examples and generalizations Geoengineering as cheap (0.001% of GDP) and fast (compared to decreasing emissions) solution
Paul Crutzen’s essay that sparked the current conversation about geoengineering as a solution
Mount Pinatubo as natural example of how geoengineering can work
Example of space aliens who present us with a box to control global temperature and CO2
Climate problem has been around for last 50 years, but nothing has been done about it so far
Lists the 3 questions about geoengineering as a solution
Recalls using specific examples Examples: and/or terminologies, but do not relate them to broader and The mechanics of geoengineering as injecting sulfates OR levitate particles substantive generalizations or into the stratosphere to reflect sunlight away & cool the planet
fertile networks of ideas Geoengineering is controversial (partially destroys the ozone layer and causes ocean acidification)
Geoengineering framed as risk control rather than action
Gas emissions are rising faster than models predicted
There are winners and losers in global warming
Figure 4 Levels of novice-expert understanding mapped onto details from the TED talk
To the left of the arrow in Figure 4, I described the levels of novice to expert understanding
from the framework as a continuum towards expertise. To the right of the arrow, I
reproduced the key ideas from the TED talk, and mapped them onto the continuum. Then,
I analyzed the participants’ summaries using the levels of understanding. Each summary was
coded for utterances that matched the list of examples in Figure 4, and an asterisk was used
69 to represent one such utterance. The columns show how each summary mapped onto the continuum of less to more expert thinking. Figure 5 shows the result of the analysis.
Recalls using structurally * * * * * * * complex, coherent and rich * * * * * * conceptual webs * * * * *
Recalls by flexibly moving between * * * * * * * * * * details and overviews, * * * * examples and generalizations
Recalls using specific examples * * * * * * * * and/or terminologies but * * * * * does not relate * * * * them to broader and substantive * * * generalizations or fertile networks of * * ideas * 01 02 04 01 04 05 01 02 03 01 02 03 GE LE GN LN
Figure 5 Mapping the summaries to the levels of understanding
The analysis revealed that even though both the experts and novices in the study included more or less the same concepts in their summaries, the way they used those concepts differed: the experts tended to locate the concepts within a rich matrix of ideas, building substantively on them, or relating them to analogous cases, while the novices either mentioned them without further using them to think about the issues involved, or discussed examples and details at length without seeing their broader significance or connections across themes.
70 Take for instance the idea of “moral hazard,” which was picked up by both experts and novices in their summaries, but used very differently. The experts used the concept within a much richer web of ideas, such as connecting it to other complex concepts like the tragedy of the commons in game theory (e.g., “This is the tragedy of the commons, the idea that if you have a commons and nobody is responsible for it, everybody will put his cow on it or her cow on it, and then it's going to be ruined and then nobody will have a common” –
GE01), and framing them analogously to other cases (e.g., “what he calls a moral hazard, is equivalent to the fact that we're less inclined to clean up the Mississippi River because we know how to extract water from it and treat it so that we can drink it” – LE01). In comparison, the novices used the same concept of “moral hazard” in less elaborated ways:
…the idea of the moral hazard that revolved around the issue, which is really important, because geoengineering, at least from what he's saying and actually from what I've heard a little bit outside, if you play with mother nature, you never know what you're going to get at the other end. (GN01)
And then I'd tell them more about how there is the issues behind in terms of the moral hazards, so, there is not an issue behind using it per se, or an issue in the engineering aspect of how to use it, but a moral issue in that if we use geoengineering, will that kind of cast a light and say, "Oh, we don't really need to cut emissions, look, everything is being solved right now"? (GN03)
Another concept that appeared in most of the summaries was “mitigation,” and was also used very differently by the experts and novices. For example, GE04 summarized it as
“a framework to think about it, in which it is rational to understand more about what is possible in terms of offsetting climate change,” while GE01 clarified the concept in the talk as “putting a counter CO2, so we have a pollutant CO2 which is increasing the temperature of the earth and there's a counter pollutant, and nature's been using it for a millennia from volcanoes.” The novices, on the other hand, focused on the literal and concrete details of mitigation:
71 He stresses the fact that it, one, must not be used as an alternative to reducing greenhouse gas emissions, which can be measured as a sum of the emissions over time. So because of the sum of the emissions over time we really do need to keep reducing it, because otherwise you're going to have to keep geoengineering further. (GN01)
So I guess he talked about mitigation at first. It would be cheaper than cutting the emissions but again that's a more extreme option and the graph was kind of frightening... But then he talked about really reducing the emissions, reducing the CO2 concentrations and really working on it. (LN01)
How Experts & Novices Respond to a Scenario Using Thinking Routines
The second task of the protocol was designed to uncover how participants think about the information given in the TED talk about geoengineering and climate change using a series of thinking routines. Participants responded to a general question designed to elicit their thinking before the thinking routines were used to engage them with the content of the video. Prior to the first thinking routine, Connect-Extend-Challenge, participants were asked what the ideas and information from the video made them think about. Prior to the second thinking routine, the 3Ys (Why does this issue matter to you, your community, and the world?) they were asked why it was important to understand the issues of climate change and geoengineering.
In this section, I first report the patterns of thinking that participants exhibited prior to the use of each thinking routine, followed by how each prompt in the thinking routines impacted their thinking.
Prior to Connect-Extend-Challenge
On the whole, the responses from the novices to the prompt – tell me what the ideas and information from the video made you think about – were shorter (between 2-5 minutes) than
72 those given by the experts (between 1-9 minutes). The novices’ responses also kept substantively close to the content of the TED talk, while the experts constantly referenced their own work as well as other cases that were relevant to the content of the TED talk.
Time Scales and Historical Lenses
In response to the question about what they thought about the ideas from the TED talk, four of the experts spoke about how it was important to view the climate issue through a much longer historical lens, rather than focus on the latest weather event, an event within the last decade., or even one within our lifetime For them, critical changes in the climate were more accurately understood when viewed across geologic timescales; placing our warming world on that scale provided crucial context and expansive framing for the perturbations that we are witnessing today. For instance, LE05 highlighted how the notion of imperiled “native” tree species in the heated arguments for mitigating global warming really ignored the fact that the temperate tree species we see in New England today are by no means “native” if one took a longer historical lens; he explained that an increase in the CO2 concentrations during the period at the end of the Younger Dryas about 12,000 years ago had precipitated the change from boreal taiga to the hardwood deciduous forest we see today. When viewed within that broader timescale, “volcanoes only have an impact on cooling the Earth for two to three years, maximum. Early 1800s, we had a little Ice Age and we had some severe frost in the middle of the summer here in New England and all that because of a series of severe volcanoes. Pinatubo was not a big volcano.”
Echoing his perspective, GE01 emphasized that geological processes play out over a longer timescale, compared to how humans are used to thinking about causes and effects:
… we have had eight large global ice ages in the last million years. The last one just ended 20,000 years ago, 14 to 20,000 years ago. The earth was just about like it is now, and yet right here there was a kilometer of ice. A kilometer of ice, and Long Island was the end of, where the ice sheet ended
73 and that's why it was piled-up with all that sand, way down there. It's like an eye blink in the history of the world.
According to him, using a broader historical lens when thinking about mitigation strategies mattered, especially because human ingenuity has a long record of surprising us with solutions to problems that were deemed insoluble not too long ago:
There's a lot of technology that can happen in the next 100 years. The idea that we're going to continue to rely on fossil fuels for an energy source flies in the face of our previous experience. There'll be technological revolutions, unforeseen. See, that's the thing. People have this idea that whatever technology we have now is what we'll have in the future. But you only have to look back to see that that's not true.
In the same vein, LE04 proposed that, as a species, we have been able to survive primarily because “the climate is in this very nice sweet spot, and we've seen in geologic time that the climate moved for long periods, both colder and warmer than it is now.” He described warming and cooling as “natural examples of geoengineering.” According to him, such a geologic lens is instructive when thinking about geoengineering, because it allows us to put into proper perspective what we consider doing to the climate since it would highlight clearly possible runaway and unintended effects and side effects.
Prior to the application of the Connect-Extend-Challenge thinking routine, none of the novices mentioned the importance of a longer historical lens through which to view climate change and geoengineering.
Complexity of Systems and Weighing Consequences
When the experts in my sample think about geoengineering and climate change, they consistently bring up the complexities involved, particularly how one action sets in motion a series of impacts that do not necessarily move in the same direction, or within the same time frame. For instance, GE04 described how “we are faced with a number of issues for understanding what will be the consequences of climate change, the magnitude of climate
74 change, and the degree to which we can respond to it.” He went on to explain how understanding the consequences of climate change must, first and foremost, include the physical changes that we should expect, in particular the global average temperature, which is in turn connected to particular consequences, i.e., how specific locations, times of the year, seasons, etc., will have consequences for our way of life. Considering the implications of climate change, according to him, demands that we pay attention to an interrelated set of factors:
So you can imagine growing food will have a different set of implications: how much snow plowing you will have to do in the winter; how hard it is to do outside labor in the summertime. That would just be temperature. Then we would also want to think about damages from high winds – tornados or hurricanes. So then we'd want to talk about the physics that govern those and how they might be influenced by climate change, which is highly uncertain. We'd want to know how sea level is going to change. We'd want to know how precipitation is going to change, kind of making a long list of all the things we would like to know. Of course, what we really want to know is how is society going to look like in 10, 20, 30, 40, 50 years, in terms of activities, economy, resources, etc., and then ask how these local specific changes in climate are going to influence those specific actions, and then we could calculate a difference. But things will be co-evolving. As climate changes, society is going to be changed. And so it becomes a coupled problem. As the climate changes, society is also going to change, to respond in certain ways. So we're attempting to do kind of a coupled prediction. And it evokes a complicated terrain and the more closely you look at everything, you see the greater complexity that's associated with it. And so it is with any one of these calculations. As you look in more detail, you realize how interconnected, how coupled, how temporal and spatial scales really make a huge difference in terms of what you'd ultimately want to know.
Similarly, GE02 emphasized how important it is to view geoengineering as a mitigation strategy within the confluence of systemic effects it is likely to have, and more specifically to unpack the example of Mount Pinatubo as a positive instance of geoengineering: “there's no question that putting particles in the atmosphere would affect stratospheric ozone. Mount Pinatubo, in addition to cooling the climate, also led to a major
75 reduction in stratospheric ozone. Another side effect is that Mount Pinatubo affected the productivity of the biosphere.”
Other experts also demonstrated a propensity to look at climate issues through a complex or systems view, from contemplating how challenging it is to “anticipate all the consequences of something that's never happened before, especially when science has a great history of generating unintended consequences that have oftentimes worse impacts than the problem that they're trying to solve” (LE01), to seeing “the connections between climate, atmosphere and ecosystems, we're talking about a wholesale reshuffling of biomes” (LE04).
Two experts also questioned whether we could really attribute climatic events that we are witnessing to the causes that have been proposed, or even control our climate:
There will be dramatic and devastating large storms that hit populated areas and kill lots of people, whether we engineer the climate or not. After we've engineered the climate, you could construct an argument that says, "Oh, that storm happened in this place because we engineered the climate," but it's very difficult to prove or disprove. Now where does that leave us? We never know. You would never be able to know. (GE02)
We change something here in this environment and it's connected to other things, and it impacts other things, and it's really difficult to predict what the outcome is. We're infantile in our understanding of climate and to think that we could have a knob that we could control rainfall patterns is ridiculous and arrogant. I don't mind some human arrogance. It gets us to the moon and stuff like that. But to say we control Earth or ecosystems, or climate… (LE05)
Among the novices, only GN01 broached the idea of a complex interplay of factors at work whenever we consider climate mitigation; according to him, mitigation strategies such as hydroelectric power might seem environmentally friendly since no fossil fuels are burnt, but they could actually cause massive erosion and destroy entire towns. To him,
“when you're playing with the entire sky, the mesosphere, and the ozone layer, and things like that, there are a lot of different places where that can go very wrong.”
76 Another way that the experts think about the complexity of climate change and geoengineering is recognizing that there are multiple levels of impact, rather than a singular consequence for every locale and time. For instance, GE01 highlights how people respond to climate change at varying levels of complexity and impact: is the issue about heat waves, or is it really a bigger issue? At what point are we justified in taking a largely untested action such as geoengineering?
Well, there've been heat waves. People will buy air conditioners and people will take precautions about the heat and so on. I mean people have lived in hot areas. People say there's going to be heat waves in Cleveland, and that because they aren't as used to it as they are in Dallas, they're all going to keel over dead, but they can learn. I don't think you might make a decision based on something like that. But he brings up this issue of the Greenland ice caps. So if you started seeing something like the catastrophic destabilization of the Greenland ice cap. Five meters of water, 15 feet of water. It'd cause a great deal of damage… If it's just a question of people moving the wheat crop down and the rye crop up because this area is becoming dryer and that wetter, well, you know, maybe you're better off with wheat than you are with rye when you get into looking at it.
All the novices, on the other hand, tended towards a less complex view of climate change and geoengineering. At the same rime, all of them also voiced their support for implementing geoengineering, albeit in varying degrees. For example, GN02 saw it as a solution that should be tested because it “makes total sense obviously, if it doesn't look like there will be irreversible consequences, which I don't think from this it would be. It might do some harm initially, though it would be really, really interesting to see.” Four of the novices also described how it would be “cool” to see geoengineering implemented.
Another consistent view across the sample of novices is the notion that as a species, we should “fix” the climate problem, mostly because we are responsible for it. From that viewpoint, geoengineering made sense to them since it gave us a great amount of control at a low financial cost. LN01 explained that, “it's okay for us to have this kind of control. Yeah, it's not particularly a natural means of it, but most of what we've done as a society has not
77 been terribly natural. I think it's our place to fix this. Yes, some of it is natural, but a lot of it is caused by our influence on the planet. It is our place to correct this and to try to work on it and to reduce some issues.” Echoing her sentiment, GN02, who described himself as someone who has “always been a fan of molding the environment,” argued that being able to control our environment is a capability that we have and should therefore use:
From the most broad conceptual level, do we as humans view the world as a sandbox, or do we as humans view the world as a hard court on which we reside and not something that we can mold like sand, right? If it's truly a sandbox, then we should be able to change every facet and basically every part of our daily lives.
Epistemic Framing
Half of the experts in my sample (N=3) highlighted how important it was to recognize the provisional nature of our understanding of climate change, and to acknowledge the “unknowable-ness” of natural phenomena given our existing tools. GE04 described climate change as a “risk landscape” that is highly diffused:
Climate change poses a whole series of risks, all of which are rather uncertain. More uncertain the further you go out in time and the more regional you go, in terms of exactly what will happen, in terms of changes in the physical climate. Layered on top of that is the uncertainty in terms of how society will be changing, both independent of climate change and dependent on climate change, in terms of adaptation, mitigation or responses to climate.
LE04 used the analogy of Pandora’s box in explaining what he thought about climate change and geoengineering, stating how there was simply no conceivable way we could actually predict unerringly the potential side effects of influencing the global climate, which raises the stakes in any conversation about geoengineering as mitigation:
Could we set up more positive feedbacks into the climate? Could we unleash positive feedbacks in the climate system which could have unintended or unknown consequences? They're just dealing with something with the stakes very high and the uncertainty very high.
78 Echoing the same view, LE01 cautioned that, “whether we can control it or whether we can anticipate and control for the unintended consequences, I think it's something that's completely unknown. And it may be something that's unknowable.”
Recognizing how incomplete our understanding of climate is, almost all the experts in my sample (N=5) considered the multiplicity of impacts of climate engineering if it were implemented, often calling for caution and more rigorous testing to gather greater insight into any mitigation strategy. LE05 likened hastening to use the first promising tool that came along to when new medicines are heralded as a cure: “you don't know what's going on with those things, and we don't know how they interact with the other drugs we're taking.”
Similarly, GE02 alluded to genetic engineering as a forewarning of how things might go very wrong with geoengineering: “You can see something similar in the biotech area where it's long been possible to manipulate human embryos, change their genetic make-up, and people have said, "We should never do that." Well it's been done now. A research that was done on gene therapy has now led to the capabilities to do it, and I think we haven't seen where that play's going to end up, and I think that can happen here too.”
Several experts (N=3) described the kind of ambivalent space that scientists must inhabit, work with, and be comfortable with as a result of the complexity that they work with: how do they weigh the unknowns against evidence and data that point to the viability of a solution? GE01 acknowledges that, “as an idea, there's no question in my mind that the science is solid, that it can be done, that it will be able to turn around the temperature effects of the CO2 increase, and that it will be economically feasible to do.” When he weighed that against the unknown impact of geoengineering, he questioned, “How should we regulate this powerful tool if you had the lever? It is a long lever. How should we use this powerful tool?
It's an important question.” Much along the same lines, GE04 observed that “the potential is
79 there to alter climate in a conscious manner, so that rather than just having inadvertent changes to climate, we could actually enact conscious changes. And we could imagine that, done in a smart way, it might have the benefit of reducing our total exposure. And so there're upsides to it. I think there're reasons to contemplate doing it.“ In fact, he stressed that the smart thing to do is not to turn our backs on the potential of geoengineering as a way to mitigate the impact of global warming, even if “the knowledge that we can do it might in some ways take the air out of our sails from wanting to actually decrease our emissions overall. So, there's a certain danger there. That doesn't mean that we shouldn't understand what the capacity is, though, nonetheless. We just shouldn't rush to saying that this is something we can rely upon.”
To three of the experts in my sample, scientists have a moral responsibility for ensuring that science is used for the good of mankind, and therefore need to hold themselves accountable to higher standards of disciplinary integrity. For instance, GE01 reiterated that the point of research should not be to “enable people to do climate engineering; it’s a more complex question about whether the research is enabling, or whether it helps you to get deeper insight,” because that insight is crucial to making optimal decisions that have wide impact. LE05 cautioned that precisely because “climate change can be pretty bad; there's going to be winners, there are going to be losers. It's mostly going to be poor people that are going to be losing. I think we shouldn't be so fearful that we do stupid things, too, or things that aren't well thought out or well tested.“ And LE01 contemplated how, as a field, “We're worried about hubris, and we're cautious about scientific remedies for improving the world. Any of the remedies can have unintended consequences that are worse than the pests or pathogens.”
80 In comparison, most of the novices (N=4) exhibited a more naïve epistemology in their responses to the issue of climate change and of geoengineering as a mitigation strategy.
For instance, they were less likely to question solutions if those solutions had been proposed by scientists. When asked why he thought the TED talk presenter Keith was trustworthy,
GN01 explained that, “He made points backed up with evidence, and even went so far, he's giving an entire 18-minute presentation, and he's discussing for a very large part of it to why it could be a bad idea. I think when somebody talks a lot about whether it's a bad idea, they actually come across as far more credible. So, because he really did stress the moral hazard point and other points in it, I think we can trust what he's saying, whether or not it would actually be a good, quick, cheap solution, that I can't say, but can I trust what he was saying there? Yes, I can.” Similarly, for GN03, the fact that the presenter was speaking at a TED talk event was enough to convince him that his ideas were sound, because “I kind of just have a trust in him watching his video, and I have a trust in also the TED organization for creating a credited speaker.”
Three of the novices also had no doubts about whether geoengineering should be implemented, arguing that it had been done in nature, and so was “natural” or part of the natural order of things. For instance, GN02 thought that geoengineering was a workable solution and not dangerous because Keith had shown how it was a natural solution already existing:
[Keith] was modeling it from something that is already found in nature, and actually at a very large level. Volcanoes are big; it's not like we're seeing some microscopic organism trying to scale up. You're scaling up a volcano, which sounds almost like an oxymoron. So, I think from that perspective, I think that the bad consequences probably wouldn't be as bad, because it really is the elements of the volcano. Well, we've seen volcanoes actually cooling the temperature around them, that's a scientific fact, right?
81 In fact, GN03 reported a relatively unproblematic view of geoengineering the global climate, explaining how geoengineering was like having “a cut on your arm and you want to use geoengineering for a Band-Aid in terms of healing it over and helping heal it over.” Such a perspective did not take into account the tenuousness of current knowledge described by many of the experts, who highlighted the unknown potential impacts on the world as a whole, and the impossibility of controlling the effects once it was implemented. LN03 took a narrower view of geoengineering, explaining how the main problem with resistance to cutting emissions was cost, and that “people just tend to go for the quick, easy money instead of thinking of the long term, so perhaps one of the best ways to start tackling climate change would be to focus on getting energy sources to the cheapness of coal. Then people will start to switch over.”
After Connect-Extend-Challenge
In general, when the Connect-Extend-Challenge thinking routine was used with the experts, all of them first referred to what they had discussed earlier as their responses to the prompts and then continued engaging with the issues of climate change and geoengineering critically. When prompted to use the thinking routine, all the novices ventured beyond the content of the TED talk to think about the issues raised in the TED talk, and also began to produce more nuanced responses. In sum, the Connect-Extend-Challenge thinking routine did push the experts and novices to engage with the issues more critically albeit at different levels of complexity.
Connections
All the experts immediately made connections between the content of the TED talk and their knowledge of the field (e.g., “I'm familiar with the basic notions of geoengineering
82 and at least roughly how it works through affecting albedo and whatnot.” - LE04), as well as to their own work, either current or in the past (e.g., GE01 pointed to an earlier published paper of his that had proposed that the last ice age was caused by the dust from “an orbital set of parameters which allowed deserts to become more dry and you got dust in the atmosphere. Once the dust got in the atmosphere, it became buoyant, suppressed the precipitation, stayed there, and caused the transition into an ice age”). LE01 also recognized how Keith’s presentation went “way beyond the topic of climate, connecting to an awful lot of scientific inquiry, scientific application, larger issues that face environmental challenges or policy challenges,” identifying with the way that Keith framed the issues within the history of the earth itself, as well as the way he talked about the challenges, the remedies, the uncertainties and the decision-making, because they were the challenges he grappled with in his own work. For LE05, the connection he made to Keith’s talk was more visceral: “I don't remember when I first heard about them injecting sulfur into the atmosphere, but I remember I was horrified.” He went on to enumerate the different red flags that came up for him in his first acquaintance with climate engineering.
All the novices similarly made connections between the content of Keith’s presentation and their own projects on the climate. For instance, GN02 (“we're taking the micro bacteria and trying to scale that up, he's trying to take a volcano and scale that up to the sky, we're trying to just scale that up from the land”) and GN03 (“connecting it to what I do with styrofoam and other environmental issues that I've looked into, so kind of the CO2 being the heart of the climate change issue and the rising temperatures”) immediately related it to their polystyrene project, while GN01 recognized the concepts of gas emissions, the injection of sulfates into the stratosphere, and the intractability of the climate mitigation problem as climate change issues that he had read about. Similarly, LN01 and LN02 also
83 made connections between Keith’s presentation and their own work in the Envirothon club.
For instance, LN01 reported how the “purely scientific facts” from the presentation had been part of the knowledge base that she read up on for the Envirothon competitions, while
LN02 remembered how his Envirothon team also routinely considered moral issues about climate mitigation, such as:
…the impacts of climate change and ways that Massachusetts can make efforts to reduce carbon footprint. So I think it connected to a lot of the issues of addressing climate change: how it influences the public's kind of opinion and actions towards climate change; you know, in what way is it best to talk about climate change, in order to get more people to care about it and take action without being misleading about what options are available.
LN03 referred to a speech that he had written and delivered on climate engineering for a competition, although that had involved using a different method to mitigate ocean acidification.
In comparison, the novices made connections that were more surface-level (e.g., recognizing the terms that were used in the TED talk, or seeing literal parallels between geoengineering and their own work), while the experts connected with Keith’s presentation in cognitively deeper ways, such as finding connections that were conceptually related; recognizing similarities in the way issues and ideas were framed; and using the same historical lens to reason about the issue.
Extensions
Three of the experts spoke explicitly about the way Keith’s presentation extended their thinking. GE02 found it “at least provocative the way he discussed why he thought it was desirable to have research on this problem,” while LE04 “was tantalized by his notion that we could get it [the sulfates] above the ozone and perhaps ameliorate that,” because he was curious about how we could suspend sulfates for the length of time needed for them to
84 have the desired effect, and wondered about the kind of spinoff effects from doing so that could result in unintended consequences. Although LE05 reported being resolutely against geoengineering, he acknowledged being convinced by Keith’s presentation that the impetus towards geoengineering was “already happening and it will get put together eventually and something will happen. We have to have that conversation.”
The novices, on the other hand, focused on the practical dimensions of geoengineering when they were invited to think about how Keith’s presentation extended their thinking. For example, GN01 reported himself to be fascinated by how cheaply and quickly geoengineering worked, and focused his response on finding another possibly more harmless substance that might be used in place of sulfates:
The idea that you could do something so quickly and efficiently is really interesting. What other geoengineering solutions could you have? Well if he's talking about reducing CO2 as being the most important thing, what if there could be a geoengineering solution to reduce CO2? Well, he talked about sending up the sulfur higher… maybe one thing you can do is find something better than sulfur. Because sulfur is acidic, so maybe you'd want to find a particle or substance that would be better than sulfur. Or maybe you'd want to find a way to do it with say water vapor or something.
Similarly, GN02 thought the volcano element was ”really cool actually,” and contemplated how the BioDissolve project did actually work on the same principle; he described how his team had invented a biofilter with Exophiala Jeanselmei fungi to treat the styrene gas released when treating polysterene, so that the gas was no longer carcinogenic:
That's one way that, you've taken something from nature, broken down with micro-bacteria, which is something from nature, something that's man-made and then treated the bad stuff again with something from nature. So, he's looked at it from the volcano element. Is there a way to stop the side effects with any sort of treatment processes with something from nature? That, I think, solving nature with nature is a really cool way of doing it.
GN03 described geoengineering as “a fantastic idea,” and reported being enlightened by the idea that sulfates could be used as a “perfect solution for lowering temperatures and creating
85 kind of a separate supplement to cutting emissions.” For LN01 and LN03, geoengineering was a game-changer in the climate change conversation because of its low cost and potential power.
Challenges
When it came to discussing how Keith’s presentation challenged their thinking, both the experts and novices focused on implementation issues, albeit at different levels of sophistication. For the five experts who responded to the prompt about what they found challenging about geoengineering, thinking about any solution to climate change necessarily involved taking into consideration multiple factors that impacted a very complex phenomenon at varying rates and to different degrees. GE01 referred to “the fine points of the mechanisms of accomplishing this” as a key challenge, i.e., how geoengineering really changes the temperature in the lower stratosphere and the upper atmosphere, which affects the deep convection of hydrological cycles. The consequence, he argued, was that both the amount of deep clouds and the evaporation of water would decrease, thus slowing down the hydrological cycle. Other experts asked:
How good of an analog is it for when a volcano goes off, to different proposals for modifying solar radiation received by the earth? And so, in those cases, you have sulfate and a number of other things that are emitted into the atmosphere. Some of it stays in the troposphere, some goes into the stratosphere. Presumably, that's not how you would do it when you are intentionally injecting reflectors into the stratosphere or other locations. (GE04)
How would that work? What I do know is that it falls out of the atmosphere quick and that we need constant replenishing. (LE04)
Three of the experts also ruminated on how the field of climate science was really riddled with unknowns, and warned that existing methods in the field were simply inadequate for providing the kind of data necessary to make informed decisions that have
86 potentially global repercussions. For instance, GE04 explained that the use of models in climate predictions, while necessary, were woefully inadequate:
You start to run experiments within the context of the model to see what happens. When you get a model result, that's always interesting. It usually leads to at least as many questions as you had going into it: "do I trust the model?" And then, "If I do trust it, do I understand what it's doing? If I understand what it's doing, is that reproducible and applicable to the real world?" In many ways it's not unlike any research question. You make progress. You understand more of the complexity. That leads to further questions.
Similarly, GE01 described our “primitive” understanding of clouds and precipitation, which occur on such small scales that current methods are not able to model them correctly. The danger is that, on the other hand, we are able to more or less accurately predict the effect of geoengineering. He explained:
We know that geoengineering is going to have powerful effects, both on the amount and the distribution of precipitation. Our ability to predict those is far more primitive than our ability to predict the geoengineering cooling. We know, we put the stuff up, it's going to reflect, and it's going to get cool. No problem. But it's going to change this whole hydrological cycle. We have very primitive tools, almost useless tools, to understand what those effects will be.
For both LE05 and GE04, an urgent challenge was how implementation was possible without serious, unintended consequences, and how as a global community, we made decisions about what we were willing to risk. In explaining the feedback systems, both positive and negative, that would be activated in the case of geoengineering, GE04 wondered:
The idea would be to control surface temperatures by reflecting more radiation such that they are not increasing as a consequence of increasing greenhouse gases. But the greenhouse gases are trapping long wave radiation, and so they're increasing the amount of long wave radiation at the surface. What you're doing is you're reflecting sunlight so you have a reduction in sunlight at the surface and you'll have an increase in this lower energy, long wave radiation. That has all sorts of implications. For instance, what does that mean for biological productions? You have less radiation for photosynthesis but you still have the same temperatures. But at the same time, you're scattering more of the light that's coming in and scattered light
87 has different implications for biology than does direct sunlight. And so, you'll quickly get into this ledger of balances. On the one hand this, on the other hand this... And how do the two weigh out? I'm not sure. And certainly, it's going to depend on where and who or what you're talking about.
For two of the experts, taking the longer historical view was particularly crucial, because the impact of climate change really became visible only through a geologically expansive perspective, and through that lens, geoengineering as a solution could be shortsighted:
If you talk to a lot of economists and you say, "Well, I'm concerned about this happening in 50 years." You start applying any conventional discount rate, and you say, "Well, economically it just doesn't matter a whole lot because of that distance." In many ways, it's the problem of climate change in general. It's the tragedy of the commons, right, but it's not even the present commons. It's the tragedy of the future commons. And so it's kind of a doubly hard problem. (GE04)
This Earth has been hit by a meteorite, and asteroids several times. Life almost stopped for a 1,000 years. Oh, glaciers covered most of the Earth for a while. Life came through. And from that perspective of life on Earth, nature will survive climate change. Bangladesh, they're going to have problems. Low-lying nations, some serious issues. New York City, I'm sorry, you're going to have to do something about your subways. There's going to be migration, there's going to be conflicts, it's not going to be pretty. But from a life perspective, life will go on. Trees will most likely still be here. (LE05)
For most of the novices (N=4), the challenge of geoengineering revolved around the mechanics of geoengineering and its actual negative side effects. For instance, a big challenge for GN01 was wrapping his head around the question of consequences:
Would there be a way to stop the process once it's happening? Would there be, "Oh wait, this is a bad idea. Is there something we can do?" You can shut down a nuclear facility, you can take down solar panels, and you can shut the dam and put it back whatever. There's no way to take something back out of the sky really quickly. Have a massive filter somewhere? I don't know. I don't see a really easy solution, there's no ‘off’ switch, so I think most solutions, you want to have an off switch, some type of an off switch.
For others like LN01, GN02 and GN03, questions abound with regard to what exactly would happen if we geoengineered the climate, and how adverse would the effects likely be.
88 Two of the novices in the sample did take a somewhat longer view of geoengineering and its effects. For instance, GN01 discussed the more generic notions of long term impact
(e.g., “Is doing one thing not going to have an effect five, 10, a 100 years from now? You really, really don't know”). Similarly, LN02, while recognizing the moral travesty that geoengineering could visit on a global scale, focused primarily on more commonsensical notions that do not reflect a systems orientation towards the issues: “I think there is a really big moral challenge in that because going through with the project like geoengineering in the atmosphere affects the entire earth; it's very difficult to know what a decision like that looks like because it would affect, you know, the earth and all the ecosystems. So it's hard to know all the impacts and even if you are able to get like all world leaders to agree on it, there's still much more to it than just like a political decision because it also has environmental decisions and people who will be harmed potentially by all these various impacts.”
Prior to the 3Ys (Why does this issue matter to you, your community, and the world?)
On the whole, when the general prompt in this part of the study – why do you think this issue matters? – was used with the participants, all of them reported that they thought the issues of geoengineering and climate change mattered, although the way they discussed them varied. While the experts unequivocally saw them as critically important issues because of their wide-ranging and systemic impact, the novices tended to be more generic in their responses, and were unable to provide a more expansive framing of impact across contexts, distance and time.
Broader versus Narrower Conceptualization of Impact
For four of the experts, the sheer scale of climate change and its potential impact warranted deep concern; to them, it was “one of the many really pressing issues we have”
89 (GE02), “a major problem” (GE01), “an incredibly important topic” (LE01), which “affects everything, simply everything” (LE04). In particular, they referred to the systemic effects of climate change, i.e., how what we are experiencing today is going to spiral into ever- deepening crises:
The climate change problem doesn't go away, it just continues to build over time. And so I think that we will cause serious changes on the earth and will damage places, and civilization will suffer. As a global society, we don't have really good ways of dealing with long-term issues; it's very difficult. We don't do it, in general. So that's why I'm concerned about it. (GE02)
He went on to explain how the globalization of ideological conflicts in the world should be a foreshadowing of how no country is immune to the grave impacts of climate change, because “you can see from the conflicts going on in the world that we're not an island, and those problems will show up back here.” Similarly, LE04 emphasized the indelible interconnections that all life on earth has with the climate, which “controls how much food we have, how much air we can breathe, the whole life support system of the planet is dictated by the climate, and we just need to look back at other periods in geologic time to know how much it affects life… So if you don't want to think this is as good as it gets, then you got to care about climate 'cause it affects everything, simply everything: food, water, all the fundamentals of life.”
On the other hand, the novices’ responses revolved around geoengineering as a solution to climate change, rather than contextualizing its impact more broadly. For instance,
GN01 described how he was “interested in it [geoengineering] to find ways that will make the world better on a large scale and better within cities and other things like that. So geoengineering typically revolves around that where you could take a specific city and make it better really quickly,” not realizing that any injection into the atmosphere will rapidly circulate globally via air streams, rather than stay within a city. Similarly limited is LN01’s
90 idea that humans will not have a huge problem adapting to the potential effects of geoengineering, unlike our natural environment which she sees as less resilient: “since people in general are pretty good at adapting their environment to themselves, but what about the environment itself? The impacts it could have on animals, plants, really any organism in the natural world could be devastating enough to change our own life, I guess.” Interestingly, as the two novices considered impact, their flawed conceptions of the effects of geoengineering are revealed.
Expansive versus Narrow Framing
The experts also focused on the importance of understanding climate issues in an informed manner, such as placing them within longer geological time scales instead of focusing myopically on the present. For instance, GE01 argues that an informed understanding of climate issues must necessarily be placed within a broader frame. While climate change has often been cast as a negative phenomenon, and rightly should be, he cautioned that, as with any issue, the larger picture is more complex, because over geologic time scales, ecologies have been shown to reach new balances:
Carbon dioxide is a carbonic acid, so it acidifies the ocean. So, that changes the whole ecology. So, it's not just that the surface warms. Lots of things are happening because the CO2 is rising. Among them, plants are being fertilized, and there's more lush, more growing, more photosynthesis. There's good. Butterflies that like lush foliage are prospering, and the ones that don't are going down in numbers. Ecological niches are switching around all the time because of this change. The Monarchs are better off and the Viceroys are worse. But this idea that because things are going to change, that any change is for the worse, is not helpful. Ecologies shift, they have balances.
Therefore, realizing that baselines for what counts as “balance” are shifting constantly is an essential concept in of our understanding of climate change.
Compounding the issues of climate change and geoengineering for both LE01 and
LE05 is the political arena that they describe as especially fraught for any informed
91 discussion and decision-making. To the former, the current political landscape “may lead us to a really frightful future, in which, there will be increased interest in applying geoengineering,” without much informed understanding of future impact, or even discussion of alternative strategies. For LE05, the obfuscation of the link between cigarettes and cancer by the tobacco industry served as a powerful cautionary tale about how any productive conversation around global warming would be politically railroaded, and argued that being able to leverage science to build towards a more optimistic future was essential:
I don't feel like the average person needs to know all the depth of climate change, but they should pay attention to scientists and the IPCC when they say, "It's going to get bad in some ways." They should take that seriously. The cigarettes cancer thing was fairly obvious even to someone who wasn't in the research at that time or studied up upon it. But climate change, they've really done a nice job of confusing people. The fact that they've made people stop saying “global warming”, and then say “climate change” was a brilliant tack because, "Oh, climate always changes." That was brilliant as a political move, but I wish there was more faith in science.
This issue of public understanding of science was picked up by GE04, who acknowledged that, to the layman, it must have been challenging to hear the back and forth in scientific findings and conclusions reported in the media. Likening it to a “whiplash” when media reports on scientific findings are later contradicted by other studies, he argued for the importance of understanding the process of scientific inquiry:
In fact, what you're seeing is the scientific process unfolding, but you're seeing it often from the level of the media at a way that you can't actually appreciate what progress actually looks like. Maybe in particular for climate science, there's a lot of ambiguity and uncertainty that's inherent to our understanding of many of the processes. There's things that we know well and you want to be able to articulate that and have people understand why we know those well, but then you also want to inform people about where the surprises are likely to be placed, where the uncertainties are. So that becomes a real important but difficult educational challenge, especially when you're speaking in a polarized political environment that tends to cast things as black or white.
92 Such expansive framing of climate change was not seen in the responses of the novices. For many of the novices, the reasons why climate change mattered were related to their immediate experiences. For instance, GN03 described how the city he was living in was in danger from sea levels rising: “Just think, in New York City, a large portion of downtown is going to be overflowing in maybe 20 years, and you see kind of the encroaching waterlines and the destroying effects.” He also recalled how in the fourth grade, he was told about the habitats of owls in the Central Park (which was located near to where he lived) being destroyed by logging. For LN03, considering future impact made knowing about climate change important because for “the people who like me are in school right now, life expectancy is going up a lot. And when we're 50, when we're 100 or something like that, the effects will be visible, because cities will be underwater. So just people need to learn to think ahead and they can't think ahead unless they know it's happening.”
Environmental Justice
While almost all the experts (N=5) alluded to the responsibility of the scientific community to not only understand, but also do something about climate change, one was explicit about the depth of responsibility he felt:
In particular I come back to the environmental justice. I think the people who will be hurt the most are not people here in the United States, but are people that are elsewhere, and we have a moral responsibility to those people. And I have children, and my children are going to be on the planet much longer than me and then maybe I'll have grandchildren. You know, I guess I feel a responsibility to future generations as well. I tend to think that we are responsible for future generations. If my grandparents hadn't left Eastern Europe, and picked up their whole family and just gotten out of there, then they all would have been killed by the Nazis and I wouldn't be here. So I owe something to them, and likewise I owe something to the next generation. (GE02)
93 After the 3Ys (Why does this issue matter to you, your community, and the world?)
Identity and Responsibility
When the thinking routine was used with the experts, a clear pattern of thinking emerged from their responses: they reasoned through a very clear and strong sense of who they were as scientists and individuals. In fact, all the experts reported how they had difficulty parsing out any substantive difference in their responses as an individual, a member of their community, and a global citizen. For instance, LE04, LE05 and GE04 reported that there was no way the prompts could be disentangled or differentiated since all of them scaled to the same point, i.e., the potential negative repercussions of climate change made it imperative for them to consider the multiple spheres of significance. Similarly, LE01 explained that, for him, why climate change and geonengineering mattered in terms of the three circles of significance (i.e., self, community, and world) came down to the same thing because “they have to deal with the fundamental relationship of human life to global environment, and that's important to me and my community and to the earth.”
All the experts also consistently pointed to their professional identity when they responded to the 3Ys. For instance, GE04 identified himself emphatically as a “climate scientist” in his response, one who wants to understand why the climate works the way it does, and one who has an indubitable responsibility to his profession and to the world to frame an informed response to geoengineering as a mitigation strategy:
Geoengineering is one of those facets of climate and climate change that I just want to understand better. So I have a personal, kind of professional interest in understanding it. Why does it matter to my community and the world? Well, here I think these answers are very similar. We want to understand how effective it might be, how useful it might be. We also want to understand how it might influence international, as well as local events. Do we need to be putting into place policy mechanisms at the national or international levels so as to govern how all these act? How strongly should those be enforced? How much do we want to include this in a potential mix
94 of mitigation scenarios? And so, you know, it's knowledge that's relevant for thinking about future climate and how we might respond to it.
In the same way, GE01 conflated his personal and professional identity in his response, explaining how the question of significance was a moral issue to him because he was bound by his role as a scientist to participate productively in the climate debate:
I'm interested in global change as a scientific problem. And understanding the relation of CO2 to climate is fundamental to climate. So, understanding the scientific issue is important. It also matters to me from the point of view of the scientific community, of which I'm a member; I owe to it to develop a clear understanding of the scientific issues and to scientifically inform the debate, because it's very easy for this debate to be taken over by people whose agenda is not really informed; in the absence of firm knowledge constituencies can take advantage of the lack of knowledge. I think that it matters to me in the sense that I have a stake and a duty as being part of the community to reduce these uncertainties.
For LE04, who spoke of his two young sons, the issue of climate change was compounded for him because “we're looking at climate changes that will make the type of life I've led impossible for the next generation, where my generation or the next generation will be the ones who feel this. So it matters quite directly to my own livelihood, my ability to live in a stable society.”
The experts’ strong sense of identity aligned with their frequent references to their moral responsibility for informing the debate on climate change and for continuing their work on understanding our ecological systems. From stressing how the scientific community had a responsibility for reducing uncertainties over climate change so that corrupt elements in our societies were prevented from capitalizing on it, to explaining the sense of environmental justice that they felt, all the experts made it clear that their response to the question of climate change stemmed from their keen awareness that there were winners and losers in the phenomenon, and that the fallout would be unevenly distributed. It was in that uneven impact that they saw their contribution as mattering, as GE01 explained:
95 I think having a scientifically informed debate is important. Then there's going to be winners and losers. And as a community, however broad we think of that, there's a responsibility of making choices knowledgeable about who the winners and losers are going to be. However the correct action is, it should be taken with full knowledge of who's going to win; who's going to lose because of this.
For the experts, their moral responsibility was fueled by a sharp awareness of how ecosystems, and how every impact of climate change would be felt on a global scale:
It's all about how we're all really connected. Climate change connects us to Bangladesh, whether we want to... Or our clothes connect us to Bangladesh, but we don't get it. But the impacts of what we do here matters far outside here. It's all intertwined. (LE05)
Action and Personal Impact
When responding to the 3Ys, the novices were generally able to contemplate the global scale of climate change. For instance, they saw climate change as powerful and capable of widespread effects (e.g., “You can affect a specific area and still have effects everywhere else” – GN01; “... how many impacts it has on ecosystems and really, just the world as a whole” – LN02). More importantly, their responses began to exhibit a more personal perspective, especially how they felt personally impacted by climate change:
To me, I would say that issue matters a lot because it will be an important part of my kids and my kids' kids, so that's what matters to me. (GN02)
I live in Manhattan, which is an island, so total issues in the future is that parts of the Hudson may be overflowing and part of my home is going to be disappearing. I just experienced the effects of Sandy and the effects that global warming brings on and how it destroys societies and going out to Sandy Hook and seeing how that destroyed and completely decimated thousands of people's homes and seeing the negative effects on that front. So that's personally why it matters to me. (GN03)
It affects everyone, but I just want to say that I myself don't want to live in a world that's going to be destroyed by something we did. (LN01)
96 When it came to why climate change mattered to the world, all the novices connected their responses back to their own lives: water levels rising and ocean acidification;
CO2 levels increasing; displacement of millions of people from their homes; and pollution.
For example, LN03 highlighted how critical it was that we began to take notice of the impact of climate change:
And well, the world is taking the brunt of all of this. I think it was more than half of all people, it's probably higher than that, they live in cities, and most of those cities are on the coast. So if the water level rises, that's half the world's population displaced. So I'd say if that's not important to the world, then nothing is.
He also went on to emphasize how he “honestly can't think of something more important than climate change in the long haul for humankind.”
A cornerstone of the novices’ responses was their emphasis on action that was needed to mitigate climate change. For instance, GN02 emphasized how “CO2 will be an increasingly larger problem if we were to stay on the track of not changing it, of not sort of reducing our levels.” Along the same lines, LN01 explained that it mattered to her that we admitted that we had made mistakes as a species, and that we should try to continuously fix those, rather than just continue on our downward spiral in damaging the environment.
Echoing her, LN02 stressed that:
I think that everyone really has a responsibility to do what they can, to not only reduce their emissions but try their best to encourage other actions to help other people reduce their emissions. You know, it’s part of being, living, on a planet that is undergoing a climate crisis. It's something that applies to everyone. (LN02)
Four of the novices also extolled the advantages of living in a planet that is not ailing: e.g., “water levels and acidity, and just generally having a cleaner and healthier world is good”
(GN01); “well obviously, if sea levels were to keep rising as he was talking about, that would be bad and if the CO2 levels were to keep rising, that would also be really bad. So, for the
97 world, I think the environment needs to stay good” (GN02). They also highlighted the importance of individual recognition of their responsibility for environmental health, and global cooperation in tackling the issue:
So, I think to the world as a whole, it is, you know, the entire world that is being affected by these decisions, so I think in order to really have the decisions go farther and more research to be done, you really need everyone on board, you know, everyone. (LN02)
There are plenty of other issues that are present in the international community now. I don't say this one should take priority over everything else, but I think it should be added on to one of the main issues that we discuss. (LN01)
I think there's also the issue of people believing that because their actions are relatively small on a world scale that that means that they don't affect the world. And I think it's really important to go into the extent of the impacts and the extent that one individual's actions that they can make a difference. (LN02)
Echoing that call to action, LN02 explained that being part of a human community meant that we had contributed to the carbon footprint, and so “it’s our responsibility to reduce its carbon footprint.” LN01 also saw that “since this issue is going to affect anyone on the planet, I think it's important to my community to be able to use at least maybe our education, our resources, to try to do something about it. Or to try to really extend our knowledge to other people or discuss it with people who live in different kind of communities, different areas.”
Comparing across Contexts
When the experts were asked what they thought people in other countries like China and Brazil would think about geoengineering and climate change, most of them (N=4) responded by first considering colleagues who they interacted with from those countries. For instance, GE02 and LE04 cited their experiences working with other scientists around the world, describing how their peers were highly anxious about climate change:
98 Brazil is a tropical country, and it's potentially vulnerable to climate change for a whole variety of reasons, and so that is a major concern to them… The Chinese people I know, who are generally scientists living here, are very concerned about climate change. But they are also concerned about air pollution, I think, as a social issue and pollution of the land too. (GE02)
Anywhere, there's going to be this population that just aren't going to think about geoengineering or much one way or the other. But I think China right now is seeing this vast transformation of their country and the ability to change things. My gut is that they would look at it more optimistically than a country that wasn't in the middle of a big economic surge where a lot of industrialization and transformation… People in Gabon, where I spent a lot of time, are not as poor as the rest of West Africa because they have oil, but they do feel screwed. They feel screwed the way the rest of the continent feels screwed. I think Africa, as if somehow they believe it would be worse for them. (LE04)
Two experts expressed the view that people in other countries were no different from those in the U.S., in that they had the same aspirations, needs and anxieties, and hence it was crucial that any actions taken on behalf of the environment must truly effect global good. GE01 believed that “people in those countries, although they have a different culture, are like me. They want to be happy and prosperous.” Similarly, LE01 expressed hope that people in the world would share his concern about climate issues, especially if it came to a situation where the power to engineer the climate feel into the hands of global powers with advanced technological capabilities.
The issue of “rights” also cropped up for GE01 and GE04, who highlighted the environmental injustice of a few countries mining the world’s resources with little regard for future impact, and often at the expense of the countries who are now suffering the brunt of climate change:
I think that they have a very good point, that the developed nations have gotten their development by exploiting the world's resources, and that they have a right to development. I mean, they have a right to a better life and a right to as good of life as other people and to be told, "Well, there's this danger now and you can't have abundant energy resources like we had because we caused the problem." I mean, you can see how if I were in Brazil or China, I would consider that to be more than unfair. (GE01)
99
Another line of thinking is to say that climate change causes harm to people and will cause net suffering around the world. That's bad, and we have an obligation to figure out ways to reduce the amount of suffering that will occur. (GE04)
When it came to considering how people in other countries might think about geoengineerng and climate change, the novices’ responses tended towards more unproblematic notions. A case in point is GN01’s belief that the promise of technology connecting people globally had in fact created a more global mindset among young people, even though there was no evidence for that:
I think a lot of people, especially because everybody's so connected now with social media and everything, feel very connected to people at the other side of the world. I think kids my age actually do feel connected to people across the globe, so absolutely, I think that there is a much more global mindset maybe than there was in previous generations. (GN01)
In the same way, LN01 assumed that opposition to mitigating the climate came from the wealthy or well connected, and all that was needed to get global support for geoengineering the climate was to give the issue enough airtime:
It's just that the people who are actively against it might have, say, more money or a higher audience to talk about it. That at least makes their representation a lot higher. Even though it's likely that it's probably a much smaller population of people who actually believe this. So I'd say possibly in those communities there are plenty of people who are interested in maybe geoengineering or at least in fixing this problem. I think if someone or if this issue is talked about enough globally, I'm sure enough people would support this idea.
To her, climate change was a “relatively simple issue to discuss, to understand, and to try to fix, and that seeing as more people do believe it exists, then it makes sense that it would exist, I'd imagine it would be easier to communicate.” In contrast, LN02 cautioned against jumping to conclusions about geoengineering, citing the many unknowns about it that should give us pause:
100 I think in anywhere in the world, there is a lot of just unknowns about geoengineering, and to a large extent, there's a lot that I don't know and there's a lot that all of us don't know about what it really would cause to happen, and what issues it would solve, and what issues it wouldn't, and what issues it might create. So I think it would be really hard to reach a consensus on what a group of people would think about it, because there's just such variance in what we know, and to some extent, we all don't know all that much about it. (LN02)
While GN02 made fewer assumptions about Brazil and China, he nonetheless demonstrated equally simplistic views on other fronts, such as expressing an uncomplicated view of who should have the right to geoengineer the climate:
So I think China needs to look at it themselves and not go to geoengineering yet, but I think for more developed countries like America, where we actually reuse the most energy of any country in the world, I think it's more acceptable. I believe South American countries are more industrializing now, so I think their focus and China's focus should be on, "How can we cut CO2 emissions the old fashioned way?" If we want to get into a geoengineering discussion later on for those countries, fine, but not now and they should focus on just cutting CO2 for now.
The Value of Thinking Routines in Supporting Thinking
In this section, I discuss the extent to which the thinking routines used in my study –
Connect-Extend-Challenge and the 3Ys – succeeded in supporting the novices’ capacity to investigate the issues of climate change and geoengineering toward more expert ways of thinking. I do not focus on the impact of the thinking routines on the experts’ thinking primarily because I am using their thinking patterns as the expert anchor levels in assessing the effectiveness of the thinking routines. Furthermore, as anticipated, the experts more or less responded to the prompts in the thinking routines without using the thinking routines, i.e., in response to the general prompts used, they discussed how the ideas in the TED talk connected to what they already knew, the ways in which they have extended their thinking, what they found puzzling or challenging about the idea of geoengineering, and why the
101 issues mattered to them personally, to their community and to the world as a whole. After I introduced the thinking routines, all of them reported that they had responded to the prompts earlier, and explicitly pointed out where their earlier statements had answered the various prompts.
Using Thinking Routines in General
In general, the thinking routines prompted the novices to spend more time thinking about the issues of climate change and geoengineering than they did when they were asked the more general prompts. When the thinking routines were used with the novices, the time most of them spent thinking about the issues of climate change and geoengineering increased by one to eight minutes. While the length of time spent on task may not always translate into actual learning (Karweit, 1984), the Connect-Extend-Challenge and 3Ys thinking routines, in this case, prompted the novices to engage substantively with the issues in ways that went beyond intuitive notions of climate change.
Using Connect-Extend-Challenge
In general, the Connect-Extend-Challenge thinking routine succeeded in engaging the novices in thinking about the issues of climate change and geoengineering for a much longer time than when the general prompt was used. More specifically, most of the novices (N=4) stayed on the issues at least twice as long as they would have without using the thinking routine. The exception was GN03, who reported that he had already discussed most of his connections and extensions prior to the use of the thinking routine, and so spent his time focusing on the Challenge prompt. Chart 1 below presents the amount of time that the
102 novices spent discussing the issues prior to and after using the Connect-Extend-Challenge thinking routine.
Taking a Wider and Longer Perspective
Prior to the introduction of the Connect-Extend-Challenge thinking routine, none of the novices considered the issues of climate change and geoengineering using a longer historical lens. Using the Challenge prompt of the thinking routine, two of the novices were able to take a somewhat historical view of geoengineering and its effects, discussing the more generic notions of long term impact as well as recognizing the moral injustice of the impact of geoengineering for different communities and locales. However, their responses still focused primarily on more intuitive notions that did not reflect a multidimensional orientation
1 03 towards the issues, and further support will need to be offered to encourage them to view the climate issue through a longer historical lens, instead of focusing on the latest weather event, or an event within the last decade, or even just within their lifetime.
Complexity of Systems and Weighing Consequences
The novices’ responses to the general prompt on the whole did not consider the complexities involved in geoengineering our climate, compared to the experts who were particularly concerned about how one action could set in motion a series of impacts that did not necessarily move in the same direction, or within the same time frame. Only one novice broached the idea of a complex interplay of factors at work in climate mitigation.
Following the use of the thinking routine, more of the novices began to consider greater complexity around geoengineering, although they focused more on the practical dimensions of implementing geoengineering (e.g., the physics involved) than on the potential multiple impacts that could happen. Whenever they considered actual negative side effects, it was specifically confined to questions about the amount of CO2 in the air or the type of sulfates that should be used, whereas the experts raised questions regarding other interrelated systems that could potentially be impacted by geoengineering, such as the atmospheric circulatory systems (e.g., clouds as an essential component of the global water cycle).
Epistemic Framing
One key area where the Connect-Extend-Challenge thinking routine did not move the novices toward expert thinking was in developing a more sophisticated epistemology about the nature of science and knowledge. The novices continued to exhibit a more naïve epistemology in their responses to the issue of climate change and of geoengineering as a mitigation strategy: they entertained no doubts about whether geoengineering should be
104 implemented, and were instead convinced that it could not be harmful since it was found in nature itself, such as volcanic eruptions.
In Summary
The Connect-Extend-Challenge thinking routine was designed specifically to help students make connections between new ideas and their prior knowledge, as well as encourage them to consider questions, puzzles and difficulties about what they were learning. In light of those cognitive aims, two important shifts in the novices’ thinking about climate change and geoengineering following the use of the Connect-Extend-Challenge thinking routine were observed. Firstly, where prior to the application of the thinking routine, none of the novices placed the issues of climate change and geoengineering within a longer timeframe, two of them adopted a longer, more historical view when Connect-Extend-Challenge was used to support their thinking. Secondly, while three of the novices discussed the practical aspects of geoengineering as a plausible mitigation strategy prior to the use of the thinking routine, the number increased to five following its use.
Using the 3Ys
In comparison to the Connect-Extend-Challenge thinking routine, the 3Ys was more varied in its success in engaging the novices in thinking about the significance of climate change and geoengineering. As shown in Chart 2, two of the novices spent less time discussing significance, compared to the amount of time they spent when the general prompt was used. Another novice spent an equal length of time discussing significance before and after the 3Ys was used. An important point to note about these three novices is that they had discussed significance at length prior to the introduction of the thinking routine; in fact, each of them spent at least twice the amount of time used by the other
105 novices in discussing significance prior to the use of the 3Ys. For the remaining three novices who showed an increase in their time spent on task after the thinking routine was used, the time used was at least three times longer.
Uncovering the significance of global issues like climate change is by no means an easy task. It requires that we understand the issues within their broader ecology of impact, as well as position ourselves within that ecology to understand how we fit into that scheme of things. What the results of my study show is that for young people who may not even have significance of issues on their mind, the 3Ys may be a very helpful strategy for focusing their attention on issues that demand understanding.
106 Responsibility and Taking Action
Following the use of the 3Ys thinking routine, the novices were comparatively better able to discuss the global scale of climate change as a primary reason why the issue mattered.
Perhaps more importantly, their responses shifted to a more personal perspective, focusing on their personal encounters with the impact of climate change. What was striking was how they framed the responsibility for climate change as a collective responsibility, and used words and phrases such as “all of us”, “everyone”, and “as people living on the same planet” when discussing collective responsibility for climate issues.
A key development in the way the novices thought about climate change and geoengineering was their increased attention to solutions: they consistently discussed the importance of taking action to mitigate climate change, and put that imperative within the broader problem space of our collective responsibility as the human race to ensure the survival of our planet.
In Summary
The 3Ys thinking routine was designed specifically to help students uncover the significance of a global issue by directing their attention to three overlapping spheres – me, my community, and the world. In light of this cognitive aim, I observed key shifts in the thinking of the novices following the use of the 3Ys thinking routine. Firstly, they were comparatively better able to discuss the global scale of climate change, and they adopted a more personal perspective on the issue of climate change. Secondly, they also became more attentive to the importance of taking action to mitigate climate change, and began to consider taking action as our collective responsibility. The latter is especially significant given that none of the novices discussed responsibility or action before the use of the 3Ys, but all of them did so after it was used.
107
Perhaps more compelling is that the use of the 3Ys prompted the experts’ (N=6) to frame their conceptions of significance through a very clear and strong sense of who they were as individuals and professionals; prior to the use of the thinking routine, none of them spoke to that.
Seeking Other Sources of Information
In the final task of the study, participants were reminded of Keith’s challenge in the
TED talk – should we have a treaty to decide who gets to do geoengineering? – and given time to think about how they might gather information for an informed response for a mixed audience of experts and non-experts. Participants then explained their strategy, and their strategies were coded for what they reported as important sources they would have consulted to frame their response. When participants’ strategies were analyzed, a few themes emerged:
• Treaties – this related to
1. Examples of successful global treaties, both relating to climate change (e.g., The
Montreal Protocol) and other political ones like nuclear agreements among
nations, global plans against human trafficking, and trade agreements (N=8);
2. The process of creating binding treaties (N=6); of particular concern to
participants who mentioned treaties in their strategy was how to bring together
a group of relevant and informed people to decide on and enforce a treaty that
would be successful.
• Geoengineering – this captured references to
1. David Keith’s website, publications and talks (N=2);
2. Public opinion about geoengineering (N=1);
108 3. The cost and technology involved (N=2);
4. Using the Internet as a source for information (N=1);
5. Examples of current field tests of geoengineering (N=2);
6. Scientific literature about geoengineering (N=5); and
7. A proposal for a small-scale test to be designed to test the impact of
geoengineering (N=1).
• Expertise – this referred to consulting experts from different fields about their
thoughts on geoengineering (N=2).
• Climate change – this was raised by 1 participant and referred to finding out more
about climate change and its proposed mitigation strategies.
Chart 3 below summarizes the broad themes of treaties, geoengineering, expertise, and climate change on a radar plot to show what each group in my sample focused on in the sourcing task.
109
The expert groups in my sample are similar in that equal numbers of them considered treaties to be a critical theme to focus on for their response. For the novice groups, equal numbers of them considered geoengineering to be a critical theme to focus on for their response. Within the expert group, more of the global experts focused on geoengineering
(i.e., scientific literature; examples of geoengineering field tests; Keith’s website, publications and talks) as an important theme in their responses, while only one local expert thought the same way (i.e., scientific literature). In the case of the novices, more global novices saw treaties as an important focus compared to the local novices. While both the novice groups had equal numbers naming geoengineering as a theme to focus on, the two groups differed in important ways: more global novices described how they would look for information in
110 scientific literature compared to the local novices. Also, the local novices referred to public opinion, cost and details about the technological aspects of geoengineering as important for their responses, while none of the global novices mentioned them.
Global/Local Orientation
When I first designed my study, I had hypothesized that the contexts within which the experts and novices worked would systematically impact the way they think about climate change. In distinguishing between what I classified as “local” and “global” experts, I used the following criteria: “global” experts were likely to frame a local issue within a global context, or systematically analyze or model global phenomena, or compare local phenomena across different cases; “local” experts were more likely to frame a local issue within its specific context, or systematically analyze or model local phenomena, or compare local phenomena and its impact in its specific context. With the novices, I categorized the
BioDissolve team as “global” novices because their project required them to study how their solution would play out in different countries, and the Envirothon team as “local” novices because they studied specific micro-processes of climate change and considered them within the context of their home state, Massachusetts.
A close analysis of the findings in my study revealed that the global experts and local experts were not substantively different in the way they framed issues, analyzed phenomena, or evaluated the impact of mitigation strategies. The experts exhibited similar trends in their thinking about climate change and geoengineering: following the interview, I asked each expert to use a “X” to mark where they would place themselves on a continuum of local to global work foci that I provided. The continuum is reproduced below:
111
frame a local frame a local issue within a issue within its global context specific context
systematically systematically analyze or model analyze or model global local phenomena phenomena
compare local study local phenomena phenomena and across different its impact in its cases specific context
The diagram below shows where the experts identified themselves on the local-global continuum. Each sphere represents where the experts chose to place themselves. In several cases, the experts reported that their work orientation could not be accurately described as only local or only global, and so they chose to place themselves more than once along the continua.
112
On the one hand, this finding testifies to how climate issues really cannot be confined to either a single locale or context. On the other hand, it throws into question my hypothesis that my study could uncover thinking patterns that are more global or more local in their orientation. I return to this latter point in Chapter 5.
The global novices and local novices similarly exhibited similar thinking patterns: they were more likely to employ linear explanations of events and processes, focused more on shorter timeframes to situate issues and ideas, and took a more solutions-oriented
113 approach to climate issues. In only one area were they different: the global novices were able to draw from a wider repertoire of examples of climate change solutions, conceivably because they had conducted research to consider different solutions to the problem of
Styrofoam. Conversely, the local novices had mainly participated in an academic competition on climate issues, the Envirothon, which involves a problem solving presentation as well as written field tests on five core subjects: aquatic ecology, forestry, soils and land use, wildlife, and a fifth annually changing subtopic. Teams did not have to actually work on a climate issue. Instead, they were asked to assess the effects of climate change in their community and to recommend steps that their city or town and individuals, including young people, should take to address the issue.
Conclusion
My study uncovered some interesting expert-novice differences in my participants’ patterns of thinking around issues of climate change and geoengineering, both corroborating what the rich tradition of expert-novice studies has uncovered to date, as well as offering some nuances in domain-specific cognitive patterns such as geological framing of events and processes. The novices’ responses also demonstrated how thinking routines might support them to push beyond intuitive ideas and begin to adopt a broader as well as more expansive perspective of climate change issues.
It is important to note that the sample size of my study is small, and while the contrasts identified warrant interest and comment, it is not clear that they would hold up as trends in a larger sample.
114 Chapter 5: Towards More Expert Thinking Using Thinking Routines
In this chapter, I discuss my findings on how novices and experts engage with complex issues like climate change and geoengineering, and suggest some implications of those findings for classroom teaching and learning. More specifically, I examine the extent to which the two thinking routines used in the study helped learners move toward the patterns of thinking exhibited by the experts when engaging with complex issues like climate change
(e.g., how they generate questions about and uncover the significance of a global issue; use sources to identify and weigh relevant evidence; analyze, integrate, and synthesize evidence; and develop compelling arguments and draw defensible conclusions), and propose some key factors that educators interested in preparing learners for more globally competent investigations of the world may want to pay attention to.
I begin by describing three key features in the way the experts in my study engaged with the issues of climate change and geoengineering that are specifically related to globally competent thinking: geological (versus ecological) framing of issues; reasoning from a clearly articulated identity and worldview; and epistemic orientation towards knowledge. I examine how these three patterns of thinking compared with the novices, and then propose why they matter as we prepare learners for our contemporary world.
Next, I discuss the ways and extent to which the thinking routines used in my study supported the novices toward the experts’ patterns of thinking, and propose ways in which educators may use the thinking routines to assess how learners are thinking about an issue, as well as support learners toward more globally competent thinking.
I then suggest some implications of my findings for educators interested in supporting more globally competent thinking in their students, with particular emphasis on
115 the use of thinking routines in making their thinking visible. I conclude by identifying some limitations of my study and propose how further research may illuminate other pressing questions about developing learners’ global competence in investigating the world.
How Experts Think about Climate Change
The novice-expert paradigm in research is not new. Across a slew of disciplines and contexts, studies using this paradigm have uncovered many rich and often surprising insights about expert-novice differences in memory ability (Chase & Simon, 1973), mental representations (Chi, Feltovich, & Glaser, 1981), and strategies for problem solving (Novick,
1988). The findings from my small-scale, exploratory study generally corroborate those insights. Here, I highlight two areas that have been especially distinctive in my study.
Firstly, the experts in my study demonstrated broader or more expansive thinking that was organized around big ideas or central concepts in the field of climate science, compared to the novices who focused their summaries on details and examples that had been used in the TED talk by the speaker to illustrate those big ideas and central concepts.
This is consonant with findings from previous studies that revealed experts’ fluency with fundamental procedural principles and knowledge meta-structures that allow them to rapidly grasp meaningful relations among seemingly disconnected pieces of information (e.g.,
DeGroot, 1965; Chi et al., 1981; Charness, 1989; Chase & Simon, 1973; Ericsson & Kintsch,
1995).
Previous novice-expert studies have also systematically revealed how experts unpack complex systems with multiple interacting components at a more sophisticated level than novices are able to (Hmelo, Holton, & Kolodner, 2000; Wood-Robinson, 1995; Graesser,
1999; Narayanan & Hegarty, 1998; Resnick & Wilensky, 1998). In my study, the novices
116 tended towards a relatively unproblematic view of climate change and geoengineering. For instance, they do not discuss the potential for unintended and irreversible consequences of geoengineering, instead focusing on how “cool” and “interesting” climate engineering is as a solution. They also lauded how it promises to be a quick and cheap fix for what they acknowledged to be a hitherto intractable problem. Half the novices in my sample also proposed a “small-scale experiment” in one city to test the effectiveness of injecting sulfates into the atmosphere, even though one of them had previously noted that air streams were likely to carry any particles to other parts of the world. In comparison, the experts were more likely to hedge their bets about geoengineering, consistently calling attention to invisible agents or actors in the system or considering emergent properties that make prediction challenging.
While the above findings from my study illustrate broader theories and previous findings about how novices perform in comparison to experts, they relate to more generic thinking patterns, rather than specifically addressing globally competent thinking. In this section, I highlight three trends in the way the experts in my sample engage with the issues of climate change and geoengineering that exemplify globally competent thinking.
Geological Framing
The experts in my study characteristically viewed climatic events and phenomena through geological time scales; in their responses to the study prompts, they frequently referred to the ways that cases and events have played out against the history of Earth or deep time to illustrate or augment their argument, cautioning how those cases and events take on vastly different significance when measured out against a timeline that goes beyond our lifetime. From highlighting how the notion of imperiled “native” tree species in New
117 England today no longer held if we looked at the transition from boreal taiga to the hardwood deciduous forest we see today as a result of an increase in the CO2 concentrations during the period at the end of the Younger Dryas about 12,000 years ago, to viewing the state of our climate today against the backdrop of the eight large global ice ages we’ve had in the last millennia, to realizing the resilience of our planet and life on it after sustaining multiple meteorite and asteroid strikes, theirs is a decidedly geological perspective that goes beyond an ecological view. The experts distinguish between “ecological timescale” and
“geological timescale”, the former referring to how one thinks about events that take place within one’s lifetime, and the other to when and how events have occurred throughout the history of
Earth. Although expert-novice studies have found that experts were more likely to take a longer, more historical view of events and causes (Jones & Read, 2005; Hmelo-Silver,
Marathe, & Liu, 2007), the experts in my study consistently used a much longer breadth of historical time as a lens to investigate climate change issues.
In comparison, the novices in my study tended to view events and issues within a more ecological timescale, i.e., they framed their responses within the perspective of events that they had experienced or heard about in their lifetime. For instance, a common reference that threaded through the responses of the global novices was the devastating impact that
Hurricane Sandy in 2012 had on New York, and the looming threat of rising sea levels that now forms their memories associated with the Atlantic hurricane season. GN03, for example, saw the significance of climate change as indubitably tied to the fate of Manhattan, where “parts of the Hudson may be overflowing and part of my home is going to be disappearing,” and immediately, and simplistically, saw it as synonymous with “the effects that global warming brings on and how it destroys societies and going out to Sandy Hook and seeing how that destroyed and completely decimated thousands of people's homes and
118 seeing the negative effects on that front.” This was despite the fact that Hurricane Sandy was measured as a Category 3, not 5, hurricane on the Saffir-Simpson Hurricane Wind Scale, which is based on sustained wind speed. Similarly for the local novices, the Atlantic storms in recent years had become the yardstick by which they understood and evaluated the impact of climate change, consistently referring to sea levels rising to endanger cities on the coast and displacing scores of people. When the novices considered the significance of climate change, they also evaluated it within how they anticipate their own lives would play out, describing what they thought would be its impact on their children and grandchildren, or themselves in a few decades from now.
Why does this matter?
What affordance might viewing events and issues through such a geological timescale have for young people today, one might ask. Firstly, the experts amply demonstrated how the way we view controversial and often bitterly contentious issues results from the baselines that we identify as the basis for measuring change. For instance, GE01 argued that the increase in CO2 is not all doom and gloom; for every one species that is adversely affected by CO2 rising, there is another species that thrives because of it. To further complexify the issue, what qualifies currently as a flourishing period for certain species like the Monarch butterflies may not be permanent, given that other variables are changing and evolving in the same period. So, “ecologies shift, they have balances” (GE01). The question of whether the increase in CO2 is advantageous or harmful, therefore, really depends not only on whose perspective we take, but also over what span of time we are referring to. If we assess global temperature from year to year, the picture that emerges will look dramatically different from one that compares it over geologic periods of one hundred million years. What counts as
“accurate” becomes a matter of parameters, or where the baseline is. Such an understanding
119 of how arguments around climate issues are framed becomes intensely important as it helps learners understand how baselines for decisions can shift depending on where one chooses to stand.
Secondly, the value of the experts’ geological perspective matters not only in their capacity to assess impact over a longer timespan and to predict from a broader base, but also in how it reveals their non-anthropocentric framing of events and issues. When the experts describe ecological shifts and balances, assess geoengineeering as a mitigation strategy, or explain why climate change matters to them, they refer to processes, variations, interactions, and evidence that revolve around what we know about the climate system. When the novices respond to the same prompts, they tend to view it from the perspective of human-related actions and impact. Such an anthropocentric perspective filters their responses through an almost exclusively human perspective. From their laser focus on how humans will be impacted by climate change to how humans need to take action to ensure their own survival, from linking all climate variations to human actions rather than recognizing that to a certain extent, the climate has always been changing, to interpreting the “world” as a human one
(e.g., “So, for the world, I think the environment needs to stay good” – GN02), the novices did not take the more expansive view that the experts demonstrated, i.e., that the earth would be here long after the humans are gone, and that the earth will survive, albeit in a different form, or it will look very different from how we know it today (e.g., it may not be habitable for humans). When the experts think about climate change, they consider the ecology of systems that interrelate and interact to both cause and ameliorate the climate we have today. In that framing, humans are just one contingent part of the picture. For the novices, humans are the center of the picture.
120 Of course, one can attribute this difference to the experts having had more complex and nuanced academic, professional and life experiences that have informed their thinking about climate change; novices are understandably less likely to have had to develop a more geological timescale in the way they view events and issues. Furthermore, the fact that the novices’ frame of reference is their own experiences is not necessarily detrimental to their learning; there is some evidence that understanding specific instances of climate change impacts can help people to better understand the phenomenon of climate change itself, and possibly motivate them to act on behalf of the issue (Clayton, Manning, & Hodge, 2014).
Researchers have also found that the phenomenon of climate change can be far removed from people’s experience and consciousness, both in time and space (e.g., Hulme, 2009;
Swim, Clayton, Doherty, Gifford, Howard, Reser, Stern, & Weber, 2009; Rudiak-Gould,
2013), and when they have opportunities to study specific cases of climate change impacts that have personal relevance to them, their understanding of the phenomenon is improved
(e.g., Akerlof, Maibach, Fitzgerald, Cedeno, & Neuman, 2013).
That being said, this distinction between novices and experts is still a valuable insight for educators designing learning experiences that put learners within the zone of proximal development (Vygotsky, 1978) for developing a more informed way of understanding complex issues and problems like climate change. When learners are given the tools and guidance to bridge the gap between viewing complex issues and problems through an ecological lens versus a geological one, they advance in the way they unpack complex issues and problems, and in the case of climate change, come closer to seeing, for instance, the challenge of the climate debate as a battle over what GE04 describes as the “tragedy of the future commons.”
121 Identity and Worldview
One consistent lens that the experts in my study reasoned from was their identity as scientists. Whether they were responding to the question of the significance of climate change and geoengineering from the position of an individual, their community, or the world, they unanimously did so from the position of scientists with a moral responsibility to not only provide scientifically accurate information to the public, but also to do so in a morally responsible way. While the novices responded to the 3Ys with qualitatively different ideas, the experts emphasized how their answers to the three prompts in the thinking routine really could not be separated. For instance, for GE04, “This is a global issue. Its repercussions will be global and we can't really understand its consequence for a community or an individual without thinking about it in a global context.” Similarly, for LE01, “the reason it matters is very similar to all three, and that maybe the attitude that one would have towards them might differ, but they're all important because they have to deal with fundamental relationship of human life to global environment, and that's important to me and my community and to the earth.” The experts’ lens for uncovering the significance of climate change was clear and stable.
While seeing the significance of climate change and geoengineering through a single identity may seem to be a narrow lens to take, the responses from the experts revealed how multifaceted that identity was, particularly in terms of what it represents and entails. All the experts referred emphatically to the moral responsibility they have to the public and the scientific community to ensure that the debates over climate change are informed by sound science, because the political decisions that result from those debates not only invariably impact the lives of many people living in different places and contexts, they also have potential impacts that persist inordinately over time, many of which are not immediately
122 obvious to us. Who stands to gain from such decisions, and who becomes marginalized through no fault of their own? The notion of environmental justice was clearly uppermost for most of the experts in their discussions about the impact of climate change and our responses to it, and they framed it unequivocally as a critical moral responsibility that scientists must shoulder:
And as a community, however broad we think of that, there's a responsibility of making choices knowledgeable about who the winners and losers are going to be. However the correct action is, it should be taken with full knowledge of who's going to win, who's going to lose because of this. (GE01)
And in particular I come back to the environmental justice. I think the people who will be hurt the most are not people here in the United States, but are people that are elsewhere, and we have a moral responsibility to those people. (GE02)
I would like to see a Supreme Court stocked with senior scientists or people of that nature. I'm not saying that these intelligent lawyers and judges can't understand the environmental issues, but there are technicalities that are more important than a patent. So you argue over a patent, someone makes a billion dollars and someone doesn't. At the end of the day, it's good for this person and not great for that person. But when you make a mistake on the environment…that affects so many people beyond. It affects innocent people. (LE05)
Resonant in their discussions was also a strident concern over how science could be appropriated for the wrong uses by those motivated primarily by monetary gains:
… it's very easy for this debate to be taken over by people whose agenda is not really informed. They have a purpose to use maybe the threat, but their agenda is not to serve anyone but themselves. You see, there are people with impure motives, with self-serving motives. They couldn't do that if our scientific knowledge was clear and they wouldn't get away with pretending something was true that wasn't. So I think that it matters to me in the sense that I have a stake and a duty as being part of the community to reduce these uncertainties. (GE01)
My concern is that if with enough money and enough spin you can keep people confused for a really long time and some cataclysm happens. So I'm concerned that we never get there and so my solution to this problem, if there is one, is to try to reform the media or try to find some way to change the way the messaging reaches people. (GE02)
123 At the same time, their similar concern about the misuse of science for nefarious purposes belies how experts do not necessarily see eye-to-eye: while GE01 strongly believed that no tainted agenda could cast doubt on an issue as long as the science about it was clear, GE02 took the opposite stance, i.e., that even if the science were clear, the general public could still be confused by clever messaging.
In comparison, how the novices positioned themselves vis-à-vis issues about climate change was more generic (e.g., “I've also felt like, I think, like a moral push to learn more about climate change because, to me, I feel a responsibility in protecting those ecosystems and doing my part in reducing my carbon footprint” – LN02; “I'm a science-oriented person. I like Earth generally” – LN03), and lacked the urgency explicit in the experts’ responses. For instance, one of the experts shared an intensely personal biography of his commitment to ensuring that we mitigate the impact of climate change:
I tend to think that we are responsible for future generations. If my grandparents hadn't left Eastern Europe, and picked up their whole family and just gotten out of there, then they all would have been killed by the Nazis and I wouldn't be here. So I owe something to them, and likewise I owe something to the next generation. (GE02)
While the novices also referred to their desire to contribute positively to mitigating the impact of climate change, they did so in much broader strokes, e.g., “So to me personally,
I'm interested in it to find ways that will make the world better on a large scale and better within cities and other things like that” (GN01); “molding the environment around us, and optimizing it, I think is a really, really good course for humanity” (GN02).
Clear markers of emotional salience were also evident in the experts’ responses, while comparatively fewer instances of them showed up in the novices’ responses. In the analysis of the transcripts, I noted the extent to which the participants in my study used emotionally salient language by coding for (1) words or phrases associated with an emotion,
124 according to Shaver, Schwartz, Kirson, and O’Connor’s list of emotions (2001) such as
“confused”, “tantalized”, “horrified”, etc.; (2) words or phrases that are generally recognized as connoting an emotional relationship, for instance when referring to ideas as “old friends”; and (3) when intensifiers such as “incredibly” and “deeply” are used to make adjectives stronger. This revealed the high level of emotional engagement most of the experts demonstrated when engaging with ideas and facts about climate change and geoengineering
(N=5), compared to the novices (N=1). I noted two kinds of emotional commitment voiced by the experts: (1) emotion about ideas related to climate change (e.g., “I hate the idea of putting sulfur into the atmosphere” – LE05; “I was tantalized by his notion that we could get it above the ozone and perhaps ameliorate that” – LE04), and (2) emotion about the facts of climate change and mitigation strategies (e.g., “I didn't like that very much” – GE02; “I guess
I am very skeptical about that” – LE01). For GE01, the ideas raised in the TED talk were
“old friends” because he had been working with them over the span of his career; whenever he spoke of them, there was a clear affection for those ideas. LE05, on the other hand, was particularly animated when discussing the facts of other proposed mitigation strategies:
And they're so absurd sometimes. "Oh, well we'll just put white blankets on Greenland." I get the concept, but really? How are you going to build a blanket that big? What kind of resource is it going to take? How much pollution are you going to produce? And then when all the pollution you produce ends up falling on top of the blanket and turning it less white, then what are you going to do? Flip it over? Vacuum it? I mean, it's ridiculous.
In comparison, the one novice who used language that was coded for emotion did so only when he talked about his personal interest in ecosystems (“I've really always loved learning about the ecosystems”), but in the same sentence, referred to his relationship with the issue of climate change in more generic ways: “I've also felt like a moral push to learn more about climate change because, to me, I feel a responsibility in protecting those ecosystems and doing my part in reducing my carbon footprint.”
125 One explanation for this difference is likely that the experts in my study, who are distinguished scientists in a prestigious research university, have made a huge commitment to the field of climate science. Therefore, some of the emotion that I identified in their responses may well reflect not only sheer depth of technical knowledge, but also self- selection into a field that they care deeply about. What is instructive, however, is that bound up with their expertise is a clarity about how their own professional and personal biographies make environmental justice a code that they feel compelled to live by. I propose that expertise, as revealed by my participants, is an interacting confluence of deep knowledge and epistemic stance (cognitive), clear professional identity in a community of experts (social), and emotional commitment to justice (emotional).
Why does this matter?
A concern that some parents and educators have about teaching currently intractable global issues like climate change, poverty, and terrorism in the classroom is the danger of such issues creating a sense of despair and hopelessness in young people. This concern is certainly not unfounded. A recent report summarizing the psychological impacts of climate change on human well-being (defined in the report as “human flourishing and resilience, beyond the absence of injury or disease”) was published by the American Psychological
Association and ecoAmerica (Clayton, Manning, & Hodge, 2014), citing research that found women, children, and older adults to be “especially vulnerable to the psychological impacts of climate change, especially those related to stress and anxiety” (p. 6). The report went on to describe the condition of ‘ecoanxiety’, where witnessing the slow but irrevocable changes in the environment and worrying about their impact on not only ourselves but future generations creates a sense of helplessness, fatalism and frustration. Earlier in March 2013, an NPR radio report on the implementation of the Next Generation Science Standards, created
126 by the National Research Council, in schools, “A Hot Topic: Climate Change Coming to
Classrooms”, (http://www.npr.org/2013/03/27/174141194/a-hot-topic-climate-change- coming-to-classrooms), quoted spokeswoman Heidi Schweingruber explaining how careful her team had to be in crafting the standards because they wanted to be sure that teachers were supported in “how to deliver what can be crushingly depressing information, without freaking kids out.” The same report also featured Mark McCaffrey of the National Center for Science sharing stories of students who, after learning about climate change in school, ended up with an acute sense of guilt over their families’ hefty carbon footprint:
"everybody's upset because the parents are driving their kids to the soccer game, and the kids are feeling guilty about being in the car and contributing to this global problem.” Clearly, there are valid grounds for the worry about engaging young people in thinking about pressing global issues like climate change.
What has been instructive about how the experts in my study reasoned about an issue is their clarity about the perspective they took and the ways in which that perspective allowed them to define the parameters of their commitments and actions. Because they clearly identified as “scientists” who have the moral responsibility of informing scientific debates and contributing to a more just world, their responses toward the issues of climate change and geoengineering aligned indubitably with that position: for them, any decision or mitigation strategy must take into account the winners and losers, i.e., environmental justice must be a key consideration. Being part of a community of scientists provided a clear lens through which they understood problems and considered solutions. That lens is also emotionally involved, as the experts candidly reveal a range of emotions in their responses, from being horrified by the proposal that we should geoengineer our world and ignore nature’s capacity to re-balance itself, to being tantalized by the idea of floating sulfates into
127 the stratosphere, to feeling grave concern at our limited understanding of its impact. The role of emotion in moral reasoning cannot be underscored enough, and is instructive as we consider how the development of a strong position or role in complex issues may help young people to grapple with challenging issues like climate change.
When discussing issues of climate change and geoengineering, the novices in my study frequently indicated their interest in finding a solution to mitigate human actions on the environment, but when pressed to elaborate on why that mattered, they fell back on more generic answers, e.g., to make the world better for all. Despite prompts used by me to specifically probe their personal views on significance (e.g., say more about why you think that way?), I was unable to get them to discuss how they viewed their personal involvement in the issue. They also rarely, if at all, used language that indicated emotion. Of course, it is conceivable that for the novices, science is a rational process of observing and describing a phenomenon, formulating hypotheses, developing predictions, and testing hypotheses – a lock-step body of techniques that are subject to principles of reasoning. In such a conceptualization, emotion or emotional thought hardly has a place. Unsurprisingly, many students today still think of science as an unemotional enterprise as they are taught the scientific method as a formulaic procedure, contrary to what research has revealed about how scientists really work. For instance, Boix Mansilla, Sato, and Lamont’s multiple case- study of experts working in cutting-edge research groups (2010) found that their research experience went beyond cognitive collaboration; the researchers reported being intellectually passionate about their work with their collaborators, excited by the social interactions in their network, and deeply engaged by their collective enterprise to explore new ground in pursuing their inquiry questions. Similarly, Dott (1998) rejected the way science textbooks portrayed the scientific method as orderly and objective, calling it a “sterile myth [that] leaves
128 no place for hunch, intuition, serendipity, prejudice, vigorous advocacy, nor rancor, all of which have played important roles in the history of science (Dott, 1998, p. 15).
Hence, if we are to adequately support young people to engage with challenging and complex global issues in globally competent ways, it becomes imperative that we do so by providing them with opportunities to develop a sense of their identity within that problem space. Who am I in this issue, and why should I care about it? If that is who I am in this issue, then what are some parameters for how I ought to make my decisions? If indeed people are more likely to orientate their behavior towards justice when they are more aware of who they are (i.e., self concept) in that situation (Skitka, 2003), and if our emotions direct our moral and social choices (Immordinio-Yang & Damasio, 2007), then it stands to reason that we should encourage young people to develop a personal stance or position within an issue or complex situation, so that they are able to feel a deeper investment in how it unfolds, and how they may play a role in it. In many classrooms, the farthest that students get to thinking about an identity when grappling with complex issues is through role-playing, e.g., taking on the roles of different stakeholders in debating the building of a power plant in a less affluent neighborhood. While there is merit to having students take on multiple perspectives, it is important to note that rarely are they invited to consider their personal identity in those learning experiences. Learning from the experts, who are able to clearly articulate their intellectual excitement in the work they do and develop a moral purpose for why the work warrants their attention, educators can design opportunities for learners to clarify why and how those issues matter to them personally, in the process inviting them to investigate the multifaceted nature of who they are and how the issue affects them, and to develop a personal stance that consistently interrogates the evolving problem space. Such authentic
129 personalizing of the issue may direct their efforts at understanding the issue, and developing a moral position vis-à-vis it.
Epistemic Orientation
While the experts in my study discussed at length their reticence about geoengineering as a mitigation strategy because of the lack of empirical data we have about its impact across multiple measures, the novices reported their interest in how cheap and quick geoengineering promised to be. Where the experts focused on what we do not know, the novices attended to what we do. For instance, the novices tended to describe geoengineering as “cool” and “interesting”, often punctuating their responses with assertions of how the human race had to find some way to solve the problem of climate change. There was much discussion of the mechanics of geoengineering – how would it work? What substitutes are there for sulfates? – and hardly any references to the unknowns in the scenario. Only one novice discussed the unknowns involved in manipulating the environment.
A theme that cropped up consistently in the responses of the experts (N=5), and less so in those of the novices (N=2), was trust: what they trust and distrust; who they trust and why; and how the notion of trust is critical to the field of environmental science and beyond.
The way the experts and novices spoke about trust was also qualitatively different. The experts consistently reported that the scientific process could be trusted: during the encoding task, they generally expressed confidence in Keith’s description of how geoengineering would work. For instance, they acknowledged that sending sulfates into the upper atmosphere would certainly deflect the sun’s rays away from the earth, and consequently lower global temperatures. They also agreed that the Mount Pinatubo example used by Keith did indeed demonstrate how the method does work. Despite that confidence, the experts
130 singularly warned about the limitations of current models to accurately predict the impact of geoengineering. A case in point is LE01, who observed that, “we have a very poor understanding of [the unintended negative effects]. And so, he's [Keith] got this big lever that he can operate and I'm sure that it will work. But whether we can control it or whether we can anticipate and control for the unintended consequences, I think it's something that's completely unknown.” Similarly, the other experts’ trust in the scientific process was consistently counterbalanced by a strong skepticism about its predictive power:
Well, what I don't like about the acid, the sulfur idea is that it falls back to Earth. Yeah, it interacts with the ozone layer, but I don't like the idea of sulfur falling to Earth at high quantities over short periods. Yeah, volcanoes do it, but that's part of nature, and that's fine. But we don't know the consequences of that falling back to Earth. What if we acidify ecosystems even further from this, ocean or land? I don't trust it. I just don't, I want to see serious research that it's not harmful. (LE05)
Do I trust the model? And then, if I do trust it, do I understand what it's doing? If I understand what it's doing, is that reproducible and applicable to the real world? (GE04)
Clearly, the experts were concerned over how much geoengineering was needed to achieve the desired cooling, as well as how current models of impact are simply inadequate for making informed decisions about issues that not only traversed multiple dimensions such as politics and ethics, but also highly uncertain and potentially wide-ranging in impact.
Consistently in their responses, they raised questions about geoengineering in terms of unintended effects that could be invisible to us given our limited tools, or so far off in the future that they currently raised no immediate red flags, or deemed inconsequential because they are happening in a place and to people far away.
These experts also highlighted the highly provisional nature of scientific knowledge itself, from how “there’s a lot of ambiguity and uncertainty that’s inherent to our understanding of many of the processes” (GE04), to the nature of climate science as simply
131 unknowable, “a miasma of not knowing and being confused” (GE01). They also constantly reference the complex ecological relationships that operate in our environment, warning how the ways one variable impacted others up and down the feedback loop were really difficult to predict, because the feedback was always happening within several interrelated nodes, and often in multidirectional patterns. Conversely, the novices tend to construct more linear causal patterns involving, at most, two to three variables in their thinking about climate change and geoengineering, generally failing to see the multiple interrelating processes that regulate ecological systems (Pimm, 1982; Ricklefs, 1993; White, 2008).
The experts’ capacity to question, slow down and deliberate on the issues involved, consider other data and not jump to quick conclusions, be comfortable in doubting, hesitating and qualifying, dramatically contrasts two of the novices’ certainty about the scientific process behind geoengineering (Kahneman, 2011; Covitt, Harris, & Anderson,
2013). Both GN01 and GN03 did not distinguish between trusting the scientific process and trusting the predictive power of scientific models; instead, they spoke of trusting the content of the presentation and the speaker David Keith in the TED talk, citing reasons ranging from how any speaker who provided an alternative point of view was trustworthy, to how
TED talks generally featured trustworthy speakers, to his position as a scientist at a prestigious university. They also rarely recognized the uncertainty in geoengineering, and were quick to accept the information in the TED talk without actually evaluating the nuances involved (Evans, 2003, 2008; Stanovich & West, 2000). This is consonant with existing developmental models of personal epistemology that suggest a general developmental trajectory in one's personal epistemology: regardless of the number of stages, positions, or perspectives posited by each model, the sequence invariably involves movement from a dualistic, objectivist view of knowledge to a more subjective, relativistic
132 stance, and ultimately to a contextual, constructivist perspective of knowing (Hofer, 2002;
Hofer & Pintrich, 1997; Schraw, Bendixen, & Dunkle, 2002; Wood and Kardash, 2002;
Schommer, 1990). I propose that the novices in my study seem to straddle an absolutist way of knowing (i.e., knowledge is viewed as certain and absolute, facts are stressed, and expertise becomes the basis for knowing) and an emerging multiplist epistemology about knowledge where they are beginning to exhibit a certain skepticism about expertise generally, and awareness that experts not only disagree but are inconsistent over time (Baxter Magolda,
2002; Hofer & Pintrich, 1997). This finding challenges theories about epistemological development that link absolutism with early childhood (e.g., Kuhn, 1993, 2005); the award- winning novices in my study attest to the likelihood that absolutist notions about ways of knowing may well persist into late adolescence, particularly when complex issues that challenge the limits of current understanding are involved.
It is important to clarify that because the interview questions in my study focused on climate change and geoengineeering, the sophistication of expert epistemology that is revealed points to a domain-specific sophistication; the way they think about complex causality and specific ambiguities involved in climate change is directly related to their deep expertise in the field. Yet, it is also conceivable that they exhibit a more sophisticated way of understanding the world in a general sense. There is no real way to separate out the two contributions of general cognitive development, and sophistication in a particular domain, to their more complex way of knowing. Instead, I acknowledge that there is likely an interplay between general epistemic sophistication and domain-specific epistemic sophistication in the way the experts in my study engaged with the issues of climate change and geoengineering.
133 Why does this matter?
Contemporary society is marked by a surfeit of information from a variety of sources that compete constantly for our attention. The capacity to interrogate carefully what counts as valid information, for what reason and context, and for when and why, becomes critical if one is to understand an issue, weigh choices, and make informed decisions. Boix Mansilla and Jackson (2011) argue that individuals will need to recognize the constructed nature of knowledge, and understand that what we know today can change when we have better explanatory frameworks for our world. As such, the skills of interrogating claims and evaluating evidence become crucial markers of preparedness for global citizenship. The experts in my study embrace the notion of scientific knowledge as humanly constructed rather than as a set of facts that are awaiting discovery in the world (Sandoval, 2005); they consistently demonstrate the practice of science as a thoughtful interplay between data and evolving theories, advanced through rigorous testing of intuition and constant experimentation (Duschl & Osborne, 2002; Erduran & Jimenez-Aleixandre, 2008; Kuhn,
1993; Kelly, Druker & Chen, 1998; Lehrer, Schauble & Petrosino, 2001).
Globally competent investigations of the world demand an epistemological understanding that recognizes the provisional nature of knowledge, engages in cognitively effortful processing of information that relies less on heuristics and more on culturally specific knowledge, and reflects on the alternatives available (Evans, 2003, 2008; Stanovich
& West, 2000; De Neys & Glumicic, 2008; Evans, 2007; Klaczynski, 2004). The novices in my study are high performing budding scientists with demonstrated curiosity about their world and the drive to work towards deeper understanding. While their experiences in their respective competitions already place them ahead of the game vis-à-vis their peers in
134 scientific inquiry, there is still much that needs to be done if they are to become enculturated into a more expert epistemic stance that is at the same time globally competent.
Thinking Routines & Globally Competent Investigations of the World
In general, the thinking routines were helpful in extending the novices’ substantive attention to the issues under discussion. The novices spent more time thinking about the issues of climate change and geoengineering than they did when they were asked the more general prompts. They also went beyond intuitive notions of climate change as they devoted more time to thinking about it. In particular, the thinking routines succeeded in supporting the novices to think more systematically about a highly complex issue, compared to when more general prompts were used.
The Connect-Extend-Challenge thinking routine generally engaged the novices in thinking about the issues of climate change and geoengineering for a longer time than the
3Ys. On the other hand, the 3Ys thinking routine was able to engage novices, who had not discussed the significance of climate change in any substantive way before it was introduced, for a substantially longer time. Given that the thinking routines are designed to scaffold different kinds of thinking, this finding underscores the important of recognizing the utility of each thinking routine, and applying them to relevant tasks.
If the endpoint of teaching students about complex global issues such as climate change is to support them to arrive at novel questions, explanations, and solutions, frame questions to explore and investigate issues of global importance, and to develop coherent and compelling responses (Boix Mansilla & Jackson, 2011), we would want to ensure that students move beyond intuitive responses that are more automatic and cognitively economical, and develop a processing system that reflects on and evaluates intuitive
135 responses in a more systematic and cognitively effortful way (De Neys & Glumicic, 2008;
Evans, 2007; Klaczynski, 2004; Stanovich & West, 2000). My study has demonstrated that thinking routines, when used thoughtfully and relevantly, have the potential to engage young people’s substantive attention to an issue or question that they might otherwise respond to in ways that are more intuitive, commonsensical, and automatic.
Connect-Extend-Challenge
The Connect-Extend-Challenge thinking routine supported at least two of the novices to take a somewhat historical view of geoengineering and its effects; following the introduction of the thinking routine, they were able to discuss the more generic notions of long term impact as well as recognize the moral injustice of the impact of geoengineering for different communities and locales. However, they were still unable to take a multidimensional orientation towards the issues that considers the issues through a longer historical lens. Of course, the Connect-Extend-Challenge thinking routine was not designed for those purposes, and in the future, it will be helpful to create one or more thinking routines to specifically address this cognitive capacity.
The Connect-Extend-Challenge thinking routine did move most of the novices to consider complexity in their responses to climate change and geoengineering. This was not the case prior to the use of the thinking routine. However, their attention remained on the more concrete aspects of geoengineering, such as the practical dimensions of implementing geoengineering (e.g., the physics involved; the amount of CO2 in the air). Ideally, we would want to move them toward more expert thinking, such as raising questions about other interrelated systems that could potentially be impacted by geoengineering. An important
136 limitation of the study is that the thinking routines are used only once, which is contrary to the spirit of thinking routines. Thinking routines are designed to be used consistently over time, and their impact is likely to be much more visible when learners are able to adopt them into their repertoire of thinking strategies.
Using the Connect-Extend-Challenge thinking routine did not result in the novices demonstrating a more sophisticated epistemology about the nature of science and knowledge. Instead, they continued to exhibit a more naïve epistemology in their responses to the issue of climate change and of geoengineering as a mitigation strategy. Many researchers have argued for science education to focus on developing learners’ epistemological understanding, citing its corresponding impact not only on learners’ academic progress in science (Leach et al, 1997; Metz, 2004; Sandoval, 2005) but also other school subjects (Buehl & Alexander, 2005; Mason & Boscolo, 2004; Mason & Scirica, 2006).
Others have proposed that a sophisticated epistemology is critical for productive living and learning in general (Kuhn, 2005; Kuhn & Park, 2005).
However, it must be remembered that the Connect-Extend-Challenge thinking routine was not designed expressly to nurture a more sophisticated epistemology in students. At most, the Challenge prompt could be seen as an invitation to epistemic complexity, although it is more a prompt for thinking about challenges of sorts, rather than a scaffold for epistemic complexity. It stands to reason that a thinking routine that specifically targets a sophisticated epistemology would have a greater impact on the novices’ responses. Unfortunately, that is beyond the scope of the current study.
137 3Ys
The use of the 3Ys thinking routine did shift the novices’ attention onto the global scale of climate change as a primary reason why the issue mattered. Their responses also took on a more personal character as they recalled personal encounters with the impact of climate change. Before the use of the 3Ys, the novices did not discuss responsibility or action before, but all six of them did so after it was used. As with the Connect-Extend-Challenge thinking routine, a more sustained use of the 3Ys is likely to have a greater impact than my study was able to have.
Additionally, 3Ys had a strong impact on the experts’ conceptions of significance: following its use, all six experts discussed the significance of climate change and geoengineering through a very clear and strong sense of who they were as individuals and professionals.
Assessment Potential of Thinking Routines
The findings from my study make a good case for thinking routines as an assessment tool. As both the experts and novices engaged with the issues of climate change and geoengineering using the thinking routines, trends in their thinking patterns became evident.
For instance, the novices’ responses to the Connect-Extend-Challenge prompts revealed how they were able to consider more complex impacts of geoengineering the climate, although they were unable to explain the complex interacting variables involved in the mitigation strategy. They were also more concrete in what they paid attention to, focusing on details about geoengineering, such as the practical aspects of implementation. The thinking routines also revealed clear gaps in the novices’ thinking, such as their naïve notions of science as fact and assumptions that anything occurring in nature must be natural and good.
138 On top of revealing student thinking, the thinking routines also clearly scaffolded thinking. In my study, both the experts and novices were pushed to engage in more critical thinking as they responded to the prompts in the thinking routines. By using a general question prior to the thinking routines to surface the novices’ assumptions about climate change, I was able to set a baseline with which to compare their thinking after the thinking routines were used. The novices’ post-thinking routine responses provided important information about how they were thinking differently, as well as where they still clung to their more naïve notions. Therefore, the use of thinking routines can provide information on whether learning has taken place, as well as productive insights into how students are thinking as they develop 21st century competencies.
In Chapter 1, I had argued that teachers looking to support students toward more globally competent thinking needed a flexible and simple assessment tool that effectively diagnosed where students were in their thinking, and that could be easily integrated into existing lessons. Thinking routines meet those criteria, because they comprise a few steps, and can be used at any point in a lesson to support and reveal student thinking.
An important next step in realizing the potential of thinking routines as assessment tools is to develop thinking routines that specifically target the learning challenges that students typically come up against when grappling with global issues. To that end, scaling up my study to include a larger sample and more thinking routines will uncover patterns of thinking that hold in the population, and consequently be the basis for a rubric with an empirically based set of descriptors that provide clear criteria for understanding where students are at in their thinking, and how they might progress to more sophisticated levels of global competence. Such a rubric is especially helpful to students, because it provides
139 important information to them about what global competence looks like and where they are vis-à-vis public criteria for 21st century competencies.
In designing my study, I had hypothesized that experts and novices working in global versus local contexts would exhibit different patterns of thinking. Much of the literature on global competence and global citizenship focuses on the concept of “global” versus “local” thinking, e.g., global thinkers are inclined to think across contexts, while local thinking focuses on a specific context. My study has shown little difference between the global experts and local experts, as well as between the global novices and local novices. Instead, as anticipated, there were considerable differences between the thinking patterns of the experts and the novices. A few possibilities may account for this finding. Firstly, it is conceivable that there really is no global-local distinction in thinking patterns, or at least none that is founded in rigorous scientific investigation. Secondly, one may also conclude that the field of environmental science makes it imperative for both experts and novices to think across contexts using cross-case analyses. Finally, my sample may have been too small to reveal clear distinctions, and the design of my study may also be less effective in targeting the distinctions. Further research will be necessary to shed light on whether the concept of global thinking makes sense, or whether the real distinction is between good thinking and thinking that is less adequate.
Implications of the Study
My study has highlighted contrasts between experts and novices when thinking about climate change that, in principle, provides directions for instruction. In particular, a few key implications about efforts to nurture global competence in the classroom are instructive.
140 Firstly, it is well worth paying attention to the three key patterns of expert thinking about complex issues such as climate change: geological (versus ecological) framing of issues; reasoning from a clearly articulated identity and worldview; and epistemic orientation towards knowledge. Focusing curricular designs on supporting learners in framing and understanding global issues using a wider and broader lens so that they are able to understand how shifting baselines impact the way we view issues and events; encouraging learners to clarify their role and stance vis-à-vis the issue of inquiry, so that they reason from an identity or self concept that matters to them; and creating opportunities for learners to understand the constructed nature of knowledge and appreciate how experts build, verify and calibrate knowledge in their domain.
The point of doing so is less about learners becoming experts per se, but rather that they begin to appreciate how expert-like thinking may shift the center of gravity in how they think about issues and problems.
Secondly, the facility of the thinking routines in revealing learners’ current thinking and in supporting their thinking is promising, at least in supporting learners toward a more complex understanding of a global issue like climate change. While my study did not find the novices moving toward more expert thinking in all areas, it does demonstrate how providing targeted prompts for learners engages them in deeper thinking about the issues of inquiry. If the thinking routines were used repeatedly over time, they have the potential for support learners’ substantive engagement with global issues, and build a strong foundation for them to grapple with complex ideas and questions.
Thirdly, the near absence of important cognitive moves, such as systems thinking and expansive framing, in the novices in my study, and the prominence of them in the experts’ thinking, call for educators to nudge student thinking in those directions. To
141 support this work, thinking routines designed for supporting learners to grapple with complexity could be designed and tested.
Limitations & Further Study
My study was designed to be an exploration into important distinctions between the patterns of thinking demonstrated by novices and experts as they engage with a complex global issue like climate change. It set out to probe the thinking of my participants, uncover trends across my subsamples, and sketch an initial trajectory of novice to expert thinking patterns that emerge from the data. These have been more or less accomplished.
While my study has shed some light on the expert-novice trajectory, my findings can only at best provide very tentative suggestions towards a rubric, as the sample used is small, and a larger sample will be needed if we want to examine trends that hold across the population. A future direction will be to conduct a content validity study using larger populations, and to begin to test the extent to which the resulting descriptions of performance across the trajectory hold with a more diverse population.
Another limitation of my study design is the single interview that was used with each participant. Thinking routines are intended for repeated use over time so that learners begin to see and use them as cognitive tools to understand the world. A one-off use is not likely to nurture thinking dispositions that come from routinized application of a series of cognitive moves that target specific kinds of good thinking. A related limitation is that a one-off use of thinking routines will not capture important developments in thinking or changes of mind that will provide information about how learners are thinking. Future research in this area may want to employ a longitudinal approach.
142 A further limitation of my study was that only two thinking routines were employed.
I had originally intended to use a wider repertoire of thinking routines in my study, but decided to stay with two because it was challenging to find participants who would consent to a longer interview. Both in my pilot study and the formal study, a substantial number of potential participants declined my invitation to be part of the study, because the two hours that I had originally designed for the study was considered too long. It was only after I reduced the interview time to an hour in my email invitation that I managed to put together a sample of 6 experts. In retrospect, employing a few more thinking routines would likely have elicited richer and more nuanced thinking from my participants, especially if the thinking routines specifically targeted complex thinking.
The expert-novice differences that my study highlighted also identified another limitation in my study. There were several key cognitive patterns demonstrated by the experts that the two thinking routines used in my study were just not designed to scaffold.
For instance, complex thinking and a more sophisticated epistemic stance stood out as being clearly important expert thinking moves, but the novices were not scaffolded adequately by the thinking routines to reach them. Future studies will need to be more thoughtful about what thinking routines to employ in order to target specific types of expert thinking about complex issues.
As explained in an earlier section, my hypothesis about global versus local thinking warrants further study. My current study is unable to shed light on whether the concept of global thinking is empirically founded, or whether the distinction is really between good thinking and thinking that is less adequate. Future investigation to clarify this will necessitate a study design that specifically targets this distinction: for instance, involving participants
143 working on a phenomenon or in a field that has clearly demarcated local and global contexts, as well as recruiting a larger sample.
Finally, an interesting puzzle generated in the course of my study was the extent to which the experts’ epistemic sophistication was domain-specific as opposed to being domain-general. Another way of slicing this pie is to hypothesize that there is an interaction between their domain-general epistemology with their domain-specific knowledge. Further study into this will yield insights into how we might want to support thinking dispositions in disciplinary, cross disciplinary, and general contexts.
144 Appendix A Study Protocol
Research Question: What similar or contrasting patterns of thinking do experts and high school students, studying climate change occurring either globally or within a country/community, demonstrate when they engage with a scenario on climate change using a series of thinking routines?
Purpose & Part of Study Protocol Procedure I. Introduction 1. Explain the purpose of the session: “I am interested in how experts/students think about climate change, so I would like to listen to how you think about the topic of geoengineering after you have watched a short video that I will screen. I am not interested in any right or wrong answers. Instead, I am interested in how (1) high school students in general think about issues related to climate change compared to adults who have more education in science. I’m not looking for individual differences OR (2) experts in climate science think about issues related to climate change compared to high school students with less education in science.”
2. Inform the participant that he or she will be videotaped: “What you say is very important, so I’m going to videotape the session so that I don’t forget anything. I will let you know when I’m turning on the camera, and when I’m turning it off. If you wish at any point for me to stop the recording, please let me know.”
II. Task 1 – Purpose: To 3. Explain the purpose of the first task to participant: “This is Encoding understand how the first task in the study. I will record this session. During information about this task, you will first watch a 12-minute video of an climate change / environmental scientist talking about geoengineering, which geoengineering is has been proposed by some climate scientists as a solution to encoded by climate change. After you have watched it, I will ask you to participants through a put together a set of notes on what you saw and heard for a neutral recall task, e.g., short oral summary. So, if you wish, you may take notes as connections among you listen and watch. ” Proceed to screen video. concepts and ideas, what stands out in 4. After screening the video, ask participant to recall the content their recall, etc. of the video for a stated purpose: “Now, imagine that your Procedure: colleague / classmate missed the screening of the video, but Participants watch a needs to have some idea of what was said. You are to provide 12-minute video on him or her with a short summary of what you saw and heard geoengineering, a in the video. Please write whatever notes you need to give a proposed but short oral summary to your colleague. Your notes should be controversial solution one page at most. ” to global warming. Then, they are asked 5. “Now, pretend that I’m your friend / colleague who missed to type out a set of the video screening. Please share with me what your key notes for a short oral takeaways from the video are. Your summary should be no summary for a more than 5 minutes. I will give you a few minutes now to
145 colleague or classmate look over your notes and think about the main ideas for your who had missed the summary. I don’t want you to read from your notes, but feel video. free to glance at them.” When the 5 minutes are up, I will ask them if they feel ready enough to begin. Before participants begin their oral summary, I’ll say: “Remember, you only have to give a 5-minute summary. To help you with the timing, I’ll raise my right hand like this (demonstrate gesture) to alert you that you have reached 4 minutes and should be wrapping up soon.”
III. Task 2 – Purpose: To uncover 6. Begin the interview with a general interview question: “Tell Scenario how participants think me what the ideas and information from the video made you about geoengineering think about. Remember that I am not interested in any right and climate change as or wrong answers, but in how you are thinking about the presented in a video ideas and information from the video.” using a series of Thinking Routines 7. When participant seems to be slowing down, or pausing for and Global Thinking longer than 10 consecutive seconds, use the thinking routine, Routines. More Connect-Extend-Challenge, to scaffold his/her thinking: “I’m specifically, I am now going to use a form of inquiry that has proven helpful in interested in these kinds of thinking. It is a thinking routine called Connect- understanding how Extend-Challenge. It comprises three prompts that invite you to participants: talk about the connections that you are making about what • connect the ideas you have watched, the ways the video has extended your and information thinking, and how the ideas are still challenging. I’ll begin with presented in the the first prompt. (1) How would you say the ideas and video to what they information presented in the video connect to what you already know, take already know? (2) What new ideas did you get from the video stock of ongoing that extended or pushes your thinking in new directions? (3) questions, puzzles What would you say is still challenging or confusing for you and challenges, and about geoengineering?” Ask the questions one at a time, reflect on what they moving on to the next only when participant says that he/she are learning; and has nothing more to say. Provide a handout with the thinking • frame the issue of routine printed on it. geoengineering, and gauge its 8. Next, use a general interview question to invite participant to significance both gauge the significance of the topic of geoengineering: “Why locally and globally. do you think this issue matters?”
Procedure: 9. When participant seems to be slowing down, or pausing for Participants watch the longer than 10 consecutive seconds, use the global thinking video on routine, 3Ys, to scaffold his/her thinking: “Another form of geoengineering again. inquiry that has been useful in these kinds of thinking is to This time, they control consider different dimensions of significance. It is called the the pace at which they 3Ys. I’m going to use that now. Tell me why this issue matters watch, and can pause to you, to your community, and to the world. Remember that at any part they want. I am not interested in any right or wrong answers, but in how Then, they respond to you are thinking about the ideas and information from the a general question to video.” Provide a handout with the thinking routine printed elicit their thinking, on it. before one Thinking 10. When participant says that he or she has nothing more to say,
146 Routine and one follow up with: “How do you think someone from another Global Thinking part of the world such as China or Brazil might think about Routine are used to this issue?” engage them with the content of the video. The interview questions will be kept comparable across the participants. IV. Task 3 – Purpose: To track 11. “In the video you just watched, David Keith proposed that Sourcing how participants we should have a treaty among nations that decides who might use sources to should be in charge of geoengineering the world’s climate. Do make sense of a you agree with him?” complex issue, e.g., what they pay 12. “Now, imagine that you have to offer a 3-minute response to attention to; how they the challenge you have selected to an audience comprising construct accounts; experts and non-experts. You now have an opportunity to what kinds of sources; gather more information for crafting an informed and what ideas and/or responsible perspective. What kind of information will you disciplines they think look for?” Give participants 10 minutes to think about and belong together; what write out possible sources. After that, pose the question: connections they are “Please walk me through your line of thinking when you were making; etc. putting together this set of resources.” Move on to more specific information, use the following prompts for each Procedure: source identified: “What is the connection that you are seeing Participants will be between this source and the issue? What part of it is relevant shown a list of 3 to you? How does it speak to the issue? What is the question challenges / puzzles you have in mind when you go to this source? Could you say that David Keith had more about what prompted you to take this path?” raised about geoengineering in the 13. During a period of inactivity, ask participant whether he or video. They are then she has completed the investigation: “Do you feel that you asked to think about have sufficient information for your critique?” If participant how they might gather says “no”, ask him or her to continue. information for an informed response to 14. If participant has completed the investigation, invite him or the selected challenge her to say what main ideas would be included in the 3-minute for an expert and non- critique: “What are the main ideas that you are likely to expert audience. include in your 3-minute critique?”
15. Ask participant: “Is there anything else you would like to say about geoengineering before we end our session?”
V. Conclusion Procedure: Conclude the study by thanking participants.
147 Appendix B Transcript of Edited TED Talk
You've all seen lots of articles on climate change, and here's yet another New York Times article, just like every other darn one you've seen. It says all the same stuff as all the other ones you've seen. It even has the same amount of headline as all the other ones you've seen. What's unusual about this one, maybe, is that it's from 1953. And the reason I'm saying this is that you may have the idea this problem is relatively recent. The fact is - - uh-uh. We've known about this problem for 50 years, depending on how you count it. We have talked about it endlessly over the last decade or so. And we've accomplished close to zip.
This is the growth rate of CO2 in the atmosphereWhat this shows is that the rate of growth of our emissions is accelerating. And that it's accelerating even faster than what we thought was the worst case just a few years back. So that red line there was something that a lot of skeptics said the environmentalists only put in the projections to make the projections look as bad as possible, that emissions would never grow as fast as that red line. But in fact, they're growing faster.
Here's some data from actually just 10 days ago, which shows this year's minimum of the Arctic Sea ice, and it's the lowest by far. And the rate at which the Arctic Sea ice is going away is a lot quicker than models. So despite all sorts of experts like me flying around the planet and burning jet fuel, and politicians signing treaties -- in fact, you could argue the net effect of all this has been negative, because it's just consumed a lot of jet fuel. (Laughter) No, no! In terms of what we really need to do to put the brakes on this very high inertial thing -- our big economy -- we've really hardly started. Really, we're doing this, basically. Really, not very much.
I don't want to depress you too much. The problem is absolutely soluble, and even soluble in a way that's reasonably cheap. Cheap meaning sort of the cost of the military, not the cost of medical care. Cheap meaning a few percent of GDP. No, this is really important to have this sense of scale. So the problem is soluble, and the way we should go about solving it is, say, dealing with electricity production, which causes something like 43-or-so percent and rising of CO2 emissions. And we could do that by perfectly sensible things like conservation, and wind power, nuclear power and coal to CO2 capture, which are all things that are ready for giant scale deployment, and work. All we lack is the action to actually spend the money to put those into place. Instead, we spend our time talking.
But nevertheless, that's not what I'm going to talk to you about tonight. What I'm going to talk to you about tonight is stuff we might do if we did nothing. And it's this stuff in the middle here, which is what you do if you don't stop the emissions quickly enough. And you need to deal -- somehow break the link between human actions that change climate, and the climate change itself. And that's particularly important because, of course, while we can adapt to climate change -- and it's important to be honest here, there will be some benefits to climate change. Oh, yes, I think it's bad. I've spent my whole life working to stop it. But one of the reasons it's politically hard is there are winners and losers -- not all losers.
So this problem is absolutely soluble. This geo-engineering idea, in its simplest form, is basically the following. You could put signed particles, say sulfuric acid particles -- sulfates -- into the upper atmosphere, the stratosphere, where they'd reflect away sunlight and cool the planet. And I know for certain that that will work. Not that there aren't side effects, but I know for certain it will work. And the reason is, it's been done. And it was done not by us, not by me, but by nature.
Here's Mount Pinatubo in the early '90s. That put a whole bunch of sulfur in the stratosphere with a sort of atomic bomb-like cloud. The result of that was pretty dramatic. After that, and some previous volcanoes we have, you see a quite dramatic cooling of the atmosphere. So this lower bar is the upper atmosphere, the stratosphere, and it heats up after these volcanoes. But you'll notice that in the upper bar, which is the lower atmosphere and the surface, it cools down because we shielded the atmosphere a little bit. There's no big mystery about it. There's lots of mystery in the details, and there's some bad side effects, like it partially destroys the ozone layer -- and I'll get to that in a minute. But it clearly cools down. And one other thing: it's fast. It's really important to say. So much of the other things that we ought to do, like slowing emissions, are intrinsically slow, because it takes time to build all the hardware we need to reduce emissions. And not only that, when you cut emissions, you don't cut concentrations, because concentrations, the amount of CO2 in the air, is the sum
148 of emissions over time. So you can't step on the brakes very quickly. But if you do this, it's quick. And there are times you might like to do something quick.
Another thing you might wonder about is, does it work? Can you shade some sunlight and effectively compensate for the added CO2, and produce a climate sort of back to what it was originally? And the answer seems to be yes. So here are the graphs you've seen lots of times before. That's what the world looks like, under one particular climate model's view, with twice the amount of CO2 in the air. The lower graph is with twice the amount of CO2 and 1.8 percent less sunlight, and you're back to the original climate.
This topic is also old. That report that landed on President Johnson's desk when I was two years old -- 1965. That report, in fact, which had all the modern climate science -- the only thing they talked about doing was geo-engineering. So I should say, I guess, that since the time of that original President Johnson report, and the various reports of the U.S. National Academy -- 1977, 1982, 1990 -- people always talked about this idea. Not as something that was foolproof, but as an idea to think about.
But when climate became, politically, a hot topic -- if I may make the pun -- in the last 15 years, this became so un-PC, we couldn't talk about it. It just sunk below the surface. We weren't allowed to speak about it. But in the last year, Paul Crutzen published this essay saying roughly what's all been said before: that maybe, given our very slow rate of progress in solving this problem and the uncertain impacts, we should think about things like this. He said roughly what's been said before. The big deal was he happened to have won the Nobel prize for ozone chemistry. And so people took him seriously when he said we should think about this, even though there will be some ozone impacts. And in fact, he had some ideas to make them go away.
There was all sorts of press coverage, all over the world, going right down to "Dr. Strangelove Saves the Earth," from the Economist. And that got me thinking. I wondered if you could use the same physics that makes that thing spin 'round in the child's radiometer, to levitate particles into the upper atmosphere and make them stay there. One of the problems with sulfates is they fall out quickly. The other problem is they're right in the ozone layer, and I'd prefer them above the ozone layer. And it turns out, I found out that there were all sorts of papers already published that addressed this topic because it happens already in the natural atmosphere. So it seems there are already fine particles that are levitated up to what we call the mesosphere, about 100 kilometers up, that already have this effect. This is a new idea that's crept up that may be, essentially, a cleverer idea than putting sulfates in. The one thing about this is it gives us extraordinary leverage. This improved science and engineering will, whether we like it or not, give us more and more leverage to affect the planet, to control the planet, to give us weather and climate control -- not because we plan it, not because we want it, just because science delivers it to us bit by bit, with better knowledge of the way the system works and better engineering tools to effect it.
Now, suppose that space aliens arrived. Maybe they're going to land at the U.N. headquarters down the road here, or maybe they'll pick a smarter spot -- but suppose they arrive and they give you a box. And the box has two knobs. One knob is the knob for controlling global temperature. Maybe another knob is a knob for controlling CO2 concentrations. You might imagine that we would fight wars over that box. Because we have no way to agree about where to set the knobs. We have no global governance. And different people will have different places they want it set. Now, I don't think that's going to happen. It's not very likely.
But we're building that box. The scientists and engineers of the world are building it piece by piece, in their labs. Even when they're doing it for other reasons. Even when they're thinking they're just working on protecting the environment. They have no interest in crazy ideas like engineering the whole planet. They develop science that makes it easier and easier to do. And so I guess my view on this is not that I want to do it -- I do not -- but that we should move this out of the shadows and talk about it seriously. Because sooner or later, we'll be confronted with decisions about this, and it's better if we think hard about it, even if we want to think hard about reasons why we should never do it.
I'll give you two different ways to think about this problem that are the beginning of my thinking about how to think about it. But what we need is not just a few oddballs like me thinking about this. We need a broader debate. A debate that involves musicians, scientists, philosophers, writers, who get engaged with this question
149 about climate engineering and think seriously about what its implications are. So here's one way to think about it, which is that we just do this instead of cutting emissions because it's cheaper. I guess the thing I haven't said about this is, it is absurdly cheap. It's conceivable that, say, using the sulfates method or this method I've come up with, you could create an ice age at a cost of .001 percent of GDP. It's very cheap. We have a lot of leverage. It's not a good idea, but it's just important. (Laughter) I'll tell you how big the lever is: the lever is that big. And that calculation isn't much in dispute. You might argue about the sanity of it, but the leverage is real. (Laughter) So because of this, we could deal with the problem simply by stopping reducing emissions, and just as the concentrations go up, we can increase the amount of geo-engineering. I don't think anybody takes that seriously. Because under this scenario, we walk further and further away from the current climate.
But here's a case which is harder to reject. Let's say that we don't do geo-engineering, we do what we ought to do, which is get serious about cutting emissions. But we don't really know how quickly we have to cut them. There's a lot of uncertainty about exactly how much climate change is too much. So let's say that we work hard, and we actually don't just tap the brakes, but we step hard on the brakes and really reduce emissions and eventually reduce concentrations. And maybe someday -- like 2075, October 23 -- we finally reach that glorious day where concentrations have peaked and are rolling down the other side. And we have global celebrations, and we've actually started to -- you know, we've seen the worst of it. But maybe on that day we also find that the Greenland ice sheet is really melting unacceptably fast, fast enough to put meters of sea level on the oceans in the next 100 years, and remove some of the biggest cities from the map. That's an absolutely possible scenario. We might decide at that point that even though geo-engineering was uncertain and morally unhappy, that it's a lot better than not geo-engineering. And that's a very different way to look at the problem. It's using this as risk control, not instead of action. It's saying that you do some geo-engineering for a little while to take the worst of the heat off, not that you'd use it as a substitute for action.
But there is a problem with that view. And the problem is the following: knowledge that geo-engineering is possible makes the climate impacts look less fearsome, and that makes a weaker commitment to cutting emissions today. This is what economists call a moral hazard. And that's one of the fundamental reasons that this problem is so hard to talk about, and, in general, I think it's the underlying reason that it's been politically unacceptable to talk about this. But you don't make good policy by hiding things in a drawer.
I'll leave you with three questions, and then one final quote. Should we do serious research on this topic? Should we have a national research program that looks at this? Not just at how you would do it better, but also what all the risks and downsides of it are. Right now, you have a few enthusiasts talking about it, some in a positive side, some in a negative side -- but that's a dangerous state to be in because there's very little depth of knowledge on this topic. A very small amount of money would get us some. Many of us -- maybe now me -- think we should do that. But I have a lot of reservations. My reservations are principally about the moral hazard problem, and I don't really know how we can best avoid the moral hazard. I think there is a serious problem: as you talk about this, people begin to think they don't need to work so hard to cut emissions.
Another thing is, maybe we need a treaty. A treaty that decides who gets to do this. Right now we may think of a big, rich country like the U.S. doing this. But it might well be that, in fact, if China wakes up in 2030 and realizes that the climate impacts are just unacceptable, they may not be very interested in our moral conversations about how to do this, and they may just decide they'd really rather have a geo-engineered world than a non-geo-engineered world. And we'll have no international mechanism to figure out who makes the decision.
So here's one last thought, which was said much, much better 25 years ago in the U.S. National Academy report than I can say today. And I think it really summarizes where we are here. That the CO2 problem, the climate problem that we've heard about, is driving lots of things -- innovations in the energy technologies that will reduce emissions -- but also, I think, inevitably, it will drive us towards thinking about climate and weather control, whether we like it or not. And it's time to begin thinking about it, even if the reason we're thinking about it is to construct arguments for why we shouldn't do it. Thank you very much.
150 Appendix C Coding Scheme (Initial & Revised Versions)
Initial Coding Scheme
1. Epistemology (naïve) Views authority as correct and/or trustworthy; scientific information as factual; issues as binary or dualistic
2. Explanation (limited) Sees cause-and-effect as linear; constructs no more than 2-3 causal relationships; offers either-or explanations
3. Solution-oriented Focuses on solutions as primary motivation; seeing the engineering of the environment as key human contribution; focuses on what is most pragmatic (instead of seeing the complexity of proposed solutions)
4. Lens 4.1. Interdisciplinarity Constructs explanations using multiple disciplines, or proposes solutions that draw from more than one discipline 4.2. Time scales or historical view Places the issue or event within a temporally extended scale, e.g., over a century or millenia 4.3. Systems view Takes a complex perspective on the issue, e.g., employing multiple variables in an explanation or causal chain 4.4. Space/distance Takes a view of the issue through geographical lens. Example: reasons by using different geographical locations or contexts
5. Identity Positions self vis-à-vis the issue. Example: takes personal/moral/professional responsibility for the issue
6. Emotion Refers to the issue in emotional terms
7. Motivation (external) Scrutinizes motivation for climate engineering solutions. Example: talks about how different parties/groups frame solutions
8. Multiple perspectives Takes different perspectives on the issue, or weighs multiple views relating to the issue
9. Epistemology (sophisticated) Takes a more developed or sophisticated epistemic stance. Example: acknowledges the provisional nature of current knowledge
10. Case comparisons References different cases to reason about the issue
11. Weighing perspectives Considers trade-offs in the issues or solutions
151 Revised (Final) Coding Scheme
1. Epistemology 1.1. Naïve Views authority as correct and/or trustworthy; scientific information as factual; issues as binary or dualistic 1.2. Sophisticated Takes a more developed or sophisticated epistemic stance. Example: acknowledges the provisional nature of current knowledge
2. Solution-oriented Oriented towards solutions or pragmatic explanations. Example: focusing on solutions as primary motivation; seeing the engineering of the environment as key human contribution; focuses on what is most pragmatic around issues related to climate change solutions (instead of seeing the complexity of proposed solutions)
3. Time scale/historical lens Places the issue or event within a temporally extended scale, e.g., over a century or millennia
4. Identity Positions self vis-à-vis the issue. Example: takes personal/moral/professional responsibility for the issue
5. Emotion When words or phrases (1) relate to emotions (e.g., "concerned", "horrified", "tantalized"); (2) connote an emotional relationship (e.g., "old friends"); (3) are intensifiers such as "incredibly" and "deeply", to refer to how he or she feels about the issues
6. Multiple perspectives Takes a complex perspective on the issue, e.g., employing multiple variables in an explanation or causal chain OR Takes different perspectives on the issue, or weighs multiple views relating to the issue OR Constructs explanations using multiple disciplines, or proposes solutions that draw from more than one discipline. Example: brings in different disciplines in explaining or thinking about the issue
152 Bibliography
Akerlof, K., Maibach, E. W., Fitzgerald, D., Cedeno, A. Y., & Neuman, A. (2013). Do people “personally experience” global warming, and if so how, and does it matter? Global Environmental Change, 23, 81-91.
American Association of Colleges and Universities (2010). Intercultural knowledge and competence VALUE rubric. Washington, DC: AAC&U.
Amsel, E., Klaczynski, P. A., Johnston, A., Bench, S., Close, J., Sadler, E., & Walker, R. (2008). A dual-process account of the development of scientific reasoning: The nature and development of metacognitive intercession skills. Cognitive Development, 23, 452-471.
Ananiadou, K., & Claro, M. (2009). 21st century skills and competencies for new millennium learners in OECD countries. OECD Education Working Papers, No. 41, OECD Publishing. Retrieved on July 12, 2011, from http://dx.doi.org/10.1787/218525261154.
Appleyard, N., & McLean, L. R. (2011). Expecting the Exceptional: Pre-Service Professional Development in Global Citizenship Education. International Journal Of Progressive Education, 7(2), 6-32.
Asia Society (2011). Graduation Portfolio System. Retrieved on November 23, 2015, from http://asiasociety.org/education/partnership-global-learning/events-services/graduation- performance-system
AT21CS (2012). Transforming education: Assessing and teaching 21st century skills. Retrieved on August 11, 2015, from http://www.atc21s.org/.
Baxter Magolda, M. B. (2002). Epistemological reflection: The evolution of epistemological assumptions from age 18 to 30. In Hofer, B. K., & Pintrich, P. R. (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Lawrence Erlbaum Associates, Publishers: Mahwah, New Jersey.
Bell, R. L., Blair, L. M., Crawford, B. A., & Lederman, N. G. (2003). Just do it? Impact of science apprenticeship programmer on high school students’ understanding of the nature of science and scientific inquiry. Journal of Research in Science Teaching, 40, 487–509.
Ben-David, A., & Zohar, A. (2009). Contribution of meta-strategic knowledge to scientific inquiry learning. International Journal of Science Education, 31(12), 1657-1682.
Bennett, M. J. (2004). Becoming interculturally competent. In Wurzel, J. S. (Ed.), Towards multiculturalism: A reader in multicultural education. Newton, MA: Intercultural Resource Corporation.
Bennett, M. J. (1993). Towards ethnorelativism: A developmental model of intercultural sensitivity. In Paige, R. M. (Ed.), Education for the intercultural experience. Yarmouth, ME: Intercultural Press.
153
Blessing, S., & Anderson, J. R. (1996). How people learn to skip steps. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, 576–598.
Boix Mansilla, V., & Gardner, H. (1998). What are the qualities of understanding? In Wiske, M. S. (Ed.), Teaching for Understanding: Linking Research with Practice. San Francisco, CA: Jossey-Bass.
Boix Mansilla, V., & IDGlobal Team (2013). Global thinking routines: Foundations for our work. Unpublished paper.
Boix Mansilla, V., & Jackson, A. (2011). Educating for Global Competence: Preparing our Youth to Engage in the World. New York, NY: The Asia Society.
Boix Mansilla, V., Sato, K., Chua, F., Hoidn, S., Ivanier, A., and Lamont, M. (2010). Building socio-emotional-cognitive platforms for interdisciplinary collaboration. Report prepared for the Canadian Institute of Advanced Research.
Boyatzis, R. E. (1998). Transforming qualitative information; Thematic analysis and code development. Thousand Oaks, CA US: Sage Publications, Inc.
Bransford, J., Brown, A. L., Cocking, R. R., & National Research Council (U.S.). (1999). How people learn: Brain, mind, experience, and school. Washington, D.C: National Academy Press.
Buehl, M., & Alexander, P. (2005). Motivation and performance differences in students’ domain-specific epistemological belief profiles. American Educational Research Journal, 42, 697-726.
Byram, M. (1997). Teaching and Assessing Intercultural Communicative Competence. Clevedon: Multilingual Matters.
Charmaz, K. (2006). Constructing grounded theory. SAGE Publications: LA.
Charness, N. (1989). Expertise in chess and bridge. In Klahr, D., & Kotovsky, K. (Eds.), Complex information processing: The impact of Herbert A. Simon. Hillsdale, NJ: Erlbaum.
Chase, W.G., & Simon, H. A. (1973). The mind’s eye in chess. In Chase, W. G. (Ed.), Visual Information Processing. New York: Academic, 1973.
Chen, G. M., & Starosta, W. J. (1996). The development and validation of the Intercultural Sensitivity Scale. Paper presented at the Annual Meeting of the National Communication Association (86th, Seattle, WA, November 8-12, 2000).
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121-152.
154 Clayton, S., Manning, C. M., & Hodge, C. (2014). Beyond storms & droughts: The psychological impacts of climate change. Washington, DC: American Psychological Association and ecoAmerica.
Collins, A., Brown, J. S., & Newman, S. E. (1989). Cognitive apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.), Knowing, learning, and instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Erlbaum.
Collins, H. (2011). Three dimensions of expertise. Phenomenology and the Cognitive Sciences, 12(2), 253-273.
Collins, A., & Stevens, A. (1993). A cognitive theory of inquiry teaching. In Reigeluth (Ed.), Instructional design theories and models. Hillsdale, NJ: Erlbaum.
Covitt, B. A., Harris, C. B., & Anderson, C. W. (2013). Evaluating scientific arguments with slow thinking. Science Scope, November, 44-52.
Curran, K. (2003). Global competencies that facilitate working effectively across cultures. Quoted in Hunter, W. (2004). Got Global Competency? International Educator, 13(10), 6- 12.
Deardorff, D. K. (2006). The identification and assessment of intercultural competence as a student outcome of internationalization at institutions of higher education in the United States. Journal of Studies in International Education, 10(3), 241–266.
Deardorff, D.K. (Ed). (2009). The SAGE Handbook of Intercultural Competence. Thousand Oaks, CA: Sage.
DeGroot A. (1978). Thought and Choice and Chess. The Hague, Netherlands: Mouton.
De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition, 106, 1248–1299.
Dott, R. H., Jr. (1998). What is unique about geological reasoning? GSA Today, October. Retrieved on May 21, 2014, from http://courses.washington.edu/ess408/Dott.pdf.
Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science Education, 84(3), 287-312.
Duckworth, R., Walker-Levy, L., & Levy, J. (2005). Present and future teachers of the world’s children: How internationally minded are they? Journal of Research in International Education, 4(3) 279-311.
Duschl, R. A. (1990). Restructuring science education: The importance of theories and their development. New York: Teachers College Press.
Duschl, R., & Osborne, J. (2002). Supporting and promoting argumentation discourse in science education. Studies in Science Education, 38, 39-72.
155
Erduran, S., & Jimenez-Aleixandre, M. P. (Eds.). (2008). Argumentation in science education. New York: Springer.
Ericsson KA, Charness N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725–747.
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211-245.
Evans, J. St. B. T. (2003). In two minds: Dual-process accounts of reasoning. Trends in Cognitive Science, 7, 454–459.
Evans, J. St. B. T. (2007). On the resolution of conflict in dual process theories of reasoning. Thinking and Reasoning, 14, 321–339.
Evans, J. B. St. B. T. (2008). Dual-process accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255–278.
Falk, D. R., Moss, S., & Shapiro, M. (2010). Educating globally competent citizens: A instrument kit for teaching seven revolutions. Center for Strategic & International Studies: Washington, D.C.
Gillian, K. J. (1995). A measure of global-mindedness at the University of Northern Colorado: An assessment of students, faculty and administrators. Dissertation Abstracts International, 56(08), 3016.
Gott, R., Duggan, S., & Johnson, P. (1999). What do practicing applied scientists do and what are the implications for science education? Journal of Research in Science and Technology Education, 17, 97–107.
Graesser, A. C. (1999). How do adults comprehend the mechanisms of everyday devices: Texts, illustrations and breakdown scenarios. Paper presented at the Annual Meeting of the American Educational Research Association, Montreal, Canada.
Green, D. W. (1997). Explaining and envisaging an ecological phenomenon. British Journal of Psychology, 88, 199–217.
Green, M.F. & Olson, C. (2008). Internationalizing the Campus: A User’s Guide. Washington, D.C: ACE (American Council on Education).
Hett, E. J. (1993). Development of an instrument to measure global-mindedness. Unpublished doctoral dissertation, University of San Diego, California.
Hmelo, C. E., Holton, D., & Kolodner, J. L. (2000). Designing to learn about complex systems. Journal of the Learning Sciences, 9, 247–298.
156 Hmelo-Silver, C. E., Marathe, S., & Liu, L. (2007). Fish swim, rocks sit, and kungs breathe: Expert-novice understanding of complex systems. The Journal of the Learning Sciences, 16(3), 307-331.
Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cognitive Science, 28, 127-138.
Hofer, B. K. (2002). Personal epistemology as a psychological and educational construct: An introduction. In Hofer, B. K., & Pintrich, P. R. (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Lawrence Erlbaum Associates, Publishers: Mahwah, New Jersey.
Hofer, B. K., & Pintrich, P. R. (1997). The development of epistemological theories: Beliefs about knowledge and knowing and their relation to learning. Review of Educational Research, 67(1), 88-140.
Hovland, K. (2009). Global learning: what is it? Who is responsible for it? Peer Review: Study Abroad and Global Learning: Exploring Connections, 11 (4). Washington, D.C.: American Association of Colleges and Universities. Retrieved from http://www.aacu.org/peerreview/pr-fa09/pr-fa09_index.cfm
Hulme, M. (2009). Why We Disagree about Climate Change: Understanding Controversy, Inaction and Opportunity. Cambridge: Cambridge University Press.
Hunter, W. (2004). “Knowledge, Skills, Attitudes, and Experiences Necessary to Become Globally Competent.” Retrieved on July 1, 2010, from http://www.globallycompetent.com/research/WDH-dissertation-2004.pdf.
Hunter, B., White, G.P., & Godbey, G.C. (2006). What does it mean to be globally competent? Journal of Studies in International Education, 10, 267-284.
Immordino-Yang, M. H., & Damasio, A. (2007). We feel, therefor we learn: The relevance of affective and social neuroscience to education. Mind, Brain, and Education, 1(1), 3-10.
International Baccalaureate (2013). What is an IB education? Retrieved on March 10, 2014, from www.ibo.org.
Jääskeläinen, L., & Repo, T. (2011). Schools reaching out to a global world: What competencies do global citizens need? Finnish National Board of Education. Retrieved on January 20, 2014, from http://www.oph.fi/english/publications/2011/schools_reaching_out_to_a_global_world
Jacobs, H. H. (2010). Upgrading content: provocation, invigoration, and replacement. In Jacobs, H. H. (Ed.), Curriculum 21: Essential Education for a Changing World, ASCD: Alexandria, Virginia USA.
157 Jerald, C. D. (2009). Defining a 21st century education. Alexandria, VA: Center for Public Education.
Jones, D. K., & Read, S. J. (2005). Expert-novice differences in the understanding and explanation of complex political conflicts. Discourse Processes, 39(1), 45-80.
Kahneman, D. (2011). Thinking, Fast and Slow. Farrar, Straus and Giroux: New York, NY.
Karweit, N. (1984). Time-on task reconsidered: Synthesis of research on time and learning. Educational Leadership, May, 32-35.
Kehl, K., & Morris, J. (2005). Differences in global-mindedness between short-term and semester-long study abroad participants at selected private universities. Interdisciplinary Journal of Study Abroad, 15, 67-79. Kelly, G. J., Druker, S., & Chen, C. (1998). Students’ reasoning about electricity: Combining performance assessments with argumentation analysis. International Journal of Science Education, 20, 849-871.
Kirkwood-Tucker, T., Morris, J., & Lieberman, N. (2011). What kind of teachers will teach our children? The world-mindedness of undergraduate elementary and secondary social studies teacher candidates at five Florida public universities. International Journal of Development Education and Global Learning, 3(3), 5-28.
Klaczynski, P. A. (2004). A dual-process model of adolescent development: Implications for decision making, reasoning, and identity. In R. V. Kail (Ed.), Advances in child development and behavior. San Diego, CA: Academic Press.
Koester, J., & Olebe, M. (1989). The behavioral assessment scale for intercultural communication effectiveness. International Journal of Intercultural Relations, 12, 233-246.
Kuhn, D. (1993). Science as argument: Implications for teaching and learning scientific thinking. Science Education, 77(3), 319-337.
Kuhn, D. (2005). Education for Thinking. Cambridge, MA: Harvard University Press.
Kuhn, D., Iordanou, K., Pease, M., & Wirkala, C. (2008). Beyond control of variables: What needs to develop to achieve skilled scientific thinking? Cognitive Development, 23, 435-451.
Kuhn, D., & Park, S. (2005). Epistemological understanding and the development of intellectual values. International Journal of Educational Research, 43, 111-124.
Maxwell, J. (1996). Qualitative research design: An integrative approach. Thousand Oaks, CA: Sage.
Miles, M. B. & Huberman, A. M. (1994). Qualitative data analysis (Second edition). Thousand Oaks, CA: Sage.
158 Lambert, R. D. (Ed.). (1994). Educational exchange and global competence. New York: Council on International Educational Exchange.
Larkin, J. H. D., McDermott, D., Simon, D. P., & Simon, H. A. (1980). Models of competence in solving physics problems. Cognitive Science, 4, 317–345.
Leach, J., Driver, R., Millar, R., & Scott, P. (1997). A study of progression in learning about ‘the nature of science’: Issues of conceptualization and methodology. International Journal of Science Education, 19, 147-166.
Lehrer, R., Schauble, L., & Petrosino, A. J. (2001). Reconsidering the role of experiment in science education. In Crowley, K., Schunn, C., & Okada, T. (Eds.), Designing for science: Implications from everyday, classroom, and professional settings. Mahwah, NJ: Lawrence Erlbaum Associates.
Lemke, C. (2002). enGauge 21st century skills: Digital literacies for a digital age. Office of Educational Research and Improvement, Washington, DC.
Levin, D. (2003). Teaching young children in violent times: Building a peaceable classroom. Cambridge, MA: Educators for social responsibility.
Linn, M. C., & Songer, N. B. (1993). How do students make sense of science? Merrill-Palmer Quarterly, 39(1), 47-73.
Liu, Y. D., & Grotzer, T. A. (2009). Looking forward: Teaching the nature of science of today and tomorrow. In Saleh, I. M., & Khine, M. S. (eds.), Fostering Scientific Habits of Mind: Pedagogical Knowledge and Best Practices in Science Education. Sense Publishers.
Longview Foundation (2008). Teacher preparation for the global age: The imperative for change. Retrieved on August 12, 2013, from www.longviewfdn.org.
Mason, L., & Boscolo, P. (2004). Role of epistemological understanding and interest in interpreting a controversy and in topic- specific belief change. Contemporary Educational Psychology, 29(2), 103-128.
Mason, L., & Scirica, F. (2006). Prediction of students’ argumentation skills about controversial topics by epistemological understanding. Learning and Instruction, 16, 492-509.
Maxwell, J. A. (2005). Qualitative research design: An interactive approach (2nd Ed.). Thousand Oaks, CA: Sage Publications.
Metz, K. (2004). Children’s understanding of scientific inquiry: Their conceptualization of uncertainty in investigations of their own design. Cognition and Instruction, 22, 219-290.
Merryfield, M. M. (2002). The difference a global educator can make. Educational Leadership, 60(2), 18.
159 Merryfield, M. M. (1994). Teacher education in global and international education. In: Sutton, M. and Hutton, D. (Eds.) Concepts and trends in global education. Washington, D.C: Office of Educational Research and Improvement.
Ministry of Education Singapore (2010). Nurturing our young for the future: Competencies for the 21st century. Retrieved on July 25, 2012, from www.moe.gov.sg.
Morais, D. B., & Ogden, A. C. (2011). Initial development and validation of the Global Citizenship Scale. Journal of Studies in International Education, 15(5), 445-466.
Narayanan, N. H., & Hegarty, M. (1998). On designing comprehensible interactive hypermedia manuals. International Journal of Human-Computer Studies, 48, 267–301.
NRC (National Research Council). (2001). Knowing what students know. The science and design of educational assessment. In Pellegrino, J. W., Chudowsky, N., & Glaser, R. (Eds.), Committee on the foundation of Assessment (Board on Testing and Assessment Center for Education Division of Behavioral and Social Sciences and Education) (p. 1-14). Washington, DC: National Academy Press.
Norris, S. (1994). The meaning of critical thinking test performance: The effects of abilities and dispositions on scores. In Fasko, D. Jr. (Ed.), Critical Thinking: Current Research, theory, and practice. Dordecht, The Netherlands: Kluwer.
Novick, L. R. (1988). Analogical transfer, problem similarity, and expertise. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14(3), 510–520.
Osland, J. S., & Bird, A. (2000). Beyond sophisticated stereotyping: Cultural sensemaking in context [and executive commentaries]. The Academy of Management Executive, 14(1), 65-79.
Partnership for 21st Century Skills (2008). 21st Century Skills, Education and Competitiveness: A Resource and Policy Guide. Retrieved January 20, 2013, from www.p21.org/.
Partnership for 21st Century Skills (2014). Reimagining Citizenship for the 21st Century: A Call to Action for Policymakers and Educators. Retrieved February, 2014, from http://www.p21.org/our-work/citizenship.
Pellegrino, J. W., & Hilton, M. L. (2012). Education for life and work: Developing transferrable knowledge and skills in the 21st century. National Academies Press: Washington, DC 20001.
Penner, D. E. (2001). Complexity, emergence, and synthetic models in science education. In Crowley, K., Schunn, C. D., & Okada, T. (Eds.), Designing for science. Hillsdale, NJ: Erlbaum.
Perkins, D. (2014). Future Wise: Educating Our Children for a Changing World. San Francisco, CA: Jossey-Bass.
160 Perkins, D. N., & Grotzer, T. A. (2000). Models and moves: Focusing on dimensions of causal complexity to achieve deeper scientific understanding. Paper presented at the Annual Meeting of the American Educational Research Association, New Orleans, LA.
Perkins, D. N., Tishman, S., Ritchhart, R., Donis, K., & Andrade, A. (2000). Intelligence in the wild: A dispositional view of educational traits. Educational Psychology Review, 12(3), 269- 293.
Peterson, B. (2004). Cultural Intelligence: A Guide to Working with People from Other Cultures. Yarmouth, ME: Intercultural Press.
Pimm, S. L. (1982). Food Webs. London: Chapman and Hall.
Posner, M. I. (1988). Introduction: what is it to be an expert? In Chi, M. T. H., Glaser, R., & Farr, M. J. (Eds.), The Nature of Expertise. Hillsdale, N.J.: Lawrence Erlbaum Associates, Inc., Publishers.
Pratt, D., & Noss, R. (2010). Designing for mathematical abstraction. International Journal of Computers for Mathematical Learning, 15(2), 81–97.
Reardon, B. A. (1999). Peace education: A review and projection. Malmö, Sweden: Malmö University.
Reimers, F. (2009). Educating for Global Competency. In Cohen, J. E., & Malin, M. B. (Eds.) International perspectives on the goals of universal basic and secondary education. New York: Routledge.
Reimers, F. (2013). Educating citizens: Public education’s vision for a tolerant and empowered society. Worlds of Education, July. Retrieved on January 10, 2014, from http://worldsofeducation.org/en/magazines/articles/190#.Uu3Af_aUKEk.
Reimers, F. (2013). Assessing global education: An opportunity for the OECD. Retrieved on February 3, 2014, from www.oecd.org/pisa/pisaproducts/Global-Competency.pdf.
Resnick, M., & Wilensky, U. (1998). Diving into complexity: Developing probabilistic decentralized thinking through role-playing activities. Journal of the Learning Sciences, 7, 153– 172.
Ricklefs, R. E. (1993). The Economy of Nature (3rd ed.). New York: Freeman.
Ritchhart, R., Church, M., & Morrison, K. (2011). Making Learning Visible: How to Promote Engagement, Understanding, and Independence for All Learners. Jossey-Bass: San Francisco, CA.
Ritchhart, R., & Perkins, D. N. (2000). Life in the mindful classroom: Nurturing the disposition of mindfulness. Journal of Social Issues, 56(1), 27-47.
Ritchhart, R., & Perkins, D. (2008). Making thinking visible. Educational Leadership, 65(5), 57- 61.
161 Roan, L., Strong, B., Gehlbach, H., & Metcalf, K.A. (2009). Social perspective taking. ARI Technical Report 1259. U.S. Army Research Institute for the Behavioral and Social Sciences.
Roberts, A. (2007). Global dimensions of schooling: Implications for internationalizing teacher education. Teacher Education Quarterly, 34(1), 9-26.
Rubio, D. M., Berg-Weber, M., Tebb, S. S., Lee, E. S., & Rauch, S. (2003). Objectifying content validity: Conducting a content validity study in social work research. Social Work Research, 27(2), 94-104.
Rudiak-Gould, P. (2013). “We have seen it with our own eyes”: Why we disagree about climate change visibility. Weather, Climate, and Society, 5, 120-132.
Saavedra, A. R., & Opfer, V. D. (2012). Learning 21st-century skills requires 21st-century teaching. Kappan, 94(2), 8-13.
Salomon, G. S. (Ed.). (1994). Distributed Cognitions. New York: Cambridge University Press.
Sandoval, W. A. (2005). Understanding students’ practical epistemologies and their influence on learning through inquiry. Science Education, 89(4), 634-656.
Sandoval, W. A. & Reiser, B. J. (2004). Explanation-driven inquiry: Integrating conceptual and epistemic scaffolds for scientific inquiry. Science Education, 88, 345-372.
Schenk, K., Vitalari, N., & Davis, K. (1998). Differences between novice and expert systems analysts: What do we know and what do we do? Journal of Management Information Systems, 15(1), 9–50.
Schommer, M. (1990). Effects of beliefs about the nature of knowledge on comprehension. Journal of Educational Psychology, 82, 498-504.
Schraw, G., Bendixen, L. D., & Dunkle, M. E. (2002). Development and validation of the epistemic belief inventory (EBI). In Hofer, B. K., & Pintrich, P. R. (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Lawrence Erlbaum Associates, Publishers: Mahwah, New Jersey.
Schwartz, D. L., Bransford, J. D., & Sears, D. (2005). Innovation and efficiency in learning and transfer. In Mestre, J. (Ed.), Transfer of Learning from a Modern Multidisciplinary Perspective, pp. 1-51. Mahwah, N.J.: Erlbaum.
Shaver, P., Schwartz, J., Kirson, D., & O'Connor, C. (2001). Emotional Knowledge: Further Exploration of a Prototype Approach. In Parrott, G. (Ed.), Emotions in Social Psychology: Essential Readings (pp. 26-56). Philadelphia, PA: Psychology Press.
Shultz, L. (2007). Educating for global citizenship: Conflicting agendas and understandings. The Alberta Journal of Educational Research, 53(3), 248-258.
162
Skitka, L. J. (2003). Of different minds: An accessible identity model of justice reasoning. Personality and Social Psychology Review, 7(4), 286-297.
Smith, C. L., Maclin, D., Houghton, C., & Hennessey, M. G. (2000). Sixth-grade students’ epistemologies of science: The impact of school science experiences on epistemological development. Cognition and Instruction, 18(3), 349-422.
Smith, D. (2008). The impact of international experience on the global mindedness of extension agents employed by the North Carolina Cooperative Extension Service. Raleigh, NC: North Carolina State University.
Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate. Behavioral and Brain Sciences, 23, 645–726.
Stewart, V. (2010). A classroom as wide as the world. In Jacobs, H. H. (Ed.), Curriculum 21: Essential Education for a Changing World. ASCD: Alexandria, Virginia USA.
Suárez-Orozco, M. & Sattin, C. (2007). Introduction: Learning in the global era. In Suárez- Orozco, M. (Ed.), Learning in the global era: International perspectives on globalization and education. Berkely, CA: University of California Press.
Swim, J., Clayton, S., Doherty, T., Gifford, R., Howard, G., Reser, J., Stern, P., & Weber, E.U. (2009). Psychology and global climate change: Addressing a multifaceted phenomenon and set of challenges. Report of APA Task Force on the Interface Between Psychology and Global Climate Change. Available at: http://www.apa.org/science/about/publications/climate-change-booklet.pdf
Tabak, I., Smith, B. K., Sandoval, W. A., & Reiser, B. J. (1996). Combining general and domain-specific strategic support for biological inquiry. Paper presented at the ITS’96 Conference, Montreal, Canada.
Tamir, P., Nussinovitz, R., & Friedler, Y. (1982). The design and use of practical tests assessment inventory. Journal of Biological Education, 16, 45–50.
Tishman, S., & Andrade, A. (1995). Thinking dispositions: A review of current theories, practices, and issues. Washington: Association of Critical and Creative Thinking International Network.
Tobin, K., Tippins, D. J., & Hook, K. S. (1995). Students’ beliefs about epistemology, science, and classroom learning: A question of fit. In Glynn, S. M., & Duit, R. (Eds.), Learning science in the schools: Research reforming practice (pp. 85-110). Mahwah, NJ: Erlbaum.
Tye, K. A. (Ed). (1991). ASCD Yearbook 1991: Global education: From thought to action. Alexandria, VA: Association for Supervision and Curriculum Development.
U.S. Department of Education (2012). Succeeding globally through international education and engagement. U.S. Department of Education, International Strategy. www.ed.gov.
163
Vygotsky, L. S. (1978). In Cole, M., John-Steiner, V., Scribner, S., & Souberman, E. (Eds.), Mind in society: The development of higher mental processes. Cambridge, MA: Harvard University Press. (Original work published 1930, 1933, 1935).
West, C. (2012). Toward globally competent pedagogy. NAFSA: Association of International Educators. Retrieved on December 13, 2013, from www.nafsa.org/epubs.
White, B. Y., & Frederiksen, J. R. (1998). Inquiry, modeling, and metacognition: Making science accessible to all students. Cognition & Instruction, 16(1), 3-118.
White, P. A. (2008). Beliefs about interactions between factors in the natural environment: A causal network study. Applied Cognitive Psychology, 22, 559-572.
Wilensky, U., & Resnick, M. (1999). Thinking in levels: A dynamic systems approach to making sense of the world. Journal of Science Education and Technology, 8, 3–19.
Willig, C. (2008). Introducing qualitative research in psychology. New York: McGraw-Hill Education.
Windschitl, M., & Andre, T. (1998). Using computer simulations to enhance conceptual change: The roles of constructivist instruction and student epistemological beliefs. Journal of Research in Science Teaching, 35(2), 145-160.
Wiske, M. S. (1998). Teaching for Understanding: Linking Research with Practice. San Francisco, CA: Jossey-Bass.
Wood, P., & Kardash, C. A. (2002). Critical elements in the design and analysis of studies of epistemology. In Hofer, B. K., & Pintrich, P. R. (Eds.), Personal epistemology: The psychology of beliefs about knowledge and knowing. Lawrence Erlbaum Associates, Publishers: Mahwah, New Jersey.
Wood-Robinson, C. (1995). Children’s biological ideas: Knowledge about ecology, inheritance, and evolution. In Glynn, S. M., & Duit, R. (Eds.), Learning science in the schools: Research informing practice (pp. 111–131). Hillsdale, NJ: Erlbaum.
Zhang, L., Lin, Z. H., Wang, J. Q., & Wang, M. (2000). China: Revitalizing education in the 21st century. Human Rights Education in Asian Schools, 103.
Zhao, Y. (2012). World class learners: Educating creative and entrepreneurial students. Thousand Oaks, CA: Corwin.
164