3 neler öğrendik? bölüm özeti Research Methods

Editor

Prof.Dr. Ali ERSOY

Authors

Dr. Zekiye YAHŞİ CHAPTER 1

Asst.Prof.Dr. Muhammet ÖZDEN CHAPTER 2

Assoc.Prof.Dr. Nihan DEMİRKASIMOĞLU CHAPTER 3

Prof.Dr. Hasan GÜRGÜR CHAPTER 4

Prof.Dr. Hanife AKAR CHAPTER 5

Assoc.Prof.Dr. Şakir ÇINKIR CHAPTER 6, 8

CHAPTER 7 Prof.Dr. Ahmet SABAN T.C. ANADOLU UNIVERSITY PUBLICATION NO: 3833 OPEN EDUCATION FACULTY PUBLICATION NO: 2642

Copyright © 2019 by Anadolu University All rights reserved. This publication is designed and produced based on “Distance Teaching” techniques. No part of this book may be reproduced or stored in a retrieval system, or transmitted in any form or by any means of mechanical, electronic, photocopy, magnetic tape, or otherwise, without the written permission of Anadolu University.

Graphic and Cover Design Prof.Dr. Halit Turgay Ünalan

Instructional Designer Musa Tömen

Assessment Editor Yaşar Özden İnceler

Graphic Designers Ayşegül Dibek Hilal Özcan Özlem Çayırlı Gülşah Karabulut

Typesetting and Composition Cansu Ersoy Kağan Küçük Yasin Özkır

RESEARCH METHODS

E-ISBN 978-975-06-3035-4

All rights reserved to Anadolu University. Eskişehir, Republic of Turkey, January 2019 3293-0-0-0-2102-V01 Contents

Philosophical CHAPTER 1 Foundations of CHAPTER 2 Planning Research Research

Introduction...... 3 Introduction...... 33 Epistemological Foundations...... 6 Planning Quantitative Research Studies...... 33 Ontological Foundations...... 8 Identifying the Research Problem...... 33 Methods of Science...... 9 Selecting the Research Design...... 35 Inductivism...... 11 Survey ...... 35 Deductivism...... 12 Experimental Designs...... 36 Paradigm Theory...... 13 Identification of the Population and Kuhn’s Legacy...... 16 Sample...... 38 The Primacy of Language...... 16 Data Collection...... 38 Implications for Epistemology and Data Analysis and Interpretation...... 39 Ontology...... 17 Planning Qualitative Research Studies...... 40 Social Sciences...... 17 Identifying the Research Problem...... 40 Quantitative Positivist Paradigm...... 18 Theoretical/Conceptual Framework.... 41 Qualitative Interpretive Paradigm...... 19 Developing Research Question(s) ...... 41 Selecting Research Design ...... 41 Selecting the Participants...... 44 Data Collection...... 44 Data Analysis, Reporting Findings, and Interpretation...... 45 Ethical Principles and Researcher’s Role...... 46 Limiting the Results and Making Analytical Generalizations...... 46 Planning Mixed Method Research Studies.. 47 The Title...... 47 Identifying the Research Problem...... 47 Identifying Research Aim and Questions...... 48 Selecting Mixed Method Research Design...... 48 Sample...... 51 Data Collection...... 51 Data Analysis, Interpretation, and Reporting the Results...... 51 Diagrams Used in Mixed Method Research...... 52

iii Writing a Qualitative CHAPTER 3 CHAPTER 4 Literature Review Research Methods

Introduction...... 63 Introduction ...... 95 Research Topic...... 63 Qualitative Research Methods and Research Questions...... 64 International Relations...... 95 Defining Variables...... 65 Basic Features of Qualitative Methods.... 95 Literature Review...... 65 Possible Contributions of Qualitative The Multiple Purposes of a Research Methods ...... 96 Literature Review...... 66 Qualitative Research Designs...... 98 Steps for Conducting a Literature Case Study ...... 98 Review...... 67 Ethnography ...... 99 Types of Literature Review ...... 68 Narrative Inquiry ...... 100 Deciding the Keywords Within Phenomenology ...... 102 the Subject Area...... 68 Grounded Theory ...... 103 Literature Reviewer ...... 69 The Process of Conducting a Qualitative Sources of Information For Literature Search..... 70 Research ...... 104 Searching the Literature...... 70 Identifying the Research Focus Selecting the Sources...... 70 or Problem ...... 104 Ask an Expert ...... 71 Selecting Participants ...... 105 Encyclopedias ...... 72 Data Collection Techniques ...... 106 Statistics ...... 72 Data Analysis ...... 109 Books ...... 72 Reporting ...... 109 The Internet ...... 73 Rigor in Qualitative Research Method ...... 110 Internet Search Engines ...... 74 Online Databases...... 74 Catalogs ...... 76 Bibliographical Databases...... 78 University Repositories ...... 78 Periodicals ...... 78 Grey Literature ...... 79 Blogs and Wikies...... 81 Journal Databases ...... 81 Ancestry Method...... 81 Organization of the Literature Review ...... 82 Common Errors and Problems in Writing a Literature Review ...... 84

iv Quantitative Mixed Methods CHAPTER 5 CHAPTER 6 Methods Research

Introduction...... 121 Introduction...... 149 Why Do We Use Quantitative Methods?.... 122 Overview of Mixed Research...... 149 Descriptive Research ...... 122 Definition...... 149 Associational Research...... 123 What Is the Reason for Using Mixed Correlational Research...... 123 Methods? ...... 151 Causal-Comparative Research...... 124 Advantages (Strenghts and Weaknesses).... 151 Experimental Research ...... 125 Limitations...... 152 Survey Research ...... 126 Determining the Goal of the Mixed Cross-Sectional Surveys...... 127 Methods Research...... 153 Cross-National Surveys ...... 127 Formulating the Mixed Research National Time-Series...... 128 Objective(s)...... 153 Election Surveys...... 129 Determining the Rationale of the Study Panel Surveys ...... 130 and the Rationale(s) for Mixing Preparing Good Survey Instruments ...... 130 Quantitative and Qualitative Tips for Writing Survey Items ...... 131 Approaches...... 155 Data Collection Procedures ...... 133 Determining the Purpose of the Study Data Analysis Procedures ...... 133 and the Purpose(s) for Mixing Quantitative Basıcs of Doing Research ...... 134 and Qualitative Approaches...... 156 Writing Research or Hypothesis ...... 134 Mixed Method Research Planning Stage .... 157 Good Research Questions ...... 135 Selecting the Mixed Sampling Design... 157 Variables ...... 136 Selecting the Mixed Research Design.... 158 Census, Populatıon, and Sampling ...... 137 Research Implementation Stage...... 163 Random Sampling...... 137 Collecting Quantitative and/or Stratified Random Sampling...... 138 Qualitative Data...... 163 Systematic Sampling ...... 138 Analyzing the Quantitative and/or Cluster Sampling ...... 138 Qualitative Data Using Quantitative Two-stage Random Sampling ...... 139 and/or Qualitative Analysis Techniques... 164 Purposive Sampling ...... 139 Validating/Legitimating the Mixed Convenience Sampling ...... 139 Research Findings...... 165 Validity and Reliability ...... 139 Interpreting the Mixed Research Findings...... 166 Writing the Mixed Research Report.... 167 Reviewing Your Mixed Methods Research Study ...... 168

v Writing Research CHAPTER 7 Research Ethics CHAPTER 8 Report

Introduction...... 179 Introduction...... 211 Understanding Research Ethics...... 179 Basic Academic Writing Skills...... 211 Research Ethics as Means of Judgment Choosing an Original Research Topic... 211 and/or Evaluation...... 179 Writing Abstract...... 212 Research Ethics as Protecting the Writing Introduction...... 213 Rights of Research Participants...... 181 Writing a Paragraph...... 214 History Of Research Ethics...... 181 How to Paraphrase...... 216 Overview of Research Ethics Scandals..... 182 How to Synthetize Articles...... 219 Establishment of Research Ethics Literature Review...... 220 Committees (RECs) ...... 186 Using Transition Words...... 221 Ethical Perspectives...... 187 Writing a Method Section...... 222 Deontological Ethics...... 187 Research Design/Model...... 223 Consequentialist/Utilitarian Ethics...... 188 Research Sample/Participants...... 224 Principle-Based Ethics (Principlism) .... 188 Research Instruments/Measures...... 225 Situation-Based Ethics (Situationism) .... 189 Reliability and Validity of Measure...... 226 Virtue Ethics...... 189 Data Analysis...... 227 Relational Ethics (Ethics of Care) ...... 189 How to Present the Results?...... 229 Other Ethical Perspectives...... 190 Presenting Data in Tables and Figures..... 230 Ethical Principles...... 190 Discussion, Conclusion and Autonomy and Informed Consent...... 192 Recommendation...... 232 Balancing Research Risks and Benefits.... 195 Privacy, Anonymity and Confidentiality... 196 Ethics In Qualitative Research...... 197 Reflexivity in Qualitative Research...... 198 Reflections on Some Ethical Dilemmas in Qualitative Research...... 198

vi Preface

Dear Students, International relations cover a vast field of by establishing a cause-effect relationship bet- study. For instance, several subjects such as ween the events, the information under this economy, terror, war, peace, commerce, law, heading might be useful. Chapter 6’s title is climate change, energy resources, environ- mixed method research. The chapter includes mental problems, immigration, trafficking, information on how to look into a research fight against drugs, history, culture and so subject both in depth and from a broader fo- forth directly or indirectly fall under the field cus by using qualitative and quantitative rese- of study of the international relations. Studies arch methods together. Chapter 7 is about the focusing on this field attempt to explain the research ethics. This title includes informati- relationships between countries, societies and on and examples about what to consider to organizations and their outcomes. By definiti- conduct an ethical research. Chapter 8’s title on, a single method cannot be used to unders- is writing research reports. This final chapter tand countries, societies and organizations includes functional information and tips on that have different cultural characteristics. how to write a scientific research report. Instead, research methods are needed that are This book wouldn’t have existed if it had not different but complement each other, criticize been for the devoted and meticulous work of or confirm. In this respect, this book entitled the chapter writers. Therefore, I would like Research Methods in your hand contains an to extend my sincere thanks to the esteemed overview of some major research methods chapter writers who contributed to this book, that can be used in international relations and Hanife Akar (Middle East Technical Univer- other fields of social sciences. sity), Nihan Demirkasımoğlu (Hacettepe Uni- The book consists of eight chapters. Chapter versity), Zekiye Yahşi (Gazi University), Ah- 1 deals with the philosophical principles of met Saban (Necmettin Erbakan University), the research methods. It is a chapter serving Hasan Gürgür (Anadolu University), Şakir as an introduction that contains prerequisite Çınkır (Ankara University) and Muhammet information required to understand the rese- Özden (Kütahya Dumlupınar University). arch methods that are based on different pa- Their contribution to the creation of this book radigms. Chapter 2 focuses on what must be is particularly very important because they known while planning a scientific research. wrote the chapters in a very short time. Also, This chapter includes information on things I would like to extend my thanks to the type- to pay attention before starting a manageable setting unit of the Anadolu University, Open research. Chapter 3 is about the inclusion of Education Faculty that typeset the book, and literature references to describe the theoreti- to my esteemed colleague at the Anadolu Uni- cal and conceptual framework of the research. versity, Faculty of Education who proofread The chapter deals with the current status of the book. the research subject that is of interest or at- I hope that this book will offer you functional tempted to be understood in the literature and useful information. I wish that this book and how to connect this information with the would serve as a resource in research conduc- research subject. Chapter 4’s title is qualitative ted on international relations and wish you all research methods. Here, there is some gene- good reading. ral information about case study, focus group and other qualitative research methods that are frequently used in the international rela- Editor tions. This chapter includes methods that will give an in-depth perspective regarding a sub- Prof.Dr. Ali ERSOY ject. Chapter 5 is about quantitative research methods. If you want to understand the rese- arch subject from a more general perspective

vii Philosophical Foundations of Chapter 1 Research After completing this chapter, you will be able to:

Define basic concepts in philosophy of Discriminate a priori proposition from a science and differentiate between the basic posteriori proposition/knowledge; hypothesis assumptions of empiricist epistemology vs. from theory; formal sciences from empirical rationalist epistemology; between the realist sciences; and natural empirical sciences from 1 ontology vs. idealist ontology. 2 social empirical sciences. Define Kuhnian terminologies such as paradigm, paradigm shift, pre-normal science, Compare inductivist methodology to deductivist normal science and revolutionary science as methodology, and discuss the problems situated within the historical context of scientific 3 associated with each. 4 practice. Discuss how the nature of social worlds are

Learning Outcomes different from the nature of natural worlds in relation with the positivist quantitative and post- positivist qualitative approaches regarding the 5 methods of study.

Chapter Outline Key Terms Introduction • A posteriori proposition/Knowledge Method of Science • A priori proposition/Knowledge Paradigm Theory • Epistemology • Hypothesis Social Sciences • Ontology • Positivism • Problem of induction • Problem of sensation • Problem of theory-ladennes • Scientific anti-realism (idealism) • Scientific realism • Theory • Hypothesis • Paradigm • Paradigm shift • Normal science • Revolutionary science • Positivist quantitative method • Interpretive qualitative method

2 Research Methods 1 INTRODUCTION “what is the reliable method to build knowledge and to test them?” Further, science in general, or Philosophy is separate field of study from the a scientific discipline in particular, cannot view science. So, one might ask the question “why should itself from a broader, historical context. Thus, it science be interested in philosophy?” However, at becomes the philosophy’s job to provide answers the time when modern science was emerging in the to such questions and situate the science within 16th and 17th centuries, there was not even a clear a broader context of a developmental trajectory. boundary between philosophy and science. In fact, That is, philosophy connects the knowledge the first scientists called themselves the “natural built in different disciplines to each other. In philosophers.” It was only in later periods of summary, philosophical accounts of science help history that science was separated from philosophy. us better understand what science is about and our However, the connection between the two relations with the various disciplines within the continues. Today, there is a sub-field in philosophy sciences. Philosophy also provides guidance to the that is particularly interested in science, known as scientific practice, providing the scientists with a the philosophy of science (Godfrey-Smith, 2003). foundational structure in which each scientist can In this chapter, we will begin our review see their practices in connection with the work of with a general introduction to a field known as others either in one’s own field or in other related the philosophy of science. Epistemological and fields; and also in connection with the overall ontological issues will be the focal points of this scientific progress. Thus, philosophy in a way introduction. After that, we will move to the becomes a hub providing connections between methodological accounts of the natural sciences, different fields of science and even between namely inductivist and hypothetical-deductivist science and other knowledge building traditions. accounts of the scientific method. Then, we will Two areas in philosophy are particularly relevant review the problems commonly associated with for the philosophical investigation of the sciences these approaches. After that, we will move to – epistemology and metaphysics. Each of these Thomas Kuhn’s (1922-1996) historical analysis areas provide answers to critical questions about of scientific practice and discuss how it changes the nature of science and the knowledge gained our perception of both the scientific practice and through scientific practice, both in general and in the methods of science. Thereafter, we will move particular scientific disciplines (Godfrey-Smith, to the social sciences. Firstly, we will review the 2003; Johnson and Christensen, 2004). positivistic roots of the philosophy of social science The term “epistemology” comes from the ancient and its preference for the quantitative methods for Greek roots, from “episteme” meaning “knowledge” studying the social worlds. From there, we will and from “logos” meaning “the study of.” Thus, move on to a review of the emergence of what epistemology can be defined asthe philosophy of came to be known as the post-positivist philosophy knowledge, or the study of knowledge. Epistemology, as an objection to the positivist tradition; and thus, is also referred to as the theory of knowledge. the interpretive paradigm and its preference for Epistemology focuses on answering such questions the qualitative methods as consequences of these as “what is the nature of coming to know?”, “what developments. are the foundations or sources of knowledge?” Science is a field of practice that aims at and “what are the limits of human knowledge?” developing reliable knowledge of the world, Metaphysics is another field in philosophy, which both natural and social worlds. Natural sciences is interested in having discussions about the nature produce knowledge of the objects of the natural of being, existence and the world. It is interested world and the social sciences produce knowledge in the ‘first causes and the principles’ of existence of the objects of the social worlds. As science and being in the world. It asks questions such as tries to develop a knowledge base of its particular “what is the nature of reality?”, “how does the world fields, it cannot critically examine itself. Thus, it exist?”, “What is the origin, or source of creation?”, cannot answer the most important questions about “What is the connection between the knowing itself, such as “what is science?”, “what demarcates mind and the external reality?”. These kinds of science from other domains of knowledge?”, or questions are known as metaphysical questions that

3 Philosophical Foundations of Research 1 cannot be answered through scientific investigation the model for studying society (in 1830s). The or through the scientific methods of observation discipline of sociology was thereby established; or experiments. Ontology is a particular focus area thus, Comte is often referred to as the founding within metaphysics, which focuses particularly father of sociology (Stanford Encyclopedia of on the study of “being” or “existence.” The term Philosophy, 2018). ontology also comes from the ancient Greek roots, In its beginning, the theoretical approaches and from “ontos” which means “being”, “to be” or “to the methods of social sciences were modeled after exist” and from “logos” which means “the study of.” the natural sciences with a desire to enjoy the same Thus, ontology can be defined as the philosophy or success that natural sciences had accomplished the study of being or existence. Thus, ontology focuses in the past. It was only in the 20th century, on questions related to abstract concepts such as concerns about the compatibility of the theoretical “what is it mean to exist?”, “What is the unchanging approaches and the methods of natural sciences substance of the existing things?”, “Is there an with the nature of the social life began to develop. It objective, unchanging reality beyond human was argued that the empirical worlds of the natural perception and conception?” and so on. These two sciences and the social sciences were necessarily areas of philosophy (epistemology and ontology) different; that is, the natural sciences are interested are not clearly separate fields, they overlap most of in the physical world (or in the natural world, or the time; that is, these are deeply interrelated areas simply in the nature) while the social sciences are of philosophical considerations (Godfrey-Smith, interested in the social life in its varieties (or in the 2003; Johnson and Christensen, 2004). social worlds) and the nature of natural world and Epistemological and ontological concerns are the social world is profoundly different from each important for understanding the nature of any other. Thus, the argument followed, the methods scientific practice and the outcome of such practices. developed in natural sciences are incompatible The ultimate purpose of science is to develop useful with the nature of the social worlds. These knowledge for our practical purposes in the world, developments were further coupled with a crisis to make our lives easier. This is true both for the within the philosophy of science, which criticized natural sciences as well as for the social sciences. the traditional epistemology for being individually In both natural science and the social science, each oriented rather than being socially and culturally theory and research approach has a collection of oriented. The argument centered around then underlying assumptions about the nature of their novel conceptualizations that the nature of human respective worlds (the world of nature vs. the worlds knowledge is social and cultural, if not for any of social life) and about the ways of coming to know other reason it is because of the involvement of such worlds (the methods). However, researchers the language in the process; and thus, coming to themselves are not necessarily explicit about these know the world is a necessarily social and cultural assumptions, as they tend to take their assumptions process. for granted as part of their practices. Thus, most of the time these assumptions are brought to the important surface by the philosophers of science. As is commonly known, natural sciences were The argument of ‘first causes’ in philosophy the first to develop in modern history; the origin of refers to an absolute cause for everything that exists; thus, existence can be tied to a the natural sciences dates back to the 16th century; creator like the God. Thus, the ‘first causes’ the process began after the helio-centric theory of argument is something like the following: the universe was proposed by Copernicus. On the “Things and the world exist in the first place other hand, the scientific study of social life began because the God created them.” at least two centuries later, in the early 19th century, with Auguste Comte’s proposal of positivism as

4 Research Methods 1 Epistemology is a concept about how we come to know Ontology is a concept that means the study about these existing things (that ontology is interested of being/existence and the nature of reality. in) and their reality. So, epistemology is interested in Ontology asks questions such as “What is investigating the sources of our knowledge of reality. the essential substance of (or the underlying And, epistemology is also interested in investigating foundation of) existence?”, “What is the na- whether we are in a state to know reality as it exits ture of reality?” and “Is it possible for us to independent of us (reality as is); and whether what we perceive reality as is?” are able to know corresponds to a reality that exists independent of us (independent of us knowing it).

example

Apple is a thing that exists. Ontology is interested in understanding the nature of an apple’s substance, i.e. of what it is made of – if its substance is material or immaterial as well as in understanding the nature of its reality. If one takes apple’s existence as having a physical, or material substance, the reality of apple is seen to be an independent reality, that means apple is taken to exist independently of our awareness of its existence. Thus, in this view apple exists objectively (independent of human perception). This is the view we assume in living our everyday lives. On the other hand, if one takes that apple exists as an immaterial thing, then its reality is taken as mind-dependent, or as an ideal reality; that means apple exists as an idea in our minds. Thus, in this view apple’s existence depends on our minds, that is, it is not an objective existence. While ontology is interested in the questions related to the apple’s existence, epistemology is interested in how we gain our knowledge of an apple. For example, how is it that we know that there is a thing called apple, and it is edible, and it has a taste, the taste can be sweet or sour, it has a particular material structure, it is juicy, it is smelly, and our knowledge that apple has health benefits, and what those benefits are and so on.

important

Helio-centric universe means the sun is at the center of the universe and all planets circle internet around the sun. This view opposes the earlier More on Ptolemy’s model of the universe: version proposed by Ptolemy an ancient Egyptian https://www.youtube.com/ philosopher, in which the earth is at the center watch?v=EpSy0Lkm3zM and all planets circle around the earth; known as http://www.polaris.iastate.edu/EveningStar/ the geo-centric view. Unit2/unit2_sub1.htm

Positivism is a philosophical theory arguing that positive knowledge is possible as long as science is based on observations and experiments, of the phenomena and the relations between different phenomena. In this context, the word “positive” means “certain”, thus ‘positive knowledge’ means ‘certain knowledge.’ Thus, according to positivism, information derived from sensory experience, interpreted through reason and logic should be the basis of all knowledge. Thus, this theory rejects inclusion of metaphysical considerations because metaphysics does not take the sensory experience as a reliable source of the external reality and doubts whether we are able to perceive reality as is. Positivism approaches reality simply as we perceive it, without further considerations regarding ontological issues that we discussed above.

5 Philosophical Foundations of Research 1 when we deduce conclusions from premises or propositions that are true, we will reach absolutely internet certain generalizations. Thus, rationalists believe reason is the only reliable source of knowledge over For full online lecture series on philosophy of the sense experience. science: by Prof. Dr. Phil Hoyningen-Huene at the Rationalism also believes in innate ideas (ideas Leibniz Universitat Hannover, Germany that come by birth or are inborn). Thus, according https://www.youtube.com/channel/ to rationalism at least some of our knowledge is UC4yBDbab-abnWC6yrysAL3w a priori, that is, independent of experience. These by Dr. Hakob Barseghyan at the University of innate ideas are, in a way, the sources of certain Toronto, Canada truths. By innate ideas they refer to the type of https://www.youtube.com/channel/ propositions that are true by definition, such UC7Tb7zZldhJfxRCxJY52uMg as “2+2=4” (two plus two equals four), “triangle has three sides”, or “all bachelors are single.” Notice that these propositions are true by the Epistemological Foundations meaning of the words (the meaning of the word ‘four’ is ‘two and two more’, the meaning of the Epistemology is concerned with the sources of word ‘triangle’ is ‘three-sided’ and the meaning of our knowledge and how we come to know about the word “bachelor” is ‘unmarried man’). Thus, the world and reality. There have been two major nobody in the right mind will doubt that they epistemological positions about the sources of might not be true. These statements are ‘a priori human knowledge. These positions are known as analytical’ propositions; a priori in the sense that rationalism and empiricism, and they defend one does not have to go to the world and make opposing views. Both of these positions have a role observations to find out whether they are true or in methods of scientific practice. false, analytical in the sense that the meaning is in the word, or true by definition. Rather, whether Rationalism they are true or not can only be known through just thinking, or reasoning about the connections Rationalism is most often associated with Rene between the definitions (meanings) of the Descartes (1596-1690) in modern philosophy. propositions. As a mathematician, who formulated However, the ancient philosopher Plato is also Cartesian coordinates, Descartes thinks that the known as a radical rationalist. Rationalism defends natural world expresses itself mathematically, so the idea that knowledge that is obtained through developing true knowledge of it requires drawing the senses lack certainty, because senses often conclusions through deductive reasoning from deceive us; for example, a stick put in water seems carefully organized premises. The disciplines to us as if it is bend, even though in reality it is like math and logic use work through drawing not bend. Further, the knowledge gained from conclusions from a priori analytical propositions. sense experience is based on inductive inference, In these disciplines if the premises are true, and if which is a generalization that is based on our past one is careful in drawing conclusions from these experiences. That is, our knowledge is induced premises, the conclusion will also be true. Thus, form particular experiences we have had in the Descartes values reason over sense experience as the past, and because the future does not have to source of knowledge. resemble the past these generalizations are not trust worthy.are Thus, our knowledge based on Rationalists also have metaphysical claims about sense experiences can only give us probabilistic existence a higher version of reality. Thus, they knowledge, which could or could not be true in hypothesize about things like an external reality future experiences. In other words, knowledge beyond human perception, which can be achieved based on sense experience lacks certainty. However, only through reasoning because the world of such the knowledge we gain through rational thinking a reality is not available to human experience. For and reason gives us certain knowledge. Thus, example, Plato defended the idea that the sense experience of our everyday lives was just a reflection

6 Research Methods 1 of the actual reality of forms (an ideal world), experience or is dependent on experience. Thus, which was innate to us but accessible only through empiricism does not have any claim about existence careful reasoning. And, our sense experiences in of any underlying reality beyond experience. In this world is only a venue towards the real world short, all our knowledge is induced from our sense of forms, however only if we can reason effectively experiences and sense experience is the only source (Stanford Encyclopedia of Philosophy, 2018). of our knowledge. For Locke, there are two kinds In summary, in rationalism reasoning is the only of experiences. The first one is the experience of way to reach an underlying, certain reality. And, the sensations (lived through sense organs), which we are given innate ideas by birth, and through is the external experience. The other type is the reason we can reach certain knowledge based on experience of the reflections, which is the internal these ideas. The reality we experienced through experience. The internal experience (reflection) our senses was doubted. However, rationalists do happens as we reflect on the data (or the ideas) we not doubt the fact that sense experiences play an gained from our sense experiences. For example, important role in gaining knowledge, but they take we can come up with an idea of a “unicorn” even reason as the reliable source for arriving at certain though we have not experienced one (which does knowledge. They also object to the claim that all not exist), through combining our idea of a horse knowledge depends on sense experience. with our idea of a rhinoceros (an animal with a single horn). Empiricism acknowledges the role of reasoning as well, as source of knowledge, however for A priori proposition/knowledge is a concept reasoning to take place, one has to have ideas first. that refers to propositions or knowledge For example, only after one develops an idea of gained through rational means like thinking numbers, then they are able to reason about the and reasoning. In other words, they don’t relations of numbers. In other words, experience come from sense experience, or they are comes first, reasoning comes in a secondary order. prior to the experience. In order for a person to reason 2 apple and 2 more apples will add up to 4 apples, they first need to have experiences with the things in the world and develop the idea of things as one or more than one, A posteriori proposition/knowledge is which is possible only through experience. Here a concept that refers to propositions or they are objecting to the claim of rationalists that knowledge that comes from sense experience, certain ideas, such as concepts of number are innate. or they are available after the experience. important

Empiricism The word “empirical” refers to knowledge or Empiricism is most often associated with John understanding that is based on experience in Locke (1632-1704); other well-known names the world. As in “empirical sciences” which include George Berkeley (1685-1753) and David means science based on experience. Hume (1711-1776). Empiricism argues that the only source of all human knowledge is the sense Empiricism ultimately leads to many forms experience (observations and experiments). Thus, of skepticism. One of which is skepticism about John Locke strongly opposes the idea of “innate the external world, according to which, since we ideas.” According to Locke, at birth, our minds experience the world only through our senses, are “tabula rasa” (“blank slates”). There are no then we cannot claim to have the knowledge of inborn ideas in our minds. We first develop ideas the external reality as it is. Because, the only thing through sense experience then our minds are filled we can have experience of is what our senses can with knowledge. Thus, for the empiricists, all provide us (can give us access to). In other words, knowledge is a posteriori, that is, it comes from what we experience is our ‘sense data’ not the world

7 Philosophical Foundations of Research 1 itself (or the external reality). Thus, empiricism each object or a living being. On the other hand, does not talk about a metaphysical reality that idealism believes in an immaterial, supernatural is independent of human experience. In fact, world; it is the world of minds, ideas, souls, and positivism that was mentioned earlier derives from God etc. Thus, according to materialists the human empiricism. In the 20th ccenentury, positivism mind has material basis, the brain; on the other evolved into Logical Positivism, which is also hand, for idealists, the mind is an immaterial thing. known as Logical Empiricism. Logical positivism Dualism is an intermediate position, defended rejects any interest in metaphysical considerations by Descartes, according to which both material because it wants to stress observable empirical data and immaterial worlds exist. The material world as the true foundation for all scientific knowledge. consists of the material things (like mountains, Thus, empiricists have taken the world simply as it earth, animals etc.) and the immaterial world (the is perceived and experienced by us, by the human world of ideas) consists of the immaterial things beings. (like souls, spirits, God etc.). Only human beings occupy both words simultaneously; our physical Ontological Foundations bodies exist in the material world while our minds exist in the ideal (immaterial) world. Thus, humans “The more plebian illusion of naïve are given a special ontological status in dualism. realism, according to which things ‘are’ as they are perceived by us through our senses… dominates the daily life of men and of animals; Nature of Reality it is also the point of departure in all of the When it comes to the nature of reality, realist sciences, especially of the natural sciences.” position takes the natural world as the only reality, Albert Einstein and refrains from claims about the existence of a superior, ideal immaterial world of ideas. There are two versions of realism. According to direct Einstein once pointed out that most of us live realism, human reality of the external world is our lives believing that what we perceive is exactly perceived directly from the natural/physical world. the same with what really exists in the external In other words, the reality of the external, physical world. However, for the philosophers concerned world exists exactly as is perceived by us, hence with the nature of reality, this is not such a straight direct. Thus, scientific realism,scientific version forward issue, it is much more complex. Their of direct realism, defends the position that our investigations focus on whether or not the external theories of the world describe the world as it really world and its reality can be objectively knowable exists outside of us, that is, independently of the by the human beings (by the human mind). After mind perceiving it. Direct realism is also known as all, human perception is confined to the human naïve realism, because it takes the reality we see and sensory abilities. There are two opposing positions perceive as the exact copy of what exists outside of about the reality – realism and idealism. us. It is seen as naïve because it does not consider the possibility that humans do not perceive the Substance of Existence external world directly; the possibility that what we perceive directly is just the sense data obtained from When it comes to the substance of existence, the external world through our sense experiences realism defends the position that the substance of in it. Therefore, according to the direct realists, all the existing things is matter, that is, all things the knowledge that our scientific theories provide are made of matter. Hence, this variation of realism corresponds to the external reality. is known as materialism. On the other hand, idealism defends the position that the substance Indirect realism is offered as a reaction to the of all existing things is the mind; thus, believes in shortcomings of direct realism (naïve realism), an immaterial existence. Therefore, according to which defends the position that our perception materialism all things in the world are made of and knowledge of the world can only represent matter and the differences between the things have the world, rather than corresponding to it. That is, to do with how matter is differently structured in what we perceive represents the world similarly to a

8 Research Methods 1 picture representing an object. A picture of an apple is not the apple itself, it is just a representation of it. In the same vein, what we take as the reality is Scientific realism is the idea that our theories a representation of the world, because we cannot describe reality as it is, independently of the directly perceive the world (we can directly perceive human mind and perception. In other words, only our sense data). So, sense data is a reflection there is a world that exists independently of of the external world, which is projected onto our the perceiving, conceiving human mind and minds. Thus, our sense data stands between our our scientific theories correspond to this minds and the world. Therefore, indirect realism reality. says, we never perceive the world directly, so we can never be certain that what we are perceiving and taking as the reality is a true copy (representation) of the external world. What we perceive could be Scientific anti-realism is the opposing view, not a truthful representation of the reality. Indirect which defends the view that we are not in a realism ultimately leads to idealism. position to know or claim that our theories Idealism argues that there is no material, or no correspond to an external reality. We cannot physical external world that is the basis of our reality. know if there is a reality existing independent Thus, our reality does not correspond to a reality of our perceptions. Notice that they are not existing independent of our minds. Further, what saying our theories fail to describe a mind- we perceive are just ideas in our minds. Thus, there independent reality. Rather, they are saying is no ground to claim that our reality comes from an we cannot know whether what our theories objective reality of an external physical world. Only describe corresponds to a mind-independent reality or not. one world exists, that is the immaterial world of the Thus, anti-realism defends the idea that minds and spiritual things. Thus, there is no mind we are engaged in science and we develop independent reality, rather all reality exists in the scientific theories only because we are mind. Everything exists only if there is a perceiving interested in what can science offer to us, mind; in other words, nothing exists outside of for its practical, instrumental benefits for the mind. Thus, our perception and knowledge of our life conditions. Science simply gives us external reality is simply images in our minds. useful tools to deal with the natural world Today, the separation of mind and body for our own purposes, thus we cannot have (dualism) is no longer a scientifically accepted view claims about its ability to represent an and the human mind is viewed as having a material external, mind-independent reality. basis – the brain. However, the active role of human mind in construction of reality and knowledge is well accepted within the disciplines like psychology. For example, a cognitive psychologist Jean Piaget is well known for his analysis of how human 1 consciousness is actively engaged in construction Why do you think we need philosophy of one’s own understanding of reality and the of science? What benefits can be drawn knowledge of the world. Further, Piaget shows how from philosophical investigation of such activity is evident starting from the early days sciences? How does the philosophy of after birth; shows how babies are actively interacting science differ from science itself? with the objects around them using their senses, thereby constructing their understandings of the Find a partner to discuss these issues. world. In Piaget’s account, mind and body works together in knowledge construction. And, both body and the mind are active in the process, rather METHODS OF SCIENCE than waiting passively for an external reality to Modern empirical sciences are distinguished project itself onto the mind. In Piaget’s account, from other forms of knowledge construction and the mind is neither able to reason without the body earlier forms of scientific tradition due to their directly interacting with the world.

9 Philosophical Foundations of Research 1 empirical nature. Thus, one needs to understand generalizations are then tested against further the difference between empirical sciences and empirical data (data gathered through observations formal sciences. Natural sciences (i.e., astronomy, or experiments). In short, today we have awareness physics, chemistry, biology) and social sciences (i.e., that the data we collected are confined to the human sociology, psychology, anthropology) are empirical sensory capabilities, so we are not in a position to sciences. However, mathematics and logic are not. pursue absolute certainty for our knowledge. We The main difference has to do with how knowledge are interested in developing scientific theories, not is constructed and the types of propositions used because they can provide us with an ultimate truth, in the process. In empirical sciences, the purpose or absolute, unchanging knowledge of the world is construction of empirically grounded theories; (natural or social), which was the position held in hence, one needs to check the propositions and any the beginning of the modern science. Rather, we other knowledge claim against the empirical world pursue scientific knowledge simply because it is (the world of experiences or the experience in the useful, because the knowledge we gain allow us to world). For this reason, the types of propositions control and predict phenomena and develop useful used in empirical sciences are called a posteriori applications for our practical purposes. Thus, today synthetic propositions, a posteriori in the sense that we came to realize that the search for an objective, the propositions require experience and synthetic ultimate, unchanging, or absolute knowledge in the sense that the meaning is not in the word as would a too naïve pursuit. in analytic propositions. However, formal sciences In short, we pursue science because science are based on purely logic; their conclusions are is beneficial. Scientific method is the means by deduced from premises that are in the form of a which science achieves theories and develops a priori analytic propositions. Thus, in order to find knowledge base. Thus, scientific method allows us out whether an analytic proposition is true or false an opportunity to construct empirically grounded one does not need to check the empirical world, it knowledge, at least in the natural sciences (we requires a careful reasoning from the premises the will further discuss the issue of methods for social meaning of which is embedded in the words used. sciences under heading “social sciences”). Scientific That is, they are purely conceptual, not empirical. method, thus, is based on empirical observations As mentioned before, these statements are true or and experiments, and measurement based on such false in terms of their definitions. For example, to experiences is an important part of the process. find out whether “2+2=5” is true or false, we do The idea of systematic method of science was not need to check the world. Reasoning will tell developed in the 17th century, by Francis Bacon us if the conclusion we arrived (5) is true or false. (1561-1626). It is systematic in the sense that it Thus, formal sciences are not empirical; they are is based on certain rules that could be followed by analytical in nature and only reasoning can be a any practitioner who is interested in replicating a reliable method for these areas of knowledge. And, study. Bacon argued that science should be based on therefore obtaining certain knowledge is possible carefully conducted observations and measurements in formal sciences, but the situation is different for of a phenomenon. The method proposed by Bacon the empirical sciences. is based on inductive reasoning, thus known In short, our scientific theories are law-like as inductivism. Inductivism prescribes a purely generalizations that provide explanation for some empiricist orientation to science, because it believes phenomena, either about the natural/physical in construction of theories based only on sense phenomena (the topic of the natural sciences) or experience, thus also known as methodological about the social phenomena (the topic of the social empiricism. Thus, inductivism carries the problems sciences). These generalizations are drawn from associated with empiricism, such as the problem of empirical experiences (sense experiences), which sensation, the problem of induction and the problem can be in the form of observations or experiments; of theory-ladennes. Inductivism is contrasted with hence they are called empirical sciences. These deductivism, which is based on deductive reasoning.

10 Research Methods 1 Inductivism In inductivism, the starting point is observations of singular facts, events, or happenings, then expressing them as singular observation statements. An important condition is that the observations must be theory- free in order to prevent contamination of pure facts with a previously accepted theory. This is important, because theory is built in bottom-up fashion, from singular cases of observations. These singular facts are then generalized to cover all cases. This process is calledinductive generalization. The aim is to induce the theory from observations of singular occurrences. The logical implications of the theory are then tested against further empirical data gained through further observations and experiments. If these implications pass the test, then the theory is verified/confirmed. However, for a theory to be qualified for verification, the researchers must meet certain conditions: 1) large numbers of observations must be made, 2) observations must be made under different conditions, and 3) not a single observation statement must contradict the generalization (the theory thereby constructed). Thus, inductivism involves generalization from observed singular cases to anall -inclusive-theory. Thus, inductive generalization involves an inference to be drawn from past observations to future observations, from what is already experienced to that which is not yet experienced. Thus, validity of any inductive conclusion assumes that nature acts in a regular way, however there is no guarantee that the things will happen in the same way in future, this issue is known as the problem of induction. It is called ‘problem’ because it is always possible that one or more future cases will have a disconfirming case. Thus, inductivist theories provide probabilistic knowledge at best, or provide knowledge that is based on what is observed so far. (see the Box 1.1 for a famous example on the problem of induction: “all swans are white”).

Box 1.1 Famous example on the problem of induction: “All swans are white” The case of “all swans are white”: The case of “all swans are white” is a famous allegory in philosophical circles, which clarifies the problem of inductive generalization. The story goes like this: One can observe every single swan they encounter, and make sure each swan they observed was white. They can keep the sample size as large as possible, observing as many swans as possible. They can change observation conditions, observing swans in all possible habitats. Not encountering a single black swan, they can inductively generalize that “all swans are white.” Notice that this generalization covers not only swans in present day, but past and future swans as well. However, there might be a single black swan this researcher has not come in contact, or there might be black swans in other parts of the world. In fact, it is now known that there are black swans in Australia. Clearly, inductive generalizations are probabilistic; they do not provide certainty. From here only one claim can be drawn with certainty: “It is likely that the next swan will be white,” which does not provide certainty.

Inductivism poses further a problem known as the problem of theory-ladennes, which states that inductivism assumes that theory-free observations are possible, however neither theory-free observations nor statements are possible. Because, without a framework for what to observe, that provides guidance for what features to pay attention to, or what to focus on, there are too many things to observe. And, an observer will be lost not knowing where to start, what to observe. Thus, any observation naturally requires a person to have a motivation, to have a framework, thus, to have a base understanding for conducting an observation. Therefore, any observation statement presumes a theory behind it. In other words, theory-free observations are impossible simply because there are no pure facts to observe; what we call facts are interpreted by us as relevant to our purposes at hand. Thus, any observation statement is necessarily theory-laden, regardless of how purely-factual it appears to be. For example, even the simple factual everyday statement such as “the sun circles around the sky” implies an earth-centered view of the universe.

11 Philosophical Foundations of Research 1 However, the scientific method based on hypothetical deductions was developed based on Problem of sensation: It states that sense the work of Hempel and Popper, which is called experience is not a reliable source of hypothetical-deductions (HD in short). The HD knowledge because senses can deceive us. method developed as a response to the problems of inductivism. In this model, theories did not have to be induced from observations, they could be produced imaginatively as long as they are testable Problem of induction: No matter how many against empirical data. Theories were formulated confirming cases are observed, no matter as universal laws, covering all possible cases. how varied observation conditions were, From these laws, scientists would try to deduce no matter how long each observation took, hypothesis that are testable against the empirical inductive generalizations are probabilistic; data. Thus, it was a top-down model. Thus, the thus, they provide fallible (not absolute) problem of inductive generalization is eliminated knowledge of the world. by focusing on the testing of a theory rather than on the construction of it. The problem of theory- ladenness is also solved, because observations required for testing comes after the theory if already Problem of Theory-ladennes: It states that formulated, so they are guided by the theory. the research is not free from the theoretical According to some, the testing should try to expectations, because any observation confirm the hypothesis. For others, confirmation statement is shaped by certain prior was problematic because it carried the problem expectations that are necessarily theoretical, of induction because the past confirmations will if not formal scientific theories then it is be taken as a proof for future cases that are not theories that guide our everyday lives and yet observed; and also, how many cases would language use. be enough for decisive confirmation was not clear. Further, being critical is seen as crucial for the science and looking for confirmation is seen Further, inductivism presents a simplistic as not being critical of the theory. It was argued version of the world, which is confined only to that finding confirming cases would be extremely sense experiences. Natural phenomenon is reduced easy if one looked for them. Thus, testing should to only observable phenomena. There is no space try to falsify all possible hypotheses rather than for hypothesizing about the unseen, unobservable trying to confirm them, and eventually try to mechanisms underlying the observable falsify the theory itself. This view was offered by phenomena. Thus, over time it was replaced with Karl Popper in the 20th century, and is known as the Hypothetical-deductive (H-D) method, which falsificationism(Popper, 1965). is based on deductive reasoning.

Deductivism important Descartes was the first philosopher in modern era who proposed a method of science based on Confirmationist approach to testing requires deductive reasoning. He defended a rationalist scientists to try to confirm a hypothesis while epistemology and believed that certain ideas falsificationist approach requires scientists to are innate and they provide the foundation for try to falsify their hypothesis. Falsification development of further knowledge through is offered as an alternative to confirmation reasoning. According to Descartes, the nature because trying to confirm takes an infinite expressed itself in the language of mathematics, number of cases and was not critical, while thus studying nature based on carefully measured falsification was a critical procedure and just observations and deducing conclusions from them a single case was sufficient for falsification. gives us the certain knowledge.

12 Research Methods 1 In HD, theory formation or hypotheses are we have to understand that these are normative not based solely on observable facts, they can be accounts, that is, as far as describing they also creatively constructed. Thus, hypothesizing about prescribe the methods for scientific practice. In unobservable events and entities are allowed other words, they are “methodological” accounts, (such as particles, molecules, gravity) as long as rather than being close descriptions of the actual their effects can be identified and tested in the methods. observed phenomena. Thus, the world/nature was Nowadays, there is a distinction drawn not taken as a simple entity that is confined only between the terms methodology and the method. to observable events. It is accepted that the world ‘Methodology’ is used to describe the philosophical is more complex than it looks and hypothesizing descriptions or accounts of methods, while ‘method’ about such complexity was seen as necessary. is used to refer to the methods or the practice Nevertheless, even falsificationist approach does that are actually employed. Thus, methodological not overcome the problem of induction despite accounts do not necessarily reflect the actual all its efforts, because when a hypothesis passes methods, because methods can be implicit, thus a test, due to its critical attitude it passes it only taken for granted. That is, there is a discrepancy temporarily. The criteria of being critical requires between what scientists say they do versus what they it to be re-tested in the future, by other individual actually. In Thomas Kuhn’s (1922-1996) historical scientists, under the light of new understandings, analysis of the actual scientific practices shows a with the help of new equipment etc. Thus, testing clear picture of such a discrepancy. Kuhn helps us becomes a continuous process. Even if a theory understand that most of the practice and methods passes the testing procedure today, it may fail it in used in actual day-to-day work of the scientists future. Thus, acceptance at any point becomes a are taken for granted by the practitioners, because probabilistic acceptance (based on the results of the the practice is guided by certain assumptions of testing so far – based on the past experiences). It a theoretical worldview which is also taken for turns out there is no way to eliminate the problem granted (Kuhn, 1962). inductive generalization. Inductivism enters the process through the back door. Thus, the resulting knowledge is still probabilistic, which forces the scientists to accept that science is necessarily 3 fallible, thus looking for or claiming an absolutely Questions to think about: certain knowledge is impossible. Let’s assume, the world did not have regularities, would inductivist inference be possible? 2 If human beings are confined to their senses in making sense of the world, With a partner compare and contrast then does it make sense to doubt the the basic principles of inductivism senses as a way of building knowledge? versus deductivism. Can we separate our senses from our ability to reason? In sum, despite the difficulties inherent in these accounts of scientific method, science is working effectively. It turns out none of these proposed PARADIGM THEORY formulations of methods (or the methodologies) is able to explain the actual workings of the scientific The traditional accounts of scientific method practice. The inductivist account was already (the methodological accounts – inductivism and abandoned by the 20th century. However, the HD) give us an orderly process of science that account of HD is still with us today. However, even follows strictly defined procedures. However, HD, as the most recent methodological account, Kuhn shows us that science is a dynamic process cannot give a faithful picture. Thus, at this point, that includes countless interrelated, circular

13 Philosophical Foundations of Research 1 activities. In Kuhn’s account, these activities are First of all, Kuhn approaches science historically, not formulaic, rather they reflect both social and dividing its development into distinctive phases: 1) communitarian activities as well as the individual Pre-science, 2) Normal science, and 3) Revolutionary scientist’s creative engagement. science. Each phase has its own distinctive practices. According to Kuhn, paradigm comes prior to Pre-science happens only once, before a field becomes any set of activities or procedures and provides the science. During pre-science, there are multiple practitioners with the current state of knowledge schools of thought, each has their own ideas about and practices of a field. A paradigm is defined as a basics of the field, definitions of the concepts, and worldview, or an intellectual or theoretical framework how the problems should be approached etc. During that provides guidance for the engagement of the this phase, the field is defined by a competition community of scientists with their phenomena of between the rival ideas. They lack consensus even interest. Thus, a paradigm provides the scientists on the most fundamental issues. When one of the with the basic concepts, problems, and exemplary rival groups develops a superior view that provides cases, as well as the basic assumptions regarding effective solutions to many of the accumulated the nature of the world and how such a world can problems, members of other schools start joining this be known and studied, all of which in turn guides group. Thereafter, the practitioners reach a consensus the entire process from observations to hypothesis and the field overcomes competition phase, thereon testing. However, Kuhn does not reject the fact a single paradigm begins to dominate the practice that there are several key features to science, such (Kuhn, 1962). as: 1) empirical observations, 2) generation and This phase that begins with achievement of testing of hypothesis, 3) construction and testing consensus is called the normal science (N). During of theories, 4) aiming to explain, predict, and normal science, there is a consensus among the possibly control phenomena in order to improve community of scientists about how to proceed, on the living conditions (Johnson & Christensen, definitions of the basic concepts of their fields, and 2004). Kuhn’s emphasis is rather on rejection of on the basic assumptions regarding the nature of formal methodological accounts that are based on the world (ontological assumptions) and how such inductive or deductive inference. Kuhn’s objection a world can be studied/known (epistemological to these accounts follows from their lack of regard assumptions). These assumptions are usually to the scientists’ paradigmatic commitments and implicit, and guide scientists in laboratories, during the socially and historically shaped nature of the observations and experiments, during hypothesis worldviews/paradigms and the methods employed. testing etc. The process during normal science is Rather, these accounts attempted simply to give similar to puzzle solving; in the sense that puzzles a generic formula for the methods of science. In provide overall rules for the game which establish a order to understand these points better, we should framework that structures an individual’s activity of review Kuhn’s work (Kuhn, 1962; Patton, 2002). solving the puzzle, the paradigm provides the same structure for the individual scientists. However, such a framework does not strictly define the moves. Rather, it allows opportunities for creative moves of Theory: Explanatory systems or the individual players, or the scientists. Once one generalizations usually expressed in the form learns the rules, they can solve any puzzle given of universal laws. they are dedicated and act creatively, even though solving some puzzles can be harder than the others. According to Kuhn, problem solving in science is similar to puzzle solving. Thus, similarly to a Hypothesis: Predictions or educated guesses player who does not doubt the framework of the about a certain phenomenon that are derived puzzle, rather takes it for granted and uses it as a from a theory. resource for their engagement, paradigm provides the same framework for the scientists during normal science. Thus, when a scientist can’t solve a problem, they doubt their own abilities rather than

14 Research Methods 1 doubting the paradigm. Further, whenever they The new phase of normal science is different encounter problems that they cannot solve, they from the previous normal science because each might put it aside until a next time, consult a peer, normal science has its own distinctive paradigm. or review the exemplary paradigmatic cases trying to Further, each paradigm gives a different lens, now understand where they are making mistakes. They the world and its reality looks different under the do not doubt the paradigm, which follows from new paradigm. That is, paradigms have the power of the currently accepted theory. Thus, Kuhn argues changing the world. And, the world can be seen from theory testing is not part of day-to-day practice of one paradigm or the other, but not from the both normal science. Here, Kuhn is strongly objecting to simultaneously. As Kuhn argued against Popper in the falsificationist account of Popper. According to a famous debate, in a same gestalt drawing (Figure Kuhn, contrary to being put under test, the theory 1.1), one can see either a rabbit or a duck, but not and the worldview it provides guides the practice both at the same time. Kuhn was directly addressing of problem solving, and it is taken for granted and Popper (who argued for falsificationism) saying that treated as a foundation making the practice possible. he and Popper look at the world of science from However, overtime, the unsolvable problems a different paradigm, saying where he sees a duck, start growing and anomalies start to accumulate. Popper would see a rabbit in the same picture. Even the most brilliant masterminds can’t solve these problems under the current paradigm. When this happens, the field enters into a phase of crisis (C). Only then, the community starts questioning the basic assumptions of their current theory. And, searches began for new ideas, for new ways of approaching problems; and competing schools of thought starts emerging, each offering their own distinctive views on how to deal with these problems. When this happens, the field enters the phase of revolutionary science (R). Revolutionary Figure 1.1 Kuhn’s duck-rabbit science is similar to pre-science in that there is a Source: https://commons.wikimedia.org/wiki/File:Duck- lack of consensus in both phases, because there Rabbit_illusion.jpg are rival ideas coming from competing schools of thought. Again, over time, an individual or a Moreover, the community holds on to any group comes up with superior ideas and their ideas previous paradigm until a better, more comprehensive begin dominating, thereby paradigm shift (PS) theory is developed. In other words, they don’t give takes place as the previously accepted theory is up a theory without a better one replacing it, despite replaced by a new theory. Thereby, the field enters its shortcomings. Because, even with its problems a next phase of normal science (N2). Thus, theory a theory nonetheless provides a framework to have testing takes place only during revolutionary the practice going, without a framework there is science. And, science has a cycle from normal no problem solving. Thus, a paradigm is valued science to revolutionary science and back to a new and cherished until a better, more comprehensive phase of normal science (N2) and to new phase of one is available. Further, a theory is refuted not revolutionary science (R2) and so on. The process through accumulated results of hypothesis testing moves in a spiral like cyclical process (see Figure (as the previous methodological accounts claimed), 1.2) (Kuhn, 1962). rather it is refuted when a better theory is available. And, the confirmation of a new theory involves risky predictions which are confirmed in future Anomaly: An anomaly is defined by Kuhn observations or experiments. A famous example as a deviation of the “paradigm-induced involves Einstein’s theory of General Relativity, expectations that govern normal science” which was accepted only after it’s prediction that (Kuhn, 1962, pp. 52-53). light bends around massive objects was confirmed during solar eclipse in 1919. (see Figure 1.2).

15 Philosophical Foundations of Research 1

Normal Science Normal Science important

Paradigm Shift Pre-science happens only once before a field is Revolution Anomaly transformed to science. After that, it is no longer a part in the process. Crisis Crisis

Figure 1.2 Spiral cycle of science

Kuhn’s Legacy Although Kuhn developed his paradigm theory of science based on his work in the natural sciences, his influence reaches far beyond the philosophy of natural science. His fame has been unprecedented particularly in social sciences, there are many important reasons for this. First of all, Kuhn established the idea that science is another social activity practiced by a community of practitioners just like other social activities. It had its own practical purposes and it has no greater claim to establishing an objective reality existing independently of the human beings (this was the philosophical claim in the beginning of the modern science). In Kuhn’s account, science was simply a practical activity with its practical benefits. Kuhn showed that science is simply another communitarian practice with its own distinctive social structure. There is a community involved, whose members engage in the practice of constructing useful knowledge. Any currently accepted theory provides the community with the concepts, solvable problems, ontological and epistemological assumptions, and general criteria for their practice. Scientific activity in this account became a creative activity, rather than being an activity strictly defined by formal rules. Kuhn argued that any paradigm provides a guidance for activity, for interaction of scientists with the objects of their interests, giving them a framework to act without necessarily engaging in long philosophical, existential (ontological), or epistemological considerations. These were issues discussed before the paradigm shift, thereon they were implicit within the accepted paradigmatic framework. Thus, individual practitioners during normal science did not have to be concerned with these most basic foundations, they were engaged in practical activity of solving problems the current paradigm produces.

The Primacy of Language In Kuhn’s account, we find the primacy of language, in definition of a field and its demarcation from other fields of study during pre-science phase; in definition of concepts and problems; in formulations of observations, in discussions and reports of the findings during normal science phase; in discussions over the basics of a discipline and the accumulated anomalies during revolutionary phase all of which involves language. We can say language is central to Kuhn’s concept of paradigm. In fact, language comes before any interpretation in the history of science that Kuhn himself provides. Kuhn highlights the fact that each individual scientist is socialized into a scientific community, into its paradigm. Thus, is trained in the process to view the world in the same way with the other community members. However, prior to this socialization, each member is socialized into a culture, a language community. That is, scientific training in any area presumes competence to a language, to its concepts. Because, the scientific language has to feed from the ordinary language. Thus, in Kuhn’s account we find how a common language of everyday life provides a common background for the scientific practice. From there, we come to see that there are no theory-free observations or reports of pure, objective, unchanging facts, because these things are defined by the concepts and meaning structures of a language in the first place. Thus, our ordinary everyday conceptualizations come first, scientific theorizing relies

16 Research Methods 1 on that. Therefore, with Kuhn we were convinced the scientific practice. This was the social, cultural that scientific activity was thoroughly theory- foundation and language was a central feature dependent, the talk about objectively standing of this foundation that science relies on. These facts or reality that stands free from theory was were also the foundations for any type of human simply a dream. Kuhn’s points were closely aligned knowledge. In short, as Kuhn showed science with the points that were already being raised by was another social practice, another interpretive the philosophers who marked the 20th century exercise, he was implying communitarian basis for with their revolutionary ideas, including Edmund knowing (for epistemology). Further, in Kuhn’s Husserl (1859-1938), John Dewey (1859-1952), account, science became another constructive Alfred Schutz (1899-1959). exercise as the knowledge was constructed by a community through language. Such an account Implications for Epistemology and rejects objective standing of a reality. Thus, in Kuhn’s accounts reality becomes human construction. Ontology He showed us how scientists were engaged in As Toulmin (1999) states, “Every cognitive construction of the natural reality (or the reality of judgement, even a direct report of perception, natural, physical world). This point itself has major is intersubjective.” That is, when I report my implications for the social sciences. What follows is observation to you, through language, you and I are that, they are too engaged in business of knowledge able to share a common meaning, through making construction, of reality construction. However, this use of the concepts of language. In other words, does not imply that the practices of two different we are able to establish a common world, and types of empirical sciences (natural sciences and language is central to this phenomenon. Similarly, social sciences) should be identical. This topic will scientists too establish a common meaning that be further elaborated in the following section. is shared by other members of their community. Thus, neither our observations, nor our concept or reports; neither our theories are an objective picture of an objectively standing, unchanging reality that 4 exists independent of us or our minds. Our ability Think about the relation between to know the reality, and have a common reality is language and our everyday reality. tied with our social and cultural practices, language Would it be possible for you to learn being a central one. Hence, we can be clear that all the things about your discipline science involves meaning making, establishing thorough experience or reasoning alone? a common ground for the external reality. Thus, science is not a distance look at an objectively existing reality. Thus, the traditional accounts of SOCIAL SCIENCES epistemology, both empiricism and rationalism, Again, modern science began in the 17th century had deep shortcomings, as they attempted to with then novel idea that we could understand approach our knowledge as if it was simply gained the nature empirically, through our experiences through individualized experiences or reasoning. in it, through observations and experiments. These accounts did not consider the socially Thus, empiricism was at the root of the scientific defined nature of experiences and reasoning, and tradition. Thus, empirical science rejected not only the central role of language in the process. Again, religious dogma as the source of knowledge about social life and its forms of meaning making predates the world, but also rejected the metaphysics (the an individual’s entry into the shared worlds, either philosophical speculations about the substance it be the world of ordinary communities or the and nature of existence and reality). It simply scientific communities. took the reality as we, human beings, experienced When Kuhn argued that science does not follow it. Empiricism ultimately yields to positivism, strictly defined methodological rules given in the which reached its peak in the early 20th century traditional accounts of methodology, he meant that in the form of logical positivism (also known as there was another type of foundation underlying ‘neopositivism’ and ‘logical empiricism’). Positivism

17 Philosophical Foundations of Research 1 insisted that only authentic knowledge is scientific second part of the 20th century. For example, knowledge and such knowledge can be achieved cognitive psychologist Piaget (1896-1980) showed through positive affirmations of theories through human cognition was active in construction of scientific method. According to positivism, in one’s understanding and knowledge of the external order to establish universal laws, observations must world. Piaget showed that individuals were actively be expressed in quantified forms. Social sciences interacting with their environments though senses, were established in the early 19th century, when starting in the early days after birth. In Piaget’s Auguste Comte coined the term “positivism” and account babies were actively constructing their proposed social sciences should follow the same, understandings of external reality through their strictly empirical and methods that were developed sense experiences. This meant that reality was in natural sciences. The tradition Comte initiated not projecting itself to a passive human mind. is continued by Durkheim, and currently by the Moreover, research on human perception also functionalist tradition. Psychological sciences also showed active engagement of the nervous system in followed the same footsteps, B.F. Skinner’s (1904- conceptualization of external reality. For example, 1990) behaviorism is the most famous example, in visual neuroscientists showed that depth perception which behavior is determined by its consequences. is a construction in our brains; in reality, our eyes Thus, social sciences have been heavily based on captures only two-dimensional representations of positivism, and its quantitative approach to the the world. These points were influential in showing study of social phenomena (Cresswell, 1994; that the human mind was not “blank slate” that Stanford Encyclopedia of Philosophy, 2018). passively takes in information from outside world. The arguments pointed to a direction that our world is not a world of free-standing, objective Quantitative Positivist Paradigm reality; neither is reality being projected into our Positivism is a philosophical position, which minds; rather our minds were active in construction states that societies or human behaviors function of our realities. Philosophers such as Ludwig based on underlying laws just like the laws of nature Wittgenstein (1889-1951), Alfred Schutz (1899- identified by natural scientists. So, according to 1959), Michel Foucault (1926-1984) and Thomas positivism, discovering these laws is the job of Kuhn (1922-1996) also argued that human activity social scientists (Campbell and Stanley, 1963). is socially, culturally and historically situated. This Basic beliefs of positivist philosophy include: meant that the reality changed over time and across 1. Scientific knowledge must be based on cultural groups. This further meant that, reality is empirically testable data. Thus, any scientific constructed; it is socially, culturally and historically claim must be empirically verifiable. If any constructed phenomena. This also meant that claim is not empirically verifiable, then it reality is multiple. Wittgenstein particularly argued must be non-scientific, or even meaningless. for how language practices, collective activities and 2. Empirical observations must be generalized life forms were central to meaning, hence to the and expressed as universal laws. Thus, process of reality construction. Schutz particularly observations must be reformulated as argued for centrality of meaning to the reality of mathematical expressions, most often in everyday activities and social life. Foucault went statistical formulations. a step further and argued how our most basic 3. There must be a unity of method across constructs like gender was socially and culturally all sciences. Thus, there is no significant constructed. These accounts collectively pointed to difference between the natural sciences and the shortcomings of the positivist approach. the social sciences. Another line of critique directed at positivism However, things started to change by the mid came from considerations of differences between the 20th century, when the accepted basic assumptions social worlds and the natural worlds. It was argued began to create a crisis, especially with regard to that the empirical world of the natural sciences are the nature of knowledge and the nature of reality. natural worlds, which are qualitatively different For one thing, the idea of direct perception of than the empirical world of the social sciences. The mind independent reality was debunked by the empirical world of the social sciences had interest

18 Research Methods 1 in were the social life and activity, relations and main objection was that structure and the order of behavior, institutions and organizations etc. As natural world and social worlds are different and Schutz argued, all the things that are part of social the positivist approach did not take this difference life (thus topic for social sciences) are organized into account. Rather, positivists treated the social around meaning that is shared by the social actors. reality as measurable, quantifiable, thus not taking That meant people act in meaningful ways towards into account that meaning as the central block of one another, their relations are meaningful, social life cannot be measured or quantified; it can organizations and institutions are structured around only be interpreted (Anderson, 1989; Bryman and meaningful practices. That further meant that the Burgess, 1994; Denzin and Lincoln, 2000). social life and its order cannot be explained in terms In short, natural scientists studied the objects of cause-effect relations as in the events of the natural of the natural world, which are physical/material world. As an example, Peter Winch (1926-1997) things made of atoms, electrons, particles etc. and argued that people stop at red light, not because red their interactions are shaped by physical forces, light causes them to stop. Rather, they stop because energies etc. So, to speak of the order of their red light signifies layers of meaning, about the order relations in terms of cause-effect is sensible. On of life, traffic, law enforcement, moral responsibility the other hand, social scientists study the social toward others etc. (cited in Cuff, Sharrock and worlds which consist of social objects, such as Frances, 1979). E.g., if I wanted to, I do not have relations, activities and practices, traditions, laws, to stop, I stop because I choose to obey the rules, organizations, institutions and so on, all of which thus, me stopping is not a causal effect. Moreover, are products of social processes. Thus, the order and all these webs of significances did not have to be structure of the social objects are products of human signified by the red light, as light did not have to be social processes that are communitarian/social and red, things could have been arranged differently and are based on shared meaning (or shared agreement, or instead of red light, purple light could have meant shared understanding). Further, language practices “stop”, or rather than light some other symbol like are central to the construction of meaning, to social a flag, or a picture, or stick could be used to convey activities, and to the shared life forms. As a result, one has to stop. In short, these arrangements are the order of the social worlds is an accomplishment not indispensable, they are not naturally so, they are of human beings, of the social actors. This means meaningful as a result of socially shared agreements. that they are not given, as the order of things in the On the other hand, the order of things is different natural world. Thus, they should be studied as such, in the natural world. As Schutz (1962) argued, the that is, as constructed worlds. order of the natural world is not organized around meaning; the chemicals do not act according to Interactive processes among the individuals meanings but people do. Thus, the methods of is central to the construction process, to which social sciences should be geared to delving into the meaning is foundational. For example, individuals meaning based structure of social life. Based on these respond to each other based on the meanings of conceptualizations, the positivists’ treatment of each other’s actions, not simply based on the social worlds in the same manner with the natural form of a behavior. An act can look same on the worlds were heavily criticized, and alternative outlook (behavioral), however may receive various programs and methods began to emerge in social forms of responses from others, because it will do sciences by the mid 20th century (Cuff at al., 1979). a different thing at each context, thereby signifying different meanings. To give an example, a request for meeting does not always receive an affirmative Qualitative Interpretive Paradigm response; the response will vary based on who is In general, interpretive paradigm is associated requesting it, where the request is taking place, with the philosophical reconceptualizations what are the conditions at the moment (e.g. regarding nature of reality and theory of respective ages, statuses, gender etc), and what is knowledge, and qualitative methods developed out the sensed purpose of the person doing the request, of dissatisfaction with the ability of quantitative all play a role. In short, meaning depends on the methods to reflect the nature of social reality. The context (Cuff et all., 1979).

19 Philosophical Foundations of Research 1 important from a first-hand experience, and interpret for their audience, hence, called the interpretive approach. Interpretive, constructivist position has Qualitative methods include approaches such some connections with idealism, however, as, having conversations with people, interviewing it differs in many respects. Constructivism them, observing their naturally occurring has no claims to existence of an immaterial interactions, collecting documents for analysis. world. They are similar in the sense that both acknowledge the role of human mind Qualitative methods are valued not only because they in human perception, human reality, and allow self-expression of the participants, also because human knowledge. they give access to local meanings, constructions and realities. However, this does not mean that there is an absolute rejection of use of numbers or statistics. However, meaning is not central to the order When needed, they are utilized for example to and structure of physical objects. As Schutz (1962) describe a population in terms of relative distribution famously stated “atoms do not mean”, they don’t of income, wealth, or educational attainment talk. Thus, the reactions of natural objects are not among sub-groups; for average ages for significant responses to a meaningful act, they are simply life events, gender distribution in occupations etc. reactions within the cause-effect chain. Therefore, However, the purpose is to use numbers descriptively, the first actors in nature who act around meaning not for prediction or explanation of a phenomenon, is the scientist (Schutz, 1962). Thus, it was the job for which not descriptive but inferential statistics are of the scientists to construct a reality of natural applied in positivist approach. phenomena, of chemical reactions, of physical In summary, the interpretive position denies the events etc. On the other hand, social scientists are possibility of an observer-independent objective not the firstmeaningful actors in social worlds. reality. Reality is viewed as thoroughly constructed, The social life social scientists enter are already by the collective activities and interactions of the meaningful, for those who occupy it. members, and interpreted by the social scientist The rejection of positivism quite naturally for an audience. Thus, social scientists are simply followed from these arguments. Quantitative aiming at reconstruction, or construction of a methods assumed reality was measurable, picture that makes the realities studied accessible quantifiable, statistically expressible and so on. for outsiders. Interpretive position is also known The positivists approached social reality as if it was as constructivism or social constructivism, independent of the human beings, from social, because of its focus on reflecting the constructed cultural and historical processes. Thus, they were nature of social reality and human knowledge of unable to reflect the contextual nature of human the constructed reality (Anderson, 1989; Dehyle, behavior and social processes. Those who opposed 1992; Denzin and Lincoln, 2000). to positivism and quantitative methods, argued In summary, the position of interpretive that social phenomenon requires that scientists qualitative approach is: enter into the worlds they study and get close to Ontologically: All reality is not given (objective) the phenomena under study in order to understand but constructed. Natural reality is constructed by the its contextual conditions. For example, staying scientists, social reality is constructed by the social as close as possible to the parties involved in a actors (members) and knowledge of it is constructed conflict, to the people reacting to an established by the social scientists who study those realities. To rule, policy, tradition or practice. Thus, rather than say reality is constructed is to say reality is social, being an objective observer of an external world and cultural, and historical (in both areas, our reality of reality that is taken as given, the social scientists are the natural world as well as our reality of the natural expected to be part of the social worlds that he/she worlds). That is, reality varies across cultures and intents to study in order to understand its dynamics throughout history. Thus, reality is multiple.

20 Research Methods 1 Epistemologically: Coming to know the world International Relations Form a is a communitarian practice in scientific practice Qualitative/Interpretive Paradigm as well as in ordinary lives. Language is central to When we consider international relations from acquiring and constructing knowledge along with a qualitative interpretive perspective, we come to sense experience and reasoning. Thus, knowledge realize that the structure of current relations and is not internal to the individuals, it is part of the the relative power dynamics are not objectively social domain; individuals gain their knowledge so, rather they are outcomes of social, cultural from the social and cultural arena. Thus, knowledge and historical dynamics. That is, the current is socially, culturally constructed, and historically reality reflects just the current situation, things situated. could have been ended in different dynamics. Methodologically: Scientific study of natural The structure of current dynamics reflects current reality through measurement, quantified and shared understanding among parties regarding statistical expressions can be suitable in natural their respective statuses and roles, which of course sciences, but for the social sciences they are not itself is a result of historical events. The functions of appropriate. Social sciences require qualitative international organizations, business connections, methods because of the already constructed nature political agreements, all are seen as constructed of social reality. thus can be reconstructed. Reconstruction does Moreover, taking social reality as objective, not only involve roles being swapped, in a sense rather constructed is further rejected due to its yesterday’s less powerful becoming a dominant implicitly ideological tendencies. Approaching force in a future time while the structure remains social reality as independent of human beings, the same. But, also development of a different and human social processes, thereby denying its type of structure altogether can be seen an open constructed nature, gives an appearance that current possibility. The current structure and how it shapes order and structure of things is unchangeable. relations, and the current power dynamics is seen For example, the class structure of a society, the as reflecting certain underlying assumptions shared structure of relations between nations, ethnic among the parties. And, because the structure is groups, states etc., the patriarchal and androcentric constructed, and because its nature is historically structure of relations is taken as naturally so. Thus, situated, the structure itself and the underlying research itself becomes politically and ideologically agreements keeping it as is are subject to the motivated activity, knowingly or unknowingly. change. Social processes within its historical path Positivism thereby ratifies current order of might result in a brand-new structure. things. Thus, interpretive qualitative approach is well aware that methodological approaches are Relate it to the world history not simply technical issues, they carry not only Think about how quickly the structure of relations philosophical assumptions (Sparks, 1989) but also changed among the states within the Union of political and ideological commitments (Anderson, Soviet Socialist Republics (USSR) in just a couple 1989). In other words, when the constructed of years between 1989-1991. nature of reality is not reflected in the methods, Try to imagine a structure for international and not discussed as an underlying philosophical relations that shows totally different dynamics. assumption in methodological accounts, the current order of social life gets ratified. The post- positivist interpretive approach is well aware of these dynamics and reflects such an awareness in the methods employed.

21 Philosophical Foundations of Research 1

5 Define the terms ontology and epistemology in connection with international relations. The nature of political reality – the claims about the things exist, what they look like, what units are involved, how these units interact with each other is called ______. The study of political ideas and knowledge is called______.

We began our review with the philosophical assumptions with the natural sciences, from its beginning in the early 17th century through Kuhn’s historically oriented analysis in the 20th century. We came to a point that there is neither a unified method of science, nor science yields absolute knowledge of objective reality that projects itself to the human mind. Rather, we came to see scientific practice as another social activity that yields useful knowledge about the physical, natural world. With Kuhn’s paradigm theory, we came to view the scientists as knowledge constructors, who work within a community of practice, under the guidance of a currently accepted paradigm. The paradigm gave practitioners basic assumptions about the nature of the physical world and how human beings can come to find out about such a world (i.e. what methods they could use), which were socially shared and implicitly followed. Thus, Kuhnian perspective brought us to a point where we started to see the shortcomings of the traditional philosophical approaches that were normative. After that, we moved to the social sciences. We reviewed how social sciences were shaped by positivism from the beginning, and thus, how social scientific methodological conceptualizations were borrowed from the natural sciences. Throughout our review, we came to realize that the research practice in social sciences did not take into account the differences between the world of nature (the physical objects and their relations) and the world of social lives (social objects, the people, their relations, interactions, institutions, organizations and so on…as well as the structure of all these social phenomena) since the beginning. Then, we discussed growing dissatisfaction with the traditional philosophical approaches, and how these movements resulted in replacement of traditional epistemological and ontological assumptions with a view that saw knowledge as constructed. Then, we discussed how the knowledge construction process needs to employ different methods in natural and social sciences due to the differences in the nature of respective worlds they study. In light of these arguments, we argued that the study of social worlds requires qualitative methods and accompanying philosophical assumptions of the interpretive approach.

22 Research Methods 1 Define basic concepts in philosophy of science and differentiate LO 1 between the basic assumptions of empiricist epistemology vs. rationalist epistemology; between the realist ontology vs. idealist ontology.

Epistemology: Study of human knowledge, of how we humans come to know the reality of our world. Ontology: Study of reality: study of the things that exist and their kinds, of the fundamental substance of those things that exist. Rationalism: An approach to epistemology, argues for existence of innate ideas and for the primacy of reason as a way in knowing the reality of our world. Empiricism: Another approach to epistemology, argues against existence of innate ideas Summary and argues for the primacy of sense experience (naturally observations and experiment) in knowing the reality of our world. Empiricism takes the sense experience as the only source (or foundation) of human knowledge. Empiricism rejects the rationalist position that we are born with innate ideas, from there we gain further knowledge through reason, and reason is the only source of certain knowledge. Empiricism claims the base of reason itself is experience, not innate ideas. We would not have reason if we did not have ideas which are gained through experiences in the world. Realism: One of the major metaphysical positions regarding the nature of reality (ontology); argues that there is correspondence between the external reality and the human perception of reality. In other words, external reality is the same as the human mind’s perception of reality. Materialism is a form of realism that defends that the substance of all things that exist are matter (not ideas.) Indirect realism is a version of realism, claiming our reality only represents the external reality (rather than corresponding to it). Idealism: Another metaphysical position regarding ontological stance of existence. Idealism rejects the material basis of existence, rather defends that existence is based on ideas, thus existing things are immaterial, they are ideas in the mind. Materialism: A realist position regarding substance of things, argues that there is only one substance for existing things, which is the matter (a monistic position). Immaterialism: An idealist position regarding substance of things, argues that there is only one substance for existing things, which is the mental idea (monistic position). Dualism: is a position between materialism and immaterialism, argues for two substances of existence, matter and ideas/mind (a dualistic position). Realist/materialist position towards the nature of reality (ontology) holds that all existence has a material basis, and our perception of these material things and their relations is objectively real. They reject the idea that our reality could be different from the reality as it exists outside of us.Idealist/immaterialist position regarding nature of reality (ontology) argues for ideal (immaterial) basis of existence, in other words thins exists as ideas in our minds, there is no material basis of existence. And, the basis of reality is mind dependent, that is, because the only reality is what we can perceive, we can’t really know if there is an objective reality independent of us.

23 Philosophical Foundations of Research 1

Discriminate a priori proposition from a posteriori proposition/ LO 2 knowledge; hypothesis from theory; formal sciences from empirical sciences; and natural empirical sciences from social empirical sciences.

A priori propositions are propositions that can be known without (or before) experience. These are propositions one can arrive at through pure reasoning, as in mathematics. E.g. one can know 2+2=4 only through reasoning. A posteriori propositions are propositions that can be known only after experience, thus they are empirical propositions. Any statement about the nature or social life is a posteriori, because we can know them only through experience, and we can also justify them only through experience. We need to test these propositions against the nature to know if they hold or not (true or false). Thus, propositions of empirical sciences are a posteriori propositions.Theory: A theory is an explanatory system or generalization about the phenomena of interest. Hypothesis: a hypothesis is a prediction or an educated guess foreseeing a certain outcome about a phenomenon. Hypotheses are usually derived from theories. Formal sciences include mathematics, logic, and

Summary other fields based on analytical thinking. These fields can provide certain (or absolute) knowledge, as long as the scientists choose premises that are true and they are careful when drawing conclusions from these premises. Formal sciences use a priori propositions; thus, reasoning is their only method. They differ from empirical sciences, as the word “empirical” implies these areas are about the world, and knowledge of the world requires empirical observations, experiments, and testing of propositions against the world. Natural empirical sciences are interested in nature, the physical world, e.g. physics, astronomy, chemistry, biology; Social empirical sciences are interested in social life (institutions, organizations, relations), e.g. sociology, political science, anthropology etc.

Compare inductivist methodology to deductivist methodology, and LO 3 discuss the problems associated with each.

Inductivist methodology of empirical sciences draws generalizations from singular observations, thus their conclusions are about the past, but once they are generalized, however, they apply to any case, including unobserved cases. This methodology follows a bottom-up direction.Deductivist methodology follows a top-bottom direction; it starts from a generalization (theory) and deduces observable implications from it, which then is tested against the empirical world. Thus, hypothesis- testing is a deductivist method used in empirical sciences. Problems of inductivism: Problem of sensation: We perceive the world through sense data, however senses are unreliable, they can deceive us. Problem of induction: We generalize from singular cases that we experienced, however, a generalization refers to past, present and future cases. Thus, the inductive generalizations are unreliable. Problem of theory-ladennes: Any observation requires prior understanding, prior expectations, thus, theory-free observations of the world is not possible. Problems of deductivism: Problem of sensation and problem of induction is still a problem in deductivism. Although, problem of theory-ladennes is eliminated, because observations are conducted under a theory, a hypothesis drawn from a theory shapes observations.

24 Research Methods 1 Define Kuhnian terminologies such as paradigm, paradigm shift, pre- LO 4 normal science, normal science and revolutionary science as situated within the historical context of scientific practice.

Paradigm: A worldview that carries basic assumptions about the nature of reality, and how such a reality can be known by human beings, although most of these assumptions are implicit in the ordinary day-to-day practices. Paradigm gives a framework, a foundation for scientific practice (indeed, to any social practice) during the period of normal science. Paradigm shift: A process of leaving one worldview (paradigm) for another more comprehensive worldview; the process which Summary involves considerations of taken for granted assumptions about the nature of reality and how such reality can be known. Pre-science: A developmental phase in history of a science, during which a field is not yet considered to be science. Normal science: A phase in history of science in which an accepted paradigm guides the practice, providing consensus among the scientists regarding fundamental issues (e.g. ontological and epistemological assumptions) and regarding the important problems and methods of working on these problems. Revolutionary science: A phase in history of science in which basic assumptions of a currently accepted theory is questioned, and rival ideas coming from competing schools of thought permeates the field.

Discuss how are the nature of social worlds different from the nature LO 5 of natural worlds in relation with the positivist quantitative and post- positivist qualitative approaches regarding the methods of study.

Social worlds are organized through cultural practices by the human beings and language is central to these social processes. Thus, any aspect of social worlds is necessarily meaningful; that is, established based on meaning. Thus, such a world requires study methods that reflect the central place of meaning in any social establishment. In other words, it should be able to reflect the constructed nature of any aspect of social life. On the other hand, the order of natural world exists independently of the human beings who live in these worlds and who develop interest in studying these worlds. Because, the aspects of nature are not centered around meaningful acts and behaviors as in social life, the study methods employed to the natural worlds does not need to be concerned with developing broad and in depth understanding. Thus, natural scientists are mostly interested in finding explanations to observed phenomenon, in other words reasoning causes (that are unseen) for seen phenomenon. For which, quantified methods can be suitable.

25 Philosophical Foundations of Research 1

1 Which of the following is an epistemological 6 Changing from Ptolemaic geocentric view position that argues against the primacy of of universe in natural sciences to the Copernican reasoning and for the primacy of sense experience? heliocentric view, and from positivist view in soci- al sciences to post-positivist view is an example of a. Rationalism b. Idealism which one of the following? c. Empiricism d. Realism e. Materialism a. Normal science b. Revolutionary science c. Pre-science Which one of the following is an ontological 2 d. Paradigm shift position that argues for material substance and mind-independent reality while arguing against e. Empirical science.

Test Yourself Test immaterial substance and mind-independent reality? 7 Which one of the following is used to desc- ribe a period in which there is a consensus among a. Realism b. Idealism the scientific community on the fundamental as- c. Rationalism d. Dualism sumptions of their discipline? e. Empiricism a. Revolutionary science b. Pre-science 3 Which one of the following is an ontological c. The period of crises. position that argues for both material and d. Empirical science. immaterial forms of existence? e. Normal science a. Realism b. Idealism c. Materialism d. Dualism 8 Which one of the following best describes e. Rationalism the reason for endorsing qualitative methods for studying social life in post-positivist interpretive 4 Which one of the following is an ontological position? position that defends the idea that human a. It provides broad analysis and in depth unders- perception of reality correspondences to the tanding of the local meanings. external reality? b. It provides a means for measurement and statis- a. Indirect realism b. Idealism tical expression of social reality. c. Rationalism d. Empiricism c. It allows for any explanation for the same social e. Direct realism phenomenon. d. It allows a way to escape complicated statistical calculations. 5 What is the reason for Karl Popper to argue for falsificationism in place of verificationism in e. It allows for employment of an accurate hypot- hypothesis testing? hesis testing. a. Falsification requires many cases but verification requires just a single case. b. Verification relies on inductivism, falsification relies on deductivism. c. Verification requires many cases but falsification requires just a single case. d. Falsification relies on inductivism, verification relies on deductivism. e. None of the above.

26 Research Methods 1

9 What is the best justification for rejection of 10 A worldview that carries basic assumptions quantitative methods and positivist paradigm in about the nature of reality and the best ways for social research? studying such a reality is referred to by Kuhn as: a. It provides opportunities for the researchers to a. Paradigm shift. socialize with local people and develop mutually b. Paradigm. Test Yourself sympathetic relations with them. c. Normal Science. b. Quantitative methods require statistical analy- d. Pre-science. sis, which is overly cumbersome. e. Empirical science. c. The qualitative differences between the nature of social life and the nature of natural world. d. Quantitative methods require large numbers of participants thus can be very difficult and ex- pensive to conduct. e. Qualitative methods can be fun as it allows for entrance to diverse populations.

27 Philosophical Foundations of Research 1

1. c If your answer is incorrect, review the section 6. d If your answer is incorrect, review the section on “Epistemology”. on “Introduction and Paradigm Theory”.

2. a If your answer is incorrect, review the section 7. e If your answer is incorrect, review the section on “Ontology”. on “Paradigm Theory”.

3. d If your answer is incorrect, review the section 8. a If your answer is incorrect, review the section on “Ontology”. on “Social Sciences”.

4. e If your answer is incorrect, review the section 9. c If your answer is incorrect, review the section on “Ontology”. on “Social Sciences”.

5. c If your answer is incorrect, review the section 10. b If your answer is incorrect, review the section on “Methodolgoy/Deductivism”. on “Paradigm Theory”. Answer Key for “Test Yourself” for “Test Answer Key

Why do you think we need philosophy of science? What benefits can be drawn from philosophical investigation of sciences? How does the philosophy of science differ from science itself? Find a partner to discuss these issues.

Philosophy of science answers critical questions about a discipline of study, such as what is its demarcation line from other related areas, what is its your turn 1 ontological and epistemological assumptions, what research methods it uses, and if those methods are suitable for the kind of reality they deal with. It also defines the fields relations with other areas of study.

With a partner compare and contrast the basic principles of inductivism versus deductivism.

Inductivism follows a bottom-up approach, while deductivism follows a top- down approach to theory building and testing. Additionally, inductivism your turn 2 believes theories emerge from theory-free observations, while deductivism rejects the idea of theory-free observations, thus deductivism defends the idea of observations being laden by a theory. Suggested answers for “Your turn” Suggested answers for “Your

28 Research Methods 1 Questions to think about: Let’s assume, the world did not have regularities, would inductivist inference be possible? If human beings are confined to their senses in making sense of the world, then does it make sense to doubt the senses as a way of building knowledge? Can we separate our

senses from our ability to reason? Suggested answers for “Your turn”

The world has to be a regular place for inductive inference. If the sun did not rise from the east everyday, we could not be able to infer that “the sun rises from the east.” Senses are the way we are connected to the world outside of us. If we are able to come to the conclusion that “senses can wrongly inform your turn 3 us about the reality, making us think that reality is same as how it appears to us” it also requires senses. Seeing the same stick in two different appearances (bent and straight) without any breaking event happening in between gives us the reasoning capacity to conclude bent look is just an appearance not the reality. Thus, our senses and our ability to reason are integrated.

Think about the relation between language and our everyday reality. Would it be possible for you to learn all the things about your discipline thorough experience or reasoning alone?

Of course not, language is central to everything in human life. Without it we wouldn’t be able to define our experiences, describe our reasoning… In your turn 4 fact, we would not be able to reason at all. Further, the definitions available in language gives us a view towards how to perceive and how to define the world.

Define the terms ontology and epistemology in connection with international relations. The nature of political reality – the claims about the things exist, what they look like, what units are involved, how these units interact with each other is called ______. The study of political ideas and knowledge is called______.

The nature of political reality – the claims about the things exist, what they look like, what units are involved, how these units interact with each other is your turn 5 called ______Ontology______. The study of political ideas and knowledge is called______Epistemology______.

29 Philosophical Foundations of Research 1

Further Reading

Benton, T. & Craib, I. (2001). Philosophy of social Godfrey-Smith, P. (2003). Theory and reality: An science: The philosophical foundations of social introduction to the philosophy of science (science thought (Traditions in Social Theory). London: and its conceptual foundations series). Chicago: The Palgrave Macmillan. University of Chicago Press. Campbell, D.T. & Stanley, J.C. (1963). Experimental Patton, M.Q. (2002). Qualitative research & and quasi-experimental designs for research. evaluation methods: Integrating theory and practice. Chicago: Rand McNally Los Angeles, USA: Sage. Chalmers, A.F. (1976). What is this thing called science? Winch, P. (1958). The idea of social science and its Queensland, Australia: University of Queensland relation to philosophy. London and NY: Routledge. Press. Chomsky, N. (1959). Review of B.F. Skinner’s, Verbal Behavior. Language, 35, 26-58.

References

Anderson, G.L. (1989). Critical ethnography in Humphreys, P. (1989). The chances of explanation: education: Origins, current status, and new Causal explanation in the social, medical, and directions. Review of Educational Research, 59: 3. physical sciences. Princeton, NJ: Princeton University Press. Bryman, A., & Burgess, R.G. (1994). Analyzing qualitative data (Edited). London: Routledge. Johnson, B. & Christensen, L. (2004). Educational research: Quantitative, qualitative and mixed Campbell, D.T. & Stanley, J.C. (1963). Experimental approaches. Boston, USA: Pearson. and quasi-experimental designs for research. Chicago: Rand McNally Kuhn, T.S. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press. Creswell, J.W. (1994). Research design: Qualitative and quantitative approaches. Thousand Oaks, CA: Patton, M.Q. (2002). Qualitative research and Sage. evaluation methods. Thousand Oaks, CA: Sage. Cuff, E.C., Sharrock, W.W. & Francis, D.W. (1979). Popper, K.R. (1965). Conjectures and refutations. 2nd Perspectives in sociology. London and New York: ed. New York: Basic Books. Routledge. Schutz, A. (1962). Collected papers I: The problem Denzin, N.K. & Lincoln, Y.S. (2000). Handbook of of social reality. Martinus Nijhoff, The Hague qualitative research (Eds.). Thousand Oaks, CA: Amsterdam. Sage. Sparkes, A.C (1989). Paradigmatic confusions and the Deyhle, D.M. (1992). Constructing failure and evasion of critical issues in naturalistic research. maintaining cultural identity: Navajo and Journal of Teaching in Physical Education, 8, 2. Ute school leavers. Journal of American Indian Toulmin, S. (1999). Knowledge as shared procedures. Education, 24-47. In Yrjo Engestrom, Reijo Miettinen & Raija- Godfrey-Smith, P. (2003). Theory and reality: An Leena Punamoki-Gitai (eds.). Perspectives on introduction to the philosophy of science (science Activity Theory. Cambridge University Press. and its conceptual foundations series). Chicago: The University of Chicago Press.

30

Chapter 2 Planning Research After completing this chapter, you will be able to:

Define concepts used in scientific research Explain quantitative, qualitative, and mixed 1 studies 2 method designs

Plan quantitative, qualitative, and mixed 3 method studies in a stage by stage fashion Learning Outcomes Chapter Outline Key Terms Introductıon • Scientific research • Quantitative Planning Quantitative Research Studies • Research problem research Planning Qualitative Research Studies • Variable • Qualitative research Planning Mixed Method Research Studies • Dependent variable • Mixed method research • Independent variable • Population and sample • Control variable • Data collection • Hypothesis • Data analysis • Research design

32 Research Methods 2 INTRODUCTION Identifying the Research Problem Scientific research is the process of systematic The very first step of a research process is and planned data collection, data analysis, problem identification, which can be distilled from assessment and interpretation, and reporting in various sources including daily life, practical issues, order to figure out reliable and valid solutions findings of previous research, and relevant theories to problems by limiting them (Erkuş, 2011). (Christensen, Johnson, and Turner, 2014). Besides, According to another definition, it is a process to a researcher’s own experience and observations, gather and analyze information so as to gain insight discussions among scientists from the same field, about a topic or issue (Creswell, 2012). Based on and different opinions can also serve as sources to these definitions, it is possible to deduce that a formulate research problems (Erkuş, 2011). After scientific research process involves 1) problem problem identification, research problem has to identification, 2) data collection, 3) data analysis, be defined. Research problem is often constructed 4) interpretation, and 5) report writing. All these as an interrogative statement that aims to analyze stages are relevant parts of quantitative, qualitative, the relation between two or more variables. For and mixed method research efforts. If one wants to example, “Is there a relation between states’ national make use of different methods, then s/he should income and the budget they allocate for armament?” consider each method’s unique specifications can be a research problem because it includes two during planning stage. In other words, both variables such as the national income of states and design and planning stages accommodate certain the budget they spare for armament, and it clearly differences for quantitative, qualitative, and mixed depicts the aim to search the relation between methods. This chapter provides information as to these two variables. A well-defined quantitative how quantitative, qualitative, and mixed method research problem has to bear some certain features. research studies can be planned. First, the variables in the problem have to explain a relation. Second, it has to be formulated as a question. Accordingly, formulas such as “Is there a relation …?”, “What is the effect of …?”, and “Does it change …?” can be employed to construct a Scientific research is a process of systematic question for a research problem. Third, a research data collection and analysis to figure out a problem has to be suitable for analysis based on solution to a problem. collected data, or it has to be testable. To put it differently, a research problem has to be compatible with empirical studies, which is the criterion to PLANNING QUANTITATIVE distinguish what is scientific from what is not. RESEARCH STUDIES Therefore, issues regarding metaphysical events, philosophy, theology, and astrology are not suitable Quantitative research is based on positivist to form research problems in a scientific sense as paradigm, which advocates that reality will they cannot be examined in empirical studies. surface precisely through logic and experiment. Primary building blocks of quantitative research Accordingly, it strives to formulate universal laws are the variables (Christensen et al., 2014). Thus, for each field of scientific study. Such a stance with they are considered as one of the key components respect to the nature of reality led to the birth of during problem identification. A variable can bear a research design to produce knowledge that is at least two different values. The term variable measurable, replicable, generalizable, intact, and is used to refer to features that can change. For value-free judgements. Unlike qualitative and instance, an individual’s age, gender, income level, mixed methods, quantitative studies entail a strict and political views can be labeled as variables. planning process that researchers have to abide step Depending on their values and control types, by step right from the beginning. Following sub- variables are classified in two groups. With respect titles provide guidance as to how a quantitative to their values, they are labeled as either discrete or research process should be planned. continuous variables. If a variable can only assume

33 Planning Research 2 certain values or options, and if it can be expressed collected. For example, a researcher investigating through only whole numbers, then it is a discrete the relation between people’s life styles and the variable. For example, gender variable can assume political parties they support may hypothesize either male or female option. Continuous variables, that “conservative people vote for political parties on the contrary, are those that can take any value on the right wing.” According to this hypothesis, between their bottom and top limits, that is to say, the researcher believes that her/his research will they can be expressed in fractions. For instance, conclude that people with conservative life styles the value that variable of weight can assume ranges vote for political parties on the right side of political from zero to infinity (Karasar, 1994). As for the spectrum. control types, variables are classified into three: dependent, independent, and control variables. Dependent variable is the one whose change is A hypothesis is either an educated guess observed in comparison to other variables, and it about the result of a problem or a temporary is generally symbolized with the letter “Y” (Erkuş, solution to a problem. 2011). Pretend that our research problem focuses on the influence that international organizations have over the level of globalization. In this case, A hypothesis is the conversion of a research the level of globalization is the dependent variable problem into a tangible form that can be examined since its change is examined in comparison to and tested. Thus, a well-developed hypothesis other variables. Independent variable, on the other should clearly point out the dependent and hand, is the one that has some kind of effect over independent variables, their levels, the research the dependent variable, and it is mostly symbolized model, and whether the relation or difference will with the letter “X”. Let’s say that a research problem be studied in that research (Erkuş, 2011). As a is formulated to figure out the effect that ethnicity matter of fact, comprehensive knowledge about the and educational background have over individuals’ field of study is indispensable to produce a well- political preferences. In this example, the variables developed hypothesis. They are mostly derived that are assumed to have a certain level of influence from theories, yet sometimes literature reviews over the result are ethnicity and educational may also lead to hypothesis formation. Besides, a background. Thus, these two are the independent researcher may base her/his hypothesis on her/his variables whereas political preferences stand out as daily life experiences. On the whole, more than the dependent variable. Control variables are those one hypotheses are developed for the solution of that can affect the dependent variable but not a problem. Regardless of the formation process among the independent variables. The researcher (either distilled from relevant theories in the field or should stabilize the control variable(s) to find based on a researcher’s daily experiences), it should out how the independent variable influences the always be worded in a way that can be confirmed dependent variable. or refuted because it is the hypothesis, not the A research may contain control variables problem, that is tested in research study. Approval pertinent to the population, research process, and denial of a hypothesis entail confirmation and and outside sources. Control variables regarding refutation processes respectively (Christensen et al. the population refer to characteristics or features 2014; Sönmez and Alacapınar, 2011). Hypotheses that individuals have such as intelligence, age, and are of two major types: statistical and research gender. Others concerning the research process hypotheses. Statistical hypothesis (H0- neutral are related with the operations during the research hypothesis, null hypothesis, absence hypothesis) such as measurement and time. Lastly, variables denotes that there is either no relation or difference peculiar to the research setting such as noise and between the variables. Research hypothesis, on the light are considered as control variables related contrary, (H1- difference hypothesis, alternative with outside sources (Karasar, 1994). Once the hypothesis) is the expression of the relation that problem is identified, the next step is to formulate a researcher hopes to confirm after relevant data a hypothesis, which indicates the researcher’s collection and analysis (Altunışık, Coşkun, educated-guess about the result before any data is Bayraktaroğlu, and Yıldırım, 2005). Since the

34 Research Methods 2 research hypothesis does not explain the direction Selecting the Research Design of the difference, it could also be written in the Research design is the arrangement necessary form of a research question. Whereas a research to collect and analyze research data in the most hypothesis is written as “There is a difference between economical way. There are two primary research … and …”, a research question is formulated as designs often utilized for quantitative research “Is there a difference between … and …?” (Erkuş, studies, which are survey and experimental 2011). Below are some examples: designs. A researcher’s choice to employ either • H0: Educated and uneducated people do survey or experimental design is highly correlated not differ in their voting preferences. with the hypothesis or research questions s/ • H1: Educated and uneducated people differ he develops in the previous stage. If a researcher in their voting preferences. writes a hypothesis or research questions directed • Research Question: Is there a difference to assess an intervention’s or innovation’s effect between the voting preferences of educated over the result, s/he should stick with experimental and uneducated people? designs. On the other hand, if a researcher aspires In a given research, hypotheses can be written to describe participants’ attitudes, opinions, as uni-directional or bi-directional. A bi-directional behaviors, and features as they are without any hypothesis reads as “there is a significant difference” interference, then s/he should make use of survey or “there is a significant correlation”. Uni-directional models. For instance, a researcher aiming to test hypothesis, on the other hand, can be written as the relation between the following presidential “The effect of method A is significantly higher than election discussions on TV and support tendency that of method B” or “Women have more democratic should prefer the experimental design. However, tendencies than men” (Sönmez and Alacapınar, another researcher trying to figure out how content 2011). For instance, the public is with the economic policies of their • There is a significant difference between government should use survey models. the vote rates of political parties employing traditional electioneering methods and Survey those utilizing technology-supported propaganda methods. (Bi-directional Survey serves to determine general tendency, hypothesis). attitude, or opinions of a population quantitatively • The vote rate of political parties utilizing or in numerical values by working on a sample technology-supported propaganda chosen from that population. The researcher methods is significantly higher than that of draws inferences about the population based others employing traditional electioneering on the data collected from the sample. Survey methods. (Uni-directional hypothesis). can be employed to assess individuals’ attitudes, Qualitative research studies do not require opinions, tendencies, and beliefs. Most frequent hypothesis writing. Because qualitative research data collection tools for survey are questionnaires, stems from post-modernist, post-positivist, and scales, or interviews. Survey can be implemented interpretive tradition, they believe that chaos rules once (cross-sectional) or at different times in the population rather than accepting that there (longitudinal). In cross-sectional study, research is an order. Therefore, qualitative research does not data is gathered from the participants once and in a aim to come up with a result that can be generalized short period of time. The participants in the sample to the entire population. Furthermore, it is not provide the research data only once (Christensen, possible to draw predictions about the data since et al. 2014). Although data is collected only once, reality is tentative. Accordingly, it is not plausible the preparation process of a cross-sectional study to set forth a de-facto principle about constantly especially in international relations should include changing phenomena, people, and objects. In data collection stage from different social groups, this sense, qualitative research is not based on ages, socio-economic classes, and genders. For hypothesis and does not require one (Sönmez and example, data should be gathered from a sample Alacapınar, 2011). composed of participants with different features

35 Planning Research 2 such as ethnicity, age, educational level, income primary experimental designs to be employed for level, and province in a survey research focusing research about political sciences and the principles on societies’ opinions about armament on an that should be taken into consideration during international scale. This way can help determine implementation (Johnson, Reynolds and Mycoff, what different groups think about armament and 2016). what the potential differences are among their opinions. The Classic Randomized Longitudinal study aims to collect data at least Experimental Design two or more times. Such studies generally last for In this design, a researcher should foresee the years. Thus, longitudinal studies may not be the formation of two groups: one (or more) is the best choice when sources and financial support experimental group consisting of participants is limited and quick results are expected. For given an experimental intervention, and the other instance, a researcher enthusiastic to examine the is the control group composed of participants changes in the tendencies that Turkish Republic who are not either given any experimental citizens display for EU membership can determine manipulation or provided with any test stimuli. the differences in Turkish society’s ideas, beliefs, or The researcher should assign the participants in opinions by collecting data from the individuals in each group randomly. Subjects cannot decide the sample at certain intervals (e.g. every two year) which group they are going to be in. This random for a certain time (e.g. ten years). assignment of participants to any of the groups is known as randomization, and placement in any group is a matter of chance. Subsequently, Survey is a quantitative research design the researcher administers the experimental aiming to determine individuals’ attitudes, intervention. In other words, s/he provides the behaviors, or opinions as they are without experimental group with relevant stimuli by any outside interference. determining the time, setting, and situation on her/his own. Using this design, the researcher establishes a dependent variable that can be important measured both before and after the independent variable is introduced. These are termed as pre-test and post-test measurements, and they indicate Longitudinal research is a design where data if the experimental process has any effect or is collected from the same sample at different not. This gauges the effect that the independent intervals at least two or more times to observe variable has over the dependent variable, namely changes in time. the main focus of that experimental research.

Experimental Designs Post-test Design Experimental designs provide the researcher Like the classical randomized experimental with the opportunity to determine the independent design, post-test design also accommodates two variable’s effect, to assign subjects to different variables (dependent and independent) and two groups, and to assess behaviors and responses. groups. Again, subjects are randomly assigned Nevertheless, mostly non-experimental designs are to each group. Whereas the experimental group preferred for research in political sciences. This is undergoes an intervention, the control group partly due to the nature of phenomena in political completes the research without any manipulation. sciences that focus on topics such as voting in real Next, dependent variable is measured for each elections rather than in experimental settings. group. Lack of pre-test is the only difference Yet, experimental design is still of significant between this design and classical randomized value in research about political sciences because experimental design. Therefore, it is not certain it scientifically unravels what to do to either if each group (experimental and control) has the confirm or support an allegation. Below are same baseline average concerning all the variables.

36 Research Methods 2 Researchers employing this design advocate that of multi-cultural social environment. One of the difference determined between the two groups these groups is assigned as the control group. No after measuring the dependent variable is a result of multi-cultural environment is established for the the experimental intervention. control group. Remaining groups are identified as experimental groups. However, multi-cultural Repeated-Measurement Design social environments built for the experimental groups have to be of different qualities. Let’s say This design is preferred when the precise time the multi-cultural social environment set for one of for the emergence of independent variable’s effect is the experimental groups consists of famous people not certain and when it is not clear when to conduct and the other hosts regular people. Following the the most reliable pre-test. Accordingly, researchers experimental intervention, members in all the administer different assessment and measurement groups are subjected to a post-test for comparison, tools before and after the experiment. This design and then it is determined if multi-cultural social may be suitable for a research in which the relation environment intervention was successful or not. between following presidential election discussions on TV and support for the candidates is tested. In this sense, two groups will be formed: one includes Randomized Field Experiments subjects that will watch the discussions on TV As obviously known, laboratory experiments (experimental group), and the other hosts those cannot be used to examine some of the phenomena who will not watch the TV shows (control group). appealing to the interest of the political scientists Then, the research process begins by randomly despite their power to derive causal deductions. Yet, assigning subjects to each group. Prior to the the foundational principles of experimental design introduction of experimental intervention to the can still provide guidance. A field experiment experimental group, multiple measurements (pre- adopts randomization and variable manipulation test) are conducted for both groups. Afterwards, the by applying the techniques of experimental design subjects in the experimental group start watching to naturally-occurring situations and units. While presidential discussions on TV (experiment) where randomly selected individuals or people clusters each candidate partakes. Lastly, in the experimental in this sense receive experimental intervention, and control groups are tested (post-test) for their the others form the control group. Following tendency to support the presidential candidates. the completion of the experiment, the researcher So, the influence that a presidential candidates’ TV conducts post-test measurements to estimate discussion has over voters’ tendency is determined. whether the intervention had an effect or not.

Multiple-Group Design Quasi-Experiments In multiple-group design, more than one Quasi-experimental designs involve similar experimental and control group is formed to processes to those of classical experimental design. compare different levels of experimental variable. The only distinction regards how participants are Use of this design is generally advised when the assigned to groups. Classical experimental design independent variable has more than one value employs randomization to assign participants to or when a researcher wants to see the potential groups whereas quasi-experimental design makes effects of manipulating the independent variable use of non-randomized procedures. To put it in different ways. Multiple-group designs may differently, the researcher utilizes already-existing contain only one post-test as well as a post-test and groups in quasi-experimental design. For example, a pre-test. Again, this design may include a time let’s pretend that we are examining the effect of sequence component. Pretend that a researcher negative propaganda over the intention to vote. aspires to examine the effect of multi-cultural Yet, imagine that we don’t use randomization for social environment over individuals’ tendency for the appointment of participants to experimental multi-culturalism. First, the researcher can form and control groups. In this case, we can stick with three groups (must be more than two) through already-existing groups. For a different example, randomization in order to analyze the influence let’s say there are two sections of students taking

37 Planning Research 2 Introduction to International Relations course. You can select one of the sections as the experimental Population is the group to which research and the other as the control group. Such research results will be relevant. studies are labeled as quasi-experimental since formation of research groups happen without the researcher’s interference. In quantitative research, the aim is to collect information about a population. When it is Intervention Analysis possible to reach every single unit of a population, The dependent variable is subjected to pre- there is no need for sampling. Yet, sampling has test and post-test procedures prior to testing the to be precisely completed if all the units of a independent variable experimentally in this version population is not within the reach of a researcher. of time series design, a.k.a intervention analysis Sample is a representative piece of the population or interrupted time series analysis. Just like the whose features are examined. Referring back to other non-randomized designs, formation of the the above example, there are approximately sixty independent variable is observed but not interfered million voters in Turkey. A research company by the researcher. For instance, poverty threshold eager to identify the distribution of vote rates before and after a political party becomes the to be achieved by all the political parties cannot ruling party can be observed in order to figure out naturally access each and every voter. Nevertheless, if that government’s economic program leads to it is still possible to guesstimate the vote rates for all any changes within the standards of living. In this political parties by examining five thousand voters example, pre-test procedure helps the researcher that share common features with sixty million determine the situation before the dependent voters. Thus, the total amount of the population variable (standards of living) is influenced by the should be calculated, if at all possible, and the independent variable (government’s economy researcher should decide if s/he will reach the entire program). So, post-test assessments to be conducted population or not during the planning stage of a with the dependent variable after the intervention quantitative study since these research efforts aim to can yield more solid conclusions. make generalizations to the population. When the entire population is out of reach for a researcher, the Identification of the Population proper size of a sample can be calculated via some and Sample statistical procedures or by examining the available sources. Finally, after deciding on the size (number Population can be defined as the group to be of participants) of the sample, the researcher associated with the results that will be distilled should consider which probability-based sampling after data analysis. Research studies contain strategy(ies) can be employed. Most often utilized two types of populations: target population and sampling strategies are random sampling, stratified accessible population. Target population is the sampling, cluster sampling, systematic sampling, one that is almost impossible to reach out. On the and convenient sampling. other hand, accessible population is the realistic choice of the researcher, and it is easily attainable (Fraenkel and Wallen, 2006). For instance, the Data Collection target population is the entire number of voters All kinds of numerical values obtained via a in Turkey for a research study held to predict the method, technique, or instrument are accepted vote rate each political party will achieve in the as data in quantitative research (Erkuş, 2011). elections. Accessible population, however, may Primary data collection techniques employed in include only several provinces or counties where quantitative research are tests and questionnaires. a poling company runs some research about the Tests are mainly utilized to measure personality, tendency of the voters living in those areas. It is skills, success, and performance. Since they are only the accessible population that is defined in standardized data collection instruments, tests the scientific reports of research studies, and it is have to be accompanied by reliability and validity referred as research population. endeavors. Furthermore, tests are also used for pre-test and post-test assessments in experimental

38 Research Methods 2 studies. Questionnaire, on the other hand, is a techniques. Primary non-parametric statistical structured interview technique aiming to collect techniques utilized in quantitative research information about more than one characteristic involve chi-square, Mann-Whitney U, Spearman of a person via various item types. Unlike tests, Rank Correlational Coefficient P, and Kruskal questionnaires are designed to obtain information Wallis Variance Analysis. Likewise, some of the about more than one feature of an individual. frequently used parametric statistical techniques Therefore, concepts such as total score, validity, are correlational analysis, regression analysis, and reliability are not applicable for questionnaires. t-Test, one-way variance analysis (ANOVA), and Questionnaires can be applied either individually multi-way analysis of variance (MANCOVA) or as a group. Besides, mail, telephone, and internet (Sönmez and Alacapınar, 2011). There are several are various channels that questionnaires could criteria definitive over which of the parametric or be administered (Erkuş, 2011). In quantitative non-parametric technique a researcher should use research, the researcher should plan how to collect for data analysis. As a ground rule, parametric relevant data before the research process begins. tests are more appropriate if the number of As a general rule, if there is already a test, scale, participants in the sample is over thirty and if the or questionnaire to gather relevant research data, data has a normal distribution. Otherwise, non- one should not develop a new one. The researcher parametric tests should be preferred. In this sense, should contact the scholars who developed the data a researcher managing a quantitative research collection instrument and grant their consent to may not always predict which of the statistical use their tool in her/his research. Yet, researchers techniques mentioned above s/he will use prior to may sometimes need to develop a new instrument data collection during planning stage. Still, s/he to measure specific knowledge, skills, and behaviors should know which of the statistical techniques s/ (Christensen et al., 2014). Providing that a he will use in two-way or multi-way comparisons researcher needs to devise a new data collection will be parametric and non-parametric, and s/he instrument, then s/he should review the literature should also know which tests to employ when the about tests, scales, and questionnaires, should data has a normal distribution and when not. For examine similar studies within the field, and should example, t-Test is a parametric test, and it should be investigate her/his capabilities to conduct necessary utilized in two-way comparisons when the number statistical procedures about data collection tools. of participants in the sample is more than thirty After completing all these steps, legal permissions and when the data has a normal distribution. The should be granted for data collection. hypothesis, for instance, that “there is a difference between democratic tendencies of educated and uneducated people” hosts two groups; “educated people” “uneducated people”. A researcher aspiring Tests are standardized data collection to determine the differences between democratic instruments designed to measure personality, tendencies of these two groups can make use of success, and performance. t-Test if the data has a normal distribution. Yet, Mann Whitney U Test would be more appropriate if the sample size is smaller than thirty and if the Data Analysis and Interpretation data does not have a normal distribution because Analysis of quantitative data is processed non-parametric counterpart of t-Test is Mann through descriptive and inferential statistical Whitney U Test. techniques. Common descriptive analysis Researchers need a standardization to interpret techniques often employed in quantitative research quantitative research data, which is labelled as include frequency, percentage, arithmetic mean, significance level. A significance level (or alpha level) weighted mean, mode, median, range, and standard is a probability level that reflects the maximum risk deviation. Inferential statistical techniques, on you are willing to take that any observed differences the other hand, are of two types: non-parametric are due to chance. It is typically set at .01 or .05. statistical techniques and parametric statistical This means that an extremely low probability value

39 Planning Research 2 will actually be observed 1 out of 100 times (or important 5 out of 100 times) if the null hypothesis is true (Creswell, 2012). During the planning stage, a Instead of expressing a data set statistically quantitative researcher should ascertain which level through numbers, qualitative research aims to of significance (.01 or .05) will be the standard for manage a detailed analysis with an interpretive her/his research. standpoint.

important Unlike quantitative research, qualitative In quantitative research, significance level is research is not composed of structured processes generally accepted as either .01 or .05. that a researcher can follow. Thus, a well-defined starting point or rigid procedures that a researcher should follow are not among the properties of a qualitative research process (Maxwell, 1996). Rather, it entails a flexible approach, and the 1 research process can be re-arranged depending on circumstances emerging during the research How should data analysis process be process. Yet, during the planning stage, researchers planned for quantitative research studies? should 1) form a theoretical framework as the foundational basis, 2) determine a systematic, manageable, and flexible research design, and, 3) PLANNING QUALITATIVE turn the research into a meaningful, consistent, RESEARCH STUDIES and reader-friendly report (Marshall and Rosman, 1999; Yıldırım and Şimşek, 2013). However, these Qualitative research strives to build up a planning stages may sound too abstract for those theory by investigating and understanding social who get in touch with qualitative research for the phenomena in their natural environments (Yıldırım first time. Following procedures should carefully be and Şimşek, 2013). During a qualitative research taken into account during planning a qualitative process, researchers focus on people’s behaviors, research study. actions, and interactions. Research process aims to comprehensively understand human behaviors in their natural environments, which requires Identifying the Research Problem the researcher be flexible, re-arrange the research Similar to quantitative research, identifying process in accordance with obtained data, adopt the research problem is the very first step of a interpretive approach, and conduct data analysis qualitative research process. Literature, researcher’s through an inductive approach. personal and professional experience, and social issues discussed in the field bear a crucial role in terms of identifying the research problem. Literature review includes reading and learning about theories, research studies, and documents, internet which eases problem identification. Personal and professional experiences can serve as fruitful List of online journals about qualitative background knowledge as they are distilled from research: Available at: a researcher’s own life and field of study. They http://www.slu.edu/organizations/qrc/ can facilitate solving important problems via QRjournals.html research. Lastly, the benefits that the research findings promise to the society make the problem

40 Research Methods 2 meaningful. On the other hand, significance and Descriptive research questions are directed feasibility should also be taken into consideration to document individuals’ communications, during planning a qualitative research. Significance conversations, or how they do things on a regular refers to the contributions that a research will bring day. Interpretive research questions, on the other into practice and the solutions that it promises hand, focus on how things work under certain about a problem. Furthermore, feasibility refers to conditions and how individuals perceive specific knowledge and experience on part of the researcher phenomena. Explanatory research questions are and time and opportunities necessary to solve the developed to depict potential relations among research problem. behaviors, situations, and events (Glesne, 2015). During planning stage, researchers should stick Theoretical/Conceptual to an explanatory level and should be careful that the research question aims to describe the focal Framework phenomenon. Theoretical framework eases defining different aspects of the research problem, developing data important collection instruments, and forming themes to be employed during data analysis. Yet, it may In qualitative research, research questions are not always be possible to postulate a theoretical not formulated to include hypotheses and background for all qualitative research endeavors. variables. One of the most important differences between quantitative and qualitative research approaches is the fact that theoretical background which has to be a part of a quantitative research because it is tested by that study may sometimes be absent from a qualitative research. Provided that there is no 2 theory that can explain an event or a phenomenon, What should be considered while writing a new theory can be developed based on the data research questions in a qualitative research? collected through a qualitative research. One of the qualitative research designs, grounded theory (Glaser and Strauss, 1967) is a main design serving to this end. Selecting Research Design Qualitative research designs guide researchers in Developing Research Question(s) terms of determining research focus, data collection instruments, and data analysis techniques. Yet, Unlike quantitative approach, there is no need these designs do not dictate a rigid guidance for hypotheses to develop research question(s) limited with certain boundaries. Qualitative in a qualitative research study (Creswell, 2003). research designs provide researchers with a flexible Likewise, the aim is not to figure out variables and approach and maintain consistency across different the relations among them. In addition, questions stages of the research process in line with a specific directed to determine specific features such as focal point. On the contrary, research designs cause-effect, relation, and difference are not used in in quantitative research approach are definitive qualitative research. The overall aim of qualitative over research questions, data set, data collection research is to comprehensively understand techniques, data analyses, and how the research perceptions and processes. Research questions will be reported. In fact, research process can vary generally begin with either how or what. A typical in accordance with the focal event, phenomenon, qualitative research question may be as follows: or situation within qualitative research endeavors “What are individuals’ perceptions about refugees?”. (Yıldırım and Şimşek, 2013). Ergo, qualitative During planning a qualitative research study, research studies should be planned with a flexible research questions should be directed to describe, standpoint. Sometimes, the research plan can be interpret, and/or explain the focal phenomenon. reviewed and even re-structured depending on

41 Planning Research 2 emerging conditions during the process. Frequently s/he is going to employ descriptive or interpretive employed research designs in qualitative approach approach. Finally, s/he should commence the are action research, phenomenology, case study, research process by identifying people who have ethnography, narrative inquiry, and grounded experience with the focal phenomenon. theory. Below are principles that researchers eager to utilize these designs should follow. important

Action Research Participants have to have first-hand experience with the focal phenomenon for a researcher to Action research is a cyclical and spiral process conduct a phenomenological study. that is conducive to development and change, that encompasses individuals’ own practice, that entails systematic data collection, that enables reflective questioning, and as a result of these, that provides Case Study opportunities to plan and apply new action. For Case study is another qualitative research design instance, Let’s say that members of a political party that can be used to obtain in-depth information have low levels of participation in the democratic about a current issue or to verify and explain processes within that party. In this regard, action effectiveness of a theory across various contexts. plan can help determine the obstacles preventing There are two basic types: single case and multiple- members from participating in the democratic case studies (Yin, 2014). For instance, pretend processes within the party and increase members’ that a think-tank offers leadership training. participation into decision making processes by This case can be accepted as an ongoing practice eliminating those obstacles through planning independent from the researcher and free from relevant actions. Action plans and implementation her/his interference. If the researcher wants to continues until participation is achieved. During describe and explain how this think-tank provides drafting an action plan design, a researcher should leadership training, case study would be an keep in mind that action plans to be administered appropriate design. While planning a case study, should be dynamic and flexible. In other words, the researcher should first decide what the case a researcher should know that s/he may have to is. Research topics examined through qualitative review, and revise action plans depending on their research approach are not formed by the researcher. success and develop new action plans from scratch. Therefore, a researcher to employ case study design Moreover, s/he should consider if s/he will include should depict what the ongoing and current case is. or exclude unforeseen issues pertinent to the Then, s/he should choose a suitable case study type research question and emerging during the process. (single or multiple) in accordance with the research question. Phenomenology Phenomenology is a qualitative research Ethnography design focusing on perceptions and meanings that Ethnography is a design rooted in social individuals develop based on their experiences. research tradition, and it analyzes the routines There are two primary approaches: descriptive regulating daily lives of individuals that form and interpretive phenomenology. The most a social group from those individuals’ point of striking feature of phenomenology design is that views. As usual, the researcher should first decide participants have to have first-hand experience with if ethnography is the most appropriate design the focal phenomenon. How refugees fleeing from to examine her/his research question during the the civil war in Syria perceive being a refugee could planning stage. Ethnography design would be be a good example for a phenomenological study. the right choice if the research aim is to explore During planning, the researcher should first think the principles that regulate daily routines such hard about what the focal phenomenon really is as language, religion, tradition, power relations, and explain that. Secondly, s/he should decide if and sovereignty of a specific cultural group. In

42 Research Methods 2 the second step, the researcher should determine Grounded Theory the approach. Though there isn’t only one way Grounded theory is a design mostly employed of administering ethnography research, two to develop a theory that would explain processes, commonly employed approaches in the literature actions, and interactions among people. In are realistic and critical approaches. Thirdly, the grounded theory, the researcher strives to researcher should identify key people who can explore social processes, behavioral patterns, and help him integrate into the cultural group, and acceptance among participants, and the meanings should seek ways to get involved with the group they form. For instance, grounded theory can be s/he wants to study. Finally, the researcher should utilized to develop a theory about the experiences be equipped with some strategies to distinguish of children in societies going through civil wars. her/his own knowledge from those s/he learns In this sense, the researcher interviews children about the cultural characteristics of the group s/he living in a country where a civil war is at large, is examining during the research process because and tries to devise a theory that would explain war ethnography research aims to describe how the with a perspective sensitive to their context and target cultural group evaluates the research topic standpoints. A researcher eager to use grounded rather than evaluating, judging, and labeling the theory design should meticulously work during the cultural group from the researcher’s standpoint. planning stage. First, it should be kept in mind that grounded theory design aims to develop a theory Narrative Inquiry about the issue a researcher wants to study. Thus, the researcher should, in a way, prove that there is Narrative inquiry can be defined as meaningful a need for a new theory in the literature. Second, and chronological presentation of human there are two approaches in grounded theory experiences after purposeful and systematic analysis. research; classical and constructivist approaches. In other words, narrative inquiry is the attempt The researcher should decide with which of these to select, arrange, relate, and evaluate events or approaches her/his study will be in line since phenomena experienced by individuals in a way that each of these approaches requires the researcher is meaningful for readers. For example, narrative adopt different roles during different processes inquiry would be an appropriate choice to focus on ranging from literature review to data analysis and experiences of an individual (or individuals) who building a theory. Third, the researcher should has worked for a non-governmental organization determine if s/he is going to conduct constant (NGO) for many years and to explain what it is comparison analysis in accordance with classical like to be an NGO volunteer via the stories that or constructivist approach. Finally, the researcher individual recounts. Likewise, lives of political should also plan if s/he is going to test the theory leaders, governors of countries, and activists who to be developed at the end of the study. Classical have major influence over international policies can approach suggests that the theory developed at the be examined via narrative inquiry, which focuses end of the research be tested whereas constructivist on stories about individuals’ real-life experiences. approach underlines that a theory is specific to its Thus, the researcher should decide if the focus will context and, hence, testing is against to the nature be on life experiences of a single person or different of qualitative research. recounts of the same event experienced by different people. Next, the type of narrative inquiry should be Due to their flexible nature, qualitative research considered; biographical study, auto-ethnography, studies deny a strict and rigid planning process. memoirs, or oral history. Thirdly, narrative inquiry Knowing that planning processes can vary across researcher should determine how to collect data different contexts, the steps outlined above for about not only the participants’ narrations but also each design should be considered as guidelines. about the social, cultural, and historical context Researchers intending to analyze their topic of that those narrations took place. Lastly, s/he should interest via a qualitative research design should define strategies about how to build recounts to be question their proficiency about qualitative research shared with readers. designs so that they can manage the planning process at its best. Simply, a researcher should have

43 Planning Research 2 good command of knowledge about qualitative for those who worked in the same and different research approach and relevant designs because each regions of the world respectively. During the of these designs serve different research questions. planning stage of qualitative research studies, the Again, each design differs in terms of approaches, researcher should roughly know the features of the researcher roles, data collection techniques, and people, institutions, documents, and data sources/ analysis processes. Providing that researchers have he will work with. Yet, there may be changes in adequate amount of knowledge and skills about data sources and participants depending on the qualitative research designs, they can proceed with emerging conditions during the process. the guidelines about how to put each design into practice, make necessary changes, or develop their Data Collection own designs. Observation, interview, and documents are main data collection techniques in qualitative research approach. In line with the research 3 aim, one or two techniques could be employed What should be considered during the simultaneously in order to describe a phenomenon planning stage of qualitative research or case as comprehensively as possible (McMillan, designs? 2004). Use of more than one technique is quite common in qualitative research. Researchers expand or eliminate the limitations of a technique Selecting the Participants via use of another one. Administering more than one data collection technique for one study is Purposeful sampling strategies are employed known as data triangulation, which improves both for participant selection in qualitative research. validity and credibility of the findings concluded So, the difference between quantitative and by a research (Yıldırım and Şimşek, 2013). qualitative approaches is best pointed out by Moreover, it is also possible to employ quantitative the sampling methods these two approaches data collection techniques such as questionnaires use. While qualitative research studies utilize a and scales in qualitative approach. For instance, rather small-scale sample outlined purposefully, a questionnaire and a scale could accompany a quantitative endeavors mostly employ probability- case study or an action research respectively. Yet, based sampling strategies (Patton, 2002). Primary interview and observation are the most frequent purposeful sampling strategies used in qualitative techniques found in qualitative studies. In research are extreme or deviant case sampling, accordance with above explanations, a researcher maximum variation sampling, homogeneous should decide which data collection technique is sampling, typical case sampling, critical incident the most relevant and has the priority during the sampling, snowball or chain sampling, criterion planning stage. Furthermore, s/he should question sampling, confirming and disconfirming case the capability of the primary data collection sampling, and convenient sampling (Yıldırım and technique of her/his choice in terms of answering Şimşek, 2013). For instance, pretend that a research her/his research question, and s/he should consider focuses on perception of identity and culture by the employing an additional quantitative or qualitative children of diplomats, soldiers, attaches, and NGO technique. volunteers who have worked for international institutions for a long time. In such a case, the researcher may make use of different sampling important techniques. For example, s/he may know someone with international work experience. Other people Simultaneous use of at least two techniques with the same experience can be reached out by (among observation, interview, and snowball or chain sampling starting from the documents) is called data triangulation in first person. Likewise, homogeneous sampling qualitative research. and maximum variation sampling may be used

44 Research Methods 2 Data Analysis, Reporting Findings, defined, and presented in a reader-friendly manner. and Interpretation In this stage, the researcher excludes her/his own ideas and opinions and presents the collected Literature presents several approaches that information to the reader in the processed form. qualitative researchers can follow during data During the interpretation of findings, on the analysis. Some of these approaches include thematic other hand, the researcher’s opinions accompany analysis, narrative analysis, discourse analysis, the findings that are analyzed and presented in- semiotic analysis, descriptive analysis, and content depth, and conclusions are drawn. In qualitative analysis (Green et all., 2007; Liamputtong, 2009). research efforts, the researcher becomes a natural However, there is no formula that can guide use part of data collection process, and her/his ideas of these techniques. Unlike quantitative research, about the research topic matters considerably. Yet, no statistical process is run to determine the these ideas should be compatible with the collected significance of the data set. No direct test is applied data. The researcher’s opinions and interpretations for validity and credibility. Nevertheless, qualitative play a crucial role over the explanation and researchers try to contextualize and understand a contextualization of research data within considerable size of data. They strive to diminish qualitative approach. In this sense, the researcher is the volume of raw data, distinguish relevant from expected to explain the relations among findings, irrelevant, identify the important patterns, and to establish cause-effect relations, and underline the convey the gist of the data set (Patton, 2002). significance of results via drawing conclusions from the findings so as to set a frame of meaning for the important data set during the interpretation of findings. With respect to data analysis in qualitative research, Unlike quantitative research, formulas are not a good command of knowledge about relevant used for data analysis in qualitative research. literature and field-specific experience of the Instead, the researcher tries to unfold the researcher are as important as her/his knowledge meanings and patterns embedded in the data and experience about content analysis, thematic set through her/his perspective. analysis, and discourse analysis. In planning the data analysis process for a qualitative research, first thing to do is to determine an analysis technique Of course, there are some principles that can that is compatible with primarily the research guide researchers in analyzing research data, questions and the body of data. Again, in this reporting findings, and interpreting. These stage, whether the research data will be converted principles are 1) coding the data, 2) forming the into digital values and if the analysis procedure will themes, 3) arranging and identifying the data be conducted through a software or not should also in accordance with codes and themes, and 4) be considered carefully. Secondly, the researcher reporting the findings. During data coding, the should decide if s/he will employ deductive or researcher divides the data into meaningful groups inductive approaches (inductive approach may after thorough examination and tries to figure out not be suitable for all kinds of research questions, what each division means conceptually. Divisions designs, or analysis techniques) for the data analysis forming a meaningful unit are named-coded by technique of her/his choice. Deductive approach the researcher. Following the coding process, these means that the researcher will analyze the data set codes are used to build patterns, in other words in accordance with a coding frame already reported themes, that can explain the data set in general and in the literature. Inductive analysis, on the other gather the codes under certain categories. Similar hand, points out that the researcher will build a codes are put together to form the themes. To put coding structure of her/his own based on the data it differently, commonalities are distilled from the set. Thirdly, the researcher should also plan if s/he codes. While arranging the data set in accordance will adopt a descriptive or interpretive approach with codes and themes, the data are explained, during the presentation of data set to the readers.

45 Planning Research 2 Ethical Principles and researcher should be able to differentiate between Researcher’s Role the research data and her/his own assumptions and presuppositions. In addition, the researcher should In qualitative research, researchers should think hard about how her/his own position could comply with some ethical principles so that potentially influence the research data across all the rights and privacy of participants can be the stages. Speaking ethically, during the planning kept confidential. In this regard, researchers are stage, a researcher should first pay attention to responsible for informing the participants about her/his ethical position compatible with either the research aim and methods clearly and correctly. qualitative or quantitative paradigms to be Researchers do not only satisfy feeling of curiosity employed during the research. If the researcher is but also they are entitled to a scientific degree or designing a quantitative research, then s/he should title with the studies they conduct. Qualitative be ready for a more objective and rigid ethical researchers should also care for the participants’ framework. If the plan is to hold a qualitative benefits and interests. The participants should be research, then s/he should take ethical dilemmas convinced to participate in the study. Furthermore, and contextual ethical issue that may come up into they should also be provided with details about account (see Chapter 7). the potential risks which may vary from legal responsibilities to being excluded by others, confronting political pressures, or to taking part Limiting the Results and Making in other phases of the research, if necessary. How Analytical Generalizations confidentiality will be established should also be Quantitative research studies need to generalize explained to the participants. In this regard, answers the results they produce. On the contrary, should be provided to clarify if the researcher will qualitative research does not accept generalization change and conceal names, places, and other details as an aim, rather aspires to conduct in-depth and and where and how long the data will be stored? comprehensive analysis of an event, phenomenon, Besides, participants should be directly informed case, or culture within a limited frame. The reason about who will have access to the data set and why why qualitative research avoids generalizations can the data set will be used (Patton, 2002). be attributed to the nature of social events since Quantitative research advocates that truth will social events are not dormant but unstable. Again, objectively surface. Thus, the researcher takes an social events may vary depending on the context outsider’s glance to the events and phenomena. they are experienced. Because it is not possible to re- In other words, the researcher looks for ways to experience a social event with the same contextual minimize his involvement to the process during and temporal outlines, trying to generalize the quantitative studies. In that, there is not even a title results of a social research to other social events will such as researcher’s role in quantificational approach not be the correct approach. In addition, it is also since it is recorded as a weakness if the researcher difficult to include enough number of individuals mentions about her/himself and her/his roles as a into the sample to represent the entire population, researcher. Yet, qualitative understanding defends which is another rationale to avoid generalizations. that truth (knowledge) is subjective, changeable, Although direct generalization is not possible, and pluralist because qualitative approach is rooted still limited generalizations can be drawn based in interpretive tradition. Therefore, the researcher on the results of a qualitative research. In doing is considered as a natural part of data collection so, the researcher should meticulously consider process, and s/he adopts a participatory role during the changing nature of social events, the frame the research. The researcher’s own observations of the research, and its limitations (Yıldırım and and interpretations are directly influential over Şimşek, 2013). This approach concerning the the results. That is why the researcher should generalizability of research results is known as supply information as to her/his position during analytical generalization. the entire research process since a participatory

46 Research Methods 2 important important

Qualitative research does not aim to make Simultaneous use of quantitative and generalizations because it works with limited qualitative data does not necessarily turn number of participants in a comprehensive every research into a mixed method research. manner and focal social events have a dynamic and changeable nature. Before planning a mixed research study, any researcher enthusiastic to use this method should PLANNING MIXED METHOD question her/his capabilities in terms of qualitative RESEARCH STUDIES and quantitative research skills and using a Various names that mixed method research multiple perspective in a single research because goes by in the literature are integration, synthesis, this competence bears a crucial and definitive role quantitative and qualitative methods, multiple with respect to planning and administering a mixed methods, and mixed methodology (Byrman, 2006; method research. If a researcher is content with Tashakkori & Teddlie, 2010). Mixed method is her/his competence to conduct such a study, then a research approach where both quantitative and following paragraphs can serve as guidelines for qualitative data sets are gathered and integrated so planning mixed method research studies (Creswell, as to find answers to research questions (Creswell, 2015): 2014). Mixed method approach accommodates philosophical assumptions of both quantitative and The Title qualitative approaches. Thus, it is more meaningful The title of a well-developed mixed method than merely collecting and analyzing quantitative research should include the research subject and and qualitative data. Besides, it offers a more solid sample. Besides, the phrase mixed method should research structure than quantitative or qualitative also be used in the title to signal the employed approach does since it entails simultaneous use of research method. Likewise, some vocabulary items both approaches (Creswell and Plano Clark, 2007). preferred in either qualitative (such as exploration, Many scholars employ mixed method so that they understanding, etc.) or quantitative (such as can synthesize and relate quantitative and qualitative relation, correlation, explanation, etc.) research data sets (Tashakkori and Teddlie, 1998) and lay titles should not be chosen for the wording of a the foundation where quantitative and qualitative title for a mixed method research. Since mixed data sets can support each other (Creswell and method stands in a zone of neutrality between Plano Clark, 2007). Furthermore, a researcher may qualitative and quantitative approaches, due simply utilize mixed method if s/he thinks that attention should be paid to formulate a neutral either quantitative or qualitative approach will be title. A title compatible with the above criteria inadequate to answer the research questions effectively could be as follows: “Analyzing Freedom of Press on its own (Creswell, 2009). Mixed method is not (Research Subject) across Developing Countries merely combining quantitative and qualitative data (Research Sample): A Mixed Method Research sets. In this approach, gathered data sets are analyzed (underlying the mixed method)” separately and findings are interpreted collectively after collecting both types of data simultaneously. Identifying the Research Problem While explaining the research problem, one should avoid merely listing relevant studies in the internet literature. Even though citing studies conducted on Journal of Mixed Methods Research Available a similar or same research problem is inevitable and at: crucial, researchers should also write about what http://journals.sagepub.com/home/mmr should be done. They should come up with more reasons than simply saying “Since there is a need

47 Planning Research 2 in the literature…” or “the gap in the literature”. important These expressions may serve as rationales to explain the need for some research problems, yet the Mixed method research questions could be research problem should be investigated in terms developed via either method-centered or of the difficulties/issues experienced within daily content-centered approaches. But, the best life. Therefore, the gaps in the literature and real mixed method research questions are those life issues should be considered at the same time that reflect both the method and content. when explaining the research problem for a mixed Such research questions are also known as method study. hybrid mixed method research questions.

Identifying Research Aim and Questions The best mixed method questions are those In general, the sentences denoting the aim of whose characteristics reflect both the method and a mixed method research are comprehensive and content. Such research questions are also known as long. Mixed method research questions should be hybrid mixed method research questions. A good developed after stating the aim. A mixed method example may be as follows: “What conclusions can research includes (i) hypothesis or quantitative be drawn from the comparison between findings research questions (ii) qualitative research questions, concerning quantitative data obtained from and (iii) mixed method research questions. As democratic living tendency scale and others distilled you know, previous sections have been dedicated from qualitatively collected data directed towards to explaining how hypothesis or quantitative and again democratic living tendency?” In this example, qualitative research questions should be written and data type (scale data and qualitative data) and what their characteristics are. Therefore, only the approaches concerning the focal concept of the features of mixed method research questions will research (democratic living tendency gathered by be examined in this part. While planning mixed either scale or interviews) are easily discernable. method research studies, research questions should be written in a way that transcends quantitative and qualitative methods and requires use of mixed method. In other words, mixed method research internet questions should not emphasize only quantitative Quality & Quantity: International Journal of or qualitative research approaches but they Methodology Available at: should combine both methods. Writing research https://www.springer.com/political+science/ questions in this way is also known as method- journal/11135 centered approach. However, writing research questions in this way entails a certain command of knowledge about mixed method designs. Selecting Mixed Method Research Accordingly, a researcher should develop her/his Design research questions compatible with the research design s/he will employ. The second approach to Different designs are employed to explain how writing mixed method research questions is labeled the research process is built in a mixed research as content-centered approach. A relevant example method. These designs include The Convergent could be “How do grown-up women’s opinions Design, The Explanatory Sequential Design, The about democracy support their attitudes about non- Exploratory Sequential Design, The Intervention governmental organizations ?” In this example, Design, The Social Justice Design,and The Multistage “opinions about democracy” composes the Evaluation Design (Creswell, 2015). Each of these qualitative aspect whereas “attitudes about NGOs” designs suggests specific steps to follow during data forms the quantitative aspect. collection, analysis, and interpretation concerning either quantitative or qualitative research approach

48 Research Methods 2 employed in a research. Accordingly, following the researcher should first formulate quantitative paragraphs provide details about the features and then qualitative research questions. Secondly, of these designs and how these features can be the researcher should plan collect and analyze reflected during the planning stage (Creswell, quantitative data, and s/he should leave it to the 2002, 2009, 2015). third step to collect and analyze qualitative data. Finally, the researcher should consider what kind The Convergent Design of integration strategies s/he will use to explain how qualitative data backs up the quantitative data This design aims to integrate the results obtained to readers. from both qualitative and quantitative data analyses. Researchers adopting this design collect both qualitative and quantitative data simultaneously. In The Exploratory Sequential Design other words, collection of neither qualitative nor This design shares many commonalities with quantitative data is prioritized. Data is collected the Explanatory Sequential Design. For example, and analyzed separately. Later, the results produced it takes two stages to complete this design, too. after data analyses are combined. So, the researcher However, unlike the Explanatory Sequential can check whether quantitative and qualitative Design, qualitative data is collected and analyzed results confirm each other. The researcher using during the first stage of research process. In the this design should consider first writing qualitative second stage, quantitative data is gathered and and quantitative research questions independently examined. Hence, qualitative aspect matters more during the planning stage. The order of these for this design. Both types of data are united and qualitative and quantitative questions—in other integrated during interpretation. This design aims words, which is developed first—does not matter. using quantitative data and findings to support Secondly, the researcher should plan collect and qualitative data interpretations. Different from the analyze relevant data separately. Thirdly, s/he should Explanatory Sequential Design, the Exploratory decide how to combine and integrate the qualitative Sequential Design is more about exploring and quantitative data sets. Data integration within the phenomenon rather than explaining and the Convergent Design entails presentation of interpreting it. Therefore, any researcher eager to results comparatively after analyzing quantitative use this design should write the qualitative research and qualitative sets. questions in the first stage and quantitative research questions in the second stage. In the third important stage, the researcher should most importantly plan how to use the qualitative results to structure the quantitative phase together with drafting how the In Convergent Design, qualitative qualitative data will be collected and analyzed. In and quantitative data sets are collected the fourth stage, the researcher should develop a simultaneously but separately. Whether the quantitative data collection tool and plan data results confirm each other is monitored collection and analysis procedures. during the integration phase.

important The Explanatory Sequential Design

This design can best be defined as supporting In the Exploratory Sequential Design, quantitative data collection and analysis processes qualitative data is collected in the first stage with qualitative data collection and analysis while quantitative data is gathered in the procedures. In this design, the weight of research second because quantitative data is utilized to is mostly positioned on quantitative data because interpret qualitative findings. this design aims to make use of qualitative data during interpretation and explanation of quantitative data. With respect to planning stage,

49 Planning Research 2 The Intervention Design The Social Justice Design This design entails examining a research problem The aim of this design is to examine the research by adding qualitative data into the experimental problem within a social justice structure. Social intervention. The researcher adopting this design justice structures that researchers enthusiastic to forms an experimental group in which procedures employ this design include feminist theory, critical concerning the independent variable whose effect race theory, queer theory, disability theory, or a is under analysis are carried out. This stage is the combination of these. The Social Justice Design quantitative phase of the research procedure. aspires to transform lives of marginalized groups Qualitative data, on the other hand, can be and disadvantageous individuals and to help them added to the procedure before, during, and after with the research results. During planning stage, the experimental intervention. Qualitative data the primary task of a researcher eager to utilize the could be used to select the subjects to participate Social Justice Design is to choose a design (The in the research, to decide about the structure of Convergent Design, The Explanatory Sequential the experimental intervention, to determine how Design, or The Exploratory Sequential Design) the subjects will join the activities within the that will answer the research question most experimental intervention, to monitor the results, effectively. After choosing one of these designs, and to provide more detailed explanation than s/he, of course, should be careful about design- statistical results can on their own. Any researcher specific planning procedures. Then, s/he should aspiring to employ this design first should choose decide which social justice standpoint (feminist one of the experimental designs compatible with theory, critical race theory, queer theory, disability the research aim (explained under Planning theory, etc.) s/he will adopt for the research, and s/ Quantitative Research Studies). Again, in this stage, he should plan how to incorporate the standpoint s/he should determine how to set up experimental of her/his choice with the research. and control groups, how to assign subjects to each group, which pre-test and post-test procedures to The Multistage Evaluation Design be employed, and the administration process of experimental intervention. Then, the researcher The Multistage Evaluation Design is a mixed should decide how to use the qualitative data method research design where the success of some set during the implementation of experimental activities or a program is evaluated in time. The intervention. In this regard, s/he could plan to strength of the Multistage Evaluation Design comes employ the qualitative data before, during, or from systematically documenting the success of after the experimental procedure or during each of a program. Planning of this design may require a these phases. Lastly, s/he should find integration project team. Accordingly, the researcher or the strategies to combine experimental results with project manager should first decide about either qualitative ones. the program whose success will be evaluated or the research team that will conduct the evaluation process. Secondly, which design(s) will be employed important for evaluation should be identified (The Convergent Design, The Explanatory Sequential Design, or In the Intervention Design, quantitative The Exploratory Sequential Design). Since needs aspect outweighs the qualitative one. analysis is a major starting point for program Experimental procedure is applied during the evaluation efforts, the correct choice could be The quantitative phase. Qualitative data, on the Exploratory Sequential Design. Thirdly, how many other hand, is used to enhance and assess the stages there will be during the evaluation should research process. be noted together with what procedures will be applied in each stage. For instance, the research might be composed of four stages. So, the stages may be ordered like qualitative, quantitative,

50 Research Methods 2 quantitative, and qualitative. Depending on the Data Collection type of these stages, the research team should Due to the fact that mixed method research plan how to complete the tasks in each stage entails using both quantitative and qualitative such as literature review, developing and applying approaches, data collection should be planned in quantitative or qualitative data collection tool, two stages. Accordingly, qualitative data collection administering experimental intervention, and etc. tools such as interview, observation, and document In the last stage, the team should evaluate the analysis and quantitative tools such as questionnaire, program and think about suggestions to improve test, and scales can easily be employed for mixed the program, if necessary. method research. Yet, the mixed method design determined in accordance with the research questions dictates which data collection technique will be employed and in which order. For example, 4 the Exploratory Sequential Design requires that qualitative data be collected first. The difficulty at What should be considered during the this point is to decide how to use the first collected planning stage of mixed method research data in the second stage. For instance, qualitative designs? data collected in the first stage could be utilized to develop a quantitative scale in the second stage (Creswell, 2014). Sample Because quantitative and qualitative approaches Data Analysis, Interpretation, and are simultaneously employed in a mixed method Reporting the Results research, the plan should be to use probability and Because mixed method design makes use purposeful sampling techniques for the quantitative of both quantitative and qualitative research and qualitative aspects respectively. The best one methods simultaneously, previous explanations that could be utilized for the quantitative aspect as to how to conduct data analysis applies here as is random sampling. Besides, simple random well. However, the order of data analysis in mixed sampling, stratified sampling, or multistage cluster method research should be the same as that of sampling can also be appropriate depending on data collection, i.e. if qualitative data is collected the research aim (Creswell, 2012). Of course, the first, then it should be analyzed first. Remember, size of the sample should also be carefully taken data analysis for qualitative and quantitative data into account for the quantitative aspect. The most sets should be conducted and presented separately. significant principle to comply with is to work with Statistical analysis to be employed for quantitative the largest sample possible because the odds to data analysis (percentage, frequency, t-test, Mann make a mistake diminishes for large scale samples. Whitney U, Anova, etc.) should be explained. As for the qualitative aspect, purposeful sampling Moreover, ideas should be generated as to how strategies will be more appropriate. In this sense, to code the qualitative data and convert it into maximum variation sampling, critical incident themes. Again, examples should be provided for sampling, snowball sampling, and confirming codes and themes emerging during qualitative data and disconfirming case sampling could be listed analysis procedures. After analyzing and presenting among the options. In qualitative research, the size qualitative and quantitative data sets separately, of a sample is determined when the research data how to merge these sets in the second stage achieves saturation. Therefore, setting a size for should be planned. In mixed method research, the sample at the beginning will not be the correct integration task can be carried out during data approach. If data collected from new participants collection and data analysis, after experimental start repeating and do not produce new codes, intervention, within discussion part or by using categories, or themes, the situation is explained tables and graphics. During data collection, with data saturation.

51 Planning Research 2 integration could be achieved by using both open Diagrams Used in Mixed Method and closed-end questions in a survey. As for data Research analysis, the researcher could first collect and Several diagrams are employed to clarify the analyze quantitative data, then obtain qualitative procedures and underline the significant points data that will help explain quantitative data, and within mixed method research. These diagrams could report the findings accordingly. Researchers display data collection, data analysis, and can do the integration task after experimental interpretation of results. Accordingly, how symbol interventions. The researcher could collect signals can be utilized is as follows (Creswell, 2015): qualitative data after the experimental intervention, and s/he could report the qualitative findings • Abbreviations such as QUAN and QUAL regarding the experimental intervention. The indicate that either a quantitative or researcher could compare and contrast quantitative a qualitative method is dominant and and qualitative data within the discussion part of prioritized in that mixed method research. her/his report. Lastly, tables and figures could be If these two approaches are evenly employed used for integration. For this type of integration, within a mixed method research, then both quantitative and qualitative findings are arranged of the upper-case initials are included. next to each other and presented. These flexible • Lower case initials such as Quan and Qual integration approaches offered by mixed method represent the less dominant or recessive research should be in the researcher’s inventory, approach. and s/he should choose the most suitable one in • Plus (+) points out the integration of accordance with her/his capabilities. However, collected data, and it is also used to mark the type of integration also differs across various quantitative and qualitative data sets are designs. For instance, first quantitative and then collected simultaneously. qualitative data sets are reported in the Explanatory • Arrow “→” shows that data is collected Sequential Design. Yet, this design calls for a consecutively. When this symbol is used, third type of interpretation. After the researcher the researcher bases the approach (e.g. presents both quantitative and qualitative data sets qualitative) s/he will utilize in the second in general, s/he should underline how qualitative stage onto the one in the first stage (e.g. findings expand and explain quantitative findings quantitative). (Creswell, 2015). • Parentheses “( )” mean that there will be an addition to some parts of the research, and square brackets “[ ]” symbolize that another independent study has been embedded into the current research.

52 Research Methods 2

LO 1 Define concepts used in scientific research studies

Some basic concepts are prerequisites to be able plan scientific research studies. Scientific research is a systematic data collection and analysis process conducted to figure out a solution for a problem. Scientific research studies are categorized into three: quantitative, qualitative, and mixed method. The take off point for each of these methods is problem identification. Quantitative research problem aims to examine the relation between two or more variables.

Qualitative and mixed method research problems are directed to investigate an event, Summary phenomenon, perception, or process via description, explanation, and/or interpretation. The concept of variable is specific to quantitative approach and it refers to properties that can take at least two different values such as age, gender, academic success, and income level. These variables are labeled as continuous and discrete variables. Likewise, they are grouped into three in accordance with their control types: dependent variable, independent variable, and control variable. Another concept utilized in quantitative research is hypothesis, which can be explained as an educated guess about the result of a problem or a temporary solution to the problem. Research design refers to planning relevant and necessary conditions to collect and analyze research data in accordance with research aim and as economically as possible. Population can be defined as the group that the results of a quantitative research will be valid for.Sample is the group that bears the same properties as those of the population. Since qualitative studies do not aim to make generalizations, population is not used. Instead, workgroup or participants are utilized. Research studies collect data from the sample or the participants in the workgroup. Data is the name for all kinds of qualitative and quantitative features obtained via a method, technique, or tool. Data analysis is the systematic way that researchers make use of to conclude a specific result by contextualizing the research data.

53 Planning Research 2

LO 2 Explain quantitative, qualitative, and mixed method designs

Surveys and experimental designs are employed for quantitative research approach. In survey model, individuals’ attitudes, behaviors, and opinions are described as they are. These models could be either sectional or longitudinal. In experimental designs, independent variable’s effect over dependent variable is tested. Experimental designs include The Classical Randomized Experimental Design, Post-test Design, Repeated-Measurement Design, Multiple-Group Design, Randomized Field Experiments, and Quasi-Experiments. Qualitative research hosts action research, phenomenology, case study, ethnography, narrative inquiry, and grounded theory. Action research is an investigation process directed to establish change and improvement. Phenomenology is a design examining meanings and perceptions distilled from experience. Case study tries to gather comprehensive information about an ongoing case or to confirm an already-established theory. Ethnography is a social research tradition studying the daily lives of a target social group Summary through their own perspective. Narrative inquiry scrutinizes human experience purposefully and systematically, and presents them in a meaningful and chronological order. Grounded theory is another qualitative research design mostly used to develop a theory that could explain the processes, actions, and interactions among people. Mixed method research studies involve The Convergent Design, The Explanatory Sequential Design, The Exploratory Sequential Design, The Intervention Design, The Social Justice Design, and The Multistage Evaluation Design. In the Convergent Design, qualitative and quantitative data sets are collected simultaneously but independently and analyzed separately. In the Explanatory Sequential Design, collection and analysis of quantitative data is supported with collection and analysis of qualitative data. In the Exploratory Sequential Design, first qualitative, and then quantitative data sets are gathered. In the Intervention Design, an experimental intervention is expanded, enriched, and assessed with qualitative data. The Social Justice Design aims to transform the lives of marginalized and disadvantageous people and help them by employing patterns such as feminist theory, critical race theory, queer theory, and disability theory across the Convergent Design, the Explanatory Sequential Design, or the Exploratory Sequential Design. The Multistage Evaluation Design is utilized to evaluate the success of a program or activities held in a certain setting in time.

54 Research Methods 2

Plan quantitative, qualitative, and mixed method studies in a LO 3 stage by stage fashion

Two basic research designs are employed for the planning of quantitative studies, survey and experimental designs. Since quantitative research aims to generalize the results to the population, the researcher should determine the size of the population and decide if s/he will reach out to the entire population. If the sum of population is not within the researcher’s reach, then some statistical procedures should be administered to define the size of research

sample (the number of participants). After determining the sample size, s/he should plan Summary which one(s) of probability-based sampling techniques will be used. Primary data collection techniques in quantitative research approach are tests, scales, and questionnaires. As a ground rule, a new tool should not be developed if there is already a test, scale, or questionnaire to collect data. However, researchers can sometimes devise new instruments to gather data about a specific knowledge, skill, and behavior. Analysis of quantitative data should be planned through descriptive and inferential statistical techniques. Following data analysis, a quantitative researcher should determine if s/he will accept .01 or .05 as the standard for data interpretation Research designs that can be utilized in qualitative research studies are action research, phenomenology, case study, ethnography, narrative inquiry, and grounded theory. In qualitative research, which purposeful sampling strategy will be used to identify the participants (extreme or deviant sampling, maximum variation sampling, homogenous sampling, typical case sampling) and why it will be used should be planned. Major data collection techniques often employed for qualitative studies include observation, interview, and documents. Thematic analysis, narrative analysis, discourse analysis, semiotic analysis, descriptive analysis, and content analysis are among the available approaches for qualitative data analysis. While planning qualitative data analysis process, the first thing to do is to determine an analysis technique that is compatible with the research question(s) and data size. Secondly, the researcher should decide if s/he will adopt a deductive or inductive approach when using the analysis technique of her/his own choice (inductive approach may not be suitable for all kinds of research questions and designs). In addition, a qualitative researcher should also be prepared and planned in terms of ethical principles that will protect the rights and confidentiality of the participants. Planning of a mixed method research begins with the selection of a suitable title that includes the term mixed method in it. Designs that are used in mixed method research are the Convergent Design, the Explanatory Sequential Design, The Exploratory Sequential Design, The Intervention Design, The Social Justice Design, and The Multistage Evaluation Design. Because quantitative and qualitative approaches are run simultaneously in a mixed method research, a researcher should plan to use probability based and purposeful sampling strategies for the quantitative and qualitative aspects respectively. In this regard, interview, observation, and documents utilized in qualitative research and questionnaire, test, and scales employed in quantitative research can serve as data collection instruments in a mixed method research. Following the analysis and presentation of both quantitative and qualitative data sets separately in a mixed method research, the researcher should plan how to integrate them. Using some figures, symbols, and diagrams to clarify the procedures and significant points in a mixed method research can facilitate the understanding of the research process by readers.

55 Planning Research 2

1 Which of the following theories/paradigms 5 Which of the following is a planning explains the features of quantitative research? principle for ethnography research? a. Interpretive paradigm a. Explaining individuals’ shared experiences b. Critical theory b. Determining strategies that would reflect the c. Positivist paradigm standpoint of the cultural group d. Post-positivist paradigm c. Simulating situations similar to the cultural e. Feminist theory situation in question d. Choosing new samples to test the research result 2 Which of the following is a planning principle e. Making action plans for the cultural for the Classical Randomized Experimental development of cultural components Design?

Test Yourself Test a. Random assignment to groups 6 Which of the following sampling strategies is b. Conducting only post-test measurements for not employed when planning qualitative studies? both experimental and control groups a. Homogeneous sampling c. Forming more than one experimental and b. Systematic sampling control groups c. Snowball sampling d. Employing purposeful sampling technique for d. Maximum variation sampling the selection of experimental and control groups e. Critical incident sampling e. Identifying the stages of the research process for the control group 7 Which of the following is not a principle that a qualitative researcher should be careful about? 3 Which of the following is not a planning principle for quantitative data analysis and a. Adopting an objective and neutral approach interpretation? b. Protecting participants’ confidentiality a. Examining normal distribution of the data c. Explaining the research aim to the participants b. Determining the level of significance to be used d. Determining how the research data will be during interpretation stored c. Determining the statistical techniques to be e. Informing participants about potential risks used during dual and multiple comparisons d. Establishing coding frame 8 Which of the following is a characteristic of e. Determining purposeful descriptive statistical the Convergent Design? techniques a. Dominance of quantitative aspect b. Collection of qualitative data at the beginning 4 Which of the following is a well-developed of the research process qualitative research question? c. Simultaneous collection of qualitative and a. Is there a relationship between gender and quantitative data sets participating in peaceful demonstrations? d. Administering experimental intervention where b. There is no difference between high and low the effect of independent variable is tested income individuals in terms of joining non- e. Determining the social justice pattern that governmental organizations. guides the research c. There is a significant difference between developed and developing countries in terms of satisfaction with life. d. Is there a difference between under-developed and developed countries in terms of military expenses? e. How is members’ participation in administrative processes encouraged within non-governmental organizations?

56 Research Methods 2

9 Which of the following is not a planning 10 Which of the following is not true for a study principle for data analysis process in mixed method with the symbols “QUAN→qual”? research? a. Research starts with quantitative data collection a. Determining statistical techniques to be b. Research employs the Explanatory Sequential

employed for the analysis of quantitative data Design Test Yourself b. Determining how to integrate quantitative and c. Qualitative aspect is less important qualitative data sets d. Data has been collected simultaneously c. Considering the compatibility between research e. Qualitative data is used to explain quantitative design and integration approach data d. Planning how to analyze qualitative data e. Determining how to utilize the data set obtained in the first stage within the second stage

57 Planning Research 2

If your answer is incorrect, review the 1. c If your answer is incorrect, review the section 6. b section “Selecting Participants” in Planning “Planning Quantitative Research Studies”. Qualitative Research Studies.

2. a If your answer is incorrect, review the section 7. a If your answer is incorrect, review the section “The Classical Randomized Experimental “Ethical Principles and Researcher’s Role” in Design”. Planning Qualitative Research Studies.

3. d If your answer is incorrect, review the 8. c If your answer is incorrect, review the section section “Data Analysis and Interpretation in “Planning Mixed Method Research Studies”. Planning Quantitative Research Studies”.

If your answer is incorrect, review the 4. e If your answer is incorrect, review the section 9. e section “Data Analysis, Interpretation and “Developing Research Question(s)” in Integration of Results” in Planning Mixed Planning Qualitative Research Studies. Method Research Studies.

5. b If your answer is incorrect, review the section 10. d If your answer is incorrect, review the section “Ethnography”. “Diagrams in Mixed Method Research”. Answer Key for “Test Yourself” for “Test Answer Key

How should data analysis process be planned for quantitative research studies?

Descriptive and inferential statistical techniques are employed for quantitative data analysis. Inferential statistical techniques are categorized into two as non- parametric and parametric statistical techniques. There are several criteria defining whether a research will utilize a parametric or non-parametric technique. As a ground rule, parametric tests are administered if the sample size is larger than 30 and the data displays a normal distribution. Otherwise, your turn 1 non-parametric tests are run. In this sense, during planning stage, a researcher using quantitative research approach may not always determine which statistical techniques s/he will make use of in advance. However, s/he should know which of the statistical techniques s/he will employ for dual or multi comparisons will be parametric and which will be non-parametric, and s/he should appropriately choose a parametric or non-parametric test when the data has a normal distribution and when does not. Lastly, while planning, the researcher should decide if s/he will comply with .01 or .05 for standardization during data interpretation. Suggested answers for “Your turn” Suggested answers for “Your

58 Research Methods 2

What should be considered while writing research questions in a qualitative research? Suggested answers for “Your turn” Hypotheses should not be developed while writing research questions for a qualitative study. Again, questions should not be directed to identify variables and the relation among them. Plus, relations, differences and cause-effect relations should not be questioned through research questions. In general, research questions are formulated with the words how and what in qualitative studies. While planning a qualitative research, research questions should aim your turn 2 to describe, interpret and explain the focal phenomenon. Descriptive research questions are formulated to understand processes and conversations among people, or the way how people complete some tasks within their daily lives. Interpretive research questions focus on how events evolve and how individuals contextualize certain phenomena under certain circumstances. Explanatory research questions aim to unravel the potential relations among behaviors, situations, and events. Researchers should pay due attention to stay on an explanatory level when developing their research questions and to formulate them in a way that describes the focal phenomenon.

What should be considered during the planning stage of qualitative research designs?

Qualitative research designs include action research, phenomenology, case study, ethnography, narrative inquiry, and grounded theory. In action research, one should consider that action plans can be reviewed and revised depending on their success, that action plans can be constructed again from scratch, and whether s/he will include or exclude previously unforeseen issues that emerge during the process. In phenomenology, the researcher should first your turn 3 think about what the phenomenon is, and should explain that. Then, s/he should decide if s/he will employ descriptive or interpretive phenomenology and how to identify those who have experienced the phenomenon. In case study, the researcher should first determine what the case is and then the type of case study design (single case or multiple cases) depending on the research questions. In ethnography, the researcher should first ascertain if ethnography is the most appropriate design for her/his study. What should be decided in the second step is to use either realistic or critical ethnography, who will be the key person to join the cultural group, and what are the ways to get involved with the target culture. In narrative inquiry, first it should be defined if the focus will be the recount of experience by one person or the recount of the same experience by more than one person. Then, the type of narrative inquiry should be established. Furthermore, it should also be settled how to gather information about the social, cultural, and historical context that the experiences took place. In grounded theory, the first thing to do is to asses if there is a need for a new theory in the field. Then, it should be stated which grounded theory approach—classical or constructivist—will be adopted. Finally, the researcher has to plan whether to test the newly developed theory or not.

59 Planning Research 2

What should be considered during the planning stage of mixed method research designs?

In mixed method research, the research process is planned in accordance with the Convergent Design, the Explanatory Sequential Design, the Exploratory your turn 4 Sequential Design, the Intervention Design, the Social Justice Design, or the Multistage Evaluation Design. While planning the Convergent Design, quantitative and qualitative research questions should be written separately. Then, independent data collection and analysis procedures should be outlined. Next, quantitative and qualitative data sets should be merged and integrated. During the planning stage for the Explanatory Sequential Design, first quantitative and then qualitative research questions should be developed in a graded fashion. Afterwards, quantitative data set should be collected and analyzed before gathering and analyzing qualitative data set so as to explain the quantitative results. Lastly, integration strategies should be chosen to explain how qualitative data set explains the quantitative one to readers. With respect to the Exploratory Sequential Design, first qualitative and then quantitative research questions should be developed. Next, qualitative data collection and analysis should be planned, and most importantly, it should be clearly drafted how qualitative results will be utilized to design the quantitative stage. In the final step, quantitative data collection should be devised, and data should be collected and analyzed. The first thing to do during planning the Intervention Design is to decide which experimental design will be used. Again in this stage, (in accordance with the characteristics of the quantitative design) how to form the control and experimental groups, how to assign participants to each group, which pre-test and post-test procedure(s) will be utilized, and the steps to administer the experimental intervention should all be drafted and planned. Then, qualitative data collection during the experimental intervention should be outlined. Finally, strategies should be determined to integrate Suggested answers for “Your turn” Suggested answers for “Your qualitative results with experimental results. As regards the planning of the Social Justice Design, first thing to do is to choose the design (The Convergent Design, The Explanatory Sequential Design, or The Exploratory Sequential Design) that will best answer the research question(s). Next, the social justice perspective (feminist theory, critical race theory, queer theory, disability theory etc.) to be adopted for the research should be identified, and how to incorporate the research with that perspective should be clearly outlined. The Multistage Evaluation Design should be planned by a team. In this sense, the first thing that the researcher or the project manager will do is to determine the program to be evaluated and the team that will conduct the evaluation process. Next, the evaluation design(s) should be defined among the Convergent Design, the Explanatory Sequential Design, and The Exploratory Sequential Design. Afterwards, the number of steps that compose the evaluation process should be drafted, and it should be clearly determined what procedures will be conducted in which step. Lastly, the researcher should think about the evaluation process and suggestions to improve the program.

Further Reading

Kittel, B., Luhan, W. J., & Morton, R. B. (2012). Druckman, J. N., Green, D. P., Kulinski, J. H., & Experimental political science: Principles and Lupia, A. (2006). The growth and development practices. Hampshire: Palgrave Macmillan. of experimental research in political science, American Political Science Review, 100(4), 627- Pennings, P., Keman, H., & Kleinnijenhuis, J. 635. (2006). Doing research in political science (2nd ed.). London: Sage. Shively, W. P. (2011). The craft of political research. (9th ed.). New York: Routledge. Barakso, M., Sabet, D. M., & Schaffner, B. F. (2015). Understanding political science research methods: Klotz, A., & Prakash, D. (2009). Qualitative methods The challenge of inference.New York: Routledge. in international relations: A pluralist guide. Basingstoke: Palgrave Macmillan. Baglione, L. A. (2015). Writing a research paper in political science: A practical guide to inquiry, Box-Steffensmeier, J. M., Brady, H. E., & Collier, D. structure, and methods. (3rd ed.). Los Angeles: Sage (Eds.) (2009). The Oxford handbook of political & CQ Press. methodology. Oxford: Oxford University Press.

60 Research Methods 2

References

Altunışık, R., Coşkun, R., Bayraktaroğlu, S., & Glesne, C. (2015). Becoming qualitative researchers: An Yıldırım, E. (2005). Sosyal bilimlerde araştırma introduction (5th ed.). Boston: Pearson. yöntemleri: SPSS uygulamalı (4th ed.). Sakarya: Green J, Willis K, Hughes E, Small R, Welch N, Sakarya Kitapevi. Gibbs, L. et al., (2007). Generating best evidence Byrman, A. (2006). Mixed methods: A four-volume set. from qualitative research: The role of data analysis. Thousands Oaks, CA: Sage. Aust N Z J Public Health, 31(6), 545-50. Creswell, J. W. (2002). Research design: Qualitative, Johnson, J. B., Reynolds, H. T., & Mycoff, J. D. quantitative, and mixed methods approaches. (2016). Political science research methods (8th ed.). Thousand Oaks, CA: Sage. London: Sage. Creswell, J. W. (2003). Research design: Qualitative, Karasar, N. (1994). Bilimsel araştırma yöntemi (6th ed.). quantitative, and mixed method approaches (2nd Ankara: 3A Araştırma Eğitim Danışmanlık Ltd. ed.). Thousand Oaks: Sage. Liamputtong P. (2009). Qualitative research methods Creswell, J. W. (2009). Research design: Qualitative, (3rd ed.). Melbourne: Oxford University Press. quantitative, and mixed methods approaches (3rd Marshall, C. & Rossman, G. B. (1999). Designing ed.). Los Angeles: Sage. qualitative research (3rd ed.). Thousand Oaks: Sage. Creswell, J. W. (2012). Educational research: planning, Maxwell, J. A. (1996). Qualitative research design: An conducting, and evaluating quantitative and interpretive approach. Thousands Oaks: Sage. qualitative research (4th ed). Pearson: Boston. McMillan, J. H. (2004). Educational research: Creswell, J. W. (2014). Research design: Qualitative, Fundamentals for the consumer (4th Ed.). Boston: quantitative, and mixed methods approaches (4th Pearson Education, Inc. Edition). Thousand Oaks: Sage. Patton, M. Q. (2002). Qualitative research & evaluation Creswell, J. W. (2015). A concise introduction to mixed methods (3rd ed.). Thousand Oaks: Sage. methods research. Thousand Oaks: Sage. Sönmez, V., & Alacapınar, F. G. (2011). Creswell, J. W., & Plano-Clark, V. (2007). Designing Örneklendirilmiş bilimsel araştırma yöntemleri. and conducting mixed methods research. Thousand Ankara: Anı. Oaks, CA: Sage. Tashakkori, A., & Teddlie, C. (1998). Mixed Christensen, L. B., Johnson, R. B., & Turner, L. A. methodology: Combining qualitative and (2014). Research methods, design, and analysis quantitative approaches. London: Sage. (12th Ed.). Pearson: Boston. Tashakkori, A., & Teddlie, C. (Eds.). (2010). SAGE Erkuş, A. (2011). Davranış bilimleri için bilimsel handbook of mixed methods in social and behavioral araştırma süreci (3rd ed.). Ankara: Seçkin. research (2nd ed.). Thousand Oaks, CA: Sage. Fraenkel, R. J., & Wallen, N. E. (2006). How to Yin, R. K. (2014). Case study research: Designs and design and evaluate research in education (6th ed.). methods (5th ed.). LA: Sage. Boston: McGraw-Hill Int. Ed. Yıldırım, A., & Şimşek, H. (2013). Sosyal bilimlerde nitel Glaser, B. G., & Strauss, A. L. (1967). The discovery of araştırma yöntemleri (9. Basım). Ankara: Seçkin. grounded theory. Chicago: Aldine.

61 Chapter 3 Writing a Literature Review

After completing this chapter, you will be able to:

Understand how researchers choose the Understand the types, steps, and purposes of 1 research topic. 2 the literature review.

Identify the sources and types of information Organize the literature systematically while 3 and their values. 4 reading and writing it. Learning Outcomes

Chapter Outline Key Terms Introduction • Literature Research Topic • Literature review Literature Review • Information sources Sources of Information for Literature Search • Literature search • Organization of the literature • Literature review errors

62 Research Methods 3 INTRODUCTION examination of a topic is possible only if the research topic is limited. However, if the research When students are expected to read academic topic is extensively limited, then it is difficult for and scholarly works during their college education, the researcher to find sources. At the same time, they come across many new concepts, a lot of the researcher may have difficulty in locating information, and different types of research. the research topic in a position within the whole It is very easy to get lost in this unfamiliar of the subject, and sometimes the research may processes unless they have a research education become meaningless. One of the important factors or a professional support. This chapter aims to in limiting the research topic is the time that the present some useful information to help students researcher can allocate for the research. Parallel to find their own ways that can guide them in their the increase in the amount of work that can be literature search, review, and writing process. It devoted the research, the researcher will be able to cannot provide a step by step formula but it can expand the scope he is working on. The researcher help develop ideas as to where to start and conduct who struggles to limit the issue of research should a research project by yourselves. Before getting first identify a “problem statement” that sets out the into how the sources for literature review can be purpose of the research. The “problem statement” searched and accessed, researchers need to find should be simple, clear, and questionable, as well and specify a research topic. For this reason, this as being restrictive. What the researcher will search chapter starts with the research topic and research for, in other words, which idea will be defended, questions. The aim of this chapter is to undertand will find its expression in the problem. The problem the literature review process as a part of research statement tells the truth that the researcher will methods and suggest useful ways and instruments defend in her/his research, test the truth, or reveal to go about doing it. the cause-effect relations. If a researcher finds the opportunity to go into the subject and evaluate a RESEARCH TOPIC large number of sources, the problem statement There is no particular strategy for students and may change (Dinler, 2016, 36). researchers to choose a research topic but there The research question guides the structure of are three well-known reasons that researchers seek the whole literature review process providing a after. They are; experience, theory testing, and concentration on what kind of writing is required replication of previous research. Past experiences to address the research topic and questions. A of professionals can be a strong inspiration for good research question should be “doable” within researchers. Researchers are mostly interested a specific time period (Aveyard, 2010, 24). When in negative experiences for the purpose of the research question is decided, it can be written improving practices and making them better for in either interrogative or declarative form (Aveyard, next generations. Theories can be used to develop 2010, 24). For example, an interrogative form can be research studies. Even a single study can be the stated as “What factors have changed international focus of a research to examine its implications. relations after the World War II”, while a declarative The replication studies refer to repeating a past form of that question can be worded as “An analysis research recognizing its methodological weaknesses of the reasons behind changes in international to do a better one. A researcher repeats a study relations after the World War II”. Aveyard (2010, carefully so that the previous limitations will not 31) suggests that a good research question should be repeated (Lodico, Spaulding and Voegtle, 2006, be clear and specific. It should be focused but not 43). Preparatory reading allows the researcher to too narrow. If the research question is too broad, it become aware of the research topic. Keeping the cannot be handled by the literature reviewer. The research topic too wide can cause the work to following examples may give an idea of focusing and remain superficial, to interfere with one another, narrowing a research question: The question “why not to know what is being explained, and finally do states go to war?” is too broad that cannot be fail achieving a reliable result. An in-depth totally answered. On the other hand, the question

63 Writing a Literature Review 3 “Why did Japan and the United States go to war in 1941?” is a more focused and manageable question Literature for a new researcher. To sum up, research questions Literature can be defined as published study such as should be focused, manageable, and answerable from articles, books, dissertations, research reports, etc. the available literature (Aveyard, 2010, 41).

Research Questions Research questions may arise from various sources. For example, one can get a research question from a discourse with colleagues or from their works and writing. Students may not be excellent source of emerging research questions but their interesting questions generally activate new paths to explore (Anderson and Arsenault, 2005, 33).

important

Anderson and Arsenault defines six types of knowledge (2005, 50): 1. Historical knowledge. In different fields, researchers use particular lenses considered appropriate for the time, and their research is dominated by this perspective. Besides, numerous topics keep their popularity within a specific period of date until another era starts with different frameworks and understandings. When considering the literature, it is important to keep its history in mind in case it evolves over time. Such an approach can keep you from engaging in a route that has been rejected in some past era. 2. Axiological knowledge. It can also be called a “theory of experience”. This kind of knowledge is created by practitioners who share their long-term experiences. Axiological knowledge may include important conclusions and generalizations. This kind of literature knowledge does not mean technicalities but the world of practice. So, the literature reviewer should notice the potential use of the axiological knowledge and avoid using it as a theory or prior research. 3. Theoretical and conceptual knowledge. Theoretical and conceptual knowledge is mostly based on the critical analysis of past research. They serve as a construct for further research questions. In other words, theoretical and conceptual knowledge lead and guide wresearchers to generate contemporary construct based on these early investigations and their formulations. Research “not supported with a theory” is generally suspected in academic circles. Thus, researchers should explore the theoretical ground and conceptual framework of the problem they focus on. 4. Prior research studies. Past research findings are essential starting points for any kind of investigation. Prior research studies have the potential to understand the historical evolution of your research problem. 5. Reviews. As can be understood by its name, reviews are based on the criticism of the important studies in a particular field. Reviews are very helpful to researchers in the sense that they gather a lot of specific pieces of research and cast them into frameworks and levels of generalization which may be more useful than the many other types of studies. 6. Academic debate. New lines of research attempts initiate academic debates in the literature. The academic debates may help a researcher to avoid the directions that have been discussed and rejected in scholarly circles. Familiarizing yourself with these six type of knowledge may help you have a good position to start and move forward with your research topic and problem.

Roselle and Spray (2011, 5) suggest some sources that may be useful to search for your research topic: • Textbooks • Scholarly literature

64 Research Methods 3 • Some scholarly articles that contain listings LITERATURE REVIEW of particular groups of cases (e.g., war, Social scientists define literature as the “published conflict, crises, particular negotiations, etc.). scholarship”. The term literature review is referred • Some scholarly websites which contain the as “the process of reading and reviewing published databases you can explore scholarship in the field” (Roselle and Spray, 2011, • Websites of respected international 16). A literature review is one of the basic parts of organizations such as International student projects, research papers, and dissertations. Monetary Fund, International Labor It empowers us to perceive how new investigations Organization; intergovernmental and research are fairly similar to building squares, organizations such as United Nations and which are laid upon the thoughts worked by others the World Trade Organization. (Oliver, 2012, 4-6). Fink (2014, 3) defines literature • Websites of governmental and non- review as “a systematic, explicit, and reproducible governmental organizations. method for identifying, evaluating, and synthesizing • Websites of government agencies such as the existing body of completed and recorded Ministry of Foreign Affairs, Presidency of work produced by researchers, scholars, and the Republic of Turkey. practitioners”. According to Schryen, Wagner, and • Reference books such as almanacs and Benlian (2015), “looking at literature reviews does books that contain helpful listings of the not only help to avoid reinventing the wheel and related field. thereby marginalizing their own work; it also allows researchers to enhance the body of knowledge by a Defining Variables process of accumulation”. Political scientists define “the circumstances, topics, policies, or other phenomena that they want to understand as dependent variables”. For instance, Literature Review when a researcher likes to investigate the reasons for a failing policy, the dependent variable of the study The selection of available documents (both is “the failure of the policy”. Another study may published and unpublished) on the topic, attempt to find an answer why a specific country has which contains information, ideas, data an suffered from several multiple coups, the dependent evidence written from a particular standpoint variable of the study is “the occurrence of several to fulfil certain aims or express certain views coup attempts in that country”. There exist multiple on the nature of the topic and how it is to be factors that influence the dependent variable of investigated, and the effective evaluation of a study which is called as independent variables. these documents in relation to the research Independent variables may cause some changes being proposed (Hart, 2005, 13). on the dependent variable. The supposition made on the relationship between the dependent and independent variables is called as hypotheses. The A literature review is also considered to be terms dependent variable, independent variable, a research methodology in its own right when it and hypotheses are frequently used in studies is carried out systematically. In other words, if a within international relations, it is important to literature review (Feak and Swales, 2009, 2): understand this terminology and the relationship • Is commenced with a well-defined research between them (Roselle and Spray, 2011, 10). question, • Follows a systematic approach to answering that question, • Clearly documents the steps that are taken 1 to write up, Explain the steps to find a research • Presents the results and discussion section, topic and research questions. it can be taken as a research methodology as well.

65 Writing a Literature Review 3 important not reserved for human subjects (Barad 2003; Latour 2005; Bennet 2010; see Passoth et. al. 2012). IR authors have used the term ‘agency’ “Social scientists refer to published scholarship as in a variety of ways. As Colin Wight (2006, literature. The process of reading and reviewing p. 178) observes, ‘attributions of agency can published scholarship in the field is referred change, not only within theories but also within to as a review of the literature.This phrase the space of a sentence’. Wight concludes that is commonly shortened simply to literature IR scholars are rarely clear about ‘what agency review. When academics talk about conducting is, what it means to exercise agency, or who a literature review, they are talking about and what might do so’ (ibid.). In our view, researching (reviewing) the scholarly literature. this situation is not necessarily the result of a When they talk about writing a literature review, lack of analytical rigor. Instead, the variety of theoretical notions of agency used in IR reflects they are talking about writing a summary of such a variety of practical understandings of agency research” (Roselle and Spray, 2011, 16). in the world. Our analytical interest is in how this practical variety influences international politics. By studying the constitution of agency In early drafts of student assignments, it is fairly as an empirical question, we seek to avoid common to see a literature review which consists attributions of agency on purely conceptual of the mention of a series of writers, each with a grounds, be it human beings, states or other few sentences devoted to them. In the worst cases, actors. Our goal is to understand how certain entities gain the ability to act—in other words, there may be no logical order to the presentation to be active and make a difference—in specific of the writers, and no apparent justification for contexts of international politics. their inclusion, other than that their work is broadly within the subject area. In cases such as this, the reader of the literature review is left to try The Multiple Purposes of A to impose some sort of rational order on what has Literature Review been written. If the reader is an examiner, then it The general purpose of literature review is might not put them in a very good mood!” (Oliver, to narrow down the focus of your research, to 2012, 5). Most researchers concur that a literature focus on it, and to help readers relate it to your writing ought to give a synthesis of the literature research. One of the main aspects of literature as obligatory commitment. Besides, a literature review is the decision about which literature review is considered to go beyond the summary of should be included and which literature should the relevant literature by assessing and interpreting be excluded. When writing a research report or a the research and theoretical knowledge (Schryen journal article, there will naturally be a word limit. et al, 2015). For instance, Braun, Schindler, and It is, therefore, necessary to decide which authors Wille (2017, 3) make a synthesis of the concept should be included and how much space should be in their paper in which they focus on “agency in allocated to each (Oliver, 2012, 4). Ridley (2012, international relations”: 43) expresses the purpose of a literature search as There is no agreement in social theory on follows: how agency should be conceptualized. For some authors, an agency is closely tied to 1. To identify the field and specific context in the idea of a human subject with a history, which the topic is located. intentions, and the capacity to make his or her 2. To decide which research and methodology own decisions. In this view, ‘a capacity for an will be adopted, which type of data may be agency—for desiring, for forming intentions, collected and analyzed. and for acting creatively—is inherent in all 3. To deepen your knowledge related to the humans’ (Sewell 1992, 20; see also Emirbayer and Mische 1998). Others have tried to move key concepts of the research topic beyond this anthropocentric understanding by 4. To avoid repeating previous studies that conceiving of agency instead as the mere quality have ben completed before your attempt of being active or vibrant—a quality that is and to produce an original work.

66 Research Methods 3 5. To avoid the mistakes that past studies important reported. 6. To discover the key people, organizations, While writing a literature review, it is critical to and papers related to your research topic. present and discuss the theories and concepts which Knopf (2006) expresses the benefits of a you plan to guide your research direction and data literature review as follows: discussion. Your own particular translation of these hypotheses and ideas ought to be clarified. 1. It can introduce a general overview of the This should be possible by first looking into how previous studies which you do not know others have analyzed and characterized these about. theories and concepts, and then by expressing your 2. It can keep away the reviewer from own particular position (Ridley, 2012, 30). “reinventing the wheel” so you won’t waste your time by repeating the similar efforts. 3. It can inspire your research by giving new A literature review can be performed in seven ideas. steps (Fink, 2014, 3): 4. It can enable you to figure out where 1. Identification of research questions. This there are issues or imperfections in current step guides the review. 2. Selecting the databases and sources. literature. There are numerous bibliographic or article 5. It can help you extend your research to databases, websites, and other resources inspire further studies based on your that can be utilized. research. 3. Specifying the key terms. Key terms are Murray states that in your literature review you the words that are used to search and get should give an account of work that has been done the related documents. before. Such a literature review should answer the 4. Applying practical screening criteria. following questions (Murray, 2006, 110): Literature reviewer can set the criteria to 1. Who has worked related to your research include or exclude the documents from the topic? review among the numerous articles. 5. Applying methodological screening 2. Who has researched your research topic? criteria. Methodological criteria incorporate 3. Who thinks your subject is worth criteria for assessing scientific quality. researching? 6. Doing the review. To ensure the reliability 4. How do you relate your subject with the and validity of the literature review, a previous studies? standardized form should be used for By answering the questions above, you are abstracting data from papers. Besides, expected to demonstrate the “gap” in the literature reviewers should be trained for abstracting to prove that your work is worth studying. If you the data and pilot testing of the process can achieve this, your research can contribute to should be reported. your field of study by adding some minor or major 7. Synthesizing the results. Literature review points (Murray, 2006, 110). results may be synthesized expressively. An uncommon sort of synthesis-a meta- analysis involves the utilization of statistical Steps for Conducting a Literature results of two or more research. Review A well-organized literature reflects the analytic Literature reviews can center on the results capacities of the writer. While reviewing the of previous research, research methods, theories, literature on a specific topic, literature reviewer application, or all these. Past scholarly works can be should focus on trends, inconsistencies or similitudes which different writers may have integrated or criticized through literature reviews missed or ignored. If the literature reviewer related to a specific topic (Cooper, 1998, 3). presents examples that different writers differon

67 Writing a Literature Review 3 the interpretation of data, or where they take 4. Focused Literature Review. Primarily, all different approaches to an issue, this can often be literature reviews are focused. This term is highly enlightening (Cooper, 1998, 9). A literature used because a more suitable term is lacking. review is more than a stage to be undertaken. Some Thus, focused literature review defines the of the questions in the review of the literature can reviews that focused on a single aspect of answer (Hart, 2005, 14): the past research such as methodology. 1. What are the key concepts, theories, and ideas related to your topic? Deciding the Keywords Within the 2. What are the epistemological and Subject Area ontological grounds for the discipline The first basic step in searching the relevant (related to your topic)? literature related to your topic is the specification of 3. What are the central questions and the keywords. If the keywords cannot be identified problems that have been addressed to date? carefully and logically, some key readings of the 4. How is knowledge of the research topic literature will be missed. To require a theoretical structured and organized? case, let’s consider a research paper entitled ‘Turkish 5. What are the origins and definitions of the Foreign Policy in a Multipolar World”. Possibly, topic? this review could have the following purposes: 6. What are the political standpoints? • To define the multipolar world. 7. What are the major issues about the topic? • To explore the Turkish foreign policy in the light of recent developments. Types of Literature Review • To analyze the transformation of Turkish Feak and Swales (2009, 2) describe the foreign policy in a multipolar world. most common types of literature review in four From the title and purposes above, several key categories: terms can be identified to search the literature as 1. Narrative Literature Review. Narrative follows: reviews are typically found in theses, • Turkish foreign policy dissertations, grant and research proposals, • Polarity and research articles. In such reviews, the • Multipolarity author selects relevant past research and • Multipolar World synthesizes it into a coherent discussion. • West-East world division Narrative reviews typically are somewhat • New-world order broad in focus compared to other • Axis-shift kinds of literature reviews, discussing After the research topic and keywords are methodologies, findings, and limitations in identified, the process of scanning different data the existing body of work. sources begins. It is possible to simplify your search 2. Systematic Literature Review. In systematic by using the keywords searching techniques. One reviews, a strict methodology is followed of them is Boolean Logic introduced by British in literature selection. While selecting the mathematician George Boole. literature, including or excluding criteria are transparent. Systematic reviews are convenient when the reviewer aims to Boolean Searching illuminate the existing literature and its is also known as keyword searching. Since there are potential implications. different catalog and database systems, there are 3. Meta-Analysis. Meta-analysis is performed variations in search methods. Yet, Boolean Searching by statistical techniques combining data will work in most databases. Even Google uses Boolean from multiple studies that have analyzed logic in its Advanced Search page. It is widely used when the same research questions. Re-analyzing different sets of data related to the same the keywords are organized into search statements and research question provides a much better combined with operators such as AND, NOT and understanding than that of a single study. OR to broaden or narrow your search (http://www. indiana.edu/~liblife/database_searching.pdf).

68 Research Methods 3 You can use these operators with an advanced emphasis on establishing a connection between search option. An example of how the Search literature review and a researcher’s ideas and research Operators function are as follows: questions. For this purpose, she suggests that the • AND. The results will contain all terms literature reviewer seek an answer to the following joined by AND. For example, modernity questions while reviewing the literature: AND democracy would return results that • What were the aims and objectives of the contain both the words modernity AND study? democracy. • In what context the study was carried out? • OR. The results will contain either term(s) • What approaches, methods and techniques joined by OR. For example, modernity OR were used in the study? democracy would return results that contain • What were the outcomes of the study? the terms modernity OR democracy. This • What was the contribution of the study to would increase the results returned when the field? compared to using the AND operator. • Does it have any link to my question? • NOT. The results will not contain the A literature reviewer can likely find a research term(s). For example, modernity NOT question from his/her practice or working democracy would return all results that environment. Abstract questions or those derived contain the word modernity and do NOT from textbooks or textbooks may not have the contain the word democracy. potential to keep your attention alive. So, different Truncation and wildcard symbols. The aspects of near environment or professional life second option to widen your search is the use of of the literature reviewer can be a good starting truncation and wildcard symbols. Most databases point to select a good topic and develop interesting use an asterisk (*) for truncation and a question research questions. Among these, the literature mark (?) as a “wildcard.” However, some databases reviewer should remain objective from the use other symbols. Truncation or wildcard symbols introduction to the conclusion and avoid making can broaden your search and allow you to look for pre-reached determinations in the matter of what variations of words. The wildcard symbol replaces you will discover. Similarly, a literature reviewer a character or characters in a word. It can be used should write the research questions in a neutral with a root word so that all variations of the word style avoiding leading questions that reflect your will be searched for, or within a word so that assumptions about what you are trying to discover alternative spellings will be included. If you search (Aveyard, 2010, 24). The position of a literature “col*r”, the search will include different spellings reviewer is not limited to searching the sources of color. Another example search for “migrat*” and presenting it in a logical order. The literature suggests different variations of the root word such reviewer should point the gap in previous studies as migrate, migrated, migrating and migration underlining the significance of his/her contribution (Ridley, 2012, 57). to the literature. A gap in literature search does Quotes. Another option for managing your not itself offer a justification for a new study. The search is the use of quotes (“ ”) when you want literature reviewer should present good reasons for to specify the phrases and reduce the results. For the importance and worthiness of the study. In example, if you search the term “global governance” other words, the literature reviewer should discuss with a quotation mark, the search engine will only the motivation of the study. match the exact terms excluding the word around Initiating a research involves several the phrase. responsibilities for the literature reviewer such as identifying potential data sources and searching information about the selected topic. In addition Literature Reviewer to these, the literature reviewer has a scholarly A literature reviewer is a researcher at the same responsibility as an original and critical thinker who time. Thus, in this chapter, the word “literature checks out and reflects on all aspects of the literature reviewer” can also be thought of as synonymous with review process and the product (Onwuegbuzie and the word “researcher” that is used very frequently Frels, 2016, 86). The literature reviewer should in the next headings. Murray (2006, 114) puts an develop knowledge within a field ordiscipline as an

69 Writing a Literature Review 3 original thinker. The contribution of the literature reviewer may be a little addition but his/her role should not be limited to repeating the previous literature. The literature reviewer should combine the thoughts uniquely by using his/her imagination and creativity. Besides being an original thinker, the literature reviewer is also a critical thinker who critically evaluates the literature sources during each step (finding, reading and writing) of literature review. Another critical attitude that 2 the literature reviewer should display is reflexivity. Reflexivity refers to the cognitive, emotional or social changes that the literature reviewer Discuss the key features of a experiences during literature review process. In other words, reflexivity well-structured literature review. is expressing what the research process means to the literature reviewer (Onwuegbuzie and Frels, 2016, 86).

SOURCES OF INFORMATION FOR LITERATURE SEARCH

Searching the Literature Each literature research includes a considerable commitment of time and energy. In a digital age, electronic sources increase rapidly. That’s why the literature search increasingly becomes computer based Literature Search an digitilazed. However, the students and researchers should not refers to discovering useful neglect the college libraries for two reasons. First, reference libraries materials that would be included are often the most knowledgeable experts who are often familiar with in or excluded from a research the terms used in databases of professional literature. Besides, they project. can guide students in the use of ever-changing electronic resources (Lodico et al., 2006, 32).

Selecting the Sources One of the most critical issues in a research attempt is the appropriateness and reliability of the information sources. It is essential to reference the primary sources in the search for the literature. However, secondary sources are also useful. Secondary sources are a synthesis of previous primary sources of theoretical and empirical studies. Secondary sources allow a quick look at the recent developments on a specific subject. In these cases, technical information is usually neglected. Primary sources are original research studies or writings. They are written by a theorist, researcher, or historian. In primary sources there is no instrument between the actual research and the reader (Balcı, 2013, 76).

important

There are three types of sources of information: primary, secondary, and tertiary sources. 1. Primary sources. The person presenting the data is actually present in primary sources (Anderson, 2005, 12). Original written works such as poems, diaries, court records, interviews, surveys and original research/fieldwork and research published in scholarly journals (http://libguides.merrimack.edu/research_help/Sources). 2. Secondary sources. When the data is reported from another author or source who is not present at the work cited, it is called secondary data (Anderson, 2005, 12). In other words, secondary sources are those which narrate and examine the primary sources/original research (http://libguides.merrimack.edu/research_help/Sources). 3. Tertiary sources. These kinds of sources organize and locate secondary and primary source. They can be summarized as follows (http://libguides.merrimack.edu/research_help/Sources): • Indexes – provide citations that fully identify a work with information such as author, titles of a book, article, and/or journal, publisher and publication date, volume and issue number and page numbers. • Abstracts – summarize the primary or secondary sources • Databases – are online indexes that usually include abstracts for each primary or secondary resource, and may also include a digital copy of the resource.

70 Research Methods 3 The Internet material can be ranked as reliable, fairly reliable and unreliable in terms of their sources. Table 1 presents this ranking (Wallace and Wray, 2013, 23).

Table 3.1 The Internet Material in terms of Reliability

Likely to be very reliable

1. Peer-reviewed journal articles that are also published in an academic journal. These should be referenced according to their paper details, rather than as an Internet resource. 2. Peer-reviewed journal articles published in genuine electronic journals. These should be referenced using their volume number and date, plus the full web address. It is possible that they will not have page numbers. 3. Already published journal articles and book chapters that have been posted, usually in PDF format, on an academic’s home page. Check, however, that it is the published version. If it says ‘submitted to’ a journal, or ‘draft’, it has yet to be peer-reviewed. You could then check if it has since been published. 4. Electronically readable books written by subject experts. 5. Official materials published on a recognized institutional website, e.g., the British Museum site, or the Institute of Linguists’ site. You can find out what site you are on by going to the homepage.

Likely to be fairly reliable

1. The pre-peer-reviewed material, as described in (3) above but track down the published version if possible. 2. Lecture or research notes on the site of an academic working at a recognized institution.

Likely to be unreliable

1. The material on the home pages of individuals. 2. The material on organization websites that is written by enthusiasts rather than experts. 3. Free-for-all post-your-views sites (unless restricted to a recognized set of academic contributors). 4. Weblogs (blogs), chatroom pontifications, etc.

As can be seen in Table 3.1, Internet materials can be ranked from unreliable to very reliable depending on their sources and the quality of their publication processes. Lodico et al., (2006, 32) state that a good researcher distinguishes the most effective types of sources from the others. Peer-reviewed sources are regarded as the strongest, implying that the published paper has been reviewed by authorities in their fields and appreciated for the quality of the methods adopted in the paper. Thus, the literature reviewer should be careful about the sources they prefer. Academic research will not be limited to one sort of publication or information source, and it should consider the full range of relevant studies. The search options can be identified in the following sub-titles.

Ask an Expert Even in this technological age, the best overall source of information continues to be people. Asking an expert who knows the field can simplify your work by providing you the key sources and unique research information, and this guidance will save a great deal of time (Anderson and Arsenault, 2005, 52).

71 Writing a Literature Review 3 Encyclopedias • education, culture, sport, and tourism The most comprehensible sources of reference • health and social protection are the encyclopedias for the selection of the topic • science, technology and information society as well as the preparation readings after the topic • justice and election selection. In serious encyclopedias, every article • transportation and communication is prepared by authors who made a name in that • economic confidence area, and in some encyclopedias there is also a bibliography on the subject. However, there is Books no doubt that other sources should be consulted Even if books constitute a significant part of after the encyclopedias in scientific studies. the sources, there are disadvantages as well as Encyclopedia publications that started in the west advantages. One of the main advantages of the in the second half of the 18th century were carried books is that the printed material is generally out in Turkey in the late 19th century. Most of our important and high-quality. Besides, books encyclopedias are adaptations of the encyclopedias integrate the other investigations in a way that will published in the west (Dinler, 2016, 57). result in a consistent knowledge base. The main disadvantage of the books is that the material is Statistics not exactly updated because it can take several years between the completion of the work and One of the sources that are appealed in the publication in the book. The best way to do a scientific studies is the statistics. The statistics book search is to look at library catalogs (Kumar, utilized by researchers lose their up-to-dateness 2011, 37). Another book option is the Google as soon as the new statistics are published. In Books that are available of Google Books or on every scientific work, the last statistics published Google.com. If the book is out of copyright or it in that area should be included. If the statistics is permitted by the publisher, it is possible to see on which a scientific research is based are not a preview of the book. The full text is available up-to-date, many theses are rejected by the jury in some cases also. There are also a buying and for this reason. Besides, the research loses a lot borrowing options that are “Buy this book” and of value. In Turkey, State Institute of Statistics “Borrow this book” by clicking on these offers on has been publishing the statistics in all areas from the webpage of the book. E-book formats are also 1962 to the present day on a regular basis. As of purchasable from the Google Play Store. To access November 18, 2005, the name “State Institute of the books or other sources from Google Books, Statistics” was changed into “Turkish Statistical go to Google Books and search the title, author, Institute (Dinler, 2006, 63). Turkish Statistical ISBN or keywords. Institute publishes various statistics covering the five-year general population censuses as well as the other statistics (annual, monthly, quarterly and monthly) on various topics such as: • employment, unemployment, and wage internet • inflation and price You can see the options that are allowed for the • foreign trade book explained below. https://support.google. • industry; trade and services com/websearch/answer/43729?hl=en; https:// www.google.com/googlebooks/about • agriculture; environment and energy • income, living, consumption, and poverty

72 Research Methods 3 The Internet to the researchers. First, researchers may access them If the author of a writing is not known, the anytime and anywhere if they have an internet credibility of the source should be questioned and access. Even if the library is closed, electronic sources it is better not to employ these as an academic are available using personal computers. On the reference. The Internet data should be utilized other hand, the researcher may have some difficulty carefully since the data found virtually on the in distinguishing whether the paper obtained Internet may be incorrect or misleading. electronically is from a scholarly journal. Deciding the scholarliness of a paper is easier when the hard copies of journal is used. Academic journals, newspapers, magazines, and industry publications The Internet are highly regarded scholarly resources. Despite this, is a system that people all around the world political scientists mostly consider only academic can load information without quality and journals to fall into the category of sources most reliability assurance. Thus, internet sources should be carefully selected in academic appropriate for literature reviews. If your professor writings. In an academic writing, the author wants you to utilize from peer-reviewed sources or should cite the references including the scholarly journals, search engines are questionable author and the date of each source. in terms of reliability. The following tips may help you decide the scholarliness of the sources accessed electronically (Roselle and Spray, 2011, 5): Limiting the sites with those such as 1. Level of detail: A scholarly journal government documents and standard reference should provide you with more detail than books of the type one may find in the library the general information contained in a is possibly the safest strategy for the internet newspaper or magazine article. searches. On the other hand, the Internet can 2. Level of discourse: Journal articles are provide less time-consuming opportunity to written in technical language. Nonscholarly access the libraries of colleges and universities resources are written for a general audience. without traveling those. Additionally, the Internet 3. Structure: Most journal articles contain provides a space for a more updated data than the an abstract with the paper’s thesis, an printed sources (Monroe, 2000, 48). When you introduction, a literature review, and a search for a theme, it is usually hard to assess the discussion of the methods used to draw quality of evidence and rely on what you find. conclusions. Sources of information will To avoid this confusion, the Internet can be used be cited. In general, you will not find as a catalog, not as a repository of knowledge. an abstract or a literature review (direct Wikipedia (https://www.wikipedia.org) is one references to other scholarly studies) in a of the very valuable sources that should be used nonscholarly publication. with care. Since Wikipedia is open to anyone to 4. Explanation of methodology: Journal contribute to it, its reliability has been highly articles explain the methods used to analyze discussed. evidence and/or data and indicate which Nowadays, numerous college libraries do not resources were used to draw conclusions. keep the hard copies of journals as in the past. 5. Footnotes and endnotes: Most journal Instead many journal subscriptions are purchased articles that are suitable for literature review in electronic form and can be accessed in electronic in the field of International Relations reviews forms. Online availability provides some advantages contain footnotes or endnotes.

73 Writing a Literature Review 3 6. Bibliography: Scholarly journal articles contain a bibliography or reference list. Magazines and newspapers rarely do. Trade publications seldom contain them. 7. Date of publication: Scholarly journals are usually published monthly, quarterly, or biannually and usually have a volume or issue number coinciding with the time of year released. Newspapers and magazines are usually published daily, weekly, or monthly. Other authors (Lodico et al., 2006, 43) explain additional issues in considering the quality of the information obtained from Web sites. They can be summarised as follows. 1. Authority. The producer, or sponsor of the website should be clear and verifiable. This information can be mostly checked by clicking the button “About Us”. If this information is not available, it is better not too use it. 2. Accuracy. Factual information such as data and names should match those found in almanacs, encyclopedias or other reference books. If the information is not consistent with the prestigious sources, the author should discuss its reasons. If the accuracy of the web source is uncertain, it should not be preferred. 3. Objectivity. The literature reviewer should be careful about the sponsor’s point of view in case of biased expressions. The website should balance different points of views. Otherwise, it should not be taken as a trusted source. 4. Currency. The website should give information about the uploading date of the document, the beginning and the end of the pages. If the document is not dated but include an bibliography, you can examine the references for up-to-dateness. The APA manual allows the n.d. usage for the unknown date of a source. 5. Coverage. The site should fully cover its topic within the limits the authors determine themselves. It is not important whether the coverage is narrow or broad but it is important that the site claim the accurate information.

Internet Search Engines Internet search engines such as Google, Bing, Yahoo, and Yandex makes many sources available directly. Yet, most of the databases are not available without a pay or a subscription via institutions or universities. Google Scholar provides a practical alternative to search scholarly literature across many disciplines and sources from academic publishers, professional societies, online repositories, universities, and other websites. It also shows the alternatives to access the complete document suggesting several sources. See https://scholar.google.com.tr.

Online Databases Some sources and databases related to international relations and/or political sciences are listed below. Some of the websites cover data on general population characteristics, including social and economic indicators. It should be noted that the Internet addresses cited below (Table 3.2) were accessible at the time of this writing and they may have changed.

74 Research Methods 3 Table 3.2

• Some online sources and databases related to international relations • British Library (https://www.bl.uk/#tabcollns) • Catalog of U.S. Government Publications (https://catalog.gpo.gov/F?RN=169571001) • Directory of Open Access Journals (DOAJ) (https://doaj.org) • European Council on Foreign Relations (http://www.ecfr.eu) • European Union Documents and Publications (https://europa.eu/european-union/documents- publications_en) • FEDSTATS (http://www.cs.umd.edu/hcil/govstat/fedstats/fedstats3.htm) • Freedom House (https://freedomhouse.org) • Google Books (https://books.google.com/?hl=en) • Google Scholar (https://scholar.google.com.tr) • High Beam Research (https://www.highbeam.com) • Institute for Foreign Policy Analysis (http://www.ifpa.org) • International Monetary Fund – Data (http://www.imf.org/en/data) • National Library of Turkey - Milli Kütüphane (http://www.mkutup.gov.tr/tr/Sayfalar/english.aspx) • NationMaster (http://www.nationmaster.com) • Open Library (https://openlibrary.org) • Research Journals Published İn Developing Countries (http://www.bioline.org.br) • ScienceDirect (https://www.sciencedirect.com) • Smithsonian Institution Research Information System (https://siris.si.edu) • Stockholm International Peace Research Institute (https://www.sipri.org) • The Center for Security Studies (http://www.css.ethz.ch/en/services.html) • The Home of the U.S. Government’s Open Data (https://www.data.gov) • The Internal Displacement Monitoring Centre (http://www.internal-displacement.org • The Library and Documentation Center of the Turkish Grand National Assembly The Library - Türkiye Büyük Millet Meclisi Kütüphane ve Arşiv Hizmetleri Başkanlığı (https://www.tbmm.gov.tr/kutuphane/ index.htm) • The United Nations Statistics Division (https://unstats.un.org/home) • The Uppsala Conflict Data Program (http://www.pcr.uu.se/research/UCDP) • The World Almanac (http://www.worldalmanac.com/index.aspx) • The World Bank- Research and Publications & Data (http://www.worldbank.org) • Think Thank Search (https://guides.library.harvard.edu/hks/think_tank_search) • Transparency International (https://www.transparency.or) • U.S. Department of Defense Publications (https://www.defense.gov/News/Publications) • United Nations Development Programme (http://www.undp.org) • United States Census Bureau (https://www.census.gov) • United States Department of Labor (https://www.osha.gov) • United States Department of State Publications (https://www.state.gov/r/pa/ei/rls/dos/221.htm) • United States Foreign Affairs Committee (https://foreignaffairs.house.gov)/ • United States Library of Congress (https://www.loc.gov) • US Joint Chiefs of Staff (http://www.jcs.mil/doctrine) • The Vision of Humanity (http://visionofhumanity.org) • World Health Organization (http://www.who.int/en) • WorldCat (http://www.worldcat.org) • 21. Century Turkey Institute (http://www.21yyte.org/en) • Eurostat (http://ec.europa.eu/eurostat/web/main/home) • OECD iLibrary-Organization for Economic Cooperation and Development (http://www.oecd-ilibrary.org) • United Nations Commodity Statistics Trade Database (https://comtrade.un.org/) • UNCTAD Foreign Direct Investment Statistics (http://unctadstat.unctad.org) • World Integrated Trade Solution (http://wits.worldbank.org/WITS)

75 Writing a Literature Review 3 Catalogs Catalogs can be thought in three categories: Library catalogs, national catalogs and publishers’ and booksellers’ catalogs. These are explained below. Library Catalogs. The library catalog enables researchers to search and locate print and online materials (books, journals, and other resources) held in the library. A journal may be available with an online link from the library catalog but some content may not be allowed to access. Instead, the library catalog may direct you to an available link that provides the content you search for (Jesson, Matheson and Lacey, 2011, 36). Library catalog or Online Public Access Catalogues (OPACs) give the bibliographical details and locations of all the publications held within a specific library.Dewey Decimal Classification is utilized by most of the UK libraries whereas the Library of Congress Classification is employed mostly in the USA libraries. In both classification systems, books are categorized as concurring to subject with particular shelf numbers designated. Possibly, the catalog is a computer-based database and it can be entered via university network. Thus, in case you’ve got access to a university computer or a Web association you do not need to be within the library to utilize it for your exploration. They nearly work in a similar way by writing the keywords as subject, title or author (Ridley, 2012, 50). A brief information about these two widely used classifications (Dewey Decimal Classification and the Library of Congress Classification) may be useful. Dewey Decimal Classification. Is also called Dewey Decimal System. It is the best known of all schemes for the classification of documents in libraries, designed by Melvil Dewey in 1873. This classification organizes the contents of a library into 10 groups with each group assigned 100 numbers (https://www. britannica.com/science/Dewey-Decimal-Classification). According to the 100 division of Dewey’s Decimal Classification system, international relation takes place under the category of social sciences as presented at the Table 3.3 (Scharma, 2008, 8).

Table 3.3 International Relations in Dewey’s Decimal Classification Dewey Number Main Classes Ten Division • Statistics • Political Science • 327 International relations • 327.1 General topics of international relations; spies • 327.2 Diplomacy • Economics 300 - 399 Social Sciences • Law • Public administration • Social problems & services • Education • Commerce (Trade) • Customs, etiquette, folklore

As can be seen from the Table 3.3, international relations is numbered with 327. One subdivision of international relations is coded as “General topics of international relations” with the number of 327.1 and the other subdivison “Diplomacy” is numbered with 327.2. The Library of Congress Classification.This classification is especially appropriate for the research universities and is used in some of the libraries in Turkey such as Middle East Technical University, Anadolu University and Hacettepe University (Dinler, 2016, 46-47). It has been developed for the Library of Congress own use. Later, it has been used by numerous libraries and has become the most widely

76 Research Methods 3 used classification system especially by the academic and research libraries in the United States and other countries.t I has been maintained and updated continually by the Library of Congress ever since (Chan, Intner, and Weihs, 2016, 2). The Library of Congress Classification system is represented with alphabet (Table 3.4). The letters I, O, W, X, Y have been excluded but could be used for future classifications (Dittman and Hardy, 2007, 17).

Table 3.4 The Library of Congress Classification Groups

A General Works B Philosophy. Psychology. Religion • J Political Science C Auxiliary Sciences of History • J General Legislative and Executive papers D History (General) and History of Europe • JA Political Science (general) E-F History: America • JC Political Theory G Geography. Anthropology. Recreation • JF-JJ Political insitutions and public H Social Sciences administration - General J Political Science • JK Polical institutions and public adminsitration - K Law United States L Education • JL Political institutions and public adminstration M Music and Books on Music - America outside U.S. N Fine Arts • JN Political Institutions and public administration P Language and Literature – Europe Q Science • JQ Political institutions and public administration R Medicine - Asia, Africa, Australia, Oceania S Agriculture • JS Local government. Municipal government. T Technology • JV Colonies and colonization. Emigration and U Military Science immigration. International migration V Naval Science • JX: International Law Z Bibliography. Library Science. Information • JZ: International relations Resources

Source: Adapted from Dittman and Hardy, 2007.

As can be seen from Table 4, the Library of Congress Classification system arranges materials by subjects. Among 20 categories, “J” indicates Political Sciences under which “JZ” represents International Relations. National Catalog. Many countries have their own national catalogs which list all the publications available in the country An example for Turkey is TO-KAT (Ulusal Toplu Katalog - National Union Catalog).t I is a database that includes all the recorded details of all the books that are available in the libraries in Turkey. The website of TO-KAT is available athttp://www.toplukatalog.gov.tr . Another national catalog for Public Libraries in Turkey is KOHA which is an open source library system that is used worldwide. KOHA is used in Turkey as an automation system that unifies all the libraries that are linked to the General Directorate of Libraries and Publications of Ministry of Culture and Tourism. It is available at (http://koha.ekutuphane.gov.tr). Publishers’ and Booksellers’ Catalogs. Publishers such as Sage, Springer, Routledge, and booksellers such as Amazon (https://www.amazon.co.uk) have their own catalogs that list all the books published. These websites can be searched like the library catalogs by using keywords like author name or subject.

77 Writing a Literature Review 3 Bibliographical Databases The function, content, and structure of bibliographic databases are different. Some are just files, containing foremost essential bibliographic data on papers (Trawick and McEntyre, 2004). The term ‘bibliographic databases’ defines the “abstracting and indexing services for the scholarly literature”. Over the past decade, technological developments expanded the horizons of the bibliographic database from abstracts to longer pieces of texts (Ridley, 2012, 51).

University Repositories University repositories are an ordinarily open-access database of the research and publication productions of an institution. The contributors are typically the academics, researchers, and university students. University repositories can be accessed via the websites of the universities or various registries, and associations which list archives on an overall premise. Directory of Open Access Repositories (OpenDOAR) is among the best known university repositories. Repositories cover archived materials such as theses and dissertations, e-portfolios, manuscripts, research reports, and book chapters. The variety of materials (Table 3.5) which may be saved in a repository will include (Oliver, 2012, 45):

Table 3.5 The variety of materials in a repository

• Experimental records • Family history archives • Datasets • Institutional history archives • Statistical records • Local history collections • Monographs • Patents Musical compositions • Books • Audio recordings • Chapters in books • Video recordings • Journal articles • Conference proceedings • Theses • Output of research societies or centres • Diaries • Specialized collections • Autobiographical accounts • Proceedings of learned societies

Periodicals Periodicals are more widely known as journals. They are the major information source of prior research literature. Periodicals are published a number of times a year and are sent to subscribers including the libraries. The journals which report the research findings are the most significant ones. The quality of journals varies greatly from journal to journal. The most qualified journals are refereed meaning that the publication is evaluated by editors and/or peers expert in their fields in terms of quality assurance. The peer reviews process imply that the reviewers and the author identities are concealed from each other. The acceptance rate of submitted papers is another indicator of the quality of the journal. The best journals accept one or two percent of the papers submitted. The higher citation rates of the journals are another indicator of the journal quality. Researchers are mostly interested in publishing their papers in the most prestigious journals (Anderson and Arsenault, 2005, 55). The list in Table 3.6 covers some leading International Relations journals and describes the scope of the publications.

78 Research Methods 3 Table 3.6 Some Leading International Affairs Journals

Some leading “International Relations” Journals 1. Australian Journal of International Affairs (https://www.tandfonline.com/toc/caji20/current) 2. British Journal of Politics & International Relations (https://onlinelibrary.wiley.com/journal/1467856x) 3. Cambridge Review of International Affairs (https://www.tandfonline.com/toc/ccam20/current) 4. The Chinese Journal of International Politics (https://academic.oup.com/cjip) 5. Ethics & International Affairs (https://www.cambridge.org/core/journals/ethics-and-international- affairs) 6. European Journal of International Relations (http://journals.sagepub.com/home/ejt) 7. International Affairs (https://onlinelibrary.wiley.com/journal/14682346) 8. International Feminist Journal of Politics (https://www.tandfonline.com/toc/rfjp20/current) 9. International Migration (https://onlinelibrary.wiley.com/journal/14682435) 10. International Migration Review (https://onlinelibrary.wiley.com/journal/17477379) 11. International Political Science Review (http://journals.sagepub.com/home/ips) 12. International Political Sociology (https://academic.oup.com/ips) 13. International Politics (https://www.springer.com/political+science/journal/41311) 14. International Relations (IR) http://journals.sagepub.com/home/ire) 15. International Relations of the Asia-Pacific (https://academic.oup.com/irap) 16. International Tax and Public Finance (https://www.springer.com/economics/public+finance/ journal/10797) 17. Journal of International Relations and Development (https://www.springer.com/political+science/ journal/41268) 18. Review of International Political Economy (https://www.tandfonline.com/toc/rrip20/current) 19. Uluslararası İliskiler-International Relations (http://www.uidergisi.com.tr)

The journals in the field of international relations Grey Literature are not limited to the list presented in Table 3. The Searching for grey literature can be tedious since list above includes some selected journals that are it isn’t normally incorporated into bibliographic indexed in Social Science Citation Index (SSCI) databases. which covers some 3000 of the leading and selective journals in the world. One of the main characteristics of the journals indexed in SSCI is that they publish the most influential scientific papers in their fields. Grey Literature SSCI is available online through the http://mjl. covers the products “Grey literature is the clarivate.com/cgi-bin/jrnlst/jlresults.cgi?PC=SS unpublished, non-commercial, hard-to- website for a fee or you can access the publishings find information that organizations such as freely via the subscribed institutions such as professional associations, research institutes, universities. The free papers via the subscribed think tanks, and government departments institutions may be limited to specific years or produce” issues. The scope of the journals referred above and the others that you might search is available on the websites of the journals under the headings such as “scope”, “aims and scope” “about us”, “journal Knowing how to look accurately for grey literature information”, and “description”. Besides the aim may enable you to optimize your time effectively and scope of the journals, other information (http://guides.lib.umich.edu). Typically, the grey (editorial board, subscription, instruction for literature includes reports, theses and dissertations, authors and publication information) take place. conference proceedings, research in progress, leaflets

79 Writing a Literature Review 3 and posters, media reports, patents, letters, and diaries (Ridley, 2012, 45). The Internet is now a major source for dissemination and retrieval of grey literature and often is a good starting point to a topic area. Some grey literature databases are available from the following sources (http://libguides.library.curtin.edu): • Trove • Agency for Healthcare Research and Quality (AHRQ) • MedNar • The Grey Literature Report • OpenGrey • GreyNet International • Cochrane Handbook Paez (2017) presents a more detailed list of sources that can be used to search of the grey literature (Table 3.7).

Table 3.7

1. Grey Literature Databases and Search Engines 1.1. Grey Matters: A Practical Search Tool for Evidence-Based Medicine 1.2. GreyNet, available via INSIT (the Institute for Scientific and Technical Information) through OpenSIGLE 1.3. Grey Literature Report (http:// http://www.greylit.org/home) 1.4. The Healthcare Management Information Consortium (HMIC) database (http://www.ovid.com/site/ catalog/databases/99.jsp) 2. Clinical Trials 2.1. ClinicalTrials.gov A service of the U.S. National Institutes of Health (https://clinicaltrials.gov) 2.2. Cochrane Central Register of Controlled Trials (http://onlinelibrary.wiley.com/cochranelibrary/ search?searchRow.searchOptions.searchProducts=clinicalTrialsDoi) 2.3. World Health Organization (WHO) International Clinical Trials Registry Platform. (http://apps.who. int/trialsearch) 3. Conference Papers 3.1. Conference Alerts (http://www.conferencealerts.com/faq) 3.2. Conference Papers Index, via Proquest (http://www.proquest.com/products-services/cpi-set-c. html) 3.3. Digital Dissertations (http://ipscience.thomsonreuters.com/product/web-of-science) BIOSIS Previews, via Web of Knowledge 3.4. Zetoc (http://zetoc.jisc.ac.uk/) 4. Dissertations, Theses, and Academic Papers 4.1. OpenGrey (http://www.opengrey.eu) 4.2 ProQuest Dissertation & Theses Global (PQDT Global) (http://www.proquest.com/products-services/ cpi-set-c.html) 4.3. WorldCatDissertations (http://www.worldcat.org) 5. Web Searches 5.1. Google Scholar (https://scholar.google.com) 5.2. Mednar (http://mednar.com/mednar/desktop/en/search.html) 6. Other Resources 6.1. The Grey Literature Report (http://www.greylit.org/about) 6.2. The University of Michigan Library, Grey Literature Overview (http://guides.lib.umich.edu/ greyliterature) 6.3 National Institutes of Health (NIH), Office of Research Management, Systematic Reviews: The Literature Search:Gray Literature (http://nihlibrary.campusguides.com/c.php?g= 38332&p=244522)

80 Research Methods 3 Among the grey literature sources, it may extent the students can utilize such a material in be useful to give some details for the websites their academic writings. These platforms cover related to dissertations. ProQuest Dissertations superficial opinions which lack rigor as well as very & Theses Global (mentioned above in 4.2) is the carefully evaluated postings. A significant number largest single repository of graduate dissertations of the issues examined related to blogs also apply and theses. It includes 4 million works from to the utilization of wikis within academic circles. universities in 88 countries. It is accessed by 3000 Wikis are progressively being utilized to empower institutions over 45000 downloads every month. students and scientists to cooperate on shared issues ProQuest offers an extensive and expanding global and to team up in terms of knowledge, research archive of dissertations which has been designated strategies, research literature, and the design of as an official digital repository for the U.S. projects. They empower groups of specialists to Library of Congress. It also provides visibility of add to hyperlinked pages, and to share thoughts breakthrough research from the world’s premier and skills. There are issues about the publication universities control of wikis to guarantee the legitimacy of the material, and this is a factor which ought to be considered before utilizing such archives in an exposition. There have customarily been worries inside scholarly circles about the utilization of internet records, for example, wikis, which empower Proquest can be accessed at numerous clients to add content to a website page. (https://www.proquest.com). There are issues about the publication control of wikis to guarantee the validity of the material, and this is a factor which ought to be considered For Turkey, a National Thesis Center is available before utilizing such records in an academic paper. on the Internet. It covers more than 49000 theses (Oliver, 2012, 48-49). with online full-text access, and these are archieved at the Databases of National Thesis Center of the Journal Databases Council of Higher Education by authors. Full-text journal databases allow online access for particular publications. They are more suitable for searching individually but they are not comprehensive. Different databases will internet complement the missing content. The journal Databases of National Thesis Center of the Council databases cover a broad range of journals but they of Higher Education can be accessed at https://tez. cannot provide all the articles you are interested yok.gov.tr/UlusalTezMerkezi/tarama.jsp. (Jesson et al., 2011, 40).

Ancestry Method Blogs and Wikies Ancestry Method refers to tracing the references Blogs are a prevalent type of correspondence, of a paper like a detective which is relevant to your empowering individuals to express themselves on research topic. The traced references may bring issues, add their sentiments to discussions, critique you new other sources that may be helpful to on the perspectives of others, ask questions, and your exploration. This type of searching may be to exchange knowledge and thoughts. Universities extremely useful in the development of theoretical usually back blogs in different fields, through their and conceptual frameworks (Anderson and virtual learning facilities, to empower the students Arsenault, 2005, 56). to reciprocate their ideas. Besides, academics regularly contribute postings to advance the exchanges. The difficult question here is to what

81 Writing a Literature Review 3 9. The literature review should explain the readers how your work might contribute to Ancestry Method the existing scholarly literature and future This method is to investigate all kinds of research. sources related to research topic as a detective.

Organization of the Literature 3 defines the process of finding the relevant If you were doing a literature review for literature, reading them, and writing up the your master thesis, what kind of literature literature (Feak and Swale, 2009, 5). do you think would be the most reliable?

To organize the literature research, grouping ORGANIZATION OF THE similar works can be a good starting point. LITERATURE REVIEW Organizing the literature search just in your mind can be more complicated than organizing In a literature review, a clear structure must it physically. Specifically, the researcher can make be organized, this structure should be explained more sheets or note cards related to the information and the reason why this order preferred should on hand and then classify them in terms of their be clarified, too (Oliver, 2012, 18). Literature relation to the dependent variable of the research preparation process includes three main steps: topic. An example can be found below (Roselle and finding the relevant literature, reading them and Spray, 2011, 53). writing up the literature review. A good literature review should have the following characteristics If you were using this example research (Feak and Swale, 2009, 5-10): question, “What domestic factors best explain the United States’ decisions to use military intervention 1. Theproblems and controversial issues related to the topic should be discussed. in Somalia’s internal conflict in 1992 but not in 2. The potential content that have not been Rwanda’s in 1994?” you may have found research covered in the literature review should be studies that discuss factors that precipitate external explained with its reasons clearly. intervention in internal conflicts by bordering 3. The literature review should concentrate on states. You may have found studies that explore latest documents since they are likely the factors that precipitate intervention by hegemonic most pertinent. states. You may have also found studies specifically 4. The literature review should be long enough about outside interventions into the internal affairs to persuade the reader that the literature of African states. All of these are related to the same was read extensively. dependent variable of international intervention 5. The gaps in the current literature should be but they represent sub-topics of this dependent revealed in the literature review. variable. Before you begin writing, systematically 6. The literature review should be more than group the articles you have read according to a list of past studies. Rather, it should give these sub-topics. Then, rather than discussing the reader a sense of what has been more each article individually, discuss the articles by significant and less significant. In other grouping. Studies most closely related to your words, the works cited in the literature dependent variable are generally discussed last with review should be critically evaluated. those more generally associated discussed first. 7. Overall, the literature review should be For the example above, the authors suggest presented chronologically. discussing the group of studies on border state 8. The literature review can ignore the studies interventions first because only the studies in this from different disciplines than your immediate discipline. group could be related to the dependent variable.

82 Research Methods 3 The group of studies related to U.S. interventions in Africa is closely associated with the dependent variable of U.S intervention into Somalia, so it should be discussed last. To sum up, while organizing the literature, one can follow this physical grouping strategy. The same grouping strategy may be applied in discussing the literature as well (Roselle and Spray, 2011, 53): 1. First group: Studies broadly related to the dependent variable. 2. Second group: Studies related to the dependent variable. 3. Third Group: Studies most closely related to the dependent variable. After organizing the thoughts in your mind, another organizing process starts with writing the literature. The main instrument in the organization of literature writing is using the headings and subheadings. The sub-division and organizing of a literature review are exceptionally imperative components since the structure provides the reader to internalize the data much more effectively when reading it, and it helps the authors to structure their arguments more effectively. There’s a variety of distinctive ways in which headings are used in organizing the thoughts. They may be utilized to supply a point by point categorization of the literature, or on a more clear level to supply a sequence to the account (Oliver, 2012, 96). Lodico et al., (2006, 56) suggest that a good structured literature review starts with a broad circle around the research topic and moves towards a more focused perspective gradually refining and narrowing it. They suggest a writing tip for the literature reviews that can help you to organize your literature. The literature review starts with a brief introduction that give an idea about the layerings of your research topic and the type of research exist in the literature.

important

You should avoid giving details about the studies you have reviewed and it should be enough to explain their methods and findings. The following steps can be adopted (Lodico et al., 2006, 56): 1. Step one: Prepare summary sheets or article abstracts. Analyze the summary or the abstracts of the studies you collected by marking the key features and motivations behind the studies. At that point, as you read over your rundown, endeavor to verbalize an outline word, theme, or expression that best applies to the article. Write these words, themes,w and expressions on the sheets that you prepared to organize your review. 2. Step two: Review the summary sheets or article abstracts. Compare and contrast the keywords and themes you have found. Put each article under the related key words and then categorize their main focus. Some articles may be placed in more than one category. 3. Step three: Decide how you can organize your literature review. While deciding you can use these questions: • How can you display a big picture of what is known about your research topic? • Which studies are most and least related to your research topic? • Which studies support or conflict with your research topic? • How can you create a meaningful framework for your research topic? 4. Step four: Begin writing your literature review using the analysis you made in the first three steps.

83 Writing a Literature Review 3 Common Errors and Problems in comments which have the potential to Writing a Literature Review develop the theory. The take-home message The inadequate training in conducting a is your point of view expressed clearly in literature review leads students learning by general discussion and abstract sections. themselves, mostly by trial and error method, “The take-home message is the capsule and this in a number of common problems and value of your paper, and everything in the mistakes (Baumeister, 2013, 125-129): paper should refer to it”. 5. Be Critical. Another common mistake is 1. Uncertain Purpose. The purpose of the the lack of criticism. In social sciences, every study should be made clear not simply by study has a limitation and they should be summarizing the past studies or findings. discussed. The criticism can be performed Instead of this, the researcher should under the sub-heading such as literature contribute to the literature by offering a new and findings. Your paper’s value will and developed understanding related to the increase if you can discuss the weaknesses selected topic. For this reason, the literature review should be clearly defined with the and strengths of the evidence you referred. goal of advancing the field’s theoretical or As a researcher/literature reviewer your role conceptual knowledge. is not that of a lawyer who seeks to prove 2. Vague Introduction and Poor the best case for one side of the argument. Organization. Many writers have difficulty Instead, your role is to consider the evidence in following the plan which dictates that skeptically like a judge or jury. Another the introduction should define the goals error related to this point is that, some clearly and then explain the theoretical writers fail to be consistent with what they framework carefully and thoroughly. evidenced and concluded. Your conclusions 3. Not Enough Information. One of the should coincide with the weaknesses and common mistakes is to fail to present the strengths of what you discussed in satisfactory information about the the literature, methods, and findings sub- literature review. Sometimes authors do not headings. give enough information about how they 6. Exceptions and Counterexamples. It reach their conclusions. A well-structured is important to control the results and literature should make a summary of the conclusions of the biases that ordinary methods and findings of the studies cited people bear for selective critique of every rather than the conclusions. You do not day processes. The literature reviewer is need to write multiple paragraphs on every likely to recognize similar tendencies when a reference you cited. But you have to give conclusion goes with prevailing wisdom and adequate information to provide the reader or seems politically correct. The literature to make up their own reasoning whether reviewer should avoid such tendencies as far the arguments make up your conclusions. as possible. The literature reviewer should Sometimes the opposite of this comes to also avoid seeking supportive examples for existence. Presenting excessive details is the research topic. This kind of approach another undesirable situation in literature will cause “confirmation bias”. To overcome reviews. The more you write, the more you this threat, the literature reviewer should risk making an error. spend time for inconsistent evidence. 4. Failing to Connect to Take-Home 7. Tell the Reader Where to Go. A successful Message. The take-home message should literature review should open up a space move the reader beyond the current state for future researchers implying the research of knowledge on a specific topic by your priorities. More specifically, the literature

84 Research Methods 3 reviewer should discuss the strong, weak Some other common literature review mistakes and ambiguous points of the paper to referred in the literature are as follows (Roselle and provide the reader with a focus on these Spray, 2011, 53): issues in the future research. 1. Discussing the studies consecutively 8. Matters of Style. One of the critical issues without making a connection with each in writing a literature review is following other the guidelines and instructions for style. 2. Neglecting to state similar findings of the First responsibility of the researcher is citing past research the author/s and the date of statements 3. Including the articles that are almost in parentheses in the text properly. The irrelevant to the research questions. format of citing should be correct in terms of referencing style to enable readers to make distinctions about different writers’ thoughts and findings. However, the literature reviewer should not let the names 4 to have a principal position in the paper. Discuss the possible steps to be taken to One rule of thumb is “Do not start a develop a well-organized literature review. paragraph with a name!”.

85 Writing a Literature Review 3

LO 1 Understand how the researchers choose the research topic.

There are several strategies students and researchers can employ while choosing a research topic These strategies are; a) past experiences, b) theory testing and c) replication of previous research (Lodico, Spaulding and Voegtle, 2006, 43). Research questions may arise from various sources such as historical knowledge, axiological knowledge, theoretical knowledge, prior research studies, reviews, and academic debate. These knowledge sources can be available from textbooks, scholarly articles, scholarly websites such as international organizations, governmental and non-governmental organizations, reference books. The research question guides the structure of the whole literature review process providing a concentration on what kind of writing is required to address the research topic and questions. Summary

Understand the types, steps, and purposes of the literature LO 2 review.

Literature is defined as the “published scholarship”. Literature review is a process of reading and reviewing published scholarship in the field (Roselle and Spray, 2011, 16). A literature review is one of the basic parts of research papers that helps researchers “to avoid reinventing the wheel” (Schryen, Wagner, and Benlian, 2015). Literature writing is a systematic work that assesses and synthsizes the past theoretical and empirical work. One of the essential aspects of literature reviewing is the choice as to which papers should be included in or excluded from your research project. This choice is critical because every project has a word limit. The literature review helps researchers develop an understanding about the key concepts of the research topic and produce an original work by replicating the previous studies and avoiding mistakes. A literature review can be conducted following seven steps: identification of research questions, selecting the databases and sources, specifying the key terms, applying practical screening criteria, applying methodological screening criteria, doing the review, and synthesizing the results. Literature review can be explained in four categories (Feak and Swales, 2009, 2): Narrative reviews are typically found in theses, dissertations, grant and research proposals, and research articles. Systematic literature reviews are convenient when the reviewer aims to illuminate the existing literature and its potential implications. Meta-analysis is performed by statistical techniques through combining data from multiple studies that have analyzed the same research questions. Focused literature review defines the reviews that focused on a single aspect of the past research such as methodology. In literature review process, literature reviewer has a critical role in connecting the literature review with the research questions. While doing this, the literature reviewer should remain objective from the introduction to the conclusion and avoid making pre-reached determinations in the matter of what you will discover. Similarly, literature reviewer should write the research questions in a neutral style avoiding leading questions that reflect your assumptions about what you are trying to discover (Aveyard, 2010, 24). The position of the literature reviewer is not limited to searching the sources and presenting it in a logical order. The literature reviewer should point the gap in previous studies underlining the significance of his/her contribution to the literature. The literature reviewer should present good reasons for the importance and worthiness of the study.

86 Research Methods 3

Identify the sources and types of information and their LO 3 values.

The search options can be identified as following: Asking an expert, encyclopedias, statistics, books, the internet search engines, online databases, catalogs such as library catalogs, national catalogs, publishers’ and bookellers’ catalogs’, bibliographical databases, university repositories, periodicals, grey literature, blogs and wikies, journal databeses, and ancestry method. One of the most critical issues in a research attempt is the appropriateness and reliability of the Summary information sources. It is essential to reference the primary sources in the search for the literature. However, secondary sources are also useful. Secondary sources are a synthesis of previous primary sources of theoretical and empirical studies. Secondary sources allow a quick look at the recent developments on a specific subject. In these cases, technical information is usually neglected. Primary sources are original research studies or writings. They are written by a theorist, researcher, or historian. In primary sources there is no instrument between the actual research and the reader (Balcı, 2013, 76). The Internet material can be ranked as reliable, fairly reliable and unreliable in terms of their sources. Peer-reviewed sources are regarded as the strongest, implying that the published paper has been reviewed by authorities in their fields and appreciated for the quality of the methods adopted in the paper. Thus, the literature reviewer should be careful about the sources they prefer (Wallace and Wray, 2013, 23).

Organize the literature systematically while reading and LO 4 writing it.

A systematic literature review should have a clear structure and this structure should be explained why this order was preferred (Oliver, 2012, 18). The literature preparation process includes three main steps such as finding the relevant literature, reading them, and writing up the literature review. Grouping similar works can be a good starting point in organizing the literature. For example, grouping the literature such as first group (studies broadly related to the dependent variable), second (studies related to the dependent variable) and third (studies most closely related to the dependent variable) can help the researchers in organizing and discussing the literature. Lodico et al., (2006, 56) suggest that a good structured literature review starts with a broad circle around the research topic and moves towards a more focused perspective gradually refining and narrowing it. They suggest a writing tip for the literature reviews that can help you to organize your literature: a) Prepare summary sheets or article abstracts, b) Review the summary sheets or article abstracts, c) decide how you can organize your literature review. Literature reviewers should pay attention to vague introduction and poor organization. Besides, the literature reviewer should move the reader beyond the current state of knowledge by presenting take-home messages. It is critical to keep in mind that the role of theliterature reviewer is to consider the evidence skeptically like a judge or jury. So, a well- organized literature reviewer should discuss the weaknesses the strengths of the current literature.

87 Writing a Literature Review 3

1. Which of the following cannot be a 1 Which of these cannot help you decide strategy for researchers to choose a research topic? 5 whether a publication is reputable? a. Experience a. Selective literature review b. Previous research b. Systematic literature review c. Replication studies c. Meta-Analysis d. Newspapers d. Narrative literature review e. Theories e. Focused literature review

2 “------refers to the studies that critics group the significant studies together in a particular field”. 6 The literature reviewer should: Which term can complete the missing word? a. Not find a research question from his/her prac- Test Yourself Test a. Historical knowledge tice or working environment b. Axiological knowledge b. Not point the gap in previous studies c. Review c. Remain objective from the introduction to the conclusion d. Debates d. Not avoid making pre-reached determinations e. Conceptual knowledge e. Not write the research questions in a neutral style 3 A literature review is: a. The dependent ariablev of the study 7 Which one of these is the most reliable b. Thecircumstances, topics, policies, or other information source? phenomena that they want to understand as dependent variables a. Primary sources c. The occurrence of several coup attempts in that b. Secondary sources country c. Tertiary sources d. Reference books such as almanacs and books d. Abstracts that contain helpful listings of the related field e. Wikis e. A systematic, explicit, and reproducible method for identifying, evaluating, and synthesizing 8 The Internet Material is likely to be very the existing body of completed and recorded reliable in: work produced by researchers, scholars, and practitioners. a. Home pages of individuals. b. The ebsitesw of organizations that are written 4 Which of these cannot express the purpose of by enthusiasts rather than experts. literature review? c. Peer-reviewed journal articles that are also pub- lished in an academic journal. a. To identify the field and specific context in d. Free-for-all post-your-views sites. which the topic is located. e. Weblogs (blogs), chatroom pontifications, etc. b. To decide which research and methodology will be adopted. c. To deepen your knowledge related to the key concepts of the research topic. d. To repeat the previous studies that have been completed before. e. To avoid the mistakes that past studies reported.

88 Research Methods 3

9 Blogs and wikis ought to be considered carefully before utilizing in academic papers. Why? 10 Which of the following features may not help you decide the scholarliness of the sources accessed a. They empower individuals to express themselves electronically? on issues a. More detail than the general information con-

b. These platforms cover superficial opinions Test Yourself tained in a newspaper or magazine article which lack rigor. b. Explanation of methodology c. None of these are posted carefully. c. A bibliography or reference list. d. They are not prevelant. d. Date of publication e. They guarantee the legitimacy of the material e. Online availability

89 Writing a Literature Review 3

If your answer is incorrect, review the section If your answer is incorrect, review the section 1. d 6. c on “Research Topic”. on “Literature Reviewer”.

If your answer is incorrect, review the section 2. c 7. a If your answer is incorrect, review the section on “Research Questions”. on “Selecting the Sources”.

3. a If your answer is incorrect, review the section 8. c If your answer is incorrect, review the section on “Literature Review”. on “Selecting the Sources”.

If your answer is incorrect, review the section If your answer is incorrect, review the section 4. d 9. b on “Purposes of A Literature Review”. on “Selecting the Sources”.

If your answer is incorrect, review the section 5. a 10. e If your answer is incorrect, review the section on “Types of Literature Review”. on “Selecting the Sources”. Answer Key for “Test Yourself” for “Test Answer Key

Explain the steps for finding a research topic and research questions.

Past experiences, theory testing and replication of previous research are three well-known reasons that researchers seek after. Past experiences of professionals can be a strong inspiration for people who spent their careers as well as students. Researchers are mostly interested in negative experiences for the purpose of improving the practices and making them better for next generations. Theories can be used to develop research studies. Even a single study can be the focus of a research to examine its implications. The replication studies refer to repeating a past research recognizing its methodological weaknesses to do a better one. The researcher repeats the study carefully so that ww the previous limitations will not be repeated (Lodico, Spaulding and Voegtle, 2006, 43). The researcher should provide a balance between a too-wide and extensively limited research topic. Keeping the research topic too wide can cause the work to remain your turn 1 superficial, to interfere with one another, not to know what is being explained, and finally to failw achieving a reliable result. However, if the research topic is extensively limited, then it is difficult for the researcher to find sources of the research topic. The researcher who struggles to limit the issue of research should

Suggested answers for “Your turn” Suggested answers for “Your first identify a “problem statement” that sets out the purpose of the research. The “problem statement” should be simple, clear and questionable, as well as being restrictive. The problem statement tells the truth that the researcher will defend in her/his research, test the truth, or reveal the cause-effect relations (Dinler, 2016, 36). The research questions that arise from the problem can be aswered by using various sources such as historical knowledge, axiological knowledge, theoretical and conceptual knowledge, prior research studies and reviews (Anderson and Arsenault, 2005, 50).

90 Research Methods 3

Discuss the key features of a well-structured literature review.

A literature review is a research methodology in its own right when it is carried

out systematically (Feak and Swales, 2009, 2). One of the main aspects of Suggested answers for “Your turn” literature review is the decision about which literature should be included and which literature should be excluded (Oliver, 2012, 4). A literature review should include the past research related with your research topic. A your turn 2 well-organized literature reflects the analytic capacities of the writer. While reviewing the literature on a specific topic, literature reviewer should focus on trends, inconsistencies or similitudes which different writers may have missed or ignored. If the literature reviewer presents examples that different writers differ on the interpretation of data, or where they take different approaches to an issue, this can often be highly enlightening (Cooper, 1998, 9).

If you were undertaking research about your master thesis, what do you think the most reliable kind of literature to use would be?

In a digital age, electronic sources incerase rapidly. One of the most critical issues in a research attempt is the appropriateness and reliability of the information sources. It is essential to reference the primary sources in the search for the literature. However, secondary sources are also useful. Secondary sources are a synthesis of previous primary sources of theoretical and empirical studies. your turn 3 Secondary sources allow a quick look at the recent developments on a specific subject. In these cases, technical information is usually neglected. Primary sources are original research studies or writings (Balcı, 2013, 76). The Internet material can be ranked as reliable, fairly reliable and unreliable in terms of their sources. Peer-reviewed journal articles, book chapters that have been posted usually in PDF format on an academic’s home page, and official materials published on a recognized institutional website are likely to be very reliable. Lecture or research notes on the site of an academic working at a recognized institution are likely to be fairly relaible internet material. Lastly, the material on the home pages of individuals, the material on organization websites that is written by enthusiasts rather than experts, free-for-all post-your-views sites and weblogs (blogs), chatroom pontifications chatroom pontifications are likely to be unreliable (Wallace and Wray, 2013, 23).

91 Writing a Literature Review 3

Discuss the possible steps to be taken to develop a well-organized literature review.

In a well-organized literature review, the problems and controversial issues related to the topic should be discussed based on the latest documents. The literature review should be long enough to persuade the reader that the literature was read extensively by critically evaluating the cited literature (Feak and Swale, 2009, 5-10). The researcher should clearly define the goal of the your turn 4 study. A well-structured literature should make a summary of the methods and findings of the studies cited rather than the conclusions. Avoding to present excessive details may manage the risk of making an error. The paper’s value will increase if the weaknesses and strengths of the evidence you referred can be discussed adequately. As a researcher, the literature reviewer should be consistent with what is evidenced and concluded. Answer Key for “Test Yourself” for “Test Answer Key

Further Reading

Aveyard, H. (2010). Doing a literature review in Hart, C. (2005). Doing a Literature Review: Releasing health and social care: A practical guide. McGraw- the Research Imagination. Sage. Hill Education (UK). Wallace, M., & Wray, A. (2011). Critical reading and Baumeister, R. F. (2013). Writing a literature review. writing for postgraduates. Sage. In The Portable Mentor (pp. 119-132). Springer, http://koha.ekutuphane.gov.tr New York, NY. https://scholar.google.com.tr Dinler, Z. (2016). Bilimsel Araştırma ve E-Kaynaklar. Bursa: Ekin Kitabevi Yayınları. https://www.proquest.com/products-services/ pqdtglobal.html Fink, A. (2014). Conducting research literature reviews: from the Internet to paper. Sage. https://tez.yok.gov.tr/UlusalTezMerkezi/tarama.jsp http://www.toplukatalog.gov.tr

References

Anderson, G., & Arsenault, N. (2005). Fundamentals Baumeister, R. F. (2013). Writing a literature review. of educational research. (London: Routledge), 2nd In The Portable Mentor (pp. 119-132) (New Edition. York: Springer), 2nd Edition. Aveyard, H. (2010). Doing a literature review in health Braun, B., Schindler, S., & Wille, T. (2018). and social care: A practical guide. (UK: McGraw- Rethinking agency in International Relations: Hill Education) performativity, performances and actor- networks. Journal of International Relations and Balcı, A. (2013). Sosyal Bilimlerde Araştırma Yöntem, Development, 1-21, At https://link.springer.com/ Teknik ve İlkeler (Ankara: Pegem Akademi article/10.1057%2Fs41268-018-0147-z. Yayıncılık).

92 Research Methods 3 Chan, L. M., Intner, S. S., & Weihs, J. (2016). Guide Ridley, D. (2012). The literature review: A step-by-step to the Library of Congress classification. (Santa guide for students. (London: Sage), 2nd Edition. Barbara: ABC-CLIO), 6th Edition. Roselle, L., & Spray, S. (2011). Research and writing Cooper, H. M. (1998). Synthesizing research: A guide in international relations. (USA: New York, for literature reviews. (USA: Sage), 3rd Edition Routledge). Dinler, Z. (2016). Bilimsel Araştırma ve E-Kaynaklar. Scharma, C. K. (2008). Practical Handbook of Dewey Bursa: Ekin Kitabevi Yayınları. Decimal Classification (Delhi: Atlantic Publishers and Distributors). Dittmann, H., & Hardy, J. (2000). Learn Library of Congress Classification (Lanham, MD Scarecrow Schryen, G., Wagner, G., & Benlian, A. (2015). Theory Press). of knowledge for literature reviews: an epistemological model, taxonomy and empirical analysis of IS literature. Feak, C. B., & Swales, J. M. (2009). Telling a research At https://pdfs.semanticscholar.org/c00b/617bf23 story: Writing a literature review (Ann Arbor: 24ddef2b8175a7337ddfe838295bd.pdf University of Michigan Press). Trawick, B. W., & McEntyre, J. R. (2004). Fink, A. (2014). Conducting research literature reviews: Bibliographic databases. At http://global.oup.com/ from the Internet to paper (USA: Sage). booksites/content /019963887x/13183238 Hart, C. (2005). Doing a Literature Review: Releasing Wallace, M., & Wray, A. (2011). Critical reading and the Research Imagination (UK: Sage). writing for postgraduates. (London: Sage), 2nd Jesson, J., Matheson, L., & Lacey, F. M. (2011). Doing Edition. your literature review: Traditional and systematic https://www.amazon.co.uk techniques. (London: Sage). https://www.britannica.com/science/Dewey- Lodico, M. G., Spaulding, D. T., & Voegtle, K. H. Decimal-Classification). (2006). Methods in educational research: From theory to practice. (USA: John Wiley & Sons). http://guides.lib.umich.edu/greyliterature. 12.05.2018 Knopf, J. W. (2006). Doing a literature review. PS: http://www.indiana.edu/~liblife/database_searching. Political Science & Politics, 39(1), 127-132. pdf Kumar, R. (2011). Araştırma yöntemleri (Çeviri http://koha.ekutuphane.gov.tr Editörü: Ömay Çokluk). (Ankara: Edge Akademi) http://libguides.merrimack.edu/research_help/ Monroe, A. (2000). Essentials of political research. Sources) (Boulder, Colo: Westview Press). http://libguides.librawry.curtin.edu.au/c.php?g= Murray, R. (2006). How to write a thesis. (UK: 202368&p=1332723 McGraw-Hill Education). https://www.proquest.com/products-services/ Oliver, P. (2012). Succeeding with your literature pqdtglobal.html review: A handbook for students. (UK: McGraw- https://scholar.google.com.tr. Hill Education). http://www.toplukatalog.gov.tr Onwuegbuzie, A. J., & Frels, R. (2016). Seven steps to a comprehensive literature review: A multimodal https://tez.yok.gov.tr/UlusalTezMerkezi/tarama.jsp and cultural approach. (Los Angeles: Sage). https://www.wikipedia.org Paez, A. (2017). Gray literature: An important resource in systematic reviews. Journal of evidence-based medicine, 10(3), 233-240.

93 Qualitative Research Chapter 4 Methods

After completing this chapter, you will be able to:

Explain the contributions that qualitative research methods can bring into international 1 Define qualitative research methods 2 relations

Explain the properties of qualitative research List the steps of conducting a qualitative 3 designs 4 research process

List the criteria for rigor in qualitative research 5 studies Learning Outcomes Chapter Outline Key Terms Introduction • International relations Qualitative Research Methods and International • Qualitative research methods Relations • Case study • Phenomenology Qualitative Research Designs • Ethnography • Narrative inquiry The Process of Conducting a Qualitative • Grounded theory • Rigor Research • Participant • Interview Rigor in Qualitative Research Method • Observation • Document • Inductive analysis • Content analysis

94 Research Methods 4 INTRODUCTION Following sections of this chapter list qualitative research designs and their properties. In addition, International Relations (IR) is described as an a sample qualitative research will be designed and interdisciplinary field that studies the relations of hypothetically conducted to give you a more solid states—accepted as primary actors—with other example through relevant steps. In this sense, we states, international or regional organizations, will start off by determining the subject to study multinational companies, international with a qualitative research approach, and continue norms, and international society. This unique with participants, data collection techniques, and structure interweaves IR closely with other analysis. Lastly, trustworthiness and credibility of disciplines such as political science, economics qualitative research methods will be studied under (international economy), history (political rigor title. history), law (constitutional law, management law, and international law), philosophy (political philosophy and ethics), sociology, geography, QUALITATIVE RESEARCH and anthropology, Due to its multi-directional METHODS AND nature, IR focuses internationally on a variety INTERNATIONAL RELATIONS of issues such as globalization and its impact on First of all, it has to be underpinned that the international society and state sovereignty, qualitative research methods are intricately related sustainable development, nuclear proliferation, with different disciplines and include many terrorism, organized crime, nationality, human concepts. Some of the concepts within qualitative rights, environmental problems, security, and research methods are ethnography, anthropology, human trafficing. Because of its topical versatility situational research, interpretive research, natural and multi-disciplinary pattern, the field of IR research, and theory development. Since all these spreads across several research traditions within concepts are used with similar meanings in terms social sciences. In other words, its multi-disciplinary of research design and analysis techniques, the term layout causes a significant level of variety in terms ‘qualitative research methods’ is widely accepted of the research designs adopted within IR (Lamont, as an umbrella term. Accordingly, basic features 2015; Levy, 2002). of qualitative research methods are explained at Together with the fact that research efforts the beginning of the chapter with the hope to based on quantitative methods built around provide details for its definition, and then potential various designs have born a crucial relevance to contributions that qualitative research methods the field of IR for a long time, the need to closely can make into the field of IR are explored. examine available situation(s) and to hold in- depth analysis to understand and describe them has also been underlined within relevant literature. Basic Features of Qualitative In this respect, the idea that qualitative research Methods methods can be functional to study specific events Qualitative research is defined as a method and phenomena has spread across the literature where perceptions and events can be pictured in (Lamont, 2015; Levy, 2002; Wiener, 2009). It is their natural environments with a holistic and possible to state that qualitative research methods realistic perspective through use of qualitative data can be a useful lighthouse to take right steps and collection techniques such as observation, interview, produce more tangible results because they can and document analysis. To further clarify, its basic only be contextually conducted and interpreted, features can be listed as follows: (1) sensitivity to and they facilitate complete understanding of natural environment, (2) participatory role of the a focal event, subject, or subject field without researcher, (3) holistic approach, (4) identifying worrying about generalizations. Accordingly, the perceptions, (5) flexibility in research design, this chapter first underlines what is a qualitative and (6) inductive analysis. Each of these features research method, what are its basic features, are explained in the following paragraphs (Bogdan and what are the contributions that qualitative and Biklen, 2007; Creswell, 2014; Simsek and research methods can make to the field ofIR. Yıldırım, 2013).

95 Qualitative Research Methods 4 Sensitivity to natural environment refers to the One of the most important features of qualitative condition that the focal phenomenon/phenomena research is that the perceptions and experiences of or event(s) of a qualitative research has/have to be participants are depicted. The goal in qualitative examined in their natural environment, or context. research is to understand how participants This specifically means that any phenomenon or perceive and interpret the external world. In this event studied via qualitative research can never be regard, researchers have to spend some time with manipulated. In this method, events/phenomena participants, talk to them, and observe them. have to be perceived in their natural environment Flexibility of design is classified among the basic as much as possible and only those findings based features of qualitative research as well. Therefore, on the real events in that context can be reported selection of appropriate method or methods because realities emerging in natural environments that would attain the goal most effectively bears are more significant in social sciences. a crucial role in qualitative research. On the other hand, focal patterns studied via qualitative important research methods are constantly prone to change, which entails flexibility in the process. In return, this flexibility makes qualitative research open to In qualitative research methods, events rearrangement and variability. have to be based on the realities emerging in their natural environment (context) Lastly, qualitative research methods are based on and only those findings can be reported. the principle of induction rather than exploring cause and effect relations grounded on a specific theory. So, a researcher employing qualitative research methods strives to distill some meaning from the Another basic feature of qualitative research collected data and to obtain patterns concerning is the participatory role of the researcher. In this the focal problem for which s/he has been collecting method, the researcher spends time in the field, descriptive and detailed information. directly interviews the participants or other sources of information and shares the same experiences As listed above, qualitative research methods have many advantages in terms of producing with the participants if necessary, and uses his systematic and sound scientific knowledge. This professional perspective and knowledge as a source part of the chapter is dedicated to picturing how of data. On the other hand, the researcher in a these specific features of qualitative research are qualitative research runs the risk of intervening different from those of quantitative research. In with the natural course of events due to its physical addition, contributions that qualitative research closeness to sources of information. Thus, the have made to various fields owing to its basic researcher has to be personally present in the features should also be mentioned duly. A list a natural environment and avoid influencing the mile long can be made about the basic features of natural course of events. qualitative research and the possible contributions Holistic data collection is among the other it can bring to the field of IR. Some of the possible features of qualitative research methods. Based contributions are explained and presented with on the fact that a whole means more than the tangible examples under the next subtitle. collection of its parts, research subject is approached with a holistic perspective. Collected with this standpoint, data should also be analyzed through Possible Contributions of a holistic approach. What lies under this idea is Qualitative Research Methods the hypothesis that the nature of human behavior It is vital to note Glaser and Strauss (1967) while is sophisticated and cannot be easily divided into speculating about the contributions that qualitative smaller parts. That is why qualitative research research methods can make to the field of IR. So, efforts focus on the unity of variables that comprise Glaser and Strauss conclude that research studies the problem and the relations among them. conducted via quantitative research methods are

96 Research Methods 4 inclined to see the facts through the same eyes all Besides, it should be kept in mind that the time, and thus, fail to explain social phenomena information and data regarding the process can that are constantly in a state of change. Yet, it also be collected in qualitative research methods. should be remembered that social phenomena Such information highlights what happened cannot be universal, are never fixed, and can change during the research process and their reflections in time. Likewise, Bennett and Elman (2006) note on the participants. In this sense, qualitative that social world is complex and only one research research methods’ adaptability to change should be approach or design is not enough to understand underpinned one more time considering the fact this complexity. Under the light of these, attempts that social phenomena within IR are never stable. made by qualitative research methods to describe, With respect to focusing on change, how the understand, and conduct in-depth analysis can improve IR’s capacity to explain the relativity and direction of relations between the USA and Russia mobility of social phenomena at hand. It is reported over Syria changed in time and the possible causes that use of qualitative methods in IR augment and of this change can be given as an example for a diversify in-depth knowledge by offering scientific qualitative study in IR. In such a study, the focus methodology variety and expanding the range of should be on the change the phenomenon displays. topics for new studies. That is to say, Bennett and Within IR, such studies conducted for a specific Elman underline that scientific knowledge can be phenomenon are labeled as process tracing, and enriched through micro analyses and focusing on it is classified as a design in some sources (Klotz, what is in the background of phenomena thanks to 2008). qualitative research methods. To make the contributions that qualitative methods can introduce into IR more tangible for the readers, one can mark three types of information internet (environment, process, perceptions) that can be gathered through use of qualitative research methods. You can visit the following URL for further First is the environmental information, which information about the significance of qualitative regards social, psychological, cultural, demographic, research methods in IR. Available at: https:// and physical characteristics. Such information may www.e-ir.info/2016/11/24/methods-war-how- serve as a foundation for patterns concerning the ideas-matter-within-political-science/ process and perceptions, and collected information can be compared with that of other settings. For instance, additional studies on compliance with Data concerning the perceptions that could international treaties and explanation of civil be determined through qualitative studies depicts wars can provide comprehensive information what participants think about the process. It should about the available situations and can clarify their be noted that the focal event or phenomenon in understanding. Furthermore, such information to qualitative research cannot be conceived out of its be gathered about a specific country can be used to context and participants’ perceptions. Therefore, compare the present situation with other countries. information and context are together valuable to understand the nature of events. One solid example important for IR can be the current status of relations between the USA and Russia—one of the hot issues in the Qualitative research methods can help collect region. In this example, it is almost impossible to comprehensive information about a focal completely understand and picture these relations topic within IR, and this information can without determining the perceptions of politicians be employed to further clarify the present and military figures, significant actors, within the situation and to make comparisons. context they exist.

97 Qualitative Research Methods 4 QUALITATIVE RESEARCH DESIGNS internet Up until this section of the chapter, definition Jacobus Fellowship Award designated and basic features of qualitative methods have been to “Protest, Social Policy and Political clarified and possible contributions they can make Regimes in Tunisia and Morocco”, which to the field of IR have been outlined. The designs was designed as a case study, points out the employed in qualitative research are explained importance of qualitative methods for IR. in this section. Accordingly, case study design, You can visit the URL below for further which is the most often utilized one, takes the details about the study. Available at: https:// lead. Subsequently, ethnography, narrative inquiry, www.princeton.edu/news/2018/03/13/ phenomenology, and grounded theory designs are jacobus-fellow-chantal-berman-studies- clarified respectively. social-protest-tunisia-and-morocco. Case Study As mentioned above, case study is the most As clearly depicted above, qualitative methods widely employed design in IR, and this is an can bring a variety of contributions to the field increasing tendency in the field. The efforts to of IR. In short, the benefits that this method can figure out complex social events or phenomena supply, due to various designs it hosts, can be can be taken as the reason for this inclination. turned into a long list. For instance, one of the A simplified definition of case study could be significant contributions that is not mentioned the attempt to seek answers to “how?” or “why?” here is the functionality of information obtained concerning a current case of interest (Gray, 2009; via qualitative method. In this regard, that the Maoz, 2002; Yin, 2014; Wiener, 2009). According information distilled through qualitative research to Yin (2014), case study refers to examination of methods is based on facts can ease its reflection a case in real life (context) when the boundaries onto the field functionally, and this can facilitate between an event and its context are obscure. producing more effective solutions for available problems. To put it differently, findings determined as a result of a qualitative study may bridge the Case study is close examination gap between theory and practice. In conclusion, of an event or phenomenon in its qualitative research—with its concern to describe, context in a given time-period. analyze, and understand comprehensively—can introduce multi-dimensional benefits for IR. Qualitative research methods can be considered as a complementary component for IR to generate The sum of definitions indicate that case study systematic scientific knowledge. is a qualitative design where a researcher collects data from various sources (e.g. observations, interviews, visual-auditory materials, documents, and reports) about an event or phenomenon in a specific context in a given time-period. Yin (2014) 1 classifies case studies mainly into two: single-case What kind of scientific information and multiple-case. The former,single-case studies, can be obtained with a qualitative can be suitable for critical, extraordinary, common, study focusing on the migration revelatory, or longitudinal cases whereas the latter, of Syrian refugees to EU member multiple-case studies, refers to more than one case. countries? At this point, it is wise to remember that case study may sometimes involve analysis of more than one case depending on the focal phenomenon.

98 Research Methods 4 As for Woodside (2010), case studies should be 2007; Maoz, 2002). Although case study is the categorized into three groups, which are theorizing most common design for qualitative research and testing, narrative or drawing, and assessment. studies in IR, debates about the focal events and Primary focus of theorizing and testing case studies phenomena are still at large. So, IR researchers is collection of empirical data for a theory whose have long been trying to answer what case would effectiveness is still blurry.Narrative and drawing be the right case to focus. For example, “Arab case studies involve sharing descriptive records Spring” or “People’s revolt in Arabic countries” can about political decisions, social structures, or seem like two distinct phenomena each of which stories of systems with relevant people following can be analyzed as a “case” because the case consists a scrutiny. Assessment based case studies refer to of other larger scale events and phenomena such informing relevant people or institutions about as multiple-context, a long time-period, power value judgements after researchers closely examine balance, power transition, aggressive actions, the aforementioned political formations, systems, and finale of people’s revolt and its aftereffects. projects, or other case studies (Koltz, 2008; Levy, Similarly, Levy (2002) emphasizes that the case 2002; Maoz, 2002). to be focused should be totally compatible with the researcher’s aim. Nevertheless, researchers to As stated at the beginning, case studies are conduct a case study analysis in IR are generally frequently utilized especially within IR and known to describe one event that is spatially and comparative political sciences. The reason for temporally restricted. All in all, researchers should this, as explained by Maoz (2002), is the flexible consider other factors that could be influential over structure of case study design, which offers analysis the context of the case while determining the focal of all sub-fields within the field of IR. On the other events and phenomena of their research, and the hand, case studies conducted in IR are known to focal phenomenon should be limited in terms of employ several quantitative data collection tools as time and place. well. Thus, we should know that not all the research studies in the field have been conducted through qualitative research methods. One more thing is that historical case studies conducted in the field are mostly classified as process tracing because they are internet actually conducted via process tracing. Regarding Conducted as a dissertation by Lugalla (2016), this, Klotz (2008) concludes that the focus of case “Effects of Political, and Governance Challenges: studies administered in IR is generally one country, Case Study of East African Community” can be taken and there is often a concealed assumption that a for a sample case design in IR. Available at: https:// historical process will be traced. Besides, it should dash.harvard.edu/bitstream/handle/1/33797391/ also be noted that case studies are widely known LUGALLA-DOCUMENT-2016.pdf?sequence=1 as “case reports” in political sciences (Levy, 2002).

important Ethnography Because of the increase in research studies Case studies are sometimes known employing ethnography design in 1980s, this as process tracing and case reports period is also known as “ethnographic transformation in IR, and they employ quantitative of global policies” (Varsti, 2008). Stemming from data collections, too. anthropology and sociology, ethnography is a research design where a researcher analyzes common behaviors, language, and action models of The contributions that case study design can an intact culture within their natural environment make into the field of IR have been reported during a long time-span. Wolcott (2008) states comprehensively in the literature. Some are listed that ethnography is a study conducted on social as lesser costs, process tracing, in-depth analysis, behaviors of a definable human group instead of a and constant comparison (Bennett and Elman, study conducted on culture.

99 Qualitative Research Methods 4 addition, Vrasti advocates that researchers’ efforts to understand and explain a restricted context Ethnography is a research design where a causes confusion, and documented data can set researcher analyzes common behaviors, researchers free by clarifying uncertainties. After language, and action models of an intact all, Vrasti—whose ideas are collectively welcome culture within their natural environment in IR—regards ethnography research design as an during a long time-span. interpretation process consisting of other data sets such as documents. Conducted by Stepputat (2012), “Knowledge production in the security–development nexus: An As clearly underlined in the definition, in ethnographic reflection” can be taken as an example ethnographic studies, ‘any human group that can be for ethnography research. Stepputat aims to analyze defined as a group can be chosen, and this group’s the trans-nationalized forms of security governance culture (way of life) regulating their everyday lives is in the postcolonial world by looking into the analyzed. Ethnography researcher enters in the natural production of knowledge aimed at increasing environment where the group lives and participates coherence between domains of security and in daily routines together with the members of that development in Western donor policies. Stepputat culture during a time labeled as fieldwork. Sharing takes an ethnographic approach to the analysis of the same culture with the members, researchers knowledge production, using his personal experience conduct studies where they interact with others of writing a policy analysis for a donor government for a considerable amount of time with minimum concerning how to ‘further improve’ the policy of intervention in order to develop noticeable study ‘concerted civil–military planning and action’. At models. In this way, the researcher observes the the end of this study, Stepputat finds that the policy cultural components in their real settings. An process is less concerned with effects on the ground indispensable principle of ethnographic studies is than with the problem of creating unity among the “the reflection of social life as it is experienced by the wide range of agents and institutions involved in the members of that group” (Wollcott, 2008). Primary emerging policy field. About this result, Stepputat role of ethnography researchers is to closely examine says, while such an approach may have potentially social organization models (e.g. social webs) and destabilizing effects – both for policy narratives thinking systems (e.g. worldviews, opinions). In this and for researchers’ authority – it responds to calls regard, ethnographic studies can focus on both the for reflections on the politics of representation and formation of structures, institutions, and policies writing in studies of international relations. that determine international relations and how they are put into practice. In line with the remark that global policies Narrative Inquiry undergo ethnographic transformation, this design We should begin by pinpointing that narrative is widely employed in the field of IR. However, inquiry basically focuses on human experiences and this design, like case studies, is not free from some stories. Relevant literature hosts narrative inquiries basic discussions. What lays the foundation for under several names such as autobiographies, these debates is the concern that researchers bear individual narrations, narrative interviews, life for the level of participation into the context but documents, life stories and histories, oral history, not for the contributions that this design can bring ethno-history, ethno-biography, auto-ethnography, into IR. Vrasti (2008) points out that researchers ethno-psychology, and personalized ethnography. are mostly occupied with how they should report Czarniawska (2004) diagnoses narrative inquiry an ethnographic study due to the vast scope design as written documents and conversations of topics in IR, which does not produce any interpreted about an event or a course of events information that can be functionally utilized in that are chronologically related. Narrative inquiry the field. Moreover, Vrasti adds that ethnography is based on sharing experiences about an event or a should be conducted as “non-local ethnography” chain of events with another or others through any to produce realistic and functional information. In channel of communication.

100 Research Methods 4 and others are shared; and where several rules, principles, pictures, metaphors, and personal Narrative inquiry is a qualitative research design philosophies are cited—are among the documents where written documents or conversations are of source utilized in narrative inquiry studies in interpreted about an event or a course of events social sciences (Clandinin and Connelly, 2000). that are chronologically related. These features promise that narrative inquiry design can bring significant contributions into IR. One of these contributions, as reported in What a researcher aspires to explore in the relevant IR literature, is the potential that narrative inquiry are the participants’ experiences. narrative inquiries bear to comprehensively analyze Clandinin and Connelly (2000) categorize them the events or phenomena and their institutional as personal/individual and social experiences. operations. Besides, obtained results may produce Noli (2013) explains that life stories are psycho- significant information for development and policy socially constructed within narrative inquiry. The formation. same study also underlines that the participant Studied by Gertsen and Søderberg (2011), is not the only shaper of his/her own life stories “Intercultural collaboration stories: On narrative (life experiences), s/he constructs these stories via inquiry and analysis as tools for research in interacting with others within a socio-cultural international business” can be taken as an example environment. Giovannoli, (year) thus, emphasizes for narrative inquiry research. The objective that narrative inquiries should focus on both of this research was presenting stories about personal and social experiences. the collaboration between a Danish expatriate Narrative inquiry is a study design where the manager and his Chinese CEO in the Shanghai researcher analyzes individuals’ lives and asks for subsidiary of a Multinational Enterprise (MNE). life stories about their lives (Riessman, 2008). Both The authors conducted interviews with expatriate witnessed and reported life stories are, therefore, managers and Chinese managers in Chinese collected from individuals (documents and group subsidiaries of these MNEs. The interviews interviews) during the research. In this design, data centered on the interviewees’ personal experiences delivered by participants is generally re-narrated of success and challenges in intercultural and re-told by the researcher along a chronological collaboration. In order to prime the interviewees, route. In narrative inquiries, no abrupt attempts the researchers provided them with a list of sample are made for the research goal of interest and questions. Western expatriate managers in China academic concerns, yet focus is tuned onto the were asked, for example: “What are your most participants experiences that span across a specific significant experiences from collaboration with period to be able to scrutinize these experiences Chinese colleagues? Can you think of something more comprehensively (Josselson, 2007). Clandinin particularly surprising, frustrating, difficult, and Connelly (2000) report that there are three positive, thought-provoking?” The authors components in narrative inquiry: temporal context, demonstrate how narrative concepts and models social context, and space. These components may help are able to elucidate intercultural collaboration deepen the understanding of a specific story; in the processes by analyzing how each member of a meantime, they are also regarded as the common dyad of interacting managers narrates the same parts of all stories. In this sense, personal experiences chain of events. The authors of this research show of the shapers in a story and their interactions with how the narratological concepts of peripeteia and others constitute character dimension. Temporal anagnorisis are well suited to identifying focal context of a story, or time frame (past-present- points in their stories: situations where change future), builds the time dimension, and the physical follows their recognizing new dimensions of their environment where a story takes place forms place conflicts, eventually furthering their collaboration. dimension (Barkhuizen, 2008). They explain how Greimas’s actantial model is Field notes—where experiences, interview valuable when mapping differences between logs, observations, narrations, letter writing, changes in the narrators’ projects, alliances, and autobiography writing, class notes, newspapers, oppositions in the course of their interaction.

101 Qualitative Research Methods 4 Phenomenology approaches. Although these two approaches aim to Another qualitative research design that can dig out personal experiences about a phenomenon, be employed in IR is phenomenology, which is they differ in terms of viewpoint and focal rooted in philosophy and psychology and where point. Accordingly, the concept of descriptive participants’ experiences about a phenomenon phenomenology was first used by Husserl in 1970s. defined by them are described. As obvious by Husserl emphasizes the importance of experiences definition, phenomenological research aims to perceived by human consciousness and states that determine individuals’ own perceptions and it should be the topic of scientific research efforts. experiences through their own standpoint (Lester, The rationale behind this, as for Husserl, is that 1999). These descriptions facilitate access to the human behaviors are influenced by anything that gist of experiences that participants have about the is perceived real, and therefore they should be relevant phenomenon (Giorgi, 2009; Moustakas described in order to understand human motives. 1994). Interpretive phenomenology, on the other hand, defines interpretation as revelation of concealed things within human relations and experiences (Lopez and Willis, 2004). Thus, interpretive Phenomenology is a research design to phenomenology focuses on meanings embedded study participants’ experiences about a in individuals’ lives rather than simple definitions phenomenon they define. of concepts and their qualities. Similar to other qualitative research designs, phenomenological research is also frequently This design focuses on phenomena which employed for studies within the field of IR. According participants are aware of but do not have enough to this, for example, policies developed for the field in-depth and comprehensive understanding about. can be examined in terms of international treaties In life, encounters with phenomena mostly occur and experiences of individuals that compromise the via various forms such as events, experiences, society. In this sense, phenomenological research can perceptions, tendencies, concepts, and situations. be utilized to depict the current status of available Primary goal of phenomenology is to reduce arrangements in IR and to devise propositions that individual experiences with a phenomenon to a could improve the process. universal explanation. A phenomenology ends in Conducted by Kholmi, Triyuwono, a descriptive part where the gist of individuals’ Purnomosidhi, and Ganis-Sukoharsono (2015), experiences is elaborated through combination of “Phenomenology study: Accountability of a “what” is experienced “how” (Lester, 1999). political party in the context of local election” can be Any human experience (event, phenomenon, taken as an example for phenomenological research. happiness, accident, relation, situation, opinion, The aim of this research is to understand the forms feeling, etc.) can be a research topic within and practices of accountability of a political party phenomenological inquiry. Advocating that under the perspective of Regional Executive Board experiences form behaviors, that behaviors are (DPD) in Jombang City. Data were collected by mostly purposeful and intentional rather than in-depth interview and by participant observation. being a simple response or reaction, that behaviors The interviewees are the head of the Political are shaped in accordance with goals, beliefs, Parties (AF), vice-chairman I (MS), vice-chairman wishes, or perceptions, that experiences and II (BC), secretary (FR), treasurer (SC), and the meanings collected by individuals about the same Advisory Council of the Party (MJ). The process phenomenon are different, and that people’s past of data analysis has three stages: (1) bracketing experience matter as much as the present ones do, (epoche`), which reveals the reality (phenomenon) phenomenological research focuses on meanings without preconceptions; (2) crystallization eidetic: that individuals derive from their experiences sorting out the essential aspects to be formulated (Smith, Flowers, and Larkin, 2009). There are two into meaning, and are grouped into certain types of phenomenological research: descriptive themes; (3) the stage of synthesized themes into phenomenology and interpretive phenomenology the narrative description. By using phenomenology

102 Research Methods 4 of interpretive paradigm as a research design, the the researcher analyzes the collected data, and the results of the study indicate that the accountability concepts derived from analyses serve as a foundation of the political party in the practices of the local for further data collection processes. What matters election of Jombang City is categorized into three at this point is to collect data and conduct analyses forms, which are accountability of the quality of simultaneously via comparing collected data politician candidates, legal accountability, and among itself. Constant comparison of research accountability of campaign funds. data facilitates theory building process because the differences and similarities among the data set are Grounded Theory always obvious. Third feature, on the other hand, is the advantage to purposefully select study sample Grounded theory is one of the qualitative within grounded theory. A sample that promises research methods where a series of methods are necessary richness in data set should be selected in systematically operated in order to develop an grounded theory since the theory will be derived inductive theory (Glaser, 2002). It is defined during the research process involving simultaneous as a process in which the researcher digs out a data collection and analysis. Labeled as theoretical general abstract theory about a process, action, or sampling, this process is peculiar to grounded theory interaction grounded in participants’ standpoint among other qualitative research designs. In their (Corbin and Strauss, 2008). study conducted to devise a theory that explains market tendencies of firms, Gebhardt, Carpenter, and Sherry Jr (2006) state that they picked up two firms that are at the beginning, in the middle, and Grounded theory is a qualitative research at the end of a change process respectively so that design aiming to find out a general abstract the sample could include enough variety to scaffold theory about a process, action, or interaction a theory. In this way, researchers produce a theory grounded in participants’ standpoint through constant comparison of the data obtained from a theoretical sample. It should again be underlined that discussions As can be followed from the definition, revolving around the use of grounded theory grounded theory makes use of processes and within the field of IR have not come to an end yet, interactions grounded in participants’ narratives, just like the other qualitative research designs. The perceptions, and experiences in order to develop a hottest issue among these discussions is whether theory that can explain the phenomenon on a wider the researchers collect their data set without conceptual level. The researcher in a grounded actually stepping into the field and based on theory tries to explore participants’ social processes, documents (historical documents, books, literary behavioral patterns, acceptances, and meanings. works, etc). What started this discussion is the In other words, researchers focus on a process or fact that a data set belonging to a focal country action that has a different step or phase formed of a grounded research study will be too vast and in time (Corbin and Straus, 2008; Glaser, 2002). large, and this will result in too heavy and dense Grounded theory research strives to devise an a data set to analyze. Moreover, this may also interpretive theory through an inductive approach. steer grounded theory research to derive more The theory constructed as a result of research is a superficial—shallow if you may—theories instead product of the process (Birks and Mills, 2015). An of sensitive and precise ones. Accordingly, Glaser important feature of grounded theory is that the and Strauss (2006) conclude that documents can concepts and categories are distilled from the data replace interviews and field notes in very large scale collected throughout the research. These concept samples. However, such a data collection technique and categories can never be established before the will limit the researcher with only documents, research commences. Another significant feature which may in turn restrict the theory to be devised is the fact that data collection and analysis are in the end. Therefore, use of grounded theory in intricately interwoven. Following the initial data IR should be confined to narrow-scale events, collection process in grounded theory research, phenomena, and areas.

103 Qualitative Research Methods 4 Concluded by Swift (2009), “A grounded theory approach to analyzing political narratives” can be noted as an example for grounded theory research. Swift used grounded theory methods to create a theoretical model for understanding the formation and development of political narratives. The analysis of six interviews with eight participants made clear that at least three super-ordinate categories must be considered in analyzing political narratives. The categories were beliefs about how the world is (beliefs), beliefs about how the world ought to be (values), and politically relevant emotional sentiments. Swift found at the end of his study that all three 2 categories influenced every participant’s narrative in important What kind of advantages do you respects. He argues that emotional sentiments were associated think case study design bears to with the amount of trust participants gave to news sources and explain a phenomenon of your different candidates. Swift inferred that values and beliefs were the own interest? Search about it. foundational principles or rationales behind the stances people took with reference to issues, parties, and candidates.

THE PROCESS OF CONDUCTING A QUALITATIVE RESEARCH So far, qualitative research method, its possible contributions to IR, and designs in this method have been elaborated. As of now, we’re starting to scrutinize how to plan, conduct, and finalize a qualitative study by considering all the designs studied in this chapter. At this point, it will be wise to remember the differences these designs pose as mentioned earlier in each design’s paragraph above. Taking these differences into account, each step of a research process is provided below. So, the subtitles in this section include identifying the research focus or problem, selecting participants, data collection techniques, data analysis, and reporting.

Identifying the Research Focus or Problem

Selecting Participants

Data Collection Techniques

Data Analysis

Reporting

Figure 4.1 Conducting a qualitative research process

Identifying the Research Focus or Problem Regardless of design, any qualitative research process starts with identification of the research topic, phenomenon, or problem. Knowledge and experience of a researcher, the context s/he is in, and her/his interests are influential over determining the research topic or problem (Creswell, 2014). The focal subject of a research is one of the definitive factors for the design of that research. However, identification of the research problem during the planning process is not enough on its own in terms of specifying the research design. Therefore, planning process of a qualitative research entails identification of the research problem together with the research aim and questions to be answered at the end of the study. Research aim and questions are endemic to the nature of the research problem, and they include the point(s) that a researcher aspires to illuminate during the process (Mertler, 2014).

104 Research Methods 4 important do people experience the x phenomenon” can be a possible question for a grounded theory research. During planning, a researcher has to Moving along the research process, a researcher can determine the research problem and the narrow the scope of questions. research aim and relevant questions to be answered at the end of the study. Selecting Participants Following the identification of the research problem, aim, and questions for a study, the next Research question(s) serve focusing on the important step is to choose participants. Thus, a research aim. In this regard, research questions researcher should select participants who can provide should describe analysis, formation of limitations, rich and versatile data for the focal phenomenon and some instructions. An aim for a research could or problem determined to be examined (Creswell, be filling in the gap available in a pile of information. 2014). No doubt that the process of selecting Following questions can guide researchers when participants depends on the focal topic, problem, formulating research questions: or phenomenon. Nevertheless, as a general rule, • e Is ther a consistency between the research several factors should be taken into account during aim and research questions? selecting participants, which are age, religion, • Are the research questions clear, correct, ethnicity, cultural elements, gender, and political and to the point? and economic factors. • Can the research questions be answered? Briefly, participants should have experience Are the sources you have enough to answer about the focal topic, problem, phenomenon, these questions? or event and they should partake in the research Of course, determining the research questions voluntarily. Moreover, the number of participants of a study may depend on the research design. For may also vary depending on the research design, instance, a researcher can formulate descriptive research problem, and research questions (Smith questions for ethnography, process-oriented et al., 2009). For example, the criteria for eligible questions for grounded theory, and meaning- participants in a phenomenological design dictate oriented ones for phenomenology. Likewise, that they have to have experience about the research research questions for a case study could be different, problem and the number cannot be fewer than 3 too. A case study commences when a specific case and more than 25. is selected. Sure, the aim to conduct a case study In this regard, there are some approaches matters as well. A case study can be administered to be employed during the planning process to describe a unique case or to comprehensively when selecting the participants of a study. understand a specific topic, problem, or issue. Accordingly, Creswell (2014) suggests criterion This case may be a tangible entity ranging from an sampling technique. If the number of people who individual to a small group of people and even to an have experienced the problem, topic, event, or institution or partnership. What is most significant phenomenon in question is high, then criterion here is whether this case in question can be defined sampling technique formed around some criteria and restricted with specific temporal and spatial could ease the work for the researcher. In such a parameters. Another example can be provided for situation, the criteria and the reasons as to choose grounded theory. Accordingly, baseline question them should be explained within the research. If for a grounded theory research has to be in the the research aims to determine the experiences of form of a why question rather than a simple yes/ a group, not individuals, then snowball sampling no format. In grounded theory, research questions can be utilized (Creswell, 2014). In snowball should be able to explain the phenomenon without sampling, the researcher finds one participant who any hypotheses and should never be derived from has experience about the topic, problem, event, available theories (through deduction). Besides, or phenomenon, and then reaches out to other research questions should direct the researcher participants through this participant. Homogenous to examine actions and the processes. So, “How sampling is another alternative technique that can

105 Qualitative Research Methods 4 be used in a qualitative research. If the aim of a research is to determine how standpoints about an Interview event or a phenomenon vary from one participant to another, and if the event or phenomenon is conceived to be influenced by an individual’s Participatory Artifacts personal features (such as political thoughts, socio- Observation economic status, etc.), then maximum variation Qualitative Data sampling could be the right technique to select Collection Techniques participants.

Data Collection Techniques Researcher Documents Throughout the chapter, we have mentioned Diary data collection techniques a few times while explaining the basic features of qualitative research and relevant designs. These techniques are given a Figure 4.2 Qualitative data collection techniques separate section to make them more tangible and to avoid repetitions. In general, data is defined as any measurement, value, phenomenon, or information Interview that can serve resolving a research problem. Data As a qualitative data collection tool, interview constitute the fundamental basis of any research as enables the researcher to gather data that observation they are “unprocessed evidence”, and they are used and questionnaire techniques cannot unravel and to understand, explain, provide experience, and to examine the participants’ feelings, opinions, formulate concepts-theories in qualitative research. perceptions, and attitudes. According to several Before moving any further about data sources, sources, some interview types include structured, the most significant point to remember about semi-structured, and unstructured interviews qualitative research is “triangulation”. Based on the (Merriam, 2013; Şimsek and Yıldırım, 2013). fact that a single data source can never be enough to answer all questions, various sources of data are consulted and they are triangulated to produce results, which is among the strengths of qualitative Interview is a data collection technique research (Bogdan and Biklen, 2007; Creswell, that enables a researcher to gather data that 2014). In this sense, researchers choose which data observation and questionnaire techniques collection techniques to employ in accordance cannot unravel and to examine participants’ with the focal problem, research aim, research feelings, opinions, perceptions, and attitudes. questions, and their own experiences in any given qualitative study. Available data collection techniques are explained in the following sub- titles. Figure 4.2 summarizes the data collection A closer look at interview types yields that techniques mentioned in this chapter. structured interview revolves around a series of questions in a specific order, and these questions are directed to each participant in the same order and In qualitative research, triangulation refers style. In other words, structured interview is face- to the use of more than one method in order to-face version of a questionnaire. Semi-structured to study, define, and explain the problem as interview, on the other hand, offers flexibility comprehensively as possible and to establish to the researcher in terms of improvising other necessary credibility (reliability) and validity. relevant or sub-questions during the interview in order to obtain details about the answers a participant provides. Unlike the former two types,

106 Research Methods 4 unstructured interview is based on natural flow of participatory observation. In a participatory interaction during which questions are raised, and observation process, one of the vital things for it is almost never predetermined where the course the researcher is to display culturally-accepted of conversation will steer during an unstructured behaviors during her/his interactions with the interview. In this type, the researcher does not members so that s/he will not influence the natural employ pre-written questions, and the interview is setting. One way of conducting observations based on exploration. is to examine and define behaviors in a specific Focal group interview should also be mentioned environment through videotape and/or detailed as one of the qualitative data collection techniques. notes. Use of videotape during observations offers Focal group interview is described as a data collection the opportunity to record research data on site, as it technique where a small group of participants is is displayed and experienced (Stake, 2010). interviewed about a topic that concerns them all with respect to their common opinions, ideas, Researcher Diary needs, and expectations. The advantage of this Researcher diary can be noted as the most technique is that data set proliferates quickly due important data source for a researcher administering to interaction among group members and group a qualitative research. Diary can be described as a dynamics. This type of interview is appropriate for source of data where the researcher writes down use with groups consisting of 6-12 people. her/his comments, feelings, and opinions about events that s/he observes and hears and where s/he Participatory Observation can add her/his reflections about her/his experience Observation means reviewing naturally- (Glesne, 2012). A researcher diary can be conceived emerging signs of an event/phenomenon as a written log of researcher’s actions, opinions, to comprehensively understand that event/ and feelings during the design, data collection, phenomenon. Another definition of observation analysis, reporting, and presentation steps of a refers to a detailed description of a behavior given research. occurring in a given setting or theory. Actually, observations cover a large portion of our daily lives. A significant majority of what we know depends Researcher diary is a source of data where the on observation, which makes participatory researcher writes down her/his comments, observation the most important data collection feelings, and opinions about events that s/he technique in qualitative research. In participatory observes and hears and where s/he can add observation, the researcher actively partakes in the her/his reflections about her/his experience. target setting or the daily routines of the target group via direct interaction with the members of that group instead of taking a distant stance. Researcher diary accommodates the interaction between subjective and objective dimensions of a research and puts these two into a productive Participatory observation is a data collection relation. It is possible to regard researcher diary as a technique that entails direct participation way to figure out the interaction emerging among into the target setting or the daily routines previous experiences, observations, readings, of the target group via interaction with the opinions, and elements concerning all the different members of that group. components of a research project (Glesne, 2012). For example, field notes can help the researcher notice points that s/he may miss afterwards. It is underlined as an effective data collection technique For instance, in an ethnography research, the because it draws attention on details and helps researcher can focus on events and phenomena creating patterns. Moreover, the diary can also be within the context to figure out local meanings utilized to explain other data sets collected during endemic to the members of a group during the research process.

107 Qualitative Research Methods 4 other hand, news videos, photographs, and movies can be grouped among audio and visual documents. internet It is possible to give more examples of such lists and classifications regarding use of documents Visit http://www.wordsinspace.net/course_ in qualitative research methods; yet, the answers material/mrm/mrmreadings/riadmIssue1.pdf for that a researcher gives to the questions concerning more information on researcher diary and field document types to be collected will serve as the notes among other data collection techniques. main criteria. What matters at this point is detailed description and deep understanding of focal events and phenomena to improve trustworthiness. Documents As a qualitative data collection technique, Artifacts documents refer to any kind of written, audio, and/or visual materials that bear information about Artifacts are another source of help qualitative the focal events or phenomena in a given research. researchers can use to answer the research Data distilled from social media applications such questions. They can also be described as physical as Facebook and Twitter should also be considered evidence for qualitative research (Merriam, among documents. Employing documents as a 2013; Onwuegbuzie, Leech, and Collins, 2010). data collection technique in a qualitative research Artifacts refer to object(s) located within the may produce the following advantages: focal context, and somehow, they appeal to the • Variety in data collection researcher’s interest. These objects may range • Aid to other data collection techniques from things that people in the focal setting use for • Quick access their daily routines and rituals to a book, chair, or • Direct focus onto the topic another piece of decoration in the environment • Low costs (Onwuegbuzie et al., 2010). Some examples of artifacts that can be used for a research within IR can be historical footsteps in a specific region, or ruins of a battlefield and a communal area. If the Documents refer to written, audio, and/or visual aim is to conduct a more contemporary research materials that bear infromation about the focal study, then some examples may include the goods events or phenomena in a given research. that refugees leave behind in their camps during migration. Artifacts that a researcher would utilize as a source of data depend primarily on the focal The documents that can be utilized in a phenomenon and the research questions. qualitative research are determined by the research aim, in other words, by the focal events and phenomena of that research. A researcher can be Artifacts are the objects within advised to ask the following questions about the the context of the focal topic, documents s/he is planning to collect during the event or phenomenon. process: • What kinds of documents do I need to accomplish my aim? • What kinds of documents should I collect Data Storage to support the data I’ve gathered so far? When conducting a research designed in • Will the documents support my accordance with qualitative research methods, explanations and descriptions? another point that should be taken into account Documents to be collected during a research fall together with the identification of data collection into either written or audio/visual materials group. techniques is how to store the data, which eases Written ones may include archive logs, letters, managing of a data set for a longitudinal study and reports, signature lists, and notebooks. On the establishing data variety. Furthermore, storing data

108 Research Methods 4 sources such as audio and visual records for later in- important depth analysis without backing them up digitally could lead to serious problems. Accordingly, Data analysis techniques vary across qualitative researchers should back up their data different qualitative research designs and control the changes in their database regardless and data collection techniques. of the research design they employ. The following can be advised about data storage and transfer: • Always back up the files in your computer. • Use a high-quality device to record the For narrative research design, on the other interviews. hand, Riessman’s (2005) classification is utilized • Develop a main list for different types of in general: thematic analysis, structural analysis, data. interactive analysis, and performative analysis. • Conceal the participants’ names in your According to this classification, in thematic analysis, data set for confidentiality reasons. a researcher gathers many narrations and concludes • Develop a data collection matrix. findings by grouping the data conceptually via an inductive approach. As for structural analysis, Data Analysis what matters more than narration is the form of narration, and it is a suitable approach to provide After a brief explanation on qualitative data details in case studies and to compare more than collection techniques, it is wise to underline one narration. Interactive analysis focuses on the that data collection, analysis, and interpretation dialogue between the narrator and listener. The should continue simultaneously throughout the focal point of a performative analysis is not the research process. Therefore, we should know that words spoken, but gestures, mimes, and moves qualitative research process is not one-way, rather accompanying narration, and the effort to convince it entails mutual interaction of data collection listeners. and analysis. To put it differently, a researcher holds participatory observations and makes daily The aim of data analysis in grounded theory notes to comprehensively understand the research research is to develop an inductive theory based problem; in the meantime, s/he conducts analyses on the data set. Accordingly, data is analyzed as to why and how those events occur based on the via a process called constant comparison (Glaser data collected until then. In addition, the same and Strauss, 2006), in which the researcher first researcher moves into a series of new observations divides the data set into manageable parts and then and analyses to check if her/his conclusions are compares each part with each other in terms of consistent with the reality. similarities and differences. Similar data is grouped under the same conceptual title. During the later There are two primary analysis approaches in stages of constant comparison analysis, concepts qualitative research, albeit many different names, developed by the researcher are re-grouped to which are inductive analysis and content analysis. form the categories. Each category is formulated Inductive analysis is labeled as thematic analysis according to its own features and sizes, and finally, in some sources. However, these two approaches several different categories are combined around branch into various data analyses depending on one nucleus (high level) category. the design and data collection tools of a qualitative research. For instance, phenomenological data analysis is the most frequently employed data Reporting analysis technique in phenomenological studies, The final step of a qualitative research process as suggested by Moustakas (1994). This approach is reporting, which includes the following sections: is of 5 stages which are (1) identification of theoretical background, method, findings, and significant remarks, (2) grouping common remarks, results. In reporting step, themes, categories, and (3) thematization of meaning clusters, (4) forming digital data are utilized; findings are related with structural and textural descriptions, and (5) the relevant literature; results are discussed, and combining structural and textural descriptions. suggestions are made (Creswell, 2014). In other

109 Qualitative Research Methods 4 words, the researcher explains what the findings mean to the readers. Van Manen (2014) states that reporting, unlike writing, is a reflection Rigor refers to assessing quality and accuracy process, and thus, the researcher should not only of research processes and conclusions. report the available data and findings, but also incorporate her/his feelings, opinions, intuitions, and interpretations. Besides, how research findings Rigor means care, attention, and diligence can be used or applied should also be explained displayed during every step of a research process in this step. The significance of suggestions in a where data about a topic is collected and analyzed, research report is that they pave the way for certain and findings are reported (Stringer, 2008). Rigor and practical approaches conducive to trial of new of a study is accepted as an indicator of its quality; ideas to tackle the focal problem (Creswell, 2014; thus, all necessary steps have to be completed and Mertler, 2014). reported meticulously (MacNaughton and Hughes, Although it is stated that research reports 2009; Yin, 2014). In this sense, the final part of the for different qualitative designs have some chapter is dedicated to concepts such as internal commonalities, it should be wisely noted that there validity, external validity, and credibility (internal are also some discrepancies. So, case studies can be and external) to make it more clear for you. the first design to start examining differences in First major criterion for rigor in qualitative report writing. In case studies, researchers have research is “internal validity” (trustworthiness), to organize themes and topics in a chronological which means questioning the true value of the order. In such a report, a cross-case analysis research for trustworthiness and to clarify the should be provided about the similarities and question “Does this research have a valid value?” differences across cases, or a theoretical model Answering the questions below could be advised should be presented. Case studies mostly end up to establish trustworthiness for a given research with conclusions, inferences, or a model built (Miles and Huberman, 1994, 28): by the researcher in accordance with the general • Are the findings of the research significant/ meaning that s/he distills from cases. In general, meaningful? a qualitative research report has to be clear and • Are the findings trustworthy for the coherent because it is the only way for researchers participants and readers? from other disciplines to understand that report. • e Is ther a genuine picture of the case we have been analyzing or trying to figure out? • Are the descriptions of cases comprehensive and meaningful enough? • Are the patterns built by the findings 3 coherent and holistic? Explain the steps of a • Do different data sources provide common qualitative research process. or overlapping findings and results? Another important criterion for rigor in qualitative research is “external validity” RIGOR IN QUALITATIVE (transferability). This refers to precise definition or explanation of the area that findings can be RESEARCH METHOD generalized to. It is not a generalization based on Rigor refers to assessing quality and accuracy of numbers as in quantitative studies, but an analytic research processes and conclusions (MacNaughton generalization in terms of similar units, places, or and Hughes, 2009). Rigor of a research involves cases (Yin, 2014). Miles and Huberman (1994) the concepts of credibility (reliability) and validity underline both transferability and suitability to (Lincoln and Guba, 1985). make an analytic generalization. Therefore, the researcher, knowing that the findings in the study cannot be directly generalized, has to provide the limitations as well as the other details, and s/he

110 Research Methods 4 has to add further explanations in research report of “non-local ethnography” among the samples of as to how the results can be transferred to other ethnography in IR area. At this point, we should individuals, units, places, or events. also know that qualitative research studies in IR Credibility (consistency) is another criterion are criticized in terms of systematy, consistency, for rigor. Credibility means that data collected and scientificalness. Nevertheless, such discussions during a research process will yield the same about use of qualitative research methods in the results if recollected and undergone the same field of IR can be taken as an opportunity to steps (Gagnon, 2010; Yin, 2014). Credibility contribute to the scientificalness of the field for the aspect involves “internal credibility” (consistency). production of more rigor studies. In this regard, Accordingly, concrete and clear explanations the best suggestion would be to establish standards should be provided, raw data should be stored, as to the rigor of qualitative research methods participants/members should be monitored during employed in IR. each step of the research, and data should be confirmed. “External credibility” (confirmability), on the other hand, refers to precise description of the roles that a researcher has throughout the entire internet process and detailed explanation of the participants and the context/social setting. Visit https://www.nsf.gov/sbe/ses/soc/ISSQR_ workshop_rpt.pdf for further details about establishing standards for qualitative research methods with an interdisciplinary view.

4 What are the criteria necessary for rigor in qualitative research methods?

In conclusion, it should be remembered that only preliminary information has been provided about a vast topic within this chapter where qualitative research methods are briefly explained. Basically, use of qualitative research methods within IR can furnish researchers with in-depth analysis, description, and understanding, and they can explain relativity and mobility of social phenomena. To put it differently, utilizing qualitative research methods in International Relations can enhance scientific knowledge with its micro analyses and by focusing on the background of events and phenomena, as underlined at the beginning of this chapter. On the other hand, there are many issues to be discussed and explained before making decisions about use of qualitative research method designs within IR. Throughout the chapter, clues have been provided as to whether qualitative research methods and relevant designs should be employed or not for the field of IR. For instance, it still does not seem possible to explain the complex situation caused by over-use

111 Qualitative Research Methods 4

LO 1 Define qualitative research methods

Qualitative research methods host a great number of concepts related with different disciplines. Several of these concepts include, but not limited to, ethnography, anthropology, situational research, interpretive research, natural research, and theory development. Since all these concepts are used with similar meanings in terms of research design and analysis techniques, the term ‘qualitative research methods’ is widely accepted as an umbrella term. Qualitative research is defined as a method where perceptions and events can be pictured in their natural environments with a holistic and realistic perspective through use of qualitative data collection techniques such as observation, interview, and document analysis. To further clarify, its basic features can be listed as follows: (1) sensitivity to natural environment, (2) participatory role of the researcher, (3) holistic approach, (4) identifying the perceptions, (5) flexibility in research design, and (6) inductive analysis. Summary Explain the contributions that qualitative research LO 2 methods can bring into international relations

Contributions that qualitative research methods can bring into the field of IR can be explained through three types of information (environment, process, perceptions) that can be gathered through use of these methods. First is the environmental information, which regards social, psychological, cultural, demographic, and physical characteristics. Such information may serve as a foundation for patterns concerning the process and perceptions, and collected information can be compared with that of other settings. The second contribution of qualitative research methods is the process. Such information highlights what happened during the research process and their reflections on the participants. Data concerning the perceptions that could be determined through qualitative studies depicts what participants think about the process. It should be noted that the focal event or phenomenon in qualitative research studies cannot be conceived out of its context and participants’ perceptions. Third significant contribution that qualitative research methods can introduce to IR is the functionality of information. In this regard, that the information distilled through qualitative research methods is based on facts can ease its reflection onto the field functionally, and this can facilitate taking timely precautions and producing more effective solutions for available problems.

LO 3 Explain the properties of qualitative research designs

Among the qualitative research designs that can be employed in IR are case study, ethnography, narrative inquiry, phenomenology, and grounded theory. Case study is the most frequently utilized design in the field of IR. Case study can be defined as a qualitative design where a researcher collects information about a focal event or phenomenon from various sources [e.g. observations, interviews, audio-visual materials (artifacts), documents, and reports] in a certain context for a certain time-period. Some of the advantages of case study design include lesser costs, process-tracing, in-depth analysis, and constant comparison. Ethnography is a research design where a researcher analyzes common behaviors, language, and action models of an intact culture within their natural environment during a long-time span. Narrative inquiry is another qualitative research design, which is defined as written documents and conversations about an event or a course of events that are chronologically related. Rooted in philosophy and psychology, phenomenology is a research design where participants’ experiences about a phenomenon defined by them are described. Grounded theory is another qualitative research design where a series of methods are systematically operated in order to develop an inductive theory.

112 Research Methods 4

LO 4 List the steps of conducting a qualitative research process

Despite various designs with their different features, it is still possible to list some basic steps to conduct a qualitative research study. In this sense, the steps in a qualitative research process can be summarized as identifying the research focus and problem, selecting participants, determining data collection techniques, data analysis, and reporting. In the first step, the research focus or problem has to be identified. Next, the aim of the research is defined together with the research questions to

be answered. Then, the participants are selected. Another major decision in a qualitative research Summary process regards the selection of appropriate data collection techniques. In this regard, some of the most common data collection techniques are interviews, participatory observation, researcher diary, artifacts, and documents, all of which should be considered to obtain data variety. In a qualitative research process, two basic data analysis approaches are inductive and content analyses. Final step of a qualitative research process is report writing where theoretical background, method, findings, and results are presented.

LO 5 List the criteria for rigor in qualitative research

Rigor means care, attention, and diligence displayed during every step of a research process where data about a topic is collected and analyzed, and findings are reported. First major criterion for rigor in qualitative research method is “internal validity” (trustworthiness), which means questioning the true value of the research for trustworthiness. Another important criterion for rigor in qualitative research method is “external validity” (transferability). This refers to precise definition or explanation of the area that findings can be generalized to. Credibility (consistency) is another criterion for rigor. Credibility means that data collected during a research process will yield the same results if recollected and undergone the same steps. Credibility aspect involves “internal credibility” (consistency). Accordingly, for consistency, concrete and clear explanations should be provided, raw data should be stored, participants/members should be monitored during each step of the research, and data should be confirmed. “External credibility” (confirmability), on the other hand, refers to precise description of the roles that a researcher has throughout the entire process and detailed explanation of the participants and the context/social setting.

113 Qualitative Research Methods 4

Events should be perceived in their natural 1 Which of the following states one of the settings as much as possible and only those findings 4 advantages that case study design offers in the field based on the facts within these settings should be of International Relations? reported. a. It has a structure that can resolve complex Which of the following properties of qualitative events in IR. research is indicated by the sentence above? b. It provides the opportunity to focus on marginal a. Sensitivity to natural setting phenomena in IR. b. Researcher’s participatory role c. It has a flexible structure conducive to examining c. Examination of participants’ perceptions all the sub-fields of IR. d. Description of participants’ experiences d. It has the potential to eliminate unresolved e. Flexibility in research design issues in IR. Test Yourself Test e. It provides the opportunity to conduct a research in IR in a short time. 2 One of the most important features of qualitative research methods is that participants’ perceptions and experiences are revealed. So, 5 It is a research design where the researcher which of the following is vital for a researcher when examines common behaviors, language, and action conducting a qualitative research study? models of a culture in its natural environment for a long-time span. a. S/he should have a flexible standpoint. b. S/he should collect realistic data about the To which of the following does the definition abo- context. ve refer? c. S/he should be experienced about qualitative a. Phenomenology research methods. b. Case Study d. S/he should employ various data collection c. Narrative Inquiry techniques compatible with the research aim. d. Ethnography e. S/he should spend time with the participants in e. Grounded Theory the research context.

6 I. Temporal context 3 Which of the following is not among the contributions that qualitative research methods II. Number of participants can bring into the field of International Relations? III. Social context a. Obtaining in-depth information about a . IV Width of the phenomenon phenomenon b. Comprehensive understanding of a phenomenon . V Spatial context c. Complete description of a phenomenon with its Which of the above are significant components of all aspects a narrative inquiry? d. Reaching to universal knowledge based on one a. I and II phenomenon b. II and IV e. Approaching a phenomenon holistically c. I, III and IV d. I, III, and V e. II, IV, and V

114 Research Methods 4

7 Which of the following is true about the 9 I. Data analysis significance of phenomenological design? II. Identifying the research focus and a. It is difficult to determine why individuals problem perform certain behaviors, and it is necessary to III. Reporting

work on a large sample to identify their relevant Test Yourself motives. . IV Selecting participants b. It is important to focus on the consequences . V Data collection rather than the behaviors individuals display, and problems could be determined by examining Which of the following is the correct representation the consequences. of steps to be followed to conduct a qualitative c. Sometimes, there is no real rational for the research? behaviors individuals display; therefore, the a. I, II, III, IV, and V focus should be on cause-effect relations rather b. I, IV, V, II, and III than motives. c. II, I, IV, III, and V d. Sometimes, there is no real rational for the d. II, IV, V, I, and III behaviors individuals display; therefore, the e. III, I, II, V, and IV focus should be on motives rather than cause- effect relations. e. Individuals perform their behaviors via 10 Which of the following definitions is true for influence of things perceived to be real, and rigor in qualitative research methods? their perceptions should be determined to a. Assessment regarding the consistency of research understand their relevant motives. process, obtained results, and suggestions b. Assessment regarding the quality and correctness 8 Which of the following is the most of research process and obtained results distinguishing feature of grounded theory among c. Assessment regarding the functionality and other qualitative research designs? reflection of research process and obtained a. The esearcherr analyzes the preliminary data results onto the field and bases collection of further data onto this d. Assessment regarding reporting and analysis. dissemination of research process and obtained b. The esearcherr focuses on participants’ results perceptions and tries to comprehensively e. Assessment regarding generalization and understand these perceptions during the noticeability of research process and obtained research process. results c. The esearcherr collects real data about the focal context and makes inferences based on this data set. d. The esearcherr collects various data about the focal phenomenon and tries to comprehensively understand the phenomenon in this way. e. The esearcherr employs inductive approach during data analysis, and hence develops a holistic standpoint.

115 Qualitative Research Methods 4

If your answer is incorrect, review the section If your answer is incorrect, review the section 1. a 6. d on “Basic Features of Qualitative Methods”. on “Narrative Inquiry”.

If your answer is incorrect, review the section 2. e 7. e If your answer is incorrect, review the section on “Basic Features of Qualitative Methods”. on “Phenomenology”.

If your answer is incorrect, review the section 3. d 8. a If your answer is incorrect, review the section on “Possible Contributions of Qualitative on “Grounded Theory”. Research Methods”.

If your answer is incorrect, review the section 4. c If your answer is incorrect, review the section 9. d on “Conducting a Qualitative Research on “Case Study”. Process”.

If your answer is incorrect, review the section 5. d If your answer is incorrect, review the section 10. b on “Rigor(ism) in Qualitative Research on “Ethnography”. Methods”. Answer Key for “Test Yourself” for “Test Key Answer

What kind of scientific information can be obtained with a qualitative study focusing on the migration of Syrian refugees to EU member countries?

In line with qualitative research methods’ sensitivity to natural settings, it is possible to deduce that the migration of Syrian refugees to EU member countries can be comprehensively examined and understood. Because s/ he will join the context personally, the researcher can easily focus on the experience and unravel the perceptions of other participants. So, the your turn 1 researcher will be able to handle the phenomenon with a holistic approach. Employing inductive approach for the analyses, s/he can portray a detailed picture based on the natural environment. Findings of such a research effort can functionally guide EU member countries about the improvements they should make in their sense of human rights and about new policies they should develop. Suggested answers for “Your turn” Suggested answers for “Your

116 Research Methods 4

What kind of advantages do you think case study design bears to explain a phenomenon of your own interest? Search about it. Suggested answers for “Your turn”

It is known that case study design is widely utilized in the field of IR and this tendency is increasing seriously. The reason for this can be attributed to the drive to comprehensively understand complex social events and phenomena in IR. As you remember, case study is defined as a qualitative research design where a researcher collects data about specific events or phenomena from your turn 2 various sources of information (e.g. observations, interviews, audio-visual materials, documents, and reports, etc.) in a given context for a certain period of time. The advantages of case study for the field of IR include lesser costs, process-tracing, in-depth analysis, and constant comparison.

Explain the steps of a qualitative research process

Fundamental steps of conducting a qualitative research are identifying the research focus and problem, selecting the participants, data collection techniques, data analysis, and reporting. Accordingly, the focal phenomenon or problem has to be determined in the first step. Then, the research aim is established together with the research questions to be answered. Next your turn 3 important step is the selection of participant(s). Afterwards, data collection techniques have to be specified. Primary data collection techniques are interview, participatory observation, researcher diary, artifacts, and documents. Inductive analysis and content analysis are two main data analysis approaches in qualitative research methods. Reporting is the final step of conducting a qualitative research study.

What are the criteria necessary for rigor in qualitative research methods?

Rigor of a study encompasses the concepts of validity and credibility. To make it more tangible, the concepts of internal validity, external validity, credibility (internal and external) should also be taken into consideration. The first criterion for rigor in qualitative research is “internal validity” (trustworthiness). Another significant criterion for rigor is “external validity” (transferability). Credibility (consistency) should be noted as another criterion for rigor in qualitative research efforts. “Internal credibility” (consistency) is a your turn 4 major component of credibility. Accordingly, concrete and clear explanations should be provided, raw data should be stored, participants/members should be monitored during each step of the research, and data should be confirmed for a solid internal credibility. “External credibility” (confirmability), on the other hand, refers to precise description of the roles that a researcher has throughout the entire process and detailed explanation of the participants and the context/social setting.

117 Qualitative Research Methods 4

Further Reading

Alaszewski, A. (2006). Using diaries for social research. Mahoney, J. (2007). Qualitative methodology and London: Sage. comparative politics. Comparative Political Studies, 40, 122–44. Eckstein, H. (2009). Case study and theory in political science. In F. Greenstein, & N. Polsby Moravcsik, A. (2014). Transparency: The revolution (Eds). Handbook of poltical science. Reading, in qualitative research. Political Science and Massachusetts, Addison-Wesley. Politics, 47, 48–53. Harrison, L. & Callan, T. (2013). Key concept in Patton, M. Q. (2015). Qualitative evaluation and politics and international relations. London: Sage. research methods. Thousand Oaks, CA: Sage. Harvey, F. P., & Brecher, M. (2002). Evaluating Todd N. T. (2016). Grounded theory generation: methodology in international studies. Ann Arbor: A tool for transparent concept development, University of Michigan Press. International Studies Perspectives, 17(4), 426–438. Lupia, A. & Elman, C. (2014). Openness in political Van Evera, S. (1997). Guides to methods for students science: Data access and research transparency. of political sciences. Cornell University Press, New American Political Science Association, 19-42. York. Retrived from https://www.researchgate.net/ publication/273202564.

References

Barkhuizen, G. P. (2008). A narrative approach to Gebhardt, G. F., Carpenter, S. G., & Sherry Jr., exploring context in language teaching. English J. F. (2006). Creating a market orientation: A Language Teaching Journal, 62(3), 231-239. longitudinal, multifirm, grounded analysis of cultural transformation. Journal of Marketing, 70, Bennett, A. & Elman, C. (2007). Qualitative methods 37-55. the view from the subfields. Comparative Political Studies, 40(2), 111-121. Gertsen, M. C., & Søderberg, A. M. (2011). Intercultural collaboration stories: On narrative Birks, M., & Mills, J. (2015). Grounded theory: A inquiry and analysis as tools for research in practical guide. Los Angeles, CA: Sage. international business. Journal of International Bogdan, R. C. & Biklen, S. K. (2007). Qualitative Business Studies, 42, 787–804. research for education: An introduction to theory and Giorgi, A. (2009). The descriptive phenomenological methods (5th ed.). Allyn & Bacon, Boston. method in psychology: A modified Husserlian Clandinin, D. J., & Connelly, F. M. (2000). Narrative approach. Pittsburgh, PA: Duquesne. inquiry: Experience and story in qualitative research Giovannoli, R. (2013). The narrative method of inquiry. San Francisco: Jossey-Bass. Retrieved from http://www.sonic.net/~rgiovan/ Corbin, J. & Strauss, A. L. (2008). Basics of qualitative essay.2.PDF research: Techniques and procedures for developing Glaser, B. (2002). Conceptualization: On theory and grounded theory (3rd ed.). Thousand Oaks, CA: theorizing using grounded theory. International Sage. Journal of Qualitative Methods, 1 (2). Article 3. Creswell, J. W. (2014). Research design: Qualitative, Retrieved from http://www.ualberta.ca/~ijqm/ quantitative and mixed methods approaches (4th Glaser, B. G., & Strauss, A. L. (1967). The discovery of ed.). Thousand Oaks, CA: Sage. grounded theory: Strategies for qualitative research. Czarniawska, B. (2004). Narratives in social science USA: Aldine Transaction. research. introducing qualitative methods. London: Glaser, B. G., & Strauss, A. L. (2006). The discovery of Sage. grounded theory: Strategies for qualitative (4th ed.). USA: Aldine Transaction.

118 Research Methods 4 Glesne, C. (2012). Becoming qualitative researchers: Mertler, C.A. (2014). Action research: Improving schools An introduction (3rd ed.). New York, NY: Pearson and empowering educators (4th ed.). Thousand Education, Inc. Oaks: Sage. Gray, D. E. (2009). Doing research in the real world Miles, M, B., & Huberman, A. M. (1994). Qualitative (2nd ed.). Thousand Oaks, CA: Sage. data analysis: An expanded Sourcebook. (2nd ed). Thousand Oaks, CA: Sage. Josselson, R. (2007). The ethical attitude in narrative research: Principles and practicalities. In D. J. Moustakas, C. (1994). Phenomenological research Clandinin (Ed.), The handbook of narrative inquiry methods. Thousand Oaks, CA: Sage. (pp. 537-567). Thousand Oaks, CA: Sage. Onwuegbuzie, A., Leech, N. L., & Collins, K. M. T. Kholmi, M., Triyuwono, I., Purnomosidhi, B., & (2010). Innovative data collection strategies in Ganis-Sukoharsono, E. (2015). Phenomenology qualitative research. The Qualitative Report, 15(3), study: Accountability of a political party in the 696-726. context of local election. Procedia-Social and Riessman, C. K. (2005). Narrative analysis. Behavioral Sciences, 211, 731 – 737. Narrative, Memory & Everyday Life, University of Klotz, A. (2008). Introduction. In A. Klotz & D. Huddersfield, Huddersfield. pp. 1-7. Retrieved Prakash (Eds.) Qualitative methods in international from http://eprints.hud.ac.uk/id/eprint/4920/ relations a pluralist guide. London: Palgrave Şimsek, H., & Yıldırım, A. (2013). Sosyal bilimlerde Macmillan. nitel araştırma yöntemleri [Qualitative methods in Lamont, C. (2015). Research methods in International social science]. Ankara: Seckin. Relations. London: Sage. Smith, J. A., Flowers, P., & Larkin, M. (2009). Lester, S. (1999). An introduction to phenomenological Interpretative phenomenological analysis. London: research. Stan Lester Developments, Taunton. Sage. Retrieved from http://www.sld.demon.co.uk/ Stake, R. E. (2010). Qualitative research: Studying how resmethy.pdf things work. New York, NY: Guilford Press. Levy, J. S. (2002). Qualitative methods in international Stepputat, F. (2012). Knowledge production in the relations. In P. Frank, F. P. Harvey, & M. Brecher security–development nexus: An ethnographic (Eds). Evaluating methodology in international reflection. Security Dialogue 43(5) 439–455. studies (pp. 111-160). University of Michigan Press. Swift, D., J. (2009). A grounded theory approach Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic to analyzing political narratives. Unbuplished inquiry. Thousand Oaks, CA: Sage. Master Thesis, Faculty of the Graduate School of Lopez, K. A., & Willis, D. G. (2004). Descriptive Vanderbilt University. Retrieved from https://etd. versus ınterpretive phenomenology: Their library.vanderbilt.edu//available/etd-03242009- contributions to nursing knowledge. Qualitative 154136/unrestricted/Swift_Thesis_Approved_ Health Research, 14(5), 726-735. Dokecki_Speer.pdf Lugalla, T. (2016). Effects of political, legal, and Van Manen, M. (2014) Phenomenology of Practice: governance challenges: Case study of East African Meaning-giving methods in phenomenological community. Master’s thesis, Harvard Extension research and writing. California: Left Coast Press. School. Vrasti, W. (2008). The strange case of ethnography and MacNaughton, G., & Hughes, P. (2009). Doing International Relations. Journal of International action research in early childhood studies: A step by Studies, 37(2), 279–301. step guide. UK: Open University Press. Wiener, A. (2009). Enacting meaning-in-use: Qualitative Maoz, Z. (2002). Case study methodology in research on norms and international relations. Internatıonal Studies: From Storytelling to Review of International Studies, 35, 175–193. Hypothesis Testing. In P. Frank, F. P. Harvey, Wolcott, H. F. (2008). Ethnography: a way of seeing. & M. Brecher (Eds). Evaluating methodology in NY: Altamira Press. international studies (pp. 161-170). University of Michigan Press. Woodside, A. G. (2010). Case study research: Theory, methods, practice. (1st ed.). UK: Emarld. Merriam, S. B. (2013). Qualitative research. A guide to design and implementation, (2nd ed.). San Yin, R. K. (2014). Case study research design and Francisco, CA: Jossey-Bass. methods (5th ed.). Thousand Oaks, CA: Sage.

119 Chapter 5 Quantitative Methods After completing this chapter, you will be able to:

Identify basic concepts that relate to basic Explain the role of reliability and validity in 1 quantitative research method. 2 quantitative research. Develop basic skills in quantitative research so that you will be able to comprehend published Have a critical view of when and how to 3 quantitative research. 4 employ quantitative research.

Chapter Outline Key Terms • Census Learning Outcomes Introduction Why Do We Use Quantitative Methods? • Data collection Descriptive Research • Data analysis Associational Research • Generalizability Experimental Research • Hypotheses Survey Research • Population Preparing Good Survey Instruments • Quantitative research Basics of Doing Research • Research questions Census, Population, and Sampling • Reliability • Sample • Validity

120 Research Methods 5 INTRODUCTION quantitative methods to sample data from the entire There exists a wide spectrum of research and population also created a lot of controversy, as in the research approaches in the field of political science, example of the 1990 Census which provoked a major international relations, and political history, and lawsuit in the US for not including the differentiated other fields as part of social sciences disciplines. As a poor living in the inner cities and the loss of their student in the field of political science research either political presentation. Further debates emerged in international relations, public policy or other at that time, in which the metaphor “statistical social science research, you must have witnessed citizenship” (Igo, 2007, p. 297) for representations that the society is exposed to numerous tables, in the society became popular. Unintentially charts or figures of election polls, opinion data in perhaps, but the discussions over the years of census the printed or broadcasted media. This chapter yielded into greater public protests and rallies for the presents a brief overview of the fundamentals of representation of biracial or multiracial citizens in quantitative research for beginning researchers. the upcoming decade (Igo, 2007). Given there is recognizable erasing amount of qualitative and mixed method studies, especially, in political sciences, international relations, and Census is data collected from every member political history research, these fields have been in the society by governments for official converted into a quantitative methodology over the purposes and could vary from general past four decades (Brown & Hale, 2014).Mansfield demographic information such as sex, age, (2002) reviewed research on international policy, education, place of birth to questions about economy, and political sciences between 1970 households. If the entire population is and 1999 in seven top journals in those areas. surveyed, meaning it is a census representing Although the studies examined differed across many 100% the population. dimensions, including the amount of quantitative research published, the frequency with which they published studies of international political economy, Briefly, this chapter is organized to help international relations or political science, and the reader understand the fundamentals of identified that in the 1970s, 20% of the relevant quantitative research and guide them to read and articles contained statistical evidence; during the understand research papers, and also to facilitate 1980s 25%; and during the 1990s, quantitative their preliminary decisions for planning for a evidence in articles raised to about 45% respectively, basic quantitative research by providing them which indicates a paradigm shift in the dynamics of with a rich collection of examples extracted from political sciences research. numerous international research studies in the field of political or other social sciences. In addition, the Lamont (2015) claims that applications of chapter also presents the basics of doing research quantitative methods in the field of international such as asking research questions or formulating relations has impacted largely the direction of North hypothesis, sampling procedures, decisions for data American political science. Similarly, Herbst (1993) collection and data analysis procedures. Although claims the same and argues that quantitative data data analysis procedures are included, it is essential has shaped public policy in the political history to underline, that the current chapter provides of the United States as quantitative measurement the reader only with a brief overview of basic or tools have proven to be enormously useful for descriptive data analysis methods rather than social scientists in offering open ways of explaining equipping the reader with the core competences in human behavior. Quantitative methods enable the statistical analysis. In that regard, the readers are researcher to collect demographic data, to quantify suggested to equip themselves with more complex and categorize behavioral acts, and collecting of or advanced inferential statistics in case they want census data. Nevertheless, the author warns that to plan or conduct more powerful quantitative it is also important to see how these measures are research for their future research aspirations. arrived at and who controls the communication of the quantitative data collected. For instance, using

121 Quantitative Methods 5 have high explanatory power and that have survived from all the tests on refutation then the levels of knowledge and experience can be confirmed 1 (Garratt & Li, 2005). In brief, Popper implied that As a political scientist examine the censuses research questions should be formulated as such taken place over the years in your country, that they are “theoretically aware and empirically have a look at how census data has been testable” (Lamont, 2015, p.22). collected over the years, and what elements Quantitative research, as part of empirical or demographics of every member in the research, enables to describe populations and society were questioned. Evaluate what phenomena in a detailed tradition based on figures purpose each category questioned serves for. or data, and enables the political scientist to make propositions out of relationships between two or more variables. Observation and experimentation are the WHY DO WE USE strategies that can be employed to test an assumption QUANTITATIVE METHODS? or a theory. Observation is the most used strategy by political scientists for international relations, or As the title indicates it is essential to answer what other social sciences while experimentation is not quantitative methods refer to and to comprehend that commonly used due to its nature of testing the underlying purpose of using them in research. (Lamont, 2015). For instance, in experimentation if Quantitative research, or also called scientific the political scientist is interested in understanding research method, is a way of obtaining information the relationship of a political theory and humans that provides the researcher with accurate and social well-being, there needs to be two equivalent reliable data that is numeric in nature and has been groups to understand the relationship or to verify obtained through testing or comparing ideas in the the theory. Consequently, experimental research public arena. As human beings we are opt to making earns to be explained thoroughly and rigorously connections among the sensory information we due to its complex nature of working with humans experience around us and interpret them (Fraenkel and the many social, cultural, anthropological, & Wallen, 2010). Quantitative research is based economic, political or historical factors that need to on observation and the measurement of repeated be considered in designing experimental research for incidences of a political phenomenon. These could social sciences. be measurement of the voting act for a political party over the years, or it could be about collecting data about the attitudes of the public toward the DESCRIPTIVE RESEARCH taxation policy of the government (John, 2010). As its meaning may indicate, descriptive studies The foundation of modern quantitative method are about describing a particular situation or case as comes from the positivist tradition, in which detailed and elaborated as possible, and it enables philosophers from the mid-1950s argue that no the classification and organization of taxonomic proposition could totally be verified through categories as in survey methods. Political scientists observation or in the absence of an external referent want to analyze whole populations, because with that might resolve the claims of truth. Popper large numbers the social scientists can make appeared with his own unique thesis and in contrast generalizations about the empirical world. The to his predecessors he highlighted that while larger the cases or subjects are, the more accurately it statements about the world can never be completely will be possible for the scientists to make inferences verified, they could be proven through scientific about the phenomenon they observe (John, 2010). inquiry (Lamont, 2015). To do that Popper In other words, information is collected so that the suggested to privilege falsification over verification social scientist can describe the characteristics of based on the idea that no matter how favorable the population that they are interested in such as the evidence may support a proposition, there can understanding the voting behaviors of the public never be enough evidence to completely verify any for a political party. conclusion. He also underscored that theories that

122 Research Methods 5 Characteristics of the data collected may vary ASSOCIATIONAL RESEARCH from opinions, attitudes, views to different beliefs Associational research enables for understanding and knowledge. Especially, during elections periods, or identification of relations between certain there are many campaigns run to understand the variables (Frankel & Walden, 2006). For a political expectations and attitudes of citizens from different scientist, for example, the researcher may like political party backgrounds, which is very particular to learn in a study related to gender issues, if to descriptive research. However, the researcher compulsory pre-school education in villages may needs to keep in mind that we cannot reach the have an impact on girls’ higher participation in entire population to collect information due to time higher education. To understand health policies, a and monetary considerations. Researchers rather politician may want to see the relation between milk select a representative sample , etither randomly consumption, and children’s bone development or through different sampling techniques, that in underdeveloped regions to reconsider the represent the population that the researcher wants conditional cash transfers to the rural poor. In such to describe. Actually, descriptive research in a way cases the researcher can make statistical analysis provides the researcher with opportunities to utilize to predict the relationships through correlational statistical analysis at a basic descriptive level. Such or causal-comparative methods, or make use of basic data enables the researcher to have a general available large datasets that are accessible. opinion of the phenomenon to being examined. It enables the researcher to utilize large sets of data to describe or summarize data and to look at average Correlational Research measures, which later can yield to more complex or Correlational studies enable the researcher to inferential statistics to see associations or relations identify how one or a set of variables are related within the population (Lewin, 2005). Ultimately, to another or enable the researcher to make the collection of sample size, means, and standard predictions. Especially in the case where it is difficult deviations may facilitate our understanding of to conduct an experimental study, the researcher the profiles of the population under study and can make use of the available data to analyze the explain their relationship with certain attitudes and relationships among different variables (McNabb, behaviors. For instance, calculation of descriptive 2004). Correlational research contradicts with statistics can be facilitated through special software experimental research in the sense that no variable packages that exist such as Microsoft Excel or SPSS is manipulated in this method. Thus, it is important that can compute large sets of descriptive data to say that correlation should not be confused with into condensed or figurative forms through tables, causal relation, it only provides an existing or non- pie charts or plot tables in a second. Becoming existing relationship between variables. user friendly with such software packages can yield pathways to conduct more complex analysis important such as associational research to understand the phenomenon researched. Associational research should not be confused with causal relation because it only important provides an explanation between existing or non-existing relationships between variables. Population or sample, which can also be called as the unit of analysis, define the group that your research will be about. The Hypothetically let us assume that we are interested more descriptive information you provide in exploring if there is a correlation between university about the sample, the more likely it will graduation and employability.In such a situation represent the population investigated. the researcher can make use of official census data Therefore, decisions about the sample is to see the correlation. Using scatterplots enable the counted as one of the most important steps researcher to see the regression line and interpret in the research process. the correlation between the variables of interest. For

123 Quantitative Methods 5 example, based on a hypothetical data Achievement Observed we found a correlation between students’ 10,00 Linear reading number of books and GPA scores. In the case the regression line is 8,00 approaching -1,00 or + 1,00 indicating there is a negative correlation for the former and a positive correlation for the 6,00 latter. In other words, if it is close to 1 there is a positive correlation between the number of books read and GPA scores. 4,00 If the correlation of the regression line shows close to number 0 zero, we state 2,00 that there is no correlation between the variables. Some helpful means that visualize the results of causational research 0,0 1,00 2,00 3,00 4,00 5,00 such as scatterplots can be facilitating Books tool to make meaning of the plot and Graph 5.1 Scatterplot indicating a positive correlation between consider implications for other research number of books read and achievement scores and practice. (see Graph 5.1) Source: Hypothetical data computed on SPSS 23, METU version. Causal-Comparative Research Causal comparative research which is also called as ex post facto (after the fact for Latin) attempts to determine the cause or consequences of differences that already exist between identified set of subjects (Fraenkel & Wallen, 2010). An example research study could be that of Weinschenk and Dawes (2017). They were interested in examining the roots of individuals’ politial engagement and explored if biological and psychological factors play a role in the formation of politcal interest.They employed two datasets on twins, where data sampling procedures happened in two phases. Part one data was a representative national sample of approximately fifty thousand households with the presence of a twin. Those who reported the presence of a twin in the family were then asked if they would participate in the MIDUS study, and 60% gave permission to contact. All respondents were invited to participate in a phone interview and to complete two self-administered surveys. The second dataset, The Minnesota Twins Political Survey, was a dataset that was collected based on a sample of twins from the Minnesota Twin Family Registry at that time. The registry contained 8000 pairs of twins born between 1936 and 1955 in Minnesota. The Minnesota Twins Political Survey data were collected using a web survey in 2008, followed by a paper-and-pencil survey collected in 2009. The survey contained a selection of rating items such as “Please rate how much obligation you would feel if the following happened to you, using a 0 to 10 scale where 0 means no obligation at all and 10 means a very great obligation.” To measure personality traits, they made use of a series of adjective-based ratings to which the participants needed to rate themselves upon given statements such as, “Please indicate how well each of the following describes you,” with the response categories being; a lot, some, little, and not at all. The adjectives were as follows: for extraversion, outgoing, friendly, lively, active, talkative, dominant, self- confident, assertive, forceful, and outspoken, for emotional stability moody, worrying, nervous, and calm. Analysis of the two datasets, enabled the researchers to find, evidence that genetic factors account for a large amount of correlation between political interest and personality traits. Although this study provides a more nuanced picture of the biological and psychological bases of political orientations (Weinschenk and Dawes, 2017), the impact of the environment or social and cultural context is worth further examining.

internet Source accessible from library databases or http://www.nyu.edu/projects/dawes/PoliticalInterest.pdf

124 Research Methods 5

2 Open Access: http://www.nyu.edu/projects/dawes/PoliticalInterest.pdf Reading the full article “The Relationship between Genes, Personality Traits, and Political” by Aaron C. Weinschenk and Christopher T. Dawes (2017). Answer the following questions: 1. Which panel data on political interest correlates highest with the one of the five traits? Please, explain.

Source: Table copied from the Open access article retrieved from http://www.nyu.edu/projects/dawes/PoliticalInterest. pdf, page 472 in Weinschenk and Dawes (2017).

EXPERIMENTAL RESEARCH As indicated earlier, experimental research is unique yet not so common in political research as the nature of experimental research demands manupulating of the independent variable in the experiment. More explicitly, it is a research type that explicitly attempts to influence a particular variable. Second, it is considered to be the best method to test hypothesis related to cause-and-effect relationships (Fraenkel & Wallen, 2010). Nevertheless, policy studies mostly cover huge sample sizes and it is very difficult to isolate participants from their social or cultural contexts to and it becomes hard for a social researcher to interpret if the results are related vis-á-vis to the effect of the treatment or experiment or if there are other intruding aspects such as time or location on the outcomes of the study. In addition, ethics is of utmost consideration because any research that deals with humans as subjects, it is important to receive consent from an official research agency (e.g., Human Research Ethics Committee) and thereafter from the individuals themselves. An interesting experimental research in political sciences could be Weber’s (2012) study who examined the political effects of several emotions based on emotional ads exposed to subjects (the volunteer cases) to see if those ads influence the voters and become politically active. Two type of tests (or instruments) were used for data collection. The first test examined emotional responses to campaign messages; the second examined whether emotions influenced political participation. Subjects were recruited from two universities: a large northeastern university and a large southern university. Subjects for the two experiments consisted of N = 287 subjects for the former and N = 274 subjects for the latter university. The treatment was as follows; on entering the lab, participants were seated at a computer terminal and instructed that they were to view a campaign ad followed with a short survey. Participants were asked to watch and react to a political web

125 Quantitative Methods 5 advertisement released from a previous congressional or questionnaire that complements data collection election. The stimulus materials consisted of videos parallel to your research questions or hypotheses created using professional video-editing equipment. formulated and former literature review. Following exposure to the ad, participants were asked a series of questions about their emotional state and their intention to participate in politics. Both experiments followed a similar logic, so they Empirical research is about any research were explained together. The results indicate anger that is based on observational data is mobilizing by increasing participatory intentions indicating that knowledge is derived from and factors related to participate. This result is then actual experience or experiment rather than replicated using ad-tracking data. The findings from a certain theory or belief. indicated that emotions are an important factor in studying campaign effects, ultimately effective tools to manipulate the voters’ actions. In the case of a need for self-constructed survey, the researcher should initiate with making decision In addition to the above research, experimental for the research problem and review the literature research in different areas may enable the thoroughly, so that he or she can verify the gap or researcher to employ different methods such as a need to study the phenomenon and provide a the Interrupted Time-Series design in the fied good rationale including operational definitions of of Public Administration (Schwester, 2015). For the concepts for the research topic. Consecutively, instance, in a given policy the policy analysist may the researcher should start to formulate the research see the effects of non-smoking policy in buildings question or hypothesis, and define the concepts that on staff health outcomes based on data collected need to be measured in addition to making decisions before the policy and after some interval time after for how to sample subjects from the target population the policy has been implemented. so that the findings can be generalized. Ultimately, the researcher may select an available validated and SURVEY RESEARCH reliable instrument or develop a data collection There is an increasing interest in utilizing survey instrument. It is important that any quantitative research in the field of political sciences as it provides research needs to include a set of information opportunities to build theory through empirical about the subjects (or cases) to make meaning of data via understanding the attitudes, behavior, the numeric findings. In other words, there should and dynamics in the society (Butt, Widdop, & be initial questions to identify certain background Winstone, 2016). It has become a common method information to have an opinion of subjects that of collecting descriptive data, and it also enables represents the population. For instance, the number the social scientist to manipulate data and do more of mothers and fathers with illiterate, middle school complex analysis to understand the relationship or high school degrees. Or using demographic among variables through associational research. data and descriptive data to help the international There are different uses of survey data. Some relations policy analysist understand and explain the ready-made data can be one of the options of study on what percentage of educated women versus analyzing the existing datasets which are accessible men are in favor of supporting the European Union for free download and to be used for statistical candidacy of their country. analysis. Such data could be that of large datasets There are different types of surveys, therefore, as shared by the OECD, for instance, for the case the researcher you need to relate your survey design of international PISA exam data. Other could be to your research questions. Butt et al. (2016) provide data collected by the researchers themselves via a list of survey types that can be utilized in political researcher-made surveys or questionnaires, e.g., science research such as cross-national surveys, Bayırbağ (2003). Especially, if the phenomenon is national time series, election surveys, and panel unique, it has been not or little examined and needs surveys. The forthcoming section includes one of the to be further researched in such occasions. The best most common survey methods, the ‘cross-sectional strategy is to develop your own survey instrument survey’, and proceeds with the ones listed above.

126 Research Methods 5 Cross-Sectional Surveys A popular strategy for doing survey research is cross-sectional research, which means collecting data through a survey across a pretermined population at one point in time (Fraenkel and Wallen, 2010). Thus, the survey is administered to a representative sample at the same time. To exemplify this, let us examine the following study. In a nation-wide study, Şen and Akar (2017) attempted to examine the impact of interregional and urban–rural population movements on schools located in areas subjected to high in- migration and outmigration flows in Turkey based on data collected from primary school supervisors (N=150). A cross-sectional survey design was utilized to examine the most pressing problems confronted in those schools, and possible actions that could be taken by various stakeholders were sought. Overall data suggested that those schools serve mostly disadvantaged populations with poor households. Among the main challenges revealed are lack of adequate school resources and poor facilities, problems caused due to ineffective and untimely implementation of the program, high personnel turnover and recruitment of largely inexperienced and relatively less qualified educational staff, and lower levels of parental involvement in children’s schooling. Accordingly, proposed recommendations focused on measures to eliminate capacity and resource constraints, improve hiring practices, and secure adequate supervision and support for educational staff, meet academic and socio-emotional needs of students and facilitate parental involvement in the education of children. Supervisor profiles reflected the following figures in the table below. Looking at the figures, could you critically describe supervisor profiles in the selected schools located in poor neighborhoods? (see Table 5.1)

Table 5.1 Demographic characteristics of survey respondents (N=150) N % Gender Female 9 6.0 Male 139 92.66

Highest degree attained Master degree 97 64.66 PhD degree 4 2.66

Years of experience Years of professional experience 14.6 14.50 Years of working in the same place of employment 13.5 10.12

Cross-National Surveys Cross-national surveys provide opportunities to collect data that is administered in various countries so that the researchers can collect data regarding how knowledge, attitudes, and behavior many vary according to difference in democracy, culture, economic conditions through comparisons. A popular recent cross- national study is the 2016 International Civic and Citizenship Education Study (known for ICCS 2016). The ‘2016 International Civic and Citizenship Education’ Study is a cross-national study that included 24 countries in Asia, Europe and Latin America. The study examined how young people are prepared to undertake their roles as citizens in the second decade of the 21st century. Also, it explores the enduring and emerging challenges of educating young people in a world where contexts of democracy and civic participation continue to change For instance, the increase in the use of social media for civic engagement, growing concerns about global threats and sustainable development, as well as the role of schools in fostering civic engagement and ways of interaction among young people. The study also tries to explore

127 Quantitative Methods 5 how cross-nations include civic education and about the structure of change helps explain why as citizenship education into their curriculum, and well as how changes have occurred over time. …. enables to understand changes or variations of helps the researcher explain the cause of unexpected civic knowledge, attitudes and engagement over fluctuations. … Second, time series analysis enables years in different nations from data collected in researchers to trace the historical behavior of past change, … which allows for extrapolating from the 2009 compared with data collected in 2016. Thus, data to make predictions about the future changes the available panel data from two periods can (McNabb, 2004, p.288) also be used to compare retentions or changes in different times, nevertheless, an internal validity National time series data in some country threat is that the samples are different and due to datasets are open access, and allow the political sampling error the populations may bear different analysist or researcher for examining the trends characteristics for both panels. You may reach the in their nation. Nevertheless, researchers are entire document from the website provided below accountable for receiving consent before they in the box. can utilize the data for their studies. An example, could be the data released from the Republic of Turkey, Ministry of Development (webpage: http://www.mod.gov.tr/Pages/index.aspx#). internet Examination of the data allows understanding W., Schulz, J., Ainley, J., Fraillon, B., Losito, the changing trends in households and changing G., Agrusti, T., & Friedman, T. (2016). birth rates with increasing female participation Becoming citizens in a changing world ıea rates into work force. ınternational civic and citizenship education An example time series study could be the study study 2016 ınternational report. https://www. conducted by researchers in Hacettepe University springer.com/gp/book/9783319739625 on Demographic and Health Surveys (DHS) starting from 1993 onwards in collaboration with official and national institutions (see Box National Time-Series 5.1). The purpose of the study was to examine The national time series surveys generally the demographics and health status of women, are data collected over the years that are in the regarding socio-economic status, fertility, interest of political scientists. Indicators may abortion, infant mortality and other aspects. Data vary from economic to social indicators, such regarding households such as sanitary conditions as income levels, education level, employment were also included. Sampling procedures were or unemployment figures by education level, conducted in collaboration with the State in or out migration trends within a nation. The Statistics Institute , that is currently called the measurements are taken in an organized manner Ministry of Development, through employing such as in fixed periods over time: these can be cluster sampling procedures. From among 12 days, weeks, months or multiyear cycles as a season NUTS (which stands for Nomenclature des or time cycle may bear history that may influence Unités Territoriales Statistiques_Nomenclature of the respondents attitudes or behaviors (McNabb, Territorial Units for Statistics) in Turkey a cluster 2004), ultimately time series analysis has two of 641 districts, 11.794 family members were major functions to consider: included in the study and women (N=9.746) between the ages 15-49 were interviewed, in which First, it provides the researcher a detailed picture they were expected to provide short answers to of the underlying structure of change in a variable, given close-ended items in a typical quantitative including certain reoccurring trends or cycles that survey design. may be present in the change. Such information

128 Research Methods 5 Box 5.1 Example for times series design study Demographic and Health Surveys (DHS) Study: The purpose of the DHS study was to examine the demographics and health status of women, regarding socio-economic status, fertility, abortion, infant mortality and other aspects. Data regarding households such as sanitary conditions were also included. Sampling procedures were conducted in collaboration with the State Statistics Institute that is called the Ministry of Development currently through employing cluster sampling procedures. From among 12 NUTS (which stands for Nomenclature des Unités Territoriales Statistiques_Nomenclature of Territorial Units for Statistics) in Turkey a cluster of 641 districts, 11.794 family members were included in the study and women (N=9.746) between the ages 15-49 were interviewed. The participants were asked close-ended questions as in a typical interview survey design. Find answers to the following questions: What is the research question in this study? What is the population? What strategies are utilized for the selection of the sample? What is the research method applied?

Source: Hacettepe Üniversitesi, Nüfus Etüdleri Enstitüsü (2014). Türkiye nüfus ve sağlık araştırması [Demographic and Health Surveys]. Report NO: NEE-HÜ.14.01, Hacettepe: Ankara.

Election Surveys According to van Holstein and Irwin (2016) elections data are useful not only for contemporary analyses but also for historical analyses. Election data provide opportunity to analyze voting behavior of individuals through combining this with ecological and contextual data to produce maps and graphs of political tendencies that explain individual behavior across geographical area and specific contexts. Face- to-face interviews have an advantage of producing high response rates but may be very expensive and time consuming. With the increase of phones, especially cellular phones, calls also have become popular trends in collecting data, yet the general preference is to send out a link to the online questionnaire by e-mail. Why do reserchers prefer sending out questionnaires by e-mail despite their low response rate? (see Box 5.2).

Box 5.2 Example for election survey

Abstract taken from the publisher Political elections can be defined as systems that contain political tendencies and voters’ perceptions and preferences. The outputs of those systems are formed by specific attributes of individuals such as age, gender, occupancy, educational status, socio-economic status, religious belief, etc. Those attributes can create a data set, which contains hidden information and undiscovered patterns that can be revealed by using data mining methods and techniques. The main purpose of this study is to define voting tendencies in politics by using some data mining methods. According to that purpose, the survey results, which were prepared and applied before the 2011 elections in Turkey by KONDA Research and Consultancy Company, were used as raw data set. After preprocessing of data, models were generated via data mining algorithms, such as Gini, C4.5 Decision Tree, Naive Bayes and Random Forest. Because of increasing popularity and flexibility in analyzing process, R language and Rstudio environment were used. Author Keywords: Data Mining; Political Elections; R Language; RStudio; Voter’s Tendency

Source: Bayir, A., Ozdemir, S., & Gulsecen, S. (2017). Determination of voting tendencies in Turkey through data mining algorithms, International Journal of E-Adoption, 9(1), 50-58, DOI: 10.4018/IJEA.2017010105

129 Quantitative Methods 5 Panel Surveys Political sciences, or international relations fields value policy decisions and the changing trends of individuals based on policy making decisions or applications, therefore through a panel study, the researcher can collect data from similar samples at different times. For instance, consider that a governing party is interested in understanding its citizens’ attitudes toward becoming a member of the European Union. After certain international trade decisions, and changes at political and economic level, the governing party may reach the same sample again to see if the attitudes of the citizens have changed over time. One major limitation of a such longitudinal study is that since there is a long period of time between the two survey administrations, the researcher is likely to lose some of his or her sample. (see Box 5.3).

Box 5.3 Example for international comparative research The 2018 Edelman Trust Barometer The 2018 Edelman Trust Barometer data in 28 Markets of ages between 25 and 64 were collected through an online survey. More than 33,000 respondents participated for three weeks end October 2017. Similar data were collected online for 7 years. The criteria set was that the participants were college educated, and earned in top 25% of household income per age group in each market. You may examine the website for Edelman Trust Barometer and examine fluctuating differences in time and space in different continents. The Table below gives an opinion of how data can be compared across countries and by trend, in this case percentage of trust in journalism is compared with trust in artificial platforms. If you wonder about the trends in your country, you may reach some data provided through open access means in the Internet. 2018 Edelman Trust Barometer (https://www.edelman.com/trust-barometer).

Source: https://www.edelman.com/trust-barometer

PREPARING GOOD SURVEY INSTRUMENTS There are three distinguished features that identify surveys according to Butt et al. (2016): a) they aim at reaching a random sample so that the researcher can generalize the findings to its population; b) they benefit from standardized closed-items so that it facilitates the measurement of attitudes and other characteristics of the respondents; and finally, c) they hold an ability to generate quantifiable data so that

130 Research Methods 5 it can be exposed to statistical analysis. In addition, it is likely that the unit of analysis selected may have some concerns such as perceived societal norms, and may respond in the light of expected norms in a society on contraversial topics such as sexual orientation or religious beliefs. Therefore, Dillman, Smyth, and Christian (2014) underscore that the researcher needs to avoid four types of errors as cornerstones of good quality surveys. • Coverage error, which refers to drawing samples form a list of irrelevant samples to represent the population • Sampling error, which exists any time when the researcher reaches some of the sample and not all members. For instance, instance, to understand the attitudes of political participation of uni- versity students the researcher only randomly selects students from one province as the available population rather than the whole country to generalize the findings to the target population; • Nonresponse error, which occurs when some particular units of analysis tend to answer some par- ticular items which are likely to influence the estimate of the findings; • Measurement error is described as the difference between the estimate produced and the true value in case the respondents may answer inaccurately to the survey questions.

Tips for Writing Survey Items After having done a thorough literature review related to the topic of study, the researcher should start writing the survey items in an organized trend. To begin with writing the items the researcher should check with the research questions or with the hypotheses formulated, examine if all key concepts are considered and operationally define and check if the variables that identify the dependent or independent variables are included in the instrument for content validity (details about validity and reliability proceeds this section). Next, the researcher needs to make sure that the questions or items included in the survey instrument are well structured. Box 5.4 reveals basic guidelines regarding how to write good survey items so that you will be able to develop your own questionnaire and improve the quality of your research design.

Box 5.4 Tips for writing good survey questions • Each question should have only one single objective. ‘I volunteer at governmental elections and NGOs’ Note, what if the participant only volunteers in one institution! • Adopt the same definitions or concepts throughout the survey. For instance, if your key term is university students, keep it like that, and do not alternate it with college students in different parts of the survey. • Avoid leading items that imply a preferred way to answer. For example: ‘I think that women make best teachers.’ 1) Totally Disagree 2) Disagree 3) Undecided 4) Agree 5) Totally agree • Avoid rating within the questions such as ‘I always attend student protests.’ 1) Never 2) Rarely 3) Sometimes 4) Frequently 5)Always Avoid loaded questions, especially during face-to-face interview survey, which may emotionally charge the respondent. For example: Do you have cancer? • Use questions that are simple and easy to understand, and cause no misunderstanding. • Avoid using technical jargon. It may discourage the individuals to continue with the survey. • Avoid negative or double negative expressions such as ‘I do not like the politicians,’ or ‘I do not dislike politics.’ • Be precise, avoid estimations such as ‘Estimated how many students are there in your class?’ ask ‘What size is your class.’

Source: Butt et al., 2016; Buckingham & Sounders, 2004; Larossi, 2006.

131 Quantitative Methods 5 To provide examples for survey items, you can examine Box 5.5 that illustrates authentic types of items or questions used in the Political Participation of Youth Study. (see Box 5.5).

Box 5.5 Examples for survey items Political participation of youth (https://www.gesis.org/index.php?id=3180&tx_eurobaromater_pi1[vol]=POLITICS,Political%20 Participation&tx_eurobaromater_pi1[pos1]=2040) Types of items for surveys (online or interviews) adapted from the EUROBarometer 2004 Frequency ratings Q. When you are with friends would you say you would discuss politics frequently, occasionally, never, or no answer Frequently ...... 3 Occasionally ...... 2 Never...... 1 No answer ...... 0 Ranking Questions Q. What are your expectations for the year to come: will 2019 be better, worse, or the same, when it comes to

Item: Better Worse Same DQ 1 Your life in general 3 2 1 2 The economic situation in your country 3 2 1 3 The financial situation of your household 3 2 1 4 The employment situation in your country 3 2 1 5 Your personal job situation 3 2 1

DQ: Decline Question Ranking based on a five-point trust Likert scale Q. Indicate your level of trust in institutions considering “1” low trust to “5” high trust

Item Ranking DQ

1 through 5 1 Education 2 Journalism (TV, newspaper, magazines) 3 Platforms (Social media, search engines) 4 Presidential system 5 Court 6 NGOs 7 Millitary

DQ: Decline Question

Source: Adapted from http://www.asianbarometer.org/pdf/core_questionnaire_wave4.pdf9

132 Research Methods 5 Data Collection Procedures In addition to underscoring that the quality of the survey instrument itself is essential, there are different techniques to collect survey data. One of the earliest, yet most expensive approach could be the face-to-face interviews which could also be done through the phone. It is advantageous in that it increases the likelihood for participation (van Holsteyn & Irwin, 2016). In addition to the interview technique, survey data may be collected through questionnaires or standardized interviews through mail, or online platforms such as the Survey Monkey or other software such as Qualtrics. Currently there exists rising interest in virtual platforms such as Twitter and other virtual means, however, there is a risk that these virtual environments may not provide the researcher with the expected return rates. Table 5.2 provides a brief documentation for advantages and disadvantages for administering different types of surveys.

Table 5.2 Advantages and disadvantages of quantitative data collection methods Direct Telephone Per mail Face-to-face Platforms administration Call interview social media Cost efficiency Low Moderate Moderate High Low Facilities required Yes No No No No Training requirement Yes Yes No Yes No Duration for data Shortest Short Longer Longest Short collection Response rate probabi- Very high Good Poorest Very high Poor lity Allows for group Yes No No No Yes administration Allows for random Yes Yes Yes Yes No sampling Encourages response to Somewhat Somewhat Best Weak Weak sensitive topics Standardization of Easy Somewhat Easy Hardest Hardest responses

Source: Note: Table initially adapted from Fraenkel and Wallen (2010); and Dillman, Smith, & Christian (2014); Larossi (2006).

Data Analysis Procedures In the introduction, I stated that the researcher should have some basic knowledge about statistics before reading empirical research or have a desire to conduct research in a field of interest and suggested that the reseacher be equipped with the essential skills in statistical analysis. Braumoeller and Sartori (2002) highlight that one advantage of such statistical knowledge facilitates political scientists to aggregate information from large numbers of cases, for instance, to understand individual attitudes, beliefs, voting or other behavior and extract even basic summary statistics from such a mass of data revealing unsophisticated analysis such as the sample mean, percentages or standard deviations as valuable, and such knowledge is obviously a great opportunity and should not be overlooked. One easy way of calculating descriptive statistics is using tools such as the Microsoft Excel, or SPSS software data analysis packages. Through using such tools, the researcher or political scientist can easily convert the data collected from the online surveys to the program or manually insert the data collected through interviews, phone calls, or filled out questionnaires. Through selecting the proper analysis boxes, the

133 Quantitative Methods 5 researcher can summarize big datasets in a press on No matter what research topic a political a button period to obtain summaries that facilitate scientist is interested in, scientific research requires understanding demographic information about the planning and thinking critically about the core participants. For instance, it enables to summarize dimensions to initiate research. The following the number of female or male participants, the section underscores the fundamental elements in number of employed or unemployed participants. doing scientific research. Through cross-tab tables, it is also possible to retrieve tables or boxes that show, what percentage BASICS OF DOING RESEARCH of the female and male population is employed or not, and other measures. For analysists using Regardless if the researcher is aiming at doing figures with visual reports such as tables, charts, or descriptive research or correlational research, there scatterboxes may appear more comprehensible as the are some basics of doing scientific research, which reader can observe the magnitude or the differences relates to asking for research questions or stating of the figures on those tables or charts. On the other hypothesis if the design selected requires one. hand, use of causal data will enable the researcher Next, it is also important to make sure that the to find out the relationships between the variables. data collected and analyzed is scientifically valid In doing data analysis based on correlational and reliable. research, the researcher must be careful and avoid understanding or reporting cause-and-effect important relationships (McNabb, 2004). In other words, descriptive statistics are useful to see the distribution No matter what research design or method of a variety of outcomes of the data and examine a researcher selects for his or her research, whether those outcomes are compatible with a to initiate a study any researcher needs to popular statistical distribution. For instance, in the be equipped with the basics of research. In case of education and gender, in most developing or other words, any research, qualitative or underdeveloped countries, education level for girls quantitative needs to start with identifying a is slightly lower or lower than education level for research problem or a purpose. boys. While causal statistics are good for comparing claims that political outcomes are responses to one or more political, social, or economic forces (Best Writing Research or Hypothesis & McDonald, 2016). For instance, in a population where there is higher social justice in income Let us exemplify what I mean by this, as a distribution, there are fewer criminal violations political analysist we may assume that the higher such as theft or bribery. educated a citizen is, our senses may say through our experiences the better paid he or she will be. Politicians may argue that the more they will important conduct public speeches in the metropoles, the more likely they will be selected. Economists may argue that drop in daily working time may In doing data analysis based on correlational lead to drop in unemployment rates nation-wide. research, the researcher must be careful and All these examples given above are predictions avoid understanding or reporting cause- that are called hypothesis in scientific research. and-effect relationships. Cause-and-effect To find scientific evidence for the phenomenon relationships are related to experimental or problem areas revealed by the politician, the research where the effect of a theory is tested policy analysist, or the economist, one needs to and may be strongly dependent on the test the hypothesis in a systematical way and offer sample selected. Therefore, the sample that reliable data to accept or refute the hypothesis we represents a population needs to be broadly stated. The scientific findings we reach at in turn explained based on descriptive statistics. will provide implications for the improvement of political science and international relations.

134 Research Methods 5 Beside stating hypothesis, curiosity may also lead For instance, rather than administering a us to scientific testing and implement quantitative questionnaire to the entire population, a research methods. We may ask several questions random sampling strategy could be selected that arise from our senses or curiosity and we and generalized to the population (Sam- want to find answers to our problem statement or pling methods are explained in the pro- a phenomenon. The scientific research problem ceeding sections). statements or research questions that lead us to Examine the following research question conduct a quantitative research may (RQ): Poor: What are the political orientations of provide greater precision to a field of study freshman higher education students’ in Eu- that explicitly takes on the task of improving our rope? understanding of a world defined by uncertainty. Good: What are the political orientations of .... in order to make predictive claims about the freshman education students’ in Europe? strategic choices actors will make under certain conditions or through the codification of data into • They should be clear so that they do not give larger datasets, in order to uncover correlations way to ambiguity. In other words, whoever between variables trough bivariate or multivariate reads the research questions should under- regressions we have been provided with a set of tools stand the same idea to initiate the study. that can help us test our assumptions about the RQ: In what type of regimes in OECD world. (Lamont, 2015, p. 111) countries are strikes seen as a civic right? • Research questions should be significant, in In case that you are planning for a quantitative other words, they should be worth to inves- research design, as the researcher you are expected tigate. Especially, they should be worth for to be equipped or at least to be familiar with the humankind, living beings or the nature. existing research in your field of specialization. You Poor: ‘What sustainable energy sourc- need to know the type of research done in your area es do not produce emission gasses that and their findings so that you can make decisions harm the nature and the food industry?’ about where and how to contribute to the literature Better: ‘What sustainable energy sources are with your study. Thus, the questions that need to harmless to the nature and the food industry? be elicited is first to start with examining what is • Research questions should be ethical, in- the available knowledge in your area in research? dicating that any research should harm no How is this knowledge reached at, and how does human being or living being, or the natu- it relate to the research design? What evidences are ral environment such as ‘How do people’s given? What are the gaps in research that need to religious beliefs affect their opinion about be further researched? If the researcher knows what Euthanasia?’ kind of answers he or she is planning to find, he • Research questions are likely to suggest, but or she initially needs to start formulating the right not always, a relationship to be investigat- research question(s) or stating the hypothesis. ed such as ‘Is there a relationship between Quantitative research questions are composed education level and expenditure on socio- of independent and dependent variables, which will cultural events? be elaborated on after examining samples of good • Good quantitative research questions research questions. Fraenkel and Wallen (2010) should be specific. provide a behaviorist approach while suggesting • Finally, good research questions should the characteristics of good research questions and be measurable; and capable of conducting state the following. rigorous statistical analysis (Garratt & Li, 2005). For instance, Do schools located in the city centers compared with schools lo- Good Research Questions cated in the city centers graduate students • Research questions should be feasible so with higher scores in national standardized that there is available time, energy, and tests? even sufficient budget to investigate them.

135 Quantitative Methods 5 To study the relationship in research questions important is a valuable approach that enables us to understand the nature of the world we live in, and to start with Gender, education, nationality are all nouns this investigation the researcher needs to find out that can be counted as variables. Numbers connections between the parts of the hypothesis can be used for variables to rank education constructed so that it allows for predicting the level such as pre-school, primary school, existence between the relationships (Fraenkel & secondary school or high school, or socio- Wallen, 2010). In other words, in order to be able economic status can be ranked as high to formulate a hypothesis that aims to study the SES, moderate SES, or Low SES, while relationship between the variables, it is essential categorical variables cannot be ranked or to understand what variables are and how they are measured numerically in conditions like used in the hypothesis. gender (male or female), or nationality.

important To make it more specificall, variables can be Research questions should be feasible, clear, defined as ‘quantitative variables’ and ‘categorical significant, specific, ethical and measurable variables., Quantitative variables are mostly able to initiate a research study. to be divided into smaller chunks such as weight in kilograms, or height in centimeters, or age in numbers, or as magnitude of interest represented Variables in figures like 1 for little interest or 5 for high A variable is a concept that may refer to any interest, or 1 for disagreement and 5 for agreement. variation in a context. For example, artifacts used All those ratings hold a continuum in themselves in elections can be a variable. Gender, education, and therefore are called quantitative. Categorical nationality are all nouns that can be counted as variables, on the other hand, are not varying in variables. For instance, let us use the example of degrees, or amounts or quantity, they rather are a voting box. If the votes in the box vary from part of a category such as gender, one is either rightist parties to liberal and leftist parties, we call male or female; or marital status, married, single, them variables. Nevertheless, if all the votes are divorced, or other, or such as nationality, or school representing the same party in the box we call them type, either public school or private school. ‘constants’. On the other hand, variables may also In order to prepare good research questions or refer to measures such as motivation, happiness, identify hypotheses it is essential to understand the well-being. In order to measure or manipulate difference between independent and dependent a variable, we need to define it specifically. For variables. Through an independent variable, a instance, we can assign ascending numbers to researcher can study its effect on one or more motivation such as 1 for low motivation and 5 for variables. Thus how the independent variable high motivation. Let us take wages, wages can be affects the dependent variable is studied. Examine defined again as low-income wages or high-income the following research questions: wages depending on how they are seen economic- RQ: Is there a relationship between mothers’ wise in a nation. education level and children’s school achievement? In the research question above, mother is the independent variable, and children’s school A variable is a concept that may refer to any achievement is the dependent variable. In other variation in a context such as artifacts used words, the independent variable affects or in elections. manipulates the dependent variable.

136 Research Methods 5 of the population. A simple formula is suggested for decision-making by dividing the population A hypothesis in its simplest form can be by the desired sample size (Fraenkel & Wallen, identified as predictions of research results 2010). For instance, assume you have a population before the research even took place of 1000 individuals, and decided to represent a 10% of your population, the calculation would be important selecting randomly 100 individuals to be included in your sample.

The independent variable is expected to Desired Sample Size = 100 = 1 affect or cause the change on the dependent Population size 1000 10 variable. “A political scientist who is interested in voting tendencies of university students and volunteer participation in Consider that you are asked to bring in figures NGOs,” is likely to examine the voting for the top management regarding the topics of tendencies as the independent variable and extra-curricular seminars the students want to their volunteering action as the presumed attend on campus. Hypothetically there are 5000 result is the dependent variable. students on campus. You want to reach 10% of the population to generalize your results to the population.

3 Population size= .10 Thus, 5000/10=500 students Read the following research Question, Sample Size ‘Do liberal governments have a greater impact on social justice? Identify the, Consequently, to generalize your findings to a) Independent variable: the population, the calculation would be reaching b) Dependent variable: a random sample of 500 students. Answer: a) liberal governments b) social There are different approaches to sampling justice processes that can be defined as random sampling (simple random sampling, stratified random sampling, and cluster random sampling methods) CENSUS, POPULATION, AND and non-random sampling methods (Systematic SAMPLING sampling, convenience sampling and purposive sampling). If the purpose of the researcher is to The individual or the sample is the unit of make generalizations to the population, the most analysis in a study. While samples are only about reliable strategy is to conduct random sampling. a part of a population, a census tries to include the entire population. Since it is not feasible and fit to reach the entire population (or target population) Random Sampling such as in a census, the sampling process is a crucial ‘Random sampling’ is a strategy that enables process to select individuals who will become every member in the population to become selected representative of the larger group, in other words, and be a part of the study. Random sampling is the actual or target population. In this case, it is essential as it avoids the likelihood to select a essential that you determine the sample size. biased sample. Consider you have a box of MMs What is the sample size that you need to consider (Chocolate pieces covered in colored glaze). If you to generalize your findings? Although there is no do not pick out one chocolate specifically, there is strict rule behind what percentage is needed, the the likelihood that you can pick out any color piece researcher may draw his or her own limitation from among 6 different colored MMs. and decide that he or she is likely to reach 10%

137 Quantitative Methods 5 Nevertheless, as the researcher you also need Systematic Sampling to decide what size is enough to consider the Systematic sampling, on the other hand, is generalization. Fraenkel and Wallen (2010) state the selection of a sample systematically from the that samples should be large enough to obtain population- it belongs to the non-random sampling reasonable expenditure of energy and time and strategy. Consider the 4000 students in a list of a suggests that the minimum numbers of subjects university campus, you may systematically select (participants) is “100 for a descriptive study, 50 for the 10th student. In other words, the researcher a correlational study, and 30 for causal-comparative selects from the list the student number 10, 20, and experimental studies” (p. 104). 30, and so on, until 4000. In case the researcher wants to make sure to avoid any biases possible, important can randomly select the first number from a box numbered 1-10, and if the figure 2 appears, he ‘Random sampling’ is the most reliable or she can select the every 12th, 22nd, 32nd and strategy to identify your sample because it so forth. If the researcher wants to implement a enables every member in the population to cluster random sampling method, the researcher become selected at an equal chance and be a can randomly selects from a cluster of student part of the study. Consequently, the findings groups, and select randomly 10 students from each of the study that are based on random cluster. For instance, if we assume there are 50 sampling can be generalized to its population. classes running every day on campus, the researcher can randomly select 5 classes, and within each class can randomly assign 30 students for the study.

Stratified Random Sampling Cluster Sampling Other sampling methods are stratified There are occasions in which it is not feasible random sampling, cluster random sampling, and to conduct a random assignment of a number of systematic sampling. In stratified random sampling individuals from a population, the reasons may be the researcher has the opportunity to exploit any multiple, for instance, the list of individuals may available information on the population to improve not be obtainable, or it is not feasible to reach out the efficiency of the sample in case they know the the random assigned individuals as it may cause population characteristics and can divide them into ethical considerations, loss of energy and time. groups or strata (Larossi, 2006) such as male and In such cases cluster sampling may be the best female, or age groups, political orientations and solution to adopt as the sampling strategy. As the other. Stratified sampling strategy is one of the most word itself indicates, cluster sampling deals with preferred methods in opinion studies in political the assignment of the subjects (or individuals) not science research as it ensures representativeness of one to one, but based on each cluster identified. In the population studied (McNabb, 2004). Assume case the researcher wants to generalize the findings there is a population of 1000 students. From to the population, he or she can select the clusters those students, 500 are in the Administrative randomly and the sampling method will be called and Political Sciences Faculty (ADMIN); 250 in cluster random sampling. For instance, take the same the Engineering Faculty (ENG), 250 students are population size for the example given above related in the Architecture Faculty (ARCH). Looking at to the hypothetical campus. We described a campus the figures you can calculate a proportion of 50% with three faculties, and a 1000 undergraduate population for ADMIN, 25% for EF and 25% for students. Let us assume that the ADMIN Faculty FA. Consider you want to select 20% of the entire has 9 classes running, ENG Faculty has 6 running, population then your sample size needs to be 200. and ARC 3 running for senior students. If we take The simple calculation will be as follows: the unit of analysis as 1 for ARC cluster, we need to consider 2 clusters for ENG, and 3 clusters 500x20%= 100 for ADMIN for ADMIN proportionately. Class sizes for each 250x20%= 50 for ENG & 50 for ARCH Faculty are almost equally distributed. Ultimately,

138 Research Methods 5 the researcher can administer the survey instrument of the individuals who are included in the study, or or questionnaire to those students from the a replication of the study with a similar sample size randomly selected clusters. reflecting more or less a similar background could be repeated. Two-stage Random Sampling Example study for a multiple sampling strategy. In the preceding sections, a set of different sampling In the case you implement a two-stage sampling strategies have been introduced. The researcher’s method, then you may consider the cluster random role is to initially adopt the strategy that will best sampling method explained above and randomly represent the population of the study. On the other select the subjects or individuals randomly from hand, the researcher may want to select a sample those randomly selected clusters. from a wider spectrum and in this case applying to one strategy only may not suit to your study. In Purposive Sampling such circumstances, you may use a combination of As its name indicates, ‘purposive sampling’ strategies given you provide an acceptable rationale method is employed when the researcher for your choice and it is always save to consult an intentionally is interested in a population with expert in the field you research. For instance, in certain knowledge, experience, or history. For a cross-sectional survey to obtain a representative instance, as a political scientist you may want to sample of schools affected by internal migration do descriptive research with the elderly who were movements, I consulted the expertise of an activist university students during the protests of urban sociologist (Akar, 2010). First, the urban 1968 in the metropols and describe their current sociologist, examined national statistical data political participation behaviors. Thus, your sample to identify patterns of internal migration in needs to be purposively selected from individuals Turkey, and a scale was developed to categorize who were at college level at that time and who were provinces according to levels of migration, with activists. Sampling may also happen for objects -10 representing the highest level of out-migration or other artefacts such as selection of schools and and +10 representing the highest level of in- therebye reaching teachers in those schools that are migration. Second, after identifying the provinces, subjected to migration flows (Akar, 2010). with the help of the Directorates of Education in each selected province, four K-8 schools with high numbers of migrant students were identified Convenience Sampling purposively. Finally, to overcome selection bias, In the cases that it is not feasible or fit two schools from each province were selected to conduct a random sampling as suggested randomly, and all teachers who were affiliated with earlier, ‘convenience sampling’ method can those schools at that time were included in the be implemented, nevertheless, it should be sample. To read the entire article you may browse highlighted that convenient sampling does not through the databases provided in your library. enable the researcher to generalize the findings In the preceding section we overviewed some from the sample to the targeted population and it key concepts in quantitative research, types of is the least suggested sampling strategy (Fraenkel quantitative methods, and how they relate to and Wallen, 2010). In this situation the sampling research in political science, international relations, strategy selected may cause internal validity threat and quantitative history research overall. Next, because the sample the researcher reached may definitions and functions of validity and reliability consist of a group of volunteers and they may feel are provided. to please the data collector and this is likely to cause bias related to the field studied, and the results will not be reliable. Therefore, when convenience Validity and Reliability sampling strategy is not avoidable, the researcher is Any quantitative research conducted needs to recommended to include a detailed description of be valid and reliable. The validity relates to the the demographic characteristics and information importance of using valid instruments, or it answers of the sample as much as possible to have an idea if the researcher is using the right instrument

139 Quantitative Methods 5 including all its items to serve for answering the research questions. The inferences the researcher can make based on the data obtained from the instrument needs to be appropriate, meaningful, correct, and useful (Fraenkel & Wallen, 2010). Thus, the participants need to be asked for the correct or exact items to collect data that will shed light to the research questions. Therefore, before any instrument that the researcher designed is administered to the sample, it is essential to pilot the instrument for general understanding and avoid any misunderstandings (Schwester, 2015). In addition, not only the instrument itself is of importance, but also the way that the data is collected matters. Let us consider that the researcher collects data at a time the employees are leaving their offices. In such a case the individuals may not think thoroughly and respond to the survey items in a hasty manner, which Validity refers to the correctness, may end with validity constraints. In brief, in order for appropriateness, meaningfulness, and a research conducted to be valid, the researcher needs to usefulness of the data collected or the control three basic methods: Look for the content-related relevancy of the data collection instrument. evidence, criterion-related evidence, and construct-related evidence (Table 5.3).

Table 5.3 Method for checking validity Method Procedure Content-related Requires expert judgment in the field of study. Decision on ‘Have I asked the most validity appropriate question/item to find answers?’ Criterion-related Enables the researcher to relate data with other criterion related evidence. For instance, validity ‘to explore the respondents’ attitudes toward political participation in ‘national elections’ the researcher may pose how many times the responded voted in their lifetime through an open-ended item and the frequency of their voting act in a close-ended item with the rating ranging from never to always, such as ‘I have voted since I became officially liable.’ (a) never; b) rarely; c) sometimes; d) Frequently; e) Always. Construct- Requires to define the variable clearly and measure it through other means to make related evidence predictions based on the literature such as eliciting academic scores of university students and their ability on statistical analysis.

On the other hand, ‘reliability’ is about the consistency of the scores obtained. For instance, if we have both open-ended items and close-ended items in an instrument that measures political tendencies of subjects, the responses need to be consisted in both methods used to check for the consistency in reponses. important In addition, data may not be reliable based on how it is collected. Consider that in a survey the respondents are If the data are not reliable they cannot yield asked for their political tendencies, in such occasion the valid inferences about the results. While data individual may not feel comfortable, especially, if the collected from an instrument can be very item includes contraversial issues, the respondent may reliable, the study may not be valid per se. feel reluctant in answering his or her actual feelings or The ultimate goal is to collect data with both attitudes. Ultimately, the data may not be reliable. high reliability and high validity.

140 Research Methods 55

Identify basic concepts that relate to LO 1 basic quantitative research method.

Quantitative research enables to collect quantifyable data and analyze it. Data can be part of demographic data or other variables. There are many concepts or terms that relate to quantitative reseach. We can talk about type of research such as descriptive, correlational and causal-correlational, or experimental rese- arch. One of the most utilized methods is the survey method for data collection, which may be collected cross-sectionally, cross-national, through panels and other. Issues of reliability and validity are important in the process of constructing valid instruments and preparing reliable reports in quantitative research.

In addition, the selection of the unit of analysis is essential. There are different techniques to identify a Summary sample representative of the population such as random sampling or purposive sampling. Analysis of the data collected can be done through basic descriptive statistics or inferential statistics.

Explain the role of reliability and LO 2 validity in quantitative research.

Reliability in quantitative research relates to data collected and analyzed in a reliable way. Reliability of the study, also refers to the consistency of the study meaning that data collected in time intervals or through si- milar or different data collection instruments needs to be consistent with each other. While, validity refers to the validity relates to the importance of using valid instruments if the if the researcher has collected data that measures the research question of the hypothesis. This means that the data collection instrument is carefully developed based on literature review or other and necessitates to check for content validty, criterion validity and construct validity.

Develop basic skills in quantitative research so that you will LO 3 be able to comprehend published quantitative research.

Quantitative research like other empirical research starts with curiosity followed by observation. The- refore, quantitiative research skills include all components, from writing the purpose, to formulating the research questions and hypotheses, to designing data collection instrumets, and data collection and analysis. Learning about the jargon or terminology of research methods and concepts, the researcher will be able to collect data and analyze data, as well as analyze research reports or research articles to benefit from them in multiple settings.

Have a critical view of when and how to LO 4 employ quantitative research.

In case basic research skills are obtained, the researcher can critically analyze published research and reflect on the decisions made by the author. Similarly, the researcher may plan for a study based on multiple aspects, from stating the purpose to how the literature is reviewed up to making decisions related to the research design, data collection procedures, data analysis procedures and reporting to all other dimensions of research.

141 Quantitative Methods 55

1 A doctoral student majoring in political sci- 5 All, except one, of the following actions are ences is interested in students’ spending money on to be done to consider a study to be scientifically cultural events during their education at university. valid. Please, select the one that does not relate to To collect data over the four-year period, which de- validity. sign would be most suitable? The content of the data collection instrument is a. Correlational study ------b. Time-series study a. subjected for expert opinion. c. Experimental study b. developed upon literature review. d. Panel study c. piloted before it is administered. e. Causal-correlation study d. composed of reverse items.

Test Yourself Test e. enables test retest options. 2 A political party has recruited a company to find out citizens’ voting tendencies for the next elections in a big city with 4 zones and 5 million 6 Survey research can be collected through se- residents. Which sampling strategy will the com- veral means. Which of the following is the weakest pany adopt to obtain a representative sample that strategy that has low return rate from respondents? can be generalized including all zones? a. Via Mail a. Cluster sampling b. Direct administration b. Purposive sampling c. Face-to-face interview c. Convenient sampling d. Telephone call d. Two-stage random sampling e. Social media call e. Systematic sampling 7 Quantitative research is a growing trend in 3 A professor in the Department of Internatio- political sciences. Which of the following is not an nal Relations wants to compare trust in institutions advantage of quantitative research? in developing countries. What type of survey rese- a. Data analysis is objective arch does she need to employ? b. Data analysis is descriptive a. Descriptive survey c. Data analysis is inferential b. Cross-sectional survey d. Data analysis is qualitative c. Cross-national survey e. Data analysis is naturalistic d. Panel survey e. Census 8 If the researcher is interested in demographic information from all the citizens it is called a ------4 In the following research question, which va- a. Census riable is the dependent variable? “What is the re- b. Population lationship between parents’ socio-economic status c. Sample and their children’s school achievement? d. Case a. Parents e. Unit of analysis b. Parents’ social economic status c. Children d. Children’s school achievement e. Children’s economic status

142 Research Methods 5

9 Which one of the following statements is a 10 If a graduate student wants to conduct a ca- correct survey item and can be used in surveys for usal-correlational study for her thesis, what will be a Likert-type agreement scale? the first step? a. Citizens of nations with social justice often pay a. Prepare a proposal Test Yourself their taxes. b. Construct a questionnaire b. I think that only men should become surgeons. c. Identify the sample c. Gender equality is a matter of democratic gover- d. Consult expert opinion nance. e. Write a research question d. Developed nations do not dislike investment on animal rights. e. International Relations is not an area that can- not be experimented.

143 Quantitative Methods 55

1. b If your answer is wrong, please review the 6. a If your answer is wrong, please review the “Survey Research” section. “Preparing Good Survey Instruments” section.

2. d If your answer is wrong, please review the 7. e If your answer is wrong, please review the “Census, Population, Sampling” section. “Introduction” section.

3. c If your answer is wrong, please review the 8. a If your answer is wrong, please review the “Survey Research” section. “Census, Population, Sampling” section.

4. d If your answer is wrong, please review the 9. c If your answer is wrong, please review the “Basics of Doing Research” section. “Preparing Good Survey Instruments” section.

5. d If your answer is wrong, please review the 10. e If your answer is wrong, please review the “Validity and Reliability” section. “Survey Research” section. Answer Key for “Test Yourself” for “Test Answer Key

As a political scientist examine the censuses taken place over the years in your country, could you have a look at how census data has been collected over the years, and what elements or demographics of every member in the society were questioned. Evaluate what purpose each category questioned serves for.

Looking at census data, especially collected during elections period could be a good example. For instance, in earlier data demographics may be more im- portant to determine who are a family members living in the home to explore your turn 1 availability of extended families, however, in the current situation household data seems to be significant. Household data may include items asking for participants the availability of dishwashers in their homes, or if they have access to the Internet, or the number of books they have. Demographics may vary from gender, age, marital status to number of children in the home. Suggested answers for “Your turn” Suggested answers for “Your

144 Research Methods 55 Reading the full article “The Relationship between Genes, Personality Traits, and Political Interest” by Aaron C. Weinschenk and Christopher T. Dawes. Answer the following questions: Which panel data on political interest correlates lowest or almost nonexistent with the one of

the five traits? Please, explain. Suggested answers for “Your turn”

In the section on “Data Collection and Data Analysis” it was explained that when the correlation of the variables get closer to number 1, there is a high your turn 2 positive correlation. A reverse relation refers to a negative correlation. In the article on page 472 the following results are reported in the given table. From the table the following can be deduced: Although small, there is some corre- lation between “Opennes” with MIDUS, National Survey of Middle Deve- lopment in the US (r=0.30), and Minnesota Twins Political Survey (r=0.26).

Read the following research Question, ‘Do liberal governments have a greater impact on social justice. Identify the independent and dependent variable.

As explained earlier, the independent variable has an impact or effect on the dependent variable. In this situation the independent variable liberal govern- ments does change the dependent variable. In other words social justice is the your turn 3 dependent variable. a. Independent variables: liberal governments, b. Dependent variables: social justice

145 Quantitative Methods 55

Further Reading

Akar, H., & Şen, D. (2017). Impact of internal Gurbuz, E., & Eris, E. (2016). The effect of personal population movements on the schooling process idealism, relativism, and machiavellianism in Turkey: Supervisors’ views. Education Policy on voting tendency: A Turkish Study, Journal Analysis Archives, 25 (13,) Retrieved from https:// of Political Marketing, 15(1), 45-69. DOI: epaa.asu.edu/ojs/article/view/2693 10.1080/15377857.2014.959683 Bayırbağ, M.K. (2013). Is there a job after college?: Le Roy, M.K., and Corbett, M. (2006). Research Higher education’s impact on transition from methods in political science: An introduction using school to work. METU Studies in Development, micro case. Wadsworth: Thomson. 40, 465-492. Schwester, R. W. (2015). Teaching Research Methods in Public Administration. Hershey, PA: IGI Global. doi:10.4018/978-1-4666-8116-3.

146 Research Methods 5

References

Akar, H. (2010). Challenges for schools in Igo, S.E. (2007). The Averaged American: Surveys, communities with internal migration flows: Citizens, and the Making of a Mass Public. Evidence from Turkey. International Journal of Cambridge: Harvard University press. Educational Development, 30, 263-276. John, P. (2010) Quantitative methods. In D. Marsh, Bayir, A., Ozdemir, S., & Gulsecen, S.(2017). & G. Stoker (Eds.) Theory and methods in political Determination of voting tendencies in Turkey science (3rd Ed.). (pp. 267- 284). New York: through data mining algorithms. International Palgrave Macmillan. Journal of E-Adoption, 9(1), 50-58, DOI: 10.4018/ Lamont (2015). Research methods in international IJEA.2017010105 relations. Los Angeles: Sage. Best, R. E., & McDonald, D. (2016). Models in Larossi, G. (2006). The power of survey design: A user’s political science: Forms and purposes. In H. guide for maging surveys, interpreting results, and Keman & J.J. Woldendorp (Eds.). Handbook of influencing respondents. Washington: The World research methods and applications in political science Bank. (pp. 295-308). Cheltenham: Edward Elgar Le Roy, M.K., and Corbett, M. (2006). Research Braumoeller, B. F., & Sartori, A. E. (2002). methods in political science: An introduction using Empirical-quantitative approaches to the study micro case. Wadsworth: Thomson. of international relations. In D. F. Sprinz & Y. Wolinsky (Eds.). Cases, numbers, models: Lewin, C. (2005). Elementary quantitative methods. International relations research (pp. 139-163). In B. Someck, and C. Lewin (2006). Research http://www.sscnet.ucla.edu/polisci/faculty/ methods in the social sciences, [pp215-225]. Los trachtenberg/syllabi,lists/harvard/moravcsik%20 Angeles: Sage. (sprinz%20wolinsky).pdf Mansfield, E.D. (2002) Quantitative Approaches to Brown, M., & Hale, K. (2014). Applied research the International Political Economy. In Sprinz methods in public and nonprofit organizations. San and Wolinsky (Eds.) Cases, numbers, models: Francisco, CA: Jossey-Bass, Web ISBN:1-118737- International relations research (pp. 164-189). 36-9. http://www.sscnet.ucla.edu/polisci/faculty/ trachtenberg/syllabi,lists/harvard/moravcsik%20 Buckingham, A., & Saunders, P. Y. (2004). The survey (sprinz%20wolinsky).pdf methods workbook. Cambridge: Polity Press. McNabb, D.E. (2004). Research methods for political Butt, S., Widdop, S., & Winstone, L. (2016). The science: Quantitative and qualitative methods. role of high quality surveys in political science Armonk, NY: M.E. Sharpe. research. In H. Keman & J. J. Woldendrop (Eds.). Handbook of research methods and applciations Schwester, R. W. (2015). Teaching Research Methods in in political science (pp. 262-280). Cheltenham: Public Administration. Hershey, PA: IGI Global. Edward Elgar Publ. doi:10.4018/978-1-4666-8116-3. Dillman, D. A., Smith, J. D., & Christian, L. M. van Holsteyn, J. J. M, & Irwin, G. A. (2016). (2014) Internet, phone, mail, and mixed-mode Studying voting behavior. In H. Keman & J. J. surveys: The tailored design method. (4th ed.). New Woldendrop (Eds.).Handbook of research methods Jersey: Wiley. and applciations in political science (pp. 248-261). Cheltenham: Edward Elgar Publication. Fraenkel, J. R., & Wallen, N. E. (2010). How to design and evaluate research in education. (7th ed). New Weber, C. (2012). Emotions, campaigns, York: McGraw-Hill. and political participation. Retrieved from: http://journals.sagepub.com/doi/ Garratt, D., & Li, Y. (2005). The foundations of full/10.1177/1065912912449697 experimental/Empirical research methods. In B. Somekh, & C. Lewin, (Eds.). Research methods in Weinschenk, A.C., and Dawes, C.T. (2017). The the social sciences (pp.198-206). London: Sage. Relationship between Genes, Personality Traits, and Political Interest, Political research Herbst, S. (1993). Numbered Voices: How Opinion quarterly, 70(3) 467–479, DOI: https://doi. Polling Has Shaped American Politics. Chicago: org/10.1177/1065912917698045 The University of Chicago Press.

147 Chapter 6 Mixed Methods Research After completing this chapter, you will be able to:

Figure out the mixed methods research designs, compare these designs in the context Explain basic process of conducting mixed of timing, priority and integrity and write your methods research and explain reasons for own mixed methods research design as in the 1 using mixed methods research 2 example

Interpret how to interprete the mixed methods Identify the data collection and analysis research data and write your own research 3 processes es of mixed methods research 4 report Learning Outcomes

Chapter Outline Key Terms Introduction • Mixed methods Overview of Mixed Research • Mixed methods research Determining the Goal of the Mixed Methods • Research design Research • Research sample Mixed Method Research Planning Stage • Collecting and analysing data Research Implementation Stage • Interpretation of data • Writing mixed methods report

148 Research Methods 6 INTRODUCTION collection and analysis and qualitative methods With the changes in social and behavioural of data collection and analysis to understand a sciences, the third methodological movement (as research problem.’’ It corresponds to the “class of Teddlie and Tashakkori mentioned) has evolved. research where the researcher mixes or combines This methodology is called as “Mixed Methods quantitative and qualitative research techniques, Research” [MMR], in which quantitative and methods, approaches, concepts or language into a qualitative methodologies are used fruitfully in single study” (Johnson and Onwuegbuzie, 2004, order to answer the research question better. In p. 17). MMR creates a remarkable opportunity for this context, within this chapter, readers can find methodologies to define and improve techniques some helpful information to conduct MMR from that are not far from what they already practice determining mixed goals to writing mixed methods indeed. report of their study. However, there is no factual MMR is the third research paradigm that recipe, it gives some clues, strengths and weaknesses partners with the philosophy of pragmatism. or direct routes and authentic examples of mixed Moreover, it provides the most informative, methods research to the readers about where to complete, balanced, and useful research results, start, how to conduct a MMR (sampling, designs, based on traditional quantitative and qualitative collecting and analyzing data etc.) by themselves. research. On the other hand, the mixed methods In this sense, firstly the researchers must learn what research paradigm offers an important approach is mixed methods research, then determine their for generating important research questions and goal and plan the research in accordance with the providing warranted answers to those questions. steps in this chapter. Therefore, this chapter opens This type of research should be used when the by describing MMR, explaining possible rationale nexus of contingencies in a situation, in relation and advantages for using it, and giving limitations to one’s research question(s), suggests that MMR of MMR. This chapter is an introduction to MMR is likely to provide superior research findings and process step by step and suggest helpful strategies outcomes (Johnson, Onwuegbuzie and Turner, and techniques to go ahead for MMR. 2007). Onwuegbuzie and Leech (2004) claim that disagreements between quantitative and qualitative research can be minimised through MMR as the OVERVIEW OF MIXED third research paradigm. MMR does not restrict RESEARCH or constrain the preferences of researchers, instead it is a good way of justifying the use of multiple Definition approaches in answering research questions. As it is not restraining, MMR characterises an extensive Mixed methods research paradigm systematically and original type of research. combines or mixes ideas from both quantitative Johnson and Onwuegbuzie (2004, p. 17) define and qualitative research. For this reason, you it as ‘inclusive, pluralistic, and complementary’, and may come across different terms referring to claim that an eclectic approach to method selection mixed research as mixed methods research, and the considering about and managing of mixed method research, mixed methodology, research can be adopted by researchers. Tashakkori multimethod research, methodological pluralism, and Creswell (2007, p. 4) suggest that mixed and multiplism. The most widely used term is methods is “the research in which the researcher MMR (Johnson and Christensen, 2004, p. 645), collects and analyses data, integrates the findings, which is used throughout this chapter. In this and draws inferences using both qualitative and sense it can be defined as ‘‘research designs using quantitative approaches or methods in a single qualitative and quantitative data collection and study or program of inquiry.” Mertens (2010, 293) analysis techniques in either parallel or sequential states that mixed methods can embody the use phases’’ (Teddlie and Tashakkori, 2010, p. 11). of both quantitative and qualitative methods to According to Clark and Ivankova (2016), the answer research questions in a single study, as well MMR can be stated as ‘‘a process of research when as ‘those studies that are part of a larger research researchers integrate quantitative methods of data program and are designed as complementary to

149 Mixed Methods Research 6 provide information related to several research questions, each answered with a different methodological approach.’ It is also possible to describe mixed research as a synthesis that includes ideas from qualitative and quantitative research (Johnson et al., 2007). The most important factor in MMR is that “the use of quantitative and qualitative approaches in combination provides a better understanding of research problems than either approach alone” (Creswell and Plano Clark, 2011). MMR can be conducted in different steps. Johnson and Christensen, 2004, p. 663) determine eight stages to conduct a MMR (see Figure 6.1). According to mixed method researchers all research begins with research questions and the rest of the process can be conducted in a linear way as in Figure 6.1 or can be diverse in accordance with the research.

internet MMR paradigm requires knowledge on quantitative and qualitative methodology. You can find more information on how to use these paradigms in a meaningful way in this study called as “Making Paradigms Meaningful in Mixed Methods Research” by Shannon- Baker, 2015 on http://journals.sagepub.com/doi/full/10.1177/1558689815575861

Step 1 Determine whether a mixed desing is Step 2 Step 8 appropriate Determine the Write the rationale for research report using a mixed desing

Step 7 Step 3 Continually Research Select a mixed research design interpret the Questions data and and mixed ndings sampling design

Step 6 Step 4 Continually Collect the data validate the data Step 5 Analyze the data

Figure 6.1 Steps in mixed methods research

All told, MMR is a comprehensive process, which starts with determining research questions based on research topic, purpose and objectives. The research questions are in central position, since they lead directly the rest of the process. After determining the rationale of the study and the research design, the researchers select the sampling, collect and analyse the data, interpret the findings and write the report

150 Research Methods 6 based on the research purpose and the objectives. to answer questions that cannot be answered by However, MMR is not always a linear process as qualitative or quantitative approaches alone. MMR shown in Figure 6.1. The researchers sometimes go can be so functional that the researchers feel free back to previous steps in order to check the process to make use of different methodologies to reach or usually match research questions up with the the research objectives (Creswell and Plano Clark, steps they have already gone through. 2011). It is possible to use words, pictures and narrative to add meaning to numbers in MMR. Some of the advantages using the MMR are listed below (Johnson and Onwuegbuzie, 2004, p. 21; Mixed methods research Johnson and Christensen, 2004, p. 646): Mixed methods research can be defined • Researchers can use numbers to add as the combination of quantitative and precision to words, pictures and narrative. qualitative approaches in order to answer • It serves the opportunity to benefit from research questions more clearly. the strengths of both quantitative and qualitative research. • A grounded theory can be created and tested by researcher by means of it. What Is the Reason for Using • By conducting MMR, it is possible to Mixed Methods? answer a broader and more complete range Greene, Caracelli, and Graham (1989) suggest of research questions as only a single method that five major objectives or motivations exist for or approach does not limit the researcher. conducting MMR. The first one is ‘triangulation’, • Regarding the specific strengths of mixed which provides validation and concurrence methods research in a two-stage sequential of results gained from various methods and design, the researcher can use the Stage 1 designs investigating the same circumstance. results to improve and brief the aim and ‘Complementarity’, as the second motivation, seeks design of the Stage 2 component. for further detailing, improvement, interpretation • Using the strengths of an additional method and description of the results obtained from one to overcome the weaknesses in another method with results from the other method. The method by using both in a research study is third one is ‘initiation’, which means unearthing also possible. the bias, and discrepancies that cause re-designing • Stronger evidence for a conclusion of the research question. ‘Development’, the fourth through convergence and corroboration of motivation, includes utilizing the findings from findings can be available through MMR. one method to deliver clues to other method. The Moreover, it may bring understanding and last motivation, ‘expansion’, seeks to expand the comprehension that might be missed when size of research with different methods for different only a single method is applied. investigation factors. • A researcher can also use MMR to increase the generalizability of the results. • It can be concluded that qualitative and Advantages (Strenghts and quantitative research blended together Weaknesses) create more entire knowledge necessary to MMR can be defined as the combination of notify theory and practice. quantitative and qualitative methods for a clear In terms of strengths, MMR also incorporates understanding of research objectives than either and grows different research strategies used with approach alone. It is because MMR balances the a specific study that broadens the dimensions and weaknesses of quantitative and qualitative research scope of the research so that the researcher can and transforms them into strength. By using all types accelerate the understanding and accomplish the of data collection tools, researchers can get more aims quickly (Morse, 2003), makes the differences comprehensive evidence for a research problem. between quantitative results and qualitative On the other hand, MMR enables researchers findings clear, promotes the active involvement

151 Mixed Methods Research 6 of the participants, fosters the multidisciplinary important interaction and finally enables extensive and prosperous data (Wisdom and Cresswell, 2013). However, conducting the mixed methods research Doing MMR study requires more time can be hard to handle at times. The researchers have and expertise as collecting analyzing, to spend a long time to conduct a MMR because and intepreting quantitative and they need to collect and analyze both quantitative qualitative data take more time than and qualitative data. Furthermore, it requires clear expected. Besides, MMR studies must explanation of procedures since the readers may be conducted by a team consisting have difficulty in figuring out the qualitative and of well-qualified researchers in both quantitative proces (Creswell and Plano Clark, quantitative and qualitative paradigms. 2011). For such reasons, conducting mixed methods research has some weaknesses, some of which result from researchers; some of which result from methodology itself. According to Johnson and Christensen, (2004, p. 647) the weaknesses of Limitations the MMR are as follows: In respect of limitations, a single researcher • A single researcher may have difficulty may have some difficulties on conducting both in carrying out both qualitative and qualitative and quantitative research when s/ quantitative research, especially if two or he intends to use two or more ways all at once. more approaches are expected to be done Thus, a research team may be needed to get the concurrently (i.e., the study might require a best results. Furthermore, learning about multiple research team). methods and approaches and understanding how • The researcher has to learn about multiple to mix them aptly should be regarded as a must. methods and approaches and understand Conducting MMR can also be more expensive, how to combine them appropriately. time consuming and may require formulating • Methodological purists contend that how to mix paradigm, analysing quantitative data one should always work within either a or interpreting the paradoxical results by research qualitative or quantitative paradigm. methodologist (Johnson and Onwuegbuzie, 2004, • It is more expensive. p. 21). • It is more time-consuming. Wisdom and Creswell (2013) underlines • Some of the details of mixed research that conducting MMR can be challenging when remain to be worked out fully by research complex interventions such as a Patient-Centered methodologists (e.g., problems of paradigm Medical Home (PCMH) model are intended to mixing, techniques for qualitatively be evaluated with it. They also argue that MMR analyzing quantitative data, how to enhances the complexity of evaluations, as they integrate data and inferences, and how to are complex to plan and conduct. Therefore, the interpret conflicting results). researcher should carefully plan all aspects of the Although MMR is not easy to conduct, as study counting the study sample, timing and Creswell and Plano Clark (2011) have stated planning as well as integrating the qualitative that, the value of MMR outweighs the potential and quantitative data. Researchers should have a difficulty of this methodology. multidisciplinary perspective who are comfortable with using either methods (Wisdom and Creswell, 2013), but it may be challenging to assure the best quality of each component of a mixed method as each method complies to its own standards certainly (Wisdom, Cavaleri, Onwuegbuzie, et al., 2011). It can finally be suggested that MMR needs a vast amount of resources and time compared to a single method (Wisdom and Creswell, 2013).

152 Research Methods 6 Additionally, Johnson and Christensen, (2004, (Johnson and Christensen, 2004). Both the p. 671) have indicated that contradictory findings qualitative and quantitative phases of each MMR between the quantitative and qualitative stages are study can be linked to one or more of these five produced in some mixed researches. Contradictory research objectives. Johnson and Onwuegbuzie findings sometimes cause the researchers to collect (2004) propose a process of MMR including eight additional data or to review the research purpose or specific steps, respectively designating the research questions. This means a new study for researchers. question, specifying whether a mixed design is appropriate, choosing the mixed method or mixed model research design, gathering the data, clarifying the data, validating the data, coming to a conclusion and writing the final report. In 1 addition to the aims of the study, the objectives in Explain the steps in researchers’ manuscripts should be made explicit conducting mixed methods by them (Leech and Onwuegbuzie, 2010). Some research by giving strengths common research objectives identified in various and weaknesses in the process. mixed researches can be stated as ‘exploration, description, explanation, prediction, influence’ (Andrew and Halcomb, 2006), and ‘exploration, DETERMINING THE GOAL description and explanation’ (Onwuegbuzie and OF THE MIXED METHODS Collins, 2007). The first type of research objective, RESEARCH exploration, comprises inductive methods to figure out an opinion, argument and the like. In Determining the goal of the study involves this way, it brings about intuition, hypotheses, making a decision about the overall, long- deductions or generalization. Description, the term aim of the investigation (Powell, Mihalas, second type, means clarifying and defining the Onwuegbuzie, Suldo and Daley, 2008). Over previous experiences, relations, and the nature the course of writing about a mixed research, a of the incident. One should develop or broaden researcher should apparently define the goals of the a theory to achieve ‘explanation’ to find out the study (Leech and Onwuegbuzie, 2010) and follow phenomena. Another type of research, prediction, multiple goals such as predicting, contributing enables the researcher to predict future events by to knowledge base, having personal, social, means of previous knowledge. The fifth type of institutional or organizational effect, evaluating objective, influence, includes ‘the manipulation of alteration, comprehending complicated situations, a variable or construct’ so as to achieve an outcome examining new thoughts, creating new ideas, (Andrew and Halcomb, 2006). notifying constituencies and investigating the past (Newman, Ridenour, Newman and DeMarco, important 2003). According to different researchers, Formulating the Mixed Research there are some steps for determining Objective(s) MMR goals. Firstly, MMR researchers The research goal directs forthrightly to the must decide on which steps they use research objective. In this step, the researcher to conduct their MMR study and should determine which of the following five must follow these steps carefully as major standard research objectives are pertinent MMR is not a kind of methodology, in for the quantitative and qualitative phases of which only quantitative and qualitative the study: a) exploration, b) description, c) research paradigms are combined in a explanation, d) prediction, and/or e) influence simple rationale.

153 Mixed Methods Research 6 In short, the goal of the study, the research purpose and the objectives have an essential role to shape research questions. On the other hand, research questions are very critical in MMR process in order to delimit the study by determining the research design, the sample, the instruments and the data analysis used (Onwuegbuzie and Leech, 2006). Thus, the framework of the study is revealed. Plano Clark and Badice (2010) have pointed out different types of research questions in MMR. Researchers can use separate questions, written in quantitative and qualitative strands of the study; general overarching mixed methods questions, addressed with both quantitative and qualitative approaches; hybrid mixed methods, in which one question is written with two distinct parts (one part addresses to quantitative and another part addresses to qualitative part); mixed methods procedural/mixing questions, directing the integration of the qualitative and quantitative strands of the study; and combination, in which questions are written separately both in qualitative and quantitative strands. From these types of questions, it is possible to say that combination type is the clearest and the most widely used one. As a result, both quantitative and qualitative research questions are combined or mixed in the MMR. In this regard, in the MMR process, in order to answer the research questions, quantitative and qualitative data are collected and analyzed either concurrently, sequentially, or iteratively (Onwuegbuzie and Leech, 2006). For example, Pritchard (2013) combined quantitative and qualitative research questions for her mixed method dissertation. She wrote her research questions in three strands: qualitative, quantitative and mixed method. In Table 6.1, there is an example from the proceedings of the MMIRA Congress, 2018 in Vienna (Çınkır, Erdemli, Kurum and Güner Demir, 2018):

Table 6.1 Example research questions in combination style for mixed methods research Dimension Example This study aims to identify the barriers for Syrian students’ integration into Turkish Goal of the study education system and suggest some strategies and policies in order to eliminate these barriers. 1. What are the barriers for Syrian students’ integration into Turkish education system with respect to language, curriculum, teacher, accreditation and certification? Quantitative 2. What, if any, policies and strategies have been developed by school managemant for Syrian students’ integration into Turkis education system? 3. What is the nature of the barriers that Syrian students’ experience as a part of integration into Turkish education system? Qualitative 4. What policies and strategies do school managers and Syrian Refugee students perceive as contributing to their integration into Turkish education system? Mixed methods 5. In what ways do survey and interview data align with one another?

important

MMR research questions can be written in a different way. Prevalently, at first the researcher can write the general goal of the study, then in accordance with this goal, qualitative and quantitative research questions are written separately. At last, MMR aim of the study is written as in the example above.

154 Research Methods 6 Determining the Rationale of the Study and the Rationale(s) for Mixing Quantitative and Qualitative Approaches After deciding that MMR is suitable for the study, the researchers determine their rationale for using a mixed design. For this purpose, Creswell and Plano Clark (2011, p. 664) have suggested the researchers to answer the following questions: What do you want to achieve by mixing quantitative and qualitative approaches? How will mixing approaches help you in answering your research questions? The researchers can find the answers to these questions with the rationale shown in Table 6.2.

Table 6.2 Different rationale for conducting mixed methods research Rationale for Conducting Mixed Explanation Methods Research involves blending quantitative and qualitative research to better the sample Participant using approaches that encompass recruiting participants, dealing with activities enrichment such as institutional review board interviews assuring that each participant chosen is eligible for involvement pertains to evaluating the convenience and/or effectuality of current Instrument fidelity instruments, by generating new instruments and observing performance of human instruments. Treatment integrity presents evaluating the level of compliance of intervention. Significance refers to interpretation and utility of findings enhancement seeks convergence, correspondence, and corroboration of results from different Triangulation methods studying the same phenomenon seeks elaboration, enhancement, illustration, and clarification of the results Complementarity from one method with results from the other method Development seeks to use the results from one method to help inform the other method. refers to discovering paradoxes and contradictions as well as providing different Initiation perspectives that may lead to a reframing of the research question or results. seeks to expand the breadth and range of inquiry by using different methods Expansion for different inquiry components.

Source: Based on Greene, Caracelli, and Graham (1989; cite by Creswell and Plano Clark (2011, p. 665); Onwuegbuzie and Johnson, 2006; Collins, Onwuegbuzie and Sutton, 2006).

A shown in Table 6.2 different writers have determined different rations for MMR. Some seek elaboration, enhancement, illustration, and clarification of the results; the others use MMR to present evaluating the level of compliance of intervention. Recently four rationales for carrying out MMR were defined as ‘participant enrichment’, ‘instrument fidelity’, ‘treatment integrity’ and ‘significance enhancement’ (Collins, Onwuegbuzie and Sutton, 2006). ‘Participant enrichment’’ involves blending quantitative and qualitative research to better the sample using approaches that encompass recruiting participants, dealing with activities such as institutional review board interviews assuring that each participant chosen is eligible for involvement. ‘Instrument fidelity’ pertains to evaluating the convenience and/or effectuality of current instruments, generating new instruments and observing performance of human instruments. Another rationale, ‘treatment integrity’, presents evaluating the level of compliance of intervention. Finally, ‘significance enhancement’ is related enabling the density and quantity of data and enhancing interpretation and utility of findings (Onwuegbuzie and Johnson, 2006).

155 Mixed Methods Research 6 Apart from the writers above, Johnson and 2006). Onwuegbuzie and Johnson (2006) Christensen (2004) have added some more reasons underline some important circumstances. The first for conducting the MMR. Firstly, the researchers is that the researcher can gather both quantitative collect quantitative and qualitative data to see if and qualitative data separately at nearly the same they get parallel results from two types of data. On point in time. The next states that quantitative and the other hand, before conducting an experimental qualitative data analyses do not depend upon each research, the researchers collect qualitative data other during the analysis level. Further, the results to design the treatment or, alternatively, to better obtained from each type of analysis are not unified recruit participants for the trial. In order to learn till both sets of data are gathered and examined more details about their quantitative survey independently. In the end, the researcher needs responses, the researchers conduct follow-up to draw a meta-inference which combines the interviews with a few individuals who participated inferences extracted from separate quantitative and in the survey. In contrast to the previous rationale, qualitative data and findings (Onwuegbuzie and in order to generalize the results got from the Johnson, 2006). interviews the researchers develop a survey In a MMR study, the reason of using mixed tool and implement it later to a sample from approach instead of a single method design should a population. Additionally, quantitative and be described in the purpose statement. When MMR qualitative approaches can be mixed throughout is examined rationales for using the mixed method the research. During the research, qualitative and such as ‘to enhance researchers’ interpretations quantitative questions are posed, both forms of of results’ (Onwuegbuzie and Collins, 2007), data are collected and analyzed, and a quantitative descriptions and appropriateness of three aspects and qualitative interpretation is made. of the model ‘participant enrichment, instrument fidelity, and significance enhancement’ (Collins, Determining the Purpose of Onwuegbuzie, Jiao, 2007) are seen to have been the Study and the Purpose(s) stated clearly. For instance, Onwuegbuzie and for Mixing Quantitative and Collins (2007) highlight in their study, that ‘participant enrichment represents the mixing of Qualitative Approaches quantitative and qualitative approaches for the It is possible to conceptualize mixed research as rationale of optimizing the sample (e.g., increasing incorporating quantitative and qualitative research the number of participants)’, and they add: ‘with in a simultaneous, subsequent, conversion (Teddlie respect to participant enrichment, the present and Tashakkori, 2003), parallel or completely researchers approached instructors/professors blended (Onwuegbuzie and Leech, 2005) before the study began to solicit participation of approach. A researcher can incorporate quantitative their students and thus maximize the participation and qualitative methods in these ways regardless of rate’ (Leech, Onwuegbuizie and Combs, 2011). the type of the study as primary research (Johnson and Onwuegbuzie, 2004) or a mixed synthesis of literature combining the findings from both Participant enrichment quantitative and qualitative studies in a common It means making sampling better by scope of empirical study (Sandelowski, Voils blending quantitative and qualitative and Barroso, 2006). Additionally, the priority of research approaches in order to select eligible any approach in the study can be disregarded in participants for the study. combining quantitative or qualitative approaches (cf. Creswell, Shope, Plano Clark and Green,

156 Research Methods 6 MIXED METHOD RESEARCH PLANNING STAGE

Selecting the Mixed Sampling Design Defining sampling design in a MMR clearly is significant in the writing of a mixed research report. In terms of sampling design, information comprises the type of sampling, the sample size considerations including a priori power (Cohen, 1988) and information abundant situations (Miles and Huberman, 1994). The use of Onwuegbuzie and Collins’ (2007) model for deciding and writing about the samples for MMR is suggested. The type of generalization, statistical, analytic (Curtis, Gesler, Smith and Washburn, 2000), or case to case transfer (Firestone, 1993) to be made must be defined as well. For this purpose, 24 sampling desings (simple, stratified, cluster, systematic, multistage random, maximum variation, homogeneous, critical case, theory-based, confirming/disconfirming, snowball/chain, extreme case, typical case, intensity, politically important case, random purposeful, stratified purposeful, criterion, opportunistic, mixed purposeful, convenience, quota, multistage purposeful random and multi stage purposeful sampling) (Patton, 1990) are identified to use quantitative and qualitative strands of the MMR. In this sense, it is possible to say that deciding sampling frame is affected strongly by MMR design, in which qualitative or quantitative side of the study is clearly explained. For this reason, Collins, Onwuegbuzie and Jiao (2006) have stated that, in MMR, researchers must determine sampling for both the qualitative and quantitative strands of the research. Furthermore, according to these writers, MMR sampling process consists of seven steps. These steps are shown in Figure 6.2 (Collins, Onwuegbuzie and Jiao, 2006).

Determine the Goal of the Study

Formulate the Research Questions

Determine the Resarch Purpose

Determine the Research Questions

Select the Research Design

Select the Sampling Design

Select the Individual Sampling Schemes

Figure 6.2 Steps in the mixed-methods sampling process

157 Mixed Methods Research 6 A shown in Figure 6.2, the sampling process in the MMR involves seven different steps. The researchers can follow the steps in a linear way. Firstly, the goal of the study is determined, and then this leads to objectives, purpose, research questions and design linearly. In the last step, the sampling design is selected. The mixed sampling design is clearly suggested as ‘The sample was selected purposively utilizing a criterion sampling scheme ...’ by Onwuegbuzie and Collins (2007, p. 123). The specific demographics of the sample are presented in Figure 6.3.

internet For MMR studies’ process of selecting sampling and sampling types, you can see “A typology of mixed methods sampling designs in social science research”by Onwuegbuzie and Collins (2007) on http://www.nova.edu/ssss/QR/QR12-2/Onwuegbuzie2.pdf

Reations between the Time orientation Description qualitative and quantitative samples Exactly the same participants are involved Identical sampling in both the and qualitative phases of the study. Quantitative and Sample for the qualitative phases quantitative and occur concurrently qualitative components Parallel sampling of the study are di erent but are drawn from the same population of interest. Quantitative and Sample members qualitative phases selected for one phase Nested sampling occur sequentially of the study represent a subset of those sample members selected for the previous phase of the research. Use of two or more sets Multilevel sampling of samples that are extracted from di erent levels of the population of interest.

Figure 6.3 Onwuegbuzie and Colins’ Model (2007) for choosing and writing about samples for MMR

Selecting the Mixed Research Design It is possible to define the specific MMR through the ordering of the application of quantitative and qualitative methods (simultaneously or sequentially) besides determining in what point the mixing methods happen. Qualitative or quantitative data collection can be held in parallel or sequential form.

158 Research Methods 6 While two types of data are gathered and examined in concurrent methods design in terms of parallel form, one type of data creates a base for gathering of another type of data in sequential form (Mertens, 2010). MMR is a field developing in a rapid succession. Thus, many more mixed research designs are still being created. There are several competing typologies of MMR. In this chapter, the best introductory designs are shown in Figure 6.4 (Johnson and Christensen, 2004, p. 658).

internet For MMR studies’ process of selecting research design, you can see “mixed methods research: a research paradigm whose time has come”by Johnson and Onwuegbuzie (2004) on https:// www.jstor.org/stable/pdf/3700093.pdf?casa_token=mK4k8EPMI3gAAAAA:EWv7_ Xt1zw1MNmhiQqpId6sGR7XVm-nJzum_QfxZDD47lI87kBUmOyDpntdg2ET7CP m5MYAed5UQjzZ6swdRIGhOTnl-9kduAkpIS_0C6b8_L7j7SBbgg

Time Order Decision Concurrent Sequential Equal Status QUAL + QUAN QUAL QUAN QUAN QUAL QUAL + quan QUAL qual qual QUAL

Decision Dominant

adigim Emphasis Status QUAN qual r QUAL + qual quan QUAL Pa

Figure 6.4 Mixed methods design matrix

As shown in Figure 4, MMR designs can be shaped by time orientation and paradigm emphasis. In terms of time orientation, qualitative and quantitative phases of the study occur almost at the same time (concurrently) or they are organized in phases over time (sequentially). When it comes to research paradigm, the qualitative and quantitative parts of the study are given virtually equal emphasis or one paradigm can be stronger than the other (a qualitatively driven design or quantitatively driven design). According to Johnson and Christensen’s (2004) typology, the researchers make two major decisions. First, they decide whether they want to operate largely within one primary paradigm or not. Second, they decide whether they want to conduct the qualitative and quantitative components concurrently or sequentially. In addition to Johnson and Christensen (2004), Creswell and Plano Clark (2011, p. 66) have pointed out that there is a multiphase combination timing in MMR designs. It occurs when the researcher conducts various stages that consist of sequential and/or concurrent timing over a program of study. In the example of multiphase combination timing, the researcher can conduct over three or more phases by combining both concurrent and sequential elements within one MMR. The mixed methods researchers choose a research design, which reflects timing, combination, interaction of quantitative and qualitative methods. In this context, Creswell and Plano Clark (2011, p. 72) have mentioned six different MMR designs, which help researchers to design their strands of MMR. Those six research designs are explained in Table 6.3.

159 Mixed Methods Research 6 Table 6.3 Characteristics of six basic mixed methods research designs Types of Mixed Methods Explanation Research Design • conducting qualitative and quantitative research concurrently • giving equal status to qualitative and quantitative strands 1. Convergent design • analysing the qualitative and quantitative data separately • making general comment by combining the results • conducting qualitative and quantitative research sequentially • begging the study with quantitative data collection and analyses 2. Explanatory design • building the second phase of the study on the results of first phase • commenting on how the qualitative phase helps to explain the quantitative phase of the study • conducting qualitative and quantitative research sequentially • beginning the study with qualitative data collection and analyses 3. Exploratory design • conducting the quantitative phase based on the qualitative phase by testing or generalizing the qualitative results • explain how the quantitative data are built on qualitative results. • consisting of major and supporting research • conducting qualitative and quantitative research either sequentially or concurrently 4. Embedded design • a qualitative research can be conducted before, during or after the major quantitative research or vice versa. • using the supporting phase in order to develop the major research in some way • shaping interaction, dominance, timing and combination of qualitative and 5. Transformative design quantitative research based on theoretical framework • theoretical framework guides the methods decisions • conducting research by combining sequential and/or concurrent qualitative 6. Multiphase design and quantitative research • using this design in order to develop, adapt and evaluate programmes

Source: The table is adapted from Creswell and Plano Clark (2011, p.72).

As Creswell and Plano Clark (2011) have mentioned, the goal of the study determines the used research design. The researchers can use the convergent design in order to make people understand the topic more completely or validate the quantitative scales. While the explanatory design is being used to explain quantitative results, the exploratory design is selected to test qualitative exploratory findings. The embedded design is preferred in experimental study for preliminary exploration or follow-up explanation or more complete understanding of the experimental trial. On the other hand, in case of the social injustice, the transformative mixed methods research is conducted to describe the phenomonolgy. Lastly, multiple phase design is implemented to develop, adapt or evaluate program, multilevel statewide studies. A sample paragraph for writing a mixed methods design into a report is shown in Figure 6.5.

internet In order to find more examples on how to write MMR methodology, you can see “Traditions and Alcohol Use: A Mixed-Methods Analysis” by Castro and Coe, 2007 on http://www.apastyle.org/manual/related/castro-2007.pdf

160 Research Methods 6 This study is in a multiphase mixed method design, organized into three strands. Strand I data collection and analysis techniques are qualitative, while quantitative methodology is being employed throughout Strand II. In the third strand, a qualitative methodology is used to get principals’ and students’ views on Names School Policies and Support Strategies on Syrian Students’ Integration into Turkish Education System. the Although qualitative methods serve an exploratory function within this mixed methods design, both design paradigms will be equally weighted throughout this research (Creswell and Plano-Clark, 2011). } The multiphase design is shown in Figure.

The overall objective is to identify the barriers for Syrian Explains the Qualitative Qualitative Qualitative students’ stages in gure Study 1 Study 2 Study 3 integration into Turkish education system and suggest some strategies and policies in order to eliminate these barriers

As a part of overall As a part of overall objective, in the rst As a part of overall objective, in the third strand, according to objective, in the strand, according Syrian students’ second strand, in to principals’ and Syrian views, What is the what level Syrian students’ views, what nature of the barriers students are aected are the possible that Syrian students by integration policies and experience as a barriers into Turkish strategies on part of integration Education System? hindering barriers into Turkish integration into education system? Turkish education system?

Figure 6.5 The process of multiphase mixed methods research design (Creswell and Plano Clark, 2011)

As shown in Figure 1, a multiphase mixed methods research design, consisting of qualitative- qualitative-qualitative data collection and analysis is organized into three strands. The Gives reason for the design rationale for this approach is that multilevel studies focus on educational issues in three Discusses different levels: students, principals and school (Teddlie and Yu, 2007). In this study, both interaction, quantitative and qualitative paradigms are equally weighted, and they are sequentially timing and conducted on the results, gathered from previous strand. Firstly, qualitative data are priority collected and analyzed. Secondly, based on the findings got from the qualitative strand, the Discusses mixing questionnaire is developed and conducted. In the third strand, depending on quantitative Cites methodological data, participants are asked for possible suggestions in order to hinder barriers. Both types of} references data are taken equally into account within the final analysis.

Figure 6.5 A sample for writing a mixed methods design into a report (A sample taken from proceeding presented at MMIRA Congress, 2018 in Vien (Çınkır et al., 2018) Though Creswell (2009) uses the term ‘concurrent’ instead of ‘parallel’, he utilizes two basic design categories to achieve possible variations in mixed methods design. Mertens (2010) agrees with Teddlie and Tashakkori (2009) about the use of the term ‘parallel’. He finds the term ‘parallel’ more inclusive than ‘concurrent’ since ‘concurrent’ means the two methods are to be used at the same time while the term ‘parallel’ implies being in ‘proximity’ to each other. Creswell’s (2009) descriptions of possible variations

161 Mixed Methods Research 6 should be regarded as well: 1) it is possible to collect inferences depending on both sets of analyses qualitative data after quantitative data to define (Onwuegbuzie and Johnson, 2006). Parallel the quantitative results in parallel (oncurrent) mixed designs involve collecting and analyzing designs which are explanatory, 2) quantitative data seperately. When considered from this point data collection follows qualitative data to explore of view, a similarity exists between parallel and a phenomenon in exploratory designs, 3) both concurrent designs. Nonetheless, the researcher is quantitative and qualitative data can be used to supposed to make inferences on both sets of data see whether convergence in findings happens in in an integrated way in concurrent designs while sequential designs, or (4) one type of data may be in parallel mixed designs each data source causes dominant although the other type has a supportive to its own set of inferences and a ‘meta-inference’ role (Mertens, 2010). (Teddlie and Tashakkori, 2003) is not intended to In basic sequential mixed designs, researcher be reached, involving the combination of both sets can use the data collected and examined from of inferences into a coherent body. Those kinds one stage of the study in order to notify the other of designs may conlude to a) two separate reports stage of the examination. In this situation, the that would be introduced or issued seperately or examination of data starts before all the data are b) two separate reports that are introduced in gathered. In sequential mixed design studies, a two different parts of the same report. However, sequence of stages including multiple approaches Yin (2006) does not agree with the view of these to data collection, examination and inference, is designs representing mixed research, and some followed at the highest level of integration. It is other researchers (Teddlie and Tashakkori, 2006; also possible to embed multiple approaches in each Onwuegbuzie and Leech, 2004) consider them stage itself and supply theoretical or methodogical as quasi-mixed designs. Regarding the formative basis for the next step in the process (Tashakkori phase of mixed research, Onwuegbuzie and and Teddlie, 1998, pp. 149-150). Applying Johnson (2006) consider, as Creswell et al., (2006) mixed designs can be possible in the course of do, these current disagreements as being beneficial carrying out Chen’s (2006) conceptualisation as for the field as it facilitates developing via hard ‘theory-driven evaluations’. Two strategies are theoretical and empitical study. followed for those evaluations. The first is ‘switch Fully mixed research designs, also known as strategy’ which involves running qualitative fully integrated mixed research designs (Tashakkori methods to evaluate the program hypothesis. and Teddlie, 2003; Teddlie and Tashakkori, 2006), The second one is ‘contextual overlaying strategy’ comprise mixing quantitative and qualitative which proposes to use qualitative approachs to approaches in an interactive way at all phases of the gather contextual information with the aim of inquiry (i.e., research goal, kind of data/operations, enabling the interpretation of quantitative data or type of analysis/inference (Onwuegbuzie and mediating findings (Onwuegbuzie and Johnson, Johnson, 2004); so at each stage, one path (e.g., 2006). ‘Conversion’ mixed designs includes the quantitative) effects the formulation of the other transformation of data in which one data form approach (e.g., qualitative). Onwuegbuzie and is transformed into the other and after that it Johnson (2006) agree with Teddlie and Tashakkori is examined (Teddlie and Tashakkori, 2006). (2006) that fully mixed (and almost fully mixed) Namely, the other data form emerges from the designs are appealing in virtue of the multiple original data form by switching the data from aspects of combination. quantitative to qualitative or from qualitative to quantitative. Furthermore, conversion takes place through techniques like ‘quantitizing’ data which involves converting the qualitative data to 2 a numerical form (Tashakkori and Teddlie, 1998) Explain the types of mixed methods or ‘qualitizing’ data which involves transforming research designs in relation to timing, quantitative data into a type that can be analyzed integration and priority and design your qualitatively (Tashakkori and Teddlie, 1998). In research as in the example (Figure 6.5). sum, the researcher analyze, re-analyze and make

162 Research Methods 6 RESEARCH IMPLEMENTATION Interviews are among the main data gathering STAGE types in mixed method design. Interviews are a popular and widely used means of collecting qualitative data. With this in mind, the inquirer Collecting Quantitative and/or aims to obtain authentic information directly from Qualitative Data some informed persons. S/he expects ‘to obtain a Zohrabi (2013) states that collecting data special kind of information and to comprehend has different procedures. Closed ended, open- what is in respondents’ mind’ (Merriam, 1998, p. ended questionnaires, interviews and classroom 71). Since the inquirer is not able to observe the observations are among the basic instruments that participants’ ‘feelings’ and thoughts, interviews are used in the MMR. Conducting various ways may play a key role to understand how people of collecting data can support each other and thus comprehend the world around them. Considering strengthens the validity and dependability of the the key role of interviews, Flick (2006) defines their data. Researchers primarily enhance the items of purpose as ‘revealing existing knowledge in a way questionnaires regarding the goals and questions of that can be expressed in the form of answers and the research. It is a fact that surveys are one of the so become accessible to interpretation’. In general, primary sources of gathering data in any research; it is possible to carry out interviews in two forms: on the other hand, it should be assured by the person to person and group or collective formats. researcher that it is ‘valid, reliable and unambiguous’ Both of these forms of interview can be called goal (Richards and Schmidt, 2002, p. 438). oriented conversation (Merriam, 1998). Questionnaires can be seen in three categories. Deciding on one type of interview instead of The first is the closed ended or structured ones, another relies on the goal of the research, the kind the second is open ended or unstructured ones, of data, or incident under the research, etc. Merriam and the last is a mixture of closed ended and open- (1998) contends that our decision counts on electing ended questionnaires. Actually it can be said that the extent of the structure craved. Four major types closed ended questionnaires give the researcher of interview, informal conversation interview, quantitative or numerical data and open-ended interview guide approach, structured open-ended ones give qualitative or text information. From this interview, closed fixed response interview, are point of view, questionnaires are grouped as ‘seven presented by Patton (1990). In this regard, the basic question types: quantity or information, researcher conducts the informal conversation category, list or multiple choice, scale, ranking, interview without prearranged questions and complex grid or table, and open-ended’ (Blaxter, without any order. Natural flow of conversation Hughes, and Tight, 2006, p. 170). A questionnaire creates the questions. Such an interview can be should comprise both closed ended and open-ended assumed to be seeking in nature and, to be honest, questions to complete each other. Generally, various hard to implement for novice inquirers. Structured ways of applying questionnaires , each of which has open-ended interview stands at the other edge of the its own benefits and drawbacks, are encountered. process. In this type, the questions are structured To exemplify, when a participant of study is distant, in advance with a nearly established order. On the researcher may send the questionnaire by post. the other hand, it can be assumed that structured But the return rate of them become quite low. open-ended interview to be too strict and to prevent Moreover, they can be delivered online. In this researcher from accessing the respondent’s point of process, the rate of return may be slightly higher as it view and perception of the world (Merriam, 1998). is easier for respondents to reply. Using telephone is Furthermore, the closed or fixed response interview also a way of implementing questionnaires. In such resembles to a closed-ended questionnaire in which a way of collecting data, the participant may have the participant only replies to the researcher’s to answer the questions. Eventually, they can also questions in a fixed form and order. Indeed, the be applied face to face during which participants participant is not free to articulate herself/himself. are relatively enforced to answer. Thus, the return Such a way of interviewing is rote and bothers rate is rich and dubious questions can be made clear the participant. Nevertheless, the interview guide instantly by the inquirer (Gillham, 2000). approach stands in the middle of the process. In

163 Mixed Methods Research 6 this way of interview, the cases and questions are Analyzing the Quantitative decided but can be paraphrased in any order on and/or Qualitative Data Using the circumstance. Comparing and contrasting the Quantitative and/or Qualitative gathered data is one of the strengths of the interview Analysis Techniques guide approach (Fraenkel and Wallen, 2003, p. 456). Data collection is quite organized and intimate in Onwuegbuzie and Teddlie (2003) present a this approach. seven-stage conception of the mixed methods data analysis process within the MMR model. Observation is a predetermined research These stages are respectively data reduction, data instrument that is conducted on purpose to help display, data transformation, data correlation, research questions and goals. It is also claimed that data consolidation, data comparison, and data observation is an intention to see the events naturally integration. The first stage, data reduction, includes (Flick, 2006), allows the inquirer to integrate it with decreasing the multi-dimensional characteristics of questionnaires and interviews for getting moderately qualitative data and quantitative data. Data display, impartial original data (Johnson and Turner, 2003, the second stage, comprises defining the qualitative p. 314) and is a type of data ‘triangulation’ so as to data visually through charts, graphs, lists etc. and verify the findings (Merriam, 1998). In this context, quantitative data through tables and graphs. Data the observers can be acknowledged to be studying transformation follows the previous stages in a way the impartial aspects justly (Fraenkel and Wallen that quantitative data are transformed into story- 2003, p. 453). After all, Nation (1997) contends that like data that can be examined qualitatively which the inquirers intend to investigate ‘representations is stated before as ‘qualitized’ by Tashakkori and of behaviour rather than the behaviour itself’. Teddlie (1998) and qualitative data are transformed Without doubt, there are strengths and weaknesses into numeric codes that can be showed statistically of observation as other means of research do have. (Tashakkori and Teddlie, 1998). The next step, data While observational data show original illustration correlation, includes ‘the quantitative data being of happenings, is held in a natural context and correlated with the qualitized data or the qualitative provide the inquirer to get contextual factors. data being correlated with the quantitized data’. Analysing obseravtional data can take so much time, Data consolidation, the fifth stage, involves combining both quantitative and qualitative because observing considerable population may not data to produce new or consolidated variables or be easy. In addition the inclination and sensitivity data sets. Data comparison comes next including of observer may exist and finally researcher may comparing data from qualitative and quantitative have an impact on the pupils and teachers. All in data resources. Data integration, the final stage, all, the observers should decide on the setting so as involves the integration of both quantitative or to conduct observation. The things to be recorded qualitative data into either a consistent whole or need to be identified next and then the researchers two distinct ones (Johnson and Onwuegbuzie, need to obtain “an initial, general presentation 2004, p. 22). In MMR, analyzing the quantitative of the field” (Flick, 2006, p. 217). Consequently, and qualitative data requires the mixing of data it is possible for the inquirer to carry out focused as mentioned in different definitions of MMR to and elected observations related to the goals and make the qualitative and quantitative data ready questions of the study. for analyzing the process the researchers mix them. Mixing the datasets enables the researcher to have a better understanding of the problem than qualitative or quantitative dataset used alone. According to Johnson and Christensen (2004, p. internet 7). The dataset can be mixed in three different ways: merging or converging, connecting and In order to see the use of triangulation in MMR embedding. A diagram that depicts these mixing study, you can see “Triangulation and Mixed data ways, is presented in Figure 6.6 (Johnson and Methods Research” by Mertens and Hesse- Christensen, 2004, p. 7). Biber, 2012) on http://journals.sagepub. As shown in Figure 6.6, when the researchers com/doi/full/10.1177/1558689812437100 merge or converge the two datasets, they actually

164 Research Methods 6 bring them together and then come to a conclusion. On the other hand, by connecting the two datasets, the researchers have one and build it on the other, and then see the results. Apart from these, the researchers can embed one dataset within the other so that one type of data support the other dataset. All in all, in MMR, analyzing quantitative and qualitative data is not enough; the researchers must mix them together in three different ways. As Creswell and Plano Clark (2011) stated that then, they form “a more complete picture of the problem than they do when standing alone”.

1. Merge the data

Qualitative data Results Qualitative data

2. Connect the data

Qualitative data Qualitative data Results

3. Embed the data

Qualitative data Results Qualitative data

Figure 6.6 Three ways of mixing quantitative and qualitative data

Validating/Legitimating the Mixed Research Findings The requirement of describing specific justifying issues which are not related to single method designs emerges due to the problems of presenting and combining in MMR. On the other hand, justifying affairs are discussed in neither Teddlie and Tashakkori’s (2003) and Tashakkori and Teddlie’s (2006) interpretation groundwork nor seem to be included in others. Therefore, a new systematic classification of legitimizing types in MMR for attention, discussion and clarification are to be laid out here. This infant classification includes nine legitimizing types as summarized in Table 6.4 (Onwuegbuzie and Johnson, 2006).

Table 6.4 Typology of mixed methods legitimation types Legitimation Type Description The extent to which the relationship between the quantitative and qualitative sampling 1. Sample Integration designs yields quality meta-inferences The extent to which the researcher accurately presents and appropriately utilizes the 2. Inside-Outside insider’s view and the observer’s views for purposes such as description and explanation. 3. Weakness The extent to which the weakness from one approach is compensated by the strengths Minimization from the other approach. The extent to which one has minimized the potential problem wherein the meta-inferences 4. Sequential could be affected by reversing the sequence of the quantitative and qualitative phases. 5. Conversion The extent to which the quantitizing or qualitizing yields quality meta-inferences. The extent to which the researcher’s epistemological, ontological, axiological, 6. Paradigmatic mixing methodological, and rhetorical beliefs that underlie the quantitative and qualitative approaches are successfully (a) combined or (b) blended into a usable package. The extent to which the meta-inferences made reflect a mixed worldview based on the 7. Commensurability cognitive process of Gestalt switching and integration. The extent to which addressing legitimation of the quantitative and qualitative 8. Multiple Validities components of the study result from the use of quantitative, qualitative, and mixed validity types, yielding high quality meta- inferences. The extent to which the consumers of mixed methods research value the meta-inferences 9. Political stemming from both the quantitative and qualitative components.

165 Mixed Methods Research 6 As shown in Table 6.4, there are different ways to enable researchers to validate the data. However, for multiple validity, researchers must be careful because it requires researchers to use both quantitative and qualitative validity types for their MMR. In accordance with the used methodology and the collected data, researchers can use internal, external, construct, and statistical conclusion validity for their quantitative strand and descriptive, interpretative, and theoretical validity for their qualitative strand of the research (Johnson and Christensen, 2004, p. 668). All in all, in MMR, establishing research validity is a circular and continuous process, in which researchers take necessary precautions by assessing the validity of the data during the study.

Interpreting the Mixed Research Findings After the first data are collected, interpreting the MMR findings starts and continues throughout the research. In a sequential study, firstly the data are collected, secondly interpreted. Then the interpretations may be used developmentally to construct the data collection and interpretation in the second phase. On the other hand, in a concurrently designed MMR study, the qualitative and quantitative data can be interpreted separately or together, depending on the research objective and rationale (Johnson and Christensen, 2004, p. 668). It is necessary to represent the importance of all findings in the MMR article (Onwuegbuzie and Leech, 2004). Regarding the significance of that, an interpretive framework which includes ten aspects of quality is drafted by Teddlie and Tashakkori (2009, pp. 301-302): 1. Appropriateness of design (i.e. ‘Are the methods of study appropriate for answering the research questions?’; ‘Does the design match the research questions?’; ‘Does the mixed methods design match the stated purpose for conducting an integrated study?’; ‘Do the strands of the mixed methods study address the same research questions [or closely related aspects of the research question]?’). 2. Reliability of design (i.e. ‘Are the qualitative, quantitative, and mixed methods procedures or design components... capturing the meanings, effects, or relationships?’). 3. Coherence inside the design (i.e. ‘Do the components of the design fit together in a seamless manner?’; ‘Do the strands of the mixed methods study follow each other [or are they linked] in a logical and seamless manner?’). 4. Analytical competence (i.e., ‘Are the data analysis procedures/strategies appropriate and adequate to provide possible answers to research questions?’; ‘Are the mixed methods strategies implemented effectively?’). 5. Analytical cohesion (i.e., ‘Do the inferences closely follow the relevant findings in terms of type, scope, and intensity?’; ‘Are multiple inferences made on the basis of the same findings consistent with each other?’). 6. Conceptual coherence (i.e., ‘Are the inferences consistent with theory and state of knowledge in the field?’); 7. Explicative concurrence (i.e., ‘Are other scholars likely to reach the same conclusions on the basis of the same results?’; ‘Do the inferences match participants’ constructions?’). 8. Analytical uniqueness (i.e., ‘Is each inference distinctively more credible/plausible than other possible conclusions that might be made on the basis of the same results?’). 9. Combining efficiency (i.e., ‘Do the meta-inferences adequately incorporate the inferences that are made in each strand of the study?’; ‘If there are credible inconsistencies between inferences made within/across strands, are the theoretical explanations for these inconsistencies explored, and possible explanations offered?’). 10. Exploratory agreement (i.e., ‘Do the inferences correspond to the stated purpose/questions of the study?’; ‘Do the inferences made in each strand address the purposes of the study in that strand?’; ‘Do the meta-inferences meet the stated need for using a mixed methods design?’) (Leech et al., 2011)

166 Research Methods 6 Although the MMR report enables readers to important meet a rich content of the related subject, the mixed methods researchers know some possible problems During the interpretation of the findings of involved in writing a MMR report. In this sense, the MMR study, the framework including the Johnson and Christensen (2004, pp. 670-671) following 10 items by Tedllie and Tashakkori have underlined four potential problems the mixed (2009, pp. 301-302), may be functional: methods researchers encounter while writing a 1. Appropriateness of design report. Firstly, writing quantitative (formal) and 2. Reliability of design qualitative (personal) reports require different styles. 3. Coherence inside the design The researchers must keep the balance between the 4. Analytical competence two forms of writing without ruining the integrity 5. Analytical cohesion of the report. Secondly, the readers might not have 6. Conceptual coherence enough knowledge on qualitative, quantitative and 7. Explicative concurrence mixed methods techniques. Therefore, the mixed 8. Analytical uniqueness methods researchers should explain the research 9. Combining efficiency terms and provide useful references, and endnotes 10. Exploratory agreement for the readers. Thirdly, MMR studies are much longer than monomethod ones. It can be a serious problem for the researchers when they want to Writing the Mixed Research publish their reports since there are strict word/page limits in scientific journals. Lastly, the MMR is a Report newly developing field, and some pure qualitative The process of writing the MMR report has or quantitative editors or reviewers might reject similar stages to mono method research reports them because of the philosophy, methodology or where you ground your study on a single research limitations. paradigm ( either entirely quantitative or entirely qualitative). The researchers firstly give the important problem, purpose, importance, and method of the study. However, when it comes to writing findings, While writing the MMR report, the researchers the researchers have two choices: presenting the can come across possible four problems. These two sets of findings and interpretations in separate problems can be explained as follows: sections or by fully integrating them in the same 1. The researchers may not be able to balance section. Then they end up the report by adding between quantitative and qualitative paradigm; conclusion, discussion and suggestions. In short, 2. The researchers may not be able to forecast the the whole parts of the MMR reports should be readers’ quantitative, qualitative and MMR integrated. The researchers ensure that the whole knowledge; sections of the report must be described sufficiently 3. The researchers may not be able to summarize (Johnson and Christensen, 2004, p. 669). A well- big quantitative and qualitative data set in written report should definitely comprise all steps accordance with journal format; and of the MMR process (Onwuegbuzie and Johnson, 4. MMR studies may be rejected by the editors 2006). The delineation of the context where the or referees, who adopt only one research research occured for the reader is vital in this kind of paradigm. study because the way quantitative and qualitative findings correlate and the degree in which meta- inferences can be accomplished become obvious. As all published works do, the reports of MMR need to be precise, entire, original without copying any other studies, and finally should incorporate all ethical considerations of the research.

167 Mixed Methods Research 6 Reviewing Your Mixed Methods Research Study Conducting a MMR can be a complex and very extensive process especially for novice researchers. For this reason, the researchers may need to check whether their MMR steps are parallel with the MMR content, philosophy, design, sampling, etc. In order to figure out if their study fits the MMR, the researchers can follow the checklist below in Table 6.5.

Table 6.5 Quick checklist for your mixed methods research Your Review (Put cross or The steps you must follow while reviewing your mixed methods study tick) 1. Evaluate your topic of the study (Is it suitable for mixed methods research?) 2. Check the philosophy and theoretical foundations of mixed methods research (Do you build up your research into this philosophy?) 3. Describe the goal/purpose/research questions of your study (Are your goal/purpose/ research questions of your study suitable for a mixed methods research?) 4. Choose your research design and organize the strands of your study 5. Be sure that you explain a sample, collecting quantitative/qualitative data, analyzing the data in your each strand in a detailed way 6. Give your rationale for collecting both quantitative and qualitative data 7. Determine timing (Is it sequential/concurrent or multiphase?) 8. Determine the type of interaction (How do you mix the strands of your study? Merged, connected, embedded, within a theoretical framework or within a program objective framework?) 9. Determine the priority (Is quantitative or qualitative dominant or equal?) 10. Explain the overall mixed methods design by citing references 11. Display each strand of your study in a detailed diagram by drawing flow of activities that occurred during your research. 12. If something goes wrong in any stage of your research, turn back to previous stage and check it, find the error, correct it and go ahead…

Note: The information in this table is arranged by means of the literature review on mixed methods and Creswell and Plano Clark (2011, p. 114)

As mentioned in Table 6.5, following these steps by checking each stage will help you to make your study better. Moreover, you can find tiny directing questions in each step, answering these questions will inform you about the progress of your study. The researchers can use these steps during or after their study. If they use them during the study, they will be aware of how their study is going on. If they use them after the study is completed, the steps will be a kind of checklist for them. On the other hand, the researchers can read the steps before doing MMR, they will inform them about genaral framework of MMR. In any case, these steps will work well for the mixed methods researchers.

3 Discuss the possible data analysis techniques in mixed methods research and explain the steps of validating data.

168 Research Methods 6

Explain basic process of conducting mixed methods research and explain LO 1 reasons for using mixed methods research.

MMR is a comprehensive process, which starts with determining research questions based on the research topic, purpose and objectives. The research questions are in central position since they lead directly the rest of the process. After determining the rationale of the study and the research design, the researchers select the sampling, collect and analyse the data, interpret the findings and write the report based on the research purpose and the objectives. However, MMR is not always Summary a linear process as shown in Figure 6.1. The researchers sometimes go back to previous steps in order to check the process or usually match research questions up with the steps they have already done. Five major objectives or motivations exist for conducting MMR. The first one is ‘triangulation’ which provides validation and concurrence of results gained from various methods and designs investigating the same circumstance. ‘Complementarity’, as the second motivation, seeks for further detailing, improvement, interpretation and description of the results obtained from one method with results from the other method. The third one is ‘initiation’, which means unearthing the bias and discrepancies that cause to re-designing of the research question. ‘Development’, the fourth motivation, includes utilizing the findings from one method to deliver clues to the other method. The last motivation ‘expansion’ seeks to expand the size of research with different methods for different investigation factors.

Figure out the mixed methods research designs, compare these designs LO 2 in the context of timing, priority and integrity and write their own mixed methods research design as in the example.

MMR designs can be shaped by time orientation and paradigm emphasis. In terms of time orientation, qualitative and quantitative phases of the study occur almost at the same time (concurrently) or they are organized into phases over time (sequentially). When it comes to research paradigm, the qualitative and quantitative parts of the study are given virtually equal emphasis or one paradigm can be stronger than the other (a qualitatively driven design or quantitatively driven design). In this process, the researchers make two major decisions. First, they decide whether they want to operate largely within one primary paradigm or not. Second, they decide whether they want to conduct the qualitative and quantitative components concurrently or sequentially. The mixed methods researchers choose a research design, which reflects timing, combination, interaction of quantitative and qualitative methods. In this context, six different mixed methods research designs, which help researchers to design their strands of mixed methods research: Convergent, Explanatory, Exploratory, Embedded, Transformative, Multiphase designs. The goal of the study determines which research design to use. The researchers can use the convergent design in order to make people understand the topic more completely or validate the quantitative scales. While the explanatory design is used to explain quantitative results, the exploratory design is selected to test the qualitative exploratory findings. The embedded design is preferred in an experimental study for preliminary exploration or follow-up explanation or more complete understanding of experimental trial. On the other hand, transformative mixed methods research is conducted to describe the only for this? Lastly, multiple phase design is implemented to develop, adapt or evaluate programs and multilevel statewide studies.

169 Mixed Methods Research 6

Identify the data collection and analysis proces of mixed methods LO 3 research.

Collecting data has different procedures. Closed ended, open-ended questionnaires, interviews and observations are among the basic instruments that are used in the MMR. Conducting various ways of collecting data can support each other and thus strengthens the validity and dependability of the data. In MMR, analyzing the quantitative and qualitative data requires the mixing of data as mentioned in different definitions of MMR. Therefore, in order to make the qualitative and quantitative data ready for the data analysis process, the researchers mix them. Mixing the datasets enables the researcher to have a better understanding of the problem than qualitative or quantitative dataset used alone. The dataset can be mixed in three different ways: merging or converging, connecting and embedding. Summary

Interpret the mixed methods research data and how to write their own LO 4 research report

In MMR, after the first data are collected, interpreting the findings starts and continues throughout the research. In a sequential study, firstly the data are collected, secondly interpreted. Then the interpretations may be used developmentally to construct the data collection and interpretation in the second phase. On the other hand, in a concurrently designed MMR study, the qualitative and quantitative data can be interpreted separately or together, depending on the research objective and rationale. A well-written report should definitely comprise all steps of the MMR process. The delineation of the context where the research occured for the reader is vital in this kind of study because the way quantitative and qualitative findings correlate and the degree in which meta-inferences can be accomplished becomes obvious. As all published works do, the reports of MMR need to be precise, entire, original without copying any other studies and finally should incorporate all ethical considerations of the research.

170 Research Methods 6

1 Which of the following terms can not refer to 5 In exploratory design, ------data collection mixed methods research? and analysis follow up with ------data collection and analysis. a. Mixed methodology b. Multimethod research a. quantitative/qualitative c. Multiplism b. qualitative/interpretive Test Yourself d. Methodological pluralism c. comparative/quantitative e. Multiphase research d. narrative/quantitative e. qualitative/quantitative 2 Which one of the following rationales can be the reason for using mixed methods research? 6 Which of the following rationales can not be a. Triangulation the reason for using mixed methods research? b. Improvement a. Insufficient for discovery and validation c. Complementarity b. Contributes to intensive knowledge d. Initiation c. Requires more budget and time in face to face e. Development interviews d. Participants’ perception of anonymity is 3 In convergent design wich of the following possibly low can not be done by the convergent design e. Time consuming for open ended items researchers? Which one of the following can be a way of a. conduct qualitative and quantitative research 7 mixing quantitative and qualitative data? sequentially b. give equal status to qualitative and quantitative a. Integrating data strands b. Merging data c. analyse the qualitative and quantitative data c. Comparing data separately d. Contrasting data d. conduct qualitative and quantitative research e. Supporting data concurrently e. make general comments by combining the 8 The researchers can legitimate the data. To do results. it, which one of the following can not be used? a. Minimizing weakness 4 In ------design, the researchers can use the supporting phase in order to develop the major b. Mixing paradigms research in some way. c. Combining quantitative and qualitative validity types a. explanatory d. Reversing the sequence of the quantitative and b. exploratory qualitative phases c. embedded e. Utilizing only observer’s views d. convergent e. transformative

171 Mixed Methods Research 6

While the researchers interpret the mixed 9 While writing mixed methods report, the methods research findings, they should follow 10 researchers may encounter some problems. Which some steps. Which one of the following can not be one of the following can not be a problem while a part of mixed methods interpretive framework? writing mixed methods report? a. Analytical competence a. Writing styles (quantitative/qualitative) b. Conceptual coherence b. Lack of knowledge about quantitative, c. Explanatory cohesion qualitative or mixed methods research d. Exploratory agreement c. The length of the report e. Combining efficiency d. Rich content e. Fear of rejection of your paper Test Yourself Test

172 Research Methods 6 If your answer is incorrect, review the If your answer is incorrect, review the section 1. e 6. b section on “Collecting Quantitative and/or on “Overvıew of Mixed Research”. Qualitative Data”.

If your answer is incorrect, review the section Answer Key for “Test Yourself” If your answer is incorrect, review the section 7. b on “Analyzing the Quantitative and/or 2. b on “What is the reason for using mixed Qualitative Data Using Quantitative and/or methods?”. Qualitative Analysis Techniques”. If your answer is incorrect, review the section If your answer is incorrect, review the section 3. a 8. e on “Validating/Legitimating the Mixed on “Selecting the Mixed Research Design”. Research Findings”.

If your answer is incorrect, review the section If your answer is incorrect, review the 4. c on “Selecting the Mixed Research Design”. 9. c section on “Interpreting the Mixed Research Findings”.

If your answer is incorrect, review the section 5. e 10. d If your answer is incorrect, review the section on “Selecting the Mixed Research Design”. on “Writing the Mixed Research Report”. Suggested answers for “Your turn” Explain the steps in conducting mixed methods research by giving strengths and weaknesses in the process.

Mixed methods research is the third research paradigm that partners with the philosophy of pragmatism. Moreover, it provides the most informative, complete, balanced, and useful research results when compared to traditional quantitative and qualitative research. Mixed methods research is a comprehensive process, which starts with determining research questions based on research topic, purpose and objectives. The research questions are in central position since they lead directly the rest of the process. After determining the rationale of the study and the research design, the researchers select the sampling, collect and analyse the data, interpret the findings and write the report based on the research purpose and the objectives. MMR also incorporates and your turn 1 grows different research strategies used with a specific study that broadens the dimensions and scope of the research so that the researcher can accelerate the understanding and accomplish the aims quickly, makes the differences between quantitative results and qualitative findings clear, promotes the active involvement of the participants, fosters the multidisciplinary interaction and finally enables extensive and prosperous data. However, conducting the mixed methods research can be hard to handle at times. The researchers have to spend a long time to conduct a mixed methods research because they need to collect and analyze both quantitative and qualitative data. Furthermore, it requires a clear explanation of procedures since the readers may have difficulty in figuring out the qualitative and quantitative proces.

173 Mixed Methods Research 6

Explain the types of mixed methods research designs in relation to timing, integration and priority and design your research as in the example (Figure 6.5).

Although many more mixed research designs are still being created, there are several competing typologies of mixed research. Mixed methods research designs can be shaped by time orientation and paradigm emphasis. In terms of time orientation, qualitative and quantitative phases of the study occur concurrently or sequentially. When it comes to research paradigm, the study can be a qualitatively driven design or quantitatively driven design. In addition to Johnson and Christensen (2004), Creswell and Plano Clark your turn 2 (2011) have pointed out that there are six mixed methods research designs: 1) Convergent design, 2) Explanatory design, 3) Exploratory design, 4) Embedded design, 5) Transformative design, and 6) Multiphase design. You can find the details in Table 6.3. You can design your mixed methods study as in the Figure 6.6 by giving the name of the design, and reasons for using it, discussing interaction, timing and priority, explaining it in figures and citing methodological references. Suggested answers for “Your turn” Suggested answers for “Your

Discuss the possible data analysis techniques in mixed methods research and explain the types of legitimating data.

The researchers can use three different techniques while analyzing mixed methods data: merge, connect or embed the data. They merge or converge the two datasets, they actually bring them together and then come to conclusion. On the other hand, by connecting the two datasets, the researchers have one and build it on the other and then see the results. Apart from these, the researchers can embed one dataset within the other so that one type of data supports the other dataset. In mixed methods research, simply collecting and your turn 3 analyzing quantitative and qualitative data are not enough; the researchers must mix them together in three different ways. As Creswell and Plano Clark (2011) have stated that then, they form “a more complete picture of the problem than they do when standing alone”. In order to legitimate the data, the researchers can use sample integration, inside-outside, weakness minimization, sequential, conversion, paradigmatic mixing, commensurability, multiple validities, political legitimation.

174 Research Methods 6

Further Reading

Creswell, J. W. (2003). Research design: Qualitative, quantitative, and mixed methods approaches (2nd ed.). Thousand Oaks, CA: Sage. Smart, J. (Ed.). (1996). Higher education: Handbook of theory and research. New York: Agathon Press. Tashakkori, A., & Teddlie, C. (Eds.). (2003). Handbook of mixed methods in social and behavioral research. Thousand Oaks, CA: Sage. Tashakkori, A., & Teddlie, C. (2006, April). Validity issues in mixed methods research: Calling for an integrative framework. Paper presented at the annual meeting of the American Educational Research Association, San Francisco

References

Andrew, S., & Halcomb, E. (2006). Mixed methods Creswell J. W., & Plano Clark, V. L. (2011). Designing research is an effective method of enquiry for and conducting mixed methods research. Thousand community health research. Contemporary Nurse: Oaks, CA: Sage. A Journal for The Australian Nursing Profession, Creswell, J. W. (2013). Qualitative inquiry & research 23(2),145-53. design: choosing among five approaches (3rd ed.). Blaxter, L., Hughes, C., & Tight, M. (2006). How to Thousand Oaks, CA: Sage. research. Berkshire: Open University Press. Creswell, J. W., Shope, R., Plano Clark, V. L., & Chen, H. T. (2006). A theory-driven evaluation Green, D. O. (2006). How interpretive qualitative perspective on mixed methods research. Research research extends mixed methods research. Research in The Schools, 13(1), 75-83. in The Schools, 13 (1), 1- 11. Çınkır, Ş., Erdemli, Ö., Kurum, G. & Güner Demir, Curtis, S., Gesler V., Smith, G., & Washburn, S. T. (2018). Syrian students’ integration into Turkish (2000). Approaches to sampling and case selection education system: Barriers, school policies and in qualitative research: Examples in the geography of support strategies. Paper accepted as presentation at Social Science & Medicine, 50 MMIRA Congress 2018 held in Vien. health. (7-8), 1001-14. Clark, V. L. P., & Ivankova, N. V. (2016). Mixed Firestone, W. A. (1993). Alternative arguments for methods research: A guide to the field. UK: Sage. generalizing from data as applied to qualitative research. Educational Researcher, 22(4), 16-23. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Flick, U. (2006). An introduction to qualitative Lawrence Earlbaum Associates. research. London: Sage Collins, K. M. T., Onwuegbuzie, A. J., & Sutton, I. L. Fraenkel, J. R., & Wallen, N. E. (2003). How to design (2006). A model incorporating the rationale and and evaluate research in education. (5th ed.). New purpose for conducting mixed methods research York: McGraw-Hill. in special education and beyond. Learning Disabilities: A Contemporary Journal, 4, 67-100. Gillham, B. (2000). Developing a questionnaire. Collins, K. M. T., Onwuegbuzie, A. J., & Jiao, Q. G. London: Continuum. (2006). Prevalence of mixed-methods sampling Greene, J. C., Caracelli, V. J., & Graham, W. F. designs in social science research. Evaluation & (1989). Toward a conceptual framework for Research in Education, 19(2), 83-101. mixed-method evaluation designs. Educational Collins, K. M. T., Onwuegbuzie, A.J., & Jiao, Q. Evaluation and Policy Analysis, 11(3), 255-274. G., (2007). A mixed methods investigation of ). mixed methods sampling designs in social and Johnson, R. B., & Christensen, L. B. (2004 health science research. Journal of Mixed Methods Educational research: Quantitative, qualitative, and Research, 1(3), 267-294. mixed approaches. Boston: Allyn & Bacon.

175 Mixed Methods Research 6 Johnson, R. B., & Onwuegbuzie, A. J. (2004). Mixed Onwuegbuzie, A. J., & Collins, K. M. T. (2007). A methods research: A research paradigm whose typology of mixed methods sampling designs in time has come. Educational Researcher, 33(7), 14- social science research. The Qualitative Report, 12, 26. 281-316. Avaliable at: http://www.nova.edu/ssss/ QR/QR12-2/Onwuegbuzie2.pdf Johnson, R. B., & Turner, L. A. (2003). Data collection strategies in mixed methods research. Onwuegbuzie, A. J., & Johnson, R. B. (2004, April). Validity issues in mixed methods research. Paper In A.Tashakkori, & C. Teddlie (Eds.), Handbook presented at the annual meeting of the American of mixed methods in social and behavioral research Educational Research Association, San Diego, (pp. 297-319). Thousand Oaks, CA: Sage. CA. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. Onwuegbuzie, A. J., & Johnson, R. B. (2006). The (2007). Toward a definition of mixed methods validity issue in mixed research. Research in the research. Journal of Mixed Methods Research, 1(2), Schools, 13(1), 48-63. 112-133. Onwuegbuzie, A. J., & Leech, N. L. (2004). Enhancing Leech, N. L., & Onwuegbuzie, A. J. (2010). The the interpretation of significant findings: The mixed research journey: From where we started role of mixed methods research. The Qualitative to where we hope to go. International Journal of Report, 9(4), 770-792. Avaliable at: https://core. Multiple Research Approaches, 4, 73-88. ac.uk/download/pdf/51087771.pdf Leech, N. L., Onwuegbuzie, A. J., & Combs, J. Onwuegbuzie, A. J., & Leech, N. L. (2005). On P. (2011). Writing publishable mixed research becoming a pragmatist researcher: The importance articles: Guidelines for emerging scholars in the of combining quantitative and qualitative research health sciences and beyond. International Journal methodologies. International Journal of Social of Multiple Research Approaches, 5(1), 7-24. Research Methodology: Theory & Practice, 8, 375- 387 Merriam, S. B. (1998). Qualitative research and case study applications in education. San Francisco: Onwuegbuzie, A. J., & Leech, N. L. (2006, Jossey-Bass. September). Linking research questions to mixed methods data analysis procedures. The Qualitative Mertens, D. M. (2010). Research and evaluation in Report, 11(3), 474-498. Avaliable at: http://www. education and psychology: Integrating diversity with nova.edu/ssss/QR/QR11-3/onwuegbuzie.pdf quantitative, qualitative, and mixed methods (3rd ed.). Thousand Oaks, CA: Sage. Onwuegbuzie, A. J., & Teddlie, C. (2003). A framework for analyzing data in mixed methods Miles, M. B., & Huberman, A. M. (1994). Qualitative research. In A. Tashakkori, & C. Teddlie (Eds.), data analysis: An expanded sourcebook. London: Handbook of Mixed Methods in Social and Sage. Behavioral Research (pp. 351–383). Thousand Morse, J. M. (2003). Principles of mixed methods and Oaks, CA: Sage. multi-method re search design. In C. Teddlie, & Patton, M. Q. (1990). Qualitative evaluation and A. Tashakkori (Eds.), Handbook of mixed methods research methods (2nd ed.), Newbury Park, CA: in social and behavioral research (pp. 189-208). Sage. Thousand Oaks, CA: Sage. Plano Clark, V. L., & Badice M. (2010). Research Nation, J. R. (1997). Research methods. New Jersey: questions in mixed methods research. In A. Prentice Hall. Tashakkori & C. Teddlie (Eds.), Handbook of Newman, I., Ridenour, C. S., Newman, C., & Mixed Methods in Social and Behavioral Research nd DeMarco, G. M. P., Jr. (2003). A typology of (2 ed., pp. 275-304). Thousand Oaks, CA: Sage. research purposes and its relationship to mixed Powell, H., Mihalas, S., Onwuegbuzie, A. J., Suldo, methods. In A. Tashakkori & C. Teddlie (Eds.), S., & Daley, C. E. (2008). Mixed methods Handbook of mixed methods in social and behavioral research in school psychology: A mixed methods research (pp. 167–188). Thousand Oaks, CA: investigation of trends in the literature. Psychology Sage. in the Schools, 45(4), 291-309.

176 Research Methods 6 Pritchard, S. F. (2013). A mixed methods investigation of Teddlie, C. & Tashakkori, A. (2009). Foundations of introductory music education courses, preservice music mixed methods research. Thousand Oaks, CA: Sage. teaching efficacy beliefs, and commitment to music Teddlie, C.; &Tashakkori, A. (2010). Overview of teaching. Unpublished Dissertation. Faculty of the Graduate School, University of Colorado. USA. contemporary issues in mixed methods research. Quantitative approaches. Thousand Oaks, CA: Sage. In A. Tashakkori, & C. Teddlie (Eds.). Handbook of Mixed Methods in Social & Behavioral Research Richards, J. C. & Schmidt, R. (2002). Longman (pp 1-41). Thousand Oaks, CA: Sage. dictionary of language teaching and applied linguistics. (3rd ed.). London: Longman. Teddlie, C., & Yu, F. (2007). Mixed methods sampling: A typology with examples. Journal of Sandelowski M, Voils C.I., & Barroso J. (2006). Mixed Methods Research, 1(1), 77-100. Defining and designing mixed research synthesis studies. Research in the Schools. 13, 29-40. Wisdom, J., & Creswell J.W. (2013). Mixed methods: Integrating quantitative and qualitative data Seliger, H. W., & Shohamy, E. (1989). Second language collection and analysis while studying patient- research methods. Oxford: OUP. centered medical home models. Rockville, MD: Tashakkori, A., & Creswell, J. W. (2007). The new Agency for Healthcare Research and Quality. era of mixed methods [Editorial]. Journal of Mixed AHRQ Publication No. 13-0028-EF. Methods Research, 1(1), 3-7. Wisdom, J. P., Cavaleri, M. A., Onwuegbuzie, A. J., Tashakkori, A., & Teddlie, C. (1998). Mixed & Green, C. A. (2011). Methodological reporting methodology: Combining qualitative and in qualitative, quantitative, and mixed methods quantitative approaches (Vol. 46). Thousand Oaks, health services research articles. Health Services CA: Sage. Research, 47(2), 721-745. Teddlie, C., & Tashakkori, A. (2003). Major issues Yin, R. K. (2006). Mixed methods research: Are the methods genuinely integrated or merely parallel. and controversies in the use of mixed methods Research in the Schools, 13(1), 41-47. in the social and behavioral sciences. In A. Tashakkori, & C. Teddlie (Eds.), Handbook of Zohrabi, M. (2013). Mixed method research: Mixed Methods in Social and Behavioral Research Instruments, validity, reliability and reporting (pp. 3-50). Thousand Oaks, CA: Sage. findings. Theory & Practice in Language Studies, 3(2), 254-262. Teddlie, C., & Tashakkori, A. (2006). A general typology of research designs featuring mixed methods. Research in the Schools, 13(1), 12-28.

177 Chapter 7 Research Ethics After completing this chapter, you will be able to:

Develop a critical understanding of the historical background of research ethics with regard to the six famous research ethics Comprehend the nature and meaning of scandals and the establishment of research 1 research ethics 2 ethics committees

Distinguish between the six major ethical perspectives (i.e., deontological ethics, Differentiate between the three major ethical consequentialist/utilitarian ethics, principle- principles (i.e., autonomy and informed based ethics, situation-based ethics, virtue consent, balancing research risks and benefits, 3 ethics, and relational ethics) 4 and privacy, anonymity and confidentiality) Learning Outcomes Recognize the unique characteristics of qualitative research in relationship to the need 5 for reflective thinking in research ethics

Chapter Outline Introduction Key Terms • Anonymity • Autonomy Understanding Research Ethics • Confidentiality • Consent History of Research Ethics • Deontology • Ethical dilemma Ethical Perspectives • Ethical scandal • Ethics committee Ethical Principles • Ethics of care • Privacy Ethics in Qualitative Research • Reflexivity • Utilitarianism • Virtue ethics

178 Research Methods 7 INTRODUCTION account when doing social research’ or ‘the set of ethical principles held by social researchers’.” Research is a systematic activity aiming to Hammersley and Traianou (2012, p. 17) also state generate new knowledge or understanding in a that the word “ethical” can refer to two things. field of study. According to European Commission First, it can refer to “The realm of considerations (2010, p. 14), for example, “Research aims relevant to determining what is good or bad, right to generate (new) information, knowledge, or wrong. Here, the contrast is with non-ethical understanding, or some other relevant cognitive matters, for example those concerned with what good, and does so by means of a systematic would be efficient, effective, prudent, expedient, investigation.” Yet a fundamental aspect of research conventional, or good etiquette.” Alternatively, the is uncertainty in relation to its potential benefits and word “ethical” can mean “What is good or right, the risks involved for the participants. As Burges as contrasted with the unethical- what is bad or (2005a, p. viii) also state, “Much of the literature on wrong.” Here the terms “right” and “wrong” refer social and educational research is concerned with to what should and should not be done regardless the design, collection and analysis of research data. of the consequences of an action, while the terms While many of the discussions focus on technical “good” and “bad” refer to the results and/or procedures, they avoid ethical questions about the consequences of that action. conduct of social investigation.” What this means is that ethical issues are involved in all stages of scientific research from the initial research design Research Ethics as Means of and formulation of the research problem through Judgment and/or Evaluation to the acquisition and dissemination/publication Research ethics is a matter of judgment and/ of research data. In fact, “The only safe way to or evaluation in terms of right/wrong actions avoid violating principles of professional ethics is (of researchers) and/or good/bad consequences to refrain from doing social research altogether” (resulting from those actions). In fact, “Making (Bronfenbrenner, 1952; cited in Hammersley and evaluations is central to all aspects of human life- we Traianou, 2012, p. vi). From this point of view, are continually engaged in judging things, including this chapter is about the ethics of scientific research ourselves and other people. Furthermore, our involving human beings. It provides a discussion evaluations can vary in several important respects; mainly on the ethics of social research in general. notably according to what sort of thing is being evaluated (for example, persons, courses of action, outcomes, institutions, cultures, societies, etc.), and UNDERSTANDING RESEARCH in terms of what standard (for instance, in the case ETHICS of actions this may be according to what is needed, In the literature, there is no single or agreed what is pleasurable, what is fair, what is courageous, definition of the term “research ethics.” This etc.)” (Hammersley and Traianou, 2012, p. 19). is because there are some ambiguities and In this regard, Patton (2002) provides a very complexities surrounding the words “ethics” and illuminating example (see Box 7.1) about the “ethical.” According to Hammersley and Traianou meaning of evaluation and/or judgment. He (2012, p. 16), for example, the word “ethics” can provides a dialogue that took place between him have at least two meanings. It can refer to “A field and a participant in a workshop about the purpose of study, concerned with investigating what is good of evaluation (with regard to the distinction or right and how we should determine this. On this between formative and summative types of interpretation, ‘social research ethics’ means the evaluation) by using the metaphor of “fishing.” As study of what researchers ought and ought not to Patton (2002) clearly illustrates in Box 7.1, every do, and how this should be decided.” Alternatively, action starts with some aims/intentions but may the word “ethics” can mean “A set of principles end up with different results/consequences. So that embody or exemplify what is good or right, or the main question here is “What should be the allow us to identify what is bad or wrong… In these priority?”: Aims/intentions for an action/activity terms, the phrase ‘social research ethics’ means ‘the (be it evaluation, fishing, or research, etc.) or the set of ethical principles that should be taken into results/consequences of that action/activity.

179 Research Ethics 7 Box 7.1 The purpose of evaluation/judgment

MQP: What’s the purpose of fishing, the desired outcome? Jack: To catch fish. MQP: What if you don’t get any fish, was the day a loss? Jack: No, because you got to spend time outdoors, maybe with family, like when I took my son fishing when he was young. MQP: So, if you don’t attain your original goal, you decide that other goals were equally or even more important? Jack: Yeah, I suppose so. MQP: … Now suppose you do catch some fish, does it matter what kind or how many? Jack: Absolutely. You want Northern pike or walleyes, unless you’re fishing for bass. They have to be of legal size, then the bigger the better. And, ideally, you’d like to catch the limit. MQP: So you have some specific targets in mind for success. Jack: Usually. MQP: And does it only matter whether you catch fish, or are there elements of how you fish that are important? Jack: Sure, it matters what kind of bait you use, what your equipment is like, how much struggle the fish gives you, how much skill it takes to land it, how good the weather is, who else is with you, things like that. MQP: So the process matters and there are qualitative dimensions to the process. Jack: You could say that. MQP: And I take it that one of the challenges is matching the right kind of bait and equipment to the kind of fish you want to catch. Jack: That’s pretty much the essence of fishing. MQP: Have you ever had an argument with someone about whether the fish you caught was due to luck or your skill? Jack: Just about every time I’ve gone fishing... MQP: … Are there times when you go out to practice fishing or improve your technique where you don’t worry so much about catching fish versus other times when expectations are high that you will bring home dinner? Jack: When I was younger that was the case. Not so much these days. I doubt that I’ll get much better or find new places to fish, but in my younger days I did a lot of practicing, especially fly fishing, and part of the thrill was finding a good new location where the fish were biting. MQP: You mentioned teaching your son to fish. Did you want him to learn anything more than just how to fish? Jack: Absolutely. I wanted him to learn patience, to love nature, and to have learned lessons about safety. MQP: So catching fish is not just an outcome, it can also be a means to other outcomes you care about... What about the cost–benefit ratio? People spend hundreds of dollars on equipment, thousands on a boat, to get something that costs $10-12 a pound at the market. Jack: You can’t look at it like that. There’s more to it than just getting the fish. MQP: So, we’re back to the goals and purposes of fishing.

Source: Patton (2002, pp. 94-95)

In the same vein, the two of the most influential theoretical frameworks about research ethics (i.e., deontological and consequentialist/utilitarian ethics perspectives) differ very much in their priorities when judging/evaluating a research proposal or activity. For example, “Deontological views insist that intentions must be evaluated in terms of conformity to general rules about right action, whereas consequentialist [and/or utilitarian] positions require that actions be assessed in terms of the desirability [and/or utility] of outcomes” (Hammersley and Traianou, 2012, p. 20).

180 Research Methods 7 important Deontology An ethical perspective that judges researchers’ One of the most important aims actions according to some absolute/universal of research ethics is to protect duties requiring certain types of action. participants from all kinds of risks. Consequentialism An ethical perspective that evaluates researchers’ actions according to their consequences. HISTORY OF RESEARCH ETHICS Utilitarianism The phenomenon of “research ethics” was An ethical perspective that judges actions mostly resulted from the revelations regarding of researchers according to which they the appalling medical experiments conducted produce most happiness and/or utility for the by some Nazi doctors in German concentration participants. camps during the World War II. For example, they “killed Gypsy twin teenagers in order to determine Research Ethics as Protecting the why some of them had differently colored eyes. Rights of Research Participants Prisoners of war were forced to drink seawater in order to find out how long a man might survive The term “research ethics” has also been without fresh water” (Schüklenk, 2005, p. 2). associated with how researchers ought to treat As a result of such experiments, firstly, the research participants in terms of protecting Nuremberg Code of 1947 (https://history.nih. their rights (such as causing them no harm, gov/research/downloads/nuremberg.pdf) was respecting their autonomy, or preserving their developed to outline some basic ethical principles privacy, etc.). According to Schüklenk (2005, (e.g., informed consent, avoiding harm, respecting p. 3), for example, “Research ethics is basically privacy) that were anticipated to guide medical about means of ensuring that vulnerable people experiments. Afterwards, the Nuremburg Code’s are protected from exploitation and other forms ethical principles were clarified and supplemented of harm.” Similarly, Hammersley and Traianou in the World Medical Association’s Helsinki (2012) conceptualize “research ethics” as a form of Declaration of 1964 (https://www.wma.net/wp- “occupational/professional ethics” and assert that a content/uploads/2016/11/DoH-Oct2013-JAMA. distinction needs to be made between “intrinsic” pdf) as well as in the National Commission’s and “extrinsic” values of research. Intrinsic values Belmont Report of 1979 in the USA (https:// (such as dedication, objectivity, and independence www.hhs.gov/ohrp/sites/default/files/the- of researchers) relate to the goal of research (i.e., belmont-report-508c_FINAL.pdf). Initially, the the production of knowledge) while extrinsic ethical codes and guidelines were aimed at only values relate to how research participants ought biomedical research, but in many countries they to be treated in the course of research (such as were extended to all respecting for autonomy, protection of privacy, research involving important and/or minimization of harm). humans, including both quantitate and The Nuremberg Code qualitative research is the most important Autonomy of all kinds (such as milestone in research The capacity of a participant to make a surveys, interviews, ethics. Because the ethic reasoned decision for himself/herself, for and/or observations of codes developed in example, whether to take part in a research human behavior). All different areas of social voluntarily. of these developments sciences were based on have also resulted the Nuremberg Code.

181 Research Ethics 7 in the production of ethics codes by some social research literature (mostly found in medicine and/or science associations as well (see the “Further experimental psychology) is vital as they have led to Reading” section for a list of some social science the development of ethical codes of conduct as well associations’ ethics codes and their web addresses). as the establishment of Research Ethics Committees (RECs) and/or Institutional Review Boards (IRBs). They also serve as a pedagogical tool in terms of representing bad/wrong actions in research ethics. internet The Declaration of Helsinki. Available at: https://www.wma.net/wpcontent/ Research ethics committee (REC) uploads/2016/11/DoH-Oct2013-JAMA.pdf A committee whose role is to conduct ethical review of research.

The Little Albert Experiment in 1920 Consent An 11-month-old infant (named Albert B.), A voluntarily and competently given without the knowledge or consent of his parents, permission to be researched. was conditioned to fear rats by associating them with fear-inducing circumstances such as a loud noise. This experiment was conducted by John B. Overview of Research Ethics Watson and Rosalie Rayner in 1920 and aimed Scandals to answer the following three questions (Harris, A widely accepted definition of “research 1979, p. 151): misconduct”, as it appears in the USA Federal • Can an infant be conditioned to fear an Policy on Research Misconduct (https://www.aps. animal that appears simultaneously with a org/policy/statements/upload/federalpolicy.pdf) is loud, fear-arousing sound? as follows: • Would such fear transfer to other animals, • Research misconduct is defined as or to inanimate objects? fabrication, falsification, or plagiarism • How long would such fears persist? in proposing, performing, or reviewing First, at approximately nine months of age, research, or in reporting research results. “Albert was tested and was judged to show no • Fabrication is making up data or results and fear when successively observing a number of live animals (e.g., a rat, a rabbit, a dog, and a monkey), recording or reporting them. and various inanimate objects (e.g., cotton, human • Falsification is manipulating research masks, a burning newspaper). He was, however, materials, equipment, or processes, or judged to show fear whenever a long steel bar changing or omitting data or results was unexpectedly struck with a claw hammer such that the research is not accurately just behind his back. Two months after testing represented in the research record. Albert’s apparently unconditioned reactions to • Plagiarism is the appropriation of another various stimuli, Watson and Rayner attempted to person’s ideas, processes, results, or words condition him to fear a white rat. This was done without giving appropriate credit. by presenting a white rat to Albert, followed by • Research misconduct does not include a loud clanging sound (of the hammer and steel honest error or honest difference of opinion. bar) whenever Albert touched the animal. After Research misconduct is also conceived as a seven pairings of the rat and noise (in two sessions, deliberate deception and/or exploitation of research one week apart), Albert reacted with crying and participants. Hence, a review of some famous avoidance when the rat was presented without the examples of research ethics scandals in the scientific loud noise” (Harris, 1979, p. 151).

182 Research Methods 7 The Nazi Experiments Between 1939 Milgram (1963) published his first findings in and 1945 a 1963-dated article revealing that 65% of a group The Nazi medical experiments were carried out of ordinary 40 males (such as postal clerks, high on concentration camp prisoners in Germany during school teachers, salesmen, engineers, and laborers, the World War II. They “included involuntary between the ages of 20 and 50) were willing to sterilization, subjection to radiation, freezing to give another person electrical shocks up to 450 induce hypothermia, infection of research subjects volts if prompted to do so by an experimenter. with malaria and tuberculosis (TB), and many However, Milgram conducted a total of 18 distinct other unethical experiments, conducted without experiments with a total of approximately 1000 the consent of the research subjects, and often subjects at Yale University between 1960 and leading predictably to extreme pain, mutilation 1963 (Cave and Holm, 2003). So in order to test and death” (European Commission, 2010, p. 16). the hypothesis of “If X tells Y to hurt Z, under what conditions will Y carry out the command In her book “Doctors from hell: The horrific of X and under what conditions will he refuse?” account of Nazi experiments on humans”, Vivien experimentally, Milgram (1974; cited in Cave Spitz, who was a 22-year-old court reporter during and Holm, 2003, p. 28-29) created the following the doctors’ trials at Nuremberg, retell in detail experimental design: the horrific human experiments conducted by 20 so-called physicians and medical assistants in Two people come to a psychology laboratory to take part in a study of memory and learning. Germany under the direction of the Nazis (Marks, One of them is designated as a ‘teacher’ and 2006). According to Spitz, the Nazi medical the other a ‘learner.’ The experimenter explains experiments “included ‘high-altitude’ experiments that the study is concerned with the effects in which concentration camp inmates were forced, of punishment on learning. The learner is without oxygen, into high-altitude chambers conducted into a room, seated in a chair, his that duplicated conditions at up to 68,000 feet; arms strapped to prevent excessive movement, removal of sections of bone, muscle, and nerves, and an electrode attached to his wrist. He is told including whole legs removed at the hips to that he is to learn a list of word pairs; whenever transplant to other victims; artificial wounding he makes an error, he will receive electric shocks and exposure to mustard gas; wounding of two of increasing intensity. The real focus of the limbs and treatment of one but not the other with experiment is the teacher. After watching the sulfonamide antibiotics; intramuscular injection learner being strapped into place, he is taken into the main experimental room and seated with fresh typhus; and collection of skeletons before an impressive shock generator. Its main from 112 live Jewish inmates who were killed and feature is a horizontal line of thirty switches, defleshed” (Marks, 2006, p. 2). ranging from 15 volts to 450 volts, in 15-volt increments. There are also verbal designations The Milgram Obedience Experiments which range from SLIGHT SHOCK to DANGER-SEVERE SHOCK. The teacher is Between 1960 and 1963 told that he is to administer the learning test to The Milgram obedience experiments were the man in the other room. When the learner conducted to investigate people’s obedience to responds correctly, the teacher moves on to the authority; in other words, the aim was to see how next item; when the other man gives an incorrect far a person would be willing to harm another answer, the teacher is to give him an electric person. However, the research participants were shock. He is to start at the lowest shock level deceived about the nature of the research and led to (15 volts) and to increase the level each time the man makes an error, going through 30 volts, believe that they were administering electric shocks 45 volts, and so on. The ‘teacher’ is a genuinely to other participants. So the Milgram experiments naive subject who has come to the laboratory have served two different purposes in writings to participate in an experiment. The learner, about research ethics: (1) as a touchstone for or victim, is an actor who actually receives no discussions of the ethics of deceptive psychological shock at all. The point of the experiment is to research and (2) as a bad example of harm to see how far a person will proceed in a concrete research participants (Cave and Holm, 2003). and measurable situation in which he is ordered

183 Research Ethics 7 to inflict increasing pain on a protesting victim. At the end of the study only 74 subjects remained At what point will the subject refuse to obey the alive, 40 of their wives had been infected and 19 experimenter? children had been born with syphilis, some of In many of the Milgram experiments, the which might have been prevented if these men participants displayed severe psychological distress had been given treatment. and extreme levels of nervousness. For example, The most widely held criticism of the Tuskegee “Many subjects showed signs of nervousness in study centers on the absence of informed consent the experimental situation, and especially upon and the exploitation of the participants. The administering the more powerful shocks. In a large Tuskegee study ended in 1972 just because the number of cases the degree of tension reached ethical concerns relating to it were printed on the extremes that are rarely seen in socio-psychological front page of New York Times by Heller (1972). The laboratory studies. Subjects were observed to national outcry that followed the shocking news sweat, tremble, stutter, bite their lips, groan, and about the Tuskegee experiment “was rekindled dig their fingernails into their flesh” (Milgram, over the years and eventually resulted in a formal 1963, p. 376). Also, one of the unexpected signs apology from President Clinton in 1997. Tuskegee of tension was the regular occurrence of nervous has been heralded as a prime example of unethical laughter, which in some participants developed and scientifically worthless research involving into uncontrollable seizures. For example, “In the deliberate non-treatment, under-treatment, first four conditions 71 of the 160 subjects showed deception, and exploitation. The study has also definite signs of nervous laughter and smiling. been labeled racist and an example of ‘genocide’” The laughter seemed entirely out of place, even (Cave and Holm, 2003, p. 33). bizarre. Full-blown uncontrollable seizures were observed for 15 of these subjects. On one occasion we observed a seizure so violently convulsive that it was necessary to call a halt to the experiment” 1 (Milgram, 1965, p. 68). The true story of the Tuskegee Syphilis Experiment was filmed The Tuskegee Syphilis Experiment in 1997 as “Miss Evers’ Boys” by Between 1932 and 1972 Joseph Sargent. Watch this movie and share your opinions with your The Tuskegee syphilis experiment was carried classmates. out by the US Public Health Service between 1932 and 1972 in Tuskegee, Alabama. According to Eu- ropean Commission (2010, p. 17): The Zimbardo Prison Experiment in About 400 mostly illiterate African-Americans with syphilis were recruited into the study as well 1971 as two hundred healthy controls. The aim of the This experiment was carried out by Philip study was to observe the natural progression of Zimbardo (1973) in 1971 as a prison experiment the disease when left untreated, and in particular in which some male college students were recruited to compare the progression of the disease in and randomly assigned the roles of “guard” and African-Americans with the results of an earlier “prisoner.” Originally, the experiment was designed retrospective study of the disease in Europeans. for a two-week time period, but ran only for six The participants were not told that they were in a days. The reason for the early termination of the medical trial, and the tests were described as ‘special experiment was because the guards had begun to free treatments’. At the outset of the trial there systematically abuse the prisoners through ridicule, was no effective treatment for syphilis. However, by 1947 penicillin had become the standard- verbal abuse, and other forms of severe treatment. effective- treatment for syphilis. The researchers The study was criticized for not informing blocked the study subjects from receiving effective participants about the risk of psychological stress, treatment, going so far as to prevent the subjects physical discomfort, and/or humiliation to which from being conscripted into the armed forces since they were ultimately exposed. As Zimbardo (1973, that would have necessitated them being treated. p. 244-245) explains:

184 Research Methods 7 Interpersonal dynamics in a prison-like other) group or individual behavior patterns environment were studied experimentally by was predictable from the medical, social or designing a functional (rather than literal) educational histories of the subjects, nor from a simulation of a prison. Environmental, battery of personality test scores. structural, institutional and social variables were manipulated in an effort to create a ‘psychology of imprisonment’ in a group of subjects who role-played being guards (for eight hours a 2 day over three shifts) and a group who acted as prisoners (for twenty-four hours a day). To The true story of the Zimbardo assess the strength of the social, situational forces Prison Experiment was filmed in on the behavior of these volunteer subjects, 2010 as “The Experiment” by Paul alternative explanations in terms of pre-existing T. Scheuring. Watch this movie dispositions were eliminated through subject and share your opinions with your selection and random assignment to treatments. classmates. A homogeneous sample of about two dozen normal, average, healthy American college males was chosen after extensive interviewing and diagnostic testing of a large group of applicants The Tearoom Sex Study in 1970 recruited through newspaper advertisements. The The Tearoom sex study was conducted by Laud subjects were from colleges throughout the United Humphreys (1970; cited in Thomas, 1996) and still States and Canada who volunteered to be in ‘a continues to illustrate controversial ethical issues study of prison life’ in return for receiving a daily wage of fifteen dollars for a projected two-week of deception, informed consent, and invasion of period. Half of these pre-selected subjects were privacy. As Thomas (1996, p. 1) explains: randomly assigned to role-play prison guards, the When Laud Humphreys (1970) published others to the mock-prisoner treatment. Neither Tearoom Trade… he drew unprecedented criticism group received any formal training in these roles- from social scientists for the ethics of his study of the cultural mass media had already provided the gay culture and lifestyles. Humphreys developed models they used to define their roles. The mock an innovative method to identify subjects and guards were impressed with the ‘seriousness’ of the gather data. First, he lingered in truck stop experiment and by the demeanor of the research restrooms and watched for gay sexual activity, staff; the prospective prisoners began to take their on occasion even serving as ‘lookout’ for the roles seriously when they were subjected to an participants. Then, he recorded the automobile unexpected arrest by the city police. After being license numbers of the participants as they left the processed and temporarily detained at the police area. From the licenses, he obtained the names station, they were escorted to the experimental and addresses of the gay participants and, several setting. Uniforms and differences in power further months later, contacted them as if they were served to differentiate the two groups of subjects. randomly selected for an unrelated sociological Continuous, direct observation of all behavioral study. Although his published works did not interactions was supplemented by video- and reveal personal or other damaging information, audio-taped recordings, questionnaires, self- did not provide any details of individuals, and report scales and interviews. All of these data put no subjects at risk, Humphreys was castigated sources converge on the conclusion that this as an unethical scholar. His study also generated simulated environment was sufficiently realistic considerable debate over the ethical obligations of and forceful to elicit intense, personal and social scientists toward human subjects. often pathological reactions from the majority According to Berry (2004, p. 325), however, of the participants. Many of the prisoner- Tearoom subjects exhibited behaviors characteristic of when the sex study was published, the learned helplessness syndrome... The guard- “the details given made it possible to identify subjects displayed a behavioral profile which was his research subjects. This controversy led to marked by its verbal and physical aggressiveness, arguments over Humphreys’ research and ended arbitrariness and dehumanization of the subjects with the sociologist, Alvin Gouldner, assaulting in the prisoner condition. None of these (and and hospitalizing him. Gouldner was later stripped

185 Research Ethics 7 of his professorship at the sociology department Guillemin and Gillam (2004, p. 268), for at Washington University, and the entire episode example, provide several reasons for justification eventually led to the closure of the department.” of RECs: “First, research ethics committees The above six reviewed research ethics scandals satisfy an obvious need to protect the basic rights (and many others discussed in the current scientific and safety of research participants from obvious research literature, such as Calvey, 2008; Goode, forms of abuse... Second, it can at least be said 2002; Pearson, 2009; Sikes, 2006, 2010) raised that procedural ethics offers researchers an ethics important ethical questions about informed ‘checklist’ by reminding the researcher to consider consent, deception, manipulation, invasion of such issues as the potential risks to participants, privacy, or some forms of harm to participants. the balancing of the benefits of the research As a result, many countries around the world against those risks, the steps needed to ensure (but mostly western countries- such as the USA, confidentiality of data, and the inclusion of consent the UK, Canada, and Australia) have established forms and plain language statements in the material some forms of Research Ethics Committees (RECs) provided to participants… Further, in fulfilling the and/or Institutional Review Boards (IRBs) to procedural obligations of this ethics checklist, the review research proposals in terms of their ethical researcher is also granted institutional credibility to acceptability for protecting the rights of research carry out the research.” However, Guillemin and participants. Gillam (2004) are also careful to recognize the gap between the “procedural ethics” and “ethics in practice.” According to them, the checklist may Establishment of Research Ethics not be so much help once the researcher is out in Committees (RECs) the field and dealing with the realities of everyday The main role of a research ethics committee research practice. In fact, the approval of a research (REC) is to regulate the ethical conduct of research proposal by a REC does not guarantee an ethical before it is carried out (i.e., whether the ethical conduct of such research because there is no direct principles of informed consent, confidentiality, relationship between the REC’s approval of a rights to privacy, and protecting participants research project and what actually happens in the from harm are taken into consideration). In some field when the research is undertaken. Ultimately, occasions, RECs may also require some changes/ the responsibility falls back to the researcher to alterations to research projects before they allow conduct his/her research in an ethical way. Hence, them to continue. Yet there are contradictory views the procedural ethics may have little or no impact about the need, roles and/or functions of RECs in on the actual ethical conduct of research. social sciences. Some researchers argue that research ethics at the procedural level have been imposed on social sciences from outside (i.e., medical science Ethical review or bioethics) while others stress the importance of RECs. The review of a proposed research by a research ethics committee (REC).

important So some social scientists have thoughtfully RECs should be the committees criticized the utility of RECs. Schrag (2011), for that guide the researchers example, organized the major complaints about and protect the rights of the RECs under the following seven themes: (1) RECs participants, who are the source impose silly restrictions, (2) the procedural ethics of information for research, in all review is presented as a solution in search of an circumstances. ethics problem, (3) RECs lack expertise, (4) RECs apply inappropriate principles, (5) the procedural

186 Research Methods 7 ethics review harms the innocent, (6) better options uncertainty and delays associated with qualitative, exist, and (7) the procedural ethics review has few ethnographic, or critical scholarship which do not fit success stories. Based on his four-year experience as easily into the existing research ethics template. The a member of a REC in Canada, Haggerty (2004) more ethical roadblocks are installed for innovative also draws attention to the following four risks of and critical research, the more we risk homogenizing the REC-based regulatory system: inquiry and narrowing vision, as scholars start to • The scope of research ethics protocols (i.e., a follow what they perceive to be the path of least bureaucratic oversight of research in the institutional resistance” (Haggerty, 2004, p. 412). sense that before a research is conducted, all researchers must apply to the REC in ETHICAL PERSPECTIVES their institutions for approval, outlining the Different ethical perspectives highlight different details of their methodology, any potential factors as being ethically significant. Hence, harms that it might produce, and the thinking about an ethical issue from multiple procedures to alleviate these harms), ethical perspectives is important in the sense that • The notion of “harm” employed by RECs one might miss the ethical point if one addresses it (i.e., evaluating the risks and benefits of the from a single ethical perspective. Thus, this section proposed research from the perspective of introduces six major ethical perspectives. medical sciences or bioethics), • The place of informed consent in research ethics (i.e., in some research settings, Deontological Ethics informed consent forms are perceived to be Deontological ethics has been the most strange, extremely formal, and sometimes influential ethical perspective in the history of impractical or difficult/impossible to research ethics. As a term “deontology” refers obtain), to the study of absolute/universal duties. So • The presumption of anonymity as a means all deontological arguments tend to center on of mitigating the risks of research (i.e., “whether or not an action-proposed or performed- the bureaucratic hurdle when studying corresponds to some command or general rule that sensitive topics and/or small local groups, an agent has a duty to obey” (Hammersley and for example, although participants may not Traianou, 2012, p. 21) such as “the duty of not to be identifiable to the general public, they harm participants”, “the duty of not to lie/deceive may well be detectable to the peers who are participants”, or “the duty of not to discriminate also involved in the research). against participants”, etc. Such obligations of researchers at the same time represent participants’ rights in the sense that a person’s duties are another Anonymity person’s rights (e.g., “the right of participants The condition of being anonymous as a not to be harmed”, or “the right of participants participant in the research, or the removal not to be deceived”, etc.). The deontological of reference to a research participant from ethics perspective is mostly based on the Kantian whom the information is gathered. (Immanuel Kant’s) thinking, which postulates that people should be able to derive a set of absolute/ universal duties by way of utilizing pure reason Accordingly, a paradox of the REC-based such as the axiom of “never treat people as means regulatory system of research ethics is that it can to your ends, but always as ends in themselves.” itself become an instrument of unethical behavior Indeed, this axiom is one of the main reasons in the long run. For example, “An unfortunate why researchers aim to obtain informed consent consequence of these [regulatory] developments from competent research participants (European will likely be that researchers will choose to employ Commission, 2010; Schüklenk, 2005). But what certain types of unproblematic and often predictable exactly does it mean to use people as means to research methodologies rather than deal with the someone’s ends? According to Rachels (2003, p.

187 Research Ethics 7 4), for example, it “typically involves violating their autonomy- their ability to decide for themselves how to live their own lives, according to their own Principle-based ethics desires and values.” A person’s autonomy may An ethical perspective that is based on be violated through manipulation, trickery, or the principles of respect for autonomy, deceit as well as by forcing him/her to do things beneficence, non-maleficence, and justice. against his/her will. A major problem with the deontological perspective is, though, that it does not specify what to do when different rights or Principle-Based Ethics duties come into conflict. (Principlism) Principle-based ethics, developed by Consequentialist/Utilitarian Ethics Beauchamp and Childress (2000; cited in European Commission, 2010; Schüklenk, 2005), proposes a The consequentialist/utilitarian ethics system of ethical reasoning based on the following perspective represents a radical rejection of four principles: deontological ethics. It argues that “Rather than • Respect for autonomy (i.e., the obligation being guided by general rules specifying duties to respect decision-making capacities of and rights, actions must be judged, prospectively, autonomous people), in terms of how well designed they are to produce • Beneficence (i.e., the obligation to provide good outcomes, and, retrospectively, according benefits and to balance them against risks), to whether or not they actually produced such • Non-maleficence (i.e., the obligation to outcomes” (Hammersley and Traianou, 2012, avoid causing harm), p. 22). The consequentialist/utilitarian ethics • Justice (i.e., the obligation of fairness in the is mostly based on the ideas of Jeremy Bentham distribution of benefits and risks). and John Stuart Mill. Rachels (2003, p. 102) summarizes these ideas in the following three simple propositions: Beneficence • First, actions are to be judged right or wrong The obligation of the researcher to bring solely by virtue of their consequences. about benefits or to avoid harms to the Nothing else matters. • Second, in assessing consequences, the participants. only thing that matters is the amount of happiness or unhappiness that is created. Non-maleficence Everything else is irrelevant. The obligation of the researcher not to • Third, each person’s happiness counts the impose harm on participants. same. So the consequentialist/utilitarian perspective is mainly about weighting up the potential benefits and In terms of research ethics, these ethical risks of research in terms of its consequences; that principles basically require researchers to is, whether the outcomes of research are considered respect the choices of competent/autonomous to be good or bad (European Commission, 2010). participants, to avoid causing them harm as It is based on the understanding that researchers’ well as to act in a reasonable manner regarding actions are judged right or wrong depending on the distribution of research risks/burdens and whether they cause more benefits (happiness) or benefits. While principle-based ethics proposes a harms (unhappiness) for the research participants. sound approach to research ethics, its critics have Hence, from the consequentialist/utilitarian ethics also charged that this approach “is unsuitable perspective, for example, informed consent may not for practical decision-making because these be sought in all research endeavors, for this would principles lack a hierarchical order, which renders only be a matter if the absence of informed consent their ranking in any given situation somewhat had bad consequences for the research participants. arbitrary” (Schüklenk, 2005, p. 4).

188 Research Methods 7 important Virtue Ethics A virtue is defined as a trait of character The duty of the researcher is to respect which is manifested in the habitual action of a the decisions of autonomous research person and believed to be good for the person to participants. This must be done to have. Virtue ethics, then, refers to a researcher’s ensure respect for autonomy. possession of certain traits/virtues as part of his/her character/personality such as being kind, generous, courageous, just, and prudent, etc. Indeed, “The Situation-Based Ethics essence of virtue ethics is that character is the (Situationism) primary object of ethical appraisal, and actions are judged according to what they tell us about the Situation-based ethics stresses that there would agent’s character” (European Commission, 2010, p. always be cases for ethical dilemmas that would be difficult to solve, and therefore there would always 25). The basic premise of virtue ethics is that virtuois be disagreements about what is good/bad or right/ researchers have the capacity to make sound ethical wrong in particular research settings/situations. judgments. So the virtue ethics perspective aims to Also, decisions for such ethical dilemmas identify the virtues that all ethical researchers ought could be hard to be made from deontological, to possess, or the vices that they need to avoid as consequentialist/utilitarian, and/or principle- well. Mostly based on the views of ancient thinkers based ethical perspectives. As Hammersley and (such as Socrates, Plato, or Aristotle), supporters Traianou (2012, p. 25) illustrates: of virtue ethics are mainly concerned with the question of “What traits of character make one a For example, imagine a situation where good person?” (Rachels, 2003, p. 173). Yet this issue a researcher has promised an informant that what he says will remain confidential, but that also represents the major objection to virtue ethics, in the course of the interview the informant claiming that it does not provide researchers with a indicates that he is going to commit a violent clear account of how they ought to act specifically crime… A deontological approach might while they are in the field doing research. In other require the researcher to keep her promise and words, virtue ethics only tells researchers what sort not disclose the information to others, but of a person they ought to be, but not how to act in consequentialism could imply that she should particular research situations. break her word in order to ward off harm to others… Consequentialism recommends that we only keep promises where the result of doing Virtue ethics so would be beneficial, or at least that we should An ethical perspective that treats the character not keep promises where the outcome is likely to of researchers as the primary focus of ethical be undesirable. By contrast, many deontological evaluation. positions insist that keeping promises is good in itself, so that we have an absolute duty to do this. The situation-based ethics perspective does not Relational Ethics (Ethics of Care) ignore any of deontological or consequentialist ethical principles (or any other universal ethical Relational ethics is mostly based on the work rules for that matter), but perceives them to be and ideas of Carol Gilligan and Nel Noddings who very abstract. It insists that such ethical principles suggested that “women’s basic moral orientation is cannot tell researchers conclusively what it is good caring for others- ‘taking care’ of others in a personal or right to do in a particular research situation. way, not just being concerned for humanity in Instead, it is the unique circumstances or features of general- and attending to their needs” (Rachels, a particular research context that need to be taken 2003, p. 164). The relational ethics perspective into consideration when deciding what counts a criticizes deontological, consequential/utilitarian “good/bad” or “right/wrong” action. and/or principle-based ethics views for their claim

189 Research Ethics 7 of universal principles and/or obligations; instead, it ethics cannot provide all that is needed for dealing suggests that researchers “ought to see people as social with “ethically important moments” that may arise, beings, nested within a complex set of relationships. especially, when undertaking qualitative research. The ethics of care concentrates on these relationships For example, while interviewing some participants and the emotions such as sympathy and solidarity may feel embarrassment if they disclose some sorts that tend to go with them” (European Commission, of their secrets. Also, some participants may not 2010, p. 26). Thus, when one approaches research want to be assigned a pseudonym in the writing up ethics from the perspective of the ethics of care (or of the research report and may insist that their real the relational ethics) perspective, one also needs to names are used. focus on the nature of the relationship between the researcher and the researched. For example, “On the Tracy’s Perspective question of whether people taking part in research should be termed ‘subjects’ or ‘participants’, those Tracy (2010) proposes a four-dimensional with an ethics of care perspective might argue that research ethics perspective: (1) procedural ethics, (2) ‘participants’ is a richer term that encourages the situational ethics, (3) relational ethics, and (3) exiting participants being seen by the researcher as people to ethics. The first dimension corresponds exactly be engaged with, not merely resources for research” with the first dimension of Guillemin and Gillam’s (European Commission, 2010, p. 26). The relational perspective, and the rest three dimensions can also be regarded to represent the characteristics of “ethics ethics perspective further values mutual respect, in practice” (the second dimension of Guillemin dignity, and connectedness between researchers and and Gillam’s perspective). For example, “situational research participants. It requires researchers to treat ethics” refers to ethical issues that emerge from a their participants as humans rather than as objects reasonable consideration of a research context’s (i.e., it is an “I-You” relationship, instead of “I-It”). specific circumstances. “Relational ethics” involves According to Hammersley and Traianou (2012, p. an ethical self-consciousness in which the researcher 29), for instance, “many advocates of the ethics of is mindful of his/her actions and their consequences care treat caring as a disposition: they argue that on the research participants. Finally, “exiting ethics” moral subjects are expected to attend to others considers ethical issues beyond the data collection with emotional sensitivity, with compassion and process in terms of how researchers should leave the empathy, with trust and intimacy. At the same time, field and share their research findings so as to avoid there is an emphasis upon reciprocal relationships- unjust/unintended consequences. on the contribution of the cared-for as much as that of the carer.” ETHICAL PRINCIPLES In the literature, there are different Ethics of Care categorizations of ethical principles. For example, An ethical perspective that focuses on the Emanuel, Wendler and Christine (2000, pp. 2703- importance of the relationships between the 2707) propose the following seven requirements as researcher and participants. a coherent framework for making research ethical: • Value (i.e., whether the proposed research has the potential to improve knowledge Other Ethical Perspectives and/or understanding), • Scientific validity (i.e., whether the proposed research is methodologically rigorous), Guillemin and Gillam’s Perspective • Fair subject selection (i.e., whether the Guillemin and Gillam (2004) propose a participants are chosen based on the two-dimensional research ethics perspective: (1) scientific objectives rather than vulnerability “procedural ethics” (requiring an approval from a and/or privilege), relevant REC to undertake research) and (2) “ethics • Favorable risk-benefit ratio (i.e., whether the in practice” (representing the everyday ethical issues benefits of the proposed research outweigh in doing research). They contend that procedural the risks for the participants),

190 Research Methods 7 • Independent review (i.e., whether the • Confidentiality- the requirement to proposed research has been approved by ensure confidentiality of research data an unaffiliated expert/institution such as a whether relating to individuals, groups or REC and/or an IRB), organizations. • Informed consent (i.e., whether the researcher • Anonymity- the protection of anonymity of informed his/her participants about the aim individuals or organizations. of the research and obtained their voluntary • Deception- the potential for deception consent), through the research process, either through • Respect for enrolled subjects (i.e., whether the lies or behavior that is misleading. researcher has taken the proper precautions • Affiliation- the need to declare any to protect the privacy of participants, to professional or personal affiliations that enable them to withdraw from the research may have influenced the research, including at any phase if they wish to do so, and to conflicts of interest and sponsorship, ensure their safety and well-being). including information about where funding Christians (2005, pp. 144-145) postulates the for the research has come from. following four ethical principles: • Honesty and transparency- the need for • Informed consent (i.e., the participants’ openness and honesty in communicating right to be informed about the nature and information about the research to all interested consequences of the research in which they parties, including the need for trust. are involved), • Reciprocity- the idea that the research • Deception (i.e., the researchers’ obligation to should be of mutual benefit to researcher design and conduct research free of active and participants or that some form of deception and/or exploitation), collaboration or active participation should • Privacy and confidentiality (i.e., the be involved. researchers’ obligation to secure or conceal • Misrepresentation- the need to avoid all personal data belonging to research misleading, misunderstanding, participants, for example, by using misrepresenting or false reporting of pseudonyms), research findings. • Accuracy (i.e., the researchers’ obligation not to fabricate, duplicate and/or omit any research data). Based on a content analysis of ethics codes internet formulated by nine social sciences associations, Bell Academy of Management’s ethics codes. and Bryman (2007, p. 71) discuss the following 11 Available at: http://ethicist.aom.org/content/ ethical principles: AOM_Code_of_Ethics.pdf • Harm to participants- the potential to cause harm through the research process and the Economic and Social Research Council’s need to ensure physical and psychological (2015, p. 3) Framework for Research Ethics lists the well-being either of research participants, following six key ethical principles: the researcher, or others. • Dignity- the requirement to respect the • Research should be designed, reviewed and dignity of research participants, researchers undertaken to ensure integrity, quality and or others and avoid causing discomfort or transparency. anxiety. • Research staff and participants must • Informed consent- the need to ensure the fully normally be informed fully about the informed consent of research participants. purpose, methods, and intended possible • Privacy- the need to protect privacy of uses of the research, what their participation research subjects or avoid invasions of in the research entails and what risks, if any, privacy. are involved…

191 Research Ethics 7 • The confidentiality of information supplied reasonably to be expected; and the effects upon by research participants and the anonymity his health or person, which may possibly come of respondents must be respected. from his participation in the experiment. • Research participants must take part voluntarily, free from any coercion. • Harm to research participants must be avoided in all instances. internet • The independence of research must be clear, The Nuremburg Code. Available at: https:// and any conflicts of interest or partiality history.nih.gov/research/downloads/ must be explicit. nuremberg.pdf In line with the above reviewed studies and drawing on the basic philosophies underlying major codes, declarations, and other documents The principle of “informed consent” stems relevant to research with human participants, this from a more general principle of respect for the section discusses in-depth the three most widely autonomy or dignity of individuals. The need to recognized ethical principles in international respect the autonomy of research participants is guidelines: (1) autonomy and informed consent, based on the idea that they should be free to decide (2) balancing research risks and benefits, and (3) for themselves what is best for them as well as to privacy, anonymity and confidentiality. exercise control over their own lives. According to Andanda (2005, p. 16), for example, “Autonomy Autonomy and Informed Consent has two facets: firstly, the requirement that those There is almost a universal agreement that who are capable of deliberating on their personal informed consent is of the most important ethical choices should be treated with respect for their principle in research ethics. By way of informed capacity for self-determination; and secondly, consent, the prospective research participant comes persons with diminished or impaired autonomy, or to an understanding of the goals of a research and those who are in dependent or vulnerable positions, of what participation in it would involve so that should be protected against harm or abuse.” So it he/she can make his or her own free decision about is important that informed consent is obtained whether, and under what conditions, to participate. voluntarily and that participants are aware that For example, according to The Nuremberg Code of they are free to withdraw from the research at any 1947 (https://history.nih.gov/research/downloads/ time without facing any consequences. According nuremberg.pdf): to Hammersley and Traianou (2012, p. 89), the issue of informed consent arises primarily in the The voluntary consent of the human subject following three situations: is absolutely essential. This means that the person involved should have legal capacity to give consent; • when researchers are seeking entry to should be so situated as to be able to exercise free settings to which they do not automatically power of choice, without the intervention of have access as ‘members of the public’ or as any element of force, fraud, deceit, duress, over- citizens, reaching, or other ulterior form of constraint or • when they are trying to obtain documents coercion; and should have sufficient knowledge that are not publicly available, and and comprehension of the elements of the subject • when they want to elicit information or matter involved, as to enable him to make an data from people through interviews, understanding and enlightened decision. This questionnaires, diaries, and so on. latter element requires that, before the acceptance of an affirmative decision by the experimental Yet there are a number of other issues that need subject, there should be made known to him to be addressed regarding informed consent. For the nature, duration, and purpose of the example, whether it is necessary to seek informed experiment; the method and means by which it is consent in all research, how much information to be conducted; all inconveniences and hazards should be given to participants, from whom

192 Research Methods 7 consent should be obtained (child, parent, spouse, Each potential subject must be adequately or only from the person himself/herself), how informed of the aims, methods, sources of consent ought to be secured (signed, videotaped, funding, any possible conflicts of interest, or oral agreement), for what consent should be institutional affiliations of the researcher, the sought (observation, interview, publication, etc.), anticipated benefits and potential risks of the study and the discomfort it may entail, post- or what counts as valid consent. The answers to study provisions and any other relevant aspects such questions may also differ according to specific of the study. The potential subject must be circumstances in which the research is being informed of the right to refuse to participate in conducted (Hammersley and Traianou, 2012). the study or to withdraw consent to participate For example, in order for informed consent to be at any time without reprisal. Special attention considered valid it must meet at least the following should be given to the specific information needs three criteria (European Commission, 2010): of individual potential subjects as well as to the • Adequate information (i.e., what a reasonable methods used to deliver the information. person would need/want to know, such as, Biggs and Marchesi (2015), on the other hand, among other things, the purpose of the examined 86 information sheets (ISs) submitted research or any potential risks involved in for research in terms of their lengths and readability order to decide whether or not to take part levels. Their results revealed that the 86 ISs ranged in the proposed research), in length from one to 35 pages and contained from • Voluntariness (i.e., the consent must not 346 to 8955 words, with a mean of 3110 words. result from coercion, manipulation, or Using a scale of 0 to 100 (with 0 meaning most undue inducements), difficult and 100 very easy to read), the 86 ISs • Competence (i.e., having enough mental showed a readability range of 28.6 to 67.5, with a capacity to understand and retain relevant mean level of 47.1, considered as difficult to read information about the research as well and/or understand. Moreover, the mean level of as to communicate his/her views on the education required to easily understand the ISs research). in 86 research applications was 11.6 (equivalent to senior secondary school) with a range of 7.5 to 14.9. So the ISs were considered to be too long to Competence be read in a reasonable time and too difficult to be The ability of a participant to understand easily understood. The researchers recommended information relevant to the proposed that an IS needed to be at a reading grade level research. of eight or below; in other words, the information provided in ISs should be such that could be easily comprehended by a person who has received only Also, informed consent is most often obtained eight years of schooling. in the beginning of the research, when the participants sign a form stating their agreement to take part in the study with their right to withdraw at any time. However, there are those who argue Participant Information Sheet that consent should be sought at different stages An information pamphlet provided to of research for several times (i.e., “process” or potential research participants prior to their “ongoing” consent): (1) before the research starts, consent/permission to be involved in the (2) during the data collection stage if, for example, research. a change has been made in the interview questions, and (3) before publishing the results of the study, etc. Again, in terms of the question “How much In deciding whether informed consent is information should be supplied to participants?”, necessary or desirable in the research, a distinction the Declaration of Helsinki (World Medical between the ethical and practical requirements Association, 2013, p. 2193) requires the following needs to be made. For example, “there may be information to be disclosed: circumstances where consent is neither practically

193 Research Ethics 7 nor ethically required, while there are others where it may be necessary, in one or both senses. At the same time, both ethical and prudential requirements are defeasible- in other words they are not absolute. Researchers may try to circumvent the practical need to obtain consent by using a covert strategy” (Hammersley and Traianou, 2012, p. 83). In this regard, Table 7.1 shows the possible pathways that might be helpful in deciding whether or not informed consent should be sought in the research.

Table 7.1 Pathways in deciding whether consent should be sought First, ask: If the answer is If the answer is “yes”, If the answer is “Is consent “yes”, then ask: 1 then ask: “Is consent “no”, then, seek ethically “Is covert access practically required?” consent required?” possible?” First, ask: If the answer is If the answer is If the answer is “yes”, If the answer is “Is consent “yes”, then ask: “yes”, then ask: 2 then ask: “Is consent “no”, then, seek ethically “Is covert access “Is covert access practically required?” consent required?” possible?” ethical?” First, ask: If the answer is If the answer is If the answer If the answer is “yes”, “Is consent “yes”, then ask: “yes”, then ask: is “yes”, then, 3 then ask: “Is consent ethically “Is covert access “Is covert access proceed without practically required?” required?” possible?” ethical?” seeking consent First, ask: If the answer is “yes”, If the answer is “no”, If the answer is “Is consent 4 then ask: “Is consent then ask: “Is covert “no”, then, seek ethically practically required?” access possible?” consent required?” First, ask: If the answer is If the answer is “yes”, If the answer is “no”, If the answer is “Is consent “yes”, then ask: 5 then ask: “Is consent then ask: “Is covert “no”, then, seek ethically “Is covert access practically required?” access possible?” consent required?” ethical?” First, ask: If the answer is If the answer If the answer is “yes”, If the answer is “no”, “Is consent “yes”, then ask: is “yes”, then, 6 then ask: “Is consent then ask: “Is covert ethically “Is covert access proceed without practically required?” access possible?” required?” ethical?” seeking consent First, ask: If the answer is If the answer is “no”, “Is consent “no”, then, proceed 7 then ask: “Is consent ethically without seeking practically required?” required?” consent First, ask: If the answer is If the answer is “no”, If the answer is “Is consent “yes”, then ask: 8 then ask: “Is consent “no”, then, seek ethically “Is covert access practically required?” consent required?” possible?” First, ask: If the answer is If the answer is If the answer is “no”, If the answer is “Is consent “yes”, then ask: “yes”, then ask: 9 then ask: “Is consent “no”, then, seek ethically “Is covert access “Is covert access practically required?” consent required?” possible?” ethical?” First, ask: If the answer is If the answer is If the answer If the answer is “no”, “Is consent “yes”, then ask: “yes”, then ask: is “yes”, then, 10 then ask: “Is consent ethically “Is covert access “Is covert access proceed without practically required?” required?” possible?” ethical?” seeking consent Source: Adapted from Hammersley and Traianou (2012, p. 83)

194 Research Methods 7 Balancing Research Risks and Benefits example threats of violence towards researchers Another most important ethical principle on drug use or on prostitution, where researchers involved in the discussions of research ethics run the risk of misidentification as a spy for the is related to the “risk of harm” that might result police, the security forces, corporate management, from the actions of researchers. It is argued a corporate competitor, etc.” As Hopper (1999, that researchers should not cause harm to their p. 332) also exemplifies, “In my studies of prison participants, but benefit them. Yet some kind (and/ inmates and motorcycle gang members, both male or degree) of harm is probably unavoidable in any and female, I was impressed by the ‘tests’ they kind of research endeavor. Also, what counts as a put me through before they would talk freely and significant/serious harm is a matter of judgment honestly to me. First, they tested my courage by on the parts of different persons (e.g., researchers, threatening to beat me up; second, they asked research participants, gatekeepers, members of a me to give them money to see if I was a fool; REC, etc.). For example, qualitative research has and third, they fed me false information to see a very open-ended character, and hence, it can be if I would inform the authorities. I had to prove challenging for a qualitative researcher to consider myself over a period of several weeks before they in advance all the contextual variables that might were comfortable with my presence.” According cause harm to participants. As Hammersley and to Hammersley and Traianou (2012, p. 62), the Traianou (2012, p. 64-65) explain: potential threats of harm arising from research can fall into the following five categories: Generally speaking, it [qualitative research] starts from a broadly defined interest in some • Pain, physical injury, and permanent topic and/or in a particular type of person or disability. place, rather than with well-defined hypotheses • Psychological damage, for instance to be tested. As a result, in the early stages at emotional distress, erosion of self- least, data collection is not highly focused but confidence, stress-related illness, and so on. has an exploratory character. Furthermore, in • Material damage of some kind, for the case of participant observation studies, and example loss of one’s freedom through sometimes even in those that are based entirely imprisonment, dismissal from one’s job, on interviews, the researcher is working in reduction in income or wealth, damage to contexts over which he or she has very limited property, and so on. control. Also, these contexts may themselves be • Damage to reputation or status, or to ones in which various kinds of danger of harm operate for those involved- physical, material, relations with significant others, for example or reputational. And, as regards the publication through the disclosure of information that of findings, researchers do not have control over was previously unknown to some relevant how others may use, and misuse, them. While audience. here, as elsewhere, precautions are possible, there • Damage to a project in which people are are limits both to the effectiveness of these and engaged, to some group or organization to to how far it would be desirable to apply them, which they belong, perhaps even to some given potentially deleterious effects, not least on institution or occupation in which they the research. participate. Also, engaging in research in some The issue of avoiding harm also becomes circumstances/contexts may cause risks of harm more important when studying some vulnerable or danger to researchers themselves. Lee (1995; participants who are under threat of harm, and so cited in Bloor, Fincham and Sampson, 2010, p. making the ethical status of their inclusion in research 46), for example, distinguishes between “ambient” becomes more difficult to ascertain. According to and “situational” dangers for researchers. “Ambient European Commission (2010, p. 51), for example, dangers are those that are present in the setting there will be some “situations where researchers for researchers and researched alike, as in the cases wish to carry out research on subjects who lack the of tropical diseases and parachute jumps, while competence needed to give valid consent or for situational dangers are those which the presence whom additional safeguards may need to be taken to of the researcher in the setting may call forth, for protect their welfare, dignity and rights. It is in cases

195 Research Ethics 7 such as these that research subjects are considered to be vulnerable and extra care needs to be taken if research involving them is to proceed ethically.” Confidentiality Such vulnerable participants, for whatever reasons, The obligation of the researcher not to reveal may have been marginalized by societies and are information about a research participant susceptible to exploitation. “They typically stand without his/her permission. in unequal power relationships with others and/ or possess substandard mental faculties, rendering them incompetent” (Lott, 2005, p. 31). They may The question of “what should be private or include the mentally disabled, impoverished persons, secret” or “what can or must be made public” can be prisoners/refugees, women/pregnant women, a contradictory matter for some research settings. children, and ethnic minorities, etc. Thus, they For example, “While there may be considerable deserve special consideration for a number of key agreement in distinguishing public places, such ethical reasons. For example (Lott, 2005, p. 50): as parks, from private places, like bedrooms, there are many uncertain cases. Are religious ceremonies • Informed consent must always be secured public events if they are open for anyone to attend? from potential research participants, Is the talk among people in a bar public or private? either directly or indirectly from a legally Similarly, what about mobile-phone conversations designated proxy. carried out in public places, or discussions in online • The research must never seek to inflict chatrooms?” (Hammersley and Traianou, 2012, actual harm on participants. p. 104). Thus, privacy can become a complex • The research should maximize possible issue involving researchers’ and others’ (e.g., benefits and minimize possible harm. participants, gatekeepers, members of a REC, etc.) judgments about what, whom, and how to study Privacy, Anonymity and in particular research contexts (such as what types Confidentiality of data to seek, what data collection methods to European Commission (2010, p. 78) defines employ, who to approach as potential participants, “privacy” as the protection of “control over how data should be reported/published, etc.). information about oneself; control over access to Anonymity and confidentiality are the two oneself, both physical and mental; and control over strategies used to maintain/protect privacy. For one’s ability to make important decisions about example, there are several ways to secure the family and lifestyle in order to be self-expressive anonymity of personal information (i.e., research and to develop varied relationships.” Privacy is also data). “The most obvious one is to replace actual related to the issue of protecting both personal names with invented ones [i.e., pseudonyms]. territory and information. As Hammersley and Another is to refer to people solely by the roles Traianou (2012, p. 99) explain: that they play: doctor, nurse, patient, relative, and so on. Alternatively, first names without second The territory involved is not only physical names, or just initials, may be used; or numbers space, itself of course always socio-culturally or letters of the alphabet: Informant 1, Informant constituted and differentiated in significant ways, 2; Doctor A, Doctor B, etc. Alongside these but also includes the kinds of ‘virtual’ space that devices, any personal characteristics of people, or exist on mobile phones, personal and institutional contextual features of places, that could lead to computers, and the Internet. Similarly, the sorts of information that are deemed personal or their identification may be omitted from accounts, private vary across cultures and contexts, and the or even changed in order to provide disguise” boundaries between these and what is public are (Hammersley and Traianou, 2012, p. 126). often subject to disagreement and change, even Likewise, confidentiality is a term related to if in all societies there is recognition that some researchers’ commitment to keep secret of the kinds of information should be under the proper people they study and to protect the research data control of particular people rather than freely they obtain. As European Commission (2010, available. p. 79) puts it, “A owes a (prima facie) duty of

196 Research Methods 7 confidentiality to B when: B (the subject) discloses • Research relations (i.e., questions of to A (the researcher) information which B regards as access, power, harm, deception, secrecy, confidential or secret; and A undertakes (implicitly confidentiality, and/or issues related to or explicitly) not to reveal this information to overt/covert research, etc.). anyone who does not already possess it. Because the • Informed consent (i.e., whether consent duty of confidentiality rests on an undertaking (by A has been obtained voluntarily, whether not to reveal B’s secrets), respecting confidentiality participants were adequately informed of can be considered as a kind of promise-keeping the aims of research and its possible risks, or contract.” Also, Tolich (2004) distinguishes or whether participants were competent to between two types of confidentiality: (1) internal consent, etc.). and (2) external. External confidentiality assures • Data dissemination (i.e., the extent to which protection against identification by outsiders (who research data can be stored, published, and/ are not the participants of the research) while or used by policy makers, etc.). internal confidentiality assures protection against Yet the character and importance of ethical insiders (who are other participants of the research). issues/dilemmas may differ with regards to the In other words, not only outsiders of the research, type of social research (such as quantitative or but also insiders may recognize what other insiders qualitative). For example, quantitative research have expressed to a researcher through interviews generally starts with a clearly-defined set of or diaries, etc. As Tolich (2004, p. 101) explains: questions and/or hypotheses, and the researcher Confidentiality is like an iceberg; only the detaches himself/herself from the research setting tip is known, but what lurks unseen, below the in the name of objectivity. In qualitative research, surface, is also a source of potential harm. The however, the researcher is the main data collection easily identified aspect of confidentiality, the tip tool during participant observations and/or semi- above the surface, I label external confidentiality. structured interviews in which he/she is expected It is well-known to all researchers and found to develop close and long-term relationships with in any ethical code. External confidentiality is traditional confidentiality where the researcher the participants. In fact, the central argument acknowledges they know what the person said in the research literature is that “the quality of but promises not to identify them in the final qualitative research reflects the quality of relation report. The less apparent aspect of confidentiality developed between the researcher and researched is internal confidentiality. This is the ability for as ‘knowing subjects’” (Gunzenhauser, 2006, p. research subjects involved in the study to identify 621). Also, qualitative researchers mostly collect each other in the final publication of the research. the study data in natural settings (such as schools, Internal confidentiality lies below the surface, organizations, etc.) rather than in laboratory going unacknowledged in ethical codes. Yet it too situations specifically set up for certain research has the potential to scuttle both researchers and purposes. So many difficulties, dilemmas and/or their informants… Examples of insiders include risks can arise when the qualitative researcher is dyads of husband and wife, same-sex couples, mentor and apprentice, or small groups like immersed in the participants’ personal territories families, classrooms, or sections of work-places. (i.e., research settings). According to Duncan, Drew, Hodgson and Sawyer (2009, p. 1691), for instance, qualitative research settings may preserve ETHICS IN QUALITATIVE distinct ethical dilemmas that deserve special RESEARCH attention of researchers, because: “(1) it is not Ethical issues are likely to arise in all kinds of always possible to predict all possible questions social research. Burges (2005b, p. 5), for example, and responses; (2) the nature of the relationship draws attention to the following four broad between researchers and participants is amenable to issues that can produce ethical dilemmas for all sensitive disclosures; (3) the process of qualitative researchers: research can make it difficult for participants to • Research sponsorship (i.e., the extent to voice concerns or withdraw; and (4) participants’ which research funders influence a research identities are generally known to researchers, activity, etc.). complicating boundary issues.”

197 Research Ethics 7 These and some other characteristics of qualitative many forms. Cunliffe (2003, p. 996), for example, research may require considerable attention when introduces three types of reflexivity: one thinks about ethical issues/dilemmas in • Self-reflexivity as a representation of the social research. Also, resolving such ethical issues/ researcher’s voice (i.e., the influence of the dilemmas may require qualitative researchers to researcher’s values and assumptions on the regularly reflect on their work so as to develop their process of inquiry). understanding of the ethical implications associated • Participant reflexivity as a representation with their research. Thus, the notion of reflexivity of the participant’s voice (i.e., asking as a conceptual tool becomes crucial when dealing participants to reflexively account for their with the day-to-day ethical issues/dilemmas arising practices). in qualitative research. • Radical reflexivity as a representation of a shared voice (i.e., exploring how both the Reflexivity in Qualitative Research “researcher as social participant” and the “social participant as researcher” create This section examines the relationship between understandings from within their ongoing, reflexivity and qualitative research ethics. It shared, and dialogical practices. especially focuses on the notion of reflective thinking as a helpful conceptual tool for understanding both Reflections on Some Ethical the nature of ethical practice in qualitative research Dilemmas in Qualitative Research and how it can be achieved. According to Rachels (2003, p. 14-15), for example: Ethical research is much more than seeking the approval of the research by a REC. Although Morality [ethics] is, at the very least, the RECs can play an important role in identifying effort to guide one’s conduct by reason- that is, to do what there are the best reasons for doing­ the major ethical issues in research proposals, their while giving equal weight to the interests of each role is necessarily limited. For example, “Research individual who will be affected by what one ethics committees cannot help when you are in does… The conscientious moral [ethical] agent is the field and difficult, unexpected situations arise, someone who is concerned impartially with the when you are forced to make immediate decisions interests of everyone affected by what he or she about ethical concerns, or when information is does; who carefully sifts facts and examines their revealed that suggests you or your participants are implications; who accepts principles of conduct at risk” (Guillemin and Gillam, 2004, p. 273). So only after scrutinizing them to make sure they are ethical dilemmas are part of every-day practice of sound; who is willing to ‘listen to reason’ even doing qualitative research. An ethical dilemma can when it means that earlier convictions may have be defined as “a situation in which there is a stark to be revised; and who, finally, is willing to act on choice between different options, each of which the results of this deliberation. seem to have equally compelling ethical advantages The essence of reflexivity in qualitative research and disadvantages” (Guillemin and Gillam, 2004, is, then, related to “a process of critical reflection p. 264). It is “a situation in which a person has both on the kind of knowledge produced from to choose from among two or more non-ideal research and how that knowledge is generated” options, when this choice may be detrimental to the (Guillemin and Gillam, 2004, p. 274). Because fulfillment of his\her professional obligations and qualitative research is primarily a shared activity principles” (Sabar and Sabar Ben-Yehoshua, 2017, of knowledge construction (that is, together the p. 409). This section presents three different ethical researcher and the participants are engaged in an dilemmas. At the end of each ethical dilemma, a active process of producing knowledge), it requires question is also raised for your self-reflection. the researcher to not only report the “facts” of his/her research, but also actively construct interpretations (e.g., “What do I know?”) while at the same time Sonia’s Case questioning how the interpretations have been Guillemin and Gillam (2004, p. 261) discuss made (e.g., “How do I know what I know?”) as the following hypothetical scenario as a typical well to request from his/her participants to reflect ethical dilemma when conducting qualitative on their experiences. Reflexivity, then, can take research.

198 Research Methods 7 Picture this scene. You are a researcher working Toby’s Case on a study examining women’s experiences of Duncan et al. (2009) describe an ethical heart disease. You are interviewing Sonia, a woman in her late 40s with diagnosed heart dilemma that arose during a qualitative research disease. Sonia lives on a remote farming property project that they carried out between 2006 and in a rural region. She is married and has one 2008 to explore the issue of “self-management teenage daughter living at home with herself and of chronic illness in adolescents.” Participants her husband. The interview is progressing well. included adolescents aged between 10 and 18 Over a cup of tea in Sonia’s kitchen, you inquire years, who had one of the following four chronic about the impact of heart disease on her life. Sonia conditions: asthma, diabetes, cystic fibrosis, stops and closes her eyes. After a few moments’ or leukemia. Data were collected through silence, you notice tears welling up in Sonia’s eyes. semi-structured interviews, which focused on Sonia tells you that she is not coping - not because of her heart disease, but because she has just found participants’ understandings of the chronic out that her husband has been sexually abusing condition, patterns of care that had developed in her daughter since she was a child. the family, how strictly the family followed advice This ethical dilemma can be interpreted in provided by health professionals, the division of terms of whether to breach Sonia’s confidentiality responsibility regarding health care in the home, to prevent harm to her daughter. Yet there may be influences on patterns of care from outside the many more immediate ethical concerns that the family environment, and perceptions about how researcher has to decide on how to react to Sonia’s any of these elements may change in the future. disclosure. For example, “Should the researcher let While conducting interviews, a 16-years-old the disclosure pass or take it up in some way? And participant (named Toby) disclosed details of poor in what way- what words to say, what tone of voice adherence with medication, which had significant to use? Turn off the tape recorder or keep it running? health implications. In effect, Toby’s disclosure of Abandon the interview plan or try to return to it? such personal information to qualitative researchers Offer to discuss the situation or offer to help in some (either purposely or accidentally) can be perceived way?” (Guillemin and Gillam, 2004, p. 273). These as a “request for help”. Prior to the study, however, questions represent some of the ethical issues as well all participants had been assured of confidentiality, as the obligations of the researcher toward Sonia meaning that the information they provided in terms of treating and interacting with her in a would not be shared with any of their family humane and non-exploitative way. For example, if members or health professionals responsible for the researcher asks more questions than what Sonia their care. Thus, the researchers found themselves feels comfortable answering, this may cause her unsure of how to proceed with Toby’s case. First, harm. On the other hand, harm may also be caused the research team decided to analyze Toby’s case by not showing enough interest to Sonia’s disclosure in detail. Essentially, there was a tension between in the sense that she might feel she has been ignored two major ethical principles in Toby’s case, i.e., and disregarded. Again, if the researcher encourages “autonomy” and “beneficence.” The ethical Sonia to seek outside help or report the situation to principle of autonomy was central to Toby’s case the police, she may feel guilty for not having done because he had a clear desire to act autonomously this before by herself or she may feel ashamed for in making decisions about his health care and being too passive. So this ethical dilemma tends to call for different actions in responding to Sonia’s wanted to decide what medications to take and disclosure, including how frequently to take them. Likewise, the ethical whether to encourage principle of beneficence (or non-maleficence) was her to talk about the also pertinent to Toby’s case. Beneficence mostly 3 suspected abuse, relates to researchers’ obligation to benefit other whether to continue people or to assist them. On the contrary, non- If you were the researcher with the interview maleficence refers to researchers’ obligation not in this study how would or turn off the tape, to harm the participants as well as to prevent you react to Sonia’s and whether to foreseeable harm or stopping harm that is already disclosure? offer Sonia some occurring. So, with regards to Toby’s case, the assistance. question was (Duncan et al., 2009, p. 1696):

199 Research Ethics 7 Is it in Toby’s best interests to have his confidentiality and autonomy respected or would it be better for him to receive the health benefits of regular medication, which might entail breaching his confidentiality? Re- phrased, would it be more harmful for Toby to continue taking the medication sporadically than it would be to breach his confidentiality? After an analysis of Toby’s case, Duncan et al. (2009, p. 1696) developed the following four action strategies, with each of their associated harms and benefits (see Table 7.2): • Doing nothing further, • Re-contacting Toby and attempting to persuade him to disclose his secret to his parents and/or doctor, • Disclosing Toby’s secret to his parents and/or doctor without Toby’s consent, • Asking Toby if he wanted researchers’ help in talking to his parents and/or doctor about his poor adherence. At the end, the researchers decided to opt for the second course of action. Later, they called Toby to explain their decision. They shared with Toby their concern about the health implications of his sporadic use of the medication and insisted that he better talks with his parents or doctor about his poor adherence with medication.

Table 7.2 Potential courses of action following Toby’s disclosure Course of action Harms Benefits • Toby’s health may deteriorate • If parents/doctor find out they may be • Toby maintains trust in researchers angry with researchers for not disclosing and maintains faith in professional 1. Do nothing information assurances of confidentiality • If Toby was actually attempting to seek help • Toby’s autonomy is respected via his disclosure, we have ignored this • Toby may choose not to disclose • Toby’s autonomy is respected 2. Attempt to the information and his health may • Toby maintains trust in researchers persuade Toby deteriorate and maintains faith in professional to disclose • Toby may feel harassed by researchers assurances of confidentiality his ‘secret’ to • Toby’s parents may become suspicious • Toby is made aware that the researchers his parents or about the reason for further contact are concerned about his health doctor and thus we may end up breaching • Toby may choose to share his secret confidentiality unintentionally and his health may improve as a result • Toby may lose trust in researchers and 3. Breach perhaps the health profession confidentiality • Toby may be angry and upset about the and disclose • Toby’s parents or doctor can take breach Toby’s ‘secret’ appropriate steps to improve his health • Toby’s parents may be angry and upset to his parents • Toby is made aware of the seriousness about the breach or doctor of his actions • Toby’s relationship with his parents/ without his doctor may be negatively impacted by consent their reaction to his ‘secret’ 4. Ask Toby if • Increases the likelihood of Toby sharing he wants our • Toby may feel harassed by researchers his secret assistance • Potentially blurs the role of researcher • Maintains trust and respects Toby’s in telling his with the role of friend, counselor or autonomy parents or clinician • Allows more of Toby’s voice into the doctor decision-making process

Source: Duncan et al. (2009, p. 1697)

200 Research Methods 7 Our study took four long years to be completed; it was only then we emailed all of 4 our participants a letter, including an attachment with their final edited story, asking for their If you were a member of the written permission to publish it. Out of 11 life research team in this study stories that were chosen to be published in full, six would you react to Toby’s secret/ agreed but asked for minor changes. Five refused disclosure in the same way or to give their consent regretting their participation. differently? Why? Their rejection came to us as a great surprise, and posed serious challenges as we felt that it may risk the entire study. Through sensitive and long negotiations, we managed to convince four Husband’s Case more to consent, pending modifications. These Sabar and Sabar Ben-Yehoshua (2017) demands for change ranged from altering the presented many ethical dilemmas linked to their gender of their children through deleting major 49 in-depth life-story interviews while researching parts of the interviews. One of the interviewees blended families in Israel, in which at least one demanded to change the text in a way that we of the couple had children from a previous felt was her way to take revenge against her ex- relationship or marriage. While analyzing their spouse. We refused that specific change. After all of the negotiations, one interviewee still did not data and during the stage of getting participants’ agree. At the stage of the final consent, she shared final written approval of their life-story interview the text with her husband. He, in turn, expressed transcripts (i.e., prior to the publication of the a total refusal to publish his wife’s life story. His study results), they tried to solve some of the ethical words, quoted at the beginning of this article [i.e., dilemmas on their own (such as whether to publish “I’ll sue you if you publish my wife’s interview”], such statements that might harm the participants were clear and left no room for negotiation. Not even though the participants themselves approved only did he threaten to sue us, he also tried to the content). Many other ethical dilemmas were convince other participants that his wife brought perceived to be more controversial and difficult to to the study, to follow him. That was extremely be solved by the researchers themselves and required nerve-wracking for us. It should be stressed that them to negotiate with their research participants. these unpleasant negotiations took place after Such dilemmas usually involved a struggle between his wife had read and accepted the transcripts of her interview, soon after it was made as did all a participant and one of his/her extended family the others. The publisher’s demand for written (that had no research relations with the researchers) consent changed things and obviously, we did not concerning the content of the interview to be include this interview. published. So the main question that Sabar and Sabar Ben-Yehoshua (2017, p. 408) most struggled with was: “Should we publish statements that we thought might harm the interviewees even though we got their approval?” Such concern came about 5 when one of their interviewees’ husband threatened If you were the researcher of this them by using such words as “I’ll sue you if you study how would you react to the publish my wife’s interview” when they requested husband’s threatening? a written consent from his wife to publish her life story in a book as part of their research. So the ethical dilemmas they discussed mostly posed potential risks not only for the participants, but also for the researchers as well. As Sabar and Sabar Ben-Yehoshua (2017, p. 417) explain:

201 Research Ethics 7

LO 1 Comprehend the nature and meaning of research ethics

All research activities should be carried out with a genuine aim of contributing to the knowledge and/or understanding of a given field. However, such research endeavors also need to be ethically sound. For example, participation in research should be voluntary; informed consent should be obtained from competent individuals; and possible risks of harm to participants should be considered and/ or eliminated, etc. So research ethics is about deciding how researchers treat their participants in terms of protecting their rights (such as causing them no harm, respecting their autonomy, or preserving their privacy, etc.). It is a matter of judgment and/or evaluation in terms of right/wrong actions (of researchers) and/or good/bad consequences (resulting from such actions). Summary

Develop a critical understanding of the historical background of research LO 2 ethics with regard to the six famous research ethics scandals and the establishment of research ethics committees

Historically, some research ethics scandals (such as the little Albert experiment in 1920; the Nazi experiments between 1939 and 1945; the Milgram obedience experiments between 1960 and 1963; the Tuskegee syphilis experiment between 1932 and 1972; the Zimbardo prison experiment in 1971; the Tearoom sex study in 1970) have raised important ethical questions about informed consent, deception, manipulation, invasion of privacy, or some forms of harm to participants. As a result, many countries around the world have established some forms of Research Ethics Committees (RECs) and/or Institutional Review Boards (IRB) to review research proposals in terms of their ethical acceptability for protecting the rights of research participants.

Distinguish between the six major ethical perspectives (i.e., deontological LO 3 ethics, consequentialist/utilitarian ethics, principle-based ethics, situation- based ethics, virtue ethics, and relational ethics)

There are six major ethical perspectives, each of which highlights different factors as being ethically significant. Hence, thinking about an ethical issue from multiple ethical perspectives is important in the sense that one might miss the ethical point if one addresses it from a single ethical perspective. For example, deontological ethics judges researchers’ actions according to some absolute/ universal duties. Consequentialist/utilitarian ethics evaluates researchers’ actions according to their consequences and/or usefulness. Principle-based ethics proposes a system of ethical reasoning based on four principles, i.e., respect for autonomy, beneficence, non-maleficence and justice. Situation-based ethics stresses the unique circumstances or features of a particular research context that need to be taken into consideration when deciding good/bad or right/ wrong actions. Virtue ethics treats the character of the researchers as the primary focus of ethical evaluation. Relational ethics focuses on the importance of the relationships between the researcher and participants.

202 Research Methods 7

Differentiate between the three major ethical principles (i.e., autonomy LO 4 and informed consent, balancing research risks and benefits, and privacy, anonymity and confidentiality)

Although there are different categorizations of ethical principles in the research literature, three major ethical principles (i.e., autonomy and informed consent, balancing research risks and benefits, and privacy, anonymity and confidentiality) are mainly stressed. The principle of informed consent stems

from a more general principle of respect for autonomy and requires a voluntary Summary and competent permission to be researched. By this principle, the prospective research participant comes to an understanding of the goals of the research and of what participation in it would involve so that he/she can make his or her own free decision about whether to participate. The principle of balancing research risks and benefits insists that researchers should not cause harm to their participants, but benefit them. Anonymity and confidentiality are the two strategies used to protect the privacy of participants. For example, to secure the anonymity of research data, researchers might replace the actual names of their participants with invented ones (i.e., pseudonyms).

Recognize the unique characteristics of qualitative research in relationship LO 5 to the need for reflective thinking in research ethics

Ethical issues tend to arise in all kinds of social research. Yet the character and importance of ethical issues/dilemmas may differ with regards to the type of social research (such as quantitative or qualitative). For example, quantitative research generally starts with a clearly-defined set of questions and/or hypotheses, and the researcher detaches himself/herself from the research setting in the name of objectivity. In qualitative research, however, the researcher is the main data collection tool during participant observations and/or semi- structured interviews in which he/she is expected to develop close and long-term relationships with the participants. The unique characteristics of qualitative research may require considerable attention when one thinks about ethical dilemmas in social research. Resolving such ethical dilemmas may also require qualitative researchers to regularly reflect on their work so as to develop their understanding of the ethical implications associated with their research.

203 Research Ethics 7

1 Which of the following is considered as the 6 Vulnerable populations are best characterized first international ethical guideline of research as those who are: involving humans? a. Mentally incompetent a. Nuremberg Code b. Relatively poor persons b. Nazi Trials c. Experiencing emotional distress c. Declaration of Helsinki d. Non-citizens of a foreign country d. World Medical Association’s Ethics Codes e. Having unequal power relationships with others e. Belmont Report 7 What is the major ethical issue that arose 2 Which of the following ethical perspectives from the Little Albert experiment? has been the most influential in the history of research ethics? a. An 11-mounth infant was used in the experiment Test Yourself Test b. The experiment was conducted without the a. Principal-based ethics knowledge of Albert’s parents b. Deontological ethics c. There was no control group in the experiment c. Virtue ethics d. Albert was conditioned to fear white rats d. Situation-based ethics e. All of the above e. Relational ethics 8 Beneficence is best achieved when: 3 The deontological ethics perspective requires the researcher not to treat the participants as mere a. The esearcher’sr rights are protected means to his/her research ends. This means that: b. The esultsr of the research benefit the society c. The ell-beingw of research participants is secured a. The esearcherr cannot study people at all. d. The risks of the esearchr outweigh its benefits b. The esearcherr should get approval from a REC e. All of the above to carry out his/her research. c. The prospective participants should not be forced to participate in research. 9 The major oler of a Research Ethics Committee (REC) is: d. The proposed research must generate more benefit than harm. a. To review the research results e. The esearcherr should conduct an overt research. b. To ensure that each research participant is willing to participate in the research 4 Informed consent refers to: c. To decide whether the researcher chose the perfect research methodology a. The ethical principle of espectr for autonomy. d. To determine if the rights of participants are b. A voluntary but uninformed decision-making. adequately protected c. A permission to choose research participants. e. To provide advice to the researcher about how d. A voluntary and informed decision by a to improve his/her research competent individual. e. A REC’s permission to conduct a proposed 10 A research proposal submitted to a REC for research. approval is expected to include: 5 Which of the following is a form of harm that a. The purpose of the esearchr might be caused to research participants? b. A description of how data will be collected c. An explanation about the potential risks to the a. Physical injury participants and how they are going to be dealt b. Psychological distress d. Information about how informed consent will c. Material damage be obtained d. Loss of social status e. All of the above e. All of the above

204 Research Methods 7

1. a If your answer is incorrect, review the section 6. a If your answer is incorrect, review the section on “History of Research Ethics”. on “Ethical Principles”. Answer Key for “Test Yourself”

If your answer is incorrect, review the section 2. b 7. b If your answer is incorrect, review the section on “Ethical Perspectives”. on “History of Research Ethics”.

3. c If your answer is incorrect, review the section 8. c If your answer is incorrect, review the section on “Ethical Perspectives”. on “Ethical Principles”.

If your answer is incorrect, review the section If your answer is incorrect, review the section 4. d 9. d on “Ethical Principles”. on “Ethical Principles”.

If your answer is incorrect, review the section 5. e 10. e If your answer is incorrect, review the section on “Ethical Principles”. on “History of Research Ethics”.

The true story of the Tuskegee Syphilis Experiment was filmed in 1997 as “Miss Evers’ Boys” by Joseph Sargent. Watch this movie and share your opinions with your classmates. Suggested answers for “Your turn”

The movie of “Miss Evers’ Boys” tells the true story of the Tuskegee syphilis experiment from the perspective of an African-American nurse named “Eunice Evers”. She is well aware of the lack of treatment being offered to participants in the experiment, but at the same time concerned her role as vital in terms of consoling them, some of whom were also her close friends. So the nurse Evers your turn 1 was faced with a dilemma of either to abandon the experiment and tell the truth to the participants or remain silent and only offer them comfort and/or relief. Although she chose the second option, she struggled with her decision throughout her life questioning her choice all the time. It is a dramatic movie to watch and can be recommended to especially those who study in the area of medicine.

The true story of the Zimbardo Prison Experiment was filmed in 2010 as “The Experiment” by Paul T. Scheuring. Watch this movie and share your opinions with your classmates.

The movie of The“ Experiment” could be interesting in understanding human behavior in terms of how it could be changed by the uniforms/dresses that certain groups of people (such as doctors, soldiers, guards, teachers, etc.) wear. So if one wants to figure out how a person perceives himself, one might your turn 2 decide to examine the uniform that the person wears. In this regard, the movie of “The Experiment” suggests that the uniforms/dresses and the roles these uniforms/dresses assigned to “guards” or “prisoners” also revealed their underneath psychological characters/personalities.

205 Research Ethics 7

If you were the researcher in this study how would you react to Sonia’s disclosure?

The ethical dilemma in Sonia’s case can be analyzed from different ethical perspectives, each of which also reveals where the researcher stands for. For example, the deontological perspective requires that the researcher should not break his/her promise for confidentiality for whatsoever the reason. This may mean that he/she should not let anyone else know about Sonia’s disclosure. your turn 3 Relational ethics perspective, on the other hand, would suggest that the researcher at least act friendly and listen to carefully what Sonia has to share with him or her in order to comfort her. So what is important here is that the researcher should make an effort to be aware of his/her ethical perspective guiding his/her actions.

If you were a member of the research team in this study would you react to Toby’s secret/disclosure in the same way or differently? Why?

If I were a member of the research team in this study I would react to Toby’s

Suggested answers for “Your turn” Suggested answers for “Your your turn 4 secret/disclosure in the same way as the authors because by attempting to persuade Toby to disclose his “secret” to his parents or doctor by himself, I would both respect Toby’s autonomy and maintain his confidentiality.

If you were the researcher of this study how would you react to the husband’s threatening?

If I were the researcher of this study, I would take the husband’s threating into consideration seriously and seek the consent of my participants before your turn 5 publishing my study results even though I had obtained their consent at the beginning of my research. I also think that even if my book publisher did not demand for participants’ written consent for the interviews I conducted, I would still seek my participants’ written consent.

206 Research Methods 7

Further Reading

American Anthropological Association’s ethics codes. Available at: http://www.aaanet.org/committees/ethics/ ethicscode.pdf American Educational Research Association’s ethics codes. Available at: http://c.ymcdn.com/sites/www. weraonline.org/resource/resmgr/a_general/aera.pdf American Psychological Association’s ethics codes. Available at: https://www.apa.org/ethics/code/principles.pdf American Sociological Association’s ethics codes. Available at: http://www.asanet.org/sites/default/files/code_of_ ethics_aug_2017_2_1.pdf Association of Internet Researchers’ ethics codes. Available at: http://aoir.org/reports/ethics.pdf Association of Social Anthropologists of the UK’s ethics codes. Available at: https://www.theasa.org/downloads/ ASA%20ethics%20guidelines%202011.pdf British Educational Research Association’s ethics codes. Available at: https://www.bera.ac.uk/wp-content/ uploads/2014/02/BERA-Ethical-Guidelines-2011.pdf British Psychological Society’s ethics codes. Available at: https://www1.bps.org.uk/system/files/user-files/ Division%20of%20Clinical%20Psychology/public/Code%20of%20Ethics%20and%20Conduct%20 %282009%29.pdf British Sociological Association’s ethics codes. Available at: https://www.britsoc.co.uk/media/23902/ statementofethicalpractice.pdf Market Research Society’s ethics codes. Available at: https://dev-mrs.cms21.ama.uk.com/pdf/mrs%20code%20 of%20conduct%202014.pdf Social Research Association’s ethics codes. Available at: http://the-sra.org.uk/wp-content/uploads/ethics03.pdf The Belmont Report. Available at: https://www.hhs.gov/ohrp/sites/default/files/the-belmont-report-508c_ FINAL.pdf The USA Federal Policy on Research Misconduct. Available at: https://www.aps.org/policy/statements/upload/ federalpolicy.pdf

207 Research Ethics 7

References

Andanda, P. (2005). Informed consent. Developing Emanuel, E. J., Wendler, D., & Christine, G. (2000). World Bioethics, 5(1), 14-29. What makes clinical research ethical? JAMA, 283(20), 2701-2711. Available at: http://www. Bell, E., & Bryman, A. (2007). The ethics of dartmouth.edu/~cphs/docs/jama-article.pdf management research: An exploratory content analysis. British Journal of Management, 18, 63-77. European Commission. (2010). European textbook on ethics in research. Available at: https://ec.europa. Berry, D. M. (2004). Internet research: Privacy, ethics eu/research/science-society/document_library/ and alienation: An open source approach. Internet pdf_06/textbook-on-ethics-report_en.pdf Research, 14(4), 323-332. Goode, E. (2002). Sexual involvement and social Biggs, J. S. G., & Marchesi, A. (2015). Information research in a fat civil rights organization. for consent: Too long and too hard to read. Qualitative Sociology, 2(4), 501-534. Research Ethics, 11(3), 133-141. Guillemin, M., & Gillam, L. (2004). Ethics, Bloor, M., Fincham, B., & Sampson, H. (2010). reflexivity, and “ethically important moments” in Unprepared for the worst: Risks of harm for research. Qualitative Inquiry, 10(2), 261-280. qualitative researchers. Methodological Innovations Online, 5(1), 45-55. Gunzenhauser, M. G. (2006). Theorizing a relational turn for qualitative research. Qualitative Inquiry, Burges, R. G. (2005a). Preface. In R. G. Burges (Ed.), 12(3), 621-647. The ethics of educational research (pp. viii-ix). Philadelphia: Taylor & Francis Inc. Haggerty, K. D. (2004). Ethics creep: Governing social science research in the name of ethics. Burges, R. G. (2005b). Ethics and educational Qualitative Sociology, 27(4), 391-414. research: An introduction. In R. G. Burges (Ed.), The ethics of educational research (pp. 1-9). Hammersley, H., & Traianou, A. (2012). Ethics in Philadelphia: Taylor & Francis Inc. qualitative research: Controversies and contexts. Los Angeles: Sage. Calvey, D. (2008). The art and politics of covert research: Doing ‘situated ethics’ in the field. Harris, B. (1979). Whatever happened to little Albert? Sociology, 42(5), 905–918. American Psychologist, 34(2), 151-160. Cave, E., & Holm, S. (2003). Milgram and Tuskegee: Heller, J. (July 25, 1972). Syphilis victims in U.S. Paradigm research projects in bioethics. Health study went untreated for 40 years: Syphilis victims Care Analysis, 11(1), 27-40. got no therapy. New York Times (page 1, column 2). Available at: https://sandyhiddenhistories. Christians, C. G. (2005). Ethics and politics in weebly.com/uploads/4/0/3/1/40313585/ qualitative research. In N. K. Denzin & Y. S. syphilis_study_40_years.pdf Lincoln (Eds.), Handbook of qualitative research (pp. 139-164). Thousand Oaks: Sage. Hopper, C. B. (1999). A comment on Erich Goode’s confession. Deviant Behavior, 20(4), 331-333. Cunliffe, A. L. (2003). Reflexive inquiry in organizational research: Questions and Lott, J. P. (2005). Vulnerable/special participant possibilities. Human Relations, 56(8), 983-1003. populations. Developing World Bioethics, 5(1), 30-54. Duncan, R. E., Drew, S. E., Hodgson, J., & Sawyer, S. M. (2009). Is my mum going to hear this? Marks, A. R. (2006). Book review. Doctors from Methodological and ethical challenges in hell: The horrific account of Nazi experiments qualitative health research with young people. on humans. The Journal of Clinical Investigation, Social Science & Medicine, 69, 1691-1699. 116(1), 2. Economic and Social Research Council. (2015). Milgram, S. (1963). Behavioral study of obedience. Framework for research ethics. Available at: https:// Journal of Abnormal and Social Psychology, 67, www.gla.ac.uk/media/media_326706_en.pdf 371-378.

208 Research Methods 7 Milgram, S. (1965). Some conditions of obedience Sikes, P. (2006). Scandalous stories and dangerous and disobedience to authority. Human Relations, liaisons: When female pupils and male teachers 18, 57-76. fall in love. Sex Education, 6(3), 265-280. National Commission. (1979). The Belmont Report: Sikes, P. (2010). Teacher–student sexual relations: Ethical principles and guidelines for the protection Key risks and ethical issues. Ethnography and of human subjects of research. Available at: https:// Education, 5(2), 143-157. www.hhs.gov/ohrp/sites/default/files/the- belmont-report-508c_FINAL.pdf Thomas, J. (1996). When cyberresearch goes awry: The ethics of the Rimm “Cyberporn” study.The Patton, M. Q. (2002). Teaching and training with Information Society, 12(2), 189-198. metaphors. American Journal of Evaluation, 23(1), 93-98. Tolich, M. (2004). Internal confidentiality: Pearson, G. (2009). The researcher as hooligan: When confidentiality assurances fail relational Where ‘participant’ observation means breaking informants. Qualitative Sociology, 27(1), 101-106. the law. International Journal of Social Research Tracy, S. J. (2010). Qualitative quality: Eight “big- Methodology, 12(3), 243-255. tent” criteria for excellent qualitative research. Rachels, J. (2003). The elements of moral philosophy Qualitative Inquiry, 16(10), 837-851. (4th edition). New York: McGraw-Hill. World Medical Association. (2013). Declaration of Sabar, G., & Sabar Ben-Yehoshua, N. (2017). ‘I’ll sue Helsinki ethical principles for medical research you if you publish my wife’s interview’: Ethical involving human subjects. JAMA, 310(20), dilemmas in qualitative research based on life 2191-2194. Available at: https://www.wma.net/ stories. Qualitative Research, 17(4), 408-423. wp-content/uploads/2016/11/DoH-Oct2013- Schrag, Z. M. (2011). The case against ethics review JAMA.pdf in the social sciences. Research Ethics, 17(4), Zimbardo, P. G. (1973). On the ethics of intervention 120-131. in human psychological research: With special Schüklenk, U. (2005). Introduction to research ethics. reference to the Stanford prison experiment. Developing World Bioethics, 5(1), 1-13. Cognition, 2, 243-256.

209 Chapter 8 Writing Research Report After completing this chapter, you will be able to:

Explain how to choose a research topic, write a structured abstract, know steps for Develop competence in writing research writing introduction and principles of writing design, choosing sample/participants, paragraphs, know paraphrase, cite and avoid discussing research instrument-procedure 1 plagiarism 2 and writing data analysis section

Develop skills in writing discussion, conclusion 3 and recommendations Learning Outcomes

Key Terms Chapter Outline • Cite • Conclusion Introduction • Data analysis • Discussion Basic Academic Writing Skills • Footnote and endnote • Measures Writing a Literature Review • Paragraph • Paraphrase Writing Research Methodology • Plagiarism • Recommendation Presenting Results • Research design • Research procedure Writing Discussion, Conclusion and • Research sample • Results Recommendation • Structured abstract • Transition words

210 Research Methods 8 INTRODUCTION the data, presenting and interpreting results/findings There is a common understanding that a writing discussion, conclusions, implications, and scientific research is not completed without it suggestions for further studies. These competencies being published. One of the foremost proofs that form the basis of academic writing. In addition to the demonstrate the completion of a research is the ability to choose the research subject, the ability of report writing. Report writing differs based on the choosing a title that will reflect this subject, writing research paradigm and methodology adopted in in a concise, brief and effective manner, correlating the research report writing, the discipline in which and constructing the introduction with the research the research is carried out and the characteristics of subject both conceptually and theoretically, writing the target audience for which the research report through scrutinizing and synthesizing the literature is written. However, there are generally accepted correlate with basic writing skills. The fact that a principles related with the form and content researcher possesses basic academic writing skills in the writing of scientific research reports. will enable him/her to write high quality research Researchers are expected to have knowledge about reports. In this sense, the researcher will start by these general principles and reflect them on the finding a title that can reflect the research subject in research reports they write. In this context, while the best and striking way. there are commonalities among the writing of a doctoral dissertation, a scientific research article Choosing an Original Research and a dissertation project at the undergraduate Topic level, there will also be divergences in terms of The research topic as well as the title is very content. For example, it is a general tendency to important to attract attention of the readers. quote within the framework of certain rules and to Finding an original and new research problem write in a certain format in all the aforementioned is the main organizing principle guiding the academic writing activities. Therefore, it is analysis of your research. It is difficult to choose important for the researchers having necessary an interesting topic without doing a preliminary knowledge, skills and competencies in finding reading in your discipline. Therefore, you should a research topic, reviewing related literature, read a lot of topics and discuss the topic then citing, writing the aim of the study and research determine the topic that you feel you find yourself problems/hypothesis, developing competencies better involved and interested in. Of course, you in writing methods, research findings, discussions need to communicate it with your supervisor. As and results are the first steps in writing an effective Alderman (2014) pointed out “Do not assume research report. that choosing a research problem to study will be a In this chapter, primarily “basic academic quick or easy task! You should be thinking about it writing skills” are demonstrated in the context at the start of the course”. According to Alderman of writing a research report. Later, “writing a (2014) there are generally three ways students literature review”, “writing research methodology”, are asked to write about a research problem: 1. “presenting results” and “writing discussion, Your supervisor provides you with a general topic conclusion, and suggestions” topics are discussed. from which you study a particular aspect; 2. Your Each topic discussed was strived to be broadly supervisor provides you with a list of possible revealed within the framework of relevant concepts, topics to study and you choose a topic from that principles and examples. list; or, 3. Your supervisor leaves it up to you and you only have to obtain permission to write about BASIC ACADEMIC WRITING it before beginning your investigation. Writing SKILLS a title that is suitable for the selected research subject is an important matter. Pattern-specific Basic academic writing skills include, writing titles may be written in quantitative, qualitative introduction/background/review of literature, and mixed method researches. When composing having knowledge about theory, different research the title, the research title generally reflects the approaches and research methods, designing and research pattern. For example, a study by Thaler developing/measurements/participants, analysing

211 Writing Research Report 8 (2017) was titled as “Mixed methods research It is important to note that basic components in the study of political and social violence and of the abstract can vary by discipline. For samples, conflict.” We can understand from this title that try to find abstracts of research that is similar to the author discusses how the mixed method your research topic. To do this, find a suitable researches will be applied in understanding journal for your topic and read manuscripts in political and social violence and conflicts, and the the journal or read recently published thesis possibilities of mixed method. in your field. Writing an abstract involves summarizing a whole manuscript and providing Writing Abstract as much new information as possible. The best way to write an effective abstract is to start with The abstract is an important component of a draft of the complete manuscript. Writing an thesis and research papers. An abstract is a short effective abstract will improve the chances of your summary of your completed research. If written thesis and manuscript being accepted, encourage well, it makes the reader want to learn more about people to read it, and increase its impact. Below your research. The abstract can be in different forms are the steps in writing an effective abstract (San and mainly depends on the written guidelines of Francisco Edit, 2018): an institution or journals. An abstract summarizes, 1. Identify the major objectives and the research usually in one paragraph of 200-300 conclusions. words or less. When writing a structured abstract, 2. Identify phrases with keywords in the consider the following sub-headings: methods section. 1. Problem statement: Why do we care about 3. Identify the major results from the the problem? What practical, scientific, discussion or results section. theoretical gap is your research topic filling? 4. Assemble the above information into a 2. Purpose: What is the overall purpose of the single paragraph. study? 5. State your hypothesis or method used in 3. Method: The basic design of the study. What the first sentence. did you actually do to get your findings/ 6. Omit background information, literature results? review, and detailed description of methods. 4. Results/Findings: Major findings found as a 7. Remove extra words and phrases. result of your analysis. What did you learn/ 8. Revise the paragraph so that the abstract invent/create? conveys only the essential information. 5. Conclusion/Implications: What is your 9. Check to see if it meets the guidelines of the concluding remark? What are the larger targeted journal. implications of your findings, especially for 10. Give the abstract to a colleague (preferably the problem/gap identified in the research? one who is not familiar with your work) and ask him/her whether it makes sense. Choosing key words. Abstracts are usually followed by a list of keywords selected by the Abstract author. The instructions for authors will state how An abstract summarises, usually one many keywords (most journals require authors to paragraph (sometimes in structured format) select 3-5 keywords (or phrases)) are required and of 200-300 words or less. In general, an may even provide a list of recommended keywords. abstract covers the overall purpose of the Choosing appropriate keywords is important study and the research problem investigated, because these are used for indexing purposes as the basic design of the study, main results, well as reaching the potential research interested implications for research and practice. in your work. Well-chosen keywords enable your manuscript to be more easily identified and cited. Always check your target journal’s instructions for

212 Research Methods 8 authors as often journals will specify that terms 1. What was I studying? used in the title of your paper should not be 2. Why was this topic important to investi- used as keywords. When selecting keywords, you gate? should choose keywords that best describe the 3. What did we know about this topic before content of your thesis or manuscript and follow I did this study? the below steps. 4. How will this study advance new knowl- 1. Focus on the main topic of your research. edge or new ways of understanding? 2. Avoid keywords that are only one word. According to Reyes (2017), there are three 3. Avoid overlapping keywords in your title overarching goals of a good introduction: and those in your keyword list. 1. Ensure that you summarize prior studies 4. Follow the journal guidelines when selecting about the topic in a manner that lays a keywords. foundation for understanding the research 5. Perform keyword research before submitting problem; your article 2. Explain how your study specifically addresses the gaps in the literature, insufficient consideration of the topic, or other deficiency in the literature; and, internet 3. Note the broader theoretical, empirical, and/ or policy contributions and implications of You can review on the website how the your research. structured abstract of a study titled “Principals’ The introduction should be very concise, well- Working Conditions, Job Stress, and Turnover structured and cover all the necessary information Behaviors Under NCLB Accountability needed to follow the development of research findings. Pressure” was written:: http://journals.sagepub. Some people recommend that the “Introduction” com/doi/full/10.1177/0013161X18785874 must be the first section written when writing a thesis or a manuscript. In general, Introduction section of an article should cover: research background, research problem/problem statement, and research Writing Introduction objective(s)/research question(s). The introduction aims to provide an overview of the research topic and explain what it is about. important The introduction leads the reader from a general subject area to a particular topic of inquiry (Hirano, 2009). It establishes the scope, context, These are general phases associated with and significance of the research being conducted writing an introduction (Redman and Maples by summarizing current understanding and 2011): background information about the topic, stating the purpose of the work in the form of the research problem supported by a hypothesis or a set of 1. Establish an area to research by: questions, explaining briefly the methodological • Highlighting the importance of the approach used to examine the research problem, topic, and/or highlighting the potential outcomes your study • Making general statements about the can reveal, and outlining the remaining structure topic, and/or and organization of the paper (Samraj, 2002). • Presenting an overview on current The Introduction needs to be very concise, well research on the subject. structured, and inclusive of all the information 2. Identify a research niche by: needed to follow the development of your findings. As Reyes (2017) highlighted “Think of • Opposing an existing assumption, and/or the introduction as a mental road map that must • Revealing a gap in existing research, answer these four questions for the reader: and/or

213 Writing Research Report 8 • Formulating a research question or problem, and/or • Continuing a disciplinary tradition. 3. Place your research within the research niche by: • Stating the intent of your study, • Outlining the key characteristics of your study, • Describing important results, and • Giving a brief overview of the structure of the paper. Below are the steps in developing an effective Introduction. However, since writing guideline of every institution and every journal is different, it is important that you look at writing guidelines and papers in your targeted journal to determine whether they use all of these steps. Steps in developing an effective Introduction (San Francisco Edit, 2018): 1. Begin the Introduction by providing a concise background account of the problem studied. 2. State the objective of the investigation. Your research objective is the most important part of the introduction. 3. Establish the significance of your work: Why was there a need to conduct the study? 4. Introduce the reader to the pertinent literature. Do not give a full history of the topic. Only quote previous work having direct bearing on the present problem. 5. Clearly state your hypothesis, the variables investigated, and concisely summarize the methods used. 6. Define any abbreviations or specialized terms. 7. Provide a concise discussion of the results and findings of other studies so the reader understands the big picture. 8. Describe some of the major findings presented in your manuscript and explain how they contribute to the larger field of research. 9. State the principal conclusions derived from your results. 10. Identify any questions left unanswered and any new questions generated by your study.

Writing a Paragraph The paragraphs are included in the whole report. The emphasis is placed here on the introduction paragraph rather than the general rules of paragraph writing. The reason for this is that in general, the first part that will be read after the summary part is the introduction paragraph. To this respect, the introduction paragraph should be striking and should make the audience feel why the coming paragraphs should be read. The opening paragraph includes five basic pieces of information, usually in this order: 1) A statement of the general topic. 2) A general statement about what the literature has found. 3) A statement about what the literature is missing or where there is an unanswered question. 4) The aim of the study. 5) A general statement of the study approach. How a sample opening paragraph is structured is given in Box 8.1.

214 Research Methods 8 Box 8.1 The opening paragraph example For a study about child development, parenting and culture Parenting style is a well established influence on child development (Bornstein, 2003). A statement of the general topic. Research indicates that different parenting styles are generally predictive of academic A general statement about what the literature and emotional adaptation in children has found. (Steinberg, Elmen & Mounts, 1989). However, some research has suggested that the A statement about what the literature is missing influence of parenting style may vary across or where there is an unanswered question. cultures and by immigration status (Frankel & Roer-Bornstein, 1982). The aim of the current study was to examine how parenting style among first-generation immigrants The aim of the study. from the African diaspora influenced child development. The study examined parenting style and child outcomes within a community of Somalian immigrants in the Northeastern A general statement of the study approach. United States.

Source: Adapted from Alice Frye (2016), Ph.D., Department of Psychology, University of Massachusetts, Lowell.

Paragraph 1 A paragraph is a group of related sentences that support one main idea. In general, paragraphs Why do you think introduction paragraph in consist of three parts: the topic sentence, body a scientific research report should be written sentences, and the concluding or the bridge different than other paragraphs? sentence to the next paragraph or section.

A paragraph is a collection of related sentences dealing with a single topic. A well-written paragraph helps your readers and keep them reading your work to the end. Driscoll and Brizee (2013) stated, “You can have fantastic ideas, but if those ideas aren’t presented in an organized fashion, you will lose your readers.” According to them the basic rule when writing a paragraph is to “keep one idea to one paragraph.” If you begin to transition into a new idea, it belongs in a new paragraph. There are some simple ways to tell if you are on the same topic or a new one. You can have one idea and several bits of supporting evidence within a single paragraph. Elements of a paragraph. To be as effective as possible, a paragraph should contain each of the following in order: argument/claim, evidence/proof, and summary. When writing a paragraph you should bear in mind that a paragraph has three elements including 1) coherence, 2) a topic sentence, and 3) adequate development (Driscoll and Brizee, 2013). Using and adapting them to your individual purposes will help you construct effective paragraphs. Coherence. The sentences should be arranged in a logical manner and should follow a definite plan for development. ... It is the unity and coherence of ideas represented in a sentence or among sentences that constitutes to a good paragraph.

215 Writing Research Report 8 A topic sentence. A topic sentence is a sentence Adequate development. The topic (which that indicates in a general way what idea or thesis the is introduced by the topic sentence) should be paragraph is going to deal with. It is good to start discussed fully and adequately. Again, this varies a new paragraph with an argument or a claim. As from paragraph to paragraph, depending on the Driscoll and Brizee (2013) states “Although not all author’s purpose, but writers should be wary of paragraphs have clear-cut topic sentences, and despite paragraphs that only have two or three sentences. the fact that topic sentences can occur anywhere in the In academic writing, most paragraphs include at paragraph (as the first sentence, the last sentence, or least three sentences, some paragraphs, although somewhere in the middle), an easy way to make sure your reader understands the topic of the paragraph rarely, may contain more than ten sentences. A is to put your topic sentence near the beginning of good general rule is two to three paragraphs a the paragraph. Regardless of whether you include an page (Driscoll and Brizee, 2013). There are some explicit topic sentence or not, you should be able to methods that can be applied in the construction of easily summarize what the paragraph is about. paragraphs (see Box 8.2).

important Topic sentence The topic sentence is the main idea of In academic writing, most paragraphs include each paragraph. It contains the focus of at least three sentences, though rarely more the paragraph and tells readers what the than ten. A good general rule is two to three paragraph is going to be about. paragraphs a page.

Box 8.2 Some methods to make sure your paragraph is well-developed

• Use examples and illustrations • Cite data (facts, statistics, evidence, details, and others) • Examine testimony (what other people say such as quotes and paraphrases) • Use an anecdote or story • Define terms in the paragraph • Compare and contrast • Evaluate causes and reasons • Examine effects and consequences • Analyze the topic • Describe the topic • Offer a chronology of an event (time segments)

Source: (Purdue University, 2018)

How to Paraphrase This section explains “what paraphrasing is?”, “why researchers do it?” and shows how to do it in an effective way. While paraphrasing researchers need to keep in mind the difference between paraphrasing and quoting. When writing, literature we take pieces of writing directly from another work and quote them and then cite the original author. In may social science writing, we often try to summarize several

216 Research Methods 8 ideas from a single article, or summarize ideas information in our own words by summarizing or from several articles. Using quotes from each paraphrasing, we must cite the original author or and every article can make it hard to present a researcher and the date of publication. Below is the clear argument. It also leads to unnecessarily long example of an extensive quote and paraphrased text. descriptions. Paraphrasing is shorter and easier to An example for a paraphrasing form an original read. But it must always include citations for the article is given in Box 8.3. original source.

Box 8.3 Quoting from one article example important Original Text: Block and colleagues described the problems with jurors assessment of reports Paraphrasing “Juror decisions are based on this memory • Paraphrasing is restating something written representation, rather than on the original raw by someone else in your own words evidence… the facts in essence become less • In social science In social sciences, relevant than what the jurors remember in their paraphrasing is preferred over quoting. reconstruction of the facts” (as cited in Block, • When we paraphrase we must still cite the Greenberg, & Goodman, 2009, p 2860). original author (Murray, 2016). Paraphrased Text: Block and colleagues described that jurors’ own memories influence their decisions (Block et al., 2009).

Paraphrasing To state something written or spoken in different words, esp. in a shorter and simpler form to make the meaning clearer 2 (Cambridge Dictionary). Discuss the relationship between citation and plagiarism in academic writing.

Quote: When paraphrasing a text we must re-read A group of words taken from a text or speech the original passage until we fully-understand it. and repeated by someone other than the original author or speaker, a formal statement After reading, we should ask some basic forms (Oxford Dictionaries). of questions such as, “what?”, “what about it?”, “who says?”, “what happened?”, “did what?”, “what happened to them?”, “who?”, “why?”, There are two types quotes “Direct Quote” and “when?”, “are doing what?” All of the following “Indirect Quote.” When we incorporate a direct sentences were taken from The Washington Post quotation into a sentence, always we must include and adopted from the presentation developed by the author’s last name, year of publication, and page Professor Jean Van Meter. When paraphrasing we number(s). Fit quotations within our sentences, generally begin the sentence with a signal phrase. enclosed in quotation marks, making sure the Some common signal phrases (Harris, 2002) we sentences are grammatically correct. If we put can use are given in Table 8.1 below.

217 Writing Research Report 8 Table 8.1 Common signal phrases to introduce a quote, paraphrase, or summary Says Yield The verb introduce the quotation as information/ The source agrees that a conflicting point is valid Adds Notes Acknowledges Believes Observes Admits Comments Offers Agrees Describes Points out Allows Discusses Remarks Concedes Emphasizes Reports Grants Explains States Recognizes Mentions Write Argues in favor Argues against The verb indicates that the source is responding The verb indicates that the source is providing critically to another source or with the position you evidence or reasons for a position. are advancing. Argues Indicates Attacks Disputes Asserts Insists Contradicts Objects Contends Maintains Criticizes Opposes Demonstrates Proposes Denies Rebuts Holds Shows Differs Refutes Illustrates Supports Disagrees States erroneously Implies The source makes a statement that you are sceptical The source presents information either tentatively about (be careful of your tone if you use these). or indirectly Alleges Implies Assumes Proposes Claims Suggests Continues Concludes The source draws a conclusion from previous You continue to refer to or quote the source. discussion Adds Concludes Continues Decides Goes on to say Determines States further Finds Agrees The verb indicates that the source agrees with another source or with the position you are advancing. Accepts Agrees Assents Concurs Parallels Supports

218 Research Methods 8

important

APA style requires the use of the past tense or present perfect tense when using signal phrases to describe earlier research (Example: “Smith (1992) found…” or “Smith (1992) has found.

How to Synthetize Articles Synthesizing articles is a crucial skill in social science research. Synthetize means that in your literature review you examine a number of studies on a shared topic and note aspects that are of interest for your own work. It also may mean that you draw and state a conclusion about the similarities and differences in the studies you review (Frye, 2016). It allows us to highlight what was important to us about those articles. Below are the steps that are important when synthetizing: 1. It is possible to synthesize multiple articles in one paragraph, 2. You note the shared issue across the articles that you want to call attention to, 3. Note any important differences that are relevant to your study, 4. And describe each article briefly in ways relevant to your study. Suppose you are writing a paper about canine aggression. You read three articles about aggressive dogs. One thing you are interested in is how aggression in dogs has been assessed, and you want your reader to understand that there is variation in assessment. So you synthesize what you have read to reflect this focus. Following is Box 8.4 showing the how to synthesize article in your study:

Box 8.4 How to synthesize article in your study

You might say something like this: Comments

Research indicates that there are a number of approaches Main idea: to assessing canine aggression. One study (Reisner, You need to know what you Shofer, & Nance, 2007) of dog aggression towards focus on in your own study in children used retrospective review of aggressive incidents order to synthesize articles for it. reported to a veterinary clinic, and included a systematic categorization of the types and circumstances of the Evidences: incidents. A study specifically focusing on comparing You need to know what parts variation in aggression across several different breeds of an article are of use to your employed a standardized survey measure of general own study, and what parts are aggressive behaviors, which is based on owner report of not. canine aggression (Duffy, Hsu, & Serpel, 2008). Another study reported the use of a standardized assessment of very specific aggressive behavior (food guarding), using Summary: an assessment method that can be used by either owners You need to draw a conclusion or professionals (Mohan-Gibbons, Weiss, & Slater, 2012). for the reader, so the reader All the methods yielded valid results as noted by each will know what is important study, but the standardized measures appeared to be about the studies you have more efficient and easier to use with non-professionals. summarized.

Source: (Frye, 2016)

219 Writing Research Report 8 During the literature review below synthesis matrix would be helpful in organizing literature. By creating a matrix you can understand how to synthesize that literature by reviewing the below Matrix 8.1.

Matrix 8.1 How to organize your literature using synthesis matrix

Topic: World War II and Its Effect on Women

Source 1: Source 2: Source 3: Source 4: Main Idea Cornelsen Stewart Bruley Scott

Main Idea-A - Women could compete with - Women joined workforce Alteration of men as equals in the sky as a break from the women’s roles because of their exemplary ordinary to help the war because of performance (p. 116) (p. 220) WWII - Women given unskilled - From the outset male pilots labor positions by resented women’s presence government because Main Idea-B in a traditionally male only seen as temporary Hardships military setting (p. 1113-4) workers, therefore no and - The Women Airforce Service reason to train them (p. oppositions Pilots (WASP) were routinely 221-2) women assigned inferior planes that - Equal pay rarely given faced were later found to have to women, even though been improperly maintained women did the same (p. 114) work (p. 221)

Main Idea-C - Women put in untraditional - Women were not affected Opposition: roles during/because of the because they still WWII did war, but back to previous remained in subordinate NOT effect subservient roles after the positions after the war (p. women war (p. 35) 217)

LITERATURE REVIEW What do we already know about this issue? (What have others said and provided evidence for?) A literature review surveys books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have explored while researching a particular topic and to demonstrate to your readers how your research fits within a larger field of study (Arlene, 2014). When writing a literature review it is important focus on three main issues including a) main objective, b) identifying main argument and evidence needed, and c) identifying the basic trends and patterns in the literature. Main objective: The purpose of literature review is to justify the research project by evaluating the state of knowledge on the topic.We must focus on, what has already been said or written, what is opposed, mistaken or wrong, what is missing, what we do not know. Identifying main argument and evidence needed: In the literature review it is necessary to identify your main argument, specific hypothesis and to develop your guide for gathering, the evidence to examine this argument/hypothesis. Identifying the basic trends and patterns: The main strategy should be identifying the main trends and patterns in the main sources (books,

220 Research Methods 8 articles, thesis, reports etc.) you read. But, you should pay attention to following questions: What theories seem to be referred to/used most often? What are the common assumptions most researchers are making? What are the common methods used? What are the points of agreement/disagreement among the authors? How has each scholar contributed with their work - are they the first to apply a particular method, first to survey a particular group, first to make a particular argument, and by doing so, how have they advanced the debate on the subject (shifted the question of interest etc.)? Do any other researchers share your views (your “working thesis”)? How has the literature on the topic evolved in recent years? (Hubbuch, 1992, p. 133). Each question must be answered carefully. One strategy for identifying the trends and patterns in the literature (suggested by Hubbuch, 1992, 133) is to make a Matrix 8.2 as follows:

Matrix 8.2 Another matrix would be helpful to focus on main sources related to research topic

Theme: International Relations Theories

Concepts

References Realism Neorealism Neoliberalism Liberalism Postcolonialism Marxism Conscructivism Feminism ...... 1. 2. 3. 4. 5. ...

Using Transition Words As mentioned earlier when writing a paragraph, we should have a main argument, plenty of supporting evidence, and a strategy for organizing our thesis or paper. Transition words are words or phrases used in the topic sentence of a paragraph that give the reader clues about the paragraph logically relates to the rest of the paper. Transition words can also reveal different types of logical relationships, and that logically relate a particular paragraph (George Mason University, 2018). In general, there are three types of transition words used when writing paragraphs (University of North Carolina, 2018). 1. Transitions between sections: Particularly in longer works, it may be necessary to include transitional paragraphs that summarize the information just covered for the reader and specify the relevance of this information to the discussion in the following section. 2. Transitions between paragraphs: If you have done a good job of arranging paragraphs so that the content of one leads logically to the next, the transition will highlight a relationship that already exists by summarizing the previous paragraph and suggesting something of the content of the paragraph that follows. A transition between paragraphs can be a word or two (however, for example, similarly), a phrase, or a sentence. Transitions can be at the end of the first paragraph, at the beginning of the second paragraph, or in both places. 3. Transitions within paragraphs: As with transitions between sections and paragraphs, transitions within paragraphs act as cues by helping readers Transitions word to anticipate what is coming before they read it. Transitions are phrases or words used to Within paragraphs, transitions tend to be single connect one idea to the next. words or short phrases.

221 Writing Research Report 8 You can see transitions and their intended use when writing paragraphs in an academic paper in Table 8.2.

Table 8.2 Transitions and their intended use when writing paragraphs For Example, If Start The Topic Sentence Of The Paragraph With You are beginning a paragraph that continues the argument of the previous Likewise, in addition, similarly, furthermore, also, moreover paragraph or makes a similar point You are beginning a paragraph that Although, yet, but, on the one hand… on the other hand, contradicts or qualifies the previous however paragraph Because/Since [idea from previous paragraph], [idea for You want to show cause and effect new paragraph]. Example: “Because the administration cut funding, the program’s effectiveness is diminished.” Logical Relationships Transitional Expression Similarity Also, in the same way, just as ... so too, likewise, similarly But, however, in spite of, on the one hand ... on the other Exception/Contrast hand, nevertheless, nonetheless, notwithstanding, in contrast, on the contrary, still, yet After, afterward, at last, before, currently, during, earlier, Time immediately, later, meanwhile, now, recently Cause and effect Accordingly, consequently, hence, so, therefore, thus You want to show additional support or Additionally, again, also, and, as well, besides, equally evidence important, further, furthermore, in addition, moreover, then You want to give examples For example, for instance, namely, specifically, to illustrate You want to emphasis Even, indeed, in fact, of course, truly Finally, in a word, in brief, briefly, in conclusion, in the end, You want to conclude/Summaries in the final analysis, on the whole, thus, to conclude, to summarize, in sum, to sum up, in summary

internet For further additional source about the transition words, please visit its web sites: 1. http://www.smart-words.org/linking-words/transition-words.html 2. http://www2.eit.ac.nz/library/ls_guides_sentencestarters.html

WRITING A METHOD SECTION The method section is critical because it allows other researchers to see exactly how you conducted your research. This is one of the most crucial parts of a research, and extreme care should be given when writing. Because you will have to do the research from the beginning if you choose a wrong research design, wrong research sample selection, or a problematic data collection tool in terms of validity and reliability, and a wrong statistical analysis technique. This will cause serious effort and time loss (Lunenburg, 2008). The methodology section describes actions to be taken to investigate a research problem and the rationale

222 Research Methods 8 for the application of specific procedures or In social studies specifically three main techniques used to identify, select, process, and research designs have been used by the analyze information applied to understanding the researchers, namely quantitative, qualitative and problem, thereby, allowing the reader to critically mixed research methodologies. The methodology evaluate a study’s overall validity and reliability. should be effectively written. As Blair (2016) The method section of a research paper answers notes the researchers should introduce the overall two main questions: How was the data collected or methodological approach for investigating their generated? And, how was it analyzed? The writing research problem. Is the study qualitative or should be direct and precise and always written in the past tense (Kallet, 2004). Subheadings in the quantitative or a combination of both (mixed method section may differ according to disciplines. method)? Are they going to take a special Therefore, there are no standard headings accepted approach, such as case study, or a more neutral and used for all disciplines. In general, we should stance? In addition, the researchers should provide detailed information on the research design/ indicate how the approach fits the overall research model, research samples/participants, data gathering design. Methods for gathering data should have instruments/equipment, materials and procedure, a clear connection to research problem. In other validity and reliability of the research instruments and words, methods will actually address the problem. data analysis. The method section should provide One of the most common deficiencies found in enough information to allow other researchers to research papers is that the proposed methodology reproduce our work. is not suitable to achieving the stated objective of paper. As Vaus (2006) stated social researchers Research Design/Model ask two fundamental types of research questions: Before beginning to write a thesis or research 1) What is going on (descriptive research)? 2) paper, the researchers need to decide how to design Why is it going on (explanatory research)? A the study. The research design refers to the overall well-developed design will achieve the following strategy that researcher choose to integrate the (Vaus, 2006): different components of the study in a coherent and 1. Identify the research problem clearly and logical way, thereby, ensuring they will effectively justify its selection, particularly in relation address the research problem (Vaus, 2006). The to any valid alternative designs that could central role of research design is to minimize the chance of drawing incorrect causal inferences from have been used, data. When designing research it is essential that 2. Review and synthesize previously published researchers identify the type of, evidence required literature associated with the research to answer the research question in a convincing problem, way. This means that collecting evidence that is 3. Clearly and explicitly specify hypotheses consistent with a particular theory or explanation [i.e., research questions] central to the only is not enough. problem, 4. Effectively describe the data which will be important necessary for an adequate testing of the hypotheses and explain how such data will be obtained, and The introduction to methodology section 5. Describe the methods of analysis to should begin by restating the research problem and underlying assumptions be applied to the data in determining underpinning the study. whether or not the hypotheses are true or false.

223 Writing Research Report 8

important

It is important that researchers are able to create appropriate research designs which are in line with the purpose and scope of their studies. One mistake often observed in research studies is the incorrect identification of the research design. For example, in a study where the relationships between research variables and group differences are investigated, the researchers often characterize the research design as a relational study, and in doing so avoid investigating important group differences. When this occurs it appears that the researchers are choosing not to place importance on differences and consequently, they fail to emphasize the comparative nature of the research. The justification for utilizing specific research designs is another point of contention because researchers very often avoid explaining their choice of research design. It is important to explain which type of mixed research design is being utilized as well as the research steps that will be followed and for what reason. In addition, it is also important not to consider all studies utilizing both quantitative and qualitative methodologies in the context of a mixed study design as well as to plan mixed study designs according to the appropriate characteristics of mixed study designs.

Research Sample/Participants population, then probability sampling is far more Sampling is the method for selecting people, useful and precise. Unfortunately, it is also much events or objects for study in research. Non- more difficult and expensive than nonprobability probability and probability sampling strategies sampling. enable the researcher to target data collection techniques. Lammers and Badia (2004) states that the way in which the researchers select a sample of individuals to be research participants is critical. internet How they select participants (random sampling) Additional source for sampling to visit website: will determine the population to which they may http://uca.edu/psychology/files/2013/08/ generalize our research findings. Ch7-Sampling-Techniques.pdf According to Lammers and Badia (2004) researchers use two major sampling techniques: probability sampling and nonprobability When writing a thesis or article the procedures sampling. With probability sampling, a used obviously vary with the type of research researcher can specify the probability of an design whether it is descriptive-experimental or element’s (participant’s) being included in the quantitative-qualitative-mixed method. Whatever sample. A probability sampling scheme is one in the types of research, as Frye (2016) suggested, which every unit in the population has a chance procedure; (greater than zero) of being selected in the sample, • Tells the reader how the data were collected. and this probability can be accurately determined. • Clearly shows the order in which things nonprobability sampling With , there is no way occurred. of estimating the probability of an element’s being • States how the sample was recruited. included in a sample. Any sampling method where • Notes who collected the data. some elements of population have no chance of • States clearly all processes or activities selection (these are sometimes referred to as ‘out of participants engaged in. coverage’/’undercovered’), or where the probability • Says what happened in the study. of selection can’t be accurately determined If • Notes where all the activities took place. generalizing the the researcher’s interest is in • May include other important details. findings derived from the sample to the general • How were participants identified?

224 Research Methods 8 • Where were they recruited? • What method, exactly, was used to recruit them? • For example, with fliers in local grocery stores asking interested volunteers to make contact by phone or email? • Using random digit dialing from a list of phone numbers from an entire zipcode? In general researchers write their participants section for a proposal in the future tense; and past tense for participant description from a completed study. 1. For a proposal (Use future tense) example: Participants will be a convenience sample of 30 couples who have been married or cohabiting at least 10 years, and are at least 30 years of age and under age 55. Both members of the couple must be employed full time outside the home. Couples may be of any sexual orientation, and any race or ethnicity. Couples with a reported or documented history of domestic violence will be excluded. There are no other exclusion criteria. 2. For a completed (Use past tense) example: Participants were 42 adults who met criteria for a diagnosis of major depressive disorder. This convenience sample was 100% Hispanic American, and included 18 men and 26 women. Participants with current alcohol or drug problems or a history of psychosis were excluded. After stating the sampling techniques and the number of participants included in the study you can provide a table illustrating demographic information about the participants.

important

Some researchers mention using random sampling methods in their studies but fail to clearly explain the process. This is important because when random sampling method is utilized it is necessary to explain in detail the nature of this method as well as how the participants are selected. Some researchers state that they use random sampling; however, when the characteristics of the selected group are taken into consideration it appears that they are actually using convenience-sampling method. In such cases, it is conceivable that some researchers may lack sufficient knowledge about sampling methods or they may have chosen specific methods to make research more attractive. In the research design, how and which method is used for selecting participants should be clearly and honestly reported. While the generalizability of the findings for studies utilizing random sampling methods is acceptable, the generalizability of the findings for studies not using these methods is limited to just the characteristics of the participants. In such studies it is useful to represent participants through many characteristics and to increase their diversity in terms of the representation of the participants. Finally, it is necessary to justify the reasons why each participant group was chosen for the study. For example, when a researcher conducts a research study on “bullying” behaviour among middle school students, s/he can justify that the rate of bullying incidents among school aged children peaks at the middle school level, and as a result, the researcher has grounds to support why research on this group is needed.

Research Instruments/Measures Research instruments are the tools and approaches used to collect data to answer the research question / hypothesis. More than one technique can be employed, the commonest are questionnaires and interviews. Research instruments or measures are the source of the actual data. These can be surveys, interviews, observation and measurement of demographic characteristics (gender, age, height, weight etc.) of the participants. Describing the measure is important, (Frye, 2016) because measures are the source of your data. All the results rest on whether or not the measures are adequate, whether measures help the reader judge the validity of the results or not. Therefore, the measures section may be a lengthy and detailed section. Research instruments/measures include the following elements:

225 Writing Research Report 8 • The construct or variable being measured, • The name of the measure, • A citation for the scale/measure if it is published, • How many items there are, • A sample item, • How items are scored, • What higher and lower scores mean, • Research or data supporting the reliability of the scale/measure, • Research supporting the validity of the scale/measure. Many studies measure constructs and variables. A construct might be depression, aggression, abuse, and agitation. There can be more than one measure for a single construct or several constructs in one study. A variable is simply something that is measured, like level of education, work experience, height, weight or income. A measure can be adapted or newly designed and developed for a specific research purpose. Constructs such as anxiety, risk behaviors, attitudes about marriage, etc., are likely (hopefully) using an existing measure. Existing measures have formal names and usually abbreviations. If it is a published measure it should be cited. When writing measures section, it is very helpful for the reader to include an actual question or item from the measure that is described. This helps the reader see how items are worded. In addition, how items are scored should be explained. Items may be scored in different ways, such as true/false; or on a Likert type scale “never, rarely, sometimes, often, always”. This helps the reader judge if the measure was adequate. In addition, we need to explain number of items/sample item/scoring when describing measures.

Reliability and Validity of Measure There must be evidence that a measure is reliable and valid. Usually this will come from the published literature. Hammersley (1987, p. 73) claims that there is no widely accepted definition of reliability or validity. According to Yin (1994, p.144)) reliability demonstrates “that the operations of a study-such as the data collection procedures-can be repeated, with the same results.” The concept of validity is used to judge whether the research accurately describes the phenomenon that is intended to describe.

Reliability of a Measure Descriptions for any scale used in the study should state support for the reliability of the measure. Different types of reliability may be important for a measure. Authors needs to provide information for 1) Test-retest reliability and internal consistency reliability for survey measures, and 2) Inter-rater reliability for observational measures. Reliability and validity in various research traditions (Bush, 2007). Table 8.3 highlights differences form the positivist and phenomenological viewpoints.

Table 8.3 Reliability and validity in different research traditions Concept Positivist viewpoint Phenomenological viewpoint Has the researcher gained full access to the Does an instrument measure what it is Validity knowledge and getting insight into what they required to measure were looking for Will the same measure yield the same results on different occasions Will similar observations be made by different Reliability (assuming no real change in what is to researchers on different occasions? be measured)?

226 Research Methods 8 Validity of the instrument used is also important. There are many types of validity. Measures do not have to be shown to be valid in all ways, but some evidence of validity is usually necessary (Frye, 2016). Bell (1999, p.104) describes validity;

Validity … tell us whether an items measures or describes what is supposed to measure or describe. If an item is unreliable, then it must also lack validity, but a reliable item is not necessarily also valid. It could produce the same or similar response on all occasions, but not be measuring what it is supposed to measure.

In summary measures needs to be clearly described so the reader can understand what how the constructs and variables of interest were measured, and judge if the measures were good. There are several components to describing a measure. Describing a measure usually requires citations from the published literature.

important

It can be said that one of the most common problems with introducing measurement tools in research studies is the failure to introduce these tools in a systematic way. In relation to the introduction of a measurement tool used in a specific research design, it is necessary to first provide information about why this tool is developed, who developed the measurement tool, the numbers of items it uses, what is the nature of the tool and which dimensions it has, the sample items, how they are answered, how they are scored as well as their validity and reliability. If the instrument is adapted to Turkish, information about the original form should be provided at the outset then information regarding the adaptation should be provided in detail. In many studies, there can be problems following this protocol and information regarding the measurement tool may be presented in a mixed up way. Another problem is not investigating the validity and reliability of the measurement instruments from relevant studies’ data sets. Many researchers consider the validity and reliability studies to be scale-specific and feel that they do not need to re-examine validity and reliability in their own studies. However, evidence of validity and reliability specific to each measure should be examined. Another problem with measuring instruments is that researchers cannot explain their reasons for utilizing the specific measuring tools for their studies. Researchers should be able to explain why they have chosen to utilize these measurement tools based on related literature as well as their psychometric features. Lastly, it is important for researchers to disclose how they used the data result scores – the total score or scores in sub-dimensions – obtained from measurement tools in their analysis.

Data Analysis

Final section of the methodology of our research 3 report should include discussions about the methods of data analysis. In this section we have to explain in a Please list and explain the most common brief manner how we are going to analyze the primary problems with measurement tools in research data we will collect employing the methods explained studies. in this book (see Chapter 4 for qualitative data analysis; see Chapter 5 for quantitative data analysis). Data comes in different shapes and sizes: questionnaires, interview, focus group, observation, documents, diary entries etc. Data analysis is commonly divided into two broad types. 1) Exploratory, and 2) confirmatory. Exploratory analysis explores the data in order to reveal what the data tell you. Confirmatory analysis seeks to establish whether you have actually got what you expected to find (Robson, 2011, p. 419). Generally, data analysis may involve qualitative and quantitative approaches, dependent on the type of data collected.

227 Writing Research Report 8 Quantitative research design is one of leading • In most cases, there are a variety of different research designs and is generally associated with methods you can choose to investigate a the positivist/postpositivist paradigm. It usually research problem. The methodology section involves collecting and converting data into of your paper should clearly articulate numerical form so that statistical calculations can the reasons why you chose a particular be made and conclusions drawn (Shirish, 2014). procedure or technique. Qualitative research is the approach usually • The reader wants to know that the data associated with the social constructivist paradigm was collected or generated in a way that which emphasises the socially constructed nature is consistent with accepted practice in the of reality. It is about recording, analysing and field of study. For example, if you are using a attempting to uncover the deeper meaning and multiple choice questionnaire, readers need significance of human behaviour and experience, to know that it offered your respondents a including contradictory beliefs, behaviours and reasonable range of answers to choose from. emotions. Researchers are interested in gaining • The method must be appropriate to a rich and complex understanding of people’s fulfilling the overall aims of the study. experience and not in obtaining information which For example, you need to ensure that you can be generalized to other larger groups (Mertens, have a large enough sample size to be able 2009; Shirish, 2014). to generalize and make recommendations Quantitative data analysis covers a wide based upon the findings. range of things, from simple organization of the • The methodology should discuss the data to complex statistical analysis (Robson, problems that were anticipated and the 2011). There are differences between qualitative steps you took to prevent them from data analysis and quantitative data analysis. occurring. For any problems that do arise, Data analysis for quantitative studies involves you must describe the ways in which they critical analysis and interpretation of figures and are minimized or why these problems do numbers, and attempts to find rationale behind not impact in any meaningful way your the emergence of main findings. Data analysis interpretation of the findings. for qualitative studies, on the other hand involve • In the social and behavioral sciences, it identifying common patterns within the responses is important to always provide sufficient and critically analyzing them in order to achieve information to allow other researchers to research aims and objectives. Comparisons of adopt or replicate your methodology. This primary research findings to the findings of the information is particularly important when literature review are critically important for both a new method has been developed or an types of studies – qualitative and quantitative. innovative use of an existing method is Therefore, it is important for you to seek advice utilized. from your supervisor regarding statistical analyses in general and the choice and application of Quantitative Data Analysis statistical software in particular. Researchers must Quantitative data analysis with the application explain how they obtained and analyzed their of statistical software consists of the following results for the reasons below (Denscombe, 2014; stages (Saunders, Lewis and Thornhill, 2012): Lunenburg; 2008): 1. Preparing and checking the data. entering • Readers need to know how the data was data into computer. obtained because the method you chose 2. Selecting the most appropriate tables and affects the results and, by extension, how diagrams to use according to your research you interpreted their significance. objectives. • Methodology is crucial for any branch of 3. Selecting the most appropriate statistics to scholarship because an unreliable method describe your data. produces unreliable results and, as a 4. Selecting the most appropriate statistics to consequence, undermines the value of your examine relationships and trends in your interpretations of the findings. data.

228 Research Methods 8 The following assumptions suggested by mainstay of early culture anthropologists, Robson (2011) to researchers were when analysing ethnography is an in-depth description of a people data. group done through “immersed” participant 1. Everyone doing research needs to observation and recorded in the vernacular of the understand how to summarize and display host society. An ethnographic study may look at quantitative data. the Cree people of Western Canada. Case Study: 2. For relatively simple statistical test specialist The most common type of qualitative research, statistical software is not essential. case study looks at episodic events in a definable 3. If you need to carry out complex statistical framework bounded by time and setting. The tests you will need to use a specialist overall purpose is generally to explain “how.” An statistical computer package. example of a case study may be how the Durand 4. You have some prior acquaintance with the Line Agreement established the political boundary basic concepts and language of statistical between the nations of Afghanistan and Pakistan analysis. in the 1890s. 5. You will seek help and advice in carrying out statistical analyses. important Data collected from questionnaires or other instruments in quantitative research methods have to be analysed and interpreted. Generally, statistical One of the frequently neglected points in procedures are quantitative data approaches (see the data analysis section of research design is Chapter 5). that lack of reports regarding the assumptive tests of the statistical analyses. If there are statistical assumptions specific to the chosen Qualitative Data Analysis analyses process, then assumption tests Researchers use qualitative research design to should be carried out and reported based answer questions asked with how and why. How on the statistical method utilized. Finally, in and why. There are five approaches to qualitative data analysis another important issue is the research (Creswell, 2009) (see Chapter 4). They selection of statistical analyses appropriate for are: Narrative, phenomenology, grounded theory, the research questions as well as to justify the ethnography, case study. Each approach to particular analyses chosen. qualitative research is not mutually exclusive. They can be used in combinations. For example, a case study approach can also incorporate grounded theory. Definitions of five research traditions in HOW TO PRESENT THE qualitative research (Creswell, 2009) and examples RESULTS? for each is given below. Narrative: This approach In most thesis and articles, the results/findings to inquiry retells someone’s story across time. It are presented in a separate chapter/section. A explores what the story means and the lessons to ‘Result’ section that presents key findings emerging be learned. An example may be to study the life from the analysis that answers the research of Mustafa Kemal Atatürk. Phenomenology: The question. It is important to prepare yourself for goal of phenomenological research is to describe presenting the results. When presenting the results participants’ experiences in a specific context and Glathorn (1998) suggests two preparatory steps. understand a phenomenon. For example, what is First, reviewing the results carefully by checking it like to be homeless in İstanbul or in Ankara? the results for accuracy. Second, checking the Grounded theory: This type of qualitative approach style manual (APA, Chicago, MLA, Harward etc.) investigates a process, action, or interaction with for the format of the tables, format for statistical the goal of developing a theory. To illustrate, a notation and figures if you will be presenting some researcher is observing the relationship between of the data in these forms. Further Glathorn (1998, workers and a managers of an international p. 162) recommends the following in the initial company across cultures. Ethnography: The planning process:

229 Writing Research Report 8 1. What will be included in the appendix section of the thesis/articles? 2. What will be included in tables? 3. What figures will be needed? 4. What are the resulting contents? One of the difficult decisions for researchers is outlining the chapter/section. These are several choices researchers may prefer when writing the results (Glathorn, 1998). Glathorn suggests that the organization of the conclusion part may be made based on five options, which are hypotheses, research questions, research methodology, chronology and variables. These may be explained briefly as follows: By hypotheses: If the study involves research or null hypotheses, you may decide to order by hypotheses: Hypotheses 1, Hypotheses 2, and so on. By research questions: If the study involved two or more specific research questions, then ordering in terms of those questions would be useful. Question 1, Question 2, and so on. By research method: If you used multiple methods, grouping the results in relation to the methods may be helpful. Interview results, survey results, and so on. By chronology: If the time order is important, then organize by chronology: Entry, Middle Phase, and Exiting. You can also use a chronological pattern in studies that relied chiefly on testing: pretest results, posttest results. By variable: If the variables are critical to the study, then order by variable, such as age, self-concept, achievement level.

Presenting Data in Tables and Figures American Psychological Association (2010) style is most commonly used to cite sources within the social sciences. According to Paiz et al., (2018) the purpose of tables and figures in documents is to enhance your readers’ understanding of the information in the document. They describe some general guidelines when using tables (Paiz et al., 2018): 1. Necessity. Ask yourself this question first: Is the table or figure necessary? For example, it is better to present simple descriptive statistics in the text, not in a table. 2. Relation of Tables or Figures and Text. Because tables and figures supplement the text, refer in the text to all tables and figures used and explain what the reader should look for when using the table or figure. Focus only on the important point the reader should draw from them, and leave the details for the reader to examine on their own. 3. Documentation. If you are using figures, tables and/or data from other sources, be sure to gather all the information you will need to properly document your sources. 4. Integrity and Independence. Each table and figure must be intelligible without reference to the text, so be sure to include an explanation of every abbreviation (except the standard statistical symbols and abbreviations). 5. Organization, Consistency, and Coherence. Number all tables sequentially as you refer to them in the text (Table 1, Table 2, etc.), likewise for figures (Figure 1, Figure 2, etc.). Abbreviations, terminology, and probability level values must be consistent across tables and figures in the same article. Likewise, formats, titles, and headings must be consistent. Do not repeat the same data in different tables. Make sure that your tables. Answer the following questions while preparing your tables: • Is the table necessary? • Is the entire table single- or double-spaced (including the title, headings, and notes)? • Are all comparable tables presented consistently? • Is the title brief but explanatory? • Does every column have a column heading? • Are all abbreviations; special use of italics, parentheses, and dashes; and special symbols ex- plained?

230 Research Methods 8 • Are all probability level values correctly identified, and are asterisks attached to the appropriate table entries? Is a probability level assigned the same number of asterisks in all the tables in the same document? • Are the notes organized according to the convention of general, specific, probability? • Are all vertical rules eliminated? • If the table or its data are from another source, is the source properly cited? • Is the table referred to in the text? Reporting statistics in APA Style is also important. The following examples in Table 8.4 illustrate how to report statistics in the text of a research report. You will note that significance levels in journal articles- -especially in tables--are often reported as either “p > .05,” “p < .05,” “p < .01,” or “p < .001.” APA style dictates reporting the exact p value within the text of a manuscript (unless the p value is less than .001) (Kahn, 2012).

Table 8.4 Reporting statistics in APA style

Statistics Reporting and presenting

Mean and Standard The sample as a whole was relatively young (M = 19.22, SD = 3.45). The average age Deviation of students was 19.22 years (SD = 3.45)

The percentage of participants that were married did not differ by gender, χ2(1, N Chi-Square = 90) =0.89, p = .35.

There was a significant effect for gender, t(54) = 5.43, p < .001, with men receiving T Tests higher scores than women

There was a significant main effect for treatment,F (1, 145) = 5.43, p = .02, and a ANOVAs significant interaction,F (2, 145) = 3.24, p = .04.

Correlations The two variables were strongly correlated, r(55) = .49, p < .01.

Social support significantly predicted depression scores,β = -.34, t(225) = 6.53, p < Regression .001. Social support also explained a significant proportion of variance in depression scores, R2 = .12, F(1, 225) = 42.64, p < .001

For quantitative data you may present frequency tables or graphs of variables of interest. When presenting qualitative findings, it is important that you do not only discuss and present a single and dominant view, but also acknowledge contradictions and disagreements within the data. Figure 6 illustrates how a qualitative researcher shows how he/she went from having descriptive codes to focusing on a few distinct, interpretative and networked themes that they can use to begin answering parts of their research question. The sample below text is taken form the manuscript published by Kurum and Çınkır (2017). The findings were presented by taking care the order of the research questions. Accordingly, the participants were first asked about the reasons for the unification of the education supervision. The reasons reported by the participants are presented in Figure 8.1

231 Writing Research Report 8 Opening Supervision to Doing the Political Same/Similar In uence (f=9) Global Themes Job (f=9)

Reasons for Reasons for Improvement the Reasons for the the Centralization of Employee Unication Unication of Unication of the Rights (f=2) of the the Education of the Supervision Education Supervision Education (f=5) Supervision Supervision Extrinsic to Extrinsic to Codes the Nature the Nature Organising Codes of the of the Themes Supervision Supervision

Other (f=1)

Figure 8.1 Reasons for Unification of the Education Supervision Source: Kurum and Cınkır, 2017.

As seen in Figure 8.1, the reasons reported by the participants for the unification of the supervision fall under two groups: reasons arising from the intrinsic nature of the supervision and reasons extrinsic to the nature of the supervision. The most significant reason arising from the intrinsic nature of the supervision was reported as “Doing the Same/Similar Job.”... You can also use the below pattern when answering your qualitative research questions raised during interviews and focus group.

Codes Basic theme Organising theme Global theme

Struggle of unemployed graduate students

Figure 8.2 From codes to organising and global themes

DISCUSSION, CONCLUSION AND RECOMMENDATION This section is probably the longest section and worth spending time on. It brings everything together, showing how our findings respond to the brief we explained in or introduction and the previous research we surveyed in our literature survey. It should be written in a discursive style, meaning we need to discuss not only what our findings show but why they show this, using evidence from previous research to back up our explanations. This is also the place to answer our research questions and theses hypotheses.

232 Research Methods 8 The purpose of the discussion is to interpret and describe the significance of your findings in the light of what was already known about the research problem being investigated, and to explain any new understanding or insights about the problem after you’ve taken the findings into consideration. The discussion will always connect to the introduction by way of the research questions or hypotheses you posed and the literature you reviewed, but it does not simply repeat or rearrange the introduction; the discussion should always explain how your study has moved the reader’s understanding of the research problem forward from where you left them at the end of the introduction (Annesley, 2010). Glathorn (1998) suggests to begin writing this section by reviewing the results of study. We do not need to include all the results but general results that we think important and have value for the readers and knowledge in the field. Main content of the discussion and conclusion section (Glathorn 1998; Annesley, 2010) is explained below in Table 8.5.

Table 8.5 Main content of the discussion and conclusion section Main section Explanation This final chapter restates the research problem and reviews the major 1. Introduction sentence methods used in the study. The major sections of this chapter summarize the results and discuss their implications. 2. Statement of the State the problem exactly as it appears in introduction section. As explained in problem Chapter…/methodology section, the study reported here was a case study of … Make sure that review of the methodology provides sufficient information for 3. Review of the a reader who has not read the detailed methodology section to understand methodology how the study was conducted. Did the research methodology and data collected answer the research question? When summarising the results, keep in mind the importance results you are going to discuss here. You do not need to include all the results but general results that we think important and have value for the readers and contribute 4. Summary of the main the knowledge in the field. Highlight the importance of your study and how results it may be able to contribute to and/or help fill existing gaps in the field. If appropriate, the discussion section is also where you state how the findings from your study revealed new gaps in the literature that had not been previously exposed or adequately described. Either compare your results with the findings from other studies or use the studies to support a claim. This can include re-visiting key sources already cited in your literature review section, or, save them to cite later in the 5. References to previous discussion section if they are more important to compare with your results research instead of being a part of the general literature review of research used to provide context and background information. How do the different findings interact? Do they explain each other or are there contradictions? 6. Discussion of the results Do the findings support our research questions, hypotheses (quantitative)?

7. Recommendations for What are the recommendations for people interested in research topic (i.e. educators, policy makers educators, policy makers…)?

Present the underlying meaning of your research, note possible implications 8. Implications for research in other areas of study, and explore possible improvements that can be made and practice in order to further develop the concerns of your research. 9. Suggestions for further Are there any areas that require further research or follow up? research 10. Limitations What were the limitations of the study and how do they affect the results?

233 Writing Research Report 8 satisfactory and why others are not. Only by giving both sides to the argument can you make your explanation convincing. 4 8. Discuss and evaluate conflicting Please explain the main similarities and explanations of the results. This is the sign differences between discussion and conclusion of a good discussion. section. 9. Discuss any unexpected findings. When discussing an unexpected finding, begin the paragraph with the finding and then Kretchmer (2008) defines fourteen steps in describe it. their paper “Fourteen Steps to Writing an Effective 10. Identify potential limitations and Discussion Section”: weaknesses and comment on the relative 1. Organize the Discussion from the specific to importance of these to your interpretation the general: Your findings to the literature, of the results and how they may affect the to theory, to practice. validity of the findings. When identifying 2. Use the same key terms, the same verb tense limitations and weaknesses, avoid using an (present tense), and the same point of view apologetic tone. that you used when posing the questions in 11. Summarize concisely the principal the Introduction. implications of the findings regardless of 3. Begin by re-stating the hypothesis you were statistical significance. testing and answering the questions posed 12. Provide recommendations (no more than in the introduction. two) for further research. Do not offer 4. Support the answers with the results. suggestions, which could have been easily Address all the results relating to the addressed within the study, as this shows questions, regardless of whether or not the there has been inadequate examination and findings were statistically significant. interpretation of the data. 5. Describe the patterns, principles, and 13. Explain how the results and conclusions relationships shown by each major finding/ of this study are important and how they result and put them in perspective. The influence our knowledge or understanding sequencing of providing this information of the problem being examined. is important; first state the answer, then 14. Discuss everything, but be concise, the relevant results before citing the work brief, and specific in your writing of the of others. If necessary, point the reader to a Discussion. figure or table to enhance the “story”. Chapter 8 was about writing a research report 6. Support your answers by explaining how for the researchers including undergraduate and your results relate to expectations and to postgraduate students as well for the practicing the literature, clearly stating why they are researchers who are working and writing in acceptable and how they are consistent or various disciplines. Murray (2017, p. 142) in fit in with previously published knowledge her book “How to Write a Thesis” sums up the on the topic. generic thesis structure and research report as 7. Defend your answers, if necessary, by follows. explaining both why your answer is

234 Research Methods 8 introduction/background/review of literature • summarize and evaluate books, articles, theses, etc. • define the gap in the literature. • define and justify your project. theory/approach/method/materials/subjects • define method, theoretical approach, instrument. • method of inquiry. • show links between your method and others. • justify your method. analysis/results • report what you did, list steps followed. • document the analysis, showing how you carried it out. • report what you found. • prioritize sections for the thesis or for an appendix. interpretation/discussion • ınterpret what you found. • justify your interpretation. • synthesize results in illustrations, tables, graphs, etc. conclusions/implications/recommendations for future research. • for future practice. • report issues which were beyond the scope of this study.

235 Writing Research Report 8 Explain how to choose a research topic, write a structured abstract, know steps for writing introduction and principles of writing paragraphs, know LO 1 paraphrase, cite and avoid plagiarism.

The most critical issue in the process of writing a research report is finding an original research topic and research title. When choosing a research topic your supervisor provides you with a general topic from which you study a particular aspect; your supervisor provides you with a list of possible topics to study and you choose a topic from that list; or, your supervisor leaves it up to you to choose a topic. The research title you choose will vary according to whether your research is a quantitative, qualitative or mixed method and should reflect your research paradigm. In general, an abstract contains 200-300 words and includes subheadings as problem statement, purpose, methods, results, and implications. In your abstract, 3-5 key words that best reflect your research should be provided. In the research report, the introduction section is important in terms Summary of explaining the need for research, describing the gap in the literature and explaining why this research is done. Writing paragraphs in the introduction and other sections is critical. For this reason, it is important to have basic academic writing skills in paragraph writing. When writing a paragraphs, the researchers should pay attention to, most paragraphs include at least three sentences, though rarely not more than ten. It is important paying attention to direct and indirect quoting rules when paraphrasing and writing a paragraph. Again, the difference between paraphrasing and quoting should not be forgotten. Thus, you should avoid plagiarism. The writing of the introduction section is also referred to as the writing of the literature review and it includes the synthesis and review of the literature related to your research topic.

Develop competence in writing research design, choosing sample/ participants, discussing research instrument-procedure and writing data LO 2 analysis section.

Writing about research design, sample/participants, measure and materials, processes, data analysis and the interpretation of data are all about writing about other research. In other words, by looking at the method section of a research report it is possible to understand whether you have chosen a suitable research design for your research, whether you have chosen the appropriate sample/participants and processes, and whether you have used appropriate data analysis techniques and you have presented findings and made interpretation. In the method section, instead of providing a detailed literature to the reader and describing the research design and data analysis strategies in detail, you are expected to explain why you chose the specific research design or analysis and why they were the most appropriate for your study. The method section of a research report should be written in the past tense. In this section, you should provide sufficient information about the design of the research, sample/ participant of the research, data collection tools/processes, validity and reliability studies, data analysis and interpretation. This section should be inspired by other researchers and should provide necessary information they need. When writing method section, it is important to keep in mind that the use of a writing style appropriate to the selected research methodology (quantitative, qualitative and mixed) reflect the specific characteristics of each methodology used in the study.

236 Research Methods 8 LO 3 Develop skills in writing discussion, conclusion and recommendations.

The final section in writing a research report is to interpret the findings in the discussion section, draw conclusions, and make recommendations. It is important that everything in this section should be based on the results of the data analysis. If the study is an empirical research study, the conclusions and recommendations must be directly related to the data that was gathered and

analyzed. It is highly recommended that do not draw any conclusion or make Summary recommendation on a topic that was not assessed by the research measures/ materials. In this section, researchers are expected to discuss not only what their findings show but why they show this, using evidence from previous research to back up their explanations. This is also the place to answer research questions and theses hypotheses. The researchers do not need to include all the results but general results that they think is important and have value for the readers and knowledge in the field. In this section, the readers also want to see recommendations for educators, policy makers, implications for research and practice, suggestions for further research, and limitations of the current study.

237 Writing Research Report 8

1 In her dissertation, Duygu wants to include 6 What is the proper order of the steps in the reason she chose her particular topic instead of writing a methodology chapter? other similar ones. In which section of the report a. Research sample, research design, instrument should Duygu include this information? and procedure, data analysis b. Research design, research sample, data analysis, a. Purpose b. Procedures instrument and procedure, c. Results d. Conclusions c. Research sample, research design, data analysis, e. Implications instrument and procedure d. Research design, research sample, instrument 2 What is the proper order of the steps in and procedure, data analysis writing a research paper? e. Research design, instrument and procedure, research sample, data analysis a. Literature review, Introduction, Discussion, Test Yourself Test Methodology, Results 7 What is the proper order of the steps in b. Introduction, Literature review, Methodology, writing an opening paragraph? A statement about; Results, Discussion a. aim of the study, the general topic, what the c. Introduction, Literature review, Methodology literature has found, what the literature is Results, Discussion, Methodology missing. d. Methodology, Literature review, Introduction, b. the general topic, what the literature has found, Discussion, Results what the literature is missing, aim of the study. e. Literature review, Methodology, Introduction, c. what the literature is missing, the general topic, Discussion, Results aim of the study, what the literature has found. d. the general topic, aim of the study, what the literature is missing, what the literature has found. What should you not do in your paper’s 3 e. what the literature has found, the general topic, conclusion? aim of the study, what the literature is missing. a. Summarize your paper’s main point or thesis (since it’s unnecessary). 8 In her assignment, if Duygu writes a sentence like “For this reason, interventions with high-risk b. Introduce a final, strong argument to support families are needed to prevent abuse of the next your thesis. generation”. How is this sentence called? c. State why the results of your research are a. Topic sentence b. Supporting sentence significant. c. Closing sentence d. Evidence sentence d. Point out where further research on your topic is e. Main sentence needed. e. Recommendations for policy makers 9 Which chapter of a research paper brings everything together including statement of the 4 What is the function of a research paper? problem, review of methodology, summary of the main results, reference to previous research? a. To entertain b. To inform a. Introduction b. Literature review c. To reflect d. To tell a story c. Findings d. Discussion e. To discuss e. Method 10 5 What types of sources are suitable for research In general, for a methodology section which papers? tenses do we use when writing a proposal and completing the report? a. Wikipedia, Blogs a. Future Tense-Past Tense b. Personal experience b. Present Tense-Simple Past Tense c. Academic journals, books, thesis c. Past Tense-Future Tense d. Webpages with no authors d. Future Tense-Present Tense e. Anecdotal evidence e. Past Tense – Present Tense

238 Research Methods 8

1. a If your answer is incorrect, review the section 6. d If your answer is incorrect, review the section on “Choosing an Original Research Topic”. on “Writing A Method Section”. Answer Key for “Test Yourself” If your answer is incorrect, review the If your answer is incorrect, review the section 2. b section on “Discussion, Conclusion and 7. b on “Writing A Paragraph”. Recommendation”.

If your answer is incorrect, review the If your answer is incorrect, review the section 3. a section on “Discussion, Conclusion and 8. c on “Writing a Paragraph”. Recommendation”.

If your answer is incorrect, review the If your answer is incorrect, review the section 4. b 9. d section on “Discussion, Conclusion and on “Introduction”. Recommendation”.

If your answer is incorrect, review the section 5. c 10. a If your answer is incorrect, review the section on “Literature Review”. on “Research Sample/Participants”.

Why do you think introduction paragraph in a scientific research report should be written different than other paragraphs?

A well-written introduction is important because, quite simply, you never get a second chance to make a good first impression. The opening paragraphs of the research paper will provide our readers with their initial impressions about the logic of our argument, writing style, the overall quality of our research, and, ultimately, your turn 1 the validity of our findings and conclusions. A vague, disorganized, or error-filled introduction will create a negative impression, whereas, a concise, engaging, and well-written introduction will lead the readers to think highly of our analytical skills, writing style, and research approach. All introductions should conclude with a brief paragraph that describes the organization of the rest of the paper. Suggested answers for “Your turn”

Discuss the relationship between citation and plagiarism in academic writing.

Plagiarism as “an object (language, words, text, diagram, graphs, ideas) which has been taken from a source (books, journals, internet, not just text), by an agent (person, student, academic) without adequate acknowledgement and with or without intention to deceive”. Therefore, “an allegation of plagiarism is intent-neutral. In other words, because the reader cannot discern whether the absence of a citation was done deliberately or you simply forgot, it is important to review your paper before you submit it to ensure you have listed all sources used during your research. This is why it is important to keep track of everything you have used during the course of writing your paper so you can easily assess your turn 2 whether all your sources have been cited properly. On the other hand, citation whether it is direct or indirect is giving credit to the sources you use by showing the author and page numbers in your paper right alongside your use of the main source. It also includes giving a correct bibliography so that your reader can go straight back to your original sources. Plagiarism is the lack of citation or inadequate citation. In general plagiarism is seen as passing off another’s words as your own, or even stealing another’s words or work. Plagiarism is an offense whether it was intentional or not. Therefore, it is important to improve your knowledge and skills in academic writing.

239 Writing Research Report 8 Please list and explain the most common problems with measurement tools in research studies?

• The most common problem is failure to introduce measurement tools in a systematic way including providing information about why this tool is developed, who developed the measurement tool, the numbers of items it uses, what is the nature of the tool and which dimensions it has, the sample items, how they are answered, how they are scored as well as their validity your turn 3 and reliability. • If the instrument is adapted to Turkish, not providing information about the original form. Another problem is not investigating the validity and reliability of the measurement instruments from relevant studies’ data sets. Many researchers consider the validity and reliability studies to be scale-specific and feel that they do not need to re-examine validity and reliability in their own studies. • Another problem with measuring instruments is that researchers cannot explain their reasons for utilizing the specific measuring tools for their studies. Researchers should be able to explain why they have chosen to utilize these measurement tools based on related literature as well as their psychometric features. • Lastly, it is important for researchers to disclose how they used the data result scores – the total score or scores in sub-dimensions – obtained from Suggested answers for “Your turn” Suggested answers for “Your measurement tools in their analysis.

Please explain the main similarities and differences between discussion and conclusion section.

The terms discussion and conclusion usually represent two separate concepts. your turn 4 However, there are some similarities between them in academic writing. The first difference is that, they both generally have a different purpose. In general under discussion, it is expected from the researchers to answer their research question. The discussion is a detailed presentation of research findings and provides scientific back-up for main arguments. It explains your findings and interprets them in context of previous work, as well as provides some suggestions for future research. The conclusion, on the other hand, is generally brief and provides just the main points of your dissertation. It can be seen as a summary of your discussion and tells the reader why your research matters. In addition, when compared to discussion, conclusion should be kept shorter and must be written straightforward.

240 Research Methods 8

Further Reading

“Documentation,” The writer’s handbook:“Graduate writing resources,” The University of Documentation styles, The University of Wisconsin- Western Ontario. Available at: http://www.sds. Madison, 2012. Available at: http://writing.wisc. uwo.ca/writing/index.html?gradhandouts. edu/Handbook/Documentation.html. Purdue University Online Writing Lab. Available at: “Plagiarism and referencing,” Presentation, Student http://owl.english.purdue.edu/owl/ Development Services, The University of Western Strunk, W. and White, E. B. (1918, 1999). Elements of Ontario. Available at: http://www.sdc.uwo.ca/ style. Available at: http://www.bartleby.com/141/ writing/index.html?grad. “The writing process: Revising, editing and proofreading,” “Quoting and paraphrasing,” The writer’s handbook: Presentation, Student Development Services, The Avoiding plagiarism, The University of Wisconsin- University of Western Ontario. Available at: Madison, 2012. Available at: http://writing.wisc. http://www.sdc.uwo.ca/writing/index.html?grad. edu/Handbook/QuotingSources.html. http://www2.eit.ac.nz/library/ls_guides_ Chicago manual of style online, The. Available at: sentencestarters.html http://www.chicagomanualofstyle.org/home.html “Effective writing at the graduate level: Academic writing style,” Presentation, Student Development Services, The University of Western Ontario. Available at: http://www.sdc.uwo.ca/writing/ index.html?grad.

References

Alderman, J. (2014). Choosing a research topic. Bush, T. (2007). Authencity in research-reliability, University of North Florida Digital Commons, validity and triangulation. In A. R. J. Briggs, & M. Coleman, (Eds.). Research methods in Annesley, T. M. (2010). The discussion section: Your educational leadership and management. (pp. 91- closing argument. Clinical Chemistry, 56,1671- 105). London: Sage. 1674. Creswell, J. W. (2009). Research design: Qualitative, American Psychological Association. (2010). quantitative and mixed method approaches. (3rd Publication manual of the American Psychological ed.). London: Sage. Association. (6th ed.). Washington, DC: American Psychological Association. Denscombe, M. (2014). The good research guide: For small-scale social research projects. (5th ed.). Arlene, F. (2014). Conducting research literature Buckingham, UK: Open University Press. reviews. University of California at Los Angeles, USA: Sage. Driscoll, D. L., & Brizee, A. (2013). What is primary research and how do I get started? The Writing Lab Bell, J. (1999). Doing your research project (3rd ed.). & The OWL at Purdue and Purdue University. Buckingham: Open University Press. Frye, A. (2016). Learning with purpose. Department of Blair, L. (2016). Writing a graduate thesis or dissertation. Psychology, University of Massachusetts, Lowell. Rotterdam/Boston. TAIPEI: Sense Publisher. George Mason University (2018). Department Block, S., Greenberg, S., & Goodman, G. (2009). of English writing guide. Available at: https:// Remembrance of eyewitness testimony: Effects of writingcenter.gmu.edu/writing-resources emotional content, self-relevance, and emotional Glathorn, A. A. (1998). Writing the winning tone. Journal of Applied Social Psychology, 39, dissertation: A step-by-step guide. California: 2859-2878. Corwin Press, Inc.

241 Writing Research Report 8 Hammersley, M. (1987). Some notes on the terms Murray, R. (2017). How to write a thesis. (4th ed.). “validity” and “reliability.” British Educational McGraw Hill: Open University Press. (Translated Research Journal, 13, 73-81. in Turkish, Çınkır, Ş. (2018). Tez nasıl yazılır. Ankara: Anı Yayıncılık). Harris, R. J. (2002). Using sources effectively: Strengthening your writing and avoiding plagiarism. Paiz, J. M., Angeli, E., Wagner, J., Lawrick, E., Los Angeles, CA: Pyrczak Publishing. Moore, K., Anderson, M., ...+ Keck, R. (2018). APA tables and figures 1. Available at: https://owl. Hirano, E. (2009). Research article introductions english.purdue.edu/owl/resource/560/19/ in English for specific purposes: A comparison between Brazilian, Portuguese, and English. Purdue University (2018). Elements of a paragraph. English for Specific Purposes, 28, 240-250, Available Purdue Online Writing Lab. Available at: https:// at: http://libguides.usc.edu/writingguide/ owl.english.purdue.edu/owl/resource/606/01/ introduction Redman, P., & Maples, W. (2011). Good essay writing: Hubbuch, S. M. (1992). Writing research papers A social sciences guide. London: Sage. across the curriculum. (3rd ed.). Harcourt Brace Reyes, V. (2017). Demystifying the journal article. Jovanovich. Open Library. Inside Higher Education. Available at: http:// Kahn, J. (2012). Reporting statistics in APA Style. libguides.usc.edu/writingguide/introduction Available at: http://users.sussex.ac.uk/~grahamh/ Robson, C. (2011). Real world research. (3rd ed.). RM1web/APA%20format%20for%20 Cornwall UK: Wiley Publication. statistical%20notation%20and%20other%20 things.pdf Samraj, B. (2002). Introductions in research articles: Variations across disciplines. English for Specific Kallet, R. H. (2004). How to write the methods Purposes, 21, 1-17, Available at: http://libguides. section of a research paper. Respiratory Care, 49, usc.edu/writingguide/introduction 1229-1232. San Franscisco Edit. (2018). Scientific, medical and Kretchmer, P. (2008). Fourteen steps to writing an general proofreading and editing. Available at: effective discussion section. Available at: http://web. https://ccc.inaoep.mx/~esucar/Clases-semdr/ science.uu.nl/biomolecularsciences/downloads/ Lecturas/All.pdf items/How_to_write_a_discussion.pdf Saunders, M., Lewis, P., & Thornhill, A. (2012). Kurum, G., & Çınkır, Ş. (2017). Marriage made Research methods for business students. England: in hell: Views of education supervisors on the Pearson. unification of education supervision in Turkey. Education & Science, 42(192), 35-57. Shirish, T. S. (2014). Research methodology in education. Solapur, India: Ashok Yakkaldevi Publication. Lammers, W. J., & Badia, P. (2004). Fundamental of behavioral research. California: Thomson Thaler, K. M. (2017). Mixed methods research in the and Wadsworth. California: Thomson and study of political and social violence and conflict. Wadsworth. Journal of Mixed Methods Research, 11(1), 59-76. Lunenburg, F. C. (2008). Writing a successful thesis University of North Carolina. (2018). The writing or dissertation: Tips and strategies for students center. Available at: https://writingcenter.unc.edu/ in the social and behavioral sciences. Thousand Vaus, D. A. (2006). Research design in social research. Oaks, CA: Corwin Press. London: Sage. Mertens, D. M. (2009). Research and evaluation in Yin R. (1994). Designing single and multiple case education and psychology: Integrating diversity studies. In N. Bennet, R. Glatter, & R. Levacic with quantitative, qualitative, and mixed methods. (Eds). Improving educational management through London: Sage. research and consultancy. London: Paul Chapman with the Open University.

242