Thematic Analysis of Survey Responses from Undergraduate Students
Total Page:16
File Type:pdf, Size:1020Kb
Thematic Analysis of Survey Responses From Undergraduate Students © 2019 SAGE Publications, Ltd. All Rights Reserved. This PDF has been generated from SAGE Research Methods Datasets. SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 Thematic Analysis of Survey Responses From Undergraduate Students Student Guide Introduction Thematic analysis is a method of examining data to gain meaningful comprehension of participant perspectives. Thematic analysis identifies patterns within the data enabling the researcher a detailed understanding of the research data. It is a useful method for analyzing qualitative data as it looks for patterns from participant communication that is not constrained by any limitations to the responses. Thematic analysis is therefore a valuable method for examining the content of responses from data collected from open-ended survey questions, focus group discussions, or interviews. In this example, a mixed-methods research study was implemented to gain student perspective on the use of technology and critical thinking development. A source of data collection was a student survey at the end of the study. Aligning with a mixed-methods approach, the survey consisted of open- and close-ended questions. The survey was used to elicit knowledge about the participants and acquire responses to specific questions directed at their attitudes, beliefs, behaviors, or emotions (Creswell, 2012; Mrug, 2010). Close-ended survey questions generally have a stem question and a set of answer alternatives to provide participants with a fixed number of responses from which they need to choose their answer from (Mrug, 2010). The open-ended questions provided a means of gaining student perspective using their own words and provided insight Page 2 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 into their comprehension and thinking about critical thinking and technology use. Thematic analysis of the student responses to these open-ended questions was thus useful in gaining meaningful understanding of the students’ points-of-view. Thematic Analysis of Open-Ended Survey Questions Surveys are used in research as a means of collecting information from a sample of participants through the responses they provide to the questions asked (Check & Schutt, 2012). Surveys encompass either close- or open-ended questions, or a combination of both. Close-ended questions using numerically rated items generate quantitative data; open-ended questions gather qualitative data; and a combination of both collecting mixed-methods data (Ponto, 2015). Surveying with open-ended questions can offer the researcher rich, unconstrained participant responses to broad questions. Open-ended questions afford participants the opportunity to express their perspective using their own language, terms, and expressions. In contrast, close-ended questions have specific, pre- selected responses from which participants must choose from. Open-ended questions therefore provide participants the freedom to respond as they think appropriate, enabling them to determine their own answers and use their own words, potentially encouraging them to share more personal and genuine perspectives. Additionally, in using their own language, the participants could also demonstrate their understanding of critical thinking in the words and terms used. While the large quantity and quality of the responses is beneficial for gaining rich understanding of the topic, the amount of data can be overwhelming. Limitations to using open-ended questions may occur in that participants may misinterpret the intent of questions if not sufficiently clear. In requiring individualized responses, greater time and thinking by participants would be required when responding (Baillou, 2008). Page 3 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 Thematic analysis has been described as a useful method of analyzing qualitative data for researchers, facilitating organization of data, and capturing valuable information (Braun & Clarke, 2006). Braun and Clarke (2006) note its flexibility in analyzing rich data and offer a guide in using this method of analysis. Thematic analysis enables the researcher to organize and analyze responses and interpret them to determine common perspectives among participants (Creswell, 2012). As with other methods of qualitative research, it is important that the researcher be aware and acknowledge personal views and biases on the topic being explored (Sutton & Austin, 2015). It is beneficial for the researcher to explicitly share their own personal stance to comprehensively position the data, analysis, and findings, and provide context and understanding to the readers. Data Exemplar: Student Perspective of Technology Use and Critical Thinking Development The data used for this example were gathered from a survey implemented with students following a university term where technology use and explicit critical thinking instruction were integrated into the course. This was one form of data collection in the study to gather student perspective on technology use and critical thinking development. Additional data collection methods and analysis included descriptive statistics with the close-ended questions in the survey and content analysis of online discussion postings. All 127 enrolled students were provided with information on the study and invited to participate as desired. The students were undergraduate learners in a beginning healthcare professions course. The technologies implemented were an in-class technology response system, Top Hat, and an online discussion posting forum on the learning management system, Desire2Learn. Of the 127 registered students, 43 students (34%) completed this end-of-term survey. Page 4 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 Thematic Analysis of the Student Responses to Open-Ended Survey Questions For this example, thematic analysis encompassed the phases as detailed by Braun and Clarke (2006): (1) gaining familiarity with your data; (2) generating initial codes or labels; (3) searching for themes or main ideas; (4) reviewing themes or main ideas; (5) defining and naming themes or main ideas; and (6) producing the report. The first question will be used as an example. It asked students to identify their preference of technology used in class and to provide a reason: “Which learning technology, Top Hat or Online Discussions, was more beneficial for your engagement in developing critical thinking? And, why?” With the first step, participant responses to the open-ended survey question were read and re-read to gain understanding and awareness of the data, and to gain familiarity with the data. Next, data that were meaningful to the study were noted, recurring messages were identified, and codes generated in the form of phrases to represent significant data. For this study, the coding was implemented by hand. The codes chosen aimed to identify the elements the students noted as important to them in their responses. The potential codes were important phrases highlighting participant ideas such as the following from responses to Question 1: Participant responses (indicating preference for classroom response system, Top Codes highlighting Hat) participant ideas because it helped me learn how to prioritize important information and choose certain Decision-making in choosing answers over others answers was more beneficial because it prepared for exams Exam preparation Page 5 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 because I got to see the type & level of questions that I would see on my exams and it also Exam questions was a great review for exams The codes were next collated to determine an overarching idea under which to organize the important phrases identified, which Braun and Clarke (2006) describe as “searching for themes” (p. 89). The themes constituting these main ideas were reviewed to ensure they each encompassed clear, accurate phrases of importance, clearly explaining the focus of the ideas. The themes were labelled and reviewed to ensure that they were appropriate and comprehensive in describing the data. From the example, two themes identified by participants preferring the classroom response system, Top Hat, were Multiple Choice as Preparation for Exams and Engagement and Participation: Participant responses Codes Themes because it was more similar to the NCLEX (National Council Multiple choice as preparation Exam preparation Licensure Examination) for exams Multiple choice as preparation it helped me study for my exam Exam preparation for exams because it was done in class time and it covered what we just In-class learning Engagement and participation learned Anonymity leading to I wasn’t afraid of answering wrong Engagement and participation participation Collaborative allowed for immediate discussion amongst peers Engagement and participation participation The following is a compilation of the responses to Question 1 of the survey with noted Codes and Themes noting preference for Top Hat: Reason for preference of Top Hat Codes: Significant