Thematic Analysis of Survey Responses from Undergraduate Students

Thematic Analysis of Survey Responses From Undergraduate Students

© 2019 SAGE Publications, Ltd. All Rights Reserved. This PDF has been generated from SAGE Research Methods Datasets. SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 Thematic Analysis of Survey Responses From Undergraduate Students

Student Guide

Introduction Thematic analysis is a method of examining data to gain meaningful comprehension of participant perspectives. Thematic analysis identifies patterns within the data enabling the researcher a detailed understanding of the research data. It is a useful method for analyzing qualitative data as it looks for patterns from participant communication that is not constrained by any limitations to the responses. Thematic analysis is therefore a valuable method for examining the content of responses from data collected from open-ended survey questions, focus group discussions, or interviews.

In this example, a mixed-methods research study was implemented to gain student perspective on the use of technology and critical thinking development. A source of data collection was a student survey at the end of the study. Aligning with a mixed-methods approach, the survey consisted of open- and close-ended questions. The survey was used to elicit knowledge about the participants and acquire responses to specific questions directed at their attitudes, beliefs, behaviors, or emotions (Creswell, 2012; Mrug, 2010). Close-ended survey questions generally have a stem question and a set of answer alternatives to provide participants with a fixed number of responses from which they need to choose their answer from (Mrug, 2010). The open-ended questions provided a means of gaining student perspective using their own words and provided insight

Page 2 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 into their comprehension and thinking about critical thinking and technology use. Thematic analysis of the student responses to these open-ended questions was thus useful in gaining meaningful understanding of the students’ points-of-view.

Thematic Analysis of Open-Ended Survey Questions Surveys are used in research as a means of collecting information from a sample of participants through the responses they provide to the questions asked (Check & Schutt, 2012). Surveys encompass either close- or open-ended questions, or a combination of both. Close-ended questions using numerically rated items generate quantitative data; open-ended questions gather qualitative data; and a combination of both collecting mixed-methods data (Ponto, 2015).

Surveying with open-ended questions can offer the researcher rich, unconstrained participant responses to broad questions. Open-ended questions afford participants the opportunity to express their perspective using their own language, terms, and expressions. In contrast, close-ended questions have specific, pre- selected responses from which participants must choose from. Open-ended questions therefore provide participants the freedom to respond as they think appropriate, enabling them to determine their own answers and use their own words, potentially encouraging them to share more personal and genuine perspectives. Additionally, in using their own language, the participants could also demonstrate their understanding of critical thinking in the words and terms used.

While the large quantity and quality of the responses is beneficial for gaining rich understanding of the topic, the amount of data can be overwhelming. Limitations to using open-ended questions may occur in that participants may misinterpret the intent of questions if not sufficiently clear. In requiring individualized responses, greater time and thinking by participants would be required when responding (Baillou, 2008).

Page 3 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 Thematic analysis has been described as a useful method of analyzing qualitative data for researchers, facilitating organization of data, and capturing valuable information (Braun & Clarke, 2006). Braun and Clarke (2006) note its flexibility in analyzing rich data and offer a guide in using this method of analysis. Thematic analysis enables the researcher to organize and analyze responses and interpret them to determine common perspectives among participants (Creswell, 2012). As with other methods of qualitative research, it is important that the researcher be aware and acknowledge personal views and biases on the topic being explored (Sutton & Austin, 2015). It is beneficial for the researcher to explicitly share their own personal stance to comprehensively position the data, analysis, and findings, and provide context and understanding to the readers.

Data Exemplar: Student Perspective of Technology Use and Critical Thinking Development The data used for this example were gathered from a survey implemented with students following a university term where technology use and explicit critical thinking instruction were integrated into the course. This was one form of data collection in the study to gather student perspective on technology use and critical thinking development. Additional data collection methods and analysis included descriptive statistics with the close-ended questions in the survey and content analysis of online discussion postings.

All 127 enrolled students were provided with information on the study and invited to participate as desired. The students were undergraduate learners in a beginning healthcare professions course. The technologies implemented were an in-class technology response system, Top Hat, and an online discussion posting forum on the learning management system, Desire2Learn. Of the 127 registered students, 43 students (34%) completed this end-of-term survey.

Thematic Analysis of the Student Responses to Open-Ended Survey Questions For this example, thematic analysis encompassed the phases as detailed by Braun and Clarke (2006): (1) gaining familiarity with your data; (2) generating initial codes or labels; (3) searching for themes or main ideas; (4) reviewing themes or main ideas; (5) defining and naming themes or main ideas; and (6) producing the report.

The first question will be used as an example. It asked students to identify their preference of technology used in class and to provide a reason: “Which learning technology, Top Hat or Online Discussions, was more beneficial for your engagement in developing critical thinking? And, why?”

With the first step, participant responses to the open-ended survey question were read and re-read to gain understanding and awareness of the data, and to gain familiarity with the data.

Next, data that were meaningful to the study were noted, recurring messages were identified, and codes generated in the form of phrases to represent significant data. For this study, the coding was implemented by hand. The codes chosen aimed to identify the elements the students noted as important to them in their responses. The potential codes were important phrases highlighting participant ideas such as the following from responses to Question 1:

Participant responses (indicating preference for classroom response system, Top Codes highlighting Hat) participant ideas

because it helped me learn how to prioritize important information and choose certain Decision-making in choosing answers over others answers

was more beneficial because it prepared for exams Exam preparation

because I got to see the type & level of questions that I would see on my exams and it also Exam questions was a great review for exams

The codes were next collated to determine an overarching idea under which to organize the important phrases identified, which Braun and Clarke (2006) describe as “searching for themes” (p. 89). The themes constituting these main ideas were reviewed to ensure they each encompassed clear, accurate phrases of importance, clearly explaining the focus of the ideas. The themes were labelled and reviewed to ensure that they were appropriate and comprehensive in describing the data.

From the example, two themes identified by participants preferring the classroom response system, Top Hat, were Multiple Choice as Preparation for Exams and Engagement and Participation:

Participant responses Codes Themes

because it was more similar to the NCLEX (National Council Multiple choice as preparation Exam preparation Licensure Examination) for exams

Multiple choice as preparation it helped me study for my exam Exam preparation for exams

because it was done in class time and it covered what we just In-class learning Engagement and participation learned

Anonymity leading to I wasn’t afraid of answering wrong Engagement and participation participation

Collaborative allowed for immediate discussion amongst peers Engagement and participation participation

The following is a compilation of the responses to Question 1 of the survey with noted Codes and Themes noting preference for Top Hat:

Reason for preference of Top Hat Codes: Significant ideas Themes: Main ideas

Anonymity leading to Engagement and I wasn’t afraid of answering wrong participation participation

because it helped me learn how to prioritize important information Decision-making in Multiple choice as and choose certain answers over others choosing answers preparation for exams

Engagement and I really enjoy using Top Hat Participation in learning participation

because I got to see the type & level of questions that I would see Multiple choice as Exam questions on my exams and it also was a great review for exams preparation for exams

Engagement; multiple because I was more motivated to answer questions than write Choosing answers choice as preparation responses for exams

Engagement and less pressure to answer these questions Safety in answering participation

Engagement and it created more discussion Participation in learning participation

I really liked using Top Hat. It was great way to test my knowledge. Engagement and Because it was graded on participation, it felt okay to make mistakes Safety in answering participation when learning

Engagement and allowed for immediate discussion amongst peers Collaborative participation participation

Multiple choice as simulated exam questions Exam questions preparation for exams

Exam question practice; Multiple choice as multiple choice was practice for the exams. You could discuss it with collaborative participation preparation for exam; classmates next to you in learning engagement

Engagement and for the immediate feedback and classroom participation Participation in learning participation

Multiple choice as better prepares for exams Exam preparation preparation for exams

because it helped us prepare for tests. It was multiple choice which Multiple choice as Exam preparation was lot more useful than discussion Qs preparation for exams

because it was done in class time and it covered what we just Engagement and In-class learning learned participation

Multiple choice as it gave examples of what tests would be like Exam preparation preparation for exams

Engagement and was more helpful Engaged in learning participation

Multiple choice as I like the multiple choice Picking answer preparation for exams

Multiple choice as because it was more similar to the NCLEX Exam preparation preparation for exams

Multiple choice as it helped me study for my exam Exam preparation preparation for exams

Anonymity leading to Engagement and Anonymity was more comforting participation participation

Multiple choice as was more beneficial because it prepared for exams Exam preparation preparation for exams

For the study, the responses to each of the questions asked in the survey were analyzed using the six steps proposed by Braun and Clarke (2006) and reviewed by an additional faculty member. Themes were identified from the responses to all the questions asked and compiled for reporting as the last step in the analysis. The main themes were used to generate a visual depiction to report the data attained from the participants presenting their perspectives on critical thinking development and the use of technology. The following is an example of the visual generated from the thematic analysis of all the survey questions to gain understanding of the “Student perspective on developing critical thinking and technology use.”

Reflective Questions

1. Using the student responses to Question 1 noting preference for the Online Discussions to facilitate critical thinking, in Exemplar A, what Themes might you generate from the Codes identified? 2. Using the responses to Question 2 of the survey, in Exemplar B, what potential Codes and Themes might you consider? 3. Exemplar C presents the responses to Question 3 of the survey, asking about the questions used. What potential themes do you note identifying types of questions which they consider helpful to

Page 9 of 10 Thematic Analysis of Survey Responses From Undergraduate Students SAGE SAGE Research Methods Datasets Part 2019 SAGE Publications, Ltd. All Rights Reserved. 2 development of critical thinking?

References Baillou, J. (2008). Open-ended question. In P. J. Lavrakas (Ed.), Encyclopedia of survey research methods (pp. 548–550). Thousand Oaks, CA: SAGE.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. doi:http://dx.doi.org/10.1191/ 1478088706qp063oa

Check, J., & Schutt, R. K. (2012). Survey research. In J. Check & R. K. Schutt. (Eds.), Research methods in education (pp. 159–185). Thousand Oaks, CA: SAGE.

Creswell, J. W. (2012). Educational research: Planning, conducting, and evaluation quantitative and qualitative research (4th ed.). Boston, MA: Pearson Education.

Mrug, S. (2010). Survey. In N. J. Salkind (Ed.), Encyclopedia of research design (pp. 1473–1477). Thousand Oaks, CA: SAGE

Ponto, J. (2015). Understanding and evaluating survey research. Journal of the Advanced Practitioner in Oncology, 6(2), 168–171. PMCID: PMC4601897. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4601897/

Sutton, J., & Austin, Z. (2015). Qualitative research: Data collection, analysis, and management. The Canadian Journal of Hospital Pharmacy, 68(3), 226–231. PMC4485510. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/ PMC4485510/

Page 10 of 10 Thematic Analysis of Survey Responses From Undergraduate Students