Utah State University DigitalCommons@USU All Graduate Theses and Dissertations Graduate Studies 5-2016 A Mixed-Method Study of Mobile Devices and Student Self- Directed Learning and Achievement During a Middle School STEM Activity Scott Bartholomew Utah State University Follow this and additional works at: https://digitalcommons.usu.edu/etd Part of the Educational Assessment, Evaluation, and Research Commons, and the Junior High, Intermediate, Middle School Education and Teaching Commons Recommended Citation Bartholomew, Scott, "A Mixed-Method Study of Mobile Devices and Student Self-Directed Learning and Achievement During a Middle School STEM Activity" (2016). All Graduate Theses and Dissertations. 4748. https://digitalcommons.usu.edu/etd/4748 This Dissertation is brought to you for free and open access by the Graduate Studies at DigitalCommons@USU. It has been accepted for inclusion in All Graduate Theses and Dissertations by an authorized administrator of DigitalCommons@USU. For more information, please contact [email protected]. A MIXED-METHOD STUDY OF MOBILE DEVICES AND STUDENT SELF-DIRECTED LEARNING AND ACHIEVEMENT DURING A MIDDLE SCHOOL STEM ACTIVITY by Scott Bartholomew A dissertation submitted in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY in Education Approved: Edward Reeve, Ph.D. Raymond Veon, Ph.D. Major Professor Committee Member Gary Stewardson, Ph.D. Victor Lee, Ph.D. Committee Member Committee Member Wade Goodridge, Ph.D. Mark R. McLellan, Ph.D. Committee Member Vice President for Research and Dean of the School of Graduate Studies UTAH STATE UNIVERSITY Logan, Utah 2016 ii Copyright © Scott R. Bartholomew 2016 All Rights Reserved iii ABSTRACT A Mixed-Method Study of Mobile Devices and Student Self-Directed Learning and Achievement During a Middle School STEM Activity by Scott R. Bartholomew, Doctor of Philosophy Utah State University, 2016 Major Professor: Edward Reeve, Ph.D. Department: Applied Sciences, Technology & Education The increasingly ubiquitous nature of mobile devices among K-12 students has led many to argue for and against the inclusion of mobile devices in K-12 classrooms. Some have conjectured that access to mobile devices may enable student self-directed learning. This research used a mixed-method approach to explore the relationships between mobile devices and student achievement and self-directed learning during a Science, Technology, Engineering, & Mathematics (STEM) activity in a middle school Technology and Engineering Education classroom. In this study, 706 students from 18 classes worked in groups of 2-3 to complete an open-ended engineering design challenge. Students completed design portfolios and constructed prototypes. Classes were randomly divided with some receiving access to mobile devices during the study while others did not. Additionally, randomly assigned classes completed the design portfolio iv electronically while others completed the portfolio on paper. Final student portfolios and products were assessed using adaptive comparative judgment (ACJ). In ACJ, judges view two artifacts (portfolios or products) electronically and choose the better of the two. Repeating this process, a number of times produced a rank-order for the artifacts. The rank order for student portfolios and products represented student achievement. Statistical analyses of student access, portfolio type, student self-directed learning, and student achievement were conducted. Thirty student interviews and five teacher interviews were conducted and interviewees were asked questions regarding mobile devices, self-directed learning, and their experience during the study. Responses from the interviews were transcribed and coded using causation and thematic coding techniques. The resulting themes from the interviews helped clarify the quantitative findings. Findings from both the quantitative and qualitative analyses showed that student access to mobile devices was significantly correlated with higher scores on student design portfolios while student achievement on design products was independent of mobile device access. This suggests that mobile devices may improve student achievement in certain types of scenarios but not in others. Student self-directed learning was independent of mobile device access. Students and teachers both commented that mobile devices may be effective at increasing student self-directed learning or achievement but only through proper instruction and demonstration. (302 pages) v PUBLIC ABSTRACT A Mixed-Method Study of Mobile devices and Student Self-Directed Learning and Achievement During a Middle School STEM Activity Scott R. Bartholomew With the increasingly ubiquitous nature of mobile devices among K-12 students, many argue for and against the inclusion of these devices in K-12 classrooms. Arguments in favor cite instant access to information and collaboration with others as positive affordances made possible through mobile devices. Self-directed learning, a process where individuals take charge of their learning and decide what they will learn, how they learn it, and how they assess their learning, has been identified as an increasingly important trait for K-12 students. The relationship between mobile device access in K-12 education settings and student self-directed learning has not been explored. This research used a mixed-method approach to learn more about the impacts of mobile devices on student achievement and self-directed learning during a Science, Technology, Engineering, & Mathematics (STEM) activity in a K-12 technology and engineering education classroom. In this study, 706 middle school students from 18 classes worked in groups of 2-3 to complete an open-ended engineering design challenge. Students completed design portfolios and constructed prototypes (products) in response to a provided engineering design challenge. Participating classes were divided with some receiving ubiquitous access to mobile devices during the study while others did not. Additionally, randomly assigned classes completed the design portfolio electronically vi while others completed their portfolios on paper. Final student portfolios and products were assessed and assigned a rank order using an innovative method of assessment called adaptive comparative judgment (ACJ). In ACJ judges view two artifacts (portfolios or products) electronically via a computer and choose the better of the two. Repeating this process, a number of times produced a rank-order for the artifacts. The rank order for student portfolios and products was used to represent student achievement. Statistical analyses of student access, portfolio type, student self-directed learning, and student achievement were conducted. In addition to the quantitative approach, 30 student interviews and 5 teacher interviews were conducted by the researcher following qualitative methodology. Interviewees were asked a variety of questions regarding mobile devices, self-directed learning, open-ended engineering design challenges, and their experience during the study. Responses from the interviews were transcribed and coded using causation and thematic coding techniques. The resulting themes from the interviews were compared with the quantitative findings. Findings from both the quantitative and qualitative analyses showed that student access to mobile devices was significantly correlated with higher scores on student design portfolios while student achievement on design products was independent of mobile device access. These findings suggest that mobile devices may improve student achievement in certain types of scenarios but not in others. Over the course of the study, student self-directed learning was independent of mobile device access. Students and teachers both commented that mobile devices may be effective at increasing student self- directed learning or achievement but only through proper instruction and demonstrations. vii DEDICATION First and foremost, I am grateful to my Father in Heaven for the gifts, blessings, trials, and opportunities He continues to provide. My Savior, Jesus Christ, is my eternal friend, mentor, and the reason for everything good I enjoy. I want to thank my dear wife and eternal companion, Julie, for her support, love, encouragement, and help through the entire doctoral process. Julie, I will be forever grateful for the way you shape my life for the better. In more ways than I can include here, this was a joint effort—through this entire process you shared the responsibilities and opportunities with a loving smile and a grateful heart. You make me smile. I am grateful to my dear children who make coming home the best part of every day. Kylie and Dallin I am proud to be your father and I am so happy that you continue to make good choices and grow to become the people our Father in Heaven wants you to be. You inspire me to be a better man through your pure love and goodness. To my parents, Ron and Kris Bartholomew, your examples, love, and never- failing support have been crucial. Dad, thanks for showing me through your example how to be a man, a father, a husband, and an educator. Mom, thanks for showing me how to be a neighbor, a friend, and a disciple of Christ. Ma and Pa, thank you for raising, teaching, and loving the greatest blessing that has entered my life. Your examples of hard-work, love, kindness, and charity continue to shape and influence me. The importance you place on education is a cherished legacy for all who know you. viii ACKNOWLEDGMENTS I want to