University of Southampton Research Repository
Copyright © and Moral Rights for this thesis and, where applicable, any accompanying data are retained by the author and/or other copyright owners. A copy can be downloaded for personal non-commercial research or study, without prior permission or charge. This thesis and the accompanying data cannot be reproduced or quoted extensively from without first obtaining permission in writing from the copyright holder/s. The content of the thesis and accompanying research data (where applicable) must not be changed in any way or sold commercially in any format or medium without the formal permission of the copyright holder/s.
When referring to this thesis and any accompanying data, full bibliographic details must be given, e.g.
Thesis: M Zinopoulou (2019) " A Framework for improving Engagement and the Learning Experience through Emerging Technologies ", University of Southampton, The Faculty of Engineering and Physical Sciences. School of Electronics and Computer Science (ECS), PhD Thesis, 119.
Data: M Zinopoulou (2019) "A Framework for improving Engagement and the Learning Experience through Emerging Technologies" The Faculty of Engineering and Physical Sciences
School of Electronics and Computer Science (ECS)
A Framework for improving Engagement and the Learning Experience through Emerging Technologies
by
Maria Zinopoulou
Supervisors: Dr. Gary Wills and Dr. Ashok Ranchhod
Thesis submitted for the degree of Doctor of Philosophy November 2019 University of Southampton Abstract
Advancements in Information Technology and Communication have brought about a new connectivity between the digital world and the real world.
Emerging Technologies such as Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR) and their combination as Extended Reality (XR), Artificial Intelligence (AI), the Internet of Things (IoT) and Blockchain Technology are changing the way we view our world and have already begun to impact many aspects of daily life. There are now connections between millions of devices from mobile phones to wearables, through wireless networks, creating smart environments and smart cities. It is predicted that over 50 billion devices will be connected by 2020. Virtual design and digital technologies enhance these smart environments to create new immersive experiences.
These developments have resulted in many industries seeking new ways to adapt and evolve to take advantage of the opportunities presented by these emerging technologies. Innovators, researchers, technologists, businesses and manufacturers are leading the way to developing and implementing emerging technologies to create smarter, more efficient homes and cities, connecting individuals, communities and operations to allow for optimisation of resources, modification and personalisation.
This thesis focuses on the opportunities presented by these technologies for developing and designing immersive smart learning environments for Higher Education and puts forward a framework for further research into how these technologies can improve engagement and enrich the learning experience. An experiment was conducted using two simulators that reinforce learning concepts. A computer based simulator and a virtual reality simulator were used. The results of the research which was conducted with Higher Education students from two universities in the United Kingdom showed an improvement in engagement levels and in the learning experience when students used an immersive smart learning environment. Table of Contents
Table of Contents
Table of Contents ...... i List of Tables ...... v Table of Figures ...... vi Research Thesis: Declaration of Authorship ...... vii Acknowledgements ...... ix Dedication...... xi Abbreviations ...... xii Chapter 1. An Overview for Smart Education ...... 13
1.1 Introduction ...... 13 1.2 Positionality ...... 17 1.3 The importance of Emerging Technology in Education ...... 19 1.4 Opportunities in Smart Education ...... 20 1.5 Structure of the Thesis ...... 22
Chapter 2. Literature Review ...... 23
2.1 Introduction ...... 23 2.2 Learning Experience Theories and Concepts ...... 24
2.2.1 Experiential Learning ...... 26 2.2.2 Constructivism – Discovery Learning ...... 26 2.2.3 Multiple Intelligences Theory ...... 27 2.2.4 Paradigm - Design-Based Research ...... 28
2.3 Technology and the Learning Experience ...... 28 2.4 Engagement and the Learning Experience ...... 31 2.5 Motivation as part of the Learning Experience ...... 36 2.6 Smart Learning Environments ...... 38 2.7 Virtual Simulators and Gaming Applications in Education ...... 43 2.8 Virtual Reality Learning Environments and Scenarios ...... 45 2.9 IoT, Augmented and Virtual Learning in Education ...... 48 2.10 Gap in the current Research and Literature ...... 50 2.11 Summary ...... 51
i Table of Figures
Chapter 3. Proposed Framework ...... 52
3.1 The Research Question ...... 52 3.2 Hypotheses ...... 53 3.3 Towards a Smart Learning Experience ...... 54 3.4 A Smart Learning Environment Framework ...... 56 3.5 Breakdown of the Framework ...... 57 3.6 Implementing a Smart Learning Environment ...... 58 3.7 Example of an Immersive Smart Virtual Learning Environment ...... 60 3.8 Implementing a Smart Learning Environment Framework ...... 61 3.9 Summary ...... 63
Chapter 4. Methodology and Design ...... 64
4.1 Research Purpose ...... 64 4.2 Evaluation Methodology ...... 66 4.3 Research Design and Rational ...... 66
4.3.1 Reliability, Validity and Expert reviews in Interviews and Surveys ...... 69 4.3.2 Participant Profiles ...... 73 4.3.3 Expert Reviews ...... 74 4.3.4 Student Groups ...... 74 4.3.5 Documentary Sources ...... 75
4.4 Analysis of Findings ...... 76 4.5 Evaluation and Engagement Metrics ...... 77 4.6 The Virtual Learning Experience Experiment ...... 78 4.7 Summary ...... 80
Chapter 5. Analysis of Results ...... 81
5.1 Overview...... 81 5.2 Qualitative Analysis Observations ...... 86 5.3 Analysis of Qualitative Research ...... 87 5.4 Summary of Sentiment Analysis across Engagement Metrics ...... 88 5.5 Expert Review Analysis ...... 90 5.6 Summary ...... 92
ii Table of Contents
Chapter 6. Conclusions and Future work ...... 93
6.1 Academic Contribution...... 93 6.2 Practical Contribution ...... 94 6.3 Educational Opportunities ...... 95 6.4 Limitations of research ...... 96 6.5 Future Work ...... 97 6.6 Concluding remarks ...... 99
Bibliography ...... 100 Appendix A Research Flowchart ...... 112 Appendix B VR Experiment Photos...... 113 Appendix C Raw Data PC Simulator ...... 115 Appendix D Raw Data VR Simulator ...... 116 Appendix E Calculations of p and Z- Score / U-Score ...... 117 Appendix F Student Questionnaire ...... 118
iii List of Tables
List of Tables
Table 2.1 Dimensions of Engagement ...... 34
Table 2.2 Online Content Delivery ...... 39
Table 4.1 Neuman’s Reliability Factors ...... 70
Table 4.2 Evaluation and Engagement Metrics ...... 77
Table 5.1 Engagement Results with z-scores and p values ...... 83
Table 5.2 Learning Environment Results with z-scores and p values ...... 83
Table 5.3 and Table 5.4 Box Visual of LE Median PC - LE Median VR Plots ...... 84
Table 5.5 Binomial Test for level counts ...... 85
Table 5.6 Student response examples ...... 87
Table 5.7 Student sentiment analysis against metrics ...... 89
v Table of Figures
Table of Figures
Figure 1.1 Head Mounted Display Figure 1.2 Sensorama ...... 15
Figure 1.3 Internet of Things, Remote Healthcare ...... 17
Figure 1.4 Hype Cycle for Emerging Technologies ...... 21
Figure 2.1 The Lewinian Experiential Learning Model ...... 25
Figure 2.2 Audio visual Methods in Teaching (1969) ...... 29
Figure 2.3 Motivation for Engagement in Student Governance ...... 36
Figure 2.4 Three new functions of student engagement ...... 37
Figure 2.5 The 6A Concept for Internet of Things ...... 38
Figure 2.6 Growth of MOOCs ...... 40
Figure 2.7 Internet of Everything ...... 42
Figure 2.8 Virtual Chemistry Lab ...... 46
Figure 2.9 Microsoft HoloLens in Education ...... 49
Figure 3.1 Towards an Enhanced Learning Experience ...... 54
Figure 3.2 and Figure 3.3 The Smart Learning Environment Framework...... 56
Figure 3.4 Connecting to a VRLE ...... 59
Figure 3.5 Diagram of a proposed Smart Lecture Theatre ...... 59
Figure 4.1 Adapted from Venkitachal Steps in questionnaire designing ...... 71
Figure 4.2 Jakob Nielsen graph ...... 73
Figure 4.3 Data gathering triangulation ...... 76
Figure 4.4 Images of VR Simulation platform and legend ...... 79
vi Research Thesis: Declaration of Authorship
Research Thesis: Declaration of Authorship
Print name: Maria Zinopoulou
Title of thesis: A Framework for improving Engagement and the Learning Experience through Emerging Technologies
I declare that this thesis and the work presented in it are my own and has been generated by me as the result of my own original research.
I confirm that:
1. This work was done wholly or mainly while in candidature for a research degree at this University; 2. Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated; 3. Where I have consulted the published work of others, this is always clearly attributed; 4. Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work; 5. I have acknowledged all main sources of help; 6. Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself; 7. None of this work has been published before submission
Signature: Date:
vii Acknowledgements Acknowledgements
I would first and foremost like to express my deepest gratitude and sincere thanks to my supervisors, Dr. Gary Wills and Dr. Ashok Ranchhod for their faith in me and their continuous encouragement and support. You believed in my abilities to undertake this challenge and you were there to guide me throughout. Losing Professor Ashok in the final year was heartbreaking and at times I considered giving up, but his teachings and spirit were strong and it kept me determined to fulfil this commitment and the trust he showed in me. With the unwavering support, insight and gentle approach from Professor Gary, I managed to get there. Thank you to my examiners, whose insight and valuable suggestions shaped my thesis. I am very grateful for your input and for the positive and kind feedback on my work.
To my son, George, you were an inspiration, your struggles with dyslexia kept me determined to seek new ways to educate. As an intelligent young man, you continue to impress me every day with your beautiful, inquiring mind. Without your support and love, I would not have been able to undertake a PhD while raising a son on my own. Thank you for making it easy for me. You are my pride and joy and I hope this research contributes towards building a new education system for amazing young people like you!
To my partner, Kristian, thank you for seeing me through this journey. Your tech insights and programming skills are amazing and you were always there to help. You were patient and encouraging. You are the anchor in my life and it is your love and intellect that power my spirit and mind every day.
My heartfelt thanks to my loving family who stood by me, cared for me, and supported me. Thank you for your encouragement and unconditional love, and for always believing in me. I am here because of all of you! Thanks for the many coffees made with love, Joyki and the entertainment, Andreas.
My thanks and gratitude to the University of Southampton for giving me the opportunity to undertake this research and supporting me throughout in every way. I could not have chosen a better place to do research on Emerging Tech than at ECS.
To my dear friends and colleagues, thank you for your support and for working with me through this. I appreciate your patience when I was not available to see you and your understanding over the years as I completed my research. You were there for me to bounce ideas and have some late night discussions. To the experts who shared their knowledge and insights freely and with enthusiasm, I thank you.
Finally to my bright and brilliant students, thank you! This research was driven forward by a determination to seek to improve the way students learn and you stepped up whenever asked to help with the experiments and discussions that followed. It was your energy and enthusiasm that kept me working harder. It is my hope that our efforts help education to continues to evolve and improve for future generations. ix To the light that guides us, thank you for keeping my path lit. Dedication
Dedication
For my father
Georgios Zinopoulos
who taught me the value of love and education
“Educating the mind without educating the heart, is no education at all.”
Aristotle
xi Table of Figures
Abbreviations
AI – Artificial Intelligence
AR – Augmented Reality
AVR – Automated Voice Recognition (AVR)
EX – Extended Reality
ICT – Information Communication Technology
MX – Mixed Reality
IoT – Internet of Things
IoE – Internet of Everything
IOTA – Internet of Things Analytics
MOOC – Massive Open Online Courses
OS – Operating System
RFID – Radio Frequency Identification
TUI – Tangible User Interface
VR – Virtual Reality
VLE – Virtual Learning Environment
VRLE – Virtual Reality Learning Environment
xii Chapter 1
Chapter 1. An Overview for Smart Education
1.1 Introduction
Technology and Education have stood beside each other as partners on the road to development yet when it comes to implementation of some of emerging technologies to create new learning environments, the two seem to be on opposite sides of that road. The opportunities to link smart and emerging technologies and to use them to facilitate new immersive learning environments is there, however, the implementation of these environments is a daunting task for many institutions.
Smart technologies working together to transmit intelligence through wireless networks was first predicted in 1926 by Nikola Tesla in his interview with Colliers Magazine (Colliers, 1926), where he envisioned a wirelessly interconnected world. He imagined that when the Internet was fully applied, the world would be ‘converted into a huge brain’ with all things being part of a whole. The initial steps towards this concept were taken many years later. The Internet protocol suite (TCP/ IP) was proposed by Robert Elliot Kahn and Vinton Cerf (1974) in the 1970s with the term “Internet” coined in 1974 in a paper by Cerf on the subject. (Cerf et al., 1974). Development followed in the 1980s with the establishment of supercomputers, interconnected at several national universities which included creating network access from educational and research organisations, with the National Science Foundation Network (NSFNET) Project, funded by the NSF (NDF, 1989). With the rise of commercial Internet Service Providers (ISP) by the 1990s more and more websites were populating the online arena.
In 1989 Tim Berners Lee (CERN, 1989), a British scientist, invented the world wide web while researching at CERN, where the Norwegian web pioneer, Håkon Lie, later proposed the concept of Cascading Style Sheets (CSS) (Lie, 1998).
Meanwhile, radio frequency identification system (RFID) technology, which used radio waves or electromagnetic fields to track information stored in tags that were attached to objects (Roberts, 2006), was also making its mark on the world. This technology was utilised in the manufacturing industry to monitor progress across an assembly line. In 1980 the first uncompensated pressure sensor was manufactured and later that decade
13 Chapter 1
Motorola Inc.1 developed the first surface micromachined inertial sensors (Yazdi, Ayazi, and Najafi, 1998) that were used by the automobile industry in airbags (Richardson et al., 2005). Some experimental connected devices at the time included a toaster and a coffee machine.
From the 50s through to modern-day the idea of connected homes and cities through smart technology became a vision for many engineers and technologists. Although the concept referred to by Tesla existed for many years, it was Kevin Ashton, another British technologist who proposed the term Internet of Things (IoT) in 1999 (Wyld, 2005) to link RIDF technology to the Internet. He described a system where the Internet connects to the physical world via ubiquitous sensors. He co-founded the first concept of IoT known as the Auto-ID Center at Massachusetts Institute of Technology (MIT) (Auto-ID labs, 2015). The rise of the Internet brought in more experimental devices. Alongside the evolution of the Internet and the IoT, other digital technologies such as Virtual and Augmented Reality were also emerging. In 1998 Mark Weiser connected a Water Fountain outside his office and (WearTech, 2008) using the water to reflect price trends of the Stock Market in real-time. In the same year the inTouch Project at was presented at MIT Media Laboratory (Brave et al, 1998) as a “system for haptic interpersonal communication based on the concept of Synchronized Distributed Physical Objects” WearCam created in by Steve Mann (1994) was one of the first cameras to appear on the world wide web as an experiment in connectivity where and was inspired by the Trojan Room Coffee Pot Camera (1991) in the computer laboratory of the University of Cambridge which inspired the world’s first webcam (Cam, 1991). Today Mann’s project has evolved to wearable computing and IoT and Augmented Reality (AR) and includes research, experiments, and mentoring. The first virtual reality example is seen in 1828 in the form of the stereoscope invented by Sir Charles Wheatstone (Bowers,2001) and allowed people to view 3D images.
1 Semiconductor Products Sector of Motorola Inc. became Freescale Semiconductor Inc. in 2004 (Eecatalog.com)
14 Chapter 1
Edwin Link, an American aviation pioneer, inventor and entrepreneur created the “Link Trainer” (Link Foundation) which became the world’s first flight simulator allowing pilots to train in a small blue box model of a plane that offered an accurate representation of how it really feels to fly and control a plane without actually doing so. This invention was followed by the humble View-Master (1939), whose simple stereoscopic design allowed users to see two identical images as a unified 3D picture.
In 1952 Morton Heilig (1952) invented the first VR-like machine that included immersive multimodal technology with a colour display, a sound system, scent emitters, and fans. Eight years later, he presented the first head-mounted display (FIG 1.1) (Heilig, 1960) and that was followed in 1962 by the Sensorama (Fig 1.2).
Figure 1.1 Head Mounted Display (Heilig, 1960) Figure 1.2 Sensorama (1962)
The first Artificial Reality Lab, called Videoplace, was built in 1974 by Myron Krueger and allowed for communication in artificial reality. The term Virtual Reality (VR) was coined in 1987 by Jaron Lanier and the term Augmented Reality (AR) followed in 1990 from Tom Caudell while working at Boeing where he presented an alternative to the diagrams that field workers were using Technology business giants like IBM and Sony were also looking at objects in relation to the internet and connectivity followed by Apple in 1998 with the first commercial iMac and who then went on to develop the iPod (2001) and the iPhone (2007).
Most industries have embraced these technologies, from healthcare to the automobile industry and electronics to retail. Since 1999 when NASA used AR to navigate its X-38
15 Chapter 1
spacecraft up to when Google launched its wearable Google Glass using EyeTap (IEEE, 2012) technology and through the last two decades, the evolution of VR and AR continues to penetrate technology-driven industries.
In 1999 movies like the Matrix (Wachowski, 1999) depicted a computer-generated/ virtual world in which humans interact while their body lies still in a dormant state. The idea of virtual worlds was becoming an inspiration for many inventors and developers.
One of the most prominent industries being gaming where the launch of the Oculus VR in 2012 through Palmer Luckey’s Kickstarter (2015) project led to the Oculus Rift headset and HTV VIVE in 2015. Both Google and Apple launched augmented reality frameworks ARCore and ARKit, respectively by 2018, allowing developers to build realistic AR experiences that blend the digital and real world.
IoT technology, in turn, allowed for smarter objects. As Mark Weiser noted in 1998, ubiquitous computing was the opposite of VR in that virtual reality places the user inside a digital world whereas ubiquitous computing brings the digital element into the physical world. The connected car has onboard diagnostics, safety sensors, and seamless connectivity with mobile devices, with research suggesting that by 2025, all cars will be connected, and by 2035, 75% of cars on the road will be autonomous (GSMA, 2013). We are building smarter cities and smarter homes. Singapore commissioned a smart city with over 1,000 sensors to monitor street lights and traffic while in 2015, Helsinki launched a fully operating Mobility service that allows users to text and buy a mobility ticket that plans their journey through the city using sensors to find the best route through private and public means of transport. More examples exist in healthcare with Apple HealthKit developed to link with their watch and iPhone to monitor and report on activity and heart rate. In one example (Fig 1.3) presented by Peter Ianace (2015), sensors attached to the body are used to give feedback to patients while reporting back to the medical professionals to ensure consistent and real-time intervention as required.
16 Chapter 1
Figure 1.3 Internet of Things, Remote Healthcare (Ianace, 2015)
In 2011 technology and mobile giants IBM and Ericsson collaborate with Cisco to produce educational and marketing initiatives on the topic of IoT following the public launch of IPV6. These examples indicate the reach of emerging technologies in various areas of life. Opportunities continue to grow and transform how people relate to each other, themselves, and the way they live and learn through new digital experiences. The unique factor that distinguishes these new technologies is that they are able to work in unison with one another to create smart objects that function in smart spaces.
1.2 Positionality
The scope and interest in this thesis stemmed from years of working in education and raising a son with severe dyslexia where his struggles became apparent from a young age. My graduate degree was centred around using gamification to learn as an avenue to integrate technology and learning to help young people with learning difficulties. As a Director of Education for the Communications Advertising and Marketing (CAM) Foundation, the digital impact on the skills and knowledge students would need in a digital technology driven environment became more apparent and I was part of the team that led the development of new qualifications to set about preparing students for future skills they would need. In the past 6 years I work as a lecturer at Imperial College Business School on the MSc Strategic Marketing program, which is rated the second best program in the world for this field. (World University Rankings, 2019) Having taught Marketing Analytics, Research and Evaluation, and more recently the modules of
17 Chapter 1
Marketing and Technology and Emerging Technologies in Marketing it was evident that emerging technologies such as Virtual Reality (VR) and Augmented Reality (AR), Artificial Intelligence (AI) and Internet of Things (IoT) are evolving. Teaching the impact of that evolution on society and how the students can use emerging technology strategies to reach their audience in future was welcomed as industry evidence showcases that this will be critical in tomorrow’s world. For the past 8 years I have also been a keynote speaker at conferences and events focused around new technology and education including MarTech and Festival of Innovation where I presented on VR in Education. From initial research and discussions with students and colleagues at the universities it was clear that education had not adopted new technologies as quickly as some other industries had. Although the universities had a good level of technology in the form of an online Hub and online materials, computer simulations and interactive online courses and computer labs, and even a 3D/ VR room, the step towards using the above mentioned technologies together to enhance the learning experience is still in its infant steps. The main area which I found was lacking was the currency and relevance of some of the content to what was happening in the real world. As an example it would have been more useful for students to use real data from external live feed channels in order to determine behaviour changes when a campaign is launched rather than using pre-exiting data that was perhaps relevant at the time but now seems out of context. Furthermore, I was interested in a method by which the students were to be able to see the results of their marketing efforts in real time. The visualisation of the product and more importantly the impact that their actions had on the consumer would encourage more engagement and enthusiasm to explore the concepts and learn at their own pace and personal style as they could use a virtual platform to interact with and construct their own learning experience. This led to the preparation of a VR platform for marketing students that could give them an initial taste of what that would be like. It is important to highlight my interest in using all emerging technologies together to enhance the learning experience and even though for the purposes of this thesis the extended elements of IoT Analytics and live feedback responses were not yet available it
18 Chapter 1 is my hope that they will form the platform for future studies toward creating smart learning environments.
1.3 The importance of Emerging Technology in Education
Virtual and Augmented Reality and IoT, referred to as smart technologies or technology accelerators, have opened significant opportunities in Education. For years, education has followed a ‘one shoe fits all’ approach (Kohn, 2001) with limited opportunities for individual customisation of how new information is presented, how student engagement is facilitated, and how students learn. In recent years, efforts to integrate technology into teaching environments were welcomed by schools through interactive whiteboards and growing ICT implementation (Starkman, 2006). However, learning materials continued targeting all students with little personalisation at the offset or during the lessons as there was little or no way of monitoring and reporting on the student performance in ‘live-mode’ and elements such as enthusiasm, engagement, and levels of interaction were still dependent only on the observation by the instructor. This type of monitoring was not only challenging to do in big classroom situations, but also time- consuming for the instructor or lecturer. In recent years an increasing number of schools have allowed for the use of iPads in class to encourage learning through applications. Apple published the results of data that was self-reported by institutions on the progress made by students using iPads in Education (Computers and Education, 2014, Research published by Apple, 2017) and indicated on a worldwide scale an increase in many elements of education from increase in reading and maths and science test scores in elementary schools to increase in attendance and a decrease in dropout rate. VUC Syd in Denmark, who introduced the one-to-one iPad program in order to build confidence and increase engagement with their students, reported a 139% increase in students seeking higher education after graduation.
19 Chapter 1
The development of simulators to reinforce learning concepts and create more interactive learning platforms has also given solutions towards more experiential learning. Examples of such online learning environments is gaming simulations such as Markstrat (Stratx Simulations), which allow scenario and simulation-based learning. These platforms have opened the doors for exploring new avenues of integrating digital environments into the classroom that introduce real scenarios through simulations and allow students to build up on their knowledge and skillset in a safe online environment.
1.4 Opportunities in Smart Education
Considering virtual and augmented reality as an opportunity to improve engagement levels in higher education (Pan et al. 2006) can prove a viable avenue for educators to consider redesigning some of their learning materials and applying formative assessment within their lesson plans and include in-process evaluation to the classroom (Edglossary.org, 2014) This is proposed to be in conjunction with other teaching methods and traditional training systems and is meant to enhance the learning process and add to it rather than substitute it.
The conjecture of the thesis is that smart emerging technologies can improve engagement and the learning experience in immersive learning environments.
Technologies such as virtual reality, artificial intelligence, augmented reality, and the Internet of Things are emerging as part of the 4th industrial revolution. Gartner (2018) conduct industry research into emerging technologies which they map across the Gartner Hype cycle (Gartner, 2018) (Panetta, 2018) and showcases evidence of the rise of these technologies on their path to the plateau of production or the point at which they reach the mass population. Ravipati (2017) suggests that the surge for VR is set to increase by 85% by 2020 as it becomes less expensive and more accessible. Jensen (2017) argued that VR platforms could be used as a collaboration tool in education by allowing better communication.
20 Chapter 1
Figure 1.4 Hype Cycle for Emerging Technologies (Gartner, 2018)
The differentiating factor in this revolution is that multiple emerging technologies will be working together to create smart objects and spaces. Individuals learn to adopt and adapt to the technologies to become a connected part of a smart system.
Immersive virtual environments are defined as simulations in virtual spaces that facilitate experiences and integrate features of technologies that fill the field of vision, placing them inside a virtual platform, allowing the user to interact in a natural way in the virtual space. (Loomis and Blascovich, 1999)
When used as an educational tool, it has the potential to increase engagement, and interaction with the content can build on several theories that are outlined in the next chapter.
21 Chapter 1
1.5 Structure of the Thesis
This chapter introduces the concepts of emerging technology such as Augmented and Virtual Chapter 1 Reality and the Internet of Things and the development of the internet which accommodates them. It also outlines the key elements which make these technologies important for the future of education. Chapter 1 includes the positionality of the researcher and concludes with an emphasis on the opportunities for education that are available through emerging technologies.
This chapter is a comprehensive literature review into the components of smart technologies, considering research and studies undertaken in the areas of IoT AR and VR and their application in education. This chapter also looks at the background of how we learn and the pedagogical Chapter 2 elements linked to engagement in learning. It then connects virtual technology and education by looking at the underpinning research and current developments in the respective areas. Chapter 2 concludes with the identification of a gap in current research.
This chapter presents a proposal for a framework for using emerging technologies to enhance engagement and improve the learning experience. It outlines an example of how this can be implemented and shows an example of a virtual learning environment that encompasses smart Chapter 3 technologies. This chapter outlines the research problem and links it to the research proposal. It presents the conjecture of the thesis and the research questions that result from the paper and presents the hypotheses that will drive the research paper. Chapter 3 concludes with a summary of the importance considering new technologies for the future of Education.
This chapter presents the methodology and design that is proposed for this paper. It considers the relevant methods and theories that are linked to the proposal and gives an explanation of why they are applicable to the design and experiment of the research. The chapter then concludes Chapter 4 with a table that links the research methods and education theories with the metrics that will measure the experiments. Chapter 4 concludes with a summary linking the virtual platform to the various schools of thought.
This chapter presents the findings of the both the qualitative and quantitative research, and Chapter 5 presents the results and an analysis of those findings while considering the research question and the hypothesis. The chapter ends with a summary of the results of the analysis
This chapter concludes with the outline of the contribution of this thesis, the limitation of the Chapter 6 research and future work. The chapter ends with concluding remarks.
22 Chapter 2
Chapter 2. Literature Review
2.1 Introduction
Education has developed in various ways, although it has not fully evolved in many years. We can consider the basic elements of learning on the simple questions of what, how, how much, where and who? What are the learning outcomes, or what do we want to teach, how will we teach it, how will we measure what they have learnt, who will teach it, where, and when?
Based on these questions, education systems worldwide outline the syllabus for millions of learners from elementary to postgraduate study. The evolution of technology and the introduction of the Internet brought about new ways to assist the learning process. These developments have impacted schools and training institutions bringing about new methods and techniques that utilised the internet and connect students and provide new resources and facilities through online portals. Massive Open Online Courses (MOOC) are an example of the opportunity presented by the internet to offer free university-level education to thousands of people via online Virtual Learning Environments (VLE). (Yuan et al. 2013) The success of these courses signifies the need for education to be more flexible and available for all. (Palloff and Pratt, 2013) Virtual Reality is now introducing another dimension to learning through immersive online environments that can transport the learner to new virtual worlds where interaction is almost limitless.
These new opportunities to create even more interactive learning environments that can utilise student activity analytics through smart technologies like IoT used to gather information that can personalise content and feed real data into these virtual worlds that are dynamic to enhance the learning materials and encourage engagement while improving the student experience. The next section discusses the various learning theories and how they fit into learning environments.
23 Chapter 2
2.2 Learning Experience Theories and Concepts
The key to education is learning how to learn. Adapting and developing specific techniques to help interpret concepts and understand their meanings and applications. Humans develop methods that help them retain that knowledge and the ability to relate it to the world around them as they grow up. The academic elements begin in the first school environments and continue throughout life. For active learning to occur within a learning environment, certain elements are prime to ensure success. Laurillard (2002) suggests in his Conversational Framework, that in order for effective learning to take place an experiential environment is required to allow the learner to adapt their behaviour within the learning environment, based on feedback both from the instructor and the environment.
This concept is linked to constructivist learning theories (Bedner, 1992), which support that individuals “construct” knowledge by interacting with the environment around them, with new insight building on, and adapting prior knowledge concepts. (Bodner, 1986). Learning can be meaningful within these environments where rules and boundaries are set and where the participant feels safe to explore and inquire. This is also mirrored within virtual learning environment, online games and Massive Open Online Courses (MOOCS), where the opportunity is given to contextualize new knowledge through useful feedback from the environment, for all learner types, at each stage of the learning cycle, and through use of different methods, activities (Yusoff, et al, 2014) and styles in order to be more effective as a learner. (Honey and Mumford, 1995)
In the paper Information Sciences, Ranchhod et al. (2014) evaluate the educational effectiveness of experiential learning (Kolb and Fry, 1975), where students learn by experiencing or doing. This theory is based on individual learning by actively participating in an environment that involves high levels of active cognitive involvement. Such environments can be found within virtual simulators or virtual labs. This process is represented well in the Lewinian model (Fig. 2.1) (Kolb, 1975).
24 Chapter 2
Figure 2.1 The Lewinian Experiential Learning Model (Kolb, 1975)
The model focuses on four elements namely (i) Concrete Experience which includes doing or having an experience, followed by (ii) Reflective Observation, which involves reflecting upon or reviewing an experience, (iii) Forming abstract concepts, which involves concluding and learning from an experience and finally (iv) Testing in new situations or Active Experimentation which includes trying out what has been learnt. According to Kolb and Fry (1975), the learning cycle can start at any one of the four points, acting more as a continuous spiral. Their studies have shown that experiential learning results are significant in conceptual understanding, critical thinking, and problem-solving skills, increased enthusiasm and application, better performance, higher levels of confidence, self-efficacy, and enhancement of learning.
This concept of learning through experience can be found in educational games such as Markstrat (StratX Simulations) published by INSEAD in 1977, a marketing management simulator that allows students to work in groups running a corporation and making management and marketing decisions for it as they compete against other groups by launching and promoting virtual products while considering multiple factors that affect a product market launch. As the game is played over several rounds or periods, students get the opportunity to observe, research, and experiment with the different factors that affect their position in the virtual market place. This simulation has proven successful in teaching complex concepts of marketing and management through gamification (Marklund, 2015) and experiential learning, as students adapt new techniques each
25 Chapter 2
round, learning from the experience of previous periods and the performance of their peers in other groups. (Dodgson, 1987) The game is successfully used in over 500 academic institutions and 8 out of 10 of the top business schools in the world. (StratX, 2016) Research by Ranchhod et al. (2014) demonstrated the value-added proposition to students when they experience learning through the Markstrat simulation.
2.2.1 Experiential Learning
The theory of Experiential learning developed by Kolb (1984) considers that learning is the outcome of knowledge which is created through the transformation of experience. In the model outlined below, we find the four main stages of learning as determined in experiential learning. The first stage includes the concrete experience (feeling) by which the learner actively experiences an activity, for example, working in a laboratory. This is followed by the reflective observation (watching) stage where the learner is consciously reflecting back on the learning experience. This stage is then followed by the abstract conceptualisation (thinking) where the learner is conceptualising or contemplating on what they have learnt, and finally, we have the active experimentation stage (doing) where the learning is actively trying to test different a model or theory and build up future experiences. This method particularly applies to simulations, augmented technology-based and IoT enhanced education environments as the interaction is based on experience and the value of learning is directly linked to the user experience. As with much of the internet and online-based environments participation and engagement are key factors for the experience to work. It is, therefore, an ideal theory to use when evaluating user engagement and experience. Xu and Ke (2016) address the concept that emerging technology such as VR can be used to facilitate experiential learning allowing the students to learn from experiences in immersive environments.
2.2.2 Constructivism – Discovery Learning
The fundamental principles behind Bruner’s discovery learning are that students must be active in order to learn and that the learning process is active and constructive. He assessed that learners should be able to identify and determine key principles for themselves rather than accepting the explanations given to them.
26 Chapter 2
The theory is closely related to work by Jean Piaget (1952) and Seymour Papert (1981).
This theory is based on the following characteristics: (Learning Theories, 2016) • encouraging active engagement when learning • promoting motivation to learn • promoting autonomy in the learning process, including responsibility and independent learning • developing creative thinking and problem-solving skills • tailoring learning experiences
This theory is relevant when considering an augmented reality environment where the learner will be motivated to construct tailored learning patterns and build on the concepts that result from them. The difficulty here is that there might be cognitive overload and some misconceptions and sometimes makes it more difficult for instructors to identify such misconceptions and set them right.
A simulated environment that is augmented could provide a solution for that issue as there is a constant flow of actionable information from the environment in the form of IoT Analytics.
2.2.3 Multiple Intelligences Theory
In his 1991 theory of multiple intelligences, Gartner identifies eight intelligences and concluded that students have different minds and therefore learn, remember, understand, and perform differently. Although we can determine that human beings are all able to learn and understand the world via language, mathematics and music, an understanding of the self and others and have the ability to create things, the degree or strength at which we can undertake these elements or profiles of intelligence as he calls them, and use and combine determines how we can solve diverse and complex problems. Gartner (1991)
This theory also fits in well with the research study as the nature of the online platforms allows for interactivity and engagement on multiple levels not just linear and can hence yield more opportunity to determine performance at the various levels of the Gardner intelligences.
27 Chapter 2
2.2.4 Paradigm - Design-Based Research
Design-Based Research (Design-Based Research Collective, 2003) is a method that involves analytical techniques to try and balance theory and practice in learning environments. It is a key methodology used when trying to understand the reasons educational innovations can work in practice. The Design Based Research method targets the connections between educational theory, the design, and practical application. (Learning Theories, 2016)
Some of the key characteristics in this method include: • Consideration of complex problems in real and authentic contexts • Integrating existing and hypothetical design theories • Conducting comprehensive testing to develop better learning environments • Link the goals of designing learning environments and developing learning theories • Maintain a continuous cycle of design, performance, analysis, and redesign • Developing theories that can be shared with other practitioners and educational designers to include any implications that result from the design • Consideration is given to how the educational design functions within authentic settings
Designed Based Research (Cobb, 2003) considers an innovation effective when design and context are successfully combined and therefore goes beyond the initial product and considers the elements the learning process in a broader sense and as part of a complex system. This method is applicable for this study in that it involves the element of real and authentic contexts when considering the effectiveness of educational design, and this is significant in augmented environments that reflect real-life situations in virtual simulated platforms.
2.3 Technology and the Learning Experience
Technology offers new opportunities for educators to create valuable learning experiences using virtual environments. (Mads, 2017) Creating smart environments that can contribute to student engagement lean in to experiential learning which dictates that
28 Chapter 2 students “learn by doing” (Lewis and Williams, 1994) Such learning environments, where reality can be reflected on online platforms, allowing students to interact and engage with content in a visually stimulating setting, can lead to more interactions and engagement and potentially improve the learning experience. The opportunities for the development of immersive learning environments is now real. (Marklund, 2015). In the 1960s, pioneer academic Edgar Dale developed the Cone of Experience (Fig 2.2) reflecting on the impact that real simulations and experience play the learning experience. (Dale, 1969) The focus of this model highlights the value of experiences within the learning journey. Cone of Experience
Figure 2.2 Anderson (2015) Adapted by E. Dale, Audio visual Methods in Teaching (1969)
29 Chapter 2
It is essential to examine these theories and principles in order to re-evaluate our perceptions and ways of learning. The opportunity exists for an immersive virtual learning environment that can impact and improve the learning experience by utilising smart spaces and smart objects within flexible scenarios to deliver practical learning outcomes.
New developments in technology now allow for substantial opportunities to enhance the learning environment. Mobile learning (m-learning) and cloud computing now give more opportunities for interaction anywhere and through various devices. (Puttini and Mahmood, 2013) The adaptation of these technologies within education has resulted in the use of new fields such as learning analytics. This encompasses the use of sensors and mechanics within digital environments to collect valuable information that can be used to impact the progress of learning and structure the pace and progression for each student. (DiCerbo and Behrens 2014). By analysing big data that is designed into the learning environment, such as adaptive assessment, instructors can also improve teaching methods. (West, 2012)
These evolving technologies can link with online learning environments allowing data for instructors and tutors to get a new perspective of the learners’ progress in acquiring knowledge and skillsets. (DiCerbo and Behrens 2014).
By adding additional dimensions of real-time data feeding into the digital environment from external sources, and monitoring the responses of the participants within the environment while allowing them to adapt to the learning targets at their own pace, the instructor and institution can work towards creating smart classrooms. (Huang, 2015) According to Ruihang Huang (2015), a smart classroom promotes smart learning, which results in critical thinking by cultivating the learner's innovative practice and application ability. We have to consider how learning is impacted when knowledge is no longer acquired through a linear method. (Siemens, 2005)
Zhiting (2009) proposes an authoritative account of what he considers smart education to be. He defines smart education as the construct of an intelligent learning environment by applying modern information technology media such as the Internet
30 Chapter 2 of Things, Cloud Computing, Ubiquitous computing and mobile networks to teach with smart pedagogy. Although previous studies by Warschauer and Matuchniak (2010) indicated that technology-enhanced schools had not shown significant improvement in student outcomes and more recent studies by Davies and West (2014) indicate that there is no ‘dramatic improvement’ in the assessment results through standardised tests, the question of whether the students’ experience improved and whether the students’ overall approach to learning and education and levels of engagement increased, was not determined.
In a document of self-reported data by institutions around the world on the use of the iPad (Computers and Education, 2014) since its launch in 2010 reported the trends (Apple, 2017) from numerous primary and secondary schools and higher education instructions which reported substantial gains when comparing standardised test scores with current student test outcomes. A high school in Southwest Ranches Florida reported 200% increase in student academic achievement evidenced by a rise in National Merit acknowledgement to 74% increase in enrolment and 139% increase in students seeking higher education after graduation reported from VUC Syd in Denmark since they launched there one-to-one iPad program in 2010. The Hillard State School in Brisbane Australia reported that 82% of parents involved in the one to one iPad programme reported an increase in student maths engagement, and 90% of the students reported having a better learning environment.
2.4 Engagement and the Learning Experience
The concept of engagement in higher education literature highlights the link between student behaviour and teaching practices. A project was set up in the United States in the 1990’s (Kuh 2009a), that sought a new measurement for quality in higher education following discontent with the existing university ranking system. It concluded that engagement was an ‘evolving concept’ that incorporated a variety of elements from teaching practices to student behaviours, satisfaction and achievement. (Kahu,2013).
Engagement is defined in a general and often varying way in higher education depending on the institution and their position (Nygaard, 2013; Bryson, 2014)) and the motivating interest as well as achievent. (Sinfield, Harrington and Burns, 2016; Kahu, 2013). Bryson
31 Chapter 2
(2014) suggests that student engagement is linked to offering ‘value’ and in an effort to improve the student experience, but also highlights that it also ‘underpins other priorities such as retention widening participation and improving student learning’ (Bryson, 2014) In his paper for the Journal of Learning Design in which he reviews multisensory technologies in a Science, Technology, Engineering Arts and Mathematics classroom, Johann Talhaard defines engagement as ‘motivation, interest, curiosity and attitude.’ (Talhaard, 2016) and the Australian Council for Educational Research (2010), sum it up as ‘time and effort students devote to educationally purposeful activities’. This perspective then broadens the perspective of what engagement is within higher education to incorporate more complex concepts with cognitive and behavioural dimensions. (Kahu, 2013).
In 2008 Richard Elmore stated that in order to improve student learning one would need also to increase the levels of students’ active engagement with the content they are taught and therefore increasing the level of content and engagement, coupled with an increase in the teachers’ skills and knowledge would result in improvement in student learning. He also highlighted that what predicts performance is not so much linked to what the teachers do but what the students are actually doing, stating that it is imperative for students to understand expectations, to know and fully understand the reasons they should want to do a task. (Reilly, 2012) This understanding, in turn, impacts the students’ willingness to work, in the same way that employees of a business invest in their work because they know and understand in most cases what they need to do and why it is important that it gets done.
Engagement, therefore, becomes a significant element in improving the learning experience and in turn, the learning outcomes. Although the subject has been discussed for centuries when considering academic achievements, it was Alexander Astin’s in his 1984 student involvement developmental theory for Higher Education (Astin, 1984) paper and later theory of involvement (Astin, 1999) where a definition refers to student involvement as the amount of physical and psychological energy that students devote to academic experience.
32 Chapter 2
The initial study that led to the theory of student involvement was a longitudinal study of students that had dropped out of college (Astin, 1975) where he was researching the reasons for students remaining in college. The conclusion of that study indicated that almost all significant outcomes could be considered through the involvement concept (Astin 1999).
Austin concludes this paper with some examples for further research in which he highlights the developmental theorist concept of locus of control (Rotter, 1966) and attribution (Weiner, 1979) where it is argued that students’ levels of involvement are linked to their perception of internal or external control and whether those factors are controllable and dependent on their efforts or uncontrollable and dependent on ability. (Astin, 1999)
More recently student engagement was part of primary educational research by other significant authors on the subject such as Coates (2008) and Kuh (2009) who defined student engagement as representing “the time and effort students devote to activities that are empirically linked to the intended outcomes of college and what institutions do to both provide these activities and induce students to participate in them” (Kuh, 2009)
A review on student engagement by The Higher Education Academy (Trowler, 2010) at Lancaster University when outlining Fredricks, Blumenfeld and Paris (2004) Dimensions of Student Engagement, highlight that it feeling and activity together this in what constitutes engagement (Harper and Quaye, 2009) and that activity without feeling is just involvement. The three key dimensions of engagement were outlined as Behavioural, Emotional, and Cognitive Engagement (Fredricks, Blumenfeld and Paris, 2004) examples of which can be seen in Table 2.1 below:
Engagement Expectations Examples of positive Dimensions engagement Behavioural Students that are Attending classes and follow behaviourally rules (Tyler and Boelter, 2008) engaged would Stay on task behave within Persistence expected norms, and High levels of effort show no negative or disruptive behaviour Respect for teachers and peers Emotional Students that are Enjoyment (Greene, 2009) emotionally engaged
33 Chapter 2
show interest and Curiosity appreciation Happiness (Libbey, 2004) Sense of inclusion (Tsai et al., 2008) Enthusiasm Cognitive Students that are Deep processing of information cognitively engaged – seeking conceptual would show interest understanding (Schiefele, 1991) in their own learning Enjoy healthy competitiveness and challenges Have personalised learning strategies
Table 2.1 Dimensions of Engagement
In his study of levels of learner engagement, Philip Schlechty (1994) outlines that an engaged student was attracted to the task they were undertaking, persisted at completing tasks despite obstacles, and were visually delighted in their accomplishments. He explained the levels of learner engagement in his book Shaking Up the Schoolhouse: How to Support and Sustain Educational Innovation (2004) as follows:
Authentic Engagement – The student associates a task they are assigned, or activity that they are undertaking with a meaningful result that has clear and immediate value to them
Strategic Compliance or Ritual Engagement – assigned work has little or no inherent meaning or immediate value to the student, but the student associates it with extrinsic results that are of value
Ritual or Passive Compliance – the student is willing to expend whatever effort is necessary to avoid negative consequences, even though student sees little meaning or value in the task
Retreatism – the student is disengaged from the task and expends little or no energy attempting to comply with demands of the task/teacher, but doesn’t disrupt others or try to substitute other activities for assigned task
34 Chapter 2
Rebellion – the student refuses to do task, disrupts others, and/or tries to substitute other activities in lieu of assigned task. He concluded that in order to understand the level of a learnings engagement one had to consider the level of attention together with the level of commitment. This is outlined in the Table 2.1 below:
Table 2.2 Philip Schlecty’s Levels of Engagement Levels of Engagement
In his research Coates (2007) formulated a matrix of student engagement styles considered along social and academic dimensions and suggested as part of that mapping that students who were “reporting an intense form of engagement” were also inclined to be “highly involved in their university study” Coates (2007) observation is also mirrored the Kuh’s (2009) study and linked to motivation.
35 Chapter 2
2.5 Motivation as part of the Learning Experience
Hunt’s (1971) Self-Control theory indicates that motivation is highest when ‘one can make one’s own choices’ (Nunan, 2015) When assessing motivations for engagement, we can assert that student motivation is their desire to participate in the learning process and ultimately achieve their goals. Careful consideration should, therefore, be given to the reasons that trigger their involvement. Students may be equally motivated, but the source of that motivation may significantly vary (Lumsden, 1994).
Intrinsic and Extrinsic Motivation are discussed by Gardner and Lambert (1972) and Gardner and Tremblay (1994). Lumsden reflects on those studies in her paper on Student Motivation to Learn, (Lumsden, 1994) that a student who is ‘intrinsically motivated’ undertakes an activity for ‘their own sake, for the enjoyment it provides, the learning it permits or the feeling of accomplishment it evokes” (Lepper, 1988). Lumsden (1994) further assessed that a student that was “extrinsically motivated” undertakes a task or activity “in order to obtain a reward or avoid some punishment external to the activity itself” (Lepper,1998).
Lizzio and Wilson (2009) identified four quadrants to illustrate motivation for engagement in student governance through extrinsic and intrinsic motivation shown below in Fig. 2.3 (Trowler, 2010).
Figure 2.3 Motivation for Engagement in Student Governance (Lizzio & Wilson, 2009)
36 Chapter 2
The motivation to learn is defined by Marshall (1987) as “the mindfulness, value and benefits of academic tasks to the learner regardless of whether or not they are intrinsically interesting” cited in Lumsden (1994) who further argues that when students are intrinsically motivated they are more likely to take on tasks that are more challenging and demand more effort and enable deeper processing of information. (Lepper, 1988). Activities that promote intrinsic motivation can include learning to create their materials, finding different strategies to enhance their learning outcomes and encouraging genuine interest in mastering a subject rather than just achieving a grade (Brown, 2001). Reeve’s (2012) research links engagement with student achievement and the learning environment and teacher’s motivational style in younger learners showcasing, as seen in Fig 2.4 below, that high student engagement is directly linked to the flow of instruction and the quality of the teacher’s motivation style. (Reeve, 2012)
Figure 2.4 Three new functions of student engagement Reeve (2012)
The importance of a blended learning environments and learning experiences is considered in a paper in the Internet and Higher Education by Garrison and Kanuka (2004) who concluded that blended learning was not only ‘consistent with the values of higher education institutions but it has the proven potential to enhance both effectiveness and efficiency of meaningful learning experiences’. The next section will cover smart learning environments.
37 Chapter 2
2.6 Smart Learning Environments
Huang (2014) sums the key characteristics of Smart Education as allowing for diversification of the means of Education and liberation of the learning style and how knowledge is constructed. He indicates that smart education changes the learner’s learning style as learning happens through multiple devices and is not limited to paper and traditional classroom learning. It allows for learning to be tailored to the requirements and abilities of the learner (Huang, 2014). This can significantly enhance the interest and engagement levels of the students that learn through these environments. In their report, Internet of Things, Strategic Research Roadmap, Guillemin and Friess (2009) sum up the connectivity concept of IoT with a six ’A’ model (fig. 2.3), describing the vision for smart elements of the Internet of Things as allowing for interconnectivity with anyone and anything at any time and place using any server or network. (Atayero, 2014).
This model implies a seamless connection between smart devices and provides opportunities for a rich and immersive learning experience through the use of real data from these devices when trying to understand particular topics (Perera et al. 2014).
Figure 2.5 The 6A Concept for Internet of Things (Guillemin and Friess, 2009)
38 Chapter 2
Atayero (2014) argues that smart technologies such as IoT will allow for the interconnectivity of anyone and anything, at any time and any place, using any service over any network. According to Guillemin and Friess, this will include focusing on some of the fundamentals of technology such as Communication, Computing, Convergence, Content Collections (Repositories), Connectivity. Envisioning the IoT elements that we could apply in a classroom (Magerkurth, 2011) using existing simulated environments through serious games such as MarkStrat we can now apply the 6 A model of Guillemin and Friess, to education and identify some of the elements that IoT can bring into the educational environment.
2.6.1 Any place / Anywhere
This technology will also allow education to break the barriers of physical and geographic constraint, clearing the way for global data to be collected by external devices to filter into the virtual classroom. An elaborate IoT ecosystem will give a more holistic picture of the subject matter for the student giving them resources that were previously only available to researchers and statisticians. Moreover, the data can be incorporated to illuminate certain theories and concepts bringing them to life, rather than dull paper- based explanations or discussions. Atayero (2014) concludes that this will offer an array of global innovation as well as create more research questions, which will expand the frontiers of knowledge (Chen et al., 2014). In his structure of a smart adult education system, Huang (2014) showcases a cloud centre that is supported by cloud computing, IoT enhanced with RFID, and an open course centre.
Table 2.2 Online Content Delivery (Allen and Seaman, 2015)
39 Chapter 2
2.6.2 Anything / Any device
It is already evident that Universities and Colleges are embracing the growth of mobile technology and connected devices and are making more significant investments in having a mobile-friendly online presence with content that is responsive and accessible from all devices, whether in the university or not. (Hanover Research, 2014) The Online Learning Consortium and Babson Survey Research group (2015) reported that over 5.8 million students in the United States are taking at least one online course, and the numbers are growing. This is supported by figures that show the number of courses offered in Massive Open Online Courses (MOOC)s increasing to 9400 in 2017 (Fig 2.6) with over 81 million learners worldwide (WES,2018) on a variety of university-supported programs on platforms such as Coursera. Some universities even offer degree pathways.
Figure 2.6 Growth of MOOCs (World Education Services, 2018)
40 Chapter 2
2.6.3 Anytime / Any context
As an IoT virtual learning environment is not bound by schedule or timetable, learning can continue to happen within different contexts at any time as long as the individual is connected and linked into the system so that information can be collated and new experiences recorded by the smart device.
2.6.4 Anyone / Anybody
Although there are limitations posed by connectivity that would limit some areas from using the internet effectively, as presented above, most of the world is now connected, and speeds and reliability is becoming a reality for most countries in the world. Industry technology giants like Google and Microsoft and social media super brand, Facebook have all been pushing for a universal internet connection to become a reality in the next few years. Google launched Project Loon in 2013. Project Loon aimed to connect every location in the world with free internet using giant solar powered, high altitude balloons that move with the wind through the stratosphere over 20,000m up. The balloon sends signals to other balloons nearby and reaches the internet provider stations on the ground, which in turn will connect to the balloon network and send signals to specialised internet antennas in homes and businesses. (Google, 2013). Facebook, on the other hand, has proposed Aquila solar-powered drones that will do a similar function as the Google balloon (Hern, 2015).
2.6.5 Any path / Any network
Ciscos give a definition for the Internet of Everything (IoE) that indicates that smart technology will link people process and data together (Fig. 2.7) to create richer experiences (Cisco, 2012), and the opportunities lie in understanding the many capabilities that are offered by this technology.
41 Chapter 2
Figure 2.7 Internet of Everything (Cisco, 2012)
2.6.6 Any service / Any operation
The overall connectivity allowed by emerging technologies means more and more businesses can connect their services to the internet and allow for dynamic integration into other systems, including education. Analytics Data collected through technology companies are now available through tools like Google Trends (Google Planning Tools, 2016) and could feed their data into simulators, making the information real and tangible for the participants. A further element that could be added to enhance this comprehensive system of IoT is that it allows learning in any way and by any method.
2.6.7 Anyway / Any method
This level of interconnectivity allows smart management and the prospect of getting a clearer picture of where the learner lacks in knowledge or engagement, clearing the way for personalised training. A smart learning environment will also impact the fundamental behaviours of the learners, and consideration should be given to the specific human
42 Chapter 2 system interaction, which might include environmental and emotional considerations such as boredom and stress levels. San Pedro et al. (2014) cites Craig et al. (2004) and hypothesises that learners do not fully engage with material that is presented to them and are less likely to retain knowledge on a topic when they are bored (Stallwood, 2016).
Integrating the concept of the 6As in education reveals numerous opportunities to use emerging technologies to create immersive learning environments. With VR devices set to increase in the next 5 years (Gartner, 2018) educational scenarios that can include gaming elements and can be implemented within virtual platforms to allow self-learning of taught concepts through the 6A principle.
2.7 Virtual Simulators and Gaming Applications in Education
The application of gaming in education (Annetta, 2009) can be improved with the introduction of cloud computing which creates an active environment for participation and communication with others, rather than the ‘normal’ methodology of a teacher conveying information or reading from a textbook or listening to a broadcast (Chung et al., 2010). If a student can manipulate an environment, learn from it, interact with others, and connect the virtual with the real, we have a compelling learning environment. Students can make decisions, evaluate them (or be evaluated within the game), set goals, and have fun while learning. In many ways, the lecturer or teacher becomes and observer and facilitator on this journey of self-discovery (Vujovi et al., 2015). In many ways, what should be considered are changes in behaviour in a more interactive atmosphere of learning. (Dickey, 2005)
Simulators and serious games in the form of business games were introduced in the late 1950s (Keys and Wolfe, 1990; Wolfe and Guth, 1975), heralding the growth of simulations as a tool for experiential learning in higher education and organisations (Curry and Moutinho, 1992; Faria et al., 2009; Parks and Lindstrom, 1995; Puskurich, 1993; Smith and Ellott, 2007). Business simulations expose students to immediate and spontaneous decision-making situations in a controlled environment, enhancing the teaching and learning of business management concepts and methodologies (Clarke, 2009; Clark et al., 2003). As technology offered faster connectivity and interactivity, simulations evolved to meet the changing formats ranging from floppy disks to university networks or online
43 Chapter 2
environments. This progressive adaptation allowed a faster adoption of simulations within universities, while researchers attempted to understand their efficacy in both academia and industry (Bell et al., 2008; Chin et al., 2009; Faria et al., 2009; Linstead, 1990).
Computer-based business simulations are usually programmed as a specific industry game, in which participants are expected to learn business skills while managing a fictitious company within a competitive environment (Summers, 2004), using economic, finance, and marketing theories (Gold and Pray, 1990; Teach, 1990). Learners make decisions by evaluating and allocating resources during several game periods, their analysis concerning either firm’s internal mechanisms/dynamics, its interaction with the business environment, or both (Summers, 2004). The widespread use of computer-based games determined an active debate among management researchers and educators regarding the educational value of business simulations (Anderson and Lawton, 2009; Clarke, 2009; Doyle and Brown, 2000; Faria, 2001; Stainton et al., 2010). Bloom’s taxonomy of learning objectives (Bloom et al., 1956) was often applied to understand the learning outcomes of business simulations (Gosen and Washbush, 2004; Wolfe, 1985). Hoover and Whitehead (1975) argued that participation in business simulation results in cognitive, affective, and behavioural learning. Cognitive learning represents the understanding of basic facts and concepts that lead a participant to develop sound decision-making abilities (Wellington et al., 1995). Although the cognitive outcomes of business simulations have been amply discussed in the existing literature (Brenenstuhl, 1975; Gosen and Washbush, 2004; Keys and Wolfe, 1990; Wolfe, 1985 and 1997), it is still difficult to document their accomplishment (Anderson and Lawton, 2009).
Affective learning is associated with participants’ perceptions, attitudes, and values regarding the topic of the business simulation and the applied pedagogical method (Parasuraman, 1980). The analysis of students’ perception regarding various pedagogical tools such as lectures, cases and business simulations (Anderson and Lawton, 2009; Anderson and Woodhouse, 1984; Blythe and Gosenpud, 1981; Miles et al., 1986) indicated a more positive attitude and acceptability towards simulations in comparison with other teaching approaches. Behavioural learning can be described as a change in
44 Chapter 2 participants’ behaviour determined by the improved capacity to understand and apply business concepts and models in real-life competitive situations (Wellington et al., 1995). To assess the external validity of behavioural learning in business simulations, several studies (Babb et al., 1966; McKinney and Dill, 1966; Vance and Gray, 1967; Wolfe, 1976) compared the playing style of students with the approach of successful managers, finding a high behavioural similarity. Moreover, Pasin and Giroux (2011) concluded that simulations are highly effective in helping students to develop decision-making abilities for complex and dynamic situations.
However, the effectiveness of computer-based educational games depends directly on the success of the process of game implementation, which can be designed to shape in a specific way the interaction between students, between students and the game, as well as between students and the coordinating professors and tutors.
Advantages of using simulators in education for hand-on training and the ability to demonstrate more complex concepts is further reinforced by the flexibility and ease of updating content and modifying modules at any time so that the platforms always remain current and valid. Research indicates that content presented in training through VR in particular can be more memorable than content presented through other formats such as video or text. (Etcourse, 2018)
2.8 Virtual Reality Learning Environments and Scenarios
More recent developments in technology have allowed the adaptation of educational games within virtual and augmented reality learning environments (VRLE) and have added the reality factors to the dimension of experiential learning (Wu et al., 2013). These environments simulate real-world situations and locations and allow the user to not only interact but also engage with and experiment with the content in the virtual environment. (Burdea and Coiffet, 1994) One example of such a development is the award-winning virtual laboratory simulation by Labster (2015) to cover STEM, used by top universities such as Harvard Medical School, Massachusetts Institute of Technology, and the University of Glasgow. The platform allows students access to a virtual 3D laboratory where they can safely learn in a virtual chemistry laboratory that supports open-end investigations, undertake experiments and execute instructions as though they are in a
45 Chapter 2
real-life environment but without the dangers of making mistakes. The environment is created with all the real-life elements that an average lab would be equipped with. The student enters a simulated virtual environment where he is guided through different stages to complete tasks, refresh knowledge from prior tasks, and even experience some aspects of what can happen when things are not done correctly. (Labster, 2016)
Figure 2.8 Virtual Chemistry Lab (Labster, 2016)
As virtual reality environments become more complex and immersive, the opportunities for application in education become significant. An example is given by the Virtual Reality Society using CAVE Automatic Virtual Environment, a cube like space in which images are displayed by a series of projectors (Virtual Reality Society, 2015). Students could potentially step into ancient Greece. Walk down the streets of the old ‘Agora’ or market place. A fully immersive experience where they can experience Athens 3000 years ago, interacting with objects in the virtual world by means of a data glove offering a new way to gain a visual understanding of things that were lost thousands of years ago and were previously not available. The learner can interact and manipulate not only objects, but complex formulae and data sets.
46 Chapter 2
An application of VR in Health Care professional training is considered in a study by (Mantovani et al., 2003) which outlined the benefits of using immersive educational environments in health care education. This can be applied to most industries. A student studying marketing could use the VRLE to segment audiences, visualise new product development, and assess the channels available for a brand promotion campaign. In addition, VRLE allows for more engaging and active interaction with peers (Hsiu-Mei, Ulrich, and Shu-Sheng, 2010) AltspaceVR is a social VR platform, built for Samsung Gear VR, HTC Vive, Oculus Rift platforms, where participants engage based on real world movements and input allowing a new level of interaction, so it feels like everyone is in the same room together. Therefore, students working on joint projects can join the VR platform at any time to participate or even chat and play games with each other. The possibility to connect people from across the world in such an immersive way enhances the opportunities to learn and interact with others. Scenarios can be part of this.
The concept of scenario based learning is effective is virtual environments and can play a major role in the overall impact that the experience has on the learner. A definition of scenarios is given by Philip Van Notten (2006) where he suggests that they are “consistent and coherent descriptions of alternative hypothetical futures that reflect different perspectives on past, present, and future developments, which can serve as a basis for action” (Van Notten, 2006). They fit in directly with Kolb’s experiential learning model and Brunner’s discovery theory as well as the Cone of Experience (Anderson, 2015). Scenarios are a compelling learning tool as it places the learning in a situation that encourages the learner to envision outcomes and work out solutions in order to solve issues, advance or apply their knowledge within a given setting. According to Scivally (2014) scenarios can give students the opportunity to apply their skills and knowledge in realistic situations.
Scenarios could also help learners remain motivated and engaged by presenting the boundaries in which their skills can be tested and enhanced.
47 Chapter 2
2.9 IoT, Augmented and Virtual Learning in Education
Studies recognise the potential in developing an immersive learning environment to educate the smart technology generation (Kortuem, 2013). The key components that support an IoT smart environment are sensors and devices and the primary driver behind the rapid expansion of IoT in many industries is that sensors have now become smaller and are produced at lower costs. The application of sensors in everyday objects such as cars, machines, and wearables, has forged the path for IoT, augmented and virtual reality to be implemented to create smarter environments. The key features of a VLE listed by Peraya et al. (1999) and cited by Stankov et al. (2010) include the allowance for multi- authoring between teachers and students, open access, online assessments, social interaction and multiple tool integration. Furthermore, there are benefits to the instructor that include enhanced administrative tools, extended communication and collaboration, more active engagement and a reduction in the face-to-face time with students. (Atayero, 2014).
The data that we now get from sensors and device connections is more accurate and more relevant. It is based on the reality of the situation in which the data was collected, and it cannot be misread or misinterpreted, increasing the accuracy and significance of the results. An example of this can be noted in smart cities such as Singapore where in a recent IoT development, sensors on the roads and in cars were used to feedback information about traffic, pollution, noise, pollen and temperature within the city allowing officials in hospitals and security to broadcast messages updating residents of the current conditions and giving support and suggestions where needed on what roads to take, when to avoid going through certain roads and how to avoid traffic. In another project at the Bosch IoT Lab at the University of St Gallen, Paul Rigger (2013) was introducing a Room Climate Monitoring System and the implementation of smart technology in classroom environments. A picture of Albert Einstein was fitted with sensors that received information on the levels of humidity, heat and oxygen while on a wall in a classroom. If the image changed colour, then the room was not properly ventilated and the students could open a window. In these examples analytics collected
48 Chapter 2 from the smart objects or assets were used to create applications that allow that data to be used in an effective and beneficial way by the community and those who serve it.
IoT Analytics is different from other types of big data analytics, and it has several characteristics that define it. Real time analytics, which is data that comes from beacons, sensors and even other devices on the ground that feeds information back as it happens. This information is heterogeneous as it comes from many different sources and irregular because it is reporting based on real world factors and not programmed patterns. As the environment changes, so does the analytics and the reports that it generates. IoT Analytics can be implemented to assist in traffic management, energy facilities and weather info structure to build smarter cities that are safer and more efficient.
A further concept that plays a role in online learning environment is Tangible User Interfaces (TUIs) which uses objects to provide data and output for a virtual world. It offers intuitive and natural interaction which provides direct and easy control and manipulation of virtual 3D objects and space (Kim et al, 2006) and can include such elements as Automated Voice Recognition (AVR).
Figure 2.9 Microsoft HoloLens in Education (Microsoft HoloLens, Accessed: 1 July, 2016
A study by Imperial College London School of surgeons at St Mary’s Hospital using Microsoft’s HoloLens headset and immersed in MR showcased that surgeons were then more successful in implementing their training in real life situations.
49 Chapter 2
Figure 2.10 AR overlay of models as viewed from remote HoloLens Philip Pratt et al (2018) Microsoft HoloLens (c) Microsoft
In the same way Augmented Reality enables interaction in real time with real and virtual objects in the real world. (Berger et al., 1999). Bringing all these concepts together, provides the necessary means to develop a new intelligent framework for immersive learning, which is explained in the next section.
2.10 Gap in the current Research and Literature
As outlined in this chapter there has been significant research into the development of new technologies and the opportunities that it offers education. Many theories indicate a link between student engagement and achievement and perceived value in education. There is a clear connection between learning and doing as outlined in (Learning Theories, 2016). The literature also revealed the growth of emerging technologies and some of their applications in education today. There is, however, a gap that has been identified in the current research and available literature in the area of experiential learning and the application of emerging technologies to improve engagement for the purpose of improving the learning experience. Although some research exists in experiential activities in an educational context and the application of technologies to benefit learning, there is little by way of using smart immersive virtual or augmented environments to improve student engagement and thus learning outcomes, through experiential learning. The use of smart virtual and augmented learning environments could potentially be a reality for education in the near future as these technology accelerators evolve and change every element of life and behaviour. The use of virtual scenarios that offer the learner a safe place to experiment, experience and learn in a self-
50 Chapter 2 learning environment that is flexible to meet their individual learning needs and develops along with the learner. Although there are efforts to understand the learning patterns using different technologies this again has not s yet been presented as a holistic approach where the technologies are seen working together towards improving the overall learning experience.
This thesis aims to contribute a better understanding of how these immersive learning environments can impact student engagement and subsequently the student experience and present a framework for future implementation of these technologies in education.
2.11 Summary
With the development in technology and more applications made to enhance and develop more immersive interactive online environments the opportunities now lie including elements of IoT within these technologically advanced platforms such as Markstat and Labster and the growing adoption of smart things, we can now consider the use of devices and smart things to enhance and create more interactive learning experiences. In an ideal educational system, we would be able to assess every individual learner’s needs and preferences, and based on the conclusions, we would have the systems in place that can propose learning solutions that are tailored to their requirements. To date this has been a very challenging task as tracking individual preferences within all learning environments would be a task too great for any institute to monitor and record. As learning continues even outside the classroom, the problems become even greater. Furthermore, we would want to provide knowledge that is current, relevant, and real. That knowledge should be dynamic and flexible. Finally, this should be facilitated through engaging and immersive learning environments that promote motivation and active learning, such as gaming simulators. Therefore, an opportunity lies in designing a framework that will incorporate emerging technologies in creating smart spaces and objects to facilitate immersive learning environments and aim to improve engagement and the learning experience.
51 Chapter 3
Chapter 3. Proposed Framework
The literature review led to many questions on the use of smart spaces to improve engagement and the learners’ experience and were key in formulating the framework.
Some of these questions are included below: • Can smart technology-enhanced virtual spaces improve the learning experience? • Does a smart virtual learning environment improve engagement levels? • Can the implementation of emerging technologies in education simulations improve the learning experience for students in higher education? • Can smart real-time analytics and metrics improve learning conditions for learners and increase engagement? • Can smart wearable technology help improve the learning experience?
3.1 The Research Question
This thesis considers an appropriate framework for an immersive educational environment based on emerging technologies used within smart classrooms. The basic premise being that by using immersive virtual learning environments the user experience and the educational journey can be considerably enhanced.
The conjecture of the thesis is that emerging technologies and the use of virtual learning environments can impact and improve engagement and the learning experience for higher education students.
The research question can be summed up as follows:
Does the educational experience and engagement of learners improve when immersed in a smart virtual learning environment?
The framework that is proposed in this study is using smart technologies such as Internet of Things and Augmented Reality to enhance a virtual learning environment as described in Chapter 3 of this paper.
52 Chapter 3
3.2 Hypotheses
There are examples where simulations have been used to enhance the learning experience which has been outlined in previous chapters. It is noted from the literature review above that these have been effective. It is acknowledged that the use of technology allows us to build and simulate business and academic environments as in the examples of Markstrat and Labster.
It is the purpose of this research to develop a design proposition for a virtual educational environment based on a scenario and identify whether simulated virtual learning environments can improve engagement levels and the student learning experience. Using emerging technologies to create digital twins in a virtual platform with which to develop simulations and apply the scenario and target for the learning. This will improve the effectiveness of the content using real and relevant data to enhance engagement and allow for personalisation and outcomes on smart learning platforms. As these platforms are dynamic the changes applied in various situations will affect the results which will be consistently changing and evolving in real time for the students. The research will aim to understand if this immersive experience improves engagement and the learning experience for students in higher education. The work that will be undertaken includes using an existing PC educational simulator and an immersive virtual learning platform developed by the researcher.
We will aim to answer the research question by confirming the following hypotheses:
Using an immersive virtual environment, we will measure the engagement levels and the learning experience of students in higher education.
Although previous studies have shown engagement through serious online games, educational virtual reality platforms and internet applications, there is no definitive study to show the engagement levels and improvement of the learning experience of higher education students when immersed in virtual smart learning environments.
The use of real and relevant information increases the ‘reality’ element in an educational environment making it more engaging leading to the following hypotheses:
53 Chapter 3
H1. The use of emerging technologies in virtual learning environments can improve engagement.
H2. Emerging technologies will improve learning experiences.
3.3 Towards a Smart Learning Experience
It is important to identify the concepts and principles that can be used to enhance the learning environment. We initially consider the existing factors that make up an interactive learning environment that is supported with technology. This can be in the form of laptops or computers, iPads and interactive whiteboards. We can then build on those to include mobile learning platforms and virtual reality and finally we enhance it sensors and beacons to create an IoT enhanced learning experience as shown below in (Fig 3.1.)
The framework (Fig 3.2) proposes to link the concepts of a smart learning environment to create an immersive education platform that is responsive to the learners needs to improve the educational experience. The connectivity will also expand the range of tools available to the educators to better control and manage that journey.
In this smart learning environment, we can facilitate learning modules that allow students to explore concepts and ideas and create their own learning resources. The opportunity exists to interlink the various technologies and use them to improve the way in which students learn, how that learning is delivered and when and monitor their progress anywhere at any time.
54 Chapter 3
Figure 3.1 Towards an Enhanced Learning Experience
The first level towards an enhanced learning experience (ELE) includes the use of PC and laptops which many education institutions already actively use today. This progresses into more integrated solutions with online modules available to the students at any time via a Hub or Virtual Leanring Environment (VLE). The next level of development would be to include mixed reality (MX) and include virtual and augmented platforms that can reinforce concepts and add a new dimention in visualisation in immersive scenarios. The final level towards the development of a fully integrated ELE would be to include objects embedded with IoT technology, sensors and beacons to develop smart spaces and use IoT Analytics and Artificial Intelligence to monitor and analyse the data towards optimisation of the content for each individual learner.
55 Chapter 3
3.4 A Smart Learning Environment Framework
Figure 3.2 and Figure 3.3 The Smart Learning Environment Framework (M Zinopoulou)
56 Chapter 3
3.5 Breakdown of the Framework
There are six key elements that make up the basis of the proposed Smart Learning Environment in order to create immersive learning experiences to improve engagement and hence the overall learning experience. Each one can then be anchored onto the learning environment and working in unison to support and contribute to the outcomes.
1. Connected Devices: These can be in the form of mobile technology and wearables linked to the personal identity of the student to allow access to content and customisation. This also allows learning to happen anywhere and at any time. 2. Education Content: This includes the educational modules with interactive material that is specific to the learning style and level of the student. Content is adapted based on how the student progresses and their own experiences with the learning material. 3. Learning Analytics: This data is monitored both in the Virtual and Augmented Environments and in the real world allowing for optimisation to allow for predictions, forecasting and Reporting. IoT Analytics can support this and AI capabilities can manage it. 4. Smart Spaces and Smart Objects: These will allow more user control of the learning experience, give added functionality and customisation. 5. Virtual Reality Platforms: Either in the form of Simulators or as VRLEs these platforms will house the learning content and facilitate the learning experience, collaboration and interaction between the student and the virtual content as well as together with their peers and other students from across the globe allowing for new opportunities to expand learning horizons. 6. Sensors and beacons: These will allow for smart spaces to be created, capturing and monitoring data and feeding it back into the virtual and real world environments allowing for big data analysis and real time updates.
57 Chapter 3
3.6 Implementing a Smart Learning Environment
The smart technology enhanced classroom allows the learner to enter into a learning environment that will identify them, log their journey and propose content and materials that are tailored to individual needs. The student enters a smart lecture theatre or space and can log into the university system using their ‘eduband’, a wearable (fig. 3.3 and 3.4) device with a unique identifier. Sensors and beacons in the room monitor activity and interaction in the room to adapt according to need and trigger. If students wish to use the interactive boards they will come on when they student is nearby and when the student interacts with the content the board will use their eduband to give access and monitor progress. Students can use VR headsets to connect and engage with online content check online notices in the virtual learning environment and complete tasks and activities at their own pace and in their own style. Students will also be able to personalise the learning process, create new content that they can share with peers allowing for collaboration and interaction. Drawing on available online content, students can cover parts of the lecture they may have missed or wanted more clarity on, they can see more case studies and examples linked to the concepts covered in class and enhance the learning outcomes. In a smart learning environment there is a dynamic control of the learning experience and both teachers and students can use data monitoring to make predictions using patterns in the data collected. For example, if a student is actively engaging with online content around a certain topic we can predict the need for additional resources or materials needed for that topic. The smart learning environment can be accessed from anywhere although some of the IoT features that involve sensors may not be available from home for example which may limit the number of interactions recorded but will nevertheless give adaptable educational modules that can be used from anywhere as long as the student has their eduband and VR headset.
58 Chapter 3
Figure 3.4 Connecting to a VRLE Figure 3.5 Diagram of a proposed Smart Lecture Theatre
Furthermore, a smart learning environment framework can include real time analytics and real world feed that can enhance the smart space with data from IoT devices, that is coming both from the environment (external real data) and from the education system in the classroom (internal) and wearable sensors. The analysis of this data will offer the opportunity for predictive analytics and for personalised offering of content that is relevant and significant for the individual learner and their pace and style of learning.
59 Chapter 3
3.7 Example of an Immersive Smart Virtual Learning Environment
The attendance register is recorded via wearable technology in a smart learning environment, and the student is entered into a virtual educational environment where they can choose to build an avatar profile. Instruction is sent to the students iPad or smartphone on the available warm up activities which demonstrate the available stages for their level and map the journey ahead showing the learning stages and outcomes for each subject and the points or achievements awarded. The available media options are explained and once the student has indicated that they have completed the instruction guide their chosen avatar profile is linked to the eduband and student record.
The student is then able to interact with the connected objects in the smart educational environment under that chosen identity creating a persona profile that can include a number of metrics both physical such as heart rate, activity and stress levels measured through the eduband to virtual monitoring of tasks completed, actions taken, media choices and scores.
The students’ options within the smart environment include the use of smartboards, virtual reality goggles, and controllers, iPads and laptops. These are linked to the education system and receive information from the sensors and beacons in the room.
This information is fed back into the education system analytics and is available to both the student and educators on a customised dashboard. This allows for the student journey to then be customised and tailored to the needs, strengths, and weaknesses of the learner. The IoT Analytics that is built into the system can record individual activity and group activity with the learning platform and combine that with the data coming from the connected objects in the room. The virtual or virtual reality learning environment will facilitate outcomes and adjust the learning modules based on the IoT analytics that is recorded.
A further element of the technology enhanced classroom is that it will allow external data to feed back to the learning platform through external sensors and API real time data
60 Chapter 3 updates. This form of real time analytics will give the learner access to real data and allows a more realistic and relative understanding of subject matter.
As a practical example, we can consider a group of Marketing students who are using a simulator such as Markstrat to understand marketing and management concepts. If the simulator was infused with data that came from real time dynamic external sources such as Google Trends (Google Planning Tools) which indicates commonly used search engine keywords, and brands that are already using IoT such as Levis, Tesla Motors and Home Depot (Walgrove, 2015) to feed in real brand information into a stagnant system of data, thus bringing it to life. This would give a new dimension to the simulation allowing students to experience the gaming elements within using data that comes from the real world. This implies that the simulator not only visually mirrors the real world but the data shared within the environment is also real time, thus enhancing the experience.
A further element that IoT allows is the data collected from the participants. This data can allow for personalisation of the learning elements that best fit every individual.
In the Labster platform for example, the eduband would identify that the student is anxious through heart beat and perspiration sensors allowing for in-simulator notifications to help with calming messages and positive reinforcement.
However, the data alone cannot be used unless it is processed through IoT analytics due to the volume and speed and frequency at which this data is available.
It will, therefore, be imperative to create selected modules that can identify what data is significant to monitor, how will that data be segmented, and how will it be distributed through the education platform. Advanced analytics is therefore needed to develop a matrix of sentiment, movement, activity recognition, fusion, behaviour analytics, tracking and clustering to then allow for more accurate monitoring, prediction, simulation, prediction and forecast to happen.
3.8 Implementing a Smart Learning Environment Framework
Some of the elements that need to be considered for this framework to work involve education, application, measurement, and security. Factors to be considered include:
61 Chapter 3
Information Channels. What channels should be utilised to present materials with the applications?
Data Access. What data can be accessed by the learner, their peers, and the educators? What data is available within the databank and how is it managed and maintained?
Control. What elements of the platform can the learner control and what are the decision options that are available based on this. How will these decisions be recorded and monitored? What elements of the system are impacted by those decisions? Reporting. How will the outcomes be reported? What reward factors are in place and what are the methods for displaying this.
Educating the educators is one of the key limitations to this study. The use of advanced technologies requires knowledge of how to implement them and how to analyse the resulting data sets to improve the learner experience. In his paper Elton (1999) suggest that the reason education systems are slow to change is linked to their hierarchical command and control systems which mean that teachers and instructors have limited means and power to improve the learning and teaching methods via technology. Selinger (2015) highlights some of the issues and possibilities linked to IoT in Education in her paper for Cisco, Education and the internet of Everything. Educators possessing the knowledge and know how to exploit IoT is outlined along with the integration of other emerging technologies such as wearables and augmented reality with IoE.
An element that is key in implementing smart immersive experiences in the classroom, is security. Privacy and security need to be addressed and clear guidelines established to ensure trust (Perera, et al, 2013) Ensuring that data is handled in a way that will not jeopardise elements of confidentiality and privacy on the part of the learner as well as monitoring the feeds that are coming from external sources. Accounts held by students on their eduband will be password protected and information linked to individual privacy such as location will be permissions based and can be deactivated when the band is not worn within a smart learning environment or for the purposes of smart learning.
Finally, it should be noted that a government led funding initiatives to explore the future of emerging technologies such as Artificial Intelligence in schools and colleges, Educ-AI-
62 Chapter 3 tion in collaboration with Nesta who published their report in February 2019 (Nesta, 2019), highlights the significance that these technologies will have and the impact on education.
3.9 Summary
Considering the available technologies and the need for a development in the education system that allows for personalized learning avenues that can utilise and manipulate real time information through virtual learning platforms linked to gaming applications it is evident that we need to find ways to link these factors together to create an immersive and fully integrated learning experience. In his 1926 interview to Colliers, where he predicted the internet and use of smart devices, Nichola Tesla stated The majority of the ills from which humanity suffers are due to the immense extent of the terrestrial globe and the inability of individuals and nations to come into close contact.”
However, he later added on subject of technology that there was a difference between progress and technology. He highlighted that while progress should benefit mankind, technology does not necessarily always do that. (Tesla, 1926). This significant statement is relevant to today’s world of constant technological advancement. It is important to identify whether we can improve the learning experience for students by implementing the Internet of Things within learning environments. This can be used to improve our perceptions and ways of learning and apply them in a way that will benefit education and humanity.
63 Chapter 4
Chapter 4. Methodology and Design
4.1 Research Purpose
Scott and Morrison (2006) state:
Methodology is the theory (or set of ideas about the relationship between
phenomena) of how researchers gain knowledge in research contexts and
why.
Paradigms are defined by Bassey (2007) as a network of coherent ideas about the nature of the world and the function of researchers which, adhered to by a group of researchers, conditions the patterns of their thinking and underpins their research actions.
In educational research two key research paradigms are positivism and interpretivism, this research study, however, adopts the mixed methods research approach (Clark and Ivankova, 2016) multiple perspectives using quantitative research in the form of questionnaires and qualitative research in the form of interviews. The paradigmatic justification, and methodological design are outlined in this chapter. For the smart virtual technology element this study will be using the Technology Acceptance Model (TAM) to evaluate the learners’ attitude to technology. (Davis, 1989)
The goal of this study is to understand the differences in engagement levels when immersive virtual environments are used within higher education institutions. The study specifically focuses on Marketing students at graduate and postgraduate level. It considers whether simulation environments that are reinforced by immersive virtual reality can enhance the learning experience by improving engagement levels. The study focuses on the interaction levels and enthusiasm of the learner to engage with the material and activities to showcase intrinsic motivation and involvement in the context of the learning environment and their willingness to continue to interact with the content on the platform.
64 Chapter 4
In order to undertake research that is linked to engagement levels we would first need to clearly define what engagement is and how it can be measured within different learning scenarios. We would need to consider the various elements linked to engagement and how they affect the outcome of interactions and performance. We can then apply this to Education environments in order to record engagement levels and changes in these levels when different technologies such as Internet of Things (IoT is applied to the education environment.
Furthermore, the study will attempt to assess the metrics that indicate any significant change in engagement levels based on motivation, effort, commitment perceived value, integration and associated outcomes.
Within business settings, David Croston (2008) observed engagement in employees as directly linked to how much discretionary effort they were prepared to invest in their work. Furthermore, he concluded that an engaged employee felt emotionally connected to their company and had a solid understanding of how they felt for the brand they worked for. They were then able to use that knowledge to improve their performance and that of their organisation. This often results in more dedicated work and vitality within working environments.
Therefore, effort and awareness can be noted as key elements of engagement in work environments. One key element to consider in this research is that engagement research indicates that engagement comes in degrees or levels and that this can dictate the outcomes and actions of those who are affected. In social setting, low levels of engagement are indicative of people who are strangers or do not share common interests or benefits. High engagement on the other hand usually indicates common ambitions, views or beliefs but could also indicate an interest to find out more or become more involved in something that we were previously not participating in.
We see many examples of the degrees of engagement in all industries from marketing and business to social welfare and education. The common denominator for them all being that increase in engagement meant more results, and in most cases this was positive.
65 Chapter 4
In education, student engagement has been found to strongly correlate with learning outcomes. Research indicates that student engagement can be observed in three ways: • Behavioral • Cognitive • Emotional
4.2 Evaluation Methodology
This study will encompass various learning methods and existing approaches to education in understanding learning engagement the learning experience and their contribution to the learning outcomes. Some of the key approaches that will be considered in the study are outlined below.
4.3 Research Design and Rational
This research proposes to evaluate engagement in learning environments on the hypothesis that when learning environments are enhanced with immersive smart technology, it can lead to increase of students’ engagement levels and improved learning experiences. A mixed method indicated by Saunders et al. (2016) of both qualitative and quantitative research was undertaken. Qualitative research was in the form of interviews both with students and experts and quantitative research was in the form of questionnaires. The questionnaire was designed using the Likert scale and was made up of two dimensions, one for interest and engagement, made up of 11 Likert items and the other for the learning environment experience made up of 15 Likert items.
The study was to observed and surveyed students, at graduate level, across two UK universities, University of Southampton and Imperial College London, studying Marketing related fields.
The study was based on two simulation platforms using gamification elements: one was a PC based online marketing simulation activity program and the other an immersive VR Marketing Experience simulator designed by the researcher.
66 Chapter 4
As part of their studies students in both universities participated in the PC Simulation activity, that involves working in teams to simulate launching a product in a virtual market, over several decision rounds in teams. The learners work on laptops or PCs where the software that is leased is accessed via password by each team. They participate as a group in making changes to product lines, updating advertising, investing in research to progress as a team at the end of every round. Feedback is available after every round to give them a status update and indicate the result of their efforts. All activity takes place on the virtual situations dashboard which is interactive and allows students to make alterations to their strategy in a bid to improve their outcomes. The purpose of using simulation learning platforms is that students get hands-on experience on the many concepts that affect marketing and business activities that ultimately can make a big difference in achieving the company goals. The students are competing against their peers, and there are clear instructions given both on how the simulation works and the expectations prior to the start of the simulation, by the instructors that are facilitating this course.
A PC simulation platform was chosen as a base experiment to evaluate engagement levels as it is online and interactive and will allow for easier comparison when considering a VR enhanced immersive environment.
The VR Digital Marketing Experience simulator is a virtual reality environment that encouraged students to participate in daily activities in order to launch a new product with the aim to use given resources to launch the virtual product.
In both groups, students were observed as they participated in the simulation platform and surveyed once they completed.
The PC Simulation platform allows for a competitive approach where students compete in their teams against the other teams taking part. This approach is meant to encourage engagement and promote competitive spirit to improve your “game play” and achieve a better position with your team at the end of every round. This does have the desired effect for the first few rounds however once the stronger teams break ahead the ones that lag behind may find it difficult to catch up and this can result in some teams losing interest altogether in the simulation and in the effort they put into the next rounds. It is also with noting that, as with every team effort, some members of the team may
67 Chapter 4
participate more than others or be more engaged that others when it comes to activity and participation.
The VRLE platform is played solo however the other students can see what the participant is doing via the screen and can collaborate or guide their peer through the platform allowing for an inclusive experience. Furthermore, when a student is immersed in a VR learning platform they actively participate and immediately begin engaging with the content available to achieve the set tasks. As they are the ones experiencing the activity their levels of activity may be higher and this may enhance engagement and the learning experience.
Despite there being less traditional team work in the VRLE platform, there is engagement with the participating student and others in the class can learn from their actions and activities which may enhance their own learning experience when it is their turn to use the VR platform.
The questionnaire was informed by the literature review on student engagement and learning experiences covered in chapter two. The focus was to monitor their participation, learning, and engagement covering some key engagement metrics outlined below in Table 4.2. The questionnaire focuses on the students’ attitudes on the learning activities environment of each simulator of the project that they are participating in. Interviews then followed with individual students and in groups for a more detailed discussion on the learning experience.
As the aim of this study is to better understand the students’ engagement with various online education platforms and to then observe if there is an improvement in that engagement when virtual immersive technology is used as an interactive platform. The key focus was to monitor and considering how students engage with the platform content to achieve the objectives set out by the simulation platform. The study was looking for positive signs of engagement that indicate that the students are eager to participate and interested in the simulation activities as well as the outcomes. The researcher recorded the way in which students interact and engage with the simulation platform in both the virtual and the PC scenarios. Questionnaires were administered in
68 Chapter 4 person following each session. During the focus groups, written notes accompanied recording of responses on video using an iPad. All data will be used for the research purposes only and deleted following the end of the study.
4.3.1 Reliability, Validity and Expert reviews in Interviews and Surveys
The following types of evaluation will be used in the research as part of the methodology and are outlined below based on their relevant theories.
Benchmarking Experimenting Observational
User Observing Comparing
The aim is to gain insight into the way in which smart technologies can impact a learning environment and enhance engagement. The data collected through interviews and surveys should produce effective measures from which we can aim to answer the research question (Neuman, 2006). At times it is feasible when designing survey questionnaires to include pre-existing questions (Hyman et al, 2006) that have been extensively tested in other learning environments but may now reveal new outcomes in smart technology enhanced learning. Beimer and Lyberg (2003) highlight that respondents may, however, be influenced to answer a question differently based on the sequence of the previous questions to the one that interests this study, and it is therefore imperative that researchers have a clear understanding of the context in which the original questions were asked and what questions preceded them (Seale, 2004). Neuman (2006) refers to three main types of reliability when assessing questions, see table below:
69 Chapter 4
Stability Reliability The question or measure gives a consistent response at different points in time. An example of this would to repeat an experiment on the same subjects at a future date and correlate results with the initial test.
Representative Reliability or Internal The question gives a consistent response when Consistency Reliability asked to different subgroups or have the same characteristics. One method used is the Split-half method.
Equivalence Reliability The question or measure gives a consistent response across indicators of the same concept. This can also refer to cases where two observers study a single phenomenon simultaneously (Venkitachalam, 2015)
Table 4.1 Neuman’s Reliability Factors
70 Chapter 4
Venkitachalam (2015) outlines the following steps when designing questionnaires:
Figure 4.1 Adapted from Venkitachalam (2015) Steps in questionnaire
Venkitachalam (2015) further reflects that when considering the validity of a design we easure the degree to which the researcher has measured what they initially set out to measure (Smith, 1991), considering whether we are measuring what we think we are (Kerlinger, 1973), and the degree to which it adequately reflects the real meaning of the concept studyied. (Babbie, 1989). We consider three types of validity (DeVellis, 2002) Content validity – Use of logical reasoning – does the question represent a concept’s full definition, Criterion related validity is linked to the extent to which a question can predict behavior and includes predictive and concurrent validity and finally Construct validity which considers to which extent a question can measure a theoretical concept which it is designed to measure.
Two methods by which experts evaluate validity Venkitachalam (2015) are shown below:
71 Chapter 4
Method 1: Average Congruence Percentage (ACP) (Popham, 1978)
Here the experts compute the percentage of questions deemed to be relevant for them, then use the average of all experts. If the value is > 90 Valid. An example would be: 2 experts (Expert 1-100%, Expert 2-80%) Then ACP = 90%
Method 2: Content validity index (Martuza, 1977)
a. Content validity Index for individual items (I-CVI)
A panel of content experts asked to review the relevance of each question on a 4-point Likert scale (min. 3 max. 10 Experts) where: 1= not relevant, 2= somewhat relevant, 3= relevant, 4= very relevant.
Then for each question, number of experts giving 3 or 4 score is counted (3,4 – relevant; 1,2 – non-relevant). Proportion is then calculated. An example would be: If 4/5 experts give score 3 or 4: I-CVI = 0.80. It is also suggested however that considering expert multipoint evaluation in two categories (relevant and non-relevant) does not always yield results that imply the full depth of the questionnaire is covered and there is the problem of chance agreement. To overcome that, it is recommended Lynn (1986) that five or fewer experts who all agree (I-CVI = 1.0) and six or more: (I-CVI should not be less than 0.78)
b. Content Validity Index for the scale (S-CVI)
This method considers the proportion of items in a question that achieved a rating of 3 or 4 by all the content experts, and here we have two approaches: S-CVI/UA – Universal agreement is the proportion of items on the scale that achieves a ration of 3 or 4 by all the experts S-CVI/Ave – Average of the I-CVIs for all the items of the scale. There is general consensus in methodological literature that content validity is a matter of judgement that involves two phases (Polit and Beck, 2006)
Administration of questionnaires is commonly done in two modes; it is either administered by an interviewer or self-administered by the participant. In this study the
72 Chapter 4 questionnaires will be self-administered and will just require distribution by the researcher. Distribution occurred in person following each of the experiments.
4.3.2 Participant Profiles
Heuristic evaluation was developed by Nielsen and Molich in 1990; Nielsen 1994) which was used as a usability engineering method in finding issues with interface design. The method includes using a small set of evaluators to examine and interface critic it against known usability principles or “heuristics”. (Nielsen, 1995) In his methodology Jacob Nielsen (1994) included the element of discounted experts review.
In determining the number of evaluators that you need to identify problems in the interface as found by heuristic evaluation based on an average of six case studies he found that single evaluator could determine only 35% of the issues. However, as each evaluator found different issues it was possible to get a better outcome by combining the evaluations from several different inspectors. In figure 4.2 we can see the proportion of issues found as more evaluators are added and can determine that most issues are found using between six and eight experts. This research study will be getting expert advice to inform the framework from seven experts in the relevant fields. The methodology for this
will be one to one scenario based interviews.
Figure 4.2 Jakob Nielsen graph (1995)
The study will also be approaching practitioners from the academic and emerging technology industry and to validate the research framework and provide feedback on the current state of the industry so as to inform the study further. This calculation will determine the sample size needed.
73 Chapter 4
For the Student sample size, the study will be using the parallel group design to a cross over design for the following advantages (Amin, 2015): • Each student group experienced different learning environments. One group will experience just the PC simulator and the other both the PC and the VR Simulator. The PC simulator group will be noted as the base group or (B) and the VR simulator group will be noted as group (V) • The groups were large is size varying from 30 for the VR group to 73 for the base group however the base group data was randomly selected to 30 for use in the statistical test. • The experiment will be for a short duration and run in same academic year. • Simple analysis of results can be computed and presented
The parallel design (Moher, 1994) calculated the sample size of a study where we are comparing two groups using different trials.
The following number of participants will be surveyed:
4.3.3 Expert Reviews
Demographic: Six to eight experts in line were interviewed in line with Nielsen’s Discounted Expert Reviews in order to validate the framework. Experts have been identified in the field of Smart Technologies with recognised industry influence and research in the field. They are based both in the United Kingdom and international locations. (United States, Norway, Sweden, Greece) and were interviewed on a one to one basis using conference calls. Notes were taken during the in depth interviews that lasted on average an hour.
4.3.4 Student Groups
Demographic: 30 – 70 students in line with Parallel Group design. Groups of students from two universities (Southampton University and Imperial College London) participating in online simulator gaming platforms as part of an interactive learning environment.
74 Chapter 4
Some of the students identified in Group B (base) will have experienced the PC Simulator only and students in Group V (virtual) will have experienced both the PC Simulator and the Virtual Marketing Simulator. The students are part of relevant Marketing courses at postgraduate level at their respective universities. The students at Southampton University and Imperial undertake the PC Simulation as part of their practical application for their Marketing degree. The PC Simulation program is used as part of the curriculum to cover hands on application of marketing elements. The participants are all over 18 and from various different backgrounds. The groups include male and female participants. This will not be a factor in the analysis of the engagement levels as it is not relevant to this study.
Students will be observed as they participate in the online environment and then surveyed for feedback on the experience. Focus groups and individual interviews will be held following each experiment to discuss the outcomes of the experience and evaluate the metrics outlined in Table 4.3 below.
4.3.5 Documentary Sources
The following additional sources of documentation will enrich the methodology. As this is a new and emerging field, there is little in the way of application smart virtual education environments and their application in creating immersive experiences, and therefore the researcher will rely on further documentation to reinforce and integrate the findings of the research with outcomes found in several online sources. Examples of sources include documentation and case studies from immersive environment developers who are pioneers in the field, presentations, podcasts, publications, webinars, infographics and lectures.
75 Chapter 4
4.4 Analysis of Findings
The study will follow a triangulation of data gathering using various documents, interviews and observation for analysis. The researcher will use Statistical Analysis via Jamovi, a Statistics Program based on R, to determine the results and outcomes of the surveys.
Figure 4.3 Data gathering triangulation
There are multiple elements that make using augmented virtual reality systems difficult to apply to conventional evaluation methods (Hale and Stanney, 2002). Similarly, research in learning environments that link in IoT data and smart technologies are still in progress and there are limited definitive results to benchmark against.
76 Chapter 4
4.5 Evaluation and Engagement Metrics
Cognitive (C) Theory and Method of Behavioural(B) Engagement Metric Thought Metric Questions Measurement Emotional (E) Leadership How much effort did Effort and Constructivism – Questionnaire C, B the student feel they investment in Discovery Learning Observation needed to put into the learning (Bruner) Discussion task/ activity? Experiential Learning (Kolb) Did the task have the Questionnaire C,B,E Attention Levels of student’s full Observation Engagement attention? (Schlechty) Paradigm - Design- Based Research Did the student feel Questionnaire Commitment and C,B,E Levels of committed to Observation Motivation Engagement completing the task? Discussion (Schlechty) Experiential Did the student see Questionnaire Perceived Value C,E Learning (Kolb) value in the task they Observation
(Elmore) were undertaking? Discussion Constructivism – Associated What did the student Questionnaire C,E Discovery Learning Outcomes think they achieved? (Bruner) To what degree did Authenticity and the task feel real and Affirmation related to real life Experiential Questionnaire C,E (Realism and situations and did the Learning (Kolb) Discussion Verification) student feel authentically engaged? Did the student want (Fredricks, to continue working Questionnaire Challenge and C,B Blumenfeld, and on the task despite Observation Persistence Paris) any challenges using Discussion deep strategies? Did the student feel a Questionnaire Experiential sense of E Accomplishment Observation Learning (Kolb) accomplishment Discussion completing the task? Did the student enjoy Experiential the activity and did E Enjoyment and Observation Learning (Kolb) they feel that Attachment Discussion (Connell and Finn) enjoyment had duration? Memorability / Multiple Was the material in Questionnaire Richness / Intelligences C the task memorable, Observation Relevance of Theory (Gardner) relatable and realistic? Discussion materials E-Learning Theory Was the content in the C Integration (Mayer) simulation well Discussion integrated? To what extent did the Motivational student feel you were C Deep Learning Learning constructing their own Discussion (Schiefele) strategies and observe the outcomes Table 4.2 Evaluation and Engagement Metrics
77 Chapter 4
4.6 The Virtual Learning Experience Experiment
For the purpose of this paper a virtual reality educational platform was created. The concept was to develop a platform that would allow students studying Marketing or Digital Marketing visualise the activities involved in undertaking marketing campaigns and the elements that could be considered when doing so. The outcomes of their efforts can be seen in the virtual environment in real time. This is similar to other Marketing computer simulations that are action driven.
The simulation platform was created using the VR development tool, Unity and could be accessed via a laptop and HTC Vive VR Headset. Student initially log into the platform with a pseudo name and choose the name of a pseudo brand. They then enter into their ‘virtual office’ and their day begins by means of clock on the wall which indicates 9am as the start of the day. They are then able to interact with content that is in the virtual environment in order to launch a new product. They can choose the features of the product or the appearance and see the visual depiction in the virtual world. The simulation also gives students a set amount of funds that can be used towards hiring advertising, buying further research to inform your decisions, research industry trends, competitors and considering corporate social responsibility. The outcome of those decisions can be seen immediately in the virtual world. Choosing the correct features for the product based on research findings will result in more customers at the store across the road buying your product. The billboards in the streets light up with your advertising and if you have chosen greener options these will in future impact the environment outside your virtual office. The students’ ability to see the potential outcomes of their efforts both in how the product turns out and the it is advertised aims to incite more engagement with Marketing students who understand the concepts but can now apply them. Visuals of the experiment can be found below:
78 Chapter 4
Figure 1 Interactive Office Desk Figure 2 Product showcased as Mobile device Figure 3 Insights Portals – Industry Research – Competitors – Advertising – Advisors Figure 4 Monitoring Board with summary of funds Figure 5 Features for product – AR - VR – NFC Figure 6 Analysis Results commissioned Figure 7 External Environment showing technologies applied Figure 8 Visual of Advertising on the digital billboards Figure 9 End of simulation period
Figure 4.4 Images of VR Simulation platform and legend
79 Chapter 4
The content of the VRLE activity was linked directly to the content taught in the lectures and asked the students to use research analysis and trends information to enhance their product features. An example of this would be choosing to undertake research in order to know what emerging technologies you should apply to the new product (in this case a mobile device) you are launching. These technologies would then be available to you to apply to your new product and added to any advertising when promoting the features of your new device. The importance of developing using market research and listening to customer needs was primary in the content taught. Seeing the results of applying this insight helped to reinforce the significance of these elements in the real world when they go into employment.
4.7 Summary
A virtual reality marketing simulator environment was used to assess engagement levels while considering the progressive methodology offered by several schools of thought in learning design and education. The key focus of this research will be to ensure that there is a combination of the creative and interactive element of the design with evidence of real engagement and response to each environment. Due to the nature of this study, the key principles that will validate engagement will be accessed via triangulation method outlined above. The reasoning behind using immersive experiences for experiential and constructive learning links with the literature review findings.
80 Chapter 5
Chapter 5. Analysis of Results
5.1 Overview
This chapter will outline the results of the Likert scale questionnaire for the two groups and showcase the analysis of the outcomes. That will be followed by an analysis of the qualitative research conducted through interviews and observations before drawing conclusions in the final chapter. As highlighted in the design the questionnaire was divided into two Likert Scales. Eleven Likert Items relating to engagement and interest are noted in the data as (IE) and fifteen Likert items relating to the learning environment are noted as (LE). There are two questions in the survey that are negative. One in each of the dimensions. These question scores have been reversed to transform them into positively framed for the purposes of analysis. The questions were put into the questionnaire to test random ticking off of responses.
Jamovi statistical software was used to conduct inferential analysis on the data from the experiment.
Using the Central Limit Theorem based on the number of participants (n ≥ 30) to compute the mean (μ) and standard deviation (σ) we plot the frequency of the means to establish a normal distribution.
The sample selection criteria were based on the following factors: whether the students had participated in the same PC Simulator and VRLE at a university in the United Kingdom and were in graduate level studies in Marketing or related fields. The participants from the group that had undertaken the activities on the PC simulator only were studying on a Master’s program at a university in the United Kingdom. The participants from the group that had experienced both the PC Simulator and the VRLE and were students on a Master’s program at another university of similar standing in the United Kingdom. Both groups were studying Marketing or Marketing related fields.
The Mann-Whitney U Test was used as a statistical test as we are dealing with Likert scales and ordinal data therefore having no true mean or standard deviation that can be
81 Chapter 5
calculated. This nonparametric test is also ideal as we have considering the difference between two student groups without assuming that the values are normally distributed.
The Wilcoxon test was not deemed appropriate for this study as the same people were not tested twice in alternative conditions. The Kruskal -Wallis test was also rejected as we do not have three or more independent groups for this analysis.
Considering Fisher’s significance testing we will consider a Null hypothesis (H01 and H02) to answer each of research hypothesis (H1 and H2) below attempting to establish a p- value as the strength of evidence against the null hypothesis P < 0.05 (5% significance) (Fisher, 1930):
H1. The use of smart technologies in virtual learning environments can improve engagement.
H₀ 1. There is no change in the engagement levels when in a virtual learning environment
H2. Smart technologies will improve learning experiences.
H₀ 2. There will be no evidence of improvement in the learning experience.
The results of the base PC experiment and the VR experiment we will either accept or reject null hypothesis 1 and or null hypothesis 2.
In consolidation for an alternative hypothesis (HA) we considered for the purposes of this research the following:
HA1. There is a change in engagement level in the VR learning environment.
HA2. There is an improvement in the learning experience levels.
The data was ranked with 5 = Strongly agree on the high end of the scale and 1 as Strongly disagree. The mid rank 3 was for uncertain or not relevant. Responses left blank were also given a rank 3 as it was assumed that the student did not know what response to give to that question.
82 Chapter 5
We then considered the differences in the ranking between the PC learning environment group or base group and our VR learning environment group. If there is a difference in rank between the two groups we will consider the alternative hypotheses to be true. If there is no difference in the distribution of the ranking between the groups we will consider the null hypothesis to be true. Below is the formula showcasing that the null hypothesis assets that the medians of the samples are identical or match closely.
The alpha level is set for the p- value at p ≤ 0.05. The critical value of U at p ≤ 0.05. The formula for the Critical value of U for a given significance level is noted as:
U critical = µ - z * σ - 0.5 where μ is the mean or average, in this case we will use the median, σ is the standard deviation and z is the desired significance level. U is therefore significant if it is less than or equal to U critical. Thus for this study with 0.05 significance level we will use z =1.96 as this is a two tailed test. Consequently, we will assess the U value, z- value and p- value for every question of the survey to note significance for every Likert item and in addition the median results for both PC simulator and VR Simulator were used to note if there is any significant difference. The results are shown below:
Table 5.1 Engagement Results with z-scores and p values
Table 5.2 Learning Environment Results with z-scores and p values
83 Chapter 5
We note from table 5.1 that the median shows a significant result for with a p value of 0.00132. We can therefore reject null hypothesis 1 (H₀ 1) as there is a change in the engagement levels when the students were using the VR Environment. Furthermore, when considering the results in table 5.2 we note that the median shows a significant result with a p value of 0.00424. We can therefore reject null hypothesis 2 (H₀ 2) as there is evidence of improvement in the learning experience when the students were using the
VR Environment.
Paired Samples T-Test
statistic df p Cohen's d
IE Median PC IE Median VR Student's t -4.29 29.0 < .001 -0.783
LE Median PC LE Median VR Student's t -3.62 29.0 0.001 -0.660
Table 5.3 and Table 5.4 Box Visual of LE Median PC - LE Median VR Plots
The results range and frequency count can be noted in (table 5.4) below. The observation indicates that on the virtual environment there were significantly more high scores of 5 across both the engagement items of the questionnaire and the learning environment
84 Chapter 5 items. There were 13 items that scored a strongly agree in the immersive environment for engagement where the computer simulator score just 1 strongly agree.
Binomial Test
Level Count Total Proportion p
IE Median VR 3 1 30 0.033 < .001
4 16 30 0.533 0.856
5 13 30 0.433 0.585
LE Median VR 3 2 30 0.067 < .001
4 14 30 0.467 0.856
5 14 30 0.467 0.856
IE Median PC 2 2 30 0.067 < .001
3 4 30 0.133 < .001
4 23 30 0.767 0.005
5 1 30 0.033 < .001
LE Median PC 2 2 30 0.067 < .001
3 3 30 0.100 < .001
4 23 30 0.767 0.005
5 2 30 0.067 < .001
Note. Hₐ is proportion ≠ 0.5
Table 5.5 Binomial Test for level counts
Similarly, we note that there were 14 items that scored strongly agree on the VR platform and only 2 on the PC simulation. It should be noted however that there were significant number of responses that were at level 4 showcasing that students appreciate these
85 Chapter 5
simulations as part of their learning experience but clearly indicate that adding the immersive element would improve the experience further to the next level.
Based on the results of the study we can reject the null hypotheses and accept the alternate hypotheses as significant.
5.2 Qualitative Analysis Observations
Students on both platforms showed engagement and were actively participating in the learning experience. Both platforms gave the student hands on interaction and they were keen to learn how to use them and how to achieve the set goals. Students on the pc simulation were in teams so there was significant team work while undertaking tasks. The VR simulator did not have team interaction built in yet however students were able to see what their peers were doing while in the VR environment as the screen on the VR laptop was shared. Students were able to talk and engage with the researcher and the other students while in the VR platform.
Most students had not experienced a virtual online platform before and the initial reactions were those of awe and surprise at the immersive elements. There was an initial period required to explain what they needed to do but the researcher attempted to keep instruction to the minimum allowing students to explore the platform and learn by experiencing each element.
The students working on the pc simulator were also learning how to progress through the content by trial and error at the start but as they got their initial feedback they were able to start to develop clearer strategies on what they needed to do. Students on the VR platform continued to learn by experience. Every action they took had an active result and they were able to immediately see the results of those actions in the virtual world. For example, when a type of advertising was chosen would then populate the billboards and digital displays outside the virtual office. There resources were also monitored. It was observed that none of the students taking part in the VR environment stated that they found the platform uninspiring and they were all eager to take part and engage with the
86 Chapter 5 virtual materials. Students used the online content well discovering ways to apply they different options and building up their marketing strategy. Several of the students had one to one discussions with the research about their experience and the outcomes of those interviews will be outlined in the next section.
5.3 Analysis of Qualitative Research
Thematic Analysis was used to analyse the qualitative data in this study (Braun & Clarke. 2006) where patterns of meaning and common themes emerged. This type of analysis is considered a suitable approach when conducting research that involves views opinions but importantly for this study, when expressing a view on an experience. When interviewing the students, we were looking for phrases that expressed their feelings towards the learning environment and analysed this by grouping similar phrases under themes of positive and less positive comments, an example of this is seen below in fig 5.6.
The impression that was expressed by the students taking part in both the pc simulator and the VR experiment in their interviews with the research was that the virtual platform was a new and engaging experience that made them want to do more and discover more on the platform. A common statement among the participants was that they felt in control of the experience. They felt they were able to interact more with the content and found it valuable that results were immediately evident as a result of their efforts. When asked whether they would want to experience more of the virtual platform all the students suggested that they would. There was a positive approach to how easy it was to understand the workings of the virtual platform and how quickly they were able to start to do things. Students expressed views that this was a more engaging and interactive way of learning. They confirmed that they enjoyed the virtual experience stating that the content was memorable and the experience would remain vivid due to the impact of the immersive environment. From a practical perspective it was clear from the discussions that students found the platform engaging and reiterated that they would be keen to try future versions of the platform that offered more content. Based on the qualitative research we were able to determine that students felt that the VR platform was engaging and added value to their learning experience. Student feedback is noted in the table below:
87 Chapter 5
PC Simulator VR Simulator Analysis
Positive • “It was interesting • “That was Use of hyperbole language was comments and I enjoyed amazing, I wish all common in the VR Simulator about the working on it.” lessons were like feedback. There were often platforms • “I liked the way we that.” comments on how they had never tried to get our • “I loved that. experienced anything like that product to the top Seeing the actions before. score every week.” I was doing in The comments on the PC practice made me Simulator were positive but the want to do more.” tone and response was more moderate. Less • “It was boring • “Wish there was The responses to the PC simulator positive after the first few more to do” were linked to a progressive lack Comments rounds” • “It was a little of interest, whereas the VR about the • “It was repetitive” scary at first” simulator experience indicated platforms progressive increase in interest.
Table 5.6 Student response examples
5.4 Summary of Sentiment Analysis across Engagement Metrics
A sentiment analysis was drawn from the questionnaires, observations and discussions with students. Positive sentiment was attributed where on average remarks were strongly in favour and contained frequent use of “strongly agree” comments in the feedback. In addition, where students used hyperbole and where no negative comments were attributed to the simulation. Neutral sentiment was attributed where on average comments were a mix of negative and positive or where comments were neutral. Example of positive sentiment include statement comments of “It was amazing” or “Wow that was great” and an example of neutral sentiment would be comments of “It was alright” or “It felt outdated”
Finally, the sentiment of Mixed is given where there were both positive and negative comments. An example of such sentiment would be in comments such as “It was good at first but then got boring”. It should be noted that observations included monitoring and recording the students’ behaviour as they took part in each of the experiments.
88 Chapter 5
Sentiment Sentiment Engagement Metric Metric Questions Analysis Analysis PC VR
How much effort did the Effort and investment student feel they needed to Positive Positive in learning put into the task/ activity?
Did the task have the Attention Mixed Positive student’s full attention?
Did the student feel Commitment and committed to completing Neutral Positive Motivation the task?
Did the student see value in Perceived Value the task they were Neutral Positive
undertaking?
What did the student think Associated Outcomes Positive Positive they achieved?
To what degree did the task Authenticity and feel real and related to real Affirmation (Realism life situations and did the Neutral Positive and Verification) student feel authentically
engaged?
Did the student want to Challenge and continue working on the Neutral Positive Persistence task despite any challenges using deep strategies? Did the student feel a sense Accomplishment of accomplishment Neutral Positive completing the task? Did the student enjoy the Enjoyment and activity and did they feel Neutral Positive Attachment that enjoyment had duration? Memorability / Was the material in the task Richness / Relevance memorable, relatable and Neutral Positive of materials realistic?
Was the content in the Integration Positive Neutral simulation well integrated?
To what extent did the student feel you were Deep Learning constructing their own Neutral Positive strategies and observe the outcomes
Table 5.7 Student sentiment analysis against metrics
89 Chapter 5
5.5 Expert Review Analysis
Industry professionals and academics alike were involved through discussions and interviews in informing the framework and the experiments. The key direction that was repeated in many of these discussions was a need for change in the education sector, both in the way that we present the content and the opportunities for new learning environments.
The general consensus among the experts was that the process should indeed be gradual as showcased in the framework. It was evident that before we can offer fully functioning smart learning environments we first need to build a solid basis and infrastructure through which these technologies can be adapted and adopted. Virtual Learning Environments are a good follow up point for these initiatives for institutions that already have sound ICT and offer blended learning options. This is the criteria both the universities that were involved in the experiments fit.
The feedback also highlighted the need for students to be prepared in the right way for what is to follow in their careers. For many of these young people technology will be a key part of their professional career path and giving them a good foundation of learning through immersive experiences will help them apply this knowhow in their future careers too.
An important aspect of creating immersive learning experiences to enhance engagement is the ability to give students a way to personalise their learning. Comments from academics that informed this research emphasised the significance of learning through doing.
A VR specialist and university academic stated that the unique opportunity posed by VR immersive experiences is that we can use them to test outcomes and see the results for experiential learning in action in a virtual environment, where it is not possible in the real world. A student can therefore build their own learning experience and see the outcomes of that experience in the virtual world allowing them to make changes and updates before they have to do the same as professionals in their career.
90 Chapter 5
The Academic Director of a prominent University Group in London gave the following statement as a summary review of the framework:
“The presented smart learning framework incorporates ontological and epistemological aspects, and has successfully amalgamated such concepts with technological advancements in the areas of virtual learning, smart spaces, simulations and associated fields. The framework looks robust and comprehensive, as it demonstrates a diverse and technology based approach to human learning.”
“The framework combines some key elements of digital and virtual environments, making it suitable for differentiated learning—stimulating interest, enhancing cognitive ability and enhancing accessibility for disadvantaged sectors of society.”
“I can see this working to improve the student experience as the framework has clearly incorporated concepts such as machine learning, modular approach to learning, virtual reality and the internet of things; all integrated in to an executable framework.”
“I do not see any major factors that are missing from the framework, but it may be worthwhile exploring ‘big data’ and `data visualisation’ as a concept to integrate to the framework.”
Dr. A.M., a Virtual Reality and Digital Technologies Specialist with expertise in designing and developing mixed reality experiences and with 10 years teaching experience at Higher Education level noted the following about the proposed framework in this thesis:
"The framework proposed represents an excellent starting point for new and more engaging teaching experience. The education system must embrace new technologies, mainly Immersive Technologies, to allow students to practically experience what the theory cannot explain. This framework well balances the new technologies with the contents to deliver. I hope this will be applied in real classrooms."
91 Chapter 5
5.6 Summary
The research covered both qualitative and quantitative analysis (mixed method approach). The findings outlined in this chapter outline the evidence that students showed more enthusiasm when immersed in a virtual learning educational environment which was noted both in observations and in the quantitative analysis. It is noted that experts of the academic and technology sphere agree that there is significant potential in exploring and applying smart emerging technology to improve the student experience.
92 Chapter 6
Chapter 6. Conclusions and Future work
6.1 Academic Contribution
This thesis presents a number of useful contributions to the field of emerging technologies in the role they play in enhancing the learning experience. The literature review contributed in collating the views and theories of several authors in what constitutes a sound learning environment and how engagement and learning and intertwined in creating a valuable learning experience. As the learning experience is a significant part of how we learn it was important to develop an alternative experience through a virtual platform in order to attempt to explore the levels of engagement of students in higher education. The findings of this research highlight the key areas where student engagement is evident and showcases the value that students place on the learning experience as part of the learning process.
When considering the factors that would contribute towards such a learning environment it was important to highlight the layers of application that would yield the final enhanced learning experience as presented in figure 3.1 where technology supported learning is enhanced initially by Virtual Simulations and finally linked in a fully smart enhanced learning experience with the Internet of Things (IoT).
The Framework proposed in Chapter 3 (fig. 3.2) illustrates the required elements for building a smart immersive learning environment. This is then used to further showcase an example of the framework when implemented (fig 3.3) to improve engagement and the learning experience and outlines the key links between each component and the opportunities and outcomes. This can be used by academics and learning institutions to create smart learning environments that can improve engagement and the learning experience.
The value of using an immersive learning experience to enhance the student journey is demonstrated through the responses both in interviews and through the questionnaire by students who had the opportunity to experience both the PC simulation and the VR Simulation. Furthermore, the framework presented in this research shows the elements
93 Chapter 6
that would make up a robust smart immersive learning environment and gives instructors and universities a basis from which to start to create these environments with an aim to improve engagement and the learning experience.
Finally, the use of an immersive platform proved to improve the levels of engagement and improve the learning experience giving further support to constructivism and experiential learning theories. Experts in in Emerging Technologies and Academics agree that how we use these digital technologies will determine the future of our society. How we educate is part of that journey. Global Vice President and Academic of a leading technology organisation, Dr. Phil Dervan stated the following testament which is in line with the framework outlined in this thesis:
“Learning needs to be understood as value creation as new data and new events create opportunities for insight and innovation. The post-industrial workplace nurtures learning as a core organizational process. Learning is an ongoing process that takes place both during and after someone’s professional career. Exploring different ways to aid learning and in context, with the use of smart technologies is a positive way forward and not just contained the classroom.”
This thesis contributes a map of components towards exploring new ways to support learning through the use of smart technologies that are mentioned above by Dr. Dervan and echoed by other academics and technologists.
6.2 Practical Contribution
The VLE used as part of this research was built using Unity. It is a cross platform game engine that is developed by Unity Technologies and available free (for revenue less than $100,000) and at a low cost of $15 a month for Learn Premium which offers the support of Unity certified instructors, on demand content and resources for creators (Unity, 2019). Business plans are also available for larger enterprises. The engine supports more than 25 platforms (2018) making it accessible to anyone wishing to create content or build learning environments using this engine. Unity also has on demand resources that
94 Chapter 6 can be tailored for different purposes allowing for a variety of scenarios to be created and for engaging environments to be built. There are live training workshops and certifications available to instructors and educators both in VR and in person. (Unity, 2019) There is direct and immediate guidance from tech leads and developers available for anyone, even with limited experience, to use the free resources available to begin creating online content for VR. This study can be replicated by any educator using Unity.
The platform created for this program was presented through Steam and was available using a VR ready MSI laptop computer and HTC Vive VR headset.
6.3 Educational Opportunities
Through the use of worldwide university partnerships already in place across the globe, there is a clear opportunity offered by immersive virtual environments to create new avenues for sharing content and interaction with no boundaries across disciplines and research fields. Interaction on virtual platforms can expand to all levels of education creating a seamless network of academic support through one’s education. Immersive learning platforms through the use of extended reality, as technology merges the physical and virtual world, could offer thousands of students the opportunity to engage and interact with virtual content that they can use to construct new learning materials for themselves and others, practice and revise existing concepts and discover new ideas and perspectives. Using virtual platforms to gain a better understanding of the taught subject can enhance the learning experience and drive up engagement levels. Immersive learning can also allow for the physical classroom to be tailored with more smart technologies to assist in monitoring and optimising of the student journey. The journey is not limited to the classroom space as student interaction with the VLE and VRLE and applications within it can continue when the learner is outside the class too. The devices they are connected to will also continue to monitor the learning journey and link results of the university VLE account for each students giving additional personalisation of the student journey and support. It is important to note that careful consideration should always be given to permissions and data security.
This research proposes the use of immersive learning environments and smart technology to improvement in the learning experience. Further interviews with experts from the
95 Chapter 6
technology industry and academics is proposed with the technology industry and educators in Higher Education to establish the kind of content that can be developed initially as we start to venture into virtual environments. An opportunity in smart virtual learning environments for higher education lies in the ability for students to develop memorable independent learning skills. This will allow for constructive education methods and give new avenues for experiential learning. Furthermore, a unique opportunity lies in students being able to see the results of their efforts on the virtual platform. This gives them hands on experience in what some possible results can be to decisions they make while still within the safety of the virtual environment.
6.4 Limitations of research
Technology developments through the internet have allowed the implementation and application of many new and innovative approach to learning effectively applied to virtual simulators which are now expanding into virtual reality worlds. These environments, however, work with information that is built in at the development stages and although they are ‘reality’ mirrored in a virtual world, the data and information they hold are only as current as the launch dates or the last update. For the environment to reach its full potential, it should be dynamic, changing with the flow of new data that comes in from the real word to enrich the learning environment. Although a VR Environment Simulator was built for the purposes of this study, it was not a full model of the platform and offered only a selection of actions and activities to undertake compared to the PC Simulator. There is potential for smart integration to bring it closer to a fully functioning VR marketing simulator.
Furthermore, the degree of engagement through monitoring levels of physical activity, personalisation to learning methods preferred by the student, and access to the learning environments outside of the classroom are still not utilised. These elements can only be made possible with Smart Technology implemented in learning environments to create dynamic, and immersive, learning experiences. As the paper focused on engagement levels in immersive environments it did not examine the depth of the knowledge obtained nor the extent to which it was retained.
96 Chapter 6
This thesis focused on higher education students. The potential for smart education learning experience in primary and secondary education was not explored in this study and the opportunity may exist for future research in this area.
Finally, it should be noted that due to the speed at which these technologies change, and although every effort has been to seek the latest information, there may be changes in the technology that the researcher may not be aware of.
6.5 Future Work
There are avenues now open to explore emerging technologies and how they can contribute to the learning experience. The opportunity is there as we face the 4th industrial revolution. Due to the nature of new technologies, that evolve at faster rates than other areas of development, it would be sensible for research to continue into these areas and ensure that the findings presented here continue to be relevant in future years. As technologies evolve so do the people who use them and this evolution is now move evident in millennials who rely on technology for their way of life (Gartner, 2018). Young generations that have grown up with computers and laptops seek new ways of engagement and soon virtual platforms will be used every day. They know and understand new technologies because they have learnt to adapt to the changes that they offer. As academics we should ensure that our institutions also follow suit and that these smart technologies and the opportunities that they offer can be utilised and applied to create a more engaging and immersive learning experience for all students.
As highlighted in the introduction and literature review the element that makes this revolution different from previous ones is that we now have several technologies that can all work together and benefit each other and the community. We see the evolution of smart cities when IoT and AI work in unison with Blockchain technologies and Extended Reality. This can be applied to the academic environment where these technologies can work as one to offer learning environments that update in real time through real time IoT Analytics, to inform immersive environments and update content on the go. Sensors can monitor and adapt learning content for experiences that are built around the needs of the individual needs of the student, by the student allowing them to develop and construct learning objectives that suit their learning approach.
97 Chapter 6
Virtual platforms would also allow collaboration across several universities giving students first hand access to experts, academics and peers from a global community for more opportunities to share ideas and experiences.
Although this study approached the element of engagement and the learning experience there will need to be further study into security and personal safety in online immersive experience especially when these platforms are to be open to a global community.
A further area of exploration is the gaming aspect that drives motivation with these smart environments. Although both the platforms in this study has educational content they were both designed around gamification concepts. There is therefore an opportunity to look deeper into the factors that drive encouragement and improve the experience and how these are inked to gaming elements.
Another consideration for further research is to consider whether VR immersive platforms suit every field of study and different levels. This research used Marketing students as participants to experience the VR environment by allowing them to visualise the product, activities and outcomes of their efforts. However, could this work for other disciplines too such as engineering or art? We already know it is being used in the medicine industry based on the examples given in the literature review. Furthermore, could students from other levels benefit from virtual environments. Students in high school for example.
It should be highlighted that there are ground breaking initiatives into the development of virtual learning platform as noted in this thesis and these initiatives and any future efforts should be supported so they can develop to facilitate the growing need that there may be in education.
98 Chapter 6
6.6 Concluding remarks
We are at the start of the 4th industrial revolution and smart technologies are soon to be common ground in all industries and part of everyday life. The difference with this revolution is that we will have multiple technologies working together towards common goals. Artificial Intelligence analysing Big Data captured from IoT enabled smart devices to feed into Virtual and Augmented platforms and interactions enabling Extended Reality.
We can use and utilise these technologies to create immersive learning experiences that will improve engagement and enhance the learning experience. We do not live in 2D world and we do not need to teach or learn using only 2D content. The opportunity now exists to bring emerging technologies into education.
The opportunities for Marketing students in higher education are limitless from demonstrating product design and branding using Extended Reality (EX) to visualising the outcomes of digital campaigns in Virtual Reality (VR) platforms.
Applying IoT Technology through connected devices and IoT analytics to capture Big Data that we can then analyse and optimise using Artificial Intelligence (AI) in order to improve practices, personalise content and predict educational needs.
99 Bibliography
Bibliography
Anderson, P.H. and Lawton, L. (2009). Business simulations and cognitive learning: Developments, desires and future directions. Simulation & Gaming 40(2): 193-216.
Anderson, P. H. and Woodhouse, R.H. (1984). The perceived relationship between pedagogies and attaining objectives in the business policy course. Developments in Business Simulations and Experiential Exercises 11: 152-156.
Annetta, L.A. (2009). ‘Video games in education: Why they should be used and how they are being used’, Theory Into Practice, 47(3), pp. 229–239
Atzori, L., Iera, A. and Morabito, G., (2010). The internet of things: A survey. Computer networks, 54(15), pp.2787-2805.
Average Connections, Akamai (2016). State of the Internet Connectivity visualization. Available at: https://www.akamai.com/uk/en/our-thinking/state-of-the-internet- report/state-of-the-internet-connectivity-visualization.jsp (Accessed: 02 June 2016).
Australian Council for Educational Research. (2010). Australasian Survey of Student Engagement. Camberwell: ACER.
Auto-ID labs (2015). Available at: http://autoidlabs.org/wordpress_website/ (Accessed: 25 April 2016).
Auto-ID Center (2002). Technology Guide. AutoID Center, Massachusetts Institute of Technology. Available at http://www.autoidcenter.org/new_media/ brochures/Technology_Guide.pdf
A Brief History of NSF and The Internet." Nsf.gov. N.p., 1989. Web. 19 Mar. 2017.
A Short History of the Web | CERN." Home.cern. N.p., 2019. Web. 23 Mar. 2017.
Babb, E.M., Leslie, M.A. and Van Slyke, M.D. (1966). The potential of business-gaming methods in research. Journal of Business 39(4): 465-472.
Bassey, M. (2007) Case Studies. In Briggs ARJ & Coleman M (Eds.) research methods in educational leadership and management. Sage, London, 142-155.
Berners-Lee, T., Fischetti, M. (1999) Weaving the Web: The Original Design and Ultimate Destiny of the World Wide Web, Harper Collins, New York
Braun V., and Clarke, V. (2006) Using thematic analysis in psychology, Qualitative Research in Psychology, 3:2, 77-101, DOI: 10.1191/1478088706qp063oa
100 Bibliography
Brophy, J. (1983). Conceptualizing student motivation. Educational Psychologist, 18, 200– 215.
Bruner, J. S. (1961). The act of discovery. Harvard educational review.
Bruner, J. S. (2009). The process of education. Harvard University Press.
Blythe, S.E. and Gosenpud, J.J. (1981). A relative evaluation of experiential and simulation learning in terms of perceptions of effected changes in students. Developments in Business Simulations and Experiential Exercises 8: 145-148.
Berger, M., Wrobel-Dautcourt, B., Petitjean, S., Simon, G. (1999). Mixing synthetic and video images of an outdoor urban environment. Machine Vision Applications, 11(3):145– 159,
Bednar, A. K., Cunningham, D. J., Duffy, T. M., & Perry, J. D. (1992). Theory into practice: How do we link? In T. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction (pp. 17-34). Hillsdale, NJ: Lawrence Erlbaum Associates
Brenenstuhl, D. (1975). An experiential study of performance in a basic management course. Simulation Games and Experiential Learning in Action 2: 83-91.
Bodner, G.M. (1986). Constructivism: A theory of knowledge. Journal of Chemical Education,63(10), 873-878.
Bowers, B., (2001). Sir Charles Wheatstone FRS: 1802–1875 (2nd ed.). IET. pp. 207–208.
Bryson, C. (Ed.). (2014). Understanding and Developing Student Engagement. London: Routledge, https://doi.org/10.4324/9781315813691
Burdea Grigore, C. and Coiffet, P., 1994. Virtual reality technology. London: Wiley- Interscience.
Carnivale, D. & J. R. Young. (2006). The challenges and benefits of requiring students to buy laptops. The Chronicle of Higher Education, May 26. Accessed from http://chronicle.com/article/The-ChallengesBenefits/2829/
CERN (2016). The birth of the web. Available at: https://home.cern/topics/birth-web (Accessed: 27 April 2016).
Chang, V. & Guetl, C. (2010). Generation Y Learning in the 21st Century: Integration of Virtual Worlds and Cloud Computing Services. In Z. Abas, I. Jung & J. Luca (Eds.), Proceedings of Global Learn 2010 (pp. 1888-1897). Association for the Advancement of Computing in Education (AACE).
101 Bibliography
Chen, M., Wan, J., Gonzalez, S., Liao, X. and Leung, V. C. M., (2014). “A Survey of Recent Developments in Home M2M Networks”, IEEE Communications surveys and tutorials, vol. 16, no. 1, First quarter
Clark, R.A., Paulson Gjerde, K.A. and Skinner, D. (2003). The effects of interdisciplinary instruction on simulation performance. Simulation & Gaming 34(1): 150-163.
Clarke, E. (2009). Learning outcomes from business simulation exercises: Challenges for the implementation of learning technologies. Education & Training 51(5/6): 448-459. "Clinical Trial Designs." Presentation. Amin, T. T. (2015) https://www.slideshare.net/Tawfikzahran/clinical-trials-designs (Accessed: 25 June 2017)
Colliers Magazine (1926). Kennedy J. B., When Woman is Boss, Interview with Nikola Tesla, New York, published January 1926, Available: http://www.tfcbooks.com/tesla/1926-01-30.htm (Accessed: 3 September 2016)
Cobb, P., diSessa, A., Lehrer, R., Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1): 9-13.
Connell, J.P. (1990). Context , self, and action: A Motivational analysis of self-system processes across life-span. In D. Cicchetto (ed.), The self in transition: Infancy to childhood. Chicago: University of Chicago.
Croston, D. (2008). Employee engagement. 1st ed. Crows Nest, N.S.W.: Moonstone Media, p.180.
Crookall, D. (2010) "Serious Games, Debriefing, and Simulation/Gaming as a Discipline", Simul. Gaming, Vol. 41, No. 6, pp 898-920.
DiCerbo, K. and Behrens J. (2014). "Impacts of the Digital Ocean on Education." Pearson, London.
Dale, Edgar. Audio-Visual Methods in Teaching, 3rd ed., Holt, Rinehart & Winston, New York, 1969, p. 108
Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1): 5-8
DeVellis, R. (2012). Scale development. Los Angeles, Calif: Sage 3 edition (20 July 2011) p.216.
Dickey, M.D., 2005. Brave new (interactive) worlds: A review of the design affordances and constraints of two 3D virtual worlds as interactive learning environments. Interactive Learning Environments, 13(1-2), pp.121-137.
102 Bibliography
Dinsmore, A. (2018). Research Backs Benefits of VR Training. Association for Talent Development. https://www.td.org/insights/research-backs-benefits-of-vr-training. [Accessed: 12 December, 2018] via [https://www.etcourse.com/simulation-tools- transform-education-and-training.html#_edn5]
Dodgson, H. (1987). "Management Learning in MARKSTRAT: The ICL Experience," Journal of Business Research, 15, 6 (December), 481-90. Domdouzis, K., Kumar, B. and Anumba, C., 2007. Radio-Frequency Identification (RFID) applications: A brief introduction. Advanced Engineering Informatics, 21(4), pp.350-355.
EEcatalog (2009). Freescale ships one billionth sensor, nears 30 years of sensor innovation. Available at: http://eecatalog.com/sensors/2009/07/13/freescale-ships-one- billionth-sensor-nears-30-years-of-sensor-innovation (Accessed: 24 March 2016).
Elton, L. (1999). 'New ways of learning in higher education: Managing the change', Tertiary Education and Management 5, pp. 207–225.
Erl, T., Puttini, R., and Mahmood Z., (2013). Cloud Computing: Concepts, Technology and Architecture. Upper Saddle River NJ: Prentice Hall.
Evans, D. (2012). How the Internet of everything will change the World for the better #IoE [Infographic]. Available at: http://blogs.cisco.com/digital/how-the-internet-of-everything- will-change-the-worldfor-the-better-infographic (Accessed: 3 January 2016).
Faria, A.J. (2001). The changing nature of business simulation/gaming research: A brief history. Simulation & Gaming 32(1): 97-110.
Faria, A.J., Hutchinson, D., Wellington, W.J., and Gold, S. (2009). Developments in business gaming: A review of the past 40 years. Simulation and Gaming 40(4): 464-487.
Finn, J. D. (1989). Withdrawing from school. Review of Educational Research, 59, 117– 142.
Fredricks, J. A., Blumenfeld, P. C., Friedel, J., and Paris, A. (2005). School engagement. In K. A. Moore and L. Lippman (Eds.), What do children need to flourish? conceptualizing and measuring indicators of positive development. New York: Kluwer Academic/Plenum Press.
Gardner, H., (2011). Frames of mind: The theory of multiple intelligences. Basic books.
Gee, J. P., (2009). Deep learning properties of good digital games: How far can they go? Serious games: Mechanisms and effects. Routledge, New York.
Gee, J. P., (2003) What Video Games Have to Teach Us About Learning and Literacy. Palgrave- Macmillan. New York.
103 Bibliography
Gosen, J. and Washbush, J. (2004). A review of scholarship on assessing experiential learning effectiveness. Simulation & Gaming 35(2): 270-293.
GlassEyes: The Theory of EyeTap Digital Eye Glass, supplemental material for IEEE Technology and Society, Volume Vol. 31, Number 3, 2012, pp. 10–14
Guillemin, P. and Friess, P. (2009). “Internet of things strategic research roadmap,” The Cluster of European Research Projects, Tech. Rep., http://www.internet-of-things- research.eu/pdf/IoT Cluster Strategic Research Agenda 2009.pdf
GSMA (2012). Automotive - connected living. Available at: http://www.gsma.com/connectedliving/automotive/ (Accessed: 23 May 2016).
Hale, Kelly S, and Kay M Stanney. Handbook of Virtual Environments. CRC Press, 2002. Print.
Hanover Research - Trends in Higher Education (2014). Available at: http://www.hanoverresearch.com/media/Trends-in-Higher-Education-Marketing- Recruitment-and-Technology-2.pdf (Accessed: 6 February 2016).
Harvard.edu "Sample Size Calculators." Hedwig.mgh.harvard.edu. N.p., 2017. Web. (Accessed: 9 June 2017)
Hern, A. (2015) "Facebook Launches Aquila Solar-Powered Drone for Internet Access." the Guardian. (Accessed: 12 August 2017).
Honey, P., and Mumford, A. (1995). Capitalizing on your learning style. Prussia, Pennsylvania: Organization Design and Development, Inc.
Hsiu-Mei, H., Ulrich, R. and Shu-Sheng, L. (2010). ‘Investigating learners’ attitudes toward virtual reality learning environments: Based on a Constructivist approach’, Computers & Education, 55(3), pp. 1171–1182.
Huang, R.H., (2014). Three realms of smart education: smart learning environment, ICT teaching model and modern educational system. Modern Distance Educ. Research, 6, 2, 3-11
Hunicke, R., LeBlanc, M. and Zubek, R. (2004) "MDA: A formal approach to game design and game research", Proceedings of the AAAI-04 Workshop on Challenges in Game AI, San Jose, CA, pp 1-5.
Hyman, L., Lamb, J. and Bulmer, M. (2006) The Use of Pre-Existing Survey Questions: Implications for Data Quality
104 Bibliography
"HoloLens concept lets you control your smart home via augmented reality", (http://www.digitaltrends.com/cool-tech/hololens-hackathon-smart-home/), Digital Trends. 26 July 2016. (Accessed: 11 January 2017)
Jpma, (2017). Questionnaire designing and validation. [online] Available at: http://www.jpma.org.pk/full_article_text.php?article_id=3414 [Accessed 16 Jun. 2017]. iPad in Education Worldwide Results, Computers and Education (2014) published by Apple, 2017.
Kahu, E., (2013) Framing student engagement in higher education, Studies in Higher Education, 38:5, 758-773, DOI: 10.1080/03075079.2011.598505
Keys, B. and Wolfe, J. (1990). The role of management games and simulations in education and research. Journal of Management 16(2): 307-336.
Kim, T. and Park, J., (2014). 3D object manipulation using virtual handles with a grabbing metaphor. IEEE Computer Graphics and Applications, (3):30–38
Kohn, A. (2001). One-Size-Fits-All Education Doesn’t work. Available at: http://www.alfiekohn.org/article/one-size-fits-education-doesnt-work/ (Accessed: 2 February 2016).
Kolb, D.A., Fry, R., (1975). Toward an applied theory of experiential learning, in: C. Cooper (Ed.), Theories of Group Process, John Wiley & Sons, London, UK, pp. 33–58.
Kolb, D. A. (1984) Experiential Learning: Experience as the Source of Learning and Development. Prentice-Hall, Inc., Englewood Cliffs, N.J.
Kolb, D. A. (2014). Experiential learning: Experience as the source of learning and development. FT press
Kortuem, G., Bandara, A., Smith, N., Richards, M. and Petre, M. (2013). Educating the Internet-of-Things generation. Computer, 46(2), pp. 53–61.
Krapp, A., “Interest, Motivation and Learning: An Educational-Psychological Perspective.” European Journal of Psychology of Education, vol. 14, no. 1, 1999, pp. 23– 40. JSTOR, www.jstor.org/stable/23420114.
Kravets, D., (2012). "Tracking School Children With RFID Tags? It’s All About the Benjamins." Wired, 7 Sept.
Kuh, G.D. 2009a. The national survey of student engagement: Conceptual and empirical foundations. New Directions for Institutional Research 141: 5 –20.
105 Bibliography
Laurillard, D., (2002). Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies (London, RoutledgeFalmer).
Laurillard, D., (2008). 'The teacher as action researcher: Using technology to capture pedagogic form', Studies in Higher Education, 33(2).
Learning Theories, "Experiential Learning (Kolb)," in Learning Theories, February 13, 2007, https://www.learning-theories.com/experiential-learning-kolb.html.
Lie, H.W. (1998). Cascading style sheets, level 2. Available at: http://www.w3.org/TR/2008/REC-CSS2-20080411 (Accessed: 12 May 2016).
Link Trainer Restoration., Web, Starksravings.com., 2019. Web. (Accessed: 3 February. 2019)Lynn, M.R. (1986). Determination and quantification of content validity. Nursing Research, 35, 382– 385.
Mads, B. (2017) Technology Introduces New Forms of Experiential Learning. [ONLINE] Available at: http://www.teachthought.com/pedagogy/technology-introduces-new- forms-of-experiential-learning/. [Accessed 13 July 2017].
Magerkurth, C., (2011). Hybrid Gaming Environments: Keeping the Human in the loop within the Internet of Things Universal Access in the Information Society International Journal, Volume 11, No 3, ISSN 1615-5289
Mantovani, F., Castelnuovo, G., Gaggioli, A. and Riva, G. (2003). ‘Virtual reality training for health-care professionals’, CyberPsychology & Behavior, 6(4), pp. 389–395. doi: 10.1089/109493103322278772.
Microsoft (2016) Microsoft HoloLens. Available at: https://www.microsoft.com/microsoft-hololens/en-us/why-hololens (Accessed: 1 July 2016).
Moher, D., Dulberg, C.S. and Wells, G.A., 1994. Statistical power, sample size, and their reporting in randomized controlled trials. Jama, 272(2), pp.122-124.
Moody, M., 2015. Analysis of promising beacon technology for consumers. Elon Journal of Undergraduate Research in Communications, 6(1).
Marklund, B. B., (2015) Unpacking digital Game-based learning—The complexities of developing and using educational games, Doctoral Dissertation, Department of Informatics, University of Skövde.
Natriello, G. (1984). Problems in the evaluation of students and student disengagement for secondary schools. Journal of Research and Development in Education, 17, 14–24.
106 Bibliography
Neuman, W.L. (2006), Social Research Methods: Qualitative and Quantitative Approaches, Boston: Allyn and Bacon.
Nielsen, J. (1995). Heuristic Evaluation. [online] Nngroup.com. Available at: https://www.nngroup.com/articles/how-to-conduct-a-heuristic-evaluation/ [Accessed 20 Jun. 2017].
Nunan, D. (2015). Teaching English to Speakers of Other Languages: An Introduction. Routledge, NY
Online Learning Consortium, (2015). Online report card - tracking online education in the United States, 2015 - OLC. Available at: http://onlinelearningconsortium.org/read/online- report-card-tracking-online-education-united-states-2015/ (Accessed: 22 July 2016).
Pan, Z., Cheok, A.D., Yang, H., Zhu, J. and Shi, J., 2006. Virtual reality and mixed reality for virtual learning environments. Computers & Graphics, 30(1), pp.20-28. Panetta, Kasey. "5 Trends Emerge in The Gartner Hype Cycle for Emerging Technologies, 2018." Gartner.com., 2019. Web. 14 March. 2019.
Parasuraman, A. (1980). Evaluation of simulation games: A critical look at past efforts and future needs. Experiential Learning Enters the Eighties 7: 192-194.
Palloff, R.M. and Pratt, K. (2013) Lessons from the virtual classroom: The realities of online teaching, 2nd Edition (San Francisco, Jossey-Bass, Wiley)
Papert, S. (1980) Mindstorms: Children, computers, and powerful ideas. Basic Books, Inc.
Perera, C., Zaslavsky, A. Christen,P., Georgakopoulos, D., (2014). ‘Context Aware Computing for The Internet of Things: A Survey’, IEEE Communications Surveys & Tutorials, Vol.16, No.1, 1st Quarter.
Piaget, J. (1952). The origins of intelligence in children (Vol. 8, No. 5, pp. 18-1952). New York: International Universities Press.
Polit, D.F, and C Beck. The Content Validity Index: Are You Sure You Know What’s Being Reported? Critique and Recommendations. Res Nurs Health. 2006
Pratt P., Ives M., Lawton G., Simmons, J., Radev, N., Spyropoulou L., Amiras D., (2018) Through the HoloLens™ looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels, European Radiology Experimentalvolume 2, Article number: 2
Ranchhod, A., Gurau, C., Loukis, E., Trivedi, R. (2013). Evaluating the educational effectiveness of simulation games: A value generation model Information Sciences, Volume 264, pp. 75-90.
107 Bibliography
Ratan, R. and Ritterfeld, U. (2009). "Classifying Serious Games", in U. Ritterfeld, M. Cody & P. Vorderer (eds), Serious Games: Mechanisms and Effects, Routerledge, New York, NY, pp 15-24.
RFID Exchange (2004). RFID 101—A Beginner’s Guide to RFID Technology and Systems. Retrieved from the web on June 20, 2004. Available at http://www.rfidexchange.com/rfid101.aspx.
RFID Technology and Its Applications In Internet Of Things (IoT) - IEEE Conference Publication." Ieeexplore.ieee.org. N.p., 2012. Web. 5 June 2018
Richardson, C., Roop, R.M., Azarian, M.H. and Ganesan, S. (2005). ‘IEEE transactions on components and packaging technologies publication information’, 28(2), pp. c2–c2. doi:10.1109/tcapt.2005.850832.
Rigger, P., Bosch IoT Lab, University of St. Galen. [Accessed: May 2017]
Roberts, C.M., 2006. Radio frequency identification (RFID). Computers & security, 25(1), pp.18-26.
Robinett, W., (1994) "Interactivity and Individual Viewpoint in Shared Virtual Worlds: The Big Screen vs. Networked Personal Displays." Computer Graphics, 28 (2), 127
Rosenberg L. B., The Use of Virtual Fixtures as Perceptual Overlays to Enhance Operator Performance in Remote Environments, USAF Armstrong Laboratory, Wright-Patterson AFB OH, 1992.
San Pedro, M., Baker, R., and Rodrigo, M., (2011). “Detecting Carelessness through Contextual Estimation of Slip Probabilities among Students Using an Intelligent Tutor System for Mathematics” in Artificial Intelligence in Education, Springer, pp. 304 –311.
Saunders, M.N.K. (2012). ‘Choosing research participants’ In G. Symon, G. and C. Cassell (eds.), The Practice of Qualitative Organizational Research: Core Methods and Current Challenges. London: Sage 37-55.
Saunders, M., Lewis, P. and Thornhill, A. (2016). Research Methods for Business Students (7th edition). Harlow: Pearson.
Scivally, A. (2014) 8 Effective Scenario Ideas for Instructional Designers. eLearning Brothers
Scott, D. and Morrison, M. Key Ideas In Educational Research. 1st ed. London: Continuum, 2007
Seale, C. (2004). “Using Data Archives for Secondary Analysis”, in Seale, C. (ed) Researching Society and Culture, London: Sage, pp. 355-365.
108 Bibliography
Selinger, M., Sepulveda, A., Buchan, J., (2015). ‘Education and the Internet of Everything’ How ubiquitous Connectedness Can Help Transform Pedagogy. CISCO
Schiefele U. (2001). The role of interest in motivation and learning. In Collis J. M. & Messick S. (Eds.), Intelligence and personality: Bridging the gap in theory and measurement (pp. 163–194). Mahwah, NJ: Erlbaum
Sharif, A and Ranchhod, A (2009). Using the Markstrat business simulation to develop strategic management behaviours.
Shah, D. (2015). By the numbers: MOOCS in 2015 — class central. Available at: https://www.class-central.com/report/moocs-2015-stats/ (Accessed: 21 March 2016).
Siemens, G. (2008). Learning and knowing in networks: Changing roles for educators and designers. Available at: http://itforum.coe.uga.edu/Paper105/Siemens.pdf (Accessed: 5 September 2016).
Sinfield, S., Harrington, K. and Burns, T. (2016). STUDENT ENGAGEMENT. In: Teaching and Learning in HE. London: Pokorny and Warren (EDS) Sage.
Stankov, S., Glavinic, V., Rosic, M. (2010). Intelligent Tutoring Systems in E-Learning Environments: Design, Implementation and Evaluation, Information Science, NY. pp. 208–236.
StratX (2016). Academic solutions for experiential learning - StratX. Available at: http://stratxcorporate.com/academic/ (Accessed: 13 February 2016). And Markstat: Strategic marketing simulation game (2016). Available at http://web.stratxsimulations.com/simulation/strategic-marketing-simulation/ (Accessed: 12 March 2016)
Statistics Solutions (2017) "Take The Guesswork Out Of Sample Size: Why And How To Conduct A Power Analysis - Statistics Solutions.". N.p., 2017.
Starkman, N. (2006). The Wonders of Interactive Whiteboards. T.H.E. Journal, 33(10), 36- 38. Retrieved June 27, 2019 from https://www.learntechlib.org/p/77102/.
Summers, G.J. (2004). ‘Today’s business simulation industry’, Simulation Gaming, 35(2), pp. 208–241. doi: 10.1177/1046878104263546.
Taljaard, Johann. (2016). A review of multi-sensory technologies in a Science, Technology, Engineering, Arts and Mathematics (STEAM) classroom. Journal Of Learning Design. 9. 46- 55. 10.5204/jld.v9i2.274.
109 Bibliography
Trines, Stefan. "Educating The Masses: The Rise Of Online Education In Sub-Saharan Africa And South Asia - WENR." WENR. N.p., 2019. Web. 2 Oct. 2019.
Trojan Room Coffee Pot, Cl.cam.ac.uk. N.p., 1995. Web. 12 July 2018.
Vance, S.C. and Gray, C.F. (1967). Use of a performance evaluation model to research in business gaming. Academy of Management Journal 10(1): 27-37.
Van Notten, P., (2006). Think Scenarios, Rethink Education. Scenario Development: A Typology of Approaches. ISBN-92-64-02363-1 OECD 2006. 69-87.
Venkitachalam, R. (2015). [podcast] Validity and reliability of questionnaires. Available at: https://www.slideshare.net/Venkitachalam/validity-and-reliability-of-questionnaires [Accessed 24 Jul. 2017].
Virtual Reality Society (2015). CAVE fully Immersive virtual reality - virtual reality. Available at: http://www.vrs.org.uk/virtual-reality-environments/cave.html (Accessed: 19 April 2016).
Vujovic, V. and Maksimovic, M., (2015). The Impact of the Internet of Things on Engineering Education. Proceedings of The Second International Conference on Open and Flexible Learning, Hong Kong
West, D. M., (2012). "Big Data for Education: Data Mining, Data Analytics, and Web Dashboards." Brookings Institution, Washington DC.
Wachowski, Lilly. The Matrix. USA: Fantasy/Sci-fi, 1999. film.
Walgrove, A., (2015). 5 brands that are using the Internet of things in awesome ways. Available at: https://contently.com/strategist/2015/03/16/5-brands-that-are-using-the- internet-of-things-in-awesome-ways/ (Accessed: 19 July 2016)
Wellington, W., Faria, A.J., Whiteley, T.R. and Nulsen, R.O., (1995). Cognitive and behavioral consistency in a computer-based marketing simulation game environment: An empirical investigation of the decision-making process. Developments in Business Simulation and Experiential Exercises 22: 12-18.
Weartech.Org, Wearcomp.Org, Wearcam.Org, Eyetap Digital Eye Glass, Weartech (TM) Wearable Technologies/Research/Consulting/Public Speaking/Inventing/Philosophy." Genesis.eecg.toronto.edu. N.p., 2008. Web. 5 Aug. 2018.
Wolfe, J., (1976). Correlates and measures of the external validity of computer-based business policy decision-making environments. Simulation & Gaming 7(4): 411-438.
110 Bibliography
World University Rankings - Masters In Business Analytics 2019." TopMBA.com. N.p., 2019. Web. 1 Oct. 2019
Wu, H. K., Lee, S.W.Y., Chang, H.Y. and Liang, J.C., (2013). ‘Current status, opportunities and challenges of augmented reality in education’, Computers & Education, 62, pp. 41– 49. doi: 10.1016/j.compedu.2012.10.024.
Wyld, D. C., RFID: The Right Frequency For Government. 1st ed. IBM Center for the Business of Government, 2005. Web. 29 Nov. 2018.
Yazdi, N., Ayazi, F. and Najafi, K., (1998). ‘Micromachined inertial sensors’, Proceedings of the IEEE, 86(8), pp. 1640–1659. doi: 10.1109/5.704269.
Yuan, L., Powell, S. and Cetis, J. (2013) MOOCs and open education: Implications for higher education 2 MOOCs and open education: Implications for higher education.
Yusoff, A., Crowder, R., Gilbert, L., and Wills, G (2014). A conceptual design for a teacher- learner framework for educational serious games. Journal of ICT in Education, 1, (1), 1-10.
Zinopoulou M, Ranchhod Ashok, Wills Gary, At lam H.F, Nik Zulkipli N.H., (2017) “IoT Security Prioritised”, IoTSF
111 Appendix A
Appendix A Research Flowchart
112 Appendix B
Appendix B VR Experiment Photos
113 Appendix C
Appendix C Raw Data PC Simulator
114 Appendix D
Appendix D Raw Data VR Simulator
115 Appendix E
Appendix E Calculations of p and Z- Score / U-Score
116 Appendix F
Appendix F Student Questionnaire
117 Appendix F
118 119