Cabin Crew VR

Home , Virtuality (product), Within (company)

BSc | Computer Science

Cabin Crew VR

Final report

Andri Sæmundsson Hartmann Ingvarsson Njáll Mýrdal Árnason Maí 2020

Supervisor: Hannes Högni Vilhjálmsson Moderator: Torfi Ásgeirsson

Abstract

Virtual Reality technology is increasingly being used by companies and institutions for training and simulation (Nee & Ong, 2016). This exploratory study aims to examine the technologies and equipment used in Virtual Reality training, and designing and developing a Virtual Reality training prototype for in-flight cabin crew members. This report will explain the different types of artificial realities and the software and hardware used in developing such realities. The Virtual Reality training prototype development process will be detailed and the results from user testing of the prototype will be examined.

Contents

Abstract 1

Contents 2

Introduction 3

Competition 5

Approach 6 User interface and interactive feedback 12 Documentation 12 Risk assessment 13

Results 14

Discussion and conclusion 20

Time to come 21

References 22

Appendix A 23

Appendix B 32

Appendix C 34

Introduction

Virtual Reality, or VR, has increased in popularity the last few years, but the VR technology is far from brand new. In the 1980s and 1990s there were several technology firms and video game companies that tried to capitalize on the VR craze at the time, but lack of hardware was a major obstacle. NASA was one of the pioneers in VR development where they developed simulations for astronauts. Technology firm Virtuality was also a pioneer in VR where they created an arcade gaming system using stereoscopic headset and hand controllers (Edwards, 2018). Both NASA and Virtuality used their own hardware which was heavy, expensive and limited.

Virtual Reality development is different today than it was in the 80s and 90s. Today, there are several VR headset creators including Oculus, Sony, Lenovo, HTC, Google and Samsung. Each system is different and created for different purposes and can be categorized as mobile, tethered or standalone. The tethered Sony VR set is, for example, to be used with Sony's PlayStation. Google and Samsung, on the other hand, have mobile phone-based headsets where users can insert their phones into a headset to experience VR. Oculus and Lenovo have standalone VR headsets, most of which have the same limited controls as mobile headsets. The Oculus Quest is one of the most popular standalone VR headsets, mainly because it has a powerful processor, motion controls, motion tracking and it is untethered giving more freedom and immersion (The Best VR Headsets for 2020, 2020).

Software used in VR development is much more available today than it was in the 90s. Some of the most popular game engines used in VR development are CryEngine, Amazon Sumerian, Unreal Engine and Unity (DevTeam, 2020). Unity and Unreal Engine are the most popular engines for VR development and were both engines explored in detail in this study.

Cabin crew training is extensive where the crew must learn through lectures and hands-on training. Much of the training material is text and/or image-based, but real-life training is also a big part of the training. The training methods and facilities vary from one airline to another. The Icelandic airline, Icelandair, has, for example, a training facility that has parts of actual airplanes where cabin crew members experience hands-on training. This facility was very expensive to build and it has its limitations, such as the number of people that can use the facility at a time. All crew members must undergo initial training as well as retraining at least once a year. Cabin crew training is costly and highly space-demanding, but with the use of Virtual Reality, cabin crew training can become more affordable, less space-demanding and potentially more effective than current training methods.

This study was twofold; the first part of the study was exploratory and focused on study and research into different game engine software, VR hardware and cabin crew training. The results from this study were used as justification of decision making when the second part of the study was conducted. The second part of the study was the design and development of a VR prototype for cabin crew members. Following the prototype development, user testing was conducted on a set of participants. Participants were asked to think aloud, or talk aloud, throughout the testing to get their insights and thoughts while in VR. The participants conducted a pre-flight check of some safety equipment and were given a questionnaire with ten questions in an attempt to measure the efficacy, satisfaction, usability and success of the training system.

Competition

There are several companies around the world creating Virtual Reality training and simulation that would be considered competition for this type of project. There are, however, no companies in Iceland offering training or simulation for cabin crew members using VR.

Bolverk XR, located in Denmark, is a company that creates Virtual Reality training and simulation for cabin crew members. Bolverk XR has several products which they label as Cross-Reality, using Virtual and Augmented Realities in their products. Bolverk services several airlines, which typically have a range of a few thousand employees to tens of thousands of employees. SAS and Quantas are an example of their clients. Bolverk XR would be considered one of the biggest competitors for this type of project being developed in this study.

Immerse is based in the United Kingdom and they create VR, AR and other types of training and simulation for some of the biggest companies in the world. Immerse has built several VR training products including a cabin crew training system for AirFrance. Immerse has other customers including Shell, General Electric and Facebook.

Virtalis has offices all around the world and has done VR projects for some impressive clients including the Royal Air Force, Lockheed/Martin and the Australian air force. Virtalis has VR products in the fields of automotive, energy, defence, aerospace, engineering and offshore oil and gas. Virtalis has made cabin crew simulations, for example a cabin crew training in VR for helicopter crew in the Australian air force. Virtalis is in the forefront of VR simulation and training and is considered a major competitor for this project.

There are also some airlines that have created and use their own proprietary VR system for their cabin crew training. Lufthansa has, for example, a specific division within their company where VR training and systems are developed. The proprietary systems used by Lufthansa and other airlines are not necessarily a major competition to this project, but it shows that most major airlines around the world are either developing their own, contracting or in the process of implementing VR training for their cabin crews.

Approach

This exploratory research project was pitched on behalf of Reykjavik Game Studios as a final assignment in the Bachelor of Science degree for the University in Reykjavik in cooperation with CADIA, Icelandair and Reykjavik Game Studios. CADIA (Center for Analysis and Design of Intelligent Agents) is a research center located at Reykjavik University which provided the technical equipment and facilities needed for this study along with guidelines, assistance and advice. Reykjavik Game Studios provided the study with a working facility along with technical equipment. The head of cabin crew training for Icelandair was willing and able to assist and provide the researchers with the training methods. The researchers also visited the training facility (see Figure 1) and were able to see how the training process is conducted in real-life. Icelandair provided the study with the cabin crew safety handbook (see Figures 5, 6 and Appendix B) which was used extensively as reference throughout the study and prototype development.

Figure 1: Visit to the Icelandair training center.

Figure 2, 3, 4: FirstAidKit, location beacon and bullhorn, visit to Icelandair.

Figure 5: map for equipment from the handbook. Figure 6: Pre-flight checklist, from the handbook

The project was split up in five sprints where each sprint was given specific tasks to be completed. The first sprint was used for research and design as well as deciding which software and hardware to use.

In the first sprint, it was decided to create two prototypes; one prototype in Unity, one in Unreal engine and compare the two. Both engines have their strengths and weaknesses; Unreal has better graphics out of the box, but also a steeper learning curve than Unity. Unity has the capability to produce similar quality graphics as Unreal, but the graphic settings in Unity must be created from scratch instead of being readily available out of the box. Unity also has a useful asset store and massive amounts of tutorials on any subject. It was quickly decided to drop the Unreal engine and Unity was chosen to be used as the development engine. The researchers were already familiar with Unity and had experience with developing in Unity.

During the first sprint of the study, the Oculus Rift VR headset was used for development. It proved to be less than optimal, since the device has to be tethered at all times. The study also experimented with the HTC Vive VR headset which also has to be tethered at all times and proved not suitable for this project. Since the virtual environment within the training system is on such a large scale, the researchers wanted to use a standalone device so that the users could move around freely without worrying about any cords. The hardware chosen for the prototype was therefore the Oculus Quest - one of the most powerful standalone headsets to date.

The second sprint focused on getting core mechanics to function properly. Core mechanics in this VR prototype included walking and interacting with objects in a virtual reality space. Experiments were conducted on a few rigs or character controllers, and an attempt was made to make a character controller from scratch. The researchers eventually decided to use a pre-build rig for all character controls for faster and more productive development pace.

In the third sprint work on core mechanics continued, as well as 3d model creation of the virtual airplane. The environment required is so specific to this project and no assets were available in the Unity asset store that mimics the interior of a Boeing airplane, so the airplane interior environment (See Figure 5, 6) had to be modeled and textured from scratch. Attention was paid to detail and all objects deemed important were included in the prototype such as overhead compartments, seats, window covers and safety equipment.

Figure 7: Earliest version of the cabin.

After the fourth sprint, all core mechanics had been implemented as well as the game-play mechanics used in the training prototype. Users could walk around within the VR airplane and conduct the pre-flight safety check (See Figure 6). Users could open overhead compartments, open windows and check items off the pre-flight checklist by discovering and touching the safety equipment placed within the airplane.

Figure 8: Final version of the cabin.

By the fifth sprint, all game mechanics were implemented to complete the first in-game mission of pre-flight checklist, checking where the user could traverse through the airplane to find the safety equipment placed within the virtual airplane. User testing was conducted on nine participants which were

airplane pilots, cabin crew members and people with no aviation experience. Users were instructed to talk aloud during the testing and asked to fill out a questionnaire. Following the user testing a report was made to disclose the results, findings and interpreted.

Figure 9: Outside view of the cabin final scene in Unity

Unity packages were used in the creation of the prototype. Several packages were studied to find parts, assets or visuals suitable for the prototype. The study experimented with the XR Rig (Unity3d, 2019) which proved to be very suitable for the prototype. Parts of a VR tutorial development scene called Escape Room (Unity Asset Store, 2020) were also used to get the core mechanics to function properly. Escape Room utilizes XR Rig which is currently the most used Virtual and Artificial Reality game controller in Unity. The XR Rig was used as a template to build a controller suitable for this prototype, while the Escape Room had added features which some were adapted to the prototype.

The prototype places the user within a 3d model replication of a Boeing 757-200 airplane, which is the same airplane primarily used in training cabin crew members at Icelandair. The objective is to check off all the items in the pre-flight checklist which the user accesses by looking at the in-game wristwatch (See Figure 10). The user walks within the airplane model to find the safety equipment.

Figure 10: The ingame wristwatch model in Unity engine

The user can open overhead compartments (See Figure 11), open windows (See Figures 12 and 13) and traverse throughout the entire plane given enough space or by using the teleport feature within the character controller.

Figure 11: Overhead compartments.

User interface and interactive feedback

In virtual reality, it is important to make the user feel as immersed within the environment as possible. There are certain rules and methods that can be followed to help make the experience feel realistic but not intrusive to the user. In a 3D or 2D game environment, the game menu system is often used as a 2D text and menu system that pops up when players push the escape key on a keyboard. Dialogs that appear in-game are often static and do not move with the player. That kind of system does not work as well in virtual reality, because within the environment users move their heads quite rapidly and quickly. It is disorienting to have the menu system appear in front of the user and the user loses the connection to the VR environment; this could lead to nauseating feeling and discomfortness (Jerald 2015, P 381). To enhance the user experience and make a system that informs the user of what they had done, a user interface system was implemented in which the user could keep track of the progress that they had made within the training program. A virtual hand-held panel design was chosen which is attached to the non-dominant hand in the form of a wristwatch that appears when you raise your hand and look at it; it does not obscure so much of the view that you feel disoriented and with a simple gesture you can make the panel disappear again (Jerald 2015, P 348). An informal sound is heard when located security objects are being touched to give the user some interactive feedback; it is important for the user to get positive feedback that something happened in the environment during the interaction. The visual feedback is also on the wristwatch panel when security objects are being checked with a visible checkmark.

Documentation

During the development of this project, the group created their own google account where all documentation was conducted. All spreadsheets for the groups presentations and all 3D modeled assets were uploaded to the google drive associated with the account. The google platform served as a mutual access for the project members to access progress reports, feature list, timesheet. The platform provided source control for the whole project (See Appendix C). The group created their own Slack message board for communication, posting reading material and relevant videos for research. The Facebook messenger app was also used for more direct communication.

Risk assessment

In the early stages of the project the group made their own risk analysis of what possible risks the project might face. Those risks were categorized by their severity and its factor, likelihood of them happening, steps to minimize them and the person responsible (See Appendix C).

Results

The design and development of the cabin crew VR prototype training system was successful. Much of the time was spent on 3D asset creation, but Unity and the packages used proved to be extremely useful for this prototype. The final product is a functioning prototype where the user takes on the role of an in-flight cabin crew member responsible for checking off all the items in the pre-flight safety checklist. The user spawns in first class at the front of the plane. The user can see the pre-flight checklist by looking at the in-game wristwatch (See Figure 16) where a user interface pops up to reveal the checklist (See Figure 17). The user can walk or teleport around the airplane. The user can open all overhead compartments (See Figures 12-13) to find the items on the pre-flight checklist (See Figures 18, 19, 20). The user can finally slide open the shutters on the windows (See Figures 14 and 15) to enjoy the view from 38.000 ft (See Figure 9).

Figure 12 to 15: All of the overhead compartments in the cabin can be opened or closed. Along with all the windows.

Figure 16: Test run of the prototype, user looks at wristwatch.

Figure 17: The wristwatch shows the pre-flight checklist and uses checkmarks for objects that have been found.

Figure 18: The user opens an overhead compartment with the controller and finds an object on the pre-flight checklist.

Figure 19: The user touches the object with the right hand controller. As the object is touched a sound file is played.

Figure 20: The user checks back with the pre-flight list. After he hears the sound and sees that the checkmark for the object has been toggled.

Usability testing

The group sought out participants who had experience in aviation and especially those who had worked as cabin crew members. Participants, who had no prior experience in the relevant fields, were also welcome. Nine participants tested the prototype at the group’s workplace. Preparations were made before they arrived; the virtual reality space had been mapped out in the real environment within the workplace using markers on the floor and chairs for the cabin seating, thus mimicking a real cabin as much as possible. Before the participants began the test, they had been given brief information about the project (See Appendix A) and how movement in virtual reality is conducted. Each participant was asked if they agreed to be videotaped during the test and asked to narrate his or her environment and actions outloud. When the run-through of the objectives was concluded the participant was asked to answer a short survey about the process (See Appendix A).

Figure 21: Participant going through the user testing of the prototype.

All of the participants who tried the prototype agreed that this type of virtual reality training would be beneficial along with real-life training (See Appendix A; Q3). However, the virtual reality training could not replace the real-life training aspects such as the physical sensation of touch, hearing and smell, which are important to the training (See Appendix A; Q4). It is difficult to imitate any form of smell and the feeling of an object's weights in one's hands in virtual reality. However, it is possible to touch objects with a controller in a virtual reality environment and to hear sounds. When asked how a participant searched for the objects they needed to find, the participants that had worked in a cabin crew said that they based it on their prior training experience. One participant, who was currently employed as a cabin crew member for an airline, commented that some of the objects were not located where they would normally look for them, e.g. the crash axe should have been at the flight deck and not in the cabin as in the prototype. Furthermore, that participant contributed valuable insight and ideas for the project, by sharing their knowledge and work experience, e.g. there should be a cabin crew seat where the location beacon is located, which is not in the prototype. They also marked where all future and current objects should be in the airplane along with other details. Participants who had not worked as cabin crew members based their search on their own intuition and in no particular order; some felt at unease how the wristwatch felt close to the face, minimizing the visual interface of the watch would fix that (Jerald 2015, P 304).

Figure 22: Another participant going through the user testing of the prototype.

When asked to rate the project on a scale from one to ten based on how close the virtual environment felt to a real life environment, the final result from the participants was average of seven, forming a measurable variable if the effects of the quality of textures, lighting and 3D-modeling have a saying on how close virtual reality feels to a reality. This will be beneficial for the group in future development where better graphics will be introduced.

Discussion and conclusion

This exploratory study set out to determine what software and hardware are best suited for simulation and training for in-flight cabin crew members. The study also set out to find out if development and production of such a training system is feasible technically, financially and timewise. After reviewing the major engines used in VR development, Unity was chosen because of the extensive documentation, tutorials and assets available for Unity. Unity proved to be an excellent engine for VR development and will continue to be used to develop the project from a prototype to a full scale project. Both Oculus Rift and Oculus Quest were used during the development process. Both devices were suitable for the development but the Oculus Quest was determined to have an advantage since it can be used untethered.

The results from this exploratory study show that it is indeed feasible to develop a VR training system. The Unity game engine and the Oculus devices also proved to be a suitable choice for this type of VR project. The user testing of the prototype showed positive results. All testers expressed positive views and enjoyed the experience. The testers with cabin crew experience also indicated that this type of training system could well be used in conjunction with the current training systems used by Icelandair.

This study shows that developing a VR training system within Unity using Oculus headsets is very achievable, financially, technically and timewise. This study shows positive feasibility of developing and using this type of system for cabin crew member training.

Time to come

This study has shown that it is indeed very feasible, technically, financially and timewise, to develop a VR training system for cabin crew members using the above mentioned tools. Work on the prototype will continue and a full VR training system will be built on the lessons learned in this study. Other missions, beside the pre-flight checklist, will be added to the training system such as turning off fires, smoke exercises and passenger control. The researchers have applied for a research grant to further develop the project into a full scale product. The product will finally be presented to potential customers and training facilities for airlines.

References

10 Great Tools for VR Development I DevTeam.Space. (2020, January 27). Retrieved from https://www.devteam.space/blog/10-great-tools-for-vr-development/

The Best VR Headsets for 2020. (n.d.). Retrieved from https://www.pcmag.com/picks/the-best-vr-headsets

Edwards, B. (2018, April 27). The Wacky World of VR in the 80s and 90s. Retrieved from https://www.pcmag.com/news/the-wacky-world-of-vr-in-the-80s-and-90s

Jerald, J. (2016). The Vr book: human-centered design for virtual reality. New York: Association for Computing Machinery.

Nee, A. Y. C., & Ong, S. K. (2016, April 21). Virtual and Augmented Reality Applications in Manufacturing. Retrieved from https://www.sciencedirect.com/science/article/pii/S1474667016342562

Slack. (n.d.). Where work happens. Retrieved from https://slack.com/intl/en-is/

VR Beginner: The Escape Room: Tutorial Projects: Unity Asset Store. (n.d.). Retrieved from https://assetstore.unity.com/packages/essentials/tutorial-projects/vr-beginner-the-escape-room-16 3264

XR Interaction Toolkit: XR Interaction Toolkit: 0.9.4-preview. (n.d.). Retrieved from https://docs.unity3d.com/Packages/[email protected]/manual/index.html

Appendix A

Script for user tests: https://www.surveymonkey.com/r/RYTND6T?fbclid=IwAR3Ux1DkMQSDcVu4ajWv6GGssStfEutm3ne npOYfs-JDLm8ttooyEZo7Rag

Velkomin/n, takk fyrir að taka þátt í þessu með okkur. Við ætlum að biðja þig um að prófa sýndaveruleika þjálfunar-kerfi fyrir flugfreyjur og flugþjóna. Ég ætla ekki að segja þér of mikið áður en þú byrjar en markmiðið er að framkvæma pre-flight check á öllum mikilvægustu öryggisatriðum sem eiga að vera til staðar. Þú ert líka með úr í leiknum sem gæti hjálpað þér.

Svo erum við að framkvæma Think Aloud test á sama tíma. Það þýðir að við viljum biðja þig um að tala upphátt allan tímann, lýsa því sem þú sérð, hvað þú ert að hugsa og hvað sem þér dettur í hug. Reyndu bara að tala allan tímann.

Þitt hlutverk: Finna ákveðna hluti og athuga fyrir flug.

Spurningar eftir prófun.

1. Hefur þú prófað sýndarveruleikabúnað áður?

2. Finnst þér augljóst hvert hlutverk þitt er í kerfinu?

3. Finnst þér eitthvað áberandi óljóst við kerfið, jafnvel villandi, ef svo er hvað þá?

4. Eru einhver sérstök atriði sem standa upp úr?

5. Hefur þú unnið sem meðlimur í flugáhöfn?

6. Mundir þú telja að þjálfun sem þessi væri gagnleg í sýndarveruleika samhliða raunverulegri þjálfunar?

7. Mundir þú telja þjálfun sem þessi gæti komið í staðinn fyrir raunverulega þjálfun?

8. Fannstu fyrir einhverjum óþægindum, t.d. sjóveiki eða annað?

Opnar spurningar:

Ef þú værir meðlimur í flugáhöfn, hversu auðvelt telur þú það að framkvæma fyrirfram ákveðinn “Pre flight” tjékklista í gegnum Sýndarveruleika, Sæmilegt, Gott, Mjög gott

Hugsaðu til baka, hvernig fórstu að því að vinna verkefnið og finna þá hluti sem þú áttir að athuga, er það byggt á reynslu þinni eða minni? Eða var það bara handahófskennt?

Skalað frá 1 til 10:

Hversu auðvelt er að hreyfa sig i flugvélinni 1 - 10

Hversu ánægð / ánægður ertu með umhverfið í sýndarveruleika? 1 - 10

Hversu líkt raunveruleikanum er sýndarumhverfið? 1- 10?

Survey result: q1

Q4:

Q5:

Q6:

Q7:

q10

Appendix B

Images and data contributed from Icelandair

Appendix C

Agile documentation - progress report, timesheet, risks