VTT TECHNICAL RESEARCH CENTRE OF FINLAND LTD

Holonovel Workshop Introduction

EAI International Conference on Technology, Innovation, Entrepreneurship and Education SEPTEMBER 11–12, 2017 | CANTERBURY, UK

Tiina Kymäläinen VTT Technical research centre of Finland Ltd. Human Factors, Virtual- and Augmented Reality Holodeck – Wikipedia definition

§ The holodeck is a plot device used in stories set within the Star Trek universe § Permit stories with locations and characters that could not otherwise exist in the Star Trek setting https://www.youtube.com/watch?v=oZwtVz7z0wM • Fictional technology – but it has been predicted that similar virtual reality tools "will become a consumer-ready product by 2024”

https://en.wikipedia.org/wiki/Holodeck

11/09/2017 2 Science Inspiration

§ The Star Trek Holodeck was inspired by inventor and holographer, Gene Dolgoff, who is also the inventor of the digital projector § His holography laboratory, built in 1964, was the first holography lab in New York City

11/09/2017 3 Episodes in Star Trek Star Trek ®, Star Trek: The Next Generation ®, Star Trek: Deep Space Nine ® and Star Trek: Voyager ® are trademarks of Paramount Pictures registered in the United States Patent and Trademark Office

§ Star Trek animated series: § "The Practical Joker” (1974), Season 2, Episode 3 § Next Generation: § "The Big Goodbye” (1988): Season 1, Episode 12 § "Elementary, Dear Data" (1988): Season 2, Episode 3 § "A Fistful of Datas” (1992): Season 6, Episode 8 § Voyager: § "Author, Author”, "Flesh and Blood”, "Nothing Human” § Star Trek: Enterprise: § “Unexpected” (2001), Season 1, Episode 5

11/09/2017 4 Applications § Starfleet personnel use holodecks for training, diagnostics and recreation § Simulate settings and events for analysis, such as to explore crime scene or for scientific experimentation § Holodeck variations of being by the sea, in a forest and being trapped in a snow-storm § An example of the holodeck's recreational functions are the holosuites

11/09/2017 5 “Technology”

§ Holodecks use two major subsystems: Holographic image and the conversion of matter § The holodeck is depicted as an enclosed room in which objects and people are simulated by a combination of: § Transported matter (Holomatter) § Tractor beams § Shaped force fields § Replicated matter § Controlled by voice commands and physical controls § A hidden computer terminal allows access to the holodeck controls https://en.wikipedia.org/wiki/Holodeck

11/09/2017 6 How does it really work?

Simulation types: § The walls can generate holographic images which appear to extend for an unlimited distance § Holograms can be projected into space § Holograms can be augmented with force beams to simulate solid, tangible objects § Holograms and force beams can be augmented with replicator technology to provide actual substance § Magnificent: If an object's status changes the simulation type will change seamlessly. For example, someone sees a distant tree (walls), approaches it (hologram), leans on it (force beams), breaks off a branch (Holodeck-matter), then picks and eats an apple (replicated).

Holodeck and Computers FAQ, Joshua Bell: http://www.calormen.com/star_trek/FAQs/holodeck-faq.htm

11/09/2017 7 Applications

§ Microsoft IllumiRoom Creates illusions of the area surrounding TV § The Void, Utah-based startup: IMAX of VR experiences, multiplayer VR environment § University of Illinois at Chicago: CAVE2: model global weather patterns and help doctors practice surgeries § Real Life Holodeck with an Oculus Rift – Computerphile § VTT: Hololens approach: shared multiuser MR experience § Etc. (growing exponentially)

11/09/2017 8 11/09/2017 9 Holonovels

§ Writing stories and plotlines for the holodeck is an activity pursued by people known as holonovelists § Most holodeck programs shown in the episodes run in first person "subjective mode” § User actively interacts with the program and its characters § Raise interesting philosophical and sociological issues of virtual realities § A holonovel "Photons Be Free" authored by the Doctor (Episode: "Author, Author")

11/09/2017 10 "Worst Case Scenario"

§ Star Trek: Voyager , Season 3, Episode 25

11/09/2017 11 Holonovels Design Principles

§ Current mixed reality environments: operates between projected reality (VR) augmented reality (AR) – transported matter – and real reality – matter/replicated matter § Combines screenwriting, dramaturgy, game design, virtual world construction and e.g. intelligent environment research § Occupies space and highlights the role of artefacts § E.g. bases on sequence construction § Instead of linear timeline follows hierarchy (IF-THEN rules) § Need: Lack of design principles for building VR/AR/HMD/Hololens applications

11/09/2017 12 Intelligent Environment (IE) design principles

§ The scene represents combination of triggering event(s), (selected preconditions) and the intended state of IE § Computer vision and pattern recognition: Environmental (real world) scene is a place in which we can move and most of the “objects” are within reach (Oliva and Torralba, 2001) § The state is the current status of the intelligent environment constructed by the state of an individual device or the aggregate states of a group of devices

Tiina Kymäläinen, Johan Plomp, Timo Tuomisto, Juhani Heinilä and Timo Urhemaa: Designing smart living for ageing Alice – and the persons next door, Intelligent Buildings International, Taylor & Francis, 2015.

11/09/2017 13 Intention Awareness

Kymäläinen Tiina, Plomp Johan, Heikkilä Päivi and Ailisto Heikki: Intention Awareness: A Vision Declaration and Illustrating Scenarios, Conference proceedings of International Conference on Intelligent Environments, IEEE Computer Society Publications, Shanghai, China, 2014

11/09/2017 14 11/09/2017 15