VTT TECHNICAL RESEARCH CENTRE OF FINLAND LTD Holonovel Workshop Introduction EAI International Conference on Technology, Innovation, Entrepreneurship and Education SEPTEMBER 11–12, 2017 | CANTERBURY, UK Tiina Kymäläinen VTT Technical research centre of Finland Ltd. Human Factors, Virtual- and Augmented Reality Holodeck – Wikipedia definition § The holodeck is a plot device used in stories set within the Star Trek universe § Permit stories with locations and characters that could not otherwise exist in the Star Trek setting https://www.youtube.com/watch?v=oZwtVz7z0wM • Fictional technology – but it has been predicted that similar virtual reality tools "will become a consumer-ready product by 2024” https://en.wikipedia.org/wiki/Holodeck 11/09/2017 2 Science Inspiration § The Star Trek Holodeck was inspired by inventor and holographer, Gene Dolgoff, who is also the inventor of the digital projector § His holography laboratory, built in 1964, was the first holography lab in New York City 11/09/2017 3 Episodes in Star Trek Star Trek ®, Star Trek: The Next Generation ®, Star Trek: Deep Space Nine ® and Star Trek: Voyager ® are trademarks of Paramount Pictures registered in the United States Patent and Trademark Office § Star Trek animated series: § "The Practical Joker” (1974), Season 2, Episode 3 § Next Generation: § "The Big Goodbye” (1988): Season 1, Episode 12 § "Elementary, Dear Data" (1988): Season 2, Episode 3 § "A Fistful of Datas” (1992): Season 6, Episode 8 § Voyager: § "Author, Author”, "Flesh and Blood”, "Nothing Human” § Star Trek: Enterprise: § “Unexpected” (2001), Season 1, Episode 5 11/09/2017 4 Applications § Starfleet personnel use holodecks for training, diagnostics and recreation § Simulate settings and events for analysis, such as to explore crime scene or for scientific experimentation § Holodeck variations of being by the sea, in a forest and being trapped in a snow-storm § An example of the holodeck's recreational functions are the holosuites 11/09/2017 5 “Technology” § Holodecks use two major subsystems: Holographic image and the conversion of matter § The holodeck is depicted as an enclosed room in which objects and people are simulated by a combination of: § Transported matter (Holomatter) § Tractor beams § Shaped force fields § Replicated matter § Controlled by voice commands and physical controls § A hidden computer terminal allows access to the holodeck controls https://en.wikipedia.org/wiki/Holodeck 11/09/2017 6 How does it really work? Simulation types: § The walls can generate holographic images which appear to extend for an unlimited distance § Holograms can be projected into space § Holograms can be augmented with force beams to simulate solid, tangible objects § Holograms and force beams can be augmented with replicator technology to provide actual substance § Magnificent: if an object's status changes the simulation type will change seamlessly. For example, someone sees a distant tree (walls), approaches it (hologram), leans on it (force beams), breaks off a branch (Holodeck-matter), then picks and eats an apple (replicated). Holodeck and Computers FAQ, Joshua Bell: http://www.calormen.com/star_trek/FAQs/holodeck-faq.htm 11/09/2017 7 Applications § Microsoft IllumiRoom Creates illusions of the area surrounding TV § The Void, Utah-based startup: IMAX of VR experiences, multiplayer VR environment § University of Illinois at Chicago: CAVE2: model global weather patterns and help doctors practice surgeries § Real Life Holodeck with an Oculus Rift – Computerphile § VTT: Hololens approach: shared multiuser MR experience § Etc. (growing exponentially) 11/09/2017 8 11/09/2017 9 Holonovels § Writing stories and plotlines for the holodeck is an activity pursued by people known as holonovelists § Most holodeck programs shown in the episodes run in first person "subjective mode” § User actively interacts with the program and its characters § Raise interesting philosophical and sociological issues of virtual realities § A holonovel "Photons Be Free" authored by the Doctor (Episode: "Author, Author") 11/09/2017 10 "Worst Case Scenario" § Star Trek: Voyager , Season 3, Episode 25 11/09/2017 11 Holonovels Design Principles § Current mixed reality environments: operates between projected reality (VR) augmented reality (AR) – transported matter – and real reality – matter/replicated matter § Combines screenwriting, dramaturgy, game design, virtual world construction and e.g. intelligent environment research § Occupies space and highlights the role of artefacts § E.g. bases on sequence construction § Instead of linear timeline follows hierarchy (IF-THEN rules) § Need: Lack of design principles for building VR/AR/HMD/Hololens applications 11/09/2017 12 Intelligent Environment (IE) design principles § The scene represents combination of triggering event(s), (selected preconditions) and the intended state of IE § Computer vision and pattern recognition: Environmental (real world) scene is a place in which we can move and most of the “objects” are within reach (Oliva and Torralba, 2001) § The state is the current status of the intelligent environment constructed by the state of an individual device or the aggregate states of a group of devices Tiina Kymäläinen, Johan Plomp, Timo Tuomisto, Juhani Heinilä and Timo Urhemaa: Designing smart living for ageing Alice – and the persons next door, Intelligent Buildings International, Taylor & Francis, 2015. 11/09/2017 13 Intention Awareness Kymäläinen Tiina, Plomp Johan, Heikkilä Päivi and Ailisto Heikki: Intention Awareness: A Vision Declaration and Illustrating Scenarios, Conference proceedings of International Conference on Intelligent Environments, iEEE Computer Society Publications, Shanghai, China, 2014 11/09/2017 14 11/09/2017 15.