FULLTEXT01.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
Designing transparent display experience through the use of kinetic interaction A Master’s Thesis by Interaction Design Master’s Programme Rafael Rybczyński School of Arts and Communication (K3) Malmö University, Sweden August 2017 MSwedenugust 2017 Designing transparent display experience through the use of kinetic interaction Interaction Design Master’s Programme School of Arts and Communication (K3) Malmö University, Sweden August 2017 Author: Rafael Rybczyński Supervisor: Susan Kozel Examiner: Clint Heyer Thesis Project I 15 credits 2017 Acknowledgements Over the lengths of this thesis project I have received support and encouragement from a many people. First of all, I would like to thank my supervisor Susan Kozel for advising me to trust my own instinct. Moreover I would like humbly to thank all the participants without whom this project would not have been possible. Also humbly I thank Alvar, Sanna, and Susanne for shelter and feedback. For proofreading my thanks goes to Jan Felix Rybczyński and Sanna Minkkinen. Finally, my deepest gratitude is reserved to my father Dr. med. Jerzy Antoni Rybczyński. Rest in peace. I would have wished for you reading this paper. Author’s notes Often during the process of doing this study I was asked why transparent displays. What’s the point? Probably I would say this goes way back to my childhood when enjoying my father reading novel stories to us as good night stories. A good example of great stories was Scheerbart’s utopian architectural vision The Gray Cloth who told about the great Swiss architect who traveled by an airship. Wherever he went, he designed buildings made from brightly colored glass, including the roof. Modern architecture has always fascinated me, and even though I did not study architecture I choose eventually to learn in interaction design to understand how close this discipline goes hand in hand with architectural principles and thoughts. Abstract This essay presents a study into the domain of architecture meeting new interaction design principles. The paper discusses future transparent surfaces to become programmable kinetic user interfaces, usable as information and communication channels to simplify our everyday environment. Based on the approach of using the five methodologies: Cultural Probes, Research Through Design, Grounded Theory, Star Life Cycle Model and Wizard of Oz; consistent data was collected to design and iterate on a visionary interface prototype to bridge the use of freehand gestures through motion sensing and moreover supported by RFID in a building structure on a see-through background. The objective of this paper is to unravel the main research question of how can people through kinetic interaction use organic interfaces on transparent surfaces? Several possible uses were ideated such as multiple shared user access, collaborative interaction on both sides. The primary research was answered through a final presented prototype combining a CV system with RFID for multiple and collaborative usages. User experiences and feedback makes an array of applications possible how a transparent interfaces with kinetic interaction can be applied to the interior and exterior such as fridge, mirror, doors, glass panels, alarm systems, games and the home entertainment. In today’s norm screens in the shape of a square are obsolete and support of new patterns, forms and materials are needed. Fieldwork concluded that kinetic interaction could flawlessly unite real world conditions with computer-generated substance, and become the design environment for future interactions to communicate with the user. We no longer seek to be bound to stiff shaped Graphical User Interfaces. Adding a transparent surface as background for such kinetic motion is underlying paradigm for the content to be projected into any ambience and surroundings. List of abbreviations 2D - Two-dimensional image 3D - Three-dimensional image AI – Artificial Intelligence AR - Augmented Reality AV - Audio/Video CP – Cultural probes CPU - Central Processing Unit CV – Computer Vision DYI – Do-it-Yourself EC - Electrochromic smart windows / Switchable glass FOLED - Flexible organic light-emitting diodes GUI – Graphical User Interface HCI - Human-Computer Interaction HOE - Holographic optical elements HUD - Head-up Display KOI - Kinetic Organic Interface KUI – Kinetic User Interfaces LCD - Liquid Crystal Display LED - Light-emitting diodes I/O – Input/Output IR - Infrared ITD - Interactive Transparent Display MIT - Massachusetts Institute of Technology MR - Mixed Reality NUI – Natural User Interface OLED - Organic light-emitting diodes OUI - Organic User Interfaces RAM – Random Access Memory RFID - Radio frequency identification Rtd - Research through Design RUI - Remote-use User Interface SDK – Software Development Kit SAR – Spatial Augmented Reality TUI – Tangible User Interface Ubicomp – Ubiquitous computing UI – User Interface VR - Virtual Reality WYSIWYG - What-You-See-Is-What-You-Get Keywords and Phrases This report will use keywords and phrases, which may require further clarification: KOI is an acronym for Kinetic Organic Interface that can have any shape or form. This new class of emerging KOI employ kinetic motion to embody and communicate information to people. (Parker et al., 2008)1 Multimodal interfaces provide users with greater expressive power, naturalness, flexibility through combining modalities such as body movements, hand gestures, touch, speech, pen, sight and sound to enrich humans experience (Oviatt, 2002) Matter is one possible future scenario of continued progress in nanoscale technology that is based to “create a physical artefact using programmable matter that will eventually be able to mimic the original object’s shape, movement. Visual appearance, sound and tactile qualities” 2 3 (Goldstein et al., 2005) Programmable physical architecture is a visionary concept for “future architecture where physical features of architectural elements and facades can be dynamically changed and reprogrammed according to people’s needs”4 (Rekimoto, 2012) Switchable glass is also known as Electrochromic (EC) smart windows. ECs are capable to control visible light and solar radiation into buildings. Moreover EC can “impart energy efficiency as well as human comfort by having different transmittance levels depending on dynamic need.” 5 Smart windows are currently being used in an increasing number of constructions in architecture. (Granqvist, 2014) 1 Oviatt, S. (1999), “Ten Myths of Multimodal Interaction”, November1999/Vol.42, No.11 Communication of the ACM 2 http://www.cs.cmu.edu/~./claytronics/ 3 Goldstein, S. et al. (2005), “Programmable Matter”, Invisible Computing, Carnegie Mellon University, pp. 99-101 4 Rekimoto, J. et al. (2012), “Squama: Modular Visibility Control of Walls and Windows for Programmable Physical Architectures”, AVI 12, May 21-25, 2012, Capri Island, Italy, Copyright 2012 ACM 978-1-4503-1287-5/12/05 5 Claes G. Granqvist, C.G. (2014), “Electrochromics for smart windows: Oxide-based thin films and devices”, Thin Solid Films 564, 2014 Elsevier B.V., pp. 1–38 “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it“ Mark Weiser Table of contents 1. Introduction ..................................................................................................... 10 2. Research Focus .............................................................................................. 13 3. Methodology .................................................................................................... 14 4. Literature Review and Related work ................................................................ 16 1. 4.1. Living and adapting to emerging technologies in the home domain ..... 16 4.2. Ubiquitous computing and the Internet of Things .................................. 17 4.3. Window – A static information medium in the peripheral background ... 18 4.4. Tangible User Interfaces ........................................................................ 21 4.5. Vision and RFID sensors ....................................................................... 22 4.6. Computer Vision, Depth-sensing cameras and dynamic environments . 23 4.7. Multi-touch, NUIs, Surface Computing, OUIs ......................................... 25 4.8. Gesture recognition ................................................................................ 27 5. Summary ........................................................................................................... 28 6. Exploratory Research & Field Studies ............................................................... 29 6.1. Brainstorming, set up of goals and requirements .................................... 29 6.2. The Scenario workshop ........................................................................... 32 7. Lo-fi experiments – Project Nimetön ................................................................... 35 7.1. Paper mock-up ......................................................................................... 35 7.2. Prototyping Nimetön ................................................................................. 36 7.3. Validation Phase ....................................................................................... 36 7.4. Evaluation Phase ...................................................................................... 38 7.5. Summary ..................................................................................................