Gesture and Action Discovery for Evaluating Virtual Environments

2018 IEEE International Conference on Artificial Intelligence and Virtual

Reality (AIVR 2018)

Taichung, Taiwan

10 – 12 December 2018

IEEE Catalog Number: CFP18O53-POD ISBN: 978-1-5386-9270-7

Copyright and Reprint Permissions: Abstracting is permitted with credit to the source. Libraries are permitted to photocopy beyond the limit of U.S. copyright law for private use of patrons those articles in this volume that carry a code at the bottom of the first page, provided the per-copy fee indicated in the code is paid through Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923.

*** This is a print representation of what appears in the IEEE Digital Library. Some format issues inherent in the e-media version may also appear in this print version.

IEEE Catalog Number: CFP18O53-POD ISBN (Print-On-Demand): 978-1-5386-9270-7 ISBN (Online): 978-1-5386-9269-1

Additional Copies of This Publication Are Available From:

Curran Associates, Inc 57 Morehouse Lane Red Hook, NY 12571 USA Phone: (845) 758-0400 Fax: (845) 758-2633 E-mail: [email protected] Web: www.proceedings.com

2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) AIVR 2018

Table of Contents

Message from the IEEE AIVR 2018 General Co-Chairs xii Message from the IEEE AIVR 2018 Program Co-Chairs xiv IEEE AIVR 2018 Organizing Committee xvi IEEE AIVR 2018 Technical Program Committee Members and Reviewers xviii

Regular Papers

Gesture and Action Discovery for Evaluating Virtual Environments with Semi-Supervised Segmentation of Telemetry Records 1 Andrea Batch (University of Maryland, College Park), Kyungjun Lee (University of Maryland, College Park), Hanuma Teja Maddali (University of Maryland, College Park), and Niklas Elmqvist (University of Maryland, College Park) Web-Based Virtual Reality Development in Classroom: From Learner's Perspectives 11 Vinh T. Nguyen (Texas Tech University), Rebecca Hite (Texas Tech University), and Tommy Dang (Texas Tech University) Uncertainty-Based Deep Learning Networks for Limited Data Wetland User Models 19 Andrew Hoblitzell (Purdue University), Meghna Babbar-Sebens (Oregon State University), and Snehasis Mukhopadhyay (Indiana University–Purdue University Indianapolis) A Natural Language Programming Application for Lego Mindstorms EV3 27 Yue Zhan (Virginia Polytechnic Institute and State University) and Michael S. Hsiao (Virginia Polytechnic Institute and State University) A Study of Situated Product Recommendations in Augmented Reality 35 Brandon Huynh (University of California, Santa Barbara), Adam Ibrahim (University of California, Santa Barbara), Yun Suk Chang (University of California, Santa Barbara), Tobias Höllerer (University of California, Santa Barbara), and John O'Donovan (University of California, Santa Barbara) BRIEF: Backward Reduction of CNNs with Information Flow Analysis 44 Yu-Hsun Lin (HTC Research and Healthcare), Chun-Nan Chou (HTC Research and Healthcare), and Edward Y. Chang (HTC Research and Healthcare)

v Evaluating Engagement of Virtual Reality Games Based on First and Third Person Perspective Using EEG and Subjective Metrics 53 Diego Monteiro (Xi’an Jiaotong-Liverpool University), Hai-Ning Liang (Xi’an Jiaotong-Liverpool University), Andrew Abel (Xi’an Jiaotong-Liverpool University), Nilufar Bahaei (Otago Polytechnic Auckland International Campus), and Rita de Cassia Monteiro (University of Taubaté) Self-Organizing Maps for Intuitive Gesture-Based Geometric Modelling in Augmented Reality 61 Benjamin Felbrich (University of Stuttgart), Gwyllim Jahn (Fologram), Cameron Newnham (Fologram), and Achim Menges (University of Stuttgart) Objective Assessment of Line Distortions in Viewport Rendering of 360º Images 68 Falah Rahim (IST-IT/), Maria Paula Queluz (IST-IT/), and João Ascenso (IST-IT/) A Compensation Method of Two-Stage Image Generation for Human-AI Collaborated In-Situ Fashion Design in Augmented Reality Environment 76 Zhenjie Zhao (Hong Kong University of Science and Technology) and Xiaojuan Ma (Hong Kong University of Science and Technology) Virtual Reality Conferencing 84 Patrick David Pazour (University of Vienna), Andreas Janecek (University of Vienna & Robimo GmbH), and Helmut Hlavacs (University of Vienna) SinkNet: Interactive Sink to Detect Living Habits for Healthcare and Quality of Life Using Private Networks 92 Hiroaki Tobita (Advanced Institute of Industrial Technology), Masaaki Matsumoto (Advanced Institute of Industrial Technology), Yasuhiro Maida (Advanced Institute of Industrial Technology), Yoshiki Eguchi (Advanced Institute of Industrial Technology), and Kayoko Terada (Advanced Institute of Industrial Technology) A Two-Level Planning Framework for Mixed Reality Interactive Narratives with User Engagement 100 Manuel Braunschweiler (ETH Zurich & Disney Research), Steven Poulakos (Disney Research), Mubbasir Kapadia (Rutgers University), and Robert W. Sumner (ETH Zurich & Disney Research) A Method to Build Multi-Scene Datasets for CNN for Camera Pose Regression 108 Yuhao Ma (China Agricultural University), Hao Guo (China Agricultural University), Hong Chen (China Agricultural University), Mengxiao Tian (China Agricultural University), Xin Huo (China Agricultural University), Chengjiang Long (Kitware Inc.), Shiye Tang (China Agricultural University), Xiaoyu Song (China Agricultural University), and Qing Wang (China Agricultural University) Understanding Head-Mounted Display FOV in Maritime Search and Rescue Object Detection 116 Susannah Soon (Curtin University), Artur Lugmayr (Curtin University & Aalto University), Andrew Woods (Curtin University), and Tele Tan (Curtin University)

vi Short Papers

The Virtual Factory: Hologram-Enabled Control and Monitoring of Industrial IoT Devices 120 Vittorio Cozzolino (Technical University of Munich), Oleksii Moroz (Soley GmbH), and Aaron Yi Ding (Delft University of Technology) Supporting the Sense of Unity between Remote Audiences in VR-Based Remote Live Music Support System KSA2 124 Tatsuyoshi Kaneko (Kagawa University), Hiroyuki Tarumi (Kagawa University), Keiya Kataoka (Kagawa University), Yuki Kubochi (Kagawa University), Daiki Yamashita (Kagawa University), Tomoki Nakai (Kagawa University), and Ryota Yamaguchi (Kagawa University) Decoding Subjective Emotional Arousal during a Naturalistic VR Experience from EEG Using LSTMs 128 Simon M. Hofmann (University of Amsterdam), Felix Klotzsche (Humboldt Universität zu Berlin), Alberto Mariola (University of Sussex), Vadim V. Nikulin (Max Planck Institut for Human Cognitive and Brain Sciences), Arno Villringer (Max Planck Institut for Human Cognitive and Brain Sciences), and Michael Gaebler (Max Planck Institut for Human Cognitive and Brain Sciences) Large Scale Information Marker Coding for Augmented Reality Using Graphic Code 132 Bruno Patrão (University of Coimbra), Leandro Cruz (University of Coimbra), and Nuno Gonçalves (Portuguese Mint and Official Printing Office & University of Coimbra) A Benchmark of Four Methods for Generating 360° Saliency Maps from Eye Tracking Data 136 Brendan John (University of Florida), Pallavi Raiturkar (University of Florida), Olivier Le Meur (Univ. Rennes, CNRS, IRISA), and Eakta Jain (University of Florida) Downsizing: The Effect of Mixed-Reality Person Representations on Stress and Presence in Telecommunication 140 Michal Joachimczak (National Institute of Information and Communications Technology), Juan Liu (National Institute of Information and Communications Technology & Osaka University), and Hiroshi Ando (National Institute of Information and Communications Technology & Osaka University) A Perceptual Evaluation of Generative Adversarial Network Real-Time Synthesized Drum Sounds in a Virtual Environment 144 Minwook Chang (Korea University), Youngwon Ryan Kim (Korea University), and Gerard Jounghyun Kim (Korea University) Exploiting the Integration of Wearable Virtual Reality and Bio-Sensors for Persons with Neurodevelopmental Disorders 149 Franca Garzotto (Politecnico di Milano), Nicolò Messina (Politecnico di Milano), Vito Matarazzo (Politecnico di Milano), Riccardo Facchini (Politecnico di Milano), Lukasz Moskwa (Politecnico di Milano), and Gianluigi Oliva (Politecnico di Milano) Machine Learning Architectures to Predict Motion Sickness Using a Virtual Reality Rollercoaster Simulation Tool 153 Stefan Hell (Kingston University) and Vasileios Argyriou (Kingston University)

vii Trajectory-Based Viewport Prediction for 360-Degree Virtual Reality Videos 157 Stefano Petrangeli (Adobe Research), Gwendal Simon (Adobe Research), and Viswanathan Swaminathan (Adobe Research)

Industry Track Papers

Intelligent Wearable Virtual Reality (VR) Gaming Controller for People with Motor Disabilities 161 Ker-Jiun Wang (University of Pittsburgh), Quanbo Liu (University of Pittsburgh), Yifan Zhao (University of Pittsburgh), Caroline Yan Zheng (Royal College of Art), Soumya Vhasure (University of Pittsburgh), Quanfeng Liu (University of Pittsburgh), Prakash Thakur (University of Pittsburgh Medical Center), Mingui Sun (University of Pittsburgh), and Zhi-Hong Mao (University of Pittsburgh) Virtual Crime Scene 165 Jeroen Nelis (Belgian Defense), Stef Desmet (KU Leuven), Jeroen Wauters (Groep T & KU Leuven), Rob Haelterman (Royal Military Academy), Erwin Borgers (3D Expertise), and Dimitri Kun (FGP Limburg) Graphic Code: A New Machine Readable Approach 169 Leandro Cruz (University of Coimbra), Bruno Patrão (University of Coimbra), and Nuno Gonçalves (Portuguese Mint and Official Printing Office & University of Coimbra) DeepLinQ: Distributed Multi-Layer Ledgers for Privacy-Preserving Data Sharing 173 Edward Y. Chang (HTC Healthcare), Shih-Wei Liao (National Taiwan University), Chun-Ting Liu (HTC Healthcare), Wei-Chen Lin (National Taiwan University), Pin-Wei Liao (National Taiwan University), Wei-Kang Fu (National Taiwan University), Chung-Huan Mei (HTC Healthcare), and Emily J. Chang (HTC Healthcare) Exploring Seismic Data through Virtual Reality and Hybrid Knowledge Base 179 Wallas Henrique Sousa dos Santos (IBM Research), Reinaldo Mozart Da Gama e Silva (IBM Research), Rordrigo Costa Mesquita Santos (IBM Research), and Marcio Ferreira Moreno (IBM Research)

Demos and Video Demos

Head-Mounted FOV Simulator for User Testing of Maritime Object Detection Tasks 183 Susannah Soon (Curtin University), Artur Lugmayr (Curtin University & Aalto University), Andrew Woods (Curtin University), and Tele Tan (Curtin University) AI for Toggling the Linearity of Interactions in AR 185 Jing Qian (Brown University), Laurent Denoue (FX Palo Alto Laboratory), Jacob Biehl (FX Palo Alto Laboratory), and David A. Shamma (FX Palo Alto Laboratory) Exploring Cultural Heritage in Augmented Reality with GoFind! 187 Loris Sauter (University of Basel), Luca Rossetto (University of Basel), and Heiko Schuldt (University of Basel)

viii HoloLearn: Learning through Mixed Reality for People with Cognitive Disability 189 Franca Garzotto (Politecnico di Milano), Emanuele Torelli (Politecnico di Milano), Francesco Vona (Politecnico di Milano), and Beatrice Aruanno (Politecnico di Milano) A Natural Language Programming Application for Lego Mindstorms EV3 191 Yue Zhan (Virginia Polytechnic Institute and State University) and Michael Hsiao (Virginia Polytechnic Institute and State University) An Augmented Reality Application Using Graphic Code Markers 193 Bruno Patrão (University of Coimbra), Leandro Cruz (University of Coimbra), and Nuno Gonçalves (Portuguese Mint and Official Printing Office & University of Coimbra) A Knowledge Oriented Virtual Reality Tool for Exploring Seismic Data 195 Wallas Henrique Sousa dos Santos (IBM Research), Reinaldo Mozart Gama e Silva (IBM Research), Rodrigo Costa Mesquita Santos (IBM Research), and Marcio Ferreira Moreno (IBM Research)

2nd Workshop on Interface and Experience Design with AI for VR/AR (DAIVAR 2018)

FoodChangeLens: CNN-Based Food Transformation on HoloLens 197 Shu Naritomi (University of Electro-Communications, Tokyo), Ryosuke Tanno (NTT Communications Corporation), Takumi Ege (University of Electro-Communications, Tokyo), and Keiji Yanai (University of Electro-Communications, Tokyo) Encompassing English Language Learners in Virtual Reality 200 Eric Nersesian (New Jersey Institute of Technology), Adam Spryszynski (New Jersey Institute of Technology), Ulysee Thompson (New Jersey Institute of Technology), and Michael Lee (New Jersey Institute of Technology) Comparative Reality: Measuring User Experience and Emotion in Immersive Virtual Environments 204 Adam Greenfeld (Curtin University), Artur Lugmayr (Curtin University), and Wesley Lamont (Curtin University) Design and Development of Interactive Intelligent Medical Agent 210 Shu-Chiao Tsai (National Taipei University), Hooman Samani (National Taipei University), Yu-Wei Kao (National Taipei University), Kening Zhu (City University of Hong Kong), and Brian Jalaian (U.S. Army Research Laboratory) Motorcycle Riding Safety Education with Virtual Reality 216 Chun Chia Hsu (Lunghwa University of Science and Technology), Yu Li Chen (Lunghwa University of Science and Technology), Wen Ching Chou (Ministry of Transportation and Communications), Shih Hsuan Huang (Ministry of Transportation and Communications), and Kai Kuo Chang (Ministry of Transportation and Communications) Converting Natural Language Text to ROS-Compatible Instruction Base 219 Takondwa Kakusa (Virginia Polytechnic Institute and State University) and Michael Hsiao (Virginia Polytechnic Institute and State University)

ix Interface and Experience Design with AI for VR/AR (DAIVAR'18) and AI/ML for Immersive Simulations (AMISIM'18) 227 Kening Zhu (City University of Hong Kong), Artur Lugmayr (Curtin University & Aalto University), Xiaojuan Ma (Hong Kong University of Science and Technology), Florian "Floyd" Mueller (RMIT University), Ulrich Engelke (Data61), Simeon Simoff (Western Sydney University), Tomas Trescak (Western Sydney University), Anton Bogdavych (Western Sydney University), Juan Rodriguez Aguilar (Spanish Research Council Catalonia), and Huyen Vu (University of New South Wales) Evaluating the Effects of a Cartoon-Like Character with Emotions on Users' Behaviour within Virtual Reality Environments 229 Diego Monteiro (Xi'an Jiaotong-Liverpool University), Hai-Ning Liang (Xi'an Jiaotong-Liverpool University), Jialin Wang (Xi’an Jiaotong-Liverpool University), Luhan Wang (Xi’an Jiaotong-Liverpool University), Xian Wang (Xi’an Jiaotong-Liverpool University), and Yong Yue (Xi’an Jiaotong-Liverpool University) Integrating Biomechanical and Animation Motion Capture Methods in the Production of Participant Specific, Scaled Avatars 237 Luke Hopper (Edith Cowan University) and Nahoko Sato (Nagoya Gakuin University)

Workshop on AI & VR in Medicine (WARM 2018)

Augmented Reality Simulation of Cardiac Circulation Using APPLearn (Heart) 241 Ryan Kyaw Thu Aung Ba (Nanyang Technological University), Yiyu Cai (Nanyang Technological University), and Yunqing Guan (Nanyang Technological University) The Prospect for the Application of the Surgical Navigation System Based on Artificial Intelligence and Augmented Reality 244 Yang Liu (University of Montpellier & Shanghai Lin Yan Medical Technology Co., Ltd.) and Pinpin Tang (Shanghai Lin Yan Medical Technology Co., Ltd.) A Virtual Reality Based Simulator for Training Surgical Skills in Procedure of Catheter Ablation 247 Haoyu Wang (University of Chinese Academy of Sciences & Shenzhen Institutes of Advanced Technology), Sheng Jiang (University of Chinese Academy of Sciences & Shenzhen Institutes of Advanced Technology), and Jianhuang Wu (Shenzhen Institutes of Advanced Technology)

Workshop on Emerging Topics in XR (ETXR 2018)

Machine-Learning Based Fitness Behavior Recognition from Camera and Sensor Modalities 249 Chih-Chieh Fang (Taipei National University of Art), Ting-Chen Mou (National Central University, Taiwan), Shih-Wei Sun (Taipei National University of Art), and Pao-Chi Chang (National Central University, Taiwan)

x Combining Leap Motion with Unity for Virtual Glove Puppets 251 Chi-Yen Lin (National Central University), Zhang-Hao Yang (National Central University), Heng-Wei Zhou (National Central University), Tsai-Ni Yang (National Central University), Hong-Nien Chen (National Central University), and Timothy K. Shih (National Central University) Towards a Music Visualization on Robot (MVR) Prototype 256 Pei-Chun Lin (Feng Chia University), David Mettrick (University of Ontario), Patrick C.K. Hung (University of Ontario), and Farkhund Iqbal (Zayed University) Omni-Learning XR Technologies and Visitor-Centered Experience in the Smart Art Museum 258 Tsang-Gang Lin (Industrial Technology Research Institute), Hsiang-Lan Shih (Industrial Technology Research Institute), Chun-Ting Lee (Industrial Technology Research Institute), Hui-Yu Hsieh (Industrial Technology Research Institute), Yi-Yuan Chen (Industrial Technology Research Institute), and Chien-Kuo Liu (Museum of National Taipei University of Education) A Combination of Feedback Control and Vision-Based Deep Learning Mechanism for Guiding Self-Driving Cars 262 Wen-Yen Lin (National Taichung University of Science and Technology), Wang-Hsin Hsu (National Taichung University of Science and Technology), and Yi-Yuan Chiang (Vanung University) Visual Augmentation of Printed Materials with Intelligent See-Through Glass Displays: A Prototype Based on Smartphone and Pepper's Ghost 267 Frode Eika Sandnes (Oslo Metropolitan University) and Evelyn Eika (Oslo Metropolitan University) Planar Simplification of Indoor Point-Cloud Environments 274 Stephan Feichter (University of Vienna) and Helmut Hlavacs (University of Vienna)

Author Index 283