KANYOK, NATHAN J., M.S., December 2019 COMPUTER SCIENCE SITUATIONAL AWARENESS MONITORING FOR HUMANS-IN-THE-LOOP OF TELEPRES- ENCE ROBOTIC SYSTEMS (80 pages) Thesis Advisor: Dr. Jong-Hoon Kim Autonomous automobiles are expected to be introduced in increasingly complex increments until the system is able to navigate without human interaction. However, humanity is uncomfortable with relying on algorithms to make security critical decisions, which often have moral dimensions. Many robotic systems keep humans in the decision making loop due to their unsurpassed ability to perceive contextual information in ways we find relevant. It is likely that we will see transportation systems with no direct human supervision necessary, but these systems do not address our worry about moral decisions. Until we are able to embed moral agency in digital systems, human actors will be the only agents capable of making decisions with security-critical and moral components. Additionally, in order for a human to be in the position that we can have confidence in their decision, they must be situationally aware of the environment in which the decision will be made. Virtual reality as a medium can achieve this by allowing a person to be telepresent elsewhere. A telepresence dispatch system for autonomous transportation vehicles is proposed that places emphasis on situational awareness so that humans can properly be in the decision making loop. Pre-trial, in-trial, and post-trial metrics are gathered that emphasize human health and monitor situational awareness through traditional and novel approaches. SITUATIONAL AWARENESS MONITORING FOR HUMANS-IN-THE-LOOP OF TELEPRESENCE ROBOTIC SYSTEMS A thesis submitted to Kent State University in partial fulfillment of the requirements for the degree of Master of Science by Nathan J. Kanyok December 2019 © Copyright All rights reserved Except for previously published materials Thesis written by Nathan J. Kanyok B.S., Bowling Green State University, 2016 M.S., Kent State University, 2019 Approved by Dr. Jong-Hoon Kim , Advisor Dr. Javed I. Kahn , Chair, Department of Computer Science James L. Blank , Dean, College of Arts and Sciences TABLE OF CONTENTS TABLE OF CONTENTS . iv LIST OF FIGURES . viii LIST OF TABLES . ix ACKNOWLEDGMENTS . x 1 Introduction . 1 1.1 Overview . .1 2 Literature Review . 4 2.1 Introduction to Literature Review . .4 2.2 Robotics . .5 2.2.1 Formalization and History . .5 2.2.2 General Features of Robots . .7 2.2.3 Sense, Think, Act Cycle . .8 2.2.4 Autonomy and Control . .9 2.2.5 Self-Driving Car . 10 2.2.6 Multidisciplinary . 10 2.2.7 Robotic Systems: . 13 2.2.8 Public Perception of Robotics . 15 2.2.9 Robot Ethics . 16 2.3 Telepresence . 16 2.3.1 History of Telepresence . 16 2.3.2 Psychology of Telepresence . 18 2.3.3 Telepresence and Teleoperation . 18 2.3.4 Modalities . 19 iv 2.3.5 Telepresence Design . 19 2.3.6 Applications of Telepresence . 20 2.3.7 Drawbacks of Telepresence . 21 2.4 Virtual Reality . 21 2.4.1 History of Virtual Reality . 21 2.4.2 Applications of Virtual Reality . 22 2.4.3 Types of Virtual Reality . 24 2.4.4 Drawbacks of Virtual Reality . 25 2.4.5 Measuring Virtual Reality . 27 2.5 Human System Interactions . 28 2.5.1 Human Machine Interaction . 29 2.5.2 Human Computer Interaction . 29 2.5.3 Human Robot Interaction . 31 2.6 Human Factors . 35 2.6.1 Definition of Human Factors . 35 2.6.2 Role of Human Psychology . 36 2.6.3 Human Factors Summary . 37 2.7 Human-In-the-Loop . 38 2.7.1 Definition of Human-in-the-Loop Systems . 38 2.7.2 Security Critical Functions . 39 2.7.3 Performance Factors . 39 2.7.4 Information Processing . 40 2.8 Philosophy . 41 2.8.1 Plato's Allegory of the Cave . 41 2.8.2 Leibniz's Possible Worlds . 42 2.8.3 Descartes' Dualism . 43 2.8.4 Ethical Theory . 44 2.9 Literature Conclusion . 45 v 3 Proposed System . 46 3.1 Proposed System . 46 3.1.1 Motivation . 46 3.1.2 Architecture . 47 3.1.3 Trust Within the System . 49 3.2 Assumptions . 51 4 Materials and Methods . 53 4.1 Materials . 53 4.1.1 Robot Model . 53 4.1.2 Software . 55 4.1.3 Hardware . 56 4.1.4 Experiment Setup . 57 5 Results . 63 5.0.1 Demographics . 63 5.0.2 Pre-Trial Health Survey . 63 5.0.3 In-Trial . 63 5.0.4 Post-Trial . 64 6 Discussion . 67 6.1 Discussion of Results . 67 6.1.1 Pre-Trial . 67 6.1.2 Health . 67 6.1.3 In-Trial . 67 6.1.4 Post-Trial . 69 6.2 Application of Results . 69 6.3 Going Forward . 70 6.4 Future Work . 71 6.5 Obstacles of Implementation . 72 6.6 Summary . 74 vi 7 Conclusion . 75 7.1 Literature Review . 75 7.2 Design of Human-in-the-Loop Telepresent Dispatch System for Autonomous Robots . 75 7.3 Experiment . 76 7.4 Future Work . 76 7.5 Summary . ..