DOCSLIB.ORG

End-User Robot Programming Using Mixed Reality

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
End-User Robot Programming Using Mixed Reality End-User Robot Programming Using Mixed Reality Samir Yitzhak Gadre1, Eric Rosen1, Gary Chien1, Elizabeth Phillips1;2, Stefanie Tellex1, George Konidaris1 Abstract— Mixed Reality (MR) is a promising interface for robot programming because it can project an immersive 3D visualization of a robot’s intended movement onto the real world. MR can also support hand gestures, which provide an intuitive way for users to construct and modify robot motions. We present a Mixed Reality Head-Mounted Display (MR- HMD) interface that enables end-users to easily create and edit robot motions using waypoints. We describe a user study where 20 participants were asked to program a robot arm using 2D and MR interfaces to perform two pick-and-place tasks. In the primitive task, participants created typical pick- and-place programs. In the adapted task, participants adapted their primitive programs to address a more complex pick- (a) A screenshot from the MR (b) After creating the waypoints, and-place scenario, which included obstacles and conditional perspective of a user program- users can visualize the robot arm reasoning. Compared to the 2D interface, a higher number of ming a robot motion. Users can motion that is planned through users were able to complete both tasks in significantly less time, specify green waypoints. the waypoints. and reported experiencing lower cognitive workload, higher usability, and higher naturalness with the MR-HMD interface. Fig. 1: Our mixed reality interface. I. INTRODUCTION For robots to become widely used, humans must be able We propose an end-user mixed reality-based visual pro- to program their actions. For example, consider the task of gramming framework for creating and modifying waypoints binning items. A roboticist might accomplish this task by to create complex, multistep robot actions (Fig. 1). Users specifying a series of waypoints in computer code for the can specify and group waypoints together to create primitive robot to visit one by one. If the action must be modified, the motions, and adapt these waypoints to perform similar tasks. roboticist would explicitly modify the waypoints specified Our interface allows users to visualize the entire motion the in the code. This method is widely popular, but will not robot plans to perform through the waypoints and have the work for end-users. The abstraction of breaking down actions robot execute it. We use a commercially available Mixed into a series of waypoints could be communicated, but Reality Head-Mounted Display (MR-HMD), the Microsoft requiring the use of programming languages to specify those HoloLens [7]. waypoints is beyond their scope. Therefore, we need an The MR-HMD at first seems to pose only advantages over alternate method of interfacing with the waypoint action other visual programming interfaces by combining the robot system. workspace and the GUI space for the end-user. However, Visual programming is one such methodology. A broad there are limitations to the technology that do not make term, visual programming refers to a class of interfaces it obviously preferable to a 2D interface. For example, the where one interacts with a visual representation of ob- HoloLens has a limited field of view, so it relies on the jects, variables, classes, etc. [1]. Visual programming has user to move around to get a full view of the MR scene. been a popular tool for programming computers by non- Furthermore, imperfect hand tracking via computer vision programmers in visual art [2], audio production [3], and makes selection and dragging gestures less reliable than education [4] because it allows users to focus on algorithmic mouse clicks, especially for novice HoloLens users. thinking, rather than the syntax needed to express their intent. We therefore conducted a user study with 20 participants In the world of robotics, RViz and the Interactive Markers and compared the effectiveness of our MR interface to package [5] allow for the creation of visual programming in- a 2D visual programming interface for two similar pick- terfaces that control and visualize a robot. Robot researchers and-place tasks. In the first task, participants programmed have also investigated how effective visual programming primitive robot motions to pick up a block and place it on is by creating and evaluating their own frameworks [6]. a platform. In the second task, participants adapted their Using our binning task from before, it would be possible primitive robot programs to sequentially pick-and-place two to create and modify waypoints visually in a keyboard and cubes on different platforms. Our results show that compared mouse interface. However, users would not see the waypoints to the 2D interface, a higher number of users were able to overlayed on the real robot environment. complete both tasks in significantly less time. Users reported 1Computer Science, Brown University experiencing lower cognitive workload, higher usability and 2Behavioral Sciences and Leadership, United States Air Force Academy naturalness with the MR-HMD interface. II. RELATED WORK VR headset, because they could navigate and interact with the scene by moving their head and hands naturally. This The traditional way to program a robot is to write code. contrasts with keyboard and mouse actions that a typical ROS [8] is an extremely powerful middleware environment 2D interface provides. Like the VR headset, the MR-HMD for roboticists. ROS includes packages to allow programmers allows users to navigate and interact with the perceived to use languages like C++ and Python to interface with robot environment using natural actions. However, an additional hardware. However, leveraging the expertise of end-users benefit of the MR-HMD is that it allows the user to also see that lack software engineering skills would help make robots the real world. more widely accessible. Rosen et al. [15] created an open-source ROS package, ROS also includes many graphical user interfaces (GUIs), ROS Reality, which enables robots to convey intent. The such as RViz [5], for visualization. RViz can display robot package allows the robot to display its intended path as a sensor data on a 2D screen and can be connected to holographic time-lapse trail to a user wearing a MR-HMD. MoveIt! [9] motion planners to enable users to program Rosen et al. [15] conducted a user study to compare the robot movement via keyboard and mouse. 2D interfaces have speed and task completion rate of novice participants using been shown to be useful for robot programming, but have a HoloLens and a 2D monitor interface to determine if a their own shortcomings with regards to immersiveness and proposed robot trajectory would collide with the environ- intuitiveness. They force users to interpret 3D information on ment. They found that the MR-HMD increased accuracy a 2D platform and use control interfaces that do not match and lowered task completion time. While Rosen et al. [15] how users interact with the world. showed the promise of using a MR-HMD for visualizing Alexandrova et al. [6] created a 2D visual programming robot motion to non-experts, they did not address how language, RoboFlow, to enable end-users to easily program effective MR-HMD is for programming these motions. The mobile manipulators to perform general tasks in open- HoloLens’s limited hand-gesture tracking capabilities pose environments. Action editing was important to resolve errors the possible issue that MR-HMD may not be an effective [6]. interface for creating these robot actions. Alexandrova et al. [10] developed a framework that en- Walker et al. [16] investigated different MR signalling ables users to provide a demonstration of a task and then mechanisms for a drone to communicate with a human. use an intuitive monitor and mouse interface to edit demon- They found quantitative task efficiency was significantly strations for adaption to new tasks. Elliott et al. [11] extended higher when using MR signals than when using physically- this work to allow for the grouping of several poses relative embodied signals. to landmarks or objects in the scene. Having an intuitive Fang et al. [17] created an MR interface to program and interface for non-experts to edit example motions for robots visualize robot trajectories. However, they do so using a 2D is especially important in Learning from Demonstration interface. Ni et al. [18] evaluated the effectiveness of an (LfD), where collecting many examples needed for learning augmented reality interface against traditional robot welding is not always feasible. In such cases, a more preferable programming methodologies. Virtual representations of the method may be to have the human perform one general robot trajectory were visualized over a video feed of the real demonstration of the task, and then adapt parts of their robot, enabling novice users to use the augmented reality programmed action to new environments. Both Alexandrova interface to program new welding paths for the robot to act et al. [10] and Elliott et al. [11] used a 2D monitor interface out. Ni et al. [18] found that the augmented reality interface to adapt robot programs. allowed users to program robot actions more quickly and Language is a well-studied modality for programming intuitively, especially for users without prior computer-aided robots because it is one of the main ways humans commu- design knowledge. However, the 2D interfaces of both Fang nicate with each other. Language has been shown to be an et al. [17] and Ni et al. [18] force users to look at a screen efficient way to parameterize a task through descriptions of and not strictly at the robot workspace, which poses safety key items and locations in the environment [12]. Forbes et issues when collaborating with a robot in close-quarters. al. [13] developed a natural language interface to allow users On the other hand, MR-HMDs allow users to both provide to program robot motions via LfD.
Load more

Recommended publications
  • The Influence of Mixed Reality on Satisfaction and Brand Loyalty: a Brand Equity Perspective
    Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 31 January 2020 doi:10.20944/preprints202001.0384.v1 Peer-reviewed version available at Sustainability 2020, 12, 2956; doi:10.3390/su12072956 The Influence of Mixed Reality on Satisfaction and Brand Loyalty: A Brand Equity Perspective Sujin Bae, Ph.D. Candidate School of Management, Kyung Hee University Kyung Hee Dae Ro 26, Dong Dae Mun Gu Seoul 02447, Korea Tel: +82 2 961 9491, E-Mail: [email protected] Timothy Hyungsoo Jung, Ph.D. Director of Creative AR & VR Hub Faculty of Business and Law Manchester Metropolitan University, Righton Building, Cavendish Street, Manchester M15 6BG, UK Tel: +44 161-247-2701, Email: [email protected] Natasha Moorhouse, Ph.D. Research Associate Faculty of Business and Law Manchester Metropolitan University All Saints Building, All Saints, Manchester M15 6BH, UK E-Mail: [email protected] Minjeong Suh, Ph.D. Adjunct Professor Graduate school of international tourism, Hanyang University 222, Wangsimni-ro, Seongdong-gu Seoul 04763, Korea Tel: +82 (0)10 5444 4706, E-Mail: [email protected] Ohbyung Kwon,* Ph.D. School of Management, Kyung Hee University Kyung Hee Dae Ro 26, Dong Dae Mun Gu Seoul 02447, Korea Tel: +82 2 961 2148, E-Mail: [email protected] * Corresponding author. © 2020 by the author(s). Distributed under a Creative Commons CC BY license. Preprints (www.preprints.org) | NOT PEER-REVIEWED | Posted: 31 January 2020 doi:10.20944/preprints202001.0384.v1 Peer-reviewed version available at Sustainability 2020, 12, 2956; doi:10.3390/su12072956 The Influence of Mixed Reality on Satisfaction and Brand Loyalty: A Brand Equity Perspective Abstract Mixed reality technology is being increasingly used in cultural heritage attractions to enhance visitors’ experience.
    [Show full text]
  • Introduction to Minitrack: Mixed, Augmented and Virtual Reality: Co- Designed Services and Applications
    Proceedings of the 52nd Hawaii International Conference on System Sciences | 2019 Introduction to Minitrack: Mixed, Augmented and Virtual Reality: Co- designed Services and Applications Petri Parvinen, Minitrack Chair Juho Hamari Essi Pöyry University of Helsinki University of Tampere University of Helsinki [email protected] [email protected] [email protected] Virtual reality (VR) refers to computer safety, privacy and confidentiality and immersive technologies that use software to generate the experiences [1, 2, 4, 5, 8, 12]. A co-created realistic images, sounds and other sensations that envisioning of an immersive experience also elevates represent an immersive environment and simulate a institutions of agreement, commonly coined as a user’s physical presence in this environment [10]. feeling of win-win. From this point of view, the Mixed reality (MR) refers to combining real and major challenge for both VR and AR technologies is virtual contents with the aid of digital devices [3]. to convince users that the added value is high enough Mixed reality is seen to consist of both augmented to compete with the current systems and offerings in reality (i.e., virtual 3D objects in immersive reality), desktops, notebooks, tablets, smartphones and related and augmented virtuality (i.e., captured features of video and game-like applications. reality in immersive virtual 3D environments) [7]. The minitrack encouraged submissions from both All these technologies have recently peaked in terms cutting edge technology and practical applications. of media attention as they are expected to disturb The minitrack showcases research on: existing markets like PCs and smartphones did when · Realistic virtual humans in immersive they were introduced to the markets.
    [Show full text]
  • Virtual and Mixed Prototyping Techniques and Technologies for Consumer Product Design Within a Blended Learning Design Environment
    INTERNATIONAL DESIGN CONFERENCE - DESIGN 2018 https://doi.org/10.21278/idc.2018.0428 VIRTUAL AND MIXED PROTOTYPING TECHNIQUES AND TECHNOLOGIES FOR CONSUMER PRODUCT DESIGN WITHIN A BLENDED LEARNING DESIGN ENVIRONMENT M. Bordegoni, F. Ferrise, R. Wendrich and S. Barone Abstract Both physical and virtual prototyping are core elements of the design and engineering process. In this paper, we present an industrial case-study in conjunction with a collaborative agile design engineering process and “methodology.” Four groups of heterogeneous Post-doc and Ph.D. students from various domains were assembled and instructed to fulfill a multi-disciplinary design task based on a real-world industry use-case. We present findings, evaluation, and results of this study. Keywords: virtual prototyping, virtual reality (VR), augmented reality (AR), engineering design, collaborative design 1. Introduction Virtual and mixed prototyping within a collaborative blended learning environment is considered an enabling design support technique that allows designers to gain first-hand appreciation of existing or near-future conditions through active engagement with a wide variety in prototyping techniques and prototypes. The product design process (PDP) follows a dual approach and method based on a combination of a blended learning environment (Boelens et al., 2017) and interactive affective experience prototyping (Buchenau and Suri, 2000). The virtual prototyping summer school (VRPT-SS) facilitates this combinatorial collaborative setting to immerse learners in the externalization and representation of their ideas quickly, exploring and communicating their ideas with others. During the course of the week a series of evaluations on the prototyping progressions are made. The spectrum of virtual and mixed prototyping is quite extensive and has multiple definitions and meanings according to literature (see Wang, 2002, and Zorriassatine et al., 2003).
    [Show full text]
  • Growth of Immersive Media- a Reality Check
    Growth of Immersive Media - A Reality Check | Growth of Immersive Media- A Reality Check 2019 1 Growth of Immersive Media- Growth of Immersive Media - A Reality Check | Table of Contents A Reality Check Table of Contents 1. Report Introduction & Methodology 3 2. Executive Summary 4 3. Overview & Evolution 6 4. Immersive Media Ecosystem 10 5. Global Market Analysis 16 6. Applications & Use Cases 21 7. Emerging Technology Innovations 31 8. The India Story 35 9. Recommendations 51 Profiles of Key Players 55 Appendix – Additional Case Studies 65 Acknowledgements 71 Glossary of Terms 72 About Nasscom 73 Disclaimer 74 1 Growth of Immersive Media - A Reality Check | Table of Contents 2 Growth of Immersive Media - A Reality Check | Report Introduction & Methodology Report Introduction 1 & Methodology Background, Scope and Objective • This report aims to provide an assessment of the current market for Immersive Media both globally and in India. • This report provides a summary view of the following with respect to Immersive Media: Brief Introduction & its Evolution, Ecosystem Players, Market Size and Growth Estimates, Key Applications and Use Cases, Analysis of the Indian Market, Key demand drivers, Factors limiting growth, along with the key recommendations that may propel the growth of the Immersive Media market in India. Details of Research Tools and Methodology • We conducted Primary and Secondary Research for our market analysis. Secondary research formed the initial phase of our study, where we conducted data mining, referring to verified data sources such as independent studies, technical journals, trade magazines, and paid data sources. • As part of the Primary Research, we performed in-depth interviews with stakeholders from throughout the Immersive Media ecosystem to gain insights on market trends, demand & supply side drivers, regulatory scenarios, major technological trends, opportunities & challenges for future state analysis.
    [Show full text]
  • Lightdb: a DBMS for Virtual Reality Video
    LightDB: A DBMS for Virtual Reality Video Brandon Haynes, Amrita Mazumdar, Armin Alaghi, Magdalena Balazinska, Luis Ceze, Alvin Cheung Paul G. Allen School of Computer Science & Engineering University of Washington, Seattle, Washington, USA {bhaynes, amrita, armin, magda, luisceze, akcheung}@cs.washington.edu http://lightdb.uwdb.io ABSTRACT spherical panoramic VR videos (a.k.a. 360◦ videos), encoding one We present the data model, architecture, and evaluation of stereoscopic frame of video can involve processing up to 18× more LightDB, a database management system designed to efficiently bytes than an ordinary 2D video [30]. manage virtual, augmented, and mixed reality (VAMR) video con- AR and MR video applications, on the other hand, often mix tent. VAMR video differs from its two-dimensional counterpart smaller amounts of synthetic video with the world around a user. in that it is spherical with periodic angular dimensions, is nonuni- Similar to VR, however, these applications have extremely de- formly and continuously sampled, and applications that consume manding latency and throughput requirements since they must react such videos often have demanding latency and throughput require- to the real world in real time. ments. To address these challenges, LightDB treats VAMR video To address these challenges, various specialized VAMR sys- data as a logically-continuous six-dimensional light field. Further- tems have been introduced for preparing and serving VAMR video more, LightDB supports a rich set of operations over light fields, data (e.g., VRView [71], Facebook Surround 360 [20], YouTube and automatically transforms declarative queries into executable VR [75], Google Poly [25], Lytro VR [41], Magic Leap Cre- physical plans.
    [Show full text]
  • Assessing the Quantitative and Qualitative Effects of Using Mixed
    23rd ICCRTS Info. Central, Pensacola, Florida 6 - 9 November 2018 Assessing the quantitative and qualitative effects of using mixed reality for operational decision making Mark Dennison1, Jerald Thomas2, Theron Trout1,3 and Evan Suma Rosenberg2 1U.S. Army Research Laboratory West, Playa Vista, CA 2University of Minnesota, Minneapolis, MN 3Stormfish Scientific, Chevy Chase, MD Abstract parate systems (physical objects, computers, paper documents, etc.) that require a significant amount The emergence of next generation VR and AR de- of resources and effort to bring into a unified space. vices like the Oculus Rift and Microsoft HoloLens Human interactions require shared cognitive mod- has increased interest in using mixed reality (MR) for els where interaction with systems must support and simulated training, enhancing command and control, maintain this shared representation, stored informa- and augmenting operator effectiveness at the tactical tion must persist the representation at a fundamen- edge. It is thought that virtualizing mission relevant tal data-level, and the underlying network must allow battlefield data, such as satellite imagery or body- these data and information to flow without hindrance worn sensor information, will allow commanders and between human collaborators and non-human agents analysts to retrieve, collaborate, and make decisions (1; 2). about such information more effectively than tradi- The emergence of Mixed Reality (MR) technolo- tional methods, which may have cognitive and spa- gies has provided the potential for new methods for tial constraints. However, there is currently little the Warfighter to access, consume, and interact with evidence in the scientific literature that using mod- battlefield information. MR may serve as a unified ern MR equipment provides any qualitative benefits platform for data ingestion, analysis, collaboration, or quantitative benefits, such as increased task en- and execution and also has the benefit of being cus- gagement or improved decision accuracy.
    [Show full text]
  • Security and Privacy Approaches in Mixed Reality:A Literature Survey
    0 Security and Privacy Approaches in Mixed Reality: A Literature Survey JAYBIE A. DE GUZMAN, University of New South Wales and Data 61, CSIRO KANCHANA THILAKARATHNA, University of Sydney and Data 61, CSIRO ARUNA SENEVIRATNE, University of New South Wales and Data 61, CSIRO Mixed reality (MR) technology development is now gaining momentum due to advances in computer vision, sensor fusion, and realistic display technologies. With most of the research and development focused on delivering the promise of MR, there is only barely a few working on the privacy and security implications of this technology. is survey paper aims to put in to light these risks, and to look into the latest security and privacy work on MR. Specically, we list and review the dierent protection approaches that have been proposed to ensure user and data security and privacy in MR. We extend the scope to include work on related technologies such as augmented reality (AR), virtual reality (VR), and human-computer interaction (HCI) as crucial components, if not the origins, of MR, as well as numerous related work from the larger area of mobile devices, wearables, and Internet-of-ings (IoT). We highlight the lack of investigation, implementation, and evaluation of data protection approaches in MR. Further challenges and directions on MR security and privacy are also discussed. CCS Concepts: •Human-centered computing ! Mixed / augmented reality; •Security and privacy ! Privacy protections; Usability in security and privacy; •General and reference ! Surveys and overviews; Additional Key Words and Phrases: Mixed Reality, Augmented Reality, Privacy, Security 1 INTRODUCTION Mixed reality (MR) was used to pertain to the various devices – specically, displays – that encompass the reality-virtuality continuum as seen in Figure1(Milgram et al .
    [Show full text]
  • CUE 21-031 Saab and Varjo Bring Virtual Reality to Flight Simulators
    NEWS FROM SAAB 7 June 2021 CUE 21-031 ´ Saab and Varjo Bring Virtual Reality to Flight Simulators Saab and Varjo Technologies work together to revolutionise flight simulators. The Finnish company Varjo has developed a human eye resolution mixed reality (XR) technology that will be integrated into all Saab Gripen E/F simulators. In addition to the highest resolution and widest field of view in the industry, the collaboration will also offer numerous new training opportunities at a fraction of the cost. Saab and Varjo Technologies, which develops virtual and mixed reality technologies for professional use and was founded in 2016, have advanced in their collaboration of many years to the point in which Varjo’s technology will be integrated in Saab’s new Gripen E/F simulators. The collaboration has its roots in Varjo’s innovation, which was critical to the successful development of state-of-the-art simulations. “When we did our first experiments with commercial devices, we received feedback from the pilots that they were unable to read text in virtual reality because the resolution was not sufficiently high. This was not a problem with Varjo’s technology,” says Stefan Furenbäck, Saab’s Head of Tactical Environment Simulation and Visualisation. “Our collaboration has been smooth and straightforward. We understand each other’s needs and how we can solve our shared technological problems. Two companies with similar cultures, with innovation high on their agenda and ready to discuss everything,” Seppo Aaltonen, Varjo’s Chief Commercial Officer describes the collaboration. Pilot training requires trainees to be able to read text and see even the smallest details.
    [Show full text]
  • A Review of Extended Reality (XR) Technologies for Manufacturing Training
    technologies Article A Review of Extended Reality (XR) Technologies for Manufacturing Training Sanika Doolani * , Callen Wessels, Varun Kanal, Christos Sevastopoulos, Ashish Jaiswal and Harish Nambiappan and Fillia Makedon * Department of Computer Science and Engineering, The University of Texas at Arlington, Arlington, TX 76019, USA; [email protected] (C.W.); [email protected] (V.K.); [email protected] (C.S.); [email protected] (A.J.); [email protected] (H.N.) * Correspondence: [email protected] (S.D.); [email protected] (F.M.) Received: 30 October 2020; Accepted: 5 December 2020; Published: 10 December 2020 Abstract: Recently, the use of extended reality (XR) systems has been on the rise, to tackle various domains such as training, education, safety, etc. With the recent advances in augmented reality (AR), virtual reality (VR) and mixed reality (MR) technologies and ease of availability of high-end, commercially available hardware, the manufacturing industry has seen a rise in the use of advanced XR technologies to train its workforce. While several research publications exist on applications of XR in manufacturing training, a comprehensive review of recent works and applications is lacking to present a clear progress in using such advance technologies. To this end, we present a review of the current state-of-the-art of use of XR technologies in training personnel in the field of manufacturing. First, we put forth the need of XR in manufacturing. We then present several key application domains where XR is being currently applied, notably in maintenance training and in performing assembly task.
    [Show full text]
  • Exploring Edge Computing in Multi-Person Mixed Reality for Cooperative Perception
    Exploring Edge Computing in Multi-Person Mixed Reality for Cooperative Perception Sihai Tang, Bruce (Haidi) Chen, Jacob Hochstetler, Jason Hirsch, Song Fu Department of Computer Science and Engineering University of North Texas, USA Abstract Traditional Mixed Reality(MR) and Augmented Reality(AR) devices support a wide gamut of sensors, but the limited com- putational resource onboard such devices make advanced tasks difficult. We introduce the use of Edge with MR devices to provide a cooperative perception capability to the MR de- vice. We base our approach on the portability and low latency of the Edge. Through our prototype system, we demonstrate the potential of devices and evaluate the performance and fea- sibility through real world trials. Our evaluation proves that the system is capable of supporting cooperative perception tasks. 1 Introduction Newer sensors as well as more sophisticated technologies are ushering in new ways of interaction with the world around us. The three different interactions that are most well known are Figure 1: Throughput in KB/s of image upload and detection. Augmented Reality(AR), Virtual Reality(VR), and Mixed Re- High resolution 16MP JPEG image is 3000x4000 pixels at 90 ality(MR). However, these devices, such as the HoloLens, are quality while the low resolution is 1.6 MP JPEG 1080x1440 limited by the hardware, especially when it comes to running at 59 quality. intensive or non-native tasks [1–4,7,9]. In our application, we seek to utilize the HoloLens in both machine learning tasks such as object detection and recognition as well as communi- is a Raspberry pi, we used the Movidius Compute Stick as cation tasks such as cooperation between multiple users.
    [Show full text]
  • Read the Report
    Ying Wu College of Computing Research Report 2020 Table of Contents Dean’s Message 3 Big Data 4 Data Science 8 Data and Software Security 12 Image and Video Processing 14 Big Knowledge, Bioinformatics and Medical Informatics 16 Distributed Systems, Mobile Computing and Networking 18 Cloud and High-Performance Computing 20 Augmented Reality, Virtual Reality Gaming and Graphics 24 Advanced Algorithms 30 Social, Mobile and Education 34 Ying Wu College of Computing (YWCC) was established in 2001, consisting of the computer science and informatics departments, and has grown rapidly to become the second-largest college at New Jersey Institute of Technology. Enrolling more than 3,200 students at all levels and graduating more than 800 computing professionals every year, YWCC is the largest generator of computing tech talent in the greater New York metro area. The tenure-track faculty and Ph.D. student population of YWCC has also grown significantly. These talented individuals are responsible for the high-quality academic research described in this report. Their research is financially supported by a variety of government agencies and corporate entities, published in top venues, and in some cases patented and commercialized. Spanning a wide spectrum of topics, from human-computer interaction to cybersecurity to sophisticated data science algorithms, the research conducted by YWCC ranges from deep mathematical theory to practical applications. Committed to sharing beyond publication, much of the software developed in these projects is made available to the general scientific community through open-source repositories. I invite you to read through the abstracts of the research projects described in this report and encourage you to reach out to the individual researchers for more details, if needed.
    [Show full text]
  • Design and Development of a Mixed Reality Application in the Automotive Eld
    Politecnico di Torino Department of Control and Computer Engineering (DAUIN) Master Degree in Computer Engineering Design and development of a Mixed Reality application in the automotive eld Supervisors: Author: Prof. Maurizio Morisio Giovanna Galeano December 2017 Contents 1 Mixed Reality1 1.1 Real Environment . .2 1.2 Augmented Reality . .2 1.2.1 Augmented Reality (AR) Categories . .2 1.2.2 Key Components to Augmented Reality Devices . .3 1.2.3 AR headsets categories . .4 1.3 Augmented Virtuality . .5 1.4 Virtual Reality . .5 1.4.1 Virtual Reality (VR) Categories . .5 1.4.2 Key Components in a Virtual Reality System . .6 1.4.3 Key Components Inside of a Virtual Reality Headset . .7 1.4.4 Performance parameters . .8 2 Context 11 2.1 Technological scouting . 12 2.1.1 Automotive eld . 13 2.1.2 Other examples . 17 3 System design 23 3.1 Helmet-mounted display design features . 25 3.2 Devices Comparison . 28 3.2.0.1 Final considerations . 29 3.3 Architecture design . 32 4 Development environment 35 4.1 Game engine components . 36 4.2 Game engines for Mixed Reality . 37 4.2.1 Unreal Engine 4 . 37 4.2.2 CryEngine . 38 4.2.3 Unity . 39 4.2.4 Unity as nal choice . 41 4.2.4.1 ARToolkit SDK . 42 4.2.4.2 Vuforia SDK . 43 4.2.4.3 Wikitude . 44 1 5 Prototype design and development 46 5.1 HoloLens hardware review . 46 5.2 Hololens inputs . 48 5.3 Hololens emulator . 49 5.4 Development basics .
    [Show full text]