Downloaded from Asset Store to a 3D Calculator Which Works Using the HTC VIVE Controllers
Total Page:16
File Type:pdf, Size:1020Kb
CALIFORNIA STATE UNIVERSITY SAN MARCOS PROJECT SIGNATURE PAGE PROJECT SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE MASTER OF SCIENCE IN COMPUTER SCIENCE PROJECT TITLE: BIOCHEMISTRY MIXED REALITY LAB SIMULATION AUTHOR: Arshia Shaikh DATE OF SUCCESSFUL DEFENSE: 05/04/2020 THE PROJECT HAS BEEN ACCEPTED BY THE PROJECT COMMITTEE IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE IN COMPUTER SCIENCE. Yuanyuan Jiang 05/06/2020 PROJECT COMMITTEE CHAIR SIGNATURE DATE Xin Ye 05/04/2020 PROJECT COMMITTEE MEMBER SIGNATURE DATE PROJECT COMMITTEE MEMBER SIGNATURE DATE BIOCHEMISTRY VIRTUAL REALITY LAB SIMULATION Arshia Shaikh Advisor: Dr. Yuanyuan Jiang California State University, San Marcos Master’s Project Report May 2020 1 ACKNOWLEDGEMENT I would like to acknowledge my advisor Dr. Yuanyuan Jiang for her guidance, support and time throughout my project. I am blessed to have her as my advisor as she has been motivating and always giving positive feedbacks. I am extremely thankful to Dr. Yuanyuan Jiang for enhancing my development skills and giving me an in-depth idea about virtual reality. I would like to thank Dr. Kambiz Hamadani, Dr. Xin Ye, other committee members and developers for being a great support and for the time they provided for this project. NSF for granting this project. This material is based upon work supported by the National Science Foundation under grant no. 1918045. I would also like to thank all the CSUSM professor for sharing their time. Thank you Dr. Hadaegh for always supporting me and guiding me in the right direction. 2 TABLE OF CONTENTS BIOCHEMISTRY VIRTUAL REALITY LAB SIMULATION .................................................................................. 1 1 Chapter - Introduction to the Problem & System ................................................................................. 9 1.1 Introduction/Background: ............................................................................................................ 9 2 Chapter - Thesis Overview .................................................................................................................. 11 2.1 Thesis overview: .......................................................................................................................... 11 2.2 My Contribution: ......................................................................................................................... 12 3 Chapter - Related Work ...................................................................................................................... 13 3.1 Paper 1: 3D Interactive Virtual Chemistry Laboratory ................................................................ 13 3.2 Paper 2: Low-Cost Virtual Reality Technology ............................................................................ 13 3.3 Paper 3: Haptic Display Technology ............................................................................................ 14 3.4 Paper 4: Mixed and Virtual Reality............................................................................................. 15 3.5 Paper 5: touch sensory feedback ................................................................................................ 15 4 Chapter - Big Project Design and Implementation ............................................................................. 16 4.1 Motion tracking hardware .......................................................................................................... 16 4.1.1 HTC VIVE Pro: ...................................................................................................................... 16 4.1.2 OptiTrack V120:................................................................................................................... 17 4.2 Simulation Software .................................................................................................................... 17 4.2.1 Unity: ................................................................................................................................... 17 4.2.2 SteamVR: ............................................................................................................................. 18 4.2.3 Motive: ................................................................................................................................ 19 4.2.4 Visual Studio:....................................................................................................................... 19 4.3 Project Flow: ............................................................................................................................... 21 4.4 Use case diagram: ....................................................................................................................... 23 4.4.1 Project flow ......................................................................................................................... 23 5 Chapter - Thesis Project and My Contributions .................................................................................. 24 5.1 My contribution implementation: .............................................................................................. 24 5.1.1 Automated Calibration of The Room .................................................................................. 24 5.1.2 Automated Detection of OptiTrack tracked Objects: ......................................................... 26 5.1.3 3D VR Compatible Calculator: ............................................................................................. 27 5.1.4 Submission Book: ................................................................................................................ 29 5.2 Documentation & collaboration ................................................................................................. 30 3 5.2.1 Class Document: .................................................................................................................. 30 5.2.2 Project Setup Document: .................................................................................................... 31 5.2.3 Chemistry Lab Lesson Plan Excel Sheet: ............................................................................. 32 6 Chapter - Contributions & Future Work ............................................................................................. 35 6.1 Contributions: ............................................................................................................................. 35 6.2 Future Work ................................................................................................................................ 36 7 Bibliography ........................................................................................................................................ 37 4 LIST OF FIGURES FIGURE 1: LESSON PLAN EXCEL SHEET MODEL. WHERE EACH MODULE IS DIVIDED INTO LESSONS AND EACH LESSON WILL BE DIVIDED INTO TASKS AND EACH TASK WILL DIVIDED WILL BE DIVIDED INTO OBJECTIVES. ... 11 FIGURE 2: HTC VIVE PRO. PACKAGE INCLUDING HEADSET, 2 CONTROLLER AND LIGHT HOUSES (CAMERAS) ........... 16 FIGURE 3: OPTITRACK V120 MOUNTED ON TRIPOD FOR TRACKING IR REFLECTIVE TRACKERS ................................ 17 FIGURE 4: SCREENSHOT DISPLAYING THE DEVELOPMENT ENVIRONMENT (SIMULATION SOFTWARE) IN UNITY ENGINE FOR VR CHEMISTRY LAB. ...................................................................................................................... 18 FIGURE 5: STEAMVR SOFTWARE WHICH IS USED TO MONITOR AND TROUBLESHOOT THE CONNECTIVITY OF VR DEVICES BY DISPLAYING THE STATUS OF EACH DEVICES. .................................................................................. 18 FIGURE 6: MOTIVE SOFTWARE TO TRACK THE IR REFLECTIVE TRACKERS. IT ALSO USED TO DEFINED THE RIGIDBODIES FOR MOTION TRACKING. ............................................................................................................. 19 FIGURE 7: VISUAL STUDIO: INTEGRATED DEVELOPMENT ENVIRONMENT (IDE) USED FOR CODE DEVELOPMENT IN C#. ...................................................................................................................................................................... 20 FIGURE 8 VR BIOCHEM LAB SIMULATION FLOW SHOWING THE FLOW OF HOW VARIOUS MODULES INTERACTING WITH EACH OTHER. ............................................................................................................................................ 21 FIGURE 9: THE DIGITAL DISPLAY: THIS DISPLAYS THE VARIOUS INFORMATION ABOUT THE APPARATUS AND CHEMICALS TO BE USED AND THE STEPS INVOLVED IN THE CURRENT OBJECTIVE. .......................................... 22 FIGURE 10 DEMONSTRATION OF CHEMICAL EXPERIMENT IN VR BIOCHEM LAB USING VR DEVICES AND ITS CORRESPONDING VR SIMULATION. ................................................................................................................... 22 FIGURE 11 USE CASE DIAGRAM FOR PROJECT FLOW DESCRIBING VARIOUS MODULES USED FOR THE VR LAB SIMULATION BY THE STUDENT AND THE CONTENT DEVELOPER ...................................................................... 23 FIGURE 12: 3D VR COMPATIBLE CALCULATOR FOR MATHEMATICAL CALCULATIONS INVOLVED DURING THE EXPERIMENT ...................................................................................................................................................... 27 FIGURE 13 TEXT BOX: GETS VALUE FROM THE CALCULATOR WHICH CAN BE USED TO SET THE PIPETTE ................. 27 5 FIGURE 14 SUBMISSION