How to Get Started with AR/MR/VR
Total Page:16
File Type:pdf, Size:1020Kb
Madeline Wrable GIS Specialist CLSTL’19 pngtree.com 1 Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started pngtree.com 2 Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started pngtree.com 3 Massachusetts Institute of Technology Getty Images 4 Rotch Library Getty Images 5 MIT GIS Services: GIS & Data Lab • GIS Lab • Additional software • GIS data repository, GeoWeb • One-on-one help • Workshops Credit: Lisa Sweeney 6 Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started pngtree.com 7 VR Pilot Project Story Adapted from “Reality Bytes: VR and AR” – 8 Matt Bernhardt, MIT Libraries Jesussanz/Getty Creative 8 VR Pilot Project Story The MIT Libraries Experimental Collections Fund Supports projects using library datasets in new ways. I used it for a drone-pilot-project. Do you have any ideas? Adapted from “Reality Bytes: VR and AR” – 9 Matt Bernhardt, MIT Libraries Jesussanz/Getty Creative 9 VR Pilot Project Story The MIT Libraries Experimental Collections Fund Supports projects using library I wonder if virtual reality (VR) datasets in new ways. would be a good fit… I used it for a drone-pilot-project. Do you have any ideas? Adapted from “Reality Bytes: VR and AR” – 10 Matt Bernhardt, MIT Libraries Jesussanz/Getty Creative 10 Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started pngtree.com 11 Facebook F8 developer’s conference, 2015 12 History of Immersive Reality Breakthroughs 21 Results and Future Directions 34 oculus quest image: facebook Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets Content Viewing Platform Examples • Google Cardboard Mobile Devices • Google Daydream • Samsung Gear VR • Oculus Rift Laptop/Desktops • HTC Vive Game Consoles • Sony PlayStation VR Standalone HMDs • FOVE VR & Eye Tracking • Google Hololens … • … 46 Mobile Device: Google Cardboard • ~$15 simple headset • Free or paid smartphone apps • “Head in a beachball” content • Phone gyroscopes used to determine view direction • Only click in center of your vision • https://vr.google.com/cardboar d/developers/ Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Mobile Device: Google Daydream • ~$80 less simple headset • Free or paid smartphone apps • “Head and hand in beachball” • Hand-held wand to separate looking and clicking • https://developers.google.co m/vr/daydream/overview Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Mobile Device: Samsung Gear VR • ~$100, requires a Samsung phone • Free or paid smartphone apps • Comes with handheld remote • Limited to Oculus software library • http://developer.samsung.com/str eam-code101/gear-vr Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Personal Computer: Oculus Rift • ~$500 hardware kit • Powerful computer required • Free or paid software applications • Cubicle-sized space • One body sensors Handheld wand or controller Headset with speakers • https://developer.oculus.com/ Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Personal Computer : HTC Vive • ~$800 hardware kit • Powerful computer required • Free or paid software applications • Cubicle or Room-sized space – Two body sensors – Two handheld wands – Headset with speakers • Note: now has a new version (Pro) • https://developer.viveport.com/us/ develop_portal/ Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Console: Sony PlayStation VR • $300 headset & $300 console • 45 million PlayStation 4 owners makes this an accessible path to VR • Easy adoption • https://www.playstation.com /en-us/develop/ Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Standalone: FOVE VR • ~$600 hardware kit • Founded by a former game producer at Sony • Headset with eye tracking • https://www.getfove.com/ developers/ Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Standalone: HoloLens • ~$3000 hardware kit • Mixed Reality • Development kits started selling February 29, 2016 • https://developer.microsoft. com/en-us/windows/mixed- reality/install_the_tools Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries full full reality virtuality mixed reality full full reality virtuality high presence full full reality virtuality low presence high presence tethered full full reality virtuality untethered low presence high presence vive pro eye tethered pimax vr vive oculus quest oculus rift htc vive cosmos meta 2 windows vr magic leap oculus dk2 hololens gear vr daydream full oculus dk1 full google glass reality virtuality cardboard smartphone smartphone smartphone untethered low presence high presence vive pro eye tethered pimax vr vive oculus quest oculus rift htc vive cosmos meta 2 windows vr magic leap oculus dk2 hololens gear vr daydream full oculus dk1 full google glass reality virtuality cardboard smartphone smartphone smartphone untethered low presence other tracking optics / displays rendering telecom senses no tracking: 3DOF larger FOV (~ 200°) greater processor greater bandwidth better ergonomic V greater resolution more efficient GPU V haptic interfaces external tracking: 6DOF greater refresh rate ray tracing ultra low latency dynamic binaural V opto electronics foveated rendering wireless local audio inside out tracking: 6DOF variable focus lenses optimization of V other senses inputs V eye tracking rendering pipeline ultra low latency general locomotion inside out tracking + location new 3D graphic techniques wireless universal tracking: 6DOF + coordinates AI in rendering processes AI in content creation + better batteries VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets Content Creation Platforms • Photo Spheres • Photogrammetry • 3D Scanning • 3D Modeling • Game Engines • … Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Photo Sphere: Ricoh Theta • ~$300 digital camera • Twin fisheye lenses • Still photographs or video • Includes streaming capability • Suitable for Cardboard or Daydream-style viewers • Output supported by Facebook, Flickr, YouTube, etc Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Photo Sphere: VUZE • ~$800 digital camera • Four pairs of fisheye lenses • Still photographs or video • Stereo 3d photo spheres • Suitable for Cardboard or Daydream-style viewers • Output suitable for YouTube Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Photogrammetry: Agisoft PhotoScan • $179 Standard & $3,500 Pro (license for only one computer) • Able to reconstruct 3D models from drone acquired photos Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries Photogrammetry: ArcGIS Drone2Map • Prohibitively expensive without educational licensing, even then ~$1,000 per seat • Part of a spatial software suite • Somewhat lacking in 3D model capabilites Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries 3D Scanning: Sense • $400 3d scanner • Time of flight & white light scanner • Useful for generating 3d models to import into VR/AR content Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries capturing modeling rendering agisoft photoscan Autodesk Revit Unreal Engine 360 video suite Solidworks CryEngine Adobe Premiere SketchUp Unity ... Rhinoceros ... Autodesk 3DS Max Autodesk Maya Strata Blender Cinema 4D ... 3D Modeling Software Adapted from “Reality Bytes: VR and AR” – Matt Bernhardt, MIT Libraries interactive synthetic photogrammetry simulations videogrammetry 3D scanning representing capturing CGI / Digital immersive Animation video non interactive hand built proceduraly built AI built VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets Game Engine: Unity • Free for hobbyists and beginners • All purpose game engine • Poor internal editing options • https://unity3d.com/learn/tutori als/topics/virtual-reality/getting- started-vr-development Game Engine: Unreal Engine • Free up to $3,000 revenue, then charges a 5% royalty • Game engine authored by Epic Games (BioShock, Assassin’s Creed) • https://docs.unrealengine.com/lat est/INT/Platforms/index.html Game Engine: Cry Engine • $10/month but no royalty fees • Game Engine designed by the German developer Crytek (Far Cry) • Licensed by Amazon • http://docs.cryengine.com/pages/v iewpage.action?pageId=25536773 VR Pipeline Content Software Content Creation + Interface Viewer 2D 3D 3D/VR headsets VR Pipeline Content Creation Software Content Viewer Content Creation + 2D 3D VR Pipeline Future Direction Content Creation Platforms Software Development Kits (SDKs) Content Viewing Platforms Content • GIS & Data Lab • VR Pilot Project Story • What is VR • VR hard/software • Getting started MIT GIS Services • GIS Lab • Additional software • GIS data repository, GeoWeb • One-on-one help • Workshops MIT GIS Services • GIS Lab » powerful computers & graphics cards (content viewer) • Additional software » 3D modeling software (content creation) • GIS data repository, GeoWeb » available datasets