sign in sign up
<<
Home , 3D user interaction, CryEngine, Facebook Horizon, Marmalade (software), OpenVR, OpenXR

Introduction to mixed realities Course 07 Content

● Recapitulation Course 6 ○ Experience - How UI interactions differ from 2D UI or classic games ○ Health concerns and good practices ● Platforms ○ Introduction, VR runtime systems, Real-Time Physics Engines, Distributed Virtual Environments, Collaborative Virtual Environments, Game Engines ● Conclusions ● Bibliography

2 R - User Experience

3 R - Illusion of VR

4 R - Interaction in VR

5 R - Health Concerns

6 VR - Software Platforms

The growing market for VR

7 VR Use Cases

It can help to create engaging games

With VR, media and entertainment companies can create more engaging content

Marketers can make their ads more interesting with the help of VR technology

Businesses can use VR to let their customers virtually try their product user experience

8 VR - Software Development Tools

1. 2. Amazon Sumerian 3. VR for everyone 4. 4 (UE4) 5. CRYENGINE 6. 7. 3ds Max 8. SketchUp Studio 9. Maya 10. Medium

9 Unity

Unity is famous for game development, but you can create VR solutions for automotive, transportation, manufacturing, media & entertainment, , construction. You can get a valuable set of tools:

● A powerful editor to create Unity 3D VR assets ● Artist and designer tools ● CAD tools ● Collaboration tools

Unity works with all popular VR platforms like Oculus, Sony, etc. 10 Amazon Sumerian

Amazon Sumerian is the VR engine from AWS. It works with all popular VR platforms like , , HTC Vive, HTC Vive Pro, , and Lenovo Mirage, moreover, it works with Android and iOS

Supports various VR use cases like employee , training , field services productivity, retail & sales, and virtual concierge

● Sumerian editor ● Sumerian hosts ● Asset management ● An ability to script the logic in the scenes you create 11 Google VR for everyone

Google offers a wide range of VR development tools. You can access these tools on the Google VR developer portal https://developers.google.com/vr

You can use these tools to develop VR apps for multiple platforms, e.g., Unity, Unreal, Android, iOS, and web

The Google VR developer platform has software development kits (SDKs) for all VR platforms it supports, e.g., Unity, Android, iOS, etc.

12 Unreal Engine 4 (UE4)

Unreal Engine 4 (UE4) offers a powerful set of VR development tools. With UE4, you can VR apps that will work on a variety of VR platforms, e.g., Oculus, Sony, Samsung Gear VR, Android, iOS, Google VR, etc.

● It offers access to its C++ code and Python scripts ● UE4 has a multiplayer framework, real-time rendering of visuals, and a flexible editor ● With the Blueprint visual scripting tool you can create prototypes quickly ● It’s easy to add animation, sequence, audio, simulation, effects, etc. 13 XR/VR Runtime Systems

OpenXR/OpenVR

WebXR/WebVR

14 OpenXR

OpenXR is an open, royalty-free standard for access to and platforms and devices

OpenXR is an API (Application Programming Interface) for XR applications

It is developed by a working group managed by the consortium https://www.khronos.org/registry/OpenXR/specs/1.0/html/xrspec.html

15 OpenXR - Architecture

Enables developers to build applications that will work across a wide variety of VR/AR devices. The fundamental elements of this API are:

● XrSpace: a representation of the 3D space ● XrInstance: a representation of the OpenXR runtime ● System and XrSystemId: a representation of the devices, including the VR or AR devices and controllers ● XrActions: used to handle user inputs ● XrSession: represents the interaction session between the application and the user

16 OpenXR - Implementations

The Khronos Group maintains the list of OpenXR-conformant platforms and products. OpenXR platforms are:

● HoloLens 2 and the Windows headsets ● Oculus PC platform and the Quest/Quest2 devices ● Varjo PC platform ● Monado Runtime for GNU/ ● Valve SteamVR support

17 OpenXR - Support

Game and Rendering Engine Support

● Unreal Engine from ● Blender ● Unity

Browser Support

● and web browsers both enable WebXR support using OpenXR by default using the code base 18 OpenVR

OpenVR is a (SDK) and application programming interface developed by Valve for supporting the SteamVR (HTC Vive) and other VR headset devices

The SteamVR platform uses it as the default application programming interface (API) and runtime

● https://github.com/ValveSoftware/openvr ● https://github.com/ValveSoftware/openvr/wiki/A PI-Documentation ● https://www.steamvr.com/en/ 19 WebXR

WebXR is a group of standards which are used together to support rendering 3D scenes to hardware designed for presenting virtual worlds (VR), or for adding graphical imagery to the real world (AR)

The WebXR Device API implements the core of the WebXR feature set, managing the selection of output devices, render the 3D scene to the chosen device at the appropriate rate, and manage motion vectors created using input controllers

● https://immersive-web.github.io/webxr/ ● https://developer.mozilla.org/en-US/docs/Web/API/WebXR_Device_API 20 WebVR

WebVR is a JavaScript API for creating immersive 3D, VR experiences in your browser. The goal is to make it easier for everyone to get into VR experiences, no matter what device you have

WebVR has been replaced by the WebXR Device API

● https://www.youtube.com/watch?v=Le8pTXQqM3s ● https://webvr.rocks/ ● https://mixedreality.mozilla.org/ ● https://developers.google.com/web/fundamentals/vr ● https://html5hive.org/vr-webvr-tutorial/ 21 Real Time Physics Engines

Infinity Tech 3D https://www.youtube.com/watch?v=fO7XhaTGDYg

Algoryx https://www.youtube.com/watch?v=jwJS9m5xhd0&t=5s

Scientific papers:

XPHEVE: An Extensible for Virtual Environments https://ieeexplore.ieee.org/document/4054758

22 Infinity Tech 3D

Physics engine for scientific and business critical applications 100% real-time - 100% interactive - 100% immersive http://infinytech3d.com/

SofaAPAPI-Unity3D is a professional physics engine combining performance and accuracy of scientific multi-physics simulation with realtime user haptic interactions in VR

23 Algoryx https://www.algoryx.se/vr-training-simulators/

24 Distributed Virtual Environments (DVEs)

DVEs are geographically distributed systems which allow multiple users to interact simultaneously in computer-generated, shared virtual environments

DVEs are widely applied in many fields, such as online role playing games, synthesis military , virtual cities, and virtual shopping malls

To support large-scale DVEs massive computing resources, storage resources, and highly efficient distributed server architectures are needed 25 Distributed Immersive Environments (DIEs)

Multi-person, multi-site shared virtual space

● Navigation ● Avatar representation ● Telephony

Where?

● Virtual Festivals ● Virtual Museums https://immerse.news/distributing-immersive-stori es-at-virtual-festivals-1048103a2292 26 Social VR

Social VR - The application of VR in the social networks. In the real-time and dynamic simulation environment, they can communicate in a multisensory way, which makes social communication immersive. Social VR games:

● VRChat https://www.youtube.com/watch?v=PWLPw4RE9Ig ● https://www.youtube.com/watch?v=5c44q1XW9p4 https://www.roadtovr.com/10-apps-hang-friends-vr/

27 Collaborative Virtual Environments (CVEs)

A CVE is a computer-based, distributed, virtual space or set of places. In such places, people can meet and interact with others, with agents or with virtual objects

CVEs might vary in their representational richness from 3D graphical spaces, 2.5D and 2D environments, to text-based environments. Access to CVEs is by no means limited to desktop devices, but might well include mobile or wearable devices, public kiosks, etc.

Facebook Spaces - “a new VR app where you hang out with friends in a fun, interactive virtual environment as if you were in the same room.” 28 Facebook Spaces https://www.youtube.com/watch?v=PVf3m7e7OKU https://www.youtube.com/watch?v=7nLtw_yqDhw

29 CVE Types

● Marketing: consumer interaction ● Training in dangerous/harmful situations ● Social entertainment, e.g. multi-player games ● Education: distance learning ● Art: interactive installations ● Built environment: architectural walkthroughs ● Medicine: surgery simulation ● Healthcare: virtual reality in phobia treatment 30 CVEs - Solutions

MeetinVR - Combine the convenience of online meetings with the interactivity of face-to-face meetings: meet in virtual where you can collaborate, create and feel truly together with your colleagues!

Virtualist - “Reduce collaboration cost, increase productivity and unleash team creativity wherever you are and whatever device you’re using.”

Glue - “We want to extend human presence over physical boundaries and advance human spatial communication and collaboration.”

Rumii - “A lot of companies claim to be making the world a better place. At Doghead Simulations, we’re actually doing it.” 31 VR Game Engines and Platforms

VART - VR Arcade Game Machine https://www.youtube.com/watch?v=Il4Gc7JI808

Gaming platforms range from the most complicated and expensive to the simplest and cheapest:

● Game Maker Studio, Coscos2D, ● Unity, Unreal ● Lumberyard ● 3ds Max Design, Maya ● CryEngine, ApertusVR, AppGameKit 32 VART - VR Arcade Game Machine https://www.vartvrarcade.com/vr-arcade-machine/

33 VR Game Platforms (Obsolete or Small Communities)

Game Maker Studio allows a developer to create games compatible with iOS, Android, Windows and even HTML5

Coscos2D is as the name suggests, a two dimensional gaming platform. It was at one point one of the most popular of all open source gaming option

Marmalade SDK is one of the better options available in the market for virtual game development

34 Unity

Unity is perhaps one of the most popular ones to date

Introduced to the market back in 2005 by , what’s unusual about this is that it provides support for the development of both two and three dimensional games

Unity provides support for all mobile, desk, web and console platforms

Reasonably priced with a very vast community of developers. With excellent editing tools easily extended via plug-ins, this platform supports a surprising huge range of asset formats

35 Unreal Engine

Unreal, a product of Epic Games, it is the game engine attributed to being behind the majority of high end, high powered games with exceptional graphic and features overall

Supporting only the C++ development language, Unreal Engine is a gaming platform that gives incredible gaming performance

It has the biggest developers community providing the best support

Support all major operating platforms for mobile and PC

36 37 Lumberyard

If you are looking for a VR game engine that offers you the full convenience of developing games, this should be the one. Offered by Amazon

It is comparatively a new entry in this segment and it is free with full source. This also means that one can tweak the engine if felt necessary

This could be an excellent platform for developing online games and don’t need to worry about hosting a robust game https://aws.amazon.com/lumberyard/ https://www.youtube.com/watch?v=nV2tm19FmYI 38 3ds Max Design

3ds Max Design software is a comprehensive 3D design, modeling, animation, and rendering solution for architects, designers, civil engineers, and visualization specialists

39 Maya 3D

Maya® 3D is used for animation, modeling, simulation, and rendering software. Maya helps artists tell their story with one fast, creative toolset

40 Conclusions

Exist many resource and tools to develop VR applications

Exist also interest for Social VR, Collaborative VR, Distributed VR, but you need more hardware resources for that

Unity and Unreal Engine are the most used tools to develop VR Games

41 Questions

1. Enumerate software tools used for VR app developing 2. What tools do you use for your project?

42 Bibliography

● Sherman, W.R., Craig, A. B. (2003) Understanding Virtual Reality. Elsevier, Science. ● Jiang, c., Xu, X., Wan, J., Li, W., You, X. (2007) Large Scale Distributed Virtual Environments on the Grid: Design, Implementation, and a Case Study. International Conference on Computer Supported Cooperative Work in Design CSCWD 2007: Computer Supported Cooperative Work in Design IV pp. 384-395.

43 Links

● Spinning Up in VR — Part 3: Virtual Reality Hardware https://medium.com/@mnrmja007/spinning-up-in-vr-part-3-virtual-reality-hardware-e91825465f4d ● Unity or Unreal – Which Is the Best VR Gaming Platform? https://appreal-vr.com/blog/unity-or-unreal-best-vr-gaming-platforms/ ● Unity vs Unreal Engine for XR Development: Which One Is Better? https://circuitstream.com/blog/unity-vs-unreal/ ● Comparing Virtual Reality Software Engines https://pixovr.com/2019/06/12/comparing-vr-software-engines/ ● Top VR Game Engines https://filmora.wondershare.com/virtual-reality/state-of-vr-games-the-game-engines-and-present-convention.html ● Best VR Game Engine Software https://www.g2.com/categories/vr-game-engine ● Understanding Collaborative Virtual Environments https://learningflourishing.com/writing-research/cve/ ● Collaborative Virtual Environments https://www.vrs.org.uk/virtual-reality-environments/collaborative.html ● Collaborative Virtual Environment – How It Works? https://appreal-vr.com/blog/collaborative-virtual-environment-how-it-works/ ● Social VR, Facebook Horizon And The Future Of Social Media Marketing https://www.forbes.com/sites/cathyhackl/2020/08/30/social-vr-facebook-horizon--the-future-of-social-media-marke ting/

● 44