3DUI,VR and Games
Total Page：16
File Type：pdf, Size：1020Kb
Editor: Graphically Speaking Miguel Encarnação Bringing VR and Spatial 3D Interaction to the Masses through Video Games Joseph J. irtual reality (VR) and 3D user interface elements in the 3D world. This is basically the tra- LaViola Jr. (3D UI) research has been going on for ditional approach; it’s how people have been inter- University more than 20 years, with limited success acting with 3D (and 2D) video games since their of Central Vin being a part of people’s everyday lives. Don’t inception many years ago. Florida get me wrong; VR and 3D UIs have been useful in The second approach simulates the real world areas such as simulation and training, scientifi c using replicas of existing devices or physical props. visualization, medicine, and psychological reha- Common examples include steering wheels, light bilitation.1 But to say that VR and 3D UI research guns, and musical instruments (for example, the has signifi cantly affected mainstream society is guitar in Guitar Hero). These devices don’t neces- undeniably false. Compared to other technologies sarily provide 3D interaction in the game but do such as the mouse and WIMP (windows, icons, provide 3D input devices that enable more real- menus, point and click) interfaces, VR and 3D UIs istic gameplay. have had little impact. The third approach is true spatial 3D tracking of However, we’re seeing a change in the utility of VR the user’s motion and gestures, where users inter- and 3D UIs, with concepts from these fi elds emerg- act in and control elements of the 3D gaming world ing on a large, mainstream scale. The video game with their bodies. This control can come through industry, as it has done for real-time 3D computer vision-based devices such as the Sony EyeToy and graphics for the last 15 years, is bringing spatial motion-sensing devices such as Wiimotes. 3D interaction into people’s living rooms through For bringing 3D UIs and VR to the masses, the rather simple, yet effective technologies such as second and third approaches hold the most prom- computer vision and motion sensing. What’s even ise and are primed to take video games to the next more interesting is that the video game industry level of innovation. has tried many times before to bring VR technology to its customers and failed miserably. Devices such Historical perspective as the Nintendo U-Force and Sega 3D glasses are Briefl y examining past video game trends will pro- just two examples of such failures. vide insight into why 3D UIs are here to stay and The question then is why have recent innova- what the future holds for them. tions such as the Sony EyeToy and Nintendo Wii been so successful? Here, I look at a possible an- Video arcades and game consoles swer and discuss the research opportunities pre- I’ve been playing video games in arcades, on con- sented by the latest commercial push for spatial soles, and on PCs for almost 30 years. So, I’ve seen 3D interaction in games. how technology has progressed from the days of Pong and Space Invaders to Grand Theft Auto IV 3D UIs and video games and Halo 3. What do I mean by 3D UIs in video games, anyway? The video game industry began in the mid-’70s. In general, 3D UIs involve input devices and interac- During these early days, if you wanted to play tion techniques for effectively controlling highly dy- video games, you had two choices: go to an arcade namic 3D computer-generated content,2 and there’s or buy a game console such as the Magnavox Od- no exception when it comes to video games. yssey, which debuted in 1972. (PCs became popu- A 3D video game world can use one of three basic lar in the 1980s but were expensive compared to approaches to interaction. The fi rst maps 2D in- game consoles.) Arcades were much more popular put and button devices, such as the keyboard and than game consoles because you could pack more mouse, joysticks, and game controllers, to game graphics, sound, and game play into a large, deco- 10 September/October 2008 Published by the IEEE Computer Society 0272-1716/08/$25.00 © 2008 IEEE Authorized licensed use limited to: University of Central Florida. Downloaded on January 7, 2009 at 12:32 from IEEE Xplore. Restrictions apply. (a) (b) (a) (c) (d) Figure 2. Arcade games that used more realistic input strategies: (a) BeachHead, (b) Football Power, (c) Manx TT, and (d) Dance Dance Revolution. Such innovative interfaces helped arcade games compete with PCs and game consoles. (b) Figure 1. Donkey Kong: (a) the arcade and It became clear that arcade games couldn’t (b) ColecoVision game console versions. Game compete with game consoles and PCs, in terms console graphics quickly approached the quality of of graphics, sound, and length of play per gaming arcade games. session. So, to compete with game consoles, the only thing video arcades could do was innovate rated, wooden cabinet than you could into a rela- at the user interface. This innovation came in the tively small console. form of a variety of input devices and strategies This trend continued through the ’80s as arcade that got players more actively involved. game graphics and sound became more sophisti- In the late ’90s and early 2000s, arcade games cated. However, game consoles weren’t far behind, introduced several particularly innovative user in- as systems such as ColecoVision, the Sega Master terface designs. For example, in BeachHead (see System, and the Super Nintendo Entertainment Figure 2a), the user wore a helmet-like device, re- System emerged. When I first played Donkey Kong sulting in a 360-degree field of regard. In games on ColecoVision, I was amazed at how close it was such as Football Power (see Figure 2b), players ac- to the arcade version (see Figure 1). tually controlled a soccer ball with their feet. In In the ’90s, things changed. Consoles became Aliens Extermination (not shown), players used faster, with better graphics and sound than arcade realistic gun props to interact in a first-person- games. If you bought one of these consoles (or your shooter-style game. In Manx TT (see Figure 2c), parents bought one for you), you didn’t have to put users rode a physical motorcycle to control a vir- a quarter into a machine every time you wanted to tual motorcycle. In Dance Dance Revolution (see play the coolest game. In addition, you could play Figure 2d), users interacted on a small dance floor. for as long as you wanted and didn’t have to travel Adopting such interfaces for game consoles wasn’t anywhere to do so. So, the standard video arcade cost effective, so the video arcade was able to live was able to move into the home. a little while longer. IEEE Computer Graphics and Applications 11 Authorized licensed use limited to: University of Central Florida. Downloaded on January 7, 2009 at 12:32 from IEEE Xplore. Restrictions apply. Graphically Speaking During the ’90s, VR technology appeared in and so on but also gameplay. As video games be- video games. VR-based games helped keep arcades came more complex by giving users a greater va- alive because they included more advanced inter- riety of things to do and controls to master, the faces employing such technologies as stereoscopic input devices for controlling them stayed relatively vision, head and body tracking, and 3D spatial constant. If we look at PC games, the argument is interaction. These games appeared mainly in ar- clear: mouse and keyboard have been the predomi- cades and entertainment centers. nant control devices since PC gaming started in One of the first VR games was Dactyl Night- the early ’80s. mare, which W Industries/Virtuality developed You could argue that console game input devices in the early ’90s as part of a suite of VR games. have improved and gotten more complex since the Players entered a pod-like structure, put on a head- Atari 2600’s simple directional joystick and but- mounted display, and used a tracked joystick to ton. Game controllers certainly have gotten more interact with the virtual world. The pod protected complex but, I would argue, not necessarily better. users from walking around in the physical world. As new generations of game consoles emerged, the The games themselves were somewhat primitive, controllers simply added more buttons and joy- but the immersive experience hadn’t been seen be- sticks to previous versions. fore in a video arcade game. One VR entertainment For example, the 1983 Nintendo Famicom used center was BattleTech, based on the BattleTech uni- a game pad with direction buttons (to replace the directional joystick) and four buttons; the Sega Master System had a similar design. In 1994, when It’s clear that game interfaces must Nintendo introduced the first 3D game console (Nintendo 64), the controller had a directional become easier to use while maintaining joystick, a directional pad, and 10 buttons. Final- ly, the Sony Playstation introduced the DualShock the high levels of expression and control controller, which has two analog joysticks, a direc- of modern console and PC games. tional pad, and 10 buttons. As controller complexity increased, so did the complexity of a game’s control scheme. These verse. The first one opened in Chicago in 1990. It schemes allowed for more expression, at the cost of aimed to provide an immersive experience where making the games difficult to learn and master. So, several users could play simultaneously not only the interfaces became tailored to hard-core gamers, at a single location but also networked across all often alienating the casual player.