Sensors in 3D UI

7/12/2011 The Wiimote Device

• Wiimote features – Uses Bluetooth for communication – Senses acceleration along 3 axes – Optical sensor for pointing (uses sensor bar) – Provides audio and rumble feedback – Standard buttons and trigger – Uses 2 AA Batteries • Supports 2-handed interaction Degrees of Freedom

• Degrees of Freedom (Mechanics) – The set of independent translations and/or rotations that specify completely any change in position or orientation of a body. • A rigid body that moves in 3-D space has 3 translational components as DOF’s and 3 rotational components as DOF’s = 6 DOF. – For example, a plane has roll, pitch, and yaw as well as X, Y, Z – Whereas, a car only has translation on one plane, 2 DOF, and rotation along one axis to face any heading – 1 DOF = 3 DOF. Wiimote – Motion Data

• Data from 3-axis accelerometer – Senses instantaneous acceleration on device (ie force) along each axis. – Units (+/- 3g) – Always sensing gravity • At rest acceleration is g (upward) • Freefall acceleration is 0 – Finding position and orientation • At rest – roll and pitch can be calculated easily • Cannot calculate yaw because there is no change in gravitational force. • In motion – math gets more complex • Error accumulation causes problems • Often not needed – gestures sufficient – Advantages • Easily detect course motions • Mimic many natural actions – Disadvantages • Abiguity issues • Player cheating • Not precise (Not a 6 DOF tracker) WiiMote Accelerometer Wiimote – Optical Data

• Data from optical sensor – Uses sensor bar • 10 LED lights (5 on each side), accurate up to 5 meters • Triangulate to determine depth – Distance between two points on image sensor(variable) – Distance between LEDs on sensor bar (Fixed) • Roll (with respect to ground) andgle can be calculated from angle of two image sensor points. – Advantages • Provides a pointing tool • Gives approximate depth – Disadvantages • Line of sight, infrared light problems • Only constrained rotation understanding. Wii Motion Plus • Multi-axis gyroscope – Provides information on changes to the Wiimote’s orientation. – Combined with the accelerometer and sensor bar, allows more comprehensive tracking of a user’s arm position and orientation – giving 6 DOF. iPhone accelerometer Kinect

• Camera tech has been around for about 15 years – available at huge expense up until very recently. • It’s made of two main parts: • A projector – That bounces out a laser across the entire field of play • And IR VGA camera. – The camera picks up the laser feedback to separate you from your sofa on what’s called a ‘depth field.’ – It’s essentially all the pixels that Kinect gets back as IR noise measured in varying color dependant on how close they are to the system. – That way bodies appear a bright shade of red, green etc, and things further away appear grey. Kinect

– The software takes this image and runs it through filters so that Kinect can work out what’s a person and what’s not. – The system follows a basic system of guidelines • such as ‘a body is from x-foot tall to x-foot tall’ and ‘a person has two arms and two legs’ to work out that your coffee table or dog aren’t extra players. • It’s also taught to be able to pick you out if you’re wearing baggy clothes or have hair coming over your shoulders. • Sorts out body parts. • Once that’s sorted, it converts body part identification into a skeleton with moving joints. • Kinect is preloaded with 200 common poses