Motion Tracking Survey Motion Tracking: No Silver Bullet, but a Respectable Greg Welch University of North Carolina at Chapel Hill Eric Foxlin Arsenal InterSense f you read the surveys of motion tracking (HMD) or on a projection screen. In immersive sys- Isystems,1-5 one thing that will immediately tems, head trackers provide view control to make the strike you is the number of technologies and approach- computer graphics scenery simulate a first-person es—a bewildering array of systems operating on entire- viewpoint, but animations or other nonimmersive ly different physical principles, exhibiting different applications might use handheld trackers. performance characteristics, and designed for differ- I Navigation. Tracked devices help a user navigate ent purposes. So why does the world need so many dif- through a computer graphics virtual world. The user ferent tracking products and might point a tracked wand to fly in a particular direc- research projects to do essentially tion; sensors could detect walking-in-place motion This article introduces the the same thing? for virtual strolling. Just as Brooks argued in his I Object selection or manipulation. Tracked handheld physical principles famous article on software engi- devices let users grab physical surrogates for virtual neering6 that there is no single tech- objects and manipulate them intuitively. Tracked underlying the variety of nique likely to improve software gloves, acting as virtual surrogates for a user’s hands, engineering productivity an order of let the user manipulate virtual objects directly. approaches to motion magnitude in a decade, we’ll I Instrument tracking. Tracked tools and instruments attempt to show why no one track- let you match virtual computer graphics represen- tracking. Although no single ing technique is likely to emerge to tations with their physical counterparts—for exam- solve the problems of every tech- ple, for computer-aided surgery or mechanical technology will work for all nology and application. assembly. But this isn’t an article of doom I Avatar animation. Perhaps the most conspicuous and purposes, certain methods and gloom. We’ll introduce you to familiar use of trackers has been for generating real- some elegant trackers designed for istically moving animated characters through full- work quite well for specific specific applications, explain the body motion capture (MoCap) on human actors, arsenal of physical principles used animals, and even cars. applications. in trackers, get you started on your way to understanding the other arti- No silver bullet cles in this special issue, and perhaps put you on track to Our experience is that even when presented with choose the type of system you need for your own com- motion tracking systems that offer relatively impressive puter graphics application. We hope this article will be performance under some circumstances, users often accessible and interesting to experts and novices alike. long for a system that overcomes the shortcomings relat- ed to their particular circumstances. Typical desires are What is motion tracking? reduced infrastructure, improved robustness, and If you work with computer graphics—or watch tele- reduced latency (see the sidebar, “Tracking Latency”). vision, play video games, or go to the movies—you are The only thing that would satisfy everyone is a magical sure to have seen effects produced using motion track- device we might call a “tracker-on-a-chip.” This ToC ing. Computer graphics systems use motion trackers for would be all of the following: five primary purposes: I Tiny—the size of an 8-pin DIP (dual in-line package) I View control. Motion trackers can provide position or even a transistor; and orientation control of a virtual camera for ren- I Self-contained—with no other parts to be mounted in dering computer graphics in a head-mounted display the environment or on the user; 24 November/December 2002 0272-1716/02/$17.00 © 2002 IEEE Tracking Latency Have you seen those so-called “gourmet” cookie stands Read in convenience stores or fast-food restaurants? They usually buffer include a sign that boasts “Made fresh daily!” Client Write Unfortunately, while cookie baking might indeed take place buffer daily, the signs don’t actually give you the date on which Network the specific cookies being sold were baked! We’ve found a related common misperception about Read delay or latency in interactive computer graphics in general, buffer and in tracking in particular. While the inverse of the estimate rate (the period of the estimates) contributes to Server Write buffer the latency, it doesn’t tell the entire story. Consider our imaginary tracker-on-a-chip. If you send its 1,000-Hz Estimate estimates halfway around the world over the Internet, they will arrive at a rate of 1,000 Hz, but quite some time later. Sample Similarly, within a tracking system, a person moves, the sensor sensors are sampled at some rate, some computation is done on each sample, and eventually estimates pop out of User motion the tracker. To get the entire story, you must consider not Time only the rate of estimates, but also the length of the pipeline through which the sensor measurements and A Typical tracker pipeline. subsequent pose estimates travel. As Figure A illustrates, throughout the pipeline there are both fixed latencies, associated with well-defined tasks such the past and knowledge of roads in general. The difficulty of as sampling the sensors and executing a function to this task depends on how fast the car is going and on the estimate the pose, and variable latencies, associated with shape of the road. If the road is straight and remains so, the buffer operations, network transfers, and synchronization task is easy. If the road twists and turns unpredictably, the between well-defined but asynchronous tasks. The variable task is impossible. latencies introduce what’s called latency jitter. Here again there’s no silver bullet. In 1995 Azuma showed that motion prediction can help considerably, to a point.1,2 The most basic approach is to estimate or measure References the pose derivatives and to use them to extrapolate forward 1. R. Azuma, Predictive Tracking for Augmented Reality, PhD disserta- from the most recent estimate—which is already old by the tion, tech. report TR95-007, Univ. North Carolina, Chapel Hill, time you get to see it—to the present time. The problem is Dept. Computer Science, 1995. that it’s difficult to predict what the user will choose (has 2. R. Azuma and G. Bishop, “A Frequency-Domain Analysis of Head- chosen) to do very far in the future. Motion Prediction,” Proc. Ann. Conf. Computer Graphics and Inter- Azuma pointed out that the task is like trying to drive a active Techniques (Proc. Siggraph 95), ACM Press, New York, car by looking only in the rear-view mirror. The driver must 1995, pp. 401-408. predict where the road will go, based solely on the view of I Complete—tracking all six degrees of freedom (posi- least seven of these 10 characteristics, and that number tion and orientation); is unlikely to shrink much in the foreseeable future. I Accurate—with resolution better than 1 mm in posi- But all is not lost! Researchers and developers have tion and 0.1 degree in orientation; pragmatically and cleverly exploited every available I Fast—running at 1,000 Hz with latency less than 1 physical principle to achieve impressive results for spe- ms, no matter how many ToCs are deployed; cific applications. We’ll start with an overview of some I Immune to occlusions—needing no clear line of sight of the available ammunition and the strengths and to anything else; weaknesses of each and then look at some specific appli- I Robust—resisting performance degradation from cations and the tracking technologies that have been light, sound, heat, magnetic fields, radio waves, and employed successfully in each. other ToCs in the environment; I Tenacious—tracking its target no matter how far or Available ammunition fast it goes; Although designers have many pose estimation algo- I Wireless—running without wires for three years on a rithms to choose among, they have relatively few sens- coin-size battery; and ing technologies at their disposal. In general, the I Cheap—costing $1 each in quantity. technologies sense and interpret electromagnetic fields or waves, acoustic waves, or physical forces. Specifical- If this magic ToC existed, we would use it for everything. ly, motion tracking systems most often derive pose esti- The reality is that every tracker today falls short on at mates from electrical measurements of mechanical, IEEE Computer Graphics and Applications 25 Motion Tracking Survey Accelerometers Gyroscopes Gyroscopes Motor Accelerometers 1 (a) Stable- platform (gim- baled) INS. (b) Strapdown INS. Motor Motor (a) (b) inertial, acoustic, magnetic, optical, and radio frequen- to know the tip position to apply appropriate forces, and cy sensors. the electromechanical devices typically used to provide Each approach has advantages and limitations. The the forces can also be used to sense the position. limitations include modality-specific limitations relat- ed to the physical medium, measurement-specific limi- Inertial sensing tations imposed by the devices and associated Inertial navigation systems (INSs) became widespread signal-processing electronics, and circumstantial limi- for ships, submarines, and airplanes in the 1950s, before tations that arise in a specific application. For example, virtual reality or computer graphics were even conceived, electromagnetic energy decreases with distance, ana- but they were the last of the six ammunition technolo- log-to-digital converters have limited resolution and gies to be introduced for computer graphics input accuracy, and body-worn components must be as small devices. The reason is straightforward: an INS contains and lightweight as possible. Although alternative clas- gyroscopes, and early high-accuracy spinning-wheel sifications are possible, we discuss the available ammu- gyroscopes weighed far too much to be attached to a per- nition using a traditional medium-based classification.