Style-Based Inverse Kinematics Keith Grochow1 Steven L. Martin1 Aaron Hertzmann2 Zoran Popovic´1 1University of Washington 2University of Toronto Abstract the space of natural poses. Moreover, these systems attempt to rep- resent all styles with a single metric. This paper presents an inverse kinematics system based on a learned In this paper, we present an IK system based on learning from model of human poses. Given a set of constraints, our system can previously-observed poses. We pose IK as maximization of an ob- produce the most likely pose satisfying those constraints, in real- jective function that describes how desirable the pose is — the op- time. Training the model on different input data leads to different timization can satisfy any constraints for which a feasible solution styles of IK. The model is represented as a probability distribution exists, but the objective function specifies how desirable each pose over the space of all possible poses. This means that our IK sys- is. In order for this system to be useful, there are a number of impor- tem can generate any pose, but prefers poses that are most similar tant requirements that the objective function should satisfy. First, it to the space of poses in the training data. We represent the proba- should accurately represent the space of poses represented by the bility with a novel model called a Scaled Gaussian Process Latent training data. This means that it should prefer poses that are “sim- Variable Model. The parameters of the model are all learned auto- ilar” to the training data, using some automatic measure of similar- matically; no manual tuning is required for the learning component ity. Second, it should be possible to optimize the objective function of the system. We additionally describe a novel procedure for inter- in real-time — even if the set of training poses is very large. Third, polating between styles. it should work well when there is very little data, or data that does Our style-based IK can replace conventional IK, wherever it is not have much redundancy (a case that leads to overfitting problems used in computer animation and computer vision. We demonstrate for many models). Finally, the objective function should not require our system in the context of a number of applications: interactive manual “tuning parameters;” for example, the similarity measure character posing, trajectory keyframing, real-time motion capture should be learned automatically. In practice, we also require that with missing markers, and posing from a 2D image. the objective function be smooth, in order to provide a good space of motions, and to enable continuous optimization. CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional The main idea of our approach is to represent this objective Graphics and Realism—Animation; I.2.9 [Artificial Intelligence]: function over poses as a Probability Distribution Function (PDF) Robotics—Kinematics and Dynamics; G.3 [Artificial Intelligence]: which describes the “likelihood” function over poses. Given train- Learning ing poses, we can learn all parameters of this PDF by the standard Keywords: Character animation, Inverse Kinematics, motion approach of maximizing the likelihood of the training data. In or- style, machine learning, Gaussian Processes, non-linear dimension- der to meet the requirements of real-time IK, we represent the PDF ality reduction, style interpolation over poses using a novel model called as a Scaled Gaussian Pro- cess Latent Variable Model (SGPLVM), based on recent work by Lawrence [2004]. All parameters of the SGPLVM are learned au- 1 Introduction tomatically from the training data, the SGPLVM works well with small data sets, and we show how the objective function can be op- Inverse kinematics (IK), the process of computing the pose of a hu- timized for new poses in real-time IK applications. We additionally man body from a set of constraints, is widely used in computer an- describe a novel method for interpolating between styles. imation. However, the problem is inherently underdetermined: for Our style-based IK can replace conventional IK, wherever it is example, for given positions of the hands and feet of a character, used. We demonstrate our system in the context of a number of there are many possible character poses that satisfy the constraints. applications: Even though many poses are possible, some poses are more likely • Interactive character posing, in which a user specifies a sin- than others — an actor asked to reach forward with his arm will gle pose based on a few constraints; most likely reach with his whole body, rather than keeping the rest • of the body limp. In general, the likelihood of poses depends on Trajectory keyframing, in which a user quickly creates an the body shape and style of the individual person, and designing animation by keyframing the trajectories a few points on the this likelihood function by hand for every person would be a dif- body; ficult or impossible task. Current metrics in use by IK systems • Real-time motion capture with missing markers, in which (such as distance to some default pose, minimum mass displace- 3D poses are computed from incomplete marker measure- ment between poses, or kinetic energy) do not accurately represent ments; and email: [email protected], [email protected], hertz- • Posing from a 2D image, in which a few 2D projection con- [email protected], [email protected]. Steve Martin is straints are used to quickly estimate a 3D pose from an image. now at University of California at Berkeley. Permission to make digital or hard copies of part or all of this work for personal or The main limitation of our style-based IK system is that it re- classroom use is granted without fee provided that copies are not made or distributed for quires suitable training data to be available; if the training data does profit or direct commercial advantage and that copies show this notice on the first page or not match the desired poses well, then more constraints will be initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To needed. Moreover, our system does not explicitly model dynam- copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any ics, or constraints from the original motion capture. However, we component of this work in other works requires prior specific permission and/or a fee. have found that, even with a generic training data set (such as walk- Permissions may be requested from Publications Dept., ACM, Inc., 1515 Broadway, New York, NY 10036 USA, fax +1 (212) 869-0481, or [email protected]. ing or calibration poses), the style-based IK produces much more © 2004 ACM 0730-0301/04/0800-0522 $5.00 natural poses than existing approaches. 522 2 Related work 2003; Howe et al. 2000; Ramanan and Forsyth 2004; Rosales and Sclaroff 2002; Sidenbladh et al. 2002]. These systems are similar The basic IK problem of finding the character pose that satisfies to our own in that a model is learned from motion capture data, and constraints is well studied, e.g., [Bodenheimer et al. 1997; Girard then used to prefer more likely interpretations of input video. Our and Maciejewski 1985; Welman 1993]. The problem is almost al- system is different, however, in that we focus on new, interactive ways underdetermined, meaning that many poses satisfy the con- graphics applications and real-time synthesis. We suspect that the straints. This is the case even with motion capture processing where SGPLVM model proposed in our paper may also be advantageous constraints frequently disappear due to occlusion. Unfortunately, for computer vision applications. most poses that satisfy constraints will appear unnatural. In the A related problem in computer vision is to estimate the pose absence of an adequate model of poses, IK systems employed in of a character, given known correspondences between 2D images industry use very simple models of IK, e.g., performing IK only on and the 3D character (e.g., [Taylor 2000]). Existing systems typi- individual limbs (as in Alias Maya), or measuring similarity to an cally require correspondences to be specified for every handle, user arbitrary “reference pose.” [Yamane and Nakamura 2003; Zhao and guidance to remove ambiguities, or multiple frames of a sequence. Badler 1998]. This leaves an animator with the task of specifying Our system can estimate 3D poses from 2D constraints from just a significantly more constraints than necessary. few point correspondences, although it does require suitable train- Over the years, researchers have devised a number of techniques ing data to be available. to restrict the animated character to stay within the space of natural A few authors have proposed methods for style interpolation in poses. One approach is to draw from biomechanics and kinesiol- motion analysis and synthesis. Rose et al. [1998] interpolate motion ogy, by measuring the contribution of individual joints to a task sequences with the same sequences of moves to change the styles of [Gullapalli et al. 1996], by minimizing energy consumption [Gras- those movements. Wilson and Bobick [1999] learn a space of Hid- sia 2000], or mass displacement from some default pose [Popovic´ den Markov Models (HMMs) for hand gestures in which the spac- and Witkin 1999]. In general, describing styles of body poses is ing is specified in advance, and Brand and Hertzmann [2000] learn quite difficult this way, and many dynamic styles do not have a HMMs and a style-space describing human motion sequences. All simple biomechanical interprepration. of these methods rely on some estimate of correspondence between the different training sequences.