What is ? Generate perception of motion with sequence of images shown in rapid succession EECS 487: Interactive • humans “see” smooth motion at 12−70 fps Must be technically excellent, but more importantly, aesthetically, emotionally compelling • violation of realism may at times be desirable Animation “pipeline”: Lecture 33: • Introduction to animation • Key-frame • Inverse kinematics and

McMillan,O’Brien

Traditional Animation 2D animation: 1. Straight ahead: draw each frame, • CADrawing and painting are now routine one frame at a time • but 2D in-betweening () • lead to spontaneity is hard to get right • great control Instead, we assume 3D model of scene • tedious: 24 fps, 1,440 frames/minute, • for each scene, vary parameters to generate 130K frames for a 1.5 hour movie desired pose for all objects 2. Pose-to-pose (developed by Walt Disney): • stop-motion: shooting miniature physical • director plans shots using models frame by frame • senior artists sketch key poses (keyframes) • typically when motion changes • interns fill in the in-between frames • all line drawings are painted on • composed in layers • background changes infrequently, can be reused • photograph finished -stack onto film

Yu,Marschner,Durand,Hodgins Some Artistic Considerations Principles of Eleven principles of traditional Goal: make characters that move in animation compiled by Lasseter: a convincing way to communicate 1. Squash and stretch personality and emotion 2. Slow in, slow out Many of these principles

Animation principles developed by 3. Timing follow indirectly from Disney in the 20’s−30’s, adapted by 4. Anticipation physics, e.g., CG , e.g., Lasseter of 5. Follow through anticipation, follow- Pixar (now Disney) 6. Overlapping action through, and many other

7. Secondary action effects can be produced The most important source of 8. Arcs traditional animation principles is by simply minimizing 9. Exaggeration the book by Thomas and Johnston, physical energy, but 10. Appeal Disney Animation: The Illusion of Life exaggerated 11. Staging Marschner

Squash and Stretch and Slow In/Out Timing of Motion Squash and stretch Timing can completely change the • rigidity/softness via distortion during motion interpretation of motion • pseudo-physics: carrying momentum • increase the sense of speed Time spent on action affects perception • try to keep the volume constant • timing indicates weight Comet • speed determines emotion Slow in and out • an extreme pose can be emphasized by slowing down as you get to it Since timing is so critical, animators (and as you leave it) • in practice, many things do not move usually draw a time scale next to abruptly but start and stop gradually keyframes to indicate how to generate • pseudo-physics: overcoming inertia the in-between frames no fx squash/stretch squash/stretch +slow in/out only

Owen Maestri Actions do not end abruptly Anticipation Follow Through • hand continues to move after throwing a ball: inertia An action can be divided into three: and Overlapping • audience likes to see resolution • anticipation of action • action Action • discontinuities are unsettling • reaction • the termination of an action Anatomical motivation: a muscle anticipates the next must extend before it can contract • overlaps indicate intentions Prepares audience for an action • don’t surprise the audience • direct their attention to what’s important Amount of anticipation can affect perception of speed and weight

Comet

Secondary Actions and Arcs Exaggeration and Appeal Use secondary actions to Exaggeration increase complexity of • get to the heart of the idea and emphasize it so the audience can see it scene, but it should not • choose which properties to exaggerate interfere with the primary action Appeal • the character must interest the audience Avoid straight lines since most • it doesn't have to be cute and cuddly things in nature move in arcs • design, simplicity, behavior Comet all affect appeal • avoid perfect symmetries • example: Luxo, Jr. was made to appear childlike freetoon.com Staging What is Animation? Present the idea so that it is unmistakably clear • audience can only focus on one thing at a time: Make objects change over time main object should be contrasted • stage action in silhouette Key technical problems are how to specify, • in dialogue, characters should face ¾ towards generate, and manipulate motion the camera, not right at each other Four alternatives: good bad 1. brute force: model each frame 2. key-frame animation: • key poses specified by hand • or poses recorded by motion capture • interpolate in-between frames

comet-cartoons.com/3ddocs/charanim/ Durand

What is Animation? Key-frame Character Animation

3. procedural/behavioral: Two approaches: describe motion algorithmically 1. blend shapes or morph targets • local rules, global emergent behavior: boids, 2. rigged characters or articulated brain-spring models • procedural texture: crack propagation in glass or concrete, metallic patina, stone aging, water Blend shapes: flow and rust • a very simple surface control scheme 4. physical simulation: • no skeleton motion according to physical laws • based on interpolating (tweening) among several key poses • particle systems for fire, smoke • given a number of base meshes, combine • mass-spring damper arrays for fluttering cloth them with time-dependent coefficients • fluid simulation

Durand Marschner Blend Shapes Rigged Character Setup: • user provides key shapes with a position (pij) for every control To support more complex character point i in shape j animation, models are often based on • k w for each frame user provides a weight j,k for each key shape jointed skeletons (articulated models) (∑jwj,k = 1), i.e., how much each key shape affects frame k • the shape for frame k can be computed from the control points: p w p ik = ∑j j,k ij Kinematics: the study of movement of Works well for relatively small motions articulated models • runs in real time, e.g., as vertex shader • began in the mechanical engineering of robots • often used for facial animation • popular for games The skeleton can be fleshed out in any way, e.g., with mesh skinning • skin deforms following bone movements

Marschner Marschner, Gillies

Articulated Model Degrees of Freedom (DoFs)

hip Setup: • rigid parts: each link (bone) in the thigh articulated chain (skeleton) is rigid Hinge/pin joint: 1 DoF • not physically accurate, but rigid • knee or elbow joint skeleton constrains movement knee • connected by joints: movement is constrained by the degree of Saddle: 2 DoFs • wrist/hand joints shin freedom at each joint

Can be animated by specifying

ankle the joint angles as functions of foot Ball/socket joint: 3 DoFs time • hip, shoulder, neck

Gillies Marschner, Gillies Key-frame Animation Key-frame Animation With the character rigged, next specify Steps: key poses at specific time steps: • each pose controlled by a set of 1. character modeling: design the geometry variables: joint angles, positions, etc. 2. character rigging: set up a bunch of parameters

• each variable changes as a function of time Key frames (created first) • joint angles, positions, etc. • for each variable, specify its key value at 3. set up key-framing “important” or key frames • specify key values of parameters at specific times • in-between frames will be created by • by forward kinematics interpolating these key values • by inverse kinematics • more generally, each variable may have • incl. motion capture a different set of key frames Time • specify an interpolation for the in-between values

Durand,O’Brien Yu

Straight ahead order of frame creation

Forward Kinematics Hierarchical Modeling

hip Could animate by moving every control point (joint) Calculate the position and orientation at every key frame: tedious, hard to get smooth, thigh of a limb’s end point (end effector) as consistent motion a function of the angles of all joints: φ p = f(Θ) Animation need to be controlled at a higher level: knee where: • “bend elbow” instead of p: position of end effector • “move left forearm one square inch” Θ: angles, positions, … of joints ϕ Model objects as a hierarchy of components shin ⎡ ⎤ ⎡ ⎤ px g(φ,ϕ,θ) • encodes topology (what’s connected to what) For example: ⎢ ⎥ = ⎢ ⎥ p h(φ,ϕ,θ) • specifies geometric relations from joints: foot ⎢ y ⎥ ⎣⎢ ⎦⎥ ankle ⎣ ⎦ tree structure θ p • each component defined relative to parent • independent of display geometry Yu Marschner,Gillies,Yu, Hodgins,Durand Moving Components Embedding Transformations Define a coordinate system at the top of the To draw an object, traverse its hierarchical model hierarchy and at each joint Two types of nodes: • object nodes: draw them (may include attributes) Each joint is thus a separate frame of reference, • transform nodes: with its own local coordinate system • multiply into Current Matrix on the way down • remove from Current Matrix on the way up Each local coordinate system is defined in the constantly changing local coordinate system coordinate system of the previous frame Allows changes at different scales We can express positions frame i • apply rotation above “Upper Body” and directions for frame i • vs. rotation above “L Arm” in the coordinate system of link i Be careful about transformation order frame i−1 by a matrix • (usually) scale before rotate transformation link i-1 • (usually) rotate before translate frame i-1 Gillies Yu

Use OpenGL Matrix Stack Example human() human’s center of mass, more { s U e1 pelvis(); realistic motion Out of two bones: glPushMatrix(); e2 L w glTranslate( ... ); glRotate( ... ); We want to model an arm as a hierarchy: lhip(); . . . where: glPopMatrix(); t glPushMatrix(); U: upper arm φ glTranslate( ... ); L: lower arm (forearm) U glRotate( ... ); rhip(); glPushMatrix(); glTranslate( ... ); User controlled parameters: L θ glRotate( ... ); φ: shoulder joint angle knee(); w . . . θ: elbow joint angle glPopMatrix(); glPopMatrix(); t: where shoulder meets torso . . . w: wrist joint location } Hodgins Yu Positioning the Forearm Attaching Forearm to Upper Arm

Initially, segments in same position Second, align corresponding elbow

s: shoulder joint location s U e1 (3) translate(e1) e , e : elbow joint locations e w L w 1 2 2 L w: wrist joint location

s U e1 = e2

First, perform elbow rotation w • translate elbow joint to origin Third, perform shoulder rotation • operate on whole arm • rotate by given angle (θ) L L w (4) translate(−s) (1) translate(−e2) s U e (5) rotate(φ) e1 = e2 (2) rotate(θ) θ 1 e 2 U φ s Yu Yu

Placing the Shoulder Converting to Hierarchy

Fourth, put shoulder in place w (6) (1) translate(−e ) (6) translate(t) 2 L (2) rotate(θ) (5) θ e1 = e2 (4) (3) translate(e1) U (3) φ (4) translate(−s) s = t (5) rotate(φ) (2)

Important things to notice (1) • limited control knobs (just the angles) (6) translate(t) • automatically handle interconnection (elbow joint)

Yu Yu Properties of Hierarchy Inverse Kinematics (x, y) Geometry is always at the leaves Forward kinematics: the angle of all joints • internal nodes are transform nodes are explicitly specified by the x = L cosθ + L cos(θ + θ ) There are two types of transforms 1 1 2 1 2 y = L sinθ + L sin(θ + θ ) • structural 1 1 2 1 2 This gets tedious … even with hierarchical modeling • fixed at design time • keeps things together Inverse kinematics: the animator drags the hands and • control knobs feet into place and inverse kinematics solves for the • variable parameters angles to achieve the final position • controlled by user

Yu

Inverse Kinematics Example Inverse Kinematics Determine joint angles from position of end effector Unfortunately, real models are much more complex: • a human has around 200 degrees of freedom First compute θ2: by cosine rule, (x, y) 2 2 2 Suppose we specify locations of end effectors C = A + B − 2 A B cosθ • the mapping of parameters to effector positions is non-linear 2 2 2 2 x + y = L + L − 2L L cos(180 − θ ) • inverting this function is not possible 1 2 1 2 2 ⎛ x 2 + y2 − L2 − L2 ⎞ • we need to calculate the relative positions of θ = cos−1 1 2 all intermediate links to achieve the pose 2 ⎜ 2L L ⎟ ⎝ 1 2 ⎠ • this is an ill-posed problem (there may be Then solve for θ by expanding x and y (from prev slide) infinitely many solutions for some chains) 1 • need to find a constrained solution, minimizing using: cos(ϕ + ψ) = cos ϕ cos ψ − sin ϕ sin ψ for example, the joint movements, maintain balance, … ? sin(ϕ + ψ) = sin ϕ cos ψ + sin ψ cos ϕ • similarly, there may not be any parameter settings that work • need to pick one that is “close enough” ⎛ −(L sinθ )x + (L + L cosθ )y⎞ θ = tan−1 2 2 1 2 2 • both involve some kind of optimization algorithm that rely 1 ⎜ (L sin )y (L L cos ) ⎟ ⎝ 2 θ2 + 1 + 2 θ2 x ⎠ on numerical methods, e.g., the Jacobian

Yu,Hodgins Yu Motion Capture Motion Capture Technologies Mechanical Instead of specifying end effector positions • measure joint angles directly manually, measure it from the real world • works in any environment • restricts motion Captures style, subtle nuances and realism

Popovic,O’Brien

Motion Capture Technologies Motion Capture Technologies Magnetic Optical Passive • tethered or wireless • strap a bunch of passive markers • transmitter emits field on subject (body, face) • trackers sense field • location of markers tracked by 8 • trackers report position and or more cameras orientation • triangulate to get marker’s 3D position retroreflective markers Disadvantages: • convert this to joint angles and map to articulated model • nearby metal objects cause distortions • high frequency (240 Hz) • limited range • restricted volume: studio size, • limited number of trackers lighting, number of cameras • low frequency (60 Hz) • occlusions are troublesome

camera with IR illuminators Popovic,O’Brien Popovic,O’Brien Optical Motion Capture Process Marker Data to Motion 1. Start with standard rest pose Motion capture gives inconvenient raw data 2. Calibrate: match skeleton, find offsets to • passive optical gives “least information” markers • accurate position, but correspondence difficult 3. Identify and uniquely label markers • which marker is which? • where are the markers relative to the skeleton? 4. Motion trial: use a short sequence that exercises all DOFs of the subject 5. Track forward through time (but watch for markers dropping out due to occlusion!) 6. Compute joint angles: explain data using skeleton DOFs an inverse kinematics problem per frame!

Popovic Popovic

Motion Capture Technologies Pros and Cons of Motion Capture

Active Optical Mocap data is very realistic • uses LEDs instead of passive markers • LEDs blink IDs correspondence automatic • timing matches performance exactly • number of markers trades off with frame rate • dimensions are exact

But it is not enough for good character animation • noise, errors from non-rigid marker mounting • contains no exaggeration • limited in the complexity of the scenes they can capture

Phase Space O’Brien Popovic,Yu Pros and Cons of Motion Capture Performance Capture

To increase versatility of mocap: Mocap is no panacea for assigning key values to • break scenes into smaller pieces and re-construct later parameters mocap data is generally a starting • gather lots of snippets of motion capture point for skilled animators to create the final • e.g.: several ways to dunk, dribble, pass product • arrange them so that they can be pieced together smoothly • at run time, figure out which pieces to play for desired Many studios regard motion capture as motion low quality, cheap motion

• no directive/creative control Automated ? Performance capture is different Problem: once the data is captured, it’s hard to • use mocap device as an expressive input device modify for a different purpose • e.g., James Cameron’s Avatar

Chenney O’Brien