Computer : Outline It’s Past, It’s Present And its Future at UW „ A brief history of „ (animation appreciation) – Video! Michael Gleicher „ Overview of the Production Process Graphics Group „ Where motion comes from – Video! Department of Computer Sciences „ Animation at UW CS University of Wisconsin, Madison „ (what we do) – Video! www.cs.wisc.edu/graphics „ Graphics and Animation Classes „ (what you can do) – Video!

Goals Caveats

„ Give you some idea of what „ This talk is skewed toward 3D Computer Animation is about Animation for film and video „ Give you some idea of where (some „ That ignores lots of interesting stuff of) the hard problems are „ 2D and image effects „ Games, Visualization, Simulation, ... „ Give you an idea of we do „ Focus on „ Try to get you excited about doing it „ Over-emphasize my heroes „ Show some fun video „ This is a rehash of some old stuff „ Little (or no) technical content

Timeline Pioneer Days (pre-1980s)

„ What was happening: ComingComing EarlyEarly ModernModern „ Early pioneers (in research labs) PioneersPioneers OfOf DaysDays EraEra started making movies with computers AgeAge „ General Story: 1982 1987 1995 „ It was hard to make pictures with „ Phases of the computers, but people did it anyway „ Why it ended: „ Computer Animation becomes feasible

1 Early Days (early to mid- Early Days (early to mid- 1980s) 1980s)

„ What was happening: „ Notable examples: „ Computer animation in film and video „ TRON (‘82), The Last Starfighter (’84) „ What it looked like: „ Many TV spots and commercials „ Computer graphics „ What it looks like today: „ Things computers could draw well „ Quaint „ Chrome! Flying Logos! „ Why it ended „ AUDIENCES grew more savvy

Works Ant Last Starfighter

Sexy Robot Flying Logos

2 Early Days (early to mid- 1980s) Luxo Jr

„ Notable examples: „ TRON (‘82), The Last Starfighter (’84) „ Many TV spots and commercials „ What it looks like today: „ Quaint „ Why it ended „ AUDIENCES grew more savvy

A critical moment in Computer Animation Comes of Age history… (late 80s – mid 90s)

„ Computer Animation not just for Luxo Jr. computer animation’s sake Pixar, 1986 J. Lasseter (dir)

„ First computer animation nominated for an Academy Award „ Notable character animation It’s the motion! (well, it looks good too) Beauty and the Beast, 1991 Jurassic Park, 1993

Glory Days of Computer Animation Another important film…

„ What will they do next? Toy Story „ Notable examples Pixar, 1995 „ Tin Toy ’88 (and other Pixar Shorts) J. Lasseter (dir) „ Terminator 2 ’91 „ Beauty and the Beast ’91 „ Feature length, 3D animated film „ Jurrassic Park ’93 „ Financially and artistically successful „ … more and more and more … „ Everybody wants to copy

3 Why is Toy Story Different? End of the Glory Days?

Tin Toy Toy Story „ No more firsts? 1988 1995 „ Glory is for good films, not cool effects „ Not just animation for animation’s sake „ Same basic technology as shorts „ Hackers are not the heroes anymore „ Different magnitude of problems „ Technology is generally available. „ How to deal with all the bits! „ (Kindof) „ Must hold attention for 90 minutes „ Artistic factors drive

Toy Story++ Shrek

„ More and more animated films „ Greater levels of complexity „ Antz, Bugs Life, Toy Story 2, … „ New artistic styles and possibilities „ Tarzan, Prince of Egypt „ Final Fantasy,… „ Current state of the art

Final Fantasy The Modern Era (’95 - …)

„ No more firsts? „ Don’t believe your eyes! „ Anything is possible! „ Seamless integration of CG and real „ Some landmarks „ Titanic ’97 „ What Dreams May Come ‘98 „ The Mummy, Episode 1, The Matrix ’99 „ Hollowman, Perfect Storm ’00 „ Pearl Harbor ’01

4 Hollowman The Modern Era (’95 - …)

„ No more firsts? „ Don’t believe your eyes! „ Anything is possible! „ Seamless integration of CG and real „ Some landmarks „ Titanic ’97 „ What Dreams May Come ‘98 „ The Mummy, Episode 1, The Matrix ’99 „ Hollowman, Perfect Storm ’00 „ Pearl Harbor ’01

How do you do it? Design

Post- Post- Design Modeling Animation Lighting Rendering Design Modeling Animation Lighting Rendering Production Production

„ “Typical” or Traditional pipeline „ One of the most important pieces „ Divides tasks „ Deciding what animation is to be „ Variants exist made

Modeling Animation

Post- Post- Design Modeling Animation Lighting Rendering Design Modeling Animation Lighting Rendering Production Production

„ What are the objects? „ Animate: to bring to life „ What do they look like? „ Making things move „ How do they move?

5 Lighting Rendering

Post- Post- Design Modeling Animation Lighting Rendering Design Modeling Animation Lighting Rendering Production Production

„ Making choices in how things look „ Let the computers color the pixels „ Term is a “Pixarism” „ The compute intensive part „ Catch-all for lots of appearance design

Post-Production Why focus on motion?

Post- Design Modeling Animation Lighting Rendering „ Production It is the heart of animation „ It’s the most unique aspect of animation „ Put the pictures into final form „ It’s the thing we have the least experience with „ It’s the thing I am most interested in

What is a character? (for the purposes of motion) What parameters?

„ Some “object” (geometry) „ It depends! „ Configuration or Pose given by a set of numbers „ Parameters „ Enough controls to be expressive „ Control knobs „ Process to (or draw) „ Few enough to be easy geometry given parameters „ Articulation: define how parameters drive geometry „ Standard methods for gross motion „ Anything goes for details

6 What are those parameters? Fortunately, we don’t care How do we model a human? about details

„ Humans are quite „ “Gross Body Motion” complex… „ Overall movement „ Small details hidden „ Under clothing „ Under simplified drawing models „ View from far enough away „ Chains of rigid segments

Animation Apreciation 101 Abstractions

„ Brilliance (Sexy Robot) „ Robert Abel and Associates, 1985 „ Early „ Early computer graphics look (chrome) „ Final Fantasy „ Square Studios, 2001 206 bones, 206 bones, 53 bones „ Realistic, animated, human characters muscles, fat, complex joints Kinematic organs, joints „ Hollowman clothing, … „ Sony Imageworks (effects), 2000 „ Complex human models, terrible dialog

Abstractions vs. Reality Abstraction of Human (skeletons vs. humans) Motion

Representation of complex human structure with varying degrees of simplification

Simple Pin Joint

„ Question of Approximating DOF’s „ Some number of connected, rigid pieces „ (usually) Complex tendon and „ Kinematic joints bone system

7 What is a motion (2) Why is this so hard?

„ A motion maps times „ We are good at looking at motion! to configurations „ Motion is very expressive m(t) ∈ℜ ⇒ ℜn „ Mood, activity, personality, … „ Vector-valued, time- varying signal „ But those attributes are subtle „ Representation comes from creation „ What makes a motion sad? Realistic? „ typically interpolation „ We lack vocabulary „ may not be convenient for editing „ Talk about motion with metaphor signal (p) Time (t)

Three main ways to make Creating Motion by Hand: motion Keyframing

„ Create it by hand „ Skilled place “key” poses „ Compute it „ Computer “in-betweens” „ Capture it from a performer „ Requires incredible amounts of talent „ But can be done extremely well „ Re-use an existing motion „ (don’t make it at all) Verdict: Produces the highest quality results, at a very high cost

Computing Motion: Procedural and Simulation Atlanta in Motion

„ Define algorithms to create motions „ Ad-hoc rules, or simulate physics „ Physics provides realism „ But how do you control it?

Verdict: Good for secondary effects, not for characters (yet)

8 Computing Motion: Baraff Horse Procedural and Simulation

„ Define algorithms to create motions „ Ad-hoc rules, or simulate physics „ Physics provides realism „ But how do you control it?

Verdict: Good for secondary effects, not for characters (yet)

Motion Capture and Performance Animation Spacetime Swing

„ Use sensors to record a real person „ Get high-degree of realism „ Which may not be what you want... „ Possibility for real-time performance

Verdict: Good for realistic human motions. Scary to animators.

Motion Capture and DD Ghosts Performance Animation

„ Use sensors to record a real person „ Get high-degree of realism „ Which may not be what you want... „ Possibility for real-time performance

Verdict: Good for realistic human motions. Scary to animators.

9 Motion Capture Technology: Motion Capture Technology: Optical Tracking Video

„ User markers and special cameras „ An interesting and open problem… „ Tracking + Math „ Limited information „ But seemingly enough „ Problem can be arbitrarily hard „ Or easy – if you make assumptions „ Video is surprisingly bad

So now you have motion... Problem: Motion is Specific

„ Transform Motion to new uses

Specific Action Specific Character different sized hand is character not here doesn’t fit

Edit motion to Retarget motion to meet new needs new character

Retargetting Motion to Transformation Basics New Characters Change what isn’t important, retain what is

„ Goal: „ Hard to define what is important one motion, „ high-level properties a cast of „ motion specific characters „ Stick to what’s easy to define „ Similar „ geometric constraints structure „ signal characteristics „ framework for better metrics later

10 Transformation as Constrained Optimization Spacetime Constraints

„ Find a motion that… 1. Meets any specific requests 2. Keeps any specific characteristics of the original 3. Is as similar as possible to the original „ Naturally posed as constrained optimization „ Consider all constraints simultaneously „ subject to meeting the constraints (1 and 2) „ NOT frame at a time minimize some objective (3) „ Solve for motions „ “best” motion that meets constraints

What does it take to do Retargeting Compendium this?

„ Setup and Solve a BIG math problem „ Non-linear, variational, constrained optimization „ Thousands of simultaneous equations „ Yes, you can do it in real time „ with some caveats...

How did we do that? Where did we do that?

„ The UW CS Graphics Group! Get Motion Annotate Compute (from library) Constraints Adapted Motion „ The initial retargeting work was Our Software done before I came here Get from library Convert (stock CD or Web) Skelleton „ There’s a graphics group here? „ I thought we only do stuff like Get Character Devise skeleton Databases and Architecture? Lighting Render (from library) for character Commercial Animation System (3D Studio MAX)

11 Graphics at UW CS UW logo

„ Target for growth in the department http://www.cs.wisc.edu/graphics „ New courses: „ Introduction to Graphics • now taught as CS638, will get new number soon „ Computer Animation • now taught as CS838, will get new number soon „ Other courses elsewhere • Art (comp. anim), ECE (image proc), ...

http://www.cs.wisc.edu/graphics http://www.cs.wisc.edu/graphics

„ History „ Faculty „ Early 80s – grad students try to do „ Mike „ Mid 90s – identified as “important new area” „ Stephen „ 1998 – Perry Kivolowicz teaches first course „ Friends (Chuck, Vadim, Nicola, …) „ 1998 – Gleicher is hired to form group „ 7 graduate students (5 mike, 2 Stephen) „ 1999 – First animation course (838) „ The only RAs in the department without • (survivors still exist) offices „ 1999 – Regular undergrad graphics course „ 4-5 undergrads „ 2000 – Chenney hired, first graphics students „ Lab shared with Computer Vision group „ 2001 – Graphics qualifying exam, … „ 1347 CS&S

http://www.cs.wisc.edu/graphics CS Graphics Courses

Mike’s Research Stephen’s Research Undergraduate & Graduate Authoring by Adaptation Scalability and Control „ CS559 – Computer Graphics „ Motion Editing of Physical Systems „ General intro of field „ Automatic Stylization „ Control of Physics „ Required for everything else „ Virtual Videography „ Scaling Physics „ The “Extra Session” experiment „ Human motion „ Terrain Synthesis „ CS679 – Computer Games Technology reconstruction „ Cartoon Physics Graduate „ „ Crowd Simulation „ CS777 – Computer Animation „ CS779 – Rendering Media Clip Elsewhere on campus…

12 Answers to Frequently Beat Down Asked Questions

„ It depends. „ Yes, 559 is a lot of work, requiring both hacking and math. „ Maybe. „ Yes, we do get undergrads involved in our research projects. „ 42 „ Stephen is in Peru this week. „ No, the software is not available for you to try yourself. „ It depends. „ C++ is the programming language of choice for 559, just as English is the lecture language of choice. It’s mainly a matter of what is most convenient. Many students pick it up as they go. „ Yes, there is one more distinguished lecture this week.

13