EUROGRAPHICS 2010 / T. Akenine-Möller and M. Zwicker Volume 29 (2010), Number 2 (Guest Editors) BetweenIT: An Interactive Tool for Tight Inbetweening Brian Whited1,2 Gioacchino Noris3 Maryann Simmons1 Robert W. Sumner4 Markus Gross3,4 Jarek Rossignac2 1Walt Disney Animation Studios 2Georgia Tech 3ETH Zürich 4Disney Research Zurich Abstract The generation of inbetween frames that interpolate a given set of key frames is a major component in the pro- duction of a 2D feature animation. Our objective is to considerably reduce the cost of the inbetweening phase by offering an intuitive and effective interactive environment that automates inbetweening when possible while allow- ing the artist to guide, complement, or override the results. Tight inbetweens, which interpolate similar key frames, are particularly time-consuming and tedious to draw. Therefore, we focus on automating these high-precision and expensive portions of the process. We have designed a set of user-guided semi-automatic techniques that fit well with current practice and minimize the number of required artist-gestures. We present a novel technique for stroke interpolation from only two keys which combines a stroke motion constructed from logarithmic spiral vertex tra- jectories with a stroke deformation based on curvature averaging and twisting warps. We discuss our system in the context of a feature animation production environment and evaluate our approach with real production data. Categories and Subject Descriptors (according to ACM CCS): Computer Graphics [I.3.5]: Computational Geometry—Geometric algorithms Computer Graphics [I.3.6]: Methodology and Techniques—Interaction tech- niques 1. Introduction possible, as most often the characters have their own, unde- fined, laws of physics and deformation. Inbetweening is a cornerstone of the 2D animation pipeline. We approach the challenging task of automatic inbe- An animator produces drawings at key frames that capture tweening with two important observations derived from the heart of the animated motion. The inbetweener then studying the traditional animation process. First, tight inbe- draws the frames between each pair of keys that fill in the tweens, those drawn between two key frames that are very intermediate motion. A typical feature-length animation re- similar in shape, are among the most laborious and time con- quires over one million inbetween drawings, with ratios of suming to produce. These inbetweens are tedious to draw be- 3-4 inbetweens for each key [JT95]. cause they require the greatest amount of technical precision The manual production of inbetween frames is both diffi- and the least amount of artistic interpretation. By focusing cult and time-consuming, requiring many hours of intensive on tight inbetweening, we pinpoint an expensive portion of labor by skilled professionals. Furthermore, its cost is a sig- the inbetweening process while restricting the problem to nificant part of the total production budget. Consequently, one that is more tractable from an algorithmic standpoint. researchers have long sought an automatic solution. Despite Our second observation is that an effective system should a large body of work in this area spanning over four decades, automate as much as possible while effortlessly deferring to no definitive solution exists. This is in part because the prob- artist control whenever needed. We never take the artists out lem is ill-posed—there is no concise set of rules for produc- of the loop. Instead we seek to make them more productive ing high-quality inbetweens, especially in cases of complex by automating the tedious and time consuming tasks so they motion and changing occlusion. The artist relies on his or her can focus their efforts on areas where their creative talents perspective of the 3D world in which the animated characters and expertise are required. are embedded, as well as on an artistic vision for achieving Based on these observations, we have developed Be- the desired aesthetics. It would not be sufficient to derive the tweenIT, an interactive environment that automates tight in- three dimensional world from the drawings, even if it were betweening when possible while allowing the artist to guide, c Disney Enterprises Inc. Journal compilation c 2010 The Eurographics Association and Blackwell Publishing Ltd. Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and 350 Main Street, Malden, MA 02148, USA. Whited, Noris, Simmons, Sumner, Gross, Rossignac / BetweenIT: An Interactive Tool for Tight Inbetweening © Disney Figure 1: Tinker Bell: Frames (a) and (e) are the given key frames and (b-d) (except for the right hand) were generated automatically. The “shadows” of previous frames are used to visualize their evolution. The topology of the right hand is not compatible across the two keys and therefore not suitable for automation: the portion indicated in red was hand-drawn. Frame (c) is overlaid on top of the original hand drawn inbetween (red) on the right. complement, or override the results. BetweenIT operates on key drawings, advanced artistic skill and interpretation are vectorized pairs of consecutive key frames that were drawn required to produce satisfactory intermediate frames. In in the program or obtained from scanned drawings. These these cases, a further “breakdown” might be done, where the key frames are segmented automatically into strokes. The artist produces those frames within the range that present strokes of one key frame are then matched with correspond- special drawing problems. Tight inbetweens require less ing strokes of the subsequent key frame. Finally, interpolated artistic interpretation – but considerable technical skill to lay strokes are generated for each requested inbetween frame. down the lines accurately. Supported user editing options include: adjusting vertex tra- The inbetweened frames represent a significant portion of jectories across frames; resolving ambiguities in the corre- the drawings, budget, and time for a production. A typical spondence; and controlling the behavior of occluded strokes. feature animation averages about four drawings per frame All interactions can be specified by the artist graphically via (for different characters, props, etc.). An 80 minute feature a drawing interface. with 24 frames per second requires 460,800 drawings per production. If a quarter of these frames are done by the 2. Background animators, then that leaves 345,600 inbetweened drawings. While some of the underlying mechanisms have benefited Since multiple drawings are often produced before arriving from the advent of digital technology, a 2D production today at the final version, the final tally can add up to over a million follows much the same basic workflow as traditional anima- inbetweened drawings per full length production [JT95]. tion [JT95]. The process begins with a storyboard, which One critical consideration during animation and inbe- provides a visual representation of the story. In layout, the tweening is arcs of motion. Natural motion always follows staging for each scene is designed, including establishing the an arc of some sort, and therefore to achieve fluid and life- setting, choosing and placing character and prop elements, like animation, artists must capture these natural arcs. This and specifying camera motion and cuts. is one of the most challenging aspects of inbetweening since Character and effects animation is then done in multiple making a drawing on an arc is much more difficult than one stages. First, animators produce the subset of drawings that placed linearly between the keys [JT95]. lay down the core of the action, the “keys.” These extreme Our approach focuses on automating the process of drawings are often “ruff” versions that capture the spirit, inbetweening animated frames post clean-up for charac- flow, and arcs of the animation. A “clean up” artist is re- ters, props, and effects. We introduce a novel interpolation sponsible for taking the ruff drawings and producing clean scheme which automatically derives natural arcs of motion lines that remain true to the original intent. from pairs of consecutive keys. Our system fits into the cur- Each key drawing has one or more timing charts asso- rent 2D pipeline with minimal change. ciated with it (e.g. see Figure 1). The animator uses these charts to specify how many drawings should be produced 2.1. Related work between keys, and at what intervals. It is the job of the inbe- The problem of automatic 2D inbetweening dates back more tweening artist to draw the requested intermediate drawings than forty years to the inception of computer graphics as a to produce seamless motion. field of research [MIT67]. To date, it has remained unsolved. When inbetweens are needed for dramatically different The many challenges in automatic inbetweening include the c Disney Enterprises Inc. Journal compilation c 2010 The Eurographics Association and Blackwell Publishing Ltd. Whited, Noris, Simmons, Sumner, Gross, Rossignac / BetweenIT: An Interactive Tool for Tight Inbetweening information loss and ambiguity inherent in a 2D projection alignments. The MeshIK system of Sumner and colleagues of a 3D character, and the topology variations between key [SZGP05] includes a boundary-based interpolation scheme frames that result from changes in occlusion. An automated that does not require alignment, but it does not address inte- approach must address several challenges comprising: es- rior texture morphing. As shown by Baxter, Barla, and An- tablishing correspondence