A System for Efficient 3D Printed Stop-Motion Face Animation
Total Page:16
File Type:pdf, Size:1020Kb
A system for eicient 3D printed stop-motion face animation RINAT ABDRASHITOV, ALEC JACOBSON, KARAN SINGH, University of Toronto Fig. 1. Given an input mesh-animation sequence (a), our system segments and deforms the 3D mesh into parts that can be seamlessly joined (b). Each part is independently used to compute a replacement library, representative of the deforming part in the input (c). StopShop iteratively optimizes the library and a mapping from each frame of the input animation to the pieces in the library. Libraries are 3D printed and each frame is assembled according to the optimized mapping (d). Filmed in sequence, the animation is realized as a stop motion film (e). Computer animation in conjunction with 3D printing has the potential to used as the preferred technique for producing high-quality facial positively impact traditional stop-motion animation. As 3D printing every animation in stop motion lm [Priebe 2011]. frame of a computer animation is prohibitively slow and expensive, 3D Faces and 3D models in general are created digitally (or physi- printed stop-motion can only be viable if animations can be faithfully re- cally sculpted and scanned) to produce a replacement library that produced using a compact library of 3D printed and eciently assemblable covers the expressive range of the 3D model. This library, typically parts. We thus present the rst system for processing computer animation containing thousands of variations of a deformable model is then 3D sequences (typically faces) to produce an optimal set of replacement parts for use in 3D printed stop-motion animation. Given an input animation printed and cataloged. Additional post-processing may be required, sequence of topology invariant deforming meshes, our problem is to out- including sanding down edges, smoothing inconsistencies, and hand put a library of replacement parts and per-animation-frame assignment of painting the 3D prints. The replacement library is then ready to be the parts, such that we maximally approximate the input animation, while used in stop-motion sequences [Alger et al. 2012]. Alternately, the minimizing the amount of 3D printing and assembly. Inspired by current 3D model could be entirely computer animated, and each animation stop-motion workows, a user manually indicates which parts of the model frame of the model independently 3D printed and post-processed are preferred for segmentation; then, we nd curves with minimal deforma- for use on a physical set. tion along which to segment the mesh. We then present a novel algorithm In either case, the cost in terms of printing and post-processing to zero out deformations along the segment boundaries, so that replacement time, material, storage and money is prohibitive. Each character of sets for each part can be interchangeably and seamlessly assembled together. Laika’s stop-motion feature lm Coraline could have as many as The part boundaries are designed to ease 3D printing and instrumentation for assembly. Each part is then independently optimized using a graph-cut 15,000 faces and up to 250,000 facial expressions [Kolevsohn 2009]. technique to nd a set of replacements, whose size can be user dened, or Paranorman required 8308 pounds of printer powder, and 226 gallons automatically computed to adhere to a printing budget or allowed deviation of ink over the course of production [Priebe 2011] (see Figure 2). from the original animation. Our evaluation is threefold: we show results This current practice for character faces (let alone complete 3D on a variety of facial animations, both digital and 3D printed, critiqued by a models) is expensive for large lm studios and completely beyond professional animator; we show the impact of various algorithmic parame- the reach of independent lmmakers. ters; and compare our results to naive solutions. Our approach can reduce Due to the tedious nature of physically moving or replacing ob- the printing time and cost signicantly for stop-motion animated lms. jects in the scene for each frame, stop motion objects are typically animated at a lower framerate (often "on twos” or every other frame). Some lms, such as Aardman’s Flushed Away or Blue Sky’s The 1 INTRODUCTION Peanuts Movie, even opt to simulate the aesthetic appeal of stop- Stop-motion is a traditional animation technique that moves a motion entirely, via computer animation. As evident by these lms, physcial object in small increments between photographed frames, the slight choppiness and lower framerate can be an intentional to produce the illusion of uid motion. As with animation in gen- artistic decision. Our research addresses both 3D printing costs and eral, arguably the most expressive part of a character is its face. animation aesthetic, providing users with a system that can produce Extensive use of clay replacement libraries for dialogue and facial animation sequences in a stop-motion style digitally, or physically expressions, goes back as far as The New Gulliver 1935. The use of a with minimal 3D printing, saving printing time and material. replacement library has become the standard approach to the stop- We present an end-to-end solution designed to optimize the 3D motion animation of expressive deformable objects, in particular for printing of a replacement library of a deformable 3D object, such facial animation. With the advent of 3D printing, replacement ani- that high-quality stop-motion approximations of input computer mation has become a bridge between the disparate worlds of digital animations can be assembled from that library (see Figure 1). At the computer animation and physical stop-motion, and is increasingly core of our system is an optimization problem (Section 3.3) whose 2019-07-23 17:05 page 1 (pp. 1-11) 39:2 • Rinat Abdrashitov, Alec Jacobson, Karan Singh focus our attention on research topics closest to our problem at an abstract level and those similar in methodology. Stop-motion. Commercial stop-motion software such as Stop Mo- tion Pro or DragonFrame, focuses on optimized camera controls and convenient interfaces for assembly and review of captured im- ages. There is long history of research on interfaces and techniques for performance animation, such as for paper cut-out animations [Barnes et al. 2008]. Stop-motion armatures have also inspired re- search into tangible devices [Bäecher et al. 2016; Knep et al. 1995] and interfaces [Singh and Fiume 1998] for posing and deforming 3D characters. Stop-motion has also been applied to study low delity prototyping for user interfaces [Bonanni and Ishii 2009]. Digital removal of seams and hands from stop-motion images has been addressed by [Brostow and Essa 2001]. [Han et al. 2014] presented a tool to aid in the generation of motion blur between static frames to Fig. 2. Modern stop-motion films such as Laika’s ParaNorman amass re- show fast motions. However, the problem of generating replacement placement libraries of thousands of 3D printed pieces. libraries for the purpose of 3D printed stop motion animation has not been addressed before. solution provides an optimal replacement library to be 3D printed Animation Compression. Although not intentional, stop-motion and a per-animation-frame assignment of pieces from this library can be seen as a compression of high-framerate or continuous ani- to reconstruct the input animation faithfully. mation into a much smaller set of frames. In computer graphics and As is common with replacement libraries [Priebe 2011], we can especially computer game development, there have been many meth- amplify the expressive range of the deformable face/object by rst ods proposed for compressing animations: for deformable meshes segmenting it into multiple parts. A user species the approximate using principal component analysis [Alexa and Müller 2000; Sattler location of the parts, and we calculate boundaries that have minimal et al. 2005; Vasa et al. 2014], for articulated characters using skeletal or zero deformation between them. An optimal replacement library skinning subspaces [James and Twigg 2005; Le and Deng 2014], is then computed independently for each part, and the object assem- or by analyzing patterns in motion capture data [Gu et al. 2009]. bled by interchangeably combining pieces from each part’s library. These methods for digital animation are free to dene interpolation The replacement library pieces also need to be instrumented with operations, eectively approximating the input animation with a connectors before 3D printing, so that repeated object re-assembly continuous (albeit high dimensional) function space. Stop motion in for stop-motion, is quick and sturdy. contrast requires a discrete selection: a 3D-printed face is either used We propose a series of algorithms to assist in the process of cre- for this frame or not. We cast this problem of extracting a printed ating a library of mix-and-matchable printable pieces and a set library of shapes and assigning those shapes to each frame of the of assembly instructions to recreate a given mesh-animation se- animation as one of sparse dictionary learning or graph clustering, quence. In particular, we introduce: a novel mesh segmentation well studied topics often used in computer graphics. In particular, method to nd near-stationary part boundaries, a deformation pro- Le & Deng [2013] use sparse dictionary learning to signicantly cess to homogenize part boundaries allowing temporal reshuing compress mesh animations, as a weighted combination of a few of segmented parts, and nally we simultaneously optimize for a basis meshes. While their weights are sparse, we must represent replacement library of printable pieces and their assignment to each every animated frame using a single physical replacement mesh, frame of an input animation. necessitating a very dierent optimization strategy. We evaluate our algorithm in Section 5 by showing compelling results, both digital and 3D printed, and a comparison to a naive ap- Stylization. Much of the work in stylizing characters pertains to proach to the above problem.