Physically-Based Modeling Techniques for Interactive Digital Painting by William Valentine Baxter III A dissertation submitted to the faculty of the University of North Carolina at Chapel Hill in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Computer Science. Chapel Hill 2004 Approved by: Ming C. Lin, Advisor Dinesh Manocha, Reader Gary Bishop, Reader Anselmo Lastra, Committee Member Michael Minion, Committee Member ii iii Oc 2004 William Valentine Baxter III ALL RIGHTS RESERVED iv v ABSTRACT WILLIAM VALENTINE BAXTER III: Physically-Based Modeling Techniques for Interactive Digital Painting (Under the direction of Ming C. Lin) In this dissertation I present a novel, physically-based approach to digital painting. With the interactive simulation techniques I present, digital painters can work with digital brushes and paints whose behavior is similar to real ones. Using this physically- based approach, a digital painting system can provide artists with a versatile and expressive creative tool, while at the same time providing a more natural style of interaction enabled by the emulation of real-world implements. I introduce several specific modeling techniques for digital painting. First, I present a physically-based, 3D, deformable, virtual brush model based on non-linear quasi-static constrained energy minimization. The brush dynamics are computed using a skeletal physical model, which then determines the motion of a more complex geometric model. I also present three different models for capturing the dynamic behavior of viscous paint media, each offering a different trade-off between speed and fidelity—from 2D heuristics, to 3D partial differential equations. Accurate modeling of the optical behavior of paint mixtures and glazes is also important, and for this I present a real-time, physically- based rendering technique, based on the Kubelka-Munk equations and an eight-sample color space. Finally, I present techniques for modeling the haptic response of brushes in an artist’s hand, and demonstrate that all these techniques can be combined to provide the digital painter with an interactive, virtual painting system with a working style similar to real-world painting. vi vii To Papa Bear, a.k.a. William V. Baxter, Sr. viii ix ACKNOWLEDGMENTS It takes a tremendous amount of dedication and perseverance to complete a doctoral dissertation. At least that’s what I’ve been told. I’m not sure I possess either of those qualities in great enough quantity to complete a dissertation, but I am fortunate in that I have been constantly surrounded by many great and motivated people whose encouragement and assistance throughout this process has made up for any attributes I lack. I would like to sincerely extend my gratitude to those individuals here. I would first like to thank my dear friend Leo Chan for setting me off on this course. I got together with Leo sometime in the fall of 1997, and he mentioned to me that he was working on computer graphics. It was his compelling description of a field combining computer science, math, and physics, with elements of visual design and art that piqued my interest and led me to apply to graduate schools. Thanks to Mark Harris and Vincent Scheib, for good discussions and inspirations relating to research topics over the years, but also for inspiration to greater heights in every form of technical and non-technical communication. Thanks as well to Kenny Hoff for his infectious enthusiasm about graphics and many stimulating discussions on that and any number of other topics. I extend my humble appreciation to Greg Coombe, Jeff Feasel, and Karl Gyllstrom for letting me make noise with them, the one endeavor that may have done the most to preserve my sanity in the last couple of years. x Thanks to all the painters who used and tested my painting systems: Rebecca Holm- berg, John Holloway, Andrea Mantler, Haolong Ma, Sarah Hoff, my wife Eriko Baxter, Lauren Adams from the Art department, and all those who gave me valuable feedback after seeing the demo. An especially large thanks to painter John Holloway for his enthusiasm, encouragement, and belief in this project, and to Rebecca Holmberg for going far beyond the call of duty to create so many delightful paintings with dAb while at the same time finishing her own degree in chemistry. I wish I could do more, but all I have to offer is my sincere thanks and an honorary degree from the “Baxter school of digital painting.” I am very grateful for all the help and the late nights given up by collaborators on this and previous research: Avneesh Sud, Naga Govindaraju, Vincent Scheib, Yuanxin Liu, and Jeremy Wendt. And thanks to those who allowed me to collaborate with them on their research as I learned the ropes: Carl Erikson, and the late-90’s Walkthrough MMR all-star gang – Dan Aliaga, Rui Bastos, Jon Cohen, Dave Luebke, Andy Wilson, and Hansong Zhang. I would especially like to thank Vincent Scheib for his help on the dAb paint model, and Jeremy Wendt for his invaluable assistance in the paint measurement effort and with the Kubelka-Munk rendering code. Thanks to Dinesh Manocha for giving me the opportunity to work with him and the Walkthrough group for my first two and a half years of graduate school. Without his willingness to take a chance on me, a kid with no computer graphics experience, I would probably still be shoveling business logic code around today. Many thanks to my advisor, Ming Lin, whose firm belief in my abilities and belief in this topic kept me going. She is tough on her students, but her tireless promotion of her students’ interests behind the scenes does not go unnoticed. Thanks to Dr. Lin and Dr. Manocha’s support, I have always been free to pursue the research ideas I wanted to pursue throughout my time here. xi Thanks also to everyone who read this admittedly long dissertation, and for their invaluable feedback. I am especially grateful to Ming Lin for working with me through- out the writing process, and to my outside readers, Gary Bishop and Dinesh Manocha. Also thanks to my other committee members, Anselmo Lastra and Michael Minion, who also took the time to read large portions and provide constructive feedback. Their efforts greatly improved this dissertation. I would like to express my appreciation also to the the agencies and foundations that have provided support for my work. First to the Link Foundation and NVIDIA for fellowships in my last two years, and to the agencies who have sponsored various research in the GAMMA group: Intel Corporation, the National Science Foundation, the Office of Naval Research, and the U.S. Army Research Office. Also, a big thanks to my parents, Suzanne and Bill, Jr., for their love and support from the beginning and especially during this time. Three days before my defense I began to find out just how much I really owe them when I became a father myself. Thanks to my sister Amy for breaking ground by becoming the first (but now not the only!) doctor in the family and for always inspiring me to try harder throughout my life. Finally, I would like to thank to my wife, Eriko, for her tireless dedication, for the sacrifices she has made, for always staying by my side, for setting a shining example with her seemingly bottomless persistence, for her unwavering belief in the value of seeing the dissertation through to the end, and for all the encouragement she provided in the times I felt hopeless. Thank you, Elly-chan. xii xiii CONTENTS LIST OF FIGURES xxi LIST OF TABLES xxv 1 Introduction 1 1.1 Painting and Computers . 2 1.2 Painting Media . 9 1.3 Applications and Benefits . 12 1.4 Thesis . 14 1.5 New Results . 16 1.5.1 Overview of Results . 16 1.5.2 Brush Dynamics . 18 1.5.3 Paint Dynamics . 19 1.5.4 Paint Appearance . 22 1.5.5 Interface . 23 1.5.6 Summary . 24 1.6 Thesis Organization . 25 2 Previous Work 26 2.1 Non-Photorealistic Rendering Overview . 27 xiv 2.2 Automatic Rendering Techniques . 28 2.3 Modeling Natural Media . 30 2.4 Painting Interfaces . 36 3 Brush Modeling 39 3.1 Previous work . 42 3.2 Introduction to Brushes . 44 3.3 Overview of Modeling Approach . 46 3.4 Brush Dynamics . 47 3.4.1 Virtual Work and Optimization . 49 3.4.2 Spine Kinematics . 50 3.4.3 Spring Energy . 52 3.4.4 Friction Energy . 52 3.4.5 Damping and Plasticity . 56 3.4.6 Derivatives . 56 3.4.7 Constraints . 57 3.5 Geometric Modeling of Brushes . 57 3.5.1 Subdivision Surface . 57 3.5.2 Bristle Strips . 58 3.6 Implementation and Results . 62 3.7 Limitations . 65 3.8 Summary . 65 4 dAb Paint: A Simple Two-layer Paint Model 67 4.1 dAb Features . 68 4.2 Mathematical Description of Contact Footprints . 69 4.3 Paint Stroking Algorithms . 72 xv 4.4 Paint Representation . 74 4.5 Details of the Paint Stroking Algorithms . 77 4.5.1 Generating Footprints on the Canvas . 77 4.5.2 Bi-directional Paint Transfer and Blending . 80 4.5.3 Updating Brush Textures . 82 4.6 Rendering . 83 4.6.1 Optical Composition . 83 4.6.2 Embossing . 84 4.7 Drying the Canvas . 85 4.8 Relative Height Field . 86 4.9 The Palette . 87 4.10 Implementation and Results . 88 4.11 Limitations . 89 4.12 Summary . 91 5 Stokes Paint: 3D Volumetric Paint 92 5.1 Previous Work in Fluid Simulation .