Paint-On-Glass Animation: the Fellowship of Digital Paint and Artisanal Control Tom Van Laerhoven, Fabian Di Fiore, William Van Haevre, Frank Van Reeth

Home , Gouache

Paint-on-Glass Animation: The Fellowship of Digital Paint and Artisanal Control Tom van Laerhoven, Fabian Di Fiore, William van Haevre, Frank van Reeth

To cite this version:

Tom van Laerhoven, Fabian Di Fiore, William van Haevre, Frank van Reeth. Paint-on-Glass Ani- mation: The Fellowship of Digital Paint and Artisanal Control. Computer Animation and Virtual Worlds, Wiley, 2011, 22 (2-3), pp.325. 10.1002/cav.406. hal-00629936

HAL Id: hal-00629936 https://hal.archives-ouvertes.fr/hal-00629936 Submitted on 7 Oct 2011

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Computer Animation and Virtual Worlds

Paint-on-Glass Animation: The Fellowship of Digital Paint and Artisanal Control

Journal: Computer Animation and Virtual Worlds

Manuscript ID: CAVW-11-0018

Wiley - Manuscript type: Special Issue Paper

Date Submitted by the 04-Mar-2011 Author:

Complete List of Authors: Van Laerhoven, Tom; Hasselt University, Expertise Centre for Digital Media Di Fiore, Fabian; Hasselt University, Expertise Centre for Digital Media Van Haevre, William; Hasselt University, Expertise Centre for Digital Media Van Reeth, Frank; Hasselt University, Expertise Centre for Digital Media

animation, 2D animation, painterly animation, paint system, paint- Keywords: on-glass

Note: The following files were submitted by the author for peer review, but cannot be converted to PDF. You must view these files (e.g. movies) online.

CASA2011_#77_figures.zip

http://mc.manuscriptcentral.com/cavw - For Peer Review Page 1 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 9 10 11 12 13 14 Paint-on-Glass Animation 15 16 17 18 The Fellowship of Digital Paint and Artisanal 19 20 21 22 23 Control 24 25 26 27 Tom Van Laerhoven Fabian Di Fiore William Van Haevre 28 29 30 Frank Van Reeth 31 32 33 Hasselt University – tUL – IBBT 34 35 36 Expertise Centre for Digital Media 37 38 39 40 Wetenschapspark, 2 41 42 43 BE-3590 Diepenbeek, Belgium 44 45 46 {tom.vanlaerhoven, fabian.diﬁore, william.vanhaevre, frank.vanreeth}@uhasselt.be 47 48 49 http://www.edm.uhasselt.be 50 51 52 53 Abstract 54 55 56 In this paper we deal with paint-on-glass animation, which is a technique for mak- 57 58 59 60 1 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 2 of 25

1 2 3 4 5 6 7 8 ing animated films by pushing slow-drying paints from frame to frame directly under 9 10 the camera. As artwork is continuously destroyed upon creating new frames, the ani- 11 12 mator is not able to rehearse or refine the animation afterwards. Furthermore, due to 13 14 impracticable issues like how to stack up layers containing wet paint or how to overlay 15 16 17 masks on the wet medium, one has to take both creative as technical decisions for each 18 19 shot. 20 21 Our approach consists of an interactive paint setup that physically simulates paint 22 23 24 media. Together with a set of digital tools the artist is relieved from the difficult task 25 26 of sustaining a constant frame-to-frame coherence while animating and is given the 27 28 possibility to modify or undo earlier paint modifications. Regarding the interaction part, 29 30 31 the artist stays in full control by employing a tangible interface. This allows artists to 32 33 push pigment with their fingers, use special-purpose digital brushes, or even to employ 34 35 real life tools like paper tissue. 36 37 38 Feedback from artists confirms the real-life behaviour of stretching and smudging 39 40 the paint as well as interacting with the setup, resulting in a natural and reality-based 41 42 methodology. 43 44 45 46 Keywords: animation, 2D animation, painterly animation, paint system, paint-on-glass 47 48 49 50 51 52 53 54 55 56 57 58 59 60 2 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 3 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 Introduction 9 10 11 In spite of its name, paint-on-glass animation is best defined as a technique for making an- 12 13 14 imated films by pushing around some kind of (wet) medium directly under the camera and 15 16 recording it frame-by-frame (Figure 1). It is categorised as an “under-the-camera-technique” 17 18 as it is nearly always undertaken by an individual artist or animator rather than by a produc- 19 20 21 tion studio using factory-like processes. As the medium is pushed around directly under the 22 23 camera and recorded frame-by-frame, each image seems to merge from the previous one and 24 25 26 melt into the next, resulting in movement that can be very fluid and organic – a continual 27 28 process of metamorphosis. 29 30 31 A well-known practitioner of the paint-on-glass animation technique is Russian animator 32 33 Aleksander Petrov. His most famous work is the award winning short film “The Old Man 34 35 36 and the Sea” which took two and a half years to complete [1] illustrating the complexity of 37 38 the technique. 39 40 41 42 Motivation. As the medium is pushed around from frame to frame, artwork is contin- 43 44 45 uously destroyed as new artwork is created. Without the ability to rehearse and refine the 46 47 animation and without the disposal of layering and masking tools, the animator must plough 48 49 50 on regardless incorporating any errors into the sequence. 51 52 On the one hand this stimulates spontaneity and delivers work that is very fresh and 53 54 55 distinctive as it celebrates the method of its making. For example, characters may move from 56 57 place to place not by walking, but by being smudged away to re-form out of the background 58 59 60 3 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 4 of 25

1 2 3 4 5 6 7 8 at the required location. On the other hand, this particular form of step-ahead animation 9 10 is a painstaking process which demands much self-discipline, endurance and concentration 11 12 as the resulting frames are never a series of clean images of the kind produced by other 13 14 15 animation techniques, but also contain a record of their making due to the pushing process. 16 17 18 19 Contribution. In this paper we present our approach to facilitate paint-on-glass anima- 20 21 tion. In close collaboration with artists and animators we developed a set of digital tools 22 23 24 that do enable them to achieve the fine quality of a finished painting for each frame, while 25 26 preserving all artisan control. 27 28 29 Primarily, our system features: 30 31 32 ∙ a real-time free-form paint tool to physically simulate the complex behaviour of dif- 33 34 35 ferent paint media (gouache, watercolour, impasto, pastel); 36 37 38 ∙ painting/animating by pushing the drawing media around resulting in a stretch and 39 40 smear effect; 41 42 43 44 ∙ a tangible interface consisting of a multitouch screen allowing the use of one’s fingers 45 46 as well as physical media including brushes and tissues; 47 48 49 50 ∙ several tools that are absent in the traditional method due to the wet nature of the 51 52 painting media such as working with layers and masks; 53 54 55 ∙ the availability of animation tools like onion skinning and flipbook animation; 56 57 58 59 60 4 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 5 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 ∙ digital tools to control the drying time, correct mistakes, remove pigment and/or water 9 10 as desired, and the ability to save, load or reuse (intermediate) results. 11 12 13 14 Paper Organisation. First, we survey work we consider related to ours. Next, we describe 15 16 our approach starting from the employed painting setup to an overview of the digital paint- 17 18 19 on-glass animation process. Finally, clarifying examples are provided in which we also 20 21 discuss our results. 22 23 24 25 26 27 Related Work 28 29 30 We start this section by giving a brief overview of traditional paint-on-glass animation. 31 32 33 Then, we elaborate on digital paint solutions that we consider related to ours. 34 35 36 37 Traditional Paint-on-Glass Animation 38 39 40 41 What we give here is an account as it might be for a practiced artist in a home studio. 42 43 We separately discuss the painting process (i.e. painting a single frame) and the animation 44 45 process (i.e. making a series of drawings). 46 47 48 49 Painting. The painting process itself resembles painting on a canvas as the initial paint is 50 51 52 added with brushes or other tools including sponges, cotton buds, small sticks, strong tissue 53 54 or even ﬁngers. 55 56 57 58 59 60 5 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 6 of 25

1 2 3 4 5 6 7 8 As the paint medium will be reused from frame to frame, the artist is constrained to work 9 10 with slow drying paint media like oil paint in order to keep working with the medium across 11 12 several hours or even days. This causes also many impracticable issues like how to stack up 13 14 15 layers of wet paint or how to overlay masks on the wet medium. 16 17 18 19 Animating. For classic 2D animation the animator draws or sets up objects one frame 20 21 at a time in sequential order until the sequence is complete, also referred to as step-ahead 22 23 24 animation. In this way there is one physical drawing or image per frame. In the case 25 26 of paint-on-glass animation the artist is working directly under-the-camera: the artist has to 27 28 29 record the current frame before proceeding to the next one as current artwork is continuously 30 31 destroyed for creating each new frame. When a sequence has been filmed, there is no raw 32 33 34 material left besides the final film. So, while animating one cannot rely on techniques like 35 36 onion skinning or flipbook animation as previous artwork is not available anymore. Hence, 37 38 39 there is also no artwork to turn to when modifying the film afterwards. 40 41 42 43 Digital Paint Systems 44 45 46 In this section we will discuss digital paint solutions that we consider related to ours. 47 48 49 50 Painting. Looking at the majority of published paint systems [2, 3, 4, 5], virtual brushes 51 52 53 are by default operated by some kind of digital stylus in combination with a digital tablet. 54 55 Although popular and basically intuitive, this way of working is nothing more than an ap- 56 57 58 59 60 6 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 7 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 proximation of ‘the real thing’. Issues such as parallax, the lack of tufts and absence of 9 10 physical tools need to be solved first. 11 12 Other paint systems explicitly target novel interfaces to bridge the gap between physical 13 14 15 and digital painting. Vandoren et al. developed IntuPaint, which is a tangible interface for a 16 17 digital paint easel, using an interactive surface and electronic brushes with a tuft of bristles 18 19 20 [6]. This approach provides natural interaction and enables detailed tracking of specific 21 22 brush strokes. Additional tangible and finger-based input techniques allow for specific paint 23 24 25 operations or effects. Okaichi et al. propose an interactive oil paint system using a painting 26 27 knife [7] that is controlled using a haptic feedback device. Microsoft Surface [8] is based on 28 29 30 the IR image capture of an IR illuminated diffuser screen underneath a transparent surface 31 32 allowing natural finger-based interaction. Presence of real objects (e.g., brushes) can be 33 34 35 detected, but only a blurred image of the brush is generated. Recently, Vandoren et al. 36 37 presented a novel interactive canvas FluidPaint allowing the use of real wet brushes [9]. 38 39 40 41 Animating. Regarding animation, only small initiatives have been taken so far. Van Laer- 42 43 44 hoven et al. introduce the concept of brushes and pigments enhanced with behavioural 45 46 intelligence that allow the user to enrich and animate interactively created images by semi- 47 48 49 autonomously embedding procedural animation into them [10]. Some commercial digital 50 51 art applications also claim to target at creating animations ([11, 12, 13]), however, the ani- 52 53 54 mation part mainly is limited to the availability of a keyframe editor and the use of animated 55 56 brushes to draw frames along a path. 57 58 59 60 7 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 8 of 25

1 2 3 4 5 6 7 8 Approach 9 10 11 In this section we give an overview of the main parts of our approach. We first describe the 12 13 14 employed digital paint setup. Next, similar to the traditional process of previous section we 15 16 will highlight both the painting and animation process. 17 18 19 20 21 Digital Paint Setup 22 23 24 For the simulation part, we make use of an interactive paint system that adopts physically- 25 26 based algorithms to simulate the complex interaction of a layered virtual canvas with dif- 27 28 29 ferent paint media, including watercolour, gouache and impasto, but also dry media like 30 31 pencil or pastel [14]. This paint system encapsulates the drying process, paint diffusion as 32 33 34 well as different pigment attributes (Figure 2(a)). Furthermore, it relies on Navier-Stokes 35 36 for defining the velocities of the fluid body and on the specialised Kubelka-Munk colour 37 38 39 mixing scheme for rendering paint layers. These features are vital elements to obtain the 40 41 much sought after ‘natural look’ in digital artwork. Every step of the simulation makes full 42 43 44 use of programmable graphics hardware to achieve interactive simulation rates. 45 46 Another important component of the system relates to its virtual brush design that relies 47 48 49 on constrained energy optimisation and on an anisotropic friction model to deform the brush 50 51 tuft and adequately translate the user’s input (e.g., originating from a tablet interface) into 52 53 realistic and predictable strokes. This design also allows for bidirectional pigment transfer 54 55 56 between the canvas and the selected drawing tool. 57 58 59 60 8 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 9 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 Regarding the interaction part, we chose to use a tangible interface to enable users to 9 10 intuitively manipulate the drawing media (Figure 2(b)). The setup consists of an interactive 11 12 multitouch surface based on the principle of frustrated total internal reflection (FTIR) [6]. 13 14 15 The contact surface of each object on the paint table is tracked by a 0.3 megapixel video 16 17 camera, equipped with an infrared band-pass filter, after which the footprint of the objects 18 19 20 touching the surface is transferred to the paint simulation for further processing and render- 21 22 ing. In contrast to the original paint table, our setup is able to track multiple objects on the 23 24 25 surface simultaneously. This way, artists can push pigment with multiple fingers at once, 26 27 use special-purpose digital brushes with tufts, or even employ paper tissue. 28 29 30 A major advantage of this setup is that it provides coinciding action and perception 31 32 spaces, resulting in a natural, reality-based methodology. In addition, it encourages multiple 33 34 35 users to work collaboratively on the same artwork. 36 37 38 39 Painting 40 41 42 43 Starting from a blank canvas, the artist creates or alters an image by adding some kind of 44 45 paint medium (e.g., gouache, impasto, watercolour, pastel) using fingers or several available 46 47 tools (brushes, pencils, spatulas, sponges, fingers). A background image can be depicted 48 49 50 as a reference and parts of the drawing canvas can interactively be masked to prevent paint 51 52 from reaching or settling down on the masked parts. Drawings can also be separated into 53 54 55 layers allowing for robust manipulations of the frame content such as depth ordering and hid- 56 57 58 59 60 9 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 10 of 25

1 2 3 4 5 6 7 8 ing/revealing objects. Furthermore, users can interactively control aspects like drying time 9 10 (including the possibility to keep paint permanent wet or pause the drying), undo mistakes, 11 12 remove pigment and water as desired and have the ability to save, load or reuse intermediate 13 14 15 results. 16 17 Watercolour images made with the system contain the typical effects that can be recog- 18 19 20 nised in images produced with real thin paint, like the dark-edge effect, watercolour glazing, 21 22 wet-on-wet painting and the use of different pigment types. 23 24 25 26 27 Animating 28 29 30 While animating, each subsequent frame in the animation is created by pushing the drawing 31 32 33 medium of the artwork in the current frame towards its desired position for the next frame. 34 35 During interaction, movement on the multitouch surface is converted to a vector field based 36 37 38 on the current velocity of the brush/finger (see Figure 3(a)), which is then applied to the paint 39 40 (both pigment and water) by means of an advection step −(⃗v ⋅ ∇)p, where ⃗v denotes the 2D 41 42 43 velocity vector field and p the amount of paint at a certain position on the canvas. As in our 44 45 paint system mass is contained in the grid cells of the digital canvas, the advection scheme 46 47 boils down to individual cells exchanging content with their neighbours (Figure 3(b))[14]. 48 49 50 For each cell, we measure the volume of paint that is exchanged with all neighbouring cells 51 52 as follows. 53 54 55 When calculating the amount of mass that flows from the centre cell to its right neigh- 56 57 58 59 60 10 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 11 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 bour with velocities ⃗vcenter and ⃗vright respectively, as shown in Figure 3(b) (left), any point 9 10 along this border has velocity: 11 12 13 14 ⃗v + ⃗v ⃗v = center right (1) 15 average 2 16 17 18 19 20 Within a given time step Δt, a distance Δx = (⃗vaverage)xΔt is travelled. In case we 21 22 are calculating the movement of water, the area covered by the volume of displaced water is 23 24 25 given by the coloured area in Figure 3(b) (left), and equals Δx(cellheight). The total volume 26 27 of displaced water then is ΔV = Δx(cellheight)w (Figure 3(b) (right)). The change in 28 i,j 29 30 water quantity is given by the following equation: 31 32 33 34 ΔV Δw = (2) 35 (cellwidth × cellheight) 36 37 38 39 40 This procedure is repeated to calculate the exchanged water quantities with the three 41 42 remaining neighbours. We add up the results and divide it by four, because each neighbour 43 44 45 contributes exactly one quarter to the total ﬂux. The same approach can be followed to work 46 47 out the movement of pigment. 48 49 50 Figure 4 depicts some stills of a user pushing paint (in this case impasto) with two ﬁngers 51 52 on an interactive multitouch surface. 53 54 55 In order to assure more control over the way the artist is obliged to make things move, 56 57 he/she can rely on digital animation tools to make manipulative operations more predictable 58 59 60 11 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 12 of 25

1 2 3 4 5 6 7 8 including a keyframe editor, onion skinning and a flipbook tool (Figure 5). The onion skin- 9 10 ning tool aids the artist to achieve good frame-to-frame coherence, resulting in smoother 11 12 frame transitions by showing trace layers of previously painted frames whereas the digital 13 14 15 flipbook tool provides immediate feedback on the status of the animation being created. 16 17 18 19 20 Results 21 22 23 24 Figure 7 depicts some snapshots of a paint-on-glass animation of a bird created with impasto 25 26 paint illustrating the stretch and smear effects. Even for the start frame, the artist chose to 27 28 29 gradually refine a basic image (painted using a brush) using his fingers only (see Figure 30 31 6). The entire animation consists of 31 frames of which each is a modified version of its 32 33 34 predecessor by only pushing paint from frame to frame. 35 36 37 38 Discussion. The target audience of our setup is skilled artists as the paint-on-glass tech- 39 40 nique itself requires some experience in the field of painting. The platform itself is targeted 41 42 43 at both traditional and digital artists as the interface is similar to the traditional environment. 44 45 Therefore traditional as well as digital painters were involved in an informal user test 46 47 48 evaluating the setup as well as creating the examples. Only basic instructions were given 49 50 (i.e. explaining the user interface). After that, the test participants were allowed to work on 51 52 53 their subjects of choice without any training. 54 55 All participants worked between two and three hours to complete their animations (43 56 57 58 59 60 12 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 13 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 frames on average) which is significantly faster than the traditional way of working (i.e. 9 10 around 20 frames a day, on average). Although the test session was rather short we can say 11 12 that all participants were very positive. They found the user interface familiar and praised the 13 14 15 real-life behaviour of manipulating the paint as well as interacting with the setup, resulting 16 17 in a natural and reality-based methodology. 18 19 20 21 22 23 Conclusions 24 25 26 In this paper we dealt with making animated films by pushing around wet media directly 27 28 29 under the camera, known as paint-on-glass animation. Without the ability to rehearse and 30 31 refine the animation as opposed to classic 2D animation, this particular form of animation 32 33 34 is a painstaking process for which animators need to persevere regardless incorporating any 35 36 errors or unwanted smudging into the sequence. 37 38 39 Our approach tackles the issues inherent to paint-on-glass animation by offering the 40 41 animator an interactive paint system that adopts physically-based algorithms to simulate the 42 43 44 complex interaction of a virtual canvas with different paint media, and a set of digital tools 45 46 that do enable him/her to achieve the fine quality of a finished painting for each frame while 47 48 49 preserving all artisan control. Regarding the interaction part, a tangible interface was chosen 50 51 to enable users to intuitively manipulate the drawing media using their own hands. 52 53 54 55 56 57 58 59 60 13 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 14 of 25

1 2 3 4 5 6 7 8 Future Work. The emphasis in this paper is on paint-on-glass animation. However, the 9 10 same approach could be applied to any “under-the-camera-technique” such as animating 11 12 using fine sand, sugar or coffee grind. For the simulation part this requires a bidirectional 13 14 15 transfer of particles; particles need to be deposited or scraped off again. Regarding the 16 17 interaction process we have to consider how to sprinkle dry media. 18 19 20 Furthermore, we strongly believe in investigating new interaction techniques which are 21 22 not present at all in traditional animation such as using simple hand gestures and finger taps 23 24 25 to easily navigate between frames or assets. Inspired by ShapeTouch [15] another option is 26 27 to exploit the shape of hands to work on the interactive surface just like when dealing with 28 29 30 real physical objects (e.g., pinning, peeling or flicking). 31 32 33 34 35 Acknowledgements 36 37 38 39 We gratefully express our gratitude to the European Fund for Regional Development and the 40 41 Flemish Government, and to Xemi Morales and Karel Robert for their artistic contribution. 42 43 44 45 46 47 References 48 49 50 [1] Aleksandr Petrov. World Wide Web, http://en.wikipedia.org/wiki/Aleksandr_Petrov_(animator). 51 52 53 [2] Suguru Saito and Masayuki Nakajima. 3D physics-based brush model for painting. In 54 55 56 Proceedings of SIGGRAPH Conference abstracts and applications, page 226. ACM, 57 58 59 60 14 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 15 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 1999. 9 10 11 [3] W. V. Baxter, V. Scheib, M. C. Lin, and D. Manocha. DAB: Interactive haptic painting 12 13 with 3D virtual brushes. In Proceedings of SIGGRAPH, pages 461–468. ACM, 2001. 14 15 16 17 [4] Nelson Chu and Chiew-Lan Tai. Real-time painting with an expressive virtual Chinese 18 19 brush. IEEE Computer Graphics and Applications, 24(5):76–85, 2004. 20 21 22 [5] Tom Van Laerhoven and Frank Van Reeth. Brush up your painting skills: Realistic 23 24 25 brush design for interactive painting applications. The Visual Computer. Computer 26 27 Graphics International (CGI2007), 23:763–771, September 2007. 28 29 30 31 [6] Peter Vandoren, Tom Van Laerhoven, Luc Claesen, Johannes Taelman, Chris Ray- 32 33 maekers, and Frank Van Reeth. IntuPaint: Bridging the gap between physical and 34 35 36 digital painting. In Proceedings of TABLETOP 2008, IEEE International Workshop 37 38 on Horizontal Interactive Human Computer System, pages 71–78, October 2008. 39 40 41 42 [7] N. Okaichi, H. Johan, T. Imagire, and T. Nishita. A virtual painting knife. The Visual 43 44 Computer. Computer Graphics International (CGI2008), pages 753–763, 2008. 45 46 47 [8] Microsoft. Microsoft surface. World Wide Web, 48 49 50 http://www.microsoft.com/surface/. 51 52 53 [9] Peter Vandoren, Luc Claesen, Tom Van Laerhoven, Johannes Taelman, Chris Ray- 54 55 56 maekers, Eddy Flerackers, and Frank Van Reeth. FluidPaint: an interactive digital 57 58 59 60 15 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 16 of 25

1 2 3 4 5 6 7 8 painting system using real wet brushes. In Proceedings of ITS 2009, ACM ACM Inter- 9 10 national Conference on Interactive Tabletops and Surfaces, pages 53–56, November 11 12 2009. 13 14 15 16 [10] Tom Van Laerhoven, Fabian Di Fiore, and Frank Van Reeth. Hand-painted animation 17 18 with intelligent brushes. Computer Animation and Virtual Worlds. Computer Anima- 19 20 21 tion and Social Agents (CASA2008), pages 365–374, 2008. 22 23 24 [11] Corel Painter X. World Wide Web, http://www.corel.com/. 25 26 27 [12] Project Dogwafﬂe. World Wide Web, http://www.thebest3d.com/dogwaffle/. 28 29 30 31 [13] Adobe PhotoShop. World Wide Web, http://www.photoshop.com/. 32 33 34 [14] Tom Van Laerhoven and Frank Van Reeth. Real-time simulation of watery paint. In 35 36 37 Journal of Computer Animation and Virtual Worlds (Special Issue CASA 2005), vol- 38 39 ume 16:3–4, pages 429–439. J. Wiley & Sons, Ltd., October 2005. 40 41 42 [15] Xiang Cao, Andrew Wilson, Ravin Balakrishnan, Ken Hinckley, and Scott Hudson. 43 44 45 ShapeTouch: Leveraging contact shape on interactive surfaces. In Proceedings of 46 47 TABLETOP 2008, IEEE International Workshop on Horizontal Interactive Human 48 49 50 Computer System, pages 139–146, October 2008. 51 52 53 54 55 56 57 58 59 60 16 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 17 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 dr. Tom Van Laerhoven is a senior researcher in computer science at 9 10 Hasselt University (UHasselt) in Diepenbeek, Belgium. He obtained 11 12 a MS in computer science in 2000 at UHasselt. In 2006, he ﬁn- 13 14 ished his PhD entitled “An Extensible Simulation Framework Supporting 15 16 17 Physically-based Interactive Painting” at the Expertise Centre for Digi- 18 19 tal Media (EDM), a research institute of Hasselt University. His research 20 21 activities are concerned with computer animation, physically-based mod- 22 23 elling and animation, non-photorealistic rendering and parallel and dis- 24 25 26 tributed algorithms. 27 28 29 Prof. dr. Fabian Di Fiore is an associate professor in computer sci- 30 31 ence at Hasselt University (UHasselt) in Belgium. He obtained a MS 32 33 34 in computer science in 1997 at the University of Leuven. In 2004, he 35 36 ﬁnished his PhD entitled “A 2.5D Modelling and Animation Framework 37 38 Supporting Computer Assisted Traditional Animation” at the Expertise 39 40 41 Centre for Digital Media (EDM), a research institute of Hasselt Univer- 42 sity. His research activities are concerned with computer animation, non- 43 44 45 photorealistic rendering and animation, and highly stylised drawn anima- 46 47 tion. 48 49 50 51 52 53 54 55 56 57 58 59 60 17 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 18 of 25

1 2 3 4 5 6 7 8 dr. William Van Haevre is a senior researcher in computer science at 9 10 Hasselt University (UHasselt) in Diepenbeek, Belgium. He obtained 11 12 a MS in computer science in 2000 at UHasselt. In 2007, he ﬁnished 13 14 his PhD entitled “Realism in Environment Sensitive Plant Models and 15 16 17 their Animation” at the Expertise Centre for Digital Media (EDM), a re- 18 19 search institute of Hasselt University. His research interests include com- 20 21 puter graphics and animation, physically-based modelling and animation 22 23 of natural phenomena, video-based animation of plants and simulating 24 25 26 highly stylised drawing techniques. 27 28 29 Prof. dr. Frank Van Reeth is a full-time professor of Computer Science 30 31 at Hasselt University (UHasselt) in Belgium and Deputy Managing Di- 32 33 34 rector of the Expertise centre for Digital Media (EDM), a research insti- 35 36 tute of Hasselt University. He is co-founder of seven spin-off companies 37 38 and a member of the board of ANDROME NV. His research interests 39 40 41 include computer graphics, computer animation, virtual environments, 42 multimedia technology and telematics. He was project leader of several 43 44 45 industrial contract research projects, technical representative as well as 46 47 project manager in various EC research projects, and involved in several 48 49 IBBT projects. He acted as evaluator of national and European R&D 50 51 52 proposals involving the above research domains. He is a member of Eu- 53 54 rographics, ACM, VRS, IEEE and CGS. 55 56 57 58 59 60 18 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 19 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 Figure 1: An artist at work using an “under-the-camera-technique” with a combination of 40 41 sand and ink ⃝c Gerald Conn. 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 19 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 20 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 (a) (b) 20 21 22 23 Figure 2: a) Physically-based paint simulation. b) The setup includes an interactive surface 24 25 on which the user can manipulate paint with multiple ﬁngers at once, or using special- 26 27 28 purpose tools. 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 20 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 21 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 (a) (b) 19 20 Figure 3: a) Pushing paint results in a velocity ﬁeld that locally manipulates the exchange of 21 22 23 pigment quantities. b) Moving water to a right neighbouring cell. The dark areas represent 24 25 the volume of displaced water. 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 21 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 22 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Figure 4: Some stills depicting a user pushing paint (impasto) with two ﬁngers on an inter- 19 20 active multitouch surface. 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 22 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 23 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 Figure 5: When drawing a new frame the animator can check temporal coherence by ﬂipping 21 22 through all previous frames. 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 23 http://mc.manuscriptcentral.com/cavw - For Peer Review Computer Animation and Virtual Worlds Page 24 of 25

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 Figure 6: The making of an entire new frame (last image) from scratch by gradually refining 37 38 a basic image (first image) using fingers only. 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 24 http://mc.manuscriptcentral.com/cavw - For Peer Review Page 25 of 25 Computer Animation and Virtual Worlds

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 Figure 7: Some snapshots of a paint-on-glass animation of a bird created with impasto 54 55 paint. Each image is a modiﬁed version of its predecessor by only pushing paint from frame 56 57 58 to frame. 59 60 25 http://mc.manuscriptcentral.com/cavw - For Peer Review