Artistic Style Characterization of Vincent Van Gogh's Paintings Using
Total Page:16
File Type:pdf, Size:1020Kb
Artistic Style Characterization of Vincent Van Gogh’s Paintings using Extracted Features from Visible Brush Strokes Tieta Putri, Ramakrishnan Mukundan and Kourosh Neshatian Department of Computer Science, University of Canterbury, Christchurch, New Zealand [email protected], mukundan, kourosh.neshatian @canterbury.ac.nz { } Keywords: Stylometry, Style Characterization, Feature Extraction, Painting Classification. Abstract: This paper outlines important methods used for brush stroke region extraction for quantifying artistic style of Vincent Van Gogh’s paintings. After performing the region extraction, stroke-related features such as colour and texture features are extracted from the visible brush stroke regions. We then test the features by performing a binary classification between painters from different art movements and painters from the same art movement. 1 INTRODUCTION tics has motivated considerable research into brush stroke analysis, which can be done mathematically Identifying artistic styles in digital paintings have and statistically with the aid of stylometry. The aim been of great interest for researchers in the field of of stylometry is to quantify artistic styles with a se- Computer Vision. It has many applications such as for ries of extracted features from the digitized artworks cultural heritage preservation (Putri and Arymurthy, (Hughes et al., 2010). An image will represented as 2010), differentiating art movement period (Johnson. a string of features of statistical, texture, colour or et al., 2008), building a style-based image retrieval shape, which will be analyzed using machine learning system (Lombardi et al., 2004) and forgery detection techniques. In this paper, we use stroke-based sty- (Rosseau, 1968). There are many factors that de- lometry as we characterize various paintings by exa- termine an artistic style. Such factors are the brush mining the statistical properties of the visible brush stroke characteristics and colour palette used by the strokes. artist and the way objects are drawn. From those fac- This work presents important image processing tors, brush stroke characteristics contribute the most methods for extracting visible brush strokes from a set to an artistic style (Zang et al., 2013). For instance, V of digital paintings by Vincent Van Gogh. Using ex- Pointillist-style consists of small, elliptical and re- tracted brush strokes, we describe feature extraction peated brush strokes that are put together in such way methods based on their texture and shape. The ex- that it will form the object when a viewer looks at it tracted features can then be compiled into a feature set from a certain distance (see Fig. 1). S which serves as the quantified brush strokes proper- The existence of many painting styles with each of ties. Since brush strokes appearances are closely re- them having several unique brush stroke characteris- lated to the artistic style of the painting itself (Strass- man, 1986), S can be seen as the style representation of V. This paper is organized as follows: In Section 2, we describe some related work in artistic style charac- terization and brush stroke extraction. Section 3 gives a detailed description of the datasets and methods used in our work. Then, in Section 4, we provide some results and discussions. Finally, Section 5 con- cludes this paper and outlines our future research di- rections. Figure 1: Example of computer-generated Pointillist ren- dering (Putri, 2012). 378 Putri, T., Mukundan, R. and Neshatian, K. Artistic Style Characterization of Vincent Van Gogh’s Paintings using Extracted Features from Visible Brush Strokes. DOI: 10.5220/0006188303780385 In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods (ICPRAM 2017), pages 378-385 ISBN: 978-989-758-222-6 Copyright c 2017 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved Artistic Style Characterization of Vincent Van Gogh’s Paintings using Extracted Features from Visible Brush Strokes 2 RELATED WORKS 2.2 Brush Stroke Extraction 2.1 Artistic Style Characterization Brush strokes are the medium used by painters to communicate what they want to convey in their pain- In a stylistic painterly non-photorealistic (NPR) sys- tings. The way they are drawn can also provide some tem, the characterization of artistic style is necessary information related to the painter, for instance the for capturing, representing, and remapping a parti- painter’s art movement and his/her emotional state cular artistic style to an input image. Every digi- (Callen, 1982). Because of this, brush stroke extrac- tized paintings can be seen as a composition of two tion has an important role in the area of digital pain- components: the style and the content (Gatys et al., ting analysis since brush strokes contain a lot of in- 2015). Artistic style characterization process extracts formation that can be used as features to represent a the style component of digitized paintings as a set of painting. features. The features are then used by the NPR sys- Li et al. (2012) described a brush stroke extraction tem as a heuristic in the painterly rendering process. method for distinguishing Van Gogh’s paintings from Research done by Hughes et al. (2010) investi- his contemporaries. Their method was used for dis- gated the characterization of artworks done by the tinguishing Van Gogh’s paintings from two different Flemish painter Pieter Bruegel the Elder using sparse periods, which are Paris and Arles-St.Remy period. coding analysis. The aim of the research was to dis- Their work consists of developing statistical frame- tinguish the authentic Bruegel paintings from the im- work for the assessment of the distinction level of itations by determining their similarity of the sparse different painting categories, brush stroke extraction model. The sparse model attempts to describe the algorithm, and numerical features for brush stroke image space by training a set of orthogonal basis func- characterization. They used the EDISON edge de- tions that effectively represent the space. Sparse co- tection algorithm developed by Meer and Georgescu ding is proven to be an effective method for feature (2001). After edges are detected, edge linking algo- modelling in drawings and in other two-dimensional rithm and enclosing operation are performed in or- media due to the sparseness of the artworks’ statistical der to close the gaps between edge segments. Then, structures that are considered to give a high contribu- the processed edges are extracted using the connected tion to the perception of similarity. component labelling. Finally, brush stroke condi- Sener et al. (2012) extracted various features for tions are defined as: the brush skeleton not severely identifying children’s book illustrators. From illustra- branched; the ratio of broadness to length is within tion samples by authors Alex Scheffler, Debi Gliori, the range of [0.05, 1.0]; and the ratio of the brush size Dr. Seuss and Korky Paul, features such as 4x4x4 to two times length times width span is within [0.5, bin RGB histograms, gist (Oliva and Torralba, 2001), 2.0]. The brush skeleton is produced by the thinning colour dense SIFT (Lowe, 2004) and gradient his- operation of the extracted connected components. tograms are extracted. Support Vector Machine with Johnson. et al. (2008) did a mathematical analy- various kernels are then used for classification. From sis for the classification of Van Gogh paintings. They their experimentation, it was found that these features examined high resolution grayscale scans of 101 pain- are useful for distinguishing one artist’s style from tings, which consist of: 82 paintings by Van Gogh, 6 another. paintings by other painters and 13 others which are The extension of the work of Sener et al. (2012) loosely classified to be Van Gogh or non-Van Gogh by Vieira et al. (2015) uses a set of 93 different fea- by art experts. In their research, they combined two tures extracted from various digital paintings by 12 kinds of features that are extracted from the paintings, artists. Among those features are image energy and which are texture-based feature obtained by wavelets entropy along with their statistical properties. Rele- and stroke-based geometric features obtained by edge vant features were selected by measuring the cluster detection. They argue that it is extremely challenging dispersion using scatter matrices. Image energy and to locate strokes accurately from grayscale images in entropy are proven to be more representative of style a fully automated manner. than any other colour-based features. This research Berezhnoy et al. (2009) elaborated a method successfully identifies the correlation between several called as prevailing orientation extraction technique Baroque painters based on their works. (POET). This method focuses on brush stroke tex- ture orientation extraction for segmenting individual brush strokes in Van Gogh’s painting. The method consists of two stages: the filtering stage and the orientation extraction stage. In the filtering stage, a 379 ICPRAM 2017 - 6th International Conference on Pattern Recognition Applications and Methods rotation invariant circular filter with good response Table 1: Notations for Eq. (1), (2), and (3). for band-passing is applied. The orientation extrac- BR Set of pixels with a radius of R referred as a blob tion stage extracted the principal orientation of brush SR Subset of BR strokes from the filtered images. The filtered images P0 Pixel location of the centre of BR were transformed into binary images using multilevel P I Pixel in an image I ∈ thresholding before the orientations were extracted. v(P) Colour component of P x y Distance value between two object x and y The evaluation of POET is based on the cross com- k − k parison between the judgments of POET and human ∆E Error threshold for colour comparison subjects. #A Number of elements in set A 3.1.2 Texture Boundary Detection 3 CHARACTERIZING ARTISTIC This method detects brush strokes by identifying dif- STYLE ferent textures in the painting image.