Spatiogram Features to Characterize Pearls in Paintings
Total Page:16
File Type:pdf, Size:1020Kb
SPATIOGRAM FEATURES TO CHARACTERIZE PEARLS IN PAINTINGS L. Platiˇsa,a B. Cornelis,b T. Ruziˇ c,´ a A. Pizurica,ˇ a A. Dooms,b M. Martens,c M. De Mey,d I. Daubechiese cGhent University, TELIN-IPI-IBBT, Ghent, Belgium; bVrije Universiteit Brussel, ETRO-IBBT, Brussels, Belgium; cGhent University, Faculty of Arts and Philosophy, Dept. of Art, Music and Theatre, Ghent, Belgium dThe Flemish Academic Centre for Science and the Arts (VLAC), Brussels, Belgium; eDuke University, Mathematics Department, Durham, NC, US ABSTRACT characteristics of the jewels in paintings, it was essential to have the spatial information involved in the analysis of pearl images. Objective characterization of jewels in paintings, especially pearls, Our contribution is threefold. Firstly, we demonstrate the abil- has been a long lasting challenge for art historians. The way an ity of the spatiogram similarity metric to quantify the similarity be- artist painted pearls reflects his ability to observing nature and his tween pearl images. Secondly, we introduce a method for matching knowledge of contemporary optical theory. Moreover, the painterly spatiograms of the images. Thirdly, we propose a set of novel met- execution may also be considered as an individual characteristic use- rics built around the spatial characteristics of the image data. As ful in distinguishing hands. In this work, we propose a set of image we will show in the paper, these techniques can be used in multiple analysis techniques to analyze and measure spatial characteristics of manners, including numerical quantification of the visually observed the digital images of pearls, all relying on the so called spatiogram image features and the degree of realism of the visual appearance in image representation. Our experimental results demonstrate good the painting, characterization of the specific properties of rendering correlation between the new metrics and the visually observed im- of different materials by an artist, or detecting copies of the artworks. age features, and also capture the degree of realism of the visual To test the performance of our proposed techniques we use both appearance in the painting. In that sense, these results set the ba- the images of painted pearls and those of photographed ones. For sis in creating a practical tool for art historical attribution and give the pearls in paintings, we look at The Ghent Altarpiece, both the strong motivation for further investigations in this direction. pearls painted by the original masters, the Van Eyck brothers (1432), Index Terms— Ghent Altarpiece, image analysis, histograms, and those of their copyists, Jef Van der Veken (1945) and Charlotte spatiograms, content similarity Caspers (2010). In addition, we consider the pearls painted by Hans Memling in his Portrait of Maria Maddalena Baroncelli (1470). In the next section, we demonstrate the benefit of spatiograms 1. INTRODUCTION over histograms in pearl image characterization and introduce the novel spatiogram-based metrics. Our experimental study results are In his 1435 treatise on the theory of painting De pictura (“On Paint- presented and discussed in Section 3. Finally, some concluding re- ing”), Leon Battista Alberti remarks that “gems and all precious marks are given in Section 4. things of that kind become much more precious by the painter’s hand” [1]. Indeed, artists have been attracted to the beauty of pre- cious stones and challenged to depict jewelery in their paintings for 2. SPATIAL FEATURES OF PEARLS IN PAINTINGS ages. As neatly reviewed by Autin et al. in the recent Jewels in Painting [2], the first pearls appear in the paintings of XVth-century For studying the images of jewels, it is essential to quantify the dis- artists. These include the Portrait of a Young Lady by Petrus Chris- tinctive features that evoke visual impression of the jewel-like luster tus and the portrait of Queen Margaret of Denmark by Hugo Van der and sheen. In The Ghent Altarpiece, pearls are often characterized Goes, in Flanders, and the portrait of Simonetta Vespucci by Piero di by a blurry highlight as a mirror image of the light source and a fine Cosimo and the portrait of Battista Storza by Piero della Francesca, glowing line at the other side indicating the delicate sheen emanat- in Italy. Famous later examples include, of course, the depictions in ing from the surface [3]. For illustration, see the pearls in Fig. 2 (a) Vermeer’s Girl with the Pearl Earring and Ingres’ Turkish Bath. and (b), or the top left pearl in Fig. 4. The way an artist painted pearls reflects his ability to observing Digital image histograms can be used to discriminate between nature, and in some cases like Jan Van Eyck, his knowledge of con- different materials in the scene: glass, wood, metal, and so on. How- temporary optical theory [3]. The painterly execution may also be ever, histograms only characterize global intensity distribution in considered as an idiosyncratic marker or an individual characteristic the image without capturing spatial relations between the image ele- useful in distinguishing hands. In this context, our method aspires to ments (pixel values). creating a tool for art historical attribution. The so-called spatiogram representation of the image data [4], We explore mathematical tools that could assist art historians adds to the histogram the spatial information about the data. We will in studying the jewels in paintings, all in the domain of digital im- show in this paper that various spatiogram features can be used to age analysis. The proposed techniques are based on the image spa- indicate more material properties (e.g. light reflectance or smooth- tiograms [4] which extend the concept of histograms to the spatial ness of the surface) but also those of the background. The details are domain. Knowing that surface reflectance is among the most notable explained later in this section. 2.1. Spatiogram and spatiogram similarity metric Table 1. New spatiogram metrics for pearls images In general, a spatiogram S is determined by three components [4]: Symbol Name Definition 1 the histogram bin count, cb; the spatial mean, µb = (µx,b, µy,b); M1 Mean(D) N i Di, i = 1, ..., B 1 and the spatial covariance, Σ σ 2,σ 2 . Similar as with P − b = ( xx,b yy,b ) 1 2 histograms, all three spatiogram components are computed for a set M2 Var(D) N i(Di M1) P − of bins, identified by index b 1, ..., B . To enable comparison M3 Rx maxi µx,i mini µx,i between regions of different sizes,∈ { all spatial} coordinates need to − be normalized to the same range. In our experiments, this range is M4 Ry maxi µy,i mini µy,i − [-1,1]. Visualization of multidimensional spatiogram data is non- trivial and, to our knowledge, it has not been addressed so far. We (a) (b) (c) (d) (e) introduce here a spatiogram visualization with three types of plots, see Fig. 5: (S1) connected centers of bins, µ; (S2) µ-positioned counts of bins (c-wide circles); and (S3) µ-positioned variances of bins (x and y error bars, respectively, σxx and σyy long). In all plots, the color identifies the bin, b±. In Fig. 5,± rows 2 and 3 illustrate the ability of spatiogram to capture the difference between Fig. 1. Test images to illustrate the properties of M1-M4 (see text). two images even when their histograms are exactly the same. 0 0 0 0 Let S = (c, µ, Σ) and S = (c , µ , Σ ) denote two spa- tiograms, each with B bins. Using m µ, Σ to represent a ( ; ) The proposed metrics are derived from the centers of spatiogram normalized Gaussian evaluated at m, weN can write the spatiogram bins, µ , and their distances, D . Especially, we define the distance similarity measure as follows [5]: i i between the centers of the adjacent bins as the Euclidean distance: 2 2 2 B D µ µ where µ µx,i, µy,i and i , ..., B . 1 i = i i+1 i = ( ) = 1 1 0 0 4 0 − − ρ = √cbcb 8π ΣbΣb (µ ; µ , Σˆ b) , (1) The new metrics M1 to M4 are summarized in Table 1. For illustra- X h | | N b b i b=1 tion of their meanings, we create simplified model images shown in Fig. 1. In these synthetic pearls, we limit the number of bins to three: 0 where Σˆ b = 2(Σb + Σb ). The similarity score is 0 ρ 1 and the area of bin b3 corresponds to the often observed mirror image of comparing any spatiogram to itself yields ρ = 1. But,≤ the≤ value of the light source, b2 depicts the main surface of the pearl, and b1 rep- ρ cannot tell us about the specific details that contribute to ρ < 1. resents the glowing sheen usually present against the outline of the Therefore, we search for additional metrics for our analysis. pearl (coming as a reflection of light from the background) [6]. The M1 and M2 metrics, respectively, are the mean and the vari- ance of the D . The M1 captures the (circular) symmetry of the pearl 2.2. Spatiogram matching based on bin-similarity i area: the smaller M1 the higher the symmetry, suggesting potentially In order to better understand the relation between the visual appear- the less sharp angle between the light source and the pearl surface. ances of pearls and the properties of their spatiograms, we devel- For example, among pearls (a)-(c) in Fig. 1, M1 is the smallest for oped an algorithm to match the spatiogram of a given pearl image (a) and the largest for (b). The next metric, M2, relates to the uni- to that of a reference pearl. We refer to the process as indirect spa- formity of “distances” between different bin areas, or the impression tiogram matching and to the resulting image as matched image.