Digital Cradle Removal in X-Ray Images of Art Paintings
Total Page:16
File Type:pdf, Size:1020Kb
DIGITAL CRADLE REMOVAL IN X-RAY IMAGES OF ART PAINTINGS Rujie Yin1, David Dunson1, Bruno Cornelis2, Bill Brown3, Noelle Ocon3, Ingrid Daubechies1 1: Duke University, USA; 2: Free University Brussels (VUB), Belgium; 3: North Carolina Museum of Art, USA ABSTRACT cradling slat perpendicular back of painted We introduce an algorithm that removes the deleterious effect to panel wood grain panel of cradling on X-ray images of paintings on wooden panels. The algorithm consists of a three stage procedure. Firstly, the cradled regions are located automatically. The second direction of wood step consists of separating the X-ray image into a textural grain of panel cradling slats in direction of and image component. In the last step the algorithm learns panel wood grain to distinguish between the texture caused by the wooden cra- dle and the texture belonging to the original painted wooden Fig. 1: Cradle of old paintings on wood panel. The results obtained with our method are compared panel: Top left: a sample patch from the X- with those obtained manually by best current practice. ray image of a cradled painting panel, illustrat- ing the artifacts caused by the cradling, with en- Index Terms— texture, art, painting, cradle. larged detail (showing cradle wood grain). Top right: sketch of the lattice of crossed perpendicu- 1. INTRODUCTION = p lar wooden slats: slats parallel to the panel wood bd = c grain are glued first; transverse slats are slotted = int 1.1. The digital cradle removal problem c in. Left: indication, on the sample patch, of the partition into different domains used in x2. Between the 12th to the 17th century, paintings in Europe were mostly created on wooden panels consisting of solid wooden boards. Until about 1950, cradling was a common restoration technique used by conservators to remediate or successfully removed by a morphological component analy- prevent structural or insect damage. In the process, the pan- sis (MCA) [4] pre-processing step. To our knowledge, there els were first thinned, and then strengthened by (permanently) is no algorithm yet to remove cradle artifacts from extended attaching to their backs hardwood lattices called cradles. portions of the X-ray images of panel paintings; experienced On the other hand, modern-day museum conservators and conservators still do this manually, in Adobe PhotoshopTM – a art historians rely heavily on X-ray imaging to study artist time-consuming process, applied to only a limited number of technique, painting fabrication, and the physical condition of paintings among the large museum collections.1 the painting (e.g. cracks in the painted surface or wooden sup- port). X-ray images of cradled works contain highly visible 1.2. Our Approach horizontal and vertical “bars”, caused by the higher X-ray ab- sorption in the thicker wood layer due to the cradle (see Fig. We formulate the digital cradle removal as a source separa- 1). This obstructs the “reading” of the X-ray image by art ex- tion problem. According to the physical composition of the perts, who would welcome a (semi-)automated procedure to wooden panel and the cradle, a raw X-ray image I can be remove these artifacts from the X-ray images. viewed as the superposition of a panel image Ip, containing In recent years, image processing algorithms have been all the information from the painting on the panel and the developed to analyze high-resolution digital images of art wooden panel itself, and a cradle image Ic induced by the 0 paintings, e.g. to classify styles [1] , to detect (and inpaint) cradling alone. We decompose Ic into two parts: Ic , piece- cracks [2], or to remove canvas artifacts [3]. In [2] cradling wise constant on the cradle support, corresponding to the cra- 2 wg artifacts in X-ray images constituted a hindrance, but since dle thickness , and Ic , a texture component reflecting the only a small portion of the painting needed to be processed, 1Alternative X-ray techniques involve filling of the voids in the cradle lat- the corresponding artifacts were easy to isolate, and were then tice by pressing putty into them, in an attempt to even out the X-ray exposure, resulting in additional risk to the paintings. We gratefully acknowledge support from SAMSI (RY), FWO (BC), NSF 2This assumes that the cradle has sharp boundaries; for less sharp bound- 0 (DD and ID), ::: aries, e.g. when the cradling supports have rounded corners, Ic is not con- cradle’s wood grain. The panel image can also be decom- fixed), followed by a Radon transform R: posed, into a “cartoon” part Icart (i.e. the different “color” p Z 1 planes and their boundaries), and a texture part Itext (i.e. p RjδIj(α; s) = jδI((s cos α; s sin α) + tτα)jdt (1) brushstroke details as well as the wood grain of the panel). −∞ Accordingly, the decomposition of the whole image into “tex- cart 0 text wg where τα = (− sin α; cos α), with α 2 [−θ; θ]. (See Fig. 2.) ture” and “cartoon” parts is I = (Ip +Ic )+(Ip +Ic ): 0 wg We use a three-stage approach to extract Ic and Ic from I. We first estimate the locations of the cradles; to detect the cradle slat angles (close to but not exactly 0◦ or 90◦), we use the Radon transform (x2.1). We also estimate the intensity difference between each single cradled region and the sur- rounding panel-only region; crossing areas (where horizontal and vertical slats overlap) need to be treated separately, be- cause they correspond to a different intensity change. We use 0 these estimates to define the function Ic , constant on the in- terior of each single or crossing cradled region (with sharp or simply modeled smoothed edges), and subtract it from I. We intrm 0 define this intermediate result as I := I − Ic . The second stage uses MCA,inspired by the digital can- intrm text cart vas removal [5], to separate I into I and Ip . To decompose F = A + B into its constituents, MCA uses that A and B each have sparse decompositions into different stan- dard dictionaries, DA and DB. In our case, the textures in wg text Ic and Ip have sparse representations in a dictionary of Fig. 2: Radon transform to locate the cradle: I here is the high frequency curvelets or shearlets; for the complimentary left part of Fig. 1. Left: peaks in RjδIj(α∗; s), for the optimal angle cart ∗ Ip , we use complex dual-tree wavelets, the same dictio- α , indicate the boundaries of the cradle. Right: a constant-angle nary as in [5] for the image “content”. section of the Radon transform of jδIj attains maximal `2 energy However, our problem is more complex than in [5], be- for α = α∗. cause Ip, which we want to recover, has components in both constituents; the third stage in the algorithm deals with this α k RjδIj(α; ·) k = R jδI(x; y)jdxdy For any , `1(R) 2 R ∗ complication. Let Dp and Dc be the panel-only and the cra- =k δI k 2 α `1(R ). On the other hand, for the optimal , int bd text ∗ dled domains (see Fig.1; Dc = Dc [Dc ). Then I jDp = RjδIj(α ; s) is more concentrated in s than for other α. Itextj Iwg = 0 D D Itextj = ∗ p Dp , since c on p. On c, we have Dc Thus the optimal α can be found by maximizing the `2(R)- wg text Ic jDc + Ip jDc . We shall use a variation of a sparse norm of RjδIj(α; ·); we set the angle of the horizontal cradle wg Bayesian factor model (see x2.2) to obtain Ic : we learn dic- slat as tionaries for the panel and cradle textures in the feature (high- α∗ = arg max k RjδIj(α; ·) k : `2(R) frequency curvelet or shearlet) space, using our observation α2[−θ,θ] of the different behavior of Itext on D and D ; Iwg is then p c c To estimate sharp boundaries we back-project the hard-thres- approximated by MCMC sampling. In this particularly chal- holded Radon data H (RjδIj(α∗; s)) to the image domain. lenging unsupervised setting, the Bayesian approach avoids λ In the case of smooth boundaries, the profiles are modeled by tuning parameters and the need for cross-validation. smooth shape (rather than step) functions fΦaga2A, with a uniform parameter choice (estimated from RjδIj(α∗; ·)) for 2. CRADLE REMOVAL ALGORITHM each transition region. The intensity difference across the cra- dle boundaries is also estimated from RjδIj(α∗; ·). 2.1. Radon transform and location estimation Suppose, for instance, that the X-ray image I on D ⊂ R2 2.2. Additive factor model contains one horizontal cradle slat.3 To even out fine scale noise and texture effects, we first apply an elongated verti- The second stage is a fairly standard application of MCA (see L−1 L text intrm 0 P P [4]) to separate out I from I = I − Ic . The in- cal Haar transform, δIi;j = l=0 Ii−l;j − l=1 Ii+l;j,(L put to the third stage, explained here, consists of Itext and stant. However, it can then be modeled or approximated, as shown below. the partition of the image domain D = Dp [Dc obtained 3 This assumption can be made without loss of generality: since cradle in the first stage. Let S be the linear curvelet (or shearlet) slats in the same direction are well separated, it is easy to consider patches text that have at most one cradle slat in each direction.