Delft University of Technology

Comparison of three 3D scanning techniques for paintings, as applied to Vermeer’s ‘Girl with a Pearl Earring’

Elkhuizen, Willemijn S.; Callewaert, Tom W.J.; Leonhardt, Emilien; Vandivere, Abbie; Song, Yu; Pont, Sylvia C.; Geraedts, Jo M.P.; Dik, Joris DOI 10.1186/s40494-019-0331-5 Publication date 2019 Document Version Final published version Published in Heritage Science

Citation (APA) Elkhuizen, W. S., Callewaert, T. W. J., Leonhardt, E., Vandivere, A., Song, Y., Pont, S. C., Geraedts, J. M. P., & Dik, J. (2019). Comparison of three 3D scanning techniques for paintings, as applied to Vermeer’s ‘Girl with a Pearl Earring’. Heritage Science, 7(1), [89]. https://doi.org/10.1186/s40494-019-0331-5 Important note To cite this publication, please use the final published version (if applicable). Please check the document version above.

Copyright Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons. Takedown policy Please contact us and provide details if you believe this document breaches copyrights. We will remove access to the work immediately and investigate your claim.

This work is downloaded from Delft University of Technology. For technical reasons the number of authors shown on this cover page is limited to a maximum of 10. Elkhuizen et al. Herit Sci (2019) 7:89 https://doi.org/10.1186/s40494-019-0331-5

RESEARCH ARTICLE Open Access Comparison of three 3D scanning techniques for paintings, as applied to Vermeer’s ‘Girl with a Pearl Earring’ Willemijn S. Elkhuizen1* , Tom W. J. Callewaert2,3, Emilien Leonhardt4, Abbie Vandivere5,6, Yu Song1, Sylvia C. Pont1, Jo M. P. Geraedts1 and Joris Dik3

Abstract A seventeenth-century canvas painting is usually comprised of varnish and (translucent) paint layers on a substrate. A viewer’s perception of a work of art can be afected by changes in and damages to these layers. Crack formation in the multi-layered stratigraphy of the painting is visible in the surface topology. Furthermore, the impact of mechanical abrasion, (photo)chemical processes and treatments can afect the topography of the surface and thereby its appear- ance. New technological advancements in non-invasive imaging allow for the documentation and visualisation of a painting’s 3D shape across larger segments or even the complete surface. In this manuscript we compare three 3D scanning techniques, which have been used to capture the surface topology of Girl with a Pearl Earring by Johannes Vermeer (c. 1665): a painting in the collection of the Mauritshuis, the Hague. These three techniques are: multi-scale optical coherence tomography, 3D scanning based on fringe-encoded stereo imaging (at two resolutions), and 3D digital microscopy. Additionally, scans were made of a reference target and compared to 3D data obtained with white-light confocal proflometry. The 3D data sets were aligned using a scale-invariant template matching algorithm, and compared on their ability to visualise topographical details of interest. Also the merits and limitations for the indi- vidual imaging techniques are discussed in-depth. We fnd that the 3D digital microscopy and the multi-scale optical coherence tomography ofer the highest measurement accuracy and precision. However, the small feld-of-view of these techniques, makes them relatively slow and thereby less viable solutions for capturing larger (areas of) paint- ings. For Girl with a Pearl Earring we fnd that the 3D data provides an unparalleled insight into the surface features of this painting, specifcally related to ‘moating’ around impasto, the efects of paint consolidation in earlier restoration campaigns and aging, through visualisation of the crack pattern. Furthermore, the data sets provide a starting point for future documentation and monitoring of the surface topology changes over time. These scans were carried out as part of the research project ‘The Girl in the Spotlight’. Keywords: Topography, 3D scanning, Optical coherence tomography, 3D digital microscopy, Painting, Cultural heritage, Image registration, Image processing

Introduction subsequent paint and varnish layers also infuence the The three‑dimensional landscape of paintings surface topography. Tis efect can be intentional—using Paintings are generally considered in terms of their the paint to create a 3D efect—or the consequence of (2D) depiction, but the physical artwork also has a third drying, hardening, or degradation. Artists, including Ver- dimension. Te substrate is rarely completely fat, and meer, deliberately created 3D textural efects on the sur- face. For instance, they used impasto to create additional refections for highlights, or used 3D efects to emphasise *Correspondence: [email protected] the textural appearance of the material they were depict- 1 Industrial Design Engineering, Delft University of Technology, Landbergstraat 15, 2628 CE Delft, The Netherlands ing. Alternatively, three-dimensional brushstrokes can be Full list of author information is available at the end of the article the consequence of a fast-paced, expressive style.

© The Author(s) 2019. This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creat​iveco​mmons​.org/licen​ses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/ publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated. Elkhuizen et al. Herit Sci (2019) 7:89 Page 2 of 22

Te topography of a painting will change under the imaging, visible light photography, ultraviolet (UV) imag- infuence of internal and external factors. Natural aging ing, X-radiography, macro X-ray fuorescence imaging, and (photo)chemical changes (e.g. the formation of metal multispectral and hyperspectral imaging, all provide soaps [1]) that occur within the diferent layers can result information about diferent modalities of a painting [2, in cracking, protrusions and/or changes in gloss. Te lay- 3]. Photographs in raking light conditions—commonly ers respond to environmental infuences: for example, an made during technical examination(s)—reveal undula- increase or decrease in temperature or relative humidity tions in the surface and visualise features like cracks (see can cause the support to expand or contract, resulting for example Fig. 1a). Although this technique empha- in cracking or deformations. Conservation treatments sises the efect of surface topography on its appearance, can also cause changes in topography. Linings, espe- it is not an exact measurement of the topography and are cially those that employ heat and pressure, can fatten the often not (well) documented. paint. Eforts to locally soften and fatten raised cracks Moreover, the efect of craquelure on a painting’s using heat and/or pressure can also cause irreversible appearance has been studied by Bucklow [5] describing changes to the 3D surface structure. Mechanical damages and classifying the types of cracks commonly found on during handling, transport or by accident can result in paintings, as well as exploring the pictorial efect of crack cracked, tenting, or faking paint. A paintings conserva- patterns [6]. Automated crack detection in digital images tor is compelled to document and address these issues, has been employed to classify paintings into geographi- but until recently, possibilities to record these changes, cal regions of origin [7] and for digital crack removal [8]. also over the long term, have been limited. A limitation of these studies is that they only consider craquelure as a two-dimensional pictorial feature, and not its three-dimensional shape. Painting documentation Tree 3D scanning methods have been demonstrated During technical examination(s), it has become general for capturing (and reproducing) the fne topographical practice to use a wide range of imaging techniques to details of painted surfaces, namely 3D laser triangula- visualise and document the condition and chemical com- tion, structured light 3D scanning, and focus variation position of a painting. Imaging techniques like infrared microscopy. Blais et al. [9, 10] captured the topography

Fig. 1 Existing documentation visualising the surface topography of ‘Girl with a Pearl Earring’. a Raking light photography is traditionally used to emphasise the topographical variations of the surface, visualising cracks and other unevenness of the surface (Courtesy of René Gerritsen Art & Research Photography). b The crack pattern of the painting was documented by manually tracing the cracks onto a transparent polyester flm, placed on the painting (Courtesy of Jørgen Wadum [4]). The red inserts show an enlargement of the area around the pearl earring Elkhuizen et al. Herit Sci (2019) 7:89 Page 3 of 22

of paintings using a ‘white’ laser spot (composed of a 1665), from the collection of the Mauritshuis1, has under- red, green and blue laser source). Tis scanner was later gone various conservation and restoration treatments in commercialised and currently used to create full-colour its lifetime. Some treatments, along with the degradation 3D printed reproductions [11] using an additional (2D) efects that can be expected of a seventeenth-century reference photograph for faithful colour rendition [12]. painting, have afected its surface topography [34]. It is Factum Arte [13, 14] and the Van Gogh Museum [15] at least certain that the painting had larger height vari- both (likely) use a combination of laser-based scanning ations in its painted surface when it left Johannes Ver- and digital photography to create facsimiles of paintings. meer’s studio. He applied small details, like the highlight However, extensive literature study did not reveal scien- on the Girl’s earring and dots on her clothing, with more tifc publications on it. impasto than the surrounding paint. In 1882, the Ant- Structured light 3D scanning, using a projector com- werp restorer Van der Haeghen lined the canvas sup- bined with either one or two cameras has also been pro- port with a starch-based paste. As part of a 1915 or 1922 posed to measure the topography of painted surfaces (e.g. treatment, restorer De Wild ‘regenerated’ the varnish by [16–21]). Te latter approaches [19–21] use a de-focused subjecting it to alcohol vapours and copaiba balsam. He projection pattern, whereby the sampling resolution presumably used the so-called ‘Pettenkofer’ process, to only depends on the camera resolution, rather than the help soften faking, brittle paint [35]. Tese treatments, (lower) projector resolution. and/or a consolidation treatment with an aqueous adhe- Focus variation microscopy has been demonstrated, for sive (date unknown), may have caused the starch-based instance to analyse punchmarks on medieval panel paint- lining to shrink. Although the paint already had some ings [22], and to evaluate its usefulness in supporting adhesion issues prior to the 1882 lining, the shrinkage painting cleaning [23]. Both studies have only used this may have caused more faking and cupping of the paint. technique to capture small areas, rather than 3D imaging In 1960, another restorer Traas relined the painting using larger regions or complete paintings. a wax–resin adhesive, applying heat and pressure. Tis Besides capturing micro-scale features, studies have wax–resin lining fattened some impasto details that been conducted to measure and monitor global shape Vermeer had applied more thickly than the surrounding variations of paintings, for example to monitor the efect paint, but ensured that the painting has remained struc- of tensioning tests conducted on a canvas painting and turally stable. gilt leather artefact [24], or monitor deformation of panel Te conservation history was documented as part of paintings over time [25]. Although these techniques are the 1994 restoration campaign, conducted as part of the used in relation to painting conservation treatments, project Vermeer Illuminated [4]. During this restoration, their resolution does not allow for capturing fne surface the crack pattern was traced onto a transparent polyester details like craquelure. flm that was carefully placed on the surface of the paint- An overview of the above-mentioned scanners and ing (see Fig. 1b). Tis was a way of documenting the sur- their specifcations can be found in Table 1. face condition, in addition to raking light photographs. Although conventionally optical coherence tomog- Te 2018 technical examination as part of the project Te raphy (OCT) is used to image the stratigraphy (layer- Girl in the Spotlight provided a new opportunity to exam- ing) of semi-transparent layers [27, 28], the uppermost ine Vermeer’s Girl with a Pearl Earring using traditional boundary also represents the topography of the surface. examination methods as well as state-of-the-art scientifc Its application for imaging the stratigraphy of cultural techniques. High-resolution digital photographs were heritage artifacts was demonstrated in various studies made of the painting in diferent lighting conditions, (e.g. [29–33]). To reach sufcient lateral and axial resolu- including raking light (see Fig. 1a). A new and innovative tion for these applications, the OCT feld-of-view is lim- part of this examination was the 3D documentation of 15 mm × 15 mm ited to a scanning area of approximately the painting’s surface using the following means: multi- By combining a high resolution spectral domain OCT scale optical coherence tomography (MS-OCT), 3D (SD-OCT) setup—a specifc sub-type of OCT using a scanning based on fringe-encoded stereo imaging (at two broadband light source (see [33])—with automated scan- resolutions), and 3D digital microscopy. ning stages, it is possible to scan much larger areas (as In this paper we compare these three imaging tech- demonstrated in this paper, reaching a single automated nologies. We fnd that the 3D digital microscopy and the 0.04 m2 scan area of ). MS-OCT ofer the highest measurement accuracy and

Case study: Girl with a Pearl Earring by Johannes Vermeer

From archival documentation and research it is known 1 39 cm × 44.5 cm Girl with a Pearl Earring by Johannes Vermeer (c.1665), , that Johannes Vermeer’s Girl with a Pearl Earring (c. invent. no. 670, Mauritshuis, Te Hague. Elkhuizen et al. Herit Sci (2019) 7:89 Page 4 of 22 - -automation, -automation, -automation, -frame, re-position-frame, - -automation, -automation, tioning tioning small samples ing(?) frame(?) tioning manual positioning frame on larger manual positioning frame on larger tioning manual Z -alignment manual on tripod, positioning per row N/A Tripod, manual posi - Tripod, Tripod, manual posi - Tripod, Microscope stage, Microscope stage, XY Scan setup Automated XY Automated Tripod, manual posi - Tripod, XY Partial XY Partial manual posi - Tripod, XY ( X ) Linear automation 2d 2 m 2 / m 2 / m 2 2 m / / m m / / 270 h 23 h 5h N/A N/A N/A N/A N/A Scan speed N/A 4h 0.8 h 5h ≈ ≈ ≈ a Flex. range Flex. N/A y y y y n n y n(?) y n y b 480 mm 300 mm 120 mm 1m 2m × × × × × 1m 2m Autom. range Autom. N/A Not automated Not automated 1 m ( X ) Not automated None used Not automated 40 mm 480 mm 300 mm ≈ ≈ Orient. N/A v v v v v v h h v v v 125 mm 100 mm 900 mm 2 mm 1.3 mm × × 80 mm 50 mm × × × × × 100 mm 170 mm 1 mm 1 mm Field of view (scan of view Field tile) N/A N/A N/A N/A N/A 60 mm 50 mm ≈ ≈ ≈ ≈ 1100 mm f m µ µ m µ m µ m µ m µ m µ m Accuracy ( Z ) Accuracy N/A N/A N/A N/A unknown 5 38 20 25 2.5 10 15

) 20 mm c m ( X ), µ m µ µ m µ m µ m µ m µ m µ m µ m µ m µ m 60 1 60 ( Y ) Resolution ( XY N/A 40 60 60 50 50 500 ≈ ≈ ≈ 100 100 Colour × × × × × × × × Top. Scan modalities × × × × × × × × × × × × triangulation projection projection microscopy microscopy tion projection projection triangulation tion projection RGB laser spot Structured light Structured light Focus variation Focus Focus variation Focus Laser line triangula - Structured light Scanning technique Laser triangulation Structured light RGB laser spot Laser spot triangula - Structured light Characteristics of scanning systems used to capture painting topography, including: the scanningincluding: of scanning Characteristics by the 3D modalities captured to captureused technique, systems topography, painting scanners, 1 e Estimated based on video footage of scanner based on video footage Estimated Reported processing time includes data [ 23 ] [ 15 ] [ 17 ] 26 ] Estimated based on projector resolution and specifed feld-of-view resolution based on projector Estimated This system can be regarded as the predecessor of the High-Res and Std-Res 3D scan systems used in this case study of the High-Res 3D scan systems and Std-Res as the predecessor can be regarded system This The fexibility of the scanning range is judged negatively, if the frame limits the size of the painting to be scanned, or in the case of using a cartesian frame, which is deemed difcult to extend or move or in the case of using a cartesian which is deemed difcult to be scanned, to frame, of the painting limits the size if the frame fexibility is judged negatively, of the scanning range The Although advertised at this level of accuracy, the authors mention that this is probably not realistic based on their fndings not realistic this is probably that the authors mention of accuracy, advertisedAlthough this level at

Laser triangulation [ 9 , 10 ] Blais et al. Structured light projection Akca [ 16 ] et al. Table scanner of the painting, feld-of-viewresolution, (or scan tile), orientation of the automated size the fexibility scanning range, to scan or no), (yes areas large and a shortscanning speed, description of the scan setup a b c d e f Breuckmann [ 18 ] Focus variation microscopy variation Focus et al. den Berg Van al. [ 22 ] Cacciari et al. Factum Arte [ 13 , 14 ] Factum Verus Art [ 11 , 12 ] Verus [ 24 ] Del et al. Sette Gogh Museum Van Karaszewski et al. [ 19 , Zaman et al. [ 25 ] et al. Palma Elkhuizen et al. Herit Sci (2019) 7:89 Page 5 of 22

Fig. 2 Multi-scale optical coherence tomography (MS-OCT) scanning setup. a Simplifed schematic layout of the multi-scale spectral domain OCT system (top view). b Annotated picture of the MS-OCT setup in lab conditions. c MS-OCT probe in front of Girl with a Pearl Earring during scanning of the painting. In all images (1) is the OCT probe, (2) the spectrometer, (3) the vertical stage and (4) the horizontal stage, β the imaging angle and l the working distance to the painting

precision. However, the small feld-of-view of these tech- specifcations and raw OCT data processing can be niques, makes them relatively slow and thereby less viable found in Table 2 and [33]. solutions for capturing larger (areas of) paintings. For Girl with a Pearl Earring we fnd that the 3D data pro- Calibration and data processing vide an unparalleled insight into the surface features of For the MS-OCT setup the desired scanned area was this painting. Tis specifcally relates to ‘moating’ around correctly aligned, by means of a built-in video camera, impasto, the efects of paint consolidation in earlier res- Te focal plane of the OCT sample arm optics was set toration campaigns and aging, and visualisation of the to 0.4 mm below the zero delay position; this to reduce crack pattern. Furthermore, the data sets provide a start- auto-correlation artefacts in the measured spectra. Ten, ing point for future documentation and monitoring of sample regions in the scanning area were captured, to the surface topology changes over time. ensure that there was no height variation larger than 1.89 mm (depth-of-feld) within the dimensions of a sin- Methods/experimental gle tile. Next, a self-developed program in LabVIEW [36] Multi‑scale optical coherence tomography (MS‑OCT) was activated, which fully automatically scans the art- Te spectral-domain OCT system has a source spectrum work and keeps it in focus. MS-OCT tile stitching was centered at 900 nm, spanning a bandwidth of 195 nm. performed of-line, based on segmented surface data Te specifed axial bandwidth-limited resolution (Z) in volume scans. Tis image stitching algorithm is not 3.0 m is µ (in air). Te OCT scan probe is mounted on dependent on tile overlap (due to high precision transla- a 3-axis stage consisting of two 200 mm (X and Y-axis) tion stages), but image segmentation artefacts can in rare scan range stages and a 100 mm (Z-axis) scan range cases result in stitching errors. stage. Te OCT probe head is mounted to the setup with the aid of a customised mount, which has 2 degrees of freedom for tilt around the X axis and Y axis, enabling 3D scanning based on fringe‑encoded stereo imaging imaging of a surface under a small angle. Tis is neces- Two 3D scanning systems, both based on fringe-encoded sary to reduce the amount of over-saturation on the stereo vision, were used to scan the complete painting. detector, caused by surface specular refections refect- Te imaging systems both consisted of an XY-frame able ing back into the lens, which are then for the most part to move the imaging module in a parallel plane to the avoided. Te scans were carried out overnight, with surface of a painting, in a horizontal and vertical motion. constant temperature and relative humidity, to mini- Figure 3a depicts the schematic layout for both imaging mise the infuence of environmental conditions on the modules. Te imaging modules consisted of two cameras scan results. Figure 2a shows the schematic depiction positioned on either side of a projector, all ftted with of the spectral-domain OCT setup, 2b the probe head polarisation flters. Tis cross-polarised setup removes and scan system, and 2c the instrument set up in front unwanted refections like highlights in the images. of the painting. More information about the system Te cameras were positioned at an angle relative to the Elkhuizen et al. Herit Sci (2019) 7:89 Page 6 of 22

Table 2 Specifcations of 3D scanning systems used in this case study, including: system design, scan modalities, system components, computational hardware used, resolution, scan area, settings used, and the confguration of the setup System model MS-OCT Std-Res 3D scan High-Res 3D scan 3D digital microscopy White light confocal proflometry Custom Custom, 2nd gen. Custom design, 3rd Hirox RH-2000 Hirox Nano Point gen. Scanner

Scan modalities(s) Topography × × × × × Colour × × × Stratigraphy × Gloss × Imaging component(s) 1 2 2 1 1 Camera model N/A Nikon D800E Canon 5DS-R Hirox JYFEL Lens model Thorlabs LSM04-BB Nikkor PC-E 85 mm Canon TS-E 90 mm MXB-5000REZ at 140× NP3 Lens extension tube(s) N/A N/A Canon EF25 II N/A N/A Filter(s) N/A 3 Hoya HD polarisers 3 Canon polarisers N/A N/A Illumination 1 1 1 2 1 component(s) Type SLD Projector Projector Fiber optics LED Fiber optics LED Model/type N/A Acer X113H AXAA M6 Hirox JYFEL Illumination intensity N/A 2800 lm 1200 lm Raking light at 100% and 100% dark feld at 10% Computational hardware Computer Dell T1700 ASUSTeK ASUSTeK Apple iMac Apple iMac Processor Intel Xeon E3-1271v3 Intel Core i7 Intel Core i7 Intel Core i5 Intel Core i5 RAM memory 16 GB 128 Gb 128 Gb 32 GB 32 GB Memory 1 TB SSD 500 Gb SSD 500 Gb SSD 1 TB SSD 1 TB SSD Scan sampling resolution Lateral resolution (XY) 8.5 µm × 67.6 µm 25 µm 7 µm 1.1 µm 2.6 µm Depth resolution (Z) 3.0 µm 27.5 µm 17.5 µm N/A < .150 µma Scan area Tile size (XY) 5 mm × 5 mm 170 mm × 100 mm 60 mm × 40 mm 2.1 mm × 1.31 mm N/A Depth-of-feld 1.87 mm ≈ 8.5 mm ≈ 1 mm 0.05 mm 1.40 mm Automated range (XY) 0.2 m × 0.2 m 1.3 m × 1.3 m 1.0 m × 1.0 m 0.5 m × 0.5 m 0.5 m × 0.5 m Settings ISO N/A 100 100 Gain 0 dB N/A Aperture N/A 1/16 1/16 Full open N/A Shutter speed N/A 0.5 s 3.2 s 1/250 s N/A Other parameters N/A N/A N/A Edge = 9, Sat. = 2, N/A Colour = 2 Images per tile 1024 26 26 ≈ 50 for 450 µm top- N/A bottom range Tile overlap N/A 41% (X), 30% (Y) 20% (X), 25% (Y) 30% N/A Confguration Painting orientation Vertical Vertical Vertical Horizontal Horizontal Imaging distance (l) ≃ 42 mm 440 mm 175 mm 10 mm 12 mm Imaging angle ( β) ≈ 5.0◦ 40◦ 21.5◦ 0◦ 0◦

The latter specifes the orientation of the painting during scanning, the imaging distance l, which is the closest point of the system to the painting’s surface, and the imaging angle β , describing the angle between the painting’s surface normal and the optical axis of the imaging components a Stated depth accuracy by manufacturer Elkhuizen et al. Herit Sci (2019) 7:89 Page 7 of 22

Fig. 3 Two 3D scanning setups based on fringe-encoded stereo imaging. a Simplifed schematic layout of both 3D imaging modules (top view). b, c Standard resolution scanner (Std-Res 3D scan at a lateral sampling resolution of 25 µm × 25 µm ) also featuring gloss scanning, where c is a close-up of the imaging components (top view). d, e High-resolution 3D scanner (High-Res 3D scan at a lateral sampling resolution of 7 µm × 7 µm ), whereby d is a close-up of the imaging components (front view). In all images (1) denotes the cameras, (2) the projector, (3) the vertical stage(s) and (4) the horizontal stage, β the imaging angle, α the tilt angle of the lenses and l the working distance to the painting

surface normal ( β in Fig. 3a). Camera lenses utilising the relative humidity, to minimise the infuence of environ- Scheimpfug principle (also known as tilt-shift lenses), mental conditions on the scan results. Based on the geo- were used to align the focal plane to the painting surface, metric layout of the system—imaging angle β and lateral rotated by angle α (see Fig. 3a). imaging resolution—the theoretical lower boundary of 17.5 m Te Std-Res 3D Scan (described in [20], based on the the axial resolution (Z) is µ . An overview of the original design by [19]), sampled at a lateral resolution specifcations of both scanners can be found in Table 2. 25 m (XY) of µ , and is shown in Fig. 3b, c. Te scan was carried out overnight, and under constant temperature Calibration and data processing and RH conditions, in complete darkness to minimise For both 3D scanning systems, the lens distortion of the infuence of environmental conditions on the scan cameras was calibrated following a multi-view geometry results. Based on the geometric layout of the system— calibration procedure [37], using a checkerboard pattern. imaging angle β and lateral imaging resolution—the Te white balance and illumination non-uniformity were 300 mm × 300 mm theoretical lower boundary of the axial resolution (Z) is calibrated using a Spectralon panel. 27.5 m µ . Te colour and 3D topography of the surface were cap- Te second system, hereafter denoted as High-Res 3D tured using a hybrid solution of fringe projection and Scan (described in [21]) sampled at a lateral resolution of stereo imaging. To measure the topography, a 6-phase 7 m 25 mm µ , and is shown in Fig. 3d, e. Two lens extend- shifting sinusoidal grey-scale pattern (fringe) was pro- ers were used in this setup to extend the focal length jected horizontally and vertically on the painting’s sur- 90 (of the commercially available tilt-sift lenses) from to face, capturing in total 24 images for a single scan area. 105 mm , allowing focusing at closer range, and thereby One additional image was captured with uniform illu- higher resolution imaging. Te scan was carried out in mination from the projector, which is used as the colour a glass enclosure, and with constant temperature and image. Tis imaging module was then moved to the next Elkhuizen et al. Herit Sci (2019) 7:89 Page 8 of 22

Fig. 4 3D digital microscopy setup. a Simplifed schematic layout of 3D digital microscope. b Annotated picture of the 3D digital microscope setup in lab conditions. c 3D digital microscope during imaging Girl with a Pearl Earring. In all images (1) denotes the Hirox microscope lens MXB-5000REZ, mounted on the motorised Z-axis focus block, (2) the LED lighting provided through optical fber(s), (3) the motorised X-axis which moves the microscope, (4) the motorised Y-axis used to move the painting platform, and l the working distance to the painting

position and the procedure was repeated. Ofine, fringe contrast in the shadowed regions, needed for the 3D unwrapping is applied and a sparse stereo matching was imaging. Te Z-axis focus-stack range (bottom-to-top) 450 m carried out to match the fringes of both camera images. was set to µ , making sure that the height variation A look-up table was generated for both cameras, encod- of the complete scan area was captured within this range. ing both images. Taking into account the camera calibra- An overview of the specifcations can found in Table 2. tion, a dense stereo matching was performed, using the principle of ray-tracing. Te RGB values of the uniformly Calibration and data processing illuminated image were then mapped onto the XYZ data- Prior to starting the scanning, the alignment of the axis points. Next, a plane was ftted through the datapoints and the planarity of movement relative to the platform and the data was sampled in a regular XY-grid at respec- (and painting) was checked using Hirox-supplied refer- 25 m 7 m tively µ or µ intervals, which approximates the ence targets. A Hirox calibration grey card was used to reconstructed resolution. If a single pixel contained more set the white-balance of the microscope. XY-scale was than one value (due to our non-parallel confguration this calibrated using the Hirox-supplied, certifed glass scale. can occur), the average of the values was taken. Missing Te Hirox software was used for programming the values were interpolated based on surrounding pixels. A automatic acquisition of the individual tiles. Te micro- detailed description of the image processing can be found scope created a series of images in the Z-direction in [19, 20]. Te output of these systems is an aligned col- capturing each focus layer individually, between the pre- our and height map. defned top and bottom set point. Tese are then com- bining into one single all-in-focus image (multi-focus or 3D digital microscopy extended depth-of-feld). Te microscope then moves Te painting was examined using a 3D digital microscope to the next position and the multi-focus capturing pro- (Hirox RH-2000), using the principle of focus stacking cedure is automatically started again. Te result for each 140× for 3D imaging, at a magnifcation of . A custom tile is a TDR fle (Hirox 3D fle format) which includes a bridge stand was built to accommodate the size of the all-in-focus colour image (JPG) as well as altitude in the 500 mm × 500 mm painting, with a automatic motorised Z-axis for each pixel. Te raw data—the individual focus XY-stage, combined with the existing automated Z-axis images—was not saved due to size limitations. Hirox of the microscope. In this confguration the painting was e-Tiling software [38] was used to stitch ofine the indi- placed horizontally on a vibration damping table and the vidual 3D tiles (providing identical functionality to the microscope moved on a motorised stand above it (see online stitching of the microscope), resulting in large Fig. 4). Te scans were carried out in a glass enclosure, 3D-stitched fles. with constant temperature and relative humidity, to min- imise the infuence of environmental conditions on the Pre‑processing and alignment of data sets scan results. A combination of (built-in, LED) directional Te height data was measured by the researchers and m and ring lighting was used to, respectively, penetrate stored in an array that is sampled in µ (rather than pix- the transparent varnish layer, and create the necessary els). Tis enables interchangeable dataset comparison. Elkhuizen et al. Herit Sci (2019) 7:89 Page 9 of 22

(a) Interpolate missing data

(b) Detrend data

Loop over all AR’s in template image (c) Select ROI

Determine cross-correlation coefficient for every iteration (d) Data alignment of the template matching function

Resample template image with best matching AR and (e) Detrend data proceed to next dataset Fig. 5 Flowchart of data processing algorithm for painting data alignment

Consequently the data was processed by the alignment protective top layer of nickel-boron. Tis was one of the algorithm, making use of the (open-source) implemen- few measurement samples that could be found with fea- mm tations of SciPy [39] (scientifc computational libraries) tures in the range, rather than the much more com- m and OpenCV [40] for Python. Te data comparison fow- mon µ range, commonly used for a variety of (contact) chart is shown in Fig. 5. First, missing data is replaced roughness measurements. Te sample has four milled by a linear interpolation result of the respective height grooves, with expected (i.e. not certifed, and found to be map (Fig. 5a). Secondly, the data is de-trended in order only a rough estimates of the feature sizes) milled depths 30 m to minimise mismatching due to angled scanning, orien- of 1000, 500, 200 and µ , and widths of 3, 2, 2 and 0.5 mm tation of the painting during measurements and bend- (see Fig. 6a). ing of the painting (which is a non-rigid object). After As the target was not certifed, we did not compare our these initial preparation steps, a region of interest (ROI) measurements to the provided dimensions, but rather is selected in which every imaging method has data and compare them to measurements made using white light contains structures which show clear topology varia- confocal proflometry (WLCP), using the Hirox Nano tions. Consequently, the height arrays are aligned with Point Scanner (NPS). Te JYFEL NP3 measurement unit a scale invariant template matching algorithm. Te sizes was mounted on and connected to the Hirox RH-2000 of the individual height arrays do not have to match with system, in a similar fashion as the 3D digital microscopy respect to the X/Y pixel dimensions (scale invariance), setup. As the height accuracy is declared at 150 nm, it but the aspect ratio (AR) of the surface map must geo- suitable as a gold standard for the other measurements, metrically be correct. Tis scale invariance is a require- which are in the micron range. An overview of the speci- ment, since a given area will be sampled with diferent fcations can found in Table 2. Te output of this system resolutions by every method. We fnd the resampled is multiple profle measurements across the grooves, matched template (Fig. 5d by computing the cross-corre- resulting in a 3D surface. lation coefcient and locate its maximum value, so that Te sample was scanned completely or in parts by the the overlapping areas are positioned on top of each other. diferent measurement instruments (depending on the Te AR with the highest cross-correlation coefcient is tile size), and local height variations were compared for deemed the best match by the system and will become the four grooves. Te rectangles in Fig. 6b show the sam- the overlapping area that is visualised. Finally, we apply pling locations of the comparison areas. We compare 2D again a detrending step (Fig. 5e), in order to remove some areas for every technique. For the NPS scanner there are local height variations in the datasets. For visualisation only 10 adjacent line-scans present, but this is ofset by purposes we align the mean height value of the datasets. the high lateral sampling and low standard deviations found for these measurements. Te data of the reference Reference target measurement comparison target was pre-processed and aligned in a similar fashion In order to compare each imaging technique’s ability to as the painting data. faithfully reproduce the height profles of the objects, we compared the measurements results of a reference target (Rubert&Co, sample no. 513E [41]). Te reference tar- get was a electro-formed nickel specimen, with a hard Elkhuizen et al. Herit Sci (2019) 7:89 Page 10 of 22

Fig. 6 Reference target with four grooves of varying depth and width. a Photograph depicting the (uncalibrated) reference target no. 513E, produced by Rubert & Co [41]. Expected groove depths (from left to right) are 1000, 500, 200 and 30 µm . b Hirox Nano Point Scanner (NPS) cross-section plot of the reference target. Blue line is the experimental NPS data. Orange dotted line is the ftted data, without non-afne artefacts. The boxes and their respective colours indicate the sampling locations for the results described in Table 4

Results Reference target scan results 30 Te reference target (depicted in Fig. 6), was scanned m) 20 by all four imaging systems. Generally, we found that all ( techniques are capable of measuring the grooves in the 10 sample, albeit the 3D scanning systems needs an non- cross-polarised setting to capture the projected fringes 0 on this metallic artefact. For the four comparison regions, the standard deviation was calculated, and the mean -10 height diferences between the respective region pairs. Std-Res 3D scan -20 We found that the top surface of the reference target High-Res 3D scan Mean error rel. to NPS 3D Microscopy was not completely fat (also in the NPS measurement). -30 MS-OCT Terefore, we ftted a curve through the NPS measure- ments, and corrected all the other measurements for this 1000 500 20030 curvature (see Fig. 6b). Table 4 provides an overview of Expected groove height (mm) the (corrected) measurements results for every scanning Fig. 7 Measurement results of reference target with four grooves. Mean error relative to the Hirox Nano Point Scanner (NPS) system. measurements (in µm ), measured locally between top and bottom Te (corrected) mean height diferences for every surface for every groove (see Fig. 6), from the deepest (left) to groove, relative to the NPS measurements, are plotted in shallowest (right) groove. Dotted lines show the theoretical scanning Fig. 7. In this fgure the dotted lines denotes the theoreti- resolution (in X) of the imaging devices (corresponding colours) cal Z-resolution of the scanning systems. Note that there is no theoretical Z-resolution available for the 3D digital microscopy. We found the smallest relative errors (abso- measurements. For the 3D digital microscopy the σ 3 m 3.1 6.3 m lute) for the 3D digital microscopy (between 0 and µ ) lies between and µ , for the MS-OCT between 6 m 1.4 7.4 m 7.2 and MS-OCT (between 0 and µ ), as compared to the and µ , for the High-Res 3D scan between 11 m 12.9 m 9.2 High-Res 3D scan (between 0 and µ ) and Std-Res and µ , and the Std-Res 3D scan between and 24 m 15.0 m 3D scan (between 8 and µ ). All errors except one µ . fall within the resolution boundaries. A mean absolute 6 m 500 m error of µ was found for the MS-OCT, at the µ Painting scan results groove, which lies outside the resolution boundaries of Using the MS-OCT, the painting was scanned in four the MS-OCT. sessions overnight, each time scanning an area of 200 mm × 200 mm As expected, we found that the NPS measurements . Te total scanned area, consisting of σ 41 × 41 have the smallest standard deviation ( ) (between 4 sets of tiles, resulting in a total scanned area of 0.42 µm 350 mm × 400 mm 0.05 and ), which were in all cases (close to or (excluding overlaps). larger than) a magnitude scale diferent to the other Elkhuizen et al. Herit Sci (2019) 7:89 Page 11 of 22

Table 3 Scan results of four 3D imaging systems for Girl with a Pearl Earring by Johannes Vermeer MS-OCT Std-Res 3D scan High-Res 3D scan 3D digital microscopy

Scan results Scanned areas Complete painting in 4 Complete painting Complete painting 10 regionsa quadrants No. of tilesb All: 41 × 41 × 4 All: 4 × 8 All: 8 × 15 Left eye: 16 × 23 Pearl Highlight: 11 × 20 Jacket: 16 × 26 Data size (per tile)c Raw data 525 Mb 2.0 Gb 1.6 Gb N/Ad Processed data 525 Mb 15.0 Gbe 11.4 Gbe 4 Mb Data size equivalent (per ­m2)f Raw data 210 Gb 111 Gb 666 Gb N/Ad Processed data 210 Gb 882 Gbe 4.75 Tbe 1.45 Tb Imaging time Capture time (per tile) 20 s 5 m 00 s 2 m 14 s 3 s Processing time (per tile) 50 s (ofine) 6 m 20 s (ofine) 8 m 45 s (ofine) 2.6 s (online) a These regions are all comparable in size to sample regions depicted in Fig. 8b and c b For the OCT scanning and 3D scanners, this denotes the number of tiles captured to scan the complete painting; for the 3D digital microscopy this denotes the number of tiles within the scan areas that were compared in the results section c Note that the tile sizes are very diferent between systems d The individual images that constitute a focus stack are not saved, due to storage limitations e This is the total data size per tile, including raw and processed image fles (i.e. conversion of camera RAW format to TIFF) f This excludes any overlap between tiles g This includes the time needed for de-trending and stitching the tiles, which was integrated in the processing of the individual tiles

Table 4 Comparison of reference target measurements, sample no. 513E by Rubert & Co [41], as depicted in Fig. 6 Groove Exp. depth NPS MS-OCT Std-Res 3D scan (µ m) N σ (µ m) (µ m) Err. (µ m) N σ (µ m) (µ m) Err. (µ m) N σ (µ m)

1 (t) 1000 1220 0.22 984 N/A 250 7.4 982 −2 22.5k 15.0 1 (b) 20 0.05 250 6.2 150 9.2 2 (t) 500 1050 0.42 496 N/A 400 5.8 490 −6 10.5k 10.2 2 (b) 10 0.12 300 1.6 2400 17.7 3 (t) 200 910 0.21 195 N/A 990 4.5 194 −1 12k 10.3 3 (b) 200 0.08 550 1.4 7500 9.3 4 (t) 30 1100 0.21 29 N/A 600 3.0 29 0 4500 10.8 4 (b) 10 0.05 400 4.1 750 10.4 Groove Exp. depth Std-Res 3D scan High-Res 3D scan 3D digital microscopy (µ m) (µ m) Err. (µ m) N σ (µ m) (µ m) Err. (µ m) N σ (µ m) (µ m) Err. (µ m)

1 (t) 1000 972 12 7000 12.9 987 3 340k 3.5 987 3 − 1 (b) 2250 8.6 240k 3.1 2 (t) 500 520 24 7750 12.2 496 0 348k 2.5 496 0 2 (b) 1000 7.2 228k 3.8 3 (t) 200 206 11 2250 8.6 206 11 560k 3.2 194 1 − 3 (b) 7000 8.3 456k 3.4 4 (t) 30 37 8 6250 10.1 35 6 125k 6.3 31 2 4 (b) 1050 10.3 155k 5.2

The Nano Point Scanner (NPS) measurements are taken as the gold standard. The grooves are labeled from the deepest (1) to most shallow (4) groove, where (t) denotes the measurement at the sample’s top surface and (b) the measurement at bottom of the groove. For every measurement region, N denotes the number of measurements, σ the standard deviation of those measurements, the height diference between the top and bottom (mean diference), and Err. the error with respect to the NPS measurement. Except N all the results are specifed in µm . The errors (relative to the NPS measurements) are plotted is Fig. 7 Elkhuizen et al. Herit Sci (2019) 7:89 Page 12 of 22

Te Std-Res 3D scanner imaged the painting overnight in the completely dark environment, and was able to 4 × 8 capture the complete painting in in tiles, in a little over 2.5 h. Te High-Res 3D scanner captured the com- 8 × 15 plete painting in tiles, with a total capturing time of approximately 4.5 h. Tis scan was carried out during open hours of the museum, with environmental lighting. Note that the scan time did not scale linearly with the resolution between these scanners. Tis is due to limita- tions in data transfer and various diferences between the two systems, including extent of scan speed optimisation and capturing of gloss as additional modality (for the Std- Red 3D scan). Due to time restrictions the 3D digital microscope scanned only ten regions of interest (ROI) on the paint- ing at a high enough magnifcation to be relevant for 3D data comparison As an indication, the Left Eye region was 16 × 23 scanned in tiles, and took roughly 0.5 h to scan. Details on the scan time and data size for every imaging system be found in Table 3. We compared the results of the four imaging systems by studying three ROIs scanned by the 3D digital micro- scope. Te location of the comparison areas are shown in Fig. 8 Locations on the painting of the three regions of interest Fig. 8. Te results per scanner of these three regions are compared. a the Pearl Highlight, b a section at the front of the Jacket, depicted in Fig. 9, where the top row shows the stitched and c the Left Eye. Detailed images of these regions are depicted in colour image obtained by the 3D digital microscope, and Fig. 9 the following rows plots of the 3D data, of the respective imaging devices. Figure 9a shows a small detail, the Pearl 7.0 mm × 6.4 mm Highlight, measuring , 9b a part of the outside the measurement uncertainty boundary of this 22.0 mm × 23.2 mm yellow/green Jacket measuring , and device, might be attributed to a data processing segmen- 21.6 mm × 21.0 mm 9c the Left Eye measuring . All height tation error [33]. 200 µm maps are plotted on a scale between 0 and . All If this segmentation fails (layers cannot be clearly dis- height maps depicted in Fig. 9 show relevant topographi- tinguished), it will introduce height map artifacts. Typi- cal details like (fattened) impasto although this efect is cally, if this efect occurs it leads to local artefacts that more difcult to distinguish in 9b, and 9b, c also clearly can be classifed as outlier data. However, if this occurs show the craquelure pattern of their particular regions; along the boundary of a tile, this could lead to a stitching these are larger areas than 9a. Figure 9b also shows an artefact, that propagates over multiple tiles in the height indentation, multiple of which can be found across the map. surface of the painting. A limitation of using a metallic reference target is that some imaging artefacts are introduced in the data, that are rare or non-existent in the painting scan data. Te Discussion refections caused by the metal surface frequently over- Scanning the metallic reference target saturate the MS-OCT spectrometer and this results in Te measurements of the reference target show rea- measurement errors. In case of scanning a (larger) metal sonable small error, as compared to the highly accurate object (automatically), the oversaturation efect might NPS measurements, and acceptable standard deviations also hamper the auto-focus algorithm. Another conse- for the features we try to capture. Te results are in line quence of its metallic surface is, that the target does not with the expected measurement uncertainty of the dif- refect light difusely. For this reason, both 3D scanning ferent measurement devices. We fnd that the 3D digi- systems were used in a non-cross-polarised setup, which tal microscopy and MS-OCT have the highest accuracy led to extra noise in the height data (stereo mis-matching and precision, compared to the 3D scanning systems features, due to specular refections). Furthermore, with based on fringe-encoded stereo imaging. We believe that the High-Res 3D scan we experienced what looked like 500 m the larger error of MS-OCT at the µ groove, lying thin-flm interference patterns, leading to further stereo Elkhuizen et al. Herit Sci (2019) 7:89 Page 13 of 22

(See fgure on next page.) Fig. 9 De-trended and aligned heightmaps of three regions of interest created by four scanning systems. a Pearl Highlight, b Jacket, and c Left Eye. The colour images are the stitched images using the 3D digital microscopy colour data

mis-matching. Tis mis-match can be explained by the able to make a meaningful comparison between the data (thin) protective coating on the reference target. Fur- sets. In both situations—between these diferent scan- thermore, the measured results are only indicative of the ning techniques and over time scan comparisons—care depth determination within a single tile of the measure- should be taken with accounting for the global shape. If ment technique (specifcally for the 3D digital micros- this global shape removal is done too rigorously, valu- copy and MS-OCT, as they cannot capture within one able information might be lost about the painting’s tile). shape (comparison or changes). On the other hand, if the removal is not rigorous enough, comparison might not Challenges with (in‑situ) painting scanning lead to any valuable insights. When we compare the scan results of the case study painting (see Fig. 9), we fnd that all three scanning tech- Alignment and stitching (of very large datasets) niques are capable of measuring its topography, captur- Te alignment between data sets was performed using ing details at the level of individual cracks. Te data sets only the 3D data, aligned at the pixel level, applying only of the three imaging techniques show broad similarities rigid transformations. A consequence of such an align- in spatial layout (XY) and height values (Z). However, the ment approach is that, if a data has lateral distortions (e.g. Std-Res 3D scan, misses the fne craqualure details, for caused by calibration errors or imaging system imperfec- instance seen on the Pearl Highlight (see Fig. 9a). Another tions), the features of the height maps will not align per- diference between the painting scan results is the subtle fectly. Whether such systematic errors occur remains to variations in the global shape for the small ROI plotted be investigated. It then also remains to be investigated in Fig. 9 (e.g. relatively lighter or darker features between if a sub-pixel, non-rigid alignment can be achieved, to the maps), despite the de-trending. (Potential) reasons improve comparison results (e.g. following the 2D align- for these and other diferences, and the subsequent dif- ment approach applied to other types of multi-modal fculties with comparing the height data are discussed on painting scan data in the Bosch Research and Conser- the following sub-sections. vation Project [42]). Additionally, as the painting was measured at diferent resolutions—and therefore needs Scanning non‑rigid paintings re-sampling—sampling errors can also infuence the One difculty with scanning and comparing data, spe- comparison between height data. cifcally for canvas paintings, is the fact that they have a Furthermore, as all systems ofer more than one imag- fexible substrate. Actions like unframing, handling, and ing modality—such as colour information—these might clamping a canvas painting onto an easel—all typical also be included in the alignment algorithm, potentially steps for a technical examination campaign—infuence leading to more robust alignment results. However, care the overall shape of a painting’s canvas. In this case study, should be taken with this, as it will almost certainly lead due to limitations imposed by the project, the painting to a diferent result, which then leads to the issue of was also moved between the diferent scans, inevitably determining which result is better: all are merely based leading to variations in global canvas shape. Addition- on optimisation. For instance, in the case of using col- ally, the painting’s orientation difered between the our data, robustness of the algorithm to illumination scans, which most certainly will also have had an efect diferences—leading to diferences in shadowing and on its shape. Tis means that there were diferences in shading—needs to be assured. how gravitational forces and forces of the (easel) fxture, On the scale of the ROI, selected from our case study infuenced the global shape of the painting. For example, painting, only MS-OCT and 3D digital microscopy are it is very likely that the central part of the painting will made up of merged tiles. For the stitched data sets, which sag more in the horizontal orientation (of the 3D digi- was either based on XY-axis displacement (MS-OCT, tal microscopy setup) compared to the vertical orienta- no blending), or done by the Hirox e-tiling software (3D tions (of the other three scans). Tese diferences might digital microscopy, method unknown) no obvious tiling be avoided in a more controlled experiment. However, a artefacts were observed. It remains to be determined, similar situation will arise when two data sets are com- which strategy is the most suitable for tile stitching and/ pared over time (i.e. for monitoring purposes). For these reasons, we will have to deal with these variations to be Elkhuizen et al. Herit Sci (2019) 7:89 Page 14 of 22 Elkhuizen et al. Herit Sci (2019) 7:89 Page 15 of 22

or blending, especially in the case when data in the over- created by the raking light illumination. Tis can be (par- lapping regions are not in agreement with each other. tially) alleviated by additional low intensity ring lighting. With the stitching of (very) large datasets, we also run However, both the raking light illumination and the ring into the issue of error propagation, in the lateral (XY) as lighting can cause light to refect directly into the lens, lead- well as axial (Z) direction. If tiles a stitched in a sequential ing to over-exposure. For these pixels the 3D digital micro- manner, small errors in misalignment (XYZ) can lead to scope will not be able to reconstruct the 3D topography, much larger errors across the complete surface. Currently leading to missing data. Careful positioning of the raking no topographical scans were made at the scale of the com- light and a low-level ring lighting minimise this efect. plete painting, or intermediate resolutions, which might be used for a global-to-local optimisation step in align- Measuring dark regions ment, (potentially) minimising the infuence of local errors As all imaging techniques rely on optics, they all have dif- and the potential for error propagation. We envision that fculty scanning areas where (almost) all light is absorbed, an approach like the one used in the Bosch Research and and therefore the spectral refectance is low (i.e. dark/ Conservation Project [42], might be extended for topo- black areas). In scanning very dark areas, the 3D scan graphical data, also dealing with the height (Z) data. systems have scanning artifacts, showing more sporadic Furthermore, we encountered issues with stitching the and noisy data. Although not specifcally the case in this High-Res 3D data set, as our current approach to stitch- painting, another limitation of the 3D scanning algorithm ing (as is used in for instance [20]), is limited by its mem- is that it relies on (some) salient features to reconstruct ory requirements for such large data sets. Also the Hirox the topography. If scanning tiles completely lack any fea- e-tiling software did not allow stitching the larger ROI at tures (completely uniform in colour), the algorithm can full resolution. fail to reconstruct the 3D shape. Given the currently unsolved challenges with the non- rigid nature of canvas paintings (also encountered in the Vibrations and illumination conditions during scanning case study data sets), and the various challenges related Other in-situ conditions in which Girl with a Pearl Ear- to alignment and stitching (i.e. sub-pixel, non-rigid align- ring was scanned also potentially infuence the scanning ment, (multi) modal alignment considerations, dealing with results. For instance, vibrations were not measured dur- stitching and blending of overlapping regions, error propa- ing the scans and only to a limited extent controlled for. gation, memory issues), we deem large-scale painting data Te High-Res 3D scanner and 3D digital microscope height comparison to be beyond the scope of this paper. were used during opening hours with museum visitors present in the same room as the technical examination, Measurement occlusions and specular refections ofering multiple potential sources of vibrations afect- All techniques have limitations in the terms of measurement ing the scanner and/or painting. For the case of the 3D occlusions. Firstly, none of the methods are (currently) able digital microscope system we expect that this will have to capture ‘overhangs’ or undercuts on a surface, meaning had a limited infuence, as the system (including paint- they cannot imaging underneath faking or cupping paint, ing) was placed on a low-vibration table and features or capture the underside of overhanging impasto (not typi- active vibration compensation for the focus stacking. cally present in Golden Age paintings). Although MS-OCT Te MS-OCT and Std-Res 3D scans were both captured would be able to detect these layer transitions, currently the during the night, limiting the vibrations induced by the segmentation for this is not incorporated in the data pro- environment. cessing. It could however be made visible in a virtual cross- Te fact that visitors were able to view the examination section using this data. Also the current method of data also introduced another potential source of measure- representation—as a 2D image—does not allow for repre- ment error, namely external illumination. Specifcally for senting these types of 3D structures. Furthermore, there are the High-Res 3D scan the external lighting (refecting of some limitations imposed by the imaging and illumination the surface) potentially infuenced the height measure- angles. Both the MS-OCT and the 3D scan systems image ment, and might explain some measurement artifacts (i.e. the surface under an angle (indicated by imaging angle β in measurement artefact of a repeating fringe pattern show- Figs. 2a and 3a, whereby deep cracks or the sides of highly ing up in some tiles). sloped areas (i.e. of an impasto) might be occluded from view, and therefore not measured. In the case of the 3D Diferences between 3D scanning techniques scan systems this might be alleviated by adding additional Lateral and axial resolution diferences cameras, providing additional imaging angles. In the 3D Each of the techniques and their implementation have digital microscopy occlusions can occur in the areas where quite diferent lateral (XY) and axial (Z) resolutions, the shading or shadows are too dark (i.e. under-exposure), where the relationship between these resolutions and Elkhuizen et al. Herit Sci (2019) 7:89 Page 16 of 22

level of fexibility in choosing a resolution are also dif- ferent. For the MS-OCT system the anisotropic lateral 8.53 m × 67.56 m resolution is µ µ /pixel and the axial 3.0 m sampling resolution is µ . Tese resolutions are lim- ited by system optics for the lateral resolution and by the broadband source for the axial resolution and are con- sequently decoupled. Te technique therefore does ofer the capability to measure at diferent lateral resolutions. For the 3D scanning the lateral and spatial resolution of the system is more fexible (in this case the High-Res 7 m 3D scan has a lateral resolution of µ and an axial reso- 17.5 m lution of µ , and the Std-Res 3D scan respectively Fig. 10 Relative scale of scan tiles. Largest tiles are the Std-Res 3D 23.1 m 27.5 m µ and µ ). Te axial resolution (Z) is deter- scan, followed by High-Res 3D scan, then the MS-OCT, and the 3D mined by the imaging angle ( β in Fig. 3) and the lateral digital microscopy generating the smallest tiles resolution (XY) (determined by the imaging distance and camera/lens combination). Te axial resolution can be increased by either imaging at a higher magnifcation, or by increasing the imaging angle. However, with increas- scan system only several hours to scan the complete sur- ing magnifcation, the total depth-of-feld will become face. Here we should note that these are not commercial smaller. Tis should, however, remain large enough to systems, and optimisation might still lead to even faster capture the full height variation of the surface within a acquisition. single tile. Te imaging angle is limited by the maximum Automated scanning range tilt angle of the lenses. In short, the 3D scanning tech- nique ofers quite some fexibility in term of resolution, Although all systems are portable (allowing in-situ but is limited by the minimal needed depth-of-feld, cam- scanning), their automated scanning range difers sub- era and lens parameters. stantially. Tese diferences are not directly technically For the 3D digital microscopy the ability to calculate limited, as all imaging techniques might be ftted on the topography is dependent on a very small depth-of- larger movement axes. Of all of the techniques, the 3D feld of all images in the focal stack. An axial resolution digital microscope would be most demanding in terms suitable of capturing details of a painting like Girl with a of step size accuracy (sub-millimeter accuracy in XY), Pearl Earring can therefore only be created at relatively whereas for the MS-OCT and 3D scan systems a lat- high magnifcations. Tis means that reaching a reason- eral stepping in the millimeter range would sufce (if able axial resolution also results in a small scanning tiles relying on other tile alignment strategies in the case of 2.1 mm × 1.31 mm ( here). MS-OCT). An additional limitation of the current 3D digital Scalability microscope confguration is that the size of the painting Te scalability of the diferent techniques is in part gov- to be scanned is limited by the dimensions of the sur- erned by the limitations in resolutions and the (related) rounding frame. Tis could be alleviated by mounting tile size (for relative scale between the imaging systems the microscope on a vertical frame. Tis would also limit see Fig. 10). Given that the microscope tiles are only the amount of dust particles and fbres that settle on the 2.1 mm × 1.31 mm it becomes very time consuming painting during scanning, which are currently visible in and data intensive to scan complete paintings. Even for the 3D digital microscopy data. a relatively small painting like Girl with a Pearl Ear- 1.1 m Aligning and focusing scanning systems ring, a scan at a lateral resolution of µ would lead 355 × 103 404 × 103 to an image of roughly by pixels (a One of the difculties with scanning paintings is the lack 144 gigapixel image). Based on the settings as used in of a-priori knowledge of the surface shape. Tis is rel- the case study this would take roughly 200 h to scan the evant on the scale of the complete painting (i.e. warp- complete painting. Dealing with the sheer size of data at ing) and at a smaller scale (i.e. impasto). For the artifact such resolutions (also in terms of alignment and stitch- to be scanned accurately, it has to be captured within in ing) also remains challenging to this day. Both the MS- the focused range (i.e. large enough depth-of-feld). Te MS-OCT system currently features an auto-focusing OCT and 3D scan systems can acquire the data at a faster 100 mm rate (mainly related to the relative tile size), where the functionality with a range of , assuring alignment MS-OCT system took four nights and the High-Res 3D for every tile; however, due to the imaging angle, the Elkhuizen et al. Herit Sci (2019) 7:89 Page 17 of 22

total height diference within a single tile cannot exceed rheological properties of the materials he used, and sub- 1.87 mm . With the High-Res 3D scan the depth-of-feld sequent efects of past treatments and aging. Figure 9a, c 1.0 mm is approximately . A careful alignment is therefore shows that the top of the white Pearl Highlight but also needed to assure the complete painting is in focus during in the white highlight in the Left Eye—which presum- scanning. Automated axial alignment might be needed to ably would have been rounded or slightly pointed when overcome larger height variations in paintings with more Vermeer painted it—is now planar and almost level with extensive warping. For the 3D digital microscopy the the surrounding paint. Directly around the impasto is a 200 m focal range need to be set for the complete scanning area, ‘moat’, for the Pearl Highlight approximately µ deep assuring every point lies within this focal range. Tis is (as measured using all three techniques), which formed also important for the total scanning time. Setting the when the paint surrounding the highlight was displaced boundaries too wide leads to an unnecessary increase of as the impasto was fattened. Tis suggests that Ver- the scanning time, while setting the boundaries too nar- meer’s original impasto would have matched or exceeded rowly runs the risk of not capturing certain areas as they the current depth of the moat to be able to make such remain out of focus. Also here automated axial alignment an impression. It appears that lining had a less efect on could help to efciently scan paintings with extensive the impasto with lower topography—for example, in the warping. Jacket—as there is no perceptible moating (see Fig. 9b). Tis data is useful for understanding Vermeer’s painting Scanning additional modalities technique, but also to assess the consequences of wax– All three techniques ofer the possibility to provide other resin lining. modalities in addition to scanning the topography. Te Te efects of using the so-called ‘Pettenkofer’ pro- MS-OCT—as is actually its main application—is capa- cess—to fatten faking paint—can also be identifed ble of mapping the stratigraphy of the (semi-)transparent for instance in Fig. 11e, using a colour map for visual paint layers, providing information about varnish- and enhancement. Te afected area of the forehead is glaze-layer thickness as well as the 3D topographies of deeper than the surrounding paint. We can also see these layers. Te 3D scanners and 3D digital microscopy in these visualisations that the cracks in the forehead both ofer colour information in addition to topography, appear to have soft, rounded edges, and the cracks which for both techniques are one-to-one intrinsically appear wider than the sharp-edged cracks throughout aligned data sets. MS-OCT, however, does not provide most of the painting. Tese scanning techniques could such colour information. Te lack of this information can therefore potentially be used to discover other regions be overcome by mapping the topographical (and stratig- that might have been treated using the Pettenkofer raphy) data to a colour image, allowing enhanced data method. It should be mentioned that determining the interpretation and ease of localisation on the artwork. exact shape (roundness/sharpness of cracks) is limited Furthermore, the Std-Res 3D scan system is capable of by the scanning resolution and measurement occlu- measuring additional modalities of a painting’s appear- sions imposed by the scanning systems. ance, namely measuring the spatially-varying gloss of the Visualisations that can be created using 3D data are surface [20]. Tis has the potential to ofer an even richer an improvement on the ways that conservators have documentation of the appearance and a painting’s state. traditionally been able to document and view craque- lure. Figure 11b show the cracks that were visible with Interpretation of Girl with a Pearl Earring data the naked eye in 1994, which the conservator traced Te scanning techniques described in this case study onto a transparent polyester flm. In the dark back- have the potential to help a conservator evaluate the ground, the dark cracks were more difcult to see, topography, condition, authenticity and conservation his- which gives the impression that the cracking might tory of an object. Tey provide a documentation of the be less pronounced or severe in the background than topography of Girl with a Pearl Earring at one specifc in the face. In comparison, Fig. 11d, e shows that the moment in time. Tis data already provides informa- relative amount of cracking is much more similar when tion about Vermeer’s technique, and the condition of the comparing the face to the background. Although rak- painting. ing light photographs (segments shown in Figs. 11a and For example, the three ROI compared as part of this 12a) are still useful for visualising the topography, it can case study—the Pearl Highlight, Jacket and Left Eye (see be difcult to interpret because the colour of the paint Fig. 9)—contain small details that Vermeer painted more can afect the visibility of the craquelure. With 3D scan- thickly than the surrounding paint (i.e. impasto). Visu- ning, the topography can be rendered as an image with- alising their height and topography in 3D reveals how out the colour information, only showing the height much paint Vermeer loaded onto his brush, and the variations (see for comparison Fig. 11d, e and detailed Elkhuizen et al. Herit Sci (2019) 7:89 Page 18 of 22

Fig. 11 Original documentation and renderings, visualizing surface height variations of ‘Girl with a Pearl Earring’. Top row shows the original documentation, part of a of raking light photograph (see Fig. 1a), and b crack tracing image (see Fig. 1b). Bottom row: rendering of 3D data from Std-Res 3D scan (c) using colour and topography data, (d) rendered as a matte, white surface, and (e) rendered using a colour map, to enhance the height variations. Note that for the renderings the height variations are exaggerated relative to the lateral scale, to increase their visibility Elkhuizen et al. Herit Sci (2019) 7:89 Page 19 of 22

Fig. 12 Details of original documentation and renderings, visualizing surface height variations of ‘Girl with a Pearl Earring’. Top row shows the original documentation, detail of a of raking light photograph (see Fig. 1a), and b crack tracing image (see Fig. 1b). Middle row: rendering of 3D data from 3D digital microscope scan (c) using colour and topography data, and (d) rendered as a matte, white surface. Bottom row: rendering of 3D data from High-Res 3D scan (e) using colour and topography data, and (f) rendered as a matte, white surface. Note that for the renderings the height variations are exaggerated relative to the lateral scale, to increase their visibility

views in 12c–f). Virtual relighting of the topography— visualizing individual cracks). However, detailed inves- rendering it as a matte, white surface—allows a con- tigation of the scan results show that resolution of the servator to see the topography more clearly than on a Std-Res 3D scan is not sufcient to capture the fner colour image. It could be used to recognise vulnerable cracks, which are of interest (see for instance Fig. 9a). areas—e.g. where the paint is lifting, faking or tent- Measurements of a reference target, also show that ing—and these areas could be earmarked to be checked these techniques have sufcient accuracy and precision regularly. to measure these fne details, although possibly not enough for capturing the fnest details with the Std-Res Conclusions 3D scan. From these measurements we can also con- Based on the case study presented in this paper—3D clude, as expected, that multi-scale optical coherence scanning Girl with a Pearl Earring by Johannes Ver- tomography (MS-OCT) and 3D digital microscopy meer—we conclude that all techniques are capable of ofer the highest accuracy and precision, as compared capturing the spatially-varying topographical features to the 3D scanning systems (based on fringe-encoded of such a painting at a relevant scale (i.e. capable of stereo imaging). We fnd that the standard deviations Elkhuizen et al. Herit Sci (2019) 7:89 Page 20 of 22

on the measurements of the reference target all lie measurement of larger (and potentially more warped) within the range that would be expected based on their paintings. As 3D scanning—based on fringe-encoded ste- respective measurement uncertainty. Although the reo imaging—appears to be the most promising in terms MS-OCT and 3D digital microscopy ofer higher meas- of scan speed to capture complete paintings, we suggest urement accuracy and precision (in the height meas- further investigation into increasing the scanning accu- urement), the single measurement areas (tiles) of these racy, precision and robustness, by for instance improving systems are very small. Tis afects the scanning speed the calibration and imaging strategy in combination with and thereby limits their suitability to scan complete further improving the data processing algorithms. paintings, even at the size of Girl with a Pearl Earring It is envisioned that 3D scans made at periodic inter- 39 cm × 44.5 cm ( ). Also the stitching of these tiles to vals have the potential to monitor long-term changes create larger topographical maps potentially increases that occur in an artwork: for instance, changes that might the measurement error. occur if there is a rapid change in temperature or relative Moreover, the results show the capabilities of all four humidity, or as a result of frequent handling or transport. systems to combine high-resolution capture of the sur- Depending on the frequency of monitoring, this could face topography, with a large planer scale, surpassing provide information about the speed and conditions existing painting scanner capabilities, either on resolu- in which these changes can occur. A 3D scan of an art- tion or scale. work created before and after a conservation treatment Given the level of detail captured (for instance shown (or of a reconstruction made using historically appropri- in Fig. 9), we argue that for the description of topography, ate materials) can also be used to reveal the efect of such the 3D data provides much richer and complete informa- treatments. Tese could involve methods that are still tion than conventional documentation techniques, and practiced (like locally fattening lifting paint with heat), which could not be reached with existing 3D scanning and those (like wax–resin lining) that are seldom used systems. Tis moves the analysis of the topography from by conservators nowadays [43]. As the used 3D imag- a subjective and—in the case of a tracing image—binary ing systems were built with standard equipment (lamps, visualisation of the craquelure, to an objective measure- cameras, projectors, microscope) that are well docu- ment of these fne details. We believe this to be true for mented by the supplier and software based on standard- all 3D scanning techniques shown in this paper. We show ised image processing and calibration procedures (using how such (objective) 3D data of a painting can be inter- either Matlab, LabView of Hirox supplied software) the preted and used (e.g. for conservation purposes) based scanners can be rebuilt in the future with the same qual- on case study data, exemplifed by three ROI and ren- ity. However, currently no (dedicated) software exists for dered visualisations of a larger region and detail of the data comparison (over time), which can adequately deal painting’s surface, showing topographical features like with the global shape variations nor the data size. It also cracks, moating and efects of paint consolidation. remains to be investigated if the accuracy of these sys- tems is sufcient for monitoring paintings over time, as Future work currently no data exists on the magnitudes of topograph- Te development of approaches to deal with high-reso- ical changes occurring in paintings, including those kept lution 3D scanning data of non-rigid artefacts, like paint- under museum conditions. ings, is considered a crucial next step to enable (over Topographical data of paintings might also be used for time) data comparison of paintings. Further investigation (more accurate) crack detection (e.g. extending studies into alignment and stitching strategies capable of large- like [7]), a painter’s brushstroke analysis, or (automatic) scale, non-rigid (sub-pixel) alignment in XY and Z are identifcation of treatment efects/degradation issues (e.g. also considered a pre-condition for meaningful compar- recognising efects of the ‘Pettenkofer’ process). ison of painting data sets, such as collected in this case Documenting the painting’s appearance and condition study. Also the use of multi-modal scan data should be is important, as the artwork itself will inevitably degrade given further consideration, to potentially improve data further in the centuries to come, unfortunately probably alignment. even up to the point of total disintegration, no matter the Te applicability of 3D scanning systems for scanning conservation eforts. 3D data might be used to create 3D complete paintings, such as discussed in this paper, might (printed) reproductions, and could also serve as a start- be further improved by increasing the scanning speed ing point for (digital) reconstructions, showing past (and and automated scanning range. Furthermore, the imple- future) states of an artwork (i.e. removing or extrapolat- mentation of an auto-focus or auto-alignment function- ing craquelure efects). ality of the focal plane for the 3D digital microscopy and 3D scanning systems will be needed to ensure accurate Elkhuizen et al. Herit Sci (2019) 7:89 Page 21 of 22

Abbreviations Received: 10 July 2019 Accepted: 18 October 2019 3D: three-dimensional; AR: aspect ratio; High-Res 3D scan: high resolution 3D scanning, referring to 3D scanning system employing fringe-encoded stereo imaging, with a lateral sampling resolution of 7 µm × 7 µm; MS-OCT: multi-scale optical coherence tomography; NPS: (Hirox) Nano Point Scanner; OCT: optical coherence tomography; RGB: Red, Green and Blue, referring to References the primary channels using in (digital) photography; SD-OCT: spectral-domain 1. Hermans J, Osmond G, Van Loon A, Iedema P, Chapman R, Drennan optical coherence tomography; Std-Res 3D scan: standard resolution 3D scan- J, Jack K, Rasch R, Morgan G, Zhang Z, Monteiro M, Keune K. Electron ning, referring to 3D scanning system employing fringe-encoded stereo imag- microscopy imaging of zinc soaps nucleation in oil paint. Microsc Microa- ing, with a lateral sampling resolution of 25 µm × 25 µm; UV: ultraviolet; nal. 2018;24:318–22. https​://doi.org/10.1017/S1431​92761​80003​87. WLCP: white light confocal proflometry. 2. van Asperen de Boer JRJ. An introduction to the scientifc examination of paintings. In: Nederlands Kunsthistorisch Jaarboek (NKJ) / Netherlands Acknowledgements Yearbook for History of Art, vol. 26. 1975. p. 1–40. The project The Girl in the Spotlight is a Mauritshuis initiative, and involves a 3. Legrand S, Vanmeert F, Van der Snickt G, Alfeld M, De Nolf W, Dik J, team of internationally recognised specialists working within the collaborative Janssens K. Examination of historical paintings by state-of-the-art framework of the Netherlands Institute for Conservation Art Science (NI hyperspectral imaging methods: from scanning infra-red spectroscopy + + + CAS), with some scientists from other institutions. The authors would like to to computed X-ray laminography. Herit Sci. 2014;2(13):1–11. https​://doi. thank Hirox Japan (Mr. Takeuchi and Mr. Nakajima) for modifying the RH-2000 org/10.1186/2050-7445-2-13. software specifcally for this project, allowing extended XYZ-movement 4. Wadum J. The Girl with a Pearl Earring—restoration history. In: Vermeer control and ofine stitching, and Mr. V. Sabatier (Hirox Europe) for his help Illuminated: a report on the restoration of the view of Delft and The in the construction and alignment of the Hirox bridge frame as well as his Girl with a Pearl Earring by Johannes Vermeer. Mannady: VK Publishing/ support during the 3D digital microscope scan. The authors would also like to Inmerc; 1995. pp. 18–21. Chap. 3.1. thank Ms. T.T.W. Essers M.Sc. and Mr. M. van Hengstum M.Sc. for their support 5. Bucklow S. The description and classifcation of craquelure. Stud Conserv. with carrying out the 3D scanning. The authors would also like to thank Océ 1999;44(4):233–44. https​://doi.org/10.2307/15066​53. Technologies B.V., a Canon Company, for their generous contribution to mak- 6. Bucklow S. Cracks and the perception of paintings. https​://www.hki.ftzm​ ing the High-Res 3D scanning system possible. useum​.cam.ac.uk/proje​cts/crack​s. Accessed 15 May 2019. 7. El-Youssef M, Bucklow S, Maev R. The development of a diagnostic Authors’ contributions method for geographical and condition-based analysis of artworks using We gathered and (pre-)processed the Standard Resolution 3D scan data, and craquelure pattern recognition techniques. Insight Non-Destructive pre-processed the High Resolution 3D scan and 3D Digital Microscopy data, Testing Condition Monit. 2014;56(3):124–30. https​://doi.org/10.1784/ to prepare them for alignment. WE carried out the literature review, and was a insi.2014.56.3.124. major contributor in writing the manuscript. TC gathered and (pre-)processed 8. Spagnolo SG. Virtual restoration: detection and removal of craquelure the MS-OCT data, carried out the alignment, and was a contributor to writing in digitized image of old paintings. In: Optics for arts, architecture, of the manuscript. AV was a contributor in writing of the manuscript regarding and archaeology III. Munich: SPIE Optical; 2011. p. 1–9. https​://doi. the introduction, case study description, and data interpretation, as well as org/10.1117/12.88829​9. suggesting revisions to the manuscript. EL gathered the 3D Digital Microscopy 9. Blais F, Taylor J, Cournoyer L, Picard M, Borgeat L, Dicaire L-G, Rioux M, Ber- data, and contributed to the manuscript, regarding this technique. YS, SP, JG aldin J-A, Godin G, Lahnanier C, Aitken, G. Ultra-high resolution imaging and JD contributed in the supervision of the work carried out, and sug- at 50 micron using a Portable XYZ-RGB Color Laser Scanner. In: Baltsavias gested revisions to the manuscript. All authors read and approved the fnal M, Gruen A, van Gool L, Pateraki M, editors. International workshop on manuscript. recording, modeling and visualization of cultural heritage. Ascona: CRC Press; 2005. p. 1–16. https​://nrc-publi​catio​ns.canad​a.ca/eng/view/accep​ Funding information ted/?id 3a76e​dd0-57ca-47ef-8c89-c2278​bd1d7​3e. The Netherlands Institute for Conservation Art Science funded the 10. Blais F, Taylor= J, Cournoyer L, Picard M, Borgeat L, Godin G, Beraldin J-A, + + + participation of the NICAS partners in the project, including use of analytical Rioux M, Lahanier C. Ultra high-resolution 3D laser color imaging of equipment and the time devoted to the project by scientists from the Cultural paintings: the ’Mona Lisa’ by Leonardo da Vinci. In: Castillejo M, Moreno P, Heritage Agency of The Netherlands (RCE), Delft University of Technology Oujja M, Radvan R, Ruiz J, editors. 7th international conference on lasers (TU Delft), University of Amsterdam (UvA) and the Rijksmuseum. The Girl in in the conservation of artworks. Madrid: CRC Press; 2007. p. 1–8. https​ the Spotlight project was made possible with support from the Johan Maurits ://nrc-publi​catio​ns.canad​a.ca/eng/view/accep​ted/?id 8f3c5​c14-2ab8- Compagnie Foundation. The high resolution 3D scanning system was made 47c4-865c-e1c76​d35ef​ d. = possible with support from Océ Technologies B.V., a Canon Company. 11. Verus Art: Verus Art textured reproductions (2017). www.verus​art.com. Accessed 15 May 2019. Availability of data and materials 12. Jackson MK, MacDonald LW. Color management in 3D fne-art painting The datasets used and/or analysed during the current study are available from reproduction. In: Color and imaging conference. Vancouver: Society the corresponding author on reasonable request. The datasets supporting the for Imaging Science and Technology; 2018. p. 396–401. https​://doi. conclusions of this article are included within the article and its additional fles. org/10.2352/ISSN.2169-2629.2018.26.396. 13. Factum Arte: Lucida: discovering an artwork through its surface. Technical Competing interests report, Factum Foundation, Madrid, Spain, 2016. http://www.factu​m-arte. The authors declare that they have no competing interests com. 14. Zalewski D. The factory of fakes—how a workshop uses digital technol- Author details ogy to craft perfect copies of threatened art. In: The New Yorker. 2016; p. 1 Industrial Design Engineering, Delft University of Technology, Landberg- 66–79. straat 15, 2628 CE Delft, The Netherlands. 2 Department of Imaging Physics, 15. Van Gogh Museum. Van Gogh museum—Relievo collection. https​:// Delft University of Technology, Lorentzweg 1, 2628 CJ Delft, The Netherlands. www.vango​ghmus​eum.nl/en/busin​ess/relie​vo-colle​ction​. Accessed 13 3 Department of Materials Science, Delft University of Technology, Mekelweg Oct 2017. 2, 2628 CD Delft, The Netherlands. 4 Hirox Europe/Jyfel Corporation, 300 Route 16. Akca D, Grun A, Breuckmann B, Lahanier C. High defnition 3D-scanning Nationale 6, Le bois de côtes, Bat. A, 69760 Limonest, France. 5 Conserva- of arts objects and paintings. In: Gruen A, Kahmen H, editors. Optical 3-D tion Department, Mauritshuis, Plein 29, 2511 CS Den Haag, The Netherlands. measurement techniques VIII, vol. II. Zurich, Switzerland; 2007. p. 50–8. 6 Department of Conservation and Restoration, University of Amsterdam, 17. Karaszewski M, Adamczyk M, Sitnik R, Michoński J, Załuski W, Bunsch E, Johannes Vermeerplein 1, Amsterdam, The Netherlands. Bolewicki P. Automated full-3D digitization system for documentation of paintings. In: Pezzati L, Targowski P, editors. Optics for arts, architecture, Elkhuizen et al. Herit Sci (2019) 7:89 Page 22 of 22

and archaeology IV, vol. 8790. Munich, Germany; 2013. p. 1–11. https​:// 30. Liang H, Cucu R, Dobre GM, Jackson DA, Pedro J, Pannell C, Saunders doi.org/10.1117/12.20204​47. D, Podoleanu AG. Application of OCT to examination of easel paint- 18. Breuckmann B. 3-dimensional digital fngerprint of paintings. In: 19th ings. Second Eur Workshop Opt Fibre Sens. 2004;5502:378. https​://doi. European signal processing conference. Barcelona: IEEE; 2011. p. 1249–53 org/10.1117/12.56678​0. 19. Zaman T, Jonker P, Lenseigne B, Dik J. Simultaneous capture of the color 31. Yang ML, Lu CW, Hsu IJ, Yang CC. The use of optical coherence and topography of paintings using fringe encoded stereo vision. Herit tomography for monitoring the subsurface morphologies of archaic Sci. 2014;2(23):1–10. https​://doi.org/10.1186/s4049​4-014-0023-0. jades. Archaeometry. 2004;46(2):171–82. https​://doi.org/10.111 20. Elkhuizen WS, Essers TTW, Song Y, Geraedts JMP, Pont SC, Dik J. Gloss, 1/j.1475-4754.2004.00151​.x. color and topography scanning for reproducing a painting’s appearance 32. Targowski P, Iwanicka M. Optical coherence tomography: its role in using 3D printing. J Comput Cultural Herit. 2019;12(4):1–19. the non-invasive structural examination and conservation of cultural 21. van Hengstum MJW, Essers TTW, Elkhuizen WS, Dodou D, Song Y, Gerae- heritage objects—a review. Appl Phys A. 2012;106:265–77. https​://doi. dts JMP, Dik J. Development of a high resolution topography and color org/10.1007/s0033​9-011-6687-3. scanner to capture crack patterns of paintings. In: Sablatnig R, Wimmer M, 33. Callewaert T, Dik J, Kalkman J. Segmentation of thin corrugated layers in editors. Eurographics workshop on graphics and cultural heritage. Vienna: high-resolution OCT images. Opt Express. 2017;25(26):1–13. https​://doi. The Eurographics Association; 2018. p. 1–10. https​://doi.org/10.2312/ org/10.1364/OE.25.03281​6. gch.20181​336. 34. Boon J, Van der Weerd J, Keune K, Noble P, Wadum J. Mechanical and 22. Cacciari I, Nieri P, Siano S. 3D digital microscopy for characterizing punch- chemical changes in Old Master paintings: dissolution, metal soap forma- works on medieval panel paintings. J Comput Cultural Herit. 2014;7(4):1– tion and remineralization processes in lead pigmented ground/interme- 15. https​://doi.org/10.1145/25944​43. diate paint layers of 17th century paintings. In: ICOM-CC, 13th Triennial 23. Van den Berg KJ, Daudin M, Joosten I, Wei B, Morrison R, Burnstock A. Meeting, Rio de Janeiro, Brazil. 2002. p. 401–6. A comparison of light microscopy techniques with scanning electron 35. Schmitt S. Examination of paintings treated by Pettenkofer’s process. Stud microscopy for imaging the surface of cleaning of paintings. In: 9th Conserv. 2014;35:81–4 Examination of paintings treated by Pettenkofer’s international conference on NDT of Art, Jerusalem, Israel; 2008. p. 25–30 process. 24. Del Sette F, Patané F, Rossi S, Torre M, Cappa P. Automated displacement 36. National Instruments: LabVIEW. http://www.ni.com/nl-nl/shop/labvi​ measurements on historical canvases. Herit Sci. 2017;5(21):1–12. https​:// ew.html. Accessed 3 Jul 2019. doi.org/10.1186/s4049​4-017-0135-4. 37. Bouguet J-Y. Camera calibration toolbox for Matlab. 2013. http://www. 25. Palma G, Pingi P, Siotto E, Bellucci R, Guidi G, Scopigno R. Deformation visio​n.calte​ch.edu/bougu​etj/calib​_doc/. Accessed 29 Oct 2019. analysis of Leornado da Vinci’s “Adorazione dei Magi” through temporal 38. Mitani Cooperation Visual System: e-tiling software. https​://www.mitan​ unrelated 3D digitization. J Cultural Herit. 2019;38:1–12. https​://doi. i-visua​l.jp/en/produ​cts/image​_gener​ation​_obser​vatio​n_suppo​rt/e_tilin​ org/10.1016/j.culhe​r.2018.11.001. g/. Accessed 3 Jul 2019. 26. Zaman T. Development of a topographic imaging device for the near- 39. SciPy for Python. https​://www.scipy​.org. Accessed 3 Jul 2019. planer surfaces of paintings. Master thesis, Delft University of Technol- 40. OpenCV. https​://openc​v.org. Accessed 3 Jul 2019. ogy, Delft, The Netherlands. 2013. http://resol​ver.tudel​ft.nl/uuid:bd71a​ 41. Rubert & Co Ltd: Precision reference specimens. 2019. http://www.ruber​ 192-eaa8-4f90-8778-b18f8​6cac7​9c. Accessed 29 Oct 2019. t.co.uk/refer​ence-speci​mens/. Accessed 2 May 2019. 27. Cheung CS, Spring M, Liang H. Ultra-high resolution Fourier domain 42. Hoogstede L, Spronk R, Erdmann RG, Gotink RK, Ilsink M, Kolderweij optical coherence tomography for old master paintings. Opt Express. J, Nap H, Veldhuizen D. Image Processing for the Bosch Research and 2015;23(8):1–13. https​://doi.org/10.1364/oe.23.01014​5. Conservation Project. In: Hieronymus Bosch: Painter and Draughtsman— 28. Wojtkowski M. High-speed optical coherence tomography: basics technical studies. Brussels: Mercatorfonds; 2016. p. 30–51. Chap. II. and applications. Appl Opt. 2010;49(16):1–32. https​://doi.org/10.1364/ 43. Ackroyd P, Phenix A, Villers C. Not lining in the twenty-frst century: ao.49.000d3​0. attitudes to the structural conservation of canvas paintings. Conservator. 29. Targowski P, Rouba B, Wojtkowski M, Kowalczyk A. The Application 2002;26(1):14–23. https​://doi.org/10.1080/01410​096.2002.99951​72. of optical coherence tomography to non-destructive examination of museum objects. Stud Conserv. 2004;49(2):107–14. https​://doi. org/10.1179/sic.2004.49.2.107.