Challenges and Results on the Digital Preservation of Aleijadinho's Sculpt

Journal of Cultural Heritage 13 (2012) 210–214

Case study

3D preserving xviii century barroque masterpiece: Challenges and results on the

digital preservation of Aleijadinho’s sculpture of the Prophet Joel

Beatriz Trinchão Andrade , Caroline Mazetto Mendes , Jurandir de Oliveira Santos Jr. ,

∗

Olga Regina Pereira Bellon , Luciano Silva

IMAGO Research Group, Universidade Federal do Paraná (UFPR) P.O. Box 19092, CEP 81531-980, Curitiba, Brazil

a r t i c l e i n f o

a b s t r a c t

Article history: We present our recent efforts in the digital preservation of a set of baroque sculptures made by Antônio

Received 8 March 2011

Francisco Lisboa, known as O. Aleijadinho, which is an important American baroque artist. The set was

Accepted 4 May 2011

made in the beginning of the xix century and is composed of 12 near real sized sculptures, hand-carved

in soapstone. These sculptures represent 12 of the 16 prophets from the Holy Bible and are part of a

Keywords:

UNESCO World Heritage Site. Our group has been collaborating with UNESCO in a project that aims to

3D digital preservation

preserve all these statues. We hereby present our initial efforts, consisting of the 3D digital preservation

3D image acquisition

of the Prophet Joel sculpture. We developed a complete 3D digital preservation pipeline composed of

3D modeling

four main stages: data acquisition, 3D reconstruction, texture generation and 3D model visualization. By

Computer graphics

evaluating our results in this ﬁrst sculpture, we discuss the improvements we conceived before applying

Image processing

our pipeline in the remaining ones. Finally, we present the 3D model of the Prophet which registers the

sculpture’s current state and will be used in restoration, research and educational activities. We believe

this contribution may be useful to guide further research on similar scenarios, showing how to avoid

some practical mistakes and achieve good results.

1. Research aims exposed to natural weathering, they are not always easily accessi-

ble and frequently may not be moved. Also, they are susceptible to

The aim of this work is to describe our experience during the 3D degradation due to aging or even vandalism, requiring important

digital preservation of an important Brazilian baroque statue which preservation measures.

is part of a UNESCO World Heritage Site. This statue is the ﬁrst of Many works in the literature focus on the 3D digital preserva-

12 to be preserved and presents several challenges, such as its dif- tion of indoor and outdoor heritages [1–5]. These works have been

ﬁcult location and large size. In order to digitally preserve it, we exploring innovative techniques to digitalize and extract new infor-

developed and applied a complete pipeline that ranges from data mation about the heritage, generating realistic 3D models. Amongst

acquisition to 3D model visualization on web browsers. We com- these works, we stress the Michelangelo’s Davi [1], the Buddha of

pare different techniques and present the problems and challenges Kamakura [2] and ancient Egyptian artifacts [3].

we faced during this task, the solutions we found, and how we are We have been working with digital preservation since 2002,

improving our pipeline in order to preserve the remaining statues. with the development of a method to combine 3D images of

The resulting 3D models of the statue may be used in several dif- archaeological objects [6,7]. Subsequently, we developed a recon-

ferent ways. For example, it may help its physical restoration, be struction pipeline that focuses on the 3D digital preservation

used in remote visualization and obtaining information about the of cultural and natural heritage [8,9], generating precise 3D

original statue. models that have been used to preserve assets from several

institutions (e.g., a Curitiba Metropolitan Art Museum and Nat-

ural Science Museum) [10]. In, we proposed a reﬁnement of

2. Experimental

this pipeline where high-resolution images are used to gener-

ate higher quality textures, and applied it on the preservation

Outdoor-located heritage sites are a challenging problem when

of artworks from UFPR’s Archaeology and Ethnology Museum.

digital preservation is concerned. Besides the problems of being

The resulting 3D models are available at our 3D Virtual Museum

(http://www.imago.ufpr.br/Museu/en index.html), which allow

realistic interaction with the models [11].

∗ We have been collaborating with UNESCO in a project that aims

Corresponding author. Tel.: +55-041-33613683; fax: +55-041-33613205.

E-mail address: [email protected] (O.R. Pereira Bellon). at preserving Brazilian baroque sculptures made by artist Antônio

B. Trinchão Andrade et al. / Journal of Cultural Heritage 13 (2012) 210–214 211

Figure 1. Bom Jesus de Matosinhos Sanctuary: parvis plant with the Prophets’ loca-

tion.

Francisco Lisboa, known as O. Aleijadinho, who is considered one

the most important American Baroque artists. These sculptures are

located in a sanctuary in Minas Gerais, Brazil, and are included in

the UNESCO World Heritage List. The sanctuary was build during

the xviii and xix centuries and is composed of a church, a parvis

and six stations regarding Christ’s life and death [12].

We focus on the preservation of artworks located at the

parvis. This place contains 12 near-natural size sculptures of Old

Testament Prophets and its plant is illustrated in Fig. 1. More

information about the sanctuary can be found at UNESCO Web-

site (http://whc.unesco.org/en/list/334). The sculptures are carved

in soapstone and located outdoors, having been found to be dam-

aged due to two main reasons: the soapstone is a soft material, so

there is a large amount of inscriptions and degradation caused by

vandalism; it is susceptible to a lichen that erodes the sculptures.

Removing the sculptures and replacing them by copies is a

polemical question as there is not a consensus among specialists

and the community strongly opposes to it. Also, the sanctuary sus-

tains the local economy, attracting tourists from all over the world.

Therefore, UNESCO and National Historical and Artistic Heritage

Institute (IPHAN) have been developing projects to preserve them.

Besides, the physical preservation and restoration, they aim at the

digital preservation of the sculptures.

This work presents our efforts in the 3D digital preservation

of the Prophet Joel, which is the ﬁrst of the 12 to be preserved

Figure 2. Data acquisition stage: (a); equipment used for data acquisition; (b): use

during this project. We use our pipeline for 3D digital preserva-

of a platform to capture the Prophet’s top.

tion, composed of stages that range from the data acquisition to

the 3D visualization of the resulting 3D models. In the 3D recon-

struction stage, we compare different state-of-the-art techniques The sculpture is ﬁxed and located on a corner of a parapet. At

and evaluate their performance on our data. The main goal of our one side it measures around 60 cm from the ground plane, while at

pipeline is to generate realistic 3D models with precise geometry the other it measures about two meters. The statue is two meters

and detailed textures. The resulting 3D models will be used to docu- high, so we had to capture data in different ground levels. We per-

ment the sculpture’s current state and also in educational, research formed the whole process of data acquisition at night as the sun

and simulation activities. light interferes with the range image acquisition. All the equip-

ments were moved to cover different viewpoints. This way, we

collect the images in 3 days using no platform on the ﬁrst day and

2.1. The digital preservation of the Prophet Joel

two different platform heights in the latter days (Fig. 2(b)).

2.1.1. Data acquisition

2.1.2. 3D reconstruction

Our data acquisition system is composed of a laser scanner

We used a 3D reconstruction method mainly composed of four

(Minolta Vivid 910), a digital camera (Canon EOS5D) and two spot-

stages:

lights (Fig. 2(a)). The ﬁrst device captures range and color images

while the camera captures high resolution color images. We devel-

(1) registration;

oped a software to better manipulate the scanner and the camera

(2) mesh integration;

simultaneously, so we can operate both of them from a computer.

(3) hole ﬁlling and;

This software enables a faster capture of the data and includes some

(4) mesh generation.

resources we needed, like plane detection [13].

212 B. Trinchão Andrade et al. / Journal of Cultural Heritage 13 (2012) 210–214

Table 1

Information about the preserved objects and their 3D models.

Voxel parametrization Processing time Number of wedges Number of faces

VRIP [14] + Davis [16] Voxel size = 3 00:20:50 1083754 1975115

PSR [15] Octree depth = 12 00:47:00 1718027 3217054

IVIA [9] + Davis [16] Voxel size = 3 03:13:05 1030497 1884484

First, we manually pre-align pairs of overlapping views in order generate the color texture by using high resolution images from

to generate a coarse approximation of their alignment. Then, we a digital camera. These images must be calibrated to the 3D model

perform the registration by using a modiﬁed ICP, which ﬁnds the and integrated into a single texture map.

precise 4 × 4 transformation matrix for each captured range image We devised a calibration approach that automatically aligns

(view) pair, achieving the alignment of all views into a unique views to photographs. It is composed of a pre-alignment stage

coordinate system. This stage is followed by a global registration which retrieves the camera parameters in relation to the scanner

[8]. data; then a local search is performed to reﬁne these parameters

After, the resulting partial meshes must be integrated in order to and obtain a better texture alignment. The resulting calibration

build a single triangle mesh for the object. For that, we compared matrices are used as input to the method presented in [10]. This

three integration techniques: volumetric range image processing method generates a texture map by projecting all the photographs

(VRIP) [14], Poisson Surface Reconstruction (PSR) [15] and our 3D on the model and merging those using weights that reﬂect the

reconstruction method, IMAGO volumetric integration algorithm correctness of each pixel color on the photographs.

(IVIA) [9]. We chose these techniques because they build precise 3D As the statue is made of soapstone, we assumed it has a Lam-

models from range images. VRIP builds a single mesh by merging bertian surface. The reﬂectance properties were manually set in

the partial meshes into a compressed volumetric grid, extracting a order to reproduce the original statue appearance, obtaining real-

surface from it. IVIA uses a modiﬁed VRIP initially and then provides istic results. The lens distortion was reduced by using a normal lens

an integrated surface by using averaged consensual distances. PSR focal distance (i.e. around 55 mm).

express surface integration as the solution to a Poisson equation.

As the deep cavities and occlusions prevent the complete capture

of the object’s surface, a hole ﬁlling stage is also required. This

2.1.4. 3D model visualization

stage and the last one were performed differently depending on

We developed a Virtual Museum [11] where users may interact

each integration technique. PSR ﬁlls holes and generates triangle

with the 3D models and access information about the preserved

mesh automatically. VRIP and IVIA were combined with Volumet-

objects. We also developed a multi platform 3D visualization tool

ric Diffusion Algorithm by [16] to ﬁll the holes and Marching Cubes

for web browsers that enables and improves access to 3D mod-

Algorithm [17] to generate the mesh.

els. Our Virtual Museum is a client-server system where the client

allows the visualization in a web browser and the server stores

2.1.3. Texture generation the high-poly 3D models. In order to reduce its transmission

Due to hardware limitations, scanner color images present time over the Internet, we reduced the high-poly 3D model of

low resolution (e.g. 640 × 480) and poor color ﬁdelity. We thus Prophet Joel using the mesh simpliﬁcation algorithm developed by

Figure 3. Rendered images of Joel’s parchment and hat using: (a) and (e): only the original views (each view has a different color); (b) and (f): RIP + Davis; (c) and (g): Poisson

method; (d) and (h): IVIA + Davis.

B. Trinchão Andrade et al. / Journal of Cultural Heritage 13 (2012) 210–214 213

Figure 4. Rendered images of Joel’s 3D model: (a): using scanner images; (b): using camera images.

Garland and Heckbert [18], as it produces simpliﬁcations quickly method that better ﬁlled holes. For further information about the

while preserving the sculpture’s geometry. comparison between these integration methods, refer to [9].

Considering beneﬁts and costs, our ﬁnal model was gener-

ated using VRIP. We post processed the model by using Meshlab

(http://meshlab.sourceforge.net) to ﬁll holes and remove outliers.

2.2. Experimental results

Fig. 4 shows our ﬁnal model with two different textures, using

scanner images and camera images, where we can notice that the

We used a computer with 2GB RAM, a Intel Core 2 Duo 2.20 GHz

camera images presented better color ﬁdelity. In order to gener-

CPU and a GeForce 7300 GT graphics card. We compared three

ate the 3D models, 161 pairs of range and color images from the

integration techniques (VRIP, PSR and IVIA), where each generated

scanner and 159 camera images were used.

3D models with different geometric characteristics (Table 1). For

each of them, we chose the parameterization that generates models

2.2.1. Work under development

that are suitable for digital preservation applications. Some prob-

We devised strategies to improve our pipeline for the digital

lems, such as views with distinct resolutions and small overlapping

preservation of the next Prophets. In the data acquisition, we are

areas between views negatively affected the alignment, generating

developing a mechanical support that moves the camera and scan-

noises to be resolved during the integration stage.

ner around the object using stepper motors. Also, we are building

To solve this problem, PSR provide a smoother, noise-reduced,

a canopy tent around each statue to protect the object and the

reconstruction [15], however, it does not ensure the creation of

devices from ambient lighting and weathering. The illumination

two-manifold meshes. Besides, it hardly ﬁnds a balance between

will be ﬁxed and constant inside the tent, avoiding shadows and

the generation of over smoothened models and noisy models. PSR

specular regions on the surface of the statue.

has a great potential of improvement as it may present better res-

In the 3D reconstruction phase, we had difﬁculties in choosing

olution when applied to less noisy data. IVIA values original details

pairs of overlapping views and their manual pre-alignment. Thus,

of the object geometry as it aims at the preservation of original

we are developing an algorithm to perform automatic alignment

data, but it is susceptible to noise. VRIP performed in a shorter

of two views. We are also studying the combination of different

time, presented a good geometry resolution and was less suscep-

mesh integration methods to obtain a superior approach with the

tible to noise. For the hole ﬁlling stage, we compared PSR and

best elements of each. The mesh simpliﬁcation algorithm produces

Davis methods (VRIP+Davis and IVIA+Davis). Davis’ method created

non two-manifold meshes and does not keep the original texture

implausible geometry in larger holes, due to noise in the borders of

coordinates of the high-resolution model. We are studying two-

the holes.

manifold approaches that maintain texture map coordinates. It will

Fig. 3 (a) and (e) show in different colors rendered images of

allow the visualization of Prophet Joel by using remote rendering

the aligned views. The ﬁrst row compares the 3D reconstruction

(feature supported by our 3D Virtual Museum). The reduction made

methods. The words on the parchment present some alignment

in the geometry will not affect the realistic visualization since the

issues due to changes on the lighting during the capture (Fig. 3(a)).

images of the high-poly 3D model are overlaid on the low-poly 3D

The second row compares performance for hole ﬁlling, where a

model.

region on the top of the Prophet hat is missing (Fig. 3(e)). VRIP

(Fig. 3(b)) presents the best results in the surface reconstruction

stage. PSR surface smoothing is noticed in Fig. 3 (c) and (g) and 3. Conclusion

the susceptibility of IVIA to noise in Fig. 3(d) and (h). VRIP + Davis

(Fig. 3(f)) obtained better results than IVIA + Davis (Fig. 3(h)) in hole This work presented the initial efforts for the preservation of

ﬁlling stage. PSR recreates a plausible surface (Fig. 3(g)), being the the 12 Prophets, a set of sculptures made by the artist known as

214 B. Trinchão Andrade et al. / Journal of Cultural Heritage 13 (2012) 210–214

O. Aleijadinho. It presents the knowledge we acquired during the [7] L. Silva, O.R.P. Bellon, P.F.U Gotardo, K.L. Boyer, Computer vision and graphics for

heritage preservation and digital archaeology, Revista de Informática Teórica

digital preservation of one Prophet and shows the problems we

e Aplicada 11 (1) (2004) 9–31.

found. The information obtained from the 3D model of the Prophet

[8] A. Vrubel, O.R.P. Bellon, L. Silva, A 3D reconstruction pipeline for digital preser-

Joel will be used in the physical preservation of the sculpture. It vation, in: Proceedings of IEEE Conference on Computer Vision and Pattern

Recognition, 2009, pp. 2687–2694.

registers the current condition of the sculpture, and may be used to

[9] J. Santos Jr, O.R.P. Bellon, L. Silva, and A. Vrubel, Improving 3D Reconstruction for

help restoring activities and obtaining new information. As future

Digital Art Preservation, in: Proceedings of the 16th International Conference

works, we intend to estimate the light position to enable a higher on Image Analysis and Processing, Ravenna-Italy, in press, 2011.

[10] B. T. Andrade, O.R.P. Bellon, L. Silva, A. Vrubel, Digital Preservation of Brazilian

ﬁdelity representation of the surface material [19]. Also, in order to

Indigenous Artworks: Generating High Quality Textures for 3D Models, J Cult

control access and prevent unauthorized distribution of 3D models,

Herit. In press, doi:10.1016/j.culher.2011.05.002 (2011).

we intend to use 3D watermarks [20]. [11] C.M. Mendes, D.R. Drees, L. Silva, O.P. Bellon, Interactive 3D visualization of

This project has been supported by United Nations Educational, natural and cultural assets, in: Proceedings of the Workshop on eHeritage and

Digital Art Preservation, 2nd Meeting, ACM, Firenze, 2010, pp. 49–54.

Scientiﬁc and Cultural Organization (UNESCO), the National His-

[12] M.A.R. de Oliveira, Aleijadinho and the Congonhas Sanctuary,

torical and Artistic Heritage Institute (IPHAN), Brazilian Ministry of

IPHAN/MONUMENTA, Brasília, 2006.

Science and Technology (CNPq) and Brazilian Ministry of Education [13] A. Vrubel, O.R.P. Bellon, L. Silva, Planar Background Elimination in Range

Images: A Practical Approach, in: Proceedings of the 16th IEEE International

and Culture (CAPES).

Conference on Image Processing, 2009, pp. 3161–3164.

[14] B. Curless, M. Levoy, A volumetric method for building complex models from

range images, in: Proceedings of International Conference on Computer Graph-

References ics and Interactive Techniques, 23rd Meeting, ACM, New York, 1996, pp.

303–312.

[1] M. Levoy, K. Pulli, B. Curless, S. Rusinkiewicz, D. Koller, L. Pereira, et al. The [15] M. Kazhdan, M. Bolitho, H. Hoppe, Poisson surface reconstruction, in: Proceed-

Digital Michelangelo Project: 3D scanning of large statues, in: Proceedings of ings of Eurographics Symposium on Geometry Processing, 4th Meeting, 2006,

Special Interest Group on Graphics and Interactive Techniques, ACM, 2000, pp. pp. 61–70.

131–144. [16] J. Davis, S. Marschner, M. Garr, M. Levoy, Filling holes in complex surfaces using

[2] K. Ikeuchi, T. Oishi, J. Takamatsu, R. Sagawa, A. Nakazawa, R. Kurazume, et al., volumetric diffusion, in: Proceedings of International Symposium on 3D Data

The Great Buddha Project: digitally archiving, restoring, and analyzing cultural Processing, Visualization and Transmission, 1st Meeting, November, IEEE, 2002,

heritage objects, Int J Comput Vis 75 (1) (2007) 189–208. pp. 428–861.

[3] M. Farouk, I. El-Rifai, S. El-Tayar, H. El-Shishiny, M. Hosny, M. El-Rayes, et al. [17] W.E. Lorensen, H.E. Cline, Marching cubes: a high resolution 3D surface con-

Scanning and processing 3D objects for web display, in: Proceedings of Inter- struction algorithm, in: Proccedings of Annual Conference on Computer

national Conference on 3D Digital Imaging and Modeling, 4th Meeting, 2003, Graphics and Interactive Techniques, ACM Press/Addison-Wesley Publishing

pp. 310–317. Co., 1987, pp. 163–169.

[4] A. Koutsoudis, F. Arnaoutoglou, C. Chamzas, On 3D reconstruction of the old [18] M. Garland, P.S. Heckbert, Surface simpliﬁcation using quadric error metrics,

city of xanthi. A minimum budget approach to virtual touring based on pho- in: Proceedings of Annual Conference on Computer Graphics and Interactive

togrammetry, J Cult Herit 8 (1) (2007) 26–31. Techniques, 24th Meeting, ACM Press/Addison-Wesley Publishing Co., 1997,

[5] M. Cabrelles, S. Galcerá, S. Navarro, J.L. Lerma, T. Akasheh, N. Haddad, Integra- pp. 209–216.

tion of 3D laser scanning, photogrammetry and thermography to record [19] C. Xu, A. Georghiades, H. Rushmeier, J. Dorsey, A system for reconstructing inte-

architectural monuments, in: Proceedings of CIPA Symposium, 22nd Meeting, grated texture maps for large structures, in: Proccedings of International

2009. Symposium on 3D Data Processing, Visualization and Transmission, 2006, pp.

[6] L. Silva, O.R.P. Bellon, P.F.U. Gotardo, K.L. Boyer, Low-overlap range image reg- 822–829.

istration for archaeological applications, in: IEEE Computer Society Conference [20] W. Kai, G. Lavoué, F. Denis, A. Baskurt, A comprehensive survey on three-

on Computer Vision and Pattern Recognition, Workshop on Applications of dimensional mesh watermarking, IEEE Transactions on Multimedia 10 (8)

Computer Vision in Archaeology, 2003, pp. 9. (2008) 1513–1527.