EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______

LIGHTING AND TESSERAE

Eva Zányi†, Carla Schroer‡, Mark Mudge‡, Alan Chalmers† † Warwick Digital Laboratory University of Warwick Coventry CV4 7AL United Kingdom [email protected], [email protected] http://www.warwickdigital.org

‡ Cultural Heritage Imaging San Francisco USA [email protected], [email protected] http://www.c-h-i.org

Abstract – A key component of many Byzantine churches was the on the curved and ceilings, which included gold and silver glass tesserae. As the viewer or the light moved within the church, these tesserae sparkled. In this we describe how we captured a Polynomial Texture Map of the at the Angeloktisti Church at Kiti, Cyprus and used it to investigate how the position of the lighting may have affected the appearance of the mosaic. Our study showed that the appearance of the mosaics is indeed significantly different when lit from various directions.

INTRODUCTION From the outside Byzantine churches look unimposing; without much decoration, no paint or precious materials. This is very different to the interior, which provided those inside the space with dramatic visual affects aiming at alleviating and engaging the viewer to approach God [14]. The architecture used light and shadow to symbolically represent different sacral hierarchies and direct the attention of the viewer. Therefore the upper parts of the churches, which represented heaven, were better lit than the lower parts. In early Byzantium this was achieved with the help of daylight through small xxxx

Figure 1. Outside of the church of Panagia Angeloktisti at Kiti, Cyprus.

22.1 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______openings in the upper parts of the walls. From middle Byzantium on, the buildings had less openings letting in natural light and these were replaced by oil lamps and candles [18]. The positioning of these artificial lights was regulated in great detail in manuals, so called typicons, in order to underline the difference between divine light and profane darkness and to let their flickering make precious materials such as the gold and silver of the , mosaics and frescoes, sparkle and draw the viewer into contemplation [1].

BYZANTINE GLASS MOSAICS The Romans perfected techniques for the design and construction of intricate floor mosaics, using natural resistant materials. The Byzantines extended these methods to mosaics and were able to now include fragile materials and more precious one such as the glass gold and silver tesserae [19]. Since so few of the mosaics are left, it has previously been assumed that they were very expensive, especially those which contained glass tesserae. However, recently James questioned this assumption and suggested that the raw material, glass, was not expensive, since Byzantine was close to regions and suppliers [5]. She further suggested that manufacturers of glass tesserae were spread all over the and in fact the setting of the mosaics was the most expensive aspect since it was very labour intense. All this means that mosaics were far more widely spread than previously believed and would also explain why small and politically insignificant churches such as Kiti, which were not situated in any major centre, were so decorated. The glass tesserae were manufactured in a number of ways and often coloured. Metallic tesserae such as gold and silver ones were made by covering a ca 6mm thick glass plate with, for example gold leaf, and then coating it with a thin layer of transparent or coloured glass. The sandwiched glass plate was then heated up until the layers fused and subsequently cut into pieces. The surfaces of the tesserae were slightly uneven and different effects could be achieved depending on if the glossy or the rougher surface was exposed on the mosaic [6.10].

SELECTION OF THE SITE Only few wall mosaics are left from the early Byzantium, since most were destroyed during the 300 years of Arab expansion and invasions, the period of Byzantine history of the 8th and 9th centuries and also because of natural disasters such as earthquakes and fires. The three extensive apse mosaics on Cyprus dating from the 6th and 7th century are consequently very unique. They show the Virgin Mary and Child and were placed in quite small and remote churches, which did not belong to the Pope or any other financially strong ruler [17]. The three churches are: the church of Panagia Kyra at Livadia, the church of Panagia Kanakaria at Lythrankomi [8] and the church of Panagia Angeloktisti at Kiti. From these three, the ones in Livadia and Lithrankomi are, since 1974, in the occupied Turkish section of Cyprus and therefore difficult to access [16]. Apart from the accessibility problem, these two were very badly damaged after the occupation. How much is left of the mosaic in Livadia is in fact not known. Kiti, close to Larnaca on the Greek part of Cyprus, has, however, a well preserved, unrestored apse mosaic and was thus chosen for our study. The mosaic comprises the Virgin Mary holding the Child with the Archangel Gabriel on the right and the Archangel Michael on the left [3.16], Figure 2. The mosaic is lit today from slightly below by spot lights, and thus a large part of the mosaic is not well lit. The church is still in full use with regular orthodox masses taking place and several visits by large tourist groups and children per day.

22.2 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______

REFLECTION TRANSFORMATION IMAGING (RTI) RTI is a term coined by Malzbender and Gelb of Hewlett Packard Labs. RTI captures the “real world” reflectance characteristics of a subject. A simple, robust, and forgiving way to capture RTI information is the use of Polynomial Texture Mapping (PTM). PTMs store surface reflection information with each image pixel. Malzbender et al., inventors of PTM [9], presented a mathematical model describing luminance information for each pixel in an image in terms of a function representing the direction of incident illumination. The illumination direction function is approximated in the form of a biquadratic polynomial whose six coefficients are stored along with the colour information of each pixel. This surface reflection information describes the subject’s surface normals. This normal information indicates the directional vector’s perpendicular to the subject’s surface at each location recorded by the corresponding image pixel. Consequently, PTMs are 2D images containing true 3D information. PTMs are also able to record approximations of other reflection-related properties including surface inter-reflection, subsurface scattering, and self-shadowing. PTMs can communicate useful shape information using purely image based transformations without full photometric stereo or other reconstruction from the surface normals using 3D geometry in Cartesian space. The normals of a surface describe its shape and are used by computer graphics lighting models to determine surface reflection properties. In 3D virtual reality representations, normals are used by lighting models to calculate how light rays will reflect off the surface of virtual 3D geometry. The normal information present in RTIs allows them to use similar 3D lighting techniques. The software used to view RTIs employs these 3D lighting models. RTI images are interactive. Their dynamic interplay of light and shadow works with the human visual system to communicate a powerful perception of the object’s shape [2.7] While RTIs can be used to communicate the effects of different illumination directions on a surface, they can also transform surface normal information to enhance the perception of surface features. This enables RTIs to not only disclose surface characteristics not visible in any of its constituent source photographs, but also reveal information not readily discernable by direct physical examination. This characteristic of RTI has been dramatically demonstrated by its recent use in revealing the nature and use of the Antikythera Mechanism, a 2nd century geared astronomical computation device [4]. A key characteristic of RTIs captured with the PTM method is that complete surface normal information can be acquired from highly shiny, specular materials such as gold without data loss associated with clipping due to specular highlights. PTMs have been demonstrated to effectively capture highly reflective surfaces without data loss due to shadows or specular highlights during the documentation of gold and silver coins as well as highly reflective stone tools [13]. This property has been used to great advantage in our project with the Byzantine mosaic and gold leafed icons at Kiti. The apse mosaic contains numerous tesserae, essentially glass gold and silver leaf ‘sandwiches’. The icons contain gold leaf and tempera painting techniques intended to produce reflective effects. Gold, silver and glass are notoriously difficult to capture through either using or active 3D range scanning, due to their highly specular nature. Our successful capture of these subjects and relighting with the simulated illumination conditions for which they were designed underscore the usefulness of PTM based RTI techniques for these classes of cultural heritage subjects.

22.3 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______

The preceding attributes of RTIs and PTMs as well as their use in cultural heritage documentation projects, the natural sciences, and law enforcement has been detailed extensively elsewhere, including [3.4.9.11.12.13.20].

Capturing RTIs using PTMs To capture single-viewpoint PTM images, the subject is photographed from a fixed camera position. Multiple photos are shot, each illuminated from a different light position. If the positions of the lights are known, the photo sequence can be mathematically synthesized into a single PTM image. The light positions can be known before the photographic session by using domes or templates that position illumination sources at pre-measured locations. There is a family of dome designs, based on the prototype built by Malzbender, which can capture a series of RTI photographs of a subject under automatic electronic control [15]. These devices, when used in conjunction with image sequence processing software, are very efficient when documenting many objects of similar size. Dome equipment is subject to several limitations which have, until recently, restricted the use of RTIs of all types and specifically PTMs to small objects in laboratory contexts. The diameter of RTI subjects is limited to approximately 1/3 of the dome diameter. As dome equipment is currently custom fabricated and the substantial cost of these instruments increases with size, RTI acquisition of larger objects or architectural features was impractical. Transport and configuration within the subject’s environmental context also becomes difficult or impossible as size increases. Highlight RTI (HRTI) is a simpler, lower cost, and more flexible method of PTM acquisition developed through a collaboration between CHI and HP Labs [11]. Eliminating the requirement for prior knowledge of the illumination positions, HRTI permits capture of the light position as the photo is shot. Highlight RTI recovers light positions from the highlights produced on one or two shiny black spheres placed in the image composition by the photographer. After the capture sequence is completed, software detects the highlights on the sphere(s) and determines the light position. Once the light positions are known, the spheres can be cropped from the image sequence prior to the final RTI synthesis. Highlight RTI have been made of objects in a wide range of sizes from two square centimeters to multiple square meters. The highlight RTI method was selected for this project because of the desire to capture various size objects, including the large apse mosaic. HRTI images are captured using a process affectionately known as the ‘Egyptian Method’. Using the Egyptian Method, an illumination radius is selected, based on the diameter of the subject. A string is measured to the radius distance. One end of the string is tied to the light source and the other end is held near to but not touching the subject at the location corresponding to the center of the composed image. For each light position photographed, the subject end of the string is positioned and the light distance is determined. The subject end of the string is then moved out of the camera’s field of view and the photo taken. This process is repeated until a representative hemispheric sample of light directions is acquired around the subject.

Capturing the apse mosaic Capturing the apse mosaic at Kiti posed several challenges. The presence in the area around the apse of light sensitive objects, including, tempura on icons, frescos, and fabrics on furnishings, mandated a low photonic damage lighting system. The 22.4 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______apse’s height off the floor, five meters at the top and over three meters at its base, offered logistical challenges. The enclosure of the sanctuary directly below the apse by a high, ornate grating both segregated the sacred space from the main volume of the church and constrained our working area. Within the working area, the locations of the alter, freestanding crucifixes, ritual objects, furnishings for practical support of ritual activities such as multiple daily masses, and our own documentary equipment made positioning of cameras, lights, colour checker charts, and reflection capturing black balls a creative problem solving exercise.

Figure 2. Capturing the PTM of the apse mosaic.

In an isolated environment, mosaic tesserae are very resistant to photonic damage, and standard flashes or other photographic lights could have been used to document them responsibly. In the apse location, the proximity of light sensitive materials meant that responsible cultural heritage practice required another approach. Our solution used a 250 watt xenon arc lamp light source designed to power a fibre optic illumination system. Xenon sources emit visible light as well as large amounts of photonically damaging ultraviolet (UV) and infrared (IR) light wavelengths. While a variety of light transmitting fibres and guides are available to carry this light, the least expensive and most widely used material is PMMA acrylic cable. PMMA acrylic acts as a band pass filter, excluding both UV and IR light and passing only visible wavelengths between 400 and 750 nm. We used a bundle of this fibre to filter our light source. The height of the apse required modification of equipment and technique, Figure 2. For the light source, the acrylic fibre bundle was attached to a telescoping monopod with the end of the fibre bundle curved 90 degrees from the axis of the monopod, much as a photographer’s light sits atop a light stand. To position the light at locations

22.5 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______necessary to sample illumination directions higher than three meters from the floor, the person holding the light stood on a four meter step ladder, acrobatically extending the light. Much of this acrobatics was caused by the limited locations in the sanctuary with sufficient free floor space for the ladder. The subject end of the Egyptian Method string was attached to a long pole, a broom handle owned by the church. The string end of the pole was cushioned with bubble wrap in case it accidentally touched the mosaic. Fortunately, this precaution proved unnecessary. Following standard procedure, the string end was held near the apse mosaic and the light person moved to the next location, the broom handle and string were moved out of the way and the image taken. Regardless of these circumstances, 79 light positions were acquired. This image sequence was satisfactory to build high quality RTI images and provide the data required for the historical lighting studies. This kind of illumination configuration can be assembled with local equipment in many places around the world, because this equipment is popular for recreational, industrial, and commercial applications. A Cypriot lighting contractor, Andreas Demetriou, loaned the equipment at no charge to our project. We used a dual camera setup with a Canon 5D and a Canon 1DS, both with 50mm fixed lenses. An RTI can be created from a single camera position. However, we chose to use a stereo pair, such that additional 3D data could be generated at a later time. Radio triggers were used to fire the cameras simultaneously. A black billiard ball for the game of snooker was tapped and placed on a boom arm, attached to a ball head on a tripod.

CONCLUSIONS Lighting was a key factor in the design and layout of Byzantine churches. In particular, the rich mosaics with their abundance of gold and silver tesserae played a major role in the Byzantine worshippers’ overall experience of the church. As Figure 3 clearly shows, the gold and silver ensured that the perception of the mosaic was very different depending on the lighting and where the viewer was standing. This deliberate effect is far less noticeable in the churches today, due to the significant alterations to the

Figure 3. How the direction of lighting affects the perception of the Child in the mosaic. buildings over the centuries, and the use of modern lighting. By capturing the PTMs of the mosaic we are able to explore in a controlled manner what sort of perceptual effects are possible depending on the angle of the light.

22.6 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______

Future work will incorporate these PTMs into a detailed computer reconstruction of the Angeloktisti church as it would have appeared in Byzantine times. In addition we will consider the nature of the light, including natural light and the candlelight, and how this may have affected the perception of the mosaics.

ACKNOWLEDGEMENTS This project was funded by the UK EPSRC grant EP7E024998/1 for which we are very grateful. A project like this requires significant additional support and effort. We would like to thank the Cypriot Department of Antiquities for giving us permission, Ioannis Eliades of the Byzantine Museum and Art Galleries for all his help, the Church at Kiti for permission and their friendly support on site, Yiorgos Chrysanthou, University of Cyprus for his support, Andreas Foulias, the author of [3] for his detailed explanation of the mosaic and Professor Rainer Warland, University of Freiburg for all his advice, especially as regards to the most appropriate literature.

References [1] BELTING, H: Bild und Kunst. Eine Geschichte des Bildes vor dem Zeitalter der Kunst, Verlag C.H. Beck, München, 1990. [2] http://c-h-i.org/examples/ptm/ptm.html [3] FOULIAS, A M: The Church of our Lady Angeloktisti at Kiti, Larnaka, Nicosia, 2004. [4] http://www.hpl.hp.com/research/ptm [5] JAMES, L: Byzantine Tesserae: Some Material Considerations, Byzantine and Modern Greek Studies, Vol. 30, No. 1, pp. 29–47, 2006. [6] JAMES L. Light and Colour in , Oxford University Press, New York, 1996. [7] JAMES, W: A saliency-based search mechanism for overt and covert shifts of visual attention. Principles of Psychology, New York: Holt, 1890. [8] MEGAW, A H S and Hawkins, E J W: The Church of the Panagia Kanakariá at Lythrankomi in Cyprus. Its Mosaics and Frescoes, Dumbarton Oaks, Center for , Washington D.C., 1977. [9] MALZBENDER, T, Gelb, D, and Wolters, H: Polynomial texture maps. Proceedings of ACM Siggraph 2001. [10] MICHAELIDES D. Cypriot Mosaics, Picture Book no. 7, Department of Antiquities, Cyprus, 1st edition 1987, revised 1992. [11] MUDGE, M, Malzbender, T, Schroer, C, and Lum, M: New Reflection Transformation Imaging Methods for Rock Art and Multiple-Viewpoint Display, Proceedings of the7th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST2006), M. Ioannides, D. Arnold, F. Niccolucci, (Eds.), Eurographics Association, pg 195-200, 2006. [12] MUDGE, M: Web Graphics/Special Sessions/ Panels, Cultural Heritage and Computer Graphics Panel, SIGGRAPH 2004 Conference Presentations, Soma Media, publisher, ISBN1-58113-950-X, 2004. [13] MUDGE, M, Voutaz, J P, Schroer, C, and Lum, M: Reflection Transformation Imaging and Virtual Representations of Coins from the Hospice of the Grand St. Bernard. In Proceedings of 6th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST2005), Mudge M., Ryan N., Scopigno R. (Eds.), Eurographics Association, pp. 29–39, 2005.

22.7 EVA London Conference ~ 11–13 July 2007 Eva Zányi, Carla Schroer, Mark Mudge, and Alan Chalmers ______

[14] PEERS, G: Sacred shock: Framing visual experience in Byzantium, Pennsylvania State University Press, 2004. [15] REDMAN, J and Mudge, M: The Simultaneous Capture of Spectral and Textural Information, Proceedings of the Society of Imaging Science and Technology Archiving 2007 conference, Scott A. Stovall, general chair, ISBN:978-0-89208-270-4, pp. 2–11, 2007. [16] STYLIANOU, A, and Stylianou, J A: The Painted Churches of Cyprus. Treasures of Byzantine Art, A.G. Leventis Foundation, Nicosia, first edition 1985, revised 1997. [17] STYLIANOU, A: Introduction in: Megaw A.H.S. and Stylianou A. Cyprus: and Frescoes, The New York Graphic Society / UNESCO, 1963. [18] THEIS, L: Lampen, Leuchten, Licht. In Byzanz – das Licht aus dem Osten: Kult und Alltag im Byzantinischen Reich vom 4. bis zum 15. Jahrhundert, Katalog der Ausstellung im Erzbischöflichen, Diözesenmuseum Paderborn, pp. 53–64, Christoph Stiegemann, Mainz, Germany, 2001. [19] WINFIELD, D: Byzantine Mosaic Work. Notes on History, Technique and Colour, Moufflon Publications Ltd., Lefkosia, 2005. [20] ZUCKERMAN, B: Every dot and tiddle: A consideration of the limitations of computer imaging for the study of Dead Sea scrolls. Double Takes: Thinking and Rethinking Issues of Modern Judaism in Ancient Contexts Garber, Z. and Zuckerman, B. (Eds), Studies of the Shoah Volume XXVI, University Press of America, 2004.

22.8