UC San Diego UC San Diego Previously Published Works

Title Drones in : Integrated data capture, processing, and dissemination in the al- ula valley,

Permalink https://escholarship.org/uc/item/8d67642f

Journal Near Eastern Archaeology, 77(3)

ISSN 1094-2076

Authors Smith, NG Passone, L Al-Said, S et al.

Publication Date 2014

DOI 10.5615/neareastarch.77.3.0176

Peer reviewed

eScholarship.org Powered by the California Digital Library University of California Drones in Archaeology: Integrated Data Capture, Processing, and Dissemination in the al-Ula Valley, Saudi Arabia

Neil G. Smith, Luca Passone, Said al-Said, Mohamed al-Farhan, and Tomas E. Levy

n late 2013, a joint archaeological and computer vision initial results and methodology including the use of UAVs and project was initiated to digitally capture the archaeological modern advances in remote sensing computational techniques. remains in the al-Ula valley, Saudi Arabia. The goal of our Iteam of archeologists and computer scientists is to integrate Dedan – A City on the Crossroads of Ancient 3D scanning technologies to produce 3D reconstructions of Arabia and Modern Technology archaeological sites. Unmanned Aerial Vehicles (UAVs) serve Once an integral part of the North-South trade artery of the Ara- as the vehicle which makes this scanning possible. UAVs allow bian Peninsula, ancient Dedan was one of the most impressive the acquisition of 3D data as easily from the air as from the and extensive eighth century b.c.e.–frst century c.e. trade cen- ground. This project focuses on the recent excavations carried ters in Saudi Arabia (fg. 1). Nestled between the rugged red sand- out in ancient Dedan by King Saud University and the coun- stone mountains and lush farms, the ruins of Dedan try’s conservation of the Lihyanite “lion tombs” carved into and their excavation make an ideal open air museum for tourists the ancient city’s cliff faces. Over the next several years this that have the rare privilege to visit the archaeological site. Te site will be used as a test bed to validate the potential of this ruins of Dedan (known locally as Al-Khuraybah) is located 4 km emerging technology for rapid cultural heritage documentation. north of the modern city al-Ula (26.6549º, 37.9131º, WGS1984). We additionally scanned several areas in Mada’in , an Dedan is strategically located in the al-Ula oasis adjacent to ancient Nabatean city filled with monumental carved sand- al-Qara where abundant water resources could be harvested for stone tomb facades, rivaled only by the capital of the Nabatean agriculture. Te ruins stretch for ca. 2 km around the eastern face empire: . of a sandstone mountain range called Jabal Dedan. Te central Under the King Saud University’s Dedan (al-Ula) Archaeo- rectangular palace at its northern end as as the“lion tombs” logical Excavation Project (al-Said 2009) and the Saudi Com- (of the Lihyanite kingdom, ffh century b.c.e.) carved into the mission for and Antiquities (SCTA), an initial digital clif face at the southern end are visible from satellite imagery. scan using low altitude fying UAVs was conducted at Dedan Te settlement of al-Khuraybah (Dedan) is the ancient capital (Khurayba) and Mada’in Saleh. In this article, we present our of the Dedanite kingdom (eighth–sixth centuries b.c.e.) and later

176 NEAR EASTERN ARCHAEOLOGY 77:3 (2014) the primary trading center of the Lihyanite kingdom (sixth–frst Te application of UAVs to archaeology is an emerging tech- centuries b.c.e.) until its collapse and abandonment in the frst nology. Lambers et al. (2007) and Guidi et al. (2008) were some century b.c.e. partially accelerated by a documented earthquake of the frst to use airborne photogrammetry in archaeological re- in the region (al-Said 2010). It was at this time that the Nabate- search to generate highly detailed 3D reconstructions, orthopho- ans took control of the area and focused the lucrative incense tos, and digital elevation models. Tis was followed by a number trade through Hegra (Mada’in Saleh) located 30 km north of De- of projects that directly sought to employ UAVs for aerial photo- dan. Over the past eight seasons of excavation an extensive area grammetric capture (Barazzetti et al. 2010; Irschara et al. 2010; of Dedan's ruins were exposed. Excavation and conservation has EisenBeiss and Sauerbier 2010; Levy et al. 2010; Remondino et been conducted by King Saud University with principal inves- al. 2011). Our own work, (Levy et al. 2010; Levy et al. 2012) has tigators Dr. Said al-Said and Dr. Abdulaziz Saud al-Ghazzi. Al- undergone a rapid evolution as new technology emerged and we though conservation techniques have been employed to protect adopted it into our feld recording methods. In 2000, we started the site, the majority of the walls constructed used mud-brick with boom systems and the frst commercially available digital and with the harsh weather conditions in al-Ula, it is inevitable cameras, followed in 2006 with our frst aerial balloon, and most that the structures will eventually decay and change in appear- recently with UAVs. ance. Considering these factors, the digital documentation of Aerial scanning’s contribution to the archaeological feld is its Dedan poses a solution both ability to achieve perspectives for documentation and vir- of architecture not possible tual site observation by both from ground scanning. Many archaeologists and tourists. cultural heritage sites and es- pecially archaeological exca- Unmanned Aerial vations contain a high level of Vehicles (UAVs) occlusion. Occlusion is detri- and Archaeology mental to scanning because A key component of our proj- anything that is concealed by ect is the development of a another object is not scanned. streamlined scanning system It results in large gaps or holes using UAVs for archaeology. in the data that can only be A UAV is an aerial vehicle resolved by conducting more (plane, helicopter, blimp) that scans from diferent loca- does not have a human pilot tions. Archaeological sites on board but is controlled re- are very complicated to scan motely. Some UAVs are capa- because they ofen consist of ble of autonomously follow- walls and structures that oc- ing a pre-programmed fight clude the majority of the site. path maintaining altitude and Even with multiple scans it is speed delivering images with not always possible to scan the correct overlap for 3D re- particular areas and achieve construction. Until recently, Figure 1. Map of northern Saudi Arabia, with research areas and sites highlighted. comprehensive capture. UAVs consisted primarily of However, whether it is ledges diferent types of planes or he- blocking the upper portions licopters that required extensive operational experience. For our of a carving or a complex system of small rooms within an ex- low altitude scanning we used a new type of UAV called a multi- cavated site, these areas are easily viewed from the air (see fgs. rotor copter. A multirotor copter uses between three to eight mo- 2–3). A large area can be captured in minimal time using aerial tor controlled counter-rotating propellers to balance itself and to scanning with limited occlusion. Additionally, aerial scanning be able to move in all degrees of freedom. A multirotor with four allows the capture of sites from the same perspective needed for propellers is called a quad-copter and by extension an octo-cop- publication: as top-down architectural drawings. Tey provide ter would employ eight propellers and motors. A multirotor re- a comprehensive layout of a site revealing how architecture and quires less fight training and with added GPS and a barometric terrain are interconnected. (altitude) sensor they hold their position even in strong winds. Similar to helicopters, a quad-copter can take of and land within The Technology Behind the UAV a small area with the ability to rotate and move in all directions. Over the past year, our team has been developing the hardware Te amount and size of propellers provides a tradeof between and sofware required to conduct automated aerial 3D scanning. payloads (e.g. size of camera that can be mounted) and difer- Te UAV provides the means to capture images from the air but ent fight times to be achieved. Tese features make a multirotor it is the sofware used to convert these 2D images into 3D mod- UAV a highly confgurable platform for aerial capture of com- els and merge them with ground scans that makes the technol- plex archaeological sites requiring multiple angles. ogy revolutionary.

NEAR EASTERN ARCHAEOLOGY 77:3 (2014) 177 Figure 2 Figure 3

computer vision algorithms that automatically detect matching features in images (SIFT: Lowe 2004; Wu 2007) and efciently compute camera intrinsic parameters using a least squares ap- proximation called Bundle Adjustment (Brown and Lowe 2005; Hartley and Zisserman 2003; Snavely 2006; Wu 2011). SfM re- constructs a sparse point cloud and calculates camera positions We use a technique from the feld of computer vision called that allow the computation of dense point clouds and trian- Structure from Motion (SfM) to do the frst stage of 3D recon- gulated meshes using Multi-View Stereopsis (Furukawa and struction. Structure from Motion refers to the method of extract- Ponce 2007; Hartley and Zisserman 2003). Rather than stand- ing a 3D structure from many overlapping digital images. Begin- ing in a fxed position and capturing 3D data, SfM algorithms ning in the twenty-frst century, SfM emerged as a new photo- use a change in camera position for each image to fnd the dis- grammetric technique for 3D reconstruction that uses robust tance (motion) between them and at the same time triangulate

178 NEAR EASTERN ARCHAEOLOGY 77:3 (2014) the 3D positions of pixels matched in overlapping images. Te Integrated 3D Scanning Results: al-Ula more motion and movement around the site, the more com- In the feld, we have three teams working simultaneously to run plete the 3D model becomes. Te collection of matched pixels the LiDAR scanner, plan fight missions and fy the UAV, and a and their calculated 3D positions become a cloud of millions third to take detailed terrestrial SfM capture. At the end of the day of 3D points, called a point cloud. From a distance the point the massive datasets generated can be merged within the same 3D cloud appears as a solid model similar to 3D models seen in space to create a comprehensive and accurate 3D reconstruction. CAD programs or video games, but as you zoom in it becomes In al-Ula we used a Faro FOCUS3D LiDAR scanner. It is pos- sible to capture within one day the main section of a large ex- cavation area (50 x 50 m), but not the small interior rooms, narrow excavation trenches, tall structures, and other high- ly occlusive areas of a site. In many cases, these areas cannot accommodate or be adequate- ly reached by the terrestrial LiDAR scanner. Te remain- ing highly occluded areas are captured using terrestrial and airborne SfM. Terrestrial SfM also allows us to take close up captures of important areas that cannot be reached using LiDAR or captured with a high Figure 4 (above). Screenshot of SfM point cloud merged with LiDAR scan using an algorithm to detect spherical markers. The red sphere represents the calculated enough resolution from the clear it is actually a collection sphere radius from its computed center. We use multiple spheres placed in the air. Te closer the camera is to of millions of points. We use SfM and LiDAR scans to merge the two scanning technologies together. the area of capture the higher several diferent algorithms to Figure 5 the density of the point cloud. combine these point clouds When we arrived at the from the air, the ground and Photograph by L. Passone. archaeological site of ancient the laser scanner to create a complete scan of the site (fg. 4). Te combination of the sofware algorithms and the UAV allow us to com- prehensively capture the site and reconstruct it in 3D with proper scale and orientation representing its structure in the real world. Later as the point cloud is cleaned and processed it can be turned into a solid model for CAD programs. Although the reso- lution of SfM point clouds is much lower than a LiDAR laser scan, it is much faster, easier to perform, and vastly more accurate than past ar- chaeological methods that re- lied on surveyors’ illustrated plans. Te combination of the two technologies allows us to take advantage of both meth- ods strong points.

NEAR EASTERN ARCHAEOLOGY 77:3 (2014) 179 Dedan we were amazed at the with multirotor copters since massive size of the settlement. they can fy at a steady pace In comparison to the area ex- of 2.5–3.0 m/s maintaining a cavated, 80% of the settlement fxed altitude insuring a sig- still remains untouched. We nifcant amount of overlap of decided to focus our scanning sharp images. Our multirotors on the two main portions of can fy between 7–9 minutes the southern excavations. Li- at a time and can cover with- DAR scanning was conduct- in this time one kilometer. ed on only the northern side Te low altitude and steady of the site. We would have to speed is sufcient to capture depend on the terrestrial and most of any large site within aerial SfM to fll in the majori- only a morning of scanning. ty of the excavations occluded Two short fights were made by the closest walls (see fg. 2). but within this short amount With the terrestrial SfM we of time we were able to cap- traversed around the entire ture the western and eastern excavation area. As a close- halves of the excavation. Te up, the cistern was captured aerial SfM reconstructed the Figure 6. Final registered point cloud model of the Dedan southern excavation (fg. 7). Te close-up scan of area combining LiDAR, terrestrial SfM, and aerial SfM. Top view (above); side entire excavation area from the ancient basin was high one end to the other. When enough resolution to identify combined with the LiDAR the texture of the sandstone and terrestrial SfM we were and the carved grafti able to situate all the scans in made by modern visitors. one global system (see fg. 6). It was not until mid-af- Te second area scanned ternoon as the sun began to was the lion tombs' carvings set that the wind died down (fg. 8). A series of switch long enough to safely fy over back carved steps lead as the site. Two UAV systems a procession up to the lion assembled by the team were tombs’ clif face. Although employed for airborne SfM: we had intended to use the a DJI Flamewheel F450 quad- LiDAR scanner, there was no copter mounted with a Canon S90 (10MP) and a 3DR Robot- sufcient place to setup the tripod and expect to capture the lion ics Y6 co-axial hexa-copter gimbal mounted with a Sony Nex-7 carvings and the tombs. However, we were able to very care- (24MP) (fg. 5). Low-altitude scanning (30–100 m) is possible fully walk along the edge to take close-ups of the tombs and the carved lions using terrestrial SfM. As the majority of our team climbed up to the clif, we could hear the buzzing sound of the UAV as it ap- proached from half a kilome- ter away preparing to capture the entire clif face, stairway, and terrain below. In just three minutes the UAV was able to capture 44 images that could be reconstructed into a dense 3D point cloud of the lion tombs. With the com- bined terrestrial SfM we were able to both capture the en- tire clif face and the detailed lion carvings. Comprehensive capture of Figure 7 highly occluded sites is possi-

180 NEAR EASTERN ARCHAEOLOGY 77:3 (2014) ble when SfM and LiDAR are integrated. Te results from the expedition to al-Ula dem- onstrates that the combina- tion of these integrated scan- ning techniques can within a short amount of time capture complex areas with survey precision measurement. In the two days of the project four areas were systematically cap- tured and have been presented here. Over the next several years these non-invasive 3D scanning techniques will be applied in order to digitally preserve these sites as they are excavated, document the in- evitable decay of carvings and structures over time, and pro- vide objective datasets for fu- ture analysis and visualization. Figure 8 Acknowledgments We are grateful for the support of the Saudi Commission for Tour- 2010. Towards Fully Automatic Photogrammetric Reconstruction ism and Antiquities, King Abdullah University of Science and Using Digital Images Taken From UAVs. PP. 65–70 in Proceedings Technology, and King Abdul-Aziz City for Science and Technol- International Society for Photogrammetry and Remote Sensing ogy. We would like to personally thank the al-Ula Museum staf Symposium, 100 Years ISPRS – Advancing Remote Sensing Science. for their hospitality and assistance in accessing the archaeologi- Lambers, K., H. Eisenbeiss, M. Sauerbier, D. Kupferschmidt, T. cal sites. We are indebted to the KAUST Visualization Laboratory Gaisecker, and T. Hanusch. 2007. Combining Photogrammetry team Daniel Acevedo-Feliz, Greg Wickham, Dwayne Edwards, and Laser Scanning for the Recording and Modelling of the Late and Paul Riker for providing access to the Faro laser scanner, Intermediate Period Site of Pinchango Alto, Palpa, Peru. Journal their technical support and participation on the project. of Archaeological Science 34(10): 1702–12. Levy, T. E. et al. 2010. On-Site Digital Archaeology 3.0 and Cyber-Ar- References chaeology: Into the Future of the Past – New Developments, Barazzetti, L., F. Remondino, M. Scaioni, and R. Brumana. 2010. Fully Delivery and the Creation of a Data Avalanche. Introduction to Automatic UAV Image-Based Sensor Orientation. IAPRSS&IS, Cyber-Archaeology, ed. M. Forte. Oxford: Archaeopress. ICWG I/V, June 15–18. Calgary, Canada. Levy, T. E., N. G. Smith, M. Najjar, T. A. DeFanti, A. Yu-Min Lin, and Eisenbeiss, H. and M. Sauerbier. 2011. Investigation of UAV Systems F. Kuester. 2012. Cyber-Archaeology in the – the Future and Flight Modes for Photogrammetric Applications. Te Photo- of the Past. Washington, D.C.: Biblical Archaeology Society eBook. grammetric Record 26(136): 400–421. Lowe, D. G. 2004. Distinctive image features from scale invariant key- Brown M. and D. G. Lowe. 2005. Unsupervised 3D Object Recognition points. International Journal of Computer Vision 60(2):91-110. and Reconstruction in Unordered Datasets. Proceedings of the Remondino, F., L. Barazzetti, F. Nex, M. Scaioni, and D. Sarazzi. 2011. Fifh International Conference on 3-D Digital Imaging and Mode- UAV Photogrammetry for Mapping and 3D Modelling – Current ling, p.56-63, June 13-16, 2005. Status and Future Prospective. IAPRSS&IS, Zurich, Switzerland, Furukawa, Y. and J. Ponce. 2007. Accurate, Dense, and Robust Vol. XXXVIII, Part 1: 25–31. Multi-View Stereopsis. IEEE Computer Society Conference on al-Said, S. 2011. Dedan: Treasures of a Spectacular Culture, in Roads of Computer Vision and Pattern Recognition, July 2007. http://grail. Arabia. Pp. 124–35 in Te Archaeological Treasures of Saudi Ara- cs.washington.edu/sofware/pmvs/bibtex.txt bia, eds. U. Franke and J. Gierlichs. Tubingen: Wasmuth Verlag. Guidi, G., F. Remondino, M. Russo, F. Menn, and A. Rizzi. 2008. 3D Snavely, N., S. M. Seitz, and R. Szeliski. 2006. Photo Tourism: Ex- Modelling of Large and Complex Site Using Multi-Sensor Integra- ploring image collections in 3D. ACM Transactions on Graphics tion and Multi-Resolution. Te 9th International Symposium on (Proceedings of SIGGRAPH 2006). Virtual Reality, Archaeology and Cultural Heritage, VAST. Wu, C. 2007. SifGPU: A GPU Implementation of Scale Invariant Fea- Hartley R. and A. Zisserman. 2003. Multiple View Geometry in Com- ture Transform SIFT. http://cs.unc.edu/~ccwu/sifgpu. puter Vision. Cambridge University Press, New York. Wu, C., S. Agarwal, B. Curless, and S. M. Seitz. 2011. Multicore Bundle Irschara, A., V. Kaufmann, M. Klopschitz, H. Bischof, and F. Leberl. Adjustment. CVPR 2011, (poster, supplemental material).

NEAR EASTERN ARCHAEOLOGY 77:3 (2014) 181