Construction Robotics https://doi.org/10.1007/s41693-020-00037-6 ORIGINAL PAPER Autonomous dry stone On-site planning and assembly of stone walls with a robotic excavator Ryan Luke Johns1 · Martin Wermelinger2 · Ruben Mascaro3 · Dominic Jud2 · Fabio Gramazio1 · Matthias Kohler1 · Margarita Chli3 · Marco Hutter2 Received: 15 June 2020 / Accepted: 9 September 2020 © The Author(s) 2020 Abstract On-site robotic construction not only has the potential to enable architectural assemblies that exceed the size and complexity practical with laboratory-based prefabrication methods, but also ofers the opportunity to leverage context-specifc, locally sourced materials that are inexpensive, abundant, and low in embodied energy. We introduce a process for constructing dry stone walls in situ, facilitated by a customized autonomous hydraulic excavator. Cabin-mounted LiDAR sensors provide for terrain mapping, stone localization and digitization, and a planning algorithm determines the placement position of each stone. As the properties of the materials are unknown at the beginning of construction, and because error propagation can hinder the efcacy of pre-planned assemblies with non-uniform components, the structure is planned on-the-fy: the desired position of each stone is computed immediately before it is placed, and any settling or unexpected deviations are accounted for. We present the frst result of this geometric- and motion-planning process: a 3-m-tall wall composed of 40 stones with an average weight of 760 kg. Keywords On site robotics · Dry stone walls · Adaptive assembly 1 Introduction * Ryan Luke Johns Computational planning and fabrication tools have the [email protected] * potential to greatly increase the sustainability of architec- Martin Wermelinger tural construction, allowing for the use of abundantly avail- [email protected] able natural and reclaimed materials that are currently too Ruben Mascaro complex or time-intensive for cost-efective widespread [email protected] application. As compared to many common building mate- Dominic Jud rials, minimally processed rock and locally recycled demo- [email protected] lition debris have extremely low embodied energy (Morel Fabio Gramazio et al. 2001; Alcorn 2003), but also have the potential to [email protected] express site-specifc materiality, revive regional vernacular Matthias Kohler building methods, and exhibit new forms of “cultural per- [email protected] formance” (Oesterle 2009). Margarita Chli This chapter presents an integrated system for construct- [email protected] ing double-faced dry stone walls in situ using a custom- Marco Hutter ized autonomous mobile hydraulic excavator. We outline a [email protected] process for mapping the environment, and localizing and 1 Gramazio Kohler Research, ETH Zurich, Zurich, digitizing irregular stones. Using this digitized information, Switzerland we algorithmically determine the position and orientation 2 Robotic Systems Lab, ETH Zurich, Zurich, Switzerland of stones to align with a designer-indicated goal surface, 3 Vision for Robotics Lab, ETH Zurich, Zurich, Switzerland and conduct grasp- and motion planning for collision-free Vol.:(0123456789)1 3 Construction Robotics Fig. 1 Stone wall constructed with autonomous hydraulic excavator placement. We demonstrate the applicability of our method and stone ofcuts, using 73% of the stock material (selected by constructing a 3-m-tall wall with a total length of 5 m out from a larger set) after dressing. In comparison, the work of 40 stones with masses between 230 and 1548 kg (Fig. 1). presented in this chapter makes use of 100% of each stone object, and uses every stone in the available set. Toward the assembly of such raw materials, the geomet- 2 Related work ric nesting of unmodifed parts has been explored in two dimensions with simulated annealing (Lambert and Ken- A number of recent projects in the domain of architectural nedy 2012). Others have demonstrated the potential of shape digital fabrication have made use of nonstandard input mate- descriptors for matching 3D shapes to existing geometries rials combined with robotic construction. Such projects have for the generation of 3D collages without constraints of fab- played a substantial role in re-establishing computational ricability (Gal et al. 2007), or for the reassembly of fractured design and fabrication as a practice that can not only mate- objects without structural constraints (Huang et al. 2006). rialize digital complexity, but also understand and adapt to The demonstrators presented by Furrer et al. (2017) and the existing complexity of the material world. Johns et al. Liu et al. (2019) both perform dry stacking of unmodifed (2018) provide an overview of recent projects in this gen- stones, making use of stationary robotic manipulators and eral domain, while a number of notable projects deal more RGB-D scanners for the localization of pre-scanned stones. specifcally with irregular stone-based assembly: “Smart Both projects consider structural stability using a physics Scrap” uses automotive digitization tools to inventory par- simulator and evaluate a cost function to determine the tially dressed limestone for facade planning (De Boer et al. validity of each potential pose. The former is focused on 2007), while “Cyclopean Cannibalism” revives ancient the construction of vertical stacks, while the latter can build polygonal masonry methods through robotic stone cutting one-layer thick walls of up to four courses with a 40 % suc- (Cliford et al. 2018). The latter fts parallel-faced polygo- cess rate in an open loop fashion (without accounting for nal elements within the constraints of demolition concrete the settling of placed stones). In this work, we overcome the 1 3 Construction Robotics design limitations of such one- and two-dimensional struc- Several projects have been developed which demonstrate tures, presenting a planner capable of building a multiple- the potential of large-scale autonomous construction with layer-thick structure following an arbitrarily specifed tar- on site robotics. Generally combining mobile platforms with get geometry. Stones are scanned in situ on the fy, and the conventional industrial robots, these compound setups have as-built wall is monitored to refne the placed-stone poses been predominantly demonstrated with standardized build- before the planner is executed again. ing components (Dörfer et al. 2016) or industrially produced chemical products (Keating et al. 2014). Recent research has Fig. 2 Setup diagram: A avail- able stones, B constructed wall, C search space, D rim path with keypoints, E grid map, F gripped stone, G LiDAR sen- sors, H GNSS antennas, I net- work antenna, J HEAP platform with axes indicated, K reference scale and payload comparison with human and industrial robot 1 3 Construction Robotics demonstrated the potential of robotic systems to construct (Fig. 2g). Before the construction process begins, the avail- auxiliary structures to achieve and maintain navigability able stones are loosely distributed on the terrain (Fig. 3). in previously unknown or untraversable terrain, but these This facilitates the detection of single stone instances by frst experiments have only been demonstrated at a small scale, segmenting the ground plane from the LiDAR map, and sub- and do not make use of naturally occurring found construc- sequently performing Euclidean clustering on the remaining tion materials (like stones), but instead use compliant bags point cloud to distinguish the single objects. These point (Saboia et al. 2019) or polyurethane foam (Fujisawa et al. clusters are used to compute an initial grasp confguration 2015) to homogenize the irregularities in their environment. on the not yet reconstructed stone segments. Once a given stone is picked up using this relatively low-resolution data, a high-resolution scan of the full stone is completed while it is held in the gripper. In the following sections, we describe 3 Methodology this stone reconstruction process, the LiDAR-based scene mapping, and the object pose refnement based on the recon- 3.1 Platform struction and scene map information. The stone wall was created using HEAP (Hydraulic Excava- Reconstruction For the 3D reconstruction, the end efector is tor for an Autonomous Purpose), a highly customized Menzi spun with a continuous velocity in front of the LiDAR sen- Muck M545 12t walking excavator that was developed sors while holding the stone. With the known geometry of for autonomous applications and advanced teleoperation the gripper, we flter out points that belong to the end efec- (Fig. 2J). The fully mobile 23-axis excavator is capable of tor while accumulating points on the stone. The complete lifting objects to a height of 9 m and can freely manipulate contour of the stone is recorded by concatenating multiple items weighing up to 3000 kg. Adaptable to complex terrain, point clouds from diferent viewing angles, and applying the machine is equipped with force-controllable hydraulic Poisson surface reconstruction (Kazhdan et al. 2006) to gen- cylinders that allow for control of ground interaction forces, erate the surface mesh from the point cloud data. and cabin- and chassis-mounted inertial measurement units (IMUs) that track machine orientation. Global cabin locali- LiDAR-based scene mapping A precise map of the exca- zation is achieved by a Leica iCON iXE3 with two global vator’s complete surrounding (including the as-built struc- navigation satellite system (GNSS) antennas and a receiver