eCAADe 2021 Towards a New, Configurable Architecture

Volume 2

Editors Vesna Stojaković, Bojan Tepavčević, University of Novi Sad, Faculty of Technical Sciences

1st Edition, September 2021 Towards a New, Configurable Architecture - Proceedings of the 39th International Hybrid Conference on Education and Research in Computer Aided Architectural Design in Europe, Novi Sad, Serbia, 8-10th September 2021, Volume 2. Edited by Vesna Stojaković and Bojan Tepavčević. Brussels: Education and Research in Computer Aided Architectural Design in Europe, Belgium / Novi Sad: Digital Design Center, University of Novi Sad.

Legal Depot D/2021/14982/02

ISBN 978-94-91207-23-5 (volume 2), Publisher eCAADe (Education and Research in Computer Aided Architectural Design in Europe)

ISBN 978-86-6022-359-5 (volume 2), Publisher FTN (Faculty of Technical Sciences, University of Novi Sad, Serbia)

ISSN 2684-1843

Cover Design Vesna Stojaković

Printed by: GRID, Faculty of Technical Sciences

All rights reserved. Nothing from this publication may be produced, stored in computerised system or published in any form or in any manner, including electronic, mechanical, reprographic or photographic, without prior written permission from the publisher. Authors are responsible for all pictures, contents and copyright-related issues in their own paper(s).

ii | eCAADe 39 - Volume 2 eCAADe 2021 Towards a New, Configurable Architecture

Volume 2

Proceedings The 39th Conference on Education and Research in Computer Aided Architectural Design in Europe

Hybrid Conference 8th-10th September 2021 Novi Sad, Serbia Faculty of Technical Sciences University of Novi Sad

Edited by Vesna Stojaković Bojan Tepavčević

Volume 2 - eCAADe 39 | iii ORGANIZATION

SPONSORS

Silver sponsor Bronze sponsor

SUPPORTED BY

TECHNICAL SUPPORT eCAADe SPECIAL SUPPORT

OpenConf (Martin Winchester) ProceeDings (Gabriel Wurzer, Wolfgang Lorenz, Bob Martens, Bojan Tepavčević, Vesna Stojaković) CuminCAD (Bob Martens, Tomo Cerovsek) Voting System (Martin Winchester)

FTN SPECIAL SUPPORT Registration System (Dimitrije Nikolić) Webpage (Marko Vučić) IT support (Igor Zečević, Rade Lučić)

iv | eCAADe 39 - Volume 2 PREFACE

This is the first of two volumes, of the 39th eCAADe Proceedings, held as a hybrid event, from 8-10 September 2021 at the Department of Architecture, Faculty of Technical Sciences, University of Novi Sad, Serbia. The two volumes together contain 117 accepted papers. All papers are also available digitally at CumInCAD (Cumulative Index of Computer Aided Architectural Design) – http://papers.cumincad.org

Theme

“The problem of the house is a problem of the epoch. The equilibrium of society today depends upon it. Architecture has for its first duty, in this period of renewal, that of bringing about a revision of values, a revision of the constituent elements of the house.” Le Corbusier, Towards a New Architecture, 1923

Context

Almost a hundred years after the first edition of the book “Towards a New Architecture”, the problem of architecture in the age of the Industrial Revolution is relevant again. A century ago Le Corbusier advocated for a new concept of architecture based on engineering logic of the Second Industrial Revolution and mass production. At the end of the 20th century, the Third Industrial Revolution brought a digital turn in architecture, reshaping the way of thinking and making. Today, industrial revolution is already changing architecture before our eyes towards a new, configurable architecture.

Volume 1 - eCAADe 39 | v How we can describe the notion of configuration in architecture?

The notion of configuration can be described as a relative arrangement of elements in a particular system, form, figure, or other formations. Configuration represents the essence of the design process. The configurable architecture may include a process of abstract thinking through design and exploitation. It can be considered from different perspectives such as design analysis and critical thinking, creative process, digital fabrication, computational design, responsive architecture and environments, material-based design, autonomous assembly or interactive design visualization.

We are already witnessing the application of configurable products in other engineering disciplines that allow end users to participate in the design process and personalize products. How will the architecture in the 21st century respond to the challenges of a new configurable approach to design production and collaboration in the following years?

From mass-customization toward mass-personalization, configurable architecture is seeking for an answer to how architecture will adjust to habitual behavior of their users in the 21st century. What are the new lessons for architects in this new age, age of mass-customization? How parametric tools can enable every person to participate in the design process? What are the new tools for the design collaboration in architecture?

“Towards a new configurable architecture” critically questions the notion of configuration in architecture and how it can be applied to rethinking architectural ideas, design process, representation, fabrication and utilization process.

vi | eCAADe 39 - Volume 1

In order to specifically address some of the questions above, we have invited researchers, professors, experts and students to discuss topics such as: Computational design Rule based systems and shape grammar Building Information Modeling CAAD and education CAAD, creativity and design thinking models Smart cities, city modeling and GIS Digital fabrication and robotics Mass customization in design Digital representation and visualization VR, AR and interactive visualization Agent-based modeling and machine learning in design Simulation, prediction and evaluation in design Structure optimization and material-based design Bionics, bioprinting, living materials Collaborative, participative or responsive design Digital design for sustainable buildings Performance based design Digital heritage

The first volume of the proceedings contains 58 papers while the second volume contains 59 papers. In addition to the accepted papers, the first volume is preceded by Keynote papers including keynote speakers' contributions concerning the themes of their keynote lectures and the workshops organized the days before the Conference. eCAADe 2021 conference chairs Vesna Stojaković & Bojan Tepavčević

Volume 1 - eCAADe 39 | vii

viii | eCAADe 39 - Volume 1 FOREWORD - eCAADe2021

Dear eCAADe friends,

The first eCAADe Hybrid conference Novi Sad 2021 “Towards a new configurable architecture” invites ‘academicians, researchers, professionals and students to address how problem-formations in architecture, will be defined within the long lasting conditions (loss of valuable natural resources, construction waste, energy consumption, ect) that generate existence threatening problems for the planet earth and calls for the urgent action. This year, our community was exited to meet with this challenging conference theme in city of Novi Sad for the 39th eCAADe conference in Europe. However, with still existing global crises, due to Covid outbreak, a joint decision was made to organize eCAADe 2021 Novi Sad conference in a Hybrid mode. The deep spirit of cooperation, and commitment of Novi Sad organization team gave a life to the eCAADe 2021 Hybrid Novi SAD Conference. The theme “Towards a new configurable architecture” critically questions the notion of configuration in architecture and how it can be applied to rethinking architectural ideas, design process, representation, fabrication and utilisation process by seeing architecture in a particular way in which architecture system’s components or services and their connectors or bindings are composed and structured into the resulting building system. A conventional architecture design and construction is characterized by tightly coupled components, that are not configurable, modifiable, and reusable without harming the actual building system and generating construction waste. As a result, the AEC sector uses the most resources and generates the most waste that harms the planet earth. This year’s conference theme “Towards a new configurable architecture” that addresses these issue’s within the frame work of eCAADe conference subjects is aligned with the Europe’s “Circular Economy Action Plan for a Cleaner and More Competitive Europe”.

Volume 1 - eCAADe 39 | ix Architecture by developing affinities and collaborations through multi- disciplinary, multi-scalar, and multi-centered approaches can transform the present and future condition of the Earth System. eCAADe conference is a unique environment for transforming tracks of thought by providing platforms for multi-disciplinary knowledge sharing. I’m looking forward to discussing these and other relevant issues in Novi Sad in physical and online environment. The World PhD Workshop 2021, that is linked to Ecaade Novi Sad PhD workshop is aimed to build togetherness and collaboration among young researchers of sister organizations, of ACADIA, SiGradi, CAADRIA, ASCAD and eCAADe, will be held online at the 6th-7th-8th of December at Federal University of Juiz De Fora. The workshop provides an opportunity for young researchers of our communities to collaborate, to experience how PhD studies are conducted in various schools across the world, and gives an opportunity to meet with prominent researchers in the Architecture and Computing field. I would like to thank to Novi Sad organization team who made eCAADe 2021 Hybrid conference event possible in this extended hard time of a global crisis. Especially, to Dr Vesna Stojaković and Dr Bojan Tepavčević, for their excellent organizational efforts, we feel welcome to Novi Sad.

Birgül Çolakoğlu President of eCAADe

x | eCAADe 39 - Volume 1 ON THE eCAADe HYBRID CONFERENCE 2021

Our eCAADe 2021 story started in 2016, when our research group, Digital Design Center at the Faculty of Technical Sciences, University of Novi Sad, got the opportunity to organize the eCAADe Regional International Symposium. The 4th eCAADe Regional International Symposium was successfully held in Novi Sad. In 2020 our school of architecture at the Faculty of Technical Sciences in Novi Sad was elected to serve as a host for the 2021 eCAADe. We were aware that special circumstances of COVID-19 will bring us many challenges in the organizational process. However, we find valuable that it would be the first eCAADe conference organized in Southeastern part of Europe.

Development

Despite the global pandemic, eCAADe2021 conference in Novi Sad aroused a lot of attention and enthusiasm globally. In the first phase (call for abstracts) we received 334 abstracts from 42 countries. Abstracts were checked in accordance with the formal quality measures set out for submissions by eCAADe.

Call for Papers results – submissions / country

Volume 1 - eCAADe 39 | xi Abstracts were evaluated by three reviewers not affiliated with the authors to avoid a conflict of interest. After the abstract review process, 216 papers were accepted and invited for full paper submission for the second stage of the double blind-peer review process.

Due to the second wave rise of COVID-19 pandemics in Europe, 130 full papers were sumbitted for the second round of reviews. Papers were blind reviewed by three reviewers. After the second review stage, 117 full papers were accepted and authors were invited to submit camera-ready papers, which can be found in two volumes of this Proceedings as well as well as online on CumInCAD database.

Conference eCAADe 2021 has been planned as a hybrid conference, hosted by the Faculty of Technical Sciences from 8-10th September 2021. Sessions are organized to frame the different specific session topics and to adapt to hybrid event organization. The four Keynotes were invited to provide specific, but complementary, visions addressing the spirit and theme of the event.

 Branko Kolarević, Dean and Professor Hillier College of Architecture and Design New Jersey Institute of Technology (NJIT)  Ammar Mirjan, Gramazio Kohler Research, Professorship for Architecture und Digital Fabrication, ETH Zurich  Hans Jakob Wagner, Research Associate at the ICD University of Stuttgart  Ivan Tomović, Associate partner, project director and executive at Werner Sobek AG, Stuttgart, Germany

Professor Milena Stavrić from TU Graz, was asked to chair a round table on the topical area “configurable design”: Architecture between digital and physical worlds in order to provide a particular opportunity for collective conversation. Along with the Conference sessions four online and six on- xii | eCAADe 39 - Volume 1 site workshops were organized bringing the opportunity to share practical development experiences.

eCAADe 2021 conference chairs Vesna Stojaković & Bojan Tepavčević

Volume 1 - eCAADe 39 | xiii

xiv | eCAADe 39 - Volume 1

ACKNOWLEDGEMENTS

First and foremost, we want to express our gratitude to all of the authors whose papers are included in the present proceedings, as well as the chairs of the sessions and roundtables, technical workshop tutors, the PhD workshop organizers and the keynote speakers, whose contributions can be regarded critical to the success of this conference. We are also grateful to the international Scientific Committee who evaluated 334 extended abstracts and 130 completed papers. Only because of eCAADe's access to the OpenConf System and the continuous support of Gabriel Wurzer and Martin Winchester was such a difficult task completed.

Organizing a hybrid eCAADe conference would not have been possible without the commitment of the eCAADe president Birgul Çolakoglu, council and committee members who kindly shared with us their experience and recommendations. It's never enough to express our special gratitude to Joachim Kieferle and Bob Martens who have always gave us decisive support and valuable suggestions. We would also like to thank Liss C. Werner, Armando Trento, Antonio Fioravanti and Jose Pedro Sousa who helped us by sharing experiences and valuable information regarding the organization of previous eCAADe conferences.

At the Faculty of Technical Sciences, we are thankful to Rade Doroslovački, the Dean of the Faculty and Jelena Atanacković Jeličić, the Head of Department of Architecture, who supported hosting the event from the beginning. We also want to express our gratitude to our local organizational team Marko Jovanović, Marko Vučić, Dimitrije Nikolić, Ivana Bajšanski, Jelena Kićanović and Miloš Obradović for their dedicated collaboration. We also wish to thank Igor Zečević and Rade Lučić from the IT center and other staff at Faculty of Technical Sciences.

Volume 1 - eCAADe 39 | xv Finally, we would like to thank our Silver sponsor - Bentley, and our Bronze sponsor - ABB for their support and our host Faculty of Technical Sciences, University of Novi Sad

Novi Sad, September 2021 Vesna Stojaković and Bojan Tepavčević eCAADe 2021 Conference chairs

xvi | eCAADe 39 - Volume 1 SCIENTIFIC COMMITTEE Brown Andre Victoria University Wellington, New Achten Henri Zealand Cagdas Gulen Czech Technical University in Prague, Czechia Istanbul Technical University, Turkey Capone Mara Agkathidis Asterios University of Liverpool, United University of Naples Federico II, Italy Cardoso Daniel Kingdom Aguiar Rita Federal University of Ceará, Brazil Carl Timo Instituto Superior Técnico, Universidade de Lisboa, Portugal FRA-UAS, Germany Carmo Pena Martinez Andressa Alaçam Sema Faculty of Architecture, Turkey Universidade Federal de Viçosa, Brazil Chronis Angelos Amorim Arivaldo Federal University of Bahia, Brazil Austrian Institute of Technology, Asanowicz Aleksander Austria Colakoglu Birgul Bialystok University of Technology, Poland ITU, Turkey Coutinho Quaresma Filipe Atanackovic Jelicic Jelena Faculty of Technical Sciences, Serbia DAU-ULHT, Portugal De Luca Francesco Azari Rahman Pennsylvania State University, United Tallinn University of Technology, States Estonia Dembski Fabian Barczik Günter Department Digital Design University High-Performance Computing Center of Applied Sciences Erfurt. Faculty of Stuttgart, Germany Devetaković Mirjana Architecture, Germany Bernal Marcelo University of Belgrade, Facuty of Perkins&Will, United States Architecture, Serbia Di Mascio Danilo Bhatt Anand ABA-NET/Architexturez Imprints, The University of Huddersfield, United India Kingdom Dokonal Wolfgang Bourdakis Vassilis University of Thessaly, Greece Graz University of technology, Austria Dounas Theodoros Braida Frederico Federal University of Juiz de Fora Robert Gordon University, Greece Dragović Magdalena (UFJF)/ Federal University of Technology - Paraná (UTFPR), Brazil Faculty of Civil Engineering, Braumann Johannes University of Belgrade, Serbia Duarte Jose UfG Linz / Robots in Architecture, Austria Penn State University, United States

Volume 1 - eCAADe 39 | xvii Eloy Sara Herneoja Aulikki Instituto Universitário de Lisboa University of Oulu, Oulu School of (ISCTE-IUL), ISTAR-IUL, Portugal Architecture, Finland Erhan Halil Herrera Pablo C Simon Fraser University, Canada Universidad Peruana de Ciencias Estévez Alberto T. Aplicadas, Peru iBAG-UIC Barcelona, Spain Hudert Markus Figueiredo Bruno Aarhus University, Denmark University of Minho; Lab2PT, Portugal Jovanovic Marko Fioravanti Antonio Faculty of Technical Sciences, Serbia Sapienza University of Rome, Italy Jović Biljana Florio Wilson University of Belgrade, Faculty of Mackenzie University, Brazil Forestry, Department of Landscape Fukuda Tomohiro Architecture and Horticulture, Serbia Osaka University, Japan Jowers Iestyn GARCIA ALVARADO RODRIGO The Open University, United Kingdom UBB, Chile Jutraz Anja Gero John National Institute of Public Health, UNCC, United States Slovenia Geyer Philipp Juvancic Matevz TU Berlin, Germany University of Ljubljana, Faculty of Ghandi Mona Architecture, Slovenia Washington State University, United Kepczynska-Walczak Anetta States Lodz University of Technology, Poland Gomez Paula Kieferle Joachim Georgia Tech, United States Hochschule RheinMain, Germany Gonenc Sorguc arzu Kilian Axel METU, Turkey MIT, United States Gül Leman Figen Klinc Robert ITU, Turkey University of Ljubljana, Faculty of Gursoy Benay Civil and Geodetic Engineering, Pennsylvania State University, United Slovenia States Kocaturk Tuba Güzelci Orkan Zeynel Deakin University | School of University of Porto, Portugal Architecture & Built Enviornment, Ham Jeremy Australia SurfCoast Architecture, Australia Koch Volker Hernández Silvia Patricia Karlsruhe Institute of Technology, FAUD. UNC, Argentina Germany Koering Dietmar Arphneotype, Germany

xviii | eCAADe 39 - Volume 1 Markovic Sladjana Kontovourkis Odysseas Ministry of Education, Science and University of Cyprus, Cyprus Technological Development, Republic Kotnik Toni of Serbia, Serbia Aalto University, Finland Martens Bob Krasić Sonja TU Wien, Austria Faculty of Civil Engineering and Medić Saša architecture, university of Niš, Serbia Faculty of Technical Sciences, Langarica Rodrigo University of Novi Sad, Serbia Universidad Anáhuac Querétaro, Medjdoub Benachir Mexico Nottingham Trent University, United Lastra Alberto Kingdom University of Alcala, Spain Meneses-Carlos Fernando Lazić Marko Escuela Radical, Mexico Fakultet tehničkih nauka, Serbia Meyboom AnnaLisa Lee Hyunsoo School of Architecture & Landscape Yonsei University, South Korea Architecture, Canada Leitao Carla Mostafavi Sina Rensselaer Polytechnic Institute, TU Delft & Dessau Institute of United States Architecture, Netherlands Leitão António Mullins Michael Instituto Superior Tecnico, Portugal Aalborg University, Denmark Liapi Katherine Mussi Andréa University of Patras, Greece IMED, Brazil Ligler Heather Naboni Roberto Pennsylvania State University, United CREATE - University of Southern States Denmark, Denmark Lima Fernando Nejur Andrei Universidade Federal de Juiz de Fora, Université de Montréal, Canada Brazil Newton David Lombardi Davide University of Nebraska-Lincoln, Xi'an Jiaotong - Liverpool University, United States China Nikolic Dimitrije Lorenz Wolfgang Faculty of Technical Sciences, TU Vienna, Austria University of Novi Sad, Serbia Marcus Adam Nováková Kateřina California College of the Arts, United Faculty of architecture CTU Prague, States Czechia Obradovic Ratko Faculty of Technical Sciences, Serbia

Volume 1 - eCAADe 39 | xix Obradović Marija Sidjanin Predrag University of Belgrade, Serbia University EDUCONS, Faculty of Oungrinis Konstantinos-Alketas Digital Production, Serbia Technical University of Crete, Greece Simeone Davide Pak Burak Webuild, Italy KU Leuven (University of Leuven), Sopher Hadas Belgium CRENAU/AAU – UMR_CNRS 1563 – Paraizo Rodrigo ENSA Nantes, France PROURB/FAU/UFRJ, Brazil Sousa Jose Pedro Park Hyoung-June University of Porto, Faculty of School of Architecture, University of Architecture + DFL/CEAU, Portugal Hawaii at Manoa, United States Soza Pedro Parthenios Panagiotis Universidad de Chile, Chile Technical University of Crete, Greece Stavric Milena Pektaş Şule Taşlı Graz University of Technology, Baskent University, Turkey Austria Pellitteri Giuseppe Stojakovic Vesna Università di Palermo, Dipartimento di Faculty of Techical Sciences, Serbia Architettura, Italy Stojanovic Djordje Pratschke Anja University of Melbourne, Australia University of São Paulo, Brazil Stouffs Rudi Rakovic Mirko National University of Singapore, FTN, Serbia Singapore Romão Luís Stulic Radovan School of Architecture, University of Faculty of Technical Sciences, Serbia Lisbon, Portugal Symeonidou Ioanna Rust Romana Department of Architecture, ETH Zurich, Switzerland University of Thessaly, Greece Sárközi Réka Tepavčević Bojan University of Pécs, Hungary University of Novi Sad, Faculty of Savić Srđan Technical Sciences, Department of Faculty of Technical Sciences, Architecture, Serbia University of Novi Sad, Serbia Trento Armando Schnabel Marc Aurel BIMTEGRA S.r.l., Italy Faculty of Architecture & Design Veloso Pedro Innovation, New Zealand Carnegie Mellon University, United Schneider Sven States Germany Vermillion Joshua Sedrez Maycon University of Nevada Las Vegas, University of Nottingham Ningbo United States China, China

xx | eCAADe 39 - Volume 1 Vizioli Simone Instituto de Arquitetura e Urbanismo da Universidade de São Paulo, Brazil Vučić Marko Faculty of Technical Sciences, University of Novi Sad, Serbia Weissenböck Renate Graz University of Technology, Austria Wiltsche Albert Graz University of Technology, Austria Wurzer Gabriel TU Wien, Austria Yazar Tuğrul İstanbul Bilgi University, Turkey Yazıcı Sevil Istanbul Technical University, Turkey Zavoleas Yannis University of Ioannina, Greece Zboinska Malgorzata Chalmers tekniska högskola AB, Sweden Zupančič Tadeja University of Ljubljana, Faculty of Architecture, Slovenia

Volume 1 - eCAADe 39 | xxi

Sibling Associations www.ecaade.org ACADIA eCAADe (Education and Research CAADRIA in Computer Aided Architectural SIGradi Design in Europe) is a non-profit ASCAAD making association of institutions CAAD Futures and individuals with a common interest in promoting good

practice and sharing information

to the use of computers in education and research in architecture and related professions. The organization was founded in 1983, and organizes an annual conference, which is hosted by a different member University each year.

papers.cumincad.org/

CumInCAD is a valuable resource for researchers, educators and others in the field. eCAADe has also collaborated with sibling

associations to create the

International Journal of Architectural Computing (IJAC).

xxii | eCAADe 39– Volume 2 Contents

7 Building Information Modelling 9 Lost in Translation Yurong Huang, Andrew Butler, Nicole Gardner, M. Hank Haeusler 21 A Software Concept for Assessing the Risk of Infection in Enclosed Spaces Clemens Lindner, Christoph Langenhan, Frank Petzold, Wulf Schneider-Brachert, Thomas Holzmann, Anca Rath, Matthias Widbiller 31 Finding Optimal Path Planning Method for Building Navigation in BIM Michał Jarzyna 39 The Use of Diagrammatic Reasoning to Aid Conceptual Design in Building Information Modeling (BIM) Nancy Alassaf, Mark Clayton 49 Operating Room Design with BIM, VR, AR, and Interactive Simulation Uwe Woessner, Joachim Kieferle, Marko Djuric 59 Structural Grid Predesign using Generative Design for Residential Building with Steel Structure on BIM Models José Luis Rodríguez Hernández, Juan Pedro Cortes Perez, Luka Gradisar, Bruno Figueiredo 67 Prototype of an Algorithm-Aided BIM Tool for Semi-Automated Generation of Modular Residential Buildings Sophia Pibal, Konstantin Khoss, Iva Kovacic 77 Safety Information Modelling to Support Planning of Archeological Restoration Site Armando Trento, Antonio Fioravanti, Daniela Borgese, Andrea Gratteri

87 CAAD and education 89 A Hard Road To Travel Wolfgang Dokonal, Patrick Scheich, Tim-Jonathan Huyeng, Uwe Rüppel 95 Augmented Quarantine Renate Weissenböck 105 Integrated Architectural and Environmental Performance-Driven Form-Finding Szende Szentesi-Nejur, Francesco De Luca, Andrei Nejur

Contents - Volume 2 - eCAADe 39 | 1 115 User-Centred Spatial Thinking in Architectural Design with Mixed Reality Beatrix Emo, Andri Gerber, Christoph Hölscher 123 A New Player Has Entered the Game Vincent Hui, Tatiana Estrina, Alvin Huang

135 Digital fabrication and robotics 137 Prvok Kateřina Nováková, Jiří Vele 145 Research on Realtime Communication and Control Workflow with Vision Feedback Integrated in Robotic Fabrication Ming Lu, Xinjie Zhou, Yifan Zhou, Liming Zhang, Weiran Zhu, Philip F. Yuan 153 3D Concrete Printing with Variable Width Filament Qiang Zhan, Hao Wu, Liming Zhang, Philip F. Yuan, Tianyi Gao 161 The reconfiguration of tool parameters in clay extruder Hülya Oral, Meryem Birgül Çolakoğlu 171 Design and Fabrication Methodologies for Repurposing End of Life Metal via Robotic Incremental Sheet Metal Forming Paul Nicholas, Ruxandra Stefania Chiujdea, Konrad Sonne, Antonio Scaffidi 181 Cyber-Physical Robotic Process for Re-Configurable Wood Architecture Anja Kunic, Aljaz Kramberger, Roberto Naboni 189 A Hybrid Robotic Construction Approach in Large Scale Odysseas Kontovourkis, George Tryfonos 199 Material-informed Formwork Geometry Joris Burger, Timothy Wangler, Yu-Hung Chiu, Chanon Techathuvanun, Fabio Gramazio, Matthias Kohler, Ena Lloret-Fritschi

209 Digital heritage 211 3Dino: Configuration for a Micro-Photogrammetric Survey Sara Antinozzi, Diego Ronchi, Fausta Fiorillo, Salvatore Barba 223 MR Game for Historical Experience Fukai Chen, Xiangmin Guo, Tiantian Lo 233 Reinterpretation of the Dougong Joint by the use of Parametric Tools and Robotic Fabrication Techniques Jiangyang Zhao, Davide Lombardi, Hanmei Chen, Asterios Agkathidis

2 | eCAADe 39 - Contents - Volume 2 243 Comparative Analysis of Open-Source and Commercial Photogrammetry Software for Cultural Heritage Isidora Đurić, Ivana Vasiljević, Miloš Obradović, Vesna Stojaković, Jelena Kićanović, Ratko Obradović 253 Open-ArcH Davide Simeone, Stefano Cursi, Antonio Fioravanti, Ugo Maria Coraglia 263 Enhancing User Experience through Interaction Design Panagiota Dania, Helena G. Theodoropoulou, Anna Karagianni, Vasiliki Geropanta, Panagiotis Parthenios 273 Evaluation of Learning Rate in a Serious Game Sepehr Vaez Afshar, Sarvin Eshaghi, Guzden Varinlioglu, Özgün Balaban

281 Rule based systems and shape grammar 283 The Market Square Grammar Marzena Nowobilski, Heather Ligler 293 Towards a Configurable Hybridity in Historic Preservation and Design Mahyar Hadighi 303 Computational Generation of Hutong Neighbourhood Maps Yuyang Wang, Asterios Agkathidis, Andrew Crompton 313 Residential Structures for the Elderly Transformation Grammar Filipe Guterres, Filipe Coutinho Quaresma 323 An Immersive Modular Museum-Archive Concept Model Katherine Liapi, Dimitra Liosi 333 Computational Generation of Hybrid Façades for a Focal Context Niloofar Zolghadrasli, Mahyar Hadighi, Eduardo Costa 341 Spaces Identity Evaluation aNd Assignment - SIENA Zhelun Zhu, Ugo Maria Coraglia, Davide Simeone, Antonio Fioravanti 351 Consistency Checker Hardik Arora, Jessica Bielski, Viktor Eisenstadt, Christoph Langenhan, Christoph Ziegler, Klaus-Dieter Althoff, Andreas Dengel

Contents - Volume 2 - eCAADe 39 | 3 359 Simulation, prediction and evaluation in design 361 Approach to Auto-Recognition of Human Trajectory in Squares using Machine Learning-Based Methods Shaoji Wu 371 Topology Reconstruction of a Discontinuous B-rep Geometry by using Form Finding Method Chao Yuan, Xiao Zhang, Shaoting Zeng, Liu Yang, Zhilong Zhao, Song Qiu 381 Towards a Free-form Transformable Structure Hussein E. M. Hussein, Asterios Agkathidis, Robert Kronenburg 391 Structural Performance of Reciprocal Structures formed by using Islamic Geometrical Patterns Asli Agirbas, Elif Feyza Basogul 401 Towards Prediction and Optimisation for Outdoor Evaporative Cooling Systems in Architectural Design Djordje Stojanovic, Milica Vujovic 409 Use of Simulation Techniques and Optimization Tools for Daylight, Energy and Thermal Performance Feyza Nur Aksin, Semra Arslan Selçuk 419 Static Shading Optimization for Glare Control and Daylight Francesco De Luca, Abel Sepúlveda, Toivo Varjas

429 Structure optimization and material-based design 431 Material Units Adam Blaney 443 Fiber Compositions Isak Foged, Jacob Hilmer 453 Thermochromic Animation Andreas Körner 463 An Autonomous Bio-Inspired Shading Façade System based on Plant Movement Principles Alexa Sharp, Georgina Blay, Janna Kholodova, David Correa 473 Configured Knitting Jane Scott, Elizabeth Gaston, Armand Agraviador 483 Ceramic AM Gantry Structures João Carvalho, Paulo J. S. Cruz, Bruno Figueiredo

4 | eCAADe 39 - Contents - Volume 2 493 A Method for Generating Regular Grid Configurations on Free-From Surfaces for Structurally Sound Geodesic Gridshells Judyta Maria Cichocka, Szymon Loj, Marta Magdalena Wloczyk 503 ClayKnit Lukas Gosch, Julian Jauk, Hana Vašatko, Elizabeta Šamec, Milena Stavric

511 VR, AR and interactive visualization 513 A Motion Vector Visualization Method on a Virtual Reality Screen Tomohiro Fukuda, Hideki Nada, Hiroyuki Fujii, Yoann Pencreach 521 Landscape Visualization by Integrating Augmented Reality and Drones with Occlusion Handling to Link Real and Virtual Worlds Naoki Kikuchi, Tomohiro Fukuda, Nobuyoshi Yabuki 529 Automatic Diminished Reality-Based Virtual Demolition Method using Semantic Segmentation and Generative Adversarial Network for Landscape Assessment Takuya Kikuchi, Tomohiro Fukuda, Nobuyoshi Yabuki 539 Mixed Reality Landscape Visualization Method with Automatic Discrimination Process for Dynamic Occlusion Handling Using Instance Segmentation Mizuki Nakabayashi, Tomohiro Fukuda, Nobuyoshi Yabuki 547 Development of an Augmented Reality System with Reflection Implementation for Landscape Design Visualization using a Planar Reflection Method in Real-Time Rendering Hao Chen, Tomohiro Fukuda, Nobuyoshi Yabuki 555 Exploring the Possibilities of a Virtual Reality Aided Architectural Design System Chaohe Lin, Tiantian Lo, Xinchuang Hu 565 Modeling Inter-dimensional Narratives Olga Mesa, John O’Keefe 575 Augmented Spaces Holly Chan, Andre Brown, Tane Moleta, Marc Aurel Schnabel

Contents - Volume 2 - eCAADe 39 | 5 6 | eCAADe 39 - Contents - Volume 2 Building Information Modelling 8 | eCAADe 39 - Building Information Modelling - Volume 2 Lost in Translation Achieving semantic consistency of name-identity in BIM

Yurong Huang1, Andrew Butler2, Nicole Gardner3, M. Hank Haeusler4 1,3,4UNSW / Computational Design 2Cox Architecture 1,3,4{m.haeusler|n.gardner|m.haeusler}@unsw.edu.au 2Andrew.Butler@cox. com.au

Custom room naming in architectural projects can vary considerably depending on the user. Having multiple and diverse names for the same room is particularly problematic for information retrieval processes in BIM-based projects. Current best practice includes either team agreement on naming labels in BIM or manual renaming to align with an office-wide standard. Both remain laborious and flawed and lead to compounding errors. This research explores how an automated naming-standardization workflow can enhance the interoperability of object-based modeling in a BIM environment and make information retrieval more reliable for a project life cycle. This paper presents research on (1) building a custom corpus specialized for architectural terminology to fit into the BIM environment and (2) devising a standard-naming system titled WuzzyNaming to save manual work for BIM users in maintaining room-name consistency. Our presented workflow applied natural language processing (NLP) technique and Fuzzy logic to perform the semantic analysis and automate the BIM room-name standardization.

Keywords: Building information modeling, Natural Language Processing, Data interoperability, Naming convention, Fuzzy logic

INTRODUCTION particularly on large architecture projects. Parame- Building information modeling (BIM) is a digital rep- terized design objects in BIM models require precise resentation of shared-data management for the de- naming in order for the system and its users to navi- sign, delivery, and production of building projects. gate to the correct entities (Pratt 2004). Information BIM works through a data repository that consists of retrieval in existing BIM software, more generally, suf- semantic information of object identification to facil- fers from issues of low efficiency and poor accuracy itate digital data-sharing and decision-making along because of the error-prone human naming and het- the project life cycle. However, semantic inconsis- erogeneous naming protocols. In accordance with tencies in BIM entity labeling can significantly impact the overarching research methodology of action re- the efficiency and preciseness of data management, search (AR) that is adopted in this project, consulta-

Building Information Modelling - Volume 2 - eCAADe 39 | 9 tions with the research industry partner Cox Architec- vit model and utilize the algorithmic methods of NLP ture have identified the BIM entity naming problem and approximate string matching in Python. Finally, as an unresolved issue that impacts the AEC indus- hosting the system in a web-based interface for user’s try. For a host of reasons, the industry has been re- access. The completed language-standardization luctant to adopt a standard BIM entity naming proto- workflow proposed aims to assist designers by au- col, meaning that inconsistencies in BIM entity nam- tomating the task of making BIM entity naming con- ing continue to compromise seamless data coordi- sistent across a project. Ensuring naming consistency nation and team and stakeholder collaboration. And and accuracy in BIM data-management contributes according to the ABAB (2018, p.226) with the “grow- to fostering the robustness of BIM project manage- ing uptake of BIM, Australia has a timely window of ment in the AEC industry. Based on the research aims opportunity to develop a common framework for BIM summarized above, the key question this research process consistency”. Yet, what is the most appro- asks is: priate and reliable method to address the problem How can NLP be applied to develop a workflow that of BIM entity naming inconsistencies? Following ac- automates BIM room naming inconsistences to align tion research principles, the research project inves- with established organizational standards? tigates and develops a computational workflow to overcome semantic differences in BIM entity nam- RESEARCH METHODOLOGY ing. More specifically, the research project imple- With ABL Group’s (1997) definition to “contribute both ments and explores the Natural Language Process- to the practical concerns of people in an immediate ing (NLP) technique to drive an automated BIM room problematic situation and to further the goals of social re-naming workflow. Ruikar et al. (2005) argue that science simultaneously”, this research project adopts the standard process needs to work in hybrid auto- the overarching methodology of Action Research mated and manual processes to avoid human natu- (AR), characterized by bridging the relationship be- ral reluctance in accepting machine automation dur- tween practice and concepts to satisfy industry iden- ing the work process. To encourage the utility of the tified problems. AR is oriented towards collabora- system, it is wrapped as a web-based API and can tion amongst stakeholders involved and aims to in- be conveniently carried out in BIM. The established corporate multiple perspectives to enhance the va- workflow harnesses NLP and Fuzzy logic to address lidity of the research (Hearn & Foth 2005). AR follows the non-uniform BIM entity naming in architectural an action-reflection cycle that is organized iteratively projects. By developing a workflow that overcomes from reconnaissance to action-execution and reflec- the barrier of language meaning conflicts, data re- tion (Lewin 1946). The essence of AR is to address a liability, and interoperability in BIM projects can be real world problem through iterative and collabora- maintained which ultimately contributes to advanc- tive evaluation, action, and reflection in a scholarly ing multi-disciplinary collaboration in architectural way. projects. The design problem has been identified by the industry partner Cox Architecture and translated into RESEARCH AIMS a research question by the academic team. With the The project’s goal is to apply semantic knowledge collaboration of an industry partner, the knowledge- to enhance data representation in collaborative de- gap between clients and researchers is overcome as sign projects, namely to develop a workflow to mit- AR gives weight to both “tacit knowledge is embed- igate inconsistent BIM entity identification naming ded in the actions and phenomenology of [the in- in architectural projects. Thus it will develop tech- dustry] participants” and “codified knowledge” pro- niques to extract entity labels for room names in a Re- duced through the research (Hearn & Foth 2005, p.6).

10 | eCAADe 39 - Building Information Modelling - Volume 2 In this research, the investigation and development ally, entity-handling remains problematic. The vari- of a language-processing workflow cycled through ous naming conventions adopted by practitioners in- three iterations. This began with the inquiry of nam- volved in the AEC industry lead to semantic differ- ing convention in the BIM working-environment, and ences of entity labels. Many scholars argue that in- the evaluation of current industry practice to inform efficiencies in architectural project organization and the potential direction of possible research action. production related to entity naming misunderstand- This uncovered that invalid room naming leads to ings between project team members could be over- inefficient synchronous exchange of documentation come through a standardized BIM naming system and conflict between design team stakeholders and (Taylor 2007; Yang & Zhang 2006). Nevertheless, de- this suggested the need for established organiza- veloping a standardized BIM naming system is a chal- tional standards for room naming in a BIM context. lenging task. It is estimated that across the 20,000 ar- To address the issue caused by inconsistent names, chitecture organizations in Australia each have devel- this research proposed an appropriate language pro- oped their own unique in-house BIM library (Duddy totype be developed in the action-taking phase. This et al.,2013). Lee et al. (2012) identify four key factors required suitable data and language models collec- that need to be addressed to develop a standardized tion and organization as the fundamental compo- BIM entity naming process including taxonomy (in- nents for executing name-standardization in work- formal description), manual classification, and defi- flow iterations. Each subsequent workflow t iteration ciency of name uniqueness. Consequently, the cur- was evaluated based on an efficiency and accuracy rent BIM environment requires harmonization of en- test by towards rationalising the BIM naming stan- tity names to limit the complexity across different dard in the workplace. The collaborative and iterative naming documentation and eliminate onerous tasks process of AR promotes the viability of translating re- in project management. Several existing methods search outcomes to address and enhance-through- and guidelines have been imposed to solve the nam- change, industry-practice. ing conflicts, ranging from numerous versions of offi- cial BIM standards (e.g. NATSPEC 2011, NBIMS 2012) LITERATURE REVIEW to industrial specifications (e.g. IFC -Industrial Foun- BIM Standardized Naming Conventions. BIM, de- dation Classes) for keeping the name conformance. fined by the National Building Information Modeling Prior to BIM name compliance, Eastman (2005) in- Standard, is an advanced process that covers multi- troduced a Design Review tool to adopt the naming ple aspects (planning, design, construction, opera- conflicts between departmental space and individual tion, and maintenance) associated with architecture base spaces. Chen et al. (2017) developed a semiau- project via implementing a standardized machine- tomatic naming system in accordance with the prop- readable information model (NIBS 2007, p.25). In erties acquired from BIM objects in Revit. Yet, while a BIM project, the predefined naming standard is it applied an auto-naming tool to ensure semantic considered as the fundamental identifier of pre- consistency, it required extra time to classify com- fabricated elements at the preliminary manufactur- ponents. Additionally, it is difficult for non-expert ing stage (Chen et al. 2015). The prevailing BIM users to retrieve the object based on the name gen- standards allow for precise object recognition and erated in this unacquainted system. These examples remedy of ambiguous definitions which maintain put emphasis on accomplishing compliance check- the shared vision and common agreement amongst ing in BIM models, but rather than impose a strin- stakeholders (Barbosa et al. 2016). Although BIM gent BIM-naming standard on thousands of organi- processes have been widely adopted in the architec- zations, an automated process could identify and rec- ture, engineering, and construction industry glob- tify multi-named BIM entities. Duan (2011) attests to

Building Information Modelling - Volume 2 - eCAADe 39 | 11 this, arguing that instead of re-adapting a new for- Kim et al (2003) synthesized all the medical terminol- mal language standard, delivering a semantic mech- ogy from biological literature into a comprehensive anism to formalize the natural-language representa- corpus with the help of NLP. Yet in the architecture, tion is more considerable and efficient. engineering, and construction (AEC), and in relation Data Retrieval in Natural Language Process- to design technology there are limited examples of ing. In order to construct a system that can extract, NLP implementations. The NLP method raises the identify, organize, and rename BIM entities accu- likelihood of organizing the information exchanged rately, this research proposes the application of natu- in the BIM project and helps to standardize the natu- ral language processing (NLP). Kumar (2011) defines ral language term of object name into specialist lan- NLP as “a field of computer science and linguistics con- guage concept. In the architectural workplace, it is cerned with the interactions between computers and time-consuming for organizations to train employ- human(natural) languages” (p.65). NLP has been ap- ees to follow new naming identifications imposed by plied frequently on information retrieval, data extrac- the unified name schema. To more effectively ad- tion, and data access with the goal of accomplish- dress the naming conflicts related to the use of differ- ing ‘human-like language processing’ (Liddy, 2001). ent BIM libraries, and to overcome issues of ambigu- It should be noted that several tools such as NLTK ity and diversity in entity naming, a generic semantic (Bird et al. 2009) or spaCy developed by Honnibal representation can be applied to represent, organize & Montani [1] are available for executing natural lan- and regulate this information in real-time. guage processing. For this research, the NLP focuses Communication of Collaborative BIM System on the semantic measure, which involves semantic with NLP. According to Succar (2009), NLP is an ap- similarity, string matching, and keyword search. Nat- propriate fit for addressing BIM entity naming prob- ural language, with the inherent property of ambi- lems, as semantic richness plays an important role guity and vagueness, is likely to provoke confusion in object-based modeling of BIM collaboration. In and imprecise outcomes for computational process- the AEC industry, the interoperability of data ex- ing. The language uncertainty can be ameliorated change is vital to facilitate the efficient sharing of by evaluating the semantic similarity. Dating back valuable information synchronously between stake- to the DISTANCE algorithm (a search technique) de- holders. But insufficient semantic representation veloped by Rada and Bicknell (1989), the conceptual leads to errors and the inability for model exchange. distance between different terms has been measured While organizations who adopt BIM approaches em- to support the information retrieval in artificial intelli- ploy different ways of working and regulations, Lo- gence. On that basis, Resnik (1999) introduced the IS- rio et al. (2016) proposed a semantic system to al- A taxonomy approach to unravel semantic ambigu- leviate information dilemmas across different plat- ity and syntactic ambiguity by measuring the word- forms, stakeholders, disciplines, and organizations. relatedness that exists in the corpus. With the rapid This system includes natural-language understand- development of computer science, NLP has become ing techniques, which interpret design issues into a transdisciplinary technique to assist semantic anal- statements with formal language use.The uniformed ysis. WordNet, an online lexical system developed outcomes avoid designers suffering from misunder- from Princeton University (Miller et al. 1990), orga- standing heterogeneous problem definitions in col- nizes and encapsulates word ontology in Synset (Syn- laborative design projects. Through applying NLP, onym set) for word identification and differentiation. an automated design-review database developed by Applying this approach, Palkovskii et al (2011) pre- Lee et al. (2012) overcame string-matching issues sented an external plagiarism detection system by in a name-based mapping process. Hence, similar employing WordNet for Word sense disambiguation. ideas can be applied from formalizing design issues

12 | eCAADe 39 - Building Information Modelling - Volume 2 to standardizing room names. The NLP interface in tation in architectural projects. Hosting the appli- CAD tools presents the advantage of replacing spe- cation on the web makes the workflow more acces- cialized terminology with natural language expres- sible across an organization. The workflow is capa- sion, which avoids the contradiction in language use ble of operating on diverse platforms for the conve- and allows easy recognition for users (Samad et al. nience of users such as Dynamo or Grasshopper in 1985). Compare with the traditional interface, an ex- the design workplace. For realizing the possibility periment proved that NLP receives a better evalua- of a controlled language system in the architectural tion of user-preference and efficiency (Biermann et al. workplace, an iterative script is programmed to host 1983). Furthermore, some studies discussed the im- the standardization process. With the ability to rec- plementation of NLP on facilitating the project man- ognize and process design terminology, it is neces- agement in BIM (Lin et al. 2016; Lee et al. 2012). In sary to train machines to be aware of the jargon and conclusion, the existing research shows that NLP has direct them into correct classification. The training found applications in architecture and engineering material was chosen from the existing stadium room- fields particularly in the domain of model-exchange name documents. Through integrating the existing or automated document classification (Venugopal et room names, the time spent on collecting special- al. 2012; Jung & Lee 2019; Salama et al. 2016). How- ized vocabulary will be saved. Employing the room ever, there are limited examples of its use in relation names used in ongoing projects will prevent the sys- to natural language processing for BIM naming con- tem from being impracticable in the workplace. To vention. Yet as argued above NLP could be worth ex- ensure the quality of the name-standardization ap- ploring to benefit the collaborative BIM system. proach, common problematic expressions during de- sign collaboration have been revealed by analysing CASE STUDY the features of received room names. Accordingly, To investigate the application of NLP in a workflow the following system put emphasis on resolving the to automate room naming inconsistencies in a BIM frequent language hurdles of room naming listed in project, this research project has adopted an indus- Table 1, for delivering explicit representation of room try case study of a finished stadium. The project semantics. proceeded in two key stages that involved firstly Bird et al. (2009) argues that “Structured col- developing a custom architectural corpus and sec- lections of annotated linguistic data are essential in ondly programming a workflow to create a web- most areas of NLP”. Thus the domain-specific corpus based naming standardization system. The architec- is of the essence in achieving the accurate evalua- tural corpus was built first to specify the architectural tion for the NLP system. Nevertheless, due to the terminology, this supports the semantic normaliza- limited application of NLP in the architecture disci- tion with more built-in knowledge of room names. pline, the current ubiquitous dictionaries are unavail- The system is wrapped as a python module and de- able for achieving the research aim. Since an un- livered as the Web-API interface for standard consul- suitable language database hinders language anal-

Table 1 Category of common language conflicts in Room-Names

Building Information Modelling - Volume 2 - eCAADe 39 | 13 Figure 1 Workflow of domain-specific corpus development

Figure 2 Fragment of the corpus hierarchy: the affiliation of room name.

14 | eCAADe 39 - Building Information Modelling - Volume 2 ysis, building a specialized corpus should be con- are specified by the industry partner to serve as the sidered as the core precondition to assure the va- benchmark of name normalization. For navigating lidity of linguistic processing in the AEC industry. the irrelevant words into the determined name class, The development of representative corpus in the ar- a python script is built to detect the semantic re- chitectural domain starts from deploying an NLP- lationship with respect to the word sense. The hi- based workflow to pre-process raw names in exist- erarchical structure of WordNet (Miller 1990) is suit- ing BIM documents. The procedure involves sev- able to compute the semantic relatedness according eral steps: cleaning, tokenization, and classification to the hypernym, hyponyms, and synonyms stored of names. Since unconventional phrases occurred in its word-Synsets (Synonym-sets). Although Word- frequently in-room identification, the heterogeneous Net performs well in decoding the semantic relations, room names are categorized as follows to avoid miss- the informal lexical entries with architectural special- ing those non-standard collocations: abbreviations, ized knowledge are absent in the WordNet database. expressions with punctuations, and normal expres- This bottleneck can be removed by seeking the syn- sions. These name groups will be processed inde- onyms of those terms manually to cover the defi- pendently to expand the richness and diversity of ciency of unconventional expression. Accordingly, architectural expressions that constitute the corpus. combining the lexical collections from both WordNet Based on the former analysis of raw documents, the and manual word-acquisition, the synonyms of im- phrases need to be tokenized and segmented to ac- portant terms are gathered as the fundamental com- quire the individual term. To improve the efficiency ponents of the terminological corpus. Supported by of word collection, only frequent concepts are re- adequate vocabulary, the corpus satisfies accurate mained to confine the corpus to a manageable size. data retrieval. The final corpus is delivered to exe- To calculate the word frequency, the word collection cute the computer-aided linguistic analysis for future needs to transform into the document-term matrix name normalization. Commencing on evaluating (DTM). The DTM technique enables the conversion of and categorizing the room terms from stadium name the linguistic text into machine-readable content in schedules, the corpus is synthesized with accumu- a vectorial semantic environment, then allows for se- lated synonym-sets under proper classification (Fig- lecting the significant terms among all expressions. ure 1). During the preparation process, manual and The frequency-counting scheme is developed to fil- computerized practice work in parallel for achieving ter the negligible words and gather valuable words. the corpus rationality and remedy the insufficiency of Hence, these shared terms used in BIM projects will technical phraseology in traditional dictionaries. be passed into the customized corpus as the basic Figure 2 depicts the lexical relations of room- lexical entries. Upon the extraction of valid terms, it names in the final corpus, which gives the machine was discovered that these words are chaotic and un- permission to accomplish the semantic understand- related to each other, this results in difficulty for the ing in accordance with the word hierarchy. (e.g.: computer to recognize and standardize the name in- Computer understands ‘Changing_place’ is synony- put based on their relations. To build a full-fledged mous with ‘Fiting_room’ as they locate in the same database, the word list requires principled storage class). Given the clear affiliation among vocabulary, within a systematic structure. In the light of this pre the word network is capable of retrieving the term requires, the corpus with explicit hierarchy must be precisely without causing sense ambiguation. The established through computing the semantic simi- name standardization is started by the basic NLP larity between terms and grouping them into indi- procedure, involving punctuation removal and word vidual sets under specified name class. Considering segmentation. Recalling the pre-analysis of current the complexity of room classification, the key terms inconsistent naming, the system gave priority to dis-

Building Information Modelling - Volume 2 - eCAADe 39 | 15 pose of spelling mistakes from the input. The python The delivered interface is through BIM software (e.g. script is developed for spell-correction, which nicely Revit & Rhino) connecting to the API. rectifies the errors. The Fuzzy logic- a computational To achieve this, a Grasshopper script is gener- algorithm assists in finding approximate strings- has ated to interact with the API and then give access been chosen to undertake the string-matching by to WuzzyNaming module. Figure 3 demonstrates comparing the pattern similarity between input text that the inconsistent room names in Revit floor- and words in the corpus. In order to automate the plan will be standardized successfully by sending the naming standardization from the corrected input, Grasshopper output to Revit. Therefore, WuzzyNam- the semantic understanding should be captured for ing should be applicable to different BIM platforms, each cleaned term. which permits the reusability of proposed system One major challenge in meeting this intention and eases user’s access with simple operational inter- is how to solve compound expressions that are re- face. currently demanded in-room naming. This synthetic language raises the difficulty for computers to com- DISCUSSION prehend the name connotation. To make sure the The standardized classification generated from the system is flexible and practical enough to standard- normalization workflow demonstrates the success of ize the random input from users, the issue of com- applying the NLP technique to solve the naming dis- plex words is approached by establishing a compre- crepancy in BIM. To validate the efficacy of the pro- hensive system to manage disparate scenarios. In the posed standardization approach accompanied with standardized processor, the complex names that dis- the customized architectural corpus, a series of room obey the linguistic rules will be tokenized into sepa- names are picked randomly from the ongoing sta- rated words, the iterative system allows conducting dium projects in Cox Architecture to be considered a thorough similarity calculation of each tokenized as the study objects for accuracy evaluation. The re- term compare with the pre-defined corpus. Once sults shown in Table 3 indicate the viability of sys- the computing completed, each word will be aligned tem with nearly 95% accuracy rate in acquiring stan- with the standard class which has the highest score in dard name class. The results are calculated by di- relevance ranking. The similarity measurement is car- viding the number of correct room classifications by ried out by evaluating both morphological and word- the total amount of room inputs. What is notewor- sense relationships to guarantee the preciseness of thy is that after training the machine progressively, normalization. Similar to the spell correction, the the computer’s ability in processing complex expres- fuzzy-logic algorithm assists to examine the word- sion has been optimized with the attempt to offer form similarity. For the word-sense relationship, the an enhanced understanding of NLP utilization in BIM, explicit word hierarchy in corpus eliminate the con- which becomes more responsive for user’s require- fusion of polysemy and associate the word-input to ment. As shown in Table 4, as iterations progress Step its standard classification if they share equivalence from iteration 1 to the next iteration, the flexibility meaning. and accuracy of the workflow improves (Table 4). After verifying the success of normalizing indi- However, the limitations of the research project vidual lexis, joining them together and output the should also be acknowledged. The time constraints standardized room name for any intuitive input. Af- of a 10-week research project duration has meant ter compiling the python module for standardization that the project could produce a proof-of-concept processor, the completed script was packaged as the demonstrating the basic utility of NLP technique for Web-API by Django. Django works as a python web use in a BIM project for the AEC industry. A key limi- framework to establish the programming interface. tation of the developed workflow uncovered during

16 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 3 Standard result in Revit room schedule

Figure 4 Overview of Standardization Workflow

Building Information Modelling - Volume 2 - eCAADe 39 | 17 Table 2 Evaluation outcome

Table 3 Reflections of Iterative testing outcome

the research process is the vocabulary deficiency of Table 4 architectural corpus, as in current stage, the devel- Standard class oped corpus is the indispensable tool for computer result in Revit room to understand word sense. schedule Given more time, this problem could be over- come with deeper investigation and collection to- Figure 5 wards the room expressions from different building Mock Revit projects (e.g. Office or Residential Building). Regard- Interface-Popup ing the user interface, the system runs independently Window in web API at this stage, this leaves space for future development by exploring the user-friendly interface (Fig. 5) to assist inexperienced designers in the archi- tectural workplace.

18 | eCAADe 39 - Building Information Modelling - Volume 2 Furthermore, the present system faces the dilemma teroperability in BIM. Without compelling users to fol- to process compound expressions by relying on the low a specific naming convention, the system pre- immature corpus created in early stage of research. vents people from struggling with the inconsistent The computational intelligence could be pushed fur- names in BIM. The naming-standardization system ther through introducing the machine learning tech- not only promotes efficient collaboration across de- nique. This shows more opportunities in which pro- sign teams but indeed lays the foundation of NLP ap- found research could be explored to broaden the us- plication in BIM to offer a more humanized system. ability of system in a larger scale. Regarding the user Overall, this research reveals the value of interdisci- interface, the system runs independent in web-API at plinary pursuits that draw on methods in the com- this stage. This leaves space for future development puter science discipline to innovate ways of working and user testing to improve usability and accessibility in the architecture discipline and AEC industry. across an organization and potentially the AEC indus- try more generally. REFERENCES Despite the shortages outlined above, the sys- ABAB, . 2018, BIM Process Consistency: Towards a Com- tem is capable of normalizing inconsistent-naming mon Framework for Digital Design, Construction and situations and deliver the standard class to Revit Operation, Australasian BIM Advisory Board schedule (Table 4). Along with the BIM populariza- Barbosa, BM, Pauwels, PP, Ferreira, FV and Mateus, ML tion in architectural domain, the system showcases 2016, ’Towards increased BIM usage or existing building interventions’, Structural Survey, 34(2), pp. the possibility of reforming BIM system in a more in- 168-190 telligence way with respect to the human language Biermann, BAW, Ballard, BBW and Sigmon, SAH 1983, ’An understanding. experimental study of natural language program- ming’, International Journal of Man-Machine Studies, 18(1), pp. 71-87 CONCLUSION Bird, BS, Klein, KE and Loper, LE 2009, Natural Language This research project has explored the application Processing with Python, O’Reilly edia Inc of NLP to develop the name standardization work- Chen, CK, ASCE, ASM, Lu, LWS, Wang, WHD, Niu, NYH flow based in BIM environment, the research re- and Huang, HG 2017, ’Naming Objects in BIM: Con- sult demonstrates that the proposed workflow can vention and a Semiautomatic Approach’, Journal of effectively deliver the standard room classifications Construction Engineering and Management, 143(7), p. 6017001 to architectural stakeholders without requiring ex- Chen, CK, Lu, LWS, Peng, PY, Rowlinson, RS and Huang, tra manual work. At the interaction of NLP and BIM, HGQ 2015, ’Bridging BIM and building: From a lit- specific computer-aided strategies are put forward erature review to an integrated conceptual frame- with the intention to mitigate risks resulted from work’, International journal of Project Management, non-standard room-naming. Recalling the primary 33(6), pp. 1405-1416 goal of this research to alleviate inconsistent nam- Duan, DY and Cruz, CC 2011, ’Formalizing semantic of natural language through conceptualization from ing, this research has adopted the NLP-driven ap- existence’, Management and Technology, 2(1), pp. proach to resolve the current room-naming confu- 37-42 sion in BIM and to increase the precision and effec- Duddy, DK, Beazley, BS, Drogemuller, DR and Kiegeland, tiveness of the normalization process, albeit with a KJ 2013 ’A platform - independent product library potential amendment. Regardless, this research pro- for BIM’, Proceedings of the 30th CIB W78 International motes further directions towards naming normaliza- Conference: International Conference on Applications of IT in the AEC Industry, China, pp. 389-399 tion. This leaves open exploration derived from this Eastman, EC 2005, ’Automated Assessment of Early Con- research, with the considerations of reducing the col- cept Designs’, Architectural Design, 79(2), pp. 52-57 laboration complexity from the view of semantic in-

Building Information Modelling - Volume 2 - eCAADe 39 | 19 Fellbaum, FS 1998, WordNet: An Electronic Lexical ceedings Database, MIT Press Pratt, PMJ 2004 ’Extension of ISO 10303, the STEP stan- ABL Group, . 1997, FutureSearchProcessDesign, York Uni- dard, for the exchange of procedural shape models’, versity, Toronto Proceedings Shape Modeling Applications, pp. 317- Hearn, HG and Foth, FM 2005, ’Action Research in the 326 Design of New Media and ICT Systems’,in Kwansah- Rada, RR and Bicknell, BE 1989, ’Ranking Documents Aidoo, KK (eds) 2005, Topical Issues in Communica- with Thesaurus’, Journal of the American Society for tions and Media Research, Nova Science Publishers, Information Science, 40(5), pp. 304-311 Australia, pp. 79-97 Resnik, RP 1999, ’Semantic Similarity in a Taxonomy: An Jung, JN and Lee, LG 2019, ’Automated classification of Information-Based Measure and its Application to building information modeling (BIM) case studies by Problems of Ambiguity in Natural Language’, The BIM use based on natural language processing (NLP) Journal of artificial intelligence research, 11, pp. 95- and unsupervised earning’, Advanced Engineering In- 130 formatics, 41, p. 100917 Ruikar, RK, Anumba, ACJ and Carrillo, CPM 2005, ’End- Kim, KJD, Ohta, OT, Tateisi, TY and Tsujii, TJ 2003, ’GENIA user perspectives on use of project extranets in con- corpus—a semantically annotated corpus for bio- struction organisations’, Engineering Construction & textmining’, BIOINFORMATICS, 19, pp. i180-i182 Architectural Management, 12(3), pp. 222-235 Kumar, KE 2011, Natural Language Processing, I.K. Inter- Salama, SDM, El-Gohary, ENM and ASCE, AAM 2016, national Publishing House, New Delhi ’Semantic Text Classification for Supporting Auto- Lai, LYC, Carlsen, CM, Christiansson, CP and Svidt, SK mated Compliance Checking in Construction’, Jour- 2003 ’Semantic-Web Supported Knowledge Man- nal of Computing in Civil Engineering, 30(1), p. agement System: An Approach to Enhance Collab- 4014106 orative Building Design’, Fourth Joint International Samad, ST and Director, DS 1985 ’Towards a Natural Lan- Symposium on Information Technology in Civil Engi- guage Interface for CAD’, Proceedings of the 22nd neering, Tennessee, pp. 1-15 ACM/IEEE Design Automation Conference, pp. 2-8 Lee, LJK, Lee, LJ, Jeong, JYS, Sheward, SH, Sanguinetti, NBIMS (National Institute of Building Science), . 2012, SP, Abdelmohsen, AS and Eastman, ECM 2012, ’De- National BIM Standard – United States V2, NBIMS-US velopment of space database for automated build- Succar, SB 2009, ’Building information modeling frame- ing design review systems’, Automation in Construc- work: A research and delivery foundation for indus- tion, 24, pp. 203-212 try stakeholders’, Automation in Construction, 18(3), Lewin, LK 1946, ’Action Research and Minority Problems’, pp. 357-375 Journal of Social Issues, 2(4), pp. 34-46 Taylor, TJE 2007, ’Antecedents of successful three- Liddy, LED (eds) 2001, Natural Language Processing, Mar- dimensional computer-aided design implementa- cel Decker Inc, New York tion in design and construction networks’’, Journal of Lin, LJR, Hu, HZZ and Zhang, ZJP 2013 ’BIM Oriented In- Construction Engineering and Management, 133(12), telligent Data Mining and Representation’, Proceed- pp. 993-1002 ings of 30th CIB W78 International Conference on Ap- Venugopal, VM, Eastman, ECM, Sacks, SR and Teizera, plications of IT in the AEC Industry. TJ 2012, ’Semantics of model views for informa- Lorio, LF, Cheong, CH, Li, LW, Shu, SLH, Tessier, TA and tion exchanges using the industry foundation class Bradner, BE 2014 ’Natural Language Problem Defi- schema’, Advanced Engineering Informatics, 26(2), nition for Computer-Aided Mechanical Design’, ACM pp. 411-428 SIGCHI Workshop, Toronto Yang, YQZ and Zhang, ZY 2006, ’Semantic interop- Miller, MGA, Beckwith, BR, Fellbaum, FC, Gross, GD and erability in building design: Methods and tools’, Miller, MKJ 1990, ’Introduction to WordNet: An On- Computer-Aided Design, 38(10), pp. 1099-1112 line Lexical Database’, International journal of lexi- [1] https://spacy.io/ cography, 3(4), pp. 235-244 NATSPEC, . 2011, NATSPEC National BIM Guide, Construc- tion Information Systems Palkovskii, PY, Belov, BA and Muzyka, MI 2011 ’Using WordNet-based semantic similarity measurement in External Plagiarism Detection’, CEUR Workshop Pro-

20 | eCAADe 39 - Building Information Modelling - Volume 2 A Software Concept for Assessing the Risk of Infection in Enclosed Spaces Exemplified by the COVID-19 pandemic

Clemens Lindner1, Christoph Langenhan2, Frank Petzold3, Wulf Schneider-Brachert4, Thomas Holzmann5, Anca Rath6, Matthias Widbiller7 1,2,3Technical University of Munich, Germany 4,5,6,7University Hospital Regens- burg, Germany 1,2,3{clemens.lindner|langenhan|petzold}@tum.de 4,5,6,7{wulf.schneider|thomas. holzmann|anca.rath|matthias.widbiller}@ukr.de

The COVID-19 pandemic has immense impact on our daily life. Precautions like facemasks and social distancing restrict the economy as well as the social life. The aim of the bachelor project was to support architects by integrating health specifications into a software environment and automatically derive specific design support by analyzing architecture. This interdisciplinary project combined architectural and medical expertise to determine the risk of SARS-CoV-2-infection in enclosed spaces by a software approach on basis of parameters provided by the Max Planck Institute for Chemistry (Mainz, Germany) and the Association of German Engineers. Subsequently, calculated risks were evaluated by hygiene experts and summarized in a traffic light scheme, which is clearly intelligible and thus allows simple use. Furthermore, the software concept provides instructions for action for architects and specialists in the field of infection control and suggests architectural tools to minimize the risk of infection. The developed concept can be applied to an existing building and contribute to the planning of new buildings.

Keywords: Building Information Modelling, Parametric Design, Design Support, COVID-19

INTRODUCTION virus 2), was identified on 7 January 2020. Since then, On 31 December 2019, cases of pneumonia of un- it has spread and developed into a global pandemic known cause in Wuhan City, China, were reported to (WHO, 2021). COVID-19 (coronavirus disease 2019) the World Health Organization (WHO). A novel coron- is an acute infectious disease caused by SARS-CoV-2. avirus, SARS-CoV-2 (severe acute respiratory disease The virus enters the body through the mucous mem-

Building Information Modelling - Volume 2 - eCAADe 39 | 21 branes (oral, nasal or conjunctival) and multiplies in ria with regard to space evaluation and infection con- the respiratory system but can also cause damage to trol. Precise guidelines and recommendations are of- other organ systems. The main route of transmission fered and summarized by a traffic light scheme on ba- seems to be the transfer of infectious virus particles sis of following parametric risk assessment: via droplets leaving the respiratory tract (J. Brown, • Instructions for action: number and distribution 2021). of persons, ventilation times, etc. Due to the current worldwide COVID-19 pan- • Structural measures: window size, window re- demic, it has been a challenge to ensure peoples bate ventilators, ventilation system, etc. health through actions like wearing facemasks and social distancing. A review by Brooks et al. (2020) shows that the psychological impact of isolation in- RELATED WORK cludes post-traumatic stress symptoms, confusion Since the beginning of the worldwide pandemic, the and anger. The data published by the Robert-Koch- role of aerosols in the transmission of SARS-CoV- Institute (German federal institute of public health) 2 is still an unsolved question. In-vitro-data has shows that intensive contact in closed rooms plays shown the ability of this virus to stay infectious in an important role in the spread of the pandemic (RKI, aerosols (van Doremalen et al., 2020). However, the 2020). The measures recommended by health au- experimental setting has been critically appraised, thorities are usually complex and depend on numer- as the testing methods do not reflect a real-life sit- ous biological and architectural parameters. While uation. Furthermore, a study involving the out- remarkably effective, these precautions lead to con- break in a restaurant in Wuhan raised the question, siderable restrictions in public spaces such as com- whether interior spaces can be used safely (Lu et panies, universities and schools, which have to be as- al., 2020). Jayaweera et al. (2020) depict in their sessed individually in each scenario and thus repre- review aerosol dynamics, how transmission can oc- sents a major challenge for all responsible parties. cur within closed spaces and why it makes a differ- This project aimed to quantify the risk of trans- ence whether and which facemasks the people in- mission objectively and individually. Rooms in exist- volved wear. Facemasks have proven to be effec- ing buildings were modelled with a BIM authoring tive in both experimental and real-life situations (Le- Tool (Revit) and analysed within a parametric mod- ung et al., 2020). Droplets are absorbed and do not elling and visual programming environment (see fig- contribute to the aerosol formation, which offers a ure 1). Therefore, the Department of Infection Con- benefit to reduce transmission even in closed spaces. trol and Infectious Diseases (University Hospital Re- However, aerosols (particles < 5-10 µm) can still pass gensburg) defined evidence-based evaluation crite- the layers of tissue to some extent and accumulate Figure 1 Parameter extraction from semantic building model into parametric modelling environment

22 | eCAADe 39 - Building Information Modelling - Volume 2 in a room (Leung et al., 2020). The formation of tween 1988 and 1989 (Warren, 1996). Besides wind aerosols and droplets appears to depend on many pressure, thermal buoyancy and airflow paths, many factors such as age, weight, speaking volume or let- parameters were considered by COMIS for multizone ters within a word (Edwards et al., 2021; Asadi et al., airflow modelling from an energetic point of view. 2019, Asadi et al., 2020). The persistence and infec- “Nat-Vent” (Valdes, et al., 2012) is a study that creates tivity of pathogens in aerosols depends on humidity, a parametric natural ventilation simulation with real- sunlight exposure, temperature or ventilation, which time geometric feedback. This parametric tool allows further complicates the evaluation of aerosol impli- estimating the natural ventilation in indoor spaces cations indoor (Marthi, 1994; Morawska et al., 2006). based on the size of rooms and windows, the height This could explain both why the published data are of stories and orientation. Autodesk CFD (Autodesk, contradictory and why social distancing and wearing 2020) is a software for the computer-aided simulation face masks are not always sufficient to prevent trans- of gases and liquids and helps with the prediction mission when there are large numbers of people in and visualization of air movements in interior spaces. an enclosed space or a long period of exposure. Due The company Hexagon took a similar approach by to ethical considerations, data on what the dynam- calculating the number of particles in a test room by ics are in such situations can currently only be de- simulating airflow and water droplets without con- termined retrospectively by evaluating previous out- sideration of the virus behaviour and compared them breaks. using two scenarios with open and closed windows There are only a few publications dealing with (Hexagon, 2020). Many papers focus on the para- the relationship between medicine and architecture. metric calculation and simulation of indoor air move- The Grasshopper add-in “COVID-19 PLANNER” (Plan- ments following different approaches but do not Finder, 2020) assists in planning the arrangement of consider the infectiological consequences. Instead, furniture in floor plans based on existing distance they provide a sufficient foundation for this project. rules mainly about 1.5 m in Germany. A similar ap- proach is used by the company accu:rate GmbH with ASSUMPTIONS their product DISTANSIM, which combines the simu- Our concept and prototypical implementation in a lation of crowds in floor plans with the prevailing dis- parametric model to support architects and users of tance rules (accu:rate, 2021). In this way, crowds can the building regarding infection control is based on be protected from infection by simulating its move- several assumptions. The transmission of COVID-19 ment in advance. In a study by the Max Planck In- is mainly based on three routes as mentioned before: stitute of Chemistry (Lelieveld, et al., 2020), which contact, droplets and aerosols. Even though it is diffi- dealt with the mathematical-chemical calculation of cult to assess, we chose to consider aerosols as main the transmission of aerosols and the risk of infection parameter as the other two routes of transmission are in indoor spaces, the probability of infection is deter- considered as constant factors that can be controlled mined based on room parameters and room use. by facemasks and disinfectants. The ventilation of rooms and the corresponding Moreover, our model is based on the assump- air exchange rate play an important role in minimis- tion, that people are distributed evenly throughout ing the risk of infection. Therefore, numerous scien- the room and that facemasks are worn permanently. tists conducted research on the calculation and sim- The scenario of entering and leaving the room was ulation of air movements in enclosed spaces. A de- not taken into consideration due to several parame- tailed study of the analysis of indoor air movement ters, which are hardly reproducible. The high simu- was done in a workshop at Lawrence Berkeley Na- lation effort and costliness in terms of software tech- tional Laboratory by 12 International Scientists be- nology for user-friendly software appear unfeasible

Building Information Modelling - Volume 2 - eCAADe 39 | 23 and are not representative for an implementation in Figure 2 the highly variable scenario of daily life. Simulations Interior perspective of indoor airflows are only useful if all parameters of the seminar are exact. However, this is not the case for assess- room of the Chair of ing the risk of infection, as the behaviour of people Architectural in the room cannot be predicted and thus, for exam- Informatics of the ple, the exhalation airflow direction cannot be simu- Technical University lated. The tool is therefore a planning software for of Munich architects and provides guidelines and instructions for action that allow the risk of infection in the room to be assessed based on the calculations of the Max Planck Institute of Chemistry (Lelieveld, et al., 2020). For the parameter “room air exchange”, the air exchange rate of the ventilation system and the win- dows are used, which assumes that the supply and exhaust air is evenly and optimally distributed and exchanged in the room. This also applies to the posi- tion of the windows in the room. An iterative adaptation of the parametric model Regarding the air exchange rate for the window and the parameters to new scientific findings is pos- openings, only the proportion of the air exchange sible and obligatory. This project is carried out us- rate that can be determined by VDI2078 (The Asso- ing the example of the COVID-19 pandemic but can ciation of German Engineers, 2015) on the basis of be adapted to any other infectious disease spread via the temperature differences between outdoor and the respiratory tract. indoor temperature is calculated. Airflows due to winds in the outdoor area are not considered here in CONCEPT order to simulate a “worst-case” scenario. The win- dow is not only part of the wall, but by definition an Parameter overview element that can be fully opened, tilted and closed to Individuals are seen as hosts of infection, which are allow air exchange between the interior and exterior defined by parameters such as proportion of speech at a predefined opening frequency. Therefore, the and respiratory volume. Based on the study of the air exchange from the indoor to the outdoor space is Max Planck Institute of Chemistry (Lelieveld, et al., significantly higher in winter. A larger window must 2020), parameters are determined and evaluated be ventilated less frequently or for a shorter period from a medical and architectural point of view (see of time. In comparison, small windows must be ven- table 1) and the object of investigation is the semi- tilated more frequently or longer in order to ensure nar room of the chair of architectural informatics (see an equivalent air exchange. The aforementioned de- figure 2). pendencies are also considered depending on the number of people or the type of event.

24 | eCAADe 39 - Building Information Modelling - Volume 2 Table 1 Method List of assessed As an example, the seminar room of the Chair of Ar- parameters based chitectural Informatics of the Technical University of on a study of the Munich (see figure 2) was modelled with a BIM au- Max Planck Institute thoring Tool (Revit) with a LOD (Level of Develop- (Lelieveld, et al., ment) of 200 and via the Revit.Inside interface, room 2020) parameters are imported directly from the semantic model into the parametric modelling and visual pro- gramming environment of Grasshopper3D. Since ob- jects from the semantic building model are used, the workflow described in the paper could be adapted to other buildings. However, the real strength of the approach is the support of the iterative character of the architectural design process. For example, if a better window type or a larger wall opening is sug- gested, the designer can immediately implement it in the digital semantic building model (BIM) and evalu- ate it again with the parametric model in the visual programming environment. Therefore, the parameters listed in Table 1, which can be adapted to different scenarios (see figure 3) by the designer via inputs into a parametric modelling environment (characteristics of the infected person, event details, aerosol properties and virus proper- ties) or imported directly from the semantic building model (room details), are processed in a Grasshop- per3D component programmed (see figure 4) based on the study by the Max Planck Institute (Lelieveld, et al., 2020) and includes the calculation of the air ex- change rate using VDI 2078 (The Association of Ger- man Engineers, 2015) (see figure 1). The addition of further parameters is possible at any time and allows us to adapt the concept to the latest state of research in terms of disease transmission and control. For ex- ample, the local incidence rates or a change in infec- tivity in case of pathogen mutation could be taken into account. This enables to determine the risk in a region according to current data and adjust it in real- time.

Building Information Modelling - Volume 2 - eCAADe 39 | 25 Figure 3 Different scenarios using the example of the Technical University of Munich seminar room with corresponding risk probability

Resulting risks of infection: Figure 4 Schematic • Individual risk of infection if another participant illustration of the is highly infectious. Grasshopper • Probability of at least one other participant be- algorithm coming infected if another participant is highly infectious • Probability of an infected person coming to the event • Individual risk of infection As an example, for different use cases, the seminar room of the Chair of Architectural Informatics at the Technical University of Munich was used to develop and test a realistic scenario. Based on three com- mon types of events (research group, regular event and kick-off event), a corresponding risk of infection was determined using the prototypical implementa- tion and generalised assumptions about the number of participants, duration, proportion of speeches and activity. Figure 3 provides an overview of the scenar- Results ios and the corresponding risks of infection if at least The room properties imported from the semantic one other participant becoming infected if another building model and the previously defined parame- participant is infectious. ters from the parametric modelling environment can then be linked directly as text inputs. The intervals Visualisation of the results were determined based on the recommendation of Current studies have not yet succeeded in estimat- the Max Planck Institute. The results are displayed ing the virulence and transmission probability of in an easy-to-understand traffic light scheme on the SARS-CoV-2 via aerosols with sufficient significance components in the BIM authoring tool. The traffic to make reliable conclusions about the risk of infec- light scheme was defined by the University Hospital tion. To estimate the risk of infection and give an easy Regensburg. to understand guidance, a traffic light system (see fig-

26 | eCAADe 39 - Building Information Modelling - Volume 2 ure 5) with regard to infection protection, depending The worst-case scenario is thus always considered on e.g. ventilation intervals or the number of people and used as a reference for simplified guidance for is proposed. group events or meetings. These primarily include educational institutions, office buildings or social and cultural meeting places. The results are assigned to Figure 5 three hazard levels: <25%, 25-75%, >75%. The traffic light “Probability of at least one other participant be- indicator visualises coming infected if another participant is highly conta- the optimisation gious” process in Revit • 0-25 % (green): A low probability of infection for another person is assumed. For this range, no further measures are recommended apart from the parameters already self-defined or de- termined by the software (masks, window venti- lation, number of participants, etc.). • 26-75 % (orange): A medium probability is as- sumed. In this case, it is recommended to intro- duce measures that can reduce the probability of transmission. In this constellation, the use of the software is suitable to determine how these parameters should be modified (e.g. a larger room or a smaller number of participants) to re- duce the probability of transmission. The risk can be specified and actively modified by the user by adaption of the input parameters. The recommendations only refer to the modification of the associated and predefined parameters. In a second step, optimisation measures can be ap- plied in this case. • 76-100 % (red): A high probability of infection for another person is assumed. For this reason, the design of the parameters should be adapted or, if necessary, the event should be cancelled or moved to other premises. This visualisation aims to be easily understand- able for designers and can be used in the future to evaluate other use cases e.g. classrooms or restau- rants with regard to reducing the risk of infection. This would enable users to make the most sensible measures with little effort. It should be emphasised that the reliability of this assessment is always depen- dent on compliance with predefined parameters (e.g.

Building Information Modelling - Volume 2 - eCAADe 39 | 27 wearing masks, keeping a distance, maximum num- room at the same time if they comply with the pre- ber of participants). viously determined speaking-time-percentage and “Individual risk of infection if a participant is highly ventilation intervals, while at the same time minimis- contagious” ing the risk of infection. This assessment can theoretically be made On the other hand, existing buildings can also be equivalent to the above. In this case, the individ- architecturally optimised or a design can be adapted ual contact with the highly contagious person must to the given infection control measures. Through also be taken into account. In static structures, where the bi-directional interface of the semantic building everyone has their assigned seat, this classification is model and the parametric modelling environment, certainly of little/no relevance. Currently, the risk of parameters from the semantic building model can transmission during events (dance evenings, sport- be used as in the previous analysis. However, pa- ing events, concerts, etc.), cannot be clearly cate- rameters can also be changed directly from the para- gorised from an infection control point of view but metric modelling environment. For example, the di- appears to be highly relevant for calculating and as- mensions of the windows or the room volume can be sessing the probability of transmission. changed, which leads to better air exchange and in turn minimises the risk of infection. Optimisation All window types, which are located as families The evaluated results can then be used bi- in the loaded Revit model, are also analysed and pro- directionally for optimisation. On the one hand, the vided in a bar chart with the infection risk corre- various parameters can be varied manually to con- sponding to the room. In this way, the different win- figure specific use cases or scenarios and either cal- dow types can be compared with each other and ex- culate action instructions for the users or support the changed in the Revit model. The mesh component in planner. Furthermore, a generic solver (Galapagos Grasshopper also visualises the corresponding infec- for Grasshopper) can be used to optimize the best tion risk by colouring the windows in Revit with the combination from the sum of the different parame- colours of the traffic light scheme (see figure 5). ters with regard to e.g. a certain protection target or the air exchange rate using evolutionary algorithms IMPLEMENTATION (see figure 4). This can be helpful when analysing Figure 6 shows how parameters can be extracted existing buildings. For example, a company can find and modified in a bidirectional way from a seman- out the maximum number of people that can be in a tic building model (BIM), e.g. Revit into a paramet-

Figure 6 Interaction between semantic building model and parametric model in a visual programming environment

28 | eCAADe 39 - Building Information Modelling - Volume 2 ric model and processed within a visual program- one other participant will become infected if another ming language (see figure 4) and suitable add-ons participant is highly infectious. Even if this probabil- for Grasshopper3D. The main contribution of this ity is assessed by infection control experts and cal- paper is a workflow to integrate the computational culated parametrically, the unpredictability of infec- methods of the Max Planck Institute using paramet- tion remains because of too many parameters (e.g. ric modelling and visual programming into an object- wind) that can’t be considered by a computational oriented BIM authoring tool and to display and eval- approach. A critical questioning of the resulting risks uate it with add-ons for the visual programming en- and a responsible implementation of the recommen- vironment. The semantic information of BIM is used dations are still required. to evaluate and propose semantic objects like win- The tool can be used for risk assessment and min- dow types to the planner. In addition to the sup- imisation in order to evaluate or improve spaces. It port of planning in terms of infectious diseases such can also be used in future planning of, for exam- as COVID-19, the information is used to evaluate ple, working areas in newly built office buildings that window types with a traffic light system (see fig- comply with current infection protection measures. ure 5) with regard to infection protection, depend- This way, working in the office becomes possible ing on ventilation intervals or the number of peo- even during pandemics. ple for example. This interaction between the two Figure 7 programmes makes it possible to benefit from the Vision of an app to advantages of both. Not only can all the properties evaluate the risk of found in the semantic building model be imported infection in indoor directly into the parametric modelling environment, spaces but these properties can also be modified accord- ingly, which plays an important role in the optimisa- tion process.

CONCLUSION Supporting the architectural design process with knowledge engineering methods is an efficient goal- oriented approach because e.g., for simulations a much higher modelling and computational effort is necessary because the built environment has to be formalized very extensively (e.g., wind, pressure dif- ferences). In the approach presented in the paper, medical and architectural issues are modelled with respect to the risk of infection with the coronavirus. In addition to supporting this use case, this approach is transferable to numerous other interdisciplinary OUTLOOK problems. The presented approach based on a Bachelor project Dealing with probabilities and risks can be seen as a first concept study. Since the ba- The developed concept aims on estimating the sis from the architectural standpoint is established, risk of infection in closed rooms of already built or just a clinical test can take place in the next step. This planned buildings on the basis of MPIC (Lelieveld, et cannot be carried out easily because of ethical con- al., 2020). It considers the probability that at least siderations and the experimental limitations due to

Building Information Modelling - Volume 2 - eCAADe 39 | 29 the current pandemic. Facemasks restrict the aerosol and risk factors of disease symptom severity’, PNAS, generation so that the data which would be col- 118(8), pp. 1-6 lected on aerosol reduction by the defined param- Hexagon, AB 2020, CFD Analysis of a small social gather- ing (COVID-19), MSC Software eters would not be transferable to a non-pandemic Verein Deutscher Ingenieure, e.V. 2015, A3 Naeherungs- situation (without facemasks). If this will be success- formeln fuer den Luftaustausch ueber Fenster fuer VDI ful in the near future, an app development, for exam- 2078, Beuth Verlag GmbH, Berlin ple for controlling ventilation systems, or a software Jayaweera, M, Perera, H, Gunawardana, B and Man- tool for planning assistance can be initiated in a pilot atunge, J 2020, ’Transmission of COVID-19 virus by project based on this preliminary study. droplets and aerosols: A critical review on the unre- solved dichotomy’, Environmental research, 188, pp. One possible example is the development of an 1-19 app that allows planners and users of rooms to de- Lelieveld, J, Helleis, F, Borrmann, S, Cheng, Y, Drewnick, termine the geometric framework conditions by us- F, Haug, G, Klimach, T, Sciare, J, Su, H and Pöschl, ing a depth camera or photogrammetry of the room U 2020, ’Model Calculations of Aerosol Transmis- and additional information on user behaviour to de- sion and Infection Risk of COVID-19 in Indoor En- termine the risk of infection of an indoor space (see vironments’, Int. J. Environ. Res. Public Health, 17 (21)(8114), pp. 1-18 figure 7). Leung, NHL, Chu, DKW, Shiu, EYC, Chan, KH, McDevitt, JJ, Hau, BJP,Yen, HL, Li, Y, Ip, DKM, Peiris, JSM, Seto, WH, REFERENCES Leung, GM, Milton, DK and Cowling, BJ 2020, ’Respi- Asadi, S, Wexler, AS, Cappa, CD, Barreda, S, Bouvier, ratory virus shedding in exhaled breath and efficacy NM and Ristenpart, WD 2019, ’Aerosol emission and of face masks’, Nat Med, 26, pp. 676-680 superemission during human speech increase with Lu, J, Li, K, Xu, C, Su, W, Yu, C and Yang, Z 2020, ’COVID- voice loudness’, Sci Rep, 1(2348), pp. 1-10 19 Outbreak Associated with Air Conditioning in Asadi, S, Wexler, AS, Cappa, CD, Barreda, S, Bouvier, NM Restaurant, Guangzhou, China, 2020’, Emerging In- and Ristenpart, WD 2020, ’Effect of voicing and artic- fectious Diseases, 26(7), pp. 1628-1631 ulation manner on aerosol particle emission during Marthi, B 1994, ’Resuscitation of microbial aerosols’, in human speech’, PloS one, 15, pp. 1-15 Lighthart, B and Mohr, AJ (eds) 1994, Atmospheric Brooks, SK, Webster, RK, Smith, LE, Woodland, L, Wessely, Microbial Aerosols, Springer, Boston, pp. 182-225 S, Greenberg, N and Rubin, GJ 2020, ’The psycholog- Morawska, L and Milton, DK 2020, ’It Is Time to Address ical impact of quarantine and how to reduce it: rapid Airborne Transmission of Coronavirus Disease 2019 review of the evidence’, The Lancet, 395(10227), pp. (COVID-19)’, Clin Infect Dis., 71(9), pp. 1-9 912-920 RKI, Germany 2020, ’Infektionsumfeld von COVID-19- Brown, J, Gregson, FKA, Shrimpton, A, Cook, TM, Bzdek, Ausbrüchen in Deutschland’, Epidemiologisches Bul- BR, Reid, JP and Pickering, AE 2020, ’A quantitative letin, 38, p. 4 evaluation of aerosol generation during tracheal in- Valdes, F and Sun, Y 2012 ’Parametric Natural Venti- tubation and extubation’, Anaesthesia 2020, 76, pp. lation Simulation with Real-time Geometric Feed- 174-181 back (Nat-Vent)’, Proceedings of the 16th Iberoamer- van Doremalen, N, Bushmaker, T, Morris, DH, Holbrook, ican Congress of Digital Graphics, Brasil, pp. 436-439 MG, Gamble, A, Williamson, BM, Tamin, A, Harcourt, Warren, P (eds) 1992-1996, Multizone Air Flow Modelling JL, Thornburg, NJ, Gerber, SI, Lloyd-Smith, JO, de Wit, (COMIS), Energy Conservation in Buildings and Com- E and Munster, VJ 2020, ’Aerosol and Surface Sta- munity Systems (ECBCS) bility of SARS-CoV-2 as Compared with SARS-CoV-1’, [1] https://www.planfinder.xyz/covid-19-planner The New England Journal of Medicine, 382(16), pp. 1- [2] https://www.autodesk.com/products/cfd/overview 4 [3] https://www.accu-rate.de/en/distansim-en/ Edwards, DA, Ausiello, D, Salzman, J, Devlin, T, Langer, R, [4] https://www.euro.who.int/en/health-topics/health- Beddingfield, BJ, Fears, AC, Doyle-Meyers, LA, Red- emergencies/coronavirus-covid-19/novel-coronavirus- mann, RK, Killeen, SZ, Maness, NJ and Roy, CJ 2021, 2019-ncov ’Exhaled aerosol increases with COVID-19 infection,

30 | eCAADe 39 - Building Information Modelling - Volume 2 Finding Optimal Path Planning Method for Building Navigation in BIM

Michał Jarzyna1 1Lodz University of Technology, Faculty of Civil Engineering, Architecture and Environmental Engineering, Department of Digital Technologies in Architecture and Urban Planning, Aleja Politechniki 6a, 90-924 Lodz, Poland [email protected]

Building's spatial structure described in a BIM model can be used for retrieving the information required for determining the shortest path within the building. The matrix method and the visibility method are the two main ways of dividing space into prime factors. Both are widely used to find the shortest path. In order to compare the performance of both methods, several tests were carried out with various versions of the floorplan modification (room area, the surface of internal walls, distance between the entrance, and exit in a straight line and within the boundary marked by walls of the maze). The results revealed significant differences between the visibility graph and the matrix method.

Keywords: BIM, Building information modeling, Facility management, FM, Routing in building

INTRODUCTION obvious “design and build” (Volk, Stengel and Schult- Navigating inside a building differs from finding the mann, 2014). Thanks to automating many decision- road between points A and B on a street map and is making processes and increasing work efficiency, it similar to finding the shortest route in an open space. can be successfully used in tasks related to the ongo- While the street grid is a ready-made material to be ing maintenance of the building (Chen et al., 2018). processed by a search algorithm, the interior of the The work aims to find the optimal solution for de- building is a bigger challenge as it is a set of rooms termining the shortest route between points in the that are not linear and that are often irregular. The building. A model made in Revit is a source of spa- BIM model, as a carrier of data about the spatial struc- tial information and a script created in Dynamo is ture of a building, is an ideal source of information al- used to determine the shortest route. The lengths of lowing to determine the shortest route between var- the mapped routes and the times needed to perform ious points in the facility. A script must translate a the calculations are recorded and compared. The re- BIM-geometry to a graph build of points connected search is conducted in rooms (labyrinths) with vary- with lines. ing degrees of complexity to find the major factors Creating an effective building navigation tool is influencing the performance of the calculations and an example of the use of a BIM model throughout perform their effectiveness. the entire life cycle of a building beyond the most

Building Information Modelling - Volume 2 - eCAADe 39 | 31 LITERATURE REVIEW more specifically its corners, which become nodal Decomposing an open space can be achieved in sev- points. Then, from each point, the maximum number eral ways. For example, transformations of space’s of lines is drawn to the corners that are “visible” from boundaries may be used to investigate it. Meth- a given place. Visible, that is, a straight line between ods based on a visibility graph (Masehian and Amin- the points can be led and this line does not intersect Naseri, 2004; Krisp, Liu and Berger, 2010) and a skele- with any elements of a building structure. ton method (Haunert and Sester, 2008) are included As a result of both transformations, space be- in this group. Another choice is to use a matrix comes a mesh, and an algorithm can calculate the method, which is built on a matrix of points con- route between the points. In this paper, both the nected to each other either by rectangular grids or abovementioned methods are compared in terms grids expanded by the Delaunay triangulation and of efficiency and applicability together with the BIM allows for Spider method routing (Hahmann et al., model. 2018). According to the research presented in (Hah- mann et al., 2018), none of the algorithms listed RESEARCH TOOL above is universal, but the visibility graph and the The analysis was carried out for a maze-shaped space Spider algorithm are the most promising. As a result, with just one entry and exit. The room’s walls were only these two methods will be considered further. entered at random. The study was carried out in Au- As the matrix method is based on a set of points todesk Revit 2018 with Dynamo 2.0.2.6833, an ad- connected with each other, there is some degree of vanced add-on for visual programming. In Dynamo, user interference in this method, as the researcher the experiments were made using the same script. typically assumes the density of the grid in compari- Since Revit is one of the most common BIM pro- son to the evaluated space (Graser, 2016; Hahmann grams and is closely linked to Dynamo, a visual pro- et al., 2018). The researcher evaluates the size of gramming application, this software environment the space and chooses between a “dense grid” and was chosen. In Revit 2020 the new tool “Path of a “loose grid” at choice. There were not found refer- travel” was added. This tool finds the way between ences describing attempts to test this method in ob- two given points on the selected level. This tool uses jective criteria and to find a balance between the ad- the visibility method to find the way, although it can- vantages and disadvantages of the grid-based matrix not operate between levels so it is a closed internal approach. It is believed that a higher grid density im- tool suitable rather for calculating evacuation times proves path mapping accuracy (Chen et al., 2018). Al- during the design phase than building navigation. though this is right, increasing the grid density is not Overall, Revit + Dynamo proved to be a useful necessarily advantageous in terms of maximizing the tool for performing the study. The computer was not planning process since it increases the computation used for any other purposes during the calculations, time (Rüppel, Abolghasemzadeh and Stübbe, 2010). and all analyses could be considered to be conducted This approach has certain primary benefits, such as in equal conditions. The Dynamo script used in the interpolating points within the open space and gen- analysis extracted information from the BIM model erating routes with lengths similar to the shortest about the room, walls (graph edges), and doors (the paths, as well as the biggest drawback of accurate entry and exit points). measurement - an improvement in calculation time The obtained input data are transformed de- (Hahmann et al., 2018). pending on the chosen research method. As men- In the second method, using the visibility graph, tioned, the method based on the visibility graph does a researcher cannot add their subjective attitude, as not require the user to enter additional information. space is analyzed in the context of its boundaries, The visibility method provides a graph created by

32 | eCAADe 39 - Building Information Modelling - Volume 2 connecting all the vertices in the room. Vertices are RESEARCH generated in Dynamo by converting the boundaries The maze variants slightly differed from one another. of the room to a polygon. The figure presented below Each of 1-4 series of experiments was conducted on a (Fig. 1) shows a visibility graph created in a labyrinth maze with the same wall structure but different door which is one of the case studies in the research. locations (Tab. 2-5). In the first sequence, the doors were on both sides of the labyrinth (Tab. 2). In con- Figure 1 trast to the first test, the doors in the second series Floorplan of the were closer to each other (length in a straight line), maze used in Series but at a comparable distance of the maze passage 1-4 with the (Tab. 3). In a straight line, the doors in the third series visibility graph on it were identical to the doors in the second series, but the path to the passage was almost twice as short as in Series 2 (Tab. 4). The Series 4 doors were designed to create the longest passageway possible (Tab. 5). The value of the coefficient x (grid density), Dynamo’s script execution time, and the route length were all Figure 2 reported for each sequence. Table 1 displays the clas- Floorplan of the sification scheme. maze used in Series All 4 layouts are presented below with the solu- 1-4 with the matrix tion for the matrix method for the grid x=30. grid on it Table 1 Description scheme for each test

The matrix method can change the density of the Table 2 grid of points. The number of segments along the x Results obtained for and y axes was the rounded down result of dividing x=30 for Series 1 the length and width of the room by the user-defined factor x representing the “grid density” (Fig. 2). The densities were lowered by 10 from 100 to 10. In both methods, it was assumed to find the shortest way connecting points. It should not be as- sociated with a path planned for humans or indepen- dent devices. In order to prepare to polygon for plan- ning routes for things with a given size, the border of the room would need to be offset.

Building Information Modelling - Volume 2 - eCAADe 39 | 33 The charts below present the data obtained for Table 3 each of the 10 variants of the variable x (Fig. 3-6). Results obtained for In the matrix method, the general trend of reducing x=30 for Series 2 route length while increasing mesh density was re- ported.

Figure 3 Visual comparison of results for Series 1

Table 4 Results obtained for x=30 for Series 3

Figure 4 Visual comparison of results for Series 2

Table 5 Results obtained for x=30 for Series 4

34 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 5 With the visibility method, the route is known to be Visual comparison the shortest because it connects in a straight line key of results for Series decision points where the change of direction takes 3 place (Tab. 6-9).

Table 6 Results obtained for visibility graph for Series 1

Although the route with the shortest distance is given by the densest mesh, the time of calculation Table 7 grows significantly after exceeding the value x = 30. Results obtained for For this reason, the value of x=30 may be identified visibility graph for as most optimal in terms of comparing the optimal Series 2 length of the road and the time of its acquisition. Value x=100 gave the shortest calculation time, but the path was relatively long. The situation is quite the opposite in the case of x=10 when the route was short but the time of calculation was disproportion- ately elongated.

Figure 6 Visual comparison of results for Series4 Table 8 Results obtained for visibility graph for Series 3

Building Information Modelling - Volume 2 - eCAADe 39 | 35 In the comparison of the calculation times, it can Table 9 be seen that they are even shorter compared to the Results obtained for value of x = 100 for the matrix grid (Tab. 10-11). visibility graph for In order to confirm the findings, a series of fur- Series 4 ther tests were conducted. For the comparison, x = 30 was chosen as the optimal choice for the matrix version and x = 100 as the variant of the shortest com- putation time. This supports the hypothesis that in the matrix method the measurement time is proportional to the scale of the space under consideration. The number of points that must be computed is directly propor- tional to the size of the space. This implies that dis- covering all possible variations is the most computa- Table 10 tionally challenging activity of the script deciding the Comparison of shortest path. The changing surface does not impact results obtained for the performance of the visibility method as it is de- matrix method pending on the number of vertices and not the dis- (x=100, x=30, x=10) tance between them. and visibility graph To validate the above-mentioned hypothesis, for Series 1-2 additional maze-complicating walls were applied to the Series 5 floorplan in Series 6. (Tab. 14-15). The length of the road rose more than twice, re- sulting in a substantial improvement in the route’s width and measurement time. The increase in com- plexity of the room outline has a direct impact on a Table 11 number of connections between edges in the visibil- Comparison of ity graph, but the objective time extension is insignif- results obtained for icant compared to the matrix method. matrix method (x=100, x=30, x=10) and visibility graph for Series 3-4

36 | eCAADe 39 - Building Information Modelling - Volume 2 Table 12 CONCLUSIONS Results obtained for As mentioned in the Introduction chapter, the study matrix method aims to broaden the knowledge of the possible ap- (x=30) for Series 5 plication of the BIM model throughout the building life cycle. By examining further potential applications of this model, the chances of further use of BIM doc- umentation prepared for construction are increased and the attractiveness of using BIM in existing build- ings increases. The findings of research into identifying the shortest path between points A and B using the point matrix in the BIM model are presented in this arti- cle. The predicted increase in computing time with increasing mesh density, as stated in the literature, was positively verified (Chen et al., 2018; Hahmann et al., 2018). The relationship between the mesh density and the efficiency of the matrix method is not linear and attempts can be made to indicate the optimal re- lationship between the accuracy of the route and the Table 13 calculation time. The second of the tested methods, Results obtained for the visibility method, turned out to be more efficient visibility graph for both in terms of route length and calculation time. Series 5

Table 14 Results obtained for matrix method (x=30) for Series 6

Building Information Modelling - Volume 2 - eCAADe 39 | 37 Polygon Routing for Pedestrian Navigation’, Proceed- Table 15 ings of 7th International Symposium on LBS & TeleCar- Results obtained for tography, Guangzhou, China visibility graph for Masehian, E and Amin-Naseri, MR 2004, ’A voronoi Series 6 diagram-visibility graph-potential field compound algorithm for robot path planning’, Journal of Robotic Systems, 21(6), pp. 275-300 Ruppel, U, Abolghasemzadeh, P and Stubbe, K 2010 ’BIM-based immersive indoor situations in build- ings’, Proceedings of the International Conference on Computing in Civil and Building Engineering Volk, R, Stengel, J and Schultmann, F 2014, ’Building In- formation Modeling (BIM) for existing buildings - Lit- erature review and future needs’, Automation in Con- struction, 38, pp. 109-127

For this reason, it is a good method to implement in a tool that allows users to search for the shortest route between rooms in the entire building. The already mentioned adaptation of the route to be traveled by a man should be implemented, by assuming an ap- propriate distance from the wall in decision points.

REFERENCES Chen, W, Chen, K, Cheng, JC, Wang, Q and Gan, VJ 2018, ’BIM-based framework for automatic schedul- ing of facility maintenance work orders’, Automation in Construction, 91(February), pp. 15-30 Graser, A 2016, ’Integrating Open Spaces into Open- StreetMap Routing Graphs for Realistic Crossing Be- haviour in Pedestrian Navigation’, GI\_Forum, 1, pp. 217-230 Hahmann, S, Miksch, J, Resch, B, Lauer, J and Zipf, A 2018, ’Routing through open spaces – A perfor- mance comparison of algorithms’, Geo-spatial Infor- mation Science, 21(3), pp. 247-256 Haunert, JH and Sester, M 2008, ’Area Collapse and Road Centerlines based on Straight Skeletons’, GeoInfor- matica, 12(2), pp. 169-191 Krisp, J, Liu, L and Berger, T 2010 ’Goal Directed Visibility

38 | eCAADe 39 - Building Information Modelling - Volume 2 The Use of Diagrammatic Reasoning to Aid Conceptual Design in Building Information Modeling (BIM)

Nancy Alassaf1, Mark Clayton2 1University of Jordan 2Texas A&M University [email protected] [email protected]

Architectural design is an intellectual activity where the architect moves from the abstract to the real. In this process, the abstract represents the logical reasoning of how architectural form is configured or structured, while the real refers to the final physical form. Diagrams become an integral part of the conceptual design stage because they mediate between those two realms. Building Information Modeling (BIM) can reallocate the effort and time to emphasize conceptual design. However, many consider BIM a professionally-oriented tool that is less suitable for the early design stages. This research suggests that architectural design reasoning can be achieved using constraint-based parametric diagrams to aid conceptual design in BIM. The study examines several techniques and constructs a framework to use diagrams in the early design stages. This framework has been investigated through Villa Stein and Citrohan House by Le Corbusier. This study addresses two roles of diagrams: the generative role to create various design solutions and the analytical one to conduct an early performance study of the building. Our research contributes to the discussion on the ways designers can use digital diagrams to support the architectural design process.

Keywords: Building Information Modeling (BIM), Performance analysis , Architectural Form, Diagram, Parametric modeling

INTRODUCTION has been given to the best way to support the con- The key aspect of architectural design is concerned ceptual design stage and capture the logic of archi- with the intellectual pursuit of creating distinctive tectural form within BIM. As a test case to see if BIM is artistic forms (Norberg-Schulz, 1965). Designers use sufficient for exemplifying the formal qualities of ar- various diagrams to develop abstract and incom- chitecture in an abstract diagram, we have mapped plete design solutions that respond to some re- the syntactic relationships and vocabulary of Le Cor- quirement considerations in the early stages of de- busier’s architecture in Villa Stein and Citrohan house sign. While Building Information Modeling (BIM) into a BIM template. Using BIM, this research ad- has gained wide acceptance in support of architec- dresses three fundamental concerns in design, using tural design production, diminutive consideration abstract diagrams to illustrate an architectural form’s

Building Information Modelling - Volume 2 - eCAADe 39 | 39 logic, generating new designs, and conducting per- solutions that aim to explore the best alternatives formance evaluation. This paper addresses the ques- (Horváth, 2000). Eastman et al. (2008, p. 203) state tion of whether BIM can support the early stages of that the stage of conceptual design in architecture design and express formal knowledge through a de- defines “the basic framework of the design to be de- fined set of syntactical tools and design vocabulary. veloped in later stages, in terms of its massing, struc- We propose that BIM supports buildings’ production ture, general spatial layout, approach to environmen- and management and can be used to represent ar- tal conditioning, and response to site and other lo- chitectural forms explicitly. cal conditions.” This research focuses on the mass- ing and the spatial layout issues of conceptual design BACKGROUND and how they affect energy performance. Our research is based on foundations in architectural design theories, BIM tools and theory, and writings of Diagrammatic Reasoning Le Corbusier’s architecture. The conceptual design process is one of the most challenging yet crucial stages of design as it neces- Conceptual Design in Architecture sitates advanced cognitive skills. In this process, dia- Architectural design is concerned with the intel- grams enable designers to simplify the design prob- lectual process of creating distinctive architectural lem into high abstraction levels. The word diagram forms. The discussion of architectural form, the result derives from the Greek dia- meaning ’across, out or of the creative process, extends beyond the material between two and gramma meaning ’figure, mark or structure or the physical presence of an object. Ac- line that is made’ (Gracia, 2010, p. 22). cording to Stiny and Gips (1978), it deals with how In the conceptual design stage, diagrams allow an existing work is organized according to unique the designer to identify and concentrate on a dis- principles that lead to new meanings and orders. In tinct characteristic of the design while maintaining that sense, the notion of form or what Langer (1967) a broad view of the whole (Baker, 1996). They pro- called ‘logical form’ transcends the object’s physi- vide a connection between the part and the whole in cal appearance to include the intentional process of design (Dulić & Aladžić, 2016). Moreover, diagrams how it was structured or configured. As a result, “make geometric articulation explicit” (Baker, 1996, Knight (1981) has argued that architectural design p. 66). They help the architect elaborate the rela- can be understood, compositionally, as a complex set tionship between the abstract formal concepts and of vocabularies governed by grammatical spatial re- the actual architectural space (Do & Gross, 2001). lations. Alexander (1964) described diagrams as “any pat- In the early phase of the design, designers spend tern, which by being abstracted from real situation most of their time postulating solutions and reason- conveys the physical influence of certain demands ing about possible design solutions. This phase is or forces.” Eisenman (1999) suggests that diagrams called conceptual design. From a methodological can bridge the abstract logical reasoning of how an perspective, conceptual design refers to the early architectural form is structured and the final physi- stage of a product development that includes inves- cal or specific form. Similarly, Dogan and Nersessian tigation, product idea generation, and some require- (2002) argue that diagrams exemplify the core of con- ment consideration (Horváth, 2000). Horváth (2000, ceptual design because they highlight form’s logic p. 4) asserts that “conceptual design is exclusively through its spatial configuration. According to Dulić based on the inherent human capabilities such as and Aladžić (2016), the diagram is a fundamental way intuition, creativity, analysis and synthesis.” Thus, to produce new knowledge through drawing in ar- this design stage provides abstract and incomplete chitecture. Diagrams, as a consequence, “allow a de-

40 | eCAADe 39 - Building Information Modelling - Volume 2 gree of artistic license” (Baker, 1996, p. 66) since they oped to offer a context for creating digital design di- can express the logic of form and the architect’s artis- agrams. Parametric systems provide an opportunity tic style (Clayton, 2012). to construct diagrams using graphical, numbers, and In architecture, the diagram has traditionally analog representations, broadening diagrams’ scope been interpreted in two ways: a generative device to include cognitive function beyond the pencil and and an explanatory or analytical device. For instance, napkin ( Gracia, 2010). In that sense, parametric sys- Eisenman (2015), in his book Palladio Virtuel, utilized tems enable designers to express design knowledge diagrams as an analytical device to uncover the for- and generate potential design solutions controlled mal logic of Palladio’s work. Moreover, in Diagram Di- through an explicit sequence of logical operations. aries, Eisenman (1999) addressed the diagrammatic Moreover, they allow designers to include geometri- techniques that provoke the emergence of form cal and no-geometrical information, such as lighting, through a sequence of operations such as transfor- at early stages (Gómez, 2010) . Nevertheless, further mations, decomposition, scaling, grafting, rotation, research into the interaction of parametric systems superimposition, shifting, etc. and conceptual diagrams to express design knowl- Even though diagrams are essential to architec- edge in practice and education is still needed (Clay- tural works, they are often obscured, implied, or even ton, 2012, 2014). disguised (Eisenman, 1999; Gracia, 2010). Besides that, there is a paucity of literature on the relationship BIM and Conceptual Design between architectural diagrams and computer-aided Today, BIM represents an epochal shift that has pro- design systems (Gracia, 2010). It has long been con- foundly changed the process and product of archi- sidered that computer-aided design systems fail to tectural design. BIM can represent architectural ele- support conceptual exploration due to their empha- ments by parameters and rules that determine geo- sis on literal representation using geometrically spe- metric and non-geometric properties from the level cific data ( Lonsway, 2002; Meniru, Rivard, & Bédard, of a single design element to the overall configura- 2003). They were not considered exploratory devices tion (Eastman et al., 2008). In contemporary practice, because they have ignored the “progression from BIM has the potential to be a single, intelligent dig- abstract and low-commitment diagrams to detailed, ital model that facilitates multiple facets of the de- specific and higher-commitment drawings” (Gross, sign process, such as performance assessment, auto- 1996). Do and Gross (2001). identified several com- mated parts and assembly production, checking for ponents that are needed for a computer program to spatial conflict, construction sequencing, material re- support thinking with diagrams. First, the program search and testing, design documentation, cost esti- should allow the designer to define design elements mating, and visualization (Garber, 2009). BIM can re- and spatial relationships between elements as con- allocate the time and effort to place more emphasis straints. Second, the program to support diagram- on conceptual design. While it has been argued that matic reasoning for design ought to perform trans- BIM can potentially support conceptual design, the formations that transform one diagram to another. use of BIM today is mainly limited to the technical as- Lastly, it should facilitate the representation of de- pects of building, such as developing construction- signs at various levels of abstractions. In other words, level information and conducting building’s perfor- a diagram is an abstract representation of a more de- mance assessments (Ambrose, 2009; Clayton, 2014; tailed physical form; therefore, each element in the Eastman et al., 2008; Marcos, 2017). For instance, abstract diagram may be represented at a more de- in a survey about the role of digital tools to sup- tailed level. port conceptual design to 241 senior architects who In recent years, various options have been devel- use computers, using Autodesk Revit (BIM tool) in

Building Information Modelling - Volume 2 - eCAADe 39 | 41 early design stages came with 0% preference (Parthe- evaluate the energy impact of various design alter- nios, 2005). According to Holzer (2011), the tech- natives. nocentric view of BIM leads many designers to be- lieve that implementing BIM is about implement- TEST CASE: LE CORBUSIER ing new software. Furthermore, Michalatos (2016) Researchers often start with a precedent in studying claimed that BIM is a sophisticated professionally- the underlying logic of an architectural work (Flem- oriented tool that is less suitable for the early design ming, 1990; Knight, 1981, 1999). According to Knight stages. On the other hand, Ambrose (2009)argued (1995), the generative description of a design logic or that BIM needs to be understood as a way of think- language enables designers to generate new design ing. In that sense, the designer can overcome the so- options with the same logic or new ones. Accord- phistication of BIM tools through strategies like inte- ingly, the proposed method of using BIM to support grating design rules, constraint modeling, diagram- diagrammatic reasoning is not limited to one way of matic thought (Al-Assaf & Clayton, 2017; Clayton, thinking. Each design logic can be exemplified in a 2012, 2014), and using divide and conquer strategy BIM template that defines the syntax and the vocab- in which design solutions can be decomposed into ulary of that logic. This research focuses on using a parts or individual partial solutions (Akin, 2015). In BIM template to generate various design options that that sense, this research aims to employ those strate- share the same underlying formal logic. In that sense, gies to illustrate that the existing data structure of our choice of the test cases is directed by the need BIM can support design thinking and allow design- to select an architectural work in which the architect ers to define a set of syntactical rules and conceptual elaborated a clear, consistent formal logic that can vocabulary explicitly. be traced throughout a family of other similar work. Le Corbusier’s work has constantly employed a clear Diagrams and Performance analysis set of organizational principles and vocabulary that The overall performance of any building is pro- can be traced throughout a wide range of buildings foundly impacted by the design decisions made dur- and defines a formal logic that has been investigated ing the conceptual design stage. Nevertheless, de- by many researchers (Baker, 2017; Eisenman, 2006; signers face difficulties exploring design alternatives Rowe, 1982). Le Corbusier developed his architec- and their effect on the building’s energy profile in ture based on the syntactic structure of the abstract an adequate manner during this stage (Gerber & geometrical form. Le Corbusier’s formal logic uses Lin, 2014; Jabi, 2014). Adopting a performance- abstracted vocabularies and a syntax that highlights oriented design approach at the early stages of de- modularity, proportion, axiality, and spatial layering sign requires several connected components such (Baker, 2017). as a building model platform, an energy simulation engine, and an optional optimization algorithm (Li, Villa Stein Liu, & Peng, 2020). The lack of appropriate model Le Corbusier’s villa Stein-de Monzie in Garches (1926- representation is one of the main difficulties that 1929) is considered a crystallization of Modern Archi- face designers. For instance, BIM models tend to in- tecture. It marks the consolidation of organizational clude many details needed to construct and manage principles and conceptual elements that Le Corbus- a building. However, building energy simulation de- ier had developed in the preceding ten years (Baker, mands an abstracted and simplified version of the 2017). model (Jabi, 2014). This research explores the rela- The form configuration of villa Stein is a ’mass tionship between abstract diagrams created in BIM system’ in which a generic solid mass can be ani- and the possibility of conducting energy analysis to mated through subtraction and addition operations.

42 | eCAADe 39 - Building Information Modelling - Volume 2 The overall geometry of the house is determined by a by a modular system (x=3 m). A square grid de- modular system (x=1.25 m). The golden section ratio termines the width and length of the main generic determines the width and length of the main rectan- mass. Although a double square controls the main gular generic mass. The mass is stratified in the lon- living space in the house, the vertical circulation layer gitudinal direction according to A-B-A-B-A rhythm, is treated as an addition to the configuration. The which Palladio’s Villa Malcontenta influences. In the house mass is stratified in the longitudinal direction transverse direction, the mass has also been strati- into four identical spatial layers with X width. In the fied into a series of volumetric planes according to C- transverse direction, the mass has also been strati- D-D-C rhythm (figure 1). The columns are arranged fied into a series of volumetric planes according to within the thin layer (C). In this spatially layered ar- 0.5X-X-X rhythm (figure 2). The longitudinal axis is chitectural expression, a transverse axis penetrates all aligned to the thin layer (0.5X), and it defines the en- the layers, defines the entry, and locates the rectan- try, the vertical circulation, and the balcony at the gular voids. The generic mass is animated through second level. Two rectangular subtractions articulate a series of subtraction and addition operations. The the main rectangular mass at the ground level and at two rectangular voids are defined by the overall spa- the roof terrace level. Two curved volumes are added tial layering in the longitudinal and transverse direc- to the ground level, and a curved chimney is added tions. This erosion of the corner distorts the rectan- to the upper levels. The interior provides a spatial gular configuration and creates a dynamic balance of contrast between open multi-volumetric living space asymmetrical distribution of elements within a sym- and the cell-like bedrooms. Finally, a series of rectan- metrical order. A staircase platform, as well as two gular and ribbon windows are added to generic mass other platforms, are added to the rectangular void. while the large windows are only placed on the front Moreover, two curved rooms are added to the roof façade. terrace level and underneath the external staircase. Figure 1 Two balconies and a canopy that define the entrance The spatial layering are juxtaposed to the front façade. Those additions system in Villa Stein are aligned to the spatial layering system in the lon- gitudinal direction. Lastly, the mass is further ani- mated by rectangular and ribbon windows. The for- mulated proportional system is extended to regu- late the facades and windows’ placement and size through what Le Corbusier called ”regulatory” lines.

Citrohan House Le Corbusier created the Citrohan House in Stuttgart Figure 2 (1920- 1927) to provide a space that is efficient and The spatial layering affordable. Therefore, in comparison to Villa Stein, Le system in Citrohan Corbusier elaborated the same formal ideas set in a House compact version. Le Corbusier developed five differ- ent design options for Citrohan House. Le Corbusier conceived Citrohan House as an ab- stract rectangular prism placed pilotis or piers to ac- centuate its independence from the surroundings. The overall geometry of the house is determined

Building Information Modelling - Volume 2 - eCAADe 39 | 43 METHODOLOGY are used to actualize the conceptual diagram and fos- This research uses a worked example approach to in- ter the production of an architectural expression. vestigate whether the data structure of BIM can be used to facilitate diagrammatic reasoning and sup- BIM(ing) Villa Stein port the conceptual design stage. An initial step To model the parametric diagram of Villa Stein, we was to select a test case. The test case was ana- divided the complex diagram into a main syntacti- lyzed to derive a set of syntactical rules and con- cal unit created in CDE and secondary units created ceptual vocabularies. Then, the syntactical rules and in FE. The syntactic unit consists of several elements conceptual vocabularies were expressed in Autodesk and syntactical relationships between them. In Villa Revit using a constraint-based modeling approach. Stein, the syntactic unit represents the house’s main This approach is achieved through devising a con- generic mass and its spatial layering system. The ceptual mass model in which reference planes and secondary units represent the vocabulary elements mass objects are related to each other through para- used to articulate the main mass through addition metric constraints. A vocabulary of conceptual el- and subtraction processes. ements was created using the family editor in Au- To model the house’s main syntactic unit, the todesk Revit to conform with Le Corbusier’s vocabu- reference planes were created to define the dimen- lary. The syntactic framework was then developed to sions of the rectangular prism and the spatial layer- place the conceptual design elements of Villa Stein ing system. Dimension parameters that regulate the and demonstrate that the framework could guide the length, width, height, and the layering system were production of an architectural formal expression. Cit- assigned. Model Line command was used to create rohan House was modeled using Villa Stein’s frame- a rectangular prism. The boundaries of the prism work to explore the generality of the syntactic frame- were aligned and locked to five reference planes. work. As a final exploration, various designed op- Then, four vertical rectangular planes were made in tions were generated from the conceptual diagram each direction to exemplify the spatial layering sys- of Citrohan House and simulated Autodesk Insight. tem. Three additional horizontal planes were also This exploration aims to produce an effective build- created (figure 3-1). In this framework, each plane ing envelope design by changing the design param- was aligned and locked to a reference plane. Ac- eters and evaluating those changes’ energy impact. cordingly, relationships and dependencies between planes were established. The secondary conceptual DISCUSSION: DIAGRAMMING LE CORBUS- elements were created in FE as simple profile extru- IER IN BIM sion. Each element’s geometry was defined through a framework of reference planes and dimension pa- This section demonstrates the use of BIM to model a rameters for the width, length, and height. Moreover, parametric diagram of Villa Stein and Citrohan House a type-parameter was created to specify whether the by Le Corbusier. The method of representing each unit is solid or void. At this stage, the way a vocab- house’s formal logic employs Revit’s Conceptual De- ulary element is constructed determines its transfor- sign Environment (CDE) to create a syntactic unit with mational behavior. a spatial layering system, reference lines, and planes, In the CDE file of the conceptual diagram, the and parametric constraints that can be saved as a main syntactic unit and the secondary units for for- template. Revit Family Editor (FE) is used to create mal additions and subtractions were loaded. We es- a vocabulary of parametric conceptual design ele- tablished the regulating framework of the concep- ments of Le Corbusier’s architecture that can be used tual diagram using reference planes and reference as additions or subtractions in CDE. In Revit Project lines, parameters (e.g., dimension parameters, for- Environment (PE), the generic architectonic elements

44 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 3 mulas, and conditional statements), and constraints Modeling the (e.g., alignment, equality constraint, and locked di- abstract parametric mension). This regulating framework allowed us diagram of Villa to define axiality, spatial layering, proportions, and Stein in CDE. modularity between the design elements in the con- ceptual diagram. For instance, we established a pro- portional system by defining a variable for a modular unit (X) and connecting it to several formulas to reg- ulate each spatial layer’s width. Additionally, dimen- Figure 4 sion parameters were created to locate the additions Modeling Villa Stein and subtractions. In this system, the semantics of de- using FE, CDE, and sign elements was also expressed. Text and material PE in Autodesk parameters were used to add names and distinguish Revit design elements. For instance, a material parameter is used to color-code elements and differentiate be- tween main mass, solids, and voids (figure 3). Lastly, for converting the conceptual diagram into a constructible building, the conceptual diagram Table 1 of Villa Stein was loaded to the Revit PE. The built-in List of parameters elements of constructions such as walls, floors, roofs, to derive the and windows were used to create a detailed archi- Conceptual tectural model by attaching and locking them to the Diagram of Villa conceptual diagram. The geometry of the concep- Stein tual diagram offered a structure of regulating lines that can be used to control the pattern of openings on the facades. Consequently, the relationship be- BIM(ing) Citrohan House tween the conceptual diagram and the detailed spe- Figure 5 cific form was preserved (figure 4). Modeling Citrohan Lastly, this conceptual diagram’s overall configu- House using FE, ration consists of a syntactic unit and solid and void CDE, and PE in objects that were related together through param- Autodesk Revit eter values. The value of these parameters can be changed to create permutations of the diagram. In other words, this BIM model is not a single instance of Villa Stein but a formal parametric system that al- lows for multiple expressive configurations. Table 1 illustrates the list of parameters that elucidate how Many of Le Corbusier’s houses express the same for- this model can be flexed. mal concepts: one main rectangular prism, a spatial layering system, a modular system, axes, and a series of subtractions and additions to animate the main mass. This logic can be found in the Citrohan House that Le Corbusier designed during the same period of Villa Stein. In this research, we used the same di-

Building Information Modelling - Volume 2 - eCAADe 39 | 45 agram of Villa Stein to create Citrohan House’s dia- can be improved by reducing the WWR for the south- gram. We changed the width, length, and number ern wall to 30%, as in scenario 4. Additionally, it can of floor parameters of the main mass, and we con- be improved by reversing the existing WWR of the nected it to the first level instead of the ground level. southern wall and the northern wall as in scenario 5 The parameters of the spatial layering system were (figure 6). adjusted to change their number and width. Addi- tionally, the location and number of the curvilinear Figure 6 and rectangular additions were changed. Then, one The effect of of the void elements was hidden, and the other one’s changing WWR in parameters were adjusted. Lastly, the conceptual di- Citrohan House agram was reloaded to the project file of Villa Stein to replace the old diagram and override its parameters. The walls, floors, and roofs are updated to the new diagram. A few more alterations were done to adjust the doors, windows, and interior walls (figure 5).

Performance study of Citrohan House While designing Citrohan House, Le Corbusier ex- B: Width-to-Length Ratio (WLR). The analysis plored five different design alternatives. Those al- shows that the house’s existing width-to-length ratio ternatives vary in many aspects, such as the width, is considered adequate compared to other design length, and height values of the main mass. Addi- scenarios. Nevertheless, it can be further improved tionally, Le Corbusier explored several scenarios for by changing the width parameter to 3X instead of the void at the roof level and the window-wall ra- 2.5X as in scenario 1 (figure 7). tio on the exterior facades. In our research, the per- formance study of Citrohan House relies on those Figure 7 changes to flex the parameter of the conceptual dia- The effect of gram and explore their impact on the energy profile changing WLR in of the house. Citrohan House Autodesk Insight inside Autodesk Revit is used to analyze the energy efficiency of Citrohan House. First, mass floors were assigned to the diagram of Cit- rohan house to split it into analytical zones and sur- faces for the energy analysis. The location and the type of the building (i.e., single-family house) were assigned too. Then, the house’s diagram was trans- formed into an accurate and reliable energy model. C: Void Study. We explored various possible scenar- This energy model was sent to the could for simula- ios to reconfigure the void at the roof level. The void tion and analysis. length, width, and height parameters were changed. A: Window-to-Wall Ratio (WWR). The analysis Moreover, the location of the void was also altered shows that the existing condition of the house con- in each design scenario. The analysis showed that sumes around 308 KWh/m2/yr. This value is less than the existing void parameters provide adequate per- the other three scenarios (1,2 and 3) in which they formance compared to scenarios 1,4,5 and 6. Never- have higher WWR. However, the energy performance theless, this performance can be improved by chang-

46 | eCAADe 39 - Building Information Modelling - Volume 2 ing the width parameter of the void into 2.5X and the lationships and dependencies between vocabularies length parameter into 3X or 4X as in scenarios 2 and 3. through a framework of reference planes and refer- Alternatively, we can also change the width parame- ence lines, parameters, and constraints. The result is ter of the void to 2.5X and align it to the north side of a conceptual parametric diagram that can be used the house as in scenario 7 (figure 8). to explore design alternatives at a conceptual level. In constructing this diagram, the divide and conquer Figure 8 strategy is employed to overcome the sophistication The effect of of BIM tools and represent complex formal systems. changing the void For instance, complex formal systems should be di- parameters in vided into subsystems of syntactical units and para- Citrohan House metric vocabularies. From this perspective, BIM as a design thinking tool can be reached by using abstract parametric diagrams that compute the formal knowl- edge of architecture. Our research illustrated how the BIM system Figure 9 contributes to diagrammatic reasoning, which con- Establish a system tributes to the more comprehensive theoretical de- between three bate on diagrams in architecture by introducing a modeling digital perspective. Therefore, this research con- environments: tributes to the discussion on how BIM can be en- Family Editor (FE), gaged in teaching formal knowledge in architec- Conceptual Design ture. In architectural education, constraint-based di- Environment (CDE), agrams for design development can direct students and Project to highlight the logic behind the form and under- Environment (PE) stand how their decisions may affect the building performance. In recent research by the author, Al- CONCLUSION Assaf (2019) explored the value of this approach to This discussion extends the role of BIM as a modeling teach second-year students formal concepts using tool to aid the production and management of build- Richard Meier’s formal language. Future research can ings into an architectural information modeling (AIM) be conducted to explore more formal styles other platform that can represent abstract formal expres- than Le Corbusier and Richard Meier. In professional sions of architecture. Using Le Corbusier’s architec- practice, the approach presented in this research al- tural expression for single-family detached houses as lows designers to externalize logic and communicate a test case, a BIM template has been devised to sup- formal ideas between the design team members. port diagrammatic reasoning and aid conceptual de- sign. Autodesk Revit proved capable of represent- REFERENCES ing abstract diagrams that can be derived using com- Akin, Ö 2015, ’Necessity of cognitive modeling in BIM’,in putational methods. Autodesk Revit allows the de- Kensek, K. and Noble, D. (eds) 2015, Building infor- signer to establish a system between three modeling mation modeling: BIM in current future practice, John Wiley & Sons, pp. 17-27 environments: Family Editor (FE), Conceptual Design Al-Assaf, N 2019, Architectural Information Modeling Environment (CDE), and Project Environment (PE) to (AIM): Teaching Formal Concepts Of Design Using derive the whole design process as shown in fig- Building Information Modeling (BIM), Ph.D. Thesis, ure(9). Thereby, designers can explicitly define re- Texas A&M University

Building Information Modelling - Volume 2 - eCAADe 39 | 47 Al-Assaf, N and Clayton, MJ 2017 ’Representing the Aes- putational environment for working with design di- thetics of Richard Meier’s Houses Using Building In- agrams’, Design Studies, 17(1), pp. 53-69 formation Modeling’, ACADIA 2017: DISCIPLINES + Gómez, P 2010 ’Generative Diagrams: Embedding Spa- DISRUPTION, Massachusetts Institute of Technology tial Data to Generate Informed Architectural De- (MIT), Cambridge, MA signs’, SIGRADI 2010 / Disrupción, modelación y con- Alexander, C 1964., Notes on the Synthesis of Form, Har- strucción: Diálogos cambiantes vard University Press, Cambridge, MA Holzer, D 2011, ’BIM’s seven deadly sins. International Ambrose, M 2009 ’Agent Provocateur–BIM and the De- journal of architectural computing’, International sign Studio: Questioning Roles of Abstraction and journal of architectural computing, 9(4), pp. 463-480 Simulation in Design Education’, The Value of Design, Horváth, I 2000 ’Conceptual design: inside and out- ACSA, New York side’, 2nd International Seminar and Workshop on En- Baker, G. 1996, Design strategies in architecture : an ap- gineering Design in Integrated Product proach to the analysis of form, Van Nostrand Rein- Jabi, W 2014 ’Parametric spatial models for energy anal- hold, New York ysis in the early design stages’, SimAUD 2014 Sym- Baker, G. 2017, Le Corbusier-an analysis of form, Taylor & posium on Simulation for Architecture and Urban De- Francis sign,, Tampa, Florida, USA Clayton, MJ 2012 ’Using BIM to aid diagrammatic reason- Knight, T 1981, ’Languages of designs: from known to ing about architectural aesthetic concepts’, Design new’, Environment and planning B: Planning and de- Computing and Cognition DCC’12 sign, 8(2), pp. 213-238 Clayton, MJ 2014, ’Chapter 3: Modeling Architectural knight, T 1995, Transformations in design: a formal ap- Meaning’, in Kensek, K and Noble, D (eds) 2014, proach to stylistic change and innovation in the visual Building Information Modeling: BIM in Current and Fu- arts, Cambridge University Press ture Practice, John Wiley & Sons, pp. 29-40 Knight, T 1999 ’Applications in architectural design, and Do, E and Gross, MD 2001, ’Thinking with diagrams in ar- education and practice’, NSF/MIT Workshop on Shape chitectural design’, Artificial Intelligence Review, 15(1- Computation 2), pp. 135-149 Langer, S 1967, An Introduction to Symbolic Logic, New Dogan, F and Nersessian, NJ 2002 ’Conceptual diagrams: York: Dover representing ideas in design’, the International Con- Li, S, Liu, L and Peng, C 2020, ’A review of performance- ference on Theory and Application of Diagrams oriented architectural design and optimization in Dulić, O and Aladžić, V 2016 ’A Note On Graphical Repre- the context of sustainability: Dividends and chal- sentations In Architecture–Diagrams Over Sketches’, lenges.’, Sustainability, 12(4), p. 1427 Contemporary Achievements in Civil Engineering 2016 Lonsway, B. 2002, ’The mistaken dimensionality of CAD’, Eastman, C, Teicholz, P and Sacks, R 2008, BIM handbook: Journal of Architectural Education, 56(2), pp. 23-25 A guide to building information modeling for owners, Marcos, C 2017, ’IM Implications in the Design Process managers, designers, engineers and contractors, John and Project-Based Learning: Comprehensive Inte- Wiley & Sons gration of BIM in Architecture.’, WIT Transactions on Eisenman, P 1999, Peter Eisenman: diagram diaries, The Built Environment, 169, pp. 113-125 Thames & Hudson Michalatos, P 2016, ’Design Signals: The Role of Software Eisenman, P 2006, The formal basis of modern architec- Architecture and Paradigms in Design Thinking and ture, Switzerland : L. Müller Practice’, Architectural Design, 86(5), pp. 108-115 Eisenman, P 2015, Palladio Virtuel, Yale University Press Norberg-Schulz, C 1965, Intentions in architecture, MIT Garber, R 2009, Closing the Gap: Information Models in press Contemporary Design Practice, Architectural Design: Parthenios, P 2005, Conceptual design tools for architects, Academy Press. Ph.D. Thesis, Harvard University Gerber, DJ and Lin, SHE 2014, ’Designing in complexity: Row, C 1982, The mathematics of the ideal villa and other Simulation, integration, and multidisciplinary de- essays, MIT press sign optimization for architecture’, Simulation, 90(8), Stiny, G. and Gips, J 1978, Algorithmic aesthetics: com- pp. 936-959 puter models for criticism and design in the arts, Univ Gracia, MJ 2010, The Diagrams of Architecture: AD Reader, of California Press Wiley Gross, MD 1996, ’The electronic cocktail napkin—a com-

48 | eCAADe 39 - Building Information Modelling - Volume 2 Operating Room Design with BIM, VR, AR, and Interactive Simulation

Uwe Woessner1, Joachim Kieferle2, Marko Djuric3 1,3HLRS 2Hochschule RheinMain 1,3{woessner|djuric}@hlrs.de [email protected]

Operating room design is a complex planning task. In order to jointly find the best solution, specialists from numerous professions are involved in the planning process, since e.g. equipment layout and airflow is crucial for optimal surgical procedures. For a better and more informed discussion, and thus better optimization and firm decisions, we have developed a method linking BIM (Building Information Modelling), VR (Virtual Reality), AR (Augmented Reality), CFD (Computational Fluid Dynamics) simulation, and a tangible user interface, so that freely configurable layouts can be tested interactively, be discussed, and optimized both in model and 1:1 scale already in early planning phases. This method has been applied to a hospital design with 33 new operating rooms, differing in sizes as well as layouts according to the different types of operations.

Keywords: Operating Room Design, Interactive Simulation, Virtual Reality, Augmented Reality, Tangible User Interface, BIM

1. INTRODUCTION duration, and a high comfort for the surgery staff An optimized equipment layout in an operating guarantees highly concentrated work even in hours room is crucial for reliable surgical procedures, and long surgeries. The interdependency between these an optimized airflow can significantly prevent infec- requirements shows that there is no one perfect so- tion risks during surgery (Sadrizadeh et al. 2016). As lution, it rather requires a joint planning effort, where the equipment layout directly influences the airflow, the specialists from each profession bring in their ex- any optimization step of the equipment layout po- pertise and jointly, interactively and iteratively work tentially requires an adaptation of the ventilations in- on finding the best solution. and outlet positions and parameters to prevent a po- In the planning phase, the airflow simulation tential contamination of the patients’ surgical area. CFD (Computational Fluid Dynamics) usually requires While high airflow volumes reduce CFU (colony form- special expertise and is outsourced to specialists. ing units) loads, they can lead to hypothermia and Thus it is common practice that any iteration step thus negatively affect the outcome of the surgery. - simulating the planned layout and bringing back Besides the patients well-being, the working proce- the simulation results into the planning for improve- dures and comfort of the surgery staff (doctors and ments - takes hours or days, disrupting the joint dis- nurses) have to be paid special attention to. Opti- cussion. There are valuable approaches to offer the mized working procedures reduce the critical surgery simulation as online services, like IngridCloud [1],

Building Information Modelling - Volume 2 - eCAADe 39 | 49 Figure 1 Distributed Application Setup

Simscale [2], BIM HVACTool [3] some of which linked with our newly developed application the number of into Revit with a plug-in for a smoother data transfer. physical mockups was reduced to one only, and a lot Butterfly [4] in Ladybug Tools is even linked closer, vi- of the optimization was realized in the virtual proto- sualizing the results in the Dynamo environment of types. Even though only a limited number of joint Revit. However all these tools lack the possibility to planning sessions could be carried out due to the cur- directly visualize the results within the full Revit con- rent pandemic situation, all participants underlined text or even in Virtual Reality. the various benefits of this method. So in this paper we show our development, that Starting with the setup, the paper first introduces provides an interactive operating room design en- the structure of the system and describes the inten- vironment by combining BIM (Building Information tions behind the development. The BIM and simu- Modeling), VR (Virtual Reality), AR (Augmented Re- lation models used are subsequently shown and ex- ality), and interactive CFD simulation as a commu- plained, followed by the interaction possibilities de- nication and optimization platform. The intention veloped for the VR- and AR-prototypes. Further de- is to build a base for an interdisciplinary discussion velopment steps are discussed in the last chapter. between planners, further specialists, operators, and medical staff, to help reducing the complexity of the 2. MOTIVATION, SETUP AND COMPO- planning process and to bring non-visible informa- NENTS tion into the discussion for an optimized solution. For this application we have developed a live link This development has already been applied in a real between a BIM-system (Revit), a tangible user inter- life project, the planning of the new central operat- face (1:20 scale architecture model), the AR and VR- ing room facility at Aachen University Hospital UKA software (COVISE) and the CFD code (OpenFOAM). (Germany) which is extending one of Europe’s largest The setup covers all aspects of a typical planning ses- hospital buildings by 33 additional operating rooms. sion, including everything from the design of the in- For the optimization of these operating rooms, terior to calculation of HVAC parameters like indoor the planners initially intended to build four physical air volume and temperature for a ventilation system mockups to discuss the various aspects. Using phys- in an operating room. All project participants (plan- ical mockups for the operating room design is a well ners, engineers, doctors, nurses, manufacturers, and proven method (Bayramzadeh et al. 2018). However others) meet in a 1:1 virtual mockup in VR (see fig-

50 | eCAADe 39 - Building Information Modelling - Volume 2 ure 1), a virtual twin of the operating room, to eval- 2.1 BIM model with customized families uate its usability. The goal of the project is to pro- The BIM model in Revit contains all the relevant infor- vide an intuitive, easy to use and direct interaction mation and is the basis for the whole application. The with the virtual model, so that fluent meeting dis- operating room is modeled as a standard project with cussions are supported. This means that the plan- all relevant building components like floors, walls, ning can be done in consultation with the actual user doors and windows. This standard BIM model also group of an operating room, the doctors and nurses, contains all relevant HVAC components like air ven- who’s requirements and expertise in the field - due tilation in- and outlets, thus providing full functional to the complexity of a planning process - might oth- access to the HVAC calculations. Furthermore the rel- erwise only be considered to a very limited degree. evant operating room equipment like operating ta- Their experience in the everyday work in a medical bles, anaesthesia machines, operating lights and me- environment, which is needed for example for the dia access points that might influence the airflow are prevention of hypothermia or the well-being of the modelled as Revit family instances. patients, might otherwise be missed. This Pre Occu- For a bi-directional link between Revit and the pancy Evaluation (POE) helps to improve the plan- AR/VR devices (Kieferle and Woessner 2015) as well as ning quality and minimises the need for potentially to define the simulation parameters, customized Re- expensive corrections after starting the operation. vit families are added to the BIM model. These fami- lies are: Figure 2 Specialists • AR-Marker family with unique marker ID. The analysing and marker ID can be either the numeric ID encoded optimizing in the 2D pattern for ArUco and ALVAR patterns operating room or an alphanumeric ID identifying the Marker in layout in a VR CAVE the tracking library. Markers are hosted on re- session spective elements in the BIM model, see figure 3, which ties them together.

Figure 3 Revit AR marker The application can be distributed among several family workstations and HPC (High Performance Comput- ing) systems. The overall setup is displayed in figure 2. It consists of a collaboration server “VRB” which handles session management and distributes mes- sages between the components. The COVISE [5] vi- sualization system is used to post-process and visu- alize the OpenFOAM simulation results. OpenCOVER is a module within the COVISE system which renders the visualizations in immersive virtual environments such as CAVEs, in HMDs (Head Mounted Displays) or in AR. All of these components and how they work to- gether are described in detail in the following chap- ters.

Building Information Modelling - Volume 2 - eCAADe 39 | 51 • Refinement family parameter to control mesh • OpenFOAM-Marker family with general Open- resolution for relevant operating equipment, FOAM settings not linked to any geometry, con- see figure 4. taining information such as simulation host, number of simulation steps, solver type, num- berOfSubdomains, and others, see figure 6. It Figure 4 also defines the “locationInMesh”, a position in- Revit refinement side the flow volume which specifies which void parameter of the geometry should be meshed.

Figure 5 Revit OpenFOAM family

And for various simulation and meshing parameters: • OpenFoamDomain family, graphically ad- justable, defining the simulation bounding box (length, width, height, position and orienta- tion), see figure 5. It defines the extents of the simulated area. In the example of the hospital Figure 6 building, this allows to select one of the op- Revit OpenFOAM erating rooms to use for the simulation. Oth- domain family erwise all rooms would be simulated together which would significantly increase the process- ing time. Another way to reduce the complex- ity of the simulation is to create a specific view called “Simulation”. If such a view is present, only the elements visible in that view will be taken into account for the simulation. This also allows to use a different view for the visualiza- tion than for the simulation. In the simulation view for example the ceiling could be removed for a better view from above while it has to be present for the simulation. • OpenFoamRefinementRegion family defining a mesh refinement level for a cubic region.

52 | eCAADe 39 - Building Information Modelling - Volume 2 Additionally, the 2.2 Simulation OpenFOAM OpenFOAM [6] has been selected for this project •Standard swirl outlet family is extended with mainly because of its reliable and easy to use mesh an additional parameter RPM (rotations per generator ”snappyHexMesh”. It can be parallelized minute) to represent the swirl component of the which is important to speed up the mesh genera- inlet velocities, see figure 7. By default, all of the tion process for complex geometries. OpenFOAM is air terminal geometry is treated as inlet or out- well established in the science community for the va- let surface. If only part of the geometry is defin- riety of solvers which are implemented and it pro- ing the inlet/outlet patch, those geometry parts vides all necessary tools for running CFD simulations have to be marked with a geometry subtype “in- such as domain decomposition and mesh generation let” or “outlet”. tools. Apart from that OpenFOAM offers a well docu- mented syntax for defining the boundary conditions Figure 7 through configuration files. Each simulation in Open- Revit swirl outlet FOAM is specified as a case folder containing config- extended family uration subfolders called ”0”,”system” and ”constant”. The time folder ”0” contains the initial parameters as config files for timestep zero. System and simulation relevant settings like the number of cores to use, the solver or mesh generation algorithms need to be de- fined in the ”system” folder. The ”constant” folder contains time independent variables like the gravita- tional acceleration ”g” or parameters for the turbu- lence model to use. In this project the ”RNG k-ϵ” tur- bulence model was used. Depending on the solver and turbulence model that is used for the simulation, the parameters in each folder vary [7]. A CFD simulation workflow is typically divided into three parts: Pre-Processing, Solving and Post- Processing. Pre-Processing comprises geometry gen- Figure 8 eration, definition of the initial state and boundary OpenFOAM conditions, domain decomposition and mesh gener- simulation ation [8]. In this project, the first two steps are done workflow in Revit, while the following steps are carried out in parallel on a remote cluster. The geometry of the hospital is built directly in Revit with the necessary air conditioning system. Most standard Revit families contain all required parameters such as flow rate or pressure drop. If necessary the families have been extended by additional parameters such as “RPM”. If such a parameter is present at an air inlet family, a swirl inlet boundary condition is defined instead of the standard “inletOutlet”. A new Revit add-in named OpenFOAMInterface extracts all geometries

Building Information Modelling - Volume 2 - eCAADe 39 | 53 and generates several stl files which are transformed Figure 9 into a computational mesh and boundary patches. Multi resolution A special family instance called “OpenFOAM-Marker” mesh generated as shown in figure 6 defines a point inside the flow from volume to select the appropriate room to simulate snappyHexMesh in if multiple rooms are available. This marker object OpenFOAM also stores general system parameters for the simula- tion. The boundaries for the simulation domain can be limited by a bounding box family instance called “OpenFOAMDomain” as illustrated in figure 5. If no such bounding box is available, the overall bounding box of all revit geometries is used which might be too large and lead to increased processing time. Mesh resolution is a trade-off between accuracy 2.3 Tangible User Interface and processing time. Therefore the mesh resolu- ATangible User Interface is used for an intuitive and tion should only be increased in areas with high geo- direct interaction with the 3D model (Ullmer and Ishii metric complexity or high gradients in the flow field. 2000). The interface consists of a typical architec- Such refinement can be defined for each object in- ture model, marker-tagged model elements, and a stance individually by adding an additional integer Logitech Brio 4K Webcam to capture the view from parameter “Mesh Resolution”, see figure 4. The pa- above. The webcam is mounted on a stand that is di- rameter specifies a subdivision level, thus a value of rectly attached to the model, see figure 10. four will subdivide the original mesh size four times resulting in a four times higher resolution. Figure 10 Air terminals are treated as inlet or outlet sur- Tangible User faces depending on HVAC system type. The add- Interface in parses these Revit families and defines boundary conditions and initial parameters for the CFD simula- tion according to the family’s parameters. All geom- etry objects are exported as STL files and the config- uration files as “dict” files. If a remote cluster is spec- ified, all data is transferred to that cluster via secure copy and the following processing steps are submit- ted to an interactive batch processing queue. The processing steps on the cluster are: domain decomposition, mesh generation (snappyHexMesh For this use case a 1:20 scale model fits best, as result shown in figure 9) and the actual solver run, ei- the model size is still conveniently handleable and all ther simpleFoam for airflow only or buoyantBoussi- relevant components to interact with fit to the user’s nesqSimpleFoam if heat transfer and convection is hand size. Floors and walls of the model are built also taken into account. After a successful solver run, with architecture model material except for the front the results are automatically transferred back from which is left open on purpose, to move the operat- the cluster to the local workstation via secure copy. ing room equipment around. The ceiling is built out The Post-Processing is done in COVISE which reads of acrylic glass on one hand to allow see through to the result of the simulation. the operating room and on the other hand as a base

54 | eCAADe 39 - Building Information Modelling - Volume 2 plate to place the model air terminals on the ceiling allow viewing the tangible interface from many dif- level. ferent directions. If multiple ObjectMarkers are vis- Pattern markers on selected model elements can ible at the same time, the accuracy of the tracking be tracked to represent the position of these ele- is increased. Revit markers allow to move the host ments. A large marker is used to register the model, family with the marker. Any change in the physi- several smaller ones representing air terminals that cal marker’s position leads to a change of position can be freely moved on the models acrylic glass ceil- of the appropriate Revit host element and the Re- ing, and one to capture the position of a sliding door vit model is updated accordingly. This change in separating the preparation room from the operating the Revit model in turn also updates the VR model. room. The sliding door has to be tracked, as the dif- A recomputation of the CFD simulation can be trig- ferent opening states of the door are relevant for the gered manually. Several seconds after starting a new airflow (Wang et al. 2021). simulation run, the results of the new configuration OpenCOVER supports different AR Tracking plu- are displayed in all connected VR and AR COVISE in- gins. Currently ARToolKit, ArUco [9] and ALVAR [10] stances and can be analyzed by the team. are implemented. For this application we are using ALVAR. These graphical (fiducial) markers allow 3D 2.4 Augmented Reality tracking of object positions. Augmented Reality is used in two places: The markers are tracked with the camera, and • In the 1:20 model to track the positions of the linked with a bi-directional OpenCover - Revit link moveable objects and to display the simulation to customized AR families, see figure 3 in the Revit results on the captured screen. model which are then linked to the respective Revit • In a 1:1 mockup to visualize the simulation re- family elements like the door or air terminals. The sults in a physical operating room or on operat- relevant parameters are MarkerSize, MarkerType and ing room equipment. MarkerID. MarkerSize specifies the size of the Marker (at Scale 1:1, not model scale). MarkerType can be ei- Tracking objects in the tangible user interface is ther “ObjectMarker” or “Revit”. ObjectMarkers define implemented in OpenCOVER, thus the tangible inter- the model origin and serve as a reference for the AR face can also serve as an AR visualization in model tracking. Any number of ObjectMarkers can be de- size. As shown in figure 11, the simulation results are fined. They all reference the same model origin and displayed in the scaled model on the tracking com-

Figure 11 Screenshot Augmented Reality in 1:20 model

Building Information Modelling - Volume 2 - eCAADe 39 | 55 Figure 12 Augmented Reality airflow in 1:1 mockup scene

puters’ screen and can be used for a direct feedback walls, doors or the virtual operating room equip- loop. ment. Four different physical mockups of operation While any one can interact with every object, rooms were initially planned for this project. How- concurrent interaction in the different linked VR/AR ever three were replaced with the virtual prototypes locations is locked on a “first come, first serve” basis as described here with only one being realized to so that only one participant can interact with one ob- test specific procedures that require haptic feedback ject at a time. such as lifting heavy patients or moving heavy equip- COVISE also supports HMDs (Head Mounted Dis- ment. To enrich the information for the physical 1:1 plays). Those could be used to bring in remote ex- mockup, AR was used to display the CFD simulations perts which otherwise could not participate in the in that 1:1 mockup, see figure 12. discussion. Even remote desktop workstations could be added as a fallback if no VR equipment is available 2.5 Virtual Reality at the remote location (Woessner and Kieferle 2016). In the VR environment used (CAVE), the operating However the advantage of co-located collaboration room and equipment is displayed at scale 1:1, so in VR in this project was rated so high, that even dur- that all participants immerse into the scene. This ing the current Covid 19 pandemic, physical meet- setup immediately fosters group discussions, all par- ings were carried out in the CAVE at HLRS. ticipants can directly bring in their everyday experi- For the analysis of the CFD simulation, various di- ence and domain specific expertise and communi- rect interactions in the VR environment are used, see cate freely much like at a site meeting. figure 13. Cutting surfaces can be placed within the Efficient optimization of the design is only pos- simulated domain to visualize sections at any place sible, when many different room configurations and and orientation, particle traces can also be started tasks can be assessed in a short period of time anywhere to visualize the airflow and iso surfaces can and modifications can easily be tried out. With the be defined not only by specifying a discrete iso-value bi-directional link between Revit and VR, one can but also by selecting a position in space with a 3D switch through different design options or interac- mouse. The iso-surface will then be computed so tively change these layouts in VR e.g. by moving that it passes through the selected point. Similar to

56 | eCAADe 39 - Building Information Modelling - Volume 2 using a 3D mouse in VR, all these interactions can also the number of face to face meetings in the CAVE be carried out using fiducial markers in the AR dis- VR environment was limited. Once the situation play. gets better, we plan to further prove the individ- ual feedback and our observations with structured Figure 13 interviews, to get a better understanding for possi- Airflow analysis in ble improvements and further interaction methods VR environment needed. Another additional step will be the cou- (CAVE) pling with a 1D simulation of the AC systems and sup- port for additional solvers (such as LES). All develop- ment is Open Source under LGPL v3 license and avail- able on GitHub [11], thus contributions and feedback would be appreciated.

REFERENCES Bayramzadeh, S, Joseph, A, Allison, D, Shultz, J and Abernathy, J 2018, ’Using an integrative mock-up simulation approach for evidence-based evaluation 3. CONCLUSION AND NEXT STEPS of operating room design prototypes’, Applied Er- gonomics, 70, pp. 288-299 Despite the high level of complexity of combining Kieferle, J and Woessner, U 01 ’BIM Interactive - About BIM, VR, AR, CFD and a Tangible User Interface, using combining BIM and Virtual Reality - A Bidirectional the system is very easy and transparent for all project Interaction Method for BIM Models in Different En- participants due to the intuitive interaction methods vironments’, Proceedings of the 33rd eCAADe Confer- applied and the discussion in the 1:1 VR environment ence, Vienna University of Technology, Vienna, Aus- tria, pp. 69-75 “as if standing in an operating room”. Thus all partici- Sadrizadeh, S, Afshari, A, Karimipanah, T, Haakansson, pants can directly relate to their everyday experience. U and Nielsen, P 2016, ’Numerical simulation of the Numerous designs have been tested and discussed impact of surgeon posture on airborne particle dis- to optimize the surgery workflow. And consecutive tribution in a turbulent mixing operating theatre’, decisions have been approved in various VR sessions. Building and Environment, 110, pp. 140-147 The meetings showed that all participants (plan- Ullmer, B and Ishii, H 2000, ’Emerging frameworks for tangible user interfaces’, IBM Systems Journal, ners, engineers, doctors, nurses, manufacturers, and 39(3.4), pp. 915-931 others) got a clear understanding of the interrela- Wang, C, Holmberg, S and Sadrizadeh, S 2021, ’Impact of tions and consequences of the planning decisions. door opening on the risk of surgical site infections in By fostering group discussion and better understand- an operating room with mixing ventilation’, Indoor ing for the other disciplines’ constrictions, solutions and Built Environment, 30(2), pp. 166-179 were found in the discussion and various substantial Woessner, U and Kieferle, JB 01 ’BIM Collaboration in Vir- tual Environments - Supporting collaboration in co- improvements have been made, planning certainty located and distributed settings’, Complexity & Sim- has been greatly improved. plicity, pp. 565-572 Compared to the usual successive and inter- [1] https://www.ingridcloud.com/wind-simulations/wi rupted steps: planning - simulation - planning - sim- nd-score ulation ... , the near real time feedback on the airflow [2] https://www.simscale.com of all planning changes helped to understand their [3] https://www.building-engineering.de/software/bim -hvactool impact and to optimize the layout. [4] https://www.ladybug.tools/butterfly.html However due to the current pandemic situation [5] https://www.hlrs.de/covise

Building Information Modelling - Volume 2 - eCAADe 39 | 57 [6] https://openfoam.org [7] https://cfd.direct/openfoam/user-guide [8] https://www.simscale.com/blog/2016/09/cfd-workfl ow-quick-guide [9] http://www.uco.es/investiga/grupos/ava/node/26 [10] http://virtual.vtt.fi/virtual/proj2/multimedia/alvar/i ndex.html [11] https://github.com/hlrs-vis/covise

58 | eCAADe 39 - Building Information Modelling - Volume 2 Structural Grid Predesign using Generative Design for Residential Building with Steel Structure on BIM Models Structural grid predesign using generative design

José Luis Rodríguez Hernández1, Juan Pedro Cortes Perez2, Luka Gradisar3, Bruno Figueiredo4 1European Master on Building Information Modelling BIM A+ 2Universidad de Extremadura, Escuela Politécnica , España 3University of Ljubljana, Faculty of Civil and Geodetic Engineering, Chair of Construction IT , Slovenia 4University of Minho, School of Architecture, Lab2PT, Portugal [email protected] [email protected] 3luka.gradisar@fgg. uni-lj.si [email protected]

Due to the more significant architectural complexity, it is helpful to include structural analysis studies in the early design stages. An architectural team typically carries out the structural grid fit in this phase. This limitation may lead to the structural distribution in the initial phase not being the most appropriate. This work aims to provide a tool for architects oriented design by optimising the cost of the structure, making an initial layout for residential buildings with the regular shape of steel structures using the generative design, which allows the creation of structural BIM models that comply with the requirements of stability and resistance for gravity design specified in the American code ASCE 360 as starting point on the conceptual design. The paper describes the computational design development for the structural building grid using multi-criteria optimisation solved by a genetic algorithm.

Keywords: Generative Design, Building Information Modelling (BIM), Structural Predesign, Structural Grid, Multi-Objective Optimisation

1. INTRODUCTION Architectural designs’ complexity tends to in- Within the conceptual design phase, the architect crease as they involve structural assessment early on; seeks to create a viable proposal that is also aesthet- this phase is typically carried out by the architectural ically pleasing and unique; during this stage, deci- team. After an initial definition of the design project, sions are constrained by general parameters such as structural engineers and other consultants typically building type, location, architectural preferences and begin work; this means that structural considerations other actions that drive the design (Mora et al., 2008). are usually subservient to architectural goals. Archi-

Building Information Modelling - Volume 2 - eCAADe 39 | 59 tects must follow up after the first iteration linear and used solutions. With a GAE such as Dynamo, compu- unidirectional structural adjustment between the de- tational design can be done by creating visual script- sign process (Mueller, 2014). There are few options ing to develop virtual models controlled by a reduced for structural input to inform or improve the initial number of critical parameters, allowing them to test concept in significant ways. numerous but related geometries without much ef- fort (Preisinger and Heimrath, 2014). 1.1. Background BIM software such as Revit allows interaction Suppose the architect’s proposal exceeds the initial with scripting and visual programming. It can use budget. In that case, the best way to reduce the costs a multi-objective optimisation; this works with Dy- might be to change the form of the structure, but namo and Generative Design, both tools embedded in later stages of the project, flexibility and tolerance within it [1]. Additionally, the interaction between toward changes are not beneficial as in architectural them allows for creating a BIM model; thus, informa- design practice, and this provokes re-works for all dis- tion can be stored, maintained, and shared with the ciplines involved (Preisinger and Heimrath, 2014). other disciplines (Sacks et al., 2018). The lack of optimisation on construction material Optimisation problems have been carried out in can be a project problem, as it represents the high- several areas, from aerospace engineering to archi- est percentage of the total building embodied CO2 tectural design (Hahn et al., 2010; Holst et al., 2013; (Kaethner and Burridge, 2012). Furthermore, a 2012 Asl et al., 2014). Multi-objective optimisation can use study[2] shows that almost 40% of the initial cost of a genetic algorithm to allow automated algorithmic a building is related to raw materials. So, a successful processes to generate, evaluate, and improve possi- early proposal at the beginning of the structure can ble design options’ performance. Some projects in- solve problems down the road. volve custom-written search algorithms and a single BIM helps to reduce the silo between stakehold- precise fitness criterion (Mora et al., 2008). ers because it is an interdisciplinary collaborative Generative Design in Dynamo(GD) works inter- methodology that combines tools, processes and nally with the different algorithms: the main one technologies (Arayici et al., 2012; Rawlinson, 2015); it is the Non-dominated Sorting Genetic Algorithm II allows the results of the parametric design to be con- (NSGA II). GD optimisation maximises or minimises nected and hosted in BIM models with transparency an objective function to achieve the best possible so- in the information. With the help of BIM tools, lution for a problem. In GD workflow, it is possible to the ideal solution would be an automated feedback have multiple goals that must optimise the entire de- loop between architectural design and structural re- sign. sponse in real-time; when the architect changes the Currently, some applications make similar pro- design parameters, they can respond to the struc- posals, with some limitations that we seek to cover tural performance. in our research. Hypar, a top 50 construction technol- The use of computational design enables simu- ogy startup, offers software as a service [3], which has lation of complex design permutations performance among its solutions one that creates structural grids and automation of iterative process (Keough and using sliders that allow creating different solutions. Benjamin, 2010). BIM Authoring software enhances However, it is impossible to optimise the structure’s the ability to develop complex processes by us- weight; one workaround is to export the JSON file to ing scripts via the API and graphical algorithm edi- the Grasshopper and create the optimisation sepa- tors (GAE) to generate parametric design-operations. rately using this tool. Asterisk is a web application Grasshopper, Generative Components, Param-O, Vi- for structural optioneering in early conceptual design sual Scripting in Allplan, and Dynamo commonly that works internally with Rhinoceros and Grasshop-

60 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 1 Parametric Schema and implementation.

per[4]. The platform is only available for the Thorn- • Give the architects the possibility to identify the ton Tomasetti employees and subcontractors, mak- costs per structure based on the initial require- ing this option available only to a small number of ar- ments. chitects. • Have multiple solutions to choose from to select the best in their project context and allow them 1.2. Research Aims and objective to make better decisions about the conceptual This work aims to provide a tool for architects to cre- architectural design based on economic feasibil- ate an initial grid structure focused on optimising the ity considering its cost. layout of steel frame residential buildings with a reg- • Create a structural BIM Model automatically af- ular shape to create structural BIM models for gravity ter choosing one option and enable it for work design specified within the ASCE 360 code. This tool in the design phase. can obtain cost estimates of the structure in terms • Reduce the re-work due to structural configura- of material costs, allowing architects to make timely tion incompatibility. decisions on the design’s critical elements, such as building form, span, and typical building height. We want to eliminate the previous work on the 2. METHODOLOGY first iteration of the structure and create an initial so- The building function has multiple variables that af- lution where the structural team can optimise the ini- fect the weight in it; in our research, we applied tial grid conception. The following objectives will be multiple fitness criteria to enhance the competing covered to complement our research: objectives in order to find optimal solutions to re-

Building Information Modelling - Volume 2 - eCAADe 39 | 61 Figure 2 Dynamo Script of the Optimisation

duce the total costs of the structural materials of the assumptions have been considered to limit the vari- building project-allowing simulation results to sig- ables to be optimised: grid spacing (X & Y), facade nificantly impact the final design result rather than offset, slab thickness, finished floor thickness, and following traditional design processes (Keough and floor height (lower, typical, and roof). Additionally, Benjamin, 2010). Because Revit is the most used com- optional variables were created that store the profile mercial tool on the market [5], we decided to focus section choice and shape in a file. the development of our study on the use of Dynamo Figure 1 shows the parametric schema and the due to the directionality they have with each other. implementation with the Dynamo Player interface. Parametric design thinking was used to develop This first step can be used to create the initial shape the schema of the initial parametric design. A para- by modifying the variables. metric schema is a tool that focuses on the visual The generative design approach consists of four and geometrical representation of the object (Ox- steps: building the computational model, generat- man, 2017). At the beginning of the process, it was ing solutions, evaluating the design, and integrating decided to provide regular shapes to help with the it into the BIM model(Mora et al., 2008). By gener- central part of the building process creation. In that ating different design alternatives, this process aims sense, it was assumed the edges of the shape have to understand better how the initial parameters im- a planar intersection with plane Z to define the exte- pact the design and its fitness scores. This knowledge rior columns and create the exterior beams. Further- helps us understand the design problem better and more, it was decided to consider only the purlin sys- decide on further improvements or find its final form tem’s weight instead of modelling it in the pre-design (Gradisar et al., 2021). phase. The main goal is to produce the grid spacing A longer span between columns equals heav- of the structure and the estimated weight. ier steel sections and increases the project’s over- An ideal parametric model would encompass all all cost that the multi-objective function must con- the variations that the designer wishes to explore sider. However, on the other side, a small span affects within the smallest dimensionality possible (Davis, the building’s function and aesthetic but constrains 2013). However, this involves several factors that can- spaces; for that reason, these two variables were cho- not be addressed. The following design limitations sen to put them in competition to find the Pareto-

62 | eCAADe 39 - Building Information Modelling - Volume 2 optimal solutions. All the geometrical operations, form the mathematical operations with textual pro- penalizations and additional features were added in- gramming, which was embedded in the nodes of side the graph shown in Figure 2. To simplify the python script as shown in figure 2. model and performance better, we decided to script the major part of the algorithm inside of the “Python 2.2. End-user process Script” node. This node allows to write in IronPython The proposed methodology for the interaction that Code and to develop the own library for the specific the architect has while using the tool includes the fol- project [6]. lowing phases:

2.1. Structural simplifications 1. The architect proposes the building’s shape, which can be created in any software that ex- The script was developed considering structural sim- ports the shape in “.sat” format. plifications to optimise the structure’s weight versus 2. Configure the fixed parameters of the proposal the grid spans based on the gravity loads of ASCE 7- in the Dynamo script. 16 for structural analysis and preliminary design. For 3. The Revit template must create the generative that case, these simplifications were applied: design where the grid optimisation study can be • Only two different shape profiles were con- selected. sidered for structural beams and one for the 4. Once this is done, the structural BIM model and columns. the pre-design information will be created. • Pseudo Analysis of the loads for residential use 5. The architectural design can begin after the recommended in the ASCE 7-16. structural proposal is created. • Predesign used the AISC Code regulation and checks only the moment ratio and deflection ra- tio for principal length distances in the beams. 3. CASE STUDY • The slab system was based on a concrete com- Figure 3 posite steel deck (One way). Shape of the main • The revision of the structure was made using the case study spreadsheet only for the critical shapes. • LRFD gravitational combination (1.2DL+ 1.6LL) was used to review the structure’s resistance and L/240 for the deflection using the ASD combina- tion (1D L+0.75LL). • It considered the releases of the moment for the The study case was divided into two main parts. The structural connection beam-column. first part was for a specific shape of the multi-criteria • Base and centre columns were used to review optimisation process. The second one was how the only the axial ratio. tool could create several solutions with different reg- • The average weight of purling beams and steel ular shapes and how it can be helpful even if we are connections was added to the total weight of not looking to apply the parametric optimisation. For the structure. the example study, it was created using a tee build- ing shape with two variable shapes represented by a These simplifications were made mainly because, in building with 17 floors with two typical floor plans, the 2021 version of the software, Robot Structural one with 2075 and another with 1615 square meters Analysis cannot be used as an iterative calculation en- (Figure 3), the fixed input for the study was 40 cen- gine within Generative Design, it was chosen to per- timetres of facade offset from the steel frame system,

Building Information Modelling - Volume 2 - eCAADe 39 | 63 first-floor height of 4.0, typical level 3.5 and roof level The BIM allows to obtain additional information of 3.75 meters with a slab thickness of 15 centimetres this process and create a piece of information for the and 10 centimetres of finishing the floor. cost estimation and documentation process to the For the case study, the first step was creating project in early stages (Figure 5). multiple studies to debug the tool and review the co- herence of the results. Although Generative Design Figure 5 for Revit is a work in progress tool, for that case, the Structural BIM code was reviewed and tested several times, the test Model result of the showed the execution time, and possible results with optimisation a population of 20 and 10 generations performed well and in less than 2 minutes; this allows raising awareness and being able to establish the population and the proposed generation for new studies as well as determining the approximate time to solve it al- though it is not always exact. 3.1. Multi design tool optionery for Archi- Nevertheless, using genetic algorithms tools do tects not result in a precise and unique result, but a range The tool works directly with Dynamo Player; a tool in of possible solutions, due to the complexity of the Revit allows changing parameters inside the script in problem it requires different sizes of the population a friendly way. Figure 6 shows several options that for the first random generation; this is because the can be modified directly on the interface, those build- parents of the next generation could be no cover the ings composed with one or more towers, with differ- evaluation area of the optimal solution, that is, due ent geometries and variable heights among others. to the limited population, sometimes is impossible to The tool works with different shapes that the archi- reach the area where the optimal solution is found, tect can conceive in the early stages before finding which results in a false optimisation. a final solution. It can apply the optionery creation After several studies, was found an optimal so- in several shapes, changing the Dynamo Player in- lution that requires less amount of steel (53 kg/m2). terface parameter or creating the generative design On the other hand, was found another solution with a study to optimize the first structural frame. higher initial cost but worked for constructability be- cause we need less labour (72 kg/m2). These optimal 4. RESULT AND DISCUSSIONS Pareto solutions are shown in Figure 4. In our case Parametric visual design environments provide a study, the steel weight obtained in the pre-design powerful and flexible tool for defining and handling phase is closed to the average on the market [2]. The complex problems of optimisation. They enable erection of the building will be faster than a short- users to express the optimisation problem without span frame due to the reduced number of beams and developing complex algorithms. The output can also columns, reducing the labour cost and the schedule be placed in the BIM authoring software. of the project. Using the genetic algorithm for the optimisation The main benefit that gives the tool is that the process requires an in-depth knowledge of the prob- grid is set for the architectural design process’s start- lem to create a good and optimised solution. It is also ing point. This task reduces the re-work between an iterative process that requires generating multi- the other disciplines by transmitting the pre-design ple populations and generations to understand the information and assumptions within the BIM struc- results better and reduce the information bias. It is tural model created; the store information is shown. necessary to know the primary purpose to obtain the

64 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 4 Generative Result Min Weight vs Max Length.

desired result, especially if it is a multivariate prob- structure. lem. One way to simplify the complexity of goals The workflow was developed with Revit and is through internal penalties on critical outputs; this Dynamo, however, could be replicated in Tekla, reduces overfitting and the computational resources Rhinoceros and Grasshopper, as well as other soft- needed for the analysis. After several tests, the study ware with genetic algorithms. was set to produce more than 1000 solutions in less As future work, it is desired to seek the imple- than 30 minutes of computation time. mentation of the Robot Structural Analysis compu- Creating the BIM model allows us to connect tation engine through the API to refine the structural the parametric design results and host them for that analysis; in addition, it is proposed to integrate the re- point of view; we improve transparency in the infor- vision by lateral loads within the pre-design area, and mation, such as the assumptions by which the initial finally, it will seek to optimize the sections of columns proposal of the structural grid was made. This pro- by levels, given the concept of constructability. vides a first initial layout that could be improved sig- nificantly in the subsequent phases with the Struc- 5. ACKNOWLEDGEMENT tural Engineer without re-work in the structural grid The first author of the research has the financial sup- fit. port of the European Master in Building Information The tool provides real-time feedback on the Modelling (BIM A+) through the scholarship offer by spacing between the grids and the required beam Erasmus Mundus (Reference project: 599172-EPP-1- depth to achieve optimal distances considering the 2018-1-PT-EPPKA1-JMD-MOB), where the main part structure’s total weight. As a result, the weight per of this study was developed during the Parametric square meter is given for each of the architect’s pos- Modelling in BIM Module. Additionally, we want sible solutions sought. to thank Autodesk for the program “Softwares and The problem was simplified in this process. How- services for Education”, this allowed us to create ever, more variables can affect the final cost of the this study without investment in the main platform

Building Information Modelling - Volume 2 - eCAADe 39 | 65 Figure 6 Shape Optionery test

where this study was done. design from early digital architectural models’’, Ad- vanced Engineering Informatics, 22(2), p. 254–270 Moreno-De-Luca, L. and Bergambre Carrillo, O. J. 2013, REFERENCES ’‘Multi-objective heuristic computation applied to Arayici, Y., Onyenobi, T. and C., Egbu 2012 ’‘Building architectural and structural design: A review’’, In- Information Modelling (BIM) for Facilities Manage- ternational Journal of Architectural Computing, 11(3), ment (FM)’’, International Journal of 3-D Information pp. 363-392 Modeling, p. 55–73 Mueller, C. 2014, Computational Exploration of the Struc- Asl, M. R., Bergin, M., Menter, A. and Yan, W. 2014 ’‘BIM- tural Design Space, Ph.D. Thesis, Massachusetts Insti- based Parametric Building Energy Performance Mul- tute of Technology tiObjective Optimization’’, 32nd eCAADe Conference, Oxman, R. 2017, ’‘Thinking difference: Theories and 2, pp. 455-464 models of parametric design thinking’’, Design Stud- D., Davis 2013, ‘Modelled on Software Engineering: Flexi- ies, 52, pp. 4-39 ble Parametric Models in the Practice of Architecture’, Preisinger, C. and Heimrath, M. 2014, ’‘Karamba - A Ph.D. Thesis, RMIT University toolkit for parametric structural design’’, Structural Gradisar, L., Dolenc, M., Klinc, R. and Turk, Z. 2021 ’‘De- Engineering International: Journal of the Interna- signing Generatively to achieve an efficient and op- tional Association for Bridge and Structural Engineer- timised solution’’, 2nd International Conference on ing (IABSE), 24(2), pp. 217-221 Construction Materials for Sustainable Future, 1, pp. Rawlinson, S. 2015 ’Estrategia del BIM para México’, Ar- 84-95 cadis Hahn, P., MacGregor Smith, J. and Zhu, Y. R. 2010 ’‘The Sacks, R., Eastman, C., Lee, G and Teicholz, P 2018, ’A Multi-Story Space Assignment problem’’, Annals of Guide to Building Information Modeling for Own- Operations Research, pp. 77-103 ers, Designers, Engineers, Contractors, and Facility Holst, M. K., Kirkegaard, P. H. and Christoffersen, L. D. Managers, Third Edition’, in Sacks, R. (eds) 2018, BIM 2013, ’‘Parametric Optimization of Hospital Design’’, Handbook, Jhon Wiley, New Jersey, p. 688 Proceedings of World Academy of Science, Engineering [1] https://help.autodesk.com/view/RVT/2021/ENU/?gu and Technology, 7(4), p. 200 id=GUID-1B0DCFD8-E834-4E26-BFD3-BA6D9614FD89 Kaethner, S. C, and Burridge, J. A. 2012 ’‘Embodied CO2 of [2] www.steelconstruction.org. structural frames’’, The Structural Engineer, 90(5), pp. [3] https://hypar.io/about/news 33-40 [4] https://core-studio.gitbook.io/asterisk/ Keough, I. and Benjamin, D. 2010 ’‘Multi-objective opti- [5] https://trends.google.com/trends/explore?date=20 mization in architectural design’’, Spring Simulation 16-06-01%202021-05-24&q=Revit,%2Fm%2F0c0tdp,Te Multiconference, pp. 1-8 kla,%2Fg%2F1211j65g Mora, R., Bédard, C. and Rivard, H. 2008 ’‘A geomet- [6] https://primer.dynamobim.org/10_Custom-Nodes/ ric modelling framework for conceptual structural 10-5_Python-Revit.html

66 | eCAADe 39 - Building Information Modelling - Volume 2 Prototype of an Algorithm-Aided BIM Tool for Semi- Automated Generation of Modular Residential Buildings

Sophia Pibal1, Konstantin Khoss2, Iva Kovacic3 1,2,3TU Wien 1,3{sophia.pibal|iva.kovacic}@tuwien.ac.at [email protected]

The research on novel design tools to enhance the production of residential buildings was the objective of this study. This ongoing research within the funded research project ``Housing 4.0'' aims to develop a BIM-based digital platform for ecological and affordable multi-storey housing. As part of the platform we developed the prototype of an Algorithm-aided BIM tool for the semi-automated generation of modular building designs.

INTRODUCTION information rich BIM-object library of modular ele- The research on novel design tools to enhance the ments and secondly the “PHD”, a parametric habitat production of residential buildings has been the ob- designer for automated generation and visualization jective of this study. This ongoing research within the of floor plan and building configurations, as well as funded research project “Housing 4.0” (Housing 4.0: calculation of investment costs, return of investment Digital Platform for affordable housing - FFG Öster- and eco-indicators. This paper focuses on the pos- reichische Forschungsförderungsgesellschaft, 2019) sibilities of a tool to generate information rich build- aims to develop a BIM-based digital platform for eco- ing designs utilizing algorithm-aided tools combined logical and affordable multi-storey housing. As part with BIM. To explore parameters, constraints, require- of this project we developed the prototype of an ments and algorithms for the production of modu- Algorithm-aided BIM tool for the semi-automated lar housing we hereby developed this prototype as generation of modular building designs. a pre-study. Our objective was to determine how the framework of such an algorithm aided BIM tool has to Research Environment and Objective be designed, which parameters, constraints and re- This study has been conducted as part of the re- quirements for modular housing have to be obtained search project “Housing 4.0”. Modular building sys- and how an evaluation of sustainability and afford- tems combined with novel digital design methods ability has to be conducted. This paper focuses on and tools aim to enhance the production of afford- the framework of the tool and the method of model able and sustainable multi-storey housing. We de- generation and will not explain the method of model velop the digital platform “Housing 4.0”, that aims evaluation. Conclusions drawn from this prototype to enable stakeholders across the value chain to uti- will further be implemented in the “PHD” tool and the lize BIM-based planning tools and BIM to produc- digital platform. tion methods for modular off-site residential build- Modular Housing & Digitization. The implementa- ings (Kovacic et al 2020). This research project in- tion of prefabricated modular housing is not focused terlinks two areas: firstly “BIM4D2P”, an interface on the housing market due to the lack of production between planning and production that inhabits an capacity and also the lack of further development

Building Information Modelling - Volume 2 - eCAADe 39 | 67 (Hairstans, 2014; Albus and Drexler, 2017). An al- linked to BIM tools and enables object-oriented mod- ternate process should provide the stakeholder with eling through generated geometries and informa- an overview of the features of modular construction tion from AAD tools (Humppi and Österlund, 2016). (Schwerdtner et al, 2018). The modular construction Algorithm-aided BIM (AAB) describes the combina- method is a type of element-based construction, as tion of algorithm-aided design and BIM. It combines not only elements, but entire spatial units are pro- generative design processes with the creation of ob- duced in the factory and transported to the construc- jects with metadata. (Humppi and Österlund, 2016) tion site (Pawlitschko, 2018). Element-based con- Different parameters are assigned different values in struction, in particular modular construction, is sup- order to evaluate and compare the resulting output. posed to be a solution to the problem of low-cost To optimize such methods, algorithmic processes can housing (Aitchison, 2018). The construction indus- be used to replace manual processes. The follow- try has a lower level of industrialization and automa- ing relevant technologies dealing with algorithmic tion compared to other industries (Albus and Drexler and digital design of buildings were identified dur- 2017). This process is to be optimized by further de- ing our research: i) Finch3D[2], as a generative de- velopment of digitization in the construction indus- sign tool for architects, which is based on visual pro- try, integral and interdisciplinary cooperation with gramming but has no interface to BIM and is mainly increased customer orientation, an increase in the concerned with the production of 3D floor plan lay- degree of prefabrication and efficient execution of outs, ii) Testfit[3] as a generative prototyping tool, the remaining construction site work. The preced- which sees itself as a building generator for feasi- ing analysis of the current situation in modular res- bility studies in project development, iii) Eva[4] , as idential construction and the task of realizing pre- a CAD-based “rapid layouting” tool, for generating fabricated housing combined with BIM show a clear area layouts and preliminary studies and iv) Roset- trend (Li et al, 2019). In order to enhance the plan- taBIM (Feist et al, 2016) that supportes generative de- ning and construction process, the development of a sign for BIM and semantic information. In contrast to digital platform for modular multi-storey housing is the proposed prototype, it can be stated that these our main objective. tools are closed-source if they are algorithm-based, Prototype Tool & Algorithm Aided BIM. The design except for RosettaBIM they do not have a simultane- process and the finding of solutions for design prob- ous (semi-automated) coupling to BIM software and lems can usually only be managed by the compari- do not specifically deal with BIM-based generation son of design variants and is often highly extensive. of sustainable and affordable residential buildings. A To automate this process, we designed the prototype similar concept has been conducted by Ostrowska- by means of algorithm aided design and to reduce Wawryniuk and Nazar (2018) for prefabricatedmulti- repetitive steps in design to the essential inputs. The family housing design by comparing multiple design main task of the framework for the prototype is there- options and supporting the decision-making based fore the algorithm-aided script that semi-automates on an analysis and splitting process of already exisit- the process of generating BIM model variants. In ing input models. In our study, an algorithm-aided addition to textual programming, visual program- design process is developed and the capabilities of ming tools (Feist et al, 2016) offer the possibility this principle are examined. The advantages of gen- of designing programs as visual scripts. By means eration of content by means of algorithmic scripts of code building blocks (nodes), which are linked can be used and together with the advantages of with wires, algorithmic design processes can be cre- BIM, can lead to a more efficient generation of vari- ated. In Graphical Algorithm Editors, like Dynamo or ants of multiple BIM models. The linked form of these Grasshopper, this visually programmed code can be two design methods currently does not have an offi-

68 | eCAADe 39 - Building Information Modelling - Volume 2 cial term. Aish (2013) calls this form “Parametric BIM”, rectly in Dynamo and Revit. Outsourcing of these Caetano and Leitão (2019) refer to this method as partial steps to Excel would also be possible and but “algorithmic-aided BIM”, and Humppi and Österlund is not the focus of this work. The script consists of (2016) name this “Algorithm-Aided BIM” (AAB). two parts. The first part being the algorithm-aided generation of the BIM model, the second part being Figure 1 the evaluation of the model. Since this prototype is Workflow of model designed to be a framework to explore the data and generation and methods needed for a fully-automated tool, we pro- software ceed the iterative process of variant generation semi- automated. The iterative process is shown in figure 2. It has to be stated that in this early stage and proto- Figure 2 type we did not strive for a fully automated genera- Semi-automated tion nor multi objective optimization. We proceeded iterative Process of the generation semi-automated; the iteration is re- Variant Generation peated until the designer is satisfied with the output.

Parameters Despite the individuality of residential buildings, RESEARCH DESIGN there are still recurring factors in every project. We In this section the research design and methods will plan and build for people and their needs. These be divided into four key aspects. We focus in the over- needs are so essential that they are defined by laws all i) framework for the Algorithm-aided BIM tool, ii) in order to meet the minimum requirements when the parameters, iii) the method of generation and the planning residential buildings (room height, accessi- iv) case studies. bility, sanitary, lighting etc.). Even if these rules leave restrictive effects, these factors provide a framework Framework Algorithm Aided BIM Tool within algorithm-aided planning can happen. Due The software used in this study has been Revit, Dy- to its recurring elements, the multi-storey residential namo and Excel. We used visual programming in- building can be typologized into basic elements such side Dynamo to script the algorithm, Revit to visual- as access, supporting structure, apartment units, fa- ize and store the BIM model and Excel as a data base, cade and balconies. By means of these categories, input tool and interface. In order to ensure verifiabil- existing buildings can be subdivided and compared ity of the entries made and also of the individual sub- according to individual aspects (Stamm-Teske et al, steps in generating the model, the input of the pa- 2012). Conversely, residential buildings in the plan- rameters is outsourced to Microsoft Excel. Dynamo ning stage can also be structured in this way ac- provides its own code blocks for this purpose, which cording to these elements in order to form differ- allow individual cells to be exported and written to ent variants and compare the results. Consequently, an Excel file. Meaning that the workflow for creating the method of generating a systematically genera- the model follows the following pattern (figure 1): In- tive residential building is based on its components put (parameters in Excel) - generation (algorithm in and their typologies. The generative design process Dynamo) - output (BIM model in Revit). When eval- follows a sequenced process in which the following uating, the process works in the opposite direction: components are generated: grid, supporting struc- model (Revit) - export (Dynamo) - evaluation (Excel). ture, access modules, apartment modules, façade Certain inputs, especially when setting up the work- and balconies. flow for the first time, are nevertheless necessary di-

Building Information Modelling - Volume 2 - eCAADe 39 | 69 Method of Generation Figure 3 In this section we explain the generative process of Method of Model the algorithm-aided BIM tool. The process of gen- Generation - erating the model is based on the individual sub- Sub-steps and elements and parameters of residential buildings. In Parameters figure 3 the sub steps of the generative process of modeling are shown schematically. The input of the parameters is divided into a grid, which serves as a basis for all further elements. Grids and floor lev- els are generated depending on the project origin. Within this generated three-dimensional system of grid and levels, the elements of the supporting struc- ture, the access and the apartment units can be lo- cated and generated. The balconies and the facades are ultimately positioned depending on the apart- ment units. The model is generated step by step and can be supplemented manually at the same time. Working Environment. In Revit, a building mass and level are modeled. The building mass in this case is cuboid and has the parameters for width, length and height. This approach theoretically makes it possible to use other volume shapes besides that of a cuboid as initial volumes. Mass families can be created in Re- vit with any basic shapes and parameters. The grid generates itself in analogy to the cartesian coordi- nate system from this point in the positive directions Figure 4 along the X, Y and Z axes of the project file. Next, the Working individual software must be linked, the mass in Revit Environment - is linked in Dynamo and the file path to the Excel file Software Chain is specified (fig. 4). Grid. A grid is used to locate the individual elements within the volume. In Dynamo, the mass volume, its dimensions as well as the grid and the floor heights are adjusted dynamically and in real time (fig. 5). This Figure 5 method is especially useful in an early design phase Grid Generation when investigating the volume dimensions at the either inside building site and favorable spans of the grid. By en- Dynamo or Excel tering the exact dimensions and grid spacing in the Excel spreadsheet, individual adjustments can also be made to individual grid lines. These constraints can be used to generate a grid in which subsequent elements can be located by grid-points and levels.

70 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 6 lected here. New component types must be created Supporting in Revit beforehand. When the entries are completed Structure inside and the script is executed all elements are generated Excel and Dynamo and placed in Revit. Access Modules. In order to be able to generate ac- cess concepts that are as individual as possible, the first step is to create the different access modules in Revit. This makes all Revit elements of the element li- Figure 7 brary available and the access can be modeled in any Access Module level of detail. Each type only needs to be created Generation inside once and then grouped into a model group. Once Excel the groups have been created, they should be listed in a table and located in the project. Again, the grid and the layers serve as a coordinate system to posi- tion the access modules. In a second step, the nec- essary openings (for stairs, elevator and other shafts) must be entered. The script in Dynamo is then used to position all elements and generate openings (fig. 7). Furthermore, rooms are also placed for each core in order to obtain information about the floor area. Apartment Modules. The apartment types and mix are first determined using Excel. Here, the apartment sizes or the minimum and maximum area per apart- Supporting Structure. After the grid has been gen- ment type are defined as well as the number of apart- erated, the first elements to be added are those of ment units (as a percentage or number). The tool the supporting structure. The table inside Excel can uses the total area minus the previously created ac- be used to create individual support structure types cess area to calculate approximately how much space in order to allow for floor-by-floor variations. The is available. Using the apartment table inside Excel, required structure is thus generated from the basic it is possible to calculate the necessary number and elements of slab, wall and column. The core usu- type per floor. With this information, floor configura- ally also has a load-bearing and bracing function, but tions can first be tested analogously. The floor plans should not be created until the next step - access - of the apartment units should be created as mod- because the affiliation of the individual elements de- ules inside Revit. Similar to the modeling of the ac- pends on the configuration of the core. In this step, cess modules, it is sufficient to draw and group each the grid structure is now used for the first time to type once. In order to keep the workload for this as locate the individual elements. Each position in the low as possible and still meet the most important pa- building model can be selected via the grid points rameters, a simplified system of lines is used for the (fig. 6) and the respective level. Since the grid is also creation and generation of the apartment modules. the construction grid, the input here does not allow This system will later also serve as a reference point any deviations from the grid lines. After the table for the façade and the private balconies that directly has been filled, the components and their superstruc- relate to the individual apartment units. Each apart- tures or cross-sections are selected in Dynamo. All ment floor plan consists of different elements that types that exist in the project file in Revit can be se- can be arranged quite differently depending on the

Building Information Modelling - Volume 2 - eCAADe 39 | 71 concept. However, externally, each apartment can the outer shell, a terrace if the apartment is on the be reduced to a few elements. First of all, each apart- first floor and a loggia if the door is inside the vol- ment is bounded by walls: a partition wall with an en- ume. The threshold value for when the prototype dis- trance door towards the access area, a partition wall tinguishes between a balcony and a loggia is given towards the adjacent apartment units, and a façade here as 50 cm from the wall, to provide a tolerance for wall with openings for lighting and ventilation (win- the thickness of the facade wall and the installation dows) or for access to the private balconies (façade depth of the doors. For the creation of the balconies, door) towards the balconies. Consequently, the floor the dimension of the balconies is first required, which plans can be reduced to three different walls delim- can be specified via the table. Either the same floor iting the apartment, as well as windows, doors and area can be assumed for all apartment units or differ- an entrance door. First, the different outlines of the ent sizes can be entered. Finally, the ceiling type and apartment units are drawn using lines. For this pur- the railing type are selected in Dynamo. pose, custom line styles were created in Revit, which define the individual functions of the boundary. Fi- CASE STUDIES nally, the lines are grouped and named according to In this section, the previously created algorithm- their modules. Excel is then used to assign the apart- aided script is applied to generate two case stud- ment modules to their positions in the model. After ies (cf. figure 8). These were designed for the the modules have been entered, the individual com- same building site as modular multi-storey residen- ponent types must be selected. This is done directly tial buildings. Due to the similar project task and in Dynamo, where all the components available in building site, the comparison resembles a compari- Revit are available for selection. Windows and doors son of variants. The primary goal is to verify the fea- are positioned in the middle of the respective line. If sibility of the algorithm-aided BIM tool. The work- several windows are to be generated along a facade, flow and manual remodeling effort have been doc- the line only has to be divided into individual seg- umented and the results are analyzed. The available ments. Each section is given an opening. project data serves as a basis for their parameter val- Facade. The generation of a facade, especially with ues and will be explained in the following. complex shapes, can be made possible by means of Case Study A. To create the algorithm-aided BIM a specially designed script. However, the aim of this model of project A, floor plans and cross-sections of work and this script is to generate a residential model the realized project were used as a basis. Using these of an early design phase and to generate the ele- plan documents the necessary information was gath- ments. Therefore, the generation of the façade is lim- ered to provide the tool with the form-giving param- ited to the wall or façade elements and openings, us- eters.The generation of the building, focuses on the ing only the wall and window families from Revit. The residential floors (1st-4th floor). Nevertheless, the in- positions of the facade walls result from the previ- dividual floor plans in the basement, ground floor ously generated and positioned lines of the residen- and top floor are an essential part of the residential tial modules. By default, the openings are positioned building. How such areas can be dealt with in the in the center of each wall element. course of generation is to be investigated and inte- Balconies. Each apartment is assigned its own bal- grated into the workflow. The basement was com- conies. When creating the floor plans of the apart- pletely constructed on site and therefore has no part ment units, the position of the doors was already de- in the prefabrication process of the building. For the fined as access points for the balconies. The script as- grid structure, four continuous axes can be identified signs a balcony to the opening based on its position. along the building, which are positioned equally in A balcony is generated if the opening is directly on the ground floor as well as in the residential floors.

72 | eCAADe 39 - Building Information Modelling - Volume 2 These are at varying distances from each other and ered to provide the tool with the form-giving param- form spans of approximately 5 m at the outer two eters.Here, just as in the generation of the model of bays and 2.85 m in the center. Along this primary case study A, the main focus of this project is also on support structure, the orthogonal axes could be ar- the residential floors. The ground floor and the top ranged at irregular intervals. There are only a few cor- floor are nevertheless included in the process, since respondences between the floor zones. For the gen- these floors, with their spaces and functions, occupy eration, the column grid is taken from the residen- an important part of the entire building. Since the tial floors and supplemented by individual grid lines basement floor is not part of the modular design, this for the ground floor. The apartment walls often do level is neglected in the generation process and only not lie in the axis line of the column grid. Such de- the above-ground floors are generated in the model. viations from the grid for apartment floor plans are This also provides better comparability to Case Study possible, therefore no additional grid lines have to A, in which only the above-ground floors were gen- be generated for this. The independent structure of erated as well. The building follows a clear grid of the development also shows different grid distances 3.40 m x 6.50 m. On the first floor, in addition to the than the rest of the building. Here, the generation is access and the load-bearing elements made of rein- resorted to grouping of building elements, which is forced concrete, there are isolated non-load-bearing, generated floor by floor. The elevator core is located space-creating walls and glass facades with a post- in the building structure and is generated as a sec- and-beam construction. These non-load-bearing el- ond access element. Finally, balconies are added to ements are to be modeled manually. The residen- the residential floors. These are mainly located on the tial floors can be fully subdivided into load-bearing southeastern facade and on the adjacent shorter fa- structure, access modules and apartment modules cade sections. A slab size of 2.20 m deep and 3.40 m and are generated utilizing the tool. Each apartment wide is envisaged. Slabs partly differ in size, adjust- is assigned a balcony on the southeastern side of ments are made manually. Finally, individual manual the building, which has the same dimension for each additions have to be made, especially on the ground apartment. The top floor, similar to the ground floor, and top floor. will be manually completed after the generation of Case Study B. The 3D model in Revit format, which access and support structure has been complete al- was created as a variant study during the research gorithmically. project serves as the basis for project B. By analyzing Feasibility. The semi-automated generation of both the Revit model, the necessary information was gath- studies can be performed with the tool but with few

Figure 8 Resulting BIM Models of Case Study A and Case Study B

Building Information Modelling - Volume 2 - eCAADe 39 | 73 manual additions. The approach of the script works above the balcony elements on the southeastern fa- and through the subdivision into its individual in- cade. This completes the entire BIM model of Project puts the residential building models can be gener- A. Manual modeling - Case Study B: Like the model of ated. A tool chain was created that provides a semi- case study A the generated model of case study B is automated design process. As expected, those com- not fully completed with just the prototype tool. The ponents of the residential buildings that have the elements, which were not generated in the model largest proportion of elements (the access and the due to their individual occurrences, still have to be apartment units). These areas are also the main el- added manually. Similar to model A from the first to ements generated algorithmically by the tool. The the fourth floor, all elements could be generated. On goal is to create an algorithmic design process that the ground floor exterior walls and facades as well as automates work steps while still allowing for the rooms excluding the generated access core had to most diverse design forms possible. For this reason, be added manually. On the top floor roof area, ex- certain manual steps, such as the design of the ac- terior walls and facades, rooms, roof terrace and the cess modules or the design of the floor plans, remain. railings of the roof terrace were not generated algo- With Project B, the steps could be completed in less rithmically. Nonetheless, the share of manually cre- time overall. This is due to the more regular grid of ated elements of a project should not exceed and the project and the subordinate elements within it, thus make an algorithmic design tool obsolete. The which can be arranged exactly on this grid. Neverthe- optimization of the design process by a tool is bene- less, the initial generation of both projects, including ficial if a modular structure is provided with the con- manual additions could be performed. cept of the design itself, which makes up a large part Manual Modeling. The creation of the algorithm- of the project and where (semi-)automation can take aided BIM tool was accompanied by an ongoing pro- place. cess between the algorithmic generation of elements Workflow. Excel was chosen as the user interface versus the manual creation of these elements in Re- to enter parameters. In terms of usability, a pro- vit. Thereby an optimization in the design process by grammed input mask that includes all necessary in- algorithmic functions should not form a restriction in put fields and data outputs while performing the cal- the design possibilities. For this reason, parts of the culations in the background would be more suitable. script were combined with a manual creation of ele- The danger of overwriting formulas or tables, both ments. The apartment and access modules are com- by the user and by an error in the script, could thus ponents, that are created in a combination of algo- be avoided. Furthermore, entering the grid points rithmic and manual modelling. In addition, the case in a table to specify the positions of individual mod- studies show that there are areas in the building that ules is not very intuitive. The specification of the are not generated by the tool. These include the in- positions on the grid via a graphical representation dividual ground floor and top floor. Manual model- of it would be clearer and would make it immedi- ing - Case Study A: From the first to the fourth floor, ately apparent in which areas overlaps or gaps oc- all elements could be generated. As a final step the cur. The used configuration of the three programs BIM model is supplemented with the necessary el- serves thereby rather a clear subdivision of the algo- ements, in particular, the missing rooms have to be rithmic expirations than user-friendliness. Since the placed. On the ground floor, individual walls are now testing of the script concept was treated with prior- added manually. On the top floor, the individual de- ity, the division into three programs is justified at the sign is modeled and the roof terrace is defined ac- expense of the usability. Despite the disadvantages, cordingly as a balcony. In addition, there are two can- a major part of the project could be generated algo- tilevered ceiling slabs on this floor, which are placed rithmically. The manual modelling was mainly lim-

74 | eCAADe 39 - Building Information Modelling - Volume 2 ited to the individual floors (ground and top floor). script can generate a multi-story residential building The residential floors as well as the access modules from individual modules. These modules can consist could be generated completely algorithmically. An of manual configurations (access modules), be gen- increased degree of repetition of elements and mod- erated by a simplified manual input (apartment mod- ules allows for a more generative workflow in the ules from line floor plans), or be completely defined project. In case study B, the design was able to be by parameter values (grid, supporting structure, fa- generated to a greater extent due to its structure con- cade, balconies). The choice of parameters deter- sistently aligned with the grid and a higher number mines the variety of results. of identical elements. Furthermore, the subdivision Workflow and Software. The combination of Revit, of the elements according to the components of the Dynamo and Excel as a software environment en- residential building represented a clear advantage in ables a clearly structured workflow of all steps in the the generation of the model. The order of subdivi- process. The linking of Revit and Dynamo enabled sion allows the focus to be initially on the supporting the data exchange due to the native interface. Code structure elements and subsequently supplemented building blocks, which are available specifically for by more flexible elements. Overall, it was possible to Revit elements, enable intervention in the BIM ele- generate a complete BIM model in both case stud- ments. When performing operations to enable trans- ies, which did not cause any incorrect overlapping of formation of Revit elements, one encounters obsta- components in the building model due to the exact cles. Dynamo’s influence is limited to adjusting indi- algorithmic positioning of the individual elements. vidual parameters. However, since the position and Only individual balcony modules had to be subse- orientation of the elements are stored in a deeper quently moved in project B. All component and area level of Revit, these can only be accessed via the so- lists were also correctly and automatically included called Revit API. In Dynamo, however, there is the in the evaluation Utilizing the tool it became obvious possibility of accessing and manipulating this con- that the whole process depends on the grid consist- tent as well via specially created scripts written in pro- ing of levels and 2D grids. gramming language. In Dynamo itself, however, er- rors occurred in the execution of the script, which CONCLUSION may be traced to a not completely mature software. AAB and Modular Housing. The use of algorith- Certain process errors could only be fixed by restart- mic processes in design often leads to a discussion ing the programs. Likewise, difficulties arose due to about the design diversity and architectural qualities the linking of Dynamo with Excel. The import and that would supposedly be limited by it. Therefore, export modules for Excel, which are available in Dy- the framework deliberately refrained from creating namo, required a new link to the target file before parameters for all elements of the residential build- each execution in order to take over the current sta- ing. The basic structure, consisting of grid, support- tus. The workflow between the three programs turns ing structure, access modules, apartment modules, out to be feasible. Especially for the creation of the façade and balconies, is contained in each project script, this environment enabled a clear sequence and can therefore be used as a superordinate pa- and allowed a step-by-step testing and adjustment of rameter group. Finally, to subdivide the individual the individual inputs of the algorithm-aided BIM tool. components into further elements in order to define During the two case studies, the separation of input, them by parameters requires a trade-off between the script, and output has also been beneficial. Despite three factors: scope of algorithmic programming, ap- graphical representation with code building blocks plicability in a large number of projects and restric- in Dynamo, the constructed script quickly becomes tion of diversity. It was shown that the developed confusing and slow. Outsourcing the input fields al-

Building Information Modelling - Volume 2 - eCAADe 39 | 75 lows the execution of the algorithmic processes to generative design for building information mod- take place in the background while the focus is on Re- elling’, CAADRIA 2016 vit and Excel. Hairstans, R 2014, Building Offsite. An Introduction., Ed- inburgh: UK Commission for Employment and Skills In the course of the study, the possibilities of- (UKCES) fered by algorithmic design have been explained. It Humppi, H and Österlund, T 2016 ’Algorithm-Aided BIM’, was shown that modular housing can be divided into eCAADe 2016 individual components and generated by parame- Kovacic, I, Pibal, S, Reisinger, J and Lorbek, M 2020 ’Dig- ters. For this purpose, the basic elements of hous- ital Platform for Affordable Housing - a Framework ing provide the first step towards parameterization. Proposal’, Engineering Project Organization Confer- ence (EPOC) 2020 - Disruptive Engineering, online, pp. They are divided into sub-elements, which are gen- 1-16 erated algorithmically. In an iterative design process, Li, X, Shen, G. Q., Wu, P and Yue, T 2019, ’Integrat- the parameters are assigned different values several ing building information modeling and prefabrica- times, thus enabling systematic improvement of the tion housing production’, Automation in Construc- results. Automated interfaces to cost data and build- tion, 100, pp. 46-60 ing physics calculation programs as well as multi Ostrowska-Wawryniuk, K and Nazar, K 2018 ’Gener- ative BIM automation strategies for prefabricated optimization processes would enable a further step multi-family housing design’, Computing for a bet- towards process optimization. The framework also ter tomorrow-Proceedings of the 36th eCAADe Confer- shows that a repetition of elements in the design is ence, 2018 not only in the sense of a subsequent serial prefabri- Pawlitschko, R 2019, ’Gestapelte Holzmodule’,in Jakob, T cation, but can also optimize the planning and execu- (eds) 2019, Modulbau. Planen und Bauen mit Raum- tion process. The use of the tool is therefore also pri- modulen und vorgefertigten Elementen. Erfahrungen aus der Praxis für die Praxis, DETAIL Business Informa- marily efficient for those projects that aim at modular tion GmbH construction or serial prefabrication of individual ele- Schwerdtner, P, Kumlehn, F and Schütte, J (eds) 2018, ments. It should be mentioned that algorithm-aided Kostengünstiger Wohnungsbau: Identifikation beste- BIM tool represents solely a prototype system for al- hender Hemmnisse für den Einsatz von Raummodulen gorithmic design of residential buildings. In further im Wohnungsbau, Fraunhofer IRB Verlag consequence, this script should be extended by addi- [1] https://projekte.ffg.at/projekt/3307426 [2] https://finch3d.com/ tions and extensions, both in generation and evalua- [3] https://testfit.io/ tion, to a more comprehensive design tool, which sig- [4] https://www.rapidlayouting.com/de/imprint/ nificantly supports the design work in the early plan- ning phases and will be implemented into the digital platform “Housing 4.0”.

REFERENCES Aish, R 2013, ’Inside Smartgeometry: Expanding the Ar- chitectural Possibilities of Computational Design’,in Peters, B and Peters, T (eds) 2013, First Build Your Tools, John Wiley & Sons, Ltd Aitchison, M 2018, Prefab Housing and the Future of Build- ing: Product to Process, Lund Humphries Albus, J and Drexler, H 2017, ’Die Potentiale vorge- fertigter Konstruktionssysteme im kostengünstigen Wohnungsbau’, in Schönig, B, Kadi, J and Schipper, S (eds) 2017, Wohnraum für alle ?!, transcript Verlag Feist, S, Barreto, G, Ferreira, B and Leitão, A 2016 ’Portable

76 | eCAADe 39 - Building Information Modelling - Volume 2 Safety Information Modelling to Support Planning of Archeological Restoration Site

Preserving workers from COVID-19 at “Venus and Rome Temple”

Armando Trento1, Antonio Fioravanti2, Daniela Borgese3, Andrea Gratteri4 1Bimtegra Srl 2Sapienza, University of Rome 3,4A61S Studio [email protected] [email protected] 3arch. [email protected] [email protected]

At the time of a global pandemic, risk management in the AEC industry faces novel challenges: to ensure continuity production in worksites, National Governments have defined general protocols. Restrictions regarding individual behaviours are based on criteria as simple as possible but, designing and planning Healthy and Safe (HS) site activities introduces a higher level of complexity to be managed. Narrowing the field to Archaeological buildings, this research aims at defining a method and implementation path for a system supporting HS designers in reducing Covid-19 risk in restoration worksites. Methodologically, an action research approach was adopted, experimenting with some engineering requirements in the case study of the ongoing restoration of ``Temple of Venus and Rome'' in the ``Parco Archeologico del Colosseo'' in Rome.Since each scheduled activity assigns a number of actors to a workspace for a time-lapse, the idea is to check HS space classes (e.g. working; resting; paths; storage; etc.) - modelled extending commercial BIM tools - against ad hoc process rules (e.g. maximum presence allowed simultaneously). This early-stage plug-in system evaluates Covid-related safety performance of designed 3D worksite layout versus 4D execution program, eventually providing advice about distancing and physical interferences.

Keywords: Safety Planning, BIM for HS, Risk Management, Archeo BIM, HS Analysis

INTRODUCTION that each decision infers on Health and Safety (HS) Designing and planning the construction of a new risks for actors involved in the worksite activities. building or restoring the restoration of an existing The design of safety presents, schematically, artefact activate a deep knowledge about the impact three elements of attention:

Building Information Modelling - Volume 2 - eCAADe 39 | 77 • identification and assessment of all concrete The investigation and technical developments risks and the consequent preparation of suitable are oriented to support the planning and manage- measures to prevent them (prevention and pro- ment of construction site for reducing personnel tection measures); safety risks as a primary interest. • communication of risks and prevention and protection measures, through information and STATE OF THE ART signs; Zou and Kiviniemi (2011), as a result of a systematic • implementation of prevention and protection and critical literature review on the state-of-the-art of measures in relation to the anticipated risks and BIM-based risk management, shows that traditional those that may arise in the executive phase. risk management implementation is still a manual According to the EU Directive (CSD) (92/57/EEC), undertaking. Moreover, he added that the assess- Health and Safety risk assessments are functions of ment is heavily reliant on experience and mathemat- physical entities on one side and, on the other side, ical analysis, and the decision making is frequently they can be linked to more abstract coordination pro- based on knowledge and experience-based intu- cess activities. ition, which leads to a decreased efficiency in the real Institutional databases, currently available on environment. the web, offer useful libraries of statistic-based re- To improve the above situation, some standards sources: typically, they assume that every element or governmental documents (e.g. ISO 31010:2009, of the building execution program can be linked to CDM regulations) emphasise on predictable risks be- some form of HS risk mitigation (author’s reference). ing identified and mitigated at an early stage. In addi- This assumption is correct, but in safety plan- tion, risk information should be documented and up- ning and design practice, the most urgent problem dated during the development process of a project. is represented by efficient interferences evaluation of These are tasks where BIM technology could pro- planned activities, not simply intended as the sum, vide efficient support. BIM platform can be used both but as the function of coincident risk conditions, es- as a systematic risk management tool in the devel- pecially in the time of COVID-19. opment process and also as a core data generator to Safety Analysis (SA) procedures, such as haz- allow other BIM-based tools to carry out further risk ard and operability analysis and effect analysis, are analysis. generally regarded as repetitious, time-consuming, BIM-based technologies for HS risk management costly and require much human involvement can be distinct in different categories, including auto- (Damarola et al.). matic rule checking, knowledge based systems, reac- Although - at this stage - human expertise is irre- tive and proactive safety systems based on informa- placeable in the conduct of effective safety analysis tion technology (e.g. 4D Cad, Virtual or Augmented procedures, there is a need to reduce the amount of Reality, GIS-based systems, etc.) (Zou 2013; Authors human effort and cost of SA. 2014). This paper discusses a method for strategically BIM-based collaboration and communication matching HS program elements to construction pro- environment could naturally facilitate the early risk cess requirements, engineered by means of explicit identification and mitigation (Dossick and Neff, 2011; relations among actors, activities and places. Ac- Grilo et al. 2010). cording to context-dependent factors, the cumula- The integration of BIM and knowledge-based tive risk for a construction process can be assessed, systems has been a research path for a small but pro- and HS program elements may be ranked and se- lific community at the global level. lected based on their ability to mitigate the risk. Deshpande et al. (2014) proposed a new method

78 | eCAADe 39 - Building Information Modelling - Volume 2 to capture, extract, and store information and knowl- ing, provisions evolve accordingly, transforming edge from BIMs, and presented a framework for clas- the layout model during the building/restora- sifying and disseminating of the knowledge. To tion/demolition progress. strengthen its practical application, Ho et al. (2013) developed a BIM-based Knowledge Sharing Manage- ment (BIMKSM) system that could enable managers PROPOSED METHODOLOGY and engineers to share knowledge and experience in The research field of this paper is knowledge-based the BIM environment to strengthen its practical ap- reasoning for automatic rule checking by develop- plication. ing reactive and proactive ontology-based safety sys- Autors’ (2007, 2014) proposed a Building Knowl- tems. edge Modelling methodology, which enriches BIM This research proposes a novel approach for information with higher levels of semantics: this on- extending existing Building Information Modelling going research demonstrated that linking the IFC that, at the state-of-art, can rely only on entities for- database to ontology knowledge-base systems al- malised “per se”, namely geometrical items fulfilled lows goal-oriented automatic reasoning for support- by isolated, object-specific information. ing project design and management. We assume that effective automation can be en- Other complementary research that belongs to hanced if design/management tools allow modelling Automatic Rule Checking category is the use of a HS knowledge and the reasoning on top of design computer program to assess a design based on ob- entities qualified by clearly established relationships. jects’ configuration (Eastman et al., 2009). Its pur- To face complex problems, BIM models should pose is to encode rules and criteria by interpretation, be able to implement and manipulate multiple sets and thus building models could be checked against of entities belonging to organically structured and these machine-read rules automatically with results, oriented (sub-) systems. for example, ”pass”, ”fail”, ”warning”, or ”unknown” In this paper, a method is proposed for strategi- (Borrmann et al., 2009). cally matching HS program elements to construction Because BIM-based risk management is an process requirements, represented by means of ex- emerging development, there are still some tech- plicit relations among actors, activities, and places. nical limitations and lack of ‘human factor’ testing in The technical implementation consists of apply- practice ing automated HS rule checking to Building Informa- According to literature and authors’ research and tion Models (BIM), namely safety space entities linked practice experiences, some limitations of the current to scheduled activities. tools and methods are: “Judgment based” (Eastman, 2009) safety evalu- ation and suggestion are performed by introducing • modelling of actors involved for specific activi- ad-hoc indicators formalized by means of data avail- ties in the spaces of the worksite is usually ne- able in the project documents / models (e.g. time glected; schedule, layout, survey, etc.). • families of HS site arrangements are not widely Indicators measure physical / non-physical at- available in BIM libraries; tributes about safety entities, not available in current • to be able to exploit the advantages of the au- BIM / IFC models. tomatic system, it is necessary to spend consid- Methodologically, an action research approach erable resources in modelling both the building was adopted for this project, for its ability to link re- and the security preparations and provisions; search to practice in order to solve a realistic problem • safety arrangements are temporary by defini- in its actual setting. tion: as the safety layout is constantly chang- In the presented case study, we primarily anal-

Building Information Modelling - Volume 2 - eCAADe 39 | 79 ysed how to represent a safety condition in terms of presented work focuses on evaluating the risk as- objects, attributes and relations needed. Afterwards, sessment plan through a rule-based engine imple- we formally defined the routine to run the risk eval- mented on top of a commercially available BIM plat- uation, intended as complex rules to be applied to form (Figure 1). the objects, attributes and relations involved, both in The research is supported by an early-stage pro- Space and Time. totype tool, which was conceptually assessed for “Covid-19 security conditions” have been de- conducting a preliminary evaluation. fined as formally engineered requirements: the en- The implementation steps are namely: riched BIM systems, relying on these formal rules • Represent a vaste library of Safety oriented (quantitative, relational, etc.), can process the rule- properties and rules within BIM families (by checking on modelled HS entities or their groups. means of API), with special emphasis of Safety Final output can be effectual at different levels: Program Entities (e.g. in Autodesk Navisworks, Visualisation in BIM of risk information; Communica- Solibri). tion to Actors involved of the HS implication; support • Represent Safety Risk entities related to Con- for designers involved in worksite planning providing struction/Restoration Activity (e.g. expressed in advice for safety risk mitigation. OWL language by means of ontology editors, e.g. Protégé). IMPLEMENTATION PATH • Link Construction/Restoration Activity (time/s- The problem here approached focuses on defining pace instances) with actual Safety Program En- a set of occurrences that dynamically happen in the tities to evaluate the Risk Assessment (e.g. map- working site. If appropriately organized, activities, ping Revit BIM database with HS knowledge on- represented by means of structured process knowl- tology base through Autodesk Revit DB Link). edge, allow planners and designers to represent us- age scenario, predicti activities risks and evaluate the The technological pipeline (Figure 1) is pre- performance of the designed working site in terms of sented and discussed in author’s previous publica- punctual elements of a safety program. tions belonging to many years work in the Sapienza’s To avoid some typical limitations of the HS risks research group (Carrara et al., 2010). evaluation, authors define and then implement some First step: design process envisages the creation classes and rules of a more general Health and Safety of a visual proposal through the definition of spaces design ontology, starting from an already tested and components using the families available in BIM. knowledge modelling theory (Carrara et al, 2010). It proved necessary to ‘customize’ Revit, creating The approach offers a viable starting point for new, specially designed shared parameters. Two new further research and development aimed at identify- parameters were defined and associated to all fami- ing potential site safety issues from the earliest stages lies: of planning activities. - Ontology Class Name: required to connect Re- It ensures that explicit knowledge contained in vit instances to an instance with a name chosen from both the requirements document and previously the knowledge structure implemented in Protégé documented projects can be used to achieve a reduc- Frames; tion in the cost / risk of HS analysis by using estab- - Ontology ID: set up to ensure that the instances lished technologies such as ontology and case-based represented had one single ID and so keep track of reasoning in the field of AEC. instances, thus allowing us to modify them and carry While testing/refining the workflow between out a versioning of the solutions put forward. Ontology and Building Information Modeling, the Second step: each of the visual instances that are

80 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 1 Project data workflow

modelled can be associated with the ‘logical concept’ Data that has been corrected can then be to which the instance represented needs to be con- exported once more using Protégé into another nected in the knowledge structure. database; the new database will therefore represent This allows us, for example, to extend the char- a further version of the design solution in progress. acterization of classes and families provided by Revit, To test the feasibility of the proposed method, a making the most of the representational potential of real case study of such a system is provided: oper- ontologies ative activities plan in a relevant heritage restoring Third step: to export the design solution devel- project are checked in order to advise and automati- oped into a suitably created MySQL database, pre- cally suggest case by case, adequate HS measures for pared earlier, involves the use of a particular Add-on, preventing COVID-19 virus spreading. known as a Revit DB Link. Fourth step: having previously defined the two CASE OF STUDY: RULES FOR REDUCING shared Ontology Class Name and Ontology ID pa- COVID-19 RISK rameters, it will therefore be possible to import the This work narrows the field of application to histor- database created from Protégé using a further plug- ical buildings, specifically to restoration of archaeo- in called DataMaster v.1.3.2. logical artifacts, where the intervention is mainly, but Fifth step: at this stage, the potential of ontolog- not exclusively, conservative, maintaining the gener- ical structures allows us to verify and check design alizability of this specific experiment. rules and constraints of all kinds - specialist, common, In this case it is not necessary to model the his- collaborative - following a proper modelling stage. torical artifact in BIM, because it is interested by a su- The definition of constraints of cardinality, calcu- perficial transformation not relevant for the goal of lation and/or inference rules allows us to query the this research. For visualising the context of interven- data imported and analyze the degree to which it tion, artefact point cloud is more appropriate (Figure complies with the semantic structure developed. 2): by dividing and naming discrete subsystems, it is The results of such processes will end up with a possible to identify the geometry of the parts to be check of logical and/or geometric data, pointing out restored. if any change of the model/plan should violate par- Differently, the characteristics concerning design ticular rules.

Building Information Modelling - Volume 2 - eCAADe 39 | 81 Figure 2 Point cloud of the artefact and its context (view from Colosseum). By dividing and naming discrete subsystems, it is possible to identify the geometry of the parts to be restored.

and planning activity are focused on the site safety- In this case the task of modelling can be carried entities: this is where semantic enrichment can sup- out with basic geometries, without representing in- port the designers’ task. depth details, as what is needed by the system for au- tomatic risk assessment, is the semantic information - those that go beyond geometry - connected to the Figure 3 fields (pure boundary boxes) that compose the afore- Worksite Layout mentioned spaces (Figure 4). allows to keep Each scheduled activity requires the presence of under control the a finite number of Actors in a given operational area interferences of the Space System for a planned Time lapse. between the main Generally, semantic information is collected by two companies synthetic indicators, intended as function of param- (OG2/Builders and eters concerning data collected by time schedule of OS2A/Restorers) work activities. and define the logistic What involves modelling in a BIM environment, on We worked on extending the Revit model/- organisation for the the other hand, are the Space System families relat- database for the project, by enriching the HS Space resting areas. ing to the main arrangements of the construction site families with an appropriate slot (linking to activity and, in particular, its safety (Figure 3). plan) regarding the number of workers inherited by the MS Project schedule. In the case of study, Actors class is made of dif- Figure 4 ferent instances of workers: Worksite Layout allows to keep • OG2, intended as the category of workers spe- under control the cialised in building activities and structural con- interferences solidation of the historical artefact between the main • OS2A, intended as the category of workers two companies specialised in architectonic and archaeological (OG2/Builders and restoration OS2A/Restorers) • Scaffolders, involved in mounting, customising, and define the logistic organisation for the 82 | eCAADe 39 - Building Information Modelling - Volume 2 resting areas. and dismounting temporary scaffolding to allow of the workers in groups of maximum n.3 worker- OG2 and OS2A to work in comfortable and safe s/each. conditions Other useful strategy to contrast COVID-19 • Technicians and external visitors, involved in spread has been to provide personal tools and per- coordinating, supervising and documenting sonal protective equipment for each worker. To each project site activities worker of each group we assigned tools and personal equipment for protection (Figure 5). At the same time, HS Space System, is made of differ- Figure 5 ent instances: Strategic rule: to • Operational Spaces: each worker of each • Transit Spaces, to move from one Space to an- group we assigned other, including horizontal distribution path and tools, personal vertical connections equipment for • Storage Spaces, according to different classes of protection, works and workers dedicate WC, etc. • HS Logistic Standard Spaces such as canteen, changing rooms, resting area, washing rooms, bathrooms, etc. according to different Actors. Since requirements engineering precedes site de- sign, it provides a convenient starting point for the identification of potential safety concerns of a sys- tem if the knowledge contained in requirement doc- uments can be extracted and used as the initial basis for Safety Analysis (O. Daramola et al.). In this case of study HS rules - or “security condi- tions” - have been imposed on HS space entities or their groups (quantitative, relational, etc.): the sys- tem acts by checking formalized parameters. Rules can be defined as formally engineered re- quirements: The enriched model can process all the components of the construction site layout, verifying the level of Covid-19 risk linked to the specific solu- tion hypothesized for its organization. To verifying the level of Covid-19 risk linked to the specific solution hypothesized for its organiza- tion we implemented some specific rules, accord- ing to the general criterium of preventing the Actors gathering in the same HS Space entity. First set of general rules regards Actors: re- quirements have been defined to divide actors in Other general rules linked groups to Logistic HS sub-groups belonging to main contractor companies Space entities. For instance, a precaution was the (OG2; OS2A; scaffolders, etc.). The most effective installation of an appropriate number of removable strategy to contrast covid-19 spread was the division site-WC. Specifically, was installed n.1 WC each n.5

Building Information Modelling - Volume 2 - eCAADe 39 | 83 workers, sanitizing it after each use. sis. Other general rules assigned general HS Logis- This paper discusses a method for strategically tic spaces for resting activities during a specific time matching HS program elements to construction pro- lapse of the working hours for preventing overlap- cess requirements, engineered by means of explicit ping in usage of the same services. The working relations among actors, activities, and spaces. groups’ break were staggered every 15 minutes to In this case of study HS rules - “security con- prevent any gathering. ditions” - have been imposed on HS space entities A more specific set of rules regarded the subdivi- (quantitative, relational, etc.): the system acts by sion of Space entities (Operational, Transit and Stor- checking formalized parameters and evaluate the risk age systems) for constraining activity during a spe- assessment plan by means of an inferential engine cific time lapse for each specific sub-group of work- implemented on top of commercially available BIM ers. For instance, the scaffolding spaces were divided platform. into multiple basic elements. Each HS Space instance Linking IFC database to OWL ontology knowl- was assigned to a dedicated workgroup of 3 people edge base, allows goal oriented automatic reasoning maximum, operating at a safe interpersonal distance for supporting design, planning and management of (Figure 6). construction site, reducing personnel safety risks as a Other rules regarded the differentiation of the main interest. vertical connections at the various level of the scaf- The results indicate that the approach on one folding. The system checked the adequate num- side enables reuse of formalised experience knowl- ber of scaffolding stairs to move from one level to edge in conducting safety analysis and, on the other, another, preventing the Actors transit interferences provides a comprehensive basis for early identifica- (Figure 7). tion of hazards when used with an appropriate do- Other rules have been implemented to reduce main ontology. workers movements to access water. Avoiding a cen- The experiment, at conceptual level demon- tral dispenser, an adequate number of water access strates that the proposed system is potentially suit- point has been defined to prevent the chance of in- able for further and more automated application in terfering with other Actors. practice by experts. At the moment, in the real site, the aforemen- To verifying the level of Covid-19 risk linked to tioned rules demonstrated its effectiveness when the specific solution modelled for site organization one of the OS2A workers resulted positive to Covid- we implemented some specific rules according to the 19 test and the Activities continued as planned, ex- general criterium of preventing the Actors gathering cluding only the actors belonging to his own specific in the same HS Space entity. sub-group, that have been preventively isolated for At the moment, in the real site, the aforemen- 10 days. tioned rules demonstrated its effectiveness when one of the OS2A workers resulted positive to Covid- CONCLUSIONS AND FURTHER RESEARCH 19 test and the Activities continued as planned, ex- As long as Safety Analysis procedures, such as hazard cluding only the actors belonging to his own specific and operability analysis, are generally regarded as sub-group, that have been preventively isolated for repetitious, time consuming, costly and require a lot 10 days. of human involvement, BIM technology can provide At the level of enhancement of automation in the an efficient platform for both, systematic risk man- existing tools, the proposed approach demonstrated agement tool and as a core data generator to allow its potentials in the evaluation and exploitation of other BIM-based tools to carry out further risk analy- knowledge contained in the project documentation

84 | eCAADe 39 - Building Information Modelling - Volume 2 Figure 6 Scaffoldings were divided into multiple basic Operational Spaces. Each HS Space instance was assigned to a dedicated workgroup of 3 people maximum, operating at a safe interpersonal distance.

Figure 7 The adequate number of scaffolding secondary stairs were defined for preventing the Actors transit interferences when moving from one level to another.

Building Information Modelling - Volume 2 - eCAADe 39 | 85 (schedule, site layout) for early identification of po- Hazard Identification and Safety Analysis’, in Maalej, tential risks. W and Thurimella, AK (eds) 2013, Managing Require- According to context dependent factors, the cu- ments Knowledge, Springer, Verlag Berlin Heidelberg Dossick, CS and Neff, G 2011, ’Messy talk and clean mulative risk for a construction process can be as- technology: communication, problem-solving and sessed, and HS program elements may be ranked and collaboration using Building Information Modelling’, selected based on their ability to mitigate the risk. Engineering Project Organisation Journal, 1, pp. 83- Further research work is generically oriented the 93 following goals: Eastman, C, Lee, J and Jeong, Y 2009, ’Automatic rule- - To plan and model proactively the sequences based checking of building designs’, Automation in Construction, 18, p. 1011–1033 of worksite tasks together with the needed safety ar- Gambatese, J, Behm, M and Hinze, J 2005, ’Viability of rangements and utilities; designgin for construction worker safety’, Journal of - To ensure that all constructions can be built Construction Engineering and Management, 131(9), without any safety threat and necessary joints exists p. 1029–1036 for fixing the safety utilities; Grilo, A and Jardim-Goncalves, R 2010, ’Value propo- - To document planned safety solutions in detail sition on interoperability of BIM and collaborative working environments’, Automation in Construction, and self-explaining way, and, this information to be 19, pp. 522-530 conveyed to Actors involved. Kiviniemi, M, Sulankivi, K, Kähkönen, K, Mäkelä, T and Merivirta, ML 2011, BIMBased Safety Management ACKNOWLEDGEMENTS and Communication for Building Construction, VTT, Finland The research has been partially funded by Sapienza, Swuste, P,Frijters, A and Guldenmund, F 2012, ’Is it possi- Athenaeum Fund 2018 project ”New technologies for ble to influence safety in the building sector? A liter- management-design works on reusing historic real ature review from 1980 until the present’, Safety Sci- estate. Case study on historical or modern build- ence, 50(5), p. 1 ings: Building definition by BIM, building reuse, stone Trento, A, Fioravanti, A and Rossini, F 2014 ’Health and component conservation, construction site manage- Safety Design by means of a Systemic Approach - Linking Construction Entities and Activities for Haz- ment. The case of study has been possible thanks ard Prevention’, Proceedings of the 32nd eCAADe Con- to FENDI technical sponsorship in agreement with ference, Newcastle upon Tyne, England, UK, pp. 633- ”Parco Archeologico del Colosseo”,in Rome (Italy). 642 Zhang, S, Teizer, J, Lee, J, Eastman, C and Venugopal, M 2013, ’Building information modeling (BIM) and REFERENCES safety: Automatic safety checking of construction Aulin, R and Capone, P 2010 ’The role of Health and models and schedules’, Automation in Construction, Safety coordinator in Sweden and Italy Construction 29, pp. 183-195 industry’, 18th CIB World Building Congress, W099— Zou, Y, Kiviniemi, A and Jones, SW 2017, ’A review of risk Safety and Health in Construction, The Lowry, Salford, management through BIM and BIM-related tech- pp. 93-96 nologies’, Safety Science, 97, pp. 88-98 Borrmann, A, Hyvärinen, J and Rank, E 2009 ’Spatial constraints in collaborative design processes’, Pro- ceedings of the International Conference on Intelligent Computing in Engineering (ICE09), Berlin, pp. 1-8 Carrara, G, Fioravanti, A, Loffreda, G and Trento, A 2015, Knowledge collaboration design: Theory tech- niques and applications for collaboration in architec- ture, Gangemi Editori, Rome Daramola, O, Stalhane, T, Omoronyia, I and Sindre, G 2013, ’Using Ontologies and Machine Learning for

86 | eCAADe 39 - Building Information Modelling - Volume 2 CAAD and education 88 | eCAADe 39 - CAAD and education - Volume 2 A Hard Road To Travel Developing tools for low-cost Virtual Reality (VR) systems in the early design phases

Wolfgang Dokonal1, Patrick Scheich2, Tim-Jonathan Huyeng3, Uwe Rüppel4 1Graz University of Technology 2,3,4Darmstadt University of Technology [email protected] 2,3,4{scheich|huyeng|rueppel}@iib.tu-darmstadt.de

This paper updates on the developments in the use of low-cost Virtual Reality devices for Architectural design. The authors established a workflow using the gaming software ``Unity'' to prepare geometry for virtual environments and developed interfaces for a more natural movement inside the virtual world. We will give a summary on the old experiments and explain about our latest develoments in refining the workflow and the interfaces within a new setting. Architects teamed up with civil engineers with a focus on computer science. Due to new hardware developments, we could change the systems from wired to wireless and added several funktions within the overall aim to keep it simple and affordable.

Keywords: Virtual Reality, Head Mounted Displays, Low-Cost Interfaces, Google Cardboard, Microcontroller

SETTING THE SCENE this field either in academia or in practice without the Designing architecture is a complicated and de- help of computers. But what we also experienced manding task but when done successfully a very re- with our students was that the change from analogue warding one. One of the key aspects that divides to digital tools somehow reduced their understand- great design from just functional design is the spatial ing of space and scale which is the opposite of what aspect. Young students in architecture tend to start we expected from these new digital tools. their design idea in floorplans to fulfil a certain brief. Until today Virtual Reality (VR) had no real impact Most of them don’t really think about creating “great in architecture into the design process of the average spaces”. architect or civil engineer because of its cost and nec- Being involved since the beginning of the intro- essary efforts. duction of the computer into architecture in bring- So, although certain VR tools are already avail- ing digital tools into architectural education we were able for a long time they were not included in the convinced that the computer would be a beneficial average architect’s or civil engineer’s toolbox - it was tool in the whole process. And we succeeded after only used by some very big offices as a costly presen- many years so it is unthinkable nowadays to work in tation tool. The Development of the new low-cost

CAAD and education - Volume 2 - eCAADe 39 | 89 and ultra low-cost VR Systems using Head Mounted turned out to be very fruitful for both sides. We will Displays (HMD) for smartphones gave us the idea report on the results of two joint workshops within for a new start to use them in the architectural de- this new setting. We were able to refine our work- sign process. From our point of view this develop- flow and our interfaces. Because of new advances in ment is a game changer and would be the chance to hard- and software we were also able to change our bring back the somehow lost understanding of space old concepts for the interfaces - we went from wired and scale to our students. This paper gives a short to wireless. summary about our past efforts to use these devices It was striking to see that the tool we developed in the architectural design process and establish an for walking in the virtual world by walking in the real easy-to-use workflow from the computer models into world instead of pressing buttons made a big differ- the VR environment. We then concentrate on pre- ence in experiencing the design. We even developed senting the new developments of refinement of our a bicycle interface to cycle in the virtual world and self-developed low-cost interfaces were able to present our interfaces at several confer- ences. INTRODUCTION For several years all our research in this field was After first experiments with the smartphone-based done as a joint effort of two schools of architecture, systems, we quickly found out that it is essential to one in Austria and one in England. We organized have an easy-to-use workflow (most students of ar- several workshops together where students of ar- chitecture are not really into scripting and program- chitecture tested our workflow and the interfaces, ming) and just the right amount of information in the we had prepared for them. They had very different Computer Models - enough to understand the qual- backgrounds most of them with very little program- ity of the space and make decisions based on that but ming and scripting experience. On the one hand not too much to bring the hardware to its limits. that was quite ok because that is the reality for most The workflow we established was based on of our architectural students - most of the students bringing the initial geometry into the Unity environ- don’t study architecture because they are very good ment and then create apps for the smartphones to in mathematics or programming. But to further de- be used in the VR environment. Unity is well known velop the workflow and the tools it was not always in our field so we don’t have to go into details here. easy to work with them in terms of solving problems Additionally, we developed several interfaces for the in theses aspects HMDs to increase the level of immersion which is cru- In fact, with this simple system, we managed to cial to understand design decisions even for experts. create a tracking system without limits - the real lim- its were only the physical world. And it was also quite surprising to see that this simple device helped a lot THE NEW SETTING in creating a much more natural movement in the vir- The original research started as a collaboration be- tual world. We even extended the idea towards cy- tween two schools af Architecture in Austria and cling in the virtual world but failed to make it possible the UK. Brexit and the retirement of our former En- to use our wired system seamlessly between walking glish partners from the University of Liverpool made and cycling as we originally had intended. changes necessary. New funding possibilities for new collaborations gave us new opportunities. We were able to change the setting and teamed up with THE PREVIOUS CONCEPTS a German university. This time we teamed up our stu- Our original concept on the workflow and the in- dents of architecture with a group of students in civil terfaces have been presented at several conferences engineering with a focus on computer science. This and can be checked at the following Cumincad links

90 | eCAADe 39 - CAAD and education - Volume 2 [9]. To summarize, we used Unity as the connecting towards cycling in the virtual world but failed to make software to bring the geometry of the design into the it possible to use our wired system seamlessly be- Virtual environment. Through Unity we could gen- tween walking and cycling as we originally had in- erate APK files that could then be copied onto the tended. smartphones and be used for google cardboard as the ultra-low-cost HMD device [1]. THE NEW CONCEPT We also developed interfaces for cardboard that The New Workflow changed the method of triggering movement within Our new setting in teaming up with the more expe- the system from pressing buttons with your hands rienced programmers allowed us to test a new work- raised to your head to a much more natural move- flow and work on new interfaces. A big improvement ment where every step in the real world equaled a was to replace the concept of using unity to create step in the virtual world. the APKs for Android phones into a much more flexi- Figure 1 ble concept. The previous This was possible due to the availability of a new systems with open-source project called WebXR from Mozilla [2]. microswitches and This software can be used together with tailored ex- wires (Photo aaa) tensions of Unity e.g., the WebXR exporter [3]. So now instead of producing APKs for only one special- ized platform we are able to make the virtual world and certain interactions that have been created in Unity available on a Web-based platform. There are several important aspects to this change: First - the system is now browser based and therefore platform independent. In the old system we had to use Android phones and google card- board. In fact, the new system can even be used with- out our specialized VR equipment with a variety of different methods. For example, you can also just use a 3D-viewer on the PC. Naturally you lose a lot of im- mersion if you work like that so it does not sound very sensible to do so. But for a check in the middle of a design process it is just a simple and quick way to ex- perience the virtual world. For more detailed checks We mounted microswitches on the soles of shoes and on the design, we would always recommend using wired it to a solenoid mounted on google cardboard the HMDs and our interfaces. More expensive but still to transfer these impulses into the virtual world (see affordable VR-Hardware (eg. Okulus Rift or HTC Vive) figure 1). In fact, with this simple system, we man- is also supported by WebXR. aged to create a tracking system without limits - the Second - It is important that the via WebXR gen- real limits were only the physical world. And it was erated website can be put on a local or global web- also quite surprising to see that this simple device server. We used NGINX webserver and a simple helped a lot in creating a much more natural move- serverside php-upload script. This allows on the one ment in the virtual world. We even extended the idea hand the collaboration between teams working on a

CAAD and education - Volume 2 - eCAADe 39 | 91 design - even when they are working in different loca- is very popular in the private maker scene, vari- tions. This is an important aspect - especially in times ous libraries and tutorials are provided. This low- of a pandemic. On the other hand, access to this web- ers the threshold for students to start in this area server can be limited to the designers only so there is of engineering. To implement the functionalities of a certain guarantee that those designs stay “private” Bluetooth devices we decided to use the ESP-BLE- - which is important for example when you do a com- Keyboard [5] l and ESP-BLE-Gamepad [6] library from petition. Github. A big advantage of the ESP is also that it is possible to connect nearly any input source. In our use-case we mostly worked with small sensors or but- Figure 2 tons which were connected to the respective pins. WEBXR Workflow We replaced the microswitches with a new sen- (Scheme From xxx) sor: a three-axis accelerometer. To make use of the This new workflow changed a lot and made things data the sensor created we had to use a small single easier - especially the huge problem we experienced board microcontroller like Arduino or Raspberry Pi. in our previous workshops with different smart- It turned out that for our task the ESP32 board with phones, different operating systems or versions of it inbuild Wi-fi And Bluetooth connectivity at a cost of and constant updates. Also we are now able to use around 10€ was sufficient. This enabled us to keep BIM-Models and add interactions and more graphi- the hardware costs of the new “Movement Box” close cal information to the models in Unity (see figure 2). to 20€. BIM-Models can be saved as IFC-file and converted The new “Movement Box” consists of the follow- to an .obj file using IfcOpenShell. We imported this ing parts: file to Unity. In the game engine, we added features • 1x ADXL335 module (three-axis accelerometer) such as physics, sound, textures, lighting, animations • 1x Lithium Polymer Battery 3,7V 1200mAh and interactions to make our BIM-models walkable • 1x ESP32-bit Dev Board Wi-Fi + Bluetooth and more real. So, after this we could work on the • 5x Push Button (4 green, 1 red) improvement of our interfaces. • 1x RGB-LED 1x 3D-printed Case • 1x Velcro strip for mounting The New Interface The old concept of the microswitches wired to the solenoid worked quite well in showing the benefits Figure 3 of a more natural way to trigger movements in the The Main virtual world. The concept of “real” steps triggering Components of the virtual steps turned out to be very important. But Movement Box “wiring” people was rather tedious and time consum- ing even after we changed from mounting it individ- ually towards pre-mounted foam clogs. At that time Bluetooth connections which was a rather obvious possibility were not stable enough and had to much input lag. Recent hard- and software developments made it possible to give it a new try - still under the aspect of affordability and simplicity. The ESP32 is programmable and compatible with the Arduino platform. For this platform, which

92 | eCAADe 39 - CAAD and education - Volume 2 Figure 4 preted as a step and the virtual button is activated. As Electronic schema mentioned above a big advantage of the new “Move- of the Movement ment Box” is the possibility to add additional sensors Box for new functions (see figure 5).

The Workshops After developing the new workflow and the new in- terfaces we planned to test our setting in two work- shops - one in each of the participating universities. The “Movement Box” simulates a normal gamepad But then we all got hit by Covid-19 and this pandemic or keyboard like it is used for PCs or Game devices also had an impact our virtual world. We can only test and is connected via Bluetooth LE to the smartphone our interfaces to the virtual world in a real world phys- (see figures 3,4). The buttons can be programmed ical setting, although this might sound strange. individually which allows for different functions and Because it was not possible to work with our stu- possible extensions. The status of the on/off but- dents face to face, we had to switch to Virtual Work- ton and the established Bluetooth connection is visi- shops. That was not a big problem in further devel- ble through the LED. One step in the real world trig- oping our digital workflow from the geometry to the gers the accelerometer and is interpreted as a “vir- VR ready applications for the smartphone. But for the tual” press on the button. testing of the new interfaces, we had to rely on our- selves until now and could not really test them with Figure 5 the students. The new box mounted with velcro CONCLUSION Our new setting is very promising in both fields - the workflow and the interfaces. It has many advantages over the previous systems but still keeps the main idea of being rather simple, easy to use and low cost. It really is a promising effort to make VR a valuable tool in every architect’s toolbox. It has the poten- tial to give designers better information about the spaces they design even in an early design phase. We can now start testing the systems with differ- ent levels of detail and interaction to see what is nec- essary for designers to have the best breaking even point in using the system. How much time is neces- sary to invest in this tool to get out the most valuable information to bring the design forward. The plan is Every 50 milliseconds the sensor checks the differ- that even with average CAD knowledge every archi- ence of movement between the gravitational accel- tect should be able to master our workflow and make eration of 9,81 m/s² which is existing due to the use of the system in a beneficial way. movement of the earth when the sensor itself is not Hopefully this can happen this summer and we moving. If the gauge is different from the gravita- can include some of results in the presentation at tional acceleration five times in a row this is inter- the conference. We improved the workflow during

CAAD and education - Volume 2 - eCAADe 39 | 93 our virtual Workshop in April and each student iden- ple and affordable. Because of that we plan to limit it tified special design problems within their individ- for now to those four buttons of interactions. In fact ual projects - from moving walls, Changing block they could trigger different events according to the heights, georeferencing the site and other interac- needs of the designers. tions.We have scheduled another virtual workshop Although the road was pretty hard to travel and for testing again with new students in June. And we some rough parts might still lay ahead, we are con- hope that face to face working with our students in vinced that we are not on the road to nowhere - a workshop in Mid July to test the interfaces will be rather the opposite! possible due to the improvements with the Covid Sit- uation. We will be able to present the results of these REFERENCES workshops at the conference. Alvarado, R.G. and Maver, T. 1999 ’Virtual Reality in Ar- chitectural Education: Defining Possibilities’, ACADIA OUTLOOK AND FUTURE WORK Quarterly, vol. 18, no. 4, pp. 7-9 Dokonal, W., Knight, M. and Dengg, E. 2015 ’VR or Not VR We have many ideas to improve the interfaces and - No Longer a Question?’, 34th eCAADe Conference - the workflow even more. The first addition to the Volume 2, University of Oulu, Oulu, pp. 573-579 movement box that we implement already this year Drosdol, J., Kieferle, J. and Wössner, U. 2003 ’The In- is a distance sensor. This will help to not collide with tegration of Virtual Reality (VR) into the Architec- the real world when you move in the virtual world - tural Workflow’, 21th eCAADe Conference Proceed- which in our system to a certain extent also means ings, Graz, pp. 25-28 Kreutzberg, A. 2015 ’Conveying Architectural Form and movement in the real world. We will test it in our July Space with Virtual Reality’, Proceedings of the 33rd workshop with the students and report at the confer- eCAADe Conference - Volume 1,, Vienna University of ence. Technology, pp. 117-124 We also plan to test our “Movement box” with a Sheehan, L.J., Brown, A., Schnabel, M.A. and Moleta, T. new version of our bike interface. The problem with 2021 ’The Fourth Virtual Dimension - Stimulating the previous version of the bike interface was that the Human Senses to Create Virtual Atmospheric Qualities’, 26th CAADRIA Conference - Volume 2, The movement with the bike was too quick for the smart- Chinese University of Hong Kong and Online, Hong phones. Therefore, it is necessary to have different Kong, Hong Kong, pp. 213-222 step sizes in a model used for walking and one used Sorguç, A.G., Yemişcioğlu, M.K., Özgenel, C.F., Katipoğlu, for cycling. This can now be changed by pressing a M.O. and Rasulzade, R. 2017 ’The Role of VR as a New button on the “Movement box”. Game Changer in Computational Design Education’, With the ability to connect up to seven devices 35th eCAADe Conference - Volume 1, Sapienza Univer- sity of Rome, Rome, pp. 401-408 to the smartphone via Bluetooth, we can also track [1] https://developers.google.com/cardboard/develop the movements of the bike as an extra device. The [2] https://mixedreality.mozilla.org/hello-webxr/ same technology that is used for cyclocomputers can [3] https://assetstore.unity.com/packages/tools/integra be used for this purpose. A magnet is attached to tion/webxr-exporter-109152 the spokes and a hall sensor or a reed contact detects [4] https://github.com/MozillaReality/unity-webxr-exp each turn of the wheel. The speed of the bicycle can ort [5] https://github.com/T-vK/ESP32-BLE-Keyboard then be calculated by an other ESP32 and send to the [6] https://github.com/lemmingDev/ESP32-BLE-Game smartphone. pad Additional changes in the virtual world from [7] https://www.bluetooth.com/specifications/specs/h lighting to changing textures and scales by using uman-interface-device-profile-1-1-1/ these triggers can now be implemented. The key is- [8] http://papers.cumincad.org/cgi-bin/works/Search?s sue will be to not overdo it and keep the system sim- earch=dokonal&x=32&y=17

94 | eCAADe 39 - CAAD and education - Volume 2 Augmented Quarantine

An experiment in online teaching using augmented reality for customized design interventions

Renate Weissenböck1 1Graz University of Technology [email protected]

This paper presents experimental research about using Augmented Reality (AR) for interactive design processes, exploring a spatial "live" design method taking place in an overlay of real space and digital models. It discusses the processes and outcomes of a seminar undertaken at Graz University of Technology in winter term 2020/2021. Due to the Covid-19 pandemic, the course was taught online, and conceptualized to allow students the biggest possible learning experience during the lockdown. Ensuring accessibility to all participants, the seminar was based on the use of ubiquitous devices. The implementation of newly developed software, such as "Fologram", enabled the students to use AR systems at home with their personal computers and smartphones. The task of the course was to design customized interventions for the students' own domestic spaces, reacting to changing conditions and needs during the lockdown. The employed workflow was driven by an instant connection between 3D-modeling (Rhinoceros3D), parametric design (Grasshopper) and holographic immersion (Fologram).

Keywords: augmented reality, remote collaboration, interactive design, customization, online teaching

INTRODUCTION a case study to gain more insight on the potentials This study is part of an ongoing research about the and constraints of employing AR in architecture edu- influence that tools and media have on architecture, cation. Additionally, this course is used as an example in particular the impact that new technologies have to reflect on a new mode of online teaching. on architectural design. In this paper, the incorpora- tion of Augmented Reality (AR) technologies in inter- AR IN ARCHITECTURE active design processes is explored, capitalizing on In contrast to Virtual Reality (VR), which takes place the interplay between digital media, human intuition in purely virtual scenes, AR allows us to overlay vir- and immersive spatiality. An experimental student tual objects with our real-world environment. Re- seminar, which was taught online during the lock- cently, Augmented Reality technologies and devices down situation of the Covid-19 pandemic, serves as have significantly developed (Wang, Ong and Nee

CAAD and education - Volume 2 - eCAADe 39 | 95 2016) and are widely available, e.g. in the form of AR recent years, some architecture schools, such as SCI- apps for mobile devices (Abboud 2014). Therefore, a Arc [3] and Graz University of Technology (Grasser new interest of architects in the use of AR can be wit- 2019), introduced AR technologies into their teach- nessed. ing. Architect Greg Lynn is one of the first architects One of few online courses employing AR is the using AR in the architectural design process (Frearson workshop “Secret Whispers & Transmogrifications” 2016). He places holographic versions of digital mod- by Garvin Goepel and Kristof Crolla (Goepel and els in existing environments and manipulates them Crolla 2021), taught online as part of last year’s “Dig- in relation to the context (Lynch 2016). Apart from ital futures world conference”. In this workshop, par- design, AR is also employed for construction. Other ticipants used their smartphone as AR device to mold architects, such as Jahn and Retsin, use immersive a digitally given object from clay, capitalizing on the and augmented systems for the manual assembly of factors of human imprecision as creative quality. In complex structures, guided by holographic instruc- this course, AR was used for fabrication, but it turned tions on the site (Jahn, Newnham and Beanland 2018; out to be very difficult in practice to build some- Retsin and Carpo 2019). thing with the smartphone. The reason for this is that In the projects of Lynn, Jahn and Retsin, the Mi- the respective person holds the smartphone in their crosoft HoloLens [1] is used as AR device. Unfortu- hands and therefore no longer has their hands free to nately, the HoloLens and similar devices are still quite manufacture. When the smartphone is attached to a expensive and therefore not available to generic tripod, the hands are free, but the viewing angle of users. Alternatively, ubiquitous mobile devices such the model cannot be changed - which is crucial when as smartphones can be used as AR tools, when working on three-dimensional objects. In order to equipped with specific apps (although of course less avoid such problems, the author decided to focus her precise and immersive). Since 2017, AR kits for course on spatiality and digital design explorations in Smartphones have been launched. Developers of AR and to exclude the actual production. AR software for architects reacted to these oppor- tunities and extended their software by applications SEMINAR: “AUGMENTED QUARANTINE” for mobile devices. One example of these is “Folo- The seminar “Augmented Quarantine” was an archi- gram” [2], developed BY architects FOR architects, tectural design workshop taught by the author in which can be run on smartphones as well as on winter term 2020/2021. The first challenge was to re- HoloLenses. Fologram works with the 3D-modeling act to the format of online teaching in a lockdown sit- software Rhinoceros3D and its integrated graphical uation, establishing the course topic and framework algorithm editor Grasshopper, which are very com- in a way that would allow the students the best pos- mon tools in contemporary architecture, and can sible learning experience. This led to three premises: therefore easily be integrated into established work- the course needed to work with software and hard- flows. ware available to all students; the task had to be re- alizable by a single person or by online cooperation; AR IN ARCHITECTURE EDUCATION and the topic should be relevant to the current lock- In 2016, King et al. coined the term ”Augmented Age”, down situation. disrupted by the internet and smartphones, leading Task. The task for the students was to use AR to de- to a change in the way we live, work and interact sign a customized intervention for their own domes- (King et al. 2016). These advancements will certainly tic space, reacting to changing conditions and needs shape architects’ minds in the future (Kaku 2014) and during a lockdown situation. The employed work- should therefore be addressed in their education. In flow was driven by an instant connection between

96 | eCAADe 39 - CAAD and education - Volume 2 3D-modeling (Rhinoceros 3D) [4], parametric design were asked to install and test the needed software (Grasshopper - Algorithmic Modeling for Rhino) [5] and hardware before the first class. The course time and holographic immersion (Fologram). was split between input lectures, software tutorials, The students experimented with a spatial “live” group discussions, group work sessions, project tu- design method, in the superimposed space of torial sessions, and project design reviews. real environment and digital model. Using Folo- In the morning of the first day, the students re- gram on their smartphone, in synchronization with ceived introductory lectures about AR and its appli- Rhinoceros3D and Grasshopper on their computer, cation in architecture, as well as information about the students were able to conceive - and iteratively course topic and structure. After team building and adjust - their designs in Augmented Reality. brainstorming in groups, the first afternoon as well as A set of conceptual base geometries with flexible the second morning were dedicated to technical pro- parameters was developed, creating a digital model ficiency lessons. These included tutorials of the Folo- that was virtually placed into the existing environ- gram app on the smartphone as well as on the com- ment. Once established, the model could be inter- puter, the exploration of example files in Rhinoceros actively adapted and precisely tailored to students’ and Grasshopper, and explanation about how to syn- living spaces - in real-time and in 1:1 scale. chronize models and parameters between the de- vices. It was also demonstrated how to position dig- Figure 1 ital objects in real space, using physical QR-codes, or Team meeting in “ArUco” markers. “WebEx”,working in In the following days, the students work pro- Rhinoceros3D (left) cess was accompanied by individual technical sup- and Grasshopper port and by regular design evaluation sessions within (right) each group. Furthermore, time slots were set for ex- changing project information between the groups. At the end of each day, the student were asked to document their process on the course blog. Friday was dedicated to the final review and extensive dis- Participants and background. The course was cussions with invited guest critics. taken by 17 students, consisting of 14 bachelor stu- Work process. The 17 participants formed 5 groups dents from our university (in the 5. semester), 2 of 2 to 4 students. At the beginning, each group master students (who transferred from other uni- developed a common design concept (geometric or versity), and 1 incoming bachelor student. 14 stu- functional) and subsequently a common 3D-model dents had previous experience with 3D-modeling in with a Grasshopper definition, establishing the vari- Rhinoceros3D, but only 4 had intermediate to ad- able parameters. vanced experience with Grasshopper. The 2 master Then, each participant within the group was students and the 1 incoming bachelor student had asked to adapt this design concept individually to some experience with 3D-modeling, but neither with their own space and needs, e.g. by moving “attrac- Rhinoceros3D nor with Grasshopper. Fologram was tor points”,sliders or curve points on the smartphone new to all 17 students, but 2 students had worked screen. Therefore, 2 to 4 variations of each idea were with AR technologies before. designed, leading to multiple sub-projects in each Structure of the course. The course time was lim- team. ited to one week, Monday through Friday the whole day. To make the most out of the time, the students

CAAD and education - Volume 2 - eCAADe 39 | 97 Figure 2 Project 1.1: “flow follows fologram”; design ideas (top), process of scanning ArUco markers on the wall with the smartphone app, and adapting parameters with sliders in the AR app, which are synchronized to the computer (Grasshopper window and Rhinoceros model window) (middle and bottom).

98 | eCAADe 39 - CAAD and education - Volume 2 Figure 3 The course was hosted on the platform “WebEx” Project 1.2: “fence (Figure 1). Besides meetings in the common follows fologram”; “room”, breakout-rooms were created for each team. concept sketch, Teachers could pass by each of the rooms to see placed ArUco the progress and answer questions. Students in markers, generated breakout-rooms could “request” the teachers to join planks and their their room if they needed assistance. adjustment (from By using screen mirroring tools (e.g. Reflec- top to bottom) tor3, ApowerMirror), the participants were able to see each other’s smartphone screen and to follow their colleagues’ interaction process live. Assessment. The criteria for the project evaluation were how well the group concept was adapted to the individual students’ specific environment, and how much design variety was achieved through these adaptations. Furthermore, it was assesed whether the interactive AR workflow was well engaged, e.g. regarding the choice of flexible parameters and the amount of interactivity during the design process. Another criterion was the actual significance of the design intervention for the student’s apartment, for example changing or improving the spatial quality or the functional use.

EXEMPLARY PROJECTS In this section, two selected projects and their sub- projects are illustrated.

Project 1: “form follows fologram” This group was interested in working with extruded shapes to generate room dividers, shelves or seating objects. The geometric concept was based on a pla- nar grid, that was placed either on the floor or the wall by using ArUco markers. In the Grasshopper definition, adjustable val- ues as well as randomized values were incorporated. These included number, size, proportions and rota- tion of rectangles, as well as height of extrusions. Project 1.1: “flow follows fologram”. “Hoarding needs” in the current pandemic inspired the student to design a multifunctional shelf, that also acts as a design object in her quite empty room: “build your own augmented reality shelf with fologram: use-

CAAD and education - Volume 2 - eCAADe 39 | 99 able for cat, booze, toilet paper” (quoted from stu- Figure 4 dent presentation). The student placed a series of Project 1.3: “frame ArUco markers on the room’s wall, which she sub- follows fologram”; sequently scanned with the smartphone. In paral- concept sketch and lel, the script generated a curve between the markers variations of the and distributed square extrusions along this curve. project in the Size, density and depth of the objects were interac- student’s living tively adjustable (Figure 2). room Project 1.2: “fence follows fologram”. Adjusting to the need of working from home, the student aimed to design a room divider to separate his work space from the kitchen. The base geometry of the interven- tion was defined by the position of multiple ArUco markers on the floor. A polyline was drawn through the scanned points, along which rectangles were distributed and extruded, creating the design of a “fence”. The variable parameters included height of the extrusions, as well as the thickness and the rota- tion of the planks (Figure 3). Project 1.3: “frame follows fologram”. This student created cubical frames for a wall shelf. The shelf con- sisted of closed cubes, usable as seat or table, as well as of open cubes, usable as storage for books or other objects. Amount and size of the cubes could interac- tively be adjusted during the process. By triggering a random command, the distribution and position of the cubes could be defined repeatedly, to explore dif- ferent geometric compositions in the room (Figure 4). Project 1.4: “fort follows fologram”. For the reno- vation of a home office, the student employed AR to design “a multipurpose furniture piece used for stor- age, as lounge and even as sound barrier in case the office is used as a recording studio” (quoted from stu- dent presentation). The geometry was defined by a square grid, which was extruded and edited by a random command in the Grasshopper definition. A variable space was created that could be adjusted by three different number sliders. Those regulated the maximum amount of blocks, the number of blocks that were randomly eliminated to create the pattern, as well as the height of the blocks (Figure 5).

100 | eCAADe 39 - CAAD and education - Volume 2 Figure 5 Project 2: “magic mushrooms” Project 1.4: “fort This project was inspired by the semicircular bay win- follows fologram”; dow in one of the students’ apartments, leading to adapting the design the exploration of revolving shapes. In their research, elements with the team discovered an installation by artist Carsten sliders, and concept Höller, where mushrooms ”grow” from the ceiling [6]. visualization Motivated by this work, the students developed the concept of a ”mushroom” usable in many ways, e.g. as lamp, table, chair, punch bag or doorknob, per- sonalized to everyone’s room. ”The form of the clas- sic mushroom is usable in so many ways, therefore we simply called it ”magic mushrooms” (no drugs in- volved) :)” (quoted from student presentation). The geometry was based on a parametric profile curve, whose shape and scale were adjustable. The posi- tion in space was defined by ArUco Markers, placed on floor, wall or ceiling. Project 2.1: “the oriel fungus”. Since the student’s apartment houses a round oriel, he conceived the mushroom-shape for different uses, in different po- sitions, orientations and scales, for example as lamp, table or round sitting opportunity. A revolving shape was defined by the profile cure, which was adaptable by moving curve points on the smartphone screen. Besides, the geometry’s size and proportions were adjustable by sliders (Figure 6). Project 2.2: “spinning mushroom”. In this project, the student created a floating chair structure with a 360 degree view around the room. The revolving structure is connected to the ceiling and consists of wooden louvres. Besides the parameters described before, the density and size of the louvres were also adjustable (Figure 7). Project 2.3: “twisted mushroom”. This student en- visioned to enhance her apartment’s atmosphere by designing light objects. As additional parameter, a twist command was added for the vertical base of the lamp, whose profile could be adjusted to multi- ple thicknesses. “The twisted mushroom lamp can be used as table lamp, ceiling lamp or floor lamp. You change the shape of the lamp and become more cre- ative just by twisting its body.” (quoted from student presentation) (Figure 8).

CAAD and education - Volume 2 - eCAADe 39 | 101 Figure 6 Project 2.1: “the oriel fungus”

FINDINGS AND CONCLUSIONS After a certain time of getting to know the pro- Closely witnessing the students‘ work as well as an- cess, the students actually “designed” directly in aug- alyzing the outcomes of the course, it can be con- mented reality - in the superimposed space of real cluded that the students’ design processes and their environment and digital model. This indicates that understanding of space benefitted from employing they got into the “flow zone” (Csikszentmihalyi 1990), AR technology, fostering their creativity as well as being immersed in the activity, plus being immersed initiating the re-consideration of architectural design into the three-dimensional design space. processes. Benefits of using the presented AR design The teaching method follows Kolb’s experiential method. Since the seminar had to be taught on- learning theory (Kolb 1983), gaining knowledge by line, the author aimed to compensate the lack of doing, by encouraging students to design with AR physical presence by employing alternative methods technology and to incorporate it into their design to visualize space. Immersive experiences enabled thinking. The course outcome shows that the stu- the students to (virtually) get in contact with spaces dents were able to employ the workflow and to come and materials. The employment of AR provided the up with creative design ideas. illusion of actual spatial objects. It aided real-time These observations correlate with the feedback evaluation and real-time modification of design pro- of the participants, stating that they could quickly ap- posals. ply the new input and got “the feeling that [they] are It could be observed that by moving around in taking the next step”, that they “already can start to their home space, the students changed their per- work with these tools!” (quote from student evalua- ception on the design, seeing it in real scale and tion). Figure 7 Project 2.2: “spinning mushroom”

102 | eCAADe 39 - CAAD and education - Volume 2 Figure 8 Project 2.3: “twisted mushroom”

in correlation to their apartment. This led them to space, matching existing elements with new ones, or change parameters, or even the design concept. designing compositions of multiple elements. Their The author would argue that the most successful knowledge about orienting and positioning design projects were those, where students didn’t take the models in a functional space, e.g. in regard to path- “first” result as final, but actually kept “designing” and ways in the apartment, was significantly enhanced. manipulating while being immersed in the overlaid Furthermore, human scale and movement could be AR scenario PLUS moving around in their space. This explored in reference to conceived design elements was especially the case in project 1, since the team (e.g. functions of seating, walking below elements...). had the best technical background. Frequently pro- The instant feedback provided by the employ- portions, scale and location were adapted, but also ment of AR encourages new design processes, since the density of elements.The team of project 2 had the adaptions and changes can be made quickly (com- least software skills, therefore they wanted to spend pared to using physical prototypes). Participating more time in exploring technical issues and had less in the seminar, the students were equipped with a time left for interactive design adaptations. Never- new set of tools as well as with another perspective theless, it is noticeable that all projects show a wide on design. Furthermore, it helped them to endure spectrum of design variations in the sub-projects, as the home lockdown, enabling them to develop con- well as a series of different user scenarios. cepts and visons for the transformation of their apart- Without using AR, it would have been harder for ments. the students to estimate the scale, proportions, etc. Remote collaboration. The experience of this re- and how the project fits in the existing environment. mote teaching class showed, that the communica- The students were able to transport their digital de- tion actually worked very well in the WebEx applica- signs out of the computer screen, and to interact tion. Since the whole meeting depends on the host’s and communicate with their physical environment. computer, technical reliability should be ensured. It They could visualize their 3D-models in 1: 1 scale is advisable to add a co-host to prevent the end of the and imagine the transformation of the spaces they in- meeting in case of network or computer problems. habit and are familiar with. Equally important are the organizational and Benefitting from immersive qualities, students moderating capabilities of the teachers, to structure got a better feeling for spatiality and location in class meetings as well as working phases. The collab-

CAAD and education - Volume 2 - eCAADe 39 | 103 oration within student teams was supported by pro- sign production.’, PROJECTIONS - Proceedings of the viding breakout rooms, as well as by screen sharing 26th CAADRIA Conference - Volume 2, Hong Kong and of the smartphones. Online, pp. 21-30 Grasser, A 2019 ’Towards an Architecture of Collabora- The use of the Fologram application also al- tive Objects’, Architecture in the Age of the 4th Indus- lowed remote students to explore the 3D-models of trial Revolution - Proceedings of the 37th ECAADe and other colleagues and place those in their own apart- 23rd SIGraDi Conference - Volume 1, Porto, Portugal, ment. Besides, it was possible to view and edit one pp. 325-332 model on multiple smartphones at the same time. Jahn, G, Newnham, C and Beanland, M 2018 ’Making in Future investigations suggest to develop a system Mixed Reality’, Proceedings of the 38th Annual Confer- ence of the Association for Computer Aided Design in with avatars of remote collaborators, which can be Architecture (ACADIA), pp. 88-97 ”beamed” them into other persons’ real space for Kaku, M 2014, The Future of the Mind: The Scientific Quest more immersive experiences. to Understand, Enhance, and Empower the Mind, Doubleday, New York King, B, Lark, A, Lightman, A and Rangaswami, JP ACKNOWLEDGEMENTS: 2016, Augmented: Life in the Smart Lane, Marshall • Course instructor: Renate Weissenböck Cavendish International, SG • Student assistant: Fabian Jäger Kolb, D A 1983, Experiential Learning: Experience as the Source of Learning and Development. 1 edition., En- • Guest critics: Urs Hirschberg (TU Graz IAM), glewood Cliffs, NJ: Prentice Hall Milena Stavric (TU Graz IAM), Gwyllim Jahn (Folo- Lynch, P 2016, ’Trimble’, ArchDaily [Online], Available gram), Ioanna Symeonidou (University of Thes- at: https://www.archdaily.com/798970/trimble- saly) sketchup-viewer-allows-you-to-manipulate- • Participating students: Bardhë Brestovci, Hen- hologram-models-in-the-real-world [Accessed: 30 rik Diekers, Paul Eichhorn, Fabian Fladischer, September 2020] Retsin, G and Carpo, M 2019, ’‘Real Virtuality’: Gilles Dorotea Josić, Yigit Kamiloglu, Branko Kuz- Retsin and Mario Carpo in conversation’, RA, Avail- manovic, Marco Linzer, Natalie Resch, Arber Rex- able at: https://www.royalacademy.org.uk/event/real- hepi, Mustafa Sari, Markus Schmoltner, Lukas virtuality-invisible-landscapes-gilles-retsin-mario- Schneeberger, Sandra Sulzer, Arlind Tahiri, Paul carpo-in-conversation [Accessed: 30 September Weberschläger, Anja Zamuda 2020] Wang, X, Ong, SK and Nee, AYC 2016, ’A comprehensive survey of augmented reality assembly research’, Ad- vances in Manufacturing, 4(1), pp. 1-22 REFERENCES [1] https://www.microsoft.com/en-us/hololens Abboud, R 2014, Architecture in an Age of Augmented [2] https://fologram.com/ Reality: Opportunities and Obstacles for Mobile AR in [3] https://community.fologram.com/t/sciarc-mastercla Design, Construction, and Post-Completion, no thesis ss-overview/484 type given, NAWIC IWD Scholarship Winner 2013 [4] https://www.rhino3d.com/ Csikszentmihalyi, M 1990, Flow: The Psychology of Opti- [5] https://www.grasshopper3d.com/ mal Experience., New York: Harper & Row [6] https://www.itsliquid.com/installations-carsten-holl Frearson, A 2016, ’Augmented reality will change the er.html/ way architects work says Greg Lynn’, Dezeen, Avail- able at: https://www.dezeen.com/2016/08/03/microsoft- hololens-greg-lynn-augmented-realityarchitecture- us-pavilion-venice-architecture-biennale-2016/ [Accessed: 13 January 2020] Goepel, G and Crolla, K 2021 ’Secret Whispers \& Trans- mogrifications:a case study in online teaching of Augmented Reality technology for collaborative de-

104 | eCAADe 39 - CAAD and education - Volume 2 Integrated Architectural and Environmental Performance- Driven Form-Finding A teaching case study in Montreal

Szende Szentesi-Nejur1, Francesco De Luca2, Andrei Nejur3 1Technical University of Cluj-Napoca 2Tallin University of Technology 3Université de Montréal [email protected] [email protected] 3andrei. [email protected]

The proposed paper presents the methodology and the outcomes of an intensive conception studio taught by the authors at the School of architecture of the University of Montreal having as objective the introduction of 3rd year architecture students to environmental evaluation and optimization techniques linked by the parametric design and the generative creation of architectural object. As opposed to mostly analysis-based approaches, an integration with architectural and urban design concepts was considered to be a more efficient method to initiate architecture students in environmental performance-driven design. The novelty of the course lays in the development of an integrative teaching method having as educational goals the development of environmental analysis skills, the creative use of digital tools, the conception of a coherent optimization process and the ability to represent a performance-driven design process.

Keywords: integrative teaching method, environmental design, performance-based design, parametric design, solar architecture, optimization

1. INTRODUCTION and orientation, the outdoor areas and the urban fab- The current trend of the building industry will lead to ric have a strong influence on the occupant’s visual a doubling of the global buildings floor area by 2060, and thermal comfort due to the diversity of solar radi- adding 230 billion of square meters of new construc- ation received by the building facades (Chatzipoulka tions [2]. A significant part of the new constructions et al. 2016, Kanters and Horvat 2012). Adequate sun will accommodate city dwellers who are expected to light access and daylight availability in buildings im- be 68 % of the global population by 2050 [5]. Humans prove also the physiological and psychological well- spend approximately 90 % of their time in indoor being of occupants and the architectural quality of places such as homes, workplaces, public places, and the interiors (Lockley 2009, Reinhart 2014). The build- schools [1]. The relation between the building form ing massing and layout together with the distance

CAAD and education - Volume 2 - eCAADe 39 | 105 and the density of the surrounding urban environ- ing performance simulations and to interpret the re- ment influencing the solar energy received by the sults, thus performance analysis is still mostly as- building envelopes are important factors which need signed to external consultants (Reinhart et al. 2012). to be taken into account in order to design low or The result of the separation of the architecture and zero-energy buildings and comfortable and safe out- the performance design phases does not allow to door spaces (Voll et al. 2016, De Luca et al. 2021a). fully exploit the potential of simulations in realizing Thus architects, planners and engineers are buildings with improved comfort and efficiency. urged to design buildings with adequate solar access, The main limitation in the actual teaching and view to the outside and controlled solar radiation inclusion of environmental simulations and building to provide occupants with comfortable and healthy performance analysis in the architecture curricula is indoor environments, to reduce resource depletion the lack of integration with studio courses. As a mat- and CO2 emissions in order to limit climate change. ter of fact, in the majority of the architecture schools, Since decades designers have the possibility to use simulations and performance are still taught in spe- computer tools in their design workflows (De Luca cific building engineering and technology courses. In and Nardini 2003). Studies investigating optimal ur- this way the students do not learn to integrate build- ban solutions proved that it is possible to design ing performance and architectural design (Charles buildings that enable the reduction of total energy and Thomas 2009). Additionally, in most classes, consumption with up to 50 % using environmen- simulations are performed by students on finished tal simulations (Kanters et al. 2014). Research work projects. As a consequence, they do not learn to fully showed that computer simulations are an increas- exploit the potential of performance simulations us- ingly used method for architects to improve daylight ing the results to guide design decisions at the be- in buildings (Reinhart and Fitz, 2006). Climatic anal- ginning of the design process, when they can have a ysis, simulation tools and new environmental design larger impact on occupant comfort and resource ef- workflows are developed at an increasing pace and ficiency. Existing studies showed that the students are available in popular digital and parametric design learn more easily and effectively the use of environ- software (Sadeghipour and Pak 2013, [3], De Luca et mental simulations if they can associate architectural al. 2021b). design tasks and performance targets (Ibarra and Digital tools for climatic analysis and computer Reinhart 2009). Recent investigations demonstrated simulations to evaluate the relation buildings and that students can perform accurately the simulations urban environments with environmental factors are and can correctly interpret the results when they in- gaining attention and their use is spreading in the tegrate the assessments with form generation prin- architecture curricula of all universities at a global ciples for the selection of optimal architectural de- scale. Studies showed that when simulation courses sign layouts (De Luca 2019). Similarly, a recent study are taught applying appropriate educational meth- showed that students can successfully design com- ods the students easily learn to perform the simula- fortable and efficient urban quarters through the cor- tions and interpret the results (Reinhart et al. 2015). rect utilization of daylight, energy simulations and However, the ability and proficiency of students at outdoor thermal comfort analysis when they develop the end of their educational path in applying cor- parametric urban models based on clear and sim- rectly climatic analysis and simulations in the design ple architectural concepts for the generation of a workflow is still insufficient, on average, to provide large number of alternatives among which they se- the project at hand with an adequate level of perfor- lect the ones with the best performance and architec- mance. This is confirmed by the still low capacity of tural quality (De Luca 2020). As a matter of fact, the architects and designers to perform correctly build- courses which integrate environmental simulations,

106 | eCAADe 39 - CAAD and education - Volume 2 Figure 1 Selected sites in downtown Montreal

building performance analysis and architectural de- optimization loops. sign are still rare, as a result so are the investigations Solar potential was defined as the possibility to on the course outcomes. use different active and passive solar technologies To contribute to the filling of this research gap, and daylighting determined by an evaluation of solar this paper presents the outcomes of the Atelier radiation and direct sunlight on the building façade Triptyque - Project n.2 Solar Geometries. Energy and roof through numerical simulations. simulation-driven high-rise design in urban context The goal of the studio was to make the students taught by the authors at the School of architecture of understand the importance of solar access of a build- the University of Montreal. The novelty of the course ing and its neighbours, including the public space as lays in the developed design and teaching method a main determinant of energy use, energy produc- which allowed the students to integrate architectural tions potential and wellbeing. Students were taught form studies and environmental analysis for solar ra- tools and techniques to maximize the solar access diation, natural light provision and view out, and to taking into consideration architectural and urban de- optimize these objectives to determine the proposed sign criteria. design solution. 2.1. Urban and architectural constraints 2. METHODOLOGY The building sites proposed for study were in close The studio was conceived as a 3-week practical intro- proximity to each other in Montreal city centre, hav- duction of 3rd year architecture students to solar en- ing similar size and development possibilities. As per vironmental evaluation and optimization in the ear- the authors interpretation of the Ville Marie urban liest stages of architectural design. The link between regulations [4], a buildable base volume was defined evaluation and optimization, as well as the basis of for each site, having certain setbacks from plot limits the whole studio work was in parametric design and and a maximal height. Besides these base volumes, generative creation of architectural object. each site had the possibility to develop a tower-like Students were required to produce a volumetric extra built volume on top of the base volume that design of a medium-high rise commercial building in also had specific setbacks from the base and a max- the dense urban context of downtown Montreal in 2 imal height. For site 1 the base volume height was phases. In the first phase students were required to limited to a minimum of 11 m and a maximum of 44 analyse 3 urban sites (Figure 1) from a solar-potential m, respecting a maximal density coefficient of 6 and point of view through a series of massing proposals the height of the tower had to be smaller than 65 m. for each site. In the second and third phase, starting For site 2 the base volume height was limited to a from one of these initial volumes a final design was minimum of 11 m and a maximum of 23 m, respect- developed through a series of parametrization and ing a maximal density coefficient of 6 and the height

CAAD and education - Volume 2 - eCAADe 39 | 107 of the tower had to be smaller than 35 m. For site 3 downtown Montreal. Three massing proposals (Fig- the base volume height was limited to a minimum of ure 2) had to be developed for each site and then 14 m and a maximum of 30 m, respecting a maximal compared from an environmental performance point density coefficient of 9 and the height of the tower of view. An important consideration in this sense was had to be smaller than 65 m. The generally applica- the solar access regulation in place for downtown ble setbacks were 3 m from the sides of the plot for Montreal that served as a base for the generation of the base volume and 4 m from the base volume lim- solar envelopes limiting the development of design its for the tower. on all 3 sites.

2.2. Environmental performance Figure 2 Nowadays there are enhanced digital tools and en- Massing studies for vironmental analysis software (RevitInsight, for in- Site 2 (stud. stance), more and more integrated in wide-use CAD Florence Lacroix) software to help designers make informed decisions in the earliest stages of design. The studio intro- duced a set of software tools for environmental anal- ysis and optimization based on the parametric/gen- erative design paradigm. Amongst these the toolset Ladybug Tools (Sadeghipour and Pak 2013) used in the course, is an open source extension for the popu- lar Grasshopper visual scripting environment housed inside McNeel’s Rhinoceros3d. The toolset provides a series of interactive 3D graphics and metrics for archi- tecture and urban design, based on local EnergyPlus Figure 3 EPW weather files and the geometry provided by the Testing the design user. for solar access requirements with 2.2.1 Sidewalks solar access. The first environmen- the generation of tal performance for the building massing to consider solar envelope was the solar access on the sidewalks surrounding (stud. Marie-Pier the three sites as required by the planning regulation. Bazinet) For site 1 it was required that a new building allows a minimum of 2 hours and 30 minutes of solar access between 12 p.m. and 15 p.m. on the equinox days. For site 2 and 3 it was required a minimum of 1 hour and 30 minutes of direct sun light on the sidewalks during the same timeframe and in the same days. To design the building massing considering this en- vironmental requirement, the students used the so- lar envelope (SE) method (Knowles 1981) through a specific component of Ladybug Tools. The method Students had the possibility to choose between allows to generate the maximum volume a building working only with the base volume or conceiving a cannot exceed in order to guarantee the required so- tower-like structure, based on the evaluation of the lar access of surrounding building facades or areas context and the typology of commercial buildings of

108 | eCAADe 39 - CAAD and education - Volume 2 Figure 4 of interests such as streetscapes, squares and play- Comparative study grounds. The inputs necessary for the SE generation of the were the latitude and longitude of the location, the environmental buildable plot, the surrounding buildings, the time performance of frame and the day. The results were irregular poly- different massing hedral volumes that the students used as constraints proposals for site 3 to develop the building massing (Figure 3). using Ladybug 2.2.2 Solar radiation benefit studies. The perfor- tools (stud. Mireille mance of the building massing proposals in relation Proulx) to solar radiation was tested using solar radiation benefit studies. With the help of tutoring by the au- thors, the students developed an algorithm which performed solar radiation simulation on the building envelope during two analysis periods, the days with air temperature < 18 °C and the warm season, i.e., the days with air temperature > 24 °C, and calculated the difference between the useful radiation and the harmful radiation. The envelope with the best perfor- mance was that with the largest difference, i.e., with the largest solar radiation benefit. In this way the stu- dents learned as well to balance possible conflicting environmental performance (Figure 4). 2.2.3 Sunlight hour and view analysis. In order to evaluate the potential of the building massing vari- 2.3. Parametrization and optimization ations to provide healthy and comfortable interiors, strategies solar access and view out were analysed on the build- Beyond the presentation of the tools pertinent for the ing envelopes. Total and average sun light hours sim- studio, students were introduced to a series of sim- ulated on the analysis grids located on the building ple optimization exercises conceived in four steps. facades and taking into account the urban context The first step was the identification of the problem were used to compare the performance of different to solve (objective) both from an environmental and design solutions, where the most performative was an architectural perspective. The second step was the one allowing the largest hours, i.e., solar access. the parametric formulation of the proposed objec- The view analysis was used by the students in two tive and the extraction of the input variables that con- ways. Some simulated the average sky view of the trol its performance. The third step involved the cou- building envelope through the Sky Exposure Factor pling of the relevant environmental simulation tools metric which assesses the portion of visible sky from with the parametric model to ascertain environmen- a point of the analysis grid on the façade. Others as- tal performance as a numerical value. The last step sessed the view out in relation to points of interests contained the creation of an optimization loop be- in the surroundings of the site as plazas and parks. tween the second and the third step by using the In this way the students had the possibility to inte- built-in optimization engine of Grasshopper. grate environmental performances related to the cli- The scope of the exercises was to expose the mate and the urban context. students to the tools and give them an overview of the relationship between the notions of shape con-

CAAD and education - Volume 2 - eCAADe 39 | 109 Figure 5 Position, height, and orientation optimization strategy (stud. Zeineb Ayed). Volumes are moved, rotated and resized to find the best solar benefits trol and parametric complexity. The students used controllable volumes on one site, additional transfor- result. the blueprint of the exercises to construct more com- mational steps for the volumes like rotation, and sim- plex and adapted strategies for their respective cho- ple parametric deformations for the proposed vol- sen sites and environmental performance directions. umes (Figure 5). 2.3.1. Moving a box. The simplest exercise involved 2.3.2. Controlling sections. A second exercise em- positioning and resizing a rectangular object on one ployed a different parametric approach, the con- of the proposed plots while keeping it within the struction of the architectural object from a series prescribed local regulation setbacks and inside the of parametrically controlled 2d sections as a lofted generated solar envelope. The goal was to maxi- skin. Specifically, the object was constructed based mize the solar benefit of the building while keeping on two independently controlled horizontal sections its total built volume constant. The input parame- (top and bottom base) and a mathematically con- ters were the coordinates of the building’s base on trolled interpolation between them. The base of the the plot: its length, its width, and the total building building was defined as a rectangle on the site de- height. Based on the resulting shape placed in its fined by the mandatory setbacks. The section con- urban context a solar benefits value was calculated trolling the top of the building was defined as an- using Ladybug Tools. The optimization goal was other rectangle parallel to the plot limits having a to find the largest possible solar benefits value for variable size, position, and height. Using a graphi- an architectural shape restricted to a given volume. cally controlled function multiple intermediary sec- For this Galapagos the evolutionary solver engine of tions were constructed between the top and bottom Grasshopper was used. The task for Galapagos was ones (one for each floor) and the object was created to maximize a fitness value calculated from the solar as a loft between them. benefits value minus the deviation of the constructed Figure 6 volume from the prescribed volume value. Section based The advantages of this exercise were its simplic- shape optimization ity and the low number of input variables. These through translation made the transformation of the architectural object and rotation (stud. both comprehensible and predictable for the stu- Olari Calina). The dents. The limitation was the reduced degree of para- rotation of the top metric control over the shape beyond resizing and of section is positioning a box primitive. This exercise template introducing a twist was extended and complexified in multiple direc- The advantages of this exercise were in the in- in the volume. tions by the students afterwards. The most important creased degree of control over the building shape us- included: the increase of the number of individually ing a relatively low number of input variables. (9 vari-

110 | eCAADe 39 - CAAD and education - Volume 2 Figure 7 Multi-faceted interior facade optimization with Galapagos (stud. Mireille Proulx). Each vertex is independently moved by the algorithm to find the best radiation ables + function graph). It also exposed the students features of the building shape a general intuition of benefits. The figure to a new way of controlling the shape. Even though the process was very accessible for the students. Fully shows only the this approach struck a good balance between con- relinquishing design control to an optimization al- interior façades of trol and input complexity it implied the use of a gorithm however, posed more problems on the ar- the model. mathematical function that was not easily optimiz- chitectural side. For this reason, this strategy was in able (transformable) in Grasshopper. The students most cases adopted by students only to control sub- extended this approach by adding new transforma- parts of the building (Figure 7). tion dimensions to the base sections like rotation of the base sections to control the twist of the building 2.4. Student project evaluations (Figure 6). The students’ activity and the results were evalu- 2.3.3. Controlling vertices. A third exercise took the ated on several criteria formulated in the studio’s syl- notion of parametric control of a shape to its limit. labus and used throughout the class instructional The exercise involved constructing a building enve- goals. The goals/criteria are serially linked and de- lope as a mesh with individually controllable vertices. termine one another in something that the authors In the exercise a building was constructed from a grid consider a typical early-solar-design workflow. The of 20 points that were connected into a mesh approx- first overarching goal of the studio was to reinforce imating a rectangular prism. Each of the 20 points amongst the students the understanding of the link was individually movable in a horizontal plane and between building geometry and solar gains. This thus controlled by 2 numerical variables one for x is a small part of a much larger pedagogical goal and one for the y coordinate. The movement of the for higher architectural education usually described vertices produced a faceted tower able to adapt lo- as performance-based design. To achieve this gen- cally and increase its solar response. Beyond the ba- eral goal students were taught and were expected sic point translation for solar reasons the exercise in- to know how to analyse, organize and compare en- cluded several architectural constraints to ensure the vironmental, architectural, and urban data. This im- feasibility of the resulting volume. The most impor- plies the ability to translate shape properties into tant were the maximum shape overhang limitation numbers and establish a set of relevant comparative and the possibility of a vertical circulation channel in- scenarios between early-stage solutions that balance side the building. architectural and environmental objectives. The biggest advantage of this technique was the Another linked goal or evaluation criterion was degree of parametric control from an environmen- the ability to deconstruct parametrically a design tal point of view. This implied many optimization problem and use parametric design tools to pro- variables. The degree of parametric complexity was duce solutions for it. For this the students had to high but since all variables controlled similar small demonstrate they know how to decompose larger

CAAD and education - Volume 2 - eCAADe 39 | 111 Figure 8 Complex approach integrating urban design, architectural concepts, and environmental performance (stud. Mathias Madelein). In this example the student used solar envelope and solar fan to shape the form of the interior courtyard. architectural problems into smaller geometric and good sense about the tools, and they could routinely mathematical concepts and solve them in a visual perform the simulations. It was clearly visible that the scripting environment like Grasshopper. The format correctness of the analysis improved from the inter- of the studio emphasized a performance-based and mediary presentations to the final submission. simulation-driven design process. As such, one of the The first part of the exercise, mostly about site student evaluation criteria and one of the targets for analysis, form studies and environmental perfor- the educational progression was the ability to docu- mance simulations can be considered a good way ment and represent such a process. to develop architecture student’s analytical thinking, data organizing and comparison skills. It was also 3. RESULTS about the identification of relevant data, mainly a Using the criteria presented in the previous section, quantitative undertaking, but it was a nice surprise the students’ activity throughout the duration of the to see students make qualitative assessments of the studio and their submissions were evaluated. The context, using nearby landmarks to optimize the majority of the submitted work was of high qual- view out from their proposed buildings (Figure 9). ity and the student feedback was positive, however Figure 9 there were some issues, recurring errors that will View analysis need special attention in future course/studio plan- considering nearby ning. landmarks (stud. 3.1. Environmental analysis skills Jamila Baldé). The optimization The proposed method can be considered successful The concept that was sometimes difficult to process took into from the point of view of teaching students how to grasp for students was the solar benefit. There consideration the perform environmental analysis. Students had to do were some confusions about the ‘good’ and ‘bad’ view of the building several environmental performance simulations, one radiation. This will need more explanations in fu- to the nearby solar radiation, sunlight hour and view study for each ture teaching undertakings, and it will need a re- square. massing proposals and for each site. This added up to evaluation for region specific weather patterns where nine iterations. That meant that by the time they got above 24 degrees C hours of the year (HOYs) are very to the optimization process, they already had a very few to justify this type of analysis.

112 | eCAADe 39 - CAAD and education - Volume 2 Figure 10 Optimization results (%) for the 12 student projects. The figures show the amount of improvement for the four criteria between the un-optimized (first) proposal and the final building shape. 3.2. Creative use of digital tools, integra- and were able to choose a good candidate for the op- tionwitharchitecturalandurbanobjectives timization process already having in mind the poten- Unfortunately, in many cases students surrendered tial variables of the design and their limitations. to the absoluteness of analysis and optimization pro- The point optimization strategies limited by vol- cess. The lack of counterbalancing with architectural ume targets and solar envelope were the simplest goals and urban design considerations, resulted in and straight-forward strategies and resulted in clear proposals that had less architectural potential. The processes. Some students chose optimization in sub- most frequent issue in this sense was due to the con- sequent steps, including size, position, and rotation. centration of all the attention on the building vol- This resulted in a more complex optimization process umes facing the sun and the complete neglection that although clearly formulated and coherent, one of the northern part. This resulted often unreason- could argue produced results dependent of the order ably tall and extended southern façades. Another fre- of the steps as much as their nature. quently occurring issue was the extreme thinness of When looking at the overall result (Figure 10) of the resulting volumes towards the top. In order to the optimization process in each student project, one maximize exposure the students tapered the build- can infer that most students managed to obtain good ings often producing unusably small floor plans to- results for solar radiation benefit. This was the main wards the tip of the proposed building. criteria for optimization although in some cases the The most successful projects were considered good result was due to a growth of building volume the ones that better integrated the design with ar- that generated a larger analysis surface, thus better chitectural concepts and respected the urban grid radiation results. But int the majority of cases the even if to the detriment of the environmental per- students managed to actually optimize the geometry formance. This balanced optimization in most cases and obtain significant environmental performance meant that only subparts of the building were con- improvements. sidered for optimization like an interior courtyard (Figure 8) or parts of the facades (Figure 7). 4. CONCLUSION The paper presented the methods and students’ de- 3.3. Conception of a coherent optimization sign outcomes and discussed the results of an envi- process ronmental performance-driven studio taught by the Most students had clearly formulated conclusions af- authors. The course succeeded in making the stu- ter the form studies and environmental simulations dents integrate architectural design principles, en-

CAAD and education - Volume 2 - eCAADe 39 | 113 vironmental simulations and optimization through Energy and Buildings, 238, p. 110831 parametric design. In this way the students learned Nations, U 2018, ’Revision of world urbanization the basis of performance-driven design which al- prospects’, United Nations: New York, NY, USA Reinhart, CF 2014, Daylighting handbook: fundamentals, lowed them to realize optimal design solutions designing with the sun, Christoph Reinhart through solar radiation, sunlight and view out simu- Reinhart, CF, Dogan, T, Ibarra, D and Samuelson, HW lations, iterative processes and form finding. Critical 2012, ’Learning by playing–teaching energy simula- aspects of the teaching method and indications for tion as a game’, Journal of Building Performance Sim- future improvements were presented to be used by ulation, 5(6), pp. 359-368 lecturer and researchers involved in the field. Reinhart, C and Fitz, A 2006, ’Findings from a survey on the current use of daylight simulations in building design’, Energy and Buildings, 38(7), pp. 824-835 REFERENCES Reinhart, C, Geisinger, F, Dogan, T and Saratsis, E 2015 Berardi, U and Jafarpur, P 2020, ’Assessing the impact of ’Lessons learnt from a simulation-based approach climate change on building heating and cooling en- to teaching building science to designers’, Interna- ergy demand in Canada’, Renewable and Sustainable tional Building Performance Simulation Association Energy Reviews, 121, p. 109681 Conference, Hyderabad, pp. 1126-1133 Charles, PP and Thomas, CR 2009 ’Building performance Roudsari, MS, Pak, M and Smith, A 2013 ’Ladybug: a simulation in undergraduate multidisciplinary edu- parametric environmental plugin for grasshopper to cation: Learning from an architecture and engineer- help designers create an environmentally-conscious ing collaboration’, Proceedings of Building Simulation design’, Proceedings of the 13th international IBPSA Chatzipoulka, C, Compagnon, R and Nikolopoulou, M conference held in Lyon, France Aug, Chambery, pp. 2016, ’Urban geometry and solar availability on 3128-3135 façades and ground of real urban forms: using Lon- Saragiannis, D.A. (eds) 2013, Combined or multiple expo- don as a case study’, Solar Energy, 138, pp. 53-66 sure to health stressors in indoor built environments, Ibarra, D and Reinhart, CF 2009 ’Daylight factor WHO, Copenhagen simulations–how close do simulation beginners ‘re- Touchaei, AG, Hosseini, M and Akbari, H 2016, ’Energy ally’get’, Building simulation, p. 203 savings potentials of commercial buildings by ur- Kanters, J and Horvat, M 2012, ’Solar energy as a design ban heat island reduction strategies in Montreal parameter in urban planning’, Energy Procedia, 30, (Canada)’, Energy and Buildings, 110, pp. 41-48 pp. 1143-1152 Voll, H, Thalfeldt, M, De Luca, F, Kurnitski, J and Olesk, Lockley, SW 2009, ’Circadian rhythms: Influence of light T 2016, ’Urban planning principles of nearly zero- in humans’,in Squire, L.R. (eds) 2009, Encyclopedia of energy residential buildings in Estonia’, Manage- Neuroscience, Elsevier, Amsterdam, pp. 971-988 ment of Environmental Quality: An International Jour- De Luca, F 2019 ’Learning Performance-driven Design. nal, 27(6), pp. 634-648 Students Approach Integrating Urban Form Studies [1] https://www.worldgbc.org/sites/default/files/UNEP and Building Performance Analysis’, Proceedings of urldate=2021-03-28 the 16th IBPSA Conference, p. 4 [2] https://www.solemma.com/climatestudio De Luca, F 2020, ’A Case Study of Teaching Method and [3] http://ville.montreal.qc.ca/pls/portal/docs/PAGE/AR Students’ Learning Experience in Integrating Urban ROND_VMA_FR/MEDIA/DOCUMENTS/01-282urldate=2 Design and Performance Simulations for Climate 021-03-28 Conscious Design in Estonia’, Acta Architecturae Nat- uralis, 6, pp. 44-63 De Luca, F, Dogan, T and Sepúlveda, A 2021, ’Reverse so- lar envelope method. A new building form-finding method that can take regulatory frameworks into account’, Automation in Construction, 123, p. 103518 De Luca, F, Naboni, E and Lobaccaro, G 2021, ’Tall build- ings cluster form rationalization in a Nordic climate by factoring in indoor-outdoor comfort and energy’,

114 | eCAADe 39 - CAAD and education - Volume 2 User-Centred Spatial Thinking in Architectural Design with Mixed Reality

Beatrix Emo1, Andri Gerber2, Christoph Hölscher3 1,2Institut Urban Landscape, ZHAW 3Chair of Cognitive Science, ETH Zurich 1,2{emob|gerr}@zhaw.ch [email protected]

We present the findings from an architecture seminar where students used heads-up mixed reality devices to design from the users` perspective. The aims of the course were to 1) help students design user-oriented spaces; 2) allow students to experience a design-analysis feedback loop; and 3) test the feasibility and effectiveness of using mixed reality in the architecture curriculum. The course required students to apply advanced spatial thinking, and so course content was specifically-developed to challenge students' spatial thinking. We present findings on the merit of using mixed reality for user-centric design based on i) student feedback and ii) a linguistic analysis of how students presented their work. Findings show that the use of mixed reality improved students` capacity to design from the users' perspective. While there is still much untapped potential for including mixed reality in architecture courses, technical difficulties are to be expected.

Keywords: user-centric design, mixed reality, spatial thinking, Hololens

INTRODUCTION sign methods (Schneider et al., 2013). Another initia- User-centred architectural design encourages archi- tive teaches evidence-based design methods to ar- tects to envisage how people actually behave in the chitecture students as an elective course (Emo, 2019). spaces that they design. Several methods exist to The core tenet of user-centred architectural design analyse this for already built spaces, such as Post courses is to teach students to consider not only their Occupancy Evaluation. The challenge in architec- aspirations as a designer, and the built environment, tural design is to propose interventions for an as-yet- but to consider, with equal importance, the role of unbuilt space that will meet the eventual needs of the the user (see figure 1). This paper explores the value user once the space is built. Despite several research of adding mixed reality (through Microsoft Hololens efforts in this direction at building (e.g. Carlson et al., devices) to such a toolkit. The use of mixed reality 2010; Dalton and Hölscher, 2017; Emo et al., 2016) in architectural education is not new; for example, and urban (e.g. Mavros et al., 2019; Emo et al. 2017) mixed reality labs have been used for teaching archi- scale, implementing such ideas in a course taken tecture for over a decade (see for example Burkhard by architecture students has proved to be a chal- and Schmidt, 2009; Wang and Schnabel 2008). What lenge. One notable teaching initiative developed a is novel in this paper is the use of heads-up mixed re- research-based design studio to test user-centric de- ality devices for teaching architecture students about

CAAD and education - Volume 2 - eCAADe 39 | 115 user-centric design approaches. ticularly relevant for architects, namely Spatial Ori- entation and Spatial Visualisation (Gerber, 2020; Berkowitz et al., 2021). Spatial Orientation is ”...an Figure 1 ability to imagine the appearance of an object from The importance of different orientations (perspectives) of the observer” the user in (McGee, 1979; in Hegarty & Waller, 2006). Spatial architectural Orientation is important, as architects are constantly design. Image after: imagining a what a space looks from the first-person Dalton and perspective even though they only view it in plan. Hölscher, 2017. The Another term for this spatial ability, suitable for the design-analysis field of architecture, could be ”perspective taking”. feedback loop, Spatial Visualization refers to the ability to imagine indicated at the ”...movements or displacements of parts...” (Lohman, centre of the 1979). Spatial Visualization is important, because graphic, is at the of an architect’s need to visualise an as-yet-unbuilt heart of our space. This requires constant shift between ego- didactic concept. centric and allocentric perspectives. This paper de- Being able to design from the perspective of the user scribes course content that was developed in order requires advanced spatial thinking. This is because to stimulate students’ spatial thinking. The extent the architect needs to imagine the same space from to which Spatial Orientation or Spatial Visualisation different user-perspectives, and needs to understand were used by the students was not measured; the fo- the implications of any design alterations on the end- cus is rather on the overall ability to design from the user. Spatial thinking relates to how humans process perspective of the user, and in mixed reality. spatial information (Newcombe and Shipley, 2015). Mixed reality encompasses elements of virtual The underlying assumption in cognitive science is reality and the real world, and so sits in the middle that all mental processes involve information pro- of the virtual to real spectrum (Milgram and Kishino, cessing. When discussing the architectural design 1994). The primary benefit of using mixed reality ver- process, this suggests that at least some part of the sus virtual reality is the capacity to include the real design process involves information processing. The world. Users of virtual reality refer to the black box mental processes behind the design process are of- effect, whereby the person is so immersed in the vir- ten discussed as part of the design thinking literature tual world that they can walk into a physical wall in (see for example Rowe 1994 and Schön 2019). The the space where they are situated. This does not hap- ability of individuals to process spatial information pen in mixed reality, given that the space in which varies; several spatial abilities test exist that measure one is situated is visible at all times. The use of virtual certain type of abilities (e.g. the Mental Rotation Test reality in architectural research and teaching is more (Shephard and Metzler 1971). Spatial abilities can be established than the use of mixed reality, providing classified in different ways, to date there is no com- motivation for our paper, that explores the value of plete consensus between approaches (see review in using mixed reality to teach architecture students. As Hegarty and Waller, 2005; Uttal et al., 2013). We fol- described below, students did not end up exploiting low the classification of types of spatial ability identi- the potential of including the real world in mixed re- fied by Carroll (1993). The question of which spatial ality. Two options would have allowed for this: abilities are relevant to architects is complex. Recent research on the spatial abilities of ar- • Projecting the model 1:1 in a real space chitects suggests that two spatial abilities are par- • Projecting the model 1:1 onto a scaled model

116 | eCAADe 39 - CAAD and education - Volume 2 Although these options were tested, no student fol- METHODS lowed this up; how to overcome this limitation will be We introduced Microsoft Hololens devices into the examined in the Discussion section. seminar “Experiments on the Spatial Perception and Given the aims of the course, it would have Spatial Cognition of Architects” held at the Institute been ideal to have the students design in CAD whilst for the History and Theory of Architecture (gta), ETH wearing the Hololens devices. However, at the time Zurich. We developed new course content to address of the course (2018/19) there was no CAD soft- the aims of the course, which were as follows: ware which allowed students to design directly in • To help students design user-oriented spaces; Hololens. As mixed reality technology develops and • To allow students to experience a design- becomes more mainstream, many more exciting op- analysis feedback process as an iterative loop; tions are becoming available, such as Fologram’s tool and for Grasshopper (Newnham et al., 2018) [1], or Min- • To test the benefit of using mixed reality devices desk’s tool for the gaming engine Unreal [2]. Further in the architecture curriculum. teaching initiatives are required to get a better under- standing of how practical it is to embed mixed reality We had access to 10 Hololens (1st generation) de- in courses for architecture students. vices for an academic year (see figure 2), during We used the mixed reality devices Hololens which the seminar was held twice. ETH Zurich’s Ed- which allow for several devices to be synced. Thus ucational Development and Technology department multiple people can view the identical model in real provided us with access to the devices and technical time, allowing for in-depth discussions whilst be- support during the project. The two semesters will ing immersed. All the participants immersed in the be referred to as Semester 1 and Semester 2. While model are assigned an avatar, so that the discussion the aims of the two semesters remained constant, can be quite interactive. For example, depending on the content of the seminar in the second semester the number of available devices, the course tutor(s) was changed to reflect the lessons learnt in the first and the students can be synced during a presenta- semester. tion so that any crit can occur whilst being immersed Figure 2 in mixed reality. This has the benefit of not having We had access to 10 to describe the feeling of being immersed, given that Hololens (1st all parties are also immersed, allowing the crit to fo- generation) cus on the content of what is being viewed. As will devices. be described below, technical difficulties meant that we had to abandon this functionality and focussed instead on each person viewing a model indepen- dently, without being synced with other course par- ticipants. The effect this had on the course outcomes is discussed in the Discussion section. We understand that as Hololens develops, the technical difficulties Hololens devices operate through gestures, which will be easier to overcome, making it possible to reap have to be learnt. With the headset on, a simple tu- the benefit of the sync functionality for future teach- torial goes through the principal gestures allowing ing endeavours. a novice to get to grips with how to use the device. Students had as much access to the devices as they wanted, and due to the limited number of devices had to reserve the devices ahead of time. Students

CAAD and education - Volume 2 - eCAADe 39 | 117 reported that they needed circa four hours to gain full Semester 1 operating knowledge of how to use the devices. Stu- Students were assigned one building out of curated dents who did not invest the time to learn how to use selection of buildings representing highlights in the the devices remained at a disadvantage for the dura- history of architecture from the Renaissance to to- tion of the semester. day. Examples included Poggio a Caiano by Giu- Students used the SketchUp Viewer plugin for liano da Sangallo (1485-1520), Casa Baldi by Paolo Hololens. SketchUp Viewer is a paid app, developed Portoghesi (1959), the Third House in Hanayama by by Trimble, which allows models designed in the Kazuo Shinoara (1977) or Villa Alem by Valerio Ol- 3D modelling software SketchUp to be viewed. The giati (2014). Over the course of the semester, stu- SketchUp Viewer for Hololens plugin allowed stu- dents had two tasks. The first task was to uncover the dents to view, but not edit, their CAD models whilst building; this involved elucidating its historical signif- wearing a Hololens device. The course was taken by icance but also describing how one might feel inside B-ARCH and M-ARCH students who were all familiar that building. The second task was to choose any as- with CAD modeling and used a CAD program of their pect of the building and propose a design modifica- choice. Note that it is possible to skip the second step tion. Students had to present their findings for both in the workflow pipeline if the base model is drawn parts of the seminar using Hololens. directly in SketchUp. The workflow pipeline was as follows: Figure 3 Photo of the final • Students developed their 3D CAD models in a crit in semester 1. CAD program of their choice One student wore a • Students exported their model as a .skp file so Hololens device that it could be opened in SketchUp and what they were • Students uploaded the .skp model to the viewing within course’s private library on SketchUp Warehouse Hololens was • This model was then visible in the SketchUp projected onto the Viewer programme on the Hololens devices. presentation wall at The Discussion section will evaluate the limitations of the front of the such a workflow pipeline for the aims of the course. classroom for the As mentioned above, no student projected the rest of the class to see. A second mixed reality model 1:1 to either a real space or to a Having been given time to learn how to operate the scaled model. Instead, the mixed reality models were student provided devices, students gave both sets of mid-term and fi- commentary on placed virtually on a surface in the classroom (often nal crits using the Hololens. In Semester 1, we ex- a desk). This limitation will be analysed in the Dis- what was being pected to use the sync capability of the devices, so visualised. cussion section. The findings reported in the Analysis that the course tutors and other students could be section are from: immersed in the model being presented. Unfortu- 1. student feedback, provided orally during class; nately, technical difficulties meant that the sync ca- 2. linguistic analysis of how students presented pabilities did not work as intended and ended up im- their work; and peding student presentations. The main challenge 3. from the grading of students’ work. was that, even though the devices were set up prior to the course and synced with one another by the teaching staff, as soon as the (synced) headset was transferred from the staff member to a student, that

118 | eCAADe 39 - CAAD and education - Volume 2 headset was no longer synced. Given that the sync- of this semester). The limitation of using the Hololens ing process takes c. 5 minutes, and student presen- devices in this way will be evaluated in the Discussion tations were only 10 minutes long, it was not feasible section. to account for all of the technical difficulties. Having been thwarted for both mid-term crits, we decided ANALYSIS to discard the sync function for the final presentation, Semester 1 where instead the immersed view of one student was Students developed their capacity to understand streamed onto a presentation wall in the classroom buildings from a users‘ perspective. This skill was (see figure 3). The impact of such technical difficul- built up through the use of mixed reality. In the first ties on the course outcomes are evaluated in the Dis- task, students explained their building whilst walk- cussion section. ing through it whilst being immersed in the building. Students reported that being able to walk through Semester 2 the building from the first-person perspective whilst The content for this semester was built around the being immersed in it helped a) their ability to under- question of what spatial abilities are unique to archi- stand the building and b) to understand how some- tects. This content was linked to the research project one else might experience the building. It was not “How do architects ‘think and design space’?” (Ger- possible to disentangle the effect of the level of im- ber, 2020). The first part of the semester was used mersion in mixed reality from the use of the first- to introduce students to the concept of spatial abili- person perspective: this would be an avenue of fu- ties and how to test them. In the second part of the ture work. The effect of the second task (present- semester, students had to develop their own spatial ing a design modification in mixed reality) strength- ability test (for an example of such a test see figure 4). ened the students’ ability to understand the build- These new tests were then taken by other students, ing from the inside out. Students reported that either as a paper/pencil test or visualised in Hololens. they reinforced the feeling that they experienced in Students evaluated the difference when taking the the first task, about how a building feels to a user. newly-developed tests in mixed reality versus as an Furthermore, students stated that they experienced analogue test. the beginning of a design-analysis feedback loop, Figure 4 given that a) they chose the type of design modifica- Example spatial tion that they would implement having experienced ability test the building from the user’s perspective and b) they developed by one could test various iterations of the design modifica- group of students tion in mixed reality. during Semester 2. Students reported that both tasks together al- The task is to tered how they presented the building, and that they identify which of type of information they chose to portray was dif- the four resultant ferent to the way they usually explained a space in models is created other crits. We analysed the language that students by the cast shown used when presenting their buildings. The goal was twofold: on the one hand to see how their way of top left. The new course content allowed us to avoid the tech- talking about architecture and space would change, nical difficulties encountered in Semester 1 as we did and on the other to discuss these findings with the not use the sync function of the Hololens devices. students in order for them to better reflect upon how Rather, each student viewed the model by them- they verbalize spatial thoughts. In order to do so, selves (figure 5 shows a photograph from the final crit

CAAD and education - Volume 2 - eCAADe 39 | 119 the presentations of the students were recorded and These aspects helped students to develop and ap- analysed through protocol analysis. The interest in ply the type of spatial thinking that is required when in this type of analysis stems from different exam- designing with the user in mind. We did not collect ples where systematic language analysis showed in- performance data on the difference in performance teresting results on cognitive processes during de- when taking a test analogue versus using Hololens: sign (Suwa and Tversky, 1997) and on the differences it would be interesting to test this in the future. between architects and non-architects (Cialone et al., 2017). In this case we were only interested in analyz- Figure 5 ing language elements describing space or the expe- Photo of the final rience of space. crit in semester 2. Students’ response to using mixed reality as a pedagogic tool was varied. The main feedback was that the initial intrigue was great, and that it was ex- citing to test a relatively new tool as part of a semi- nar. Students reported that it took circa four hours to learn how to use the devices properly; if they spent less time with the devices, they were frustrated due to basic operational problems that were instead eas- ily overcome for the students who spent more time learning how to use the devices. Overall, students noted that, because they were not able to design in mixed reality, but only view a model that was de- signed on a desktop, the use of the Hololens in this DISCUSSION semester was more like a visualisation tool for pre- The aims of the course were threefold: sentation purposes. The switch from the originally- 1. To help students design user-oriented spaces; intended mode of presentation of the final crits (us- 2. To allow students to experience a design- ing the sync function) to the workaround (streaming analysis feedback process as an iterative loop; the immersed view onto the presentation wall im- and pacted how students could interact with mixed real- 3. To test the benefit of using mixed reality devices ity, although it was not possible to measure this im- in the architecture curriculum. pact systematically. Semester 1 addressed all three goals, whereas Semester 2 Semester 2 addressed only the last goal. For both The use of mixed reality helped students visualise semesters, all goals were met. In Semester 1, stu- the spatial tasks required in the student-developed dents reported that they experienced and presented spatial tests. When taking a test as a paper/pencil the building in a novel fashion that they had not test, the objects had to be visualised mentally. Using previously experienced in their studies. They stated Hololens to solve the test helped for several reasons: that they believed such a user-centric perspective would help them design better spaces in the fu- • Students could see the objects in 3D ture. Students stated that they experienced a design- • Students could touch and rotate the objects analysis feedback loop, even though in cases where • Students could change their standpoint in rela- students only did one design alternative, the effect tion to the objects, by walking around them. of such a feedback loop was limited. Students ex- pressed thanks for having been exposed to such a

120 | eCAADe 39 - CAAD and education - Volume 2 design workflow and stated they thought it would The language analysis did not follow our hypoth- help them in the workplace after their studies. A ma- esis because students did not use more spatial lan- jor limitation is that no systematic data was collected guage when presenting their project in mixed reality. where we could measure to what extent students One reason for this might be that the students pre- learnt user-centric design methods, and as such there sented their work many times over the course of the is no statistical threshold for evaluation. Further- semester and so concentrated on the fundamental more, grading differences between students were aspects of their work. Nevertheless, we observed one negligible and did not prove a useful way of distin- major change in the way they talked about space; guishing to what extent the course outcomes were when describing projects analogically by print out learnt differently between students. students would describe the spaces more locally - The value of using mixed reality devices in the “here you can find this”, “there you have that”. How- architecture curriculum had mixed results, given the ever, when describing the project in mixed reality many limitations of the course set up. First and students referred more to the movement through the foremost, it was not possible to design directly in building, for example, “and then you go there and Hololens, but only to view the model that had been you can see that...”. This suggests that the experience designed on the desktop. Therefore, the use of mixed and presentation of space in mixed reality appears to reality was categorised by many students as a presen- be based more on the movement in space, than by a tation tool. The huge potential of mixed reality for fixed observer. This fits with the user-centric aims of architectural design is still untapped; this will change the course. once it is possible to edit a CAD model whilst wear- In conclusion, we found that the use of mixed ing the device. A second and related limitation is reality improved students’ ability to think from the the fact that, given that it was not possible to de- users’ perspective. Students reported that their spa- sign in Hololens, this resulted in a cumbersome work- tial thinking was challenged by the course content. flow pipeline, with the CAD model having to be ex- More work is needed in order to be able to measure ported from the native software and imported into to what extents students are successful in design- Hololens. As has been stated before, user-centred ar- ing from the users’ perspective. We hope that more chitectural design must be seamlessly integrated in courses using mixed reality will be offered to archi- an architect’s native workflow for it to become main- tecture students in the future, as this seems to help stream (Emo, 2019). Third, the time constraints of the students understand their models from different per- course meant that it was not possible to fully test the spectives, including the user perspective. real effect of mixed reality as opposed to virtual re- ality, given that no student projected a model 1:1 in ACKNOWLEDGEMENTS a real space, or onto a scaled model. Fourth, tech- This project was supported by an ETH Innovedum nical difficulties prevented the desirable sync func- grant #1869. We thank ETH Zurich’s Educational De- tionality of Hololens to be used effectively during the velopment and Technology department for lending course. We understand that this has been addressed us the Hololens devices and their excellent techni- in the subsequent version of the device, Hololens 2. cal support. Thanks to also Julian Wäckerlin and Se- Nevertheless, such technical difficulties are to be ex- bastian Oswald for their assistance during the course, pected when using novel technologies, and despite and to the students who participated in the seminar. the need to change the mode of presentation, stu- dents valued being able to explore the use of mixed reality in their studies. REFERENCES Berkowitz, M, Gerber, A, Thurn, CM, Emo, B, Hölscher, C and Stern, E 2021, ’Spatial Abilities for Architecture:

CAAD and education - Volume 2 - eCAADe 39 | 121 Cross Sectional and Longitudinal Assessment With Mavros, P, Li, H, Dubey, R, Jazuk, K and Hölscher, C 2019, Novel and Existing Spatial Ability Tests’, Frontiers in ’Architectural Psychology for Mixed-Use Cities’, in Psychology, 11(4096), pp. 1-16 Cairns, S and Tunas, D (eds) 2019, Future Cities Lab- Burkhard, R and Schmitt, G 2009, ’Visualising future oratory Indicia 02, Lars Muller Publishers cities in the ETH Value Lab’, in M, WXS (eds) 2009, McGee, MG 1979, ’Human spatial abilities: Psychometric Mixed Reality In Architecture, Design And Construc- studies and environmental, genetic, hormonal, and tion, Springer, pp. 205-218 neurological influences’, Psychological Bulletin, 86(5), Carlson, LA, Hölscher, C, Shipley, TF and Dalton, RC 2010, pp. 889-918 ’Getting Lost in Buildings’, Current Directions in Psy- Milgram, P and Kishino, F 1994, ’A taxonomy of mixed chological Science, 19(5), pp. 284-289 reality visual-displays’, IEICE Transactions on Informa- Carroll, JB 1993, Human Cognitive Abilities. A survey of tion and Systems, E77D(12), pp. 1321-1329 factor-analytic studies, Cambridge University Press Newcombe, NS and Shipley, TF 2015, ’Thinking about Cialone, C, Tenbrink, T and Spiers, HJ 2018, ’Sculptors, Ar- spatial thinking: New typology, new assessments’,in chitects, and Painters Conceive of Depicted Spaces Gero, J (eds) 2015, Studying visual and spatial reason- Differently’, Cognitive Science, 42(2), pp. 524-553 ing for design creativity, Springer, pp. 179-192 Dalton, RC and Hölscher, C (eds) 2017, Take One Building: Newnham, C, van den Berg, N and Beanland, M 2018 Interdisciplinary Research Perspectives of the Seattle ’Making in mixed reality’, Proceedings of ACADIA, pp. Central Library, Routledge, Taylor & Francis Group, 88-97 London Rowe, PG 1994, Design Thinking, MIT Press Dalton, RC, Hölscher, C and Turner, A 2012, ’Understand- Schneider, S, Kuliga, S, Hölscher, C, Conroy-Dalton, R, ing Space: The Nascent Synthesis of Cognition and Kunert, A, Kulik, A and Donath, D 2013 ’Educating the Syntax of Spatial Morphologies’, Environment architecture students to design buildings from the and Planning B: Planning and Design, 39(1), pp. 7-11 inside out’, Proceedings of the 9th international space Emo, B, Al-Sayed, K and Varoudis, T 2016, ’Design, cogni- syntax symposium, pp. 363-75 tion & behaviour: usability in the built environment’, Schön, Donald 2019, The Reflective Practitioner: How Pro- International Journal of Design Creativity and Innova- fessionals Think In Action, Routledge tion, 4(2), pp. 63-66 Shephard, RN and Metzler, J 1971, ’Mental Rotation of Emo, B and Hölscher, C 2020, ’The potential of cogni- Three-Dimensional Objects’, Science, 171, pp. 701 - tive science for architecture’,in Gerber, A (eds) 2020, 703 Training Spatial Abilities. A Workbook for Students of Suwa, M and Tversky, B 1997, ’What do architects and Architecture, Birkhäuser, pp. 18-20 students perceive in their design sketches? A pro- Emo, B 2019, ’Why Evidence-Based Methods Are Useful tocol analysis’, Design studies, 18(4), pp. 385-403 for Architectural and Urban Design’, in Leopold, C, Uttal, DH, Meadow, NG, Tipton, E, Hand, LL, Alden, AA, Robeller, C and Weber, U (eds) 2019, Research Cul- Warren, C and Newcombe, NS 2013, ’The malleabil- ture in Architecture: Cross-Disciplinary Collaboration, ity of spatial skills: a meta-analysis of training stud- Birkhäuser, pp. 173-182 ies’, Psychol. Bull., 139, p. 352 Emo, B, Treyer, L, Schmitt, G and Hölscher, C 2017 ’To- Wang, X and Schnabel, MA 2018, Mixed reality in archi- wards defining perceived urban density’, Proceed- tecture, design, and construction, Springer Science & ingsofthe35thInternationalConferenceonEducation Business Media and Research in Computer Aided Architectural Design [1] https://fologram.com/ in Europe: ShoCK, pp. 637-646 [2] https://mindeskvr.com/ Gerber, A 2020, Training spatial abilities: a workbook for students of architecture, Birkhäuser Hegarty, M and Waller, D 2005, ’Individual Differences in Spatial Abilities’,in Shah, S and Miyake, S (eds) 2005, The Cambridge Handbook of Visuospatial Thinking, Cambridge University Press, pp. 121-169 Lohman, DF 1979, Spatial ability: A review and reanaly- sis of the correlational literature., Stanford Univ Calif School of Education

122 | eCAADe 39 - CAAD and education - Volume 2 A New Player Has Entered the Game

Gaming tropes as a harbinger of architectural praxis

Vincent Hui1,Tatiana Estrina2, Alvin Huang3 1,2,3Ryerson University 1,2,3{vincent.hui|testrina|alvin.huang}@ryerson.ca

Video games as a medium for entertainment have evolved overtime to become impactful and relevant in current culture, industry, and education. Although architecture is a cultural commodity that has been present since the dawn of civilization, within the span of less than 50 years, video games have had an incredible impact on architectural praxis. This paper posits a dyadic relationship between computer games and architecture, exploring the influences video game design attributes have had on nascent designers within the industry. This paper documents a case study examining architectural students' studio projects from Canada's largest architecture program, identifying the influences of video games that emerge within their designs. The paper discusses the effect video games have made on architectural education through passively impacting current pedagogical practice, design tendencies, and inevitably molding the architects of the future.

Keywords: Architecture, Emergent Design Trends, Education, Gaming, Digital Proxy, Experiential Learning

INTRODUCTION also creating a cultural influence in design method- Video games have become a ubiquitous medium ex- ologies seen predominantly in student architectural tending well beyond recreational use and increas- work. Given the inherent cultural dynamic between ingly gaining traction in educational and professional architecture and cultural shifts as inculcated in cur- applications. Video games have rapidly expanded rent media, video games have become an emergent their versatility from recreational outlets for a range player in shaping nascent architects’ design attitudes of casual and competitive gamers of all demograph- and approaches. ics through to advanced simulation and education Architecture is a cultural commodity that has tools in a diversity of disciplines. As a tool for stu- been present since the dawn of civilization. Yet dents, video games are known to be a unique, en- within the span of less than 50 years, video games joyable, and effective way to learn about the con- have had an incredible impact on architectural praxis. cepts related to architecture when integrated with As a cultural medium, architecture has served as a educational design exercises. In additional to tech- milestone for any civilization’s technological, cultural, nical advances in video games merging with digital social, and economic zeitgeist. It is both provoca- tools used in architectural industry, video games are tive in its initial conception and taken for granted

CAAD and education - Volume 2 - eCAADe 39 | 123 as it becomes acclimatized in its urban context over vius lays the groundwork in defining the architect’s time. Yet as with all cultural commodities, architec- role as more than construction and building, but in- ture evolves as each successive generation creates stead demands that it is a synthetic discipline. This a future that is literally and figuratively built upon has been refined over the centuries where architects and within the past based upon current contexts have a mandate to explicitly serve as ”cultural inter- and tools. This scaffolding has rendered the current mediaries” to this era (O’Callaghn 2017). paradigm of architecture as an increasingly collabo- Within architectural praxis, this responsibility is rative discipline. That both the technological tools of why there is such a great value (both in terms of representation and the cultural concentration in the credit, time, effort, accomplishment, etc.) ascribed video game industry have concurrently evolved with to the design of the built environment, it serves as architecture is not surprising. To find 3D modeling a cultural artifact that both epitomizes a specific era, and production tools, responsiveness from artificial demographic, technological advances, and perspec- intelligence and algorithms, and increasingly immer- tive as well as establishes the groundwork for sub- sive technologies in both industries is tacit to con- sequent generations of architects. Unlike other cul- temporary practice. This conjoined evolution over tural artifacts such as music, fashion, or literature, last half century has resulted in hidden and overt con- built architectural work is a tangible and monumen- nections between the two industries’ respective out- tal medium that serves as a milestone that persists puts. well after the societies that have erected them have Architecture is both a multifaceted, creative out- come to pass. Beyond the advances in the means let that elicits delight, and a discipline anchored in of production and visualization, there are a myriad an obligation to durability and utility. It serves as of overlaps that are increasingly emergent in these a cultural medium and mediator. Architecture has convergent mediums. Under the hood, there are never existed in isolation, as social, economic, tech- also many digital tools such as artificial intelligence nological, and ecological shifts have simultaneously (as seen in non-playable characters (NPCs) in games altered and drawn from the built environment. All and in architectural simulations), optimization (such architecture, from mundane factories to grand civic as the appropriate ways to display an entire virtual buildings, reflect contemporary cultural values. Like world of highly detailed content to determining best cultural artifacts throughout history and around the configurations to adapt to variable user traits), deci- world, architecture epitomizes the zeitgeist of any sion making (including decision-tree logic matrixes given society. In his ”10 Books on Architecture”, Vit- and customization), and procedurally generated con- ruvius presents the fundamentals of architecture as ditions that empower game developers and architec- durability, utility, and beauty. It is important to note tural designers alike. that even over two thousand years ago, the patri- What is more evident is the impact of the cur- arch of architectural discourse indicated the need rent state of architectural praxis on architectural de- for a multifaceted education to excel in architecture sign. Video games are now influencing the design of on account of the various cultural and technologi- current and future architectural work. This virtuous cal shifts currently at play when describing ”the ideal cycle between these two different cultural mediums architect should be a man of letters, a skillful drafts- is not only rapid but mutually reinforcing. If gam- man, a mathematician, familiar with historical studies, ification is the integration of elements from game a diligent student of philosophy, acquainted with mu- design in non-game contexts, then architecture has sic, not ignorantof medicine, learned in the responses of progressively witnessed this in both the production jurisconsults, familiar with astronomy and astronomi- and workflows of current praxis, but certainly in its cal calculations“ (Vitruvius and Morgan 1960). Vitru- pedagogy. The authors of the paper do not contend

124 | eCAADe 39 - CAAD and education - Volume 2 that this is a widespread or direct reaction, however cuses on the influence of video games on work by they posit that there are overt and subtle conditions architecture students through several lenses includ- where video games are seeping into and influencing ing: planar emphasis, dramatic artificial lighting, the architectural design. There is an evolving symbiosis use of ornamental versus utility assets, understand- between the two disciplines. Where once architec- ing of materials and applications, and organizational ture served as a background for video games to de- disconnect of architectural designs. velop a framework in their medium, the authors con- In order to quantify the effect video game design tend that a dyadic relationship is emergent. has had on emerging designers, the studio project‘ from a second-year architectural science class of 90 VIDEO GAME INFLUENCES ON ARCHITEC- students were surveyed and compared against typi- TURAL DESIGN cal video game attributes that emerged from an ini- tial analysis of the work against gaming tropes. The Despite video game technology contributing to the students each produced two projects throughout development of modeling, visualization, and repre- their Fall semester, a pavilion and a concert hall. The sentation of architecture, the video game industry music pavilion, sited in Grange Park, Toronto, served has also proved to be a imagination and design cat- as a warm-up project to get students re-acquainted alyst. Historically, popular media have had a signifi- with architectural tools. The month-long assignment cant impact on popular culture, and therefore design, challenged the students to design a sculptural open- and video games are not being an exception (Ata- air pavilion that allows for gathering and the enjoy- man 2000). With the growing number of generations ment of music, with a focus on formal and mate- raised by the navigation of digital spaces, the spa- rial representation of the students’ architectural con- tial design within video games has begun to trans- cepts. Due to the site’s close proximity to Frank form the way that architecture students, profession- Gehry’s AGO and Will Alsop’s OCAD U Design, the als, and the general public perceives and imagines students were expected to engage with imaginative architecture. This shift in spatial understanding and ideas unconstrained by contextual influences. The design is explored further in this paper via surveying second assignment challenged students to design a of projects created by architecture students, and the concert hall or performance space for a particular au- observations of phenomena that emerge based on dience or music style, ranging from pianists to bur- architectural gaming tropes. While the authors do lesque to choir groups. Lasting for three months of not posit that these design approaches are solely the the term, students were required to design within result of video game influence, they discuss correla- a designated urban site and address various techni- tions between the two as a means to begin to exam- cal issues alongside the conceptual representation ine the influence of this particular cultural occurrence of the musical genre. The survey results are pic- on the design industry at large. tured in Table 1. While some of the in-game tropes In recognizing the duality of influences between seemed to be more prevalent in the pavilion-scale architectural design and video games, many tropes or the building-scale projects, the overall survey indi- become evident in student work. Utilizing second cated numerous instances of the aesthetic influence year design studio projects completed by students of video games in the student’s work. within Canada’s largest accredited architecture pro- Due to the students’ early introduction to the gram, an analysis of architectural design project im- software, the vast majority of student projects sur- agery and drawings in comparison with a myriad of veyed chose to use the Enscape plug-in for their vi- video games reveals a clear connection between the sualization, rather than other available engines such cultural influences of video games impacting how as V-Ray or Corona Renderer. The software uses video students are designing buildings. This case study fo-

CAAD and education - Volume 2 - eCAADe 39 | 125 Table 1 Results of various video game tropes found in the 180 second-year architectural science design projects surveyed.

game engines in order to visualize models in real- to be very unusual, not only disconnecting the in- time. Although this gives students the ability to in- terior program from the exterior of the building ar- stantly export views, the software’s gaming origins chitecturally but also frequently completely ignoring are evident in the graphic styles of the imagery the architectural boundaries of physical enclosure. This students produce and may skew their design and vi- leads to the occurrence of two formal discrepancies sualization sensibilities towards those discussed, po- within video game design; the disconnect of enve- tentially distorting the results. lope from internal spaces and the disengagement of utility from spatial organization. Organizational Disconnect Due to the non-physical constraints imposed on When building design process occurs in plan, design- digital spaces designed for games, many in-game ers closely examine both spatial relationships in pro- models do not abide by the external envelopes for grammatic terms and scale of rooms in relation to the design of interior spaces. One such example is the human body. Here, designers begin to cross an- the game Control (505 Games, 2019), which was dig- alyze how visitors will feel when inhabiting spaces itally reconstructed by Extra Office to examine the and begin to tweak spatial qualities in both section spatial relationships to the brutalist designs featured and plan drawings, all the while examining the per- in the game. The 3D spatial arrangement proved that spectives of the spaces. Modern architectural design despite its fidelity to the architectural style and the focuses on the expression of the functional program numerous successes with regards to the architectural architecturally, signaling the activity from within the design of the game, a large portion of the spaces do space to the building’s exterior. Video games on not fit within the main building’s envelope (Figure 1). the other hand, are generally not physically occu- According to the reconstructed model of the levels, pied by the player (apart from virtual reality games) the game’s interior spaces not only venture outside and instead serve as a contextual background for the of the building’s exterior boundaries but also do not game’s action and narrative. In addition, video game fill the entirety of the volume of the building (Scav- designers are unconstrained by boundaries of phys- incky 2020). A similar occurrence can be found in ical space. The combined lack of these constraints other games, such as The Path (Tale of Tales, 2009) leads to video game interiors and floor plan designs where the vast interior spaces within ‘Grandmother’s

126 | eCAADe 39 - CAAD and education - Volume 2 Figure 1 Reconstruction of volumes in Control (left) (Scavincky 2021). Impossible interior spaces in The Path (right). Figure 2 Student projects exemplifying exterior formal gestures with little ramifications inside .Figure 3 /Animal Crossing/ (left, middle) and /Sims 4/ (right) exemplifying the perimeter utility phenomenon. Figure 4 Projects exhibiting ‘plan Tetris’ approach, with programmatic elements on the perimeter leaving central empty areas. Figure 5 Student project designed in three-dimensions (left), projects emphasizing planar nature of surfaces (center, right). Figure 6 /Cyberpunk 77/, /Hitman/, and /Star Wars Battlefront 2/ gameplay images showcasing the exceedingly reflective or glossy materiality.

CAAD and education - Volume 2 - eCAADe 39 | 127 House’ cannot possibly be confined within the small currence among the concert hall projects, with 51% forest hut that is presented from the exterior. of the projects exhibiting these traits. A manifes- This phenomenon emerges frequently in archi- tation that is also termed ‘plan-Tetris’ sees the stu- tectural design, with many architects known for final- dents place all of their programmatic elements on izing their formal design before thinking about inte- the perimeter of the building envelope, leaving them rior programming. While leading to fascinating ex- with large open volume in the center. This leads terior geometries and unique architectures, this ap- to large open spaces that are disproportionate from proach frequently does not lend to functional or in- the program requirements for the building, which habitable spaces. Among the projects surveyed in becomes problematic for the occupants. Students this study, 25% had an envelope disconnected from have the tendency to place ‘lounges’ our ‘breakout the interior spaces. Two of such projects are pictured rooms’ in leftover floor areas, such as in Figure 4, in Figure 2, representative of the tropes that students which tend to be more intimate private spaces in fell into. The first project, with a Voronoi-covered the building and do not function well in large echo- veil overtop of a rectilinear building, exemplifies the ing volumes. The furnishing of awkward spaces is students’ desire to present ’innovative’ forms largely only made more accessible to students with available without any rationale. Rather than designing an or- tools and more importantly, an unfortunate precon- ganic façade to enclose the interior spaces, the stu- ditioning from video games that furnishings denote dent designed the two elements in isolation, result- space. Occupants begin to congregate on the pe- ing in a building envelope having very little relation- riphery of the spaces making the scale of the space ship with the occupiable spaces inside. This severity unusable. Similarly, many of the student projects, of detachment is antithetical to architectural praxis. such at the floor plan on he right of Figure 4, fill the Similarly, the second project presents a façade design leftover void with a large atrium space. While atria that does not respond or interact with its interior pro- can be useful in visually and audially connecting lev- gramming. els and occupants, they do not serve a functional pur- The disconnect between utility and space fre- pose and should not be occupying much of the floor quently emerges when players are able to gener- area of a functionally programmed building. ate their own spatial layouts, such as when play- ing the likes of Animal Crossing (Eguchi and Nogami, Planar Emphasis 2020) or Sims 4 (Redwood Shores Studio of Maxis, First person shooters (FPS) have emerged as a very 2014). When engaging with architectural design popular sub-genre of video games, generally follow- within video game environments, players frequently ing the player’s character, either from their own point find themselves placing assets and furniture along of view or from above, as they shoot and battle their the perimeter of rooms while leaving the center way through the game. Typical first person shooter empty (Figure 3). Such an approach seems to be (FPS) games provide the players with a unique spa- the most straightforward method to design a room tial experience, exploring space from the perspec- to many, as it creates the most functional use of the tive of the character. Thus, the navigational capa- space. However, in most cases a large open space bilities of such games place value on the navigation in the center does not provide significant useful area within the architecture rather than isolated ‘money and instead becomes a large unused portion of the shots’ popular among the architecture community. room. This is especially true in video games, where For this reason, the FPS genre uses a unique architec- the digital occupant does not require the space to tural approach, as spaces are designed specifically to move around. be used as a battlefield. Much of the game’s design This design approach was the most frequent oc- focuses on the ground plane and objects that the

128 | eCAADe 39 - CAAD and education - Volume 2 player can use to hide and protect themselves with, since the dawn of design. From writings by Vitru- such as walls, garbage cans, and tables (Guttler and vius to contemporary architectural dialogue by Peter Johansson 2003). Meanwhile the horizontal surfaces Zumthor, materiality has been explored for its struc- themselves, especially the ceiling conditions, are un- tural capabilities and aesthetic finishes within a build- derdeveloped and largely ignored as they are infre- ing (Vitruvius and Morgan 1960) (Zumthor 2006). Al- quently within the player’s line of sight. Therefore, though many historical video games focus on accu- without the need for player interaction, verticality is rate recreation and reconstruction of the materials often left underappreciated in both the game devel- found in the built environment, games that present oper and player’s perspective. As seen in popular FPS fantastic or futuristic conditions begin to experiment games such as Counter-Strike and Overwatch a major- with the materiality of their conditions. The Assassins ity of the maps focus on fighting on the horizontal Creed series (Ubisoft) is an example of a game that fo- plain with distinct lanes throughout maps (Willatzen cuses on accurate historical representation through and Karakurt 2020). This design approach is largely reconstruction previously built cities with details pre- contradictory to that used by architects as occupants cise, from the replication of buildings to the type and of any given space experience the design at varying size of stones used. On the other hand, the Halo scales and from differing angles, forcing architects series (343 Industries, Bungie) is a future projection to examine, account for, and design every surface with explorations in their world building question- within an architectural project, rather than neglect- ing ”What if?”. The result is a world where alien and ing any they are not fond of. human species coexist, creating a built environment FPS video games have begun to instigate an ap- that surpasses the boundaries of construction today, proach to architectural design where students disre- and in these explorations is the play on materiality. gard surface design and instead focus on the con- The creation and use of dubious materials as a vi- finement and enclosure of the space. It is not infre- sual emphasis have become an increasing trend in quent to see students present empty and underde- video games, from gameplay to marketing graphics, veloped ceiling conditions despite the multitude of it has started an emergence in architectural design fixtures and systems that are generally found there in tandem. Through a comparison between various (Figure 5, left). Still, many students fail to recognize video game titles, there are several tropes found in the abnormality of a blank ceiling and instead use the treatment of materiality. The attributes of incor- it as a means to frame their rendering, similar to its rect material properties, dubious materials, and in- purpose in FPS games. A reoccurring theme among compatible material applications are very commonly the student’s projects is the emphasis of the verti- seen in the surveyed students’ projects. cal planes over the horizontal, such as in the right- A selection of video games ranging from FPS most image of Figure 5. The designer of this project genres to roleplaying games were examined on uses emphasized the wall surfaces with dark colored slats, of materiality in architecture (Figure 6). The inclu- while framing them with the continuous horizontal sion of excessive reflective materials, as well as the planes of the ceiling and floor. Rather than capitaliz- inclusion of dubious and unknown materials are re- ing on the three-dimensionality that architectural de- occurring conditions present in games such as Cyber- sign affords, the student uses the FPS design model punk 2077 (CDPR, 2020) and Star Wars Battlefront 2 to generate their pavilion. (EA, 2017). In Cyberpunk 2077, the interiors of build- ings, especially in the wealthy districts of the game’s Dubious Materiality city, are finished with highly reflective surface on The discussion of materiality within architecture has the flooring, walls and ceiling and used in conjunc- been an essential element in the design of buildings tion with the lighting in the scene. The intention

CAAD and education - Volume 2 - eCAADe 39 | 129 Figure 7 Student projects featuring materials with peculiar and unnatural properties.

Figure 8 Student projects using fictitious materials and unresolved construction details. Figure 9 Student projects where lighting was unrealistically dramatized and over-emphasized.

Figure 10 Student projects utilizing linear lighting to emphasize formal characteristics of the designs. Figure 11 In-Game captures of NPCs being used as filler assets suggesting functions in urban conditions. Figure 12 Student projects showcasing the misuse of virtual assets including furniture as utility assets to suggest function within a space as a compromise to developing an 130 | eCAADe 39 - CAAD and education - Volume 2 in-depth design. was to create a ‘flashy’ presence in wealthy districts on alien technologies and infrastructure, the fantas- where the design language followed a sleek, bright, tical approach offers unique opportunities for unre- and minimal aesthetic and relates directly to hyper alistic buildings for entertainment purposes that are wealthy districts within the game’s narrative (Scavin- unavailable to architects due to the constraints of sky 2021). budget, time, physics, and more. Video games fo- When examining the projects from the second- cus on stimulating people’s attitudes or behaviors year studio, it was found that 38.9% of students rather than understandings, thus producing persua- also used excessively reflective materials within their sive products of entertainment for interactive inter- projects. In addition, twice the amount of concert est (Bogost 2009). The result is games with worlds halls had excessive reflectivity compared to the pavil- that do not need logic but instead a fantasy to be ion projects. While some projects utilized glossy ma- immersed in. In contrast, architecture is bound by terials in similar applications to flooring, it was found the laws of reality and designs are subject to numer- to be misused more often in disregard for longevity ous physical factors that affect material choices, from in the structure to an overwhelming use of reflec- safety codes to climate to building detailing. Where it tivity in efforts to glamourize project facades and may be permissible for in-game buildings to contain spaces, ultimately falsely representing real-life mate- unspecified façade textures, architects cannot afford rials. In the examples shown of the exterior pavilion to be ambiguous in their architectural propositions. project, many instances of reflective tiling, flooring The student projects also portrayed various in- and wall textures are seen in the renderings used in stances of fictitious materials on architectural ele- conjunction to lighting conditions at night and dra- ments of focus, such as façade materiality and wall matic daytime light (Figure 7). However, reflective cladding. In the survey conducted, 35% of projects materials used in exterior structures where foot traffic seemed to showcase fictitious or unspecified materi- and weather conditions occur frequently are imprac- als. Many student designs were unresolved with re- tical due to wear over time and the requirement of spect to building construction, methodology, or as- maintenance becoming a costly to operate. In video sembly knowledge, leading them to bypass resolu- games, the thought of longevity, wear in spaces, and tion and apply ambiguous textures. As seen in the maintenance are rarely a main concern as a time- pavilion projects, various structures utilize distorted less medium is crafted to pull the user in with gaudy forms with continuous reflective materials applied as graphics, highlighting ray-trace technologies that vi- finishes, lacking precision to the constructability of sually replicate realism in efforts to enhance game- the projects. The lack of resolution from joint lines to play experiences. how materials seem to be glowing in daytime con- In addition to the aforementioned unrealistic ditions spawn confusion in the students’ design in- material properties, unspecifies or entirely fictitious tentions (Figure 8). Among the concert hall projects, materials permeate video game architecture. This many designs similarly glossed over some material phenomenon also emerges from games’ atheistic identification. In many cases, floor and ceiling ma- culture, as computer games are freed from the need terials were omitted, with students placing more at- for constructability. Most prevalent in games pro- tention on the walls. In the third project pictured in jecting future urban conditions, such as in Halo 4’s Figure 8, the student went as far as applying a black forerunner alien architecture. It is evident that in and white pattern to the interior walls of the lounge these designs, dubious materials and questionable space and on the glass, causing much confusion for structural assemblies are made obvious, even occa- viewers and occupants alike. Moreover, an exterior sionally glowing, reflecting their surroundings, and view of the same project reveals a discontinuity in floating (Figure 9). As Halo 4 is a video game based the material finish, suggesting an unresolved appli-

CAAD and education - Volume 2 - eCAADe 39 | 131 cation of the material in efforts of drawing interest in predominated by the Hollywood Illusionistic conven- the space. The use of the graphic applique becomes tion of motivated lighting (Brand and Knight 2003). a compromise to visually occupy unresolved spaces This desire to bring emphasis to one’s project within the design. Rather than emphasizing the spa- permeates through the surveyed students’ work, tial and tectonic components of the architecture, the with a total of 21% of projects using overly dramatic student resorted to interior decorating. lighting in their projects. The students used lighting primarily to highlight their designs within a darker Overlay of Dramatic Artificial Lighting and dull context, similar to the tactics used in the Animation, architectural visualization, and video games proper. One such example is in the project games all engage with the simulation of lighting pictured on the left in Figure 12, where the bright- within digital architectural environments, a difficult ness of the light emitting from the building brings and critical element in the representation of spaces. additional attention to the structure. As the graphic Lighting serves as not only a way to render a space possibilities of the software supersede realism, stu- in a photorealistic manner but is also a large factor dents are led to over accentuate their designs. Simi- of the tone and the emphasis in architectural repre- larly, the central image in Figure 11 presents a project sentation. Generally, in architectural design, lighting where lighting was overused to accentuate the de- is divided into task lighting, which aids occupants in sign and its occupants. Here, in addition to the highly performing specific tasks in brightly lit spaces, and contrasting light the project does not present any ambient lighting, which provides general light within emission sources for the lights, leading them to ap- the space. On occasion, architects are also able to en- pear to appear miraculously. This phenomenon was gage with decorative lighting which operates prin- observed frequently throughout the project set, as cipally to emphasize architectural conditions rather many students fail to appropriately design lighting than serving the building’s occupants. As video fixtures within their projects and so revert to simply games are neither constrained by user needs nor throwing spotlights onto their designs in the closing budgets, they are more prone to abstain from task moments. The final rendering in Figure 11 presents lighting and capitalize primarily on decorative light- similar situation, where colored light bathes the de- ing fixtures to light scenes. In addition, many games sign, but without any discernible source. need not even explicitly depict the source of the light, In particular, a popular lighting trend seems to and instead just light surfaces with the assumption have emerged from video game architecture. While of the presence of a fixture. One such game is Need previously, architects would not use linear LED strips for Speed Heat (EA, 2019), in which the player trav- to emphasize folds or curves, due to constrained els rapidly through urban conditions. The surround- lengths, cost, and lack of purpose. However, video ing buildings are lit up using numerous lighting types game designers were not limited by these issues and and in a range of colors, which highlights the con- began to make heavy use of LED strip lighting in or- text for the player, drawing attention to the infras- der to emphasize formal gestures. Such lighting can tructure (Figure 10). The forced emphasis with light be found in a multitude of games, from Halo to Cy- to contrast the dark portions of the screen guide the berpunk 77. Recently, this desire to emphasize re- viewer’s eye and guide players through the course. veals has permeated both into student projects as This bight lighting aids the game retain its players well as professional work. Of the surveyed projects, in dark conditions, aiding users to feel engaged with 16% contained linear lighting strips, all of which the game. A study conducted by Brand and Knight, were included by students to decorate their architec- of numerous platform games indicated that 73% of ture rather than serving any explicit function (Figure the surveyed games observed high-contrast lighting, 12). This lighting technique has also become popu-

132 | eCAADe 39 - CAAD and education - Volume 2 lar among built projects, with firms such as Zaha Ha- danger of using NPCs as filler assets is the easily de- did Architects and Partisans Architects making use of ceiving nature as a method to communicate func- long LED strip lighting to emphasize their curvaceous tions of a space rather than developing a design that designs. explains the use for itself. Another misuse of virtual assets is the use of fur- Asset Showcase niture as filler in the architectural student projects With the immersion of world building in video with 21% misuse in utility assets and 10% in misuse of games, the addition of assets and accessories in decorative assets. Similar to the suggestion of NPCs, the scenes are utilized as a secondary story telling much of the student designs use furniture to indicate element often referencing timelines, functions of the purposes of the spaces rather than designing for spaces, and aesthetic qualities. In many instances the the function. The problem can be seen in designs use of assets in video games becomes a filler of space, containing vast large rooms fitted with lonesome used as a tool for spatial direction in a player’s path or desks and coined as an office space. The concert hall to dress the background set of the world. Moreover, spaces prove to be more obvious, with some fitted the interactive aspects of assets become purpose- with only chairs and tables to suggest a restaurant fully used to guide players in advancing in the game, (Figure 15). Reliance on the furniture assets to de- with a majority of filler assets used only for orna- scribe spaces appeared in a third of the concert hall mental purposes. With the development of architec- design projects. Ornamental assets are also a com- tural visualization technologies, the techniques for mon use seen in the student work and raises ques- visualization are merging. As seen from the student tions of practicality and necessity. In an example with projects surveyed, assets are used in three distinct the plants located in the middle of the atrium bay categories: ornamental decoration, assets to suggest of one of the student concert hall projects, an inten- utility, and the population of characters, or NPCs. tion of decorating with green life is common practice In Spider-Man PS4 (Insomniac, 2018) the use of however, the excessive use of greenery can be seen NPCs serves as a means for the game designers to in- as an ineffective use of space within the building’s dicate the uses of spaces and to occupy the built envi- atrium (Figure 14). Without the assets, the spaces are ronment. The NPCs largely act as a filler in the game, diminished to a room empty of design and therefore as the principal Spider-Man character is rarely able reveal a reliability on assets as a shortcut method to to interact with them meaningfully. Cyberpunk 2077 design. has a similar use of NPC placement around Night City where massive crowds are seen roaming streets with CONCLUSION many loitering without interactive purpose. In addi- The influence of contemporary cultural values and tion, the NPCs are added to fill many spaces to feel media always permeate and are epitomized in ar- more visually lively. In student projects the use of chitecture. The rapid ubiquity of video games as a people is important to showcase the scale of the de- popular medium has followed an accelerated trajec- signs resulting to the addition of people within archi- tory similar to the influences of literature, cinema, tectural visualizations. In many cases NPC placement and television in architectural design. The inher- is utilized as a filler within the structures and can be ent commonalities and intertwined nature of archi- seen in the pavilion projects where the people sug- tecture and video games extend beyond tools and gest a concert in the structure yet without the NPCs, techniques and are experiencing a symbiotic evolu- the pavilions struggle to explain their function (Fig- tion. Video games as a medium are influencing not ure 13). Throughout the survey, 43% of projects had only the methods of production and presentation of issues of NPC placement in the visualizations. The nascent architects work, but also their design prece-

CAAD and education - Volume 2 - eCAADe 39 | 133 dents, priorities, and paradigms. Through this ini- tial survey of emergent architecture student design work, the authors present a taxonomy of features drawn from video games that pervade their work. Though this is not an indictment on the value of video games as a medium, the paper presents cau- tionary observations on issues that may prove to be endemic in future praxis.

REFERENCES Aguilera, M and Mendiz, A 2003, ’Video Games in Education’, Computers in Entertainment, 1(10.1145/950566.950583), pp. 1-10 Annetta, L 2008, ’Video games in education: Why they should be used and how they are being used’, Theory in Practice, 47(10.1080/00405840802153940), pp. 229-239 Ataman, O 2000 ’Measuring the Impact of Media on Ar- chitectural Design’, 4th SIGRADI Conference Proceed- ings, Rio de Janeiro, pp. 93-97 Bogost, I 2009, ’Persuasive Games: The Expressive Power of Videogames’, American Journal of Play, 2, pp. 230- 232 Brand, J and Knight, S 2003, ’The diverse worlds project: narrative, style, characters and physical world in popular computer and video games’, Digital Games Research Conference Guttler, C and Johansson, T 2003 ’Spatial principles of level-design in multi-player first-person shooters’, NetGames ’03: Proceedings of the 2nd workshop on Network and system support for games O’Callaghan, J 2017, ’Architecture as Commodity, Archi- tects as Cultural Intermediaries: A Casey Study’, Ar- chitecture and Culture, pp. 221-240 Vitruvius, MP and Morgan, MH 1960, The Ten Books on Ar- chitecture, Dover Publications, New York Willatzen, J and Karakurt, E 2020, Competitive Level De- sign: A study on Counter-Strike: Global Offensive Level Design, UPPSALA UNIVERSITET Yildiz, S and Yildz, E 2011, ’A study on pc - Video games in terms of the space awareness from childhood to youth’, Procedia - Social and Behavioral Sciences, 28, pp. 796-800 Zumthor, P 2006, Thinking Architecture, Birkhauser [1] Scavincky,R.2021,‘ArchitectureCriticonControl-it’sm orethanBrutalism!’,availableat:https://www.youtube.co m/watch?v=4gbWOhxTpr8&t=724s

134 | eCAADe 39 - CAAD and education - Volume 2 Digital fabrication and robotics 136 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Prvok An experiment with 3D printing large doublecurved concrete structure

Kateřina Nováková1, Jiří Vele2 1,2CTU Prague, faculty of achritecture 1,2{bruhokat|velejiri}@fa.cvut.cz

In this experimental research project we report on the manufacturing process of the first full-size 3D printed concrete structure in our country. The house was 3D printed by an ABB IRB 6700 robot whose range we made fit with the requirements for transportation size and also, its range determined the size and geometry of the house. During the transformation process from sketch to code we involved students to apply computational design methods. We designed the main load bearing structure which had to be thinnest and lightest possible together with its insulation features and printability. We were aware of the world-wide research in this field started by NASA centennial Challenge called 3D-printed-habitat [Roman,2020] as well as start-ups derived from this research [1,2,3,4]. During the project, we investigated the following matters: (1) the relationship between geometry of the wall in model and in practice (2), setting of the robot and the mixture; and (3) stress test of the wall. With the results of the test we aimed at contribution to standardisation of 3D printed structures in ISO/ASTM 52939:2021. The finalized structure, named ``Prvok'', was made to prove printability of the mixture and stability of the design.

Keywords: 3D printing, robot, concrete, grasshopper, experiment, house

INTRODUCTION tings of the ABB robot hardware site and experiment- We describe Prvok (Fig. 1), an experiment of 3D ing with wavy samples. Our manufacturing site will printed load bearing structure in form an inhabit- be described, where the settings of robot, mixer and able floating object made of cement-based mixture. pump influence the shape/size possibilities of the In order to describe the processes, we use the term printed object. The house had to be manufactured buildability for describing behaviour of the mixture very fast, even though its geometry was “sculptural” in terms of printability and pumpability. Our system and thus not suitable to be constructed from regu- for 3D printing is fixed robotic hand, being inspired lar ready-made prefabricated panels. First hypothesis by Gosselin et al., 2015 or CyBe, 2018. Among many of our experiment was refuted. We anticipated that others we also followed experiments made by Lis- a digital model of the house can bring us improved bon School of Architecture (Craveiro, 2020) with set- time/cost savings: not only customisation and com-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 137 Figure 1 Prvok - the f3D printed floating house

plexity comes at no extra cost, but also the exterior Having read articles about sample small scale test and interior can be manufactured at the same time. printing, we also present issues, which surprised us Furthermore, the aesthetic of the outer wall was im- when printing large irregularly curved walls. portant because it stresses the possibilities of addi- tive manufacturing method. By avoiding moulds we MATERIALS saved time and expenses connected with manufac- The use of additive manufacturing in building indus- turing and removing complicated forms. Last but try is a recent phenomenon of the last decade. (De not least, optimised shape of the construction re- Schuttera, 2018) points out that in 2018, no regu- duced energy losses (Lowkea, 2018) and helped sta- lar working mixtures for AM were available on the bility of the wall while constructing it by 3D printing. market. This gave rise to many developments of Finally with the presented shape we achieved small own printable cement mixtures, such as CYBE [5], Be- overall thickness of the wall together with preserv- more3D [6], Apis Core [7], etc. We already experi- ing its static and thermic qualities. The design of the enced the 3D printing smaller objects, but for such wall also saved the material and thus had good envi- a large object as Prvok, we needed to test and verify ronmental impact. The prefabricated cement com- the few mixtures for 3D printing available in Czech posite we used to print both the interior and exte- republic. The first material we tested was Weber - rior wall was developed and promoted in the frame- Beamix [8]. In June 2019 the Saint Gobain company work of this project together with the company Mas- from Netherlands addressed us to test their mate- ter Builder Solutions (MBCC). While the load bear- rial developed together with Eindhoven University ing construction and envelope was manufactured by of Technology and planned for project Milestone [9]. the robot, the interior panels for floor and ceiling Initially, the material was too thick for this type of 3D were CNC-cut from ply-wood. Second hypothesis we printing. After successful printing with adding wa- wanted to prove was the buildability, printability and ter and polypropylene fibres manually, we observed processing stability of the double curved walls. These cracks on the ready samples after only 3 days. Get- were designed by students in Rhino-Grasshopper. ting the right constitution proved also difficult, either

138 | eCAADe 39 - Digital fabrication and robotics - Volume 2 the sample was dry with immediate cracks or liquid. Robot ABB IRB 6700 The next material was from ERC Tech company, con- The ABB robot we used is ABB IRB 6700 with con- sisting of recycled concrete mixture for 3D printing troller software IRC5. Manipulator provides sufficient [10]. The mixture proved printable, but the it had too reach of 3,2 meter and handling capacity of 150 kg. much variation of liquidity and lacked stability. Sim- It can turn 340° around leaving a 20° blind angle ilar results were obtained from printing with Isotrop- [14]. Robot programming was done in Grasshop- ico Rapido material from private Czech Mateo com- per plug-in Robots.We found out that the robot con- pany. We also tried to print quick-setting and hard- troller cannot handle the large amounts of data for ening, anti-shrinkage hydraulic cement binder mate- move instructions. For 3D printing large structures rial called Lampocem form Mapei [11]. Keeping the we used approx 50 000 lines of move instructions, recommended setting caused cracking when drying, but the touch pendant froze around command 10 because the overall volume was too thick for this ma- 000, meaning that we could affect robot speed with terial and the drying of the surface was too quick. Fi- the touch pendant when we had a larger number of nally Master Builder Solutions (MBSS) company ad- command lines. As a result, the robot kept work- dressed us for a cooperation in testing the printability ing without us having the possibility to control it. of their mixture [12]. It had lack of solid particles big- We tried out two options to prevent this problem. ger than 0,5mm and thus it was not fulfilling the crite- The first and easiest is to divide the print object into ria of concrete. Nevertheless, MBSS introduced a pre- smaller pieces. The second is to run a program that fabricated UHPC mixture with polypropylene fibres, splits printing instructions into smaller sets of data plasticizers and fasteners incorporated and called it and feeds the computer of the robot with these sec- Master Flow 3D 100. The final version of this material tions. This allows it to print large pieces without hav- had following satisfying qualities: ing divisions. With the aim at prolonging the printing Granularity up to 0,5mm Processability - time in order to let out mixture stiffen, we chose the up to one hour / 20°C3,9 liter water per 25kg first option. Even then, with reduced amount of data, bag of the mixtureMinimal operable temperature the upload of the data into the robot still takes about +5°CDurability: 6 months 10 to 15 minutes.

METHODS Description of the manufacturing site For our task of manufacturing the house we chose Preparation of the site before printing itself is vital for a fixed Robot ABB. [13]. 6-Axis industrial robots of- seamless concrete 3D printing. The type of machine fer flexibility, precision and reliability. Thanks to used for printing plays a crucial role, especially its po- their mass production they are also relatively cheap sitioning on site and setting of other helping tools in comparison to other concrete printers. Many such as the pump and mixer. There were many issues universities researched and developed softwares for on the transport of the material, starting from the well known industrial robots brands to use them bag, to the mixer, and then by a tube to the robots’ as CNC machines, 3D printers, interactive machines, nozzle. Another important aspect is the storage of etc. That makes them well established in the field of the printing material. Since everything was new for 3D concrete printing. The availability of ABB Rhino us for such a big project, we worked on purpose with grasshopper plug-in helps architects to transmit the manual control over the system. This allowed us to model into code. understand its behaviour in order to possibly auto- mate it later. We experimented with a manual mixer and a screw pump. This solution offered visual con- trol of mixture during the printability tests. However,

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 139 the manual service of the machines was demanding uid than the other. This affected the structural stabil- and stressful. When we were confident about the ity and also lead to an imprecision of the wall, of up to amount of added water, we tried to automate this 0,5 cm. These aspects should be taken into account process as much as possible and tested the M-Tec during printing. The so-called printability window Duo mix, which mixed dry mortar with water and also represents the longest time when a mixture can be had a screw pump to pump it into the hose. Unfor- printed with acceptable print quality. It is related to tunately the tiny 20mm long polypropylene fibres of the blockage limit, which is the longest time when a the mixture kept clogging the machine, which pre- mixture can stay in a pump, hose, nozzle before clog- vented continuous printing. We ended up with the ging. Buildability is the ability to resist deformations mixer M-Tec D30 and pump M-Tec P50 which proved during printing. to be the most reliable solution.We worked inside a warehouse with a portal crane that transported both DESIGN the robot and the bags of material above the mixer. This reduced manual labour by half. We used a 10 Figure 2 meter long hose. In our open-space test lab we were from the left: curves not having control over air temperature and mois- as attractors, ture. By test-printing we found out that the mixture preliminary result, won’t stiffen at temperatures below 15°C. During the chosen pattern print of Prvok, we struggled with the decrease of tem- used in Prvok perature inside of our hall and we had to regulate it Students were asked to create 3D printable designs, with mobile gas heaters. taking into account the shape freedom of irregular- ity, and referring to water as an inspiration (since Pr- Printing conditions vok was conceived as a floating object on the water). The print quality is closely connected with the visual They had to design a wall surface and structure which appearance of printed material, which ideally should fulfils both artistic demands and stability functions. have no cracks. The printed layers should have similar (Fig. 2) A code for detecting overhangs of the sur- thickness (Kazemian, 2017). The print quality is very face together the length of the of the single “over- sensitive to temperature and humidity fluctuations hanging” 3D printed row had to be scripted and re- (Papachristoforou, 2018). We found out that it was spected by the resulting design. Direct paths over possible to 3D print with our mixture only at temper- 90 cm in length repeating more than 20 times above atures from 18 to 25 degrees and humidity of 40 to each other had to be avoided. 60%. If temperature dropped under 15 °C, or humid- Figure 3 ity rose above 80%, the activator inside the mixture From given shape did not start chemical reaction. In these conditions of the building the concrete was liquid and there was a high chance trough attractor of collapse during the printing. If temperature rose definition to a above 25 °C, the material hardened in the hose and curved surface caused clogging. Humidity below 40% caused incor- rect concrete curing and the final object had lesser strength. The print material was very sensitive to added water. We noted that the material M-Tec D30 Given the floating objective, the wall surface should was inconsistent with respect to adding water, caus- resemble waves, drops or any kind of water state ing fluctuations of one layer to be dryer or more liq- of matter. Second, the surface should be double-

140 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 4 curved with an overhang up to 13° in order to achieve Division of the better stability during the printing process. We knew model into pieces from previous testing, that our imperfect printing printed separately processes can cause collapse at straight surfaces. Stu- dents were given the basic shape and dimensions. With additive manufacturing the use of parametric architecture is quite clear, since the code can be transmitted through an ABB plugin directly into the code for robot production. At the start, the students were introduced to cod- ing in Rhino Grasshopper. The effect of the participa- tion on a real project made students design possible solutions within one or two lectures. Attractor and sweep functions were used the most (Fig. 3). Begin- ning with a simple attractor effect, in order to achieve a wavy surface, whole curves were manipulated ac- Elliptic windows are printable up to ¾ of their height cording to the proximity to the points using the at- without any support. Firstly, we aimed at the use of tractor. As an end result the surface was modified us- adaptable mould developed by Schipper (2015). The ing both manipulated curves and attractor points. robot ran always one side at a time, offering us the most control of the path. This was so due to tiny un- predictable irregularities caused by liquidity of the PRODUCTION mixture, the cast had to be of variable position and Prvok was printed in the city of Budweis (150 km from always respect the actual thickness of the path of the Prague) and then transported to Prague by long ve- robot, which had a measured imprecision of 2,5 cm. hicle transporters. This ]limited the building width At the top part we casted the openings layer by layer, to 3,5 meter. Thus the concept of the house was in order not to block the robots’ movements (Fig. 5). designed to have two 3D printed load bearing con- Figure 5 crete walls divided by a wooden middle piece. Its Casting of the 13,5 meters of length was split into 3 parts and trans- window, ported separately. The height of the structure was preparation of the out of reach of the robot, which we solved by placing casts (left) to ready it on a special produced steel table one meter above opening (right) ground. This allowed us to manually prolong its noz- zle with 1 meter. As soon as the robot printed the first 1,5 meter height of the walls, we reduced nozzle DESIGN VERSUS REALITY length. The geometry of the model does not fully reflect As mentioned earlier, to reduce data amount, the manufactured reality. Wolfs (2018) states that which blocks the process from continuity (Borg Con- his model of the structural behaviour analysis suf- stanze 2018), we split the walls into ten print parts fers from geometrical and material imperfections in- (Fig. 4). This speeded up the process of data prepara- duced by the printing process. We found out that tion and reduced upload time. Different script were also the coded behaviour of the robot itself does not used for parts 2 - 9, which eliminated heat bridges fully respect the given design. around windows.

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 141 5.1 Between design and code 5.2 Between code and reality General assumption, that 3DCP can print objects that correspond with its 3D models, might be mislead- Figure 7 ing. During conversion of the 3D model into code Fly by point in there are two steps, where shapes of the 3D model sketch, Fly by point are slightly modified. First conversion happens in in reality Grasshopper. For each curve a specific number of points is distributed equally. Planes are assigned to those points and a polyline is created, which is to be followed by the robot. Number of points used should be adequate to the complexity of the geometry. Too Inconsistency may also happen during the print. many points generate excess number of move in- Ideal thickness of our printed wall was 4,5 cm (Fig. structions for the robot and slow down the process of 7). But if water mixing was inconsistent, some layers data preparation.Robot composes trajectory in three might have been drier, causing the final layer to be different ways (move), but for 3DCP only MoveL is ap- thinner, up to 3,5 cm. More liquid mixture extruded plicable. MoveL command stands for Move linearly faster, more material was deposited and therefore to a specified target. The robot has to stop at each the layer was thicker, extremely up to 6 cm. When point and change its trajectory in order to follow the irregular to one side the 3D printed walls also incline exact polyline path predefined in Grashopper. That is to tiny deformations before concrete starts gaining not possible for 3DCP, where the robot has to move its strength. Fluent movement from one layer to an- with as smallest velocity changes as possible. Con- other also influences geometry. Closed curves are stant speed of the robot is crucially important, be- printed more equally when material at every point on cause the amount of pumped material is the same the curve is deposited at the same time in the conse- all the time. To move regularly, the robot blends cor- quent layer. Each closed curved layer of Prvok took ners and flies-by defined points (Heim, 1999). Each approximately 3 minutes to print. Open curves have point has a circular horizontal zone, which robot has a small curing window and the next layer is deposited to cross. The bigger zone the radius has, the smaller immediately at the points above each other. Irregu- change of the velocity is. But it also causes greater lar squeezing of consequent layers occurs .Therefore deflection from the designed path (Fig 6.1). We set one layer is squeezed more than another. Left pic- zone around programmed positions to 15 mm. ture below shows a squeezed layer on top of which was immediately deposited next layer. Then it took Figure 6 2 minutes for a printer to come back, therefore the From the left: second layer had enough time to solidify and didn’t Geometry squeeze. respecting the range of the robot STRESS TEST and size of the Static load test of compressive strength and test of transportable piece, stability of load-bearing structure were made at the elevated robot with faculty of civil engineering in Prague. A real scale prolonged nozzle, model of the structure was printed on 2.4.2020 and elevated robot with tested on 7.5. 2020. The test was dome by by shortened nozzle 300mm long hydraulic cylinder INOVA AHS 630-300 with max. press force of 630 kN and tension force of

142 | eCAADe 39 - Digital fabrication and robotics - Volume 2 130 kN. Although preliminary static calculation was 3D printed structures were missing. With the results made with satisfactory results for the location of mid- of the testing at Experimental centrum at Fsv CTU we dle Europe (29kN), exposing the structure in public contributed to standardisation of 3D printed struc- was not possible without the press experiment prov- tures in discussions about ISO/ASTM 52939:2021. ing the stability. Relying on calculation is also impos- Due to missing commercial services in our coun- sible due to the lack of standards for 3D printed con- try we successfully engaged students in the design crete structures. Potentiometric sensors with a pulley and production process of a commercially built struc- with a path of 100 mm and a cable with a diameter of ture using the novel technology of 3D printing with 0.2 mm connected as part of a chain for measuring Master Flow 100 prefabricated cement mixture. We the path were used to measure deformations. The found no professional architect being able to design DEWE5000 measuring control panel, which we used, 3D printed doble-curved structure for 3D printing is intended for measuring the static and dynamic be- concrete together with its preparation for the robot. haviour of monitored objects (Fig. 8). Since the funding was limited, no professional 3D printing company could be invited from abroad to Figure 8 manufacture the designed structure. We hope the From the left: act of showing the possibilities of the technology horizontal deviation opened a deep discussion about the implications of mesuring unit, steel 3D printing concrete in the Czech republic. Prvok was frame distribiting publicly exhibited for several months with the pos- the load, Measuring sibility of exploration by lay people. Hereby we in- switchboard creased the trust of public in this technology. We DEWE5000 with also hope we have increased the attention of Czech microcomputer Prvoks inner wall bearing the roof was loaded with universities with the importance to invest in com- 500kN - the maximum capacity of the INOVA AHS putational designing in pedagogy. A novel mixture 630-300. We could observe the maximum of verti- was developed throughout the project, which has cal deformation of 2,36mm (Fig. 9). The static load now been approved to the market thanks to our test- test proved the load-bearing capacity and stability ing of printability. We refuted hypothesis n.1. The of the printed PRVOK structure even under the sim- real outcome does not fully reflect the geometrical ulated snow load for the Prague area and even when model due to two factors: inconsistency of printing this load was increased by a random load. The sta- quality and the robot not exactly following the path. bility and load-bearing capacity of the structure was Thus, the real outcome has deviation from the model proven even with more than ten times overload of geometry on an average deviation of 2,5 centime- the required load-bearing capacity. tres. The production of interior equipment of the house is recommended to be only done after the 3D Figure 9 printed walls are ready. To obtain the precise dimen- Graph of sions of the print, it is possible to use the very same deformations robot equipped with a 3D scanner. Hypothesis n.2 was successfully proven. Additionally we developed a method for circular openings with the use of cheap reusable polystyrene moulding. Finally, stress tests were made that proved the durability and stability of CONCLUSION the construction. The experiment was very novel in terms of testing 1:1 model and proving its stability where standards for

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 143 FUTURE WORK crete construction – Possibilities and challenges’, Ce- Together with the students we are engaged in ment and Concrete research, volume 112, pp. 50-65 a longer term observation of the cracking phe- Papachristoforou, M, Mitsopoulos, V and Stefanidou, M 2018 ’Evaluation of workability parameters in 3D nomenon, which is natural behaviour of non- printing concrete’, Procedia Structural Integrity 10, reinforced concrete. Each student had to identify pp. 155-162 a potential crack and install a plaster target of 2 x 2 x Roman, MC, Fiske, MR, Nazarian, S, Yashar, M, Ballard, J, 0,3 centimetres over the risky areas. We are running Bentley, M and Adams, AM 2020 ’3D-Printing Lunar these measurements since October 2020 and the re- and Martian Habitats and the Potential Applications sults are awaited in October 2021. Future research for Additive Construction’, International Conference on Environmental Systems, Texas will be devoted to study of the influence of the ge- Schipper, H R 2015 ’Double-curved Precast Concrete El- ometry of the wall on frequency on the cracks and ements - Research Into Technical Viability of the their frequency depending on their position on the Flexible Mould Method’, InstitutionalRepositoryofTU walls. Next, we will study how thermal conductivity Delft reflects various geometrical versions of the wall to- De Schuttera, G, Lesagea, K, Mechtcherineb, V, Nerellab, gether with various insulation materials. We want to VN, Habertc, G and Agusti-Juanc, I 2018 ’Vision of 3D printing with concrete — Technical, economic and find out how the single curved and double curved environmental potentials’, Cement and Concrete re- surface influences the thermal behaviour, and how search, volume 112, pp. 25-36 much the reality differs from simulation calculation. Wolfs, RJM, Bos, FP and Salet, TAM 2018 ’Early age me- Last but not least, we will perform fire test on the wall chanical behaviour of 3D printed concrete: Numeri- samples and compare them with walls of a different cal modelling and experimental testing’, Cement and geometry in the next generation of Prvok. Concrete research, volume 106, pp. 103 - 116 [1] http://www.ticarchitecture.fr/index.php/habitats/yh nova/ REFERENCES [2] https://www.dezeen.com/2018/06/04/eindhoven-u Borg Constanze, C, Ahmed, Z Y, Schipper, H R, Bos, F P, niversity-technology-project-milestone-3d-printed-con Knaack, U and Wolfs, R J M 2018, ’3D Printing Con- crete-houses/ crete on temporary surfaces: The design and fabri- [3] https://www.3dnatives.com/en/apis-cor-3dprinted- cation of a concrete shell structure’, Automation in house-060320184/ construction, 16, pp. 395-404 [4] https://www.pci.org/PCI_Docs/Papers/2018/32_Fin Craveiro, F, Bartolo, PJ, Bartolo, H, Duarte, PJ and Nazar- al_Paper.pdf ian, F 2020 ’An automated system for 3D printing [5] https://cybe.eu/technology/3d-printers/ functionally graded concrete-based materials’, Addi- [6] https://bemore3d.com tive Manufacturing 33 [7] https://www.apis-cor.com Gosselin, C, Duballet, R, Roux, PH, Gaudillie฀re„ N, Morel, [8] https://www.cz.weber/zakladni-omitkove-maltove-s Ph and Dirrenberger„ J 2016 ’Large-scale 3D print- mesi ing of ultra-high performance concrete – a new pro- [9] https://www.saint-gobain.com/en/will-printers-buil cessing route for architects and builders’, Materials d-homes-tomorrow and design, volume 100, pp. 102 - 109 [10] https://www.erc-tech.eu/ercconcrete/ Heim, A and Von Stryk, O 1999 ’Trajectory Optimization [11] https://www.mapei.com/it/en/products-and-soluti Of Industrial Robots With Application To Computer ons/products/detail/lampocem Aided Robotics And Robot Controllers’, Optimization [12] https://www.master-builders-solutions.com Kazemian, A, Yuan, X, Cochran, E and Khoshnevis, B 2017 [13] https://new.abb.com/products/robotics/industrial- ’Cementitious materials for construction-scale 3D robots/irb-6700 printing: Laboratory testing of fresh printing mix- [14] https://fzi-robot-interface-proposal.readthedocs.io ture’, Construction and Building Materials 145 /en/latest/industry/ABB.html Lowkea, D, Perrotc, E, Wegerd, A, Gehlend, CH and Dil- lenburgere, B 2018 ’Particle-bed 3D printing in con-

144 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Research on Realtime Communication and Control Workflow with Vision Feedback Integrated in Robotic Fabrication

Ming Lu1, Xinjie Zhou2, Yifan Zhou3, Liming Zhang4, Weiran Zhu5, Philip F. Yuan6 1FabUnion Architectural Technology and Digital Fabrication Co., Ltd 2,4,5,6College of Architecture and Urban Planning, Tongji University 3School of Architecture, University of Hawai'i at Mānoa [email protected] 2,4,5,6{2032187|1810140|huweiran|philipyuan007}@tongji.edu.cn [email protected]

On-site construction is one of the main research directions of robot construction. Due to the complex and everchanging construction environment on the site, traditional offline programming and simple conditional programming cannot meet the needs of robot on-site construction at all. Realtime adjustment of the robot's operating program for the real-time situation in the field is the appropriate solution. Therefore, the real-time communication and control of robots has become a key issue for robot on-site construction. This article discusses in turn the way of robot offline program control through EthernetKRL and grasshopper. A remote real-time communication and control method for an on-site construction robot is studied, and the application of the method in the on-site construction process of an actual robot is introduced with the Wuzhen coffee kiosk project as an example. Based on the above methods, remote real-time monitoring of the construction robot on site can be realized, which provides a reference for the actual engineering application of the construction robot on site.

Keywords: on-site, robot, fabrication, communication, sensor

1 INTRODUCTION ture researches with the requirements of the growing 1.1 Background construction industry. Among the on-site applica- Since the early 1980s, on-site applications of robotics tions, mobile robot on-site construction is one of the have remained an evolving research topic. Beyond main branches of robot construction. The unstruc- the economic and sociological challenges brought tured operating environment at the construction site by the innovative process, major technical issues still has become the source of many problems in mobile need to be noted in order to compare the past and fu- robot field construction research, and the represen-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 145 tative ones are: that must be solved in the field robot con- struction research. Realtime communication be- 1. The contradiction between the excessive data tween robots and between the robot and the processing brought by the environmental per- control server is the basis for this process. ception of unstructured construction operations and the mobility and miniaturization of con- struction robots on site. The rationalized per- 1.2 Motivations ception of unstructured environments has not The real-time communication and control workflow yet matured and perfected solutions. The un- of the field-built robot proposed in this paper is structured environment of the building con- based on our existing robot field operation practices. struction environment is very high. Therefore, In the existing practical experience, In the existing the rationalization of the construction environ- practical experience, the biggest challenge is the on- ment at this stage cannot be completely depen- site positioning and post-positioning program ad- dent on the robot itself. The intervention of justment after the on-site construction robot moves. an appropriate amount of manpower and other This process mainly involves the positioning of the external equipment is a more feasible solution. robot and the world coordinate position, and the pro- On the other hand, in order to achieve flexible gramming simulation of the robot’s target path. The operations on complex construction sites, on- former can ensure that the robot can complete the site construction robots require higher mobility task accurately, and the latter can ensure the safety and miniaturization. Therefore, the robot itself of the robot during the construction process. In the cannot carry other human and external equip- previous practice, we mainly adopted two methods: ment. It can be seen from this that the real- 1. Redefine the way the robot defines the base co- time communication between the on-site mo- ordinates in the field. When the robot arrives bile construction robot and external human and at the working position, the base coordinates equipment is very important; of the task to be completed are defined accord- 2. The need for realtime adjustment of operating ing to the reference point of the scene environ- procedures brought by the variability of operat- ment, and then when the operator performs the ing conditions. The working conditions at the robot programming, the program will be writ- construction site are variable, and the traditional ten according to the redefined scene polar co- offline programming robot operation program- ordinates. The advantage of this method is that ming method cannot meet the adaptability re- the process is simple, and when the program is quirements of robot field operations. Therefore, finished, the security of the code can be verified the robot needs to adjust the operation program immediately. However, its disadvantage is that according to the external situation in real time. it requires the field operators to have high robot The adjustment of the operating program often operation ability and robot programming abil- requires the integration of multiple external in- ity. In a large scale robot field operation environ- formation, so the general operating program ad- ment, it is not realistic to equip operators with justment instructions are also issued by the ex- the same number of robots; ternal controller, and the robot needs to receive 2. Perform program output in advance, that is, and execute in real time; perform reasonable position and attitude plan- 3. The need for collaborative control when work- ning on the robot’s operation points, and output ing with multiple robots. There are many types the robot operation code and the preplanned of construction site operations, and multi-robot Base coordinates at the planned points. Dur- cooperative operation is one of the problems

146 | eCAADe 39 - Digital fabrication and robotics - Volume 2 ing the actual construction of the robot, the on- its field operations. Similar concepts apply to most site construction robot moves to the designated on-site construction. This article will introduce the planned position, and the relative relationship research of real-time communication technology un- between the robot and the working position is der this concept , then use a practical case to illustrate measured by the total station (Fig. 1). After the application of this workflow in actual engineer- modifying the Base coordinate value and check- ing. ing the safety through a preset program, per- form the robot’s on-site operation. Although 2 COMMUNICATION METHOD this method simulates the robot’s motion path in 2.1 Communication of KUKA the planning and actual implementation stages, Based on the above ideas, the communication func- this simulation can be performed by a preset tion of the robot is needed. Communication is an program. In the existing project practice, us- essential function in some construction projects that ing this method can make a professional opera- carry sensors and require interaction. Almost all large tor perform the program output and verification robots, such as KUKA, ABB have their own communi- of 7 robots, and the remaining human-machine cation functions. collaboration can be completed by ordinary op- This article takes the KUKA robot as an exam- erators. ple. KUKA has three communication methods, which are RSI [KUKA RSI,2009], mxAutomation [KUKA mx- Figure 1 Automation,2018], and EthernetKRL [KUKA Ether- On-site robotic netKRL,2016]. fabrication with RSI has high requirements on the real-time per- electronic total formance of communication instruction input, and station. it needs to input strictly according to a certain fre- quency to work. This requires the cooperation of real-time operating system. Additionally, RSI requires the upper system, like Grasshopper, to send mo- tion commands every 4ms or 12ms, which means each brief motion commands need to be divided into thousands of interpolation commands and send one In contrast, the second of the above two methods is by one to the robot. This obviously makes it much more efficient and feasible. The shortcoming of this harder for naive users. There’s another risk that KUKA method is that the data exchange between the robot, robots are completely controlled by the upper sys- the total station and the computer for field program- tem in RSI mode, and the teach pendant also lacks ming is done manually, which greatly reduces the ef- the setting operations for speed control. Therefore, ficiency of the entire workflow. Therefore, based on if the robot moves too fast, it may cause safety acci- the second mobile robot construction method and dents. robot real-time communication technology, we con- The mxAutomation does not have high require- sidered the establishment of the following workflow: ments for real-time performance, but requires the co- After the robot moves to the working position, the operation of a PLC or soft PLC to run, and ethernetKRL total station transmits the positioning data to the does not have the above-mentioned difficulties. It is programming computer, and after the programming a communication method that does not require high computer adjusts the program for the robot position, external hardware. It only requires a computer. With- then send realtime instructions to the robot to guide out the real-time nature of the above two methods,

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 147 fortunately, real-time nature is not a primary issue in Now you can use the online transmission in- most application scenarios of robot construction. structions to send instructions to the robot through Most of the ethernetKRL is used only to transfer grasshopper. In this way, you can print unlimited sensor data to the robotic arm. If you want to trans- commands theoretically, regardless of the storage mit command data, there is no ready-made solution size limit. for ethernetKRL. The only thing that can be found is The specific process is: first, a large number of a processing-based example [Gordon 2018]. Obvi- program instructions need to be sent, and the first ously, it can’t be required because we want to use one is sent in order. After the robot executes the grasshopper to do robot control. first instruction, the robot will send the current status So we write a src program residing in the robot message to grasshopper. Knowing that the robot has controller, and the corresponding grasshopper plu- just completed an instruction, it starts sending new gin to achieve two-way communication, this plug- instructions to the robot. in has been integrated into the FUROBOT [FUROBOT Due to experimental time constraints, we tested 2020]. that more than 9,000 instructions were sent and ex- The ethernetKRL communicates in the following ecuted automatically in sequence. We believe that processes: more instructions will not be a problem. There’s Run the src program we wrote on KUKA. This pro- no problem with this method to achieve various 3D gram communicates with grasshopper, accepts in- printing tasks. struction data from grasshopper, and parses the re- ceived instruction data into robot instructions and 2.2 Sensor makes the robot run this instruction immediately. Af- The simple command data and robot state data can- ter the robot runs this instruction, it will collect some not meet most of the requirements. In most cases, it data of the robot itself and send them to grasshop- is necessary to combine sensor data. per. These sensors can be fixed to the end of the Grasshopper then began to accept xml data from robot or outside the robot. If it is fixed on the robot, the robot’s state parameters and parsed it. Gener- because the distance is relatively short, the sensor ate new instructions based on the receiving data, and data can be transmitted to the robot. However, this then turn the instructions into instruction xml data method is more troublesome. It needs to process and send them to the same robot (Fig. 2). the sensor data inside the robot. Each robot is pro- So, the robot started to read the instruction xml grammed differently, and the sensors required for data again, and so on. each project are different. It also increases the work- load of programming. The received robot data needs to be sent back to Grasshopper for processing, so we Figure 2 choose to send the sensor data directly to Grasshop- Exchange data per. The advantage of sending the data collected by the sensor directly to grasshopper is that there are some off-the-shelf solutions. For example, firefly can collect sensor data from arduino, even without ar- Experiment.. When performing large 3D printing, duino. The sensor can directly connect to the com- due to the storage limitation of KUKA’s controller, of- puter where grasshopper is located, and then write fline files larger than 10mb cannot be accepted. At the components from USB Read the sensor data dur- this time, we generally use segmented printing to ing the connection. Most sensors can use the above split the file.

148 | eCAADe 39 - Digital fabrication and robotics - Volume 2 two methods to solve communication, but the use of between the robot and grasshopper is used to trans- vision sensors is slightly different. mit the sensor data, it is a feasible solution, but the same problem still exists, different sensors and differ- 2.3 Vision sensor ent project requirements. So, we have to take a new The vision sensor generally uses a USB camera. It is approach to achieve complete wireless communica- different from ordinary sensors in that the vision sen- tion. sor is used in a wider range, and the amount of data We use the ROS system as a bridge between the transmission is larger than that of most sensors. It sensor and grasshopper to achieve wireless commu- is very difficult to transmit images with ethernetKRL, nication between them. because the communication protocol does not have The biggest function of ROS system is to man- a good support for image transmission. age the communication between nodes. Here differ- So, we connected the camera directly to the ent sensors correspond to different topics, and other computer where grasshopper is located. And devel- nodes can obtain sensor data by subscribing to top- oped some grasshopper components to get and pro- ics. Because ROS is generally used on Linux, you can cess images. The basic process is to first connect run a ROS system on a Raspberry Pi or a small indus- the camera to the computer, then start grasshopper trial computer. They are relatively small and can be to obtain real-time images, and then perform corre- installed on mobile robot devices(Fig.3). sponding processing based on these images, such as Figure 3 basic binarization, reverse, filtering, etc. Finally, the ROS collects sensor key information of the image is obtained, and the cor- data , and send responding robot instructions are formed, which are them to sent to the robot through grasshopper. grasshopper. 2.4 Wireless communication In most industrial and prefabricated robot construc- tions, communication relies on wired network con- nections, but in the construction of buildings, espe- We connect the sensor to the device where ROS is lo- cially when using mobile robots, wired connections cated, and then use rosbridge to send the acquired may not be convenient. We tried the wireless con- sensor data to the specified port in the form of JSON nection of the robot in the same local network, and text. Then we use bengesht [Behrooz,2019]to get verified that both the robot and grasshopper could this data. We tried sending laser scanner data to send and receive instructions like a wired connection. grasshopper (Fig. 4) Generally speaking, sensors are mostly fixed on Figure 4 mobile robots. If the sensor is directly connected to Data transfer of the computer through a USB cable, even if a wireless laser scanner connection is used for bidirectional command data between ROS and transmission at this time, it cannot be called remote Grasshopper control at this time, because the computer and the robot still have a USB cable connected In addition, We found that the video data transferred to due to the limitation of the USB cable length, this pre- grasshopper using rosbridge was difficult to con- vents the computer from being too far away from the vert from JSON text to images. We then adopted mobile robot. another ROS function package: web-video-server. We have found that if the sensor is directly con- Using this function package, we sent the image data nected to the robot and the wireless communication

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 149 through the http server. Then we wrote a plug-in to ages, and then corresponding processing is per- receive this image data. The received image can still formed according to these images, such as basic bi- be processed and extracted by the image process- narization, inversion, filtering, etc. Finally, the key in- ing plug-in we wrote previously, and finally generate formation of the image is obtained, and the corre- instruction data to send to the robot (Fig. 5). sponding robot instructions are formed, which are sent to the robot through grasshopper. In this ex- periment, we used FUROBOT to communicate with Figure 5 KUKA, and used a visual plugin self-developed to ac- Data transfer of USB quire and process images(Fig.7). camera between In addition, we show an experiment on real-time ROS and control, using KUKA RSI real-time control module. Grasshopper Under RSI, we need to write control instructions to satisfy the smoothness of the trajectory. We used a cubic polynomial to perform the axis of the robotic arm. Space trajectory planning. In this experiment, At this point we can say that wireless LAN commu- we used almost the same equipment as the previ- nication between grasshopper, KUKA robot and sen- ous experiment to demonstrate the robotic arm tool sors is fully realized. head to track the moving wooden block(Fig.8). Com- pared with ethernetKRL, the advantage of RSI is that 2.5 Experiment the real-time performance is relatively high, but the In this section, a robot experiment will be performed disadvantage is that it needs to be relatively high to test the effectiveness of the closed-loop commu- for the host computer hardware, because it needs nication method: to maintain a relatively high instruction sending fre- Content: the robot judges the distance to a piece quency (250hz). Therefore, in some cases where of paper with a black circle through recognition vi- real-time requirements are not high, ethernetKRL is sion to control the robot’s up and down movement, a more appropriate choice. so as to keep the distance between the robot and the piece of paper within a certain range. Figure 6 Tool: KUKA KR6R900 robot, vision sensor (USB Image processing in cam), black paper with solid white circle, computer grasshopper Process: We perceive this solid white circle with a visual sensor. Vision sensors have a wider range of uses than other sensors. At the same time, the amount of data transmission is larger than that of Figure 7 most sensors. It is very difficult to transmit images Experiment of using ethernetKRL, because most robots do not have ethernetKRL a good communication protocol for image transmis- sion. We connected the camera directly to the com- puter where grasshopper is located. And developed some grasshopper components to get and process images(Fig.6). First, the camera is connected to the computer, then grasshopper is started to obtain real-time im-

150 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 8 robot grabs the special-shaped 3D printed board. Fi- Experiment of RSI nally, there is a dimensional error in the pre-installed steel structure, and the dimensional accuracy of the 3D printed decorative panel is much higher than the installation accuracy of the steel structure, which makes it impossible to accurately position the 3D printed decorative panel using the steel structure. 3D printed decorative panels need to find additional po- sitioning reference.

Figure 9 Wuzhen Internet Conference outdoor 3D printed coffee kiosk 3 PROJECT PRACTICE 3.1 Challenges The real-time communication technology was ap- plied to the construction of a mobile robot on site at the Wuzhen Internet Conference outdoor 3D printed coffee kiosk(Fig.9). Due to the complex scene envi- ronment, real-time adjustments to the robot’s oper- ating program are required for the changing scene environment. This adjustment needs to be com- 3.2 Workflow pleted in a higher-performance computer. Therefore, In the end, we used the steel structure foundation real-time communication technology is one of the as the positioning reference point for positioning the important basic technologies for mobile robot op- mobile robot, adjusted the operating program ac- eration on-site. The coffee kiosk has an area of 120 cording to the different positions of the robot, and square meters, the main structure is a steel structure, transmitted the adjusted operating program to the and the surface is covered with a 3D printed decora- robot in real time for work. The final workflow is as tive panel. The 3D printed decorative panel and the follows: steel structure are linked by a secondary keel. The in- stallation and positioning of the special-shaped 3D 1. Use a total station. Based on the steel structure printed decorative panel has become a major prob- foundation position, the 3d site of total station lem on-site construction. The difficulties are mainly will be built, and the steel structure foundation manifested in three points: firstly, the spatial posi- position will be reconstructed in Rhino. Then im- tioning of the special-shaped panel, we plan to use port the designed installation model of the 3D the mobile robot field assembly method to solve the printed decorative panel into the reconstructed problem of spatial positioning; secondly, the posi- on-site model to confirm the relative position tioning of the 3D panel by the robot. Since the between the 3D printed decorative panel and 3D printed panel is a special-shaped plate, it is dif- the on-site steel structure; ficult to confirm the relative position of the special- 2. Move the robot into the field and prepare for shaped plate and the robot. We plan to solve the sec- robot positioning. After the robot reaches the ondary positioning of the 3D printed version after the vicinity of the working position, use the total sta-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 151 tion to measure the positions of the three TCP higher than manual installation. There is still room for points on the fixture, and then calculate the po- further improvement in the installation efficiency of sition of the robot body based on the coordi- the on-site fabrication robot. nates of the robot’s axes. 3. Grab the 3D printed decorative panel and 4 CONCLUSION perform the secondary positioning of the 3D This article discusses a kind of communication printed decorative panel. The robot grabs the method and application workflow which is suitable special 3D printed decorative panel and moves for the construction of on-site fabrication robot, and the decorative panel into the visible range of the verifies the effectiveness of this method with a prac- total station. The total station locates the spatial tical case. positions of the 3 3D-printed decorative panel Further research will be carried out from the fol- feature points, so as to obtain the position of the lowing aspects: 1. Further improve the correspond- captured 3D printed decorative panel relative to ing speed of communication, improve the real-time the robot; performance; 2. Expand the communication range 4. Solve the robot operation program. According from the local area network to the external network, to the grasping position of the 3D printed dec- improve the communication range; 3. Explore the in- orative panel and the target installation posi- tegration of robot communication method and con- tion of the 3D printed decorative panel known ventional construction process, and improve the fea- in the model, the robot operation program can sibility of on-site robotic fabrication under the exist- be obtained. The operation program is transmit- ing construction conditions. ted to the robot through real-time transmission, and the robot can complete the installation of a board. If the installation of the next board does REFERENCES Chai, H, Zhang, L and Yuan, PF 2020, ’Advanced Timber not require a mobile robot, repeat steps 3 and Construction Platform Multi-Robot System for Tim- 4. If the installation of the next board requires a ber Structure Design and Prefabrication’,in Yuan, PF, mobile robot, repeat steps 2, 3, and 4 (Fig. 10). Xie, YM, Yao, J and Yan, C (eds) 2020, Architectural In- telligence, Springer, pp. 129-144 Chen, CY, Chang, TW, Hsiao, CF and Huang, HY 2019 ’De- Figure 10 veloping an interactive fabrication process of maker based on “seeing-moving-seeing” model’, The Inter- Workflow of on-site national Conference on Computational Design and robotic panel Robotic Fabrication, pp. 312-321 assembly work Craig, J 2017, Introduction to Robotics: Mechanics and Control, Pearson, NY NY Shi, J, Wang, Y and Liu, S 2020, ’Designing an Architec- tural Robot: An Actuated Active Transforming Struc- ture Using Face Detection’, in Yuan, PF, Xie, YM, Yao, J and Yan, C (eds) 2020, Architectural Intelligence, Springer, pp. 253-269 Spong, MW, Hutchinson, S and Vidyasagar, M 2005, Finally, the robot completed the assembly of 120 3D Robot modeling and control, John Wiley & Sons Inc printed panels. The average assembly time of each [1] https://github.com/behrooz-tahanzadeh/Bengesht board was 11 minutes. The relative installation error [2] https://github.com/mjgordon/kuka_ethernet_strea of edge points was less than 7 mm. Compared with ming manual installation, the installation efficiency is al- [3] https://www.food4rhino.com/app/furobot most the same, but the installation accuracy is much [4] www.kuka.com

152 | eCAADe 39 - Digital fabrication and robotics - Volume 2 3D Concrete Printing with Variable Width Filament

Qiang Zhan1, Hao Wu2, Liming Zhang3, Philip F. Yuan4, Tianyi Gao5 1,2,3,4Tongji University 5University College London 1,2,3,4{1932153|2032186|1810140|philipyuan007}@tongji.edu.cn 5ucbqtga@ucl. ac.uk

Traditional mold-based concrete construction methods are inefficient, poor quality, and labor-intensive. Concrete 3D printing technology is expected to replace traditional methods as an emerging intelligent construction method due to its flexible, automatic, fast, and mold-free features. Concrete 3D printing is a method by extruding and selectively laminating construction materials onto a specific path, relying on fine-grained control of the printing material and the printing device. However, the maximum printing resolution is limited by the width of the toolpath. Filament width and printing resolution are two main factors that need to balance. In this paper, a variable width printing method is proposed using the active nozzle speed control method. The width of the print path can be adjusted according to the model details. A width control algorithm is proposed. The general workflow of variable width printing, including model preparation, toolpath planning, robotic fabrication, is also introduced, and a concrete bench is printed for experimental validation. The result shows great application potential for surface decoration and structural reinforcement. The efficiency, feasibility, and problems encountered in printing are analyzed and summarized.

Keywords: 3D concrete printing, variable filament width, robotic fabrication

INTRODUCTION the research team of Xu from Tsinghua University in Concrete 3D Printing is one of the most promising 2019. (XU et al. 2020). A lightweight concrete pre- technologies in the construction of building compo- stressed was also printed in Tongji University. nents. It allows mold-free manufacturing of complex 3D concrete printing technology provides a geometry. Like other additive manufacturing tech- novel approach to the massive production of cus- nologies, particle-bed (Lowke et al. 2018) and fila- tomized components. However, low resolution is ment extrusion (Khoshnevis 2004) are two primary one of the shortcomings of extrusion-based concrete printing methods. Because of the relatively simple printing. For extrusion-based technique, the fila- design of the end-effecter, the extrusion method is ment is extruded in constant flowrate by the noz- widely used in many construction practices. For ex- zle of fixed geometry. In order to prevent material ample, a concrete prestressed pedestrian bridge was clogging, a relatively large nozzle diameter should be printed by TU Eindhoven in 2018 (A. M. Salet et al. adopted. In other words, the maximum printing res- 2020), and a large span arch bridge was printed by olution is limited by the filament width and nozzle di-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 153 ameter. METHOD Fresh concrete is a kind of flexible material dur- Control algorithm ing pumping and extrusion. It is possible to change In extrusion-based 3D concrete printing, the filament the filament shape during the printing process. Com- width is directly affected by material flowrate, noz- pare to regular constant width extrusion, printing zle travel speed, and nozzle geometry. To figure out with variable filament width can effectively reduce the section geometry. A sample with nine layers the toolpath length, improve printing efficiency and of filament was printed and sawed. The section is enhance the structural integrity by reducing the in- shown below (See fig.2(left)). The filament was ex- ner cavity. (see fig.1) truded from a round nozzle with a diameter of 30mm. The cross-section can be approximated as a rectangle with circular shape corners, and the filament width Figure 1 is represented as w, layer height is represented as h. toolpath See fig.2(right). comparation For a solid extrusion without an inner cavity, the between constant functional relationship between nozzle speed and fil- and variable ament width can be described as filament width Q v = f(w) = (1) Research about filament shape control has been con- π − 2 ( 4 1) h + wh ducted by varying the nozzle geometry. A control- lable shape nozzle was developed by Xu to change In this formula, v is the nozzle velocity, Q is the mate- the filament width while printing. (Xu et al. 2019) A rial flowrate, w is the filament width, h is the filament shape control device was also described for section width. Therefore, for a given flowrate and filament geometry control and surface quality improvement. height, the nozzle velocity can be directly calculated (Lao et al. 2021) However, the complexity of the noz- from the filament width. zle setup limited the application scenario. This paper is aimed to propose a simple variable-width printing Toolpath generation algorithm For variable width printing, both nozzle position and method by robot speed control. In the following sec- nozzle velocity should be included in the toolpath in- tion, the general control algorithm is introduced, the formation. In this paper, a Grasshopper-based robot design process and fabrication workflow are shown, toolpath generation program is developed. The several showcases are printed for experimental vali- generation algorithm and grasshopper program are dation. shown (see fig.3). The generation process starts with closed NURBS Figure 2 geometry with variable thickness. Then the model the cross-section of was contour sliced. Then a simple geometrical the test sample method is used for toolpath generation. As is shown (left) calculation in fig.3, curve A and curve B represent the outer and model (right) inner contour, the tool center point (TCP) of the print head should be kept at an equal distance between curve A and curve B. Circles are drawn in tangent to both curves A and B by keeping the centers of the cir- cles along the toolpath with the diameter equaling the filament width. Centers of the circle Cn are con- nected by the interpolated curve M. As observed, the

154 | eCAADe 39 - Digital fabrication and robotics - Volume 2 curve shows good fitness as the toolpath. EXPERIMENTAL VALIDATION Material preparation Figure 3 A bicomponent fiber-reinforced printing material is the toolpath developed to get the optimal extrudability and build- generation ability for 3D concrete printing. Retarded mortar was algorithm pumped to the nozzle and mixed with a liquid accel- erator right before extrusion. The mixture used in the study includes river sand with a maximum aggregate size of 2mm, ordinary Portland cement with a water to cement ratio of 0.3, 5% of silica fume. 0.6% of polycarboxylate-based su- perplasticizer is added to improve the rheology of the mixture. An accelerator is actively dosed at 2% by weight of the mortar using a dosing pump controlled Design and optimization by PLC (Programmable Logic Controller). A group of benches with a width of 1m is designed to verify the variable width printing process. At first, Machine setup a rectangular shape shell of the bench was designed. A robot-based printing system is built to perform the The middle part of the bench was slightly curved to variable width printing test. The system contains two give a comfortable seat. Then a pattern is morphed major parts, the mixing and delivering unit for mor- onto the shell to create an uneven thickness rein- tar mixing and deposition, the control unit for robot forcement. Several reinforcement patterns are de- motion control. The general workflow and machine signed, and patterns with straight lines (Sample A) setup are shown in fig.5. and cross lines (Sample B) are chosen for test print- The mixing and delivering unit contains a batch ing. (See Fig.4). To prevent over extrusion and fila- mixer to mix the material, a screw-type mortar pump ment rupture, a printability test is performed to fig- to transport and extrude the material, and a contin- ure out the proper range of printable filament width uous mixer for accelerator mixing. The print head is (see section “Robotic fabrication”), which is set to mounted on a KUKA industrial robot with a rated pay- (30mm,57mm). load of 500kg and a maximum reach of 2826mm. All the dry powder materials are premix in a fac- Figure 4 tory and packed into 25kg bags for easy material perspective view preparation in the mixing process. Then dry material and side view of is mixed with water in the batch mixer. After 2min of sample A (top) mixing, fiber was added and mixed for another 2min. perspective view The mixing process remains the same for each batch and side view of to ensure material consistency. sample B (bottom) After preparing fresh mortar, the mixture was pumped through the hose to the printhead, where the mortar is mixed with the accelerator. In terms of flowrate data, mortar is a high viscosity material with abrasive content, which is hard to deal with for most flowmeters. It is difficult for a long time measure- ment of the real-time flow rate of the mortar. There-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 155 Figure 5 the system diagram

fore, in this study, the flow rate of the mortar pump Figure 6 is pre-calibrated by the motor speed, which is con- over extrusion can trolled by VFD (Variable Frequency Drive). During the be observe in test printing process, the motor speed remains the same printing of sample B to ensure a constant flow rate.

Robotic fabrication The printing device is pre-tuned for continuous ex- trusion. The samples were printed on a preprint base covered by plastic film. All the samples were printed at the same material flow rate, while the nozzle speed To figure out the proper printable range, six strips varies. of samples were printed at different speeds, and the The printing process starts from sample B. The width is measured by measuring tape. (see fig.7) process took 1350 sec. However, surface discontinue The filament width and corresponding nozzle speed can be observed obviously on the external surface. are listed in table 1. The surface quality is also an- Filament over extrusion occurs in the area with larger alyzed. From the result, it can be found that fila- thickness. According to the initial model of sample B, ment width in the range of (30mm, 57mm) shows the range of filament width is (30mm,100mm), which acceptable surface quality. Therefore, the original exceeds the maximum printable range. The surface width range (30mm, 100mm) was linear remapped to quality is unacceptable for a bench. (see fig.6) (30mm, 57mm).

156 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Table 1 the relationship between nozzle velocity, filament width and surface quality

Figure 7 After the remapping process, samples were test sample for reprinted, and no over extrusion was observed af- printability ter adjustment. For sample A, the printing process assessment took 900s, with a vertical buildup rate of 4m/h. For sample B, the printing process took 1125s, with a ver- tical buildup rate of 3.2m/h. The printing process is shown below (see fig.8).

RESULT AND DISCUSSION In terms of surface quality, from the overall visual as- Figure 8 sessment, the filament width is changed along with the printing process the patterns, no significant discontinuity or filament and result over extrusion is observed. The aim of variable width printing is realized. In terms of geometric consis- tency, the inner surfaces of both samples show good agreement with 3D models. (see fig.9) However, the reinforcement patterns also appear on the out- side surfaces (see fig.10), which are supposed to be smooth patterns. To further investigate the cause, both samples are 3D scanned and compared with the original 3D models. 3D scanning is performed after 21 days of the curing process at room temperature. In this re- search, an industrial handheld laser 3D scanner is used for scanning. High reflective positioning tar- gets are stuck to the target surface by referring to the standard scanning process (see fig.11). The re- sult is stored in point cloud data for further analysis. The original 3D models and the 3D scanned result are aligned and compared. Geomagic Control X, a scan- ning data process software from 3D System, is used for model post-process and error analysis.

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 157 Figure 9 the inner surface of sample A (left) and sample B (right)

Figure 10 the exterior surface of sample A (left) and sample B (right)

The displacement distribution is shown in fig.12. Figure 11 Although the result is acceptable for large scale Front view of building components, there is still space for fur- sample A, high ther improvement. From the cross-section view (see reflective fig.12(top)), it can be found that the major displace- positioning targets ment distributes mainly in the corner area. are stuck to the The displacement is relatively larger for sam- target surface ple B. The overall displacement is among (- The 3D data of the inner surface cannot be obtained 26.85mm,26.43mm), with an average displacement due to the limitation of operation space. There- of -4.37mm and a standard deviation of 5.58mm. The fore the geometric comparison is only performed displacement distribution is shown in fig.12. From for the outer surface. In sample A with a straight- the cross-section view (see fig.12(bottom)), there is line pattern, the overall displacement is among (- significant displacement in the upper corner area, 17.85mm,17.85mm), with an average displacement which is the leading cause for the surface pattern. of -2.96mm and a standard deviation of 3.35mm. The nozzle shape and the inaccuracy of filament

158 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 12 displacement analysis of sample A (top) and sample B (bottom)

width control may be the major causes. The curved especially in the corner area. In terms of filament toolpath may influence the material extrusion, which quality, there should be a proper printable range for can cause uneven material deposition between the width variation. The relationship between printabil- inner and outer sides of the toolpath. ity and other printing parameters remains for further investigation. In terms of efficiency, layer deposition CONCLUSION AND PERSPECTIVE usually is a time-consuming process. Increase the This paper presented a novel concrete 3D printing nozzle speed may induce other problems such as me- method by actively changing the filament width dur- chanical vibration and abrasion. Variable width print- ing printing. Although there still lots of aspects to be ing provides a simple and straightforward method improved, this study shows a promising approach to with a shorter toolpath, significantly reducing print reduce the print time, improve the printing resolu- time. In terms of structural performance, weak Layer tion and enhance the structural integrity. bonding is one major weakness for additive manufac- First, this study proposed a filament width turing. This method provides a simple way to reduce changing the method by nozzle speed control. Due the amount of gap inside. Variable thickness compo- to the simple design, there is no need for a complex nents can be printed in a single path, which can en- shape-forming device mounted on the nozzle. Thus hance structural integrity. high robustness can be achieved. This study also In addition, this paper also presents a feasible presents a simple algorithm converting the thickness method to analyze geometric consistency. By 3D data into nozzle speed. However, according to the laser scanning, the deformation of the print sample experimental validation, there still significant errors, can be presented in full-scale with high precision. Although this study only shows an overall deforma-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 159 tion assessment, with proper experiment setup, both the inside and outside geometric information can be obtained. Future research may focus on integrat- ing multi-process parameters, such as the real-time flowrate control and toolpath generation.

ACKNOWLEDGEMENTS This research was supported by National Key R&D Program of China (Grant No. 2018YFB1306903), Na- tional Natural Science Foundation of China (Grant No. U1913603), and Shanghai Science and Technology Committee (Grant No.18DZ1205604).

REFERENCES Khoshnevis, B 2004, ’Automated construction by con- tour crafting—related robotics and information technologies’, Automation in Construction, 13, pp. 5- 19 Lao, W, Li, M and Tjahjowidodo, T 2021, ’Variable- geometry nozzle for surface quality enhancement in 3D concrete printing’, Additive Manufacturing, 37, p. 101638 Lowke, D, Dini, E, Perrot, A, Weger, D, Gehlen, C and Dil- lenburger, B 2018, ’Particle-bed 3D printing in con- crete construction – Possibilities and challenges’, Ce- ment and Concrete Research, 112, pp. 50-65 A. M. Salet, T 2018, ’Design of a 3D printed concrete bridge by testing’, Virtual and Physical Prototyping, 13, pp. 222-236 Xu, W 2020 ’FABRICATION AND APPLICATION OF 3D- PRINTED CONCRETE STRUCTURAL COMPONENTS IN THE BAOSHAN PEDESTRIAN BRIDGE PROJECT’, Fab- ricate 2020: Making Resilient Architecture Xu, Jie, Ding, L, Cai, L, Zhang, L, Luo, H and Qin, W 2019, ’Volume-forming 3D concrete printing using a variable-size square nozzle’, Automation in Construc- tion, 104, pp. 95-106

160 | eCAADe 39 - Digital fabrication and robotics - Volume 2 The reconfiguration of tool parameters in clay extruder

Hülya Oral1, Meryem Birgül Çolakoğlu2 1Architectural Design Computing, Istanbul Technical University 2Architecture, Is- tanbul Technical University 1,2{oralh|colakoglumer}@itu.edu.tr

Reconfigurable manufacturing systems (RMS), which have emerged with mass customization in the industry, allow the tool or system parameters to be adjusted in a controlled manner to produce customized products. Although the reconfigurable manufacturing systems have been applied in building and construction for waste, time, and cost reduction in producing specific mold forms at a prototype level, their utilization in additive manufacturing (extrusion) has not been explored. This paper presents, initial steps of ongoing research on the development of reconfigurable tools and workflow by transforming the tool's rigid parts with controlled parametric movable parts. The clay extruder tool used in crafts is transformed into a reconfigurable extruder tool that allows making customized forms. In the experimental setup, die combinations and tool head rotations are examined with the proposed Extrusion-based Making Grammar (EbMG). Produced forms illustrate the variations of the design space in relation to the rule sets. Here, a rule-based approach is found to be efficient for controlling the making parameters. This study explored the potentials of transforming a rigid craft tool into a customizable tool that allow the generation of product variation. It presents the preliminary stage of transforming craft tools into further digital craft tools.

Keywords: Hands-on making, craft, reconfigurable tools, extrusion, tool making

1 INTRODUCTION facturing systems (FMS and RMS). These systems are Customization strategies have already been inte- employed to respond to the user demands in terms of grated into the architecture, engineering, and con- product variation in the industry. The reconfigurable struction (AEC) industry in terms of housing (Mat- manufacturing systems have been applied in build- sumura et al. 2019), urban spaces (Verebes 2015), and ing and construction for waste, time, and cost reduc- building components (Tu and Wei 2013). It has re- tion in producing specific mold forms (Khabazi and cently expanded to the area of the design and devel- Budig 2016; Raun and Kirkegaard 2015) at a proto- opment of on-demand or bespoke tools which pro- type level. However, their utilization in additive man- duce a specific family of products by implementing ufacturing i.e., extrusion has not been explored. Yet, the strategies of flexible and reconfigurable manu- the integration of customization strategies into craft

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 161 practices in terms of tool development is overlooked. parts with manual, machine or robot control, recon- The customization of craft tools is a promising area figurable tools allow experimentation with the ma- of research that enables in-situ fabrication by trans- terial, resulting in novel production workflows (Asut, forming manual rigid tools into reconfigurable mech- Eigenraam, and Christidi 2018; Raun and Kirkegaard anisms. 2015; Tessmann and Mehdizadeh 2019). This study aims to parameterize the craft tool In the literature, the customization of tools use to control the making process and produce cus- mainly relies on the moveable mechanisms which en- tomized designs. As a case study, a manual clay able the variation of building components. These extruder is analyzed to determine the part-function mechanisms are designed by parameterization and relations and control the variation in outputs using reconfiguration of existing tool parts and functions Extrusion-based Making Grammar (EbMG). We uti- (Oral and Colakoglu 2020). Pin-based tooling, parts- lized experimentation and a rule-based method to based molds, polymer, and fabric-formed molds are discover and define the parameters of the die-tool developed to reduce lead-time, cost, and waste in head component as the first phase of this study. The producing differentiated units out of one mecha- outputs produced with combinations of tool parts nism. In pin-based molds, double-curved surface and functions extend the defined solution space of geometry is controlled via pins moving on Z-axis the rigid tool. (Kelkar, Nagi, and Koc 2005; Schipper et al. 2014). Parts-based molds consist of moveable planar parts 2 BACKGROUND to change the size or form of the components The idea of mass customization is to provide an in- (Khabazi and Budig 2016; Shaffer 2017). The varia- creased variability with the flexible and reconfig- tion can also be produced by flexible materials like urable manufacturing systems (FMS and RMS) at an elastic fabrics and polymers controlled by move- affordable cost unlike crafts (Piller 2013). The RMS able mechanical parts (Khan 2008; Tessmann and can respond promptly to the changes in the market, Mehdizadeh 2019). Changes in the orientation of but in a defined design space (Bortolini, Galizia, and tool parts directly affect the overall size and form in Mora 2018). The FMS can adapt to any changes in a defined solution space. the production workflow in the long run, because In crafts, every product needs individualized pro- of the use of digital fabrication tools such as CNC, duction as a result of lacking control defined as “the robotic arm, and 3D printers. Digital fabrication tools risk in making” (Pye 1968). Craft practices mainly offer a certain degree of flexibility in terms of varia- rely on tacit knowledge gained through hands-on tion. However, the FMS has a higher investment cost making with material and the tool. The function compared to the RMS, which uses the combinatorial of the tool is derived from the affordances of its logic of assembly (Koren 2006). Moreover, digital fab- parts and the expertise of the craftsman. Consider- rication tools have limitations such as file-to-factory ing the hidden knowledge in crafts, it is important to production, single tool operation, limited bed size, parametrize the making process. The rule-based ap- limited manipulation of tool parameters and parts proach is a widely used method to decipher existing (Koren 2006; Oxman 2007; Peek and Moyer 2017). designs (Duarte 2005; Colakoğlu 2005) and recently Having been low-cost and easy-to-operate, reconfig- the fabrication processes (MacLachlan and Jowers urable tooling strategies can be employed to trans- 2016). Formalizing the manual-making processes form craft tools. It enables the manufacturing of a can enhance creativity through shape computations family of products rather than one-off and unique (Knight 2018), resulting in the emergence of outputs. products with available tools and techniques (Oral Tool reconfigurations leading to emergent use of the and Colakoglu 2020). By using changing or moving tool can be explored with a grammar based on com-

162 | eCAADe 39 - Digital fabrication and robotics - Volume 2 binations of tool subparts and functions rather than to discover moveable, changeable, and combinable shape computation. A making grammar can be gen- parts of the manual extruder. RQ2 will be discussed erated by focusing on not the outputs, but on the through the physical outputs produced with the process of forming the material with the tool. A man- Extrusion-based Making Grammar. ual tool can be parametrized to be controlled and re- configured to generate variation as in the paramet- 4 RECONFIGURING THE PARAMETERS OF ric model. Especially for non-expert designers and THE EXTRUSION multi-operators, visually described rules can create a medium to formalize the making process and facili- 4.1 Parameterizing the Making Process The manual clay extruder is chosen as a craft tool to tate the execution of the grammar. be examined. It is used in workshops to produce coils out of soft materials such as clay, polymer clay, and 3 METHODOLOGY playdough. It is selected for its adaptability to 1:1 The methodology of this paper consists of the ex- scale being able to produce extruded building units perimental method to decipher the use of the craft and components. The sub-parts of the extruder are tool and the rule-based method to explicitly define the tool head holding the polygonal metal dies, the the tool use. In the first stage, a set of experi- pressing disk controlled by the rotation of the screw, ments are conducted with the die-tool head com- handle, and the tube for the material (Figure 1). ponent to explore the reconfiguration scenarios hid- Figure 1 den in tool use. Die combinations, tool head rotation, The sub-parts of the and die change operations, which are embedded in craft tool. tool affordances. are discovered in this phase. The Extrusion-based Making Grammar (EbMG) is gener- ated by adding the discovered functions to the rigid tool ruleset. In this study, die combinations and tool head rotations are examined and variable forms are physically produced with the tool and playdough material. The outputs of this study are the novel use of the craft tool with the reconfiguration of tool pa- rameters unveiled through hands-on experimenta- tion and variable extruded units with differentiated surface patterns. The physical outputs are discussed In the conventional use of the tool, first, the soft ma- regarding the effect of die shapes and rule combina- terial is put in the tube and the die is placed into tions on the surface and the boundary of extruded the tool head. When the handle is rotated, the disk forms. pushes the clay down through the die to make the The following questions will be examined in this material dislodged. The output is a straight extru- study. sion that is built layer by layer. It is cut to the defined RQ1: Which parameters of extrusion as a hands- length by a wire. on making process can be identified in the context of The Extrusion-based Making Grammar (EbMG) reconfigurable tools? consists of pre-press, press, and post-press opera- RQ2: How the parameters can accommodate tions, which are represented as a visual grammar to product variation from the reconfiguration of the tool be applied easily (Figure 2). In the pre-press, the die head and die parts? is placed, combined with other dies, or omitted from RQ1 will be explored by conducting experiments the tool head (1a,2a, and 2b). In the extrusion phase,

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 163 Figure 2 Extrusion-based making grammar (EbMG).

two types of pressing are shown with continuous and 4.2 The Experimental Setup stepped surface qualities. The single rotation of the As a first phase of analyzing the part-function rela- screw determines the extruded layer height repre- tion, several experiments are conducted with the die- sented with t1 (3a and 3b). The tool head rotating tool head component. In the experimental setup, the in CW and CCW directions and variable angles can tool is operated manually, and playdough is used as be combined with continuous and interval pressing. a material. Dies found in the toolset are concave and The post-press operations are limited to separating convex polygons cut from thin metal plates. A con- the extruded form from the material by cutting (5a). cave chamfered pentagon is 3D printed and added In the ruleset application, two operators are to the existing toolset to explore the die customiza- needed for controlling the rotation of the screw and tion. The pitch of the screw defines the height of the tool head synchronously, especially in combined op- extruded layer in one revolution of the screw. The ex- erations. Thus, the grammar guides both the experi- truded forms are generated with 4 revolutions of the mental process and the operators who apply the rule- screw. set, resulting in a controlled making process even In the preliminary experiments, it is discovered with hands. that the rigid die-tool head component can be re- configured in terms of die combinations, tool head rotation, and die change (Figure 3), which are not

164 | eCAADe 39 - Digital fabrication and robotics - Volume 2 available in conventional tool use. While die combi- the extruded forms by reducing the size of the die nations generate emergent surface patterns on ex- gap (7a and 9a). The combination of identical dies truded forms, tool head rotation forms twist extru- also creates variation by superimposing them in dif- sions. Die change during the extrusion process cre- ferent angles (6a, 8a, and 10a). Two star-shaped dies ates tapered extrusions. Due to the small effect of die are combined in 50 and 18 degrees of relative orien- change operations on this scale, it is out of the scope tation forming a decagram (6a and 8a). The extruded of this paper. forms of these combinations have variable lengths of corners. Figure 3 The placement of the dies in relation to each The die-tool head other is visually shown in the EbMG to be able to component repeat the process. The non-symmetric dies create reconfigurations. curved extrusions by bending the output (9a). The longer the extrusion, the more the bending occurs, which can be corrected by hand during the process. The sharp-edged polygons like stars tend to have de- formations on edges compared to chamfered poly- In the experiments, tool head and die parts were ex- gons such as the three-circle die. The variation of amined in terms of moveable, changeable, and com- outputs is increased with the number of die combi- binable features to create variation by reconfiguring nations. tool parts. In the first experiment, selected concave 4.2.2 Experiment 2: The Tool Head Rotation. The and convex dies are combined in specific orienta- second experiment focuses on die rotation to dis- tions and stuck to the tool head with clay. In the sec- cover the potentials of the die-tool head component. ond experiment, the nuts-like tool head is utilized for The rectangular and three-circle dies are used by uti- rotating the dies during the extrusion. lizing the rotational ability of the tool head shown 4.2.1 Experiment 1: The Die Combinations. Fol- in Figure 5. Following the straight extrusions, twist lowing the extracting and discovering of the die-tool extrusions with different surface properties are pro- head component parameters and functions in the duced by manipulating the pressing, rotation angles, first experiment, a set of forms is produced by recon- and die combination parameters. By sticking the figuring dies in a controlled way (Figure 4). By us- material to the pressing disk, a continuous extruded ing rectangular and three-circle dies, the straight ex- form is obtained without the need for rotating the trusion is obtained and cut with the wire in defined tool head (3a). Due to the changes in pressure ap- length (1a and 2a). Then, selected convex and con- plied to the material, the sections of the extruded cave dies are superimposed in different angles to ex- form oscillate between larger to smaller. The rotation amine the effect on extruded forms. The combina- of the tool head can also generate this form by fixing tion of the star polygon with the circular polygon the die to the tool head (5a). The frequency of twisted generates again star shape extrusions with cham- edges is lowered in this sample compared to Figure fered edges due to the similar die gap (3a). The com- 5.3a. The variable rotation angles can be used also in bination of the hexagon and chamfered pentagon the same extrusion process (7a and 8a). The synchro- creates grooves only at the side of the form due to the nization between rotation and press creates twist ex- die shift during extrusion as an emergent behavior trusions (5a and 6a). When the press and rotation (4a). The three-circle and star-shaped dies form again operations are applied consecutively, multi-step ex- modified star extrusions with linear grooves (5a). Cir- trusions are produced with sliced appearance due to cular and half-circular dies are useful for scaling down the displacement of the die deforming the surface (4a

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 165 Figure 4 The product variation obtained by die combinations.

166 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 5 The product variation obtained by the rotation of the single and combined dies.

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 167 and 8a). The combination and rotation of dies also the extruded form is bent in the non-symmetrical generate variation both in the section and the sur- polygons such as half-circle bend due to the unequal face. As shown in Figure 5.9a, continuous press with pressing applied to the material. The bending also in- 360 degrees of tool head rotation introduces variable creases when the number of operations and height of grooves on the surface. The multi-step extrusion with the extruded forms rise. The rotation of the tool head equal press duration creates equally distributed par- and different angular superimposition of the concave titioning on the form (10a). polygons creates variable grooves on surfaces. These The tool use is visually represented to coordinate grooves are effective in enlarging the surface area synchronous operations during the making. Inter- and creating visually appealing patterns. val extrusion operation with rotation is represented Considering the intricate surface details of the by extruded form with horizontal lines, while contin- outputs, problems are encountered in the produc- uous extrusion operation with rotation by extruded tion of these surfaces with rigid or reconfigurable form with no lines. Degrees shows the approximate molds. Two or more mold parts with smaller surface angles of rotation due to manual making. The special protrusions are required to produce such detailed condition as emergent behavior in which material is surface elements in CNC molding with high cost and stuck to the pressing disk is shown in Figure 5.3a. waste. When liquid material is poured into the mold, The rules and their combinations can be increased by the material can be stuck in small protrusions. In re- using different polygonal dies and rotation-press se- configurable molding, part-based or flexible mate- quences. rial molds must be used to produce 3-dimensional forms rather than pin-based tooling. In the literature, 5 RESULTS AND DISCUSSION part-based molds produce different sizes of the same The produced forms are straight, twist, and multi- unit, whereas, in polymer and fabric-formed molds, step extrusions with variable surface patterns. These intricate surfaces with certain precision are difficult to forms can be modeled by lofting the series of orthog- produce since the fabric behavior is inconsistent. onal sections with bending deformations. Consider- The extruder has the same advantages of addi- ing the dimensions of the hand-held tool, the pro- tive manufacturing technique as low waste and pro- duced forms can be used in the early design and pro- ducing complex surface details. In addition, the ex- totyping phase to experiment and define the toler- truder has a 2-dimensional die cost, which is very ances of extrusion in terms of material properties, die low compared to the multi-part and complex molds shape and dimensions, haptic feedback, and press- used in rigid and reconfigurable molding. In addi- ing force. tion, diversity can be increased with modifiable die The findings of the experiments are related to designs and combinations. Although the forms are the effect of die shapes and combinations on sur- directly produced with the reconfigurable extruder in faces and the boundary of extruded forms. The layer this study, the extruder with a circular die can be uti- height is calculated as 3 mm in square die and 2,22 lized to extrude coils of discrete tool paths as in the mm in there-circle die related to the die gap area. A 3D clay printer. The robotic arm, which is one of the larger die gap area results in smaller extruded mate- digital fabrication tools, can perform customized pro- rial. To achieve the same height in different dies, the duction workflows through different end effectors. number of screw revolutions should be increased. The reconfigurable extruder can be integrated into Surface deformations are high in extrusions with con- robotic production as in clay printing, but this time cave polygons compared to convex ones due to the the nozzle is changeable and combinable owing to sharp edges. Circular and half-circular dies are useful the dies. New 3D printed pentagonal die can gener- in reducing the width of the components. However, ate interlocking units with protruding parts fitting to

168 | eCAADe 39 - Digital fabrication and robotics - Volume 2 each unit by eliminating additional joints. Unveiling tool parameters and define them explicitly enables the design and development of novel tools 6 CONCLUSION and procedures derived from available tools, materi- This study aimed to decode the parameters of a als, and techniques. Furthermore, for each rigid af- craft tool use to control and reconfigure the tool fordance of tool parts, variable affordances emerge parts for producing customized units as initial steps from experimenting with them. Parameterizing and of the ongoing research. A manual clay extruder controlling the manual tools enable producing diver- was examined to define part-function relations that sity in components. Using manual tools as an input can be changeable, moveable, or combinable by fol- for developing customized tools provides a gradual lowing the reconfiguration methods in the indus- improvement in craft practices as a result of integrat- try. The methodology based on experimentation ing computational thinking into making processes. and a rule-based approach is found to be effective to parametrize the intuitive making process to fur- ACKNOWLEDGEMENT ther control the workflow. The novel part-function We would like to thank Istanbul Bilgi University for fi- relations are discovered which is not included in the nancially supporting this presentation. conventional use of the rigid tool such as die com- binations, tool head rotation, and die change. Vari- REFERENCES able dies and part-function combinations are exam- Asut, S, Eigenraam, P and Christidi, N 2018 ’Re-flex Re- ined by benefitting the combinatorial logic in terms sponsive Flexible Mold for Computer Aided Intuitive of reconfigurable tooling. Design and Materialization’, Proceedings of the 36th The series of experiments shows the product eCAADe Conference, Lodz, Poland, pp. 717-726 variation generated by the minimum number of the Bortolini, M, Galizia, FG and Mora, C 2018, ’Reconfig- urable manufacturing systems: Literature review dies. The use of a 3D printed die as an extension and research trend’, Journal of Manufacturing Sys- of the toolset shows the CAD/CAM tools can also be tems, 49, pp. 93-106 convenient to enhance available tools to generate Colakoglu, MB 2005, ’Design by Grammar: An Interpreta- new forms. The diversity is achieved in terms of sur- tion and Generation of Vernacular Hayat Houses in face variations compared to the rigid tool as a re- Contemporary Context’, Environment and Planning sult of the numerical explorations with the material. B: Planning and Design, 32(1), p. 141–149 Duarte, JP 2005, ’Towards the mass customization The resulting die shapes and operations are repre- of housing: the grammar of Siza’s houses at sented visually for the use of non-expert designers Malagueira’, Environment and planning B: Planning and multi-operator applying the ruleset. The limita- and Design, 32(3), pp. 347-380 tion of this study is related to the experimental setup. Kelkar, A, Nagi, R and Koc, B 2005, ’Geometric algorithms Playdough is used as a material in the experiments in for rapidly reconfigurable mold manufacturing of which clay, polymer, and air-drying clay can be exam- free-form objects’, Computer-Aided Design, 37(1), pp. 1-16 ined further. Khabazi, Z and Budig, M 2016 ’Cellular Concrete Cast- Future studies will be focused on: ing Using Digital Moulds’, Proceedings of the 34th • Controlling the tool with motors to improve eCAADe Conference, pp. 83-92 Khan, O 2008 ’Reconfigurable Molds as Architecture Ma- the accuracy and precision in produced compo- chines’, Proceedings of the 28th Annual Conference of nents. theAssociationforComputerAidedDesigninArchitec- • Developing an interface to control and simulate ture (ACADIA), pp. 286-291 the tool use. Knight, T 2018, ’Craft, Performance, and Grammars’, in • Using different materials in experiments to eval- Lee, JH (eds) 2018, Computational Studies on Cultural uate the effect on the forms. Variation and Heredity, Springer Singapore, Singa-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 169 pore, pp. 205-224 Tu, KJ and Wei, HY 2013, ’Emerging trends and concepts Koren, Y 2006, ’General RMS Characteristics. Com- of mass customisation in Taiwan’s housing industry’, parison with Dedicated and Flexible Systems’, in in Piroozfar, P.A.E. and Piller, F.T. (eds) 2013, Mass Dashchenko, AI (eds) 2006, Reconfigurable Manufac- customizationandpersonalizationinarchitectureand turing Systems and Transformable Factories, Springer construction, Routledge, New York Berlin Heidelberg, Berlin, Heidelberg, pp. 27-45 Verebes, T 2015, ’Cities and Their Specificities: Standards, MacLachlan, L and Jowers, I 2016, ’Exploration of multi- Customs and the Making of 21th Century Urbanity’, material surfaces as weighted shapes’, Graphical Architectural Design, 85(6), pp. 8-17 Models, 83, pp. 28-36 Matsumura, S, Gondo, T, Sato, K, Morita, Y and Eguchi, T 2019, ’Technological developments of Japanese prefabricated housing in an early stage’, JAPAN AR- CHITECTURAL REVIEW, 2(1), pp. 52-61 Oral, H and Colakoglu, MB 2020, ’Flexible Tools in Mould and Formwork Making: A Review’, J. of Design Re- search, 18 (3/4), pp. 173-195 Oxman, N 2007 ’Digital Craft Fabrication Based Design in the Age of Digital Production’, Workshop Proceedings for Ubicomp 2007: International Conference on Ubiq- uitous Computing, pp. 534-538 Peek, N and Moyer, I 2017 ’Popfab: A Case for Portable Digital Fabrication’, Proceedings of the Tenth Interna- tional Conference on Tangible, Embedded, and Em- bodied Interaction - TEI ’17, Yokohama, Japan, pp. 325-329 Piller, F 2013, ’Three capabilities that make mass cus- tomisation work’, in Piroozfar, P.A.E. and Piller, F.T. (eds) 2013, Mass customisation and personalisation in architecture and construction, Routledge, New York, pp. 17-30 Pye, D 1968, The nature and art of workmanship, Univer- sity Press Cambridge Raun, C and Kirkegaard, PH 2015 ’Adaptive Mould– A Cost–Effective Mould System Linking Design and Manufacturing of Double–Curved GFRC Panels’, 17th international congress of GRCA-GRC, Dubai Schipper, R, Grünewald, S, Eigenraam, P, Raghunath, P and Kok, M 2014 ’Optimization of The Flexible Mould Process for The Production of Double-Curved Concrete Elements’, CIC 2014: The 1st Concrete Inno- vation Conference, Oslo, Norway Shaffer, M 2017, ’Developing robotic formwork: en- hancing formwork mobility and variability through mechanization’, Construction Robotics, 1(1-4), pp. 77-83 Tessmann, O and Mehdizadeh, S 2019 ’Rotoform: re- alization of hollow construction elements through roto-forming with hyper-elastic membrane form- work’, Proceedings of the Symposium on Simulation for Architecture and Urban Design, pp. 1-7

170 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Design and Fabrication Methodologies for Repurposing End of Life Metal via Robotic Incremental Sheet Metal Forming

Paul Nicholas1, Ruxandra Stefania Chiujdea2, Konrad Sonne3, Antonio Scaffidi4 1,2,3,4Royal Danish Academy, School of Architecture 1,2,4{paul.nicholas|rchi|asca}@kglakademi.dk [email protected]. dk

This paper investigates an integrative approach to robotic incremental sheet metal forming (RISF), which connects the registration of variable material properties and geometries to the re-forming of pre-made components beyond their initial formulations. Re-using rather than recycling metals can save the significant energy costs that come with having to melt, purify and re-manufacture products, as well as saving the costs of the new object it replaces. In this paper, we describe a workflow that connects 3d scanning, design automation and fabrication. The method goes beyond state of the art for RISF by challenging the assumption of starting from a flat unused sheet of metal, opening up the potential of RISF for material reuse. Our approach is demonstrated through the fabrication of a series of bench seating elements from oil drum geometries, however is generalisable to other input materials and output geometries. 3d scanning is used to register varying geometric features such as rolled beads, irregularities such as dents and holes, and material properties such as corrosion.

Keywords: robotic fabrication, re-use, upcycling, incremental sheet metal forming

INTRODUCTION Current research explores a variety of design and The growing impact of material consumption and manufacturing-led approaches that aim to minimise waste on human health, economy and the natu- waste, including design for disassembly, design for ral ecosystem is motivating an increasingly urgent bio-materials, and design from reuse. In this paper shift to resource efficient practices across architec- we work with the concept of material reuse, which ture, construction and design. Central to this shift is addresses existing products and places focus on ex- the move from a linear to a closed-loop economy - a tensions to their life cycle, and with a specific mate- circular economy that reduces waste to a minimum rial focus on scrap steel. The production of steel is a via reusing, repairing and recycling. highly energy-intensive process, and has maintained

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 171 Figure 1 Registration, design and fabrication workflow

a relatively constant CO2 intensity over the past two the registration of relevant material properties, then decades [1]. Steel has the advantage of being 100% detail our methodology for forming non-flat metal recyclable and scrap is used within both primary and sheet into new products informed by local geomet- secondary production processes, where it can result rical and material properties. We demonstrate this in up to 70 - 90% reduction in the energy cost for workflow (see figure 1) on the case of a bench seat, secondary production. However, secondary produc- although the method is developed to be more widely tion is still highly energy intensive, much scrap steel applicable to other initial input objects and architec- not recycled, and large quantities of material are lost tural cases. Lastly, our paper outlines the wider possi- or deemed ‘unrecoverable’ with each use cycle and bilities of this approach, which are found in re-use but excluded in the calculation of recycling rates [2]. A also hybrid manufacturing scenarios, and the further lack of methods for processing reclaimed elements research that is required. through automated fabrication is recogised as a bar- rier to reuse of this material [3]. FABRICATION PRACTICE AND PROPERTIES Our paper introduces an innovative fabrication Robotic Incremental Sheet Forming (RISF) is a vari- process - robotic incremental sheet metal forming ant of the flexible Incremental Sheet Forming (ISF) (RISF) - and investigates how RISF can enable the di- manufacturing process first developed in Japan in rect reuse of scrap steel to make other products. Re- the 1980s and 1990s. A manipulative freeform pro- using rather than recycling metals can save the sig- cess for imparting 3D form onto thin metal sheet, nificant energy costs that come with having to melt, RISF moves a simple tool from either one or two purify and re-manufacture products, as well as saving sides causing a highly localized plastic deformation the costs of the new object it replaces. We demon- (Jeswiet et al 2005). A single sided approach typi- strate that as a tool for re-use, RISF opens up new pos- cally utilizes one tool and a blank, while double sided sibilities for old scrap, and dramatically expands for- forming provides further flexibility for forming out of mal possibilities where no or only subtractive alter- plane in opposing directions. Where prior architec- ations are the norm. We firstly describe state of the tural research has investigated the structural and ex- art in architectural incremental sheet metal forming,

172 | eCAADe 39 - Digital fabrication and robotics - Volume 2 pressive potential of single sided (Bailly et al 2014, is visible to the naked eye), while in Japan a 5 grade Nicholas et al 2016, Nicholas et al 2018) and dou- scale based on visual examination and measure of ble sided (Nicholas et al 2017) incremental forming, surface thickness is used. Research by Chun et al 2013 the starting geometries are always flat, new sheets of has aimed to standardise the Japanese scale, which metal. they characterise as highly subjective and qualitative, In this research, point cloud scanning is used through the application of image processing tech- to capture the geometry and surface properties of niques and logistic regression. Similar research by non-flat, non-new thin sheet metal objects. Within Yan et al 2014 also uses machine vision and connects industry, structured light, time of flight and pho- the same classification system to a machine learning- togrammetric scanning methods are commonly used based approach. More recent research such as Pet- to measure the dimensional accuracy of parts and en- ricca et al 2016 continues this approach, connect- able geometry and surface inspection, typically for ing image-based analysis to more powerful machine the purpose of quality control (Nicholas et al 2020a) learning. This connection is a next step for our re- or for reverse engineering. In these applications, the search, which in this paper is constrained to image- captured 3d point cloud is used to create or update based analysis using pixel values. This approach is a 3d model from scan data of an unknown or uncer- vulnerable to changes in lighting condition, where tain physical part (Hale et al 2020, Brandt 2005). Re- darker pixels can be mistakenly interpreted or com- cent research seeks to close the loop by connecting pared as lighting conditions change. scanning to the re-manufacture or repair of damaged parts (Buican et al 2014), or the continuing robotic METHODOLOGY manufacture onto unknown geometries (Nicholas et Our methodology connects the property registration al 2020b). of a found material with its further working. To sum- To form metal into a desired shape, the RISF marise: 1. Shape and material properties are reg- process induces local strains, activating the mate- istered through 3d scanning, 2. Any zones of rust rial’s capacity for plastic deformation under tensile are classified into 3 material property parameters, 3. stress. Corrosion, an environmentally driven process Additional geometry to be formed into the base ge- of chemical degradation, can negatively impact this ometry is introduced in a CAD environment, 4. A capacity in two ways. Corrosion can be either local structural analysis is performed to generate a rigdi- - ‘pitting’ - or uniform, which affects a larger area of sation pattern for areas with high von mises stress, 5. the surface. Uniform corrosion can be simply cal- Robot toolpaths are extracted from the model, and culated as a reduction of thickness, and has a min- 6. New geometries are formed into the found geom- imal impact on the ultimate tensile strength. How- etry. These steps are expanded upon in the sections ever corrosion also causes brittleness, with rusted below. areas of metal becoming less ductile and losing its ability to have its shape changed. This behaviour Registration is more difficult to calculate but much more signifi- 3D scanning is performed with Intel’s RealSense Li- cant from the perspective of the fabrication process. DAR scanner. The found metal object is scanned in Corrosion damage can be classified via visual exam- the forming frame. An anti reflective spray is applied ination. For example, in Sweden corrosion is classi- to the metal to improve the quality of the scan and re- fied from grades A (Steel surface covered completely duce noise caused by reflectivity. A toolkit was devel- with adherent mill scale and with little if any rust.) oped to record and read the pointclouds (see figure through grade D (Steel surface on which the mill scale 2) in Rhino3D and Grasshopper, software used fur- has rusted away and on which considerable pitting ther for the digital workflow. The scanner can also

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 173 be used to register the fabrication process itself. By The geometry to be formed is manually posi- recording frames at a certain time interval, the fab- tioned on the Nurbs model of the found object. First, rication process can be optimized by analyzing the an over-sized region of interest is selected in which recorded mesh against the model in the CAD envi- the geometry should be placed, for example only ronment to measure real time forming tolerances. in the upper area of the original object. The selec- tion of this area is however only approximate, so that the position of the geometry can be adjusted to ac- Figure 2 count for local material properties. In order to ad- Registration of just this positioning, all vertices of the mesh that material properties lie within the region of interest are interrogated for their level of corrosion, with visual feedback given to the designer. The most important criterion is level 3 corrosion, which is not formable. Once positioned, the Nurbs model is converted into a mesh to per- form a structural analysis. The grasshopper plugin Processing of material properties Karamba3D is used for structural analysis. Depend- The pointcloud provides geometric information for ing on the case, different structural information can the found metal object including any dents, beads, be output, including the magnitudes and directions holes etc. Material properties impacting the sur- of principal forces. This output introduces the possi- face quality and structural properties are extracted bility to generate new geometries informed by struc- via analysis of the colour of each point within the 3d tural feedback. After the final geometry has been de- scan. Working within an HSL colour space, colours fined, the print path is generated. associated with rust are identified through their lu- minance values . As these are variable according to Robotic fabrication setup light levels, these values are determined on a case by The method has been developed and tested on two case basis. robotic setups - a UR5e and an ABB IRB 1600. Ini- All points within the pointcloud are then cate- tial testing of the fabrication process explored RISF gorised into one of three different property levels. on non-flat metal sheets. The setup for this experi- Level one means that it is almost rust-free and the ment uses a UR5e robotic arm with an 18 mm diam- forming process can be fully executed. Level two evi- eter forming tool operating with a speed of 45 mm/s dences a corrosion process from a low to medium ex- from the convex side of the curved sheet. The tool tent and forming is only possible to a limited wall an- forms a 0.3 mm aluminum sheet fixed in a jig that gle and depth. Level 3 is classified as advanced cor- clamps it to the shape of a semi-cylinder (see figure rosion in which the material has no stability and is of 3). The target geometry is half of an ellipsoid. The a brittleness that makes it unsuitable for forming. second stage of the investigation used a ABB IR160 robotic arm with an 18 mm diameter forming tool op- Placement and sizing of new geometries erating with a speed of 32 mm/s. The holding jig in- The design workflow processes two different mod- cludes three rigid belts which clamp a 0.5 mm mild els in parallel - nurbs and mesh based. The entire 3d steel sheet in the shape of a semi-cylinder (see fig- model is built as a Nurbs model and all geometric ma- ure 4). These initial tests showed that non flat form- nipulation takes place in this model. However, colour ing was possible, however that additional strategies information is obtained from the mesh vertices, and were required to prevent buckling. a structural analysis is performed on the mesh.

174 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 3 ROBOTIC STRATEGY TO COUNTERACT UR5e fabrication BUCKLING setup In order to counteract buckling, our initial strategy is to use a 3D printed holder tool attached to a second robot, which would locally support the sheet in the forming area from the concave side (see figures 5-7). The holder tool has two components, one which is fixed to the robot arm and an interchangeable head which varies with the size and shape of the formed geometry (see figure 5). The curvature of the edge touching the metal sheet is the same as the curvature given by the rig. This stops the sheet from buckling, however it also limits the freedom of design of the ge- ometry as a new head would need to be printed for each different geometry. A second design of this tool consists of a single component tool with four sup- porting points (see figure 6). This upgrade allows for more freedom of shape, but it is limited in its ability to form larger areas.

Rig-based strategy Our second strategy is the redesign of the rig further used in the second setup (see figure 8). The curved Figure 4 sheet is supported at top and bottom by metal rigid ABB fabrication belts. Two intermediate belts are integrated with setup flexible placement according to the forming location. This introduces intermediate clamping that reduces the distance spanned by the metal sheet reducing the possibility of macro scale buckling of the barrel.

Figure 5 First design of holder tool with interchangeable head

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 175 Toolpath-based strategy Figure 6 The fabrication toolpath is created by contouring Second design of the geometry in the CAD model and spiralizing the holder tool with curves for continuous movement. In the case of four supporting larger irregular geometries, it was found that the points metal sheet tends to pillow where the distance from center to edge is highest. Our strategy to reduce this tendency is to introduce a cone perimeter en- casing the initial geometry. This perimeter is formed first and the toolpath progressively transitions to the target geometry. In addition, the metal is pressed into the geometry every 15 layers (see figure 9). The metal is thus better controlled and takes on the de- sired shape.

INCORPORATING MICRO BUCKLING Figure 7 The largest geometry is formed first. In our testing, Forming using local it is typically positioned near the center of the metal support tool (back sheet. The forming of this feature increases the stiff- view) ness of the entire sheet. Secondary geometries are then formed: rigidisation pattern consisting of cone- shaped geometries informed by the structural anal- ysis from the digital workflow. Micro buckling ap- pears within the rigidsation pattern when the form- ing area is already tensed. The design language of the micro-buckling can be changed by following certain rules such as the forming order of the cones, the posi- tioning of the cones on the metal sheet, the proxim- ity between them and their geometrical aspect such as base diameter and depth. For instance, when the cones are positioned on a linear or concentric grid the buckling seam tends to follow a line-shape in be- Figure 8 tween the cones. Whereas when the cones are po- Metal rig with sitioned in a diagrid, the buckling seam follows the intermediate rigid grid pattern. By controlling the proximity between belts the cones and their size we can change the depth of the buckling seam. Therefore, when cones are closer and larger, the seam is more prominent, whereas when the geometries are further from each other and smaller there it fades away.

176 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 9 Fabrication toolpath strategy

Figure 10 CASE STUDY - DESIGN FOR A BENCH Deliberately This methodology is tested through a case study, to induced design and fabricate a bench seat from a barrel which micro-buckling reached its end of life. In this paper, the digital devel- (back view) opment and initial prototyping of this case study is emphasised. Our design speculation consists of two scenarios, using a 55-gallon and a 100-gallon stan- dard barrels. In our digital model, the barrels are cre- ated using standard dimensions and bead position- ing, whereas the rust pattern has been generated us- ing noise patterns in order to imitate randomness of weathering. The classification of the material prop- erties are divided in 3 parts according to the method described above: no rust, light rust, which allows for limited forming and heavy rust where no alterations can be made. We assume that the bead rolls on the 55-gallon barrel are formable, since they are less than Figure 11 10 mm in depth, whereas the ones on the 100-gallon Deliberately barrel are more prominent, hence not formable. Two induced different seat geometries are used, adult size and micro-buckling children size. These geometries design are fixed in or- (front view) der to follow the ergonomic aspect of a chair. Tubular legs are attached to the reinforcing beads of the bar- rel.

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 177 Figure 12 Formed rigidisation geometries

The methodology described above is applied to 0.01mm. The determination of the seat position and this setup to generate a design geometry and robotic the feet allows an exact structural analysis of the force toolpaths including the initial properties, the seat ge- transmission to the geometry. ometry and a rigidisation pattern (see figures 10 - 12). In the next step, the pattern is generated which In the case of the 55-gallon barrel (see figure 14), two is divided into several operations. First, an unequal children-sized seats are formed according to an anal- grid is generated in the Nurbs model between the ysis of the existing material and geometric aspects. seat and the edge of the barrel. Cones positioned on In the case of the 100-gallons barrel (see figure 15), this grid are sized according to the structural model, one adult-size bench is formed, however a smaller where the force in each nearest vertice is calculated. size could be a second iteration for the same rust pat- This model generates a specific force transfer on each tern. position which we use to determine the radius and Inputs to the structural analysis include the load depth of a cone of the pattern. The cones act to of an adult (80kg) or child (30kg) distributed equally stiffen and rigidify the thin metal skin under load- over the seat geometry, restraints at the leg posi- ing - a baseline comparison shows that a half-barrel tions, and material specifications defining the thick- geometry without cones has 43% performance un- ness of the material. L1 = 1mm, L2 = 0.5mm, and L3 = der a sitting load compared to the same geometry

Figure 13 Workflow for design for a bench

178 | eCAADe 39 - Digital fabrication and robotics - Volume 2 with the determined distribution of cones. Next, the proach is demonstrated through the design and pro- position of the cones is reconfigured according to totyping of bench seating elements from oil drum the rust classification. If a cone is located in an un- geometries. 3d scanning is used to enable incre- formable area (L3), it is eliminated, whereas if a cone mental forming with non flat geometries and par- is on a constrained deformable area (L2), its size is re- tially corroded materials. The forming geometries duced. Their position further depends on the reach aim to increase the structural performance of the oil of the robot arm, since in our case, the cones can- drum, while moving away from a symmetrical geom- not be formed within 60 mm from the edge of the etry in order to integrate the ergonomic function- half-cylinder. The size and depth of each geometry ality of a seat. Here, our vision-based approach to is thus determined before the final nurbs model is incorporating material properties has established a generated, from where the toolpath is extracted. It is basic pipeline for information from the found ob- then converted back into a mesh to be colourised for ject into the digital representation, and further work visual purposes and, if necessary, a second structural is planned to incorporate machine learning based analysis is performed. Toolpaths are extracted and models to categorise corrosion. The localised sup- fabrication achieved using the toolpath-based strat- port tools and toolpath based methods developed to egy described above. avoid buckling are still in development, but our use of them in these experiments has shown that both ap- Figure 14 proaches could be needed to form with non flat, non- Case study tensioned sheet metal. Further work will focus on examples - 55 testing more varied metal objects through the work- gallon barrel flow, on developing further iterations of the support tool that enable even greater forming flexibility, and altering the toolpath approach to better avoid pillow- ing during the forming of deep geometries by mov- ing iteratively outwards and deeper through multi- ple passes until the target depth is reached. Finally, in the context of material reuse, our workflow could Figure 15 support highly localised reuse and remanufacturing Case study cycles. These methods could be expanded into a examples - 100 larger scale assembly of re-used metal sheets across gallon barrel a variety of architectural and construction applica- tions, drawing on different material streams. Many potential input sources come in standardized sizes such as household appliances (fridge, dishwashers, washing machines, etc.), spiral or square duct vents, corrugated sheets, or car/airplane skin. Applications could include façade cladding, screens or canopies. Where we have worked here with single objects, in DISCUSSION AND CONCLUSION next steps formed panels will be assembled into free- This paper has introduced a workflow that connects standing spatial structures with the forming process 3d scanning, design automation and robotic fabrica- used to embed additional functionalities related to tion. The workflow aims to support the direct reuse of water flow, reflection or shading. In more conven- scrap metal through incremental sheet metal form- tional applications, the developing methods could ing, avoiding the energy cost of recycling. This ap-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 179 also enable hybrid manufacturing processes in which cremental Sheet Forming’, Robotic Fabrication in Ar- initial forming was achieved via traditional metal chitecture, Art and Design 2016, pp. 62-77 forming techniques such as stamping or deep draw- Nicholas, P, Voorderhake, D and Schork, T 2018 ’Full- scale prototype of a lightweight and robotic incre- ing, to which subsequent detailed forming is then ap- mentally formed copper facade system with stand- plied. This approach would connect the speed of in- ing seam connections’, Proceedings of the IASS Sym- stantaneous forming processes to the customisation posium 2018 achievable via incremental sheet metal forming. Nicholas, P,Zwierzycki, M, Nørgaard, E, Leinweber, S, Sta- siuk, D, Ramsgaard Thomsen, M and Hutchinson, C 2017 ’Adaptive robotic fabrication for conditions of REFERENCES material inconsistency: Increasing the geometric ac- Bailly, D, Bambach, M, Hirt, G, Pofahl, T, Herkrath, R, Hey- curacy of incrementally formed metal panels’, Fabri- den, H and Trautz, M 2014 ’Manufacturing of Innova- cate 2017 tive Self-supporting Sheet-Metal Structures Repre- Petricca, L, Moss, T, Figueroa, G and Broen, S 2016 ’Corro- senting Freeform Surfaces’, Proceedings of the Inter- sion Detection Using A.I : A Comparison of Standard national Conference on Manufacturing of Lightweight Computer Vision Techniques and Deep Learning Components - ManuLight 2014 Model’, The Sixth International Conference on Com- Brandt, J 2005 ’Skin That Fits: Designing and construct- puter Science, Engineering and Information Technol- ing cladding systems with as-built structural data’, ogy, pp. 91-99 Proceedings of the 2005 Annual Conference of the As- Yan, B, Goto, S, Miyamoto, A and Zhao, H 2014 ’Imaging- sociation for Computer Aided Design In Architecture Based Rating for Corrosion States of Weathering Buican, G, Oancea, G and Manolescu, A 2014 ’Răzvan, et Steel Using Wavelet Transform and PSO-SVM Tech- al. “Remanufacturing of Damaged Parts Using Se- niques’, Journal of Computing in Civil Engineering lective Laser Melting Technology.”’, Applied Mechan- [1] https://www.iea.org/reports/iron-and-steel ics and Materials, vol. 693, p. 285–290 [2] http://www.dovetailinc.org/report_pdfs/2015/dove Chun, P, Funatani, K, Furukawa, S and Ohga, M 2013 tailsteelrecycling0315.pdf ’Grade Classification Of Corrosion Damage On The [3] https://www.steelconstruction.info/Recycling_and Surface Of Weathering Steel Members By Digital Im- _reuse#Barriers_to_reuse age Processing’, Proceedings Of The Thirteenth East Asia-pacificConferenceOnStructuralEngineeringAnd Construction (Easec-13) Hale, L, Linley, E and Kalaskar, DM 2020 ’A digital work- flow for design and fabrication of bespoke orthoses using 3D scanning and 3D printing, a patient-based case study.’, Sci Rep 10, 7028 Jeswiet, J, Micari, F, Hirt, G, Bramley, A, Duflou, J and All- wood, J 2005 ’Asymmetric Single Point Incremen- tal Forming of Sheet Metal. DOI: 10.1016/S0007- 8506(07)60021-3’, CIRP Annals, Volume 54, Issue 2, Nicholas, P, Rossi, G, Papadopoulou, I, Tamke, M, Brandt, N and Hansen, L 2020a ’Precision Partner: Enhanc- ing GFRC craftsmanship with industry 4.0 factory- floor feedback. In’, Proceedings of eCAADe 2020 Nicholas, P,Rossi, G, Williams, E, Bennett, M and Schork, T 2020b ’Integrating real-time multi-resolution scan- ning and machine learning for Conformal Robotic 3D Printing in Architecture.’, IJAC Vol 18, Issue 4 Nicholas, P, Stasiuk, D, Nørgaard, E, Hutchinson, C and Ramsgaard Thomsen, M 2016 ’An Integrated Mod- elling and Toolpathing Approach for a Frameless Stressed Skin Structure, Fabricated Using Robotic In-

180 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Cyber-Physical Robotic Process for Re-Configurable Wood Architecture Closing the circular loop in wood architecture

Anja Kunic1, Aljaz Kramberger2, Roberto Naboni3 1,3CREATE Group - University of Southern Denmark 2SDU Robotics, The Maersk Mc-Kinney Moller Institute - University of Southern Denmark 1,3{kunic|ron}@iti.sdu.dk [email protected]

The concept of circularity implies that materials, components, systems can be re-utilized to reduce their environmental impact by extending their life-cycle. This paper discusses an approach to circular construction that revolves around transformable wood architecture. What if we can make buildings that can be assembled, disassembled, and re-assembled by robots in infinite circular loops of reconfigurations? To explore this concept, a robotic process is developed to automate the reconfiguration of timber structures, considering the material, geometric and processual challenges involved in the operations. This method entangles establishing a cyber-physical process based on visual and force feedback, the development of wood construction elements suitable for the process, the deployment of design algorithms for semi-autonomous online construction. The paper describes this setup and demonstrates its functionality through a set of experimental prototypes conceived and evaluated in a three-phase collaborative process of assembly-disassembly-reassembly.

Keywords: Robotic timber construction, Circular wood architecture, Cyber-physical systems, Robotic timber re-assembly

INTRODUCTION on how materials, components, and systems can be Material circularity is one of the critical challenges for rendered reconfigurable in time. Along with this ma- the built environment of today and the future, which terial agenda, architectural robotics plays a key role involves an in-depth reconsideration of our current in this view by enabling automated, reproducible, practices for building (Sack-Nielsen 2018). Contem- and complex construction assembly, reconfigurable porary computational tools for design and fabrica- in time. tion offer opportunities to enable circularity at dif- From an architectural technology standpoint, ferent levels (Studio RAP 2017, Jensen and Sommer the idea of reconfigurable architecture, made of dis- 2018). Design for circularity extends the temporal crete parts, is considered a premise for enhancing the horizon of architecture and requires considerations automation level in constructions by re-examining

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 181 the whole production chain (Retsin 2019). The re- bly routines, this task proves to be particularly chal- search in computational design and architectural lenging within the construction field because of the robotics looks into automated reconfigurability of high level of customization, the material impreci- wooden constructions to shift the paradigm from a sions, the assembly tolerances, the elements stabil- linear and deterministic to a circular and open-ended ity, and the forces applied during the assembly. Our architecture. Innovative solutions are emerging in approach involves using online design algorithms the design of building blocks which enable combi- to compute assembly sequences based on assembly natorial and reconfigurable assembly logics (Retsin feedback obtained through camera vision. Design et al. 2017, Sanchez 2019) and in the conceptual- for disassembly principles has been implemented in ization of assembly robots where building elements designing the wooden elements to facilitate recon- and robot mechanism work towards a symbiotic con- figurability and assembly automation. struction process to allow for continuous building and re-building of the structures (Leder et al. 2019, Robotic setup Tedbury 2017, Huang et al. 2018). The presented research is conducted using a collab- The CREATE Group at the University of Southern orative robotic cell that allows for flexible and adap- Denmark has been studying different approaches to tive assembly and disassembly. Two UR10e robots computationally enhanced circular construction, in- are mounted on reconfigurable welding tables at a volving the use of performative design (Naboni and distance that will enable their workspaces to over- Kunic 2019), reversible construction kits (Kunic et al. lap and form a collaborative area (Fig. 1). Within 2021), the use of robotics and humans for the assem- this shared space, the robots can work on the same bly of reconfigurable and reusable structures (Naboni structure and perform assembly and consequent dis- et al. 2021). assembly procedures. While most current research studies are focus- ing on the conceptualization of the new robot and Figure 1 building typologies without specific considerations Collaborative of the material and construction challenges, in this robotic setup study we investigate the possibility of a robotic circu- lar construction process for assembly- disassembly- reassembly of complex structures based on vision and haptic construction feedback and human-robot collaboration. In particular, we focus on develop- ing a cyber-physical robotic process that supports the continuous reconfiguration of timber structures while tackling in real-time emerging environmental and construction changes. The two robots are equipped with a multi-phase col- laborative assembly tool, comprising an electric 2F- METHODOLOGY 85 Robotiq gripper; a Robotiq Wrist Camera which is mounted above the gripper along the same axis; an This work explores the robotic re-assembly of trans- electric Dessouter screwdriver which is mounted per- formable timber construction. We demonstrate a pendicularly between the gripper and the camera in novel process where robots are autonomously as- a custom-made holder; a custom-made wrist switch sembling, disassembling, and reassembling sets of for human-robot interaction (Fig. 2). An additional wood construction elements. While industrial robots 3D printed screw holder with an embedded magnet are specifically well-performing in repetitive assem- is attached to the screw bit to better hold and trans-

182 | eCAADe 39 - Digital fabrication and robotics - Volume 2 port the steel bolts from the pick-up station to the as- struction process is updated in response to this feed- signed hole and vice-versa. back. Finally, the robot and screwdriver controllers make possible the execution of the robot procedures Figure 2 and the communication of the inquired construction Multi-phase end feedback back to ROS (Fig. 3). effector Figure 3 Cyber-physical workflow for adaptive reconfigurable construction

Cyber-physical communication Two types of feedback are inquired from the physi- cal domain: (i) visual feedback from the wrist cam- The computational design environment of Grasshop- era about the holes and bolts coordinates in the as- per for Rhinoceros is used as the central planning built construction; (ii) force/torque feedback from the and communication tool, where the data of both the screwdriver during the screwing and unscrewing op- physical construction setup and digital design are erations. The new coordinates of the holes and bolts stored. A cyber-physical system is established based are sent back from ROS to Grasshopper, and the dig- on a closed-loop, feedback-driven communication ital model is updated to facilitate further stages of between the two domains to coordinate the digital disassembly and re-assembly. On the other hand, and physical worlds and allow for their reciprocal de- the force and torque feedback from the screwdriver pendencies. is controlled only by ROS. This is used to understand The overall design-to-construction process is if the bolt is inserted or removed correctly and call based on the information exchange between the human assistance if the execution fails. The human three bodies: (1) Grasshopper, (2) Robot Operat- interaction is well integrated into the digital work- ing System (ROS), and (3) the controllers of the two flow through the collaborative wrist switch, and its in- UR robots and the screwdriver. In the Grasshop- volvement does not interrupt the process but allows per environment, the procedures for iterative design, for its seamless continuation. robotic simulation, and generation of the robot tar- gets are developed and communicated further to Construction system for robotic re- ROS through the Bengesht plug-in (Bengesht). These assembly include the pick, place, and travel targets related The developed assembly system is an evolutive step to manipulating the blocks, the pick/unscrew, in- from the authors’ previous works (Naboni et al. 2021). sert and travel targets related to the steel bolts, and It is based on a generic set of discrete blocks that al- the hole/bolt detection targets. The automated pro- low for reconfigurability and flexibility of the struc- grams for screwing/unscrewing and hole/bolt detec- tural layout. These are intended to aggregate orthog- tion are developed as robot skills in the ROS environ- onally with each other, and with differential density, ment. Furthermore, the feedback received from the as enabled by the connectivity patterns and the lay- physical setup is managed through ROS, and the con- out of the holes. The blocks are designed with pyra-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 183 midal morphological features (Fig.4a), which facili- Figure 4 tate the process of layered robotic assembly and dis- Performative assembly in multiple ways: features of the - Mounting precision. The male-female interlock construction between two consecutive layers guides the robotic system positioning, as the blocks tend to snap geometrically into the intended position (Fig.4b). This creates an intrinsic interdependency between the blocks that minimizes typical construction tolerances; - Rotational Locking. In each connection slot are present three-dimensional pyramid-like geometries. These are used to prevent the blocks from spinning around the bolt axis or slightly rotating out of place during the screwing process. - Improved shear resistance and moment capac- ity. As discovered in the authors’ previous work, the developed features provide stiffness to the bolted connection and enhance the ductile behavior of the assembled blocks under applied loads. This signifi- cantly contributes to the reusability of the blocks. - Minimized fabrication toolpath. The material wasted in the process of robotic milling is minimized to what is necessary to obtain a connection fea- ture that would perform as desired mechanically and assembly-wise. The milling toolpath is programmed using a single round tool that moves along paral- lel contouring trajectories (Fig.4c). Both sides of the block are milled in a single robot program. In this way, there is no need for tool changing and addi- tional operations, increasing the machining time. The blocks are intended to aggregate orthogo- nally with each other with differential density, as en- abled by the connectivity patterns and the layout of the holes.

184 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Robotic assembly / re-assembly experiment block positioners in the cell and executes the assem- The cyber-physical construction framework pre- bly steps previously described, i.e., pick and place, sented in this work is tested and evaluated on five hole detection, screwing (Fig.5-A 1-2-3) to complete experimental prototypes. The prototypes consist of a structure. twelve generic construction blocks which are aggre- Phase 02 - After this is completed, Robot 2 per- gated in different randomly generated layouts, as- forms the disassembly of the structure, following de- sembled through a layered sequence. The design, as- fined steps, i.e., bolt detection on the assembled con- sembly, and disassembly procedures are performed figuration, unscrewing, and releasing the bolts into through an adaptive and feedback-based construc- the bolt-release mini gripper, pick and re-place of the tion process which operates as a bi-directional com- blocks (Fig.5-B 1-2-3). The placing targets of the pre- munication between the design environment and vious assembly cycle become the picking targets of the real-time construction execution. the new dis-assembly cycle. Assembly and Dis-assembly. The assembly se- Phase 03 - The blocks are then re-assembled into quence and commands are planned directly from the a new structure, following the assembly sequence design model, simulated, and communicated to the steps again. robots. These involve the (1) picking and placing The three phases are repeated to obtain five dif- of the timber elements, (2) detection of the holes ferent prototype structures. where the blocks are connected, and (3) the screw- Hole Detection. The holes and bolts are de- ing procedures. Similarly, the disassembly process tected through the Robotiq wrist camera, attached is based on scanning the pre-assembled structure, directly to the robot flange. The detection is based the robotic unscrewing, extracting the timber ele- on: (1) the isolation of the hole or bolts through Mor- ments, and repositioning into a new layout, which is phological Image Processing methods, namely Dila- defined as re-assembly. These actions are performed tion and Erosion (Efford 2000), (2) the hole detection in the following sequence to construct five consecu- through contour-based object detection method tive prototypes: (Schlect and Bjorn 2011), and the (3) transformation Phase 01 - Robot 1 takes the blocks from the of the pixel information to the real-world coordinates Figure 5 Toolpath planning and sequences for the assembly (A) and disassembly (B) of the timber blocks

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 185 of the hole by homography (Negahdaripour et al. totypes (Fig.9). Vision- and force-based feedback 2005). Once the hole coordinates are detected, they were successfully implemented to overcome typi- are compared to the digital information, and an effec- cal assembly challenges due to unexpected physical tive offset is calculated (Fig.6). This is further used as changes, such as tolerances and material imperfec- a new coordinate for the screwdriver, and the digital tions. Furthermore, human assistance was used as a model is updated accordingly. secondary integrated level of target correction if the feedback-driven automatic execution failed to per- form the tasks. This ensures a highly resilient system Figure 6 that is ready to adapt to various changes and condi- Hole detection from tions while keeping the level of automation high. the wrist camera To facilitate the process of robotic assembly and at the same time assure advanced mechanical perfor- mance and reusability of the wooden blocks, a con- struction system with specific features was designed. Three-dimensional morphological features showed to be essential during the assembly processes, as the blocks were sliding in the correct position regardless of how the robot gripped them. They prevented the rotations and spinning of the blocks around the bolt axis while screwing was performed. Robotic Screwing and Unscrewing. The screwing Both assembly and disassembly processes relied and unscrewing processes are defined as robot skills heavily on computer vision feedback and showed an inside ROS. The procedure is based on two princi- average success rate of 88.7% and 83.8%, respec- pal flows, one which detects the hole and ensures tively. The disassembly procedure performance is its position is available for later use, and the second sensitively inferior (Fig.7b) because of the more chal- one which aligns the screwdriver with the detected lenging camera detection task for the bolt socket hole. To understand if the screwing and unscrewing head (3mm), in comparison to a 6mm wide free hole are performed correctly, force and torque values are (Fig. 7a). The most common reason for needing hu- monitored from the screwdriver controller. In case of man assistance is faulty hole/bolt detection due to failure, the robot automatically switches to the free- wood knots or similar dark features that cause mis- drive mode and allows an operator to assist the robot leading vision. Therefore, the hole/bolt detection by dragging the tool to the corrected position. Af- method will need further refinement to provide more ter this, a wrist switch button is used to resume the robust performance. robot process without discontinuing communication Besides the minor flaws, the system is highly rel- between the physical and digital worlds. evant and crucial to perform an autonomous robotic assembly and disassembly of discrete wood struc- RESULTS tures. The developed cyber-physical construction Extending previous research from the authors, a system was highly beneficial in realizing the pre- novel process for collaborative robotic assembly, dis- sented prototypes. It was responsible for approxi- assembly, and re-assembly based on a cyber-physical mately 96% of the successfully performed assembly setup was developed and tested. The system suc- and disassembly operations - estimated with devia- cessfully operates on different assemblies, five of tions higher than 3 mm, resulting in outside-the-hole which are presented in this paper as physical pro- projection (Fig.8).

186 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 7 Success rate of assembly and disassembly procedures

Figure 8 Distribution of deviation in the holes coordinates across the experiments

Figure 9 Three construction prototypes on which robotic collaborative assembly, dis-assembly and re-assembly were tested

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 187 CONCLUSIONS AND DISCUSSION Naboni, R and Kunic, A 2019 ’A computational frame- Circularity is a priority topic within the contempo- work for the design and robotic manufacturing of rary architectural agenda. A radical rethinking of complex wood structures’, Proceedings of eCAADe and SIGraDi 2019, Porto, Portugal, pp. 189-196 the design and construction processes is required to Naboni, R, Kunic, A, Kramberger, A and Schlette, C achieve such an aim. Architectural robotics plays an 2021, ’Design, simulation and robotic assembly of important role in this, by providing instruments and reversible timber structures’, Construction Robotics, workflows for the automation of circular construc- 5, pp. 13-22 tion. The joint application of design computing and Negahdaripour, S, Prados, R and Garcia, R 2005 ’Pla- a cyber-physical system of this research opens op- nar homography: accuracy analysis and applica- tions’, IEEE International Conference on Image Pro- portunities for easily reconfigurable wood structures. cessing 2005, Genova, Italy The presented research contributed in this sense by Retsin, G (eds) 2019, Discrete: Reappraising the Digital in introducing a robot setup and an online process that Architecture, AD, Architectural Design performs complex assembly, and for the first time dis- Retsin, G, Garcia, MJ and Soler, V 2017, ’Discrete com- assembly and re-assembly tasks, autonomously. The putation for Additive Manufacturing’, in Menges, A, impact of such a demonstration is bifold. On the one Sheil, B, Glynn, R and Skavara, M (eds) 2017, Fabri- cate, UCL Press, pp. 178-183 hand, robots are measurably upskilled to take into ac- Sack-Nielsen, T (eds) 2018, Circularity City — Shaping Our count intrinsic material complexity during assembly Urban Future, 1st Edition, Circularity City, Denmark processes, overcoming typically found assembly lim- Sanchez, J 2019, ’Architecture for the Commons. Partici- itations. On the other hand, robots begin to acquire patory Systems in the Age of Platforms.’, in Retsin, G the capacity of intervening on existing structures, of- (eds) 2019, Discrete. Reappraising the digital in Archi- fering perspectives for future automation of building tecture, AD, Architectural Design , pp. 22-29 Schlecht, J and Ommer, B 2011 ’Contour-based object and structural reconfiguration and repurposing. detection’, Proceedings of the British Machine Vision Conference, pp. 50.1-50.9 ACKNOWLEDGEMENTS Tedbury, I 2019, ’Semblr, 2017’, in Retsin, G, Jimenez, M, The work has been developed by CREATE Group from Claypool, M and Soler, V (eds) 2019, Robotic Building. Architecture in the Age of Automation, DETAIL, pp. 84- the University of Southern Denmark, in collaboration 85 with SDU Robotics and Industry 4.0 Laboratory. The [1] https://studiorap.nl/#/abnamro authors would also like to thank Mathias Neerup and [2] https://www.food4rhino.com/app/bengesht Antonios Karageorgiou for their contributions to de- [3] https://vimeo.com/304108480 veloping the cyber-physical communication system.

REFERENCES Efford, N (eds) 2000, Digital Image Processing: A Practical Introduction Using JavaTM, Pearson Education Jensen, GK and Sommer, J (eds) 2018, Building a Circular Future, 3rd Edition, 3XN Architects, Denmark Kunic, A, Naboni, R, Kramberger, A and Schlette, C 2021, ’Design and assembly automation of the Robotic Re- versible Timber Beam’, Automation in Construction, 123, p. 103531 Leder, S, Weber, R, Wood, D, Bucklin, O and Menges, A 2019 ’Distributed Robotic Timber Construction’, Pro- ceedings of ACADIA 2019: Ubiquity and autonomy, Austin, Texas, pp. 510-519

188 | eCAADe 39 - Digital fabrication and robotics - Volume 2 A Hybrid Robotic Construction Approach in Large Scale

The example of a tree-like timber branching structure development

Odysseas Kontovourkis1, George Tryfonos2 1,2Department of Architecture, University of Cyprus 1,2{kontovourkis.odysseas|tryfonos.george}@ucy.ac.cy

This paper demonstrates an integrated approach for the development of a complex timber structure through a hybrid model of robotic and conventional construction. Specifically, a tree-like branching structure is parametrically developed and optimized in terms of material waste reduction during robotic cutting of variable structural components in size and angle. Also, the position and angle of joints are determined through robotic marking of wooden components in a continuous robotic workflow. This is followed by their conventional assembly into larger structural elements and then into an overall structure. The physical results are evaluated in terms of cutting and assembly accuracy of wooden parts based on a 3D scanning approach. The results show manufacturing deviations, both in cutting and assembly stages, which are executed by the robot and the carpenters respectively. These results provide useful inputs that enables a more thorough and productive consideration of the application of robotic technology and human involvement in the construction industry.

Keywords: Hybrid construction, Tree-like branching structure, Robotic cutting, Manual assembly, Accuracy

INTRODUCTION fer to off-site processes, where the use of automa- The application of automation and robotics in tim- tion and robotics is used to optimize the production ber construction industry is rapidly evolving during process of wooden members, which are then trans- the last decades with examples worldwide being de- ported on-site for assembly. The benefits of this pro- veloped and implemented in large scale (Schwinn et cess over conventional woodworking methods are al., 2012; Willmann et al., 2016). Also, during this obvious and are related to increased accuracy but period, research has been done on the effective use also productivity along with other benefits that the of automation techniques for the industrial produc- use of wood itself can offer such as reduced environ- tion of conventional standardized structures, such as mental impact in relation to other materials (Altaf et prefabricated houses, but also for the development al., 2014). of bespoke shapes, such as complex structural sys- One of the first examples discussed relevant is- tems. In both cases the majority of examples re- sues was the work by (Leonard et al.,1990), where

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 189 an expert process planning system based on a semi- man assistance. intelligence process selector was integrated with an Most of the examples that deal with robotic tim- industrial robotic arm, which was applied for cutting ber construction have examined and deepen into and assembly in wood truss fabrication. Since then, specific aspects of research and experimentation, other examples have been developed in different di- which were evaluated through small-scale construc- rections, mostly for cutting and assembling wooden tions in the laboratory. This is due to the limitations structures, for instance in the work by Gramazio and and challenges that a robotic woodworking process Kohler, both in the level of experimentation with ex- as a whole can involve. These may be related to the amples such as ‘The Sequential Wall’ in 2008, as well level of technological readiness, the resources avail- as in the level of application in construction sector able and the emphasis placed on specific aspects of such as ‘The Sequential Roof’ in 2016 (Willmann et al., the process in each case. Also, the scale of structure, 2016). the time required for completion, the available hu- Thus, recent studies have focused on exploring man workforce and the construction budget, which the possibilities offered by automation and robotics influence the decisions taken during real scale sce- in terms of process accuracy and productivity, along narios. with the introduction of emerging technologies, Faced with similar challenges, this project which could offer additional possibilities for improv- demonstrates the fabrication of a large-scale tree- ing the timber construction processes. In the work like timber branching structure using robotic prin- by Kyjanek et al. (2019) the use of Augmented ciples by referring to a hybrid approach, where au- Reality (AR) technology was introduced in order to tomation and traditional construction processes are achieve efficiency and productivity but also an effec- integrated into a single methodological framework. tive workflow of human and robot collaboration in Aim is to understand and deepen into the correlation the assembly of timber structures. In other examples, between accuracy during robotic execution and tol- 3D scanning is used to check raw timber parts, which erances occurred due to conventional tools and ma- were used in the construction of wooden structures terial failures but also due to the human involvement. in order to increase productivity (Someya et al., 2020), Ultimate goal is, through a hands-on procedure, to to accurately produce timber structures consisting of understand possible advantages and disadvantages raw timber material (Vestartas and Weinand, 2020) of the automated process over the conventional one but also to evaluate the accuracy of complex wooden that has similar goals; to design and construct one- structures after their production through the correla- off complex structural systems. Finally, in a broader tion of scanned result with the initial design (Hasan context, this work aims to discuss the sustainable et al., 2019). potential of robotic timber construction in the con- Apart from all the above, an integral part of a struction sector. holistic automated procedure in robotic timber con- The structure of the paper consists of the fol- struction is design and optimization. In the work lowing sections. Initially, the section that describes by Søndergaard et al. (2016) topology optimization the hybrid robotic construction workflow, which in- principles were applied for the design of a truss- volves three distinguished steps that are described based bespoke timber structure and then assembled in respective sections dealing with different aspects using robotic principles. Similarly, in the example of the process. Then, each section is presented by Kunic et al. (2021) Finite Element Analysis (FEA) separately, firstly, the parametric design and opti- and Topology Optimization (TO) was performed for mization of tree-like timber branching structure, sec- the optimization of a timber beam structure using a ondly, the robotic and conventional construction ex- workflow that incorporated robotic assembly and hu- ecution and thirdly, the evaluation of results. Finally,

190 | eCAADe 39 - Digital fabrication and robotics - Volume 2 some general conclusions are drawn. larger structural elements at the construction site. Then, the wooden components are joined together HYBRID ROBOTIC CONSTRUCTION WORK- in order to complete the overall structure. FLOW In this paper, issues under investigation are related to the cutting and assembly accuracy of The work presented in this paper demonstrates an in- wooden parts. The results, both in terms of individ- tegrated approach for the development of a complex ual timber members but also the overall structure, are timber structure through a hybrid model of robotic evaluated by correlating the shape and size of digital and conventional construction. The development of structural members as well as their join angles with the structure follows a series of steps: a. Paramet- the respective physical results through a 3D scan- ric design development and optimization, b. Robotic ning procedure. Results provide useful ground for and conventional construction, which is divided into the quantitative/qualitative assessment of the devi- three sub-tasks; Robotic cutting and marking, Con- ations occurred during robotic and conventional ex- ventional assembly of trusses, and Conventional as- ecution throughout the process till the final assembly sembly of the overall structure, and finally c. Cutting on-site. and assembly accuracy evaluation (Figure 1).

Figure 1 PARAMETRIC DESIGN AND OPTIMIZATION The methodology Tree-like branching structure design for the The structure refers to a shelter with maximum gen- development of the eral dimensions of approximately 3.5m wide, 7m complex timber long and 3m high, which functions as seating space structure for disabled and their accompanied persons in the forest park of Limassol city (Figure 2). The design of structure considers the needs of the people hosted and more specifically metric characteristics related to the dimensions of wheelchairs and access of both disabled and accompanied persons.

Figure 2 The tree-like timber branching structure that creates sitting Specifically, the parametric design development and place under the optimization of the tree-like branching structure en- roof for both ables the control of variable structure’s components disabled and size and angle. Simultaneously, this informs the accompanied robotic cutting process, achieving in parallel accu- persons. The racy and material waste reduction. At the same time, disabled persons the suggested robotic approach determines the po- The goal is to supports sociability and interaction can use the area sition and the angle of the connection of the mem- between different users with special needs without between the seats. bers through their marking in a continues workflow their isolation and alienation by the community. The so that timber components are ready for assembly. design of table and seats serve users‘ needs for rest- At a later stage, the automated process is followed by ing and the design of roof (with dimensions slightly the conventional assembly of the components into larger than the seating area) serves the need for basic

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 191 sun protection. Hence, the basic concept of shelter is (Figure 3.D). formulated as a tree-like branching structure that al- The primary structural elements of trusses are ac- lows free space underneath in order to meet users’ curately joined together based on a predefined join- needs. ery system that consists of double and single timber The main parts of the branching structure re- members in each joint. Hence, the dimensions of fer to the 2 verticals columns that are located sym- timber members are calculated to allow an effective metrically in the middle of it and the primary struc- robotic cutting process off-site based on the reach tural elements in the form of trusses, which are sup- length of the robotic arm but also based on the ef- ported by the 2 columns. Consequently, the pri- ficient transportation and assembly on-site. The dis- mary trusses support the seats, the table and the tribution of single or double timber members in a roof. The columns are consisted by 4 timber mem- primary truss is determined by the number of nodes bers in rectangular cross section 75x75mm, which representing joints. These members are allocated al- are connected with 3 cross elements with dimensions ternately (single-double timber members) in order to 200x50x75mm, creating the rectangular columns allow rigid joint development between each other with overall section 200x200mm. Also, the structure without the addition of metal joints. Thus, the dis- consists of 30 structural trusses, which are joined to- tance between nodes and hence joints determines gether to make the entire structure. These primary the length of the timber elements but also their rota- trusses differ in their dimensions and their placement tional angle. In addition to the differences occurred angle based on their static role in the overall struc- in length, the cutting angle of members is differen- ture. More specifically, 10 structural trusses support tiated due to their join angle and this results in a the roof, 10 trusses the table and 10 trusses the seats large number of bespoke members with shape vari- (Figure 3A). ability. Briefly, the total number of timber members for the primary truss structure is 311, consisting of double members with rectangle section of 75x25mm Figure 3 and single members with rectangular dimension of The primary and 75x50mm. These members exist in various lengths secondary parts of ranged from 424.41mm to 1693.35mm and in various the branching angles ranged from 29o to 76.84o. structure: A. The 2 main columns and Parametric control and optimization of tim- the 30 primary ber members structural trusses, B. The overall structure and the individual timber mem- The secondary bers are designed and parametrically controlled via structural system the Grasshopper environment [1] (plug-in for Rhino for the table and The secondary structural systems consist of triangu- [2]). Initially, exact cross section dimensions and cut- the roof, C. The lar units that are used for the stability of the table and ting angles for each timber member based on prede- timber the roof (Figure 3.B). These consist of timber mem- fined nodes’ position are defined. This allows further traverse/transverse bers in different dimensions and join angles, which control of every single timber member as regard its members, D. The 6 are assembled together to formulate the horizontal size (length and cross section), its accurate placement tribbles of timber plate for the table and the three-dimensional plate and rotational angle in trusses but also allows shape members and the for the roof. On top of them, timber traverse/trans- optimization of the overall structure. The implemen- cross beams. verse members formulate the table and roof surfaces tation of parametric design and control principles, (Figure 3.C). Finally, 6 tribbles of timber members for- has an additional twofold purpose, firstly, to optimize mulate the seats, which are supported by crossbeams

192 | eCAADe 39 - Digital fabrication and robotics - Volume 2 the use of timber materials by packing all timber el- in order to avoid collisions of timber members with ements, achieving minimum material wastage, and the robot, the tools (circular saw and marker) and the secondly, to seamlessly connect the robotic cutting boundary walls. Also, attention was given on the po- and marking process of timber members using the sitioning of available raw timber rods/containers to robotic simulation program Taco ABB [3], a plug-in for be inside the range of robot reach length (Figure 4). Grasshopper. Figure 4 The highly diversified timber members as regard The robotic set-up their length and join angles, but also the need to re- showing the duce the cost of materials and to minimize material placements of all wastage, lead to the introduction of a methodology tools. With red for optimal utilization of the largest possible percent- colour is the area of age of available wood. This is achieved through the wood placement. implementation of a bin-packing algorithm (Crainic With magenta are et al, 2008), which is applied by using the PackRat the raw timber plug-in [4] (plug-in for Grasshopper). The aim of this rods/containers procedure is to minimize the used space by packing that are picked by rectangular shapes in rectangular containers, an idea the gripper before that is extended in this study, where the use of mini- cutting (image mum available material and material wastage are the shows the position primary objectives for cost and environmental im- of timber members pact reduction (see previous work done by authors The Taco ABB plug-in is used for the offline simula- after cutting based (Reference will be added after blind review)). on the bin-packing tion of the available ABB industrial robot IRB2600- algorithm). With 20/1.65. Apart from the robotic movement program- ROBOTIC AND CONVENTIONAL CON- green colour is the ming, custom commands that control gripper be- circular saw STRUCTION haviour (opening and closing) are incorporated into position and with Robotic cutting and marking process the overall program, which is generated in RAPID pro- blue is the marker Robotic simulation. The robotic involvement in the gramming language and fed into the IRC5 controller position. All the construction of the proposed structural system as for further execution. tools are placed on the next step in this hybrid robotic construction pro- In particular, the generated program of robotic a metallic base cess begins with the robotic simulation. This involves construction behaviour involves: picking the timber members based on the appropri- together with the • Picking the appropriate timber element for cut- ate length defined by optimization process, cutting robot and in a ting based on the distribution order of avail- the members in the right length and angles of joints specific reach able raw timber rods/containers, which is spec- by a circular saw, and finally, marking in both edges length in order to ified during bin-packing optimization using a of timber members for screwing and hence joining allow an effective custom-made gripper (Figure 5.B). elements in larger structural elements. execution of the • Cutting the length and join angles in both edges Initially, the robotic set-up in the laboratory, robotic process. of timber members using a circular saw. This be- which involves the right positioning of all tools par- haviour differs according to the timber member ticipating in the process and the calibration with the and is related to the number and angle for each robot during its movement behaviour, has been con- joint (Figure 5.C and Figure 5.D). ducted. Also, technical specifications of the robot • Marking each timber edge based on the number that were related to the reach length but also the of screws defined in each joint using a custom- boundaries of the laboratory have been considered

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 193 made marking tool (Figure 5.E). is used for marking both edges of timber members. •Placing the timber members on the robotic base As it has been described in the robotic simula- by the robot and then numbering and classifica- tion part of this paper, the robot with the assistance tion by the robot operator (Figure 5.F). of the gripper picks the appropriate raw timber rod from the available material container. The selection is done automatically based on the results of bin- Figure 5 backing optimization that attempts to pack as many Basic steps in as possible timber elements in the available timber robotic simulation. containers. The order of picking the available timber A. The initial parts is determined by the dimension of the existing position of the leftover rods. robot. B. The Then, the robot places the wood rods on the saw algorithm moves to cut the timber members in the appropriate length the robot at the and in a direction perpendicular to cutting direction. picking position This procedure is repeated twice as the number of and activates the timber elements edges. Consequently, the robot ro- gripper. C. The tates the timber element on Z axis to cut the edges of algorithm moves the wood at the right angle in order to achieve pre- the robot near the cise joinery contact of the timber member in each circular saw. D. The joint (Figure 6). Due to the length of timber rods wood is cut at the very beginning, the robot operator may assist accordingly to the the robot by holding the remaining material after this length and joint is positioned on the circular saw. Also, after timber angle parameters. member cutting the robot operator is responsible to E. The wood join is replace the leftover timber rod on its initial position been marked. F. A seamless workflow is used for all the above steps for repeating the cutting process. The member is that is executed as a single toolpath, which involves Finally, the robot holds and moves the timber el- placed on the base several Tool Centre Points (TCPs), waiting time peri- ement to the marking tool, which consists of a marker by deactivating the ods and custom IO inputs for gripper control. The at an appropriate height. The timber member is gripper. G. Toolpath toolpath is distinguished according to the cross di- marked based on screwing points at both ends with generation for the mensions of each member under robotic execution. the assistance of robot. Finally, the robot places the processing of two The waiting time period is added in order to allow member at the base of the shop floor for numbering (2) wood members. necessary time for robotic assisted tasks, mainly by and classification by the robot operator. the robot operators, and are related to; a. reposition of remaining raw timber rods/container after each Conventional assembly of trusses timber member cutting, b. removal of leftover parts After cutting, marking, numbering and classification after robotic cutting of timber member. of the timber members, these are transported to a storage area near the site, which is located in the De- Robotic execution. The robotic execution of cutting partment of Forest in Limassol city, for their assembly and marking is achieved by the available ABB robot into trusses and then into an overall structure. Ini- together with the use of a custom-made gripper tool tially, the timber members are classified based on the mounted at the edge of the robot for handling the numbering occurred during robotic execution in the timber members. Also, a conventional circular saw is laboratory. Then, members are installed and joined used for material cutting and a custom-made device

194 | eCAADe 39 - Digital fabrication and robotics - Volume 2 together with clamps for the temporary creation of starts from the placement of the two columns. Then, the trusses until their screwing. The correct place- the primary timber trusses, both for the roof, the ta- ment of the timber elements in their proper posi- ble and the seats. Then the structural elements are tion based on their cutting angles and the configu- joint together in the middle of the structure. In the in- ration of their joints is an important aspect towards stallation process, unavoidable deviations compared the configuration of the overall structure as this de- with the initial digital model occurred, both in the termines the correct assembly of the parts. This is length and in the angle of installation due to the in- done manually and involves several difficulties, such accuracy determination of angles during the assem- as inaccurate determination of explicit joint angle of bly of primary trusses. The placement of the trusses the timber members as there are no available tools is followed by the on-site placement of secondary tri- that could help towards this direction, other than the angular units, both for the table and the roof. Finally, guidance provided by members’ cutting angles. the timber traverse/transverse members for the roof and table are measured, cut and assembled on the Figure 6 spot based on the existing dimensions. This is fol- Robotic cutting lowed by the placement of timber members for the process using a seating area (Figure 7). traditional circular saw. CUTTING AND ASSEMBLY ACCURACY EVALUATION The digitization of the structure for accuracy eval- uation is conducted by using the 3D laser scanner Faro 70M. The use of laser scanner determines the actual georeferenced 3D model and the accuracy re- Figure 7 lated to the robotic cutting and conventional con- On-site assembly of struction process. In regard to the georeferenced tree-like branching 3D model, 12 scan locations are performed. Then, structure. the scan locations are merged in a single unify point cloud using Faro Scene software [5] and georef- erenced using the Cyprus Geodetic Reference Sys- tem 1993, CGRS93/Cyprus Local Transverse Mercator (LTM) (EPSG:6312) [6]. In order to evaluate the ac- curacy of the structure and to compare the physical prototype with the digital model, the results are ex- ported in E57 point cloud format and imported in Rhino software. This allows comparison of the 3D point cloud with the 3D geometry of the structure based on quantitative/qualitative observation of the Assembly of the overall structure deviations occurred in order to check and evaluate In the final part of the process, the trusses are trans- the construction accuracy. ferred to the construction site for their assembly and completion of the entire structure. This stage is per- formed by two to three people including a carpenter, an assistant and the robot operator. The assembly

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 195 The detail (A) shows the connection of a column with Figure 8 trusses related to the table and the roof, the detail (B) Top. The 3D laser shows the connection of a truss with the roof in the scanned model and middle of structure, the detail (C) shows the result of the overall accuracy a truss assembly and the detail (D) shows a selected evaluation. Bottom. joint of a timber truss. Four selected Figure 9 shows the detail (A), where the column details of the placement on the ground has been affected by the tree-like structure ground slop. In addition, the connection with truss for accuracy shows a maximum assembly deviation of 47.47mm, evaluation. which is due to a small displacement of column at Z-axis and tolerance of ±5-10mm occurred during wood planning of timber members by the carpen- ter. These deviations can be avoided in future wood- working construction by considering the wood plan- ning from the initial design stage.

Figure 9 The selected detail (A) of a column that shows the maximum deviation of 99.91mm. This is due to the wrong placement of the 3 benches placement affected by sloping ground. Also, detail (A) shows a maximum deviation of 47.47 caused by Figure 8 shows the overall evaluation of the tree-like the planning of branching structure that indicates a maximum de- timber members viation of 99.91mm. This occurs at the bottom of and the small the structure between the 2 main columns and the 3 The detail (B) in Figure 10 shows the connec- displacement of benches. During the conventional assembly on site, tion of trusses in the middle, formulating the roof column. this part of the structure was anchored with the slop- structure. An assembly deviation is observed that ing ground, a stage that was not considered in the has been caused by wood planning or wood thick- parametric design phase of the process, leading to ness tolerance of 12.26mm. Furthermore, small devi- unpredictable large deviation results. Subsequently, ations are observed due to assembly inaccuracies by the accuracy evaluation involves 4 main selected de- the carpenters that affect the accurate placement of tails of structure in different level of scaling for a com- timber traverse/transverse members. This results to prehensive observation of the results obtained. The a maximum of 3.90 of angle deviation in the joint of bottom image in Figure 8 demonstrates the 4 details.

196 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Figure 10 The details Left.(B) and Right.(C) shows the results of accuracy evaluation related to the trusses in the middle of roof structure, the timber length and angle.

truss. CONCLUSIONS The evaluation of the selected truss in detail (C) The physical results derived from the suggested hy- that is demonstrated in Figure 10 shows minor devi- brid robotic construction approach show great po- ations occurred due to the robotic algorithm, which tential of implementation. This is obvious in cases was responsible for the robotic cutting of timber where the design and construction tasks are com- members in the given lengths and angles. In detail, plex, and particularly when there is large variability the deviations of robotic cutting angles have been in timber members that consists the overall structure. calculated from 0o to 2.56o and of cutting lengths In this case, robotic involvement is important for an from 1.9mm to 8.08mm. This can be improved in accurate, reliable and fast production process of in- future studies by considering other parameter in- dividual timber members. However, when the pro- volved in robotic cutting process such as the circular cess involves assembly tasks on-site, due to their in- saw disk thickness and vibrations of the gripper end- creased complexity the contribution of human work- effector tool. force is desirable. On the other hand, in a fully au- Additionally, the evaluation indicated that major tomate process, the robotic set-up as well as the deviations are caused due to the assembly of trusses, processing time might be unaffordable, especially in the differences between the timber cross sections in large scale structures. Nevertheless, as it has been digital and physical environment and the tolerances observed from the hands-on experiment that is pre- occurred due to the wood planning. In the detail sented in this paper, either semi or fully automated (D) the comparison shows that the cross section of construction approaches might result in a number digital members is 4.2mm - 6.3mm larger than the of errors and deviations between physical and dig- scanned physical members. In addition, in the de- ital models. These might be related to innumer- tail (D) the digital model of the double timber mem- able factors that among others include materials and ber in section has an area of 3750mm2 whereas the tools tolerances, errors during human involvement as same member in scanned model has an area approx- well as metric differences between physical and dig- imately 2555,89mm2 that is 1194,11mm2 or approx- ital space of implementation. The above challenges imately 68% lower. This results inaccuracy in joints could give rise to further research for a future fully au- and consequently deviations in rotation angles of tomated process. members that cause assembly errors. Regarding the suggested workflow that apart

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 197 from hybrid robotic construction stage deals with in Timber Prefabrication’, 36th International Sympo- parametric design and control as well as optimiza- sium on Automation and Robotics in Construction (IS- tion, this is found to be useful for a number of rea- ARC 2019), pp. 1223-1230 Leonard, EB and Livingston, EE 1990, ’A prototype for sons. These include the ability of the process to con- intelligent computer integrated wood truss fabri- trol effectively the dimensions of timber members cation’, Robotics and Autonomous Systems, 6(4), pp. and the overall structure during different stages, the 337-349 opportunity to obtain optimum control of robotic Schwinn, T, Krieg, OD and Menges, A 2012, ’Robotically picking and cutting of members and hence achiev- fabricated wood plate morphologies. Robotic pre- ing minimization of cost and material wastage, and fabrication of a biomimetic, geometrically differen- tiated, lightweight, finger joint timber plate struc- finally the seamless process from design intention to ture’, in Brell-Çokcan, S and Braumann, J (eds) 2012, robotic execution. These benefits might open the Robotic fabrication in Architecture, Art and Design, discussion on the use of robotic approaches versus Springer, p. 48–61 conventional ones, and in generally, might demon- Someya, S, Ikeda, Y, Hotta, K, Tanaka, S, Hayashi, M, strates its sustainable potential in the construction Jokaku, M and Takahashi, T 2020 ’Research on a sector through measurable indicators. Method to Consider Inspection and Processing for Atypical Wood Members Using 3D Laser Scanning’, 37th International Symposium on Automation and ACKNOWLEDGEMENTS Robotics in Construction (ISARC 2020), pp. 1203-1210 We would like to thank the Department of Forests Søndergaard, A, Amir, O, Eversmann, P, Piskorec, L, Stan, and the University of Cyprus for their great support. F, Gramazio, F and Kohler, M 2016, ’Topology opti- mization and robotic fabrication of advanced timber Also, we would like to thank Kyriakos Kyriakou, Anto- space-frame structures’,in Reinhardt, D, Saunders, R nis Yiapanis, George Vessiaris, Loucas Louca, Philip- and Burry, J (eds) 2016, Robotic fabrication in Archi- pos Michael, Elena Koufopavlou, Panagiota Konatzii tecture, Art and Design, Springer and all other people who helped in different stages Vestartas, P. and Weinand, Y 2020 ’Laser Scanning with of the project. Industrial Robot Arm for Raw-wood Fabrication’, 37th International Symposium on Automation and Robotics in Construction (ISARC 2020), pp. 773-780 REFERENCES Wagner, HJ, Alvarez, M, Kyjanek, O, Bhiri, Z, Buck, M and Altaf, MS, Al-Hussein, M and Haitao, Y 2014 ’Wood- Menges, A 2020, ’Flexible and transportable robotic Frame Wall Panel Sequencing Based on Discrete- timber construction platform – TIM’, Automation in Event Simulation and Particle Swarm Optimization’, Construction, 120, p. 17 Proceedings of the 31st International Symposium on Willmann, J, Knauss, M, Bonwetsch, T, Apolinarska, AA, Automation and Robotics in Construction and Mining Gramazio, F and Kohler, M 2016, ’Robotic timber (ISARC) construction. Expanding additive fabrication to new Crainic, TG, Perboli, G and Tadei, R 2008, ’Extreme Point- dimensions’, Automation in Construction, 61, p. 16– Based Heuristics for Three-Dimensional Bin Packing’, 23 INFORMS Journal on Computing, 20(3), pp. 368-384 [1] http://www.grasshopper3d.com/ Hasan, H, Reddy, A and TsayJacobs, A 2019 ’Robotic Fab- [2] https://www.rhino3d.com/ rication of Nail Laminated Timber’, 36th International [3] http://blickfeld7.com/architecture/rhino/grasshopp Symposium on Automation and Robotics in Construc- er/Taco/ tion (ISARC 2019), pp. 1210-1216 [4] http://yconst.com/software/packrat/ Kunic, A, Naboni, R, Kramberger, A and Schlette, C 2021, [5] https://www.faro.com/en/Products/Software/SCEN ’Design and assembly automation of the Robotic Re- E-Software versible Timber Beam’, Automation in Construction, [6] https://georepository.com/crs_6312/CGRS93-Cypru 123, p. 17 s-Local-Transverse-Mercator.html Kyjanek, O, Al Bahar, B, Vasey, L, Wannemacher, B and Menges, A 2019 ’Implementation of an Augmented Reality AR workflow for Human Robot Collaboration

198 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Material-informed Formwork Geometry

The effects of cross-sectional variation and patterns on the strength of 3D printed eggshell formworks

Joris Burger1,Timothy Wangler2, Yu-Hung Chiu3, Chanon Techathuvanun4, Fabio Gramazio5, Matthias Kohler6, Ena Lloret-Fritschi7 1,5,6Chair of Architecture and Digital Fabrication, ETH Zurich 2Institute for Build- ing Materials, ETH Zurich 3,4MAS in Architecture and Digital Fabrication, ETH Zurich 7Chair of Architecture and Digital Fabrication & Institute for Building Ma- terials, ETH Zurich 1,5,6,7{burger|gramazio|kohler|lloret}@arch.ethz.ch [email protected]. ethz.ch 3,4{yuchiu|ctechathu}@student.ethz.ch

Fused deposition modelling (FDM) 3D printing of formworks for concrete has the potential to increase geometric freedom in concrete construction. However, one major limitation of FDM printed formworks is that they are fragile and often cannot support the hydrostatic pressure exerted by the concrete. The research project `Eggshell' combines robotic 3D printing of formwork with the casting of a fast-hardening concrete to reduce hydrostatic pressure to a minimum. Eggshell can be used to fabricate architectural-scale building components; however, knowledge of the influence formwork geometry has on the hydrostatic pressure resistance is still sparse, resulting in unexpected breakages of the formwork. This paper presents an empirical study into the breakage behaviour of FDM printed formworks when subjected to hydrostatic pressure. Firstly, the study aims to give a first insight into the breakage behaviour of formworks with a constant cross-section by casting a self-compacting concrete into the formwork until breakage. Then, we investigate if three-dimensional patterning of the formwork can have a beneficial effect on the breakage behaviour. Finally, the preliminary results are validated through the fabrication of two full-scale columns. The empirical results point towards the fact that sharp corners in formworks are weaker compared to rounded corners. Although the presented results are still preliminary, they mark an important step in the development of reliable design and fabrication strategies using 3D printed formworks.

Keywords: 3D Printing, Formwork, Fused Deposition Modelling, Digital Concrete, Hydrostatic pressure, Eggshell

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 199 INTRODUCTION Therefore, the research described in this paper In recent years, research on 3D printing of formworks aims to: for concrete using fused deposition modelling (FDM) 1. Give a first insight into the breakage behaviour has emerged, as this technique enables to expand of formworks with a constant cross-section. the geometrical possibilities of concrete (Jipa et al., Data is collected on the breakage height (max- 2017; Leschok and Dillenburger, 2019; Naboni and imum height of concrete that can be filled be- Breseghello, 2018; Peters, 2014). However, one ma- fore breakage) by casting self-compacting con- jor limitation of FDM 3D printed formworks is that crete into the formworks without using the fast- they are typically fragile and need additional sup- setting concrete. port structures to withstand the hydrostatic pressure 2. Investigate if patterns can have a beneficial ef- exerted by the fresh concrete. Indeed, addressing fect on breakage behaviour. It has been shown the hydrostatic pressure is the biggest challenge that in previous studies that surface patterning of projects using FDM 3D printed formworks face. formwork can aid to counter shrinkage occur- In contrast to the beforementioned projects, the ring from the 3D printing process (Burger et research project ‘Eggshell’ (Burger et al., 2020a) ad- al., 2020a). To investigate if similar patterns dresses this challenge by combining robotic FDM 3D can have a beneficial effect on the breakage printing with the digital casting of a fast-setting con- behaviour of the formworks, several patterned crete (Lloret et al., 2015; Reiter et al., 2020). Us- formworks are tested and compared. ing the fast-setting concrete, hydrostatic pressure 3. Validate the preliminary results through the fab- can be reduced, and a thin, 3D printed shell can rication of full-scale demonstrators. be used as formwork. The Eggshell process con- sists of the following steps: 1) 3D Printing of a form- work, 2) casting of fast-setting concrete, and 3) plac- MATERIALS AND METHODS ing reinforcement. It is important to note that hy- Design drostatic pressure is not entirely eliminated by using The formworks are designed using the 3D modelling the fast-setting concrete but merely reduced to an tools Rhino 3D and Grasshopper, with the aid of the amount that the thermoplastic formwork can with- Python programming language. The designs were stand. The reduction of hydrostatic pressure is es- made to be parametric, enabling fast modification of sential to the process since this allows for filling tall the design based on the results from casting. One ad- formworks without the need for printing additional ditional advantage of scripting the designs is that the support, which significantly increases printing time. developed patterns can now be applied to any input To be able to fill a formwork with fast-setting con- surface, not just the geometries described in this pa- crete successfully, it is crucial to have information per. Additionally, the fabrication data is directly gen- on the effect of the hydrostatic pressure of the still- erated from the Grasshopper scripts, eliminating the setting concrete on the formwork. Hydrostatic pres- need for the additional step of slicing the geometry. sure on conventional wooden and steel formworks has been studied extensively and is a common con- 3D Printing sideration in the design of standard formworks. How- The designs are 3D printed using the Eggshell print- ever, the effect of hydrostatic pressure on thin, 3D ing setup described by Burger et al. (2020a). A printed formworks has, so far, not been studied. For Universal Robots UR10 robotic arm with a custom- 3D printed formworks to be adopted as a viable con- developed extruder printhead is used. The printing struction method, it is essential that more knowledge setup allows for the fabrication of parts measuring is available on their breakage behaviour. 1.2 x 1.2 x 1.2 m. Polyethylene terephthalate glycol

200 | eCAADe 39 - Digital fabrication and robotics - Volume 2 (PET-G) filament with a diameter of 2.85 mm is used age height (maximum height of concrete that can be as the printing material. Printing parameters were filled before breakage) by the setting time (defined as kept constant throughout all the experiments pre- the time in minutes after which the concrete can sup- sented: a layer height of 1.0 mm, printing speed of port its own weight), resulting in Eq. (1): 45 mm/s, nozzle diameter of 1.5 mm, and nozzle tem- breakage height [mm]) perature of 210 degrees were used. The layer width max filling rate [mm/min] = setting time [min] used is 1.5 mm, and since all formworks consisted of a single layer for the wall, all formworks shown have (1) a constant wall thickness of 1.5 mm. Following Eq. (1), a higher maximum filling rate can As a result of the printing process of the thin, be obtained in two ways: single-walled formworks, it has been observed that 1. increasing the breakage height (by designing a slight print deformations can occur when long, stronger formwork) straight lines are printed (Burger et al., 2020a). This 2. decreasing the setting time of the concrete (by warping behaviour has also been verified by others using a faster setting concrete) (Wijnen et al., 2018). In this paper, this will not be studied in-depth, but will be mentioned when rele- In this paper, the parameter under investigation is vant for an experiment. the breakage height, with the setting time of a mate- rial considered as a given. A discussion on adjusting Casting the setting time of concrete can be found in Reiter The evaluation of the printed formworks (Section et al. (2020). For the experiments described in ‘Con- ‘Constant cross-section’ and ‘Patterns’) was done by stant cross-section’ and ‘Patterns’ the setting time of filling them with small batches of a retarded, self- the self-compacting, retarded concrete is very long compacting concrete at a constant filling rate. It is (around 8 hours). This ensures no setting of the con- important to note that, unlike in the regular Eggshell crete takes place during the experiments. fabrication process, no fast-setting concrete was In ‘Demonstrators’, two different concrete mixes used, as hydrostatic pressure needed to be present are used to cast the two demonstrators: Mix A to evaluate breakage behaviour. The retarded self- (mix design undisclosed), and Mix B, as described in compacting concrete (SCC) mix design used is the Burger et al. (2020a). Mix A has a setting time of same as used by Burger et al. (2020a) and has a den- 10 minutes, whereas Mix B has a setting time of 50 sity of 2.29 g/cm3. For each experiment, the form- minutes. The filling rate is, therefore, adjusted ac- work was placed in a box and connected to the base cordingly. Demonstrator 1 was cast with Mix A, us- with hot glue. The concrete was then cast in inter- ing a digital casting system currently under develop- vals of 1 minute and 30 seconds, with each batch of ment at ETH Zurich as part of the Innosuisse project: material corresponding to 100 mm of height in the “Robotic fabrication of standard and non-standard formwork. concrete structures”. The casting setup consists of a For the casting experiments, it was assumed that concrete pump, mixing chamber, and material out- the pressure on the formworks is purely hydrostatic, let hose and is similar to the system used to fabricate and therefore can be calculated using the equation the SDC mullions for the Dfab House (Lloret-Fritschi for hydrostatic pressure: P = ρ · g · h. The max- et al., 2019). Demonstrator 2 was cast with Mix B, us- imum filling rate (vertical speed at which a formwork ing a manual filling process described by Burger et al. can be filled with concrete without breaking), used to (2020b). define the processing parameters in Section ‘Demon- strators’, can be established by dividing the break-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 201 Figure 1 Selection of the printed formworks tested in ‘Constant cross-section’. The left image shows the printed formwork, the right image shows the formwork at the EXPERIMENTS hibit breakage if filled with concrete to a large height. moment of Constant cross-section breakage (if The first series of experiments aimed at establish- Patterns applicable). C1) ing the breakage height for several formworks with The second series of tests aimed to investigate if square a constant cross-section: a square, a hexagon, a three-dimensional patterns added to the formwork cross-section, C2) decagon, and a circle (Figure 1). The circular form- can improve the breakage behaviour of the form- hexagonal work was expected to have the best performance works. It has been previously shown that patterns cross-section, C3) (highest breakage height). By changing the circum- have a positive impact on the 3D printing process circular radius, cross-sectional area was kept constant at 661 (Burger et al., 2020a); however, the impact of these cross-section. cm2 to ensure that the amount of concrete cast per patterns on hydrostatic pressure resistance has not batch was the same for each formwork. Each was yet been investigated. If the patterns could prove printed to a height of either 500 mm (for the square beneficial, they could potentially serve as a ‘func- and the hexagon) or 1000 mm (for the decagon and tional ornament’: allowing for a faster vertical casting the circle) and was cast with concrete until break- rate and providing aesthetically pleasing features. age. Due to the long straight lines of the square and A range of patterns was applied to the square hexagonal formwork, a deformation of several mil- formwork tested in the previous section to investi- limetres can be observed in the prints (as described in gate the effect of patterns on breakage behaviour. Section ‘Printing’). However, this was assumed not to It was chosen to apply the patterns to the square influence the casting results strongly. The results can formwork since this cross-section was more likely to be seen in Table 1. The square formwork (C1) broke exhibit breakage at lower heights (see Table 1), and at a height of 405 mm, the hexagonal formwork at a therefore more likely to yield results. Applying the height of 500 mm, the decagonal formwork (C3) at patterns to the circular formwork might have yielded a height of 730 mm, whereas the circular formwork better comparative results, but this would have re- (C4) was able to fill to a height of 1000 mm, without sulted in very long printing times (due to the large showing breakage (Figure 1). formwork height required) and high concrete use The results show that cross-sections with sharper (due to the large volume of the tall formworks), and corners, and therefore lower interior angles, are less was therefore deemed unfeasible for this study. performant. The circular formwork stands out as the Several patterns ranging from a regular division most performant, showing no breakage. This corre- into triangles to an organic pattern are tested. As this sponds to existing knowledge on the design of pres- is the first study into the effects of patterns on break- sure vessels such as fuel tanks, as such pressure ves- age behaviour of 3D printed formworks, it was cho- sels are generally designed without sharp corners, sen to conduct tests on a wide variety of patterns, which act as stress concentrators. However, it can be ranging from high- to low resolution and from a tri- assumed that the circular formwork would also ex- angular to rectangular and hexagonal division. The

202 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Table 1 Experimental data obtained in the tests described in “Constant cross-section”.

Figure 2 Patterned formworks tested in ‘Patterns’,the top figures show the printed formwork, the bottom figures show the formwork at the moment of breakage.

square formwork with an edge length of 257 mm and cal casting rate constant between formworks, the an area of 661 cm2 was used as an input for the pat- amount of concrete cast per batch was slightly mod- tern generation. The different patterns were mod- ified for each formwork. However, since the differ- elled using custom Grasshopper scripts based on a ence in volume was less than 8%, it was assumed that rough mesh of the square formwork. This mesh was this difference would not largely impact the breakage then subdivided depending on the required resolu- behaviour. The results of casting the formworks with tion of the pattern, and various operations (trans- concrete are summarized in Figure 3. lation, subdivision, relaxation) were applied to the Comparing the results to the reference geom- mesh faces to form the final patterns. The designs etry (C1, Figure 3), P1 and P2 show a lower break- were then printed to a height of 750 mm (Figure 2, age height, whereas P3 - P6 have a higher breakage top). height when compared to the reference. The low Unlike the experiments conducted in the previ- breakage height of P1 could perhaps be explained ous section, the resulting formworks all had slight by the fact that the structural system relies on rect- variations in volume due to the different applied pat- angles instead of - typically stronger - triangles. Fur- terns (see Figure 3). Therefore, to keep the verti- thermore, since the cell size is small (40 x 40 mm),

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 203 Figure 3 Results of experiments conducted in “Patterns”. Top: Comparison of breakage height of the patterns tested, the grey area indicates the breakage height of the reference formwork (C1). Middle: Location of breakage in the formworks. Bottom: Experimental data obtained on breakage height and vertical printing rate.

many sharp corner situations are created that have within a certain amount of time) of the patterned been shown to be weak in the previous section. Test formworks and the reference. As the formworks have P2 has a triangular pattern division but only has a a constant wall thickness of one layer, the VPR is only limited structural depth (10 mm), making its perfor- influenced by the length of the cross-section without mance similar to that of the reference square. any travel moves present. P1 and P2 have a similar Tests P3 and P4 show that it is possible to achieve VPR but also a lower breakage height. P3, P5, and P6 a 20% higher breakage height than the reference, by have higher breakage height and a (much) lower VPR creating patterns with large depth (25 mm), both in due to the length increase of the printing path. The a high- (P3) as well as low-resolution (P4). Test P5 only pattern with a similar VPR to the reference (but shows a pattern with smooth corners with a fillet ra- higher breakage height) is P4 due to its relatively sim- dius of 10 mm, while still creating structural depth. ple geometry. Test P6 features a diagrid structure outside the form- Generally, the results of the experiments indi- work, a structural concept known for creating stabil- cate that some patterns have a positive effect on ity in tall buildings. The application of the diagrid also the breakage height, whereas some of the patterns means there are no sharp interior angles, except for have a negative effect. The results point towards the the corners of the formwork. Both P5 and P6 give a fact that patterns with rounded corners have better breakage height increase of around 50%, most likely breakage behaviour. Furthermore, all of the applied due to the lack of sharp corners and realization of patterns have a positive effect on the 3D printing pro- structural depth. cess of the formworks: much less warping can be ob- Lastly, Figure 3 also shows the vertical printing served in the patterned formworks (Figure 2), com- rate (VPR: height of formwork that can be printed pared to the square formwork (Figure 1), due to the

204 | eCAADe 39 - Digital fabrication and robotics - Volume 2 elimination of long straight print lines. strator 2, pattern P5: organic y-shape, (see ‘Patterns’) was applied to the geometry. This pattern was cho- Final demonstrators sen since it showed a stark increase in breakage The last experiments described in this paper aimed height, as well as a reasonable VPR. to apply the preliminary findings on breakage height FABRICATION. The column formworks were printed made throughout the previous sections, to the fabri- in three parts (with a height of 0.9 m, 1.2 m, and cation process of two full-scale demonstrators. 0.9 m) since the 3 m height of the column exceeded DESIGN. The design of the columns made use of the building height of the available fabrication setup. the cross-sections described in Section ‘Constant The printing time for Demonstrator 1 and 2 was, re- cross-section’. The formwork cross-section vertically spectively, 13 hours and 17 hours. Both demonstra- morphs between a circle, a decagon, an octagon, a tors were cast using the fast-setting concrete. As de- hexagon, and finally, a square (Figure 4, left). The scribed in Section ‘Casting’, Demonstrator 1 was cast strongest cross-section (the circle) was used at the using Concrete Mix A (setting time 10 minutes), using bottom, where more pressure acts on the form- the digital casting system. In contrast, Demonstra- work, and the weakest cross-section (the square) was tor 2 was cast with Concrete Mix B (setting time 50 placed at the top. These shapes were then lofted to- minutes), using a manual casting process, since the gether to form the base geometry for both demon- digital casting system was not available at the time strators. While both columns shared this base geom- of casting Demonstrator 2. etry, two different patterns were applied to validate The filling rate was calculated using the data the effect of the patterns. on breakage height collected in the experiments on formworks with a constant cross-section (see ‘Con- Figure 4 stant cross-section’). To do the calculation, the col- Left: Design of the umn was divided into five sections that would each column based on have a different filling rate (Figure 4). The filling rates tested were calculated using Eq. (1), with the addition of cross-sections. a variable scale factor. It is necessary to adjust for Middle: Division of scale since experiments conducted on the breakage the column design height of formworks with the same cross-sectional in different sections shape but in different scales have shown that cross- for establishing sections with a larger area have a lower breakage fabrication height (experiments not discussed in this paper). parameters. Right: Since there was no data on the breakage height of Filling rate for the the octagonal formwork, it was chosen to use the av- sections of the two erage of the hexagonal and decagonal formwork. For demonstrators, the circular formwork, a breakage height of 1000 mm A simple pattern consisting of sine waves that form calculated using the was assumed since it has been shown that this would shallow (5 mm) undulations was applied to the top data on breakage not result in breakage. Furthermore, since Demon- part of Demonstrator 1. The intent of this pattern was height. strator 2 featured the organic y-shape pattern, which not to provide additional resistance against hydro- has shown to increase the breakage height by 50% static pressure but instead to make the square cross- (see Section ‘Patterns’), the resulting filling rate from section feasible to print without warping since long, the calculation was increased by 1.5 for the sections straight lines without any undulations are prone to of the column where the pattern was applied. The warping. Therefore, the pattern was applied only to maximum filling rates for both demonstrators can be the uppermost section of the column. For Demon-

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 205 Table 2 Fabrication parameters for the two demonstrators. * The breakage height of the octagon was not measured but is instead an average seen in Table 2. sharp corners, likely due to stress peaks that con- of the hexagon and Demonstrator 1 was filled using the digital cast- centrate in the corners. For this reason, sharp cor- the decagon. † The ing system. It was chosen to simplify the filling rates ners should be avoided in the design of 3D printed circle did not show to 3 L/min for the bottom part, 2.25 L/min for the formworks. Instead, filleted corners or smooth, curvi- breakage when cast middle part and 1.5 L/min for the top part (Figure linear shapes should be used. If a certain design to a height of 1000 4, right). The total formwork was filled in around 65 does require sharp corners, the filling rate must be mm, so this value minutes. Demonstrator 2 was filled using a manual reduced, which increases fabrication time. Therefore, was chosen as the casting process. Due to the longer setting time of the this study shows that 3D printed formworks are espe- breakage height. material used (50 minutes instead of 15 minutes), fill- cially suitable for non-standard cross-sections, rather ing rates were lower, and therefore the casting pro- than the orthogonal shapes that can already be eco- cess longer. The full formwork was filled in around nomically produced with conventional formworks. 330 minutes. After casting, each demonstrator was In Section ‘Patterns’, we have seen that applying left to cure for three days before the formwork was geometrical patterns to the formworks can have both removed. The casting process can be seen in Figure a negative and a positive effect on the breakage be- 5 and the two finished demonstrators can be seen in haviour. Generally, patterns that introduce sharp cor- Figure 6. ners into the design are likely to weaken the form- work and should be avoided. On the other hand, pat- DISCUSSION terns that use smooth curves could benefit the form- This paper attempted to: 1) give a first insight into work, but this will need to be further investigated in the breakage behaviour of formworks with a con- future work. A next step for the research into differ- stant cross-section, 2) investigate if patterns can have ent patterns could be a thorough exploration of a sin- a beneficial effect on the breakage behaviour, and 3) gle pattern, testing how different parameters (reso- validate the preliminary results through the fabrica- lution, depth) of the pattern influence its breakage tion of full-scale demonstrators. What is important behaviour. The application of patterns does result to note is that most of the experiments in Chapter 3 in higher control of deformation of the printed form- were limited to a single specimen. The results point works. For example, applying a slight undulating pat- towards the fact that the geometry of the formworks tern with a depth of 5 mm onto a square formwork has indeed an impact on the breakage height. How- has shown to fully eliminate visible warping of the ever, to provide conclusive results, a series of exper- print. This confirms that the addition of a pattern iments with a larger number of specimens will have can be used to avoid the warping of prints with long, to be conducted. straight lines. From Section ‘Constant cross-section’ we can The application of patterns results in a longer see that prototypes with sharp corners experienced toolpath compared with the basic shapes, which breakage at lower height, than prototypes without means printing time is longer and, therefore, the VPR

206 | eCAADe 39 - Digital fabrication and robotics - Volume 2 lower. An additional aspect of patterns that should However, several assumptions were made to be mentioned is the aesthetic value that these can simplify the design- and fabrication process of the add to the finished elements. Patterns such as vis- two Demonstrators: 1) it was assumed that the break- ible on the final demonstrators (Figure 6) can add age height of a formwork that morphs between appealing ornaments to the concrete element while cross-sections A and cross-section B linearly interpo- potentially providing additional strength and bene- lates between the breakage height of cross-section A fits during fabrication. Again, this is contrary to con- and cross-section B, and 2) it was assumed that the ventional formwork fabrication, where a similar orna- octagon had a breakage height that is the average of ment would only add complexity, cost, and waste to the hexagon and the decagon. Calculating according the fabrication process. to these assumptions proved feasible for the fabrica- tion of the two columns; however, this will have to be Figure 5 verified in further research. Left: Casting of Demonstrator 1 CONCLUSION AND OUTLOOK using the digital This paper provides a first insight into the breakage casting system. behaviour of several 3D printed formwork shapes. Right: Casting of This is an important step in the development of the Demonstrator 2 Eggshell fabrication process since it is necessary to using manual adjust the vertical filling rate according to the break- casting. age height of a formwork. Although the range of cross-sections tested is limited, the vertical filling rate can now be calculated for shapes ranging from a polygon to a circle. Furthermore, the presented re- sults might also be used to predict breakage for other fabrication processes that involve 3D printed form- Furthermore, it is striking that most of the tested work. The preliminary results presented gives an in- formworks could only support around half a meter dication that curvilinear shapes are more performant of concrete, even though calculations of hoop ten- than orthogonal shapes, which could inspire designs sion imply that cylindrical formworks made of PET-G, for 3D printed, thin shell formwork. with a tensile strength of 50 MPa, could support sev- As an outlook to the research, testing of addi- eral meters of concrete pressure. This points towards tional cross-sections is needed, as well as increased degradation of the PET-G formwork, possibly caused sample size. Additional research into the potential by the high pH value of concrete, as also described degradation of the formwork by the high pH value by Ruffray (2020, pp. 115-125). of concrete is needed to give more insight into the The successful fabrication of the two columns stresses responsible for formwork breakage. Further- shows that it was possible to apply the preliminary more, different shell thicknesses, as well as differ- findings to a full-scale building element. The filling ent materials, could still be explored. Finite Element rate calculated according to the results from the con- Analysis (FEA) could also be conducted on the form- stant cross-sections proved to be suitable for avoid- works to investigate if analytical methods will yield ing breakages in the formwork while still maintaining similar results to the empirical results presented in a relatively fast casting rate. It was possible to use two this paper. As a long-term outlook, it might be pos- concrete mixes with a different setting time, showing sible to develop an analytical model that can accu- that Eq. (1) can be applied for different materials. rately predict breakage of the formworks.

Digital fabrication and robotics - Volume 2 - eCAADe 39 | 207 Thin, 3D Printed Formwork’, Second RILEM Interna- Figure 6 tional Conference on Concrete and Digital Fabrication, The two final Cham, pp. 1104-1115 demonstrators Jipa, A, Bernhard, M and Dillenburger, B 2017 ’Submil- limeter Formwork. 3D-Printed Plastic Formwork for Concrete Elements’, TxA Emerging Design + Technol- ogy Conference, Austin, Texas, USA Leschok, M and Dillenburger, B 2019 ’Dissolvable 3DP Formwork’, Ubiquity and Autonomy – Proceedings of the ACADIA Conference Lloret, E, Shahab, AR, Linus, M, Flatt, RJ, Gramazio, F, Kohler, M and Langenberg, S 2015, ’Complex con- crete structures: Merging existing techniques with digital fabrication’, Computer-Aided Design, 60, pp. 40-49 Lloret-Fritschi, E, Scotto, F, Gramazio, F, Kohler, M, Graser, K, Wangler, T, Reiter, L, Flatt, RJ and Mata-Falcón, J 2019 ’Challenges of Real-Scale Production with Smart Dynamic Casting’, First RILEM International Conference on Concrete and Digital Fabrication – Dig- ital Concrete 2018, pp. 299-310 Lloret-Fritschi, E, Wangler, T, Gebhard, L, Mata-Falcón, Although still in an early stage of research, this J, Mantellato, S, Scotto, F, Burger, J, Szabo, A, Ruf- study has provided a first step in the direction fray, N, Reiter, L, Boscaro, F, Kaufmann, W, Kohler, M, of fabrication- and material-informed formwork de- Gramazio, F and Flatt, R 2020, ’From Smart Dynamic Casting to a growing family of Digital Casting Sys- sign. Eventually, this can allow us to produce non- tems’, Cement and Concrete Research, 134, pp. 1-16 standard, optimised concrete structures with a min- Naboni, R and Breseghello, L 2018 ’Fused Deposition imal amount of formwork and, thus, a minimal Modelling Formworks for Complex Concrete Con- amount of waste. Furthermore, it has already been structions’, Blucher Design Proceedings, São Carlos, shown that it is possible to recycle and reprint a form- BR, pp. 700-707 work several times. Therefore, this process will en- Peters, B 2014 ’Additive Formwork - 3D Printed Flexi- ble Formwork’, Proceedings of the 34th Annual Con- able the reduction of concrete use and result in the ference of the Association for Computer Aided Design reduction of construction waste generated. Finally, in Architecture (ACADIA), Los Angeles Eggshell could expand geometrical freedom within Reiter, L, Wangler, T, Anton, A and Flatt, RJ 2020, ’Setting concrete construction, leading to an increased aes- on demand for digital concrete – Principles, mea- thetical value for the built environment. surements, chemistry, validation’, Cement and Con- crete Research, 132, pp. 1-14 Ruffray, N 2020, From Fresh Concrete Microstructure to REFERENCES Digitally Fabricated HPFRC: A Challenging Journey Burger, J, Lloret-Fritschi, E, Scotto, F, Demoulin, T, Geb- up From the Nanoscale in Search of Precious Digital hard, L, Mata-Falcón, J, Gramazio, F, Kohler, M Macro-Applications, Ph.D. Thesis, ETH Zurich and Flatt, RJ 2020, ’Eggshell: Ultra-Thin Three- Wijnen, B, Sanders, P and Pearce, JM 2018, ’Improved Dimensional Printed Formwork for Concrete Struc- Model and Experimental Validation of Deformation tures’, 3D Printing and Additive Manufacturing, 7(2), in Fused Filament Fabrication of Poly Lactic Acid’, pp. 48-59 Progress in Additive Manufacturing, 3(1), pp. 193-203 Burger, J, Lloret-Fritschi, E, Taha, N, Scotto, F, Demoulin, T, Mata-Falcón, J, Gramazio, F, Kohler, M and Flatt, RJ 2020 ’Design and Fabrication of a Non-standard, Structural Concrete Column Using Eggshell: Ultra-

208 | eCAADe 39 - Digital fabrication and robotics - Volume 2 Digital heritage 210 | eCAADe 39 - Digital heritage - Volume 2 3Dino: Configuration for a Micro-Photogrammetric Survey

Applying Dino-Lite microscope for the digitalization of a cuneiform tablet

Sara Antinozzi1, Diego Ronchi2, Fausta Fiorillo3, Salvatore Barba4 1,2,4Università di Salerno 3Politecnico di Milano 1,2,4{santinozzi|dronchi|sbarba}@unisa.it [email protected]

Close-range photogrammetry, due to the possibilities offered by the technological evolution of acquisition tools and, above all, the relative original challenges posed to surveyors and the theory of measurements, deserve constant critical attention. The new opportunities to detect and represent reality are mostly focused on historical architecture, referring to consequent orders of magnitude and restitution scales. On the other hand, the formalization of relevant practices for very small objects is not frequently addressed. In recent tests carried out using two Dino-Lite handheld digital microscope models, polarized light digital microscopes generally used in medical and industrial fields, we proved the potential of using these imaging systems also for Cultural Heritage documentation, highlighting, however, some issues related to the depth of field and the consequent acquisition geometry. Therefore, this study aims to solve these problems, increasing the performance of microscopic photogrammetry by optimizing the acquisition procedures with the design of custom accessories for micro-photogrammetry (e.g. a calibrated plate). These developments will be carried out as part of a technology transfer agreement with the Dino-Lite company pointed to codify a protocol for high accuracy photogrammetric documentation of small artefacts.

Keywords: Digital Heritage, Small artefacts, Detailed 3D shape, Handheld microscope

INTRODUCTION the network of relationships of complementarity and The configuration of new survey approaches for Cul- interaction between the different disciplines related tural Heritage’s digitization in terms of conservation, to imaging and three-dimensional processing, in- management, and promotion is gradually asserting cluding manufacturing industry, medical sciences, itself within the so-called “fourth industrial revolu- entertainment and, of course, Cultural Heritage (Pier- tion”. The possibilities offered by Industry 4.0 thicken accini et al. 2001). In particular archaeological disci-

Digital heritage - Volume 2 - eCAADe 39 | 211 plines are greatly interested in using emerging and ing photographs taken by a digital microscope as a available digital techniques and technologies pro- dataset for photogrammetric processing revealed a vided by Geomatics aiming at rigorous objects de- not negligible potential for small objects modeling, scription (Bitelli et al. 2007) and preservation of hu- reaching about 0,1 mm accuracy (Esmaeili and Ebadi man heritage, at least in digital format, by develop- 2017). A digital microscope is similar to a traditional ing the concept of “preventive and planned conser- optical microscope except that it is equipped with a vation”, for the restoration field, here declined in or- CCD camera so that it can output a digital color im- der to establish a “preventive digital memory” (Car- age to a monitor, and among the main types of a lucci 2016). Furthermore, given the recent inaccessi- microscope are the most flexible and the least ex- bility conditions, rethinking representation and visu- pensive (Atsushi et al. 2011). Digital portable micro- alization phases of the assets constitute a possibility scopes, born for inspection, documentation, and dig- for remote sharing, as well as the establishment of a ital metrology analysis - already popular in the manu- vast system of information storage, editable and im- facturing and quality control industry, as well as used plementable at any time (Apollonio et al. 2021), as in the medical field - are easy to handle and versa- well as the representation of real-world heritage for tile capturing systems. On the other hand, the im- interaction and virtual experience (Djuric et al. 2019). ages they provide do not have both: very high res- Usual procedures for documenting heritage ob- olution (usually not exceeding 5 MP) and wide dy- jects are mainly addressed to orders of magnitude namic range. In addition to these problems, the nar- and architectural restitution scales, but still little ori- row Field Of View (FOV) and shallow Depth Of Field ented towards the coding and formalization of rele- (DOF), which could compromise close-range pho- vant practices for very small objects. In fact, inter- togrammetry, should be mentioned. These limita- facing with small finds represents a not easy chal- tions could, in fact, affect point matching, the num- lenge determining a change in the representation ber of conjugate points computed, and the resolu- scale and a rethinking of the acquisition operating tion of the 3D model (Kontogianni et al. 2017), also systems, not always in comfortable contexts (Plisson determining an increase in acquisition times because and Zotkina 2015). In particular, applications for Cul- it makes necessary to move the microscope often to tural Heritage have stricter requirements on morpho- cover the object. In this case, the acquisition automa- logical details, whose correct reproduction necessar- tion would make the entire process more advanta- ily refers to other factors such as object’s size and geous (Mitchell and Kniest 1999). shape and surface’s reflective properties - whether it is opaque, glossy or translucent (De Paolis et al. Case study 2020). Several solutions for micrometric applications One of the current challenges is to configure a high- (Hansen et al. 2006), including range-based ones performance micro-photogrammetric system using (Tolksdorf et al. 2017), are increasingly succeeding in the now widely available on the market digital the field of Cultural Heritage study. However, not in- portable microscopes. The main aim is to test their frequently, certain optical or mechanical limitations, validity on micro-scale survey, overcoming the obvi- the need for specialized operators and the consider- ous operational difficulties, especially in the acquisi- able costs direct the interest towards precision pho- tion phase. For this purpose, in this current work, the togrammetry, which offers the possibility of obtain- authors will focus on the systematization of individu- ing the three-dimensional (3D) coordinates of an ob- ally available hardware tools to explore microscopic ject from two-dimensional (2D) digital images in a photogrammetry and define a standard procedure rapid, accurate, reliable, flexible and economical way for the acquisition of very small objects. Therefore, (Yilmaz et al. 2008). In detail, the possibility of us- it is proposed the procedural solutions identified to

212 | eCAADe 39 - Digital heritage - Volume 2 carry out the digital survey of a 3D printing replica digital 3D model. Compared to the canonical close- [1] in geopolymer of a tablet with cuneiform writings. range photogrammetry acquisition, a redesign of the The text content concerns the administrative infor- acquisition hardware system at the dimensional scale mation for fish deliveries (a modern correspondent is required to solve some critical issues strongly am- of a packing slip). Its dimensions are approximately plified at a sub-millimeter level. In detail, the inno- 20 x 22 x 8 mm, and the characteristic wedge-shaped vative micro-photogrammetric scanning system pro- impressions have a depth of 1-3 mm (see Figure 1). posed in this study, called “3Dino System”,is based on the combined use of a three-dimensional calibrator Figure 1 and Dino-Lite handheld digital microscopes. Recto and verso One of the main problems is the need to include views of the 3D in the scene metric references or calibrated objects printed copy of the and, even by solving this difficulty, to be able to pro- cuneiform tablet duce an adequate calibration pattern according to the magnitude of the object to be measured (Lavec- chia et al. 2017). To address these issues, a calibrated plate - designed by the authors and built for the occa- sion - was used to optimize, align and for scaling pro- cedures (see Figure 2). This calibrator, called “3Dino Plate”, consists of a PLA plate obtained with 3D fila- This paper focuses on studying the most suitable ment printing, characterized by an orthogonal pat- 3D measurement instrumentation and method to tern of 99 truncated cone holes, with a countersink achieve a digital replica of this tiny, complex, and de- angle of 60° and a smaller base diameter of 0,35 mm. tailed object. For the survey tests, it was used the The calibrator accuracy, based on the 3D printing set- 3D printing replica, as challenging as the original, tings [2] and the conformity of the actual position of with the intention of preserving the original spirit of the holes with respect to the project file, has been es- multidisciplinary research that aims to use a physi- timated at 0,1 mm, which can be assumed as an in- cal (or digital) replica of the tablets for scientific pur- strumental error value (i.e., markers accuracy). poses. To achieve an adequate descriptive quality Figure 2 and evaluate results soundness, the survey was con- The ducted with two different Dino-Lite portable digital three-dimensional microscopes (www.dino-lite.eu). The quality, speed calibrator “3Dino and cost-effectiveness of this first method were then Plate” and detail of compared with the data obtained from a Nikon full- the section of one frame camera combined with macro optics. All pho- of the truncated togrammetric results were compared with an active cone holes sensor, the SCAN in a BOX structured light 3D scan- ner, used as a reference.

MATERIALS AND METHODS 3Dino Plate The coordinates of each hole, in a local reference sys- tem, are therefore known to the operator and can Digital close-range photogrammetry exploits several be imported into the photogrammetric project and photographic captures taken from different view- linked to acquisitions. This allows the use of the per- points for accurately measuring objects and build a forated pattern as a constraint points (GCP) grid - ho-

Digital heritage - Volume 2 - eCAADe 39 | 213 mogeneously distributed throughout all the area - object is acquired from various focus planes by mov- to optimize cameras alignment. To guarantee suf- ing the microscope on a micrometric vertical rail to ficiently robust results during the alignment phase, focus on different planes without changing the dis- an adhesive geometrically and chromatically non- tance between the object and the optical centre. repetitive pattern, equipped with coded targets, was printed and applied on the calibrator. Dino-Lite digital microscopes The difficulty of manually conducting micro- Dino-Lite digital microscopes provide a powerful, scope acquisitions represents a further obstacle. To portable, and functional solution for detail inspec- facilitate and speed up the acquisition phase, a por- tion. The models compared for this first exper- tion of the calibrated plate was set on a plane allow- imentation, Universal (AM4113ZT) and High Speed, ing the object movement. Using the digital micro- (AM73915MZTL) belong to two different categories, scope housed on a vertical bracket at a fixed angle, a both in terms of technical characteristics and of spe- series of images with sufficient overlap can be taken cial features, consequently falling into very different just by sliding the calibrated plate manually once the price ranges (see Table 1). magnification rate has been chosen (see Figure 3). The main problem encountered in microscopic Table 1 optics is the short Depth Of Field (DOF), and it is em- Dino-Lite phasized when increasing magnification. This allows microscopes having in focus only a small portion of the artefact technical so that only a small portion of the image appears specifications for sharp enough to be used for 3D reconstruction (Clini the models used et al. 2016). In fact, the sharpness, joint with density and the resolution of the photoset, will determine the quality of the output point cloud data (Westoby et al. 2012).

Figure 3 “3Dino Plate” here combined with the Dino-Lite microscope in the “3Dino System”

To overcome the problem of the short DOF, one pos- sibility would obviously be to close the aperture of The element that clearly distinguishes the two micro- the optics as much as possible within the limits of scopes is the Working Distance (WD), i.e. the linear the diffraction phenomena (Sapirstein 2018). Since distance between the tip of the microscope nozzle this possibility is not available on portable digital mi- and the subject (see Figure 4). This factor directly croscopes, where the diaphragm is fixed, it is neces- affects the Magnification Factor (MF), Field of View sary to use alternative solutions. For this purpose, the (FOV) and Depth Of Field (DOF).

214 | eCAADe 39 - Digital heritage - Volume 2 Figure 4 tant parameter from a practical point of view and in- Working Distance evitably binds the use of different magnifications in reached by the High the operational phase. Speed microscope The lighting conditions, based on the polarized (top, 75,5 mm) and light of the microscopes, have been improved with by the Universal the adoption of two LED illumination rings, one microscope which is around the object and the other above it. (bottom, 21,7 mm), The light does not directly affect the object because housed on the of a cylindrical panel consisting of diffusing mate- stand and rial, placed between the object and the light source: operating at the thus, diffused light conditions neutralize the shadow same magnification cones - without variations in the intensity of shadows, (30x). Specifically, lights and colours. the acquisition of a Both Universal and High Speed models were com- relative portion of bined with the 3Dino Plate System, fixing the micro- the calibrator and a scope in the special Dino-Lite RK-10-EX stand and detail of the pattern connecting it to a portable workstation (see Figure (ZOOM). 5). Hence, the acquisitions were carried out with the same magnifications, 30x, operating with the dedicated DinoCapture 2.0 procedure (www.dino- lite.eu/index.php/it/software-dino-lite).

Figure 5 “3Dino System” with the special LED ring lighting set-up

Acquisition with Nikon camera The validation of Dino-Lite digital microscopes re- It is trivial to observe that a right compromise must sults was supported by the photogrammetric survey be established between these factors, bearing in with a Nikon D810 SRL and the calibrator 3Dino Plate, mind that the further you are from the subject (high also suitable for digital cameras. WD), the lower the Magnification, with a wider Field The best optical systems for small objects are the Of View and more Depth Of Field; conversely, the macro lens and, as well as allowing the close focus in closer you are to the subject (low WD), the greater order to maximize the data quality, they also repre- the Magnification, with a smaller Field Of View and sent, among the types of optics, those optically less less Depth. Therefore, the WD is an extremely impor- affected by distortions.

Digital heritage - Volume 2 - eCAADe 39 | 215 In our case, an AF-S VR Micro-Nikkor 105 mm Table 2 f/2.8G IF-ED was used, which allows a minimum fo- Recto Agisoft cusing distance of 0,314 m. In this case, it was pos- Metashape survey sible to manage the shallow Depth Of Field and the data comparison related problem of blur starting from the concept that the Depth Of Field increases with the decrease of the aperture with the consequent reduction of the amount of light passing through and the relative in- crease in exposure times (Greenleaf Allen 1950). So, an aperture of f/36 was used, shooting at about 35 cm, thus ensuring a DOF equal to about 2 cm. A camera trigger system was used to remotely activate the shutter on the camera and reduce the vibrations, which are not negligible due to the relatively long ex- Table 3 posure time (see Figure 6). Verso Agisoft Metashape survey Tie points error reduction and optimization data comparison The datasets obtained by Dino-Lite microscopes and by Nikon camera were thus processed in a SfM soft- ware, Agisoft Metashape, according to the general photogrammetric workflow (see Table 2 and Table 3) and the processing was completed with the same graphic workstation equipped with Intel I9 9900k CPU, RTX2080ti GPU, and 64GB of RAM. The sparse point cloud, made only by tie points (TPs), is the starting point for the realization of a com- plete 3D model. However, the removal of low qual- The evaluation of image orientation quality within ity TPs is appropriate because their presence affects SfM methods can be performed using the Gradual Se- the results of the next steps, which consist in the re- lection filter tool by Agisoft Metashape, which - like computation of the orientation parameters, and the most photogrammetric software - allows you to fil- creation of the final dense cloud (Barba et al. 2019). ter the sparse point cloud based on some quality pa- rameters. The parameters considered in this study were: Reconstruction uncertainty, Reprojection error Figure 6 and Projection accuracy. “3Dino Plate” and Nikon setup REFERENCE MODEL EQUIPMENT configuration for The instrument employed - to have a reference the photos model and check the overall geometrical dimen- acquisition. Take a sions of the photogrammetric reconstructions - is the look at the digital SCAN in a BOX (@2015 Open Technologies SRL). The model here: system is equipped with two high-resolution indus- https://skfb.ly/on67U trial USB cameras and a high definition light projec- tor (ASUS S1). All components are fixed on a thick

216 | eCAADe 39 - Digital heritage - Volume 2 aluminium rail, but the cameras can be moved in cal- and the distance of the cameras on the plate. For our ibrated positions (base) to customize the work field case study, a Field Of View of 100 x 80 mm was set (see Figure 7). that implies the working distance scanner-object of The main steps for the 3D measurements work- about 200 mm, the minimum base between the two flow are: i) calibration of the optical setup, ii) range cameras and the minimum point spacing (resolution) maps acquisition, iii) raw alignment, iv) alignment on the surface 0,078 mm. optimization and v) meshing. The digital reconstruction of the object surface Whenever the scanner is mounted or the cam- and details was performed acquiring: 16 nadiral era setting on the support bar changes, the system scans on both sides (8 for the verso and 8 for the needs to be configured and calibrated. The coded recto), 16 tilted (8 for each side) and 8 nadiral to the calibration master has three different patterns based four edges (totally of 40 range maps). The scanning on the working area selected. The optical setup pro- time for each range map is less or equal to 4 seconds. vides for projector and cameras configuration (posi- The raw in progress alignment process was very use- tion, orientation, focus, exposition). The calibration ful to check real-time scan completeness. This ini- allows calculating the optical parameters of the spe- tial registration is then optimized using an Iterative cific setting. Closest Point (ICP) algorithm. The final mesh model (formed by about 400.000 polygons) have fewer de- Figure 7 tails of the correspondent photogrammetric models The “SCAN in a but can be used to ensure a reference check of their BOX” system global measures (see Figure 8).

DATA ANALYSIS The aim of this analysis is to compare the procedures adopted to digitize of the cuneiform tablet with the various technologies. The first consideration concerns the difficulty with the current 3Dino configuration and camera set- ting in acquiring the two sides of the object in a sin- gle set of photographs. This problem is related to the acquisition geometry and the calibrator shape. Two sets of images, one for recto and the other for verso side, do not have enough points in common to be merged into a single model. It would have been nec- essary the acquisition of the very thin edge. In this case, we would have had a depth of field problems that could have compromised the alignment of this possible set of additional images. A further consideration concerns the manage- ment procedures of the images in Agisoft Metashape for data and products obtainable with microscopes and camera. More specifically, among the most obvi- According to the object dimension and the resolu- ous and limiting digital microscopes problems, there tion required, it is to choose the scanning area more is the absence of Exif data. This information associ- suitable, strictly connected to the working distance

Digital heritage - Volume 2 - eCAADe 39 | 217 Figure 8 “SCAN in a BOX” polygonal model (1 mm grid background)

218 | eCAADe 39 - Digital heritage - Volume 2 Figure 9 ated with the pixel matrix, as well known, retains data Global and relative relating to the camera model, sensor size and focal deviations of micro- length, and are essential variables, in the absence of photogrammetric laboratory calibration, used by any software to per- dense clouds form adequate calculations for the internal and ex- compared with the ternal camera calibration. Therefore, the uncertainty “SCAN in a BOX” generated by the absence of Exif data must be con- used as reference sidered as a further detrimental parameter for the model (verso side) alignment processes. To provide a robust comparison [3] for the point clouds, a cloud to cloud registration has been per- formed using the data coming from the SCAN in a BOX as reference. The registration was carried out in two steps, providing: first, a manual registration using homologous points and then a global registra- tion by automatic alignment algorithms (ICP[4] ). The photogrammetric survey accuracy assess- ment was carried out using CloudCompare C2C (cloud to cloud) command. This tool searches, for each point being compared, the closer one in the ref- erence entity, thus defining a shift value of the first with respect to the second. Using the cloud gener- ated by the SCAN in a BOX as a reference, the clouds obtained from photogrammetry were compared. The Figure 9 shows in false colours the devia- tions among the clouds. Through the calculations performed it is noted that the mean and standard de- viations do not exceed the instrumental accuracy of the 3Dino Plate (0,3 mm). The comparison procedure showed a difference on the Z axis of the model compared with the refer- ence one.

Digital heritage - Volume 2 - eCAADe 39 | 219 can be shorten with the design of an automatic cap- Figure 10 turing system. Colour per vertex of verso side acquired with the different CONCLUSIONS AND FUTURE DEVELOP- passive sensors MENTS The experimental tests conducted with Dino-Lite in- strumentation are valid for defining a first approach to the photogrammetric use of portable USB micro- scopes. The quality/price ratio of Dino-Lite microscopes, combined with the obtainable high resolution, un- doubtedly represent the method strengths. Portabil- ity and ease of use are two additional and significant qualities.

Figure 11 On the left, a comparison of point clouds sections generated with different datasets: 1) by “SCAN in a BOX”,2) by Nikon D810, 3) by microscope Dino Universal and 4) by microscope Dino This difference could have been determined by both High Speed. On the the registration procedure and the higher resolu- right, the point tion obtained from the photogrammetric survey per- cloud obtained formed with Nikon and Dino Universal. The noise, from the Nikon due to alignment problems, of the photogrammet- D810, with the ric set acquired with Dino High Speed is evident. Ab- positioning of the solutely comparable, and perhaps of slightly higher slice chosen for the quality than the SCAN in a BOX, are the results com- comparison ing from Dino Universal (see Figure 10 and Figure 11). A final consideration concerns the parameters: cost, time and quality of output. In this regard, with a cost of a few hundred euros and a quality compara- ble to that obtained by Nikon and SCAN in a BOX, the Universal microscope is one of the most interesting instruments. The flaw concerns the acquisition time, still long (about 3-4 hours for 500-600 captures) that

220 | eCAADe 39 - Digital heritage - Volume 2 On the other hand, it is necessary to specify that, to ACKNOWLEDGMENTS pursue the survey primary objective, that is the ver- We would like to thank, sincerely, IDCP Digital Inno- ifiability of metric data, some key aspects are still to vation - in the persons of Jan Boers, Danielle van Dui- be more rigorously defined. In addition to software jvendijk and Ivo Manders - and, of course, Dino-Lite house changes to enable the Exif data to be saved, Digital Microscope for the support, availability and there are several changes required to the calibrator equipment offered for the research. Special thanks in Figure 2. Firstly, reducing the makers dimensions to Mirko Surdi (Ghent University), Catherine Mitter- and spacing will allow more points of known coordi- mayer (Department of Antiquity Sciences, University nates to be identified. of Geneva), Francesco Fassi (3D Survey Group, Po- Moreover, the geometry of the plate could be litecnico di Milano) and Marinella Levi (+LAB, Politec- modified to fit a motorized turntable to facilitate pos- nico di Milano) for sharing dataset, materials and re- sible tilted-axis acquisitions. However, the 3Dino searches. Plate is suitable for objects up to 2-3 mm thickness, NOTES i.e. mainly two-dimensional objects such as coins. [1] The cuneiform tablet used as a test is the 3D print- In such cases, it would not be necessary to change ing replica in geopolymer of the 724 tablet realized the geometry of the plate. It should be borne in by the +LAB (www.piulab.it - Politecnico di Milano). mind, however, that the use of the plate always re- The original 724 tablet was 3D measured by the 3D quires two sets of acquisitions. The problem of shal- Survey Group (Politecnico di Milano). The interdisci- low depth of field remains and can really be solved plinary research aims to obtain a digital and physical by reshaping the acquisition, e.g. with Focus Stack- clone of the cuneiform tables with high details and ing techniques, using a micrometric slide to move high accuracy for education and scientific purposes the optical system. For these reasons, and in order (e.g. shared analysis and studies among a team of ar- to streamline the acquisition procedures and ensure chaeologists to decipher a text content). a more rigorous workflow, the authors are designing [2] The specific print settings for making the plate a system that involves reformulating the plate geom- are listed below: print used: Creality CR-10 v 1.0; di- etry in order to adapt it to a rotary table system that mensions: bed 300x300 mm x height 400 mm; bed makes the object rotate - and not translate - with re- temperature: 50 °C; nozzle temperature: 200 °C; noz- spect to the sensor. zle diameter: 0,3mm; material used: PLA 1,75 mm ± Although still with limited architectural applica- 0,05 mm; layer height: 0,12 mm; infill: 20%; infill pat- tions, this experiment showed that, verifying their tern: cubic; printing speed: 50 mm/s; approximate great accuracy, it is possible to adopt this system time for printing: 13 hours. for photogrammetric survey of tiny artifacts. Conse- [3] To perform a homologous comparison for quently, in further applications, this procedure could both active and passive techniques, the procedure be extended to different acquisition scales, such as was performed in CloudCompare 2.10.2. architectural ornaments and friezes, possibly evalu- [4] The ICP (Iterative closest point) algorithm, be- ating the integration of macro optics too. cause of its iterative nature, can only guarantee the Despite the problems within the framework of convergence to a local minimum. The error in the fi- modern micro-technologies, more interesting ideas nal registration is expressed through RMSE, or RMS for the three-dimensional documentation of very of the Euclidean distance between the match point small artefacts that would otherwise be difficult to pairs of the alignment process. For each ICP iteration represent prevail from this work in progress. the maximum number of matches was set to 50.000 and the number of iterations for the process equal to 60.

Digital heritage - Volume 2 - eCAADe 39 | 221 REFERENCES 421-425 Greenleaf Allen, R 1950, Photographic optics, Macmillan, Lavecchia, F, Guerra, M G and Galatucci, L M 2017, ’Per- New York formance verification of a photogrammetric scan- Apollonio, F I, Fantini, F, Garagnani, S and Gaiani, M A ning system’, The International Journal of Advanced 2021, ’Photogrammetry-Based Workflow for the Ac- Manufacturing Technology, 96, pp. 4267-4279 curate 3D Construction and Visualization of Muse- Mitchell, H L and Kniest, H T 1999, ’Digital Photogramme- ums Assets’, Remote Sensing, 13(3), pp. 1-40 try and Microscope Photographs’, Photogrammetric Atsushi, K, Sueyasu, H, Funayama, Y and Maekawa, Record, 16, pp. 695-704 T 2011, ’System for reconstruction of three- De Paolis, L T, De Luca, V, Gatto, C, D, G and Paladini, G I dimensional micro objects from multiple photo- 2020, ’Photogrammetric 3D Reconstruction of Small graphic images’, Computer-Aided Design, 43(8), pp. Objects for a Real-Time Fruition’,in De Paolis, L T and 1045-1055 Bourdot, P (eds) 2020, Augmented Reality, Virtual Re- Barba, S, Barbarella, M, Di Benedetto, A, Fiani, M, Gujski, ality and Computer Graphics, Springer International, L and Limongiello, M 2019, ’Accuracy Assessment pp. 375-394 of 3D Photogrammetric Models from an Unmanned Pieraccini, M, Guidi, G and Atzeni, C 2001, ’3D digitizing Aerial Vehicle’, Drones, 3, pp. 79-97 of cultural heritage’, Journal of Cultural Heritage, 2, Bitelli, G, Girelli, V A, Remondino, F and Vittuari, L 2007 pp. 63-70 ’The potential of 3D techniques for Cultural Heritage Plisson, H and Zotkina, L 2015, ’From 2D to 3D at macro- object documentation’, Proceedings of SPIE - The In- and microscopic scale in rock art studies’, Digital Ap- ternational Society for Optical Engineering, pp. 244- plications in Archaeology and Cultural Heritage, 2, pp. 253 102-119 Carlucci, R 2016, ’Dalla scomposizione della realtà alla Sapirstein, P 2018, ’A high-precision photogrammetric memoria digitale preventiva’, Archeomatica - Tec- recording system for small artifacts’, Journal of Cul- nologie per i Beni Culturali, 1, p. 3 tural Heritage, 31, pp. 33-45 Clini, P, Frapiccini, N, Mengoni, M, Nespeca, R and Rug- Tolksdorf, J F, Elburg, R and Reuter, T 2017, ’Can 3D scan- geri, L 2016 ’SFM Technique and Focus Stacking for ning of countermarks on Roman coins help to recon- Digital Documentation of Archaeological Artifacts’, struct the movement of Varus and his legions’, Jour- The International Archives of the Photogrammetry, Re- nal of Archaeological Science: Reports, 11, pp. 400- mote Sensing and Spatial Information Sciences, pp. 410 229-236 Westoby, M, Brasington, J, Glasser, N, Hambrey, M Djuric, I, Stojakovic, V, Misic, S, Kekeljevic, I, Vasiljevic, I, and Reynolds, J 2012, ’‘Structure-from-Motion’ pho- Obradovic, M and Obradovic, R 2019 ’Church Her- togrammetry: A low-cost, effective tool for geo- itage Multimedia Presentation: case study of the science applications’, Geomorphology, 179, pp. 300- iconostasis as the characteristic art and architec- 314 tural element of the Christian Orthodox churches’, Yilmaz, H M, Yakar, M and Yildiz, F 2008 ’Digital pho- 37 eCAADe and XXIII SIGraDi Joint Conference, “Archi- togrammetry in obtaining of 3D model data of irreg- tecture in the Age of the 4Th Industrial Revolution, pp. ular small objects’, The International Archives of the 551-560 Photogrammetry, Remote Sensing and Spatial Infor- Esmaeli, F and Ebadi, H 2017 ’Handy Microscopic Close- mation Sciences, pp. 125-130 Range Videogrammetry’, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, pp. 65-67 Hansen, H, Carneiro, K, Haitjema, H and De Chiffre, L 2006, ’Dimensional Micro and Nano Metrology’, CIRP Annals, 55(2), pp. 721-743 Kontogianni, G, Chliverou, R, Koutsoudis, A, Pavlidis, G and Georgopoulos, A 2017 ’Enhancing close-up image based 3D digitisation with focus stacking’, The International Archives of the Photogrammetry, Re- mote Sensing and Spatial Information Sciences, pp.

222 | eCAADe 39 - Digital heritage - Volume 2 MR Game for Historical Experience A study on the interplay between tangible and intangible heritage in Chaozhou ancient town

Fukai Chen1, Xiangmin Guo2, Tiantian Lo3 1,2,3Harbin Institute of Technology (Shenzhen) [email protected] [email protected] [email protected]

The tangible and intangible heritage, defining the local identity and spirit of place, is relatively less connected. We have proposed a novel method, to enhance visitors` interests, resulting in better engagement and understandings of the heritage. A MR game was developed to create a multidimensional experience by combining intangible meaningful and creative contents, such as comics, with immersive interaction in HoloLens 2. The contents were then superimposed onto the tangible heritage on-site. A test was conducted on 22 users selected at random. The results revealed that embedding the intangible, much amplifying the details of the tangible, made the experience more dramatic, immersive and engaged, arousing users' further interests in tangible heritages. We believed the proposed method can be an effective way to better understand the tangible and intangible aspects of heritages.

Keywords: Mixed Reality, Digital Heritage, Chaozhou Ancient Town

INTRODUCTION 2010; Z. Noh et al., 2009; H. Rua et al., 2011; Z. He et al., Due to the impact of COVID-19, lots of heritages 2018; A. Leopardi et al., 2019). The connection of the (i.e. museums and ancient towns) have accelerated tangible and intangible characteristics helps to es- digital transformation. Immersive technology opens tablish meaning and disseminate significance (Rush- up new possibilities in changing heritage practice to ton et al., 2018). Yet the relationship between the tan- conserve the relationship between the tangible and gible and intangible is relatively weak in existing dig- intangible qualities embodied within heritages. MR ital heritage, just presenting the physical exhibitions is defined as the blend of virtual elements (e.g., holo- into a three-dimensional virtual format. The paper grams or projections) with the real world, and, com- aimed to strengthen that relationship to create a mul- pared with AR, can achieve a broader goal in enhanc- tidimensional experience by a MR game based on a ing our understanding of the real and virtual environ- new kind of contents and interaction. ment. (M. Mengoni et al., 2018; Bekele & Champion, Many methods have been applied to the study 2019). Over the years different studies have proved of the tangible and intangible heritage particularly the feasibility of different application in cultural her- with regard to the heritage site, contents and interac- itage based on VR, AR and MR (M. Carrozzino et al., tion. For heritage sites, many studies took the tests in

Digital heritage - Volume 2 - eCAADe 39 | 223 museums (Pollalis et al., 2018; Chen, 2020). For con- ogy. The applications and literature are from the tents, some interpreted the intangible into 3D mod- area of digital heritage and games. And then we els (Garagnani et al.2016; Rushton & Schnabel, 2020). will discuss cases from the perspective of contents, For interaction, some made it as a game based on individual-interaction and social-interaction the mobile-phone or PC devices (Varinlioglu et al., 2019; Ferreira & Amélia, 2018; Chen, 2020). However, Contents compared with the artefacts confined in display cab- Intangible contents, forming the spirit of place to- inets, the ones in outdoor heritage site (i.e. build- gether with the tangible, however, was less consid- ings and monuments) could be more tangible thus ered in some research. A study of enhancing engage- enrich the contrast between the tangible and intan- ment by interacting with hologram of artefacts was gible. For the contents, more creative and meaning- presented by Pollalis (2017). There’s a considerable ful ones, such as cartoon characters, could be consid- information overlap between the hologram and the ered. For the interaction, it would be more immersive tangible, making it less meaningful and hardly lead- with the new MR hardware such as HoloLens 2. ing to a better understanding. However, some stud- We strengthened the interplay between the tan- ies have embedded the intangible value, such as a gible and intangible heritage, resulting in a multidi- digital model linked to open WebGIS maps (Garag- mensional experience. To explore a better tangible nani et al.,2016) and a 3D model capturing the effect experience, a public plaza in the center of Chaozhou of time on architecture with the oral history (David Ancient town, China, was chosen as a case study to Silcock et al., 2018). The embedding of the intangible test. Meanwhile users could roam the heritage with in these studies made the heritage more meaningful the guidance of meaningful and creative contents, than before, while the contents weren’t creative and such as cartoon characteristics and comics, and got attractive enough. Some later experiments showed a more immersive interaction within a serious MR the potential of serious games in digital heritage by game based on HMD hardware HoloLens 2. a construction game in wood-architecture exhibition We start by reviewing the current literature in (Chen, 2020). However, visitors can hardly experi- digital heritage about the tangible and intangible, ence the game unless on-site, which limits the fur- then aggregate them into 3 aspect: creative and ther dissemination. The contents, embedded in the meaningful intangible contents, individual interac- intangible, creative and easy to disseminate, are sel- tion between the tangible and intangible heritage dom considered. and social-interaction. Then this study will take Chaozhou Ancient Town in Guangdong, China as an Individual-Interaction example to verify the potential of MR applications. Hand-held device, due to it portability, was widely Four stages were carried out: 1) contents creation chosen as the AR hardware platform in digital her- based on Chaozhou; 2) development of the game itage. For example, with AR, the exhibition “David prototype in Unity; 3) an experiment on the spot of Bowie Is” was transferred into an application on mo- Chaozhou; 4) analysis of the users’ behaviors and at- bile phone and the sculpture exhibition “KAWS: HOL- titude towards the game. The proposed method us- IDAY” could be experienced in 12 places all around ing MR can be an effective way to better understand the world. However, the experience isn’t immersive the tangible and intangible aspects of heritages. enough due to the screen size of mobile phones and the distraction from the artifacts when focus on de- RELATED WORK vice screens (Natalia Vainstein et., 2016). The increas- This section discusses further the related work in ex- ing number of head-worn AR devices (Marco Mason, hibiting digital information with AR or MR technol- 2016) opens up new probabilities to augment that experience. Some studies applied HoloLens to digital

224 | eCAADe 39 - Digital heritage - Volume 2 Figure 1 heritage, resulting in better engagement and learn- Chaozhou Paifang ing effects (Pollalis et al., 2018). However, these stud- Street, consisting of ies using HoloLens seldomly developed the full po- many memorial tential in interplay among the user, the tangible and archways the intangible.

Social-Interaction Connections to other people are much valued, both for their own sake and as a way to establish a meaningful experience. However, many studies only conducted the individual experience (Garagnani et al.,2016; Pollalis et al., 2017). While to enhance the engagement, some studies developed applica- tions (ArtLinks and Imprints) with social interaction through sharing DIY contents for artifacts in the mu- seum (Dan Cosley et al., 2008; Kirsten Boehner et al., 2005). Meanwhile some recent AR game has made some significant attempts. HADO, an AR sport game that allows 6 participants to make into 2 teams to play, proposed a new method of multiplayer inter- action. So far, the method is rarely applied to digital heritage, which needs careful integration with mean- ingful contents.

METHODOLOGY Intangible data collection Study area This section will explain the procedures adopted in Chaozhou, having more than 700 cultural relics and our experiment of intangible data. The intangible historic sites and 5 national key cultural relics pro- information, which represents the cultural and his- tection units, is an excellent city with self-contained toric figure of the ancient town, was mainly collected and unique culture,. For example, Chaozhou Paifang by visiting different heritage spots on-site and inter- Street, consisting of many memorial archways, has a views with the natives. With the work above, the fol- high value both on the tangible and intangible as- lowing intangible information was presented: pects (Figure 1). However, in light of the shortages in the guidance system, some of the intangible cul- • Archways were constructed to glorify the elites tural value, such as the oral history of this spot or or aristocrats who made great contribution to the time effect on that, is important but hard to be the whole town, such as the resisting the inva- captured by outsiders, preventing them from further sion or leading to build a significant infrastruc- learning about the abundant cultural value. There- ture. fore, the case study took Chaozhou ancient town as • Traditional houses in Chaozhou have obvious or- an example to assess the feasibility of strengthen- ders in space. For example, the main building ing relationship between the tangible and intangible is located on the central axis, while the subor- through the creative contents and immersive interac- dinate buildings are arranged on sides and cor- tion integrated in a MR game. ners. The distribution of buildings could indicate

Digital heritage - Volume 2 - eCAADe 39 | 225 one’s family status in a way. HoloLens 2 was chosen as the hardware platform for •Festivals in Chaozhou also represent the cultural development. value. For example, Spring Festival is the most important one when the family member will go USER TESTING back to hometown and celebrate it together. Overall, the following emergent themes were identi- A significant festival activity “burning the tile fied from 3 aspects: favorite contents, main attitude tower” will be held during The Mid-Autumn Fes- towards the game and the collectivity of users. The tival to pray for pest reduction. case study area was selected through investigations on the urban context and preliminary visits to the an- Prototype implementation cient town. The center plaza of town, which is located Embedded in the intangible, meaningful and cre- at the intersection of the two main roads, was chosen ative contents, such as several cartoon characteris- for this experiment to implement a test on how the tics and a series of comics strip, were made to better tangible and intangible information could be inter- arouse the users’ interests in local culture, leading to played in the MR game. The participants, mainly the further dissemination of the heritage. (Figure 2). tourists, were chosen randomly. Totally we had 22 The system software in this study was developed volunteers for this testing. The holograms of archway by a game engine, Unity. Unity is a cross-platform and board are based on the archway in real scene. game development engine, with a hierarchical, com- The holograms of comics and cartoon model were prehensive development environment, including vi- designed to introduce Chaozhou Ancient Town (Fig- sual editing, detailed attribute editor and dynamic ure 4). And simple questionnaires were made for ba- game preview functions. It is also used to make sic data of the users, along with their favorite con- games or develop game prototypes. Thus, the proto- tents and main attitude towards the game (Figure 5 type was completed in Unity (Figure 3) and the ver- & Figure 6). sion adopted in this research was 2019.4.20f1c1. • First, the 3D model and creative contents, “Arch- As for the selection of hardware devices, the way Model”, “Cartoon Model” and “Comics”, are technology of mixed reality is applied to Microsoft more favorable than others (Figure 5). HoloLens 2. Considering of making users get a big- • Second, the users, who thought the game was ger field of view (FOV), compared with smartphones, interesting, accounted for the largest proportion Figure 2 Handrawing comics with intangible information such as Spring Festival

226 | eCAADe 39 - Digital heritage - Volume 2 Figure 3 The interfaces of Game prototype in Unity

Figure 4 User testing in the center plaza of Chaozhou ancient town

in this survey. (Figure 5). find after the testing of the former user. The second is •Third, compared with individual users, collective that the scale of some user interface design should be users accounted for a larger proportion in the considered carefully. For example, the audio button survey (Figure 6). and the tags of models in our project were designed too small to tap it easily. The third is that the color There were some precautions taken that should be of interface would get dark somewhere, which might pointed out. The first is that sometimes the hologram be affected by the ambient light, making it unclear to content would be moved into somewhere hard to

Digital heritage - Volume 2 - eCAADe 39 | 227 Figure 5 The data of users’ favorite contents and main attitude towards the game

Figure 6 The basic data of users (gender, age and whether collective)

read. heritage through this way, some user expressed that We aware that our method may have some lim- it was hard to recognize the detail things due to the itations. The first is that some users found it hard rainbow effect of the picture, which would reduce the to learn HoloLens due to the picture quality, ges- interest in further learning. The second is the gesture ture mistake and the limitation of user view. When mistake. Many users would mistake the two gestures, asked whether it would help better understand the “air-tap” and “touch”. “Touch” gesture is more applied

228 | eCAADe 39 - Digital heritage - Volume 2 to the near hologram. It would be instinctive for users fers other attempts by exhibiting not only the tradi- to “touch” the hologram far away, for which “air-tap” tional contents, such as the 3D-model, but also the is more suitable. The third is the limitation of user creative ones such as the appealing comics and cute view. There were troubles and misunderstandings in cartoon characteristics. For the video, audio and the guiding the user since the hologram was only shown image and words were less adopted, it seems to be on the user’s eyes. due to the overlay of the pre-existing counterpart in the real environment. DISCUSSION We have presented a new method in strength- The main goal of this experiment was to attempt to ening the relationship between the tangible and in- find a method to strengthen the interplay between tangible by creative and meaningful contents. The the tangible and intangible heritage to create a mul- results suggest that they could arouse users’ interest tidimensional experience. We will discuss that from and result in better engagement. This kind of con- 3 perspective: contents, individual-interaction and tents could also be applied to another immersive en- social-interaction. vironment such as AR or VR. There’s also some limita- tion in this part. The main one is that the animation Favorite contents of cartoon characteristic is relatively simple. We are in the process of enriching the contents by not only Figure 7 embedding the animation on cartoon characteristic Users’ favorite but also adding the architectural models on different contents in this MR scales, for example, the whole ancient town model game on the city scale, different kinds of traditional houses on the group buildings scale, the archways model on the individual building scale, and some node model such as Dougong on the detail scale.

Individual-interaction

Figure 8 Users’ main attitude We made the surveys of “Your favorite contents”, fig- towards this game uring out that the “Archway Model” was most wel- comed, which was adopted by 9 users, 40.9% of the all. The “Comics” and “Cartoon Model” were also welcomed as the second and the third favorite one, while no one adopted the “Audio” and “Text and Im- age” (Figure 7). Prior work has embedded some in- tangible contents into the digital heritage. Garag- nani (2016), for example, shared a 3D-map to show the ephemeral nature of the intangible. And Rush- ton (2020) made a whole experience from the phys- ical models to the digital models. The 3D model is Second, although there were some difficulties in commonly chosen in these studies and most wel- learning HoloLens, most users found it interesting comed in this experiment. This is thought to be due towards this kind of interaction. The result of sur- to the three-dimensional attribute that is well suited vey “your main attitude towards this game” revealed to immersive environment. However, our study dif- that, 13 users, 59% of the users showed interest in

Digital heritage - Volume 2 - eCAADe 39 | 229 this game (Figure 8). This is probably because the Social-interaction HoloLens is novel to participants. and thus, in spite of the unfamiliarity of the gesture-based interaction, Figure 9 we did not find a significantly high rate of frustration The number of the with this game. Prior work has tried to create a better collective and interaction based on hardware and contents. Pollalis individual users (2018) made the immersive exhibition with HoloLens, while the intangible embedded into the contents wasn’t much different with the existing information. Chen (2020) combined the gamification with AR ex- hibition based on hand-held devices. However, the immersive experience is limited due to the screen size and relatively simple interaction (i.e. just click- ing on the screen). Our study differs from these at- tempts by combining MR, which could blend the tan- gible and intangible together, with meaningful and creative contents that embedded intangible informa- tion. For example, it allowed the user to scale and rotate the archway model by double hands gesture with illustrative tags pertaining to the model when Third, we found many users were more collective selected. And the cartoon characteristics would grow than individual. It resulted in that 19 users, 86.4% bigger with prompt tone when selected. This study is of the all, were collective users(Figure 9). One of the the first step in trying to blend the tangible and intan- reasons is probably that it is more common for peo- gible based on interaction within MR environment, ple to travel to the heritage with several friends or which might also be useful for the facade changing colleagues than by oneself. However, the limitation practice in ancient town by visualizing and compar- at this aspect was that the users could neither share ing several design projects without damaging the their views with other users nor to the public due to heritage. the development time limitation on the prototype. As expected, there were three limitations. The Cosley(2008) and Boehner (2005) developed an ap- first was that the interaction could be more auto- plication on hand-held device (ArtLinks and Imprints) matic since the existing way, mainly by clicking tags, to enhance the engagement with social interaction was much similar with the interaction on mobile through sharing personalized tags for artifacts in the phones. The second was that the tangible attributes museum. These works surely opened up new possi- in the interplay were not well-developed. The third bilities on our future work. To further research, the was that the picture quality of HoloLens wasn’t ideal. first step is to develop the spectator view on a screen, Research into solving the above problems is already showing what is happening in HoloLens to the pub- in progress. First, it would be better to pop-up the lic for the better dissemination of the intangible her- contents automatically by AR recognition of a certain itage. The second is to achieve the sharing of same object when touring. Second, to better connect the holograms on multiple HoloLens devices to enhance tangible and intangible, it is much needed to utilize the social interaction. spatial mapping function in HoloLens. Third, the de- fect of picture quality, may be solved by trying other HMD MR hardware such as Nreal.

230 | eCAADe 39 - Digital heritage - Volume 2 CONCLUSION Boehner, K, Thom-Santelli, J, Gay, G, Hall, J. S and Barrett, Stronger relationship between the tangible and in- T 2005 ’Imprints of place: creative expressions of tangible heritage results in better understandings the museum experience’, CHI’05 extended abstracts on Human factors in computing systems, pp. 1220- of the heritage. We assessed the potential of 1223 the MR game in strengthening that relationship in Carrozzino, M and Bergamasco, M 2010, ’Beyond virtual Chaozhou, China. Various intangible meaningful and museums: Experiencing immersive virtual reality in creative contents, such as comics, were superim- real museums’, Cult. Herit, 4, p. 452–458 posed onto the tangible heritage on-site by HoloLens Chen, Z. R 2020 ’The Guidance System of Gamification 2. The result was that it was the embedding of intan- and Augmented Reality in a Museum Space.’, Pro- ceedings of the 25th International Conference of the gible information, much amplifying the details of the Association for Computer-Aided Architectural Design tangible, that made the experience more dramatic, Research in Asia (CAADRIA), pp. 671-680 immersive and engaged, leading to more interaction Cosley, D, Lewenstein, J, Herman, A, Holloway, J, Baxter, back with the tangible heritages. J, Nomura, S and Gay, G 2008 ’ArtLinks: fostering In the light of the significant findings, we believe social awareness and reflection in museums’, Pro- it may contribute to helping visitors get more posi- ceedings of the SIGCHI conference on human factors in computing systems, Florence, Italy, pp. 403-412 tive engagement with the heritage and a more com- Ferreira de Barros Filho, G and de Farias Panet Barros, A prehensive understanding of its tangible and intan- 2018 ’A gamified perspective in the learning of pat- gible aspects. This study is the first step towards the rimonial architecture’, Proceedings of SIGRADI 2018, issue and the game clearly has some limitations due São Carlos, Brazil, pp. 1120-1127 to hardware or lack of programming knowledge. De- Garagnani, S, Arteaga, J and Bravo, L 2016 ’Understand- spite this we believe our findings may significantly ing Intangible Cultural Landscapes-Digital tools as a medium to explore the complexity of the urban improve the interpretation and dissemination of the space’, Proceedings of ascaad 2016, London,United heritage. Future work will mainly cover the develop- Kingdom, pp. 431-436 ment of the additional features for the game, such He, Z, Wu, L and Li, X.R 2018, ’When art meets tech: the as animation for the cartoon characters and social- role of augmented reality in enhancing museum ex- interaction among multiple HoloLens users. periences and purchase intentions’, Tour. Manag, 68, p. 127–139 Leopardi, A, Ceccacci, S and Mengoni, M 2019 ’Dynamic ACKNOWLEDGMENT projection for the Design of an adaptive Museum This research was supported by the following funds: Guide’, International Conference on Design, Simula- National Social Science Foundation of China (No. tion, Manufacturing: The Innovation Exchange, pp. 19FXWB026); Youth Project of National Natural Sci- 85-94 Mason, M 2016, ’The mit museum glassware prototype: ence Foundation of China (No. 51908158); Science Visitor experience exploration for designing smart and Technology Planning Project of Guangdong glasses’, Journal on Computing and Cultural Heritage Province (No. 2017A020220011); General project of (JOCCH), 9 (3), pp. 1-28 Shenzhen University Stability Support Plan; Harbin Mengoni, M, Ceccacci, S and Leopardi, A 2018, ’Spa- Institute of Technology virtual Simulation Experi- tial Augmented Reality:an application for human ment Teaching project (No. : AY11000014). work in smart manufacturing environment’, Procedia Manufacturing, 17, p. 476–483 Noh, Z, Sunar, M.S and Pan, Z 2009 ’A review on aug- REFERENCES mented reality for virtual heritage system’, Interna- Bekele, M. K. and Champion, E. R. I. K. 2019 ’Redefin- tional Conference on Technologies for E-Learning and ing mixed reality: userreality-virtuality and virtual Digital Entertainment, pp. 50-61 heritage perspectives’, Proceedings of CAADRIA 2019, Pollalis, C, Fahnbulleh, W, Tynes, J and Shaer, O 2017 Wellington, New Zealand, pp. 675-684 ’HoloMuse: Enhancing engagement with archaeo- logical artifacts through gesture-based interaction

Digital heritage - Volume 2 - eCAADe 39 | 231 with holograms’, Proceedings of the Eleventh Inter- national Conference on Tangible, Embedded, and Em- bodied Interaction, pp. 565-570 Pollalis, C, Gilvin, A, Westendorf, L, Futami, L, Virgilio, B, Hsiao, D and Shaer, O 2018 ’ARtLens: enhancing mu- seum visitors’ engagement with african art’, Proceed- ings of the 2018 ACM Conference Companion Publica- tion on Designing Interactive Systems, pp. 195-200 Rua, H and Alvito, P 2011, ’Living the past: 3D models, virtual reality and game engines as tools for support- ing archaeology and the reconstruction of cultural heritage–the case-study of the Roman villa of Casal de Freiria’, Journal of Archaeological Science, 38(12), p. 3296–3308 Rushton, H and Schnabel, M. A 2020 ’Exhibiting Digital Heritage-The Curation of Un-Mediated Experiences in Museums’, Proceedings of CAADRIA 2020, pp. 130- 140 Rushton, H, Silcock, D, Rogers, J and Schnabel, M.A 2018 ’The tangible and intangible: interpreting mod- ern architectural heritage in virtual realities’, Amps proceedings series 15, Conference Tangible–Intangible Heritage(s)–design, socialandculturalcritiquesonthe past, present and the future, London, pp. 13-15 Silcock, D, Rushton, H, Rogers, J and Schnabel, M. A 2018 ’Tangible and Intangible Digital Heritage-Creating Virtual Environments to Engage Public Interpreta- tion.’, Proceedings of eCAADe 2018, pp. 225-232 Vainstein, N, Kuflik, T and Lanir, J 2016 ’Towards us- ing mobile, head-worn displays in cultural heritage: user requirements and a research agenda’, Proceed- ings of the 21st international conference on intelligent user interfaces, pp. 327-331 Varinlioglu, G and Halici, S. M 2019 ’Gamification of Her- itage through Augmented Reality’, Proceedings of eCAADe 2019, pp. 513-518 [1] https://everylittled.com/article/134905 [2] https://hado-official.com/en/

232 | eCAADe 39 - Digital heritage - Volume 2 Reinterpretation of the Dougong Joint by the use of Parametric Tools and Robotic Fabrication Techniques

Jiangyang Zhao1, Davide Lombardi2, Hanmei Chen3, Asterios Agkathidis4 1,3,4University of Liverpool, Architecture School 2Xi'an Jiaotong - Liverpool Uni- versity ,Department of Architecture 1,3,4{psjzha32|Hanmei.Chen|a3lab}@liverpool.ac.uk 2Davide.Lombardi@xjtlu. edu.cn

Traditionally, Chinese architecture was based on the use of timber frameworks as structural system. The Dougong joint is amongst the typical connection typologies, widely applied in the timber heritage buildings in China. Each component of the Dougong (bucket-arch joint) conforms to a strict structural proportion in addition to simple but efficient connection methods between its different components. However, the spread of the structure in modern architecture is limited due to high labour cost. Parametric design and digital fabrication techniques have greatly promoted the development of complex timber structures in recent years, which could be introduced in order to reinterpret the Dougong joint. In continuation of our research on exploring the application of robotic technologies for the fabrication of traditional Chinese timber joints, our paper will investigate the feasibility of the structural logic of the Dougong and how it could be applied in a modern timber framework structure.

Keywords: Dougong joint, timber structures, parametric design, robotic fabrication, optimization algorithm, topology optimization

INTRODUCTION very long history. Being one of the prominent fea- The Dougong joint tures of traditional Chinese timber frame architec- The Dougong (bucket-arch), also known as Kegong, ture, the Dougong was widely used during the Han Douke, Zantiao, Puzuo, etc, is a key structural ele- Dynasty, 202 BC - 220 AD, (Dehua, 2011). The Dou ment of traditional Chinese architecture (Ma, 2003). is a bucket-shaped wooden block, and the Gong is a Located between the top of the column and the bow-shaped short piece of wood. The Gong is placed crossbeam transfers the load of the building’s eaves on top of the Dou, protruding outwards, and the Dou to the column. Besides China, the bucket arch is is placed on the end of the Gong, thus it is criss- a key component of the East Asian wood frame crossed layer by layer, forming a structure with a big building structure (Yuan et al., 2011). The emer- upper and a smaller one (Figure 1), (Dehua, 2011). gence and development of the Dougong joint had a

Digital heritage - Volume 2 - eCAADe 39 | 233 Research aims Figure 1 An inexperienced craftsman will need a long time The structure of the to train his skills before becoming experienced, Dougong which would be required for the construction of the Dougong as it is the most complex wooden tectonic detail in traditional Chinese buildings. There are even explicitly trained craftsmen named ‘Dougong car- penters’ explicitly trained for this purpose (Ma, 2003). But if the proportions and size of the timber tecton- ics are stored in the form of the parametric model and CNC/robotic technologies could be applied to produce the structural elements based on that data, even an inexperienced craftsman could quickly pro- duce the corresponding structural components (Will- The use of Dougong in ancient Chinese mann et al., 2016). A parametrized, robotic system buildings could potentially not only save time significantly but The Dougong has four main functions: load transfer, also transfer traditional design knowledge to con- extending the end of the roof overhang, earthquake temporary design solutions, which would enable the resistance and decoration. The Dougong is located fully automated production of complex timber com- between the column and the beam (Figure 2), (Dehua ponents (Willmann et al. 2016). Based on the above 2011). The load from the roof and the upper frame hypothesis our paper will answer the following ques- is transmitted to the column through the Dougong, tions: and then from the column to the foundation. It also 1. What is the structural logic of the Dougong? transfers the load of the roof overhang ending to the 2. How can we utilise Dougong’s structural logic pillar, thus extends its length. The Dougong plays by using parametric tools, and apply it in modern ar- a role in earthquake resistance as well (Yuan et al. chitectural solutions by using robotic tools? 2011). Its elements are connected to each other by To answer these questions, we will first analyse mortise and tenon joints, which releases the energy different case studies which have introduced meth- transmitted to the building by an earthquake and ods of how to utilise traditional timber structures into greatly reduce the seismic load of the entire house modern architecture by using parametric tools and (Wu, Song, and Li 2018). Due to its aesthetics and robotic fabrication techniques. We will then propose exquisite structure, it is amongst the most character- a computational design and fabrication framework istic decorative elements of traditional Chinese archi- that will integrate topology optimization, a genetic tecture. algorithm and robotic fabrication after analysing the underlying principles of the Dougong set. In this paper, we will mainly focus on the feasibility of the Figure 2 structural logic of the Dougong using parametric The locations of the tools. Dougong joints on a traditional Chinese building

234 | eCAADe 39 - Digital heritage - Volume 2 MATERIALS AND METHODS industrial robot. They firstly analyzed the connec- Literature review tion methods of the three modular timber systems We will conduct a comparative analysis of three dif- and then designed new modular units based on the ferent cases studies, two of which have analyzed the analysis result. The 5-axis ABB industrial robot used structural logic of timber tectonics of traditional Chi- a spindle and a milling end-effector in order to pro- nese architecture and applied it in contemporary de- duce the new units. Their research provides a good signs, while the third case introduces the voxeliza- insight into the process of robotic fabrication of the tion, topology optimization and stress analysis of tim- re-invention of traditional modular units. However, ber tectonics. the researchers did not explain the underlying prin- At the University of Tongji, the research team of ciples of the three material systems in a systematic Philip Yuan explores the possibility of the application way and did not try to optimize the size of the new of a traditional Chinese eaves rafter in modern ar- structures. chitecture by fabricating an umbrella-shaped, mod- Naboni and Kinics’s (2019) developed an opti- ular installation with a 5 axis CNC router (Yuan and mized bridge consisting of modular timber units. Hua, 2019). They developed a workflow, including They explored a new design and robotic assembly the prototype research of the eaves rafter, the sim- method including techniques such as topology op- ulation and optimization of structural performance, timization, voxelization, and robotic fabrication. Af- and digital fabrication. Through simulation, anal- ter designing the bridge out of a modular unit, they ysis, and comparison of the bending moments of applied ‘Millipede’ to analyse the stress lines of the the eaves rafter under three kind of overhanging ra- bridge’s structure. After that, the structure is subdi- tios, the researchers found out that the original ratio, vided into square boxes for making sure each stress which is the underlying principle of the eaves rafter, line is applied to each box. Each unit is following is reasonable. They developed a new modular unit the orientation of the stress lines inside of the square and applied it to design an umbrella-shaped struc- box. The new bridge design is lighter than the orig- ture based on the principle that a support point di- inal one, despite having the same mechanical prop- vides the eave’s rafter into two parts while the pro- erties as the previous bridge. Two UR10 robots were portion between the two parts is being maintained used to produce the units and assemble the structure at 0.46-0.48 (Yuan and Hua, 2019). Before fabricating following the coordinates of the square boxes. Their the umbrella structure, the cross-section of the unit method provides a useful reference for our research. was optimized using the Millipede plug-in for Rhino/- However, the researchers do not take into consider- Grasshopper, which helped to reduce the structure’s ation further aspects such as the design method and weight. The fabrication process included the intro- size optimization of the unit. duction of a 5-axis CNC router. Their methodology provides a good insight into the field of reinterpre- Methodology tation of traditional timber tectonics. The wood ele- Having analysed these case studies, our research in- ment is produced by a robotic arm, but the structure troduces a new methodology that includes three was assembled manually. main phases (Figure 3), which are: the study of the At the University of Hong Kong, Lange (2017) underlying principles of the Dougong, the design and his studio are exploring the reinterpretation of and optimization of the unit, the optimization of the three kinds of traditional Chinese tectonic structures structure and the robotic production of the unit. The such as the Dougong, a reciprocal structure found in paper will continue our previous research (Zhao et al., woven, timber arch bridges in China, and the Chidori 2020) to analyse the proportion of the Dougong ele- system by using Rhino/Grasshopper and a 5-axis ABB ments and will present a new conceptual method of

Digital heritage - Volume 2 - eCAADe 39 | 235 Figure 3 The design to fabrication framework

optimizing a small timber footbridge. workflow, topology optimization, voxelization, multi- The first phase is about understanding the un- objective optimization, and robotic fabrication are derlying principles of the Dougong joint by study- integrated with design, analysis and structural opti- ing and analyzing the historic rulebook Gongchen mization in a Rhino/Grasshopper environment. Dur- Zuofa Zeli recording the size, the technology, and the ing the experiment process, the weight and displace- structure of different kinds of Dougong joints. In this ment of the research project are set up as fitness ob- phase, the size and proportions of the Dougong el- jects. The section size of the unit designed following ements are being defined and the reasonability of the underlying principle of Dougong is set up as the the ratio of the distance between each adjacent force genetic data. The multi-objective optimization algo- point of the components of the Dougong are being rithm will calculate the smallest value of the weight tested through structural simulation in ‘Karamba3D’ and displacement of the research object by adjusting (Preisinger, 2013), a plug-in for structural analysis in the section size of the unit. The research will explore Grasshopper. the feasibility of the workflow by verifying it on a lin- The second phase is about designing and opti- ear and a non-linear structure. A wooden bridge has mizing the principal unit based on the mechanical been chosen as the design object on which our com- rules of the Doungong joint in the Rhino/Grasshop- putational framework will be applied. per environment. In this phase, ‘Millipede’, a plug-in Phase one: Analysing the underlying rules of the for topology optimisation and ‘Galapagos’ a plug-in Dougong. Through studying and analyzing the his- for genetive algorithms were used to analyze and op- toric rulebook Gongchen Zuofa Zeli, the proportion timize the cross-section of the unit as described in our rules and the connection methods of different kinds previous paper (Zhao et al., 2020). of Dougong joints were revealed. Each Dougong el- In the third phase, we developed a computa- ement is equally divided into 2*n parts [n>0, n฀N (in- tional workflow aiming to simplify the process of ap- teger), one part=3 Dou] according to the distances plying the unit for the design of architectural so- between two adjacent load and support points. Both lutions such as a wall, a bridge or a pillar. In our

236 | eCAADe 39 - Digital heritage - Volume 2 Figure 4 Option A: both sides of the element extending 2 parts outwards from the support point of the edge (left), Option B: both sides extending 1 part (right)

sides are either extending two parts or one part out- thus the width of one Doukou is 19.2cm. Each class wards from the support point of the edge (Figure is linked to a particular building function and any 4). There are three kinds of angles between two ele- change from class to class is based on the formula de- ments, such as 120°, 135° and 90°(Ma, 2003). The con- scribed by Zhao et al. (2020): nection of the different elements in the same layer 1 Dou = 19.2cm + (1- X) * 0.5 Cun, takes place by the crossing of three straight timber (1Cun=3.33cm), X: the class of Cai (1, 2, 3, 4, ..., 11);Cun joints or a cross-shaped joint (Figure 5). The connec- is the traditional unit of length in China tion of the elements in the different layers takes place An element of Dougong is chosen as an object to by using straight tenons. According to the studies test the feasibility of the traditional proportion rules of Ma (2003) and Liang (2006), the size of the cross- by using ‘Karamba3D’.We choose the first-class Cai (1 section of most of the elements in the Dougong is Dou=19.2cm) as the standard unit. The dimensions set as 1 Dou in width and 2 Dou in height or 1.25 of the element are 19.2cm in width, 24cm in height Dou in width and 2 Dou in height. The ratio of height and 22.5cm in length, which can be considered as to width equals 1:2 approximately. Therefore, we set a beam to tested. Options A and B are both tested the unit’s preoptimization size to 1 Dou in width and 2 in ‘Karamba3D’. In the simulation, the figuration of Dou in height. There are eleven classes in the Doukou the displacement plays an important role to evalu- system, the width of the first-class Dou is 19.2 cm, ate the fitness of the structure. According to the stiff-

Figure 5 The Dougong connection methods

Digital heritage - Volume 2 - eCAADe 39 | 237 Figure 6 The simulation-results of option A, where both sides of the element are extending 2 parts outwards from the support point of the edge (left), and version B, where both sides extending 1 part (right)

238 | eCAADe 39 - Digital heritage - Volume 2 ness calculation,κ=F/σ (κ: the stiffness, F: the force on are 1:5:3:3:5:1, the displacement is smaller than the the body, σ: the displacement), a smaller displace- element (1:1:1:1:1:1), but both sides outwards of the ment means that the structure is more stable with the structure are reduced compared with the element given forces. (1:1:1:1:1:1). After a comprehensive comparison, the For option A we assume that the element is sub- original force ratio appears to be the most reason- jected to point loads of 1 kN. When A: B: C: D: E: F able. equals 1: 1: 1: 1: 1: 1, the displacement of the el- Phase two: the design and optimization of the ement is 0.002491cm. When A: B: C: D: E: F equals unit. We have explored strategies to simplify the 3: 4: 2: 2: 4: 3, the displacement of the structure is Dougong elements for making them more suitable 0.005011cm. The displacement of the latter is larger for modern architecture in our previous paper (Zhao than the former, which means the stiffness of the ele- et al., 2020). We simplified two elements of the ment has been reduced. Thus, the stiffness will be re- Dougong by adjusting their dimensions to integers duced when the length of the overhanging structure and removed the decorative elements and their com- is increased. When A: B: C: D: E: F equals 3: 1: 5: 5: 1: plex curves (Figure 7). At the same time, we have de- 3, the displacement of the element is 0.00206cm (Fig- signed two units based on the mechanical properties ure 6). The displacement of the structure is smaller and proportions of the Dougong and explored as- than the original structure. However, the length of sembly possibilities following the connecting rules as the overhanging part is reduced. In summary, when presented by Zhao et al. (2020). After that, the cross- the proportion of the distance between the original section of the units was optimized with Millipede and structure’s stress points is 1:1:1:1:1:1, the structure is Galapagos. reasonable relative. Phase three: the optimization of a wooden arched In option B, the displacement of the element is footbridge. A small arched footbridge was chosen as 0.001003cm, 0.005011cm and 0.00206 cm, when the an object to evaluate our proposed method of opti- proportion of the distance between the load points mization and robotic assembly (Figure 8). Firstly, the is 1: 1: 1: 1: 1: 1, 4:2:3:3:2:4 and 1:5:3:3:5:1 respec- arch structure is discretized into cubic cells, the size tively. When A‘: B’: C‘: D’: E‘: F’ equals 4: 2: 3: 3: 2: of which is set by us. In the process of topology op- 4, the displacement of the element is larger than the timization, the topology optimization algorithm cal- original proportion, which means that the fitness is culates which cells should be removed, based on the not better than the original element of the Dougong utilization rate within the structural system accord- (Figure 6). At the same time, when the proportions

Figure 7 The new simplified elements of the reinvented Dougong unit

Digital heritage - Volume 2 - eCAADe 39 | 239 Figure 8 The topology optimization process of the footbridge.

240 | eCAADe 39 - Digital heritage - Volume 2 ing to a given set of loads, boundary conditions, and where A: B: C: D: E: F equals 3: 4: 2: 2: 4: 3, the dis- constraints which include the live load of 3kN/฀which placement was larger than in the original Dougong has been applied to the footbridge as well as set- element, which means that the stiffness of the ele- ting the supporting points of the footbridge to be ment is smaller than the original one. The displace- on the ground and not being able to move along ment is smaller than the original ratio when A: B: C: or rotate around the XYZ axes. Topology optimiza- D: E: F equals 3: 1: 5: 5: 1: 3, but both sides of the ele- tion is a mathematical method for structural opti- ment extend outwards reduced in comparison to the mization of material layout within a given area ac- original element of the Dougong. This weakens the cording to a given set of loads, constraint conditions properties of the Dougong because one function of and performance indicators (Bendsoe and Sigmund, it is to extend the overhang. In the situation of option 2013). Taking material layout as the optimization ob- B with the same constraints, when A‘: B’: C‘: D’: E‘: F’ ject, through topology optimization, the algorithm equals 3: 4: 2: 2: 4: 3 or 3:1:5:5:1:3, the displacement can calculate the most efficient material layout of the is larger or smaller than the one (1:1:1:1:1:1). How- design space (Bendsoe and Sigmund, 2013). Sec- ever, this doesn’t appear reasonable compared to the ondly, a cubic cell was set as a boundary to con- original proportion (1: 1: 1: 1: 1: 1), because of the trol the position of the units within the bridge arch. same reasons observed in the simulation of option A. Thirdly, the weight of the arch structure was reduced The results of the simulation show that the ratio of by optimizing the cross-section of the unit by us- 1:1:1... :1 is the optimal standard. ing Wallacei, a plug-in to evolutionary algorithms in The paper also explored a new method of opti- Grasshopper3D. The plug-in is used to calculate the mization and robotic fabrication of a timber arched smaller value of the weight and displacement of the footbridge by analyzing and optimizing it within a structure by adjusting the unit’s cross-section. After given set of loads. The weight of the wooden bridge the optimization, the tool path of the robotic assem- was reduced from 16.592 kg to 2.954 kg after the bly of the units was generated automatically based optimization with Millipede and Wallacei, due to its on the parametric model of the unit within the vi- transformation from a solid structure to a frame struc- sual programming environment. We have adjusted ture being able to carry the same loads. This proves the robotic assembly tool path based on the fabrica- that our method could be useful in the field of op- tion simulation in RoboDK, a firmware for industrial timization and robotic fabrication, which should be robots and offline programming. During the whole explored further in future work. process, the data of each phase was delivering timely feedback to all other phases and ensured that the ar- REFERENCES chitect can control the whole design and manufac- Bendsoe, M.P and Sigmund, O 2013, Topology optimiza- turing process. tion: theory, methods, and applications, Springer Sci- ence & Business Media Dehua, P 2011, Dougong, Nanjing: Southeast university CONCLUSION University Press Our research has revealed and verified the underly- Ma, BJ 2003, Chinese ancient architecture woodwork con- ing principles of the Dougong by analyzing the orig- structiontechnology, China Science Publishing & Me- inal materials and simulating them in Karamba3D. A dia moderately sized component, width 1 Dou in height, Preisinger, C 2013, ’Linking structure and parametric ge- 1.25 Dou in width and 18 Dou in length, was cho- ometry’, Architectural Design, 83(2), pp. 110-113 Willmann, J, Knauss, M, Bonwetsch, T, Apolinarska, A, sen as the object to analyze the feasibility of the Apolinarska„ F and Kohler, M 2016, ’Robotic tim- Dougong element’s proportions. We have tested two ber construction—Expanding additive fabrication rules, option A and B. In the simulation option A, to new dimensions’, Automation in construction, 61,

Digital heritage - Volume 2 - eCAADe 39 | 241 pp. 16-23 Wu, Y, Song, XB and Li, K 2018, ’Compressive and rack- ing performance of eccentrically aligned Dou-gong connections’, Engineering Structures, 175, pp. 743- 752 Yuan, J and Hua, C 2019, ’Experimental research on bracket set models of Yingxian Timber Pagoda’, Jianzhu Jiegou Xuebao, 32, pp. 66-72 Yuan, P.F and Hua, C 2019, ’Reinterpretation of Tradi- tional Wood Structures with Digital Design and Fab- rication Technologies’, in Bianconi, F and Filippucci, M (eds) 2019, Digital Wood Design, Springer, Cham, pp. 265-282 Zhao, J, Lombardi, D and Agkathidis, A 2020 ’Application of robotic technologies for the fabrication of tradi- tional Chinese timber joints’, eCAADe 38, pp. 351- 360

242 | eCAADe 39 - Digital heritage - Volume 2 Comparative Analysis of Open-Source and Commercial Photogrammetry Software for Cultural Heritage

Isidora Đurić1, Ivana Vasiljević2, Miloš Obradović3, Vesna Stojaković4, Jelena Kićanović5, Ratko Obradović6 1,2,6Computer Graphics Chair, Faculty of Technical Sciences, University of Novi Sad 3,4,5Department for Architecture and Urban Planning, Faculty of Technical Sciences, University of Novi Sad 1,2,3,4,5,6{isidoradjuric|ivanav|milos_obradovic|vesna100|kicanovicj|obrad_ r}@uns.ac.rs

Recently, photogrammetry has become a relatively easy and low-cost method for cultural heritage 3D reconstruction. Plenty of free and open-source photogrammetry programs have been developed, but not all of them provide an adequate solution for the 3D reconstruction of cultural heritage. In this research, an overview of the state-of-the-art open-source photogrammetry programs was done. In order to investigate whether among the open-source software packages there is an adequate alternative to the commercial software for the purpose of cultural heritage 3D reconstruction, the open-source software - AliceVision Meshroom was compared to the most commonly used commercial software Agisoft Metashape. The programs were compared on the examples of cultural heritage objects and according to the predefined criteria that are important for achieving detailed and accurate 3D reconstruction. The results of testing and comparing the 3D reconstructions obtained using Meshroom and Metashape were illustrated and explained through two case studies of antic objects of cultural heritage, characterized by similar materialization, but different geometric shapes.

Keywords: photogrammetry, software, cultural heritage, Meshroom, Agisoft Metashape

INTRODUCTION menting cultural heritage (Bartoš et al. 2014; Kersten With the advancement of image processing, pho- and Lindstaedt 2012; Gonizzi-Barsanti et al. 2012), togrammetry has become a relatively easy and in- providing users with different levels of automation, expensive method for 3D reconstruction of cultural visualization and integration with other technolo- heritage (Nocerino et al. 2017). Consequently, there gies, such as virtual reality (Obradović et al. 2020). is a tendency to develop software solutions, avail- Today, photogrammetry software allows image pro- able through free, commercial and open-source pro- cessing to be performed by non-professional users grams, that generate reliable 3D solutions in docu- by having only an integrated camera on their mo-

Digital heritage - Volume 2 - eCAADe 39 | 243 bile devices (Tanskanen et al. 2013). However, not SELECTION OF CULTURAL HERITAGE OB- all programs provide an adequate solution for the JECTS 3D reconstruction of cultural heritage (Remondino This paper is a part of the wider research on develop- et al. 2012; Gagliolo et al. 2018). Parameters such ing a strategy for cultural heritage digitization, sup- as high level of precision, reliability in texture gen- ported by the Ministry of Culture and Information of eration, data compression are key items that image the Republic of Serbia (Stojaković et al. 2020). The processing technology must meet in order to repre- research also presents some of the results of the col- sent a reliable and visually acceptable form of recon- laboration between the university team and the Mu- structed objects (Menna et al. 2016; Nocerino et al. seum of Vojvodina in Novi Sad on digitization of antic 2012; Cipriani and Fantini 2017). objects of cultural heritage and the development of The aim of this paper is to investigate whether the virtual museum for the interactive 3D visualiza- among the open-source software packages there tion (3D Scene: Interactive 3D Walkthrough. 2020). is an adequate alternative to the commercial soft- The objects used in the research were selected ware for the purpose of the detailed and accurate according to the predetermined criteria, designed to 3D reconstruction of cultural heritage. In this re- single out the most common types of cultural her- search, state of the art open-source photogramme- itage held by museums and galleries in Serbia. Based try programs were analyzed and compared accord- on the survey conducted among the 45 cultural in- ing to general characteristics such as the ability to stitutions, the different types of cultural heritage ob- provide the full photogrammetric workflow (photo jects were analyzed (Stojaković et al. 2020). The cri- alignment, building point cloud, 3D model and tex- teria for analysis were: object size and shape, mate- ture generation, orthophoto and DEM generation), rial, portability, and visual accessibility. According to processing time, complexity, intuitiveness, and ease the given criteria, the representative examples of cul- of use, etc. Moreover, the parameters that are im- tural heritage objects held by the majority of cultural portant for the digitization of cultural heritage were institutions have been marked. The most common analyzed, such as the ability to provide highly accu- types of cultural heritage are characterized as objects rate results for small-scale details. The analysis com- that are between 10 centimeters and 1 meter in size, pared results among the most representative open- made of opaque materials that do not have a high source software packages. Finally, the open-source gloss, whereas most of them are portable and fully software - AliceVision Meshroom (AliceVision Mesh- visually accessible (Stojaković et al. 2020). Room 2021) was compared to the most commonly The shape and size of the objects of cultural used commercial software Agisoft Metashape (Kings- heritage used to test the programs were selected land 2020). The programs were compared on two to present representative specimens with different case studies of antic objects of cultural heritage dis- shape features. Two antic objects of cultural her- tinguished by similar materialization, but different itage, approximately one meter in size, distinguished geometric shapes, which were previously selected by similar materialization, but different geometric as the most representative examples of cultural her- shapes were used. Case studies present objects of itage objects held by the cultural institutions in Ser- immovable cultural heritage from the Ancient Col- bia. A comparative analysis between the testing re- lection of the Museum of Vojvodina: The honorary sults showed that, unlike the commercial software, base erected in honor of Emperor Valerian (Figure 1a) the modeling results in open-source software vary and the tombstone made of white marble (Figure 1b). depending on the shape of the object and the ap- The first object presents a stand-alone monument proach of photogrammetric surveying (convergent of a compact, three-dimensional shape, while the or parallel). second one is characterized by a mainly flat shape.

244 | eCAADe 39 - Digital heritage - Volume 2 In relation to the given shapes of objects, two ap- • Unlike commercial software, none of the open- proaches to photogrammetric surveying were ap- source software provides a full photogrammet- plied and tested: convergent and parallel. Both ob- ric workflow (from building a point cloud to the jects were surveyed using terrestrial photogramme- orthophoto or DEM generation). try. • All of them provide an option for a point cloud generation and export in different file formats, Figure 1 that can be further used for mesh generation in- Case studies: (a) side some other program, such as MeshLab. The honorary base • Programs that allow mesh and texture genera- erected in honor of tion are AliceVision Meshroom, 3DF Zephyr Free, Emperor Valerian; Regard3D, and Multi-View Environment. The (b) The tombstone rest of them require a model to be generated made of white in some other 3D software by exporting a point marble, Museum of cloud. Vojvodina, Novi Sad • Only a few of them allow 3D model optimization (AliceVision Meshroom, 3DF Zephyr Free, Re- gard3D), and edit (3DF Zephyr Free, Multi-View OVERVIEW OF OPEN-SOURCE PHO- Environment), while none of them provide an TOGRAMMETRY SOFTWARE option for scale and measure a 3D model, which The analysis of photogrammetry programs was done can be one of the main disadvantages compared to determine whether among the open-source soft- to the commercial software. ware packages may be an adequate alternative to • AliceVision Meshroom stands out for its intu- the commercial software for the detailed and accu- itive use and high-quality node-based interface. rate 3D reconstruction of cultural heritage. The fol- Unlike most free programs, the Meshroom en- lowing available open-source programs were com- ables almost the entire photogrammetry work- pared to each other according to the parameters flow: photo alignment, building sparse point that are important for the digitization of cultural her- cloud, high-quality mesh and texture genera- itage: COLMAP (COLMAP 2021), OpenMVG (open- tion. Still, the disadvantage of this program MVG 2021), AliceVision Meshroom (AliceVision Mesh- is the impossibility to generate and visualize a Room 2021), Regard3D (Regard3D 2021), 3DF Zephyr dense point cloud (only a sparse point cloud) Free (3DF ZEPHYR 2021), MicMac (Rupnik et al. 2017), within the program workspace itself. A dense VisualSFM (Wu 2011; VisualSFM 2021), Multi-View En- point cloud can be exported, which requires ad- vironment (Fuhrmann et al. 2014). Since every open- ditional work with nodes. source software has certain limitations compared to For further evaluation, only the programs that pro- the commercial one, special attention was paid to vide an option for full photogrammetry 3D recon- highlight whether some of them provide advanced struction (from point cloud to texture mesh) were se- options such as the ability to provide the full pho- lected and further tested on the selected objects of togrammetric workflow, the ability to introduce scale cultural heritage. They were tested on the quality of from 3D models, measure, edit or optimize generated 3D reconstruction, processing time, complexity, intu- point cloud. itiveness, and ease of use, etc. Based on the given In the following, the main advantages and limi- analysis, testing results, and the author’s previous ex- tations of open-source programs are summarized: perience in photogrammetric modeling, the AliceVi- sion Meshroom software was selected as an appro-

Digital heritage - Volume 2 - eCAADe 39 | 245 priate solution for the needs of cultural heritage pho- Special attention was paid to recording the real state togrammetric modeling. The 3D reconstruction re- of the object. sults produced by Meshroom showed a high level of The level of recording a real material/color of the a real shape and texture representation of the digi- object was analyzed by visual assessment of the gen- tized objects. erated high-resolution textures (4K). In this case, spe- cial attention was paid to recording possible cavities MESHROOM VS METASHAPE and irregularities in the reconstructed texture. In order to test and evaluate the efficiency of the most prominent software among the open-source Case study 1 - Honorary base erected in ones, the AliceVision Meshroom was compared to honor of Emperor Valerian, Museum of Vo- the commercial software Agisoft Metashape (Agisoft jvodina 2021). The 3D model reconstructed by Metashape The first object is located in the western part of the had served as a referent model for the evaluation of garden of the Museum of Vojvodina and represents the open-source software Meshroom. a stand-alone object of compact three-dimensional The objects 3D reconstruction results obtained shape. The approximate dimensions of the object by Meshroom were compared to the modeling re- are 80 cm (width) and 160 cm (height). Concerning sults from Agisoft Metashape according to the fol- the given shape of the object, a convergent type of lowing criteria: photogrammetric surveying was applied. The main limitation during the surveying was the high vege- • Accuracy / scale of the resulted 3D reconstruc- tation near the monument, which conditioned the tion maximum distance of the camera from the object of • The level of recorded details on the object shape 1.5 meters. Due to that, a focal length of 18 millime- • The level of recording a real material / color of ters was used. The object was captured from several the object. different heights so that all parts of the monument were recorded. Figure 2 presents the camera poses The accuracy of the resulted 3D reconstructions was and the 3D model created as a result of a convergent determined by defining a real scale of the created 3D approach to photogrammetric surveying and pho- models. To introduce the scale of digitized 3D mod- togrammetric modeling by the commercial program els, during the survey, scale bars were placed on the Agisoft Metashape. monument’s pedestals (See Figure 1). The accura- cy/scale of the resulted 3D reconstruction is repre- Figure 2 sented by the mean square error. The mean square Textured 3D model error represented the deviation from an object’s real and camera poses size. Also, the 3D reconstructions obtained by both, obtained using Meshroom and Metashape were analyzed in Cloud- Agisoft Metashape Compare program, in order to estimate the precision of the open-source software in relation to the com- mercial one. The given precision is represented by the values of the mean distances of the points and the standard deviation. The level of recorded details on the object shape was estimated by comparative analysis on the visual appearance of the textured meshes generated using both, the open-source and commercial programs.

246 | eCAADe 39 - Digital heritage - Volume 2 Table 1 Accuracy / scale of the resulted 3D reconstruction. Analyzed Since the open-source program Meshroom does not photogrammetric provide an option to scale the 3D model, the 3D modeling reconstruction obtained by Meshroom was scaled parameters for within the Agisoft Metashape program, as well. The Metashape and deviation, i.e. mean square error (Table 1), of the Meshroom digitized 3D model from the real object shape in (Stojaković et al. the case of Meshroom was 0.288 mm, which was al- 2020) most the same value as for the 3D model created in Metashape. It can be noted that Meshroom provided a high level of accuracy of the resulted 3D recon- struction. This can be related to the compact, three- dimensional shape of the object and the convergent type of object surveying, which provide data that is more favorable for an accurate photogrammetric re- construction. Table 2 The resulting values of the mean distance and Deviation of the 3D standard deviation are shown in Table 2. By com- model from puting distances between the point clouds in Cloud- Meshroom in Compare, it was estimated that the absolute devia- relation to the 3D tion of the 3D model created in Meshroom was 3.5 model from Agisoft millimeters compared to the 3D model reconstructed Metashape by Agisoft Metashape. Based on that, the value of (Stojaković et al. the mean distance was -0.784 mm, while the stan- 2020) dard deviation of the 3D model created by Meshroom was 1.210 mm. These values can be related to the Figure 3 lower capability of the Meshroom to generate dif- Overlapped point ferent small texture depths, compared to the com- clouds from mercial program Agisoft Metashape. The compara- Meshroom and tive analysis of the overlapped 3D models created by Metashape with the Meshroom and Metashape (Figure 3) illustrates the signed mean accuracy of the 3D reconstruction obtained by open- distances source software compared to the commercial one. It (Stojaković et al. can be seen that the deviations of the 3D model re- 2020) constructed by Meshroom are mostly present on ap- proximately flat, smooth, and insufficiently contrast- ing parts of the object.

Digital heritage - Volume 2 - eCAADe 39 | 247 The level of recorded details on the object shape. Figure 4 The level of recorded details on the object shape was Front view of the 3D estimated by a comparative analysis of the textured reconstruction of meshes reconstructed by both - free and commercial the object created programs. By comparing whole textured 3D mod- by using (a) Agisoft els from both programs, obvious differences cannot Metashape, image be observed (Figures 4a and 4b). The image GSD size 2478 x 4096; (Ground Sample Distance) in both 3D models was and (b) Meshroom, around 0.3 mm. The 3D model created by Meshroom image size 2432 x provided a faithful representation of the real state of 4096 (Stojaković et the object without the presence of cavities or irregu- al. 2020) larities on the mesh or texture. It can be seen that all the texture details are noted, such as specific engrav- ings. The level of recording a real material / color of the object. The real material and color of the object were estimated by the visual quality assessment of the tex- Figure 5 ture, paying special attention to the smallest visible Texture details: (a) details. By observing the enlarged parts of the tex- Agisoft Metashape ture generated by Metashape (Figure 5a) and Mesh- vs (b) Meshroom room (Figure 5b), it can be seen that general repre- (Stojaković et al. sentation of the real state of the stone texture, char- 2020) acterized by specific small engravings, was noted. Still, minor differences in the material representa- tion between the two generated textures can be ob- served. The texture color and very small holes on the original stone material were slightly lower quality in the case of Meshroom.

Case study 2 - Tombstone made of white marble, Museum of Vojvodina The tombstone made of white marble is located in the eastern part of the Garden of the Museum of Vojvodina and represents an object of a mainly flat shape. The approximate dimensions of the object are 135 cm (width) and 100 cm (height). Due to the given shape of the object, the monument was surveyed by a combined method of parallel and convergent type of photogrammetric surveying. Due to location re- strictions, the maximum distance of the camera from the subject was 2.5 meters. The object was captured from two heights, using a tripod and landscape cam- era mode. All the camera parameters were manually

248 | eCAADe 39 - Digital heritage - Volume 2 Figure 6 set, using the 30 millimeters focal length. Figure 6 Textured 3D model presents the camera poses and the 3D model created and camera poses as a result of the parallel approach to the photogram- obtained using metric surveying and modeling within the commer- Agisoft Metashape cial program Agisoft Metashape. Accuracy / scale of the resulted 3D reconstruc- tion. Unlike the high accuracy results that Meshroom showed in the previous case study, in the case of the second object, the value of the mean square error was 0.6 millimeters higher than the resulted value of the Metashape 3D reconstruction (Table 3). Also, observing the parameters of photogrammetric modeling, it can be seen that the Meshroom 3D re- construction resulted in a significantly smaller num- ber of points and polygons, compared to the Ag- Table 3 isoft Metashape. This can be related to the mainly Analyzed flat object shape, and the applied parallel type of photogrammetric photogrammetric surveying, which, compared to the modeling convergent type, usually results in a fewer number parameters for of the created photos. Consequently, an insufficient Metashape and number of photos cause fewer data required for a de- Meshroom tailed 3D reconstruction. (Stojaković et al. After the comparison of both 3D reconstructions 2020) in CloudCompare (Figure 7), the value of the abso- lute distances of the overlapped point clouds was 5 millimeters (Table 4). The value of the mean distance was 0.098 mm, while the standard deviation of the 3D model created by Meshroom was 0.914 mm (Table 4). A deviation of approximately 5 millimeters repre- sents cavities or arbitrarily interpolated parts, proba- bly resulted due to insufficient data for Meshroom to generate the 3D model. Table 4 Deviation of the 3D model from Meshroom in relation to the 3D model from Agisoft Metashape (Stojaković et al. 2020)

Digital heritage - Volume 2 - eCAADe 39 | 249 The level of recorded details on the object shape. Figure 7 The level of the Meshroom 3D representation of the Overlapped point real state of the object (Figure 8b) in comparison to clouds from the Agisoft 3D reconstruction (Figure 8a), showed Meshroom and that the general shape of the object, characterized Metashape with the by specific engravings, was faithfully reconstructed. signed mean The image GSD (Ground Sample Distance) in both 3D distances models was around 0.2 mm. (Stojaković et al. 2020) The level of recording a real material / color of the object. By analyzing texture details, it was noted that the textured 3D model created using Meshroom had 5 millimeters cavities on the texture (Figures 9b and 10), which were not present on the 3D model cre- ated in Agisoft Metashape (Figure 9a). Also, as in the Figure 8 previous case study, the differences in the material Front view of the 3D and color representation of the texture generated by reconstruction of Meshroom were obvious. Such irregularities can be the object created related to the generally achieved accuracy and preci- by using (a) Agisoft sion of the 3D reconstruction (See Accuracy / scale of Metashape, image the resulted 3D reconstruction). size 4096 x 3043; and (b) Meshroom, image size 4096 x CONCLUSION 2907 (Stojaković et In this paper, detailed research on open-source pho- al. 2020) togrammetry programs for the purpose of cultural heritage digitization was done. The one who stood Figure 9 out the most among the others - AliceVision Mesh- Texture details: (a) room was compared to the most commonly used Agisoft Metashape commercial software Agisoft Metashape. The com- vs (b) Meshroom parative analysis and the evaluation of Meshroom in (Stojaković et al. relation to Metashape, was performed on the two 2020) representative case studies of objects of cultural her- itage. The results of the object’s 3D reconstructions Figure 10 were compared and analyzed according to the pre- Meshroom texture viously established criteria. The main conclusions re- detail with the lated to the results of the photogrammetric model- presence of cavities ing by using the open-source Meshroom software are of 5 millimeters as follows: (Stojaković et al. • In the case of the compact, three-dimensional 2020) object and the convergent approach to pho- togrammetric surveying, Meshroom provided almost the same quality precision results as Ag- isoft Metashape.

250 | eCAADe 39 - Digital heritage - Volume 2 • In the case of the mainly flat object and the par- However, the precision of the 3D reconstructed allel type of photogrammetric surveying, the ac- models of cultural heritage still mostly depends curacy of the reconstructed 3D model (mean on the surveying process used for acquiring pho- square error) was significantly lower compared tographs (Remondino and El-Hakim 2006; Barsanti to the 3D model created by the commercial pro- et al. 2013; Sapirstein 2016; Djuric et al. 2019; gram. Obradović et al. 2020). This implies that, when it • The level of recorded details on the recon- comes to detailed and accurate cultural heritage dig- structed object was higher in the case of objects itization, profound knowledge of photogrammetric of a three-dimensional shape, while in the case surveying and modeling is required. of the predominantly flat object shape, certain shortcomings and cavities in the 3D model were ACKNOWLEDGMENTS observed. The research was supported by the Ministry of Cul- • In both cases, the visual representation of real ture and Information of the Republic of Serbia, under characteristics of materials and colors of the tex- the Project for Digitization of Cultural Heritage of the tures generated by Meshroom slightly differed Republic of Serbia (Project name: Strategy Develop- from the same one created by Metashape. ment for Standardization of Creation of Photogram- Based on the above mentioned, it can be concluded metric 3D Digital Objects of Cultural Heritage - Pha- that, unlike commercial software, modeling results in seI). the open-source software can vary depending on the The authors would like to thank the Museum of shape of the object and the approach to photogram- Vojvodina, Novi Sad for the collaboration on the dig- metric surveying (convergent or parallel). That may itization and the virtual presentation of cultural her- also be related to the number of photos used for the itage objects from the Museum’s Ancient Collection. 3d reconstruction, concluding that, unlike commer- cial software, free software may require more data to REFERENCES achieve a successful result. Given that, it can be con- Barsanti, S G, Remondino, F, Fenández-Palacios, B J and cluded that the Meshroom may be an adequate al- Visintini, D 2014, ’Critical factors and guidelines for ternative to commercial programs in the case of sur- 3D surveying and modelling in Cultural Heritage’, Int. Journal of Heritage in the Digital Era, 3(1), pp. 142- veying free-standing and mainly three-dimensional 158 objects. Depending on the required level of detail of Barsanti, SG, Remondino, F and Visintini, D 2013 ’3D Sur- a reconstructed 3D model, minor irregularities may veying and Modelling of Archaeological Sites-some occur. It should also be mentioned that, compared critical issues’, Proceedings of the XXIV International to Agisoft Metashape, the Meshroom processing of CIPA Symposium, Strasbourg, France, p. 145–150 the final mesh generation was, in both cases, quite Bartoš, K, Pukanská, K and Sabová, J 2014 ’Overview of Available Open-Source Photogrammetric Software, demanding for the processor. its Use and Analysis’, International Journal for Innova- Nevertheless, having an alternative solution to tion Education and Research (IJIER), pp. 62-70 the professional photogrammetry software is signifi- Cipriani, C and Fantini, F 2017 ’DIGITALIZATION CUL- cant for cultural institutions which can not afford pro- TURE VS ARCHAEOLOGICAL VISUALIZATION: INTE- fessional licenses. In such cases, most of the open- GRATION OF PIPELINES AND OPEN ISSUES’, The In- source programs provide a solution to partially pre- ternational Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Nafplio, pp. serve data on the existing objects, by creating their 195-202 point clouds. The improvement of the open-source Djuric, I, Stojakovic, V, Misic, S, Kekeljevic, I, Vasiljevic, I, software performances would probably contribute to Obradovic, M and Obradovic, R 2019 ’Church Her- their more frequent usage by non-professionals. itage Multimedia Presentation - Case study of the

Digital heritage - Volume 2 - eCAADe 39 | 251 iconostasis as the characteristic art and architectural 21(115), pp. 269-291 element of the Christian Orthodox churches’, Pro- Remondino, F, Del Pizzo, S, Kersten, T P and Troisi, S ceedings of the eCAADe and SIGraDi 2019 Conference, 2012 ’Low-Cost and Open-Source Solutions for Au- Porto, Potugal tomated Image Orientation – A Critical Overview’, Fuhrmann, S, Langguth, F and Goesele, M 2014 ’MVE- Euro-Mediterranean Conference, Berlin, Heidelberg, A Multi-View Reconstruction Environment’, GHC, pp. pp. 40-54 11-18 Rupnik, E, Daakir, M and Deseilligny, M P 2017, ’MicMac– Gagliolo, S, Ausonio, E, Federici, B, Ferrando, I, Passon, D a free, open-source solution for photogrammetry’, and Sguerso, D 2018 ’3D CULTURAL HERITAGE DOC- Open Geospatial Data, Software and Standards, 2(1), UMENTATION: A COMPARISON BETWEEN’, The In- pp. 1-9 ternational Archives of the Photogrammetry, Remote Sapirstein, P 2016, ’Accurate measurement with pho- Sensing and Spatial Information Sciences, Riva del togrammetry at large sites’, Journal of Archaeological Garda, pp. 347-354 Science, 66, pp. 137-145 Kersten, T P and Lindstaedt, M 2012, ’Automatic 3D ob- Stojaković, V, Obradović, R, Đurić, I, Vasiljević, I, ject reconstruction from multiple images for archi- Obradović, M and Kićanović, J 2020, ’Strategy De- tectural, cultural heritage and archaeological appli- velopment for Standardization of Creation of Pho- cations using open-source software and web ser- togrammetric 3D Digital Objects of Cultural Her- vices’, PFG Photogrammetrie, Fernerkundung, Geoin- itage - Phase I’,in Project Report, Faculty of Technical formation, 6, pp. 727-740 Sciences, University of Novi Sad, Serbia (eds) 2020, Kingsland, K 2020, ’Comparative analysis of digital pho- Projects for Digitization of Cultural Heritage of the Re- togrammetry software for cultural heritage’, Digital public of Serbia, Ministry of Culture and Information Applications in Archaeology and Cultural Heritage, 18, of the Republic of Serbia p. e00157 Tanskanen, P, Kolev, K, Meier, L, Camposeco, F, Saurer, Menna, F, Nocerino, E, Remondino, F, Dellepiane, M, O and Pollefeys, M 2013 ’Live metric 3d reconstruc- Callieri, M and Scopigno, R 2016 ’3D DIGITIZATION tion on mobile phones’, Proceedings of the IEEE Inter- OF AN HERITAGE MASTERPIECE - A CRITICAL ANAL- national Conference on Computer Vision, pp. 65-72 YSIS ON QUALITY ASSESSMENT’, The International Wu, C 2011 ’VisualSFM: A visual structure from motion Archives of the Photogrammetry, Remote Sensing and system’, http://www. cs. washington. edu/homes/c- Spatial Information Sciences, Prague, pp. 675-683 cwu/vsfm Nocerino, E, Lago, F, Morabito, D, Remondino, F, Porzi, [1] https://www.cs.cmu.edu/฀reconstruction/colmap.h L, Poiesi, F, Rota Bulo, S, Chippendale, P, Locher, tml A, Havlena, M, Van Gool, M, Eder, M, Fötschl, [2] https://openmvg.readthedocs.io/en/latest/software A, Hilsmann, A, Kausch, L and Eisert, P 2017 ’A /softwares/ smartphone-based 3D pipeline for the creativity [3] https://micmac.ensg.eu/index.php/Accueil industry - the REPLICATE EU project’, 3D VIRTUAL [4] https://alicevision.org/ RECONSTRUCTION AND VISUALIZATION OF COMPLEX [5] https://www.3dflow.net/3df-zephyr-photogrammet ARCHITECTURES, pp. 535-541 ry-software/ Nocerino, E, Menna, F and Remondino, F 2014 ’Accu- [6] https://www.regard3d.org/ racy of typical photogrammetric networks in cul- [7] http://ccwu.me/vsfm/ tural heritage 3D modeling projects’, International [8] https://www.gcc.tu-darmstadt.de/home/proj/mve/ Archives of the Photogrammetry, Remote Sensing & [9] https://www.agisoft.com/ Spatial Information Sciences, Riva del Garda, pp. 465- [10] http://www.muzejvojvodine.org.rs/images/Rimsko 472 %20nasledje%20u%203D/index.html Obradović, M, Vasiljević, I, Đurić, I, KIćanović, J, Sto- jaković, V and Obradović, R 2020, ’Virtual Real- ity Models Based on Photogrammetric Surveys—A Case Study of the Iconostasis of the Serbian Ortho- dox Cathedral Church of Saint Nicholas in Sremski Karlovci (Serbia)’, Applied Sciences, 10(8), p. 2743 Remondino, F and El-Hakim, S 2006, ’Image-based 3D modelling: A review’, The Photogrammetric Record,

252 | eCAADe 39 - Digital heritage - Volume 2 Open-ArcH An open knowledge-based system for architectural heritage representation

Davide Simeone1, Stefano Cursi2, Antonio Fioravanti3, Ugo Maria Coraglia4 1,2,4AGENCY - Agents in Digital AEC 3Sapienza University of Rome 1,2,4{d.simeone|s.cursi|um.coraglia}@aec-agency.com 3antonio.fioravanti@uniroma1. it

Although HBIM is nowadays a common practice in processes and activities aimed at the investigation, documentation and conservation of architectural heritage, some doubts are emerging regarding its capabilities of fulfilling architectural heritage representation and semantics requirements. In this context, this work presents Open-ArcH, a digital approach for architectural heritage representation that relies on the integration of HBIM methodology with a knowledge base developed by means of graph databases, with two major objectives: 1) it supports the formalization of the complex semantics that it is necessary for a full comprehension of the artefact, currently poorly managed by HBIM application, and 2) the openness and flexibility ensured by the graph database technology allow for its adoption in heritage investigation and conservation processes by the different actors involved, providing them with the capability of collaborating to the construction of the knowledge base of the artefact.

Keywords: BIM, Architectural heritage, graph databases, knowledge-based system

INTRODUCTION sis, historical documentation and so on. Slowly but Building Information Modeling is nowadays a com- constantly, the digital approach to built heritage has mon practice in processes and activities aimed at been diverging from a simplified BIM representation the investigation, documentation, and conservation schema - initially embedded in the HBIM acronym of architectural heritage. Although BIM was orig- (Murphy et al., 2009) - towards a multitude of specific inally introduced in the architectural heritage field approaches, often integrating different technologies as a mere transposition of an element-based mod- and methodologies already adopted in the heritage eling schema, its process-based nature progressively field, within the attempt of fulfilling all the repre- has emerged, embracing the complexity of this field sentation needs that emerge during heritage investi- and the representation and informative needs of the gation, documentation, and conservation activities. different disciplines involved such as decay analy- In the last years, different ontologies and dataset

Digital heritage - Volume 2 - eCAADe 39 | 253 schemas have been developed or adapted to drive ing the knowledge base by directly intervening in the and knowledge formalization within BIM processes, knowledge networks without the necessity of trying favouring structuring knowledge for its reuse and to fit - and sometimes force - the information into not computability. Nevertheless, there are some limits suitable data schemas. This is not a negation of the regarding their actual applicability to built heritage ontology-based approach but rather a way to make processes, motivated by the difficulty of managing, them effective in the architectural heritage represen- in the BIM representation, the unicity of the artefact, tation: where applicable, information ontologies can of its history and configuration, as well as the neces- be used to drive the formalization of entities and con- sity to consider the interpretation of the heritage ex- cepts in the project while the flexibility and openness perts for its correct documentation. As a result, often of the graph databases approach ensure the capabil- data, information and knowledge gained during the ity of including knowledge areas not covered by cur- architecture investigation are forced into inefficient rent ontologies or to customize, where necessary, the information schema (i.e. vaults defined as “floors” representation following the progressive emergence in BIM authoring software) generating inconsisten- of the informative model during the heritage activi- cies and reducing the advantages of the adoption of ties.To assess the efficacy and the capabilities of the BIM approach within the built heritage field, also to Open-ArcH approach to architectural heritage repre- the eyes of cultural heritage specialists.In this con- sentation in BIM processes, this paper shows in the text, our work investigates the adoption of a more following sections some experiments performed dur- flexible approach to built heritage knowledge for- ing the investigation of the Church of Saints Abbon- malization - defined Open-ArcH - that relies on the dio and Abbondanzio located in Rignano Flaminio integration of BIM with graph databases techniques, (Italy); a building of Christian worship built on the re- aiming at tailoring the knowledge base to the speci- mains of a cell belonging to a pagan temple of the ficity and uniqueness of the architectural heritage Roman era. artefact, its evolution during the time and the in- formative needs of the different specialists involved. LITERATURE REVIEW The key principle of the Open-ArcH approach is the In literature, there is vast research on the topic of use of graph databases to formalize and manage all knowledge-based data enrichment through seman- the heritage semantics in the BIM process and inte- tic web technologies in the AEC field (Pauwels et al., grating the resulting knowledge base with the BIM 2017). But only some approaches have focused on model of the artefact, aiming at providing a tridi- the architectural heritage representation. De Luca mensional, semantically enriched representation of et al. (2011) proposed the development of an infor- the building. This represents a shift from the HBIM mation system at the scale of architecture, consid- paradigm and its initial application of a set of pre- ering the relationships that can be established be- defined, hierarchy-organized classes/families -often tween the representation of buildings (shape, dimen- not even conceived for architectural heritage- to a sion, state of conservation, hypothetical restitution) project-based definition of classes, attributes and re- and heterogeneous information from various fields lationships specifically conceived to ensure a coher- (such as the technical, the documentary or still the ent and fully comprehensive representation of all we historical one), while another domain-specific ontol- need to know regarding the building, to effectively ogy deal with cultural heritage alteration documen- support the design and conservation decisions. From tation, its diagnosis, and the intervention for possi- the perspective of the heritage process, Open-ArcH ble restorations (Cacciotti et al., 2015). To extend is essentially a bottom-up approach that allows the the application of the CIDOC CRM template (Croft et different involved specialists to contribute to shap- al.,2010) - the main ontological reference in a cultural

254 | eCAADe 39 - Digital heritage - Volume 2 heritage context - the Architecture Metadata Ob- promote the research, teaching, and dissemination ject Schema (ARMOS) defines a conceptual model for of UNESCO World Cultural Heritage site (Wang et al., the cataloguing of architectural heritage, focusing, in 2020). particular, on the formal aspects of architectural de- sign (Agathos and Kapidakis, 2013). In terms of the LIMITS OF CURRENT HBIM APPLICATION investigation process, Mecca et al. (2006) proposed TO ARCHITECTURAL HERITAGE a very specific ontology for a diagnostics workflow The recent application of the Building Information regarding earthen architecture to formalize various Modelling (BIM) process to architectural heritage, collections of information guidelines. To manage the highlights several issues (Volk et al., 2014), related information used in the process of conservation and both to the modeling principles and the adherence restoration of historical monuments, Messaoudi et al. to heritage process requirements. The most obvi- (2018) work relies on an ontological model for record- ous limitation is found in the difficulties encountered ing and integrating multidisciplinary observations of in modeling the specificity of a historic building by the conservation state into structural data spatialized using proprietary BIM software, which often relies within a semantic-aware 3D representation, integrat- on the parametric modeling of the objects made ing information about material and alteration phe- available on the standardization that characterizes nomena into a unique spatial model.Furthermore, building components designed for new construction some research focused on the integration of knowl- (Construction Specifications Institute, 2019). This edge base regarding architectural heritage, created limitation is also visible in the relationships estab- through information ontologies, with a BIM environ- lished between the components that, intending to ment (Simeone et al., 2019; Beetz et al., 2016; Mai- simplify the work done by users, tend to impose con- etti et al., 2018; Pauwels et al., 2013; Quattrini et al., straints that reflect design principles and construc- 2017). Recently Yang et al. (2019) propose a semi- tion rules that are generally always valid for new automatic conversion mesh-to-BIM through Dynamo construction but that constitute strong limitations in and translation of the model in IFC owl, while Bon- the documentation of historic buildings (for exam- duel et al. (2019) investigated the applicability of Re- ple, the horizontality of floors, the precision of ver- source Description Framework (RDF) literals in a her- tical surfaces or their constant thickness).These de- itage context built to include a wide variety of exist- sign standards turn out to be unsuitable for the sit- ing geometric schemas. Moreover, Borin et al. (2020) uations found in the digital restitution of a histori- develop a model for the visualization of the Harris di- cal building thus limiting the potential use of such agram (a methodology used for dating walls’ layers) models in this field (Brusaporci and Maiezza, 2016). through the use of a BIM-based ontology. Alternative To deal with the historical and morphological com- to relational DBs and ontologies are graph databases, plexity of existing buildings, current HBIM processes whose recent diffusion is mainly due to their proven make extensive use of tools for the management of efficiency in terms of data indexing, querying, filter- point clouds, obtained by laser scanner survey, us- ing capabilities and flexibility in adding new kinds of ing the latter as a reference for the modeling of para- relationships or adapting a data model to new her- metric components in BIM environments (Centofanti itage digitalized process requirements. In the cultural et al., 2016). However, there is not yet an integra- heritage field, graph databases have been applied tion between the extremely accurate representation to represent the CIDOC CRM template (Bruschke and of geometries, which is obtained through point cloud Wacker, 2014), while, emerging research employs processing, and the parametric modeling tools made knowledge graph approach to realize the represen- available by proprietary BIM software, since the au- tation and association of information resources, and tomatic acquisition of these data is not yet very ef-

Digital heritage - Volume 2 - eCAADe 39 | 255 fective; therefore, these resources, at the information edge that characterizes the processes regarding the level, remain essentially independent. A further diffi- built heritage (Saygi and Remondino, 2013). culty is found precisely in the modeling approach: in the case of a new design, the need for accurate and OPEN-ARCH: GRAPH DATABASES FOR complete knowledge of what to model goes hand in KNOWLEDGE FORMALIZATION IN THE hand with the information consistency imposed by BIM environments, providing valid support for con- FIELD OF ARCHITECTURAL HERITAGE struction and management of a building; in the case Research methodology and Open-ArcH of processes that characterize the activities of investi- modeling principles gation and restitution of an existing asset, the knowl- To overcome these obstacles, the Open-ArcH ap- edge we have of the latter, in addition to coming proach relies on the integration of a BIM environment from extremely heterogeneous sources, is subject to with an artefact-specific knowledge base aimed at continuous changes, reinterpretations, inconsisten- fully covering the information elaborated in a her- cies and gaps that may remain until the end of the itage process and necessary for its documentation process and beyond (Simeone et al., 2014). These and for aware decisions related to its conservation. effectively opposing approaches make current HBIM At this stage, the proposed integration within the processes extremely time-consuming. Considerable Open-Arch scope has been applied to the representa- efforts are required, either due to the lack of fam- tion of some key heritage knowledge domains such ilies specifically designed for heritage components as the historic documentation of the artefact (meant with the consequent need to use and adapt families as a whole), the evolution of its configuration dur- conceived for new construction or to have to model ing the time, the analysis of degradation and dam- them ad hoc from scratch for existing components, ages, and the definition of potential interventions for or to obtain the right compromise between the accu- the preservation of the artefact and its components. racy of geometric and informative data and the sim- Within the Open-ArcH approach, the formalization of plification due to the use of parametric tools made knowledge related to these disciplines has been inte- available by the software (Antonopoulou and Bryan, grated with a component-based representation pro- 2017). Another aspect that characterizes the pro- vided by the BIM environment and necessary to con- cesses concerning the Built Heritage, difficult to solve trol geometrical and technological features of the ar- through the current HBIM modeling techniques, con- chitectural heritage artefact. In this process of inte- cerns the representation of changes, of different na- gration of a BIM model with a graph database, a deli- ture, undergone over time by the building, also about cate aspect to be carefully considered is related to the the present and future interventions, or the absence definition of the right location to formalize the infor- of ancient building elements yet interpreted through mation in, as well as which information can be used historical documents (Cursi et al., 2015).Although to link the BIM area with the knowledge base to en- much research is focusing on a solution to the prob- sure interoperability between the two systems and to lems discussed above, the aspect that makes par- limit the duplication of data. Therefore, in the Open- tial the effective applicability of BIM on the exist- ArcH system, the BIM environment is dedicated to: ing, limiting the semantic spectrum representable 1. Representation of the artefact configuration, in in this type of environment, lies in the fundamen- terms of its current state and the previous con- tal principles of BIM, namely the excessive focus on figurations during history (where applicable); the technological components of the artefact that 2. Integration with other representation and visu- leaves aside other aspects that concern the represen- alization approaches used in the heritage pro- tation of indirect and highly heterogeneous knowl- cesses;

256 | eCAADe 39 - Digital heritage - Volume 2 3. Formalization, through objects attributes, of the crete), to represent the information necessary for its information from different domains that can be documentation and, at the same time, provide each attached to single artefact components and that information piece with its necessary interpretation can be useful to have formalized in a tridimen- context. The first advantage in this approach is re- sional environment. lated to the possibility of assigning different labels to a node representing a component of the build- While the graph database area is dedicated to: ing, ensuring its right definition without being lim- 1. Formalization the intangible knowledge of the ited by the families defined in the BIM environment, artefact; usually forced by the representation hierarchy of the 2. Control of relationships among artefact compo- authoring software. At the same time, the graph nents, concepts, and other entities database technique allows to assign multiple labels 3. Enrichment of elements’ semantics in terms of to a node/object, and this is particularly useful in ar- definitions, attributes, complex set of relation- chitectural heritage processes where often construc- ships tion elements assume different roles - and as a con- 4. Making computable the entire knowledge re- sequence refer to different classes - in the evolution lated to the artefact, providing a homogenous of the artefact during the time. By managing the methodology for its formalization architectural heritage semantic within a knowledge 5. Provide a highly flexible environment for knowl- base implemented using graph databases, it is pos- edge modeling that can allow different special- sible to homogeneously formalize and integrate all ists to include their interpretations and assump- the data and information related to the artefact and tions in the heritage documentation. produced by the different specialists during investi- gation and conservation activities. This represents an As shown in figure 1, the dialogue between these two important advantage in the heritage field since it al- environments is ensured by the transposition, in the lows to efficiently manage an incremental represen- graph database, of the set of instances that compose tation of the artefact and make it available in an open the building model, using a unique identifier that en- way to current and future scholars, ensuring coher- sures the correspondence one-to-one and that can ence of the documentation and access to the most be used as a reference for the development of ad hoc updated knowledge of the artefact. Knowledge frag- solutions to transfer data from one environment to mentation is one of the major issues of current in- the other. Although the globally unique identifier - formation management in the architectural heritage the GUID (ISO/IEC 11578:1996), even in its ifc stan- field, that often hinders heritage specialists’ efforts in dardization - can technically play this role, in our im- re-collecting data to be used as bases for further in- plementation we adopted a codification parameter, vestigations and studies, especially in terms of intan- defined as OAH-key, customizable in its structure (i.e. gible knowledge. in the form xxx-yyy-zzz where xxx is a discipline code, yyy is a family code and zzz is a progressive num- The knowledge base development: balanc- ber for the element) and that can be more accessible ing specialist’s experience and discipline even in proprietary software, machine-readable and ontologies understandable by the user. The graph database data model relies on four main The replication of the HBIM entities within the primitives: 1) labels (meant as classes, i.e. BIM fami- knowledge base, formalized in terms of nodes, em- lies), 2) nodes representing entities (abstract or con- beds the artefact and its components in the graph crete), 3) relationships that define links between en- database, allowing the domain specialists to link tities and 4) properties-attributes (name-value pairs them with other concepts, entities (abstract and con-

Digital heritage - Volume 2 - eCAADe 39 | 257 that are used to add qualities to nodes and relation- the availability of ontologies definition by integrat- ships). Using this structure, the different specialists ing them with the concepts and information that, are capable of building their knowledge network, although not manageable using the strict applica- defining their labels, identifying concepts and enti- tion of existing ontologies, are necessary to com- ties, and creating different relationships with the in- plete documentation of the artefact. By integrating tent of describing comprehensively and unambigu- graph databases and Building Information Modeling, ously the results of their investigation on the build- it is possible to enrich the semantics of the repre- ing. As per the multidisciplinary feature of architec- sentation, including for instance the relationships be- tural heritage processes, often involving many spe- tween the different components of the artefact in its cialists with their knowledge, experiences, the pro- evolving configurations (see figure 2). This is a rele- cess of development of the knowledge base is re- vant improvement in the representation since it al- quired to be collaborative - with the different spe- lows for the possibility to include relationships such cialists operating in the same knowledge graph - and as adjacency, assembly-of, part-of, made-by that can- tailored on the uniqueness and specificity of the ar- not be formalized through the BIM representation chitectural heritage artefact, of its history and its fea- schema, essentially based on the is-a relationships tures. In this process, the Open-ArcH approach pro- between families and instances. vides a formalization environment where each spe- In the light of these reflections, the modeling cialist can model information both relying on refer- of the Church of Saints Abbondio and Abbondanzio ence ontologies (such as the ifc-OWL, or the CIDOC- aimed to develop a geometric and informative HBIM CRM), develop each own data model or used a hy- model (using proprietary BIM software Autodesk Re- brid approach that exploits the standardization and vit) consistent with the technical-constructive recon-

Figure 1 The Open-ArcH framework, based on the integration between an HBIM environment and graph databases for full documentation of the building, its historic documentation, and the conservation activities.

258 | eCAADe 39 - Digital heritage - Volume 2 Figure 2 The adoption of the graph databases, within the Open-ArcH system, to enrich BIM semantics by formalizing artefact configuration, historical documentation, construction knowledge.

struction derived from the analysis of the building the main wall system and of the transformations un- and formalized in the knowledge graphs. The com- dergone over time, as well as of the building tech- plexity of the geometries of most of the construc- niques and the masonry typologies (Carbonara, G., tive and technical elements of the building has nec- 1994). Therefore, considering the traditional work- essarily implied a compromise between the simplifi- flow of the constructive analysis, stratigraphic anal- cation of the geometric representation, adapting the ysis and a wall study were carried out first, provid- degree of adherence to reality to a level sufficient ing information related to all the actions that con- for this research, and the informative and paramet- tributed to the current configuration of the wall; fol- ric value of the objects used. This is because, unlike lowed by the identification of the decay analysis and AEC where geometry is strictly generated through then the rehabilitation interventions. As shown in fig- rigid parameters and allowed operations embedded ure 3, the Open-ArcH approach can comprehensively in the family structure, architectural heritage requires formalize knowledge related to the decay analysis a more flexible system for geometry representation and intervention providing both geometrical repre- that allows diversity in the input system, partially de- sentation and non-geometrical information in an in- taching geometry from family parameters.The Open- tegrated and computable way. ArcH approach is conceived also to support the con- From a software perspective, the Open-ArcH im- servation activities that are performed on the arte- plementation relies on the integration of different fact to ensure its persistency in the future. This is platforms and apps, relying on the use of Autodesk usually managed by attaching to the BIM model en- Revit as main BIM environment and the Neo4J graph tities/nodes some additional knowledge items that database editor (table 1). formalize the typologies of decay discovered during the artefact analysis and including in the knowledge CONCLUSIONS base the interventions required by those decay ty- In this paper we presented Open-ArcH, a digital ap- pologies. As an example, in the case of the Church proach for architectural heritage representation ori- of Saints Abbondio and Abbondanzio, the studies ented at fulfilling the representation requirements and investigations on the historical phases of the overcoming the difficulties of the HBIM approach in building allowed the most accurate identification of

Digital heritage - Volume 2 - eCAADe 39 | 259 Figure 3 Representation of architectural decay and conservation interventions in the Open-ArcH approach: BIM models of decay (top-left), intervention mapping in technical drawings from BIM (bottom-left) and process knowledge formalization in terms of nodes formalizing walls, decay surface areas, decay typologies and interventions comprehensively formalize the vast knowledge nec- viding two major advantages: on one side, it sup- (right). essary to fully document an architectural artefact. ports the formalization of the complex semantics The proposed framework, that connects a HBIM en- that it is necessary for a full comprehension of the vironment with graph databases, shown great capa- artefact, currently poorly managed by HBIM applica- bilities for understanding and accessing complex and tion; on the other, the openness and flexibility en- rich datasets in many different knowledge domains sured by the graph database technology allow for integrating them with building entities, thus signif- its adoption in heritage investigation and conserva- icantly facilitating the efforts of semantically enrich tion processes by the different actors involved, pro- the HBIM representation. If compared to current viding them with the capability of collaborating to HBIM applications, the Open-ArcH approach is pro- the construction of the knowledge base of the arte-

Table 1 Platforms and apps integrated into the Open-ArcH implementation

260 | eCAADe 39 - Digital heritage - Volume 2 fact. As future developments, the research will focus (eds) 2020, Graphical Heritage, Springer Series in De- on the possible implementation of the proposed ap- sign and Innovation, Springer International Publish- proach through desktop and web applications and ing, pp. 757-770 Brusaporci, S and Maiezza, P 2016 ’e-loading bim: Be- web resources, favouring its adoption during the per- tween spatial and database information modeling forming of activities on the field. Additional efforts for architectural heritage documentation’, XIII Con- are also needed in progressively integrating exist- greso Internacional de Expresión Gráfica aplicada a la ing ontologies in a reference knowledge base, to en- Edificación, APEGA 2016, pp. 835-847 sure some standardization and common terminolo- Bruschke, J and Wacker, M 2014 ’Application of a Graph gies in the knowledge bases of different conserva- Database and Graphical User Interface for the CIDOC CRM’, CIDOC 2014 Annual Conference tion projects. In our opinion, this further step could Cacciotti, R, Blaško, M and Valach, J 2015, ’A diagnos- be very beneficial, since it will allow for potential tic ontological model for damages to historical con- connections among multiple artefacts’ knowledge structions’, Journal of Architectural Heritage, 1, pp. bases, providing a new dimension to the potentials 40-48 and the possibilities of open access to architectural Carbonara, G 1996, Trattato di restauro architettonico, heritage data. UTET Centofanti, M, Brusaporci, S and Maiezza, P 2016 ’Tra “Historical BIM” ed “Heritage BIM”: Building Informa- REFERENCES tion Modeling per la documentazione dei beni ar- ISO/IEC 11578, : 1996, Information technology – Open chitettonici’, ReUso 2016, Pavia Systems Interconnection – Remote Procedure Call Crofts, N, Doerr, M, Gill, T, Stead, S and Stiff, M 2010, (RPC)specification, ISO Definition of the CIDOC Con-ceptual Reference Model, Agathos, M and Kapidakis, S 2013, ’A Meta - model ICOM/CIDOC Documentation Standards Group, eCI- Agreement for Architectural Heritage’, in Garoufal- DOC CRM Special Interest Group lou, E and Greenberg, J (eds) 2013, Metadata and Se- Cursi, S, Simeone, D and Toldo, I 2015, ’A Semantic mantics Research, Communications in Computer and Web Approach for Built Heritage Representation’, in Information Science, Springer International Publish- Celani, G, Sperling, D and Franco, J (eds) 2015, CAAD ing, p. 384–395 Futures 2015: Computer-Aided Architectural Design Antonopoulou, S and Bryan, P 2017, BIM for Heritage: De- Futures. The Next City - New Technologies and the Fu- veloping a Historic Building Information Model, His- ture of the Built Environment, Springer International toric England, London Publishing, Berlin, Heidelberg Beetz, J, Blümel, I, Dietze, S, Fetahui, B, Gadiraju, U, Construction Specifications Institute, CSI 2019, Omni- Hecher, M, Krijnen, T, Lindlar, M, Tamke, M, Wessel, Class® Introduction and User’s Guide, CSI M and Yu, R 2016, ’Enrichment and Preservation of De Luca, L, Busayarat, C, Stefani, C, Véron, P and Flo- Architectural Knowledge’,in Münster, S, Pfarr-Harfst, renzano, M 2011, ’A semantic-based platform for M, Kuroczyński, P and Ioannides, M (eds) 2016, D Re- the digital analysis of architectural heritage’, Com- search Challenges in Cultural Heritage II, Lecture Notes put. Graph., Virtual Reality in Brazil, 35, p. 227–241 in Computer Science, Springer International Publish- Maietti, F, Medici, M, Ferrari, F, Ziri, AE and Bonsma, P ing, pp. 231-255 2018, ’Digital Cultural Heritage: Semantic Enrich- Bonduel, M, Wagner, A, Pauwels, P, Vergauwen, M ment and Modelling in BIM Environment’, in Ioan- and Klein, R 2019 ’ncluding widespread geome- nides, M (eds) 2018, Digital Cultural Heritage, Lecture try schemas into Linked Data-based BIM applied to Notes in Computer Science, Springer International built heritage’, Proc. Inst. Civ. Eng. - Smart Infrastruct. Publishing, p. 104–118 Constr., pp. 34-51 Mecca, S, Masera, M and Cirinnà, C 2006 ’Knowl- Borin, P, Bernardello, RA and Grigoletto, A 2020, ’Con- edge management approach for conservation of necting Historical Information with BIM Ontologies. Earthen Architecture’, Conference Proceedings of HBIM Methods for the Visualization of Harris Ma- Joint CIBW65/W55/W86 Symposium, Rome trix for the Torrione in Carpi’, in Agustín-Hernández, Messaoudi, T, Véron, P, Halin, G and De Luca, L 2018, ’An L, Vallespín Muniesa, A and Fernández-Morales, A ontological model for the reality-based 3D annota- tion of heritage building conservation state’, Journal

Digital heritage - Volume 2 - eCAADe 39 | 261 of Architectural Heritage, 29, pp. 100-112 Murphy, M, McGovern, E and Pavia, S 2009, ’Historic building information modelling (HBIM)’, Struct. Surv, 27, p. 311–327 Pauwels, P, Bod, R, Di Mascio, D and De Meyer, R 2013 ’Integrating building information modelling and se- mantic web technologies for the management of built heritage information’, 2013 Digital Heritage In- ternational Congress (DigitalHeritage), Marseille, pp. 481-488 Quattrini, R, Pierdicca, R and Morbidoni, C 2017, ’Knowledge-based data enrichment for HBIM: Ex- ploring high-quality models using the semantic- web’, JournalofArchitecturalHeritage, 28, p. 129–139 Saygi, G and Remondino, F 2013, ’Management of Archi- tectural Heritage Information in BIM and GIS: State- of-the-Art and Future Perspectives’, Int. J. Herit. Digit. Era, 2, p. 695–714 Simeone, D, Cursi, S and Acierno, M 2019, ’BIM semantic- enrichment for built heritage representation’, Au- tomation in Construction, 97, pp. 122-137 Simeone, D, Cursi, S, Toldo, I and Carrara, G 2014 ’B(H)IM - built heritage information modelling’, Proceedings of the 32nd eCAADe Conference, Delft, the Netherlands, pp. 613-622 Volk, R, Stengel, J and Schultmann, F 2014, ’Building In- formation Modeling (BIM) for existing buildings — Literature review and future needs’, Automation in Construction, 38, pp. 109-127 Wang, X, Chang, W and Tan, X 2020, ’Representing and Linking Dunhuang Cultural Heritage Informa- tion Resources Using Knowledge Graph’, Knowl. Or- gan., 47, pp. 604-615 Werbrouck, J, Pauwels, P, Bonduel, M, Beetz, J and Bek- ers, W 2020, ’Scan-to-graph: Semantic enrichment of existing building geometry’, Automation in Con- struction, 119, p. 103286 Yang, X, Lu, YC, Murtiyoso, A, Koehl, M and Grussen- meyer, P 2019, ’HBIM Modeling from the Surface Mesh and Its Extended Capability of Knowledge Representation’, ISPRS Int. J. Geo-Inf, 8, pp. 301-312

262 | eCAADe 39 - Digital heritage - Volume 2 Enhancing User Experience through Interaction Design

Rethinking the municipal agora of Chania through AR narratives

Panagiota Dania1, Helena G. Theodoropoulou2, Anna Karagianni3, Vasiliki Geropanta4, Panagiotis Parthenios5 1,3,4,5School of Architecture, Technical University of Crete 2Athena-Research and Innovation Center in Information, Communication, and Knowledge Technologies [email protected] [email protected] 3akaragianni1@isc. tuc.gr 4,5{vgeropanta|parthenios}@arch.tuc.gr

This paper examines the relationship between designing interactive experiences based on new technologies and the process of architectural narration. It highlights the idea of rethinking a building with historical, architectural and functional value, as an experience and a journey. Referring to the historic building of the Municipal Market of Chania, Greece and using the conceptual idea of designing through narratives, it delineates the process of integrating new technologies into the process of designing a spatial and temporal experience. Exploiting Augmented Reality, we design an application implementing a digital layer with architectural and historical content, that is integrated into reality, improving the on-site visit, providing enhanced understanding of the building and introducing experiential visitor-building interaction. The application is available through mobile devices and the proposed system is evaluated by a group of users showing the positive effects of the use of interactive technologies in redesigning the experience of a space.

Keywords: Mobile AR, cultural dissemination, architectural narration, interactive spatial experiences, interactive visualization

INTRODUCTION: AGORA AND ISSUES use of new technologies. This work presents the RAISED study of designing and developing a new digital layer User experience in the built environment is increas- (effect on physical space) by designing through nar- ingly discussed, observed and visualized in recent ratives (Lehman, 2018; Achten, 2014), implemented scholarly articles, studies and experiments. In most with Augmented Reality technology, aimed at en- cases, the goal is to acquire a holistic picture of how hancing user experience in the Municipal Agora of users interact with the built space. However, there Chania, commonly known as the “Agora”. The Agora, are few examples that aim to enhance user perspec- is a historic building, located at the in-between zone tive and offer an augmented experience through the between the old and new city of Chania. Being a sig-

Digital heritage - Volume 2 - eCAADe 39 | 263 nificant point of interest for tourists and visitors, it is the experience of the users can grow and evolve. Cul- used as a central market on a daily basis, thus creating tural Heritage (CH) dissemination has been enhanced a commercial hub, fully integrated into the everyday for decades with Information and Communication life of the citizens. Previous research has shown that Technology (ICT) tools. Experts manage to preserve visitors of the Agora experience lack of orientation and/or recreate and make information about histor- and absence of comprehension regarding the his- ical artifacts, buildings and sites accessible, exploit- torical and architectural value of the building (Kara- ing techniques and tools for recording, documenting gianni, 2019). Based on the experience mapping of and visualizing content (Ruffino et al., 2019). In the the existing building setup, the weakest points of field of Architectural Heritage (AH), ICT tools are used the overall experience are being identified and a new to complete and highlight spatial and morphologi- perceptual route is designed by connecting interac- cal elements, reconstructing the form and nature of tive setups at nodal points of the site. The aim is to ex- damaged or illegible architectural features, present- perientially highlight the dual role of the Agora and ing the current state and approaching the artifacts’ its urban and monumental value. original state (Parthenios et al., 2014). CH profession- als are the presenters, and people are the observers RELATED RESEARCH - TECHNOLOGY in the effort to preserve and disseminate the value In the field of Architectural Heritage, buildings are of a historic structure. Ιn recent years, we have ex- usually treated entirely as monuments, thus the his- perienced the emergence of Extended Reality (XR), torical and / or architectural value of the space re- which refers to Virtual Reality (VR), Augmented Real- mains easily distinguishable. However, in historic ity(AR), and Mixed Reality(MR), along with real time sites like the Municipal Agora of Chania that are still interactive technologies (Fast-Berglund et al., 2018). used by the public on a daily basis, the duality of Exploiting these technologies, experts have at their their character may become blurred and it is diffi- disposal tools that can turn their audience from an cult to recognize particular historical and architec- observer, into an actor and spatial user. Kiourt et tural features. Often, such buildings are dominated al. (2020) contend that XR technologies have the by their functional character and the promotion of ability to reveal hidden, unknown or illegible ele- their cultural and structural value must be revived ments and stories, thus, attracting the visitor’s atten- using targeted techniques. In order to understand tion, offering him/her the satisfaction of a deeper the need to preserve and highlight elements of the understanding of the site. Augmented Reality (AR) history and architecture of buildings with dual char- refers to the actual Reality enhanced by computer- acter, it is important to identify the spatial, histori- generated content (Azuma et al., 2001) and is ideal for cal and morphological qualities and rethink the site. sites with architectural-historical value that need to This activity supports that the visitor’s experience is be highlighted or interpreted experientially on-site, redesigned based on them, enabling her/him to un- enabling the visitor to visualize virtual information derstand the current state of the building as well as its (spatial, historical, morphological, structural) within initial form. Lehman introduces the idea of rethinking the real space, resulting in an in-depth understand- a space by creating a narrative with the help of tech- ing of it. In addition, given the low cost and increas- nology (Lehman, 2018). The work proposes a new ing use of mobile devices, the application of Aug- typology for a public library in which the ameliora- mented Reality through mobile applications-Mobile tion of the experience comes through a new narra- AR (MAR) (Angelopoulou et al., 2011) is an easily ac- tive. In this case, using technology to develop a de- cessible state-of-the-art solution to enhance the on- signed digital layer into the concept of a library as it site experience of visitors to an area of interest. To- is known until today, creates a fertile ground in which day, several studies focus on the use of MAR to en-

264 | eCAADe 39 - Digital heritage - Volume 2 hance the visitor’s experience in places of interest, scale, the symmetry of the building and the absence thus taking advantage of its potential. Panou et al. of information, they commented on the inability to (2018) introduce a MAR application to demonstrate orientate themselves. The users failed to recognize sites with historic value located in the old town of not only the layout but also the duality of the place Chania, Crete, Greece. Angelopoulou et al. (2017) because some of them were attracted to the mar- proposed a multi-user MAR educational tool for the kets selling tourist products and thus restricted their archaeological site of Sutton Hoo (UK). KnossosAR visits to only certain parts of the building. As such, (Galatis et al., 2016) is a MAR educational tool for sup- they did not notice the monumental aspect of the porting the experience of the archaeological site of Agora, and they concentrated only on the center with Knossos (in Crete, Greece). Furthermore, AR can be- its information kiosk. However, the majority of users come the means to better understand and evaluate who had the opportunity to browse the place virtu- an architectural idea for the purpose of intervening ally before visiting gained a better understanding of or redesigning/reusing an existing space, improving the dual role of the Agora, and its architectural form. current practices of visualizing the design and con- Many users commented that large parts of the Agora ceptual process, and creating a realistic spatial rela- need to be improved in order to be more attrac- tionship between the virtual version of the architec- tive and to ensure an understanding of its dual role. tural idea and the physical space.The present study As a result of all these observations, the aim arose focuses on exploiting MAR technology to design a to tackle most of the flaws in order to offer a great digital spatial, chronological and architectural narra- number of interesting stimuli for the users through tive to enable the visitor to understand the duality of which they could enhance their experience and their the character of the building of the Agora through knowledge. The technology at our disposal gave us an on-site experience. The purpose of this study is to the opportunity to design a narrative through a dig- propose a solution through AR technology for high- ital layer and offer users a more constructive experi- lighting qualities that are difficult to extract from the ence. The research implemented mobile-AR so that history and architecture of the building, thus assem- the application would be easily accessible to the visi- bling a complete narrative image of the present, past tor with a mobile device (phone or tablet, Android or and future of its existence in time and space. iOS), with a focus on exploiting marker-based AR and markerless AR (Bekele et al, 2018) (Kiourt et al., 2020). METHODOLOGY Specifically, the marker-based technique is a method This experiment is part of an ongoing research on the of recognition of a 2D image (image target), such experience of the Agora. The first part (Karagianni et as an image, a QR code, colored or monochrome, al., 2019) examines the relationships between inter- placed in the physical world acting as a marker that active design strategies and the process of architec- can be recognized by a camera. When the cam- tural design, through an IPS location tracking exper- era detects the marker, the predefined virtual con- iment and a survey on the users’ experiences of the tent is displayed on the top of the marker. Marker- building. As such, a number of issues emerged from based AR has the advantage that it can support user this research. The general experience was rated as interaction with virtual content by detecting hand positive but the users pointed out some shortcom- gestures over a specific area of the marker which ings of their visits. Some users found the place quite acts as a virtual trigger (Marker-based Virtual Trigger- uncomfortable due to the crowds, the odors in the ing)(Theodoropoulou et al., 2020). When the camera markets and the bad ventilation of the building. Be- detects the interruption of the camera’s visual con- cause of these factors, they intentionally avoided vis- tact with the virtual trigger (like a hand movement iting the rest of the building. Furthermore, due to the over the virtual trigger), then it can trigger a prede-

Digital heritage - Volume 2 - eCAADe 39 | 265 fined function related to the virtual content. Mark- and visitability. Thus, aiming to enhance the expe- erless AR is based on the recognition of features and rience of visitors throughout the entire building, re- shapes of the physical object without the need for a inforcing this area is very significant. This point of static image target. The present study chose to utilize interest is the exit that connects the Agora with the both types of Augmented Reality, depending on the old town of Chania and this connection plays a very point of interest of the building, the type of virtual important role in all the concepts of the establish- content and the type of spatial experience involved. ment of the building. It must be stressed that the After the application was developed, it was evalu- whole project of the Agora sought to beautify the city ated by a group of 15 users who were randomly se- of Chania, becoming an emblem of liberation from lected during their visit to the site, in order to obtain the Ottoman yoke. The third point of interest, even feedback from their experiences and to compare this nowadays remains the most visited and gives the feedback with that of the previous survey. Because most complete piece of information about the Agora of the fact that the Agora is a busy place and that in since an information kiosk is available. The project the previous evaluation the crowds and the temper- aims to keep its importance and reinforce this point ature were viewed negatively, month and time was by turning it into a crossroads of spatial and tempo- chosen (February, morning) with mild weather con- ral architectural content. To complete the route, two ditions and a moderate number of visitors. additional points of interest were identified at the ex- its of the second corridor-axes of the Agora following DESIGNING THE ROUTE the cross-shaped structure and, in order to offer the To enhance the users’ experience within the Agora, whole experience of the place to the visitor. (Figure a narrative route is designed to reinforce the percep- 1). tion of the dual role of the Agora and attempt to alle- Figure 1 viate some of the problems noticed in the previous The designed route survey (Karagianni et al., 2019). The design of the of the narrative into narrative was based on the basic form of the build- Agora ing, utilizing the its cross shape . The narrative con- cerns the whole building from the bottom to the top. The implementation of Augmented Reality uses the entire height of the building, applied firstly on the floor at the entrance of the building (markerless AR), as well as to the highest point of it (markerless AR), ANALYSIS OF EACH POINT ARCHITEC- its roof. In addition, the basic historical, morpholog- TURALLY AND TECHNICALLY ical and functional elements that emerge as part of The narrative within the Agora was expressed the narrative scenario of the space were identified. through the design of five distinct interactive points The beginning of this narrative is set at the main en- in the form of a route across the whole building (Ta- trance of the site, where the Agora is being presented ble 1). as an architectural whole. Specifically, information about the spatial, historical, architectural importance Point 1: The main entrance of the site is being presented, through an AR interac- Digital storytelling-narration begins with the experi- tive experience giving the users a first impression of ence of three AR implementations to the main en- the building. The second point was chosen because trance (South) of the Agora. Initially, by focusing on of the low rating it received from the users in the pre- the ground in front of the entrance with the camera vious survey, in terms of experience, understanding of the mobile device, the user can have an overall

266 | eCAADe 39 - Digital heritage - Volume 2 Table 1 Analysis of each point (architecturally & technically)

view of the building and understand the scale and the digital platform, did not visit the northern part of shape of the Agora. Specifically, using the markerless Agora. Thus there was a need to enhance this point technique of plane detection, an accurate 3D model of Agora so that the users could perceive the whole of the building is displayed on the ground (Figure experience of the building and its size. At this point, 2_1b). To highlight and understand the scale of the the aim is to make the users understand the impor- building, a model of a human figure has been placed tance of the position of the Agora in relation to its sur- at the entrance of the 3D model, signaling the exis- rounding urban area. The Agora can be considered tence of the visitor him/herself in the natural space as a strong link between the new and the old city. It at the entrance of the Agora. Secondly, by aiming at must be stressed that while the main entrance of the a facet of the building, the camera detects its shape, building is in the new city, after crossing the build- displaying a video player proportional to the facet’s ing the visitor exits to the old city. Understanding the dimensions and then with a short video playback, a need to demonstrate this connection, marker-based section of the building is highlighted that enables the AR is being implemented. When the camera of the user to acquire a first impression of the inside(Figure mobile device of the visitor detects the specially de- 2_1c). Passing through the main entrance, the visi- signed image, a map of the area emphasizing this link tor finds a specially designed image (image target). of the old with the new city through the Agora is dis- Implementing marker-based AR, when the user aims played. Additionally, the visitor acquires historical in- the camera of the mobile device at the image, she/he formation about the Agora as a monument. (Figure acquires important information about the history of 2_2a) Moreover, at this location of the building, tech- the building, its materiality and architectural design nology also serves as a means of giving the visitor (Figure 2_1a). As such, the visitor becomes familiar more time to stay in a place that initially seemed in- with the dual role of the Agora, as a monument and different. Specifically, at this point the user has the as a market. Virtual content at this point also acts as a ability to experience interaction with the virtual con- guide to the narrative path, highlighting the chosen tent by activating the hand gesture detection over a route and the locations of other points of interest. Virtual Trigger functionality. To implement a Virtual Trigger, the user passes her/his hand over it, and a Point 2: Exit of the Agora 3D model of the Agora appears with additional infor- The idea of the second point emerged from the fact mation about the building as a monument but also that during previous research, most users, especially about its orientation (Figure 2_2b). those who did not acquire any information through

Digital heritage - Volume 2 - eCAADe 39 | 267 different areas of the site and display 3D models with Figure 2 architectural information. By turning the camera to- Point 1 & point 2 wards the roof of the building, the user has the op- through AR portunity to see the structural features of the roof in application its original form (Figure 3_3a). Additionally, focusing the camera towards the wall, the user can see a 3D representation of the current state of the building, re- vealing illegible morphological features such as the symmetry of the openings of the facades. Finally, by pointing the camera at the opposite wall, a three- dimensional model of a redesigning proposal of the interior of the building is displayed. Figure 3_3b, de- picts the display of the architectural proposal on top of the real environment and highlights the initial con- ceptual approach to focus on the illegible morpho- logical symmetry of the structure of the Agora, as one of its major architectural elements (Figure 3_3b).

Point 4 and 5: East and west ends of the Point 3: The center of the Agora Agora The center of the building is selected as one of the The points of interest 4 and 5 of the spatial narra- most important points of the route. Currently, an in- tive are placed in the second corridor of the Agora formation kiosk constitutes a pole of attraction of- (East-west). An important issue of the previous sur- fering visitors the opportunity to learn more on the vey (2019) was the difficulty of orienting users not building’s history. Furthermore, from this location only inside but also outside the Agora. Thus, it was the users can have a panoramic view of the building, observed that visitors were forced to return to the but users lack spatial perception and orientation. The main entrance of the building to continue their tour perception of the building´s large scale is overshad- of the city, as they did not have enough information owed by the narrow corridors resulting from the ex- about their path if exiting from another exit. Utilizing tensions of the market stalls, preventing visitors from the new digital level that has been designed, the vis- seeing architectural details. To highlight the func- itor can better understand the routes of the interior tional importance of this point, the design of the spa- of the building, but also the connection of the Agora tial narrative focuses on strengthening the visitor’s with its surrounding area. Points 4 and 5, acting as perception on the morphological and structural ele- points of concentrated spatial and urban informa- ments of the Agora. Moreover, this point is selected tion, aim to facilitate user orientation regarding the as the point where the interventionist architectural landmarks and facilities of the city of Chania. Point 4 idea of this project is on-site demonstrated to the vis- refers to the east exit of the building while point 5 to itors of the building. Thus, this point, apart from the the west. There, using marker-based AR and through spatial crossroads, plays the role of the time cross- maps, users can understand a part of the structure of roads, since here with the help of Augmented Real- the city around the building. When the camera of the ity, parts of the architectural past, present and future user’s mobile device detects the specially designed of the building are simultaneously experienced. Im- images, maps appear on top of the image, showing plementing markerless AR, the visitor has the ability specific roads, facilities and attractions. Introducing to aim the camera of her/his mobile device to three

268 | eCAADe 39 - Digital heritage - Volume 2 Augmented Reality in these places, visitors can con- the building of the Agora was presented as a clear tinue their journey in the city of Chania, leaving these shape without the details of the markets, and this lack exits and without having to return to the main en- of details differentiated and altered the appearance trance. (Figure 3_4a,b). of the whole building. Furthermore, the other points on the route were also given high evaluations by the Figure 3 users. The main purpose of reinforcing underappre- Point 3, point 4 & ciated parts of the Agora, such as the north exit to point 5 through AR the old town, was achieved. A great number of users application noticed that it was the first time that they had real- ized the importance of the placement of the Agora and the strong connection between the new and the old town through the main corridor and the north exit. The last two points (4 and 5), situated at the sec- ondary corridor, enabled the users to become more familiar with the surroundings, through the mapping offered by the AR application. The east and the west exits of the Agora lead the user in a buffer zone be- tween the old and the new town. With the aid of the application, the users claimed that they managed to find their bearings concerning the whole city of Chania and to choose how to continue their visits to other city landmarks. In comparison to the aforemen- ASSESSMENT AND EVALUATION (OLD tioned survey (2019), the authors noticed that the in- AND NEW EXPERIMENT) terest of the users was augmented because of the The evaluation of the experiment was achieved stimuli of the AR technology. The narrative helped through a questionnaire that the users completed af- the users follow a certain route and thus enjoy the ter their experience within the Agora. The sections whole building. This guideline was not offered in the of the questionnaires were set around the evaluation previous survey and, as a result, the users avoided of the route, the application, and the enrichment of places that seemed far from the main entrance or the users’ knowledge (Figure 4) .There are two points very crowded or with unpleasant odors. One more which the users appreciated the most. The first point advantage of the introduction of AR was the ability of preference was point 1, the main entrance to the to orientate oneself. Compared to the users in the Agora. The fascinating element about this point was previous survey, users of the AR application claimed that, through AR technology, the users better appre- that the mapping offered by the application helped ciated the scale, the form of the building and its ar- them find their bearings around not only the Agora chitectural details. The second point of preference but also its surroundings. To summarize, after con- was point 3, the center of the Agora. The superim- ducting the experiment and considering its results, posing of the states of the building through the ap- the authors observed that the idea of introducing plication impressed the users and helped them to ap- something ”new” into something ”old” is the optimal preciate the architectural value of the building, such stimuli for users and it is an opportunity to attract as its symmetry and its spatial difference in compari- more people to visit the place. Thus, It is a pleasant son to its current state. The main observation of these way to rethink the monument and collect informa- preferences is that the users appreciated the fact that tion on site in real time. The building interacts with

Digital heritage - Volume 2 - eCAADe 39 | 269 Figure 4 Results of user evaluation (questionnaire)

the user and this interaction is more constructive. highlight the building’s dual character. At the de- Furthermore, concerning the crowd and the temper- sign level, the section was used as the tool to define ature, the circumstances were better in comparison the issues and to appreciate the qualities that the au- to the previous survey. The overall assessment was thors wanted to reveal in each point. The section very good and the users showed a great interest in is not used only as a representation technique, but this new narrative with the aid of a new technology. also as a projective tool for digital spatial interven- tion. The ease of the section process establishes a CREATING SECTION TAXONOMIES means to translate the relationship between digital The narrative and the experience of the users within space and material form (Lewis et al., 2016). Each sec- the Agora, defined the spatial trace of the AR technol- tion depicts the internal spatial qualities and presents ogy, discovering a series of taxonomies (Figure 5). Ev- the immaterial AR intervention and reconstruction. ery point of the route had a unique spatial blueprint The goal is to present the design initiative, based on and the selection of each point intended to recog- the premise of not affecting the physical space of the nize and resolve noticeable spatial issues within the building. Furthermore, taxonomies are a designed space. The chosen tool in order to resolve these is- explanation of the spatial narrative and at the same sues is AR technology, which alters the user’s per- time the narrative itself comes from the emergence ception of the space. The choice of the points and of these taxonomies. Each taxonomy shows the main the narrative throughout this route achieved the seg- idea of enhancing or highlighting each point and menting of the space into individual themes (histor- how AR technology is involved.One of the main diffi- ical, architectural, functional) but at the same time culties that the visitor faces with the Agora is the un- the defragmenting of the space, thus managing to derstanding of its scale, its shape as well as the orien-

270 | eCAADe 39 - Digital heritage - Volume 2 tation of the visitor. With the creation of spatial sec- new design and research methods to enhance user tions and their analysis, the central points in which experience in space. Through this superimposition the alteration of the complete experience of the vis- a new experience design scheme emerges: on the itor is felt, are highlighted. Specifically, at point 1 lo- one hand, the new digital layer that intends to max- cated at the main entrance, the size and shape of the imize knowledge, human senses and their develop- Agora is not easily recognizable by the visitor, so Aug- ment; on the other hand, the section analysis serves mented Reality helps to understand the overall scale. as a basis where space, building conditions and mate- At point 2 low traffic is the key element that needs to rial intersect with human experience. This combina- be reinforced. So, the visitor is led to visit this point, in tion of tools - AR and sectional taxonomies - provides order to balance her/his experience throughout the a flexible approach to the digital design of the experi- whole building and to facilitate the understanding of ence within the Building of the Agora. The availability the building’s relationship with the old city’s bound- of new technologies offers new, unexplored method- aries. At point 3, despite the fact that the visitor is ologies of combining architectural design principles in the center of a cross shaped building and normally with new digital techniques.The way the digital nar- should have an overall picture of the whole space, hu- rative is spatially built is directly related to the way the man intervention (regarding the roof and the exten- visitor approaches and experiences the place but also sions of the shops in the corridors) prevents her/him to the particular architectural, spatial and historical from understanding the size of the space and the ge- characteristics of the building. Although the build- ometric and morphological characteristics of the in- ing is a single object, it looks like it tells two different side of the building. Finally, the axis between points stories. On the one hand, its historicity which in the 4 and 5 is illegible in terms of its role as a passage to eyes of the visitor ceases to be clear and on the other different parts of the city without the visitor having hand its very existence as part of the daily life of the to return to the main entrance to orient. city. The purpose of this project is to design a dig- itally guided route within the real site that will help Figure 5 visitors appreciate the dual role of the Agora, make Taxonomies them admire the whole building and not just parts sections for every of it that are legible and, finally, to reinforce illegible point of interest elements that through the division of space into sec- tors, came to the surface. Starting from the main en- trance and utilizing all dimensions of the building, re- vealing spatial and temporal correlations per prede- fined area, the experience of the visitor balances and reaches completeness. Thus, a historical and spatial route is set with a beginning, course and ending and the user can understand the nature of the Agora as a building and as a journey but also its importance for the city of Chania. At the technological level, the pro- cess of identifying experiential data, combining them with empirical observations and using AR technology CONCLUSIONS AND FUTURE DIRECTIONS as a design strategy that alters the experience with- out affecting the materiality of the building opens This paper presents a multidisciplinary approach that up new possibilities in two discrete fields: firstly AR superimposes interactive strategies, experience de- becomes a valuable user-friendly tool for designing sign and architectural analysis towards identifying

Digital heritage - Volume 2 - eCAADe 39 | 271 spatial experiences; secondly, altering user experi- Guides in Cultural Heritage’, The 8th EAI International ence without affecting the materiality of the build- Conference on Mobile Computing, Applications and ing, yet built on architectural principles promotes Services Karagianni, A., Geropanta, V. and Parthenios, P.2019, ’Ex- the contemporary practice of architecture, correlated ploring the ICT Potential to Maximize User - Built and harmonized with technology. Future directions Space Interaction in Monumental Spaces: The case of this work will expand the methodology of creating of the municipal agora in Chania, Crete’, Sustainabil- experimental tools for enhancing user experience in ity + Cultural Heritage, 2, pp. 603-61 built space and integrate new technologies, design Kiourt, C., Theodoropoulou, H. G., Koutsoudis, A., Ioan- principles and user experience metrics through gam- nakis, J. A., Pavlidis, G. and Kalles, D. 2020, Exploiting Cross-Reality Technologies for Cultural Heritage Dis- ification. By developing a new tool to alter user per- semination, IGI Global ception and knowledge on space, the findings of the Lehman, M.L. 2018, ’Future-proofing the public library’, presented experiments aim to offer insight on how PUBLIC LIBRARY QUARTERLY, 37(4), p. 408–419 the built environment can become the stimuli to al- Lewis, P., Tsurumaki, M. and Lewis, D. 2016, Manual of ter and improve human perception and experience Section, Princeton Architectural Press, New York of it. Panou, Ch., Ragia, L., Dimelli, D. and Mania, K. 2018, ’An Architecture for Mobile Outdoors Augmented Real- ity for Cultural Heritage’, SPRS International Journal REFERENCES of Geo-Information, 7(12), p. 463 Achten, H. 2014 ’One and Many: An Agent Perspec- Parthenios, P., Mania, K., Oikonomou„ A., Mallouchou, F., tive on Interactive Architecture Gerber’, Proceedings Ragia, L., Patsavos, N. and Dimitriou, M. 2014 ’Using of the 34th Annual Conference of the Association for WEBGL to design an interactive 3D Platform for the Computer Aided Design in Architecture, Los Angeles, monuments of Crete’, Proceedings of the 1st CAA GR California, pp. 479-486 Conference Angelopoulou, A., Economou, D., Bouki, V., Psarrou, A., Ruffino, P., Permadi, D., Gandino, E., Haron, A., Osello, Jin, L., Pritchard, C. and Kolyda, F. 2011 ’Mobile aug- A. and Wong, Ch.O. 2019, ’DIGITAL TECHNOLOGIES mented reality for cultural heritage’, Proceedings of FOR INCLUSIVE CULTURAL HERITAGE: THE CASE the 4th International Conference on Mobile Wireless STUDY OF SERRALUNGA D’ALBA CASTLE’, ISPRS An- Middleware, Operating Systems, and Applications, pp. nals of Photogrammetry, Remote Sensing and Spatial 15-22 Information Sciences, IV-2/W6, pp. 141-147 Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S. Sdravopoulou, K., Gutiérrez Castillo, J.J. and Muñoz and MacIntyre, B. 2001, ’Recent advances in aug- González, J.M. 2020, ’Naturalistic approaches ap- mented reality’, IEEE Computer Graphics and Applica- plied to AR technology: an evaluation’, Education tions, 21(6), pp. 34-47 and Information Technologies, 26, p. 683–697 Bekele, M., Pierdicca, R., Frontoni, E., Malinverni, E. and Theodoropoulou, H.G., Kiourt, C., Lalos, A.S., Koutsoudis, Gain, J. 2018, ’A Survey of Augmented, Virtual, and A., Paxinou, E., Kalles, D. and Pavlidis, G. 2020, ’Ex- Mixed Reality for Cultural Heritage’, Journal on Com- ploiting Extended Reality Technologies for Educa- puting and Cultural Heritage, 11, pp. 1-36 tional Microscopy’,in Bourdot, P., Interrante, V., Kop- Davila Delgado, J. M., Oyedele, L., Demian, P. and Beach, per, R., Olivier, AH., Saito, H. and Zachmann, G. (eds) T. 2020, ’A research agenda for augmented and vir- 2020, Virtual Reality and Augmented Reality. EuroVR tual reality in architecture, engineering and con- 2020. Lecture Notes in Computer Science, Springer, struction’, Advanced Engineering Informatics, 45, p. pp. 149-162 101122 Fast-Berglund, A., Gong, L. and Li, D. 2018 ’Testing and validating extended reality (xr) technologies in man- ufacturing’, proceedingsofthe8thSwedishProduction Symposium, pp. 31-38 Galatis, P., Gavalas, D., Kasapakis, V., Pantziou, G. and Zaroliagis, Ch. 2016 ’Mobile Augmented Reality

272 | eCAADe 39 - Digital heritage - Volume 2 Evaluation of Learning Rate in a Serious Game

Based on Anatolian cultural heritage

Sepehr Vaez Afshar1, Sarvin Eshaghi2, Guzden Varinlioglu3, Özgün Balaban4 1,2M.Sc. Landscape Architecture, Faculty of Architecture, Department of Land- scape Architecture, Istanbul Technical University 3Associate Professor Dr., Izmir University of Economics, Department of Architecture 4Delft University of Technol- ogy, The Netherlands 1,2{Afshar19|Eshaghi18}@itu.edu.tr 3,4{Guzdenv|ozgunbalaban}@gmail.com

Cultural heritage conservation has two aspects, tangible and intangible, both of which contribute greatly to the understanding of ancient inheritances. Due to the role of education in the preservation process, and the strength of the new media in the current era, serious games can play a key role in conservancy by transmitting the target culture. There is a gap in the serious game field in relation to Turkey's cultural heritage on the Silk Roads, underlining the motivation of this research. Hence, this study proposes the Anatolian Journey serious game, which is developed in the Twine platform, designed to transmit Turkey's tangible and intangible cultural heritage, providing comprehensive information on the Seljuk caravanserais, located on the Silk Roads. Moreover, the research compares undergraduate and graduate students' gains in knowledge of heritage data while playing a serious game and encountering the same content in text form with an online survey.

Keywords: Digital Heritage, Serious Game, The Silk Roads, Anatolian Caravanserais, Learning Rate

INTRODUCTION ethics as intangible heritage (Bouchenaki 2003). Ac- Cultural heritage demands a high level of preserva- cording to Goodarzparvari and Bueno Camejo (2018), tion, due to its ability to allow assessment of an- education plays a dominant role in cultural heritage cient times through its relevant artifacts and records preservation. Furthermore, they propose that chil- of adventures (Tanselle 1998). The understanding of dren and youth are the most critical age group, due this great value has been expanded in the last three to their influence on upcoming trends. In the current decades, from the appreciation of monuments and era, digital preservation also seems to be a proper sites to architectural constructions as the tangible way to gather these data, and conserve them for the heritage to a society’s rituals, notions, mysteries, and next generations (Ikeuchi et al. 2007). In this re-

Digital heritage - Volume 2 - eCAADe 39 | 273 spect, as a type of new media, serious games, also of studies exists regarding the available text-based referred to as educational games, have more than platforms and their pros and cons. Twine [3] platform pure entertainment value, and have been of interest as a free and open-source tool supported by a non- since 2002 (Alvarez and Djaouti 2011; Mouaheb et al. commercial community is one of the samples of in- 2012). These games deliver educational content, as teractive fiction tools with a helpful community and well as entertainment, in a less restrictive way com- tutorials. The outcomes of this platform are in the pared to purely educational ones (Stege et al. 2011). form of web pages ranging from quite simple texts Hence, we can use serious games, with their educa- to complicated designs enriched with images, audio, tional and digital aspects, as an educational tool to and videos. The game on this platform is concluded increase awareness of the heritage issue for the next from passages producing the scenes of the story. It generation. Moreover, according to Cui (2016), gen- can also be integrated with Twine macros, HTML, CSS, erally, digital games are influential on players‘ beliefs and JavaScript to generate more complex game me- and transmit culture as an art form for the internet chanics. It has a branching system that forces the generation. One of the most well-known instances designer to think about the varied stories and end- in this field, World of Warcraft, delivers an encyclope- ings, in contrast to the traditional linear writings. It dic range of culture (Cui 2016). Furthermore, accord- can also be combined with professional tools such as ing to Hamdaoui et al. (2014), designing a serious Unity and DooM [4] to build the storytelling part of game is challenging, due to the balance that should a videogame (Alstergren et al. 2020; Anderson and be maintained between the ludic aspects and the in- Smith 2021; Barbara 2020; Boom et al. 2020; Cruzeiro formation intended to be transmitted. Hence, a seri- 2020; Hargood 2018; Moulthrop 2020; Not and Pe- ous game’s educational effectiveness is the dominant trelli 2019; Vrettakis et al. 2019). Strikingly, it is taught factor in understanding the serious games’ advan- in several mostly few-hour workshops and Game tages over traditional textbooks in supporting the Jams, inspiring different participants to dig into the learning process. (Stege et al. 2011). However, ac- capabilities of this platform to convey their field of cording to Boom et al. (2020), being aware of the study through it (Boom et al. 2020). Various fields types of historical games people play is the other are taking advantage of the Twine platform as an ac- important step in developing serious games in this cessible and cost-effective tool; such as in pharmacy domain. Additionally, taking into account the vio- in teaching clinical decision making (Morningstar- lence in the existing games is dominant in the cases Kywi and Kim 2021), developing the branching sto- that the game is going to be used as an educational ryline in a conversation chatbot for teaching the En- tool. Considering the development part of the issue, glish language (Muhammad 2020), in disseminating the game industry follows the strategy of the cinema the cultural heritage (Kaleja 2020; Not and Petrelli by involving large groups of multidisciplinary profes- 2019; Vrettakis et al. 2019), and in developing other sions (Moulthrop 2020), using the leading game en- educational storytelling tools namely, BEACONING gines in the market like Unity [1] and Unreal Engine [2] (Cruzeiro 2020), StoryPlaces (Hargood 2018), Dream- (Barbara 2020). In contrast, with the existence of the Scape (Alstergren et al. 2020), Narralive (Vrettakis hypertext games generating interactive fictions, the et al. 2019). Other available commercial text-based need for the varied range of professionals is usually tools can be named as Inform 7 [5] using English syn- reduced to few authors letting express their individ- tax to generate virtual worlds (Moulthrop 2020) and ual perception with less restriction imposed from the Padlet [6], which is somehow similar to Twine but presumptions of the economy. This phenomenon more education-oriented (Cruzeiro 2020).This paper also overpasses the gap between programmers and investigates the role of serious games in the conser- non-programmers (Moulthrop 2020). A wide range vation of cultural heritage, and evaluates their edu-

274 | eCAADe 39 - Digital heritage - Volume 2 cational efficiency in comparison to the traditional 2020, in which players follow the route of Marco Polo education systems, via developing and testing the in the 1270s, journeying from Venice to China, with Anatolian Journey game, which transmits the tangi- choices of routes leading to different endings, re- ble and intangible cultural heritage of the silk roads sembling The Oregon Trail game [7]. Other examples of Anatolia during the Seljuk period. There follows a include Silk Road Match 3, Silkroad Online, Caravan, comprehensive literature review of the previous seri- and Caravanserail. Regarding the literature review, ous games and earlier assessments comparing edu- despite some available serious games related to the cation of the traditional and novel types. Silk Roads, we diagnosed a global gap about the Silk Roads and, specifically, its Anatolian caravanserais in LITERATURE REVIEW the Seljuk period. Hence, the issue needs a more Many studies are investigating the existing serious comprehensive study to address the gap, with the es- games both commercially and academically (Eshaghi sential surveys to test its value in education. and Örnek 2020; Forte et al. 2012; Johnson and Va- lente 2009; Lercari et al. 2015; Örnek and Seçkin METHODOLOGY 2016; Poplin 2011; Varinlioglu 2017); The Oregon Trail History is considered to exemplify this genre. This game has As a cultural heritage game, the first step before start- been played for approximately twenty years by ele- ing the game process is to investigate the history of mentary school students, targeting the 1990s’ pio- the game’s target period. Hence, understanding the neer life on the Oregon Trail (Becker 2007; Bigelow history of the Silk Roads, caravanserais, and Anato- 1997; Caftori and Paprzycki 1997; Kane 2020; Regal- lian Seljukid period is of great importance in this re- ado 2017; Slater 2017). Addressing specifically the search. Now known as the Silk Roads, this network of rate of learning, Stege et al. (2011) compared the abil- trade routes was in addition to the route of merchan- ity of a group playing a serious game named E and dise, also the route of culture, beliefs, and knowl- Eve’s Electrical Endeavors and another group, reading edge transmission. The exchange of the caravans’ lo- a textbook about the same issue of electrical engi- cal products for domestic goods from the cities they neering in a high school. They concluded that seri- passed through enhanced cultural conveyance, in ous games led to a remarkably better result, although addition to the merchants’ material and wealth. The it is not possible to conclude with certainty that they caravans’ stopping points were the caravanserais, were more motivated than they would have been if constructed from the 10th until the 19th century, and reading the same content as a text. Regalado (2017) became vital in facilitating journeys [8]. For the cur- and Kane (2020), both in Trinity College, ran a study rent study, we chose the Seljuk caravanserais, a rel- assessing students’ learning ability using games. The atively under-researched period of Turkey’s history first is based on The Oregon Trail game, and the sec- compared to the Roman, Byzantine, and Ottoman ond is designed for a course curriculum, using a game periods. However, there is doubt over the exact loca- developed in 2019 by Aldrich, Kane, and Ky, named tion of the Seljuk caravanserais. After their establish- The Silk Road, following the rules of The Oregon Trail ment in the 12th century, the Anatolian Seljuks be- game (Kane 2020). Hamdaoui et al. (2014) proposed came important traders and built many caravanserais using the e-learning standard to generate more ef- leading to the development of the area (Önge 2007). fective serious games and thus standardize the ed- ucation process, by facilitating communication be- Data Management tween the instructor and game designer. As a sam- In terms of the data gathered for this study, the as- ple of commercial games about the Silk Roads, Silk sembled Seljukid Anatolian Caravanserais’ data are Roads: Caravan Kings is a newly released game in roughly the most comprehensive available dataset

Digital heritage - Volume 2 - eCAADe 39 | 275 promoting the study’s background. This dataset was vanserais in the current research (Figure 1). Finally, to generated during a data digitization process in a identify the connections between the caravanserais, Ph.D. course, using a high accuracy method. Initially, based on the Silk Roads’ routes in the target period, the researchers in the course consulted the available we superimposed the tourism route Anatolia by Car- books (Acun 2007; Erdmann 1961; İlter 1969), an ex- avanserai, proposed by Bektaş (1999, p.53). However, isting article (Özergin 1965), a map named Silk Road we reduced the number of caravanserais to improve - Culture Road [9], a website [10], and the OWTRAD the game map’s readability. project [11], covering the whole caravanserais of Anatolia. The essential data from these sources were Game Design digitalized. To locate the gathered caravanserais on According to the literature review, regarding archeol- a Google Earth Project, first, the known caravanserais ogy and history, non-linear storytelling seems to be were found on Google Earth, and second, we tried to significantly fitted due to its capability in engaging locate the others using the naked eye on the satel- the player with the past in a multi-perspective man- lite images, through the information provided in the ner. Additionally, the individuals or groups develop- researches. Third, where this was not possible, we ing their story in such a branching system will be in- determined an estimated location. In the next stage, volved in deep research of the intended topic and we classified the gathered information via a four-level gain the knowledge meanwhile (Boom et al. 2020). classification. We defined the levels as follows: level Hence, we developed the game on the Twine plat- 1, the registered historical buildings by the Ministry form as an alpha version. This research is an exten- of Culture and Tourism, state museums or their web- sion of our recent study in the ASCAAD2021 confer- sites, and the local authorities [12, 13]; level 2, re- ence based on Iran’s Safavid caravanserais named, searched historical buildings determined by local au- The Sericum Via: A Serious Game for Preserving Tan- thorities and academic institutions; level 3, informa- gible and Intangible Heritage of Iran (Eshaghi et al. tion from travelers and locals; and level 4, Çekül’s 2021), following the similar methodology of The Ore- map (2012). We entered all the information into the gon Trail in the context of Anatolia. As mentioned in PostGIS online database to generate a caravanserai the future work of the previous research, the Anato- map in the QGIS. However, to ensure the reliability lian Journey game uses the gathered information of of the information, we used only the level 1 cara- the Anatolian caravanserais and the intangible her- Figure 1 Data Management Process

276 | eCAADe 39 - Digital heritage - Volume 2 Figure 2 Gameplay Screen

itage of the map’s each area in the game’s related ing style, with questions and fill-in-the-blanks during chapters. The player takes on a merchant’s role, trad- the gameplay. To select the content that we want to ing among cities, based on the assigned duties. Un- convey throughout the game, we looked for a strik- like the Sericum Via, this time, the player is not lim- ing point throughout the history of Anatolia in the ited by time or health, and instead, the final score Seljuk Period to start the journey. During that time, is based on the duties completed and the number Konya was the capital of Anatolia and a critical ac- of coins gained by answering questions. This game commodation and trade center located on the Silk mechanic allows the game to have more educational Roads. It was also an important location in terms impact in an entertaining way. As the learning ob- of intangible heritage [14]. Considering the selected jectives, the game engages the player in the realm caravanserais to be included in the game’s map, we of living as a Seljuk era trader, who encounters pass- decided to start the journey from Antalya to reach ing caravanserais. Additionally, in contrast to the Ser- Konya city. Hence, as the representative section of icum Via, whose survey described the target player as the whole game, the first chapter takes place in the in the 11-20-year-old range, we designed the Anato- surroundings of Antalya, giving valuable information lian Journey game specifically for our target, under- about the city in the Seljuk period, obtained via a lit- graduate and graduate students. Moreover, as we erature review, as the player talks with locals (Demir noticed that the players tended to ignore the pro- 2018; Dinç 2017)(Figure 2). vided data, we designed the game in a more engag-

Digital heritage - Volume 2 - eCAADe 39 | 277 The images we used in the game are under the Cre- Form questionnaires, one for each group, containing ative Commons license, including some from the the same ten questions, with links at the end of the SALT Research online website, Ali Saim Archive [15], game and the text file. The questionnaires tested the which we made some modifications to them (Figure conductors’ comprehension rate. The survey pop- 3). ulation was Graduate and Undergraduate students of the Izmir University of Economics (IEU). The ques- tionnaire was delivered online due to the Covid-19 Figure 3 pandemic and consent was sought before participa- Gameplay Screen / tion in the survey; we also applied for a Scientific Re- Image Source: SALT search and Publication Ethics Form of the IEU. 22 par- Research, Ali Saim ticipants were equally separated into groups A and Ülgen Archive [15] B. The results show that group A, which played the game, scored on average 8 out of 10, and group B’s average was 5 out of 10. Additionally, 5 of the game players’ correct answers were among the identical questions in the game (Figure 4). Hence, the results indicate that providing cultural heritage educational data, in the form of a serious game, raises the degree of data retention. This implies that the players can better memorize the data encountered in the form of Figure 4 in-game questions rather than in readings. Survey Results

CONCLUSION The focus point of this paper is the preservation of cultural heritage via serious games. Through these games’ transitional and educational nature, the tar- geted tangible and intangible heritage can be con- served. Serious games such as the Anatolian Jour- ney game, proposed here, can replace the traditional educational tools, such as textbooks. This game en- gages players with Turkey’s heritage in the Seljuk pe- riod, and assists the retention of data via a recre- PLAYTESTING AND EVALUATION SURVEY ational activity. Also, the conducted surveys high- Same as the previous study, we conducted an open light the game’s teaching potential. However, the full alpha playtest. While in the Sericum Via game, we version of the game may show even greater improve- sought player feedback in terms of the game inter- ments due to continuous gameplay. As a further face, content, and target audience; in this research study, we can merge the Sericum Via game and the we evaluated the students‘ learning rate while play- Anatolian Journey game, enabling the players to un- ing the Anatolian Journey game compared to their dertake a cross-cultural journey along the Silk Roads, comprehension of observing the same content in a passing through various countries. An advantage of traditional textbook. We designed our survey for the platform used for game development is the abil- two groups; group A played the game, and group B ity to extend the game as desired. We are aware of read the provided text file containing the same con- the limitations of the game we developed due to the tent as texts and images. We prepared two Google

278 | eCAADe 39 - Digital heritage - Volume 2 lack of time and labor as a small group; however, one Becker, K 2007, ’Digital game฀based learning once re- of the main goals of the study is to demonstrate the moved: Teaching teachers’, British Journal of Educa- tional Technology, 38(3), pp. 478-488 capabilities of the Twine tool as a game development Bektaş, C 1999, A Proposal Regarding the Seljuk Car- platform in a cost and time-efficient manner with no avanserais Their Protection and Use, Yapı Endüstri need for a considerable amount of effort to learn it Merkezi Yayınları Bigelow, B 1997, ’On the road to cultural bias: A critique and any big multidisciplinary group. It means that of The Oregon Trail CD-ROM’, Language Arts, 74(2), everyone can be a Twiner and generate his/her do- pp. 84-93 main’s game to disseminate it joyfully and effortlessly Boom, Krijn HJ, Ariese, C. E, van den Hout, B, Mol, A. A and Politopoulos, A 2020 ’Teaching through play: Using to the broadest possible community. Furthermore, video games as a platform to teach about the past’, the final game exported as an HTML file is able to Communicating the Past 2020, p. 27 be played in any browser and any device. It means, Bouchenaki, M 2003 ’The interdependency of the tan- it can be available for masses in any socioeconomic gible and intangible cultural heritage’, 14th ICOMOS General Assembly and International Symposium range as educational content. Meanwhile, thanks to Caftori, N and Paprzycki, M 1997 ’The design, evaluation its possibilities to be integrated into other engines, and usage of educational software’, Society for Infor- enhanced by using more complicated codings, and mation Technology & Teacher Education International Conference. Association for the Advancement of Com- adding professionally designed graphics or videos, puting in Education, pp. 23-27 any developed game, can be turned into a commer- Cruzeiro, T.J.L 2020, Inven! RA-Platform for authoring and cially perfect video game. tracking of Inventive Activity Plans, Master’s Thesis, UNIVERSIDADE DO PORTO Cui, C 2016, ’A Study of Digital Games as a New Me- ACKNOWLEDGMENT dia of Cultural Transmission’,in Pan, Zhigeng, Cheok, Adrian David, Müller, Wolfgang and Zhang, Ming- We prepared this paper with the dataset assembled min (eds) 2016, In Transactions on Edutainment XII, in a doctorate course held by Assoc. Prof. Dr. Güzden Springer, pp. 48-52 Varinlioğlu and Dr. Özgün Balaban, at Istanbul Tech- Demir, M 2018, ’CONQUEST OF ANTALYA BY THE SELJUKS OF TURKEY’, Cedrus, 6, pp. 555-571 nical University, the Architectural Design Computing Dinç, G 2017 ’Cumhuriyet’in İlk Yıllarında Antalya’da İhra- Graduate Program. Hence, we would like to espe- cat ve İthalat’, Uluslararasi 9. Türk Deniz Ticaret Tarihi cially thank all the student researchers on the course Sempozyumu, İstanbul, pp. 37-38 for their valuable contribution to this study. Erdmann, K and Erdmann, H 1976, Das anatolische Kara- vansaray des 13. Jahrhunderts/2/3 Baubeschreibung. Die Ornamente, Mann REFERENCES Eshaghi, Sarvin, Vaez Afshar, Sepehr and Varinlioglu, Acun, H 2007, Anadolu Selçuklu dönemi kervansarayları, Guzden 2021 ’THE SERICUM VIA: A Serious Game for Kültür ve Turizm Bakanlığı Preserving Tangible and Intangible Heritage of Iran’, Alstergren, Iza 2020, DreamScape, a Platform for Creating, The 9th International Conference of the Arab Society Sharing and Listening to Interactive Stories, Master’s for Computer Aided Architectural Design, Egypt, pp. Thesis, Uppsala University 306-316 Alvarez, J and Djaouti, D 2011, ’An introduction to Seri- Eshaghi, S and Ornek, M 2020 ’‘Yedikule Bostans’: A Se- ous game Definitions and concepts’, Serious Games rious Game For Cultural Heritage’, IDU SPAD’20 Inter- & Simulation for Risks Management, 11(1), pp. 11-15 national Spatial Planning and Design Symposium Anderson, Barrett R and Adam, M. Smith 2021 ’The Un- Forte, M, Lercari, N, Onsurez, L, Issavi, J and Prather, E dergraduate Games Corpus: A Dataset for Machine 2012 ’The Fort Ross virtual warehouse project: a se- Perception of Interactive Media’, Proceedings of the rious game for research and education’, 18th Interna- Thirty-Fifth AAAI Conference on Artificial Intelligence tional Conference on Virtual Systems and Multimedia, (AAAI-21) pp. 315-322 Barbara, Jonathan 2020 ’Twine and DooM as Authoring Goodarzparvari, P and Bueno Camejo, FC 2018, ’Preser- Tools in Teaching IDN Design of LudoNarrative Dis- vation of cultural heritage via education of children, sonance’, nternational Conference on Interactive Dig- utilizing visual communication: Persepolis as a case ital Storytelling. Springer 2020, Cham, pp. 120-124 of study’, Scientific Research Publishing, 9, pp. 141- 151

Digital heritage - Volume 2 - eCAADe 39 | 279 Hamdaoui, N, IDRISSI, MKHALIDI and BENNANI, S 2014, Tanselle, GT 1998, Literature and artifacts, Bibliographical ’Serious Games in Education Towards the standard- Society of the University of Virginia ization of the teaching-learning process’, Advancesin Varinlioglu, G, Aslankan, A, Alankus, G and Mura, G 2017 educational technologies, 2, pp. 174-181 ’Raising Awareness for Digital Heritage through Se- Hargood, Charlie, Mark, J. Weal and David, E. Millard 2018 rious Game’, 35th International Conference on Educa- ’The storyplaces platform: Building a web-based tionandResearchinComputerAidedArchitecturalDe- locative hypertext system’, Proceedings of the 29th on sign in Europe., p. 31 Hypertext and Social Media, pp. 128-135 Vrettakis, E, Kourtis, V, Katifori, A, Karvounis, M, Lougiakis, Ikeuchi, K, Oishi, T, Takamatsu, J, Sagawa, R, Nakazawa, A, C and Ioannidis, Y 2019 ’Narralive–Creating and ex- Kurazume, R, Nishino, K, Kamakura, M and Okamoto, periencing mobile digital storytelling in cultural her- Y 2007, ’The great buddha project: Digitally archiv- itage’, Digital Applications in Archaeology and Cul- ing, restoring, and analyzing cultural heritage ob- tural Heritage jects’, International Journal of Computer Vision, 75(1), Önge, M 2007, ’Caravanserais as symbols of power in pp. 189-208 Seljuk Anatolia’, Power and culture : identity, ideology, Johnson, WL and Valente, A 2009, ’Tactical language and representation, 306, p. 21 culture training systems: Using AI to teach foreign Örnek, MA and Seçkin, YC 2016, ’Development of an Edu- languages and cultures’, AI magazine, 30(2), pp. 72- cational Video Game for the Teaching of Landscape 72 Grading Principles’, Journal of Digital Landscape Ar- Kaleja, Laine 2020, Linking archive materials through in- chitecture, 1, pp. 308-315 teractive storytelling for innovative spatial exhibition Özergin, M 1965, ’Anadolu’da Selçuklu kervansarayları’, design in cultural heritage, Master’s Thesis, Aalborg Tarih Dergisi, 15(20), pp. 141-170 University Copenhagen Kane, M 2020 ’A Comparison of Online and Offline Mea- İlter, I 1969, Tarihi Turk Hanlari, Karayolları Genel Müdür- sures of Metacognition: Featuring The Silk Road lüğü Game’, Society for Information Technology & Teacher [1] https://unity.com/ Education International Conference [2] https://www.unrealengine.com/en-US/ Kim, Rory E 2021, ’Using Interactive Fiction to Teach Clin- [3] http://twinery.org/ ical Decision-Making in a PharmD Curriculum’, Med- [4] http://www.doombuilder.com/ ical science educator, 31.2, pp. 687-695 [5] http://inform7.com/ Lercari, N, Onsurez, L and FORTE, M 2015, ’M. Multimodal [6] https://padlet.com/ Strategies for Virtual Reconstruction of Landscapes [7] https://store.steampowered.com/app/1401840/Silk for Serious Heritage Games’, ACM Journal on Com- _Roads_Caravan_Kings/ puting and Cultural Heritage, 9, pp. 1-20 [8] https://en.unesco.org/silkroad/about-silk-roads Mouaheb, H, Fahli, A, Moussetad, M and Eljamali, S [9] https://www.cekulvakfi.org.tr/files/dosyalar-haber/i 2012, ’The serious game: what educational bene- pekyolu_harita_min.pdf fits?’, Procedia-Social and Behavioral Sciences, 46, pp. [10] http://www.turkishhan.org/ 5502-5508 [11] http://www.ciolek.com/owtrad.html Moulthrop, S 2020 ’Developing English Conversation [12] https://muze.gov.tr/muzeler Chatbot Using Dialogflow’, In 2020 International Elec- [13] https://www.kulturportali.gov.tr/ tronics Symposium [14] https://konyakultur.gov.tr/index.php?route=pages Not, E and Petrelli, D 2019, ’Empowering cultural her- /pages&page_id=1 itage professionals with tools for authoring and de- [15] https://archives.saltresearch.org/handle/12345678 ploying personalised visitor experiences’, User Mod- 9/46 eling and User-Adapted Interaction, 29, pp. 67-120 Poplin, A 2011 ’Games and serious games in urban plan- ning: study cases’, In the International Conference on Computational Science and Its Applications, pp. 1-14 Regalado, AM 2017, A Think-Aloud Assessment of Metacognition in 8th Grade Students: Using The Ore- gon Trail, Master’s Thesis, Trinity College Slater, K 2017, ’Who Gets to Die of Dysentery?: Ideology, Geography, and The Oregon Trail’, Children’s Litera- ture Association Quarterly, 42(4), pp. 374-395 Stege, L, Van Lankveld, G and Spronck, P 2011, ’Serious games in education’, International Journal of Com- puter Science in Sport, 10(1), pp. 1-9

280 | eCAADe 39 - Digital heritage - Volume 2 Rule based systems and shape grammar 282 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 The Market Square Grammar Reconstructing the facades of Warsaw’s old town

Marzena Nowobilski1, Heather Ligler2 1,2Pennsylvania State University 1,2{mvn5252|hml5521}@psu.edu

During World War II, eighty-five percent of Warsaw was destroyed. After the war, the city's Old Town buildings and monuments were ambitiously reconstructed to replicate the prewar configuration - an effort dependent on a vast amount of documentation in the form of photographs, paintings, drawings, and more. Warsaw's Rynek Starego Miasta (Old Town Market Square) was a significant part of the reconstruction effort and continues to be an iconic landmark of Warsaw which exemplifies the architectural style and character of the city's past. This paper considers the way in which the Old Town Market Square can be further preserved, recreated, and newly understood through a parametric shape grammar that formalizes the characteristic facades of the historic square and subsequently generates two-dimensional elevations to celebrate this heritage.

Keywords: shape grammars, digital heritage, procedural modeling, generative design

SHAPE GRAMMAR AS A TOOL FOR CUL- search. For example, projects to date have focused TURAL APPRECIATION AND EDUCATION on using shape grammars as a basis for the rapid The complex history of Warsaw’s Old Town, especially procedural modeling of facades (Müeller et al, 2007), in the reconstruction of the Market Square facades, combining shape rules with three-dimensional LI- is matched by a complex visual history of interpreta- DAR scan data to determine facade attributes (Rip- tion and reinterpretation in a variety of media rang- perda, 2008), and using shape grammars to develop ing from measured drawings to imprecise paintings. rulesets that semi-automatically generate heritage The goal of this research is to illustrate how shape building information models (HBIM) consisting of fa- grammars (Stiny and Gips, 1972; Stiny, 2006; Knight, cade data (Dore and Murphy, 2014). While these ef- 1994) can be instrumental in formalizing these inter- forts have paved the way, a key challenge in this re- pretations while simultaneously laying the ground- search area is to capture custom details in facades work for new understanding of this history and its (Hohmann et al, 2009). The research here is partic- possible futures. ularly interested in how the characteristic details of Similar approaches applying the shape grammar each individual facade can be documented and re- formalism to facade analysis and generation form a configured, leading to multiple rules providing op- foundation for developing and differentiating this re- tions for elements in each composition. With this per-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 283 spective in mind, a primary appeal of shape gram- ary of 1945. Within BOS there were two opposing mars is their visual language specified with replace- views on reconstruction - one for replication and ments, substitutions, and transformations that can the other for modernization. The replication argu- capture the structure of a language of designs in as ment was led by Professor Jan Zachwatowicz, who ar- much detail as desired. This structure yields compu- gued that the buildings and monuments of Warsaw tations that are intuitive and explanatory, suggesting should be accurately reconstructed based on the ex- how the formalism might be useful to curatorial and isting documentation. On the other hand, Roman Pi- educational practices in digital heritage, where pro- otrowski, along with his deputy Józef Sigalin, wanted cesses of reconstruction are challenging to describe, to “modernize” the city. In the end, it would be de- interpret, and evaluate. cided that Warsaw’s Old Town would be carefully re- constructed to follow the historic character of pre- war Warsaw by Zachwatowicz and his team [1]. This Figure 1 proved to be a formidable task and Old Town Warsaw View of Strona is now celebrated as a UNESCO Heritage Site due to Dekerta (photo by the phenomenon of the near-total reconstruction of author). the urban area and its material, functional, and sym- bolic value [2]. Even more, Zachwatowicz’s contribu- tion in this effort is recognized even to this day, as a statue of him in Old Town Warsaw was unveiled in March 2021 [3].

RECONSTRUCTION DOCUMENTS AND AU- THENTICITY To start the reconstruction process, a vast amount of documentation of the Old Town that existed in the form of drawings, photographs, and paintings pro- duced both before the war and secretly during oc- THE DESTRUCTION AND RECONSTRUC- cupation were studied. However, the historic veduta TION OF WARSAW paintings of Bernardo Bellotto emerged as the main Nearly all of Warsaw was destroyed during World source of inspiration for the reconstructions because War II - with only fifteen percent of the city retain- his paintings reflected Old Town during an indepen- ing its original fabric and character. A city that once dent Poland and were believed to have captured the had over a million residents was left with only a few city center in its most beautiful form. Bellotto was the thousand. It seemed almost impossible that Warsaw court painter to the last King of Poland, Stanisław Au- would be rebuilt, so how did it happen? gust Poniatowski, and painted over twenty highly de- Beginning January in 1945, former residents and tailed vedutas of Warsaw’s Old Town during the late displaced persons started coming back to Warsaw 1700’s. Great attention to detail was paid by Bellotto, looking for a new start. These inhabitants started combined with a tendency to alter the buildings in rummaging through Warsaw’s rubble and attempt- his paintings in order to “improve” them [4]. Due to ing to rebuild the city themselves. To regulate the this creative license, Warsaw’s Old Town is not an ex- method and aesthetic of this adhoc reconstruction, act replica of what previously existed, but rather a the Biuro Odbudowy Stolicy (BOS or Office for the Re- reimagined Warsaw as seen through the eyes of Bel- construction of the Capital) was established in Febru- lotto and the team of urban planners and architects

284 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 who interpreted the paintings alongside other frag- set of the reconstruction effort to aid in understand- ments of documentation for the reconstruction ef- ing this complex heritage of Warsaw. This is pro- fort. gressed through a formal study on the characteristic After the completion of the reconstruction, citi- facades in Warsaw’s Old Town Market Square. These zens of pre-war Warsaw began to notice that there facades are distinctive in how they offer a link be- was a disconnect between the Old Town they re- tween the pre-war city, Bellotto’s imaginary city, and membered and the newly reconstructed Old Town the current post-war city. More precisely, the work [5]. These discrepancies continue to be unpacked to here is focused on interpreting the character of the assess the nature of the reconstruction, which actu- reconstructed city center formally and document- ally combined efforts of replication and moderniza- ing the current Old Town Market Square through a tion due to the interpretation of the pre-war docu- visual-computational approach that can accommo- mentation as well as other constraints of materiality date replication, modernization, and more. This is and function. Reflecting on all of these influences, achieved in two parts, each focusing on the facades Kuźnicki contends that “The reconstruction has man- in the Old Town Market Square as illustrated in two- aged to successfully revive the spirit of Warsaw’s his- dimensional elevation drawings. The first part of the tory, with eclecticism of all historical layers and styles research analyzes the facades to interpret the for- present in the city. Due to several technical changes mal structure of their organization. The second part introduced into the urban plan of the district, the en- builds on this analysis to develop a generative spec- tire Old Town remains an integral part of the living ification which can produce the current facades and organism of the city. As a residential area, it serves other reconstructions in a similar character, suggest- the same traditional purpose it has served over many ing a method that can both appreciate the city as it is years. The Old Town remains true to its function and and as it might be reimagined. nature. All of these characteristics contribute to the authenticity of the Old Town” (2013). The research here aims to look closer at a sub-

Figure 2 Elevation of Strona Dekerta (drawing by author).

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 285 Figure 3 Analysis of levels in the eight individual facades of Strona Dekerta.

ANALYZING WARSAW’S OLD TOWN MAR- sions, ratios, and spacing of architectural elements KET SQUARE automatically” (Müeller et al, 2007). Following this Warsaw’s Old Town Market Square is the central ur- concept, each facade is broken down into horizontal ban space of the Old Town and has maintained this levels which correspond to floor levels in the overall role since the city was founded in the late 13th cen- design. Strona Dekerta is composed of eight levels tury. The square consists of four sides, each oriented in total, with individual facades ranging from five to to a cardinal direction and named after a prominent eight levels. These levels are: a) ground level, labelled figure in Warsaw’s history (Lewandowski). The sides in red; b) first level, labelled in dark blue; c) second are: a) Strona Dekerta (Dekert’s Side) to the north; level, labelled in light blue; d) third level, labelled in b) Strona Barssa (Barss’ Side) to the east; c) Strona dark green; e) first roof level, labelled in light green; f) Zakrzewskiego (Zakrzewski’s Side) to the south; and second roof level, labelled in pink; g) fourth level, la- d) Strona Kołłątaja (Kołłątaj’s Side) to the west. The belled in yellow; and h) third roof level, labelled in or- scope of the current study focuses on Strona Dek- ange. To study vertical relations between levels, each erta for two reasons. First, because it is the side is further labelled with a letter (A, B, C, or D) to indi- of the square which had the most remaining frag- cate the proportional rhythms of various heights in ments post-war; and second, because it houses the each individual facade. Based on this labelling, the Muzeum Warszawy (Museum of Warsaw) in which facades are categorized into five types according to the Destruction and Reconstruction of the Old Town is their vertical proportional patterns: a) the simple fa- exhibited today (Figures 1 and 2). cade, with an order of A-B-B-C-A-A; b) the complex Strona Dekerta can be decomposed into eight facade, with an order of A-B-B-B-A-C-D-D; c) the hy- individual facades as illustrated in Figure 3. To ana- brid facade, with an order of A-B-B-C-D-D; d) the or- lyze and categorize these facades formally, a subdi- namental facade, with an order of A-B-C-B-D; and e) vision of levels and tiles is used. This method is in- the ornamental asymmetric facade, with an order of spired by a similar approach used in the image-based A-B-C-D-B-C. procedural modeling of facades, where the order un- derstood through subdivision is reinterpreted to cre- ate a set of rules based on element placement within a facade structure that can “derive the exact dimen-

286 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 4 PARAMETRIC SHAPE GRAMMAR OF SIM- Tiling analysis of PLE FACADES simple facades 1 Drawings, paintings, and photographs - the docu- and 2. mentation relied on for the post-war reconstruction of Warsaw - all foreground visual understanding as a primary means of designing, archiving, preserv- ing, imagining, and rebuilding. The shape gram- mar formalism likewise privileges generative descrip- tions that elaborate visual and spatial relationships to achieve novel computations. Shape grammars are distinctive in how they emphasize shapes, rather than symbols, as the basis for defining rules and their respective computations (Stiny and Gips, 1972; 2006; Knight, 1994). Here, this visual-computational method is used to develop a parametric shape gram- mar for the simple facades of Warsaw’s Old Town Mar- ket Square. Inspiration for the Market Square Grammar builds on the ice-ray grammar, which formalizes Chi- nese lattice designs. To develop a simple lattice, a ruleset starts with a frame as an initial shape. Through a series of rule applications, the frame is modulated with tiles, each oriented with labels spec- ified in the rules. These tiles fill-up the frame and the To further narrow the scope here, the simple facades grammar features replacement rules that transform are emphasized as the basis for the current study. the tiles into patterns (Stiny, 1977). Following a sim- These facades are further decomposed to analyze the ilar logic, the levels and tiles analyzed from the sim- relations in each level, as shown in Figure 4. These ple facades of the Old Town Market Square can be re- smaller subdivisions effectively result in a tiling of the worked as shape rules to generate the facades that facade that corresponds to the characteristic element currently exist as well as newly imagined facades in types placed on each level, including: a) doors; b) the spirit of Bellotto’s revisions. windows; c) stringcourses; d) dormers; and e) sky- The Market Square Grammar is a parametric lights. Each level is broken down with 1, 2, or 3 shape grammar that produces two-dimensional el- equivalent tiles depending on the arrangement of fa- evations in the language of simple facades recon- cade elements within each of the eight distinctive structed at Warsaw’s Old Town Market Square. To levels. The interested reader will notice that some of begin, the parameters for the main vocabulary ele- the levels may contain a shift of elements, as shown ments within the grammar are defined: y indicates in simple facade 2. With the simple facades of the the height of the initial level, x indicates the width Strona Dekerta decomposed in terms of levels and of the levels, ya indicates the new heights of the the placement of elements, likewise corresponding first, fifth and sixth levels, yb indicates the new height to these levels, the essential vocabulary and spatial of the second and third levels, yc indicates the new relations of this class of facades are described. This height of the fourth level, d indicates the distance analysis is the basis for the shape grammar devel- of the horizontal shift within a level, e indicates the oped in the next section.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 287 x x − d width of a tile which can be defined as or , GENERATING SIMPLE FACADES WITH THE 3 3 f indicates the vertical positioning of an element MARKET SQUARE GRAMMAR within a tile, g indicates the height of an element, To generate a simple facade, the rules of the Market h indicates the width of an element, and i indicates Square Grammar are applied sequentially. Figure 8 the horizontal positioning of an element within a tile illustrates the derivation for a simple facade design (Figure 5). The Market Square Grammar is organized that matches the composition of simple facade 1 in in eight stages that describe the progression from an Figure 4, an existing reconstructed design of Strona initial shape to a simple two-dimensional facade de- Dekerta. A sequence of forty-six shape rules is ap- sign. These stages are: a) stage 1 to define the ini- plied to achieve this design. The process begins with tial shape; b) stage 2 to generate vertical levels of an rule 1.1 to initialize the design. Then, rule 2.1 is ap- individual facade; c) stage 3 to relabel the levels in plied five times to generate six levels. Moving into terms of the colors defined in the analysis; d) stage stage 3, rules 3.1, 3.2, 3.3, 3.4, and 3.5 are applied 4 to proportionally scale the levels to follow to A-B- to label each level. Next, rule 4.1 is applied. This B-C-A-A rhythm of the simple facades; e) stage 5 to rule is unique in how it uses a parametric schema to divide each level into tiles; f) stage 6 to place ele- scale the levels, which are revised in this application ment boundaries within tiles; g) stage 7 to redefine to achieve the A-B-B-C-A-A rhythm of the simple fa- tile boundaries so that only a facade outline remains; cade (Stiny, 1977; 2006; 2011). Rules 5.1.2, 5.2.1, 5.3.1, and h) stage 8 to replace element boundaries with fa- 5.4.1, 5.5.2, and 5.6.1 are applied next to produce cade elements. While the stages identify a simple se- the next level of subdivision in the grammar. Then, quence of rule categories for placing vocabulary ele- the application of rules 6.1.2, 6.1.3, 6.2.1, 6.2.2, 6.3.1, ments, each stage includes multiple rules that elabo- 6.3.2, 6.4.1, 6.4.2, 6.5.1, 6.6.1, 6.7.1, and 6.7.2 gener- rate the detailed options at each step of the process. ate the element boundaries within each tile. To re- In all, the grammar consists of 98 shape rules. These calibrate the overall frame by removing boundaries, rules are all depicted in Figures 6 and 7. rules 7.1.2, 7.1.3, 7.2.1, 7.2.2, 7.3.1, 7.3.2, 7.4.1, 7.4.2, 7.5.1, 7.6.1, 7.7.1 and 7.7.2 are applied. Lastly, the rules of stage 8 complete the grammar by placing fa- Figure 5 cade elements with rules 8.1.1, 8.2.1, 8.3.1, 8.4.1, 8.5.1, Main vocabulary 8.6.1, 8.7.1, and 8.8.1. elements of the The benefits of the grammar can be immediately grammar and their appreciated through this derivation, which acts as a parameters. frozen animation of the production. Subsequently, using different initial parameters, scaling parameters, and rule sequences, a variety of simple facades can be produced. These are generated as individual fa- cades in the current version of the grammar, though additional rules can be developed to string the fa- cades together and study their overall urban char- acter. A sample of eight simple facades produced with the grammar strung together in this manner to reimagine a side of the market square is given in Fig- ure 9.

288 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 6 Shape rules of Stages 1-7 of the Market Square Grammar.

DISCUSSION inary vision of the city. When the grammar is con- The utilization of shape grammars to study the Mar- strained to the actual facade details and related pa- ket Square facades allows for a range of generative rameters, it can act as visual documentation to em- outcomes - from the precise to the experimental. phasize a precision appropriate to record the current Due to the intuitive and visual nature of the formal- site. On the other hand, replacement rules allow for ism, this ruleset is accessible for users of all levels who substitutions that explore new configurations rang- can decide whether to adhere to an accurate or imag- ing from the subtle to the exaggerated, just as Bel-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 289 Figure 7 Shape rules of Stage 8 of the Market Square Grammar.

290 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 8 Derivation to produce a simple facade from Strona Dekerta following the sequence of the 8 stages in the Market Square Grammar.

lotto’s paintings included a personal perspective of gests how future work with the grammar might con- an ideal city later realized in the reconstructed build- tinue. The pictorial nature of the grammar, along ings. This flexibility suggests how shape computa- with its implementation, allows the Market Square tion prompts new discussions on the nature of recon- grammar to be interactively utilized and understood. struction and possibilities of visualizing past, present, This suggests a tool for preserving and interpreting and futuristic histories of the city. the recreation of the facades - one that also acts as The current version of the grammar is all man- a medium for cultural appreciation and education ually defined and produced by hand drafting the in which users can create their own reimagining of shape rules and their computations. In addition, as the facades. Future work will aim to build on this proof-of-concept, the authors are developing an im- preliminary implementation to integrate the hard- plemented version in Python Rhinoscript that sug- coded grammar with the Shape Machine interpreter

Figure 9 New simple facade designs generated with the Market Square Grammar.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 291 (Economou et al, 2021; Ligler and Economou, 2019), Economou, A, Hong, TCK, Ligler, H and Park, J 2021, allowing users to define their own rules for facade ’Shape machine: a primer for visual computation’,in designs by drawing their own tailored details and Lee, JH (eds) 2021, New Perspective of Cultural DNA, Springer, Singapore, pp. 65-92 embellishments, continuing the spirit of reinvention Hohmann, B, Krispel, U, Havemann, S and Fellner, D 2009 inherent to Bellotto’s emblematic venduta paintings ’CityFit: High-Quality Urban Reconstructions by Fit- and their impact on the history of Warsaw. This ef- ting Shape Grammars to Images and Derived Tex- fort could be further developed as an interactive app tured Point Clouds.’, Proceedings of the 3rd ISPRS In- in the Destruction and Reconstruction of the Old Town ternational Workshop 3D-ARCH exhibit at Muzeum Warszawy (Museum of Warsaw), Knight, TW 1994, Transformations in design: a formal ap- proach to stylistic change and innovation in the visual located in Strona Dekerta, for visitors to be able to arts, Cambridge University Press, Cambridge, MA recreate their own Market Square and look a little Kuźnicki, K 2013, ’The Authenticity of the Reconstructed closer to compare it to the existing Market Square Old Town of Warsaw: A Reflection’, e-conservation, they see surrounding them. Journal 1(10.18236/econs1.201306), pp. 24-33 Ligler, H and Economou, A 2019 ’From drawing shapes to scripting shapes: Architectural theory medi- CONCLUSION ated by shape machine’, Proceedings of the Sympo- The resilience of Warsaw’s Old Town is an intriguing sium on Simulation for Architecture and Urban Design case to study how architectural style and character (SimAUD), Atlanta, p. 278–286 are tied to cultural identity and their reimagining in Murawski, M 2009 ’(A)political buildings: ideol- the visual arts - and therefore, greatly valuable for all ogy, memory and Warsaw’, Joint Conference of DOCOMOMO-International and The Architectural sorts of interpretations, appreciation, and education. Heritage Society of Scotland, Scotland In order to preserve and allow for the recreation of Müller, P,Zeng, G, Wonka, P and Van Gool, L 2007 ’Image- a regional architectural style at a micro and macro Based Procedural Modeling of Facades’, ACM Trans, level, the Market Square Grammar offers a first take p. 9 on how visual computations can be utilized to doc- Ripperda, N 2008 ’Determination of Facade Attributes ument the proportions and characteristic elements for Facade Reconstruction’, International Archives of Photogrammetry, Remote Sensing and Spatial Infor- of the simple facades along with their placements. mation Sciences, pp. 285-290 This formalized interpretation of an important her- Stiny, G 1977, ’Ice-Ray: A Note on the Generation of Chi- itage site enables the historic facades to be newly nese Lattice Designs’, Environment and Planning B: replicated. Furthermore, the research sets up a struc- Planning and Design, 4(10.1068/b040089), pp. 89-98 ture that allows for users of the grammar (and its po- Stiny, G 2006, Shape: Talking about Seeing and Doing, MIT tential implementations) to create their own varia- Press, Cambridge, MA Stiny, G 2011, ’What Rule(s) Should I Use?’, Nexus Journal: tions of re-envisioned facades with similar relations Architecture and Mathematics, 13, pp. 15-47 and parameters to the original ones. Future work will Stiny, G and Gips, J 1972 ’Shape grammars and the gen- aim to build on precisely how these potential interac- erative Specification of Painting and Sculpture’, In- tions can be fostered to support novel descriptions, formation Processing, p. 1460–1465 interpretations, and evaluations of the postwar re- [1] https://culture.pl/en/article/how-warsaw-came-clos construction in Warsaw. e-to-never-being-rebuilt [2] https://whc.unesco.org/en/list/30/ [3] https://warszawa.naszemiasto.pl/pomnik-jana-zach REFERENCES watowicza-odsloniety-architekta-odbudowanej/ar/c15- Dore, C and Murphy, M 2014, ’Semi-automatic Gener- 8167207 ation of As-Built BIM Facade Geometry from Laser [4] https://99percentinvisible.org/episode/episode-72- and Image Data’, Journal of Information Technology new-old-town/ in Construction, 19(2), pp. 20-46 [5] http://www.warszawska.info/srodmiescie/plany_uli c/rynek-plan.html

292 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Towards a Configurable Hybridity in Historic Preservation and Design

Mahyar Hadighi1 1Texas Tech University [email protected]

This paper fulfills the dual purpose of developing a systematic methodology for designing new constructions and adding to or revivifying existing buildings in historic neighborhoods and demonstrating an effective pedagogy in regard to historic preservation at the undergraduate level. A previously developed methodology for verifying and analyzing hybridity in architectural design is expanded as a foundation for designing an addition to a historic building in a famous urban context, i.e., a registered historic structure in the museum district of New York City. Shape grammar as a computational design methodology is used to analyze the historic fabric of the urban area and to create a configurable hybrid design that is both compatible with that context and reflective of the needs and design of the contemporary urban setting. The validity of shape grammar as a methodology for designing configurable hybrid constructions in historic neighborhoods and its effectiveness in relation to a teaching focus on historic preservation are considered through an analysis of projects from an upper-undergraduate-level architectural design studio. The students used the shape grammar methodology to analyze and understand historic contexts and features in order to generate new designs for the given context.

Keywords: Configurable hybridity, Historic preservation , Shape grammar, Neue Galerie, Vienna Secession

Introduction East Coast of the United States, the author observed Hybridity in architectural design, in general, is a mod- that the residential architecture produced by some of ern phenomenon. Although the International Style the faculty members practicing in the mid-twentieth architecture has been criticized for its alleged place- century in this specific context incorporate shapes lessness, many of the modernist architects and theo- and features from both European modern architec- reticians, especially outside Europe, the birthplace of ture and American traditional architecture. The au- modernism, adapted the principles of modernism to thor explored this hybridity between European mod- the context in which they practiced. In the process ern architecture and American traditional architec- of documenting examples of mid-twentieth-century ture in the work of William Hajjar, a member of Penn architecture in a selection of college towns on the State’s architecture faculty in the mid-century period

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 293 who practiced in State College where Penn State’s impact; and (5) aspects of the social and technologi- main campus is located. And, through this investi- cal context were identified to explain such influences. gation, the author developed a systematic method- To test the effectiveness of the proposed ology for identifying, explaining, and re-creating hy- methodology for generating designs appropriate bridity in architectural design. to historic neighborhoods especially in the peda- The questions of how to design new construc- gogy of historic preservation, the steps described tions and how to add to or revivify existing buildings were explained to students in an upper-level under- in historic neighborhoods have always met with con- graduate architecture studio who were then asked troversial responses in terms of the extent to which to (1) perform a contextual analysis; (2) study code any new design should reflect or contrast with the and zoning; (3) understand the needs of contem- characteristics of the context. Finding a way to ap- porary life in terms of analyzing the architectural proach these questions is especially challenging for program; (4) develop their own rules or modify the students in a design studio environment where they rules previously developed for Hajjar’s grammar or are presented with the challenge of designing for a the grammar of his likely influences, based on the historic neighborhood. In the present paper, the sys- first three previous steps with the goal of generating tematic methodology developed to identify and ex- configurable hybrid designs appropriate for an addi- plain hybridity in architectural design is modified in tion to an existing historic structure; and (5) use their order to serve as a system for the design of config- architectural intuition and program requirements to urable hybrid designs in historic settings. The theo- develop a design generated by the grammar. retical outcomes highlight the effectiveness of shape grammar as a computational design methodology Shape Grammar not only in terms of verifying and describing hybrid- Shape grammars in computation are a specific class ity in architectural design and generating designs ap- of production systems based on an initial shape, or a propriate to historic neighborhoods but also in rela- set of finite shapes, and transformational shape rules tion to achieving an effective pedagogy of historic (Stiny & Gips, 1971). Since the 1970s, as a compu- preservation. tational design theory, the concept of shape gram- mar has been used in architectural analysis when a Methodology pattern of design characteristics or a stylistic repe- The present paper is part of a larger-scale project in tition of shapes in architecture is evident. This the- which shape grammar is used to verify, describe, and ory has been used in relation to many examples of generate hybridity in architectural design, with Haj- historical architecture, such as the Palladian villas as jar’s architecture featured as a case study showing hy- analyzed by Stiny and Mitchell (1978), Frank Lloyd bridity between European modern architecture and Wright’s Prairie houses as analyzed by Koning and American traditional architecture. The steps taken to Elizenberg (1981), bungalow houses as analyzed by identify and analyze the hybridity in Hajjar’s work are Downing and Flemming (1981), Queen Anne houses as follows: (1) Hajjar’s life and practice were traced as analyzed by Flemming (1987), and Alvaro Siza’s to identify likely influences on his work; (2) a shape houses at Malagueira as analyzed by Duarte (2005). grammar was developed for the houses he designed In the past three decades, the idea of compar- and built in State College, PA; (3) the grammars of ing grammars and the notion of stylistic evolution some of the likely influences on his work were iden- have been explored in multiple ways in the litera- tified and developed; (4) Hajjar’s grammar was com- ture. For example, such comparisons have been inte- pared and contrasted with the grammars of these in- gral to endeavors to explain transformation in design fluences to determine the nature and extent of their (Knight, 1993) and the notion of composite gram-

294 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 mar (Chase & Ahmad, 2005), to investigate the adap- A Shape Grammar for Hybridity tation of existing traditional houses to new require- In a previous paper, the life and architecture of Abra- ments (Eloy & Duarte, 2011), to determine Alberti’s ham William Hajjar were traced given his career as a influence on Portuguese architecture (Figueiredo et faculty practitioner at Penn State University produc- al., 2014; Kruger et al., 2011), and, most recently, to ing residential architecture in the local area (Hadighi generate a generic housing grammar (Benros, 2018). & Duarte, 2018). Hajjar (1917-2000) studied archi- As part of the larger-scale study, in previous papers, tecture at the Carnegie Institute (now Carnegie Mel- shape grammar was used to identify, describe, and to lon) (1936-1940) and at the Massachusetts Institute some extent generate hybridity in architectural de- of Technology (MIT) (1940-1941). He took a faculty sign (Hadighi & Duarte, 2018, 2019, 2020). In the position at the Pennsylvania State College (now the present paper, however, the methodology is used Pennsylvania State University or simply Penn State) in to design hybrid constructions in a historic area and 1946. From 1952, when the first house he designed its effectiveness in relation to historic preservation for State College was built in the area, until 1963, teaching is examined. when he moved to Philadelphia on a leave of ab- sence from Penn State to work with a friend from MIT, Hajjar designed and built thirty-three single-family Figure 1 houses in the Penn State area. Key to his success in Comparison of local residential work was his use of shapes, rules, selected rules of the elements, and principles from both European mod- three grammars ernism and American traditional architecture, the lat- ter of which was the dominating style in the context. To identify and describe this hybridity between Euro- pean modernism and American traditionalism, three grammars were created: (1) a grammar for Hajjar’s single-family houses in the Penn State area (Hadighi & Duarte, 2018); (2) a grammar for the work of his major influences from European architecture, Walter Gropius and Marcel Breuer, both of whom Hajjar had got to know during his year at MIT and via a studio- type arrangement between MIT and Harvard’s Grad- uate School of Design in which students, including Hajjar, from both institutions worked together and benefited from critiques given by Gropius, Breuer, and other faculty (Hadighi & Duarte, 2019); and (3) a grammar for traditional houses in the context in which Hajjar taught and practiced (Hadighi & Duarte, 2019). All three grammars were compared and con- trasted to identify the extent to which Hajjar’s ar- chitecture was influenced by European modernism- specifically by the work of Gropius and Breuer-and the American traditional architecture of the context (Hadighi & Duarte 2020). Figure 1 shows a com- parison between the rules of the three grammars

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 295 stated. Although the proposed methodology was Hybrid Grammar in Historic Preservation developed to meet the central goal of analyzing Design and Pedagogy the qualities of Hajjar’s architectural language to de- As noted, one of the challenges in designing a con- termine how it reflects the influences of traditional textualized building, especially in teaching a design American architecture and European modern archi- studio, is that of developing a design that is both tecture (the latter represented by the Gropius-Breuer compatible with the context-in terms of form, ge- partnership), the shape grammar methodology pro- ometry, and architectural language-and reflective of vided a way to quantify these influences. Figure 2 present-day lifestyles and architectural trends. This is shows the percentage of Hajjar’s architectural lan- even more challenging when the new design is for a guage that reflects influences from and similarities historic neighborhood or an addition to a registered between the grammars. historic building. Understanding the effectiveness of It remains important to understand that al- shape grammar as a design methodology in identify- though similarities and differences between the two ing and describing hybridity in architectural design, grammars reflect similarities and differences be- the author used this methodology to generate con- tween the work of two architects or two architectural figurable hybridity in an upper-level undergraduate styles, they also relate to the respective lifestyles of historic preservation and design studio at Texas Tech the two periods. For instance, the size and propor- University College of Architecture in which eleven tion of the interior spaces, such as the kitchen, bed- senior-level students participated. rooms, and living room in designs generated by Ha- Titled “The Neue Galerie, New York: Understand- jjar’s grammar and the grammar for traditional ar- ing the Past; Envisioning the Future,” the studio chitecture may relate to a change in family size be- project was to design an addition to the privately tween the two periods, to a change in social relation- owned Neue Galerie in New York City. Founded in ships between co-workers in the mid-century United 2001, the Neue Galerie is located at 1048 Fifth Av- States, and/or to a change in the role of women in enue, at the intersection of Fifth Avenue and 86th these social relationships. Street, in an area known as Museum Mile. This mile- long stretch of Fifth Avenue is home to some of the most well-known museums in the world, including

Figure 2 Similarities between the three grammars in a diagram

296 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 the Guggenheim and the Metropolitan Museum of tute for Jewish Research. The current owners (and co- Art. The defining goals of the studio project were founders of the gallery), Ronald S. Lauder and Serge to create a design that expressed the importance of Sabarsky, purchased the property in 1994 and com- the historical context and demonstrated a nuanced missioned Annabelle Selldorf, founding principal of understanding of the historical structure of the exist- New York City-based architecture practice, Selldorf ing Neue Galerie in order to secure its integrity for Architects, to renovate and rehabilitate to render it New York City into the future. Designed by Carrère a worthy of the Neue Galerie (Figure 3). The archi- & Hastings, a prominent firm that had also designed tect’s work “involved restoring 1048 Fifth Avenue to the New York Public Library, the existing Neue Ga- its original state, while adapting it to the most strin- lerie building has been designated a landmark by the gent museum standards with regard to the display New York Landmarks Commission and is generally and preservation of works of art” (neuegalerie.org considered one of the most distinguished buildings Mission Statement). ever built on Fifth Avenue. As stated on the Neue Galerie website, the original building, commissioned by industrialist William Starr Miller, has been put to many uses, including as the residence of Mrs. Cor- nelius Vanderbilt III and as home to the YIVO Insti-

Figure 3 Neue Galerie New York, north elevation (as seen from 86th Street)

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 297 take all the issues commonly considered in any de- Figure 4 sign studio into account, such as the concept, materi- Portrait of Adele ality, lighting strategies, spatial organization, and en- Bloch-Bauer by vironment, it required them to do so with special con- Gustav Klimt sideration afforded to context, historical importance, footfall, and relationship with the existing building.

Figure 5 A computational system developed by a student (Jonathan Gonzalez) to identify the key features of Gustav Klimt’s Portrait of Adele Bloch-Bauer. The system divided the portrait into a grid and distorted it The Neue Galerie collection encompasses a range The students proceeded to take all the usual steps depending on the of media, including painting, sculpture, works on of performing a site analysis and a precedent analy- location of the paper, decorative arts, and photographs created sis, considering the historical background, determin- section of the in Austria and Germany between 1890 and 1940. ing the zoning requirements and applicable codes, portrait. This The collection-especially the Austrian material- and understanding the design guidelines. However, diagram created a emphasizes the special relationship between the fine to create a foundation for their design in light of the variation of voids and decorative arts mainly in Vienna circa 1900. The demanding nature of the challenge given its empha- and curves. collection includes work by major artists practicing sis on history and compatibility-specifically to help fine arts such as Gustav Klimt, Egon Schiele, Oskar them achieve a better technical understanding of the Kokoschka, Alfred Kubin, and Richard Gerstl, whereas influences already in play-the students were intro- the decorative arts are represented by pieces from duced to the theory and application of shape gram- the Wiener Werkstätte (Vienna Workshops) and the mar as a computational design methodology. Stu- designs of such well-known figures as Josef Hoff- dents, individually, developed rules based on the mann, Koloman Moser, and Dagobert Peche. The historical context-particularly based on the historical architects Adolf Loos and Otto Wagner are also rep- design guidelines. Simultaneously, to reflect the con- resented. Figure 4 shows Klimt’s Portrait of Adele tent of the building, new rules were developed based Bloch-Bauer, one of the most well-known artworks- on the formal analysis of the Austrian art known as and even considered the most important piece-in Vienna Secession (Figure 5). The rules were then iter- the Neue Galerie’s collection. ated based on the rules of the grammars developed For the studio project, students were set the chal- in previous work focused on hybridity in Hajjar’s resi- lenge of designing a contemporary building addition dential work as described (Figure 6). compatible with the existing historic structure on a prominent site within a dense historic context. Thus, not only did the assignment require the students to

298 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 6 Final Design Another student After developing a configurable hybrid grammar, (Ryan Reyes) used a each student generated twelve to twenty-four de- three-part strategy signs (Figure 7). This process was performed manu- to design the ally in some cases and in other cases by developing addition using grasshopper scripts based on the developed gram- space, structure, mar. The final result of the students’ work, how- and skin. Using the ever, is not configurable: each student chose a de- developed rules, he sign among twenty-four generated designs and de- generated designs veloped it to become a final product. The design for the structure selected was based on the individual student’s ar- and the skin. chitectural intuition, a program analysis, and com- ments received from the studio instructor and other reviewers (Figure 8). In an ideal world with more time in the studio, the final result could be chosen/pro- duced through a process of computational optimiza- tion. This should be a future development of the pedagogical methodology described in the present paper. In the successful cases in the studio, the fi- nal product is absolutely contemporary following the present-day architectural trends, while at the same time compatible-in terms of geometry and design significance-with the historical context.

Discussion As part of a larger-scale study, an analysis of Hajjar’s architectural language including through a compar- ison with the architectural language of Gropius and Breuer and the American traditional architecture of the local context was performed. Through that com- parison analysis, shape grammar was shown to be an effective computational design methodology for verifying and describing hybridity in architectural de- sign. The focus of the present paper, however, was to demonstrate that through further development this methodology will become a highly effective system- atic methodology for the production of hybrid de- signs in historic contexts as demonstrated in a preser- vation design studio setting. Traditionally, shape grammar has been used to analyze examples of his- torical architecture related to a particular architect, architectural style, or period, It can also be used to generate hybrid designs as shown herein-i.e., designs

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 299 Figure 7 Examples of designs generated by a shape grammar developed by a student (Josh Ruiz) in relation to program analysis. The final design (right) was chosen based on the program analysis.

Figure 8 The final product developed by a student (Ryan Reyes) based on the configurable hybrid grammar.

300 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 that are compatible with the context-particularly by of Housing: The Grammar of Siza’s Houses at reflecting the historic design guidelines-and are at Malagueira’, Environment and Planning B: Planning the same time contemporary and so compatible with and Design, 32, p. 347–380 Eloy, S and Duarte, JP 2011, ’Transformation grammar for present-day lifestyles, building codes, zoning regula- housing rehabilitation’, Nexus Network Journal, 13, tions, and architectural trends (Figure 9). pp. 49-71 Figueiredo, B, Sousa, L, Duarte, JP and Kruger, M 2014, Figure 9 ’Alberti Digital on Portuguese Architecture: shape A section of the grammar transformation as a computational frame- final product work to determine the influence of Alberti legacy on Portuguese Renaissance churches’, Joelho, 5, pp. 43- developed by a 51 student (Laura Flemming, U 1987, ’More Than the Sum of Parts: The Chavez) Grammar of Queen Anne Houses’, Environment and representing the Planning B: Planning and Design, 14, pp. 323-350 relationship Hadighi, M and Duarte, JP 2018 ’Adapting Modern Ar- between the chitecture to a Local Context: A Grammar for Ha- jjar’s Hybrid Domestic Architecture’, Proceedings of historic building eCAADe 2018, Lodz, pp. 515-524 and the new Hadighi, M and Duarte, JP 2019 ’Using Grammars to addition. Trace Architectural Hybridity in American Mod- ernism, The Case of William Hajjar Single-Family House’, Proceedings of eCAADe 2019, Porto, pp. 529- 540 Hadighi, M and Duarte, JP 2020 ’Tracing Hybridity in Lo- The design studio’s focus was on hybridity through cal Adaptation of Modern Architecture: The case of a formal expression of architectural languages, ge- A. William Hajjar’s Single Family Architecture’, Ninth International Conference on Design Computing and ometries, shapes, features, and building code and Cognition (DCC’20), Atlanta zoning regulations. However, it is certainly reason- Hadighi, M and Duarte, JP 2020a ’Local Adaptation of the able to expect the socio-economic context and build- International Style: Contextualizing Global Architec- ing technology available to be important in deter- ture between East and West’, Proceedings of eCAADe mining hybridity in architectural design. 2020, Berlin, pp. 331-340 Hadighi, M and Duarte, JP 2019, ’Bauhaus Interna- tionalism to College Town Modernism: Exploring REFERENCES Bauhaus Culture in Hajjar’s Hybrid Architecture’, in Benros, D 2018, A generic housing grammar for the gener- Lee, JH (eds) 2019, Computer Aided Architectural De- ation of different housing languages: a generic hous- sign “Hello, Culture”, Springer, Daejeon, pp. 429-443 ing shape grammar for Palladian villas, Prairie and Knight, T 1983, ’Transformations of languages of de- Malagueira Housesdapting Modern Architecture to a signs: part 3’, Environment and Planning B: Planning Local Context: A Grammar for Hajjar’s Hybrid Domes- and Design, 10, pp. 155-177 tic Architecture, Ph.D. Thesis, London’s Global Univer- Koning, H and Elizenberg, J 1981, ’The Language of the sity Bartlett School of Graduate Studies Prairie: Frank Lloyd Wright’s Prairie Houses’, Environ- Chase, S and Ahmad, S 2005 ’Grammar transformations: ment and Planning B, 8, pp. 259-323 using composite grammars to understand hybrid- Kruger, M, Duarte, JP and Coutinho, F 2011, ’Decoding de ity in design with an example from Medieval Islamic re aedificatoria: using grammars to trace Alberti’s in- courtyard buildings’, Proceedings of CAAD Futures, fluence on Portuguese classical architecture’, Nexus pp. 89-98 Network Journal, 13, pp. 171-182 Downing, F and Flemming, U 1981, ’The Bungalows of Stiny, G and Gips, J 1971 ’Stiny, G., & Gips, J. (1971). Shape Buffalo’, Environment and Planning B, 8, pp. 269-293 Grammars and the Generative Specification of Paint- Duarte, JP 2005, ’Towards the Mass Customization ing and Sculpture. IFIP Congress.’, Proceedings of IFIP

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 301 Congress Stiny, G and Mitchell, J 1978, ’The Palladian Grammar’, En- vironment and Planning B, 5, pp. 5-18 Çolakoğlu, B 2005, ’Design by grammar: an interpreta- tion and generation of vernacular Hayat houses in contemporary context’, Environment and Planning B: Planning and Design, 32, pp. 141-149 [1] https://www.neuegalerie.org

302 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Computational Generation of Hutong Neighbourhood Maps A shape grammar for procedural modelling to represent the historical Beijing urban structure

Yuyang Wang1, Asterios Agkathidis2, Andrew Crompton3 1,2,3University of Liverpool 1,2,3{Y.Wang179|a3lab|Andrew.Crompton}@liv.ac.uk

The Hutong neighbourhoods, composed of Chinese courtyard dwellings (Siheyuan), are historically and socially significant urban spaces that embody traditional Chinese philosophy. There is a global interest in these spaces many of which face oblivion, this study presents a formal grammar for parametrically generating Hutong neighbourhoods, which replicates the morphological characteristics of historic cases. This research investigates traditional principles of urban planning of ancient Beijing, based on examples on the historical map Qianlong Jingcheng Quantu, to derive the lost design rules. These rules are then used to build up a procedural modelling process, which reveals the development of Beijing urban structure from Yuan (1271-1368) to Qing (1368-1911) dynasty. The grammar presented in the procedural modelling process contributes to the understanding of the generation of Hutong neighbourhoods. In support of traditional Chinese urban design, the grammar will be translated into an algorithm to be implemented in a computational environment by visual scripting that will generate new instances of Hutong neighbourhoods.

Keywords: urban morphology, Siheyuan neighbourhood, procedural modelling, shape grammar

BACKGROUND formed due to population increase, after which they Hutong neighbourhoods are historically and socially were composed of variants of a vernacular dwelling significant part of Chinese urban heritage that em- type- Chinese courtyard housing (Siheyuan). Since bodying traditional Chinese philosophy. The ur- these neighbourhoods usually existed between two ban area of ancient Beijing was divided into four- Hutong (alley), they were called Hutong neighbour- sided sectors as residential neighbourhoods under hoods, whose basic units are Siheyuan. A typical Si- the constraint of an orthogonal grid system. Dur- heyuan consists of axially aligned courtyards sym- ing the Qing dynasty, these neighbourhoods were re- metrically surrounded by individual buildings, con-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 303 nected by orthogonally located corridors, walls, and yards of Siheyuan into five categories to understand gates, as illustrated in Figure 1. how the different Siheyuan layouts were deformed to adapt themselves in the various site contexts within the Hutong neighbourhoods (Figure 2). Figure 1 One common conclusion of these studies is that Beijing Siheyuan, the rules underlying the evolution of Hutong neigh- Ma, 1999. bourhoods are complex. However, computational approaches are powerful to make generative rules explicit and to develop the design iterations. The Shape Grammar, invented by George Stiny (1980) to interpret the generative rules of objects graph- ically, was implemented by Rudi Stouffs (2018) us- ing algorithms developed in Rhino/Grasshopper en- vironment. An analytical shape grammar has its ad- vantage in straightforwardly demonstrating the mor- phological iteration of shapes, which was employed It is noted that current architects lack physical ex- to explain historical styles or languages of designs in amples to study their design knowledge. Duan’s procedural modelling approaches, such as ‘Rome Re- (2007, 2016) measuring surveys indicate most his- born 2.0’ project (Dylla et al., 2008) and urban blocks torical Siheyuan and Hutong neighbourhoods have of nineteenth-century Camden London Flora et al. been damaged and even destroyed. Meanwhile, it is (2018). noted there is an increasing interest in Hutong neigh- Figure 2 bourhood and Siheyuan, but, due to the linguistic A section of difficulties in understanding historical literature, it is Qianlong hard for architects and urban designers to learn the Jingcheng Quantu original design knowledge of traditional Chinese ar- (Qianlong Capital chitecture (Di, 2016). To explicate the lost historical Map, 1750) design principles for contemporary architects and ur- indicating the ban designers and provide a tool for rapidly gener- urban planning of ating Hutong neighbourhoods replicating the tradi- historical Beijing is tional morphological characteristics may help them based on a grid in their design work. system. Since the Hutong neighbourhoods were com- posed of Siheyuan dwellings containing a large num- ber of variants, morphological studies of the neigh- bourhoods were linked to the typological analysis of Siheyuan. Li (2009) revealed the principles explain- ing how the archetype of Siheyuan evolved variants that compose Hutong neighbourhoods on Beijing’s AIMS, METHODS, AND MATERIALS orthogonal grid. Similarly, Liu (2019) investigated Our overall aim is to investigate the design rules many Siheyuan precedents within five Hutong neigh- underlying historical Hutong neighbourhood design bourhoods drawn on Qianlong Jingcheng Quantu in order to translate them into an algorithmic tool (Qianlong Capital Map, 1750) by classifying the court- to generate variations of traditional Beijing Hutong neighbourhoods. To achieve this aim, the following

304 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 questions are addressed: entation, as demonstrated in Rule 2a in Figure 3. We defined these neighbourhoods as ‘Type A Neigh- • How could the historical rules of Hutong neigh- bourhood’ (TAN). It is noted that the form of some bourhood generation and development be neighbourhoods was re-planned in the Qing dy- translated into a shape grammar to simulate nasty. In the south-north orientation, two neigh- its procedural modelling? bourhoods were combined as a whole and were di- • How could such a shape grammar be imple- vided into three new neighbourhoods, with an al- mented as an interactive algorithmic tool in ley width of 10.01 metres between each of the two a computational environment to generate Hu- adjacent new neighbourhoods as shown in Rule 1 tong neighbourhood variants replicating their in Figure 3. We defined these new neighbourhoods original morphological characteristics? as ‘Type B Neighbourhoods’ (TBN). The neighbour- The current paper is focusing on the first question. hoods planned in the Yuan dynasty (TAN) were di- The derivation of rules come from historical litera- vided into 67.67x67.67m Siheyuan plots. Although ture (E, 1739, and Song, 1369) and secondary sources the development of neighbourhoods in the Qing dy- from other studies, such as the morphological studies nasty changed the depth of both Siheyuan plots and on Hutong neighbourhoods by Li (2009), Li and Wang neighbourhoods, the width remained. Rule 2b repre- (2006), Liu (2019), and Ni (2009). The Qianlong Cap- sents the division of a TBN into Siheyuan plots. ital Map recorded the urban plan of Beijing around 1750 at a scale of 1: 650, including all the Hutong Defining Siheyuan plots neighbourhoods and Siheyuans they contained. This Division of plots into sub-plots. A Siheyuan plot in map is also used for rules derivation. the Yuan dynasty could be divided into two to four sub-plots in the east-west orientation. Each sub-plot THE HUTONG NEIGHBOURHOOD GRAM- could include several courtyards aligned in south- north orientation (Type A sub-plot, TASP) or an empty MAR space (Type B sub-plot, TBSP). Based on this finding, By observing the Qianlong Capital Map, it is noted Li and Wang’s (2006) investigation of examples on that there were a large number of Hutong neigh- the Qianlong Capital Map concluded on several types bourhood variants, which resulted in the diverse ur- of sub-plots. As shown in Rules 3 in Figure 4, there ban structure of ancient Beijing within its orthog- are six dividing patterns, in which the plot is consti- onal grid system. We chose the most representa- tuted of one or two sub-plot types. We found that tive the east-west oriented Hutong neighbourhoods, the width of most TBSP is within the range or much from the Yuan dynasty to be our case study. smaller than TASP. Therefore, we define the width of TASP ranges from 12 to 35 metres and TBSP from 5 to Generation of Hutong neighbourhoods 12 metres. The six division patterns of the plots are plan and Siheyuan plots defined as Rules 3a to 3f respectively. In the Yuan dynasty, each neighbourhood’s depth (length in south-north orientation) was planned to Sub-plot combination. Some TASPs and TBSPs were be 67.76 metres and its width (length in east-west combined to regenerate new sub-plots. According orientation) was 677.6 metres. The Hutong alley be- to Liu’s investigation (2019) of selected Siheyuan ex- tween two adjacent neighbourhoods was 9.24 me- amples shown on the map, we inferred the principles tres wide. These key urban elements and dimen- that one TASP could be combined with one adjacent sions are determining the initial shapes of the Hu- TBSP to become a combined sub-plot, called Type AB tong grammar. Each neighbourhood was averagely sub-plot (TABSP). Then the TABSP could be used to divided into ten plots aligned in an east-west ori- construct a Siheyuan. This combination is defined as

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 305 Figure 3 Rule 1 trasnforms TAN into TBN and Rule 2 divides TAN/TBN into plots.

Rule 4a and Rule 4b, shown in Figure 4. Since the TAB- south orientation, constrained by the depth of the SPS are wider than other types, it enables a Siheyuan neighbourhood, is 3 or 4 in most cases. For simpli- to be built on it and contains two courtyards in the fication, we assume it is 4 in TAN and 3 in TBN. The east-west orientation. Normally, on a TABSP, a set of number in the east-west orientation, constrained by courtyards are aligned in a south-north orientation, the width of the sub-plot, is 2 or 3 mostly. We define it with a side courtyard next to these courtyards in par- as 3 if the width is bigger than 20 metres, otherwise allel and connecting the south and north boundaries. 2. Rule 5 is defined as the disaggregation of TABSP Sub-plot disaggregation and recombination. The or TASP. Considering the neighbourhood depth and Siheyuan with a side courtyard usually belonged type of sub-plot, the rule includes variants such as to rural upper-class owners. In many cases, the Rule 5a, Rule 5b, Rule 5c, and Rule 5d. The recombi- TABSP disaggregated into many smaller sub-plots, nation of two TABdSPs is defined as Rule 6a and two called a Type ABd sub-plot (TABdSP), on which a TAdSPs is defined as Rule 6b. Except for the above one-courtyard Siheyuan is constructed. The disag- disaggregation, there was another mode that a TASP gregation may also happen to some TASPs, whose separates into two Siheyuans by dividing the bound- disaggregated sub-plot is called a Type Ad sub-plot ary of the second and the third courtyard, in which (TAdSP). Two Type Ad sub-plots or two Type Abd the original type of each courtyard remains. This is sub-plots adjacent in north-south orientation may defined as Rule 7. We call the separated sub-plot on be recombined to become a new sub-plot, called the south side as Type ASS sub-plot (TASSSP) and the Type Ad2 sub-plot (TAd2SP) and Type ABd2 sub-plot one on the north as Type ASN sub-plot (TASNSP). The (TABd2SP) respectively. In the disaggregation, the rules are shown in Figure 4. number of generated TAdSPs or TABdSPs in a north- These Siheyuans built on disaggregated sub- plots are clustered, in which many Siheyuans are not

306 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 4 Rule 3 divides a plot into sub-plots; Rule 4 and Rule 6 re-combine sub-plots; Rule 5 and Rule 7 disaggregate sub-plots; and Rule 8 shrink sub-plot edges to generate an alley.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 307 adjacent to the urban fabric. To create access to these on a TAdSP or TABdSP usually contains one courtyard Siheyuans, the west and east edges of disaggregated (defined as Rule 11a), on which no shape iteration sub-plots shrink to give space to generate a small happens on the plan view, while on a TAd2SP or a south-north oriented alley. The alley may cross the TABd2SP two non-standard courtyards connected in neighbourhood to connect the alleys on the south a north-south orientation, is defined as Rule 11b. Si- and north side of the neighbourhood or not cross. To heyuan built on a TASSSP always divides the site into simplify, we assume that an alley crosses the neigh- two courtyards: a standard one on the north and a bourhood in all cases. In the shrink, as defined in Rule non-standard one on the south. This iteration is de- 8, the movement distance of each edge is different, fined as Rule 12a. On the contrary, on a TASNSP the which makes the alley geometrically irregular. division may allocate a non-standard courtyard on the north and a standard one on the south, which is Courtyard division and type definition defined as Rule 12b. The type of sub-plot determines After the sub-plot type is defined, it is divided into the courtyard division by defining courtyard types courtyards depending on the Siheyuan to be built on and numbers, which, consequently, determines the the sub-plot. When describing the word ‘courtyard’ type of the Siheyuan. in the Siheyuan context, it usually means the space enclosed by walls, which includes both the open Moving the edges of the courtyard courtyard space and all the buildings surrounding In the initial neighbourhood planned in the Yuan dy- that space. In many cases, some parts of the ‘court- nasty, the south and north edges of each Siheyuan yard’ are not completely enclosed by walls. Instead, plot were ideally divided into segments from the cor- the rear boundary of a building is extended in east- responding edges of the neighbourhood, which are west orientation to define the boundary of a court- straight lines, enabling the elevations of Siheyuans yard. Borrowing the idea of categorizing courtyard arrayed smoothly. After the reconstruction in the into types based on the courtyard location, the room Qing dynasty, the edges of many Siheyuans have type that a courtyard contains (Wang et al., 2019), been moved to a south-north orientation, which we classify courtyard types as northern boundary caused the courtyards adjacent to the urban fab- courtyard (NBC), southern boundary courtyard (SBC), ric in south-north orientation to extend or shrink. east side courtyard (ESIC), west side courtyard (WSIC), The shrinking and extension are defined as Rule standard courtyard (SC), and non-standard courtyard 13 and Rule 14 respectively. The movement of (NSC). A TASP is used to build a Siheyuan with 3 or edges also took place to the east and west edges of 4 courtyards aligned in a north-south orientation, non-standard courtyards and boundary courtyards. in which the southernmost and northernmost ones These edges could move in east or west directions to are non-standard courtyards while the middle one or extend or shrink the courtyard area. Since Siheyuans two are standard courtyards. The only difference be- were adjacently aligned in the east-west, this move- tween as TASP a TABSP is that a side courtyard is ad- ment consequently influences adjacent Siheyuans. ditionally aligned to the east or west. Generation of In this movement, the east edge of the courtyard a three-courtyards Siheyuan on a TASP is defined as moves to the east or the west edge moves to the west Rule 9a and a four-courtyards one as Rule 9b. Three to extend the courtyard area, and the correspond- courtyards with an east side courtyard on a TABSP is ing adjacent Siheyuan’s edge moves accordingly to defined as Rule 10a, three courtyards with a west side shrink its area. This took place in courtyards adjacent courtyard as Rule 10b, four courtyards with an east alleys. The movement is defined as Rule 15, as shown side courtyard as Rule 10c, and four courtyards with in Figure 6. a west side courtyard as Rule 10d. A Siheyuan built

308 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 5 Rule 9-12 divide a sub-plot into courtyards.

Figure 6 Rule 13-14 defines shrink or extension of a boundary courtyard in north or south and Rule 15 defines the shared edge movement of two adjacent courtyards in east-west.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 309 Figure 7 Rule 16-20 defines room layout patterns.

Figure 8 Rule 21 defines room movement; Rule 22 defines room rotation; and Rule 23 defines entry gate location patterns.

Room layout pattern west boundary. A standard courtyard normally con- Borrowing the approach of Wang et al. (2020) to cat- tains a primary room on the north and two secondary egorize courtyard type, the room layout patterns on rooms pairwise axisymmetric about the courtyard’s the courtyard are identified based on two criteria: central axis. This layout contains more variants re- courtyard location (on the southernmost, on the mid- garding the criteria of the existence of wing rooms dle, and on the northernmost) and the room type(s) and secondary wing rooms. Secondary room com- the courtyard contains. Specifically, a boundary ponents have not been considered in this research courtyard normally contains an east-west crossing due to their low importance in Siheyuan. For non- row room, which is located on the south if the court- standard courtyards, the determination of the room yard lies on the south boundary while it is located on layout was improvised by craftsmen. We concluded the north if the courtyard lies on the north bound- on eight variants by observing examples on the Qian- ary. Boundary courtyards may also contain one or long Capital Map. The generation of room layout pat- two secondary rooms. Similarly, a side courtyard con- terns in the plan view is defined as Rules 16-20, as tains a south-north crossing row room, which locates shown in Figure 7. on the east if the courtyard lies on the east boundary while it locates on the west if the courtyard lies on the

310 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 9 The procedural modelling process of two Hutong neighborhood examples.

Location of building elements means the rooms could be slightly moved and ro- The algorithmic underlying Gongcheng Zuofa Zeli tated. The movement is defined as Rule 21 and the (Structural Regulation, 1734) defines a parametric sys- rotation is defined as Rule 22, as shown in Figure 8. tem that generates variants of single buildings, many The location of the Siheyuan entry gate is deter- of which are the rooms in a Siheyuan. Once the mined by the urban fabric. For most Siheyuans it is rooms are generated, they are located according to located in the southeast of the front courtyard. If the layout patterns shown in Figure 7. The constraint the south edge of the site is not connected to urban of the layout pattern to room location is loose, which space such as a Hutong alley or a street, the entry

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 311 gate is located elsewhere. There is a sequence to rank Dylla, K, Fisher, B, Muller, P, Ulmer, A and Haegler, S the priority of four patterns of entry gate location, re- 2008, ’Rome Reborn 2.0: A Case Study of Virtual City garding the urban fabric context. The sequence is Reconstruction Using Procedural Modeling Tech- niques’, Computer Graphics World, 16(2008), pp. 62- east of south edge> south edge of east edge> south 66 of west edge> west of north edge. Four patterns of E, E 1739, Ba Qi Tong Zhi [The eight banners annals], Qing location of an entry gate are defined as Rule 23a, Rule dynasty of China, Beijing 23b, Rule 23c, and Rule 23d, as shown in Figure 8. Flora, R, Polly, H and Andrew, H 2018, ’The Use of Histori- Once the rules are captured, the grammar for its cal Data in Rule-Based Modelling for Scenarios to Im- procedural modelling is constructed. The Figure 9 prove Resilience within the Building Stock’, The His- toric Environment: Policy & Practice, 9(3-4), pp. 328- shows the procedural modelling process of two Hu- 345 tong neighbourhoods, which includes 12 steps. Li, J 2009, ’Qianlong Jingcheng Quantu Zhong de Heyuan Jianzhu Yu Hutong Jiefang Kongjian Tanjiu DISCUSSION AND CONCLUSION [research of courtyard building and Hutong neigh- borhood modes on the Qianlong Capital Map]’, Jour- Re-visiting historical design with a computational nal of China Architectural History, 2, pp. 249-282 perspective can inspire us not just to design the con- Li, D 2016, Computational Re-interpretation of Heritage temporary world by exploring new forms but also to Architecture, Ph.D. Thesis, University of Liverpool inherit traditional architectural knowledge. This pa- Liu, D 2019, ’An Intensive Analysis of the Typical Ur- per revealed the mathematical considerations em- ban Micro-Fabrics of Beijing in Middle Qing Dynasty: bedded in the generation of Siheyuan neighbour- Taking the Area in Northeast of the East Four Arch- ways in Qianlong Jingcheng Quangtu for Example’, hoods by a shape grammar corresponding to their in Jia, J (eds) 2019, Jianzhu Shi [Architectural History] original generative rules, which grasped the essence Vloume 43, China Building Industry Press, Beijing, of understanding their historical iteration. In future pp. 130-152 work, the rules of the proposed grammar will be im- Ma, B 1999, Beijing Siheyuan Jianzhu [Beijing Siheyuan plemented in Rhino 3D/Grasshopper environment to Building], Tianjin University Press, Tianjin procedurally generate the virtual three-dimensional Ni, Y 2009, Study on Typology of Beijing Hutong Siheyuan, China Building Industry Press Hutong neighbourhood models. The Grasshopper Song, L (eds) 1369, Yuan Shi [history of Yuan dynasty], script, in association with the grammar, is a new way Ming Dynasty of China, Beijing to understand Hutong neighbourhoods. Meanwhile, Stiny, G 1980, ’Introduction to shape and shape gram- in support of Hutong-style neighbourhood design, mars’, Environment and Planning B: Planning and De- the script could be a digital tool to generate neigh- sign, 7(3), pp. 343-351 bourhood models. Stouffs, R 2018 ’Where Associative and Rule-based Approaches Meet a Shape Grammar Plug-in for Grasshopper’, Proceedings of the 23rd CAADRIA, pp. REFERENCES 453-462 Anon, Anon 1733, Gongcheng Zuofa Zeli [Structural Reg- Wang, Y, Agkathidis, A and Crompton, A 2020, ulations]., Qing Dynasty of China, Beijing ’Parametrising historical Chinese courtyard- Chan, C and Xiong, Y 2007, ’The features and forces dwellings: An algorithmic design framework for the that define, maintain and endanger Beijing court- digital representation of Siheyuan iterations based yard housing’, Journal of Architectural Planning Re- on traditional design principles’, Frontiers of Archi- search, 24 (1), pp. 42-64 tectural Research, 9(9), pp. 751-773 Duan, B 2007, Beijing Hutong Zhi [Beijing Hutong Chroni- cle], Beijing Press, Beijing Duan, B 2016, Beijing Siheyuan Zhi [Beijing Siheyuan Chronicle], Beijing Press, Beijing

312 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Residential Structures for the Elderly Transformation Grammar

Filipe Guterres1, Filipe Coutinho Quaresma2 1,2Universidade Lusófona de Humanidades e Tecnologias [email protected] [email protected]

The increment of the average life expectancy and the birth rate reduction in developed countries generates invariably a massive global population ageing. Assuming that residential houses are adaptable for elder citizen requirements, it will be important to provide the quality of life and social support for this fringe of population, maintenance of its use and avoiding abrupt space changes for nursing houses, for example. Our research, using a Shape Grammar from the portuguese housing legislation and mobility principles, proposes a generative tool that will allow to (re)design residential houses. Our goal is to provide designers an intuitive document that explains in a systematic way to enable architects to address norms in an intuitive way. the application of norms according to Portuguese housing legislation and referring to social equipment and technical rules related to accessibilities

Keywords: adaptative housing, shape grammar, generative design, Portuguese housing legislation, transformation in design.

Introduction idential dwellings by giving specific living conditions According to the Portuguese National Statistics to the houses of elder citizens, it will be important to Institute (INE), in 1970 the population aging rate was provide the maintenance of its use and avoiding the around 33.97 for 100 children. In 2010 there was al- particularly hard transition of their relocation to other ready a proportion of 129 seniors for 100 children. It residences, i.e., nursing house.Our research proposes Is expected that in 2060 the proportion will raise to 3 a generative tool that will allow to (re)design residen- seniors/1 child (INSTITUTO DE SEGURANÇA SOCIAL, tial houses, using a shape grammar (STINY, 2011), a 2011). As consequence the nursing homes are not set of descriptions and parameters (DUARTE, 2001) responding so efficiently to this big challenge, not extracted from the Portuguese housing legislation to promoting an inclusive and inter-generational soci- adapt the Portuguese design parameters and recom- ety, and rather contributing to augment the distance mendations of nursing homes (MINISTÉRIO DA SOLI- of the elderly population from society. And the con- DARIEDADE E DA SEGURANÇA SOCIAL, 2012) into the sequence is a constant loss of social contacts and self- residential home model, based on accessibility and identity resulting in a broken neighbourhood net- mobility principles and focused on programming as- work in our communities. (CLASSEN, 2007).Assuming pects of the domestic house. A transformation gram- that it is possible to adapt the existing urban and res- mar (KNIGHT, 1994) will allow to understand and ma-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 313 nipulate a given set of rules from two legislation which includes the revision of fundamental concepts, codes and standards by generating the transforma- until the demanding requirements and legal disposi- tion and rehabilitation of some private apartments tions. In the second phase a documentation search and proposing an optimized adaptation of the house was done for specific buildings in the Palmela coun- (ELOY, 2012). It Is to be noted that this research aims cil (Setúbal district). The Palmela City Council hosted to transform a family home for a family with no hand- this study, (see figure 1) and made available regu- icaps, transforming it in such a way that it will be latory means of a practical case study for investiga- adapted to people with some specific needs related tion (a model of a rebuilt home). The case study is a to mobility constraints. Our main goal is to provide paradigmatic home rebuilt in the Palmela City Coun- designers a tool that will help them to rehabilitate cil, located in Pinhal Novo Urban Rebuilt Area: delim- and transform dwellings in a legal and adapted mode ited area which can be improved by using tax bene- hoping to propose an inclusive way of domestic liv- fits to improve the quality of the buildings. These ar- ing to its inhabitant needs (DARÉ, 2010), and pro- eas have a significant set of effects, among which, it is viding a comfortable living throughout the citizens noteworthy, the tax benefits associated with munic- ageing.In this regard, the goal of this study is to pro- ipal property taxes. It also follows from that act that vide ergonomics, comfort, durability, and the elim- the owners have access to tax and financial support ination of architectural barriers to residential build- and incentives for urban rehabilitation. The third and ings which are adapted to the “future of the senior last phase consists in the application of Shape Gram- generation” of the XXI Century.With this in mind, not mar based on the home model chosen in the previ- only the technical recommendations are the angular ous phase. stone of this study, but also the application of a set of Shape Grammar Rules, Conditionals and Descrip- Portuguese norms, highlighting the idea of adapting tors these same norms to other areas that are not contem- These rules of this grammar are composed of a plated, for example the room compartment and the schemata, a set of conditionals related to the draw- indoor furniture.It is intended to create a new archi- ing parameters and a set of descriptors. These de- tectural model and residential design adapted to this scriptors come directly from the regulations in ac- new reality, generating solutions that frame accessi- cordance with the Portuguese legislation. The rules bility parameters and comfort areas, by rehearsing a are referring to U, V and W. The algebras U are re- series of rules and parameterization of these defini- lated to geometric elements of the design rule , i.e., tions via Shape Grammar, and with focus on the di- Point, lines, etc., and V with labels. For instance, the mensioning of resting areas (rooms), and of the re- label composed by a wheel chair circle, the design of maining compartments such as the kitchen, living a wheel chair, the segment of straight [m, m1], etc. room and common areas. The algebra W is related with the weight, it’s used Methodology in hatches (placed to restrict, or make the furniture The adopted methodology covers three phases: placement, it is impossible to do the inverse mode in the first phase a necessary bibliographic gather- in Shape Grammar “The Generation of Hepplewhite- ing was done to understand the subject of the study, Style Chair-Back Designs” (Knights 1980). If in Knights such as the existing legislation: Quality and Social grammar the hatch schema was a “legal” area to gen- Answers Manuals: Technical Recommendations for erate a curve in a certain style, on this grammar our Social Equipment (RTES) for nursing homes; General hatcher is embedded with the function of “not allow regime for Urban Edification (RGEU) shown in the Law to invade that area, subtracting all elements over lap- 38382 of 07/08 1951, and Technical Rules for Acces- ping it.In this regard we built 3 differentiated set of sibility shown in the Decree Law 163/2006 of 08/08 rules that works only on the floor, because the great

314 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 1 Apartments in rehabilitation and transformation using Shape Grammar

goal of this residential structures for the elderly trans- - Nursing Homes“; When 4br inserts a shape a chest formation grammar is not only to generate project al- of drawers is recognized. It applies the descriptor: ternatives according to the law, but also to provide ”The wardrobe or storage room, according to the di- designers an intuitive document that explains in a mensions shown on point 8.6.4 of Chapter III Techni- systematic way, the application of norms according cal Recommendations for Social Equipment (RTES) - to Portuguese housing legislation and referring to so- Nursing Homes“; when 5br inserts a chest of draw- cial equipment and technical rules related to acces- ers the twin bed is recognised; When 6br inserts a sibilities.At this point, we have the grammar that al- single bed when mirror twin bed and double bed; lows us to generate a bedroom, kitchen and toilet When 1k inserts a Point P a window from an initial for people with reduced mobility. This grammar has shape is recognized It applies the descriptor: ”There the following rules:When 1br inserts a Point P a door should be a manoeuvre for a wheel chair to rotate from initial shape is recognized. It applies the de- 360º (point 1) of 4.4.1-DL 163/2006 from 08/08. When scriptor: ”There should be a manoeuvre for a wheel 2k inserts a circle with x diameter, point P is recog- chair to rotate 360º (point 1) of 4.4.1-DL 163/2006 nized. It applies the descriptor: “There should be from 08/08“; When 2br inserts a circle with x diam- manoeuvre for a wheel chair to rotate 360º (point 1) eter, point P is recognized. It applies the descriptor: of 4.4.1-DL 163/2006 from 08/08” (point 1) of 3.3.1- ”There should be manoeuvre for a wheel chair to ro- DL 163/2006 from 08/08; When 3k inserts a A-B-C- tate 360º (point 1) of 4.4.1-DL 163/2006 from 08/08“; D points, a point P is recognized; When 4k inserts a When Kbr inserts a Point P a hatch from initial shape kitchen right countertop, a point A-B are recognized; is recognized. It applies the descriptor: ”Accessible When 5k inserts a left kitchen countertop, a point C- room according to point 8.6.2 Chapter III of the Tech- D are recognized. When 6k inserts a right kitchen nical Recommendations for Social Equipment (RTES) countertop, the watch sink, fridge and stove are rec-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 315 Figure 2 Accessible bedroom rules

316 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 3 Friendly kitchen furniture rules

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 317 Figure 4 Friendly kitchen furniture rules

318 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 5 Accessible toilet rules

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 319 Figure 6 Shape grammar derivation rules for a residential plan

320 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 ognized. When 7k inserts a left kitchen countertop, ment (RTES) - Nursing Homes and all other techni- the washing machine is recognized; When 1lr inserts cal normatives. Figure 6 shows the T1 plan deriva- a Point P a door from initial shape is recognized. It tion.Drawings are being generated and detailed in applies the descriptor: “There should be a manoeu- plan starting from a empty T1 appartment, populat- vre for a wheel chair to rotate 360º (point 1) of 4.4.1- ing it with basic furniture respecting legal circulatory DL 163/2006 from 08/08”; When 2lr inserts a circle spaces. with x diameter, point P is recognized. It applies the Conclusion descriptor: “There should be manoeuvre for a wheel In terms of housing rehabilitation, the shape chair to rotate 360º (point 1) of 4.4.1-DL 163/2006 grammar rules applied in this case of study show that from 08/08”; When 3lr inserts a table a hatch from ini- it’s possible to increase the typological diversity of tial shape is recognized; When 4lr inserts a sofa, when the rehabilitation strategies by incorporating other a rectangle is recognized. It applies the descriptor: emerging types of housing, in line with the Decree- “Free zone of wheel chair permanence of 4.1.1 of DL- Laws of elderly home cares into a residential home. 163/2006 of 08/08. When 1wc inserts a Point P a door This will promote a diversity of typologies and com- from initial shape is recognized. It applies the de- munal areas for single and couple elderly people, to scriptor: ”A free zone of wheel chair to rotate 360º of improve integrative cities. This review increase the 4.4.1-DL 163/2006 from 08/08“; When 2wc inserts a viability of rehabilitation work for the building type circle with x diameter, point P is recognized. It applies studied, promoting a response within the housing the descriptor: ”A free zone of wheel chair to rotate sector and various other sectors in society. This is the 360º of 4.4.1-DL 163/2006 from 08/08“; When 3wc in- moment to invest on the inclusion of people with re- serts a point P, a washbasin is recognised; When 4wc duced mobility, guaranteeing their rights supported inserts a toilet a rectangle is recognized. It applies the by technology. This type of adaptable buildings is descriptor: ”There should be a free zone which satis- the answer for the future of the elderly society in our fies the shown dimensions of one of the sides of the cities. frontal part of the toilet according to paragraph 2) of 2.9.4 of DL 163/2006 from 08/08.“; When 5wc inserts REFERENCES a shower a rectangle is recognized. It applies the de- Classen, A. (eds) 2007, Old Age in the Middle Ages and the scriptor: ”There should be a free zone which satisfies Renaissance, de Gruyter the shown dimensions of one of the sides of the base Classen, A 2010, ’Old Age in the Middle Ages and the of the bathtub according to paragraph 1) of 2.9.7 of Renaissance: Interdisciplinary Approaches to a Ne- DL 163/2006 from 08/08; 6wc it’s an erase rule -> {} glected Topic’, The Journal of English and Germanic Philology, 109(1), pp. 1-84 (empty set). Daré, AC (eds) 2010, Design Inclusivo: o Impacto do Am- Rebuilt Home in Pinhal Novo - Palmela, applying biente Doméstico no Utilizador Idoso, Universidade Shape Grammar Rules Lusíada Editora, Portugal In order to fill a given T1 plan in Palmela Dis- Duarte, JP 2001, Customizing Mass Housing: A Discursive trict, it’s evoked the bedroom (br), Kitchen (k) and Grammar for Siza´s Malagueira houses., Ph.D. Thesis, Toilet (wc) shape grammar applying all the elements Massachusetts Institute of Technology Eloy, S 2012, Transformation Grammar Based Methodol- needed. All grammar rules are applied in a simul- ogy For Housing Rehabilitation: Meeting Contempo- taneous mode to its derivation. Is to be noted rary Functional and ICT requirements, Ph.D. Thesis, In- that the grammar derivation is not the end of the stituto Superior Técnico generation process until being obtained a satisfac- Weissman Knight, T 1980, ’The generation of tory and meaningful result, i.e., a legal aplication of Hepplewhite-style chair-back designs , Environ- the of Technical Recommendations for Social Equip- ment and Planning B: Planning and Design,’, Envi- ronment and Planning B: Planning and Design, 2(7),

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 321 pp. 227-238 Weissman Knight, T (eds) 1994, Transformations in De- sign: A Formal Approach to Stylistic Change and Inno- vation in the Visual Arts, Cambridge University Press, England Quaresma, FC 2014, Gramática da Forma das Sistematiza- ção da Coluna de Alberti (Alberti, Ph.D. Thesis, Facul- dade de Ciência e Tencnologia Instituto de Segurança Social, ISS (eds) 2011, Recomen- dações Técnicas para Equipamentos Sociais – Lares de Idosos, Publicações Segurança Social, Portugal Stiny, G 1991, ’The Algebras of Design.In Research in En- gineering Design’, Doi, 2(3), pp. 171-181 Stiny, G 2011, ’What Rule(s) Should I Use?’, Nexus Network Journal, 13(1), pp. 15-47 Stiny, G and Gips, J 1972, Shape Grammars and the Gener- ativeSpecificationofPaintingandSculpture, Proceed- ings: C V Freiman (ed.) Information Processing 71 , Amsterdam [1] https://dre.pt/home/-/dre/553657/details/maximize d [2] http://www.seg-social.pt/pedido-de-licenciamento

322 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 An Immersive Modular Museum-Archive Concept Model An integration of shape grammars with virtual reality

Katherine Liapi1, Dimitra Liosi2 1,2University of Patras, Greece [email protected] [email protected]

Τhe paper presents a concept model of a modular exhibition that is designed to function as a museum in the physical environment and as an archive of architectural projects in the virtual. A design method based on shape grammars and inspired by the Bauhaus teaching, has been used to generate both the museum spaces and the archive which expands endlessly as the acquisition of architectural exhibits keeps increasing. Specifically, rules that control the geometric organization, tectonic expression and linear or radial expansion of a modular assembly of interlocking physical or virtual spaces, leading to various self-similar spatial patterns, have been developed. Several scenarios of the visitors' interaction with the exhibits and the museum's space with the use of VR and AR have been explored. In the actual museum scenario, physically present visitors can interact with the exhibits which are enriched with digital information. In the case of the archive, the Unity platform is used for the development of a virtual experience of the visitor who interacts with the expanding space and exhibits in a purely immersive manner.

Keywords: virtual reality, augmented reality, shape grammars, golden ration, modular museum, archive, Bauhaus teaching

INTRODUCTION challenges separately or combined. A reoccurring challenge that museums in our cities Modular museum buildings that permit the addi- face is the need for a continuous spatial expansion in tion of new exhibition spaces in a systematic manner order to satisfy the acquisition of new exhibits. This can partly provide an answer to increasing numbers challenge is often difficult to overcome, due to all of exhibits over time. However, it is obvious that the kinds of constraints, including site boundaries, code expansion of any modular space in an actual urban restrictions, structural limitations etc. An additional context, has to conform to various spatial limitations challenge that always existed, but has become a ma- and constraints; on the other hand in a virtual con- jor issue in our days, is the public’s limited physical text, the limitations may be far less compelling, and access to exhibition spaces due to the pandemic cri- the museum ideally might expand endlessly as the sis. There is therefore a need to address both these acquisition of artefacts increases.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 323 The use of cutting edge technologies, such as project was to tackle the above mentioned real world Virtual Reality (VR) and Augmented Reality (AR), can problem, additional educational objectives were set. provide solution to accessibility difficulties due to the The experimentation with shape grammars as a de- occurrence of the pandemic or other physical access sign method, as well as the exploration of the applica- constraints. It can also accommodate the lack of bility of AR and VR processes in conjunction with the space for the display of support material for the ex- proposed design method, towards the development isting or for other acquired exhibits. of the modular museum concept, were among the Although the discourse on the value of a vir- educational objectives and challenges of the project. tual reproduction of exhibits poses the issue of “au- Furthermore, on the occasion of the celebration of thenticity”, as stated by Benjamin (Benjamin, 1936), the 100 years of Bauhaus, and with the intention to in recent years both VR and AR have found signifi- re-examine and reveal the timeliness of the Bauhaus cant applications in museums (Chen, 2020; Shehade design education, the use of the shape grammar and Stylianou-Lambert, 2020), on the ground that au- method was expected to draw inspiration from the thenticity can be achieved to some degree by the Bauhaus teaching on studies on form. The emphasis developments in information technology (IT) which on the development and use of rules for form genera- “enable visitors to fully explore and appreciate muse- tion and transformation in the Bauhaus teaching was ums moving beyond time, space, and language bar- the main reason for the coupling of shape grammar riers” (Chung, Lee, Kim and Koo, 2018). with Bauhaus design methods. Museums and exhibitions of architectural mod- els and drawings constitute a particular case, as, on EXHIBITION-ARCHIVE CONCEPT MODEL one hand, the displayed drawings are only a frag- AND SHAPE GRAMMARS ment of the total number of the drawings produced A concept model that will permit both the develop- on a project, and, on the other, the scaled physical ment of an actual exhibition space in the physical en- model or models, in most cases, convey valuable in- vironment and that of an archive in the virtual envi- formation about the site, geometric and volumetric ronment that expands endlessly as the acquisition of relationships, structural organization etc., but they architectural exhibits keeps increasing, is proposed. hardly ever convey the experience of a moving ob- For the development of this spatial model, shape server around or inside it. VR and AR applications in grammars, introduced by Stiny in the late seventies such cases could either enhance the actual experi- (Stiny, 1980), have been chosen as the preferred de- ence of the displayed item, or provide an entirely vir- sign method. tual experience that replaces the experience of the Shape grammar has been defined as “a formula- visitor in the physical place of the museum. In this tion composed by geometric shape rules or transfor- case there are no accessibility barriers and the display mations that, once applied recursively and ordered in of additional information is possible. Yet the con- successive steps, can produce a family of designs that stant addition of exhibits that could gradually build share the same design principles, features or style” an archive, requires the development of a new spa- (Benrós, Eloy and Duarte, 2015). tial concept that will permit to the visitor to experi- Of decisive importance for this study was the ence the expanded virtual museum as a spatial con- main property of the shape grammars method ac- tinuum. cording to which ”a set of a finite number of rules The development of a concept model of a mod- and shapes may generate an indefinite number of de- ular museum that addresses these challenges has sign solutions” (Tepavcevic and Stojakovic, 2012).The been conducted in the context of an academic intention in this project was to use shape grammar project. While the main objective of the design rules towards the development of various self similar

324 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 spatial systems. Self-similarity, defined as ”symmetry has been selected as the basic module for the devel- across scale”,which ”implies recursion, pattern inside opment of the modular museum concept model are of a pattern” (Gleick, 1998), was a desired feature of shown in Figure 1, while the unit at its final stage is the design method that had to be followed for the de- shown in Figure 2. velopment of the modular museum geometric con- Figure 1 cept. Hence the use of a proportioning system that Shape grammar would generate all planar and spatial elements of the process for the modular museum was set as a main project require- exhibition unit ment. geometric and Departing from this requirement, rules that spatial could control the geometric organization, tectonic configuration expression and expansion of a modular assembly of interlocking physical or virtual spaces had to be de- veloped. For the formation of the shape grammar method, rules that were influenced by the Bauhaus teaching and studies on form and in particular on the use of the Golden Ratio (Fitzerald, 1991; Lerner, 2005) were used. The emphasis in the Bauhaus teaching on the development and use of rules for the tectonic or- ganization and transformation of 3D forms was the main reason for the coupling of shape grammars with Bauhaus design methods. Taking into account that the Golden Section rect- angles, ‘replicate through endless divisions’ (Fletcher, 2008) in a self-similar repeating pattern, the golden means was chosen as the preferred proportioning system to be used for the generation of both the sur- face elements, the spaces and the overall geometric configuration and tectonic expression of the modu- lar museum. Once the main rules were set, the first stage of this study was focused on the exploration of the form of the basic exhibition unit. So, for the generation of the spatial geometry of the basic unit, specific rules that determine the manner in which golden, or «φ» rectangles, placed in the direction of the three Carte- sian planes, are joined to each other at their sides, or intersect each other at a line that derives from the harmonic decomposition of each «φ» rectangle, were set. By following consecutive steps that reflect these rules, several configurations of the basic unit have been generated. The steps in the process that has led to the geometric configuration of a typical unit that

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 325 At a following stage, rules that determine the Figure 2 manner in which the exhibition units are connected Exhibition unit at its to each other in order to create interlocking and com- final stage municating spaces, have been set. The main function of the rules this time was to ensure that the propor- tions of the generated spaces also fall within the «φ» proportioning system. The application of these rules permit the devel- opment of the structure in a linear or a radial manner and/or in a vertical or a horizontal direction, leading to various self-similar spatial patterns. However, al- though these rules may be applied for the develop- ment of interlocking spaces in both the physical and the virtual world, the limitations of the application of the rule in the physical space due to all kinds of constraints are obvious; instead, in the virtual space models can keep expanding endlessly. Examples of linear and radial arrangements of interlocking spaces are shown in Figures 3 and 4. Using these rules in a programming environ- ment has facilitated the generation of alternate geometric configurations of the modular museum model. The construction of small scale physical mod- els has been used to test the correctness in the defini- tion of the rules and the effectiveness of the process. At the last stage of this process, on an effort to transfer this geometric concept to an initial mu- seum/archive design, several arrangements of inter- locking exhibition units were examined. A radial or- ganization of units that may allow for a vertical ex- pansion was selected as the most appropriate for an urban site. Accordingly, a spatial arrangement of four units in a radial manner that allows at the same time for the development of a vertical core at the centre of their in-between space, was selected. This arrange- ment of the exhibition units also permitted the un- limited vertical development of the virtual museum structure (Figure 5). The consecutive steps that were followed for the development of the initial architectural design of the basic exhibition unit and the modular museum, included: scaled adaptation of the geometric con- cept model of the unit in the museum, addition of

326 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 3 Linear arrangements of interlocking units

Figure 4 structural members and circulation elements such Radial as ramps, stairs and elevators, addition of openings arrangements of such as windows and doors, definition of the func- interlocking units tion of the sub-spaces and the horizontal surfaces within the main volume of the exhibition space, i.e. for model and drawings display or lecture or perfor- mance areas. Supplementary rules for the addition of spatial elements to the basic exhibition unit that would render them consistent with the same propor- tioning system had to be set.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 327 INTEGRATION OF AR AND VR TECHNOLO- Figure 5 GIES Vertical In recent years Augmented Reality has found many development of the applications in exhibitions and museums (Chen, virtual modular 2020). By adding digital data to the physical data, AR museum contributes to the enhancement of information of- fered to the visitors in real-time and the enrichment of their experience (Kesim and Ozarslan, 2012). As Chen claims, AR that provides a better understanding of the exhibit and its interaction with its surroundings can transform exhibitions to “social educators”. The use of AR in this study, has been informed by the processes applied at an Architecture Exhibition of Ancient Machinery at the Southern Taiwan University of Science and Technology. AR was used to create a game related to the physical exhibits, which would actually introduce the visitor to an enhanced envi- ronment enriched by additional information on an- The steps that were followed in the development of cient machinery (Chen, 2020). By following a simi- the initial design of the exhibition unit and the mu- lar process, the architectural drawings and models in seum are shown in Figure 6. Figure 7 highlights the the exhibition spaces of the modular museum could effective use of the development method in critical be enriched by digital information provided by the design decisions regarding the placement, propor- application of AR. Hence several scenarios of the visi- tions and dimensions of the openings and the circu- tors’ interaction with the exhibits and space with the lation elements which also determine the spatial or- use of VR and AR have been explored. ganization and the tectonic expression of the modu- The main difference between AR and VR is that lar museum. AR interacts with the physical world, and VR provides The final outcome of this effort has indicated that a completely immersive experience (Milgram, Take- the methodology that has been followed, has greatly mura, Utsumi and Kishino, 1994). As the modular mu- facilitated the development of the geometric con- seum has been developed only in a virtual environ- cept of the basic exhibition unit and the modular mu- ment, an actual application of AR has not been con- seum as well as the development of the initial design ducted.Therefore in this study VR is used in the con- and schematic drawings of the project. text of the virtual model of the developed modular museum. Accordingly, in the virtual environment, the modular museum will function as a digital archive of architectural models and will be accessible through VR technology. Users of VR technology will visit the archive and will be able to interact with the architec- tural building models, enter inside these models and freely roam in them (Figure 8).

328 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 6 The integration of VR technology in this project Exhibition unit: and the workflow followed for the development of transition from the the digital archive is demonstrated in the flowchart basic geometry to shown in Figure 9. the architectural model

Figure 7 Interior of the modular museum

Figure 8 Virtual Reality application for experiencing the architectural project

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 329 user remains at the same position. Figure 9 The user’s interaction with the architectural VR platform design model can be done in 3 ways: 1. by approaching each model, press a button, aim at the description button of the exhibit and acti- vate its description, and access details related to the project. 2. by clicking on the model, the user is teleported inside the model and interacts with it on a real scale. In order to create the archive in the virtual environ- 3. by pointing at the red sphere and pressing an- ment, the virtual representation of the modular mu- other button, the user is able to view an ex- seum as a 3D model, converted to a .fbx file format ploded version of the model. was completed first. The .fbx file has been imported The application of VR was tested with students of the in Unity, a video game development platform. Then Department of Architecture at the University of Pa- the 3D building models that would be used for the tras, who experienced the modular museum’s space exhibition were imported in unity as well. The vir- and the displayed architectural drawings and mod- tual environment in which the archive was located, els through the headsets. The tests that were con- was configured through an editing process in Unity ducted led to several improvements in the definition by adding additional elements such as lights, materi- of the Unity code as well as in the evaluation of the als etc. The commands for user interactions with the circulation patterns, observation points inside the ex- environment were written in C # programming lan- hibition spaces, views to the exterior, view of the dis- guage. Finally, the project in Unity was “baked” so played architectural drawings and models, as well as that it can be visible through the headset and thus movement through and around the models, experi- the experience of VR would be completed. enced in the actual size of the architectural space or Το interact with the virtual environment, which building they represent. is a three-dimensional space, the users need to wear the VR headset and hold the controllers. The phys- ical hands of the user are simulated in VR and their CONCLUSIONS movements are tracked by the controllers they hold. The developed concept model of the modular exhi- Teleportation was selected as the user’s way of bition successfully addresses the challenges and the navigating the virtual environment of the museum- design objectives set at the beginning of the project. archive. The users can also navigate inside or around The use of shape grammars as a design method- the displayed models experienced, in the actual size ology for the development of the design concept has of the architectural space or building they repre- been proven very helpful. The intention was to use sent. The interface allows them to be teleported from the shape grammar rules towards the development point to point, by pressing a button that activates a of self-similar spatial systems. Coupling shape gram- laser beam, which targets the point where the user mars with rules that derived from the Bauhaus in- wants to go. As they release the button, they are struction and studies on form, and in particular, with transferred to this exact point. The choice of be- rules that could generate a three-dimensional con- ing teleported instead of walking was made to avoid ception of the geometry of the Golden Section, made dizziness, possibly caused by continuous walking in a possible the proportioning of the parts as well as the virtual environment, while in the physical world, the generation of self-similar interlocking spaces, a re-

330 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 quired feature of the project. Furhermore this cou- ACKNOWLEDGEMENTS pling has led to the formal coherence of the devel- We would like to thank Petroula Gkanidou for the as- oped modular museum design. sistance that she offered to the development of the The application of these rules in a programming VR application. environment has facilitated the generation of alter- nate configurations of the basic exhibition unit as REFERENCES well as for the development of arrangements, both Chen, Zi-Ru 2020 ’The Guidance System of Gamification linear and radial, of interlocking units that form the and Augmented Reality in a Museum Space’, Pro- modular museum. The final outcome of this effort ceedings of the 25th CAADRIA Conference, pp. 672- has also indicated that the use of shape grammars 680 to control the geometric organization, and the tec- Chung, Namho, Lee, Hyunae, Kim, Jin-Young and Koo, Chulmo 2018, ’The Role of Augmented Reality for tonic expression and expansion of the modular mu- Experience-Influenced Environments: The Case of seum, has assisted in decisions regarding the place- Cultural Heritage Tourism in Korea’, Journal of Travel ment, proportions and dimensions of openings, cir- Research, 57, pp. 627-643 culation and other space defining elements. Duarte, José Pinto, Eloy, Sara and Benrós, Deborah 2015 The developed modular museum is expected to ’Re-inventing ceramic tiles: Using shape grammars function as a museum in the physical environment, as a generative method and the impact on design methodology’, Proceedings of 16th International Con- and as an archive of architectural projects in the vir- ference CAAD Futures 2015, p. 469 tual. Linking VR with the spatial model has made Fitzerald, Astrid 1991, ’Harmony by Design’, PARABOLA- possible the experience of a continuous space that The Golden Mean, 16, pp. 37-41 grows in a systematic manner as new exhibits are Fletcher, Rachel 2008, ’Dynamic Root Rectangles Part added. It has also permitted the exploration of sev- Two: The Root-Two Rectangle and Design Applica- eral scenarios on the visitors’ interaction with the ex- tions’, Nexus Network Journal, 10, p. 149 Gleick, James 1998, Chaos: The Amazing Science of the hibits. This application was tested with students who Unpredictable, Vintage, London were transferred via VR inside the modular museum’s Kesim, Mehmet and Ozarslan, Yasin 2012, ’Augmented space and were teleported inside the displayed ar- Reality in Education: Current Technologies and the chitectural models. The tests led to several improve- Potential for Education’, Procedia - Social and Behav- ments in the definition of the Unity code, as well as, ioral Sciences, 47, p. 298 in the evaluation of the functionality of the modu- Lerner, Fern 2005, ’Foundations for Design Education: Continuing the Bauhaus Vorkurs Vision’, Studies in lar museum. The automatic update of the expanding Art Education, 46, pp. 217-224 model of the museum in the Unity Code, would be Milgram, Paul, Takemura, Haruo, Utsumi, Akira and a desired improvement of the code that will be ex- Kishino, Fumio 1994 ’Augmented reality: A class plored as a future study. of displays on the reality-virtuality continuum’, Pro- Eventually, the integration of shape grammar ceedings of SPIE-The International Society for Optical rules based on the «φ» proportioning system for the Engineering, pp. 282-283 Shehade, Maria and Stylianou-Lambert, Theopisti 2020, spatial organization and the tectonic expression of ’Virtual Reality in Museums: Exploring the Experi- the design, with VR and AR technologies, was sug- ences of Museum Professionals’, Applied Sciences, 10, gested by the nature of the modular museum project p. 1 itself. At the same time, this approach served as a Stiny, George 1980, ’Introduction to shape and shape testbed for the success of the synthesis of earlier and grammars’, Environment and Planning B: Planning current design methods with cutting edge technolo- and Design, 7, pp. 343-351 Tepavcevic, Bojan and Stojakovic, Vesna 2012, ’Shape gies in one single project. grammar in contemporary architectural theory and design’, Facta universitatis - series: Architecture and

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 331 Civil Engineering, 10, p. 170 Walter, Benjamin 1936, ’L’oeuvre d’art a l’epoque de sa reproduction mecanisee’, Zeitschrift für Sozial- forschung, 5, pp. 40-68

332 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Computational Generation of Hybrid Façades for a Focal Context The case of Naser-Khosrow Street in Tehran

Niloofar Zolghadrasli1, Mahyar Hadighi2, Eduardo Costa3 1,2Texas Tech University 3Bath University 1,2{Niloofar.Zolghadrasli|mhadighi}@ttu.edu [email protected]

The purpose of this paper is to use shape grammar methodology to generate contextualized urban façades within the focal context of Naser-Khosrow Street in Iran's capital city of Tehran. The history of Naser-Khosrow Street, which is considered a significant historic urban space, begins in the Safavid period (1735-1501). Yet, the urban history and historical significance of this street have been neglected in recent years. Evidence of a sense of turmoil and incompatibility between the modern and traditional architectural façades featured on the street is easily observed. Against this background, this paper offers façade models generated based on the computational methodology developed by Hadighi and Duarte (2019, 2020) to determine and capture the hybrid expression of European modernist and American traditional styles. In this spirit, the systematic methodology of shape grammar is expanded to the focal urban area of Naser-Khosrow Street to generate new façade layouts referencing the characteristic features of some of the iconic buildings located there.

Keywords: Shape Grammar, Naser -Khosrow Street, Façade Design, Generating Hybridity

INTRODUCTION trict and the parish developed. On this basis, a new Naser-Khosrow Street began its history during the urban texture came into being such that the Naser- Safavid Empire (1735-1501) as a narrow and signifi- Khosrow pathway took on greater importance given cant pathway aligned with the main citadel walls of that it now flanked the new urban areas of Udlajan Tehran, which was not yet Iran’s capital city. It was and Marvi. In this extended form, the pathway be- not until the Qajar dynasty (1796-1925) that Tehran came Naser-Khosrow Street and even boasted signif- became the capital, an event that proved to be a turn- icant new architectural buildings such as Darolfonon, ing point in the development and extension of the Shams-Ol-Emareh, and Saraye Roshan. Darolfonon Naser-Khosrow pathway. During this dynasty, Tehran was constructed in 1852, during the second Pahlavi became an important urban center, and over time period (1941-1978), to serve as the home to the pio- new urban organizational structures such as the dis- neer of modern schools in Iran. A few years later, in

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 333 1861, the five-story Palace of Shams-Ol-Emareh, the along the street’s length. Thus, in the present study, a highest building in Iran of this period, was erected. In systematic method is proposed to generate façades 1894, during the Pahlavi era under Reza Shah’s rule, following the historical spirit of Naser-Khosrow Street the two-story Saraye Roshan was built for commer- while also observing the rules and regulations of the cial purposes. As Naser-Khosrow was (and remains) present-day urban context. Thus, this study draws a commercial street, Saraye Roshan became an in- on the computational methodology developed by spirational design for other commercial buildings in Hadighi and Duarte (2019, 2020) to describe and re- the area. However, although the construction of the create the hybrid expression of European modernist three buildings spanned 42 years, they all, similar to and American traditional styles. In the present paper, the overall style of the old buildings on the street, fol- the proposed methodology is further developed to low the same architectural style. This style is Iranian generate a hybrid architectural façade design for the eclecticism, the base of which is a fusion of hybrid- historical urban texture of Naser-Khosrow Street. ity between traditional Iranian architecture and Euro- Figure 1 pean architecture, which was ubiquitous during the Left: Location of the Qajar era. Figure 1 shows the locations of Darolfonon, corpus on Shams-Ol-Emareh, and Saraye Roshan on the street. Naser-Khosrow Against this background, Naser-Khosrow Street be- Street, Tehran, Iran. came a commercial focus given the many activities Right: From top to that took place there. Especially from 1921, when bottom: due to a population increase in Tehran and significant Darolfonon, economic growth, commercial buildings, especially Shams-Ol-Emareh, those used for wholesale services in the center of the and Saraye Roshan. city, started to develop. However, in terms of its his- Photographs from tory, architecture, and urban texture, Naser-Khosrow top to bottom Street has been neglected in recent years. Perhaps, a taken by Neda lack of appreciation for the street’s cultural and his- Alipour, Pedraam torical importance led to the construction of new (Persian Wikipedia), buildings (built from the late 20th century) on the and Fulvio Spada. street with façades that are plainly incompatible with those of the old buildings such as the three briefly de- scribed. Modern styles stand in stark contrast with Shape grammar the historic styles on the street. The old buildings are Shape grammar is a computational methodology orange brick with horizontal and vertical divisions in used to describe shape transformations by applying the facades; the new buildings are uniform with flat rules to an infinite shape (Gu & Behbahani, 2018). In light-colored façades without any additional façade this procedure, these transformations are illustrated divisions. Where the old buildings have curved de- through the schema of A฀B, whereas a finite rule is tails and elaborate carvings, the new buildings al- used to convert a left-hand-side shape (LHS) into a most entirely lack embellishment. Recent modifica- right-hand-side shape (RHS) (Gu & Behbahani, 2018; tions rupture the historical site’s appearance, with no Teboul et al., 2011). The transformation design draws apparent effort made to achieve cohesiveness be- on the grammar to transform existing design styles tween the modern architectural styles and the his- into new types (Knight, 1983). Examples include torical architectural elements of the street. This in- Coutinho et al.’s (2011) description of the influence of consistency creates a strong sense of visual confusion the historical architecture of Alberti’s work on local Portuguese architecture through a shape grammar

334 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 and Eloy and Duarte’s (2012) development of a trans- mars to generate new façade designs; (4) Generate formation grammar for housing rehabilitation to cre- new designs; and (5) Compare the generated de- ate a new style adapted to contemporary life. The signs with the original designs to determine the ex- idea of using grammar transformation to establish tent to which the new designs capture traditional correlations between classical buildings and mod- elements. The present paper’s focus is to generate ern life culminated in the shape grammar methodol- hybrid façades through the shape grammar devel- ogy developed by Hadighi and Duarte (2018, 2019) oped based on the iconic façades of Naser-Khosrow to create a hybrid design. The background of us- Street. The grammar-based analysis focuses on a par- ing shape grammar to analyze the existing traditional ticular architectural design, referred to as a corpus of urban context includes Duarte and Rocha’s (2006) grammar. The architectural façade design of the se- study of Medina of Marrakesh through this method- lected iconic buildings on the street will be captured ology. Barros et al. (2013) used the methodology to in other papers. On this basis, the extracted grammar create an adaptive model based on Mapotus’s slums is extended to generate new façade layouts drawing in a different direction. A study by Ke (2016), which on the style of existing corpora and modern-world is currently in progress, will describe the methods needs. Future papers will focus on related concerns used in the grammar in relation to a neighborhood within the larger-scale study. in Beijing, China. The historical importance of the Figure 2 buildings, their development in modern times, and Analysis of façade the objective of providing variations of combinations corpus of to preserve the local culture are issues considered Darolfonon, in this study. In a similar vein, Verniz and Duarte Shams-Ol-Emareh, (2017) extended the methodology to describe the and Saraye Roshan. complex urban structures of Favela Santa Marta in Rio de Janeiro.In the present paper, the hybrid method- ology (Hadighi & Duarte, 2018, 2019) is extended to generate a new hybrid façade solution for the his- torical urban texture of Naser-Khosrow Street. The idea is to revitalize the traditional urban space by cre- ating harmony between the new façade design and the characteristic features of the street’s existing tra- ditional buildings.

METHODOLOGY The present paper is part of a larger-scale project. Several steps are taken to develop hybrid façades Analyzing the selected buildings compatible with the traditional buildings on Naser- In addition to an analysis taking into account the Khosrow Street and its contemporary needs. The corpus’s façades based on geometrical criteria, fac- steps to take to achieve that goal are as follows: tors such as building codes and cultural context con- (1) Analyze the characteristic features of the façades tributed to the assessment of the façades. For in- of iconic buildings located on the street; (2) Cap- stance, as the design guideline of the context lim- ture the rules of and develop a grammar for se- its the height of new construction to a maximum of lected façades on Naser-Khosrow Street as the cor- two grades, the grammar is developed for two-grade pus of the project; (3) Develop the extracted gram- buildings. In the analysis of the five-story Shams-

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 335 Ol-Emareh, an exceptional building that exceeds the Extracting the grammars of the selected two-grade limit, the whole façade was fractioned into buildings three separate horizontal sections. Analyzing the cul- The focus at this stage is deriving rules, referred to as tural context revealed that the need to prevent vi- the corpus, from the selected buildings. The deriva- sual access into the building’s interior from the out- tion process is further categorized into two sets of side is a significant consideration in the façade de- principles-a lower-level grammar focused on orga- signs. This consideration constituted an important nizing the architectural elements in each unit and parameter even a limitation, for the design in relation a higher-level grammar focused on arranging the to the configuration of architectural elements such as fundamental units (Barrios, 2005). The combination the kinds of openings that could be included. As an of the lower- and higher-level grammars results in example, even for a very large unit, a sizeable con- the step-by-step derivation of each building’s façade. tinuous window could not be considered as it would Figure 3 shows the step-by-step derivation of Saraye not accord with the site’s cultural criteria. There- Roshan. fore, based on the cultural context, the windows were grouped into three major classifications for the anal- GENERATING NEW FAÇADES FROM THE ysis: clerestory, standard, and basement windows. EXTRACTED GRAMMARS The clerestory windows were categorized as either When the corpus’s grammar has been extracted, circular or square edge. The standard windows were the next step is to generate new façade design defined as one of three distinctive shapes: square solutions through a derivation strategy (Mohamed, edge, top round arch, and top pointed arch. Figures 2005). This process paves the way for an architect to 2 shows the façade analysis of Darolfonon, Shams-Ol- design a new façade design for the local context of Emareh, and Saraye Roshan. Naser-Khosrow Street. At this stage, the architect is equipped with possible computational solutions for

Figure 3 Step-by-step derivation of Saraye Roshan.

336 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 new façades on the street, making it possible to or- facade with the existing façade of Saraye Roshan is ganize a new façade composition following the cor- evident based on proportion, division of façade ele- pus style. (e.g., Chase, 2002; Ruiz-Montiel et al., 2013). ments, and geometry. Thus, the generated grammars reflect the historical Figure 4 themes of the site. Given that this is the case, the Selected rules of iconic buildings’ influence might be expected to be stage (0). conspicuous in the corpus. However, the generated façade is new, reflecting those themes but not repli- cating them (e.g., Knight, 1994; Moon, 2007).Like the extraction process, the generation procedure is exe- cuted via a two-step process. In the first step, the split parametric grammar rules are applied to the initial shape to split the basic shape into a wedge of fun- damental units. The split parametric grammar is a specialized type of grammar with split rules for a ba- sic shape (Wonka et al., 2003). The rules in this step are drawn from the rules of the derivation process, whereas the configuration of fundamental units in the corpus is defined through the higher-level gram- mar. In Figure 4, the proposed rules for splitting the basic shape to generate new fundamental units are presented through a parametric split grammar. The next step in the generative process follows the rules Figure 5 of the lower-level grammar in the derivation process. Selected rules of In this step, applying respective rules to the funda- stage (0-0). mental units generates the façade, and the applica- tion rule schema is defined through the parametric grammar (Figure 5). In phase 1, labels I and II are generated from the initial shape. In phase 2, II trans- forms into W for windows and S represents clerestory which are small windows near the roof. In phase 3, I transform into D, the label for a door and small win- dows above it. In parametric grammars, the layouts of the shape transformation are defined through vari- ables and equations. Ultimately, defining the gener- ated rules results in possible solutions for generating new, thereby providing a multiple-decision diagram from which architects can select a façade that corre- sponds to their design strategy. The shape grammar of the two generated facades is illustrated in Figures 6 and 7. In Figure 8, the generated façade, and the historical façade of Saraye Roshan are illustrated. In this comparison, the compatibility of the generated

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 337 Figure 6 Generated façade through shape grammar methodology.

Figure 7 Generated façade through shape grammar methodology.

338 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 8 Fig. 8. Generated façade compatible with the existing façade of Saraye Roshan.

CONCLUSION AND FUTURE WORK REFERENCES Shape grammar was explored as a computational de- Barrios, CR 2005 ’Symmetry, Rules and Recursion: How sign methodology relevant to the façade language to design like Santiago Calatrava’, In: 23nd eCAADe of three iconic buildings on Naser-Khosrow Street Conference Proceedings Barros, P, Beirão, J and Duarte, J.P 2013 ’The language of in Tehran, Iran. This exploration resulted in obtain- Mozambican slums.’, In: eCAADe Conference Proceed- ing a grammar schema for each selected façade and ings presenting a derivation tree diagram as a computa- Chase, S.C 2002, ’A model for user interaction in tional solution for generating new façade designs. grammar-based design systems’, Automation in con- This strategy constitutes a useful tool for creating un- struction, 11, pp. 161-172 limited new hybrid façade designs for Naser-Khosrow Coutinho, F, Duarte, J.P and Kruger, M 2011 ’A computa- tional interpretation of De re aedificatoria: Translat- Street’s urban context and can be applied to his- ing Alberti’s column system into a shape grammar’, toric neighborhoods elsewhere. This process facili- In: 29th eCAADe Conference tates the creation of numerous façade variations from Duarte, J.P and Rocha, J 2006 ’A Grammar for the Pa- which the architect chooses the design most appro- tio Houses of the Medina of Marrakech-Towards a priate to and expressive of the needs of a given site Tool for Housing Design in Islamic Contexts’, In: 24th and the design challenge overall. However, the op- eCAADe Conference Eloy, S and Duarte, J.P 2012 ’Transformation Grammar for timization of the generated façades will be covered Housing Rehabilitation: from a specific to a general in future studies focusing on establishing parame- grammar’, In: 30th eCAADe Conference. ters to guide the architect’s selection of a design from Gu, N and Behbahani, P.A 2018, Shape Grammars: A Key multiple options.The purpose of the continuing re- Generative Design Algorithm, Springer search project is to analyze the existing façade archi- Hadighi, M and Duarte, J.P 2018 ’Adapting Modern Ar- tecture of Tehran by performing a comparative anal- chitecture to a Local Context-A Grammar for Hajjar’s Hybrid Domestic Architecture’, In: eCAADe Confer- ysis of the façade grammar of the historical context ence. of Naser-Khosrow Street with reference to and as an Hadighi, M and Duarte, J.P 2019 ’Bauhaus Internation- influence on the modern texture of Iran’s capital. alism to College Town Modernism’, In: International Conference on Computer-Aided Architectural Design Futures, Singapore, pp. 429-443 ACKNOWLEDGEMENT Hadighi, M and Duarte, J.P 2020 ’Local Adaptation of the The preliminary research for this study is based on the International Style.’, In: 38th eCAADe Conference first author’s thesis written at Tarbiat Modares Univer- Ke, G 2016, ’Shape grammar as a method for describ- sity in Tehran, Iran, and supervised by Mohammad ing order of Chinese traditional urban pattern’, Ur- Reza Bemanian and Mohammadjavad Mahdavine- ban form at the edge: proceedings from ISUF2013, 2, jad. This research was also supported in part by Texas p. 187 Tech University.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 339 knight, T.W 1983, ’Transformations of languages of de- signs: part 3’, Environment and Planning B: Planning and Design, 10, pp. 155-177 knight, T.W 1994, Transformations in Design: A Formal Ap- proach to Stylistic Change and Innovation in the Visual Arts., Cambridge University Press Mohamed, M.S 2005, Creative approach to design formu- lation: Shape grammar as a tool in architecture design analysis & synthesis, Ph.D. Thesis, Alexandria Univer- sity Moon, J 2007, Shape grammar for Mies van der Rohe’s high rise apartment, Ph.D. Thesis, Massachusetts Institute of Technology Ruiz-Montiel, M, Boned, J, Gavilanes, J, Jiménez, E, Mandow, L and PéRez-De-La-Cruz, J.L 2013, ’Design with shape grammars and reinforcement learning’, Advanced Engineering Informatics, 27, pp. 230-245. Teboul, O, Kokkinos, I, Simon, L, Koutsourakis, P and Para- gios, N 2011 ’Shape grammar parsing via reinforce- ment learning.’, Institute of Electrical and Electronics engineers (IEEE), Rhode Island Verniz, D and Duarte, J.P 2017 ’Santa Marta Urban Grammar-Towards an understanding of the genesis of form’, In: 35th eCAADe Conference Wonka, P, Wimmer, M, Sillion, F and Ribarsky, W 2003, ’Instant architecture.’, ACM Transactions on Graphics (TOG, 22, pp. 669-677

340 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Spaces Identity Evaluation aNd Assignment - SIENA

Aducktypingapproachforautomaticrecognitionandsemanticenrichment of architectural spaces

Zhelun Zhu1, Ugo Maria Coraglia2, Davide Simeone3, Antonio Fioravanti4 1,4Sapienza University of Rome 2,3www.aec-agency.com 1,4{zhelun.zhu|antonio.fioravanti}@uniroma1.it 2,3{um.coraglia|d.simeone}@aec-agency.com

This paper presents the development of SIENA - Space Identity Evaluation aNd Assignment - based on duck typing for automatic recognition and semantic enrichment of the architectural spaces. This method is known in computer science as a form of abductive reasoning and leverages on the observable features of an object in order to establish its recognition. As result, the spatial identity is object-oriented and can be dynamically defined. In this research, the duck typing approach has been achieved with the support of BIM methodology and graph database. The former allows information-based modeling of an architectural project while the latter makes possible the representation of the knowledge along with their relationships. Consequently, this research may have many possible applications, especially as a valid design support tool in the very first design stages. Furthermore, an efficient spatial identity detection could contribute to the development of further human-machine interactions and therefore a possible optimization of the design process.

Keywords: Semantics, Graph database, Duck typing, Space identification

INTRODUCTION misunderstandings among involved actors due to So far in architectural design, the room identitifi- different meaning that these people attribute to dif- cation process must be statically and manually de- ferent symbols, icons, graphic conventions, measure fined by designers according to his/her experience units and terms. Therefore, the knowledge-based and building functional program. This means that analysis and simulations concerning the space iden- the attribution method does not automatically con- tity have a limited level of reliability and objective- sider its semantical functions and properties and, as ness (Wu & Zhang, 2019). consequence, the identity attribution process must BIM methodology relies on information of con- be manually updated and upgraded as soon as the stituent elements and their relationships (Autodesk, project changes. Moreover, it should be considered 2002). Nevertheless there is still a certain arbitrary

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 341 Figure 1 Consequence of designer- dependent attribution of space identity

level related to the spaces identity assignment. As a ductive reasoning and it allows the identification of matter of fact, Figure 1 shows an example of an incon- an object by its observable characteristics (Ramalho, sistent manual definition of a room identity obtained 2015). The ”duck test” is defined by this expression ”if in a BIM software, in this case Revit. As we can see, I see a bird that quacks like a duck, walks like a duck, has the name assigned by the designer to this room does feathers and webbed feet and associates with ducks, I’m not correspond to its function and there would not certainly going to assume that it is a duck”. The ap- be enough space for furniture for that scope such as plication of this logic in computer science has led to a bed, even in case of future renovation. This means a dynamical establishment of an object’s semantic that the semantic attributes of the spatial identity through its very properties (Milojkovic et al., 2017), relies on two aspects: its capability and its endow- rather than the type that the object has been manu- ment. The first indicates that a room is characterized ally assigned. by enough dimensions and predispositions for that On the other hand, the data structure based on function. The second aspect, on the other side, indi- graphs allows the information storage as well as the cates that the room is equipped with necessary furni- representation of both properties and relationships ture and devices that allow it to perform that specific between the elements. This feature embodies the use. key factor in the proposing methodology together This research wants to present an innovative ap- with its characteristic to provide real-time graphical proach, called SIENA - Spaces Identity Evaluation aNd visualization of the relationships. Furthermore, the Assignment, based on the implementation of duck duck test can be obtained by comparing the similar- typing method within graph database formalizing ity between two graphs, the project one and the ty- the building, its spaces and its components. pological one. Consequently, the room’s identity can In computer science, duck typing is the applica- be semantically established and assigned once it sat- tion of the ”duck test” to determine the identity of isfies the features of being a typological room such as an object based on its features. It is a form of ab- a kitchen, bedroom or bathroom.

342 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 OBJECTIVE path (Fig. 2). This research inquires about the semantic meaning In graph theory, two structures can be consid- of buildings’ digital entity by its constituent compo- ered equivalent even if the shape of an edge or the nents and functional spaces. Hence, entity semantic position of the vertices are different. And this brings attributions are related to the identification of its own to the concept of the isomorphism which means that features, and relations linked with other entities. This two graphs are isomorphic to each other if they can process could lead to an automated spatial semantic be transformed, by re-organizing vertices, in such identification and enhancement, improving the co- a way that they become equivalent graphs. This herence between the assigned name and its effective implies that two isomorphic graphs have the same essence. Thus the proposed approach could reduce number of vertices, the same number of edges, the manual room’s identity corrections issues in case of same maximum and minimum degree and the same frequent changes and then achieve a more efficient length of the longest simple path. design process. Figure 2 Finally, the knowledge-based identity assign- Example of ment could be vital for further developments in subgraph (2.a) and human-machine interactions. This feature can be bipartite graph achieved in case the system is able to understand the (2.b). (Marcus & actual meaning and functions assigned to the room America, 2011) and consequently to respond to the designer’s re- quests.

STATE OF ART This research proposes the application of graph structures for the “duck test”, as the enbodyment of the SIENA approach. As defined in the graph theory (Marcus & America, 2011), the structure of a graph G = (V, E) is composed by a set of vertexes V = {V1, V2, ..., Vn} and a set of edges E = {E1, E2, ..., Em}. Each edge Ek is identified with a pair of vertices (Vi, Vj) and the degree of a vertex is the number of edges that occurs at that vertex. A subgraph of the graph G is a graph that is con- tained within G; all vertices and edges of the sub- graph are included in G. A graph is defined bipartite if its vertices can be separated into two sets X and Y in such a way that ev- So far, the application of graph databases in architec- ery edge in the graph has one endpoint in each set. ture consists in representing and analyzing the rela- A path is a sequence of vertices and edges in tionships between rooms of a building or different a graph such that the sequence alternates between areas of the city for design development and evalu- vertices and edges, starting and ending with vertices; ation (Dawes & Ostwald, 2013). As a matter of fact, each edges in the sequence joins the vertices that oc- the graph theory has been the vehicle of space syn- cur immediately before and after in the sequence. A tax analysis which focuses on the spatial configu- path that contains no repetitions is called a simple ration and their relationships (S. Bafna, 2013). Fig.

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 343 3 shows an example of graph visualization of habi- METHODOLOGY tation’s organization in “Architect’s Data” by Ernest The basic idea is that if there is a sufficient level of sim- Neufert (1980). ilarities between a typological room’s graph (duck) and a specific project room’s graph, which identity is still unknown, thereby it is possible to assign the Figure 3 identity of the typological room to the project room. Graph This process can be synthesized with Fig. 4. representation of In this research the adopted graph database is an habitation, NEO4J which is an open source software. Architect’s Data by The first step is the individuation of meaningful Ernest Neufert properties of “ducks”, which in this case have been considered as typological rooms: a kitchen, a sin- gle bedroom, a double bedroom, a WC and a bath- room. Therefore, features, and consequently their value, like spatial dimensions, minimum quantities of openings, required devices and equipment have been analysed. The adopted typological room standards are taken from the Building Code of Rome, Italy, which provide quantified requirements needed for the ob- tainment of the habitability licence (Regolamento Generale Edilizio Del Comune Di Roma, 1934) [1]. Then the requirements of the considered typo- logical rooms have been extrapolated from the Build- ing Code and therefore transferred into the graph database. As pointed out in the Introduction chap- ter, there are two kinds of requirements: capability and endowment. Hereby this aspect has been taken in consideration during the creation of the relation- However, the space syntax theory describes the spa- ships. In order to facilitate data comparison and ex- tial features and configuration of buildings or ur- portation in the next phases, these features are de- ban areas but does not recognizes their identities by fined according to the Revit classes nomenclature means of a semantic attribution. Therefore this paper and the corresponding graphs that have been cre- presents an implementation of the graph database ated in NEO4J. The Fig. 5 shows in particular the re- in the automatic assignment of the spatial identifica- quirements of a bathroom, according to the building tion process. The exploitation of the graphs in this standards of the City of Rome. process provides graphical visualization of the rela- On the other hand, many testers have been cre- tionships between rooms and their semantic attribu- ated to be compared with the “ducks” in order to tions. Furthermore, a graph database allows queries verify their identity. In this research, these rooms and logical operations which could provide needed have been modelled in Revit and the data extrapo- functionalities in the SIENA process. lation achieved through Dynamo. With the support of the adopted visual programming language, values of above individuated features of testers, have been

344 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 4 Schematical reasoning of the duck test in SIENA

organized and then transformed into CSV - Comma not the focus of the present research. The simplifica- Separated Value - file and thus imported into the tion consists in the manual assignment of the name graph database. This research has considered as sig- of objects under the ”Comment” parameter, in the nificant tester three different rooms, shown in Fig- ”Data Identity” section. This process is a common ure 6. The result of the comparison process will be practise among design teams in order to coordinate presented in the next paragraph, followed by discus- and uniform parameters of BIM objects coming from sions. different origins, in the same project. In order to facilitate the data extrapolation, this work has adopted a BIM objects management re- garding the constituent elements’ identity, which is

Figure 5 Graph representation of the “ducks” (Typological Rooms)

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 345 RESULTS: Figure 6 About the graphs structure: Utilized testers in The resulting graphs, both from typological rooms this research and project ones, are bipartite and endowed with the same length of the longest simple path. Further- more, if the project’s graph G can be matched with the graph of its “duck” D, the latter is subgraph and isomorphic to G. During the data extrapolation process from Dy- Thereafter, these labels have been summoned in Dy- namo of this work, no sorting process has been done namo by their parameter’s name and therefore orga- except for the walls. Hence the degree of G corre- nized into a list of lists. The sorting process consid- sponds to the number of elements exported from the ers the CSV file’s format and the order needed for the project’s model minus one, which is the vertex rep- Cypher Statement, essential in afterward data impor- resenting Family_Rooms. This is the reason why the tation in NEO4J. In this work the creation of the graph degree of G is higher to D’s, and there are unmatched database has been subdivided into many steps, cor- vertexes within the G. These are furniture like tables, responding distinct Cypher operation: chairs and other furniture that don’t contribute to the Starting with the creation of the nodes (Fig. 7, satisfaction of the Building Code, but are present in step 1), this process involves information from the the project model. project room. As a matter of fact, nodes contain nec- essary data in order to provide a knowledge-based About the comparison: database. Then, each project room’s graph can be The graph representation offers a visual and imme- created by connecting the room identity’s node with diate solution for the SIENA problem. As can be ob- the feature ones (Fig. 7, step 2). Consequently, served in the Figure 9, where a room’s identity ver- graphs of the project rooms are ready to be com- tex of G is coloured with blue and the D ones are pared with the typological rooms ones. lighter blue, the satisfaction of the duck test can be However, in order to avoid useless cartesian easily verified. The dashed line distinguishes the G products during the comparison process between graph from the D ones and the correspondence be- nodes, which would lead to a consistent compu- tween project nodes and the typological ones are tational effort, a conditional matching method has highlighted by orange-color link. The result of this been adopted (Fig. 7, step3). Even though all comparison can be exported through a matrix via requirement-fulfilled relationships are wanted. CSV file and thereby analyzed in other software, in- In synthesis, the adopted workflow can be de- cluding Dynamo. Thereby it is possible to establish a scribed by the process diagram in Figure 8. reverse data flow from the graph database, with the results from this process. Figure 7 Moreover, the comparison process in the graph Cypher statements database allows the representation of relationships to create graph among all requirement-fulfilled vertexes. The eval- nodes and their uation of these similitudes, in space capabilities and relationships device equipment, could provide the flexibility of the comparing room to change its functional destination in the future and thereby offer solutions for renova- tion, with least operations.

346 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 8 Workflow of the adopted SIENA approach

Hereafter there are been reported the result of the opening must be at least 1.125mq. In the case of the comparison process of the testers (Fig. 9). In par- the P_Room1 the room surface is about 5mq and its ticular, there are going to be discussed in detail. windows area is 1.05mq therefore does not fulfill the Result of the tester 1: As can be observed, all required standard for bedrooms. requirements to be a bathroom have been satisfied, Result of the tester 2: All requirements of a the P_Room1 is therefore a Bathroom, even though Kitchen have been satisfied, then P_Room2 is a it could be a WC at the same time. This is because Kitchen. On the other hand, the tester 2 differs from the graph of WC is the subgraph to the Bathroom’s the bedroom because the ventilation and the light- one. Moreover, it is endowed with more equipment ing requirements needed for a bedroom’s window rather than a WC like a Sink, a Bidet as well as a Bath- are not satisfied (1/8 of the room’s area) while it is en- tub. In particular, this element can fulfill the require- dowed with many special equipments like Refrigera- ment of the Building Code which requires the endow- tor, Chimney, Cooktop and Sink. Another interpreta- ment of a Bathtub OR a Shower in the bathroom (as tion of this result is that in case of functional changes, mentioned in Fig 5). On the other hand, its surface the P_Room2 can be transformed into a bedroom area, as well as its windows area, doesn’t satisfy all re- working only on the window’s properties rather than quirements to be a bedroom. As a matter of fact, the other functions which would require more opera- Building Code of Rome establishes that the minimum tions. surface of the bedroom must be at least 9mq in case Result of the tester 3: P_Room3 satisfies all re- of single bed use and 14mq in case of 2 beds. At the quirements for being both a single and double bed- same time, the area of the window of bedrooms, in room. These are isomorphic to each other. Accord- order to provide a good air/light refurnishing, must ing to the Building Code, bedrooms have the same, be at least 1/8 of the room’s surface. This means that but more restrictive requirements in case of a double

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 347 bed use (area single bedroom must be at least 9 mq FUTURE DEVELOPMENTS while area double bedroom must be minimum 14mq For instance, this research could be extended from in order to provide enough breathable air for 2 peo- “space identity” to “building identity”, which means ple). It is also possible to observe, that the tester sat- an upper hierarchy of organization. It is possible isfies the capacity requirements to be a kitchen, but it therefore, changing the scale of the graph database, is not equipped with the necessary devices (Endow- to define the semantic meaning of entire buildings. ment). This means that the P_Room3 can be poten- The emerging challenge represents the establish- tially transformed into a Kitchen in case of a future ment of the relationships between spaces which is vi- renovation, as long as endowed with all necessary tal for the correct performance of activities. devices and therefore their system connections (gas, Moreover, the referencing “ducks”, which in this plumbing, water and so on). work belong to the Building Code of the city of Rome, can be any other technical standards like hy- DISCUSSION giene requirements of special facilities in hospitals This work has considered many typological models, or building’s fire prevention and protection regula- obtaining its semantic definition by essential require- tions as well as building programme with stakehold- ments from the Building Code of Rome. Neverthe- ers’ needs. Thereby, it is possible to automatically less, there can be many additional functional and di- evaluate the conformity of a design to the required mensional features to be take into consideration. For standards. example, the accessibility of a room which absence As mentioned during the results’ discussion, an- compromises its functionality. The question here is other potential application of the SIENA approach is that the entrance into a space can be achieved both finding out similar functions in the case of renovation from a vertical opening as well as thorough stairs or and functional conversion process. The similitudes elevators. From an operative point of view, those ele- analysis could provide valid design support pointing ments should be considered within the same seman- out those functions achievable with minimum oper- tic class while so far they belong to different classes in ations and therefore most affordable ones. As result, Revit (they are called “Families”). On the other hand, the proposed approach could bring to time and en- the adopted area and volume requirements doesn’t ergy, therefore cost saving, during the design process consider the proportions between dimensions which for the restoration of an existing room. could affect the comfort of internal spaces. These On the other hand, further development of this particular cases require a deeper analysis of this ap- work is its reversed side. The idea is to take ad- proach. vantage of the synthesized semantical features of Moreover, in order to focus on the final objec- spaces which can be defined by designer’s creativ- tives of this research, and imagining what happens in ity as well as through technical requirements (dimen- the actual design process, this methodology has con- sional, functional, structural, fire prevention and so sidered only terminal equipment (like cooktop, sink, on), and import them into the project. Consequently, chimney) but not the actual ducts and/or plugs (like the resulting design is naturally endowed with se- gas connection, water supply and draining, chim- mantic meanings and it can satisfy a priori all nec- ney pipe). This feature could be solved only in a essary requirements. This achievement can provide very detailed model creation which is rare in a com- valid design support in answer to the intertwisted mon design process, especially during its very first technical requirements and allow the management stages. Nevertheless, it could be inquired in the fu- of complexities in modern projects. ture progress of this approach in order to improve knowledge-based spatial recognition processes.

348 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 9 Result from the comparison

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 349 CONCLUSION The SIENA approach based on the “duck test”, could represent an innovative tool to support design, espe- cially in the very initial stages of the design process. Through this strategy, the spatial identity can be es- tablished by its semantic attributions taking into con- sideration its dimensional capabilities and device en- dowment. Therefore the SIENA approach can config- ure, in the meanwhile, as a useful checklist of techni- cal requirements as well as design purposes. Eventu- ally, human designers may be relieved to verify the compliance of their project with the building stan- dards, and therefore focus on more creative activities. The proposing strategy aims to the automatised and objective mode of the spatial recognition which could provide remarkable improvements in further human-machine interactions. As a matter of fact, es- tablishing the semantic meanings of spaces could be the further step of KAAD, Knowledge-based Assistant for Architectural Design.

REFERENCES Autodesk, . 2002, ’White paper building information modeling’, Autodesk Building Industry Solutions, /, pp. 1-7 Bafna, S. 2003 ’SPACE SYNTAX, A Brief Introduction to Its Logic and Analytical Techniques’, Environment and Behavior, Vol 35 No.1, January 2003, pp. 17-29 Dawes, M.J. and Ostwald, M.A. 2013, ’Applications of graph theory in architectural analysis: past, present and future research’, NOVA - University of Newcastle Research Online, /, pp. 1-36 Marcus, D.A. and America, M.A. 2011, Graph Theory: A Problem Oriented Approach, Mathematical Associa- tion of America Milojkovic, N., Ghafari, M. and Nierstrasz, O. 2017 ’It’s Duck (Typing) Season!’, IEEE International Conference on Program Comprehension, pp. 312-315 Neufert, E. 1980, Architect’s Data, Blackwell Science Ltd. Romalho, L. 2015, Fluent Python, O’Reilly Media Inc Wu, J. and Zhang, J. 2019, ’New Automated BIM Object Classification Method to Support BIM Interoperabil- ity.’, Journal of Computing in Civil Engineering, 33(5), pp. 1-56 [1] https://www.comune.roma.it/PCR/resources/cms/d ocuments/CC5261_34.pdf

350 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Consistency Checker

An automatic constraint-based evaluator for housing spatial configurations

Hardik Arora1, Jessica Bielski2, Viktor Eisenstadt3, Christoph Langenhan4, Christoph Ziegler5, Klaus-Dieter Althoff6, Andreas Dengel7 1Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) / Go Reply GmbH 2,4,5Technical University of Munich (TUM) 3,6Deutsches Forschungszentrum für Künstliche Intelligenz (DFKI) / University of Hildesheim 7Deutsches Forschungszen- trum für Künstliche Intelligenz (DFKI) [email protected] 2,4,5{jessica.bielski|langenhan|c.ziegler}@tum.de 3,6,7{viktor. eisenstadt|klaus-dieter.althoff|andreas.dengel}@dfki.de

The gradual rise of artificial intelligence (AI) and its increasing visibility among many research disciplines affected Computer-Aided Architectural Design (CAAD). Architectural deep learning (DL) approaches are being developed and published on a regular basis, such as retrieval (Sharma et al. 2017) or design style manipulation (Newton 2019; Silvestre et al. 2016). However, there seems to be no method to evaluate highly constrained spatial configurations for specific architectural domains (such as housing or office buildings) based on basic architectural principles and everyday practices. This paper introduces an automatic constraint-based consistency checker to evaluate the coherency of semantic spatial configurations of housing construction using a small set of design principles to evaluate our DL approaches. The consistency checker informs about the overall performance of a spatial configuration followed by whether it is open/closed and the constraints it didn't satisfy. This paper deals with the relation of spaces processed as mathematically formalized graphs contrary to existing model checking software like Solibri.

Keywords: model checking, building information modeling, deep learning, data quality

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 351 INTRODUCTION (Elango and Devadas 2014a). Inspired by the works of The development and design approaches of archi- Elango and Devadas (2014a), a DL approach is devel- tecture are characterized by continuous change due oped to suggest design variations, based on a large to social, ecological, and furthermore technological dataset of reference buildings, for architects in the conditions. Contemporary and future design qual- early design phase. However, there seems to be no ity assessment take place against the background method to evaluate highly constrained spatial con- of these ever-changing framework conditions. In- figurations for specific architectural domains based spired by system theory, the first computer-based ap- on basic architectural principles and everyday prac- proaches were developed revolving around design tices. The necessary data quality for a DL approach rules (Purcell et al. 1990), but were unable to exhaus- not only refers to quantitative pieces of information tively formalize the complexity of architectural de- but also refers to the qualitative aspects. The lack of signs. The second generation of the design method- such an evaluator can have severe effects on the qual- ology movement in the 1970s, represented by Horst itative performance of different DL approaches. This Rittel and others, viewed the design process as an in- paper introduces an automatic constraint-based con- dividual process that could only be described incom- sistency checker to evaluate the coherency of spatial pletely (Richter 2010). From the current perspective, configurations of housing construction according to the formalization of complex cases in architecture is a set of rules based on general principles of architec- not sufficiently solved and is referred to as data ac- ture as the data used for training the DL approach quisition bottleneck. To remedy these shortcomings, should be coherent with architectural principles and Langenhan (2010;2013;2017) introduced the novel everyday practices. approach ‘semantic building fingerprints’ that facil- itates spatial relationships. The digital semantic fin- PROBLEM STATEMENT gerprint of buildings describes the main semantic The process of designing spatial configurations is features of design, forming the basis for similarity as- an amalgamation of ill-defined design problems and sessment to deal with ambiguities and complexities non-linear decision-making. Horst Rittel and Melvin of architecture. However, beginning in the 1980s, Weber (1973) stated that design problems them- the digital approaches of case-based reasoning (CBR) selves are wicked problems as they can’t be defini- have been introduced to the field of Computer-Aided tively described. “The design process is complex Architectural Design (CAAD), influencing the build- due to its content, context, stakeholders, ill-defined ing design and leading to artificial intelligence (AI) problems, and multifaceted interactions. Further- research in the building industry in the form of case- more, each design process has special characteristics based design as early as the 1990s. Because of the which are not easily standardized. Gann et al. (2003) gradual rise of AI and its increasing visibility archi- stated the difficulty of quantifying the quality of ar- tectural deep learning (DL) approaches are being de- chitectural design since it consists of both tangible veloped and published on a regular basis such as re- and intangible facts and objective-subjective com- trieval (Sharma et al. 2017) or design style manipula- ponents.” (Harputlugil et al. 2014, p.139). Simultane- tion (Newton 2019; Silvestre et al. 2016). The origi- ously, the decision-making processes in architecture nal architectural design process without any compu- are non-linear and highly interactive. It is a mix of ra- tational support suggests a wide range of different tional and intuitive decisions, and there are no step- designs as inspiration (Richter 2010) and for gener- by-step procedures (Elango and Devadas 2014b, p.1). ating alternatives as a common practice, which can Architectural data is complex and error-prone due consequently be identified as an incremental learn- to the difficulties in defining both design problems ing process with iterative evaluation by the architect and design decision-making, as mentioned above. It

352 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 is necessary to ensure the data quality if it is used cally done through post-occupancy evaluation (POE), for training Machine Learning (ML) or Deep Learn- which is a systematic evaluation of the occupant af- ing (DL) approaches. The quality of data determines ter a certain period of time of inhabitation. How- the performance of any ML or DL approach (Sessions ever, created for satisfaction as an empirical basis and Valtorta 2006). The data quality not only refers to improve existing buildings, it is deemed unfit for to quantitative pieces of information but also refers early design stages (Harputlugil et al. 2014). Differ- to the qualitative aspects. The lack of such an evalua- ent strategies, such as the Design Quality Indicator tor can have severe effects on the qualitative perfor- (DQI) (CIC 1999) and Laseau’s (2000) variables for typ- mance of different DL approaches. ical design problems, have been applied to assess the design quality of architectural design decisions RESEARCH CONTEXT & RELATED WORK in the early phases. The design decisions and pri- During the currently running research project [metis- oritizing of the different variables within these first II] (2020-2023), we examine and develop DL-based design phases have a significant impact on ensuring methods and approaches for the support of the early the quality of the end product. In the same way as conceptual phases in architectural design. Taking the ”information on design quality important is [dur- into account the vagueness and uncertainty of ar- ing these early stages] to expanding the capabilities chitectural design data in the form of graph-based of the design team to make well-informed choices” spatial configurations, we investigate how auto- (Harputlugil et al. 2014, p.140), it is needed to en- completion of floor plans (comparable to, for exam- sure the data quality for deep learning. Drawing from ple, sentence completion on modern mobile gad- the Vitruvius’ principles, the DQI, and Laseau’s (2000) gets) can be achieved using artificial neural net- variables, Harputlugil et al. (2014) divide and criterias works. Based on early sketches of the building de- of the architectural design process as follows: func- signs, rooms and the possible relations between tionality, built quality, and impact, depending on the them are suggested to the architect to enhance the culture, society, and technology of the era. The func- early ideation process. The auto-completion meth- tionality has been shown to be further divided and ods are intended to be a helpful tool for architects prioritized by use e.g., fit for functionality, access e.g., during the early conceptual process to help them local and interior access, and finally space e.g., rela- overcome design bias. Providing them with the dif- tion of spaces. Overall, the architectural design de- ferent design continuation options is intended to cre- cision process can be depicted as an Analytic Hierar- ate interaction patterns to assess their own design chy Process (AHP) with Multi-Criteria Decision Mak- decisions and examine the possible further develop- ing (MCDM), focusing on the use of the building, its ment of the current spatial configuration state. Suc- individual spaces, and their interrelations, in the early cessful approaches for purposes other than ours use design phases. architectural image data: as examples, search for sim- Constraint-based approaches are generally used ilar designs (Sharma et al. 2017), modification of the in architecture to design spatial configurations. Lay- design style (Newton 2019; Silvestre et al. 2016), or out planning is a historical design activity that af- estimation of the layout in 3D (Sun et al. 2019) can fects the characteristics and performance of a build- be named. Even though new DL approaches are re- ing throughout its lifecycle (Shikdar et al. 2010). Ac- searched in the field of architecture, there still lacks a cording to the authors, the design constraints are a method to evaluate the architectural design quality guide to search spatial solutions. Constraint-based to ensure good data quality for training different DL approaches allow the designer to interact with the approaches. layout planning process, which simulates the itera- Architectural design quality assessment is typi- tive nature of the creative design (Shikdar et al. 2010).

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 353 Damski and Gero (2006) suggested a system to de- APPROACH velop space layout topologies for spatial configura- This paper introduces an automatic constraint-based tions using an evolutionary approach. They consid- consistency checker to evaluate the coherency of se- ered spatial layouts as a set of topological and direc- mantic spatial configurations of housing construc- tional constraints, which was used as a fitness func- tion according to a set of rules. The consistency tion in the evolutionary system. checker informs about the overall performance of a The constraint-based approaches mentioned semantic spatial configuration followed by whether above only provide a solution to the design prob- it is an open floor plan (‘open’/‘closed’) and the con- lems of spatial topologies. However, currently, straints it didn’t satisfy. Semantic spatial configura- no constraint-based approaches are used to evalu- tion informs about the semantics associated with a ate a semantic spatial configuration. In this work, spatial configuration (i.e., building floor plan). Se- a constraint-based consistency checker was devel- mantics refers to the information regarding different oped to ensure coherency of spatial configurations connection types connecting different room types (see figure 1) based on quantifiable and countable in a spatial configuration.In our current work on a criteria, which results in improved data quality and DL-based spatial layout auto-completion system that quantity for training different DL approaches. makes recommendations for architects in the early design phases, data quality plays an essential role.

Figure 1 Main objective and approach to ensure data quality.

354 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 The data used for training the DL approach should another room, e.g., bathroom-passage- sleeping be coherent, otherwise, the auto-completion system room-passage-living room. can recommend rooms, and connections that seem • If a room has only one connection type, it should implausible to architects, e.g., connect a bedroom be a passage, door, or entrance. If a room has with a kitchen via a window. Due to the lack of more than one connection, it needs to have at such an evaluator, the consistency checker was de- least two direct connections, i.e., via passage/- veloped. The semantic spatial configurations are for- door, except if the connection type is the en- malized as graphs, where the nodes of the graph rep- trance. resent the room types, and the edges between the A web-based tool was developed to ensure that nodes of the graph represent the connection types. architects can reexamine the rules suggested for We developed a set of 11 rules to evaluate the con- the consistency checker. The web-based tool dis- sistency of the data at our disposal. The main idea plays information regarding the spatial configura- behind these 11 rules is to omit the semantic spa- tions present in the database in a tabular format. tial configurations which don’t adhere to the general Each counted row starts with the name of the Ar- rules from the final dataset (dataset for training DL chitectural GraphML (AGraphML) file (Langenhan, approaches). Each spatial configuration has to pass 2017). This file is taken from a dataset that was pre- each of the consistency rules. The consistency rules viously created by combining different databases for are the following: e.g. geometric, topological and lexical data (Roith, • No spatial configuration should contain a room Langenhan & Petzold, 2019). The following column with no connection to any other room. shows the consistency score, i.e., the number of rules • No spatial configuration should contain the passed by the spatial configuration. Following, the room-type building services. number of checks the spatial configuration failed are • Every spatial configuration must contain at least mentioned within square brackets. The next column one bathroom/toilet, one kitchen, and one presents the number of rooms in the spatial con- sleeping/living/generic room, except if it is a figuration, and finally whether a spatial configura- one-room-apartment. tion is open/closed (see figure 2). Furthermore, the • Every spatial configuration must contain at least graph-based representation (i.e., image) of the spa- three rooms. tial configuration is depicted, which is created using • A passage shouldn’t be allowed to connect a a Python module known as Graph-tool. The last four kitchen and a sleeping room. columns of the web-based tool consist of radio but- • A passage from the bathroom/toilet should only tons for each architect - in this scenario: the two ar- be allowed when it connects to a sleeping room. chitects A1 and A2 - to approve or disapprove. Each • No spatial configuration should have direct ac- architect evaluates two different topics within their cess (i.e., via passage/door) between the bath- respective column: room/toilet and kitchen. • Does the spatial configuration of the graph vi- • Each room should have at least an edge connec- sualisation adhere to architectural principles for tion to another room using a door, passage, en- floor plan layouting of residential housing? trance, or wall. • Is the apartment type, consisting of the room • If a room connects with another room via a wall, count for habitable rooms and floor type it must connect to another room via a door/pas- (‘open’/‘closed’), correctly labelled by the sys- sage. tem? • There should be no passage connection be- tween the living room and bathroom/toilet via

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 355 Figure 2 Overview of the columns of the web-based tool.

Architects also have an option to select ‘Unsure’ for the inclusion in the final dataset), then the row rep- evaluating the spatial configuration about whether it resenting the spatial configuration has a white back- should be included or not in the final dataset. There- ground. If both the architects approve it, then the fore, they could separately discuss the said floor plan. row’s background color should be green (see figure If none of the architects select an option (i.e., ap- 3). If both the architects disapprove of it, then the proval or disapproval of a spatial configuration for row’s background color should be red (see figure 3).

Figure 3 The web-based tool with both architects either approving or disapproving the spatial configuration.

356 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Figure 4 The web-based tool with either of the architects is unsure or there is a disagreement regarding the approval or disapproval of the spatial configuration.

If both the architects have different opinions, i.e., one files, which were also approved by the architects of approves, and the other disapproves, then the row’s our research group. background color should be grey (see figure 4). If ei- ther of the architects is unsure of it, then the row’s CONCLUSION AND FUTURE WORK background color should be blue (see figure 4), and The main idea of this work is to ensure the use of se- the name of the AgraphML gets added to a different mantic spatial configurations for training different DL list accessible via “List of unsure files” on the home- approaches and check the results. The consistency page. checker is an important module for the application As mentioned above, the consistency checker of DL in architecture. Since a trained neural network evaluated semantic spatial configurations, followed is a black box, the consistency checker helps in evalu- by a manual evaluation by architects of our research ating the results of the trained neural network. In this group. The dual evaluation ensured that spatial con- work, we presented an automatic constraint-based figurations approved by both the architects and the consistency checker to evaluate the coherency of se- consistency checker are in the final dataset. After the mantic spatial configurations of housing construc- first round of manual evaluation of spatial configura- tion according to a set of rules. The data evaluation of tions the rules of the consistency checker were reex- the consistency checker and manual evaluation per- amined. Due to the reexamination, we were able to formed by the architects was found to be coherent. revise the consistency rules for the second iteration. However, the rules of the consistency checker were The second iteration with additional and improved formalized by the architects of our research group, consistency rules resulted in more approved spatial who also evaluated the consistency checker. configurations for the final dataset. Additionally, the proposed constraint-based A total of 597 AgraphML files representing the se- consistency checker will be integrated into an ex- mantic spatial configurations were evaluated using isting DL pipeline as a use case, which allows both the consistency checker with the revised set of rules. the training data and the recommendations of the The consistency checker approved 286 AgraphML

Rule based systems and shape grammar - Volume 2 - eCAADe 39 | 357 Neural Network to be validated by the consistency stages’, Advanced Engineering Informatics, 27(4), pp. checker. In our current work on a DL-based spatial 413-426 layout autocompletion system (metis-II) that makes Laseau, P 2000, Graphic Thinking for Architects and De- signers, John Wiley & Sons recommendations for architects in the early design Newton, D 2019, ’Generative Deep Learning in Architec- phases, data quality plays an essential role. The maxi- tural Design’, Technology|Architecture + Design, 3, pp. mum number of architects using the web-based tool 176-189 is currently two , while in the future, we aim to scale Purcell, PA, Mitchell, WJ and McCullough, M 1990, The the web-tool to accommodate a larger diaspora. The Electronic design studio: Knowledge and media in the updated version of the web-based tool will also allow computer era, MIT Press Richter, K 2010, Augmenting Designers’ Memory: Case architects to add new constraints. Based Reasoning in der Architektur, Ph.D. Thesis, Bauhaus-Universität Weimar REFERENCES Rittel, HWJ and Weber, MM 1973, ’Dilemmas in a General Damski, JC and Gero, JJ 1997 ’AN EVOLUTIONARY Theory of Planning’, Policy Sciences, 4, pp. 155-169 APPROACH TO GENERATING CONSTRAINT-BASED Roith, J, Langenhan, C and Petzold, F 2017, ’Sup- SPACE LAYOUT TOPOLOGIES’, CAAD Futures 1997 porting the building design process with graph- Elango, M and Devadas, M 2014 ’Identification of Param- based methods using centrally coordinated feder- eters for Evaluating Architectural Design. - A Case ated databases’, Visualization in Engineering, 5, p. Ar- of Housing in Hot and Humid Region, Chennai City, ticle 20 India’, Proceedings of ARCHDESIGN / Architectural De- Sessions, V and Valtorta, M 2006 ’The Effects of Data sign Conference on Design Methodologies Quality on Machine Learning Algorithms’, Proceed- Elango, M and Devdas, M 2014, ’Multi-Criteria Analysis of ings of ICIQ 2006 the Design Decisions In Architectural Design Process Sharma, D, Gupta, N, Chattopadhyay, C and Mehta, S during the Pre-Design Stage’, International Journal of 2017, ’DANIEL: A Deep Architecture for Automatic Engineering and Technology, 6, pp. 1033-1046 Analysis and Retrieval of Building Floor Plans’, 14th Gann, D, Salter, A and Whyte, J 2003, ’Design Quality In- IAPR International Conference on Document Analysis dicator as a tool for thinking’, Building Research and and Recognition (ICDAR), 01, pp. 420-425 Information, 31, pp. 318-333 Shikder, S, Price, A and Mourshed, M 2010 ’Interactive Harputlugil, T, Gultekin, A, Prins, M and Topcu, YI 2014, Constraint-Based Space Layout Planning’, no source ’Architectural Design Quality Assessment Based On given Analytic Hierarchy Process: A Case Study’, Journal of Silvestre, J, Ikeda, F and Guéna, F 2016 ’Artificial imagi- The Faculty of Architecture, 3, pp. 139-161 nation of architecture with deep convolutional neu- Langenhan, C 2017, Datenmanagement in der Ar- ral network “Laissez-faire”: Loss of control in the es- chitektur. Untersuchung zur Organisation von quisse phase’, CAADRIA 2016, 21st International Con- Entwurfsinformationen in IT-Infastrukturen und ference on Computer-Aided Architectural Design Re- Nutzungsmöglichkeiten in wissensbasierten Syste- search in Asia - Living Systems and Micro-Utopias: To- men, Ph.D. Thesis, Technische Universität München wards Continuous Designing Langenhan, C and Petzold, F 2010, ’The Fingerprint of Sun, C, Hsiao, CW, Sun, M and Chen, HT 2019 ’Horizon- Architecture: Sketch-Based Design Methods of Re- net: Learning room layout with 1d representation searching Building Layouts through the Semantic and pano stretch data augmentation’, Proceedings of Fingerprint Of FloorPlans’, JournalofArchitectureand the IEEE Conference on Computer Vision and Pattern Modern Information Technologies, 4, pp. 4-13 Recognition 2019 Langenhan, C and Petzold, F 2015 ’BEYOND THE BUBBLE- [1] https://www.dqi.org.uk/ Computer-aided topological analysis and paramet- ric design of room configurations in university edu- cation’, Proceedings of eCAADe 2015, Vienna Langenhan, C, Weber, M, Liwicki, M, Petzold, F and Den- gel, A 2013, ’Graph-based retrieval of building in- formation models for supporting the early design

358 | eCAADe 39 - Rule based systems and shape grammar - Volume 2 Simulation, prediction and evaluation in design 360 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Approach to Auto-Recognition of Human Trajectory in Squares using Machine Learning-Based Methods

An application of the Yolo-v3 and the DeepSORT algorithm

Shaoji Wu1 1Tianjin University [email protected]

The square plays an essential role in contemporary urban space. Researchers had explored many methods to record the distribution of people in it. However, few of them study this issue using fine data. This study proposes a method recognized of the human trajectory using a machine learning-based computer vision algorithm, which can be divided into the following three steps. (1) the acquisition of video and the method of obtaining human trajectory. (2) cleaning of the raw human trajectory data. (3) to visualize the trajectory data. Based on the existing methodology, we take three example squares within the Tianjin University campus to illustrate it. We use trajectory map, people distribution heat map, and people walking speed heat map as visualization methods. The following two conclusions are drawn from the three examples. First, it is found that the human trajectory data derived from this method is more accurate when the UAV is flying at a lower altitude. Second, this study demonstrates that a passive Real Time Locating Systems (RTLS), based on a deep learning computer vision algorithm, can effectively obtain human trajectory data in a square. Third, this paper proves that the visualization method we used is effective.

Keywords: Human Trajectory, Squares, Machine Learning, DeepSORT, Yolo-v3, UAV

INTRODUCTION spection, cannot effectively reflect people’s location The square plays an important role in contemporary in squares due to their high human cost and low urban space, hosting optional activities and social ac- accuracy. This paper proposes a deep learning- tivities in cities and communities (Gehl J, 2011). Re- based method to derive the distribution of people in searchers have explored ways to derive a more favor- squares accurately and takes three squares within the able square layout through the distribution of peo- Tianjin University campus as examples. ple. However, traditional methods, such as record- ing people’s distribution through human visual in-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 361 Research Streams Yang, Chen, and Deng et al. have described that This paper traces three major research streams as fol- RTLS systems used in the field of architecture can lows. The first stream analysis the existing quantita- be classified into five main categories, namely GPS, tive studies on squares. The second research stream RFID, Bluetooth, Wi-Fi, and UWB (2019). Among these discusses why a vision-based deep learning algo- five systems, UWB is an active positioning system, rithm is chosen. The third research stream investi- and it requires a large amount of equipment for their gates studies using human trajectories to quantify use, such as the base stations (Anchor), Tags, and the built environment. Routers (Zetik, Sachs, et al. 2007, Huang and Wu, The Quantitative Study of Squares. Current re- 2017). The human trajectory data obtained from the search on squares has mainly focused on qualitative UWB is more accurate, but it not only takes longer to studies. For instance, Zakariya, Harun, and Mansor be deployed but also affects the subject’s response. et al. used Melbourne as an example to sort out the Passive positioning systems can also be divided into temporal evolution of events in squares (2014). two main categories: radio signals, such as GPS, and Current quantitative analyses of squares can visual signals, such as videos. RTLS using radio has be divided into the following three categories. many drawbacks, such as inaccurate positioning of Firstly, the square’s spatial characteristics are ex- GPS and the small application range of Bluetooth plored through the simulation of its physical environ- data (Yang, Chen, et al. 2019). ment. For example, Li, Jiang, and Wang et al. stud- This study attempts to build an RTLS using a pas- ied the relationship between the wind environment sive positioning system using a computer vision al- of a square and its surrounding urban built area us- gorithm based on deep learning, and there are three ing the wind network index (Rw) (2019). Secondly, advantages as follows. Firstly, this method allows the Mathematical theory is used to analyze the spatial collection of human trajectory data without affect- form and layout of squares. A good example of this is ing the subjects. Secondly, this system is very cost- that Liang, Hu, and Sun used fractal theory to evalu- effective, requiring only a UAV(Unmanned Aerial Ve- ate the green space’s spatial layout in four squares, hicle) to collect video data without requiring equip- including Union Square in San Francisco, USA฀May ment such as base stations. Thirdly, the system is Square in Buenos, Argentina฀Central Square in Qiqi- fast to deploy, making it suitable for large-scale, long- har, China and Seven-Star Square in Jincheng, China duration experiments. (2013). Thirdly, Virtual reality technology was used The specific algorithms used in this study to to compare the effects of squares with different sizes construct the passive positioning system are Deep- and Depth-Width ratios on the subjects and derive SORT (Simple Online And Realtime Tracking With A the relationship between its spatial scale and spatial Deep Association Metric) proposed by Wojke, Bew- quality (Xu, Liu, et al. 2012). ley, Paulus (2017), and Yolo-v3 (You Only Look Once) None of these studies used human trajectory presented by Farhadi and Redmon (2018). DeepSORT data to assess the square layout, which is the research is an improvement to the SORT algorithm proposed gap this study seeks to fill. in 2016 (Bewley, Ge et al. 2016). Kapania, Saini, and Goyal et al. used the DeepSORT algorithm for Mul- Methods of Obtaining Human Trajectory. A large tiple Object Tracking (MOT) of videos captured by number of researches have been done on Real Time UAVs and found that it performed well on the VisUAV Locating Systems (RTLS). RTLS can be divided into 2018 dataset (2020). Ahmed, Rodrigues, and Jeon et two main categories. One is active positioning sys- al. used DeepSORT and Yolo-v3 to establish an auto- tems, and the other is passive positioning systems. mated surveillance system for social distance during Using RTLS, researchers can obtain human trajectory the Covid-19 pandemic (2021). data.

362 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 1 Built Environment Evaluation Using Human Tra- Geographical jectory. With the increasing computing power, hu- distribution of man trajectory data is increasingly being used by re- research examples searchers. Mohamed, Qian, and Elhoseiny et al. pro- and their images pose a method to predict pedestrians’ social behav- ior in videos by human trajectories using Graph Con- volutional Network (GCN) (2020). Using Bluetooth data, Yoshimura, Sobolevsky, and Ratti et al. ana- lyzed the characteristics of visitor trajectories in the Louvre and found significant differences in behav- ioral patterns between long-stay and short-stay vis- itors (2014). Grauwin, Sobolevsky, and Moritz et al. used mobile traffic records to analyze people’s trajec- tories in three cities: New York, London, and Hong Kong (2015). Huang and Lin studied people’s social interaction behavior in an innovation incubator in Beijing using Wifi-derived crowd location data (2018). Following these researches, this study exploits the Spatio-temporal distribution characteristics of the human trajectory data derived from videos and attempts to find out the relationship between it and the square’s spatial layout.

Research Examples Figure 1 shows the three research examples, all of which are located on the campus of Tianjin Univer- sity. In this study, these examples are referred to as A, B, and C, which can be subdivided into two cate- gories. The first one is A and B, which are surrounded by buildings, and there are several entrances and ex- its around them. The second category is represented by C, which is surrounded by roads and water and has internal structures such as sculptures and memorial METHODOLOGY pavilions. Figure 2 is the technical roadmap of this paper, which can be divided into the following three steps. First, the acquisition of video and the method of obtain- ing human trajectory from it. Second, cleaning of the raw human trajectory data. Third, to visualize the tra- jectory data.

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 363 Figure 2 The technology roadmap

Obtaining Raw Human Trajectory Data Us- this study is labelImg, which is a graphical image an- ing UAV notation tool and label object bounding boxes in im- The UAV device used in this study is Dji mini2, which ages. We developed a dataset contains 756 images captures 4k (3840*2160) video as the base data for for the next step. analysis. This study uses the deep learning-based (2) Training the model. We use 80% of the images Yolo-v3 and DeepSORT algorithms to calculate each (i.e., 605) in the dataset for training and the other 20% person’s trajectory in the video. This section is pro- of them (i.e., 151) for evaluation. Due to the small size grammed on Python by TensorFlow and Keras as ma- of the dataset, we resize the images to different sizes chine learning packages and OpenCV as the com- every few batches. The resized image width takes val- puter vision package. This section can be divided into ues from the interval [320,608]. He, Zhang, Ren, et al. the following two steps. proposed that using this method could improve the Transfer Learning. We use a transfer learning(TL) ap- training model’s robustness and accuracy (2015). The proach to training the neural network to recognize batch size in this study is eight, and the epoch (i.e., pedestrians in squares. TL is an approach used to the number of training steps ) is 1,000. transfer information from one machine learning task Obtaining Raw Trajectory. There are three steps in to another, and it was proposed by Pan and Yang this section. (2009). In this study, we transformed the Yolo-v3 pa- First, we use Yolo-v3 for pedestrian detection, rameters downloaded from the official website into a which calculates every pedestrian’s position and gets model that can recognize people from the top view. their predict boxes in each video frame. We can ob- The two subdivided steps as follows. tain four data u, v, r, and h to represent the predict (1) Labeling the data. The labeling tool used in boxes, where (u, v) is its center coordinate, r is the

364 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 scale of it, and h is the aspect ratio. Second, due to its shortcomings, the Hungarian Second, we use DeepSORT to track people pass- algorithm may make errors when matching predict ing through in the entire video and obtain their tra- boxes between frames in videos. This error can result jectories. This algorithm uses a Kalman filter to pre- in the distance between two adjacent points in some dict each predict box’s motion, which works best for trajectories being so large that it is beyond the dis- linear systems, such as the human walk process in a tance that a normal human walking speed can reach. ⋆ ⋆ ⋆ ⋆ video. An octave vector (u, v, r, h, x , v , r , h ) To solve the problem, we remove a box plot’s outliers is used to represent the trajectory’s state of people (Figure 3(d)), which draw from the distances between ⋆ ⋆ ⋆ ⋆ at a given moment where x , v , r , and h are all two adjacent points in trajectories in a video. the velocities of u, v, r, and h between two adjacent frames. Then, we use the Hungarian algorithm to Visualization of Trajectory Data match predict boxes between frames to determine This paper takes three approaches to the visualiza- if the two persons detected in adjacent frames are tion of trajectory data: trajectory maps, people dis- the same one. Finally, each pedestrian’s raw trajec- tribution heat maps, and pedestrian walking speed tory can be drawn using the coordinates of its predict heat maps. This section is programmed on Python box’s geometric midpoint in different frames. by Matplotlib and PIL as packages. For further details The last step is to export the raw data as a CSV about the resulting image of this research, we refer to file, which is the basis for subsequent data analysis. 3.1. The method of drawing and interpretation of the three is as follows. Data Cleaning (1) The trajectory map visualizes the data- The Yolo-v3 cannot detect all the pedestrians in the cleaned trajectory data on an image, where each video, and the DeepSORT algorithm has problems trajectory is 1 pixel wide and trajectories are dis- with identity switches. Therefore, the raw trajectory tinguished from each other using random colors. data obtained from 2.1 should be cleaned before This kind of image clearly shows the trajectory of all analysis. Figure 3 illustrates the the boxen plots and pedestrians in the video. box plots used to clean the data in this study. The (2) The people distribution heat map shows how boxen plot (Figure 3(a-b)) is used to show the distri- people gathered in the square during the video bution of the data. And the box plot (Figure 3(c-d)) shooting. It rasterizes the video capture range and is used to show the data intervals that are removed. counts the number of predicted box centroids in This section is programmed on Python by pandas and each raster. The lighter the color of a grid, the more NumPy as data analysis packages and Matplotlib as pedestrians passed through the area. Conversely, the visualization package. This section can be di- the darker the color of it, the fewer people passed vided into the following two steps. through the corresponding area. First, the Yolo-v3 algorithm may have an error de- (3) The pedestrian walking speed heat map tection. Some errors occur in just a few frames, such shows pedestrians’ velocity in the square during the as recognizing a passing person’s shadow as a pedes- video. It also uses the same rasterization method as trian. Others appear throughout the video, such as the people distribution heat map. The lighter the recognizing a maintenance hole cover as a person. color of a grid, the faster the average speed of pedes- We remove the trajectories that occur less than ten trians passing through the corresponding area. Con- times to solve the former problem. Then, we draw versely, the darker its color, the lower the average ve- the box plot in terms of each predict box’s occurrence locity of people passing through it. number and then remove the errors by removing its outliers (Figure 3(c)).

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 365 Figure 3 (a) the boxen plot of each predict box’s occurrence number; (b) the boxen plot of the distances between all two adjacent points in trajectories; (c)the box plot of each predict box’s occurrence number; (d) the box plot of the distances between all two adjacent points in trajectories; RESULT AND ANALYSIS trajectory distribution in the three examples and pro- This study uses three squares within Tianjin Univer- vides an intuitive graphical interface to understand sity to illustrate how UAV can be used in obtaining people’s path selection tendencies in squares. As pedestrian trajectories. The videos used in this paper shown in Table 1, the trajectory directions in Exam- were taken on 14 and 15 March 2021, both of which ples A and B are mainly connected to their entrances were sunny days. To eliminate the effects of coinci- and building entrances around them. Example C’s dence, we shot three videos of 20min duration at dif- trajectories are divided into two main paths, one ferent times in each square. This chapter is divided connecting the northern and southern entrances to into two sections. One is the characteristics of the the square and the other connecting its east and pedestrian trajectory distribution in the three exam- north entrances. For the trajectory map, the Iden- ples. The other is an analysis of trajectory distribu- tity Switches phenomenon is more severe in Exam- tion’s causes. ple C than in Examples A and B, showing a relatively incomplete trajectory. The reason for this is that the The Human Trajectory Distribution in accuracy of the trajectory recognition is related to Squares the height of the image taken by the UAV. Specifi- Table 1 illustrates the trajectories of pedestrians in cally, the lower the height of the image captured by the three example squares of this study. Based on the the UAV, the clearer the pedestrian in the video. In three case studies, it can be demonstrated that the this condition, the higher the accuracy of the image method proposed in this study can effectively derive recognition algorithm will be as well. For example, the human trajectory data from the video and quan- Examples A and B were captured at a lower height. tify it. This section describes it respectively using tra- Therefore the people in them are clearer than Exam- jectory map, people distribution heat map, and peo- ple C in videos. As a result, the pedestrian trajectory’s ple walking speed heat map. accuracy derived using DeepSORT increases as the The trajectory map shows pedestrians’ precise shooting height decreases.

366 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Table 1 The trajectories of pedestrians in the three example squares

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 367 The people distribution heat map shows the strongest connection intensity links the north-west pedestrian density in the square in the three exam- and south-east entrances to the square. Another ex- ples. In this type of image, a grid with a lighter color ample is Square C, which has a low connection inten- means that its pedestrian density is higher. Con- sity between its southern and eastern entrances de- versely, a darker color indicates a lower density of spite having a connecting path. pedestrians in it. The path with the highest con- Second, The people walking speed heat map centration of pedestrians in Example A connects the characterizes pedestrians’ spatial distribution with north-west and south-east entrances to the square. different velocities in squares. A typical example is In Example B, this kind of path is distributed to the Square A, in which cyclists are traveling at high speed, south of the square. There are two highly distributed mainly on the path that connects the north-western pedestrian paths in Example C. One connects its east- to the south-western entrance. ern to the northern entrance. The other connects the north and south entrances, and its western side, DISCUSSION which is along the lake, has a significantly higher den- This study takes three squares within Tianjin Univer- sity of pedestrian distribution than the eastern side. sity as examples and uses deep learning algorithms The three people distribution heat map corresponds to derive pedestrian trajectory data. Accordingly, the to reality. relationship between the pedestrian trajectories and The people walking speed heat map shows the the square layout is discussed. However, three fac- distribution of pedestrian velocity in the three ex- tors might affect our analysis. amples. In this kind of image, a grid with a lighter First, this study only took videos in early spring, color means that its pedestrian speed is higher. Con- when Tianjin’s temperature was low (3฀-13฀ at the versely, a darker color indicates a lower density of shooting time). Fewer people were engaged in ac- pedestrian speed. Example A has a clear area with tivities in the square at this time. Therefore, the con- high velocity, which connects the north-western to clusions drawn in this paper may not apply to human the south-western entrance. This is because some activities in other seasons. pedestrians on the path are cycling, while others in Second, When shooting video with the UAV, it the square are predominantly on foot. The pedes- cannot remain perfectly stationary because of the trian walking speeds in both Example B and C are high winds at altitude. This results in some slight vari- mainly similar, with no clear areas of high pedestrian ations in the videos. The solution to this problem re- speed distribution. quires the use of a better performing UAV. Third, The deep learning algorithm used in this Representation of Spatial Phenomena study cannot completely calculate all pedestrians’ By comparing the spatial phenomena exhibited by trajectories in the video. The Yolo-v3 algorithm may the distribution of pedestrian trajectories in these have an error detection, and the DeepSORT algo- three example squares in 3.1 and their representable rithm might generate identity switches. Although spatial characteristics, we can find the following two cleaning the raw trajectory data can mitigate this ef- points. fect. However, other algorithms with better robust- First, the intensity of the connections between ness and accuracy are needed to improve the relia- the entrances and exits around the square can be bility of the methodology proposed in this paper. characterized using the people distribution heat map. Paths with a denser distribution of pedestrians can represent more vigorous spatial connection in- CONCLUSION tensity. For example, in Square A, the path with the The following two conclusions can be drawn from this paper.

368 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 First, it is found that the human trajectory data ban environments, Springer, Cham, pp. 363-387 derived from this method is more accurate when the He, K., Zhang, X., Ren, S. and Sun, J. 2015, ’Spatial pyra- UAV is flying at a lower altitude. When the UAV is fly- mid pooling in deep convolutional networks for vi- sual recognition’, IEEE transactions on pattern analy- ing at a high altitude, the pedestrians in the taken sis and machine intelligence, 37(9), pp. 1904-1916 video are less clear, which leads to a reduction in the Huang, W. and Wu, M. 2017, ’Dimensions of Information accuracy of pedestrian detection using Yolo-v3. This in Big Data Acquired by Indoor Positioning Systems: leads to a reduction in the accuracy of DeepSORT A New Perspective for Environmental Behavior Re- when calculating the pedestrian trajectories. search’, Time + Architecture, 2017(05), pp. 50-53 Second, this study demonstrates that a passive Kapania, S., Saini, D., Goyal, S., Thakur, N., Jain, R. and Na- grath, P.2020 ’Multi object tracking with UAVs using Real Time Locating Systems (RTLS), based on a deep deep SORT and YOLOv3 RetinaNet detection frame- learning computer vision algorithm, can effectively work’, Proceedings of the 1st ACM Workshop on Au- obtain human trajectory data in a square. Using hu- tonomous and Intelligent Mobile Systems, Bangalore, man trajectory data, we can obtain information on pp. 1-6 the pedestrian’s position and speed at a given mo- Li, B., Jiang, C., Wang, L, Cai, W. and Liu, J. 2019, ’A para- ment. Because of the low equipment cost and the metric study of the effect of building layout on wind flow over an urban area’, Building and Environment, convenience of data collection. The method pro- 160, p. 106160 posed in this study has the potential to quantify peo- Liang, J., Hu, Y. and Sun, H. 2013, ’The design evaluation ple’s behavior in public spaces, such as squares in the of the green space layout of urban squares based on future. fractal theory’, Nexus Network Journal, 15(1), pp. 33- Third, this paper proves that the visualization 49 method we used is effective. The visualized image Lin, Y. and Huang, W. 2018 ’Social Behavior Analysis in Innovation Incubator Based on Wi-Fi Data - A Case shows the areas with the highest pedestrian flows Study on Yan Jing Lane Community’, Proceedings of and the fastest speed in the square. We believe that the 23rd CAADRIA Conference, Tsinghua University, this visualization approach will provide researchers a Beijing, China, pp. 197-206 new quantitative perspective on public space. Mohamed, A., Qian, K., Elhoseiny, M. and Claudel, C. 2020 ’Social-stgcnn: A social spatio-temporal graph con- volutional neural network for human trajectory pre- REFERENCES diction’, Proceedings of the IEEE/CVF Conference on Ahmed, I., Ahmad, M., Rodrigues, J. J., Jeon, G. and Din, S. Computer Vision and Pattern Recognition, pp. 14424- 2021, ’A deep learning-based social distance moni- 14432 toring framework for COVID-19. Sustainable Cities Pan, S. J. and Yang, Q. 2009, ’A survey on transfer learn- and Society’, Sustainable Cities and Society, 65, p. ing’, IEEE Transactions on knowledge and data engi- 102571 neering, 22(10), pp. 1345-1359 Bewley, A., Ge, Z., Ott, L., Ramos, F. and Upcroft, B. Wojke, N., Bewley, A. and Paulus, D. 2017 ’Simple online 2016 ’Simple online and realtime tracking’, 2016 IEEE and realtime tracking with a deep association met- international conference on image processing (ICIP), ric’, 2017 IEEE international conference on image pro- Phoenix, Arizona, pp. 3464-3468 cessing (ICIP), Beijing, China, pp. 3645-3649 Farhadi, A. and Redmon, J. 2018, ’Yolov3: An incremental Xu, L., Liu, N. and Sun, C. 2012, ’Scale of Square, Calibre improvement’, Computer Vision and Pattern Recogni- of Space:Virtual Study on the Relation of Space Pref- tion, arXiv preprint arXiv, p. 1804.02767 erence and Image with Area of Square and Depth- Gehl, J. 2011, Life between buildings: using public space, Width Ratio’, Architectural Journal, 2012(02), pp. 74- Island press 78 Grauwin, S., Sobolevsky, S., Moritz, S., Gódor, I. and Ratti, Yang, L., Cheng, B., Deng, N., Zhou, Z. and Huang, W. C. 2015, ’Towards a comparative science of cities: Us- 2019 ’The Influence of Supermarket Spatial Layout ing mobile traffic records in new york, london, and on Shopping Behavior and Product Sales-An ap- hong kong’, in Helbich, M., Arsanjani, J. J. and Leit- plication of the Ultra-wideband Indoor Positioning ner, M. (eds) 2015, Computational approaches for ur- System’, Proceedings of the 24th CAADRIA Conference,

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 369 Victoria University of Wellington, Wellington, New Zealand, pp. 301-310 Yoshimura, Y., Sobolevsky, S., Ratti, C., Girardin, F., Car- rascal, J. P., Blat, J. and Sinatra, R. 2014, ’An analysis of visitors’ behavior in the Louvre Museum: A study using Bluetooth data’, Environment and Planning B: Planning and Design, 41(6), pp. 1113-1131 Yu, H., Yang, B., Liu, J. and Yu, G. J. 2018 ’Passive human trajectory tracking study in indoor environment with CSI’, 2018 International Conference on Network- ing and Network Applications (NaNA), Xi’an, China, pp. 372-377 Zakariya, K., Harun, N. Z. and Mansor, M. 2014, ’Spatial characteristics of urban square and sociability: A re- view of the City Square, Melbourne’, Procedia-Social and Behavioral Sciences, 153, pp. 678-688 Zetik, R., Sachs, J. and Thoma, R. S. 2007, ’UWB short- range radar sensing-The architecture of a baseband, pseudo-noise UWB radar sensor’, IEEE Instrumenta- tion & Measurement Magazine, 10(2), pp. 39-45

370 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Topology Reconstruction of a Discontinuous B-rep Geometry by using Form Finding Method

Chao Yuan1, Xiao Zhang2, Shaoting Zeng3, Liu Yang4, Zhilong Zhao5, Song Qiu6 1,2,3,4,5,6Tsinghua University [email protected] [email protected] [email protected] [email protected] [email protected] [email protected]

In the field of industrial manufacturing and building design procedure, B-Rep (Boundary Representation) model is often used to design and fabricate building components or molds, but in the finite element analysis(FEA) procedure, engineers often need to use the F-Rep (Functional Representation) model files. So, converting two file formats back and forth from one to another is a very important topic in architectural design and manufacturing process. However, there are still some limitations to carry out the conversation process efficiently on discontinuous B-rep geometries with existing software and plug-ins. In this paper, authors introduce an efficient retopology method with kangaroo physics plug-in based on Rhino platform to convert a B-Rep file into a F-Rep file (a continuous uniform mesh infinitely approached to the original geometry with a controllable face numbers) within limited steps. Thus, designers and engineers can do creative parametric design or finite element analysis continuously without surface boundary limitation. Furthermore, the mesh converted by the method introduced in the paper has a better regularity on each single face and better homogeneity of all faces than the built-in ``QuadRemesh'' function in Rhino-7.

Keywords: Form-finding, Retopology, Mesh Mapping, Finite Element Analysis, Shape Quality

INTRODUCTION ogy” and introduces the idea of “low poly mesh with This paper is mainly divided into three chapters: “Ba- the same topology”. At the same time,it is introduced sic Principle”, “Method and Practice”, “ Breakthrough the principle of using “kangaroo physics” plug-in in and Advantage”,which explains the theme “Research the topology reconstruction workflow, explains the on Discontinuous B-rep Geometry Topology Recon- principle of keeping the mesh naked edge on the B- struction in Form-Finding Method ” in detail. rep surface boundary , keeping the mesh vertices fit In the chapter of “Basic Principles”, the author on the initial surface, keeping the length of the mesh first explains the definition of “B-rep geometry topol- edges consistent in the workflow.

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 371 In the chapter of “ Method and Practice ”, the need to be transformed into functional representa- author, through two cases, explains how the topol- tion (F-Rep) models to get finite element analysis ogy reconstruction method mentioned in this paper done (Hetting, Kosinka, 2020). In such a manner, de- combines with the existing manufacturing level to signers can perform topology optimization or tex- create a way for parametric designers to intervene ture study according to the data on the mesh ver- in the field of sneaker industrial design. And, in the tices (Fig. 1). In the computational aided design project of “a folded plate shell design and topology field, the relationship between design and data is optimization”, how to efficiently convert the open getting closer. Whether the data is imported from B-rep model of discontinuous surfaces into a F-rep external sensors or the finite element analysis of the model and carry out the topology optimization work. model itself, it is necessary to convert a B-rep model At the same time, the author shows the scenarios and into a mesh model, so that it can “wear” the data. the value of the method this paper introduced in the Although there are many existing topologies tools existing architectural reconstruction. in the market, most of them are concentrated in In the chapter of “ Breakthrough and Advan- the field of computer animation or digital sculpture, tage ”, it is explained that the methods of mesh eval- such as MAYA, Blender or Z-Brush, and most of them uation, and compares the advantages through the have been packaged as tools, which is not a flexi- method proposed in this paper in“ Mesh quality” and ble method for architecture designers to use. These “ Face Quality” over the “Quadremesh” function in tools are generally hard to deal with discontinuous B- rhino7. In fact, discontinuous B-rep models should rep models. In practical design projects, the continu- be avoided or optimized in engineering projects. If ity of B-rep models is often destroyed in reciprocat- all models in an engineering project can strictly fol- ing modification, so it is often required to optimize low UV continuity, then B-rep models can be eas- and preprocess the B-rep surface continuity before ily transformed into meshes. But in practical work, the model converted to F-Rep models, which is time- many project modeling work after repeated modi- consuming. fication, often cannot achieve geometric continuity, and many F-rep based work needs to be carried out Figure 1 in real time, so the method proposed in this paper Optimization of is very suitable. The working method introduced in infill pattern of the this paper is not only the conversion method of file folded plate format, but also reduces the obstacles for the con- structure bridge version of B-rep model based production, finite el- according to finite ement analysis and performance-based design, and element analysis provides more room for designs and researches in the and the on-site limited time of practical engineering. construction process in Tongji University with 1. BACKGROUND Block Research In the process of industrial production of building Group(Swiss components, models of boundary representation฀B- On the Rhino platform, the mainstream retopol- Federal Institute of rep฀method are widely used(Farin,1984)., such as ogy tools include the “Mesh” function in the soft- Technology in NURBS based continuous surfaces (Piegl, Tiller, 1996). ware itself, “ STARLING ” published in 2012 (by Ma- Zurich), 2019 B-rep method can well preserve the continuity in- teus Zwierzycki, 28299 downloads), “InstantMeshes” formation of the surfaces. However, in the field published in 2019 (by Dale Fugier), “GOPHER” pub- of performance-based design, B-rep models often lished in 2017 (by mnewberg, 4312 downloads)and

372 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 2 The limitations of mainstream retopology tools in Rhino

the latest function “Quadremesh” in Rhino 7. In this nore the discontinuity bug of the initial B-rep model, paper, we take a discontinuous open B-rep model in and form a uniform and regular mesh, which pro- an actual design project as an example, and use these vides an easy access for finite element analysis and five methods to do topology reconstruction work. It performance-based design work. can be seen that they all have some problems(Fig. 2): “Mesh” function in Rhino itself , “InstantMeshes” 2. BASIC PRINCIPLE plug-in, and “Quadremesh” function in Rhino 7 failed 2.1 Topological Structure Classification of to join meshes together, “STARLING” plug-in failed to B-rep Surfaces produce normal mesh; mesh produced by “GOPHER” The basic principles of this workflow are as follows: plug-in failed to fit the target B-rep well. Firstly, operators need to have a very basic under- In the field of computational graphics field, standing of topological geometry: B-rep surfaces can there are many basic methods for mesh subdivi- be divided into “Surface” and “Multi-Surface B-rep”. sion, such as ”Delaunay method” (Fortune,1995), “Surface” includes open surface and closed surface. ”advancing front method”(Li,Ji ,2018) and ”Teich- “Multi-surface B-rep” include “Open Multi-surface B- müller extremal map”method (Choi, Gu, Lui, 2017). rep” and “Closed Multi-surface B-rep”. In this paper, Some researchers have done some research on the openness of initial geometry can be classified the form of architecture, ”Guide Curve Offsetting easily (Fig. 3). method”(Wang,Gao,Wu,2019), ”free-form structures” Figure 3 method(Gao,Li,Ma,2018), but these methods have Open Multi-surface not solved the problem of geometric discontinuity B-rep (left), Closed of target B-rep model. In this paper, a dynamic form Multi-surface B-rep finding method is proposed to re-mesh and texture (right) the discontinuous B-rep model. This method can ig-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 373 2.2 Low Poly Fundamental Mesh Topologi- 2.3 Form Finding cal Design The form finding method used in this paper is based Taking Stanford bunny model as an example, it can on “kangaroo physics” in grasshopper. Developed be defined as a “closed multi-surface B-rep”. Its basic by Daniel piker, Kangaroo2 can carry out interac- topology model is a sphere, no matter how compli- tive physics simulation, optimization and form find- cated the details are, it can be abstracted as the con- ing. This plug-in can be given different physical con- cave convex relationship of the mesh vertices on the straints to the vertices and lines in mesh files. In this sphere mesh (Fig. 4). project, the author defines the vertices on edges of the mesh as “ anchored on the target edge”, which ensures that the reconstructed mesh fits the edge Figure 4 of the target B-rep model strictly; and the lines in- Stanford Bunny side the mesh are defined as “mutual constrained model in B-rep file spring force”, so as to prevent the position of each (left), Stanford point from moving too much during the fitting pro- Bunny model in cess. The attraction force can ensure that the recon- F-rep file after topology However, taking the sole model of a pair of sports structed mesh strictly adsorbed on the target B-rep reconstruction shoes as an example, it can be defined as an “open surface, and the elastic collision force can prevent the (mid), Mesh ball in multi-surface B-rep”, and its basic relationship can mesh from self-folding and affect the reconstructed same topology be summarized by a plane mesh. Regardless of its mesh smoothness. relationship with concave and convex bottom, it can be abstracted as Stanford Bunny the coordinate changes of each vertex on the plane 3.METHOD AND PRACTICE mesh model (right) mesh. For the target model, the genus is the most The topology reconstruction method provided in this important topological element, and the basic mesh paper (Fig. 6) can be roughly divided into three steps: model must describe the genus information (Fig. 5). 1. Build basic (less mesh faces, clear topological rela- Therefore, the operators of topology reconstruction tionship) mesh; 2. Subdivide the basic meshes and work need to have the recognition of the topological apply form finding algorithm to fit it to the target B- structure of the initial B-rep and summarize the tar- rep surface; 3. Set the number of faces and edges get model with a basic mesh. As for mesh subdivision of the new mesh according to the follow-up require- and other operations, they can use the mesh subdivi- ments. In this process, the operation of open and sion algorithm instructions built in Rhino, or they can closed B-rep surface is slightly different, which will be use plug-ins (such as weaverbird). Basic mesh design explained in detail in the paper. is a type generalization work, which provides basic direction for follow-up optimization work. 3.1 Topology Reconstruction and Texture Design on a Closed B-rep Surface The closed B-rep surface has no naked boundary, so Figure 5 based on mesh mapping, the topological structure of Target B-rep model polyhedron or its derivative is fitted to the target B- (left), mesh rep surface by imposing physical constraints. Firstly, containing same the corresponding basic mesh shape is selected ac- opening topology cording to the characteristics of the target B-rep, and information with then wrap the target B-rep surface. Then the mesh is B-rep model (right) subdivided according to the follow-up requirements, and then the mesh is fitted to the target B-rep sur-

374 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 6 Topology reconstruction process

face to complete the mesh reconstruction. The phys- able, mesh topology reconstruction method is used ical constraints imposed are: 1. The spring force con- according to the open B-rep surface specific charac- straint of each line segment. 2. constraint of all ver- teristics. tices of the mesh (each vertex of the mesh should be 3.2.1 Plan-like Open B-rep Surface. When the tar- constrained on the target body) 3. Volume collision get open B-rep surface only has large deformation force of all vertices of the mesh (Fig. 7 ). in two dimensions in the spatial coordinates, the de- formation of the third dimension can be ignored. Figure 7 The topological structure of the mesh can be recon- Closed target B-rep structed according to the planar projection profile of surface ( left), basic the open B-rep, and then the reconstructed mesh can mesh after be fitted to the target B-rep surface itself by apply- subdivision ing physical constraints. As shown in the figure (Fig. warping the target 3.2 Topology Reconstruction and Texture 8), the process of topology reconstruction and tex- B-rep surface ( mid), Design on an Open B-rep Surface ture design can be divided into 3 steps: 1. Extract the mesh after Open B-rep surfaces are very common in industrial edge of the target open B-rep surface and compress topology production, so it is of great value to have an efficient it to the coordinate axis with smaller change dimen- reconstruction ( method to topology reconstruct them. Since the sion; 2. Generate uniform and continuous basic mesh right) forms represented by open B-rep surfaces are vari- through plane closed profile; 3. Wrap the plane ba-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 375 sic mesh on the target surface through form finding • Set a mount of equidistant points according method. to the characteristic line (for applying anchor forces). Figure 8 • Generate amount of random points on the pla- Plane-like open nar mesh and distribute the random points B-rep surface(left), evenly on the plane mesh through dynamic sim- plane mesh(mid), ulation and generate uniform plane triangular topology mesh through Delaunay algorithm. reconstructed mesh After the topology reconstruction work of the ini- • Remap the planar mesh back to the 3D form and model (right) tial open B-rep surface, the author interferes the new do finite element analysis. mesh through the trigonometric algorithm, forming It is obvious that the results of (a) are obviously af- a water ripple like effect (Fig. 9). This effect has far ex- fected by the uniformity of the mesh topology, while ceeded the modeling ability of traditional mold fac- the results of (b) show that if the mesh is uniform, sim- tory, and the modeling process can interact with the ilar errors will not appear (Fig11). mold factory well. The topology reconstructed mesh After the topology reconstruction work and fi- can be used as the basic model for later parametric nite element analysis, the researchers calculate the design work. stress concentration area of the shell model, and use the volume modeling method (plug-in: dendro) to Figure 9 generate a mesh, and then use the mesh to seg- Water ripple pattern ment the shell model, and get the topology opti- in Rhino(left), Water mized shape. Then the optimized model is expanded ripple pattern on and divided into several planar pieces. Finally, the re- out sole by 3D searchers use laser cut to fabricate it, and complete print(right) the assembly of a sphere-like shell structure(Fig12).

4. TECHNOLOGICAL BREAKTHROUGH 3.2.2 Sphere-like Open B-rep Surface. In the fi- AND ADVANTAGES nite element analysis for a B-rep shell model, it is 4.1 Technology Breakthrough: Universality often necessary to transform it into an F-rep model Improvement at first. Therefore, the shape quality of the mesh In the field of industrial production, “boundary rep- directly affects the accuracy of the analysis. How- resentation (B-rep)” is a method for computer to save ever, the existing retopology plug-ins are often un- the information needed for model description ac- able to directly convert discontinuous B-rep models cording to the Eulerian topological boundary (Field into uniform and continuous meshes: the converted D A, 2000), which is beneficial for mechanical equip- results either lack in detail, or take serious deforma- ment to complete the mold work for product sur- tion (Fig10). faces by means of contour crafting. The disadvan- For sphere-like open B-rep surfaces฀using the tage of this geometric description method is very ob- method proposed in this paper can produce a fine F- vious: since the need to have a clear definition of the rep model. The basic method is as follows: edge of the surface, the “discontinuity” of the bound- • Squash the initial F-rep model (transformed by ary often appears when the surface is connected and “B-rep mesh” function in Rhino) into a planar chamfered. This kind of discontinuous characteristic mesh. often needs designer or digital modeler to spend a

376 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 10 The reconstructed models made by other re-topology plug-ins in Rhino

Figure 11 (a) :The “Mesh Brep” function in Rhino makes reconstructed mesh uneven and affects analysis results;(b) : the mesh reconstructed by lot of time to repair and optimize. peated modification will undoubtedly increase the the method However, in the process of re topology, geomet- burden of designers and slow down the develop- proposed in this ric discontinuity will cause big inconvenience. As ment progress. There are still some re topology tools paper is uniform, shown in figure 13, the exposed edges (shown in in computer animation field (such as blender, Maya) which makes the purple) shows that many surfaces are not connected or digital sculpture software (ZBrush), but they also analysis results (Fig.13, left). If the model is directly re-meshed, the have the same limitations. more reliable. computer will recognize the whole object as many surfaces and mesh them one by one. Taking the built- Figure 12 in re-mesh algorithm in rhino7 as an example, the in- Fabrication and put object can only achieve its own uniformity in a assembly of a shell single mesh, but cannot achieve the global mesh uni- model formity. Although there are many existing topology reconstruction tools in the software market, such as HyperMesh, CreateQuadMesh and so on, they have their limitations. Many edge vertices that can’t be corresponded one by one need to be repaired manu- ally by designers before topology reconstruction. Re-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 377 The method proposed in this paper is based on the method to calculate the shape quality of the quad design of the whole target B-rep surface (the un- mesh, as shown in formula (2) (Park C, Noh J S, Jang I connected parts of the surface can be ignored, and S, et al,2007): there is no need to repair the surface), and then the a1a2 b = (2) form finding can ensure the uniform mesh effect. a3a4 Compared with the traditional software, the effect In formula (2), The closer the value of “b” is to 1, the of this method is essentially different. And the plat- closer the shape is to rectangle. By calculating the form(Rhino) used is the same as that widely used by average value and variance of each mesh face, differ- the mold factory. ent mesh optimization methods can be compared di- rectly. Taking a sphere-like mesh as an example, In this Figure 13 evaluation, the quad mesh of the target B-rep surface Non-continuous is reconstructed to get the mesh M_ 1 (method in this open B-rep(left), the paper), and M_ 2 (QuadRemesh). As shown in the non-continuous figure below (Fig.14), the shape quality of the topol- mesh generated by ogy reconstructed mesh completed by this method QuadRemesh in 4.2 Mesh Face Quality Advantages is better than the mesh generated by the built-in Rhino7(right) For an optimized mesh, since the mesh generated on “QuadRemesh” function in rhino7 (the red area rep- the surface cannot ensure consistency of each ele- resents mesh in poor quality, and the blue area rep- ment mesh, the more “regular” the shape of each face resents the mesh in good quality). is, the better the shape quality will be; taking triangu- lar mesh face as an example, the closer the shape is to Figure 14 equilateral triangle, the higher the shape quality will Shape quality of be. For a mesh as a whole, the more approximate the M_1(method in this area of all its faces is, the better the overall uniformity paper) (left), shape is. This data is reflected in the variance of gradient de- quality of gree. Therefore, for a mesh optimization algorithm, M_2(QuadRemesh) shape quality and gradient degree are the two major (right) technical indicators to evaluate the algorithm. 4.2.2 Mesh Gradient Advantage. Good mesh size gradient is very important in the process of standard- 4.2.1 Shape Quality Advantage. The shape quality ized component manufacturing. If the area ratio of of triangular mesh can be evaluated by the ratio of each two adjacent mesh surfaces is the same, that is area to edge length. The formula is ฀1฀ (Lo S H,1989): to say, all mesh faces change linearly, then the gra- 4√3Area dient degree of the mesh is the most uniform.Since a = 2 2 2 (1) l1 + l2 + l3 each face of a mesh has its adjacent faces, the aver- In the formula (1), the range of α is between 0 and 1. age value of mesh face area to its adjacent mesh faces The larger “a” is, the closer the mesh is to an equilat- can be taken as the gradient value of the whole mesh, eral triangle. and the formula is (3): The evaluation of quad mesh shape quality can 1 ( Ai Ai Ai ) Ri = + + . . . + (3) be described by distortion coefficient “b”. Divide the n Ad1 Ad2 Adn quad mesh into four triangles according to the diago- In formula (3), “A_i” is the area of a face ,“Ad_j” is the nal, calculate the α value of the four triangles respec- area of its adjacent faces and “n” is the number of ad- tively, and find a geometric average value, that is, the jacent faces; for a quad mesh (except the faces at the

378 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 edge), n = 4; the closer the value “R” is to 1, the smaller faces to F-rep uniform meshes becomes an important the mesh change degree is. Different mesh optimiza- work, which has a great impact on the commercial tion methods can be compared quantitatively by cal- design cycle. culating the value “R” of each mesh face and calculat- By using the method of “ topology reconstruc- ing its average value and variance. tion in form-finding method ” proposed in this pa- It can be seen from the data in the figure be- per, the B-rep model made by mold factory can be low (Fig. 15) that the variance of the mesh gradi- quickly transformed into a continuous mesh with ent obtained by the method in this paper is signif- uniform distribution, regular shape and controllable icantly smaller than that obtained by rhino7 origi- number of faces. This method not only saves the ba- nal method, which indicates that in the topology re- sic skills of mold factory, but also provides the model construction work of the target B-rep surface, this interface for more emerging digital designers in the research method can obtain more uniform effect, future, avoiding the repeated labor of independent which is more conducive to the follow-up work. processing of “discontinuous B-rep surface”. In the design process, the basic B-rep model can be quickly Figure 15 transformed into a mesh model, which can be eas- Mesh Quality ily involved in the finite element analysis in the early Comparison: design process, and directly modify and adjust the comparing the mesh through parametric control to speed up the it- Shape Quality and At the same time, the method of this paper is more erative optimization process. In addition, it can even Degree of Gradient flexible in dealing with the number and location of access user data collection as the basis of basic mesh of two singular points, which the mesh reconstruction in- change to realize the rapid customized design work. reconstructed structions of other software find it hard to deal with. The efficiency of model conversion seems to be a pro- meshes, the smaller The method of this paper is on the left, and the dis- cess of quantitative change. In fact, it is the “lubri- the value, the tribution of singular points is more uniform than that cant” of product data chain: in a short design cycle, better quality it has. on the right (Fig. 16). let more digital designers and engineers participate Figure 16 in the project, so that the product has better quality More uniform and unprecedented image. singularity distribution M_1(by ACKNOWLEDGEMENTS the method of this This research is funded by the National Social Science paper) (left), less Fund of China (Grant No.17ZDA019). uniform singularity distribution M_1(by 5. CONCLUSION REFERENCES QuadRemesh) At present, in the field of design in developing coun- Les A, Piegl and Wayne, Tiller 1996, The NURBS book[M], (right) Springer Science & Business Media, German tries, the computer mostly takes on the role of “aided Choi, Chi Po, Gu, Xianfeng and Lui, Lok Ming 2017 ’Sub- design”, and the product geometric model is often division connectivity remeshing via Teichmüller ex- completed by the technical workers in the mold fac- tremal map[J]’, Inverse Problems & Imaging, United tory. At the same time, due to the constraints of States, pp. 825-855 modeling ability of manufacturers, many discontin- Farin, Gerald 1984, A History of Curves and Surfaces uous B-rep surfaces are produced, so that the tex- in CAGD[J], Computer Science and Engineering, United States ture design and data-oriented geometric modeling Field, David A 2020, ’Qualitative measures for initial that many designers want are difficult to achieve. The meshes[J]’, International Journal for Numerical Meth- topology reconstruction of models from B-rep sur-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 379 ods in Engineering, 47(4), pp. 887-906 GAO, Boqing, LI, Tierui, MA, Teng and YE, Jun 2018, ’A practical grid generation procedure for the design of free-form structures[J]’, Computers & Structures, 196, pp. 292-310 Jan Hettinga, Gerben and Kosinka, Jiří 2020, ’March 2020Computer Aided Geometric Design 77:101832 DOI: 10.1016/j.cagd.2020.101832 Authors: Gerben Jan Hettinga University of Groningen Jiří Kosinka University of Groningen’, Computer Aided Geometric Design, 77, pp. 89-106 Li, Yongjun and Ji, Shunying 2018, ’A geometric al- gorithm based on the advancing front approach- for sequential sphere packing’, in Herrmann, Hans J., Jaeger, Heinrich and Luding, Stefan (eds) 2018, Granular Matter, Springer, Heidelberg, pp. Vol. 20, Iss. 4, 1-12 Mizobuti, Vinicius and Junio, Luiz C.M. Vieira 2020, ’Bioinspired architectural design based on structural topology optimization[J]’, Frontiers of Architectural Research, 9(2), pp. 264-276 Owen, Steven J 1998 ’A survey of unstructured mesh generation technology[J]’, 7th International Meshing Roundtable, United States, pp. 239-267 Park, Changhyup, Noh, Jae-Seung, Jang, Il-Sik and Kang, Joe M 2007, ’A new automated scheme of quadri- lateral mesh generation for randomly distributed line constraints[J]’, Computer-AidedDesign, 39(4), pp. 258-267 Piacentino, Giulio 2013, ’Weaverbird: Topological mesh editing for architects[J]’, Architectural Design, 83(2), pp. 140-141 WANG, Qisheng, GAO, Boqing and WU, Hui 2017, ’An Architectural Grid Generation over Free-Form Sur- faces Based on Guide Curve Offsetting[J]’, JOURNAL OF SHANGHAI JIAO TONG UNIVERSITY, 53, pp. 1040- 1044

380 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Towards a Free-form Transformable Structure A critical review for the attempts of developing reconfigurable structures that can deliver variable free-form geometries

Hussein E. M. Hussein1, Asterios Agkathidis2, Robert Kronenburg3 1Architectural Engineering Department, Zagazig University, Egypt 2,3Liverpool School of Architecture, University of Liverpool, UK [email protected] [email protected] 3r.h.kronenburg@liv. ac.uk

In continuation of our previous research (Hussein, et al., 2017), this paper examines the kinetic transformable spatial-bar structures that can alter their forms from any free-form geometry to another, which can be named as Free-form transformable structures (FFTS). Since 1994, some precedents have been proposed FFTS for many applications such as controlling solar gain, providing interactive kinetic forms, and control the users' movement within architectural/urban spaces. This research includes a comparative analysis and a critical review of eight FFTS precedents, which revealed some design and technical considerations, issues, and design and evaluation challenges due to the FFTS ability to deliver infinite unpredictable form variations. Additionally, this research presents our novel algorithmic framework to design and evaluate the infinite form variations of FFTS and an actuated prototype that achieved the required movement. The findings of this study revealed some significant design and technical challenges and limitations that require further research work.

Keywords: Kinetic transformable structures, finite element analysis, form-finding, deployable structures, Grasshopper 3D, Karamba 3D

INTRODUCTION sub categorised to mobile, transformable and incre- Transformable systems in architecture are defined as mental. the systems that can “alter their forms to have differ- Transformable solutions can be categorised in ent spatial configurations to be employed for space- different ways as described in the classifications of, saving and utilitarian needs” (Fox & Kemp, 2009). for instance, C.J. Gantes (2001), Felix Escrig (2010), These systems are considered a sort of dynamic ki- Maziar Asefi (2010), and Esther Adrover (2015). These netic architecture, based on Fox’s (2009) classification classifications revealed some common mechanisms of kinetic architecture, which has three categories: utilised in deployable (portable) and transformable embedded, deployable, and dynamic, which is also (i.e. not portable) solutions such as ’spatial bar struc-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 381 tures (Asefi, 2010), which are so-called ’scissor-like el- ometry to another (figure 5), which is the scope of this ements (SLE)’ (Escrig & Sánchez, 2010) or ’latticework’ research. (Hanaor, 2009; Adrover, 2015). Figure 1 Spatial-bar structures mostly share the same fea- Typologies of tures of space trusses; they are composed of linear spatial-bar elements (i.e. bars or struts) assembled in three- structures: dimensional configurations and flexible joints at the pantographic ends or intermediate points of these elements (Asefi, (linear and 2010; Hoberman, 2006), covered by flexible materials angulated) and (e.g. PTFE) or lightweight panels (e.g. Polycarbonate) reciprocal (bars (Gantes, 2001). (Larsen, 2008) and Spatial-bar structures have two typologies, ’pan- plates (Rodriguez, tographic’ that employs scissor-pair mechanisms et al., 2009)). with straight or angulated bars and ’reciprocal’ struc- tures with bars or plates in closed-loop formations Figure 2 (figure 1) (Asefi, 2010; Hanaor, 2009). The transforma- Transformation tion morphologies of spatial-bar structures, accord- morphologies of ing to Escrig (2010), have six typologies (figure 2): spatial-bar ’umbrellas’, ’bundles’, ’rings’, ’polyhedral’, ’planes’ and structures: umbrella ’double-arched. (Escrig & Sánchez, According to the mentioned classifications and 2010); bundles the morphologies mentioned by Escrig (2010), it can (Ibid); planes This unique morphology of transformable structures be noticed that the possibilities of the form varia- (Schumacher, et al., is named in this research as “free-from transformable tions achievable by spatial-bar mechanisms are lim- 2010); polyhedral structures” (FFTS). They share similarities with space ited and based on the modification of primitive 3D (Hoberman, 1991); trusses which are supported by their corners or edges shapes (e.g. box, cylinder) or platonic solids. Addi- rings (Ibid) and and are not fully supported/attached/suspended tionally, spatial-bar mechanisms are not common in double-arched by/to another structure, unlike the HypoSurface or architectural applications; this can be for two major (Escrig & Sánchez, Margolin’s sculptures (figure 6). factors, their cost and complexity (Asefi, 2010). 2010). Despite their issues, spatial bar structures of- fered sophisticated architectural solutions, such as Figure 3 the works of C. Hoberman (figure 2-d&f) and Santi- An expandable ago Calatrava (figure 2-c). Moreover, recently, some free-form surface designers attempted to extend the possibilities and with angulated morphologies of spatial-bar structures. For instance, scissor-bars C. Hoberman (2015) developed a ‘kinetic block’ that (Hoberman, 2015). can achieve foldable free-form geometries (figure 3). Additionally, other prototypes were developed to Figure 4 create interactive free-form surfaces, such as the Hy- Contour kinetic poSurface (Dunn, 2012), and the kinetic sculptures of sculpture made by Reuben Margolin (figure 4) [5]. Finally, some prece- Reuben Margolin dents attempted to create transformable free-form [5] structures that alter their forms from a free-form ge-

382 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 5 Forms that can be achieved by FFTS as described in this research.

Figure 6 a). The motion is achieved by changing the lengths The possible of struts using linear actuators and flexible joints transformation of called CMS (Concentric Multilink Spherical), which free-form surfaces: were based on scissor mechanisms to achieve con- a) expandable; b) centric movement of the struts. supported by The main concern with FFTS systems is their capabil- In 2015, Robert Read commenced a project another structure; ity to deliver infinite form variations which may make called ‘The Gluss’ (figure 7-b), which aimed to make c) supported by its their design and evaluations process more compli- the Tetrobots mechanisms cheaper and smaller, and ends, which is the cated and challenging compared to common trans- easier to control using Arduino [3]. He employed Ac- scope of this formable structures, which move within a predefined tuonix L16 actuators for the adjustable struts, and research. series of states (e.g. from compacted to expanded). 3D printed adjustable turret joints, which were previ- Therefore, this research aims to highlight the ously invented by Song, Kown and Kim (Song, et al., essence and possible functions of FFTS and define 2003). Read made an open-source parametric digi- the techniques employed to achieve this kind of tal model of the turret joints, which can also be easily movement and reveal the key design and evalua- fabricated by 3D printing. tion challenges and considerations. Thus, Why is Figure 7 the free-form transformation of spatial-bar structures The Tetrobots : A) needed? How can this transformation morphology The tetrobot be achieved? How these precedents faced the design (Hamlin & and technical challenges of FFTS? Sanderson, 1998, p. In order to answer these questions, the research 142); B) the Gluss investigated eight precedents with six different ap- [3]; C) Morphs [2]. proaches sorted chronologically. Each approach is presented in two sections: the first is a brief descrip- tion of the precedent (e.g. structure system, mech- anism, materials), and the second is a critical review and evaluation.

PRECEDENTS OF FREE-FORM TRANSFOR- Afterwards, in 2015, a team from BMADE Robotics MATION Lab at UCL proposed an interactive tetrahedral Tetrobots model called ‘Morphs’ (figure 7-c) [2]. It was designed to move within public spaces and respond to its en- In 1994, G. Hamlin and A. Sanderson (1998) proposed vironment. The structure has 12 linear actuators and a robot that can walk within rough terrains based on spherical cast polyurethane joints, which enables the the tetrahedral modules of space-trusses, and they structure to shift its CG, making it able to crawl. called it Tetrobot (i.e. tetrahedral robot) (figure 7-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 383 In terms of design, tetrobots were mainly pro- signers also suggested that the structure’s operation posed to make robots that can move over terrains process is self-learning as it could adapt to the chang- that wheeled machinery cannot access. They have ing needs of its users (Luciana, 2013). similarities with space trusses and can be employed In terms of design, topo-transegrity offered a to obtain a structure with free-form transformation. simple approach to achieve free-form transformable However, Tertobots technically have two major is- structures. Within the transformation process, the sues: first, they rely on a large number of linear ac- modules remain rectangular from the plan-view, tuators, which may increase the complexity of the and the structure elevates upwards without any un- structure’s compositions and its operation, which can wanted deformation. However, the structure only negatively affect its maintenance cost and life ex- offers forms that look like steps and cannot offer pectancy. Second, The proposed joints have many smooth free-form surfaces. Additionally, there are small pieces which may require an intensive mainte- many concerns in employing pneumatic actuators, as nance plan (e.g. lubrication), especially if these joints they are not reliable nor convenient in structural ap- were exposed to dusty, rainy or snowy environments. plications due to the compressibility of air when sub- jected to loads (Hamlin & Sanderson, 1998) and their Topo-Transegrity complicated technical requirements. In 2002, the ‘5subzero’ design group, founded in Lon- don, presented a prototype in the Latent Utopias Actuated Tensegrity Exhibition, Graz, 2002 of a structure called topo- In 2003, T. Sterk proposed a reconfigurable struc- transegrity that can reconfigure itself based on the ture system based on tensegrity structures called ac- changing conditions (figure 8) [1]. The structure was tuated tensegrity (figure 9). He presented that sys- proposed to renovate the courtyard of the Barbican tem in some conceptual projects such as the ‘Frais’ Arts Centre in London as a responsive surface for this [6] and the ‘Prairie House’ [7]. Additionally, he pre- public space that can offer a real-time transformation sented a prototype of this system in 2004 made of to host changing activities, events, and behaviours cast aluminium and wires of shape memory alloys ac- (Neumayr, 2006). tuated by pneumatic actuators (Schumacher, et al., 2010). The designer claimed that the structure could transform to control its aerodynamics, respond to the Figure 8 changing loading conditions (e.g. wind), control so- Topo-Transegrity lar gain, and reduce the CO2 emission (Sterk, 2015). prototype Additionally, he also claimed that the structure could (Neumayr, 2006). shake itself to drop any accumulated snow on its sur- The presented prototype was a 1:10 scaled model face. for a manipulatable space-frame structure with ad- Figure 9 justable struts using ‘Festo’ pneumatic actuators. The the kinetic blocks of structure’s modules can be considered deformable Actuated Tensegrity boxes, and accordingly, its joints are simple; each is (Sterk, 2003). a cross with four one-degree of freedom (DOF) revo- lute joints. The designers claimed that the structure could transform into stairs, walls with adaptive lou- vres, roof openings or changeable routes and deliver Sterk’s approach was ambitious, and he presented a network of included planes that allow access from multiple functions and advantages of free-form every point of the public space to another. The de- transformable structures. Additionally, his selection

384 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 for tensegrity structures can reduce the total struc- composition were simple and subtle; they did not tural weight and reduce the power needed for op- employ complex joints nor struts. The structure’s sim- eration accordingly. However, there are many con- plicity can also enable mass production of the com- cerns about the reliability and durability of tensegrity ponents and reduce its cost accordingly. Unfortu- structures in architectural applications (Motro, 2003; nately, the number of actuators necessary to achieve Asefi, 2010). Moreover, there are many issues with us- the required transformation is huge and can increase ing pneumatic actuators, as explained before. Finally, the complexity of the operation process and mainte- his proposed kinetic modules can increase the move- nance. Additionally, the structure’s span can change ment limitations as each module changes its size in during the transformation process, which means that all directions within the actuation process, which can this kind of structures is not feasible in structures with be limited by the movement of its adjacent modules. fixed spans or supports.

HybGrid Double Scissor-Pair structures In 2009, D. Rosenburg, in his PhD at MIT, developed Figure 10 a transformable structure mechanism based on pan- Transformation of tographic structures for a partition that responds to the HybGrid the unexpected user’s needs (Rosenberg, 2010) and control the flow between two spaces and can be con- trolled either manually or using AI system. His pro- posed structure can achieve multiple curvatures by shifting the mid joint of the scissor mechanism of In 2003, Jordi Truco Calbet and Sylvia Felipe Marzal, the pantographic structure (figure 11). Each mod- the founders of Hybrida studio in Barcelona, pro- ule of the mechanism has eight scissor compositions, posed a structural system called HybGrid in their two for each side; that is why this approach called a MArch study at the Architecture Association (Hensel, double-scissor pair structure. et al., 2010), and they registered this structure system Figure 11 as a patent in 2007 (Marzal & Calbet, 2007). The Hy- Double SLE bGrid system can deliver free-form surfaces by tesse- (Rosenberg, 2010) lating the surface into a rhombus-shaped grid. Each line of the grid consists of three strips of flexible fibre composite, and between these strips, the actuators were placed. By adjusting the actuator, the form of the gridline can be changed (figure 10), and the con- figuration of the structure changes accordingly. The system was firstly proposed for fixed struc- In terms of design, the double-scissor pair mecha- tural configurations, and it was employed in projects nism was entirely made of simple 1-DOF joints; How- such as the ‘Hybermembrane’ pavilions at the Design ever, the structure’s composition itself is complex. HUB museum in 2013 and the Belloch Parc, Santa I Additionally, according to the available documenta- Cole, in 2016 [8]. Additionally, the team presented tion, the designer focused on the structure’s respon- some 3D renderings and proposals for transformable siveness to the users’ needs and its operation rather HybGrid structures, and they suggested some appli- than the technical issues of his solution, as there cation like controlling solar gain. are some concerns and doubts about the ability of In terms of design, the HybGrid geometry and the utilised servo motors to shift the pivots as their

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 385 torque may not be sufficient to make the required FREE-FORM TRANSFORMABLE STRUC- movement. TURE (FFTS) According to the previous investigation of the prece- Modified Scissor-Like elements M-SLE dents, and their issues, a novel approach to de- Y. Akgün (2010) proposed a structural mechanism sign and evaluate free-form transformable structures that improves the flexibility of pantographic struc- was developed. This approach is formulated into tures and reduces the actuators required to achieve an algorithmic framework and a Grasshopper script this transformation (figure 12). His approach to (figure 13) that organises the development process achieving this is by dividing the structure into a num- of these structures and perform the required sim- ber of deployable arc-shaped segments using Mod- ulations seamlessly and effectively. The proposed ified scissor-like elements (M-SLE). Each M-SLE has framework was developed in multiple iterations; the four bars instead of two, with one intermediate pivot. first one was presented in eCAADe 2017, based on an He made studies, calculations and finite element sim- investigation on the available design and evaluation ulations on his proposal of M-SLE on a vault structure frameworks (Hussein, et al., 2017). The final version of with three arc-shaped segments and dealt with it as this framework and script, presented here, was vali- a four-bar linkage (Akgün, et al., 2010). He suggested dated by testing it on the design and evaluation of some applications for his structure, such as making an arbitrary 10x10 meters FFTS. The script was also adaptive interactive roofs and control solar gain. employed in operating a physical model of a single active linear element ALE of FFTS. Figure 12 The FFTS design and evaluation processes have A 3D view for a two approaches, top-down and bottom-up. The top- pavilion with M-SLE down approach aims to define the design concept and perform the evaluation processes to extract the components’ design data (e.g. maximum required length of a strut); Then, the bottom-up approach In terms of design, the M-SLE approach has two ad- aims to build and operate the physical model of the vantages: they offer structures with simple joints and structure based on the extracted data. the lowest number of actuators, which accordingly could decrease the overall cost and the complexity of The top-down approach the construction, operation, and maintenance of the The research focused on making an FFTS for a pavil- structure. Unfortunately, the supports of the struc- ion with a rectangular layout (figure 5), which is ture cannot move upwards, which can limit the struc- based on the mechanisms of Tetrobots as they are ture’s flexibility. Additionally, his proposed solution similar to space trusses and can be employed for var- only fits the case he proposed; the calculation he ious architecture applications. The structure’s sup- presented cannot be generalised for structures with ports are by its edges, and it can be subdivided into a more than 2 M-SLEs. Finally, according to the doc- number of active linear elements (ALEs) that can con- umented structural simulation results of this prece- trol the form and the movement of the structure (fig- dent, the structural simulations were performed on ure 14). three presumed form variations for four actuator con- The form of the ALEs can be controlled by ad- figurations (12 forms in total), which are not sufficient justing the lengths of the columns and the top to reveal the worst-case scenarios and maybe not ac- and bottom-layer struts to maintain the span of curate. the structure. There are two approaches for actu- ation: top/bottom-layer actuation (TBLA) and opti-

386 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 13 The developed GH script

mised TBLA with a reduced number of actuators, as designers can determine the design data of the com- shown in figure 14. ponents, such as the maximum or minimum length required for the linear actuators of the struts and the Figure 14 maximum tension or compression stress applied to The Active linear these actuators (figure 15). Then the designers can elements (ALE) of move to the bottom-up approach and build the ac- FFTS. tuated physical model.

Figure 15 A sample of the worst-case scenarios extracted from the GH script and the GA solver. Afterwards, a parametric model was created using Grasshopper 3D; the script was necessary to gener- ate the form variations of FFTS and seamlessly inte- grate the inverse & forward kinematics analyses (us- ing Kangaroo 2 plugin) and the finite element simu- lations (using Karamaba 3D) within a user-friendly in- The bottom-up approach After extracting the components’ design data, de- terface to easily generate, manipulate and evaluate signers can design and fabricate the FFTS compo- the FFTS. nents, i.e. the adjustable struts and the flexible joints. By employing the developed GH script and ge- Due to some limitations in this research project, an netic algorithms (GA) solvers (e.g. Galapagos), the actuated model for one active linear element with

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 387 five pyramid modules of the FFTS was created based from each configuration to the other (Figure 17); a on the design data extracted from an arbitrary 10x10 video recording for the prototype movement is up- meters pavilion used to validate the functionality of loaded to the link [4]. the GH script. The joint design was based on the turret joint CONCLUSIONS investigated in the tetrobots section, and so it was This research investigated six approaches that at- named as the MT (modified turret) joint (figure 16). tempted to create free-form transformations of This joint achieved the concentric rotation of the spatial-bar structures. The study highlighted the struts around the joint’s centre, and it controls the essence of these structures as they can be employed movement ranges and behaviours of the struts. The as (1) Roof structures with customisable form vari- digital model of the MT joint was parametric, and it ations (e.g. M-SLE), interact with users (e.g. Actu- was fabricated using 3D printing. ated Tensegrity), control solar gain (e.g. M-SLE) and achieve structural stability in different loading condi- Figure 16 tions (e.g. HybGrid) (2) Partitions that respond to the The MT joint after unexpected users‘ needs (e.g. Double Scissor-pair) fabrication (3) Flooring that controls the users’ flow in architec- tural or urban spaces (e.g. Topo-Transegrity) (4) Inter- active elements that interact with users within urban spaces (e.g. Morphs). Moreover, the study revealed some design is- sues, challenges, and considerations; a common is- sue is the FFTS ability to reconfigure into unex- pectable form variations, which can make the eval- For the adjustable struts, Actuonix actuators L16-R uation processes more challenging. Additionally, the with a gear ratio of 1:150 were utilised in the actu- precedents revealed the following:- ated prototype of FFTS, the ones with stroke length 100 mm for the struts and 140mm for the columns. • The tetrobots highlighted the potential of utilis- These actuators were controlled using the developed ing space frame structures to create FFTS; how- GH script and Arduino boards using the Firefly plugin. ever, their mentioned applications were not rel- Afterwards, some modifications were made to evant to architecture. the parametric model and the GH script to match, for • The topo-transegrity offered a sophisticated so- instance, the prototype scale, actuators’ movement lution and simple joints, but it was dependent ranges and adding the calculations necessary to cal- on unreliable pneumatic actuators and could ibrate the actuators. not achieve smooth free-form surfaces. Then, the FFTS model was assembled and op- • The actuated tensegrity presented a lightweight erated, and the operation process revealed some is- solution with ambitious applications; however, sues. The most critical issue was that the structure’s it was also reliant on pneumatic actuators, and span was changing, which was solved by a python there were some concerns about the limitation script to control the actuators’ speeds to have a more of its module design and the reliability and dura- stable and consistent movement and maintain the bility of its tensegrity system. structure’s span. • The HybGrid revealed a novel approach and sys- Finally, the structure was operated effectively tem to achieve FFTS; however, it will be compli- and achieved form variations and moved smoothly cated to design and operate due to the number of actuators needed for operation, and the struc-

388 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 17 Some of the form variations achieve by the FFTS prototype.

ture span is not fixed, which may reduce its ap- structure after assembly effectively. plicability. FFTS achieved a reliable and durable reconfig- • The double scissor pair offered a solution with urable structure with some features of space trusses joints; however, the composition was complex, and their modules, which can be widely employed and they are some concerns about the feasibil- In architectural applications. Its movement achieved ity of its proposed actuation. by a lower number of actuators compared to the • The M-SLE offered a sophisticated solution with tetrobots and the gluss project and maintained its simple joints and a reduced number of actua- span, unlike the HybGrid. Moreover, The structure tors; however, the solution has many limitations supports can move vertically and achieved more flex- the affected its flexibility. Furthermore, there ibility compared to topo-transegrity and the M-SLEs. were some concerns about the accuracy of the Furthermore, our approach achieved a seamless inte- documented structural simulations of M-SLE. gration of form generation, kinematic and structural simulations with stochastic investigations to obtain In order to solve these issues, the research presented more accurate results compared to the M-SLE. our approach to achieving FFTS, using a novel al- Although the proposed framework and script gorithmic design and evaluation framework and a dealt with some of the issues and challenges of GH script that seamlessly integrate the form gener- the design and evaluation of FFTS, Further research ation, kinematic analyses and structural simulations work is necessary to improve its reliability and con- in a user-friendly interface. Moreover, genetic algo- venience. Additionally, some improvements are nec- rithm solvers were employed to determine the criti- essary for the design of the proposed FFTS to reduce cal form variations (e.g. worst-case scenarios) and ex- its complexity and enhance its flexibility. Finally, Fur- tract the components’ design data (maximum stress ther research can improve the capabilities and func- on a strut). tionality of the proposed system. Based on the extracted design data, the actu- Consequently, this research can be considered a ated prototype components were defined, such as base for further research work in the field of trans- the linear actuators for the struts and the developed formable structures and can be beneficial in architec- 3D printed MT (modified turret) joints. Afterwards, ture from academic and practical perspectives. the developed GH script was modified to operate the

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 389 ACKNOWLEDGEMENTS Hussein, H. E. M., Agkathidis, A. and Kronenburg, R. 2017 The authors would like to acknowledge the Egyp- ’Free-Form Transformation of Spatial Bar Structures: tian Ministry of Higher Education for the scholarship Developing a Design Framework for Kinetic Surfaces Geometries by Utilising Parametric Tools.’, eCAADe given to the first author; In addition, they acknowl- 2017, Rome, pp. 747-756 edge the University of Liverpool for their support. Larsen, P. O. 2008, Reciprocal Frame Structures, Architec- tural Press, Oxford REFERENCES Luciana, P. 2013, Contagious Architecture: Computation, Aesthetics and Space, MIT Press, Cambridge Adrover, E. R. 2015, Deployable Structures, Form+ Tech- Marzal, S. F. and Calbet, J. T. (eds) 2007, Construc- nique, Laurence King, London tion System, Procedure for Shaping it, And structural Akgün, Y. 2010, A Novel Transformation Model for De- section for a Construction System, Patent No. WO ployable Scissor-Hinge Structures, Ph.D. Thesis, Uni- 2007/138141 A1., Madrid, Spain versität Stuttgart Motro, R. 2003, Tensegrity: StructureSystemsofTheFuture, Akgün, Y., Gantes, C. J., Kalochairetis, K. E. and Kiper, G. Kogan Page Limited, London 2010, ’A Novel Concept of Convertible Roofs with Neumayr, R. 2006 ’Topo-Transegrity’, Responsive Architec- High Transformability Consisting of Planar Scissor- tures: Subtle Technologies, Toronto, Ontario Hinge Structures’, Engineering Structures, 32, pp. Rodriguez, C., Chilton, J. and Wilson, R. 2009, ’Flat Grids 2873-2883 Designs Employing the Swivel Diaphragm’,in Motro, Asefi, M. 2010, Transformable and Kinetic Architectural R. (eds) 2009, An Anthology of Structural Morphology, Structures: Design, Evaluation and Application to In- orld Scientific Publishing Co. Pte. Ltd, Singapore telligent Architecture., VDM Verlag Dr. Muller Ak- Rosenburg, D. 2009, Designing for Uncertainty: Novel tiengesellschaft & Co. KG, Saarbrucken Shapes and Behaviours Using Scissor-Pair Trans- Dunn, N. 2012, Digital Fabrication in Architecture, Lau- formable Structures, Ph.D. Thesis, MIT rence King Publishing Ltd, London Rosenburg, D. 2010, ’Indeterminate Architecture: Escrig, F. and Sánchez, J. 2010 ’A General Survey of Scissor-Pair Transformable Structures’, Delft Archi- Deployable Structures with Articulated Bars’, IASS, tecture Theory Journal, pp. 1-23 Shanghai Schumacher, M., Schaeffer, O. and Vogt, M. 2010, MOVE: Fox, M. and Kemp, M. 2009, Interactive Architecture, Architecture in Motion-Dynamic Components and El- Princeton Architectural Press, New York ements, Birkhäuser GmbH, Berlin Gantes, C. J. 2001, Deployable Structures: Analysis and De- Song, S., Kwon, D.-S. and Kim, W. S. 2003, Spherical Joint sign, WIT Press, Southampton forCouplingThreeorMoreLinksTogetheratOnePoint, Hamlin, G. J. and Sanderson, A. C. 1998, Tetrobot: A Mod- US, Patent No. US 6,568,871 B2 ular Approach to Reconfigurable Parallel Robotics., Sterk, T. D. 2003 ’Using Actuated Tensegrity Structures to Springer Science+Business Media, LLC., New York Produce a Responsive Architecture’, ACADIA22 Hensel, M., Menges, A. and Weinstock, M. 2010, Emer- Sterk, T. D. 2015, ’Beneficial Change: The Case for Re- gent Technologies and Design: Towards a Biological sponsiveness and Robotics in Architecture’, in Ko- Paradigm for Architecture, Routledge, Oxon larevic, B. and Parlac, V. (eds) 2015, Building Dynam- Hoberman, C. 1991, Radial Expansion/Retraction Truss ics: Exploring Architecture of Change, Routledge, Tay- Structures., US, Patent No. 5,024,031 lor & Francis Group, Oxon Hoberman, C. 2015, ’Transformable Building Structures [1] http://www.5subzero.at/stuff/topotransegrity.html that Change Themselves’,in Kolarevic, B. and Parlac, [2] http://www.interactivearchitecture.org/lab-projects V. (eds) 2015, Building Dynamics: Exploring Architec- /morphs ture of Change, Routledge, Oxon [3] https://halckemy.s3.amazonaws.com/uploads/attac Hoberman, C. 2006, ’Transformation in Architecture and hments/188839/Gluss.pdf Design’, in R., Kronenburg (eds) 2006, Transportable [4] https://youtu.be/GQvyn8H_0tE. Environments 3, Taylor & Francis, London [5] https://www.reubenmargolin.com/waves/contours/ Hussein, H. E. M. 2020, Transformable Spatial-bar Struc- [6] http://www.orambra.com/~frais.html tures: An Algorithmic Design and Evaluation Frame- [7] http://www.orambra.com/~prairieHouse.html work to Develop Free-Form Transformable Structures [8] http://www.hybridarch.net/hypermembrane-demo (FFTS), Ph.D. Thesis, Liverpool School of Architecture

390 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Structural Performance of Reciprocal Structures formed by using Islamic Geometrical Patterns

Asli Agirbas1, Elif Feyza Basogul2 1Department of Architecture, Fatih Sultan Mehmet VakifUniversity, Istanbul, Turkey 2Institute of Graduate Studies, Fatih Sultan Mehmet Vakif University, Istanbul, Turkey [email protected] [email protected]

Many Islamic geometric patterns consist of stripes which are recognizable in the two dimensional patterns. These stripes systematically pass over or under each other, thus they create a tessellation. This system has the same principle with reciprocal frame structures. Considering this situation, in this study, it is aimed to lift the two dimensional Islamic geometric patterns to the third dimension with the principle of reciprocal frame structures. A selected Islamic geometric pattern has been lifted to the third dimension in the reciprocal structure principle, and structural analyzes have been performed.

Keywords: Reciprocal frame structures, Islamic geometric patterns, Structural analysis

INTRODUCTION (2008) mentioned, it can be suitable for emergency Reciprocal frame structures consist of mutually sup- situations, since it can be built quickly with local ma- ported rigid elements. A module in reciprocal frame terials. In addition, Popovic Larsen (2014) points out structure is a short piece of the whole structure that the benefit of the reciprocal frame structure is (Popovic Larsen, 2008; Anastas et. al. 2016). Recip- that in symmetrical configurations, all joints and all rocal frame structures are also known as Nexorades members are identical. Moreover, Popovic Larsen (Baverel, 2000). (2019) mentions that the joints are reversible and al- Although it is an interesting type of structure, it is low disassembly and reusability, and reciprocal frame not widely used in construction industry today. Anas- structure is lightweight and easy to carry, and in tas et. al. (2016) explain the reason for this as the dif- gridshell-like reciprocal frame structures, the failure ficulty in adapting to the holistic form of the structure of a piece or connection does not destroy the whole as it goes from module to global scale. system. On the other hand, reciprocal frame structures In this study, it has been determined that Islamic have many advantages. Large openings can be cov- geometrical patterns with polygonal system and re- ered via bringing together a large number of short ciprocal frame structures are constructed with the pieces. Therefore, reciprocal frame structures are also same principle. In the both system, the pieces of the known as one of the structure types used to cre- patterns systematically overlap each other. ate large spans in buildings. In addition, as Di Carlo In Islamic geometric patterns, which are carved

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 391 on stone and wood, there is a semi-three dimension- constructed. For example, Archaeological Shelter ality, which is formed by passing the strips over each in at Bibracte, France, built with the principle of other. This situation can be observed at the works Reciprocal Frame structures in 2008, was designed of many researchers who had conducted studies on by architect Paul Andreu and engineers Bernard Islamic geometrical patterns (Bonner, 2017; Agirbas, Vaudeville and Simon Aubry (Gelez et. al 2011). 2020). However, to date, there is no study on the po- Frans Masereel Centrum in Kasterlee, Belgium was tential relationship between reciprocal frame struc- designed by architects LIST, Hideyuki Nakayama Ar- tures and Islamic geometric patterns with polygonal chitects and engineers Bollinger + Grohmann and system. Therefore, in this study, it is aimed to lift the Bureau Bouwtechniek, using reciprocal frame struc- two dimensional Islamic geometric patterns to the tures on the roof (Bergis and De Rycke, 2017). Mount third dimension with the principle of reciprocal frame Rokko-Shidare Observatory (Japan, 2010) was de- structures and to compare them as means of struc- signed with the reciprocal frame structure principle tural performance. Thus, based on the traditional by architect Hiroshi Sambuichi and Ove Arup and elements, contemporary reciprocal structure models Partners (Popovic Larsen, 2014). Pizzigoni (2009) with high structural performance can be obtained in proposed fibre-reinforced concrete based recipro- a variety of different forms. cal frame structure for Italian World Expo Pavilion in Shanghai 2010. Kreod Pavilion by Pavilion Ar- BACKGROUND chitecture and Rambøll London (London, UK, 2012), Reciprocal frame structures Seiwa Bunraku Puppet Theater by Kazuhiro Ishii (Japan,1990s), Forest Park Pavilion by Shigeru Ban, It is known that reciprocal frame structures were used Cecil Balmond and ARUP (St Louis, Missouri, 2007), in the twelfth century in Chinese and Japanese archi- Coca Cola Beatbox Pavilion by Asif Khan and Pernilla tecture (Di Carlo, 2008). These structures, built us- Ohrstedt (London, UK, 2012) can be given as other ex- ing timber as a material, were specifically built as roof amples of buildings with reciprocal frame structures. supporting systems, and are also known as mandala roof (Kohlhammer and Kotnik, 2011). The works of Villard de Honnecourt (around Figure 1 1240), Sebastiano Serlio, Leonardo da Vinci and John Un-notched Wallis are known as the early examples of the recipro- elements by Da cal frame structures (Houlsby, 2014). There are notes Vinci, notched about the use of multiple beams in the Leonardo’s elements by Wallis, Codex Atlanticus fol. 899v, which includes reciprocal aligned axis by frame structure drawings (Williams, 2008). It is possi- Serlio (Parigi and ble to group these early examples according to their Pugnale, 2012) In addition, various researches are carried out on re- construction type. Parigi and Pugnale (2012) men- ciprocal frame structures. For example, Douthe and tions that Da Vinci used un-notched elements, Wal- Baverel (2009) proposed a form-finding method for lis used notched elements, and Serlio used aligned reciprocal frame structures by using dynamic relax- axes for constructing reciprocal frame structures, and ation algorithm. Parigi and Kirkegaard (2014) de- relates the reasons of different usage of elements to veloped the Grasshopper plug-in, named Recipro- their proximity to the plane (Figure 1). For example, calizer, which can create various types of recipro- in the version with the usage of the un-notched ele- cal frame structures. Thonnissen (2014) described a ments, a dome-like situation will inevitably occur. form-finding tool for reciprocal structures in his pa- It is possible to give contemporary examples of per. Song et. al. (2014) proposed an interactive tool various reciprocal frame structures that have been for designing reciprocal frame structures. Anastas et.

392 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 2 al (2016) developed a parametric model that allows A tessellation in the reciprocal frame pattern to be adapted to various Ahlat cemetery surface forms. Gerber and Pantazis (2016) focussed (Agirbas, 2020) on form-finding process of reciprocal frames which are informed by structural and environmental analy- ses. Moreover, Xian et. al. (2020) conducted a study on robotic fabrication of a double-curved reciprocal frame wall.

Potential of Islamic geometric patterns Two-dimensional Islamic patterns are mathemati- cally defined patterns. It can be easily seen that some of these two-dimensional patterns have overlaps that give a sense of third dimensionality. For example, as Agirbas (2020) mentioned, three-dimensionality of the Islamic patterns on Ahlat tombstones (dates back to 12th-15th centuries) can be seen as relief. It Figure 3 can be said that the reason why these patterns can Example of Islamic be seen as relief, is due to the material used. As can geometric pattern be imagined, the three-dimensionality can be given (Schneider, 1980) to the pattern by carving stones. Three-dimensionality in Islamic patterns is in the form of overlapping. The overlaps are defined in a very regular and systematic way. To understand this order, we can consider one of the strips that form stars in Islamic patterns. If a strip passes under next strip, it will pass over another strip in the next en- counter. This system repeats and a network-like tes- sellation is formed (Agirbas, 2020). An example of this type of pattern can be seen in Figure 2. Schnei- der (1980) has clearly described this situation in the METHODOLOGY drawings of Islamic geometric patterns in his book. In this study, six-pointed star polygon pattern tessel- In the drawings, although a pattern was created with lation in Schneider’s (1980) book was used in lifting Is- a single line place to place, Schneider (1980) made lamic geometric patterns to the third dimension and the system in the pattern understandable by show- transforming them into a reciprocal frame structure. ing the lines passing under the other lines (strips) The selected six-pointed star polygon pattern can be with a dashed line (Figure 3). seen in the upper left part of Figure 3. In Schneider’s When we consider the patterns in three dimen- (1980) drawings, it is determined how the third di- sions, we can easily say that the patterns consist mension of Islamic patterns is or could be (the stripes of many pieces. The systematic formation of many pass from the bottom or the top). This is important pieces by overlapping is similar to the principle of for the organization of the reciprocal frame structure. forming reciprocal structure. The selected six-pointed star polygon pattern can be created with using reference circle units. The

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 393 units can be multiplied and patterns with multiple been defined in order to ensure load transfer at the units can be formed. In this study a pattern with 7 joints of the star arms. The heights of the intermedi- units, a pattern with 19 units and a pattern with 37 ary columns were defined as 2 cm. These were also units have been created (Table 1, Figure 4). defined as beams to the script. This method is the method used by Gustafsson (2016) in his reciprocal structure study using Karamba. Figure 4 In this study, the load transfers at the points, Pattern with 37 where the beams overlap at their center points, were units excluded. No intermediary columns were added to these points. With the “support” component, how the struc- ture will be supported was defined. Different num- bers of supports were added to the modelled pattern and various tests were carried out. The translational degrees of freedom (Tx, Ty and Tz) and the rotational degrees of freedom (Rx, Ry and Rz) of the supports were locked. Various constants were determined for the tests. These are the force of gravity and beam sec- tion type (rectangular, 3x 2 cm).

RESULTS Results of the pattern with 7 units The pattern with 7 units is 200 cm in diameter. 4 tests were conducted for structural performance analysis. After creating three-dimensional models in The constants in these tests are given in Table 2. The Grasshopper [1], structural performance analyzes of variables in these tests are the number and location the models were made using the Karamba [2] pro- of the supports. In addition, structural performance gram. Karamba is a structural analysis program that simulations were performed with 3 types of materi- works as an add-on to Grasshopper. Comparisons of als, namely timber, steel and concrete, and the results the modelled structures were made by using differ- were compared (Table 2, Table 3). ent materials (timber, steel, concrete) and different • In the tests for the pattern with 7 units, it can be number of supports were used in order to see struc- seen that there is a little displacement value in all tural performance difference. results. In all tests, it was seen that steel showed Set-up less displacement value than the other materi- als. However, it should be noted that this system The patterns modelled in Grasshopper were given as is a very small system of 2 meters in diameter. input to the structural analysis script. For this, the • In the analyses where the material is kept con- “LineToBeam” component was used and it was con- stant in the tests and the number of supports nected to the “Assemble” component. “LineToBeam” (24, 12 and 6 pieces) is used as a variable, we component can transform lines defined in Grasshop- naturally see that the maximum displacement per into beams. With this component, the star arms in result increases as the number of supports de- the Islamic geometric pattern were defined as beams. creases. In addition, extra intermediary columns have

394 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Table 1 The reference circles and the patterns

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 395 Table 2 The results of the tests made with the pattern with 7 units

• In each test, the results of maximum compres- steel and concrete were used as variables (Table 4). sion, maximum tension, maximum moment and According to the results of these tests: maximum shear were the lowest in the timber • The results of maximum displacement values systems. were between 0.29 cm and 0.61 cm. • It was seen that steel showed less displacement Results of the pattern with 19 units value than the other materials. The pattern with 19 units is 390 cm meters in diame- • The results of maximum compression, maxi- ter. 2 sets of tests were conducted for the structural mum tension, maximum moment and maxi- performance analysis. In the first set of tests, timber, mum shear were the lowest in the timber sys-

396 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Table 3 Graphic representation of the tests (timber)

Table 4 The results of the material-based tests made with the pattern with 19 units

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 397 Table 5 The results of the profile cross-section based tests made with the pattern with 19 units (timber material)

tems. Results of the pattern with 37 units •In the comparison of the 7-unit (200 cm) and 19- The pattern with 37 units was 582 cm meters in di- unit (390 cm) models, the larger the diameter ameter. 2 sets of tests were conducted for structural of the model gives the larger the maximum dis- performance analysis. In the first set of tests, timber, placement value. steel and concrete were used as variables (Table 6). According to the results of these tests: In the second set of tests, profile cross-section was used as a variable. Various dimensions (3x2 cm, • The results of maximum displacement values 2x4 cm, 3x3 cm, 3x4 cm and 4x4 cm) for the cross- were between 0.48 cm and 1.02 cm. sections were used (Table 5). The material was fixed • It was seen that steel showed less displacement as timber. According to the results of these tests: value than the other materials. • The results of maximum compression, maxi- • As the dimension of cross-section increases, the mum tension, maximum moment and maxi- maximum displacement value decreases. While mum shear were the lowest in the timber sys- the highest result was in the test with 3x2 profile tems. cross-section, the lowest result was in the test • In the comparison of the 19-unit (390 cm) and with 4x4cm profile cross-section. 37-unit (582 cm) models, the larger the diame- Table 6 The results of the material-based tests made with the pattern with 37 units

398 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Table 7 The results of the profile cross-section based tests made with the pattern with 37 units (timber material)

ter of the model gives the larger the maximum value than the other used materials in every tests. displacement value. In each test, the results of maximum compres- sion, maximum tension, maximum moment and In the second set of tests, profile cross-section was maximum shear were found to be the lowest in the used as a variable. Various dimensions (3x2 cm, 2x4 timber systems. cm, 3x3 cm, 3x4 cm and 4x4 cm) for the cross-sections There are many similar types, more complex or were used (Table 7).The material was fixed as timber. simpler Islamic geometric patterns in the literature. According to the results of these tests: Similar studies can be performed using various Is- • As the dimension of cross-section increases, the lamic geometric patterns. Thus, new ones can be maximum displacement value decreases. While added to the reciprocal frame structure morpholo- the highest result was in the test with 3x2 profile gies. cross-section, the lowest result was in the test According to the results of the structural per- with 4x4cm profile cross-section. formance analyses, the created reciprocal structure works effectively. As a limitation of this study, it should be noted that, the load transfers at the points, CONCLUSION where the beams overlap at their center points, were In this study, based on Islamic geometric pattern, re- not included in the structural performance analysis. ciprocal frame structure was created. This study was In the case of inclusion of the load transfer at those carried out with the selected Islamic geometric pat- points, the system probably will be more rigid. Also, tern tessellation. The pattern created with 7 units has various dimensions of the cross-sections should be been lifted from the 2nd dimension to the 3rd dimen- tested. sion. Afterwards, reference circles were systemati- It should be note that this is a simulation-based cally added around this pattern, and patterns with 19 case study and may have intrinsic limitations and also and 37 units were created. Throughout this replica- may have biases based on the tool and method. tion process, the principle of reciprocal frame struc- tures has been adhered to. It has been observed that REFERENCES the system has a dome-like shape. Agirbas, A 2020, ’Algorithmic Decomposition of Geo- When the system was enlarged with increasing metric Islamic Patterns: A Case Study with Star Poly- the units, it was observed that the maximum dis- gon Design in The Tombstones of Ahlat.’, Nexus Net- placement values in the system normally increased. work Journal, 22, p. 113–137 It was also seen that steel showed less displacement Anastas, Y, Rhode-Barbarigos, L and Adriaenssens, S

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 399 2016, ’Design-To-Construction Workflow For Cell- Parigi, D and Kirkegaard, PH 2014, ’The Reciprocalizer: Based Pattern Reciprocal Free-Form Structures’, an Agile Design Tool for Reciprocal Structures’, Nexus Journal of the International Association for Shell and Network Journal, 16, p. 61–68 Spatial Structures, 57(2), pp. 159-176 Parigi, D and Pugnale, A 2012 ’Three-dimensional Recip- Baverel, O 2000, Nexorades: a Family of Interwoven Space rocal Structures: Morphology, Concepts, Generative Structures, Ph.D. Thesis, University of Surrey Rules’, The Proceedings of IASS (International Associa- Bergis, L and De Rycke, K 2017 ’Reciprocal Frame for the tion for Shell and Spatial Structures) Annual Symposia, Roof of the Franz Masereel Centre’, The Proceedings University of Aalborg of IASS (International Association for Shell and Spatial Pizzigoni, A 2009 ’A High Fiber Reinforced Concrete Pro- Structures) Annual Symposia, University of Hafencity, totype for Reciprocal Structures of Demountable Hamburg Building’, The Proceedings of IASS (International Asso- Bonner, J 2017, IslamicGeometricPatterns: TheirHistorical ciation for Shell and Spatial Structures) Annual Sym- Development and Traditional Methods of Construc- posia, Politecnica de Valencia, Valencia, pp. 1895- tion, Springer, NY 1906 Di Carlo, B 2008, ’The Wooden Roofs of Leonardo and Schneider, G 1980, Geometrische Bauornamente Der Seld- New Structural Research’, Nexus Network Journal, 10, schuken in Kleinasien, Reichert, Wiesbaden pp. 27-38 Song, P, Fu, CW, Goswami, P, Zheng, J, Mitra, NJ and Douthe, C and Baverel, O 2009, ’Design of “nexorades” Cohen-Or, D 2014, ’An Interactive Computational or “structures with mutually supporting elements” Design Tool for Large Reciprocal Frame Structures’, with the dynamic relaxation method’, Computers & Nexus Network Journal, 16, p. 109–118 Structures, 87 (21-22), pp. 1296-1307 Thonnissen, U 2014, ’A Form-Finding Instrument for Re- Gelez, S, Aubry, S and Vaudeville, B 2011, ’Nexorade or ciprocal Structures’, Nexus Network Journal, 16, p. Reciprocal Frame System Applied to the Design and 89–107 Construction of a 850 m2 Archaeological Shelter’, Williams, K 2008, ’Transcription and Translation of Codex International Journal of Space Structures, 26 (4), pp. Atlanticus, fol. 899 v’, Nexus Network Journal, 10, p. 303-311 13–16 Gerber, D and Pantazis, E 2016 ’Design Exploring Com- Xian, Z, Hoban, N and Peters, B 2020 ’Spatial Timber plexity in Architectural Shells: Interactive form find- Assembly: Robotically Fabricated Reciprocal Frame ing of reciprocal frames through a multi-agent sys- Wall’, Proceedings of eCAADe, TU Berlin, Berlin, Ger- tem’, Proceedings of eCAADe, University of Oulu, many, pp. 403- 412 Oulu, Finland, pp. 455-464 [1] https://www.grasshopper3d.com/ Gustafsson, J 2016, Connections in Timber Reciprocal [2] https://www.karamba3d.com/ Frames, Master’s Thesis, Chalmers University of Technology, Department of Civil and Environmen- tal Engineering, Division of Structural Engineering, Gothenburg, Sweden Houlsby, GT 2014, ’John Wallis and the Numerical Analy- sis of Structures’, Nexus Network Journal, 16, p. 207– 217 Kohlhammer, T and Kotnik, T 2011, ’Systemic Behaviour of Plane Reciprocal Frame Structures’, Structural En- gineering International, 21(1), pp. 80-86 Popovic Larsen, O 2008, Reciprocal Frame Architecture, Architectural Press, Oxford Popovic Larsen, O 2014, ’Reciprocal Frame (RF) Struc- tures: Real and Exploratory’, Nexus Network Journal, 16, p. 119–134 Popovic Larsen, O 2019, ’Reciprocal Timber Structures and Joints’,in Hudert, M and Pfeiffer, S (eds) 2019, Re- thinking Wood: Future Dimensions of Timber Assem- bly, Birkhauser, Basel, pp. 88-99

400 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Towards Prediction and Optimisation for Outdoor Evaporative Cooling Systems in Architectural Design

Djordje Stojanovic1, Milica Vujovic2 1University of Melbourne 2Universitat Pompeu Fabra [email protected] [email protected]

The paper presents a blueprint of a simulation model and structures the processing of information that sensors in real life would provide to improve the efficiency of the cooling system and increase the thermal comfort of occupants in an outdoor environment. It outlines an evidence-based technique for evaluating perceived benefits that would arise from the introduction of Wireless Sensor Networks (WSNs) and Computational Techniques (CTs) to evaporative cooling systems. The initial results indicate how data that would be gathered by sensory devices on occupancy and microclimatic conditions can be employed for the prediction and optimisation of the system to narrow the time gap between peak usage and nozzle activation to minimise resource use and maximise occupants' comfort. The larger objective of the study to help mitigate the effects of rising temperatures in urban environments and support the future use of outdoor public spaces. In the longer run, the study aims to explore the role of computationally enhanced microclimate control at a large scale, integral to innovative approaches to outdoor public spaces design.

Keywords: Human-Building Interaction, Interactive Architecture, Responsive Environments, Evaporative Cooling, Outdoor Public Space

INTRODUCTION search reported aims to contribute to the improved On warm days, parks are often regarded as the most efficiency of an existing evaporative cooling system pleasant city areas because the water evaporating to help improve thermal comfort in outdoor areas, from plants’ leaves cools down the surrounding air. using Computational Techniques (CTs) and Wireless Much like in nature, human-made evaporative cool- Sensor Networks (WSNs). ing solutions remove heat from the atmosphere by This paper outlines the research as follows. An emitting water droplets that change phase from liq- introduction to outdoor ambient cooling systems is uid to vapour and reduce the air temperature in the followed by a literature review focusing on recent ef- process. However, known ambient cooling solutions forts employing CTs and WSNs to study the interac- are rarely applied at the scale that could impact large tion between human behaviour, microclimate, and urban environments in the same way parks do. By the built environment. A brief examination of chal- drawing on a natural process of transpiration, the re- lenges and opportunities follows the research aims

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 401 are formulated. The study explores what informa- spite the overwhelming evidence that rising temper- tion can be acquired on human behaviour and how atures in cities compromise the quality of public life it can be processed to reduce cooling systems’ water (e.g., Dwivedi 2019; Taha 1997; and Yang et al. 2016). consumption and increase occupants’ thermal com- The evaporative system is composed of the wa- fort. To that end, research objectives are addressed ter pump, filter, supply tubing, and different nozzles. by the formulation of the simulation modelling tech- Nozzles use air to break up the fluid into a spray pat- nique to examine potential improvements to an ex- tern. Previous studies on evaporative cooling sug- isting evaporative cooling solution ”Static Mist Line gest that water release patterns have the biggest im- System” produced by Ozmist leveraged by the em- pact on the system’s efficiency (Nunes et al. 2011, ployment of sensing devices and computation. In and Yamada et al. 2008). Design guidelines on evap- the results section, the paper presents how the com- orative cooling suggest that the maximum dispersal putational system can use data that would be gen- angle is 70˚, and each source installed at the recom- erated through occupancy detection and microcli- mended height of 3m may affect the area measuring matic monitoring to control the nozzles’ emitting wa- up to 5m in diameter (Osmond and Sharifi 2017). This ter vapour. The initial results indicate how the sys- study is based on the existing Static Mist Line Sys- tem’s ability to learn from the environment can be tem developed by Ozmist (Figure 1). Its manufactur- used to optimize its operation and develop a predic- ers pride themselves on the ability to ultra-fine mist tive capacity. And why computational techniques for droplets that evaporate before wetting floor surfaces. prediction and optimisation could be utilized to nar- It is most commonly applied along the perimeter of row the time gap between peak usage and systems’ the area, using stainless-steel tubes to create a cur- activation to minimize the use of resources and max- tain of mist. When combined with fans, dispersing imize occupants’ comfort. In the end, we discuss the the airflow, the system can reduce outdoor air tem- study results, the challenges to further development perature up to 10°C, according to the manufacturer’s of the fully functional prototype, and its contribution product guide. The solution pressurises tap water to to the larger objective to help mitigate the effects of 1,000 [PSI] and forces it through a tiny nozzle that rising temperatures in urban environments and sup- shatters the stream and creates droplets that are just port the future use of outdoor public spaces. 1/10th the thickness of a human hair (Ozmist 2020). The solution is put into function with a simple switch Outdoor Ambient Cooling Systems or by a thermostat, and it operates with a single level Evaporative cooling is one of the known solutions of intensity. for micro-climate control in outdoor areas. The sys- tem uses water that changes the phase from liquid to vapour to remove heat from the atmosphere. The ambient air temperature is reduced by forcing water through small nozzles under high pressure, produc- ing fine mist clouds that absorb ambient heat (Os- mond and Sharifi, 2017). Recent research confirms the efficiency of the system and its ability to lower air temperature in outdoor environments with minimal energy consumption (Farnham, Emura and Mizuno 2015; Farnham et al. 2017; Frederick and Khosla 2014). However, evaporative cooling is still rarely ap- plied at the systemic level in outdoor public areas, de-

402 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Figure 1 ciently explored for the enhancement and upscaling Static Mist Line of evaporative cooling systems in outdoor environ- System produced ments. by Ozmist Previous Research The research reported in this paper builds on the previously published framework for the responsive operation of an evaporative cooling system that ad- dresses the relationship between human behaviour and micro-climate (Stojanovic and Vujovic, 2020). Our initial research aim was to include a diverse set of input parameters related to human activities in outdoor public areas provided by pressure sensors. Originally the approach was based on the location of occupants. The proposed system would be able to establish the speed of movement and proxim- ity between occupants and therefore indirectly help understand human behaviour. In parallel, a locally based weather station was planned to provide micro- climate data, including air temperature, wind veloc- ity and direction, and humidity. The outcome of that study was the information flow matrix that sat the grounding for further research to develop compu- tational techniques to process information and im- prove the evaporative cooling system’s efficiency. The framework showed that a wide set of parame- ters could be used to enhance the ambient cooling system’s operation in an outdoor environment. How- ever, the two important conclusions were that not all input parameters would have equal relevance for Wireless Sensor Networks the optimisation of the system’s operation and that Groups of sensors linked with wireless media have too many input categories increase computational found diverse applications in the built environment complexity and require excessive programming ef- linked to optimising the use of resources and occu- forts. Therefore, the research presented in this paper pants’ wellbeing (Wu and Noy, 2010). WSNs are in- focuses on two input categories, occupancy count creasingly used to study human behaviour and mi- and air temperature, to outline an optimisation pro- croclimate in the built environment (eg. Biswas et al. cedure while allowing for the inclusion of other pa- 2016, Guo et al. 2019, Kim et al. 2016, Malkawi and rameters in future. Srinivasan 2005, Munitxa and Bogosian 2015). Ad- vanced computational techniques for data structur- ing and prediction have enabled improved accuracy, RESEARCH AIMS robustness, and reduced operating costs of WSNs The study presented in this paper is based on the (Dai et al. 2020). However, the full potential of WSNs initial desktop research that recognises the benefits and advanced computational techniques is not suffi- of ambient cooling and the perceived potential of

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 403 WSNs and CTs to add ‘intelligence’ to the existing ing solution by using current air temperature and evaporative cooling solution, commonly applied at a occupancy measurements and take their previous small scale. The paper presents a new step and an and predicted values into account. This input re- important stage of the ongoing study by outlining places the planned acquisition of data in real life to a technique for evaluating perceived benefits that enable the research phase where we begin establish- would arise from the introduction of WSNs and CTs ing the innovative procedure for optimisation and to a specific evaporative cooling solution. This paper prediction. In real life, the input information, struc- addresses the following research questions. How to tured into two datasets, would be provided by WSN, structure the processing of information that would whereby data set one would consist of information be provided by sensory devices to improve the effi- provided by a thermostat recording local air temper- ciency of the cooling system and increase the ther- ature values, and data set two would be provided mal comfort of occupants in an outdoor environ- by pressures sensor that would be integrated into ment? And, how to examine the validity of the po- the pavement surface to record the number of oc- tential improvements to evaporative cooling systems cupants within the area affected by a segment of an that WSNs and CTs would enable? evaporative cooling system in an outdoor environ- The study answers the question by presenting ment. The occupancy count within established area a blueprint of the simulation model to generate limits and local air temperature would be recorded datasets that would be used to control the percent- in ten-minute intervals. age of active nozzles and water release periods in re- The model examines how the use of two datasets sponse to input information consisting of the tem- can help optimize the system’s operation to use perature and occupant’s count datasets provided by fewer resources. The study identifies three optimiza- WSN. The paper outlines a procedure for correcting tion levels to establish how responsive operation of the system’s operation by taking current air temper- the cooling system can minimize water consump- ature and predicted occupancy count into consider- tion while controlling local air temperature in an out- ation and sets the grounding for prototype devel- door environment. The system’s actuation is via wa- opment. The initial results indicate how data that ter vapour emission intensity expressed in percent- would be gathered by sensory devices on occupancy age of active nozzles [%], and the duration of release and microclimatic conditions can be employed for periods defined in minutes [min], which are directly the prediction and optimisation of the system. The proportional to water consumption and could be ex- presented outcomes would help narrow the time gap pressed in litres [l]. between peak usage and nozzle activation to mini- The model also examines how the use of two mize resource use and maximize occupants’ comfort. datasets can help improve occupants’ thermal com- This research aims to enable the upscaled ap- fort by reducing the time gap between peak occu- plication of evaporative cooling systems, reduce wa- pancy and the system’s activation. As the cooling sys- ter and energy consumption and increase occupants’ tem becomes fully effective only after a period of ac- thermal comfort in outdoor public spaces. tivity, the duration of this period may impact the sys- tem’s ability to respond to occupants’ demands and METHOD make the best possible use of resources. Reducing The proposed simulation model uses a randomly this gap could be of great importance in an outdoor generated occupancy count and approximated environment, where the air temperature is less sta- weather statistics to structure information flow to ble and more difficult to control than indoors. Rather optimise the evaporative cooling system’s operation. than activating the cooling system when occupants’ The goal is to enhance the existing evaporative cool- presence is detected, the model sets grounding for

404 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 the technique to adjust system operation in advance age periods. If current measurements match the es- based on predicted occupancy count. tablished indicator values and their correlation, the water vapour emission and its release patterns in- RESULTS crease to minimise the potential gap between the The three optimisation levels identified are compat- system’s full performance and highest occupancy ible but differ in the way data on occupancy, and air levels. The proposed simulation technique aims to temperature is used. examine the increase of water consumption through Level one is the most basic optimisation level, a pre-emptive upsurge of the vapour emission and its yet it operates with a complexity requiring a com- release patterns that would increase occupant’s ther- putational approach. Rather than switching the sys- mal comfort during predicted peak usage. tem off to reduce water consumption, the approach The following table shows the structure of the maintains operation at the level proportional to the database that would be generated by the simulation difference between the locally measured and desired in the form of a spreadsheet (Figure 2). temperature values. It uses temperature values to de- The first column one shows the date and time termine the optimal use of water through calculated when the measurements would be taken in real life. adjustment of the intensity of vapour emission and Values would be recorded at 10-minute intervals, be- the duration of release patterns. The proposed simu- tween 9:00 and 18:00h, for 12 weeks. Therefore, the lation technique aims to determine if longer periods simulation model uses 4536 values from each of the with lower intensity could be more efficient and con- two input categories to generate data that would en- sume less water than shorter periods with higher in- able examining the efficiency of the proposed opti- tensity of vapour emission. misation and prediction method. At level two, current measurements that WSN To enable conception of the simulation model, would provide are compared against previous val- air temperature values in the second column can ues of the same dataset to establish the increasing be retrieved from historical hourly weather statistics or decreasing trend. Based on the rising or declining specific to the relevant geographic location. The tendency of air temperature, the system predicts the planned use of WSN would offer more prices and temperature of the next hour and corrects the inten- more frequent data measurements that would im- sity of the water vapour emission and its release pat- prove the accuracy of results. The third column of the terns to ease the fulfilment of its objective to main- spreadsheet is for the occupancy count. The number tain the predicted air temperature. The proposed of occupants would be determined by non-invasive simulation technique’s objective is to determine if occupancy detection techniques relying on pressure the timely reduction or upsurge of the water vapour sensors embedded into the floor surface in real life. emission and its release patterns could additionally For the initial simulation modelling, the number of reduce overall water consumption and enable eas- occupants can be generated randomly. It is assumed ier matching between systems operations and occu- it would be limited to 10 people per 30 [m2] area or pant’s thermal comfort. a microclimate zone controlled by the existing evap- At level three, the previous occupancy data is orative cooling unit taken as a departing point of the analysed, and peak usage times are identified, i.e., study. when the simultaneous presence of 6 or more peo- Columns four and five show the level one op- ple is recorded within the area limits. Values of the timisation, or the first correction based on the cur- two datasets, the air temperature and occupancy rent air temperature and occupancy count. The cor- count leading to the peak usage period, are estab- rections would be shown as amended water vapour lished as indicators of probable or predicted peak us- release periods and the percentage of active noz-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 405 Figure 2 An overview of input information and datasets that would be generated by the simulation

zles. These parameters would be translated into wa- or the cost of thermal comfort improvement. ter consumption and expressed as numeric values to The final two columns show the level two op- enable evidence-based evaluation of systems perfor- timisation or corrections related to predicted occu- mance. pancy peaks. The current occupancy and air temper- Columns six and seven show the level two opti- ature measurements are compared to previous mea- misation or corrections based on the air temperature surements to identify a string of data pointing to- trend. If the temperature is increasing, the percent- ward occupancy peak. When the peak is identified, age of active nozzles and the pattern of release peri- the percentage of active nozzles and release patterns ods are intensified to match anticipated climate con- are increased to ease the system’s objective to main- ditions and improve systems’ readiness to respond tain the outdoor temperature at the desired thresh- to occupant’s needs. Similarly, if the temperature is old, i.e., 24 [C]. The increase also allows for a shorter decreasing, if the current temperature is lower than gap between occupancy detection and realisation of the previous temperature value, the system would re- the system’s full potential. The simulation method duce the percentage of active nozzles and the pat- aims to establish the main principles of the predic- tern of release periods. These parameters would be tion technique and examine the resulting increase in used to calculate water consumption during such a water consumption. changed mode of operation to allow comparison and evidence-based examination of anticipated savings

406 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 DISCUSSION AND CONCLUSION prototype. The first research question on how to use informa- Future studies include analysis of the data gen- tion that would be provided by sensory devices to erated through simulation to enable data and the improve the efficiency of the cooling system and development of CTs for prediction and optimisation. increase the thermal comfort of occupants in an Prototyping is planned to enable the acquisition of outdoor environment is answered with the simula- real-life data in an outdoor environment. In the tion blueprint and an information processing proto- longer run, future research will examine how im- col. The three optimization levels are outlined, each proved evaporative cooling systems enhanced with implying a correction to the system’s performance, WSNs and CTs, and applied at a large scale, can be- based on the interpretation of occupancy and air come an integral part of innovative design methods temperature information. The outlined computa- for outdoor public spaces in response to rising tem- tional approach is based on current and past mea- peratures and climate change. surements and predicted values for both input cat- This study’s contribution is in developing a sim- egories. The simulation model’s purpose is to gen- ulation model to enable structuring a computational erate datasets defining the percentage of active noz- approach to optimizing the evaporative cooling sys- zles and water release periods per ten-minute inter- tem and developing the evidence-based technique vals. for examining potential improvements that would The second research question on how to ex- arise from WSNs and CTs. amine the validity of the potential improvements to evaporative cooling systems introduced with REFERENCES WSNs and CTs is answered through empowering of Biswas, P, Moon, S, Qi, H and Dey, AK 2016, ’Computa- evidence-based technique, for comparison of water tional frameworks for context-aware hybrid sensor consumption between the three proposed levels of fusion’, International Journal of Image and Data Fu- optimisation, and the default operation of the sys- sion, 7(1), pp. 83-102 tem controlled by the thermostat only. The proposed Dai, X, Liu, J and Zhang, X 2020, ’A review of studies applying machine learning models to predict oc- simulation model’s results in the form of datasets that cupancy and window-opening behaviours in smart define the percentage of active nozzles and water re- buildings’, Energy and Buildings, 223, p. 110159 lease periods per ten-minute intervals can be trans- Dwivedi, A 2019, ’Macro- and micro-level studies using lated into the graphs illustrating water consumption Urban Heat Islands to simulate effects of greening, and offering themselves to further investigation and building materials and other mitigating factors in optimisation of the system. Mumbai city’, , Architectural Science Review, 62(2), pp. 126-144 Limitations of this study arise from the inclusion Farnham, C, Emura, K and Mizuno, T 2015, ’Evaluation of of only two input categories, while many other cli- cooling effects: outdoor water mist fan’, Building Re- matic parameters such as air humidity, solar radia- search & Information, 43, pp. 334-345 tion, wind direction and speed, may impact or drive Farnham, C, Zhang, L, Yuan, J, Emura, K, Alam, AM and the performance of outdoor evaporative cooling sys- Mizuno, T 2017, ’Measurement of the evaporative tems. Simultaneously, occupancy metrics used in cooling effect: oscillating misting fan’, Building Re- search and Information, 45(7), pp. 783-799 this study do not address difficulties in predicting hu- Guo, Z, Wang, X and Yuan, F 2019 ’Sensing Human Be- man behaviour. Other limitations arise from the lack havior in the Built Environment’, Proceedings of 18th of real-life data collection, making presented results International Conference, CAAD Futures 2019, Dae- difficult to evaluate. On the other side, the benefits of jeon, pp. 378-388 the simulation model include the possibility of devel- Kim, B, Kang, CM, Kim, J, Lee, SH, Chung, CC and Choi, JW oping a computational system ahead of the real-life 2017 ’Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network’,

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 407 Proceedings of IEEE 20th International Conference on Intelligent Transportation Systems (ITSC), Yokohama, pp. 399-404 Malkawi, AM and Srinivasan, RS 2005, ’A new paradigm for human-building interaction: the use of cfd and augmented reality’’, Automation in Construction, 14(1), p. 71–84 Munitxa, M and Bogosian, B 2015 ’Sensing Urban Mi- croclimates’, Proceedings of ACADIA 2015: Computa- tional Ecologies: Design in the Anthropocene, Cincin- nati, pp. 483-496 Nunes, J, Zoilo, I, Jacinto, N, Nunes, A, Torres-Campos, T, Pacheco, M and Fonseca, D 2011 ’Misting-cooling systems for microclimatic control in public space’, EFLA Regional Congress of Landscape Architecture, Talin Osmond, P and Sharifi, E 2017, Guide to Urban Cooling Strategies, Low Carbon Living CRC Stojanovic, D and Vujovic, M 2020 ’Algorithmic Frame- work for Correlation Between Microclimate Control and Space Usage in Outdoor Public Spaces’, Proceed- ings of eCAADe 2020, Berlin, pp. 517-524 Taha, H 1997, ’Urban climates and heat islands: Albedo, evapotranspiration, and anthropogenic heat’, En- ergy and Buildings, 25, p. 99–103 Venkiteswaran, VK, Liman, J and Alkaff, SA 2017, ’Com- parative Study of Passive Methods for Reducing Cooling Load’, Energy Procedia, 142, p. 419 – 426 Wu, S.N and Noy, P 2010, ’A conceptual design of a wire- less sensor actuator system for optimizing energy and well-being in buildings’, Intelligent Buildings In- ternational, 2(1), pp. 41-56 Yamada, H, Yoon, G, Okumiya, M.A and Okumiya, H 2008, ’Study of cooling system with water mist sprayers: Fundamental examination of particle size distribu- tion and cooling effects’, Building Simulation, 1, p. 214–222 Yang, L, Qian, F and Zheng, k 2016, ’Comparative Study of Passive Methods for Reducing Cooling Load’, En- ergy Procedia, 169, p. 419 – 426 [1] https://www.ozmistoutdoors.com.au/

408 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Use of Simulation Techniques and Optimization Tools for Daylight, Energy and Thermal Performance

The case of office module(s) in different climates

Feyza Nur Aksin1, Semra Arslan Selçuk2 1,2Gazi University, Department of Architecture 1,2{feyzanursariyildiz|semraselcuk}@gazi.edu.tr

In recent years, performance-based design has become the key issue behind design decisions in the construction industry towards reducing energy consumption. Various simulation techniques and optimization tools have started to be used together for performance objectives to reach optimal solutions for complex design process. In the sector, one of the most energy-consuming buildings is offices. This study examines the effects of integration of simulation programs and optimization tools on the daylight, energy and thermal performances of office buildings on different climates. Two cities, Ankara and Izmir, in Turkey selected as locations. The study is carried out with total of thirteen parameters. With Rhinoceros/Grasshopper software, Honeybee, Ladybug and Octopus plug-ins used for daylight, energy and thermal simulation and performance optimization. With the results obtained, the optimal configurations related with selected parameters are determined for reducing energy consumption while improving daylight and thermal performance on different climates.

Keywords: daylight, energy and thermal comfort performance, multi-objective optimization, performance-based design, office buildings

INTRODUCTION comfort for occupants gains a great importance in of- Today, buildings are responsible for almost 40% of fice buildings. When these buildings are examined in global energy consumption and 36% of greenhouse general, it is seen that the need for lighting, heating gases (IEA 2019). For these reasons, especially in re- and cooling constitutes the large portion of energy cent years, building sector has focused on especially consumption. In particular, insufficient efficiency of the concept of performance, because of its potential daylight and thermal performance also contributes contribution to current problems in the built environ- greatly to this consumption. For example, failure to ment (Ng 2013: 4). In the sector, one of the most provide sufficient amount of sunlight in buildings in- energy-consuming buildings is offices and consider- creases the use of artificial lighting (Nazaroff 2014). ing that a significant portion of the daily time period According to Wong (2017), electrical lighting consti- is spent in workplaces, energy efficiency and indoor tutes up to 40% of the annual energy consumption

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 409 of buildings. So, the real functionality of energy effi- search for the reduction of annual electricity con- cient buildings also depends on the visual and ther- sumption and predicted percentage of dissatisfied mal comfort of the building occupants under ambi- on a single office room. With the results obtained, it ent conditions (Delgarm et al. 2016). While reduc- is stated that multi-objective optimization produces ing energy consumption, ensuring indoor visual and more suitable solutions than single objective opti- thermal comfort conditions at an optimum level not mization. In another study, a total of fifteen variables only reduces the harmful effects of buildings on the of an office building are optimized for four differ- environment, but also ensures that the occupants are ent climate zones of Australia to reduce energy con- more productive and keep them away from the psy- sumption (Bamdad et al. 2017). chological and physiological problems that arise due In some studies, optimization has been per- to insufficient indoor conditions (Chen et al. 2015; formed on different parameter values of windows Cheong et al. 2020). to improve various objectives such as energy per- Daylight, energy and thermal performance in formance, daylight performance, and quality of view buildings change depending on many different pa- (Shahbazi et al. 2019; Pilechiha et al. 2020). Zhai rameters and these parameters can affect each other. et al. (2019), in their study on an office room in For this reason, a parametric design is needed to Xi’an, China, optimized window properties and ori- make an evaluation based on performance. With entation in order to increase thermal comfort and vi- the help of parametric modeling and simulations, de- sual performance while reducing energy consump- pending on the performance target, optimum solu- tion. Fang and Cho (2019), optimized the building tions can be determined for building design (Eltaweel size, roof, skylight, louver and window properties in and Su 2017). Also, the high level of complexity that order to reduce energy consumption while increas- results in balancing the performance requirements of ing the useful daylight illuminance value for three dif- buildings increases the demand for multi-objective ferent climate zones. Zhao and Du (2020), on the optimization. For a high-performance building de- other hand, optimized the building orientation, win- sign, it is necessary to optimize integrated strategies dow and shading design of an office building for dif- and technologies. ferent climate zones in China in order to minimize In recent years, it is seen that there are various heating, cooling, lighting and discomfort hours. optimization studies in the related literature regard- Although it has been observed that there are de- ing the performance evaluation of the offices. For tailed studies among the related literature, it is note- example, in various studies, different parameters of worthy that there are generally few studies combin- shading or louvers have been optimized for different ing multiple parameter and objective options. In objectives such as reducing CO2 emissions, energy studies, either the number of parameters is kept consumption, increasing daylight and visual com- high or the number of performance objectives is in- fort performance (González and Fiorito 2015; Man- creased. For this reason, unlike other studies, the zan and Padovan 2015; Lavin and Fiorito 2017; Uribe number of parameters and performance objectives et al. 2017). Méndez Echenagucia et al. (2015) car- has been increased in this study in order to make a ried out an optimization study to reduce the heat- more accurate assessment of building performance. ing, cooling and lighting energy need for an open With a total of 13 parameters: width, depth and space office over windows and wall thickness to pro- height of the module, orientation, window to wall ra- vide shading. According to the results, it is observed tio (WWR), depth, count, angle and direction of shad- that the window arrangement in general is highly ef- ing, wall insulation thickness, glazing type, cooling fective on energy efficiency. Delgarm et al. (2016) and heating set points, a simulation based optimiza- have done single objective and multi objective re- tion study was carried out to increase the useful day-

410 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Table 1 Design parameters and possible values

light illuminance (UDI100-3000lx) while reducing en- lated to the outside. The exterior wall is fixed to the ergy use intensity (EUI) and predicted percentage of module’s width and rotates with the module with the dissatisfied (PPD) in order to increase energy, thermal change of the orient parameter. Design parameters comfort and daylight performance. In addition, two and possible values are shown in Table 1 and Figure different climatic zones were evaluated in the study, 1. and the effects of climate were also examined. The methods used and the results obtained are discussed Daylighting Simulation in detail in the next sections of the study. Honeybee and Ladybug tools were used for the day- light simulation model of the study. First of all, Radi- METHOD ance materials to be used were assigned to glazing, In this part of the study, an optimization process was shading, walls, floor and ceiling elements separately carried out together with parametric modeling and on the parametric geometry of office module (Table simulation analysis in order to evaluate daylight per- 2). The daylight performance metric to be used in this formance based on different parameters. Simula- study is UDI value, as stated before. UDI metric is the tions and optimization were performed for two dif- percentage of the time during the active occupancy ferent cities, Ankara with temperate-dry climate and hours that the point in the space receives the target Izmir with hot-humid climate. range of illuminance defined as useful and provided by daylight. Nabil and Mardaljevic (2005), while in- Parametric Modelling troducing this metric, proposed the UDI range be- A parametric model of a single office module was cre- tween 100 and 2000 lux, describing situations that ated using Rhinoceros/Grasshopper. After the initial are not too dark or too bright, but they updated the model was created, the developed parametric model upper threshold value as 3000 lux in a later publica- was completed with Honeybee tool. The determined tion (Mardaljevic et al. 2012). In this study, for an parameters for this study are width, depth and height achieved illuminance range, UDI100-3000lx value is of the module, orientation, WWR, depth, count, an- selected to evaluate the daylight performance of of- gle and direction of shading, wall insulation thick- fice module. ness, glazing type, cooling and heating set points. It After the materials were assigned, the test sur- was assumed that only one side of the module is re- face was prepared, and Radiance parameters’ val-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 411 Figure 1 (a) Isometric view of the parametric module (b-c) section of shading elements with varying parameter options

Table 2 Material properties of the module

ues (Table 3) and weather data of the location were Table 3 defined in the daylight model. The grid size and Radiance distance from surface values used for test surface simulation are 0.5 m x 0.5 m and 0.85 m respectively. For the parameters daylight model the suggested values from Reinhart (2019) were used. When the installation of the day- Energy Simulation light model was completed, the simulation phase For energy simulation, material properties for wall was started. With the daylight model created in this and glazing were set to the model. All surfaces, ex- section, a simulation study was carried out on the cept for the wall with glazing, were assumed as adi- office module via Radiance/Daysim. Then, the data abatic. For office module, equipment loads per area, obtained were converted to UDI100-3000lx metric lighting density per area and number of people per within the range of 09.00-18.00 occupancy hours and area were set to 7.65 W/m2, 11.85 W/m2, 0.15 ppl/m2 the lunch hours assumed to be 12.00-13.00. respectively. Annual profiles obtained from daylight simulation were used for the lighting schedule. The

412 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 occupancy schedule was matched with the daylight RESULTS AND DISCUSSION simulation and kept the same for the equipment In this part of the study, solution alternatives ob- schedule. EnergyPlus simulation engine was used for tained as a result of simulation-based optimization the simulation. Fan coil units + DOAS was used as a were evaluated. In order to evaluate the obtained HVAC system. Cooling and heating set points are also pareto-front solutions more effectively, the follow- defined as parameters in different value ranges in or- ing fitness function formula introduced by Konis et al. der to make a more accurate evaluation for different (2016) and previously used by Pilechiha et al. (2020) climatic zones. After simulation was done, annual en- was used. ergy usage values in terms of cooling, heating, light- FF = (UDIi − UDI min)C1− ing and equipment loads were obtained and these 1(EUIi − EUI min)C2− (1) were transformed into a total EUI value. EUI indicates 1(P P Di − PPD min)C3 energy consumption per unit area for each year. Where; Thermal Comfort Assessment i: result of iteration min: minimum value of optimization set After the energy simulation results are obtained, max: maximum value of optimization set PPD value is calculated for thermal comfort evalu- 100 ation. PPD is “an index that establishes a quan- C 1 = − (2) titative prediction of the percentage of thermally UDI max UDI min 100 dissatisfied people determine form predicted mean C2 = (3) vote”(ASHRAE 2021). Values less than 10% are recom- EUI max −EUI min mended for PPD. For calculation, dry bulb tempera- 100 C3 = (4) ture, mean radiant temperature and relative humid- PPD max −PP D min ity values were obtained from energy simulation re- sults. After the metabolic rate was set to 1 met and Results for Ankara the clothing level to 1 clo, the calculation was com- For Ankara, all individuals and pareto-front individ- pleted and the PPD value was obtained. uals obtained as a result of optimization are shown in Figure 2. When the results were examined, it Optimization was seen that the EUI values were between 115.25 For the optimization of the office module, in this part kWh/m2 and 58.28 kWh/m2. Individuals with the of the study, a total of 13 parameters (see Table 1) highest EUI performance values are generally north and three performance metrics (UDI100-3000lx, EUI, oriented, having 0.7 WWR and a cooling setpoint of PPD) determined in the previous sections were evalu- 27oC. For these individuals, either the no shading op- ated. UDI value was tried to be maximized while EUI tion or the 0.2 m shading depth option usually comes and PPD values were minimized. With the Octopus to the fore. Although WWR is high, the glazing type plug-in used for optimization, a logical balance has being triple loE and the insulation thickness being been tried to be established between parameters in high were effective in reducing the EUI value. While order to improve performance metrics. HypE algo- the UDI100-3000 lx values of these individuals gen- rithm was used for optimization. Optimization was erally took values above 90%, their PPD values ex- started for ten generation by setting mutation rate ceeded 14%. It was observed that the choice of low 0.1, mutation probability 0.5, crossover rate 0.8, and insulation thickness and double clear glazing type population size 50 and the steps of the case study was effective for the individual with the worst EUI were completed. performance value. As a result of the optimization, the PPD per-

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 413 Figure 2 All individuals and pareto-front individuals obtained from the optimization results of Ankara

formance metric took values between 21.56% and with high performance, while WWR values are gener- 5.41%. WWRs of individuals with the best PPD per- ally 0.2 or 0.3 in individuals with low performance. formance generally took values between 0.5 and 0.7. When the pareto-front individuals are examined, The daylight performance metric values of these in- it is seen that the east and west directions are not dividuals are generally around 75%, and their en- included in the parameter values. Insulation thick- ergy consumption values are generally close to 80 ness generally took the two highest values. 18oC is kWh/m2. It has been observed that there is an in- not included in the values for the heating setpoint verse relationship between PPD and EUI. No shad- parameter. In Table 4, pareto-front individuals with ing option was observed in individuals with high PPD the highest ten fitness function values according to performance. Orient parameter stands out as south. equation 1 are listed. Looking at the table, it is possi- Cooling setpoint got the lowest value of 23oC. ble to observe that certain parameter values prevail. For Ankara, UDI100-3000 lx performance metric It has been observed that all individuals are north ori- took values between 93.79% and 56.40%. The most ented and have a triple loE glazing type. Insulation effective parameter for the UDI values to differ was thickness has its highest values. With the thermal and WWR. WWR parameter values are high in individuals optical properties of triple loE glazing and high insu- Table 4 Top ten solutions with the highest fitness function value among pareto-front individuals for Ankara

414 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 lation thickness, the energy performance is generally viduals with the worst energy performance have no improved by about 50% compared to the individual shading and double clear glazing type. In addition, with the worst energy performance. High window these individuals are west-oriented. Due to the loca- ratios were effective in increasing UDI values. While tion of Izmir, solar rays coming from the west caused the height parameter of the module takes 3.6 m and an increase in energy consumption. above values, a deeper module form has emerged. PPD performance metric for Izmir took values be- Module sizes generally have medium and higher val- tween 5.39% and 27.47%. It has been observed that ues. Figure 3 shows the module view of the optimum individuals with the highest thermal comfort perfor- solution for Ankara. mance are generally south-oriented with a value of 0.3 WWR. Although insulation thickness is high in Figure 3 these individuals and the glazing type is triple-loE, Isometric module the increase in heating and cooling loads caused an view of the increase in EUI values. The small size of the WWR optimum solution generally pulled the UDI to about 75%, causing a de- for Ankara crease in daylight performance. Having the cooling setpoint value of 27 was effective in the emergence of the individual with the worst PPD performance. The fact that the individual parameters with the best thermal performance and the individual parameters with the worst thermal performance generally take the same values, except for the glazing type and cool- ing setpoint, has revealed that the glazing type and cooling setpoint are very important for Izmir in terms of thermal comfort performance. Daylight performance metric took values be- Results for Izmir tween 47.10% and 91.06%. Depth parameter hav- For Izmir, as can be seen from Figure 4, which shows ing the lowest value and WWR being 0.7 were effec- all solutions and pareto-front solutions obtained as tive in obtaining the best UDI100-3000lx value. The a result of optimization, the EUI performance met- depth parameter of individuals with high daylight ric has values between 111.14 kWh/m2 and 63.68 performance generally takes the value of 6 m, while kWh/m2. It has been observed that individuals with the width parameter takes 5.2 m and above, and the the highest energy performance do not have dou- height parameter takes 3.6 m and above. The in- ble clear in the glazing types. It is seen that the crease in width and height in value has increased the low U-values and SHGC values of other glazing types glazing area on the façade, thus increasing the UDI are effective in this. In these individuals, the cool- values. ing setpoint value received the highest value of 27oC, When the pareto-front individuals are examined, while the heating setpoint received the lowest value it is seen that the east and west directions are not in- of 18oC. PPD performance metrics of solution indi- cluded in the parameter values for Izmir also. Apart viduals with high energy performance generally took from that, the double-clear glazing type is not in- values around 17% and remained below the recom- cluded in the parameter values of the pareto-front mended value. The fact that these individuals have solutions. Usually north-oriented module appears as high WWR values has ensured that their daylight per- the dominant parameter value. In Table 5, pareto- formance is also high. It has been observed that indi- front individuals with the highest ten fitness function

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 415 Figure 4 All individuals and pareto-front individuals obtained from the optimization results of Izmir

values among optimization results for Izmir accord- Figure 5 ing to equation 1 are listed. When looking at the pa- Isometric module rameter values of the individuals in this table, it is view of the seen that the depth of the module has low values, optimum solution and the width and height generally have high val- for Izmir ues. With the high WWR, while the daylight perfor- mance was improved, it was ensured that PPD and EUI got lower values with the other parameters bal- ancing them. One of the effective parameters here is the use of the triple loE glazing system with the low- est U and SHGC values. No shading option is not in- cluded among the parameter values of the most op- timal solutions for Izmir. This reveals the importance of shading for Izmir. Figure 5 shows the module view of the optimum solution for Izmir. Table 5 Top ten solutions with the highest fitness function value among pareto-front individuals for Izmir

416 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Comparison CONCLUSION When the optimization results are examined, it has In this study, a framework and method are presented been observed that the best performance metric for the designers to obtain the daylight, energy and values for Izmir are generally lower than those for thermal performance feedback of the buildings and Ankara. When the solution individuals in Pareto-front to support the decision-making process at the early were examined, it was seen that the most effective design stage. In general, in the study consisting of parameters in the UDI values of both were WWR and parametric modeling, simulation and optimization the height of the module. The most effective param- stages, an office module was analyzed for two differ- eters in the PPD value of both are the cooling setpoint ent climate zones. Width, depth and height of the and the glazing type. module, orientation, WWR, depth, count, angle and Although the change of the width of module pa- direction of shading, wall insulation thickness, glaz- rameter does not have effective results for the Ankara ing type, cooling and heating set points parameters climate, as this parameter value increases for Izmir, selected for optimization study to minimize the EUI the EUI and PPD values generally decreased and the and PPD while maximize UDI100-3000lx value. UDI100-3000lx performance metric value increased. According to the optimization results, the EUI While the reduction of the shading count provides a values for Ankara vary between 115.25 kWh/m2 and decrease in EUI values for Ankara in general, such an 58.28 kWh/m2, while the PPD performance metric is effect has not been observed in Izmir. While it is seen 21.56% and 5.41% and the UDI100-3000lx values vary that no shading is frequently chosen in the pareto- between 93.79% and 56.40%. These values for Izmir front solutions emerging for the Ankara climate, this are between 111.14 kWh/m2 and 63.68 kWh/m2, be- option is quite rare for Izmir. In this case, it can be said tween 5.39% and 27.47% and between 47.10% and that shading is more important for the Izmir climate. 91.06%, respectively. It has been observed that the Another parameter where the effect of climate best performance metric values for Izmir are gener- difference on parameters can be read is insulation ally lower than those for Ankara. It has been also thickness. While the upper two values are gener- observed that different parameters for both climates ally chosen for Ankara’s climate, the lower two op- have different effects on performance metrics, and at tions come to the fore for Izmir. In addition, while the the same time, some parameters have similar values increase in insulation thickness generally affects the and effects. thermal comfort performance negatively in Ankara, Depending on the decisions to be made in the such an effect has not been observed in Izmir. For early design stages, parameter values can be selected both climates triple-loE glazing type has been the according to design priorities. Also, with the results, most chosen parameter value. This is because, for it is seen that with the help of parametric modeling, both climates, a balance is achieved between perfor- simulations and optimization process, depending on mance metrics with low U-value and SHGC value de- the performance target, optimum solutions can be spite high WWR rates. Choosing this type of glazing determined for building design. In future studies, for and shading in Izmir has been effective in reducing getting more optimal and accurate building perfor- cooling loads by preventing direct sunlight and re- mance, the number of parameters and optimization ducing solar gain. In Ankara, on the other hand, the objectives can be increased. increase in insulation thickness and the effect of glaz- ing type are reflected in both heating and cooling REFERENCES loads. International Energy Agency, IEA 2019, 2019 Global Sta- tus Report for Buildings and Construction:Towards a zero-emissions, efficient and resilient buildings and construction sector, United Nations Environment

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 417 Programme between useful daylight illuminance and daylight ASHRAE, ASHRAE 2021, ANSI/ASHRAE Standard 55-2020 - glare probabilty?’, 1st Building Simulation and Opti- Thermal Environmental Conditions for Human Occu- mization Conference BSO12, Loughborough, UK pancy, ASHRAE Nabil, A and Mardaljevic, J 2005, ’Useful daylight illu- Bamdad, K, Cholette, ME, Guan, L and Bell, J 2017, ’Ant minance: a new paradigm for assessing daylight in colony algorithm for building energy optimisation buildings’, Lighting Research & Technology, 37(1), pp. problems and comparison with benchmark algo- 41-57 rithms’, Energy and Buildings, 154, pp. 404-414 Nazaroff, WW 2014, ’Illumination, lighting technologies, Chen, X, Wang, Q and Srebric, J 2015, ’Model predic- and indoor environmental quality’, Indoor air, 24(3), tive control for indoor thermal comfort and energy pp. 225-226 optimization using occupant feedback’, Energy and Ng, R, Patel, S and Ng, R 2013, ’Introduction: Experimen- Buildings, 102, pp. 357-369 tal Performances: Materials as Actors.’, in Ng, R and Cheong, KH, Teo, YH, Koh, JM, Acharya, UR and Man Yu, Patel, S (eds) 2013, Performative materials in architec- SC 2020, ’A simulation-aided approach in improv- ture and design, Intellect Books, pp. 1-18 ing thermal-visual comfort and power efficiency Pilechiha, P, Mahdavinejad, M, Pour Rahimian, F, in buildings’, Journal of Building Engineering, 27, p. Carnemolla, P and Seyedzadeh, S 2020, ’Multi- 100936 objective optimisation framework for designing of- Delgarm, N, Sajadi, B and Delgarm, S 2016, ’Multi- fice windows: quality of view, daylight and energy objective optimization of building energy perfor- efficiency’, Applied Energy, 261, p. 114356 mance and indoor thermal comfort: A new method Reinhart, C 2019, ’Daylight performance predictions’, in using artificial bee colony (ABC)’, Energy and Build- Hensen, JLM and Lamberts, R (eds) 2019, Build- ings, 131, pp. 42-53 ing performance simulation for design and operation, Méndez Echenagucia, T, Capozzoli, A, Cascone, Y and Routledge, pp. 221-269 Sassone, M 2015, ’The early design stage of a build- Shahbazi, Y, Heydari, M and Haghparast, F 2019, ’An ing envelope: Multi-objective search through heat- early-stage design optimization for office build- ing, cooling and lighting energy performance anal- ings’ façade providing high-energy performance ysis’, Applied Energy, 154, pp. 577-591 and daylight’, Indoor and Built Environment, 28(10), Eltaweel, A and Su, Y 2017, ’Parametric design and day- pp. 1350-1367 lighting: A literature review’, Renewable and Sustain- Uribe, D, Bustamante, W and Vera, S 2017, ’Seasonal op- able Energy Reviews, 73, pp. 1086-1103 timization of a fixed exterior complex fenestration Fang, Y and Cho, S 2019, ’Design optimization of build- system considering visual comfort and energy per- ing geometry and fenestration for daylighting and formance criteria’, Energy Procedia, 132, pp. 490-495 energy performance’, Solar Energy, 191, pp. 7-18 Wong, L 2017, ’A review of daylighting design and imple- González, J and Fiorito, F 2015, ’Daylight Design of Office mentation in buildings’, Renewable and Sustainable Buildings: Optimisation of External Solar Shadings Energy Reviews, 74, pp. 959-968 by Using Combined Simulation Methods’, Buildings, Zhai, Y, Wang, Y, Huang, Y and Meng, X 2019, ’A multi- 5(2), pp. 560-580 objective optimization methodology for window Konis, K, Gamas, A and Kensek, K 2016, ’Passive perfor- design considering energy consumption, thermal mance and building form: An optimization frame- environment and visual performance’, Renewable work for early-stage design support’, Solar Energy, Energy, 134, pp. 1190-1199 125, pp. 161-179 Zhao, J and Du, Y 2020, ’Multi-objective optimization de- Lavin, C and Fiorito, F 2017, ’Optimization of an Exter- sign for windows and shading configuration consid- nal Perforated Screen for Improved Daylighting and ering energy consumption and thermal comfort: A Thermal Performance of an Office Space’, Procedia case study for office building in different climatic re- Engineering, 180, pp. 571-581 gions of China’, Solar Energy, 206, pp. 997-1017 Manzan, M and Padovan, R 2015, ’Multi-criteria energy and daylighting optimization for an office with fixed and moveable shading devices’, Advances in Building Energy Research, 9(2), pp. 238-252 Mardaljevic, J, Andersen, M, Roy, N and Christoffersen, J 2012 ’Daylighting metrics: is there a relation

418 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Static Shading Optimization for Glare Control and Daylight

Francesco De Luca1, Abel Sepúlveda2, Toivo Varjas3 1,2,3Tallinn University of Technology 1,2,3{francesco.deluca|absepu|toivo.varjas}@taltech.ee

Daylight and solar access influence positively building occupants` wellbeing and students' learning performance. However, an excess of sunlight can harm the visual comfort of occupants through disturbing glare effects. This study investigated, through multi-objective optimization, the potential of static shading devices to reduce glare and to guarantee daylight provision in a university building. The results showed that the reduction of disturbing glare was up to more than twice the reduced daylight, which nevertheless, was provided in adequate levels. View out and energy performance were also analyzed. Detailed results of optimal shading types and classrooms layout indications are presented.

Keywords: Daylight, Visual comfort, Shading, Multi-objective optimization

INTRODUCTION sunlight on the desk and monitor contrast found Daylight and solar access are important factors for that the space was considered intolerably uncom- building occupants‘ comfort and wellness. Stud- fortable for many occupied hours (Jakubiec and Rein- ies showed that sunlight and the perception of day- hart 2016). Field studies conducted in office spaces night alternation improve health facilitating the cor- underlined the importance of the occupant distance rect entrainment of humans’ circadian rhythm (Lock- from the windows and view direction to control glare ley 2009). Research works proved that daylight (Kong et al. 2018). The studies about the use of shad- increases the workers‘ satisfaction and productiv- ing to eliminate visual discomfort mostly focused ity (Andersen et al. 2012) and improves the stu- on operable internal shades or blinds investigating dents’ learning performance in educational buildings materials, geometrical configurations and controls (Heschong 2002). Additionally, studies showed that (Chan and Tzempelikos 2013, Koo et al. 2010). through a correct design natural light in buildings re- There is yet a limited focus in analyzing opti- duce energy consumption through consistent cut of mal configurations of building massing and enve- electric lighting energy and reduction of heating en- lope to admit daylight (De Luca 2017), and types ergy also at Northern latitudes (De Luca et al. 2016, of external shading devices to control glare. If on Voll et al. 2016) without significant cooling energy the one hand the static shading glare reduction is increase (De Luca et al. 2018). smaller compared to the internal operable ones, on On the other hand, an excess of daylight and the other hand they present the advantages of a con- direct sunlight can significantly decrease the build- stant though reduced view out and higher electric ing occupants‘ visual comfort due to glare effects. lighting energy reductions because not dependent A study conducted in a students’ studio open space on occupants’ operation (Reinhart 2004). Static shad- taking into account sun in the field of view, direct ing proved to be an efficient and economic strategy

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 419 to control daylight distribution (Hans and Voss 2011). Using computer simulations, glare is assessed at the Being visual comfort and daylight potentially con- height of the eyes and in the view direction of the oc- flicting performances, they need to be analyzed si- cupant in a seating position at approximately 1.2 m. multaneously in the early design stages to find op- Input of the simulations are the interior surfaces, the timal and trade-off shading solutions. glazed areas and the external obstructions, the mate- rials reflectance and the visible transmittance values. Glare analysis Glare simulations are performed for a single moment Glare can be caused by an excessive luminous inten- (point-in-time) using a specific sky condition or a cli- sity and by the contrast between the different lumi- mate based sky, or for the entire year. The point-in- nance level of the light sources and background. The time simulation output is the fisheye view present- level of glare is affected also by the location of the ing the luminance values (cd/m2) and the glare as- main light source inside the field of view of the ob- sessment through the metric used (Figure 1). Annual server. Two glare levels can be distinguished: dis- glare simulations require the additional inputs of the comfort glare which causes eye strain and disability statistical weather data and occupancy hours. The glare which prevents a person to see the surrounding output is a chart showing the visual discomfort lev- environment (Reinhart 2018). Discomfort glare met- els for each hour of the year. rics are based on the glare index which expresses the contrast between a glare source characterized by its Figure 1 size, luminance and position inside the field of view, Point-in-time glare and the average luminance of the background. Ac- assessments in one cording to the glare index, larger and or brighter light classroom of the sources located in the center of the field of view in- study using a clear crease glare, whereas a brighter background attenu- sky with sun at 11 ates the glare effect (Jakubiec and Reinhart 2012). a.m. on April 4 – Several glare metrics have been developed, such DGP 32 % (left) and as the Daylight Glare Index (DGI) (Hopkinson 1972) at 11 a.m. on and the Daylight Glare Probability (DGP) (Wienold January 4 – DGP 43 and Christoffersen 2006). DGI advanced the initial % (right). metrics developed for the small sources of electric Daylight analysis lighting taking into account large glare sources such Daylight availability metrics date back to the end of as daylight through windows. DGP, which is one the 19th century. One of the most used metrics is the most recent metrics, adds the measure of the scene Daylight Factor (DF) which predicts interior natural brightness (saturation effect) as possible visual dis- light levels as a ratio of exterior illuminance. DF is a comfort source in addition to the contrast used by simple metric to use through formulas and computer the previous indices (Reinhart 2018). The DGP in- simulations. Its reliability is limited because it consid- dex is based on four levels of probability that a per- ers only the geometrical characteristics of the room son would experience visual discomfort in the spe- surfaces, glazed areas and external obstructions and cific setting, i.e., imperceptible (DGP ≤ 34 %), percep- the materials’ reflectance and glazing transparency. tible (34 % < DGP ≤ 38 %), disturbing (38 % < DGP ≤ Climate based daylight metrics such as Daylight Au- 45 %) and intolerable glare (DGP > 45 %). tonomy and Useful Daylight Illuminance have been Glare analysis is particularly important in edu- developed to accurately predict through computer cational and work premises because the occupants simulations the annual percentage of time during cannot change seating position and view direction. which an interior point meets the daylight thresh-

420 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 old, using also the building orientation and statistical the classrooms’ layout. View out and energy perfor- weather data (Reinhart et al. 2006, Nabil and Mardal- mance using the shadings are also presented. The jevic 2006). novelty of the study lies in the assessment of glare, Spatial Daylight Autonomy (sDA) is a recently to control together with daylight through static shad- developed annual daylight metric introduced in the ing, for the entire room area and multiple views in- method LM-83-12 and adopted by leading interna- stead than for a single view as in existing literature. tional standards such as LEED (Illuminating Engineer- ing Society 2013). sDA assesses annual daylight METHODS availability as the percentage of occupied floor area The study was conducted through three- where the illuminance threshold of 300 lux is reached dimensional modeling of the classrooms and build- for at least 50 % of the time (sDA300/50%) between ings, measurement of the optical properties of the 8 a.m. and 6 p.m. regardless of the function of interior materials, parametric modeling of the shad- the building. LM-83-12 requires minimum 55 % of ing devices, daylight modeling, multi-objective opti- sDA300/50% to consider a room acceptably daylit. mization, view out and energy modeling. The build- Daylight is simulated on a horizontal plane (sim- ing used in the study is the Academy of Architecture ulation grid) located at the desk height of approxi- and Urban Studies of TalTech located in Tallinn, Esto- mately 0.75 m using sensor points. Other inputs of nia (Lat. 59°26’N Lon. 24°45’E). the annual daylight analysis are the room surfaces of floors, walls, ceiling, window glass and frames, Building and classrooms model 2 main furnishing and the external obstructions, their The classrooms 46 of 45.9 m southerly oriented 2 reflectance and visible transmittance values, the il- and 41 of 52 m easterly oriented, which have the luminance threshold (lux), the occupancy hours and same windows and tables layout, were selected to the annual weather data. analyze glare and daylight for different orientations This study investigated through multi-objective and opposite buildings’ height and distance (Figure optimization the potential of different types of ex- 2). The classrooms were located at a height of 13.25 ternal static shading devices to improve visual com- m. Detailed three-dimensional models were realized fort while guaranteeing adequate daylight provision in Rhinoceros (McNeel 2021) including tables, cup- in two classrooms of Tallinn University of Technol- boards, cork wall boards and the main appliances as ogy (TalTech). The study was conducted using two projector and whiteboard. The relevant surrounding simulation planes to provide useful information for buildings were also modeled (Figure 2).

Figure 2 The Academy building with location of the classrooms used in the study and the surrounding buildings. Grayscale rendering of classroom 46.

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 421 Figure 3 The shading device types overhang with vertical fins, horizontal louver, vertical louver and hexagonal pattern (from left to right).

Shading parametric models Figure 4 For the study four different shading device types The reflectance were modeled: overhang with vertical fins; horizon- measurement tal louver; vertical louver; and hexagonal pattern (Fig- equipment 3nh - ure 3). For each type an algorithm was realized in Spectrophotometer Grasshopper (Rutten 2021) to generate the shading YS3060. using different parameters. The overhang with vertical fins shading only pa- rameter was the depth, variable from 0 m to 2 m. The parameters of the horizontal louver were the slats spacing starting from 0.1 m to the full window height Table 1 (no slats), the depth from 0 m to 0.3 m and the rota- The interior surfaces tion with hinge on the top edge of the slat from 0° reflectance (R) and (open) to 89° downward (closed). The parameters of windows visible the vertical louver were the slats distance from 0.1 m transmittance (VT) to the window width (no slats), the depth from 0 m values. to 0.3 m and the rotation with hinge on the internal edge from -89° (closed cw) through 0° (fully open) to +89° (closed ccw). The parameters for the hexago- Material characterization nal pattern shading were the radius of the aperture The optical properties of the interior surfaces and from 0.1 m to the window width (no shading), and windows were measured to realize a reliable daylight the depth from 0 m to 0.3 m. model and to obtain accurate occupant visual com- The shadings were located on the exterior of the fort and daylight availability predictions. The light re- window frame inside the window recess of 0.22 m, flectance (R) of the opaque surfaces was obtained us- except the overhang with fins which was attached to ing the calibrated equipment 3nh - Spectrophotome- the building façade. The overhang had two fins for ter YS3060 (Figure 4). The visible light transmittance the south facing room and only the one toward south (VT) of the two-pane glazing was calculated as the for the east facing room as recommended by rules- ratio of the vertical illuminance measured with the of-thumb. The windows were 2.28 m and 2.35 m (w), window closed and open, which constitutes a simple and 1.71 m and 1.74 m (h) in size in classrooms 46 and method to approximate VT (Reinhart 2018). The VT 41, respectively. Custom components were created measurements were conducted using the calibrated and used in the parametric model for the glare and Luxmeter MSC-15. The R and VT values are presented daylight simulations, the multi-objective optimiza- in Table 1. tions, the view out and the energy assessments.

422 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Daylight model in room 41, for grid offsets 0.5 m and 1 m, respec- The daylight model was realized using the software tively. ClimateStudio performed annual hourly glare ClimateStudio in Grasshopper (Solemma 2021). Cli- simulation for every view. The output was the spatial mateStudio is based on the validated daylight simu- discomfort glare (sDG) based on the metric DGP, i.e., lation software Radiance (Ward 1994) and the novel the fraction of views which present a DGP level above path tracing technology which allows daylight sim- 0.38 (disturbing glare) for more than 5 % of the oc- ulations hundreds or thousands of time faster than cupied hours (Figure 5). The 5 % exceedance time in previous Radiance-based software without compro- glare assessments is defined in the European daylight mising the accuracy. The daylight model presented standard EN 17037 (CEN 2018). two sections, one for glare and the other for daylight For daylight availability simulations the grid was availability simulations. Both used as inputs the class- located at 0.75 m from the floor, and was consti- rooms‘ three-dimensional models and the materials tuted by single points with the normal facing up- definition realized using the measured interior sur- ward. Among the several daylight metrics available faces’ optical properties. For the surrounding build- in ClimateStudio, sDA300/50% was used as intro- ings and ground were used standard reflectance val- duced by LM-83-12 (Figure 5). Although the current ues, i.e. 35 % and 20 %, respectively. The material Estonian daylight standard requires daylight assess- used for the shading was a metal with reflectance ments through DF, sDA was used because existing lit- 49.8 %. Additional inputs were the statistical annual erature proved that DF is not reliable to predict day- weather data of Tallinn in epw format, the occupancy light levels in Estonia (Sepúlveda et al. 2020). The schedule from 8 a.m. to 6 p.m. during weekdays, main Radiance parameters used in the simulations which are the hours during which lessons take place were: - ab 6 - lw 0.01 -ad 1. The path tracing param- in the classrooms and as well those recommended by eters were: sample rays per sensor per pass 64 and the Estonian building regulations for energy and day- max number of passes 100. light analysis of educational buildings. The simulations were conducted using grids of Multi-objective optimization points spaced 0.5 m, with two offsets from the walls Multi-objective optimization was used to investigate and windows, approx. 0.5 m and 1 m, to analyze the optimal trade-off shading configurations to reduce performance of different classroom used areas. For glare while guaranteeing daylight availability. The glare simulations the grid was located at 1.2 m and software used was Opossum, a model-based opti- presented 8 view directions for each grid point, for a mization tool for Grasshopper (Wortmann 2017). The total of 1320 and 768 in room 46 and 1496 and 832 Figure 5 The analysis grid for glare simulations with views (left) and for daylight availability (right) in classroom 46 (0.5 m from walls), as generated by ClimateStudio with results visualization.

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 423 algorithm used was RBFMOpt (Radial Basis Multi- RESULTS Objective Optimization). The objectives were the The results of the study are presented in three sec- minimization of sDG and the maximization of sDA tions. In the first the optimal types of shading de- through minimization of the result of the subtraction vices to reduce glare and provide adequate daylight of the simulation result from 1 (1=100 % of the sen- are presented and the performances are discussed. In sor points receiving DA300/50%). The parameters of the second the influence of the shadings on the view the shading devices were used as the variables of the out is presented. In the third the energy consump- optimization process. tion variations using the shadings are analyzed.

Energy and view out models Optimal shading devices Energy simulations were performed without and To find the optimal trade-offs allowed by the different with the Pareto-optimal shading types with the best types of shading multi-objective optimization was performances using the energy tools of ClimateStu- used for each shading type in the two classrooms 46 dio based on the software EnergyPlus (NREL 2019). and 41 using the two simulation grids as presented in The simulations parameters are presented in Table 2. the section Methods. To compare the performance, the Pareto front solutions of each shading type were used because these represented the optimal trade- Table 2 offs of glare protection and daylight availability. The Energy simulation most performative shading types of each classroom parameters. EW = were those which permitted the largest sDG reduc- external walls, IW = The scope was to investigate the effect of the differ- tion and at the same time an sDA of minimum 55 %. internal walls, F = ent shading devices on the energy performance. The Taking into account classroom 46 the sDG and floor, C = ceiling, A occupancy schedule, the climatic data and the day- sDA in the actual condition (no shading) were 38.4 = adiabatic, W = light setpoint were the same used for the daylight % and 99.4 % respectively, using the simulation grid window, VT = model. The view out allowed by the shadings was at 0.5 m offset from the walls, and 41.7 % and 100 % visible analyzed through the Sky Exposure Factor (SEF) us- respectively, using the grid with 1 m offset from the transmittance, ing the plug-in Ladybug Tools (Sadeghipour and Pak walls. The results are presented in Figure 6. The two SHGC = solar heat 2013). The SEF metric calculates the visible portion most performative shading types were the horizontal gain coefficient, Ut of the sky from points of surfaces as a ratio of the sky louver and the overhang with vertical fins. Consider- (W/m2K). hemisphere visible without any obstruction. ing the simulation grid with the maximum extension,

Figure 6 Plots of the Pareto front trade-off solutions of the shading devices of classroom 46 for analysis grid with distance from walls 0.5 m (left) and 1 m (right).

424 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 the best shading was a horizontal louver (A) which re- tively, using the analysis grid at 0.5 m from the walls, duced the sDG to 12.7 % while guaranteed an sDA of and 21.1 % and 91.4 % respectively, using the grid at 62.4 %. The geometrical parameters of the slats were 1 m from the walls. The most performative shading 0.13 m spacing, 0.14 m depth and 4.8° rotation. The types were the overhang with fin, the hexagonal pat- second best performance was recorded also for the tern and the vertical louver. The results are presented type horizontal louver (B) characterized by the slats in Figure 7. Considering the grid closer to the walls, parameters of 0.14 m spacing, 0.14 m depth and 3.5° the first and the third best shading types (G-I) were rotation. This shading allowed to reduce sDG to 13.9 overhang with fin which allowed to reduce the sDG % and at the same time to provide an sDA of 67.3 %. to 6.6 % and 6.8 % while guaranteed an sDA of 55.6 The third most performative shading was of the type % and 56.7 %, respectively. Their depths were 0.60 m overhang with fins (C) which allowed to reduce sDG and 0.61 m, respectively. The second best shading (H) to 14.4 % while guaranteed an sDA of 58.8 %. The was of the type hexagonal pattern with an aperture depth of the shading was 1 m. radius of 0.46 m and a depth of 0.3 m. It allowed to Considering the simulation grid with the largest reduce sDG to 6.8 % as the third best shading of type distance from walls and windows, the most perfor- overhang with fin but allowed slightly more daylight mative shading type was the overhang with vertical provision with an sDA of 58.3 %. fins with two configurations. The first (D) had a depth Considering the grid at 1 m from the walls, the of 1.19 m and permitted to reduce glare in the class- three most performative shading types were all over- room so that only 9 % of all the views recorded a dis- hang with fin (J-K-L). They permitted to reduce the turbing glare (sDG) and at the same time guaranteed sDG to 2.9 %, 3.5 % and 3.9 %, respectively, while they a daylight provision of sDA 55.2 %. The second (E) allowed an sDA of 57.7 %, 58.6 % and 61.5 %, respec- had a depth of 1.24 m and allowed a reduction of sDG tively. Their depth were 0.75 m, 0.71 m and 0.67 m, to 9.5 % and an sDA of 57.3 %. The third most perfor- respectively. For classroom 41 and considering the mative shading was a horizontal louver (F). It allowed small grid, two other shadings had performance sim- a reduction of sDG to 10.7 % and at the same time ilar to the third best. A hexagonal pattern shading guaranteed an sDA of 66.7 % using the slats geomet- (M) with 0.1 m of aperture radius and 0.07 m of depth rical parameters of 0.25 m spacing, 0.21 m depth and reduced sDG to 4 % and allowed an sDA of 62.5 %. A 13.9° rotation angle. vertical louver (N) with slats spacing of 0.9 m, a depth Taking into account classroom 41 the sDG and of 0.24 m and a rotation of 58.7° CCW reduced the sDA without shading were 18.8 % and 85.0 % respec- sDG to 4.1 % while allowed a sDA of 55.8 %.

Figure 7 Plots of the Pareto front trade-off solutions of the shading devices of classroom 41 for analysis grid with distance from walls 0.5 m (left) and 1 m (right).

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 425 View out analysis Table 3 The view out analysis as well as the energy analy- Reduction of sDG sis were used in the study to evaluate the influence and sDA obtained of the shading devices on other aspects of occupant through the comfort and building performances. For the view shadings analyzed out analysis SEF was calculated for the same sensor (A-N) for the two Table 3 summarizes the shading devices perfor- points as for the glare simulations (Figure 8), using classrooms using mance for glare reduction and consequent decrease both analysis grids. The average SEFs of the class- the two analysis of daylight which was anyway adequate according rooms without shading and with the 14 most perfor- grids. to the most advanced standards. In classroom 46, mative shadings analyzed were compared (Table 4). due to its southerly orientation, the most performa- tive shadings were the horizontal louver (A), which Table 4 reduced annual glare by 66.9 % and daylight by only Average Sky 37.2 % when the large grid was used, and the over- Exposure Factor hang with fins (D), which reduced glare by 78.4 % and (SEF) values without daylight by a much lesser 44.8 % when the small grid shading (ns) and In classroom 46 using the larger analysis grid (0.5) the was used. In classroom 41, due to its easterly orienta- with the shadings three most performative shadings (A-B-C) reduced tion, the most performative shadings were the over- analyzed (A-N) for the view to the sky by values between 38.5 % and hang with fin and the vertical louver, with close per- the two classrooms 44.8 %. Similar reductions, between 40 % and 48.3 formances. However, the first was the most perfor- using the two %, were recorded when the smaller analysis grid mative using both the large and the small grid (G-J) analysis grids. was used (1) and with the related most performative reducing glare by 64.9 % and by 86.3 % and daylight shadings (D-E-F). In classroom 41 the reduction of SEF by a much lesser 34.6 % and 36.9 %, respectively. was between 24.1 % and 28.7 % when the large grid Thus evidence showed that using static shading was used (0.5) with the related optimal shadings (G- the reduction of visual discomfort outperformed the H-I), and was between 23.3 % and 31.6 % when the decrease of daylight availability. Results also showed smaller grid was used (1) and the related most perfor- that the analysis grid, representing a possible tables’ mative shadings (J-K-L-M-N). The smaller SEF reduc- layout, further from the walls presented higher glare tion in classroom 41 was due to the possibility to use and daylight without shading, being the further sen- smaller and more open shadings because the east- sors closer to the windows, but also recorded the erly orientation caused smaller visual discomfort. larger glare reduction using the shadings. Figure 8 Sky Exposure Factor analysis in classroom 41 without shading (left) and with the shading overhang with fin G (right).

426 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Energy analysis CONCLUSION The energy simulations were performed for the main The research investigated the potential of static shad- types of consumption which can be influenced by the ing to reduce visual discomfort and to guarantee external static shading, i.e., heating, cooling and elec- adequate natural illumination in two classrooms of tric lighting (Figure 9). The results showed that the TalTech Academy of Architecture and Urban Studies use of the shading caused a small increase of total with different orientations. Parametric variations of energy consumption in comparison with the much four types of shading (overhang with fins, horizontal larger visual comfort increase. In both classrooms the and vertical louver, hexagonal pattern) were used to 14 most performative static shading types analyzed minimize disturbing glare and maximize daylight au- (A-N) increased the heating energy and decreased tonomy through multi-objective optimization. View the cooling energy due to reduced solar gains, and in- out and energy analyses were used to further eval- creased the electric lighting consumption due to re- uate the optimal shadings. Two analysis grids with duced daylight. different offsets from the walls were used to obtain In classroom 46 the average increase of total en- useful information for the classroom tables’ layout. ergy was 8 %. The average increase of heating en- The results showed that the shadings allowed a ergy was 9.8 %, the average decrease of cooling en- reduction of disturbing glare by up to 78.4 % and 86.3 ergy was 70.5 %, and the average increase of electric % and at the same time reduced daylight availabil- lighting was 8.3 %. In classroom 41 the average in- ity of 44.8 % and 36.9 % in the southerly and east- crease of total energy was 2.5 %. The average heating erly oriented classrooms, respectively. Nevertheless, and electric lighting energy increase was 2.6 % and adequate daylight levels were provided. Thus the 4.9 %, respectively and the average cooling energy study proved the potential of static shading in im- decrease was 25.5 %. Being heating the largest con- proving visual comfort while guaranteeing daylight sumption, the difference of energy increase between provision. However, the most performative shadings the two classrooms was due to the smaller solar gains reduced the average view out by 48.3 % and 26.6 %, of classroom 41, due to its easterly orientation. and increased the energy consumption by 7.9 % and 2.3 % in the two classrooms, respectively. Figure 9 The outcomes also showed that using the Energy simulation smaller grid, which represented a compact tables’ results for the layout, the shadings performed better in reducing classrooms without glare. Moreover, the shadings horizontal louver and shading (ns) and overhang with fins were the most performative for with the shadings the southerly oriented classroom and the latter also analyzed (A-N). for the easterly oriented classroom together with the vertical louver. Future work will analyze classes with different sizes and orientations and will use multi-objective optimization of pairs of performances between glare, daylight, view out and energy. The resulting sets of data will be used to develop prediction formulas to be applied as the basis of a coupled method to in- form design decisions about optimal shading types and sizes for specific room sizes, orientation and lay- out, during the early design phase.

Simulation, prediction and evaluation in design - Volume 2 - eCAADe 39 | 427 ACKNOWLEDGEMENTS De Luca, F, Voll, H and Thalfeldt, M 2018, ’Comparison The research was supported by the grants ZEBE 2014- of Static and Dynamic Shading Systems for Office 2020.4.01.15-0016 of ERDF, and Finest Twins 856602. Buildings Energy Consumption and Cooling Load Assessment’, Management of Environmental Quality: An International Journal, 29(5), pp. 978-998 REFERENCES McNeel, R 2021, Rhinoceros, https://www.rhino3d.com Andersen, M, Mardaljevic, J and Lockley, SW 2012, ’A Nabil, A and Mardaljevic, J 2006, ’Useful Daylight Illumi- framework for predicting the non-visual effects of nances: A Replacement for Daylight Factors’, Energy daylight – Part I: photobiology-based model’, Light- and Buildings, 38(7), pp. 905-913 ing Research & Technology, 44(1), pp. 37-53 NREL, National Renewable Energy Laboratory 2021, En- CEN, European Committee for Standardization 2018, EN ergyPlus, https://energyplus.net 17037:2018: Daylight in Buildings, CEN, Brussels Reinhart, CF 2004, ’Lightswitch-2002: a model for man- Chan, YC and Tzempelikos, A 2013, ’Efficient venetian ual and automated control of electric lighting and blind control strategies considering daylight utiliza- blinds’, Solar Energy, 77(1), pp. 15-28 tion and glare protection’, Sol. En., 98, p. 241–254 Reinhart, CF 2018, Daylighting Handbook II. Daylight Hans, O and Voss, K 2011 ’OPTIShade potentials in opti- Simulations. Dynamic Facades, Building Technology mization of passive climate protection for buildings’, Press, Cambridge (MA), USA In Proc. Climate and Construction, Karlsruhe, pp. 1-10 Reinhart, CF, Mardaljevic, J and Rogers, Z 2006, ’Dynamic Heschong, L, Wright, RL and Okura, S 2002, ’Daylighting Daylight Performance Metrics for Sustainable Build- Impacts on Human Performance in School’, J. of the ing Design’, Leukos, 3(1), pp. 7-31 Illuminating Engineering Society, 31(2), pp. 101-114 Sadeghipour Roudsari, M and Pak, M 2013 ’Ladybug: a Hopkinson, R 1972, ’Glare from daylighting in buildings’, parametric environmental plugin for grasshopper to Applied Ergonomics, 3, pp. 206-215 help designers create an environmentally-conscious Jakubiec, JA and Reinhart, CF 2012, ’The ‘adaptive zone’ design’, In Proc. of BS 2013, Chambery, p. 3128–3135 – A concept for assessing discomfort glare through- Rutten, D 2021, Grasshopper, Algorithmic modeling for out daylit spaces’, Light. Res. & Tech., 44, p. 149–170 Rhino, https://www.grasshopper3d.com Jakubiec, JA and Reinhart, CF 2016, ’A Concept for Sepúlveda, A, De Luca, F, Thalfeldt, M and Kurnitski, J Predicting Occupants’ Long-Term Visual Comfort 2020, ’Analyzing the fulfillment of daylight and over- within Daylit Spaces’, Leukos, 12(4), pp. 185-202 heating requirements in residential and office build- Kong, Z, Utzinger, MD, Freihoefer, K and Steege, T 2018, ings in Estonia’, Building and Environment, 180, p. ’The impact of interior design on visual discomfort 107036 reduction: A field study integrating lighting envi- Illuminating Engineering Society, IES 2013, Approved ronments with POE survey’, Building and Environ- Method: IES Spatial Daylight Autonomy (sDA) and An- ment, 138, pp. 135-148 nual Sunlight Exposure (ASE), IES Koo, SO, Yeo, MS and Kim, KW 2010, ’Automated blind Solemma, LLC 2021, ClimateStudio, Solemma, control to maximize the benefits of daylight in build- https://www.solemma.com ings’, Building and Environment, 45, pp. 1508-1520 Voll, H, Thalfeldt, M, De Luca, F, Kurnitski, J and Olesk, Lockley, SW 2009, ’Circadian Rhythms: Influence of Light T 2016, ’Urban planning principles of nearly zero- in Humans’, in Squire, LR (eds) 2009, Encyclopedia of energy residential buildings in Estonia’, Manage- Neuroscience, Elsevier Academic Publishing, Amster- ment of Environmental Quality: An International Jour- dam, pp. 971-988 nal, 27(6), p. 634–648 De Luca, F 2017 ’From Envelope to Layout. Buildings Ward, GJ 1994 ’The RADIANCE lighting simulation and Massing and Layout Generation for Solar Access in rendering system’, In Proc. of the 21st SIGGRAPH Con- Urban Environments’, In Proc. of 35th eCAADe, Rome, ference, Orlando (FL), USA, p. 459–472 vol. 2, pp. 431-440 Wienold, J and Christoffersen, J 2006, ’Evaluation meth- De Luca, F, Voll, H and Thalfeldt, M 2016, ’Horizontal ods and development of a new glare prediction or Vertical? Windows’ layout selection for shading model for daylight environments with the use of devices optimization’, Management of Environmen- CCD cameras’, Energy and Buildings, 38, pp. 743-757 tal Quality: An International Journal, 27(6), pp. 623- Wortmann, T 2017, ’’Model-based Optimization for Ar- 633 chitectural Design: Optimizing Daylight and Glare in Grasshopper’, Tech. Arch. + Des., 1(2), pp. 176-185

428 | eCAADe 39 - Simulation, prediction and evaluation in design - Volume 2 Structure optimization and material-based design 430 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Material Units Uploading information into matter via stimuli and the challenges of determining feedback

Adam Blaney1 1Lancaster University [email protected]

Associative and generative design processes are capable of creating complex digital models, which can have their digital material properties (size, aesthetics, performance) infinitely adapted or radically transformed, relative to design demands, if they remain in their digital environments. Imagine, if physical materials and structures had these abilities, where design updates could be uploaded into a structure's physical material makeup at molecular resolutions. This could begin to enable a physical structure's matter to be reprogrammed so they can adapt across their length scales with high sensitivities and multi-material properties. To leverage these abilities, novel design and fabrication processes need to be developed, which enable interrelationships between design parameters, assembly mechanisms and material properties. This paper presents key findings and implications of two final prototypes, from a series, which developed a design and fabrication approach termed tuneable environments that enables interrelationships and design information to be uploaded into matter at granular resolutions.

Keywords: tuneable environments, programmable matter

INTRODUCTION mechanical loading demands in this context induce Biological design and fabrication processes main- stimuli (proteins/hormones), which informs cellular tain continuous discourse and interrelationships be- activity to update the material properties of bone tween material properties, design demands and fab- (Hadjidakis and Androulakis, 2006). Importantly, rication processes. As a result, structures can contin- the bone remodelling process demonstrates the role ually tune and adapt their global (shape) and local stimuli can play in generating interrelationships and properties (material, composition, location, volumes) in programming a structure’s material properties. across time. This is particularly evident in bone re- Significantly, these interrelationships and material modelling, where consistent mechanical loading de- processes enable highly desirable physical abilities, mands (e.g. running vs sedentary) inform the prop- such as; scalability, multi-adaptive structures, self- erties of an individual’s bones (Frost, 1990). The healing and material cycles, all at high material res-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 431 olutions. These desirable abilities are yet to be fully BACKGROUND leveraged within artificial design and fabrication ap- Previous approaches to Programmable Matter have proaches as typically, form is still imposed upon mat- highlighted new potentials within design and fab- ter and linear modes of design, fabrication and con- rication processes. The various examples discussed sumption dominate. This raises the question; how within this section can all be seen as a form of pro- can desirable abilities present within biological pro- grammable matter based on how it has previously cesses/structures be instilled with artificial process- been defined; “a material that has the ability to per- es/structures? form information processing much like digital elec- The motivation behind this research is to fun- tronics” (Tibbits, 2017, pp.14). By examining dif- damentally rethink how design and fabrication pro- ferent approaches to programming material units/- cesses interact with matter, so multi-adaptive struc- components, the role of scaffolds and the poten- tures can be developed at high material resolutions. tials of stimuli to programme matter becomes in- To do this, Research through Design (Frayling, 1993) creasingly apparent, especially when form is not im- is employed as a flexible approach (Gaver et al., 2004) posed upon individual material units and resolution to investigate; how updates from digital design tools increases. However, challenges of feedback between can be uploaded into matter to reprogram it at high design tools and material units become apparent material resolutions? when hardware is not directly embedded into indi- The paper presents two final prototypes from vidual material units, which makes it difficult to deter- a series, which collectively developed a design and mine what material properties have been generated fabrication approach termed tuneable environments and what further design updates would need to be (Blaney et al, 2019). These prototypes investigate uploaded so desirable/intended materials properties how stimuli can be used to program matter at high can be generated. material resolutions. To do this, tuneable environ- Programming matter by embedding hardware ments modulate stimuli parameters (duration, lo- directly into individual material units has enabled cation, magnitude), which have interrelationships reconfigurable structures as units can have self- with design parameters and material properties. As movement, self-inspection and self-adhesion abili- a result, 2D patterns and 3D structures can have ties (Romanishin et al., 2013). The hardware enables multi-material properties updated because informa- digital information to be uploaded into each material tion from design tools can be ’uploaded’ into the unit and actions can be physically executed (Gilpin structures’ matter (Blaney, 2020). To contextualise and Rus, 2012). External stimuli do not play a role this approach and reflect on its implications and op- within these hardware-based approaches if the ma- portunities the paper is divided into four sections: terial units have self-moving abilities. However, this Firstly, previous approaches to programming matter embedded hardware comes at a cost, as material and the role stimuli plays within them are discussed. resolution (size) and scalability become limited. Re- Secondly, a final mineral accretion prototype is pre- moving this self-moving ability can somewhat minia- sented that investigates the implications of feedback. turise each material unit (Gilpin et al., 2010) but re- Thirdly, a final diffusion prototype is presented that sults in the requirement of external stimuli being sup- investigates the impacts of ’scaffolds’. Finally, key plied to the units so they can move around and in- principles and implications from the two prototypes teract with one another. The approach of program- are discussed. ming matter by embedding hardware highlights sev- eral main challenges of interest: 1) scalability due to cost, 2) material resolution, 3) only geometric recon- figurations are possible.

432 | eCAADe 39 - Structure optimization and material-based design - Volume 2 MIT’s self-assembly lab has developed an alter- limited. This issue could potentially be avoided if ma- native approach to programming matter through a terial processes are based on assembly. Importantly, series of practice-based investigations (Tibbits, 2016). Ayers reveals how stimuli can be utilised to create Instead, material units are pre-programmed by de- continued interactions with matter, so information signing the geometries and the material interfaces of from design tools can potentially be uploaded into individual material units, which still enables compu- matter i.e. using stimuli to iteratively programme tational processes such as self-error correction (Pa- matter. padopoulou et al., 2017). It also enables increased These approaches to programming matter high- material resolution and an increasingly scaleable pro- light the potential role stimuli can play in guiding/in- cess due to material unit cost and ease of fabrication. forming various material properties across a range of However, because the hardware has been removed, material platforms, especially when form/geometric external energy/stimuli (e.g. fluid agitation) must be properties are not imposed upon individual material supplied to the material units to enable autonomous- units. assembly (Papadopoulou et al., 2017). Tolley and Čejková et al further demonstrate the role of Lipson demonstrated that by controlling the energy stimuli plays in systems to do not impose form upon supplied to these types of material units the rate of material units, with initially simple decanol droplets assembly can be increased (Tolley and Lipson, 2010). dynamically changing shape based on evaporation This illustrates the beneficial role modulating param- of water, which results in complex multi-armed struc- eters of external stimuli can play in fabrication pro- tures being created at high resolutions (Cejkova et al., cesses and potentially, how it could be employed to 2016; Čejková et al., 2018). These complex structures programme matter. The removal of hardware raises are small in size, typically occurring on the nano- the challenge of enabling a discourse between de- scale to cm-scale, unlike the meter-length scales of sign tools and material units, which prevents new approaches that pre-design the geometries of mate- information from being uploaded from design tools rial units. This obstacle of scale reveals the role and and determining the properties generated. Being tradeoffs ‘scaffolds’ play to overcome issues of scale. able to ‘upload’ information into matter is desirable Smith et al demonstrate how product-scale as it would enable increasingly flexible material sys- structures can have highly complex colour and sur- tems that can accommodate unforeseen design de- face texture patterns generated when subjected to mands, which would avoid redundancies and signif- stimuli post-fabrication (Smith et al., 2019). In this icant material waste. This opens up two main chal- instance, the structures are 3D printed using growth lenges at this point: 1) how can stimuli begin to pro- mediums, which have ‘preprogrammed’ bacteria in- gramme matter? 2) How can a discourse between de- corporated into them. Here, the 3D printed struc- sign tools and material units be achieved based on tures act as a scaffold to guide material proper- modulating stimuli? ties generated. Additionally, within the developing Ayers demonstrates a continued discourse be- field of fungal architecture, scaffolds are created us- tween design representations and material units can ing growth substrates to grow large-scale mycelium- be achieved if design tools are used to modulate based structures (Goidea et al., 2020). Previously, parameters of stimuli (water pressure) that deform McGilivary and Gow demonstrate how stimuli can the global shape of metal components (Ayres, 2011). be used to alter material properties of mycelium, if These global shape changes are reversible up to the kept alive, as hyphal branching can be tuned by ex- elastic limit of the material but beyond this point, de- posing it to an electric field (McGillivray and Gow, formations are permanent and the ability to upload 1986). Importantly though, to leverage desirable design information into the material units becomes abilities, such as self-healing, the material must be

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 433 kept alive (Adamatzky et al., 2019) but this can re- anisms used to develop a closed-loop control system. sult in scaffolds being digested by the mycelium ma- The mineral accretion process is essentially the terials (Adamatzky et al., 2021). This highlights a electrolysis of seawater. Hilbertz demonstrated it balance between guiding material growth via stim- is a multi-material system that can aggregate vol- uli, the longevity of pre-designed scaffolds and the umes of either calcium carbonate or magnesium hy- need to continually supply the material units with re- droxide crystals upon large cathode scaffolds sub- sources to enable growth. The later challenge high- merged within seawater (Hilbertz, 1978). The pre- lights the need for these bio-material systems to be- dominant means of starting and stopping the ag- come integrated with existing biological ecologies gregation of matter in the mineral accretion process and material cycles so resources can be shared and is by switching on or off voltage supplied between replenish one another. an anode and cathode. Longer durations produce Surveying these various and diverse approaches increased volumes (Hilbertz, 1978). Voltage mag- to programming matter highlights the important role nitude informs the material type grown, with 1.23- stimuli can play in enabling interactions with mate- 1.5 volts predominantly producing calcium carbon- rial units at highly granular resolutions. However, ate and over 3 volts producing magnesium hydroxide the trade-off between increasing material resolu- (Goreau, 2012). Importantly, the mineral accretion tions and removing hardware results in the discourse process results in multiple chemical reactions being between design tools, the structure and its material generated, which can proliferate and affect material properties being lost. properties in a closed system if they are not offset. I have investigated how modulating parameters Goreau also highlights that as material volumes grow of stimuli can be used to programme matter at high they insulate against the electrical stimuli (Goreau, material resolutions through a series of iterative pro- 2012). Importantly, this insulating property acts as a totypes (Blaney, 2020). This approach to program- contrasting effect to the stimuli induced. Because of ming matter via stimuli is termed tuneable environ- this phenomenon, it provides a way to begin to deter- ments and enables information from design tools to mine associations between material volumes grown be uploaded into matter (Blaney et al, 2017, Blaney relative to dropping electrical current values, which et al, 2019). These previous prototypes represent can be monitored. It also highlights how materials open-loop control systems and are based on pre- can emit signals, so directly embedding sensors may determined associations between stimuli, design pa- not be needed to determine associations and estab- rameters and material properties. Two final proto- lish feedback. Hilbertz et al illustrated how this con- types are now presented with the first investigating trasting material effect can be used to monitor ma- how feedback between material properties and stim- terial growth by embedding an array of sensors (ex- uli could be determined, with the aim of develop- posed wires) at set distances perpendicular to the ing a closed-loop control system. The second proto- cathode scaffold (Hilbertz et al., 1977) i.e. effectively type presented investigates how material units can the sensors are embedded in the material units be- be interacted with without using constraining scaf- cause of the cathode scaffolds required. Addition- folds, with the aim of increasing material resolution ally, this set-up was carried out in the open ocean and flexibility. (i.e. open system) so conditions generated during the chemical reaction did not proliferate, for example, in- MINERAL ACCRETION PROTOTYPE creasing pH levels resulting in magnesium hydroxide To discuss the parameters of this final prototype a growth predominating. brief overview of the mineral accretion process is dis- The previous experiments Hilbertz et al carried cussed in order to highlight the processes and mech- out did not use design tools to alter stimuli param-

434 | eCAADe 39 - Structure optimization and material-based design - Volume 2 eters that inform material aggregation properties, et al, 2019). The challenge from these previous pro- which means the properties of the material volumes totypes is that they represent open-loop control sys- grown are not ‘programmed’. This provides an op- tems, which means there was no feedback between portunity to extend this mineral accretion research stimuli and the material properties generated. Be- to understand how stimuli can programme matter at cause there is no feedback, it is not possible to ro- molecular resolutions by uploading information from bustly determine if desirable material properties (e.g. design tools. To illustrate the development of this ap- volumes of material) have been generated relative to proach; firstly, the parameters of the mineral accre- design parameters. tion process are discussed. Secondly, previous pro- Figure 1 totypes and their challenges are discussed. Thirdly, 3D cathode scaffold a final mineral accretion prototype is presented that highlighting investigated the challenge of determining feedback volumes of material between the stimuli and material properties gener- that can be updated ated. simultaneously in Previous prototypes I have developed using the 3D space. mineral accretion process have informed how mat- ter can be programmed by pre-defining associa- tions between design parameters, stimuli parame- ters and material properties. Previous prototypes have demonstrated; 1) cathode scaffolds composed of physically separated elements enable localised stimuli to be induced, which informs local material properties (Blaney et al, 2017). 2) Inducing localised stimuli makes it possible to grow low-resolution 2D shapes with variable material compositions and vol- This final mineral accretion prototype investi- umes informed by analogue and digital design tools gates how material properties generated relative to (Blaney et al, 2019). 3) Material aggregation can oc- stimuli can be determined, which is a fundamental cur volumetrically, which makes it possible to tune challenge for developing functional material proper- material properties across the whole of a 3D structure ties, adaptive structures and reprogrammable matter at various times, unlike 3D printing processes that are at high resolutions. To achieve high material resolu- constrained to layer-by-layer build-ups (see figure 1). tions, sensors will not be directly embedded into the These past prototypes highlight an alternative individual material units. This final prototype will in- approach to programming matter, where material vestigate how feedback can be determined based on units are not pre-programmed by imposing form materials emitting signals that contrast with the stim- upon them. Instead, they demonstrate how mat- uli induced, so a closed-loop control system can be ter can be materialised and programmed on-demand created. For these reasons, this prototype incorpo- at molecular resolutions by modulating stimuli in- rates multiple sensors that are external to the cath- formed by design parameters. Additionally, tuneable ode scaffold (see figure 2). environments open up the potential for discourse To begin to determine relationships between with design tools to be maintained with a structure’s the intended stimuli supplied (voltage) and material material properties, so shapes and volumes of mate- growth volumes produced the conditions generated rial can have their properties updated based on aug- from the chemical reaction need to be maintained mentations to design tools (Blaney et al, 2017; Blaney within threshold ranges because a closed system will

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 435 be used. This is done so associations between in- liquid to dose into the solution. Solution tempera- duced stimuli, corresponding electrical current sen- ture is monitored as it has a direct effect on solution sor values and material growth volumes generated conductivity values but the temperature values will can be analysed reliably. In order to determine as- not induce a system action because the heating ele- sociations between stimuli and material properties, ment used self-regulates. Figure 3 highlights all the electrical current sensor values will be compared with system’s actions governed by an Arduino as a way to time-lapse photography to document material vol- potentially enable discourse between stimuli, sens- umes grown relative to sensor values. ing, design parameters and material properties. Previous prototypes were relatively complex as MineralAccretionSetupandSystemActions they were composed of multiple cathode wires, The initial solution for the prototype is made up of which makes them not suitable for determining ini- 39.5 litres of tap water with 1.5kg of marine salts tial relationships between stimuli and material prop- added, which creates initial electrical conductivity erties. For this reason, only two cathodes will be and pH values. To maintain the solutions initial con- used in this prototype (see Figure 2). To gener- ditions, a pH and electrical conductivity (EC) sensor ate a more robust data set, the prototype will be will be used (see Figure 2). The real-time pH and EC carried out 3 times with three different cathode sensor values inform dosing pump actions and which types. Firstly, carbon cathodes with a smooth sur-

Figure 2 Set-up diagram of a mineral accretion prototype with all sensors external to the material units

436 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 3 System actions and pre-defined relationship diagram

face texture (40x6.3mm). Secondly carbon cathodes types, considerably more growth occurred when us- with a rough surface texture (40x6.1mm). Finally, ing an aluminium cathode over the same duration, aluminium cathodes with a rough surface texture which is frustrating (see Figure 4). (40x1.2mm). A set voltage of 4.00 V will be supplied Figure 4 via a bench power supply for 24 hours, as this gener- Time lapse ated comparatively large volumes of material growth photography in previous prototypes (see Figure 4). highlight minimal material growth Mineral Accretion Results over a 24hr period. Time-lapse photography for all three studies high- Image C compares lighted minimal material growth and compared to Figure 5 plots the various sensor values and sys- material growth the carbon cathodes, the aluminium cathode gener- tem actions recorded during the aluminium cath- that occurred over a ated the most material growth. In previous proto- ode growth. Although solution temperature stayed 24hr duration from a previous prototype. Video: https://vimeo.com/380528892 Structure optimization and material-based design - Volume 2 - eCAADe 39 | 437 within an accepted range, the solution conductivity Figure 5 continually rose irrespective of temperature fluctua- Sensor values tions. The system actions attempted to offset the so- recorded over a lution conductivity by dosing reverse osmosis water. 24hr period using This condition appears to be significant as average an aluminium current, voltage and power values all decreased even cathode. Notably, with very minimal material growth occurring. Addi- as solution tionally, comparing the values with all three cathode temperature varied types highlighted that no direct linear-associations there was no could be created between stimuli and current sen- impact on solution sor values as no linear cause and effect relationship- conductivity as this s/trends can be identified continually rose. The minimal material growth and varying sen- The average current sor values generated from this final mineral accretion sensor values also prototype raised personal reflections on what consti- decreased over tutes a successful investigation. The aim was to un- time, which is derstand how feedback could be determined and de- unexpected based veloped within the system so matter could ultimately on previous be reprogrammed. Defining if the prototype is suc- research. These cessful based on only growing material volumes does results highlight not fully capture the system’s intricacies. This is be- non-linear cause using stimuli to interact with material units has behaviours being effectively given them agency and multiple degrees generated when of freedom. Because of this, the material units will sensors are not only do what they deem suitable in their fabrication directly embedded environments. In this case, not accumulating increas- into material units. ing volumes of matter irrespective of how long the This illustrates that stimulus is supplied for. The fact to no material has design been grown highlights that ideal conditions must be tools/processes created for growing materials even when ‘non-living’ based on materials are used. This reveals implications of ro- pre-defined bustness in this initial system and the ramifications form is not imposed upon matter. Because of these associations are of turbulence within the environment in order to cre- non-linear traits, digital design processes based on unsuitable/not ate ideal environments (Kelly, 1994). Alternatively, non-linear associations will be required to begin to reliable for Hilbertz et al demonstrate linear associations and ro- programme matter. Richards and Amos demonstrate programming bust growth can be achieved within these material how non-linear design tools can programme digi- matter at high systems if sensors are directly incorporated into the tal material units (voxels) to generate high-resolution resolutions. material units/scaffolds (Hilbertz et al., 1977). Com- and multi-material structures (Richards and Amos, paring with Hilbertz experiments, the results demon- 2016). Further work is needed to determine how digi- strate that monitoring relationships between stim- tal design processes based on non-linear associations uli and material properties using external sensors to can be incorporated and if they could more robustly monitor materials emitting signals leads to complex programme matter at high material resolutions. and non-linear associations being generated when These non-linear behaviours based on program- ming matter via stimuli highlights several opportu-

438 | eCAADe 39 - Structure optimization and material-based design - Volume 2 nities but the challenges appear to be shared issues loop control system. The aim is only to investigate within similar architectural research areas that do not the implications on material properties and material impose form upon matter. These will be outlined fur- resolutions when rigid scaffolds are removed. ther in the discussion section. The next prototype Figure 6 presented is used to investigate further how the ma- Ink diffusion terial resolution could be increased by removing the prototype set-up constraining cathode scaffolds and the implications of this.

DIFFUSION PROTOTYPE A personal area of interest with the mineral accretion prototypes is that the aggregated material is con- strained to the pre-designed and rigid scaffolds. This effectively results in low-resolution, pixelated struc- tures being created at the global scale of the system. To investigate how materials could be interacted with via stimuli with less constraining scaffolds a proto- type was developed based on 3D ink diffusion pat- terns.

Diffusion Porototype Setup: The diffusion prototype injects 4 different water- based ink colours into 3 different volumes (5L) of sup- Diffusions Results port liquid to investigate how ink diffusion patterns Figure 7 documents the various patterns generated can be interacted with via stimuli. Figure 6 highlights as inks are deposited into the three support liquids the system’s set-up and components. The three sup- and then agitated. Water resulted in the ink clouds port liquids are water, sugar syrup (1 parts sugar to homogenising and proliferating much faster than the 2 parts of water) and vegetable glycerine. The sup- sugar syrup, which is also highlighted by the flow port liquids vary in density and viscosity, with water meter values. Because the vegetable glycerine is so being the lowest and vegetable glycerine being the dense and viscous the inks dink not sink and could highest. The same volumes of ink are deposited at set not be manipulated by solution agitation (pumps), intervals using dosing pumps controlled via Arduino. which is also highlighted by the flowmeter values. The stimulus in the system is liquid agitation, which is Instead, the vegetable glycerine patterns had to be induced via two submersible pumps and the parame- generated by manually manipulating the inks. The ters (magnitude, duration, intervals) are defined by a photographs reveal delicate patterns suspended in user interface created using the software Processing. time (see Figure 7). The combined results from each This loosely enables information from design tools to support liquid highlight how different forms of scaf- be uploaded into matter again. The stimulus is moni- folds and the potential of using stimuli to guide, tune tored via a flow meter attached to one of the pumps, and suspend complex 3D diffusion patterns. which will highlight energy transferred to the mate- Interestingly, the comparative properties of the rial units (ink) within each support liquid but signifi- support liquids and patterns generated highlights cantly, the stimulus is not associated with any design the possibility of changing the material properties of demand and there is no attempt to develop a closed- scaffolds to inform their flexibility. For example, us-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 439 Figure 7 Ink diffusion comparing 3D forms generated in support solutions with varying viscosities and densities. Video link: https://vimeo.com/367431096

ing heat to reduce the viscosity of the support liquid These two prototypes along with previous inves- to rapidly generate 3D ink patterns, induce stimuli tigations highlight the potential of iteratively pro- to manipulate ink patterns with higher fidelity, and gramming matter at high resolutions using tuneable then temporarily freezing these patterns in time by environments. They also raise key considerations and increasing the support liquid’s viscosity by cooling it potentials for developing design and fabrication pro- back down. Based on these insights, it could be imag- cesses based on interrelationships instead of impos- ined how tuneable environments could be incorpo- ing form upon matter. rated with novel 3D printing developments, such as Rapid Liquid Printing (Hajash et al., 2017). Imagine, DISCUSSION materials deposited in support liquids to create pre- This research demonstrates how tuneable environ- cise structures and then the support liquid is used as ments can be used as an alternative approach to a tuneable environment to fine tune or update the programming matter. Tuneable environments con- 3D printed structure’s properties via stimuli by plac- tribute to programmable matter as it enables infor- ing them back in the tank. Developing precision and mation from design tools to be iteratively uploaded functional properties of this prototype is a long way into a structures’ material makeup. Meaning, struc- off and it was not the aim. However, it does begin to tures can be updated and radically transformed at illustrate how modulating stimuli can be applied to highly granular resolutions (molecular). This is pos- various material platforms and open up new ways to sible because tuneable environments do not impose interact with and begin to programme matter at high form upon matter but instead, creates design and material resolutions and increased flexibility. fabrication processes based on interrelationships,

440 | eCAADe 39 - Structure optimization and material-based design - Volume 2 which can leverage desirable abilities present within pended in time if the material properties (viscosities) biology, such as self-healing and potentially adap- of the scaffolds can be varied via stimuli. This opens tive structures if feedback is achieved. Furthermore, up the potential of incorporating tuneable environ- tuneable environments also appears to act as a uni- ments with 3D printing processes, such as Rapid Liq- versal language for interacting with matter, as high- uid Printing (Hajash et al., 2017), so 3D printed struc- lighted by the role stimuli can play in enabling dis- tures could be repaired or updated once fabricated course across a range of material platforms. if they are placed back in their tanks/fabrication en- Iterative prototyping has been used to investi- vironments. Imagine future medical prosthetics or gate subjective areas of interest as the process of pro- splints that are rapidly fabricated by submerging a gramming matter via stimuli develops. The two fi- patients’ limb into a tank, which can also be further nal prototypes in this series specifically investigate is- updated as the patient heals or grows. A main obsta- sues of feedback and the roles of scaffolds. Their re- cle to overcome in beginning to achieve these mate- sults highlight possible application areas along with rial abilities is the ability to upload and temporarily future implications of programming matter via stim- freeze assembly instructions into the deposited ma- uli, which appear to be emerging in other areas of re- terial, so structures can be taken out of the tank but search. also updated when subjected to stimuli again.

Implications of Feedback and Adaptive Ar- CONCLUSION chitecture This research highlights how matter can be inter- The mineral accretion prototype highlights issues of acted without imposing form up material units. This feedback when attempting to determine linear asso- is important as current linear modes of fabrication ciations. Instead, the results reveal non-linear associ- impose form upon matter. Resulting in significant ations occurring between stimuli and material prop- material waste, damaged ecosystems and green- erties generated when sensors are not directly em- house gas emissions, most notably witnessed within bedded into matter. Further investigations are re- the fashion (EDGExpo) and construction industries quired to understand how digital design tools can be (Fisher, 2020). Developing design and fabrication utilised to determine material properties generated processes capable of updating multiple properties of in non-linear systems. Achieving feedback within a structure at high resolution would address these is- the system would lead to high resolution physically sues as circular materials would be possible. This is adaptive and self-healing structures. But this then because matter could be infinitely reprogrammed, so opens up the implication of; what would constitute fluctuating and unforeseen design demands can be desirable adaptations, especially within complex de- accommodated. Future research aims to investigate sign issues, such as urban design? Adamaztky et al how matter can be reprogrammed at high resolu- also discuss a similar issue of feedback so functional tions so adaptive structures can be generated, which material properties can be developed within fungal are capable of sharing material resources and lead to architecture (Adamatzky et al., 2019). This highlights new material ecosystems. a shared and emerging issue when interacting with materials via stimuli. REFERENCES Applications and Developing Repro- Adamatzky, A, Ayres, P, Belotti, G and Wösten, H 2019, ’Fungal Architecture’, International Journal of Uncon- grammable Matter ventional Computing, 0(1), pp. 1-15 The diffusion prototype illustrates how support liq- Adamatzky, A, Gandia, A, Ayres, P, Wösten, H and Tege- uids can act as flexible scaffolds, so complex 3D laar, M 2021, ’Adaptive Fungal Architectures’, LINKs- patterns can be rapidly generated, tuned and sus- series, 5(10), pp. 66-77

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 441 Ayres, P 2011, ’Free-from Metal Inflation & the Persistent ing and Additive Manufacturing, 4(3), pp. 123-132 Model’, in Glynn, R and Sheil, B (eds) 2011, FABRI- Hilbertz, W 1978 ’Electrodeposition of Minerals in Sea- CATE:MakingDigitalArchitecture, UCL Press, London, water’, OCEANS, Washington DC, USA, pp. 699-706 UK, pp. 70-73 Hilbertz, W, Fletcher, D and Krausse, C 1977, ’Mineral Blaney, A 2020, Designing Parametric Matter: Explor- Accretion Technology: Applications for Architecture ing adaptive self-assembly through tuneable environ- and Aquaculture’, Industrialization Forum, 8(4-5), pp. ments, Ph.D. Thesis, Lancaster University 75-84 Blaney, A, Alexander, J, Dunn, N and Richards, D 2019 Kelly, K 1994, Out of control: The new biology of machines, ’Designing Parametric Matter’, IASDR: Design Revolu- social systems, and the economic world, Basic Books, tions. International Association of Societies of Design New York, USA Research Conference 2019, Manchester, UK McGillivray, A and Gow, N 1986, ’Applied electrical fields Blaney, A, Dunn, N, Alexander, J, Richards, D, Rennie, polarize the growth of mycelial fungi’, Microbiology, A.E.W and Anwar, J 2017, ’Directing self-assembly 13(9), pp. 2515-2525 to grow adaptive physical structures’, International Papadopoulou, A, Laucks, J and Tibbits, S 2017, ’From Journal of Rapid Manufacturing, 6(2-3), pp. 114-133 self-assemblies to evolutionary structures’, Architec- Cejkova, J, Hanczyc, M and Stepanek, F 2018, ’Multi- tural Design, 82(4), pp. 28-37 armed droplets as shape-changing protocells’, Arti- Richards, D and Amos, M 2016, ’Encoding Multi- ficial Life, 24(1), pp. 71-79 Materiality’, in Grogoriadis, K (eds) 2016, Mixed Cejkova, J, Stepanek, F and Hanczyc, M 2016, matters: a multi-material design compendium, Jovis, ’Evaporation-Induced Pattern Formation of Decanol Berlin, Germany, pp. 40-49 Droplets’, Langmuir, 32(19), pp. 480-485 Romanishin, J, Gilpin, K and Rus, D 2013 ’M-blocks: Fisher, K 2020, ’UK Statistics on Waste’, Deparment for En- momentum-driven, magnetic modular robots’, vironment, Food & Rural Affairs, 1(1), pp. 1-22 IEEE/RSJ International Conference on Intelligent Frayling, C 1993, ’Research in Art and Design’, Royal Col- Robots and Systems, IROS, Tokyo, Japan, pp. 4288- lege of Art Research Papers, 1(1), pp. 1-5 4295 Gaver, W, Boucher, A, Pennington, S and Walker, B 2004, Smith, R, Bader, C, Sharma, S, Kolb, D, Tang, T, Hosny, A, ’Cultural probes and the value of uncertainty’, Inter- Moser, F, Weaver, J, Voigt, C and Oxman, N 2019, ’Hy- actions, 11(5), pp. 53-56 brid Living Materials: Digital Design and Fabrication Gilpin, K, Knaian, A and Rus, D 2010 ’Robot pebbles: of 3D Multimaterial Structures with Programmable One centimeter modules for programmable matter Biohybrid Surfaces’, Advanced Functional Materials, through self-disassembly’, IEEE International Confer- 30(7), pp. 1-14 ence on Robotics and Automation, Waikoloa, Hawaii, Tibbits, S 2016, Self-Assembly Lab: Experiments in Pro- pp. 2485-2492 gramming Matter, Routledge, New York, USA Gilpin, K and Rus, D 2012 ’A distributed algorithm for 2D Tibbits, S (eds) 2017, Active Matter, MIT Press, Mas- shape duplication with smart pebble robots’, IEEE In- sachusetts, USA ternational Conference on Robotics and Automation, Tolley, M and Lispon, H 2010 ’Fluidic Manipulation Minnesota, USA, pp. 3285-3292 for Scalable Stochastic 3D Assembly of Modular Goidea, A, Dimitrios, F and Andréen, D 2020, ’Pulp Fac- Robots’, IEEE international conference on robotics and tion: 3d printed material assemblies through micro- automation, Anchorage, Alaska, United States, pp. bial biotransformation’, in Burry, J, Sabin, J, Shiel, B 2473-2478 and Skavara, M (eds) 2020, FABRICATE: Making Re- [1] https://edgexpo.com/fashion-industry-waste-statist silient Architecture, UCL Press, London, UK, pp. 42-49 ics/ Goreau, T 2012, ’Marine Electrolysis for Building Mate- rials and Environmental Restoration’, in Kleperis, J and Linkov, V (eds) 2012, Electrolysis, IntechOpen, Norderstedt, Germany, pp. 273-290 Hadjidakis, D. J and Androulakis, I. I 2006, ’Bone remod- eling’, Annals of the New York Academy of Sciences, 1092(1), pp. 385-396 Hajash, K, Sparrman, B, Guberan, C, Laucks, J and Tibbits, S 2017, ’Large-Scale Rapid Liquid Printing’, 3D Print-

442 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Fiber Compositions Development of wood and textile layered structures as a material strategy for sustainable design

Isak Foged1, Jacob Hilmer2 1,2The Royal Danish Academy - Architecture, Design, Conservation [email protected] [email protected]

This study examines composite compositions based on fiber-based materials. It focuses on organic textiles of Jute, Hemp, Wool, Flax, and Glass fiber as a synthetic textile, combined with the lightweight wood species Paulownia. By creating novel composites, the study aims to investigate methods and generate design knowledge for material strategies to improve and reduce material waste in the built environment, further enabled by the use of small elements that can be sourced from waste wood and reclaimed wood. Research is conducted as a hybrid material-computational methodology, developing and testing probes, prototypes and a full-scale demonstrator assembly in the form of a wall seating composition. The results find that the proposed method and resulting composites have significant potentials for both expressive and functional characteristics, allowing tectonic articulation to be made, while creating minimum material structures based on assembly of small elements to larger complex curvature building parts.

Keywords: Wood, Textile, Composite, Computational Design, Environmental Design

INTRODUCTION tons by 2060 (OECD, 2018). As the design fields tar- While the available energy on the planet is thou- get materials with environmentally friendly and re- sandfold larger than what we need globally, mate- generative profiles, such as wood and plant species, rials are a limited resource, and a fragile one. The it becomes evident that we need new design knowl- planet cannot support the current use of materials, edge, strategies and methods to advance into a sus- where we currently use the available resources per tainable material practice. With the mentioned ma- annum within 2/3 of the year passed [1]. Accord- terials being a scarce resource, with raw wood only ing to the world leading organization that tracks ma- providing approximately 4.5% of the global supply terial use, The Organization for Economic Coopera- (UN, 2020). Furthermore, wood and plants are mate- tion and Development (OECD), this is combined with rial systems that needs assembly of many smaller ele- a projected increase in material use in the built en- ments into larger structures in architecture, request- vironment from a massive 89 Gigatons to 167 Giga- ing new methods for such designed structures of nat-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 443 ural materials. This research studies methods and tures. In similarity, textile architectures have been strategies through experimental design studies, for studied as structurally articulated systems (Otto and the assembly of small wood pieces based on an ultra- Raush, 1996) and as subtle architectural membranes lightweight and high carbon dioxide wood speci- from artistic approaches (Ramsgaard et al, 2019). men, Paulownia (280 kg/m3), combined with organic Towards technical studies and the combination of textiles into natural fiber composites with high CO2 these approaches (Ahlquist et al, 2017, Deleuran et sequestration potentials (Pervaiz and Rain, 2003, Icka al, 2016) present studies which can be understood et al, 2016)[2]. This aims directly at addressing our as a search towards structural lightness (Beukers and current problems of the material energy-pollution Van Hinte, 2013). Technical studies investigating the source footprint, and the challenge of designing as- layering of textiles and wood into composites have semblies into novel composites through smaller el- also been conducted, focusing on both organic and ements, supporting the use of waste wood and re- synthetic textiles fibers, with a study focusing on the claimed wood in structures. joining of wood elements, through a point connec- tion approach [3].The background of potential solu- METHODS tions in this study is to examine and develop wood- The research employs a design-led experimental textile composites as a structural, forming and artic- methodology, focused on experimental prototypes ulating element. The material make-up and articu- and demonstrators (Ramsgaard and Tamke, 2009, lation thereof are thereby based on a material core Groat and Wang, 2011). Concretely, studies are fo- of lightweight wood with varying outer textile lay- cused on a three-step approach, with firstly study- ers bonded together to form a tectonic composition, ing wood-textile composites for deflection and stiff- where expressive characteristics are aligned with ness characteristics, then wood-textiles composites structural material-environmental potentials.The pa- of surface elements, analysing ultimate limit state per presents the base idea and strategy of combining conditions, and lastly a wood-textile composite of lightweight, CO2 sequestrating materials into func- a design-driven undulating monoclastic surface ge- tional and expressive composites. Studies are con- ometry as a field tested demonstrator with varying ducted through a design-led experimental research load cases. All three levels are developed as par- approach, where methods are presented, before core allel physical-computational design investigations, results, discussion and finally conclusion of the study using a Rhinoceros-Grasshopper-Karamba compu- is included. tational framework, integrating material-geometric- To get an initiating understanding of the tex- structural analysis into the design-led modelling in- tile impact on simple wood geometries, a series of vestigations. 30 samples is created in dimensions 30x14x1000mm. Jute, Flax, Hemp, Wool and eGlass fiber plain woven Deflection Studies textiles are combined with the wood core in five dif- The idea of combining smaller wood elements into ferent ways. Wool is the only non-woven textile, and a larger structure is not new. The German archi- studies are conducted along the grain of the paulow- tect and engineer Friedrich Zollinger invented the nia wood, figure 1. This provides a composite ma- Zollinger System, combining short planks into a di- terial matrix of variations in composites (textile type agrid structure (Schlaich et al, 2003). Focusing on and paulownia wood) and geometric arrangement material use and resulting articulation, systems such of the textiles in respect to the wood. Adhesion be- as tensegrity (Fuller, 1961, Lalvani, 1996) and recip- tween wood and textiles are throughout the studies rocal structures (Balmond, 2002) have been devel- based on white glue (carpenters glue) and compos- oped by pioneers of articulated structural architec- ites are merge by simple lay-up layering without pres-

444 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 1 Materials used of the studies. A: eGlass, B: Non-Woven Wool, C: Flax, D: Hemp, E: Jute, F: Paulownia wood.

surised formwork. For the deflection testing all ele- Ultimate Limit State Studies ments are fixed at one end of the element and ex- Following the initial studies, paulownia elements are posed to a point load in the other end (1 kg dead cut and assembled into a 250x450mm surface with load) figure 2, and as computational simulation stud- 8mm depth, including eight pieces of 250x8x60mm ies, figure 3. The stiffness of each element can then dimensions, glued on the short edge with stan- be evaluated as a result of measured deflection, with dard wood glue. Textiles of Flax and Wool are then the knowledge employed in the following studies. mounted with standard wood glue, with one refer- ence surface kept without textile mounted. Each sur-

Figure 2 Physical prototype deflection studies with samples merged into one photo.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 445 Figure 3 Computational simulation studies by Finite Element Method (FEM) analysis of deflection behaviour based on material composites. A: eGlass, B: Wool, C: Flax, 1: Covered with textile, 2: Diagonal textile, 3: Top textile, 4: side textile, 5: mid textile.

446 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 4 Textile material properties from (Petrone and Meruane, 2016)(Shah et al, 2016)(Bodros and Bailey, 2008) (Dhaduti and Mathad, 2019), used for FEM simulations. face is then placed in a steel frame, where a bolt DESIGN EXPERIMENTATION and 50mm washer is mounted in the geometry cen- Design-led experiments are driven by the empirical troid. This setup allows testing the composite mate- measurement of the previous studies embedded into rial properties by a vertical pull force, forcing the tex- the computational design model and the data from tile into tension and the wood into compression, fig- the literature. The aim of the design model is to sup- ure 5. Measurements are conducted for all surfaces as port the investigations of relations between the ar- Ultimate Limit State (ULS) testing, giving a direct un- ray of planar wood elements with varied surfaces of derstanding of the strength comparison properties textile layers, applied on both ‘outside’ and ‘inside’ of between the different composites. the geometry. By intuitively changing the load cases, figure 7 and 8, textile patterns are formed revealing Design Model Studies the relations between the material composite, the Based on the understanding of the previous two geometric articulation and the use condition. These studies, a demonstrator geometry for an undulating studies provide insight to how the textile-wood com- wall mounted seating is developed. The intent is to posites can steer expressive characteristics and min- maintain a simple form geometry, monoclastic, fo- imizing material use at the same time. From these cusing on the material-structural relations between experiments, fabrication patterns are directly gener- textile and the simple wood elements forming the ated of the 2D cut lines of textiles and the 3D cut lines curvature of the geometry. In this way, the textile op- of the wood elements, forming the resulting curva- erates to create joining between wood elements, dis- ture and strength and flexibility of the geometry. tribute tension forces, and form the articulating sur- faces that the human is seated against. The studies are driven by the developed computational model, where varying stress concentrations from two load cases, figure 6, steer the geometry and application of textile layers. The load cases include a person sit- ting well into the geometry, leaning towards the back of the geometry, and another person sitting on the front part of the geometry, figure 7 and 8. The loads are set to 70 kg, through points on the front of the seating, or as distributed on the seating and against the back surface.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 447 Figure 5 A1-A4: Paulownia without textile, B1-B4: Flax-Paulownia composite, C1-C4: Wool-Paulownia composite.

448 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 6 RESULTS Computational The initial series of experiments with deflection stud- design studies ies of linear element, fixed in one end, and exposed model using the to a point load, presented little bending variation be- Rhinoceros, tween the elements during simulation. The simula- Grasshopper, tion studies were then confirmed in the physical pro- Karamba totypes, despite the difference in material proper- parametric ties between the textiles, with results showing only modelling and minor deflection differences measured. Stiffness of simulation the composites were therefore assessed almost iden- framework. tical based on the geometry variations and load case presented. Through physical measurements of the surface composites (with 8mm structural depth), Figure 7 it became evident how much the textile changed Top: Tension and the strength properties when combined with the compression forces paulownia base elements. Unsurprisingly, the sam- on front and back ple without textile mounted breaks at a much lower side of geometry. force than the wood-textile composites, at 80.9N. Center: Stress The Wool/Paulownia composite at 128.4N. The Flax/- concentrations. D: Paulownia composite does not break, but the wood Resulting is crushed under compression forces at 159.3N. In- generated textile terestingly, hence, the latter does not actually break layers distribution at the maximum stress possibly within the measure- on front and back ment setup. Instead, the paulownia wood, in com- side of the pression, is pressed together, shortening the surface geometry under to a degree, which makes the sample move away load case with from its edge fixtures. This means, that the structural person sitting on issue of the composite is not the composites tension outer part. forces, but the material deformation that occurs in the wood layer during the ULS test. This suggest that wood species with low densities (as can be found through the Janka test method) is less usable for the composite structures proposed in this study. Lastly, the field testing of the full scale demonstrator, figure 9 and 10, reveals that the structure with fracture, but some deformation, resist different load cases, as pre- sented above, and with a high load of 85 kg, as op- posed to the computational design experiments us- ing a 70 kg force as input. At 150 kg load on the built composite, with pressure points moved from the cen- tre of the geometry to the sides, the structure delam- inate in the lower section, leading to fracture of the wood assembly.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 449 In specific, Paulownias low density is a result of its Figure 8 growth characteristics providing the material with a Top: Tension and very good environmental impact, but also with a low compression forces density to resist compression forces. It is possible on front and back that higher density woods would enable a significant side of geometry. higher composite strength profile, thereby enabling Center: Stress a reduction of the thickness of the wood layer, and concentrations. D: in turn be the same or lower combined weight of Resulting the composites. The computational model studies generated textile focused on simulation and modeling of textile layers layers distribution and the effects on the combined composite. These on front and back studies could in future studies be advanced by con- side of the sidering synclastic and anticlastic surface curvatures, geometry under by variation in textile layer properties, variation in ad- load case with hesive elasticity whereby both bonding and local in- person sitting on terlayer dynamics of the composite may be strate- mid part leaning gically controlled to design varied stiffness and al- backward. ternate fixture positions of the structure, particularly when considering the composite for other function- alities than presented in this research. Further studies of the interrelations between textile and wood layers appear very relevant to pursue as a means to material focused environmental design strategies.

Figure 9 Resulting textile-wood composition with combined load cases.

DISCUSSION The studies use a hybrid physical-computational ap- proach, with the intent to combine validation of sim- ulation models and explorative experimental studies through design-led investigations. This also means that the material has been selected by a focus on positive material-environmental impact, combined with low weight and tensile qualities. More stud- ies with a greater variation of materials, both textiles and wood species will be able to strengthen the re- sults obtained and enable a potential broader un- derstanding of the proposed composite properties.

450 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 10 Field test of demonstrator with varying load cases. A: No loads, B: Load case 2 (65 kg), C: Load case 1 (65 kg), D: Load case 2 (85 kg), E: Load case 1 (85 kg), F: Assymetric and dynamic load, G: Load away from center (150 kg) leading to delamination, H: Load leading to fracture (150 kg).

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 451 CONCLUSION Lalvani, H 1996, ’Origins of Tensegrity: Views of Em- From the studies presented, it is clear that the merich, Fuller and Snelson’, International Journal of proposed composite, and its specificities unfolded Space Structures, 11, pp. 27-55 Otto, F and Rausch, B 1996, Finding Form: Towards an Ar- through material and computational prototypes and chitecture of the Minimal, Edition Axel Menges demonstrator studies present ways to develop and Pervaiz, M and Sain, M 2003, ’Carbon storage potential design structural assemblies of smaller parts through in natural fiber composites’, Resources, Conservation the computational process. The studies therefore and Recycling, 39(4), pp. 325-340 present knowledge, methods and concrete design Petrone, G and Meruane, V 2017, ’Mechanical proper- models, which enable material-environment efficient ties updating of a non-uniform natural fibre com- posite panel by means of a parallel genetic algo- assemblies of smaller parts, such as waste wood, rithm’, Composites Part A: Applied Science and Man- reclaimed wood and reclaimed and restitched tex- ufacturing, 94, pp. 226-233 tiles. While demonstrated through a small architec- Ramsgaard, M, Sinke, Y, Monteiro, F and Lienhard, J 2019 tural design scale, the studies could be imagined di- ’Systems for transformative textile structures in CNC rectly applicable in architectural scales such as the knitted fabrics - Isoropia’, Proceedings of the Tensinet building envelopes and spatial partitions, where re- Symposium Ramsgaard, M and Tamke, M 2009, ’Narratives of Making: duced weight will reduce the load on load-bearing thinking practice led research in architecture’, Com- elements, and being easier to handle and transport municating (by) Design, January, pp. 1-8 during construction. Schlaich, M, Stavenhagen, L and Krüger, G 2003, ’Die HanseMesse in Rostock - Zollinger mit moderner Technik’, Bautechnik, 80, pp. 279-284 REFERENCES [1] https://www.overshootday.org Ahlquist, S, McGee, W and Sharman, S 2017 ’Pneu- [2] https://news.bio-based.eu/natural-fibres-show-outs maKnit: Actuated Architectures Through Wale- and tandingly-low-co2-footprint-compared-to-glass-and-mi Course-Wise Tubular Knit-Constrained Pneumatic neral-fibres/ Systems Conference’, Proceedings of ACADIA 2017 [3] https://tu-dresden.de/ing/maschinenwesen/itm/for Balmond, C 2002, Informal, Prestel schung/forschungsfelder/anwendungsbereiche/faserv Beukers, A and Van Hinte, E (eds) 2013, Lightness: The erbundwerkstoffe/holzverbunde?set_language=en Inevitable Renaissance of Minimum Energy Structures, Nai010 Publishers Bodros, E and Baley, C 2008, ’Study of the tensile proper- ties of stinging nettle fibres (Urtica dioica)’, Materials Letters, 62, pp. 2143-2145 Deleuran, A, Pauly, M, Tanke, M, Tinning, I and Rams- gaard, M 2016, ’Exploratory Topology Modelling of Form-active Hybrid Structures’, ProcediaEngineering, 155, pp. 71-80 Dhaduti, S and Method, S 2019, ’Preparation and Analy- sis of Mechanical Properties of Short Sisal and Glass Fiber Reinforced Composites’, MetabolicEngineering, 5(3), pp. 1009-1016 Fuller, B 1961, ’Tensegrity’, Portfolio an Art News Annual, 4, pp. 112-127 Groat, L and Wang, D 2011, Architectural Research Meth- ods, Wiley Icka, P,Damo, D and Icka, E 2016, ’Paulownia Tomentosa, a Fast Growing Timber’, Annals ”Valahia” University of Targoviste - Agriculture, 10(1), pp. 14-19

452 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Thermochromic Animation Thermally-informed and colour-changing surface-configurations

Andreas Körner1 1Universität Innsbruck, Institut für Experimentelle Architektur [email protected]

All factors of thermal comfort are invisible to humans and do not (yet) impact visual navigation in the built environment. Thermochromic materials change their colour relative to temperature. In architecture, their applications as responsive ornaments and as intelligent composite systems are discussed. Nonetheless, design research on their use together with computational design is scarce. This study investigates thermochromics concerning architectural surfaces. Design and material experiments were conducted to test the hypothesis that thermochromic animation can be configured to visualise invisible parameters of thermal comfort. Scale prototypes were fabricated from different materials and coated with thermochromics. They varied in layer number and sub-coatings. The colour change was observed with several instruments. Heat transfer simulations of digital doppelgangers accompanied the physical experiments. The results suggest that this method can be used to configure thermochromic animation. This can be implemented into a procedural design model for porous and multi-layered thermochromic surfaces in the future. In this, digital simulation and material-based design are combined in a method that advances the use of thermochromic materials in the context of digital architectural design.

Keywords: thermochromics, fabrication, simulation, materials, colour

INTRODUCTION Thermal comfort Buildings create micro-climates and one of their Comfort can be defined as being satisfied with the key functions is to provide a certain level of ther- ambient (Hens, 2016). The interior climate quality is mal comfort. The following paragraphs investigate quantified by the inhabitant’s sensation of warmth. a method to design, predict, visualise, and fabri- This is the result of four physical parameters: air tem- cate thermochromic response (TR). This can help to perature, mean radiant temperature, relative airflow, communicate invisible parameters of thermal com- and humidity. All four can be affected by architec- fort and create an environmentally responsive, poly- ture. Two more factors exist that can be controlled chromic ornament. individually: metabolic heat production and clothing (CIBSE, 2006). Special attention is given to the direc-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 453 tion of exposure. Non-uniform exposure, e.g., turbu- sive stimulation to active perception (Gibson, 1966). lent draughts or heat from local radiators, increase discomfort. Defining a specific model for thermal Figure 1 comfort in our dynamic urban environments is chal- The parameters lenging (Greenberg et al., 2017). Concepts of adap- affecting thermal tive thermal comfort are increasing in popularity. comfort are In the context of climate change and sustainability, invisible to human non-uniform models of thermal comfort and the role eyes. Therefore, of the variability of indoor conditions are discussed inhabitants rely (Mishra et al., 2016). Until recently, low energy prices either on allowed levelling a lack of environmental design by knowledge to find artificial air-conditioning. Consequently, the changes comfortable areas in the global economy and the looming climate crisis or evenly temper have reinvigorated the study of thermal comfort over the entire space. the past two decades (Nicol & Roaf, 2017). In the case Santa Maria degli of naturally ventilated buildings, thermal adaption Angeli e dei Martiri, by occupants can help to achieve comfort without Rome, February air conditioning (Etheridge, 2012). None of the fac- 2020. tors is directly visible to humans (Hens, 2016). There- fore, they currently do not impact visual navigation in large spaces (Figure 1).

Perceiving the invisible This research investigates a design and fabrication method combining thermochromic materials and digital simulations for architectural elements. They can communicate invisible parameters of thermal comfort to a building’s inhabitants. It specifically Information and ornament aims to identify key parameters for design, fabrica- The potentials of smart- or information materials tion and simulation which can constitute a foun- have been highlighted by others (Kretzer, 2017) dation for future research into parametric design (Addington & Schodek, 2005). Their use in archi- methods. Therefore the paper explores the rela- tecture includes interior and exterior applications. tionships between temperature, topology, and ther- Thermochromic response (TR) has been studied ex- mochromic materials by designing, fabricating and tensively and some applications in architecture have testing a series of prototypes for evaluation. Cup- been suggested. Nonetheless, its position in the ar- kova et al. clarified, that surface patterns and topol- chitectural discourse is strongly tied to that of build- ogy can significantly modulate the thermal mass and ing physics, environmental performance, and sus- heat transfer behaviour of cast panels (Cupkova & tainability. Since many such materials lead to colour Azel, 2015). While the colour of a surface is visible, changes a closer examination of the role of colour is its temperature is not. Our skin on the other hand required. Surface colour or lighting has no significant senses temperature but not colour. Visualising fac- impact on thermal comfort (CIBSE, 2006). Any psy- tors of thermal comfort through material configura- chological effect of colour in architecture is outside tions would change its mode of perception from pas- the scope of this paper. In architecture, colour has always played an important role. During the poly-

454 | eCAADe 39 - Structure optimization and material-based design - Volume 2 chromy debate in the 19th century, it has been ar- referred to as ‘substrate’ (Grafstein et al., 1968). Two gued whether or not ancient Greek architecture was conventional thermochromic systems are liquid crys- mono- or polychromic (Semper, 1834) (Figure 2). De- tals and leuco dyes (Kretzer, 2017). Several different bates were whether whole surfaces or only selected thermochromic pigments with varying phase change features were coloured (Ziesemer, 2003) and if orna- temperatures can be mixed to achieve a more com- ment ‘animates’ the dead surfaces and rigid articu- plex TR. In the context of architecture, applications lations (Wölfflin, 2000). Some colours have an im- of thermochromics as responsive ornaments and as pact on the environmental performance of a building intelligent composite systems have been explored (Azarnejad, 2017). As ‘surface conditions’ ornament (Meagher et al., 2013). Electric circuits can be embed- has the potential to substitute tectonics as means of ded inside a solid material to partially heat it. Once spatial organisation (Picon, 2010). A difference must coated with thermochromics, patterns can be made be made between smart materials, as materials that visible on its surface (Cupkova et al., 2018). In con- have an inert intelligence, and a smart application of struction, thermochromics find a commercial appli- materials, as informing the placement and configura- cation as a smart, dynamic glass to control heat loads tion of materials. This paper investigates an overlap through glass facades (Seeboth & Lötzsch, 2013). between the two. From this review the following re- From this review the following research question can search question can be derived: How can material in- be derived: How can TR be programmed, configured, telligence be used to visualise factors of thermal com- and fabricated? fort? Heat transfer and thermal infrared images Figure 2 ‘Research on the performance of various build- Detail of the ing components has constituted a significant and Parthenon, long-standing domain within architectural research’ Acropolis, Athens. (Groat & Wang, 2013). Burry et al. investigated the Painted by Gottfried integration of heat transfer performance and ther- Semper. 1836. mochromics for façade design (Burry et al., 2013). The Public Domain. role of thickness and thermal mass in relationship to geometric surface morphologies has been exten- sively researched by Cupkova et al. (Cupkova & Pro- moppatum, 2017). They have investigated the heat transfer through concrete panels triggered by solar loads (Cupkova & Azel, 2015) and a fabrication pro- cess using robots to shape concrete panels (Bard et al., 2019). While previous research investigated ra- diation’s impact on a surface, this paper presents an investigation of the heat flux through a surface that is heated from the back. Heat transfer is the interac- tion of energy caused by a difference in temperature; within a medium or between media (Ghoshdastidar, 2004). Aviv et al. have investigated the use of thermal Thermochromic response infrared (TIR) cameras to inform robotic fabrication Materials that visibly change colour relative to tem- (Aviv & Teitelbaum, 2017). Access to TIR cameras has perature are called thermochromic materials. When been eased with the availability of low-cost products. applied as a coating, the underlying material is often

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 455 Figure 3 This research combines several approaches for the base geometry. The three key strategies are grooves/ridges, plateaus/terraces, and cavities/voids.

Such cameras have numerous applications across all erature review, this paper investigates how the layer- scales (Acorsi et al., 2020). From this review the fol- ing of colours, patterns, and thermochromic coatings lowing research question can be derived: How can can be configured to extend the response toolset designers predict and plan such TR? of a single thermochromic ink by utilising the ma- terial inert mechanism of delay. The role of cavities and undercuts has been explored since most exist- Figure 4 ing work deals with solid, high-density, extruded and A method diagram relief samples (Figure 3). Often, the thermochromic outlining the coating acts either as an observation and measuring relationships tool (Cupkova & Promoppatum, 2017) or as an actu- between ated display (Cupkova et al., 2018). The hypothesis is, simulation, design, that through this process, the repertoire of responses fabrication, can be extended towards thermochromic animation; observation, and beyond the binary colour change while relying on a evaluation. coating that performs one phase transition.

METHODS Design and material experiments were conducted to test the hypothesis of thermochromic animation. Various panels were scripted and digitally fabricated from multiple materials (high-density foam, PLA, clay, plaster, cement, sand binder-jet) with varying den- sities. The prototypes were then coated with ther- mochromic pigments. The coatings themselves var- ied in the number of layers (thickness) and vari- ous sub-coatings (colour and pattern). The colour Thermochromic animation changes, resulting from heating and cooling, were documented using TIR cameras, video, and laser Departing from previously gained knowledge on thermometers. Outcomes were analysed and com- CFD-driven thermochromic articulations and the lit- pared with heat transfer simulations of digital dop-

456 | eCAADe 39 - Structure optimization and material-based design - Volume 2 pelgangers (Figure 4). The observations then inform original geometry was accepted due to time con- the next design iteration. This combination of ex- straints. A 0.2mm layer height was used. Conse- periment and simulation in ‘sequenced phasing’ is quently, the resulting panels had a lower resolution common in environmental design research (Groat & than the original polygon mesh. The prints were used Wang, 2013). to make reusable silicone moulds. Plaster, concrete, and white cement were used for casting. Some of the Designing with CFD substrates were mixed with yellow thermochromic An air-flow simulation of an arbitrary wall was set up pigment powder at different ratios (8g, 4g, 2g per using Autodesk CFD Ultimate 2019. On this surface, a model) to investigate an alternative response strat- series of geometric features were placed. They rep- egy to coating. Pouring material with different den- resent abstract façade protrusions that interfere with sities into the surface depressions has been explored the air flows independent of the flow direction. The as well, but is not part of the results shown here be- results are exported as an image file. The smooth cause of the experiments pre-mature state. The re- colour gradients are posterised and contoured. The sulting panels were 3d-scanned with photogramme- resulting curves are sorted by length, displaced per- try using Epic’s Reality Capture. The deviation be- pendicularly to the image plane, and meshed using tween the original meshes and the scanned ones was VDB (Figure 5). analysed by overlaying the meshes in Rhino6. Across each panel, it ranged between 1-2mm. Since the re- Figure 5 sulting meshes had to be reduced heavily for the sim- The colour data of ulations, this deviation was accepted. the air-flow pattern is converted into Heat transfer simulation curves. They are The photogrammetry meshes were reduced from an displaced average of 80k to around 300 faces using the ‘_Re- depending on their duceMesh’ command in Rhino6. The scope of the length in a direction simulations was to understand which parts heat up normal to the quicker than others. The focus was on the graphic surface plane. The output to determine different zones and to iden- results are meshed tify patterns. Therefore, a simple setup was chosen and shaded for with a heat flux boundary condition of 200 W/m² ap- preview. plied to the back. For all other surfaces, a film co- efficient boundary condition of 20 W/m²C was ap- plied for convection with a 10°C reference tempera- ture. The simulation type was set as a transient with 20 timesteps. For initial simulations, during the de- sign process, ANSYS Discovery Live 2020 was used to get quick visual feedback. Since the academic licence Fabrication of ANSYS has only limited export functionality, Au- This paper focuses on PLA 3d-printing and casting. todesk CFD 2019 was used in some cases (Figure 6). Cnc-milling, clay, and binder-jet 3d-printing are not It allows exporting results as FBX files with an add-in discussed. A series of 1:10 scale models (120x60mm) (Showcase Exporter 3). The files include the simula- were printed with an FDM 3d-printer (Anycubic i3M) tion results as vertex colours. for colouration strategy tests. A deviation from the

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 457 to identify patterns. Therefore, a sufficient differenti- Figure 6 ation of data across an image was satisfying and pa- Digital heat-transfer rameters like the accuracy of temperature readings simulations allow were ignored. The orthogonal top views of the spec- us to predict and imens were compared to the digital simulations us- design the ing computer graphics; both single images and se- heat-map patterns quences. The evaluation criteria for the TR were con- on the opposite trast, diversity, and speed. side of the relief.

Figure 7 A series of ‘hot’ panels just before cooling down. The different substrate patterns and colours are visible.

Coatings The black thermochromic ink has a phase transition temperature of 27°C. This is close to the temperature, which, together with high levels of humidity, gen- Figure 8 erally cause discomfort (CIBSE, 2006). This research Experiment setup. used liquid crystal thermochromics. It consisted of two components; a 1:1 mix of thermochromic slurry and sprayable lacquer. The substrate coatings be- neath were either applied with marker pens, ther- mochromic powder (28°C), or acrylic spray paint.

Experiments and comparison For observation, the scale parts are placed 40-50mm over an infrared heating plate (150W). Each series of panels was exposed for ten minutes. The colour change was documented using a mounted DSLR camera, a handheld laser thermometer, a hand- held smartphone TIR camera (FLIR One iOS Gen 3), and a USB TIR camera (FLIR Lepton 3 + GroupGets Purethermal-2) mounted 150mm below the DSLR (Figure 8). The focus of the observations lied on the relationship between geometry and TR. The thermo- graphic data was used for control and reference and

458 | eCAADe 39 - Structure optimization and material-based design - Volume 2 RESULTS DISCUSSION The substrate-inert colour has a big influence on the The results suggest that the relationships between fabrication process. Neutral, pale material colours - material thickness, topology, and TR can be used such as plaster, cement and white PLA - perform bet- to configure thermochromic animation. Ther- ter than more saturated ones. The latter requires ex- mochromic pigments change colour at a specific tensive priming which adds steps to the process. The temperature. When directly exposed to heat, e.g. experiments show that an even thermochromic coat- from solar radiation, the colour change occurs once ing is crucial. If the coating layer is too thick, the the surface temperature exceeds this material- colour does not sufficiently shine through. (Figure specific value. When the backside of the surface 7) Patterned substrate coatings tend to work better is heated, the colour change is triggered once the than solid ones because they use the natural contrast substrate material’s temperature exceeds it. Digital between substrate material and pattern. They reduce simulations are ubiquitous tools common in archi- the required thickness of the thermochromic coat- tectural and environmental design (Groat & Wang, ing. Tool traces of the printing nozzle yielded a higher 2013). Besides material experiments, they are key surface complexity leading to intricate features. This for thermochromic animation. The results show that yielded a more effective TR compared to smooth ar- the hypothesised method to configure and program eas. The TR was stronger, and more visible, wherever TR is feasible. Thermochromic surfaces can be pro- small elements prevailed. The same applies to imper- grammed by informing the composition of the sub- fections from the casting process such as enclosed strate layers with heat transfer simulations. The sim- solids (gravel) or voids (bubbles). The effect is gener- ulations can successfully predict the response and ally stronger the thinner intricate a plate’s topology the method can be used to highlight surface fea- is (Figure 9). Designs with relatively deep protrusion- tures. Prototypes can be fabricated in various ways. s/depressions tend to be more successful than flat Creating moulds for casting appears to be a sustain- but patterned ones; depth outperforms thickness. able solution since digital fabrication per part is re- duced. Response variety can be increased by using Figure 9 different casting materials, inlays, and pigments. A Snapshots of a comparison of the outcomes suggests that the direct ten-minute long correlation between geometry thickness and TR can exposure to IR be directly used for design without extensive sim- radiation from ulations. Some software solutions allow exporting below. simulation results as coloured meshes or CSV tables. This data can be imported into CAAD platforms us- ing the FBX file format or via node import for design purposes. In general, the following can be said about thermochromic animation and its configuration: • Intricate surface articulation leads to a stronger TR. Imperfections, tool traces, paint-pooling, coating evenness, and thickness show potential as design strategies; diversity, contrast. • Thin and porous geometries perform better since their thermal mass is comparatively low. They also have a higher surface area; speed, con- trast.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 459 • Cavities and inclusions have an impact on the TR. sensor does not provide radiometric data. Techni- The trapped air insulates and impacts the heat cally, temperature data can be read from the result- transfer path inside the geometry; speed, con- ing TIFF files and converted into RGB values using Im- trast, diversity. ageJ. This requires manually calibrating the camera • Flat surfaces (backsides) and reliefs perform using temperature references. Future experiments worse than geometries that are equally intricate must be done using a Lepton 3.5 which provides ra- on both sides; speed, contrast. diometric data at a comparable price. The image size of 160x120px is too low. Artificial intelligence Critical reflections can be used to enhance the output. 3d-printing of- fers great design freedom. To up-scale the process, Some of the panels heat up unevenly. TIR images robotic clay or concrete 3d-printing appear feasible. show that the heating bed is not warming up con- Their downside is reduced design freedom due to sistently. As a result, panels close to the border re- fabrication constraints (continuous paths). Binder-jet main cooler than the ones in the centre. The loca- 3d-printing could be a possible alternative, although tion of a panel concerning its neighbours has an ob- the results tend to be extremely porous with high ab- servable effect on the evenness of the colour change sorption rates. too. Panels in the centre of an arrangement warm up quicker than those on the perimeter. The sides of Future studies each panel should be insulated for evenness in future The implementation of cavity spaces must be ex- experiments, as suggested by others (Cupkova & Pro- plored further. In the next steps, the outcomes can moppatum, 2017). be implemented into a parametric model for the Limitations generation of thermochromic surfaces according to environmental data. Developing a method to im- Strategic placement of layers reduces UV degra- plement the thermal analysis back into a paramet- dation of the thermochromics by shielding certain ric model is key. On this basis, digital simulation parts of the geometry from light (Figure 10). The and material-based design are combined in a tech- software interface between thermal simulation and nique that advances the use of thermochromic ma- generative design is not sufficiently developed yet. terials in the context of architectural design (Figure The thermal simulation results can be exported as 11). The outcomes suggest investigating how such a meshes containing vertex colours or as node data. A response can be combined with augmented reality. workaround is to export the uncoloured mesh data Complex environmentally responsive trackers could and then texture map orthographic images onto the be designed to act as input devices for AR applica- mesh using computer graphics. The Lepton 3 TIR Figure 10 A multi-layered curtain wall element would allow merging environmental performances such as shading and insulation with an integrated TR.

460 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 11 From left to right: 3d-model, thermographic image (TIR), thermochromic response (TR), schematic sections, thickness, digital simulation. Three key strategies are tions on smartphones or glasses. Future, and ongo- ACKNOWLEDGEMENTS suggested: cavities, ing, research will focus on larger scales and 1:1 ap- This research is part of the project Thermochromic undercuts, layers. plication. In the context of the climate crisis envi- Topologies funded by the Tiroler Wissenschafts- ronmentally friendly and non-toxic solutions must be förderung. Principal investigator: Andreas Körner; investigated. Studies on non-toxic thermochromic Assistant: Catalina Tripolt; Contributors: Jan Contala, pigments exist (Seeboth et al., 2013). Cnc-milling of Philipp Schwaderer, Kilian Bauer, Julian Edelmann, wooden formwork and clay 3d-printing should be in- Ernest Hager. All figures except Figure 2 by the au- vestigated. thor.

CONCLUSION REFERENCES A combination of digital simulations and material Acorsi, MG, Gimenez, LM and Martello, M 2020, ’Assess- experiments can be used to configure the ther- ing the Performance of a Low-Cost Thermal Camera mochromic response of architectural surfaces. Lay- in Proximal and Aerial Conditions’, Remote Sensing, 12(21), p. 3591 ers, cavities, overhangs, and coating affect the re- Addington, DM and Schodek, DL 2005, Smart Materials sponse. Reliefs offer limited possibilities due to the and Technologies in Architecture, Elsevier, Burlington high thermal mass of the base surface. The experi- Aviv, D and Teitelbaum, E 2017 ’Thermally Informed ments investigated the topic on both a material and Bending: Relating Curvature to Heat Generation an architectural scale. By thermochromic material Through Infrared Sensing’, Humanizing Digital Real- programming, surface articulations can visually high- ity, Singapore, pp. 362-371 Azarnejad, A 2017, Impact of Building Facades’ Color on light formerly invisible parameters of thermal com- Building and Urban Design, Ph.D. Thesis, TU Wien fort over distance. The environmental condition at Bard, J, Cupkova, D, Washburn, N and Zeglin, G 2019, the surface datum is therefore extended into space. ’Thermally Informed Robotic Topologies’, in Will- The observable increase in complex geometries in ar- mann, J, Block, P, Hutter, M, Byrne, K and Schork, chitecture together with higher demands for envi- T (eds) 2019, Robotic Fabrication in Architecture, Art ronmental performance will require advanced meth- and Design 2018, Springer International Publishing, Cham, pp. 113-125 ods to simulate and design. Thermochromic anima- Burry, J, Salim, F, Williams, M, Pena de Leon, A, Burry, M tion allows programming the behaviour of respon- and Sharaidin, K 2013 ’Understanding Heat Transfer sive surfaces in architecture. In doing so it con- Performance for Designing Better Facades’, Proceed- tributes to the fields of thermodynamic architecture ings of ACADIA 2013, Cambridge, pp. 71-78 and digital ornament. A closer interlinkage between CIBSE, - 2006, Environmental Design, Chartered Institu- material, geometry, colour, and texture will extend tion of Building Services Engineers, London Cupkova, D and Azel, N 2015, ’Mass Regimes’, Interna- the architectural vocabulary to better communicate tional Journal of Architectural Computing, 13(2), pp. between the natural and the built environment. 169-193

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 461 Cupkova, D, Byrne, D and Cascaval, D 2018 ’Sentient Con- crete’, Proceedings of CAADRIA 2018, Beijing, pp. 545- 554 Cupkova, D and Promoppatum, P 2017 ’Modulating Thermal Mass Behavior Through Surface Figuration’, Proceedings of ACADIA 2017, New York, pp. 202-211 Etheridge, D 2012, Natural Ventilation of Buildings: The- ory, Measurement and Design, Wiley, Chichester Ghoshdastidar, PS 2004, Heat Transfer, Oxford University Press, Delhi and Oxford Gibson, J 1966, The Senses Considered as Perceptual Sys- tems, Houghton Mifflin, Boston Grafstein, D, Burkowski, RP, Kornblau, M and Flint, WL 1968 ’Thermochromic Displays’, Recent Advances in Display Media, Washington D.C., pp. 53-61 Greenberg, E, Mavrogianni, A and Hanna, S 2017 ’Toward a Spatial Model for Outdoor Thermal Comfort’, Hu- manizing Digital Reality, Singapore, pp. 408-416 Groat, L and Wang, D 2013, Architectural Research Meth- ods, Wiley, New York and Chichester Hens, H 2016, Applied Building Physics, Ernst & Sohn, Berlin Kretzer, M 2017, Information Materials: Smart Materials for Adaptive Architecture, Springer, Cham Meagher, M, van der Maas, D, Abegg, C and Huang, J 2013, ’Dynamic Ornament’, International Journal of Architectural Computing, 11(3), pp. 301-318 Mishra, AK, Loomans, M and Hensen, J 2016, ’Thermal comfort of heterogeneous and dynamic indoor con- ditions’, Building and Environment, 109, pp. 82-100 Nicol, JF and Roaf, S 2017, ’Rethinking Thermal Comfort’, Building Research & Information, 45(7), pp. 711-716 Picon, A 2010, Digital Culture in Architecture, Birkhäuser, Basel Seeboth, A and Lötzsch, D 2013, Thermochromic and thermotropic materials, Pan Stanford Seeboth, A, Lötzsch, D and Ruhmann, R 2013, ’First exam- ple of a non-toxic thermochromic polymer material – based on a novel mechanism’, Journal of Materials Chemistry C, 1(16), p. 2811 Semper, G 1834, Vorläufige Bemerkungen über bemalte Architectur undPlastikbei denAlten, Johann Friedrich Hammerich, Altona Wölfflin, H 2000, ’from Prolegomena to a Psychology of Architecture’,in Frank, I (eds) 2000, The theory of dec- orative art, Yale University Press, New Haven, pp. 332-339 Ziesemer, J 2003, ’Gottfried Sempers Bedeutung für die Wahrnehmung von Architekturpolychromie im 19. Jahrhundert’, ICOMOS, 39, pp. 128-132

462 | eCAADe 39 - Structure optimization and material-based design - Volume 2 An Autonomous Bio-Inspired Shading Façade System based on Plant Movement Principles

Alexa Sharp1, Georgina Blay2, Janna Kholodova3, David Correa4 1,2,3,4University of Waterloo 1,2,3,4{ajsharp|glblay|jkholodo|david.correa}@uwaterloo.ca

Utilizing existing principles of plant movement, we can design climatic-responsive facades made of hygroscopic materials. This paper investigates the use of a double actuating system to create an architectural façade. Several adaptive façade strategies have been previously developed using wood bilayers, but there has not been significant investigation into the application of multiple actuation points in a single unit. The paper presents a façade that is responsive to the surrounding environment via the kinematic amplification of hygroscopic wood expansion. The kinematic amplification uses the biomechanical principles from both the Water Lily (Nymphaea) and the Purple Shamrock (Oxalis triangularis). Acting as an adaptive shading mechanism, the façade system - arranged using Lindenmayer system principles - can improve occupant comfort by controlling solar radiation . The developed prototypes use climate-responsive wood bilayer actuators. The aesthetic and functional features of the bio-inspired mechanism promote a visual awareness between our built environment and environmental conditions.

Keywords: Adaptive Façade, Biomimetics, Plant Movement, Responsive Architecture, Hygroscopic, Stimulus-Responsive Materials

INTRODUCTION 2012), or smart 4D printing mechanisms (Gladman et Plants offer valuable inspiration to designers and al., 2016; Correa et al., 2020; Poppinga et al., 2020) engineers. Adaptation strategies, form and func- - among many others. Abstract branching algo- tion relations, and large systemic principles can pro- rithms based on plants known as L-systems (Linden- vide valuable lessons on integrative systems. This mayer systems) have been used in the development is particularly important in the case of Architecture of large scale infrastructure, street networks, and as buildings must integrate multiple systems while façades in digital reconstruction applications (Peter, providing an inhabitable and comfortable environ- 2017; Becker et al., 2013). The research presented ment for their occupants. Plants have been used as in this paper takes inspiration from both abstracted the source of inspiration for adaptive façades (López models of organization and more direct biomimetic et al., 2017), engineered materials (Elices, 2000), bio- transfer of plant biomechanics principles. inspired construction principles (Knippers and Speck, The façade mechanism combines biomechani-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 463 cal principles from the Water Lily (Nymphaea) and term (”Architecture | Institut du monde arabe,” n.d.). the Purple Shamrock (Oxalis triangularis) to achieve Buckminster Fuller was once asked about the exces- the kinematic amplification of a hygroscopic actua- sive maintenance required for his geodesic structures tor. The Oxalis uses a two-way folding mechanism and replied: ’If you build it like a machine, you must while the Water Lily uses a ‘furling’ and ‘unfurling’ maintain it like a machine, not like a building” (Bald- movement to open its petals. The combination of win, 1997). For example, the façade screen of the In- both mechanisms increases the kinematic actuation stitut du Monde Arabe is made of a series of many in- and corresponding solar exposure coverage of the terconnected replaceable panels, each of them with mechanism in relation to humidity fluctuations. The sensors, controllers and actuators. Similar to many way that plants arrange themselves in nature, from buildings with extensive electromechanical compo- branch structure to leaf cluster, inspired the structure nents, the required maintenance was extensive and that holds up the prototypes that make up the ar- unfortunately this façade screen is no longer func- chitectural façade. This is commonly referred to as a tional (”Architecture | Institut du monde arabe,” n.d.). branching. As a system, the actuated façade mecha- As defined by Stuart Brand in his book How Build- nism will be referred to here as an adaptive façade. ingsLearn (Brand, 1994), there is ‘high’ and ‘low’ archi- tecture when considering obsolescence and change: Related Work ‘high’ architecture values permanence and tends to The ability of the building to condition the inte- avoid components that will malfunction or degrade, rior environment is essential for occupant comfort. while ‘low’ architecture is associated with flexible re- Since the weather conditions constantly change, one sponsiveness to change over time and has a relatively can describe this characteristic of the building as an rapid rate of obsolescence and decay (Meagher, adaptive behaviour. Façade systems play a key role 2014). The two primary drivers of change in build- in negotiating interior and exterior conditions. Sev- ings are: the rate at which the physical materials eral adaptive strategies have been previously devel- wear out and need to be replaced, and the frequency oped using active (requiring electromechanical in- with which changes in fashion dictate the reconfigu- put), passive, or hybrid constitutive systems, using ration of the existing infrastructure (Meagher, 2014). both material properties and electromechanical logic When building and maintaining adaptive façades, it controllers and actuators (Addington, 2010; López et is the rate at which the electromechanical compo- al., 2017). However, while climate adaptation can be nents wear out and fail that is of concern (Meagher, a valuable asset to a building, it is not feasible if it oc- 2014). curs at the expense of an excessive operational cost. The biological structures of plants have a novel During the building’s operational lifetime, mainte- perspective to offer here. Instead of using electrome- nance is time consuming, occurs at irregular intervals, chanical systems with multiple moving components, and can be more expensive than the cost of construc- we can use a compliant mechanism - something that tion. Moreover, the more mechanically complex the achieves motion through elastic material deforma- building, the more expensive the maintenance. For tion. One advantage of compliant mechanisms is instance, the cleaning of complex façades can be very the reduction in necessary maintenance (Wagner et difficult, expensive, and therefore, not a high priority al., 2020). Several plants have structures that show for building owners (Andaloro et al., 2017; Madureira reversible, climate adaptive actuation using compli- et al., 2017) The inclusion of various mechanical parts ant mechanisms. However, it is a long-lasting engi- in an adaptive façade can allow for increased range neering tradition to reduce unwanted movement in of motion and movement in the short-term, but of- wood, which manifests itself in the production pro- ten leads to extensive mechanical failures in the long- cess of plywood; wood veneers are laminated cross-

464 | eCAADe 39 - Structure optimization and material-based design - Volume 2 wise to become more dimensionally stable and less actuation based on daily and seasonal fluctuation cy- anisotropic (Eder et al., 2020). These plants use bi- cles. The aesthetic qualities provided by the shape- layer architectures that change shape in response to change actuation of the mechanism is also proposed changes in moisture - even after the tissue is dead as a performance characteristic that can elicit public (Martone et al., 2010; Forterre, 2013; Stahlberg and awareness of the environmental changes in humid- Taya, 2006). Seed dispersal mechanisms of the pine ity. Such a façade would also create an architecturally cone or the wheat awn use hygroscopic swelling of refreshing space that could increase occupant com- cellulose tissues to create specialized material struc- fort. tures capable of autonomous shape-transformation (Rüggeberg and Burgert 2015; Dawson et al., 1997; MATERIALS Reyssat and Mahadevan 2009; Elbaum et al., 2007). In The materials used were beech veneer, teak veneer, building construction, the anisotropic swelling char- non-water soluble glue, cellulose acetate film, and acteristics of wood have been historically viewed as a PLA. Beech veneer was used as the active expansion drawback when compared to isotropic or man-made layer, as it is a locally sourced wood in Canada and materials (Eder et al., 2020). Therefore, there is cur- Europe. Of particular importance for hygroscopic ac- rently limited research on the architectural applica- tuation, beech has a high swelling coefficient and a tions of such dimensional fluctuations. Systems that relatively high flexural strength. Teak, another com- use hygroscopic materials instead take advantage of monly available wood, was used as the resistive layer. the properties of compliant mechanisms. Beech was also extensively used in previous literature Hygroskin is a project by Correa et al. (2013) that for shape changing bilayers by the authors (Wood et takes advantage of the environmentally responsive al. 2016) and others (Grönquist 2019; Abdelmohsen hygroscopic properties of wood, developing aper- et al. 2019). tures that open and close in response to changing humidity levels. The façade uses the anisotropic and Plant Biomechanical Principles hygroscopic properties of wood to create bilayer ac- The most prominent feature of the Oxalis triangularis tuators (Correa et al., 2013). Similarly to the Hy- is its leaves. They come in threes, radially arrayed at groSkin project by Correa et al. (2013), HygroScope the end of a long stalk. The leaves have a distinct by Menges et al. (2014) manipulates the surface of triangular shape and are a deep, slightly variegated wood veneer composite to create the opening and purple. These leaves are attached at the pulvinus (the closing of the façade. The use of the material itself bulb at which the leaves attach) to the stalks. The Ox- for the actuation provides design simplicity and re- alis achieves movement in its leaves in response to duces the number of mechanical parts needed. Ide- sunlight through changes in turgor pressure in the ally, fewer parts will also translate to lower mainte- pulvinus (Minorsky, 2019; Kumke, 1981). The leaf nance costs, thus financially benefitting the owner of movement is shown in Figure 1. The turgor pressure the building. It is this wood bilayer actuator strategy changes the shape of the pulvinus which creates two that will be used in the presented research. amplified movements: the raising or lowering of the leaf as a whole and the folding of each leaf in half. HYPOTHESIS Making the small displacement of the pulvinus into a A hygroscopic façade mechanism is presented and more exuberant unfolding. tested using physical prototypes. The goal was to de- The Hardy Water Lily (Nymphaeaceae) is well sign a unit that can change based on local humidity known for its opening and closing in response to conditions with minimal human or electromechani- sunlight and temperature cues, seen in Figure 2 (van cal input. Relative Humidity is used as the driver of Doorn and Uulke, 2003). Flower opening is driven

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 465 Figure 1 Oxalis leaf movement in response to light.

Figure 2 Lily petal movement in response to light.

by internal changes, both cell elongation and divi- of branches in different kinds of plants (Rodkaew et sion (Schleicher et al., 2015). The timing of the open- al., 2004; Leitner et al., 2010; Boudon et al., 2012). ing due to cell elongation is due to a variety of fac- Although not used to mathematically optimize this tor such as temperature, quantity and quality of light, façade, the L-System is used as a starting point in and the duration of light and darkness (van Doorn the distribution of the tensile cable structure for the and Uulke, 2003). The inner surface of the petal elon- façade. As plants are extremely efficient in their gates rapidly when temperatures increase (opening growth to survive, the idea of efficiency in the struc- the flower in the morning), while cooling results in a ture to hold the prototype was also considered. The reversal of the cell elongation (closing the flower at 60-degree intersection angle (Figure 3) of the tensile night) (van Doorn and Uulke, 2003). cable structure was chosen as it follows a logic such L-Systems (Lindenmayer systems) are mathe- as those found in the branching patterns. While the matical models that predicts the efficient and opti- units were developed within full biomimicry param- mal growth patterns for roots and branches in plants eters, the structure supporting the units has a bio- (Rodkaew et al., 2004; Leitner et al., 2010; Boudon inspired approach, which is an approach based on et al., 2012). The mathematical model incorporates observation of biological systems rather than trans- length, size, position, and spacing of the biologi- lating biological function into a viable model (Cruz et cal plant elements to accurately predicts the angles al., 2021).

466 | eCAADe 39 - Structure optimization and material-based design - Volume 2 RESULTS content during lamination. While the grain of veneer Prototype Introduction layers in plywood is laid cross-wise to achieve dimen- The autonomous petal-like structure uses two cou- sional stability and to reduce anisotropy, for this ap- pled bilayer mechanisms that allow the petal to si- plication anisotropy is the desired effect. Previous lit- multaneously bend and fold. The folding follows the erature on wood bilayers indicates that the lamina- amplification principle of the Oxalis, while the petal tion should take place cross-wise (Wood et al. 2016; shape-change mimics the cellular elongation of the Holstov et al. 2015; Rüggeberg and Burgert 2015). Water Lily. The two actuations combined allow for an However, for this application we found that the small increased range of motion for each petal. The petals scale of our bilayer strips made it necessary to lami- are radially arrayed around a central hub, with three nate the layers with the grains parallel. This parallel “petals” on each side, to form a flower-like compo- lamination allowed for an increased radius of curva- nent. ture when compared to the perpendicular grain ap- proach. It is speculated that this parallel lamination Figure 3 reduced the flexural stiffness caused by the passive Axonometric view layer’s grain. of unit (top). Shape In addition to the bilayer shape-change actua- of the petals tors, each petal has a non-hygroscopic surface com- (bottom). ponent referred here as the “petal”. Each individual petal is a cut-out piece of semi-translucent 0.1mm white cellulose acetate film cut to shape, seen in Fig- ure 3. The shape is largely reminiscent of the petals of a Water Lily, but with slight changes to allow for a bending movement: there is a slit coming from the base of the petal in which one bilayer strip is inserted, and a circular cutout in the centre to avoid splitting of the material due to this slit, also seen in Figure 3. A connection piece was 3D printed in order to be able to connect the two moving bi-layers together without limiting their range of motion. This connec- tion point sits parallel to the wood grain of the bilayer to maximize the amount of physical contact, seen in Figure 4.

Figure 4 Connection piece between bilayer strips (left). Petal connection piece (right). Fabrication Each petal of the unit uses two small bilayer strips. The bilayer material is composed of a layer of beech veneer and a layer of teak veneer. Both layers have the same grain direction. A non-water soluble glue is used to avoid changes in the material’s moisture

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 467 Figure 5 Movement of the prototype according to Relative Humidity.

Another 3D-printed connection piece was public use buildings. Figure 5 presents the different printed to hold the six petals in place, seen in Fig- kinematic states of the prototype during changes in ure 4. This connection piece is hexagonal to allow Relative Humidity (high and low). for a set of three radially-arrayed petals to be held To assess the potential for the façade system to towards the exterior of the façade, and another set of operate in North America, we have evaluated the cli- three radially-arrayed petals to be held towards the matic fluctuation within a determined region. In Fig- interior of the façade at an offset angle. ure 6, the data shows temperature and humidity fluc- The connection piece (Figure 4) consists of three tuations in the Waterloo Region (Ontario, Canada) parts: a central piece through which the steel cables in June and January. There are four data points for of the support system run, and two end caps to se- each day, averaging out six hours’ worth of humidity cure the petals into place. The central piece is thicker, fluctuations. The Relative Humidity has a much big- to leave space for two cables to pass through it, with ger range of fluctuation during the summer months a 60-degree angle between them. The two end caps (from June to August) than it does during the winter are the same: they consist of a flat hexagonal plate months (from December to February). Relative Hu- the same size as the central piece with an additional, midity tends to drop during the day and rises back raised, smaller hexagonal plate on one side. Offset up at night. Taking the average of each of the four from each edge of the smaller plate are two small sections of the day, graphed onto an average daily pegs, which are used to provide additional security graph, shows how humidity affects the module pro- to the petal attachments. The end caps are glued totype (Figure 7). to the central piece, pegs facing inwards. The point The application of this project was explored by at which a petal attaches to the central connection looking at a colder climate. In the winter, when the piece is one end of the bilayer strip. The two holes Relative Humidity does not fluctuate as much and at the end of this strip fit onto the connection piece’s stays high, the petals would generally stay closed, let- pegs. ting in lots of needed winter sunlight. In the sum- The façade responds to humidity changes in the mer, the petals would generally stay open and pro- weather with minimal human intervention. The abil- vide shade due to the dry air during the hottest parts ity for the system to change shape on its own is an of the day. As shown in Figure 7, there is significant adaptive feature that is useful for public spaces and seasonal and daily fluctuation in this climatic zone for

468 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 6 Monthly humidity fluctuation in winter vs. summer 2020 in Waterloo Region.

the device to have an impact in the solar gains of a of approximately 8%. The façade system is likely to space. The shading system is also an engaging visual deteriorate when directly exposed to precipitation or indicator of Relative Humidity, which cannot other- rain and is therefore most suitable for applications wise be seen. It is speculated that the mechanism can within a double skin glass façade environment. be tailored to address specific geographic regions Potential implementation of the adaptive façade and their distinct seasonal changes. People spend at within a building is shown in Figure 8. The arrange- least 80% of their time indoors, which means interior ment of units on the façade was done with the com- spaces have a huge impact on their psyche (Chan- fort of the occupants in mind. Since shading is not a non, 2018). This façade allows for lots of natural light priority below waist level, smaller units - which shade in a controlled manner and reminds the building oc- less - were placed near the bottom. Similarly, since cupants of nature with its flower petal design. These light at the top of an atrium space typically hits the are facets of good design for occupant pleasure and walls rather than the occupied space, smaller units comfort, as noted in Happy by Design by Ben Chan- were also placed there. Lastly, the spacing of the ten- non (Channon, 2018). sile structure varies according to the size of the units The bio-inspired principles give the architectural to account for the weight of the structure. This dis- design feature the ability to modulate solar exposure tribution approach is inspired by L-systems, as plant within daily and seasonal cycles. As seen in Figure 8, root systems space themselves out according to nu- the maximum shaded area the petals provide is ap- trient needs (Leitner et al., 2010). proximately 46%, with a minimum shaded coverage

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 469 We hope to be able to further extend the number Figure 7 of parameters that this façade mechanism can ad- Average humidity dress and therefore be able to expand its function- during the day in ality. Furthermore, a better understanding of the January and June performance of the façade under multiple conditions 2020 in Waterloo will be critical. Extensive testing and validations are Region. goals for further research; they will allow a precise quantitative analysis of the façade on a large scale.

Figure 8 Exterior render of the façade (top). Shading study (bottom).

DISCUSSION The materials used in the presented prototype are very light weight and therefore fragile. In its current configuration, the petals must be light enough for the veneer strips to lift them; this causes a degree of fragility that requires the façade modules to be shielded from high winds or precipitation. Further applications of this system could include vents that could operated by occupants to achieve a more cus- tomized range of motion. There is, however, poten- tial for the use of other lightweight materials for the petals, as it is the shape and the ability to flex that is important, not the specific material. There is also the mechanical vulnerability of the parallel lamination of Lastly, while the ability of the petal to flex is integral the bilayer material. It is possible that larger versions to the movement of the façade, the exact geometric of the unit may be able to use larger scale wooden bi- features of the petal shape are a potential area for fur- layers or hybrid metal and wood composites (Abdel- ther exploration. Different petal shapes could impact mohsen et al. 2019); further material testing will be the effectiveness of the shading, as well as change needed in order to upscale the mechanism. Similar to the aesthetic qualities of the façade as a whole. This other bio-inspired technical applications, the mecha- is something that could be adjusted according to a nisms are designed to target a very limited set of per- specific building’s needs, both practical and visual. formance criteria; only about 13.4% of biomimetic building skins are designed to address multiple inter- CONCLUSION dependent parameters (Cruz et al., 2021). The pre- Wood bilayers, while they have have been explored sented work is part of a small group of adaptive fa- in various architectural applications, still have oppor- cades that aims to address both visual interest and tunities for further novel development. Particularly, climate adaptation as intrinsic performance criteria. there is considerable room for exploration with us-

470 | eCAADe 39 - Structure optimization and material-based design - Volume 2 ing multiple integrated pieces of bilayer material to work for Modeling Plant Architecture Development actuate a mechanism, such as a façade system com- Based on a Dynamic Language’, Frontiers in Plant Sci- ponent. ence, 3(10.3389/fpls.2012.00076), pp. 1-20 Brand, SB 1994, How Buildings Learn: What Happens After The façade units resemble flowers, alluding to They’re Built, Viking Press their origins, and are thus aesthetically pleasing to Channon, B 2018, Happy by Design: A Guide to Architec- occupants. The façade requires minimal human in- ture and Mental Wellbeing, Riba Publishing, London put while contributing to a visually engaging en- Correa, D, Krieg, OD, Menges, A, Reichert, S and Rinder- vironment, a key factor in greater occupant com- spacher, K 2013 ’Hygroskin: A Climate-Responsive fort (Minucciani and Sağlar Onay, 2020; Holstov et Prototype Projected Based on the Elastic and Hygro- scopic Properties of Wood’, ACADIA 2013: Adaptive al., 2015). Similar to the bio-inspired models, the Architecture, Waterloo, Canada, pp. 33-42 prototype unit reacts to the daily cycle, as Relative Correa, D, Poppinga, S, Mylo, MD, Westermeier, AS, Humidity fluctuates throughout the day correlating Bruchmann, B, Menges, A and Speck, T 2020, ’4D with sunlight exposure. The presented prototype pine scale: biomimetic 4D printed autonomous achieves the goals set out in the hypothesis: a sun scale and flap structures capable of multi-phase shading device designed and derived from biological movement’, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sci- principles, that requires minimal human input. We ences, 378, p. 20190445 further expect that the aesthetic and functional fea- Cruz, EC, Hubert, TH, Chancoco, GC, Chayaamor- tures of the bio-inspired mechanism will promote a Heil, NCH, Cornette, RC, Badarnah, LB, Raskin, visual awareness between our built environment and KR and Aujard, FA 2021, ’Design processes and environmental conditions. multi-regulation of biomimetic building skins: A comparative analysis’, Energy and Buildings, 246(https://doi.org/10.1016/j.enbuild.2021.111034), REFERENCES pp. 1-16 Abdelmohsen, S, Adriaenssens, S, El-Dabaa, R, Gabriele, Dawson, C, Vincent, JFV and Rocca, AM 1997, ’How pine S, Olivieri, L and Teresi, L 2019, ’A multi-physics ap- cones open’, Nature, 390, pp. 668-668 proach for modeling hygroscopic behavior in wood van Doorn, WG and van Meeteren, U 2003, ’Flower Open- low-tech architectural adaptive systems’, Computer- ing and Closure: A Review’, Journal of Experimental Aided Design, 106, pp. 43-53 Botany, 54(10.1093/jxb/erg213), pp. 1801-1812 Addington, MDA and Schodek, DLS 2010, Smart materi- Eder, ME, Schaffner, WS, Burgert, IB and fRATZL, PF 2020, als and new technologies for: for the architecture and ’Wood and the Activity of Dead Tissue’, Advanced design prefessions, Architectural Press, Amsterdam; Materials, -(10.1002/adma.202001412), p. 2001412 Boston Elbaum, R, Zaltzman, L, Burgert, I and Fratzl, P 2007, ’The Andaloro, AA, Mazzucchelli, ESM, Lucchini, AL and Pede- Role of Wheat Awns in the Seed Dispersal Unit’, Sci- ferri, MPP 2017, ’Photocatalytic self cleaning coat- ence, 316, pp. 884-886 ings for building facade maintenance. Performance Elices, M (eds) 2000, Structural biological materials: de- analysis through a case-study application’, Journal sign and structure-property relationships, New York : of Facade Design and Engineering, 4(10.3233/FDE- Pergamon, Amsterdam 160054), pp. 115-129 Forterre, Y 2013, ’Slow, fast and furious: understanding Baldwin, JB 1997, BuckyWorks: Buckminister Fuller’s Ideas the physics of plant movements’, Journal of Experi- for Today, John Wiley & Sons mental Botany, 64, pp. 4745-4760 Becker, SB, Peter, MP, Fritsch, DF, Philipp, DP, Baier, Sydney Gladman, A, Matsumoto, EA, Nuzzo, RG, Ma- PB and Dibak, CD 2013, ’Combined Grammar hadevan, L and Lewis, JA 2016, ’Biomimetic 4D print- for the Modelling of Building Interiors’, Inter- ing’, Nature Materials, 15, pp. 413-418 national Archives of the Photogrammetry, Re- Grönquist, P, Wood, D, Hassani, MM, Wittel, FK, Menges, mote Sensing and Spatial Information Sciences, II- A and Rüggeberg, M 2019, ’Analysis of hygroscopic 4/W1(10.5194/isprsannals-II-4-W1-1-2013), pp. 1-6 self-shaping wood at large scale for curved mass Boudon, F, Pradal, C, Cokelaer, T, Prusinkiewicz, P and timber structures’, Science Advances, 5(9), pp. 1-7 Godin, C 2012, ’L-Py: An L-System Simulation Frame- Holstov, A, Bridgens, B and Farmer, G 2015, ’Hygro-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 471 morphic materials for sustainable responsive archi- elling of facades’, ACM Transactions on Graphics, tecture’, Construction and Building Materials, 98, pp. 26(10.1145/1276377.1276484), p. 85 570-582 Peter, MP 2017, ’Modelling of Indoor Environments Knippers, J and Speck, T 2012, ’Bioinspiration & Using Lindenmayer Systems’, International Archives Biomimetics’, Bioinspiration & Biomimetics, 7, p. of the Photogrammetry, Remote Sensing and Spa- 015002 tial Information Sciences, XLII-2/W7(10.5194/isprs- Kumke, JMK 1981, ’Oxalis leaf movements driven by two archives-XLII-2-W7-385-2017), pp. 385-390 separate systems’, Journal of Interdisciplinary Cycle Poppinga, S, Correa, D, Bruchmann, B, Menges, A and Research, 12(10.1080/09291018109359728), pp. 87- Speck, T 2020, ’Plant Movements as Concept Gen- 96 erators for the Development of Biomimetic Compli- Leitner, D, Klepsch, S, Knieß, A and Schnepf, A 2010, ant Mechanisms’, Integrative and Comparative Biol- ’The algorithmic beauty of plant roots – an L-System ogy, 60, pp. 886-895 model for dynamic root growth simulation’, Math- Reyssat, E and Mahadevan, L 2009, ’Hygromorphs: from ematical and Computer Modelling of Dynamical Sys- pine cones to biomimetic bilayers’, Journal of The tems, 16(10.1080/13873954.2010.491360), pp. 575- Royal Society Interface, 6, pp. 951-957 587 Rodkaew, Y, Chuai-Aree, S, Siripant, S, Lursinsap, C Lienhard, JL, Schleicher, SS and Knippers, JK 2015, Bio- and Chongstitvatana, P 2004, ’Animating plant inspired, Flexible Structures and Materials, Springer growth in L-system by parametric functional sym- International Publishing bols’, International Journal of Intelligent Systems, López, M, Rubio, R, Martín, S and Croxford, B 2017, ’How 19(10.1002/int.10147), pp. 9-23 plants inspire façades. From plants to architecture: Ruggeberg, MR, Burgert, IB and Pisignano, DP 2015, ’Bio- Biomimetic principles for the development of adap- Inspired Wooden Actuators for Large Scale Applica- tive architectural envelopes’, Renewable and Sustain- tions’, PLOS ONE, 10(10.1371/journal.pone.0120718), able Energy Reviews, 67, pp. 692-703 pp. 1-16 Madureira, SM, Flores-Colen, IFC, de Brito, JDB and Rüggeberg, M, Burgert, I and Pisignano, D 2015, ’Bio- Pereira, CP 2017, ’Maintenance planning of facades Inspired Wooden Actuators for Large Scale Applica- in current buildings’, Construction and Building Ma- tions’, PLOS ONE, 10, p. e0120718 terials, 147(10.1016/j.conbuildmat.2017.04.195), pp. Schleicher, SS, Lienhard, JL, Poppinga, SP, Speck, TS 790-802 and Knippers, JK 2015, ’A methodology fr trans- Martone, PT, Boller, M, Burgert, I, Dumais, J, Edwards, ferring principles of plant movements to elas- J, Mach, K, Rowe, N, Rueggeberg, M, Seidel, R and tic systems in architecture’, Computer-Aided De- Speck, T 2010, ’Mechanics without Muscle: Biome- sign, 60(https://doi.org/10.1016/j.cad.2014.01.005), chanical Inspiration from the Plant World’, Integra- pp. 105-117 tive and Comparative Biology, 50, pp. 888-907 Stahlberg, R, Bar-Cohen, Y and Taya, M 2006 ’What can Meagher, MM 2014, ’Responsive Architecture and we learn from nastic plant structures? The phy- the Problem of Obsolescence’, International tomimetic potentiality of nastic structures’, Smart Journal of Architectural Research: ArchNet-IJAR, Structures and Materials, San Diego, CA, p. 616802 8(10.26687/archnet-ijar.v8i3.498), p. 95 Wagner, MAW, Huang, JLH, Okle, PO, Paik, JP and Spole- Menges, AM, Reichert, SR and Krieg, ODK 2014, ’Mete- nak, RS 2006, ’Hinges for origami-inspired structures orosensitive Architectures’, in Kretzer, M and Hoves- by multimaterial additive manufacturing’, Materi- tadt, L (eds) 2014, ALIVE: Advancements in Adaptive als & Design, 191(10.1016/j.matdes.2020.108643), p. Architecture, Birkhauser 108643 Minorsky, PVM 2019, ’The functions of foliar nyctinasty: Wood, DM, Correa, D, Krieg, OD and Menges, A 2016, a review and hypothesis: Functions of foliar nycti- ’Material computation—4D timber construction: nasty’, Biological Reviews, 94(10.1111/brv.12444), Towards building-scale hygroscopic actuated, self- pp. 216-229 constructing timber surfaces’, International Journal Minucciani, V and Sağlar Onay, N 2020, Well-being de- of Architectural Computing, 14, pp. 49-62 sign and frameworks for interior space, Information [1] https://www.imarabe.org/en/architecture Science Reference Muller, PM, Zeng, GZ, Wonka, PW and Van Gool, LVG 2007, ’Image-based procedural mod-

472 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Configured Knitting

Grafting as an assembly process for knitted architecture

Jane Scott1, Elizabeth Gaston2, Armand Agraviador3 1,3Newcastle University 2Northumbria University 1,3{jane.scott|Armand.agraviador}@newcastle.ac.uk 2Elizabeth.gaston@northumbria. ac.uk

There is a growing interest in knit as a material system for architectural research in a workflow that integrates computation and digital fabrication in the design and specification of highly engineered fabrics. However, the dimensional limitations of industrial machines mean that large scale work may require assembly from multiple pieces. Reconfiguring knitted fabric by joining fabric panels disrupts the performance of the material, challenging the computational model when fabric characteristics are transformed at the seams.The aim of this research is to evaluate the potential for grafting, a traditional joining method for knitted fabric, as an assembly technique for architectural scale knitted prototypes. The paper presents an overview of knitted loop geometry focusing on the impact of loop construction in textile joins. The paper presents experimental research conducted using unconventional off-machine techniques at two scales, demonstrating how grafting can be used to assemble 3D structures without compromising the integrity of the material. Findings highlight the significance of this technique and suggest how the work could translate to digital fabrication.

Keywords: Knit, Grafting, Computational Form Generation, Textile Design

INTRODUCTION performance of the knitted material throughout an This research emerged from an investigation into the architectural form. design and fabrication of complex knitted forms con- This paper presents an overview of the design structed at a building scale. The project brings to- methodology, one that integrates textiles sampling, gether researchers working in textile design, specif- form-finding and large-scale prototyping. At the ically knitted fabric specialists, with computational first stage of this research knit preforms (branch- design, and architecture. The aim of the research is ing structures composed of multiple knitted tubes) to reposition the design challenge of knitted archi- are defined through knit sampling. Computational tecture from a textile perspective, focusing on the de- form finding is used to generate a branching dome sign opportunity afforded by joining processes used topology integrating multiple knitted preforms. This within traditional knitwear in order to maintain the topology is explored at both a sample size and as

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 473 an experimental architectural prototype. The re- potential of knit-tube configurations and prototype search challenge is to find a suitable joining mech- fabrication. The prototype was fabricated off ma- anism to assemble the knit preforms that preserves chine, using unstructured yarns and freehand loop the integrity and performance of the fabric. The re- construction processes. The findings of this research search undertaken for this paper evaluates grafting, are significant to the role of knit in architectural re- one method of assembling knitted fabrics used in the search, particularly in the shift from interior to exte- knitwear industry, applying the findings to architec- rior, and temporary to permanent applications. tural prototypes. The research question asks how can The following sections discusses the technical as- graftingbeadaptedasajoiningtechnologyformultiple pects of textile joining processes specifically tailored textile components in architecture? Results describe to weft knit assembly. the benefits of grafting as a joining process for archi- tectural applications. KNITTED LOOP GEOMETRY The basic production and structure of a weft knitted BACKGROUND fabric is described in Albaugh et al (2019). In this pa- The interest in knit as an architectural material has per the horizontal rows of loops are termed courses expanded over the last decade as international re- and the vertical rows of loops are termed wales, fol- search groups focus on integrating computation, ma- lowing technical knit terminology (Spencer 1989). terial performance and digital fabrication using in- The qualities of knitted fabric that make it attrac- dustrial knitting technologies (Ahlquist 2016, Bara- tive as a material are also those that pose challenges novskaya et al 2016, Gengnagel et al 2018, Lui et in accurate modelling. These qualities include low di- al 2019, Popescu et al 2019, Sabin and Jones 2017, mensional stability, the specification of fabric struc- Scott 2013). One challenge of working with knit at ture at individual loop level through varying stitch an architectural scale is the dimensional limitations type and the ability to produce multi-material fab- of fabrics manufactured using industrial machines rics. Early work on modelling knitted fabric dimen- with a needle bed width of 1.85m. Large scale work sions (Chamberlain 1951, Doyle and Hurd 1953, Leaf can be assembled from multiple pieces, however re- and Glaskin 1955, Munden 1959) assumes stability of configuring knitted fabric by joining fabric panels material use, whereas later work acknowledges the compromises the performance of the material, lim- effect of material use on the mechanical properties iting multidirectional extensibility and the inherent of fabric (Choi and Lo 2006, Liu et al 2017). All how- strength the material (Brackenbury, 1992). Signifi- ever recognise that the single unit of a knitted fabric cantly, the challenge of computational modelling of is a loop, and loop length is the fundamental unit for knitted fabric is exacerbated when fabric characteris- fabric dimension. Loop length is controlled during tics are transformed at the joins. knitting by precise control of the needles during loop formation but also by control of the tension of the DESIGN CONCEPT yarn that is supplied to the needle and the take down To address the gap in knowledge that relates to join- tension applied to the fabric once knitted (Spencer ing processes for knitted architecture a project was 1989). designed to explore seamless joining processes used The loop construction of a knitted fabric pro- in the construction of an experimental knit architec- duces inherent, multi-directional extension through ture based on an intersecting tubular topography. loop deformation under tension produced from low The parameters explored in this paper include: gen- loads (Munden 1959). Plain knit is anisotropic (figure erative form-finding based on geometric constraints, 1) and when extended in one direction knitted fabrics knit sampling to explore the formal and structural decrease dimensionally in the perpendicular direc-

474 | eCAADe 39 - Structure optimization and material-based design - Volume 2 tion (Albaugh et al 2019); under multi-dimensional dimensional differences in varying stitch structures force forms a double curved form. Theoretical ex- (Liu et al 2017). tensions are limited in practice by the diameter of the yarn used and the friction resistance incurred be- TEXTILE JOINS tween yarns at the points of contact between loops Whilst complex form can be produced in knit with- (Choi and Lo 2006). During loop extension the loop out seaming, additional joins extend the shape lexi- length remains constant whereas other methods for con significantly. This paper focusses on joins at the specifying the dimensions of a knitted fabric such as top and bottom of knitted fabrics. The joining of tex- courses/cm and wales/cm, stitch density or tightness tiles can be categorised into four classes, mechani- factor, all commonly used in the knitwear industry, cal, thermal, solvent and adhesive all of which influ- are adversely affected by loop deformation (Leaf and ence the characteristics of the fabrics being joined Glaskin 1955 Munden 1959). (Busiliné et al 2017). A successful seam is strong and Despite stability of loop length, fabric dimen- secure with no grinning between elements, produc- sions are also affected by loop shape (Leaf and ing a join that has dimensional stability (Jones and Glaskin 1955 Munden 1959 Leaf 1960 Spencer 1989). Stylios 2013). Knitwear is conventionally joined with In a fully relaxed state of minimum energy, loop stitched joins where one or more threads interlaces shape is a product of loop geometry and not physi- between two or more knitted components. To pre- cal properties of yarn or loop length (Munden, 1959). serve the extensibility of a knitted fabric, the interlac- Each loop is three dimensional and bends behind ing includes looped elements to increase the exten- and in front of loops of the previous and subsequent sibility of the seam (Brackenbury 1992) courses (Leaf 1960). The sinusoidal elastica, forming A further consideration in seaming knit fabrics is the head of the loop, is rotationally symmetrical to the quality of fabric edge. An unsecured edge of a the sinker loop (ibid), benefitting ease of modelling. knitted fabric will quickly deteriorate under any ap- plied stress. Increasingly machine knit fabrics are Figure 1 completed with a binding off process that replicates Knit Geometry. Left: casting off in hand knit and secures the loops. Fab- 3D Profile of Plain rics may be bound off separately then joined by other Knit, Centre methods or may be bound off together, joining and Vertical/Wale-wise finishing the fabrics simultaneously. In both cases extension. When However, in production and fabric usage, loop shape this method of finishing the knitted fabric can pro- stretched vertically is additionally a product of tensions within the loop; duce restriction of lateral extension (Choi et al, 2015). a loop theoretically tensile energy (external loads applied to the fabric), Un-cut but un-secured knit fabrics are conven- extends to half the bending energy (the bends in yarn produced during tionally joined with an intra-looped, single chain full loop length. the knitting process) and torsional energy (the result (BS100) using linking machines. Here, the seam Right: of yarn spinning and the knitting process) (Liu et al thread passes through each individual loop of two or Horizontal/Course- 2017), and yarn parameters; fibre content (which af- more fabrics in the seam. wise extension. fects surface friction and bending rigidity), spinning The loop configuration of the stitch allows When stretched processes, (which affects yarn diameter), torsional greater extensibility of the seam, which is close to the horizontally a loop rigidity, tensile modulus and yarn compression (Choi extensibility of a plain knit fabric (Tamke et al, 2018) theoretically and Lo, 2006). Whilst these multi-variables can be but is limited by the run-in ratio of the thread and the extends to half the specified during production, minor modifications in thickness and compressibility of the fabric. Reduced full loop length the individual loop length may have a large effect on stitches/cm and thread tension will increase the run- (adapted from fabric quality (ibid) resulting in inaccurate computa- in ratio of the seam (length of thread absorbed by the Spencer 1989). tional modelling capability. This is compounded by

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 475 seam: length of seam) and therefore the extensibility GRAFTED JOINS of the seam but this will reduce seam integrity (Choi In a grafted knitted seam, the joining thread repli- et al 2015). cates the structure of the knitted course exactly (Van Linked seams produced with inter-looped dou- Zandt 2006), ensuring no variation in the perfor- ble chain stitch configurations (BS 400) are produced mance of the fabric and no bulk (figure2). This is ben- from two threads, a needle thread and a looper eficial in both fabric functionality and accuracy of fab- thread. This class of seams are more robust than ric modelling. The additive, grafted course can repli- single chains but extensibility are further reduced cate any fabric structure increasing its versatility. (Brackenbury, 1992). The most extensible of the conventional stitched Figure 2 seams are the BS 500 class overlocked seams where Tubular knitted the configuration of three or more threads closely re- samples joined sembles a plain weft knit construction, with the loops using a grafted of each thread resembling a knitted course (ibid). seam. Position of This allows the seam to be theoretically extended to grafted joins does the total length of the needle thread. In practice this not affect Currently grafting is only achievable as a hand pro- is limited by the stitch density and thickness/com- behaviour of seam cess and is not discussed in academic, technical lit- pressibility of the fabrics. A benefit of an overlocked under load. erature however it is widely understood by the hand seam is that the two looper threads act as a cover to Elongation of seam knit craft community (Chamberlain 1952 Lee 2007). a cut fabric edge and so joining and sealing two fab- mirrors elongation An expert hand knitter will tacitly recognise the ef- rics in one action (Jones and Stylios 2013). This how- of fabric. fects of loop geometry and variation in stitch struc- ever produces a bulky seam. Four or five-thread over- ture on the dimensions and properties of a knitted locked seams, in which additional needle threads in- fabric. This knowledge, and the common use of 3D teract with the looper threads, produce more secure production make hand knit a responsive and adapt- seams but reduce the seam extensibility (Uçar 2002). able process for investigative prototyping (Gaston The overlocked seam adds the most bulk to the fabric and Scott, 2020). and can produce less precise seams. In most cases of mechanically stitched seams, to SPECIFICATION OF GRAFTED SEAM ensure seam integrity the thread run-in is lower than A series of knitted tubes joined at different positions the course length that has been stitched and so the along the length of the fabric were analysed to test course-wise extensibility of the fabric is restricted at the performance of a grafted seam. Fig 3 illustrates the seam (Brackenbury 1992). that when force is applied the fabrics respond in the Technical developments in seam technology for same way, regardless of the position of the join, and knitted architecture include extensive work by Jenny that the grafted fabrics can withstand the same force Sabin Studio and Dazian who have developed a as a knitted fabric with no join. folded seam system integrating knitted tubes into woven nylon webbing (Sabin et al 2018). In their Figure 3 recent project Lumen, winner of MoMA and MOMA Tubular knitted PS1’s 2017 Young Architects Program the canopy de- samples joined sign relies on the woven nylon net’s ability to with- using a grafted stand the primary tension forces. seam. Position of grafted joins does not affect behaviour of seam under load. Elongation of seam mirrors elongation 476 | eCAADe 39 - Structure optimization and material-based design - Volume 2 of fabric. Table 1 Generative form-finding with branching structure topology.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 477 For knitted architecture this is significant both for the Table 2 modelling of structures, and in fabrication where it Technical is necessary for seams to not impact the multidirec- Specification of tional stretch of the fabric under tension. Knitted Preforms

MATERIALS AND METHODS DESIGN CONCEPT To address the gap in knowledge that relates to join- ing processes for knitted architecture a project was designed to explore seamless joining processes used in the construction of an experimental knit architec- ture based on an intersecting tubular topography. The parameters explored in this paper include: gen- erative form-finding based on geometric constraints, knit sampling to explore the formal and structural potential of knit-tube configurations and prototype fabrication. The prototype was fabricated off ma- chine, using unstructured yarns and freehand loop construction processes.The findings of this research are significant to the role of knit in architectural re- DOME BRANCHING TOPOLOGY search, suggesting a method for knit construction To explore how the branching structures devel- that retains fabric performance across multiple fab- oped as knit preform nodes could be configured as ric components. This could be particularly important an architectural prototype a generative form find- in the shift from interior to exterior, and temporary to ing exercise was undertaken to generate branching permanent applications. dome topologies (Table 2). The visual programming through Grasshopper defines a sequential script in- KNITTED PREFORM NODE DESIGN cluding the following logic. A series of latitudinal A series of three initial knitted preforms were pro- “strata” are defined from origin point (0,0,0) using duced using hand-knit processes. The preforms in- diameters defined by minimum and maximum val- clude a high level of structural integrity achieved by ues. A series of branch quantities are defined for the adapting course length and stitch length require- centre point (apex) and each stratum. Each quantity ments. The preforms are specified with full techni- dictates the number of lines intersecting at any one cal notation in table 1. What is significant here is point along the corresponding stratum. The values of that the individual tubes are intrinsic to the fabric branch quantities are translated into a series of total preform, forming the fabric into an integrated node. intersections (or “nodes”) to be distributed onto each The fabrics can be joined together at any point along stratum. Because of the branching structure from the a tubular ‘branch’. This allows huge freedom in de- apex, the number of nodes at each stratum must be sign, using the variables of tube length and num- the product of the total nodes of the preceding in- ber of branches, to generate increasing complex knit terior stratum and the desired number of outward topologies. branches at each of those nodes. All lines are collected to form the branching topology in 2D (Table 2 column 1). A 3D dome is gen- erated with the diameter of the outermost stratum.

478 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 4 The topology is extruded into a 3rd dimension and Small scale corresponding lines are created at the intersection of experimental knit extrusion and dome surface. This new network of branching lines represents the 3D topological arrangement of topology. the branching system (Table 2 column 2). Initial tests of the branching topology were ex- plored in hand knit at a small scale using knitted nodes with 8 branches grafted together with joins positioned in the middle of each branch (fig 4). Com- plex shaping was integrated into the node rather than required at the join this simplified assembly and enabled the prototype to retain fabric properties Figure 5 throughout. The grafted seam was placed centrally 2D branching on each spar. topologies developed to PROTOTYPE DESIGN maximise inflated The architectural prototype was developed using a structural integrity. large scale freehand knit construction process re- fined by the team during previous site-specific in- Figure 6 stallations including the Cocoon Series (Gaston and 3D topological Scott 2015-2019). The prototype was constructed arrangement from 20kg Cheviot British Wool, (1.5cm diameter, un- developed for twisted). structural integrity. Prototype design evolved from topology 1 (ta- ble 2) composed of 4 knitted preforms grafted to- gether along the horizontal axis, and at the apex. The branching structure was refined to maximise struc- Figure 7 tural integrity, incorporating two beams crossing at Arrangement of the apex, with 4 secondary arches (figs 5&6). This was Nodes (with constructed from 4 identical preforms grafted at the inflatable beams) top edges. Each preform was composed of a node with 3 branches 20cm diameter (8 wales) at the base and 3 branches 15cm diameter (6 wales) at the top (fig 7). During assembly inflatable beams were manually in- serted into the fabrics to enable the prototype to be self-supporting for exhibition (this aspect of the con- struction is discussed in a separate research paper). The positioning of the inflatable beams is illustrated in figure 7.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 479 Figure 8 Prototype inflated for display.

Figure 9 Prototype: detail of stitches including grafted seam and joins

Figure 10 Detail : grafting a seam (left) and illustrating knitted stitches after inflation (right)

480 | eCAADe 39 - Structure optimization and material-based design - Volume 2 RESULTS AND DISCUSSION CONCLUSIONS The prototype measures 1.5m high with a 2m diam- This interdisciplinary research adopts specialist in- eter at the base. The branching structure is sup- sight from knit design researchers to consider the ported using 2 inflatable beams crossing at the apex problem of joining knitted fabrics for architectural and 4 shorter beams producing stabilising secondary applications. To maintain the integrity of a knitted arches (fig 8). The use of freehand loop construction material grafting is examined as a traditional joining processes has enabled very clear visual analysis of the process for knitting which, despite a lack of techni- behaviour of knitted fabrics under tension with the cal and academic reference, offers a method to join beams, both within the tubular fabric itself and at the knitted fabrics seamlessly, with no disruption to the grafted seams (fig 9). In figure 9 there is no observ- structural integrity and multi-directional extension of able difference between stitches within the knitted the fabric. tube, and the stitches that make up the join. In a re- By integrating a methodology that included tex- laxed state the stitch dimensions are identical. Un- tiles sampling, form-finding and large-scale proto- der tension all stitches have distort in the same way, typing, potential applications for grafted seams have demonstrating excellent width-wise extension under been explored at a sample scale and within an ex- tension from the inflated beam. The integrated shap- perimental architecture. What is significant about ing in the design of the knitted node has enabled the the seam is that the grafted join produces no varia- inflated sections to expand, as the beams lie along- tion to either the look or the performance of the fab- side each other within the knitted node (fig 9). ric. In fact, the grafted seam is only visible in the ar- The stitch distortion observed in the prototype chitectural prototype as a minor change in colour, supports the theoretical model of stitch geome- offering a seamless appearance to the knit. Graft- try whereby multi-directional extension is observed ing is a straightforward process for prototyping off- through loop deformation under tension. Whilst the machine with unconventional materials, as demon- loop length remains constant, the stitch density is re- strated within this experimental work. However, to duced through the impact of tension via the inflat- translate this process for digital fabrication requires able beams. In addition, the loop shape is impacted further research. Digitally fabricated knitted fabrics after inflation. Figure 10 illustrates the change in the are produced with stitch densities of multiple stitches geometry of the loop after inflation, with the stitch per centimetre, and grafting must be accurate at the density in the inflated beam reduced by 40% in com- level of the individual stitch. The development of parison to the fabric in a relaxed state (stitches in a new mechanisms to support unsecured fabrics at relaxed state measured 7cm in height and 5cm in both edges of the seam during the grafting processes width, after inflation an individual stitch measured are required to enable the transition to automated 7cm in height and 8cm in width), reducing the cover processes. However, if these challenges are over- of the fabric. come, grafting offers the potential to scale knitted In the prototype the grafted knitted seam repli- fabric without disturbing the performance of the ma- cates the structure of the knitted course exactly, the terial, enabling the translation from computation to seam is composed of the same yarn (bleached for vi- fabrication without seam restrictions. sual identification) as the rest of the fabric therefore As knitted architecture shifts from temporary in- there is no variation in the look or the performance stallations into permanent applications, and from in- of the fabric (fig 7). In addition, there is no additional terior to exterior, there is a need to find robust joining bulk to the fabric as the grafted seam produces an mechanisms for this adaptable and responsive ma- addition knit course seamlessly integrated into pro- terial system. Grafting offers one such mechanism totype (fig 8). demonstrated here using large scale knitting. Fur-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 481 ther research is needed to address the application of ples and Applications, Woodhead Publishing , Cam- grafting to machine knitted fabrics, and the devel- bridge opments of new technologies that enable grafting Leaf, GAV 1060, ’Models of the Plain-Knitted Loop’, Jour- nal of the Textile Institute Transactions, 51(2), pp. 49- to become an automated part of the knit fabrication 58 process. Leaf, GAV and Glaskin, A 1955, ’The Geometry of a Plain Knitted Loop’, Journal of the Textile Institute Transac- REFERENCES tions, 46(9), pp. 587-605 Liu, D, Christe, D, Shakibajahromi, B, Knittel, C, Cas- Ahlquist, S 2016, ’Sensory Material Architectures: Con- taneda, N, Breen, D, Dion, G and Kontsos, A 2017, ’On cepts and Methodologies for Spatial Tectonics and the Role of Material Architecture in the Mechanical Tactile Responsivity in Knitted Textile Hybrid Struc- Behavior of Knitted Textiles’, International Journal of tures’, International Journal of Architectural Comput- Solids and Structures, 109, pp. 101-111 ing, 14(10), pp. 63-82 Liu, Y, Li, L and Yuan, PF 2019 ’A Computational Ap- Albaugh, L, Hudson, S and Yao, L 2019 ’Digital Fabrica- proach for Knitting 3D Composites Preforms’, In pro- tion of Soft Actuated Objects by Machine Knitting’, ceedings of CUMINCAD 2019, pp. 232-246 Proceedings of CHI 2019, Glasgow Munden, DL 1959, ’The Geometry and Dimensional Baranovskaya, Y, Prado, M, Dorstelmann, M and Menges, Properties of Plain-Knit Fabrics’, Journal of the Textile A 2016 ’Knitflatable Architecture’, Proceedings of Institute Transactions, 50(7), pp. 448-471 eCAADe 2016, FINLAND, pp. 571-580 Popescu, M, Rippmann, M, Van Mele, T and Block, P Beyer, B, Suárez, D and Palz, N 2019 ’Microbiologically 2017 ’Automated Generation of Knit Patterns for Activated Knitted Composites: Reimagining a col- Non-developable Surfaces’, Proceedings of CUMIN- umn for the 21stcentury’, Proceedings of eCAADe and CAD 2017, pp. 271-284 SIGraDi2019, Porto, pp. 541-552 Sabin, J and Lloyd Jones, P 2017, LabStudio: Design Re- Brackenbury, T 1992, Knitted Clothing Technology, Black- search Between Architecture and Biology, Routledge, well Scientific Publications, Oxford New York Busilienė, GAV, Strazdienė, E, Urbelis, V and Krauledas, S Sabin, J, Pranger, D, Blinkley, C, Strobel, K and Liu, J 2018 2017, ’The Investigation of Knitted Materials Bonded ’Lumen’, Proceedings of ACADIA 2018, Mexico City, Seams Behaviour upon Cyclical Fatigue Loading’, pp. 444-445 Materials Science, 23(2), p. 1392–1320 Scott, J 2013 ’Hierarchy in Knitted Forms: Environmen- Chamberlain, J 1951, The Principles of Machine Knitting, tally Responsive Textiles for Architecturesign in Ar- The Textile Institute, Manchester chitecture’, Proceedings of ACADIA 2013, Cambridge, Choi, W, Kim, Y and Powell, NB 12015, ’An investiga- Ont, p. 361–366 tion of seam strength and elongation of knitted- Spencer, DJ 1989, Knitting Technology second edition neck edges on complete garments by binding-off A Comprehensive Handbook and Practical guide to processes’, Journal of the Textile Institute, 106(3), pp. ModernDayPrinciplesandPractices, Pergamon Press, 334-341 Oxford Choi, KF and Lo, TY 2006, ’The Shape and Dimensions of Tamke, M, Baranovskaya, YS, Monteiro, F, Lienhard, J, La Plain Knitted Fabric: A Fabric Mechanical Model’, Tex- Magna, R and Ramsgaard Thomsen, M 2020, ’Com- tile Research Journal, 76(10), pp. 777-786 putational knit – design and fabrication systems for Doyle, PJ and Hurd, JCH 1953, ’Fundamental Aspects of textile structures with customised and graded CNC the Design of Knitted Fabrics’, Journal of the Textile knitted fabrics’, Architectural Engineering and Design Institute Transctions, 44(8), pp. 561-578 Management, 16, pp. 1752-7589 Gaston, E and Scott, J 2020, ’Colour Trans:Form:Ation the Underwood, J 2009, The design of 3D shape knitted pre- Application of Knit as Knowledge’, Journal of the Tex- forms, Ph.D. Thesis, RMIT tile Design Research and Practice, 8(2), pp. 193-208 Uçar, N 2002, ’Grinning of ISO 514 Stitched Seams on Gengnagel, C, La Magna, R, Ramsgaard Thomsen, M and Knitted Fabrics Under the Effects of Repeated Exten- Tamke, M 2018, ’Shaping hybrids – Form finding of sion and Recovery’, Textile Research Journal, 72(11), new material systems’, International Journal of Archi- pp. 944-948 tectural Computing, 16(2), pp. 91-103 Van Zandt, E 2006, The Knitters Handbook, Hamlyn, Lon- Jones, I and Stylios, GK (eds) 2013, Joining Textiles Princi- don

482 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Ceramic AM Gantry Structures

Discretisation and connections between beams and columns

João Carvalho1, Paulo J. S. Cruz2, Bruno Figueiredo3 1,2,3School of Architecture, University of Minho, Lab2PT - Landscapes, Heritage and Territory laboratory 1,2,3{joao.carvalho|pcruz|bfigueiredo}@arquitetura.uminho.pt

The manufacture of architectural components driven by digital design tools and Additive Manufacturing (AM) allows the achievement of highly evolved constructive systems, more integrated into a specific reality to which it is intended to respond, resulting in unique and adapted solutions with high geometric and material performances. Considering the application of these methods to common structural elements, namely beams and columns, for which there are already several examples demonstrating their feasibility, we find that it is necessary to provide a sound answer to an element that is fundamental for these proposals to function together as a single system - the moment of connection between beams and columns. In this sense, this paper proposes the design and test of a set of connections with adapted geometry between beams and columns, produced through ceramic Liquid Deposition Modelling (LDM), applying logics of topological optimization. This work foresees the development of a constructive system that incorporates reversible and irreversible connections, being formalised in a set of gantry structures formed by two vertical elements and a horizontal one, giving the comparative model between digital design and manufacture methods and the traditional ones.

Keywords: Ceramic AM, Performative design, Computational design, Connections, Ceramic gantry structure

1. INTRODUCTION ing production processes. The potential that AM can bring to architecture has In addition to the potential benefit regarding the been highlighted during the last few decades. Tradi- economy of means and sustainability, AM also allows tionally called 3D printing, AM consists of the produc- for mass customization, without any restriction about tion of objects by the successive addition of material the plurality of geometries that is possible to obtain. layer by layer, where it is necessary, making it a sus- Since the material is only deposited where it is neces- tainable system, in contrast to subtractive methods sary, being this process controlled by a computer, the where, generally, there is a big waste of material dur- time and cost of producing different objects will not

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 483 depend on the existence of accessory components ples of columns such as Concrete Choreography (An- (moulds), and it will take the same time to produce ton et al., 2020) or Modular Column [1] and beams either they all have the same geometry or different and bridges like WASP Reinforced Concrete Beam [2] ones. In fact, with AM the mass production of ar- or the Concrete Bridge [3] present us with architec- chitectural components with unique geometries will tural elements that explore design and manufactur- have the same cost and take the same time as the ing logics that clearly demonstrate the potential of mass production of identical copies. this process. However, these are isolated studies that Supported by computational design logics, AM consider both beams and columns independently, allowed new fields of research and development without the complementarity that exists in a real con- in architectural design, building materials and con- text. struction industry, enabling geometric freedom that The use of ceramic materials to produce ad- can result in the design of geometries with specific vanced architectural components is not yet an effec- requirements, optimizing the design and functions of tive reality and there are numerous issues that need traditional systems (Sarakinioti et al., 2020). further study. This research aims to contribute to the In 2016, a report from the World Economic Fo- mitigation of these failures and to seek solutions so rum (Renz et al., 2016) related to the future impact that the effective use of this technology/material is and likelihood of new technologies, based on the re- possible in a real context. sults of the “Future of Construction Survey”, states In this sense and based on the extensive knowl- that the “contour crafting of buildings” is unlikely edge resulting from previous research projects devel- and is considered to have a low impact on the con- oped at the Advanced Ceramics R&D Lab, this investi- struction industry. Comparatively, the probability gation proposes the development of a set of ceramic of the production of components by AM and “ad- gantry structures, with load-bearing capacity, com- vanced building materials” is considered moderate, posed by hybrid (ceramic in collaboration with other and the production of prefabricated components is materials) beams and columns. extremely high. This may justify why many research teams are 2. METHODOLOGY working in this field, using AM technologies to pro- Considering the implications associated with the duce advanced materials and components “follow- LDM of ceramic materials, namely the need to fire ing a line of thinking that is based on discrete ele- the components and the limited working volume ments”. With this logic, instead of the need for ma- of the production/firing equipment, the entire sys- chinery covering a bigger area than the area of the ac- tem was discretized into smaller elements to meet tual building, each element is discretised according these requirements. Thus, the methodology applied to project requests and existing machinery character- in this study begins by analysing the discretisation istics (Cruz et al., 2020). This idea is corroborated by of the general model into components, proceeds to Mario Carpo (Carpo, M., 2019), pointing that “instead the interconnections between each of these compo- of printing bigger and bigger monoliths, it may be nents considering the material constraints and ends conceivable easier to start with any number of parts, with the dynamic behaviour of the system when sub- as many as needed and as small as needed”. jected to load-bearing efforts. Despite that, and all the improvements on the The used methodology follows a set of rules con- use of these technologies, there is still a lack of sidered fundamental to reach relevant data for the research on a fundamental aspect of construction, definition of fully functional systems, capable to re- which is the connections between components, spond to mechanical requests, be easy to produce namely when they have a structural role. Exam- and assemble, as well environmentally friendly com-

484 | eCAADe 39 - Structure optimization and material-based design - Volume 2 paring to systems with similar behaviour. For that For the execution of the necessary additional and taking into consideration the restraints that ce- components was defined a material palette that ramic LDM have, namely geometric deformations, on could compensate the previously mentioned ce- these initial tests the system must be as simple as ramic downsides, like its fragility, and help to get the possible, both functional and geometrically, focus- better behavior of the entire system. Wood (laser ing on the definition of joints and overall structural cut oak), rubber (SBR), mortar (Sika adhesive mortar scheme. for ceramic), glue (acrylic adhesive) and concrete (C- In addition, the discretization methods that may 30), as well metallic elements for the nodes and post- vary in future conceptions are strict to simple slices of tensioning were tested. These materials are used the overall component, avoiding unnecessary com- either to separate ceramic pieces, to glue them to- plexity, that could compromise the results of prelam- gether or to give resistance to traction efforts. inar tests. The manufacturing method presented below and the computational models for the generation of 2.1. Problem definition geometries follow these material configurations and Considering the problems related to the use of ce- any change to the composition or type of material re- ramic AM on architecture and construction, that goes sults in the necessary adaptation of method and ge- from material composition to the limited built vol- ometry. ume of the production equipment, this study aims to contribute with solutions in some specific fields, try- 2.3. Fabrication ing to produce real scale high-performance ceramic LDM comprises the formation of objects from a paste gantry structures. material that is extruded on a regular or irregular ba- In this sense, and since we are dealing with ele- sis, layer by layer. The extrusion of the ceramic ma- ments at different scales (discretised component and terial is controlled by a rotating spindle that regu- full structural system), there is a need to make these lates the outflow of material. The material intake sys- two scales compatible. For that, it is important to tem uses compressed air to compact and transport study the reaction that the material has when in- the ceramic material to the extruder, where the au- creasing or decreasing the size of the components tomated spindle is located. Depending on material as well the response of the assembled systems when characteristics the equipment settings, fundamen- subjected to efforts. tally the deposition flow, speed, layer high and layer Additionally, there is a common issue condition- thickness must be changed accordingly. The produc- ing the use of ceramic materials on the construction tion equipment allows the control of each parameter of structures which is its fragile condition and the bad automatically (from the numerical control code) and behaviour when submitted to traction efforts. manually in real-time (from the machine’s control in- terface). 2.2. Material The produced ceramic components use basic The material used to produce the ceramic compo- configurations for the execution of mid-size com- nents was a fine stoneware paste (130-MP) without ponents. Extrusion thickness of 3mm and 1,5mm chamotte and with 35% of water content, as it proved height. The pressure applied to the ceramic paste to be the most suitable and compatible with the type on the cartridge was 5,0bar with a printing speed of of extruder present in the production equipment (Lu- 50mm/s. These values, namely the air pressure, have tum® 4.0XL). Tests carried out on previous studies in consideration the plasticity of the ceramic paste. show that this specific material when fired at 1260ºC can reach up to 175 MPa of compressive strength (Cruz et al., 2019).

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 485 3. PROTOTYPES Figure 1 During the investigation, several prototypes with the Cylindrical stacked objective of solving the problems mentioned above ceramic pieces and were developed. The executed elements were used separation material. to classify the materials as to their effectiveness or de- gree of complementarity in relation to ceramic mate- rials, either to serve as a separate element of ceramic components or as a traction or post-tensioning ele- ment for the built set. Prototypes to study the struc- ture of beams and columns components were also made.

3.1. Complementary materials The first set of tests was carried out having in mind the search for the best material to serve as a sepa- rating element between ceramic components, some- thing fundamental considering that ceramic is a ma- terial that, although extremely resistant to compres- sion efforts, is also very fragile, especially if the ap- plied tension is not uniform, which can cause early Three test pieces were developed for each series, ruptures, jeopardizing the integrity of the system. making a total of 51 test specimens. The main objec- For this purpose, cylindrical ceramic pieces with tive of these tests was to determine which material approximately 70mm in diameter and 40mm in is best suited to work together with ceramics having height were developed, consisting of three 3mm as a base point the results obtained by the concrete thick walls, where the inner wall rise above the re- specimens. The concrete used in the test specimens main two in height, serving as a connecting element. was a traditional C-30 mixture. Later, these pieces were associated with materials Figure 2 that could serve as separation elements (Figure 1). Complementary As previously mentioned, the inclusion of other materials test materials was tested, namely wood, rubber, mortar, specimens. glue, and concrete. Based on this aggregation, a se- ries of test specimens were developed, comprising: a) simple ceramic element; b) two stacked ceramic elements; c) three stacked ceramic elements; d) two ceramic elements stacked with a layer of separation material; e) three ceramic elements stacked with two layers of the separation material; f) two ceramic ele- ments filled inside with concrete; g) three ceramic el- ements filled inside with concrete; h) cylindrical test pieces of solid concrete; i) cylindrical hollowed con- crete test specimens (Figure 2).

486 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 3 Column prototypes. A - Structural reinforcement; B - Thermal insulation; C - Infrastructures; D - Ventilation; E - Compound stave (second type of discretization); F - Example of column assembly.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 487 3.2. Columns bution of masses and the consequent distribution of During the investigation, 5 types of columns were de- types of material, as a function of the applied forces, veloped (Figure 3), each having a different purpose is obtained using a topological optimization algo- and representing an additional performative func- rithm that after making a global analysis of the ele- tion. Although they are presented as being of differ- ments (column, beam and load), makes the reduction ent types, in their constitution and form of operation of the initial volume of the beam and presents the ar- there is no change, being the formal variation an ele- eas that will be in compression or traction. The re- ment that points to the study of the possible addition duction of mass applied to the test models was 60% of functionalities. of the initial volume of the analysed element. The columns, in addition to the structural role, which later will relate to the beam, there was also a Figure 4 demand to give these elements functions that were Beams topological not limited to the load support, as are the cases of optimization with a) structural reinforcement; b) thermal insulation; c) the type of applied passage of infrastructural networks; d) ventilation. forces. Additionally, two types of discretization were also tested: a) one component at each level; b) three components per level. Here, on columns prototypes, Figure 5 the focus was mainly on the functional issue and on Example of the the increase in the performance of ceramic architec- beam using the first tural components, leaving out the structural perfor- type of structure to mance since the responses to this performance are build the expressed by the tests presented in 3.1. components.

3.3. Beams This research points to the development of a set of beams composed by discretized ceramic elements tensioned using post-tension steel cables and ten- sioners or static elements resistant to traction efforts such as wood. Each of these types of stresses will Figure 6 result in a different materialization as the structural Shell and internal scheme varies according to the characteristics of the structure stress/aggregation elements (Figure 4). geometries for load Considering the characteristics inherent to the compression tests. ceramic material, which normally presents a good re- sponse to compression efforts and poor performance when exposed to tensile forces, care was taken to make a mass distribution that followed the structural scheme in each situation, resulting in a higher con- centration of ceramic material in areas expected to be compressed and materials resistant to traction in areas where this type of request is more prevalent. Additionally, to study the most suitable geometry for The structural scheme that results in the distri- the shell and internal structure of the ceramic staves

488 | eCAADe 39 - Structure optimization and material-based design - Volume 2 that make up the beam, three alternative possibili- ceived materializing different conditions to the struc- ties were defined (Figures 5 and 6). The first stands ture. The first type (Figure 7a) is the simplest and is out for the use of straight walls to define the contour based on the support and direct discharge of forces of the part and internal structure, and an inner cylin- (with separating material between ceramic pieces), drical channel for the passage of post-tension cables. without any type of structural blocking. The beam The second examines the use of a triple wavy con- and the column remain independent and are the tour wall that alternates between two external walls gravity and inertia of each one of them to provide the with an internal and an external wall with two inter- balance of the system. nal, forming a latticed alveolar structure. There is no The second type (Figure 7b) is characterized by internal structure in this component. The third for- the use of the tensioning elements as load transferors mat consists of the use of an alveolar corrugated wall for the column, with no direct contact between the and an internal structure defined by a triply periodic main (ceramic) body of the beam and the column. In minimal surface. this case, there is the use of a special piece on top of For each of these structural drawings, the com- the column to receive the tensioning elements and pression will be tested in two directions in order to transfers the loads. There is a structural blockage in understand the behaviour of the component in a real at least one axis of rotation. situation. These tests simulate the force exerted by The third type (Figure 7c) provides blocking in the post-tension (longitudinal direction of the beam) one direction and is characterized by the use of metal and by the applied loads (vertical direction). pieces to connect beams and columns and a wood frame on top of the column to receive the beam. 3.4. Connections between beams and Type fourth type (Figure 7d) consists of a wooden columns interconnection that enters beam and column ele- The connections between beams and columns are ments in cavities specially developed for the purpose. perhaps the most sensitive element in a frame struc- The design of the wood elements that go inside the ture like the one this research aims to develop. In beam is conceived to be deformed according to de- addition to serving as a transfer point for any loads flection and avoid stress points on ceramic compo- that are being applied to the beam, the connecting nents. element also has the task of ensuring the stability of For all previously mentioned types, the compo- the assembly by providing the blocking of rotation nents positioned next to the connections are rein- efforts in the various possible directions. forced with concrete inside. The wood material used This preponderance manifests itself mainly in is birch plywood considering its bidirectional fibers structures composed of several independent ele- and uniform composition, by opposition to natural ments, as is the case presented here, of a system com- raw materials. posed of discretized ceramic elements, in contrast to a reinforced concrete structure in which the efforts 4. RESULTS AND DISCUSSION of the joining moment can be easily diluted in the The developed system, the principles that rule it and system. In this specific case, with the use of either the set of practical results obtained during the pro- steel cables to tension the ceramic pieces, or wooden totyping of the various models, renew the conviction beams in articulation with ceramic components, the that it is possible and feasible to produce medium- tops of each of the composite elements, beams and scale ceramic architectural components and their columns, are also the moments of consolidation of subsequent application in a real context, redefining the composed element, giving extra complexity. the built environment, making it more consistent and In this sense, four structural schemes were con- environmentally friendly.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 489 Figure 7 Types of connection between beams and columns. A - first type; B - second type; C - third type; D - fourth type.

The results obtained during the compression the materials studied by specimens C2M1/C3M2 and tests presented in 3.1. clearly show the potential C2W1/C3W2, respectively mortar and wood, present of using this material in structures with bearing ca- very positive values in the way they avoid part of the pacity (Figure 8). Based on the values achieved by weaknesses of the ceramic material and can present concrete, the most used material for the construc- resistance values higher than those presented by tion of supporting structures, we realize the enor- concrete specimens. mous difference in the capacity that exists between both ceramic and concrete. Even without the in- Figure 8 troduction of elements to separate ceramic pieces, Example of the as are the cases of C2 (approx. 80mm height) and compression tests C3 (approx. 120mm height), respectively with two carried out during and three stacked components, which could reduce the research. drastically the capacity of the ceramic set, the resis- tance values (in MPa) were substantially higher than those presented by the equivalent concrete speci- mens, CM/CH80 and CM/CH120. Also, the results obtained from test pieces that used rubber (C2R1/C3R2) and acrylic glue (C2A1/C3A2), as a separating element between ce- ramic pieces, show the instability that these two ma- terials give to the system by causing accentuated deformations and misshapen settlements, leading Regarding the tests of shell geometry and inter- to total rupture of the ceramic pieces. In contrast, nal structure to be applied to each component, it is important to report the deformations presented by

490 | eCAADe 39 - Structure optimization and material-based design - Volume 2 the test pieces that use linear elements as being quite ACKNOWLEDGEMENTS expressive which, in some cases, as reported by Fig- This work was financed by the Project Lab2PT - Land- ure 9, lead to the uselessness of the component, un- scapes, Heritage and Territory laboratory, reference der pain of compromising the structural integrity of UIDB/04509/2020 through FCT - Fundação para a the system. Ciência e a Tecnologia and the FCT Doctoral Grant As future work, looking to the conclusion of the with the reference SFRH/BD/138062/2018. We are present study, we aim to carry out the remain listed also grateful to Instituto de Design de Guimarães tests, namely regarding the definition of the geome- for hosting and supporting the activities of the Ad- try of each component and effective full-scale test of vanced Ceramics R&D Lab on the use of their facilities the complete system. and equipment, and for all the assistance provided by the technician Samuel Ribeiro. Figure 9 Beam component REFERENCES exploring the use of Anton, A., Bedarf, P., Yoo, A. and Dillenburger, B. 2020, linear shell and with ’Concrete Choreography’, in Burry, J., Sabin, J., Shiel, cavities for the B. and Skavara, M. (eds) 2020, Fabricate 2020, UCL introduction of Press, London, pp. 286-293 wood connection Carpo, M. 2019, ’The Age of Computational Blutalism’, elements and in Retsin, G., Jimenez, M., Claypool, M. and Soler, V. post-tensioning (eds) 2019, Robotic Building: Architecture in the Age of Automation, Detail Business Information GmbH, Mu- steel cables. Here is nich, pp. 8-9 possible to see the Cruz, P. J. S., Camões, A., Figueiredo, B., Ribeiro, M. J. deformations after and Renault, J. 2019, ’Additive manufacturing ef- firing. fect on the mechanical behaviour of architectural

Table 1 Complementary material compression test results.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 491 stoneware bricks’, Construction & Building Materials, 238, pp. 1-17 Cruz, P. J. S., Figueiredo, B., Carvalho, J. and Campos, T. 2020, ’Additive Manufacturing of Facade Construc- tion’, Journal of Facade Design and Engineering, 8 (1) (doi.org/10.7480/jfde.2020.1.4725), pp. 1-19 Renz, A. and Solas, M. Z. 2016, ’Future of Construction Survey Results’, in Almeida, P. R., Buhler, M., Gebert, P., Castagnino, S. and Rothballer, C. (eds) 2016, In- dustry Agenda, Shaping the Future of Construction - A Breakthrough in Mindset and Technology, World Eco- nomic Forum, Switzerland: World Economic Forum, pp. 51-54 Sarakinioti, M-V., Konstantinou, T., Turrin, M., Tenpierik, M., Loonen, R., Klijn-Chevalerias, M. L. and Knaack, U. 2020, ’Development and prototyping of an inte- grated 3D-printed facade for thermal regulation in complex geometries’, Journal of Facade Design and Engineering, 6 (2), pp. 029-040 [1] http://oliviervanherpt.com/wallpaper-handmade/ [2] https://www.3dwasp.com/en/concrete-beam-creat ed-with-3d-printing/ [3] https://www.ugent.be/ea/structural-engineering/e n/research/magnel/research/concreteprinting/3dbridg e.htm

492 | eCAADe 39 - Structure optimization and material-based design - Volume 2 A Method for Generating Regular Grid Configurations on Free-From Surfaces for Structurally Sound Geodesic Gridshells

Judyta Maria Cichocka1, Szymon Loj2, Marta Magdalena Wloczyk3 1Wrocław University of Technology and Science (WRUT) 2Laboratory of Digital Fabrication (LabDigiFab) 3The Eugeniusz Geppert Academy of Art and Design in Wrocław [email protected] [email protected] [email protected]

Gridshells are highly efficient, lightweight structures which can span long distances with minimal use of material (Vassallo & Malek 2017). One of the most promising and novel categories of gridshells are bending-active (elastic) systems (Lienhard & Gengnagel 2018), which are composed of flexible members (Kuijenhoven & Hoogenboom 2012). Timber elastic gridshells can be site-sprung or sequentially erected (geodesic). While a lot of research focus is on the site-sprung ones, the methods for design of sequentially-erected geodesic gridshells remained underdeveloped (Cichocka 2020). The main objective of the paper is to introduce a method of generating regular geodesic grid patterns on free-form surfaces and to examine its applicability to design structurally feasible geodesic gridshells. We adopted differential geometry methods of generating regular bidirectional geodesic grids on free-form surfaces. Then, we compared the structural performance of the regular and the irregular grids of the same density on three free-form surfaces. The proposed method successfully produces the regular geodesic grid patterns on the free-form surfaces with varying curvature-richness. Our analysis shows that gridshells with regular grid configurations perform structurally better than those with irregular patterns. We conclude that the presented method can be readily used and can expand possibilities of application of geodesic gridshells.

Keywords: elastic timber gridshell, bending-active structure, grid configuration optimization, computational differential geometry, material-based design methodology, free-form surface, pattern, geodesic

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 493 INTRODUCTION have proposed various methods of covering the sur- In the last few years there has been a growing in- face with geodesics. There are two known methods terest in elastic timber gridshells due to their high for generating geodesic curves on a surface. One economy and possibility to achieve structurally effi- of them, which is commonly known from the defini- cient double-curved structures with the use of ini- tion of a geodesic (a geodesic curve is the shortest tially flat members. Elastic gridshells are “shell struc- path between two points lying on a surface), shows tures composed of a single- or multi-layer grid of how to create a geodesic using a start point and continuous profiles, which are initially straight and an endpoint. The second method allows to create will be progressively bent until achieving an archi- a geodesic curve with a start point and a direction tecturally and structurally satisfactory geometry” (La- vector. Both methods have been largely employed fuente Hernandez 2015). The bending process can to create geodesic patterns on form-found and free- be done simultaneously to all members in site-spung form surfaces (Harding et al. 2014; 2017; Adiels et al. gridshells, or elements can be bent one by one in the 2018; Cichocka 2020; Adiels, Ander & Williams 2017). sequentially erected systems. The site-sprung timber The design of geodesic configurations for elastic tim- gridshells have been gaining importance in recent ber gridshells has been mostly elaborated through years due to their regular grid configuration, which built prototypes (Harding et al. 2014; 2017; Adiels et is an important factor in achieving structural stabil- al. 2018; Schulitz 2017; Cichocka 2020). ity. Despite their relatively high stiffness, their erec- The literature on geodesic patterns for built pro- tion method is costly, complex and usually requires totype gridshells shows a variety of approaches for post-stiffening (Cuvilliers et al. 2017). The sequen- integrating structural performance with strict geo- tially erected elastic timber gridshells made of laths metric behaviour with flat laths made of continu- are usually called geodesic gridshells, as the geome- ous wooden strips (Adiels et al. 2018). The other try of laths is restricted to follow geodesic curves (So- researchers proposed a different method of cover- riano 2017). They can be based on the free-form sur- ing the surface with short geodesic curve segments faces, therefore they have much wider applications. joined to each other at nodes (Geodesic Segment However, the design of geodesic patterns, which pro- method) (Schulitz 2017). The design and deployment vides enough stiffness of the system remains unex- of the geodesic grid structures have been also pre- plored. sented by Pillwein et al. (2020). Authors of this pa- per assume creating a geodesic gridshell by simulta- LITERATURE REVIEW neously creating a planar grid of the structure and In the last decade the problem with generating reg- a spatial grid that approximates the design surface. ular geodesic patterns has attracted much atten- This allowed them to build a spatial form from a flat tion both from scientists researching discrete sur- grid of slats by lifting the entire grid of the model and face optimization and structural engineers. Creat- anchoring it in appropriately spaced supports. How- ing an evenly distributed and regular grid based ever, we assume cases of forms that are impossible on the geodesic curves lying on the free-form sur- to assemble from flat grids due to their complex ge- face is quite a complex problem (Pottmann et al. ometry and the intersection of curves that introduce 2010). Free-form surface is an irregular surface, which triangular fields into the grid system, preventing the can follow an asymmetric shape. It is not possi- creation of a flat grid based on a spatial grid. ble to keep the spacing equal between the curves The most interesting approaches to cover free- due to the varying curvature of the free-form sur- form surfaces with the continuous geodesics have face (it is possible only if the curvature of the sur- been proposed by Pottmann et al. (2010), who de- face is equal 0). To solve this issue, many researchers scribes methods of finding the distances between

494 | eCAADe 39 - Structure optimization and material-based design - Volume 2 geodesic curves located on a given surface and the regular grids on the free-form surfaces is a chal- shows a variety of approaches for creating geodesic lenging task (Pottmann et al. 2010). Several practical patterns. The first method of creating a geodesic questions arise when dealing with design of geodesic 1-pattern is using the Jacobi field to determine the gridshells with regular grids: 1) firstly, it is impor- distance between two adjacent geodesic curves. tant to identify an effective method for generation This method allows covering a given surface with of regular grids on free-form surfaces, 2) secondly, it geodesic curves with an offset directed by the se- is key to assess how regular grids can improve struc- lected vector field. The second method creates a tural performance in comparison to irregular ones of geodesic 1-pattern by evolution of the curve on the the same density, 3) thirdly, it is crucial to establish triangle mesh. This method also uses the Jacobi field which free-form surfaces are suitable for geodesic for offsetting the curve, but the geodesic curve is rep- gridshell design. To answer all these questions, 1) resented as a polyline and the Jacobi field is com- we present an innovative approach which allows the puted for the control points of this polyline. The third generation of regular geodesic grid configurations method uses geodesic vector fields that are repre- on free-form surfaces, 2) we apply these patterns on sented as unit vectors attached to the centres of the three free-form surfaces and compare their structural mesh faces that represent the surface. By analysing performance (max. displacement, max. initial bend- these vector fields, it is possible to select the direc- ing stress and strain energy) with irregular patterns of tions of geodesic curves and define patches of the the same density and 3) we indicate the importance surface in which the curves have a common direction. of finding structurally feasible free-form surfaces be- Summarizing, several ways to create geodesic fore application of any grid with equivalent contin- patterns have been presented in the previous stud- uum technique. ies (Adiels et al. 2018; Schulitz 2017; Pottmann et Based on the hypothesis that regular grid config- al. 2010; Adiels, Ander & Williams 2017, Pillwein et urations can structurally perform better than usually al. 2020). However, most of them do not take into applied irregular grids, this paper proposes to em- account the structural performance of surface shape ploy a mathematical method of offsetting geodesic and generated grids, when applied to the design of curves by computing vector field for a given curve, gridshells. Firstly, this paper implements methods in- which is used to find the next geodesic and which spired by Adiels, Ander & Williams (2017) to gener- allows generating regular geodesic grid configura- ate geodesic curves based on the start point and the tions for free-form surfaces. Furthermore, regular direction vector and by implementing ways of creat- geodesic grids are generated for three different free- ing geodesic patterns described by Pottmann et al. form surfaces and are evaluated with Finite Element (2010) to cover the free-form surface with the bidirec- Analysis. The results are compared with analysis of tional grid. Secondly, the method is validated in the the gridshell with irregular grids of very similar den- geodesic design application by generating regular sity to study how the regular grid distribution in- geodesic grids on three free-form surfaces to demon- fluences the structural performance. Incorporation strate the influence of grid regularity on the structural of means of discrete surface optimization for the performance of gridshell structures in comparison to design of geodesic gridshells can help in achieving irregular ones of the same density. higher bearing capacity and longer spans with mini- A hypothesis that more curvature-rich shapes mized use of materials. Consequently, the presented are much stiffer has been recently presented by Hard- method can bring novel applications of geodesic ing et al. (2017). Regularly distributed grids have gridshells in the built environment. potential to better represent the continuous (shell) behaviour of these shapes. However, generation of

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 495 METHODOLOGY ditional parameter for the {g} curve, such that for the We started by investigating a method of creating and g(s) curve there is a variation g(s, E), where g(s) = offsetting geodesic curves on any untrimmed sur- g(s, 0)(see figure 1). This allows us to control the face which, when unfolded into a plane, are straight geodesic curve offset in the g(s, 0) − g(s, 1) range. curve-like lines. This method is inspired by the so- lutions presented by Pottmann et al. (2010), which Figure 1 describes methods of finding the distances between Jacobi field geodesic curves {gi} and {gi + 1} located on a given between two surface {S}. geodesic curves. First geodesic curve: To perform the offset of the geodesic curve on the surface, we need to cre- ate the curve {g0}. In this case, it is the first geodesic curve located on the surface {S}. To create the curve {g0}, we use the point {p0} as the start point which is located on the edge of the surface {S} and the direc- tion vector {v0}, which is a vector lying on the plane {P }, tangent to the {S} at the point {p0}. The vec- Geodesic precision: For tracing a geodesic curve in tor {v0} indicates the direction of the shortest path a given direction we use a variable parameter which traced on the surface {S} from the point {p0}. The abil- describes the step between two control points of ity to rotate the direction vector gives us design free- the curve lying on the surface (the smaller step, the dom, and the direction depends on the design intent. greater accuracy of the curve). The resulting curve {g0} will be called a “base curve”. With regard to obtained curve {g0} we perform an off- Figure 2 set for the next curve {g1}. Trimmed geodesic Jacobi field: The offset is performed with the curves in place of derivative vector {v} for the parameter {s}, which is negative curvature. the length of the geodesic curve. This vector is called the Jacobi field. We assume that it is a vector perpen- dicular to the geodesic curve and is expressed by the formula (Pottmann et al. 2010): π v(s) = w(s)R (gi(s)) (1) 2 w” + Kw = 0 (2) Dealing with the curvature: It is known that the where: constant distance between geodesic curves is pos- w(s) is the derivative of the geodesic curve for sible only if the curvature of the surface is equal 0, the parameter {s}, which means that the surface is developable. In other R is the rotation angle of the tangent vector for cases, when the curvature of the surface is not equal the parameter {s} (in this case the rotation angle is 0, it is impossible to keep equal spacing between π/2 to make sure that the curve {gi + 1} is offset par- geodesic curves. To avoid the situation when curves allel to {gi}), lying on the surface intersect with each other (it hap- K is the Gaussian curvature. pens when the local curvature of the surface is nega- The curve offset method described in Geodesic tive) which would lead to unnecessary clutter in the Walks on Polytopes (Lee & Vempala 2017) uses an ad- pattern, we simply trim the curve at the intersection

496 | eCAADe 39 - Structure optimization and material-based design - Volume 2 with the first encountered curve (see figure 2). and g(s) = g(s, 0) (see figure 1). By specifying E = (0, 1) and knowing the offset length expressed as Figure 3 the length of the vector v(s), we can determine the Placing additional desired offset length. For such offset endpoints we geodesic curves in trace another geodesic curve. Created in this way, place of positive the curve {gi + 1} determines a geodesic parallel curvature. curve relative to the curve {gi}. Therefore, the tan- gent vector at the point {P i+1} at {gi + 1}(s) is the direction vector {vi+1} for the next curve traced from the point {P i+1} lying on the curve {gi + 1} (see fig- ure 4).

Figure 4 Methods of tracing In the case when curvature of the surface is positive, geodesic curves. geodesic curves move away from each other. This Left: with start and causes too much space between the geodesic curves, end points. Right: which is disadvantageous due to the regularity of the with start point and pattern and its structural performance. To preserve direction vector regularity of the pattern and to avoid creating con- Curvature-richness: Before application of the reg- preserving an equal siderable space between the geodesic curves in the ular and irregular grids, we started by investigating geodesic offset. region where the curves are moving away from each the feasibility of the proposed free-form surfaces. All other, we introduce the parameter d. This parame- shapes have the same maximum span of 5 m and the ter is the place of the point P0 lying on the curve {gi same height 2.5 m. The shapes are distinguished by +1} at the point where the curve {gi + 1} moves away the curvature richness parameter, which is calculated from the curve {gi} by too large a distance (which can based on the Gaussian curvature. The Gaussian cur- be determined earlier). vature is the combination of the principal curvatures This point is translated to the adjacent curve {gi} and is also called total curvature (Kreyszig 1991). As by the shortest path to make sure that point P 0 is we consider for this study only the change of the translated perpendicular relative to the curve {gi+1} curvature, the absolute value of the Gaussian curva- to create point P 2. For points created in this way, we ture is measured for 900 evenly distributed points calculate a point halfway between the point P 0 and of each surface to evaluate the curvature-richness of P 1 such that P b = (P 0 + P 1)/2. P b is the place the shape. where the next geodesic curve is created, and its di- Continuum Structural Analysis: To compare rection is the sum of the tangent vectors of adjacent the structural soundness of the proposed free- curves at the point P 0 and P 1 successively (see fig- form surfaces for gridshells, we employ the equiv- ure 3). alent continuum technique (Malek, Wierzbicki & Implementation: We take curve {gi} or its seg- Ochsendorf 2014). According to this method a grid- ment and use the above formula (Jacobi field) for shell can be represented by a continuous shell with its endpoints described as P s = g(0) and P e = the equivalent thickness, which volume satisfies the g(L), where L is the parameter described by the following relation: length of the curve g. Approximate offset distance between the curve {gi} and {gi + 1} equals E|v(s)|, Vcont = Vgridshell (3) where geodesic curve g(s, E) is a variation of g(s) teqS = A ∑ l

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 497 Figure 5 The curvature richness comparison for the analysed free-form surfaces.

A ∑ l teq = kg/m3, Bending Strength (fr) = 53.6 MPa. S For simplicity, it is assumed that the material is where: isotropic, and the shells are unstrained. The analysis ∑ l is a sum of all the members’ lengths, of equivalent continuum shells can be a useful tool S - the total area of the free-form surface, in the feasibility pre-assessment of shapes meant to A and - a cross-sectional area of the member serve as base surfaces for geodesic gridshells before The gridshell density is defined as the propor- the grid configuration is specified. tion of members’ lengths per m² of the surface. Let ρgridshell be given by: RESULTS ∑ l ρgridshell = (4) In this section the structural soundness of the pro- S posed free-form surfaces as shells and based on them Insertion of (4) into (3) yields: gridshells is evaluated. A structurally sound structure teq = Aρgridshell (5) is stiff and feasible. The structure stiffness is mea- We will make the following assumptions: all grid- sured as the internal strain energy, while structural shells based on three different, free-form surfaces feasibility means that the structure is in equilibrium should have an approximate density of 3.75 m/m² and meets the maximum displacement and stress re- and be made of timber laths of rectangular cross- quirements. section 100x6.5 mm. The approximate shell thick- Analysis of approximated shells: The three ness can be calculated as follows: free-form surfaces with different curvature richness parameters were analysed. The surfaces are meshed teq= 0.1 m x 0.0065 m x 3.75 m/m² = 0.0024375 m = 2.44 mm with the resolution 10 cm and the analysis is con- For the purpose of this study we consider ducted under one load case 1.4 Dead Load. The thick- geodesic timber gridshells to be constructed of one ness of shells was calculated with the equivalent con- of the most often used materials for these types of tinuum method, and equals to 24.4 mm. The anal- structures - Baltic Birch (Cichocka 2020). The follow- ysed shells are assumed to be unstrained made out ing material properties have been used for the struc- of Baltic Birch. The supports are assumed to be pin tural analysis: Young Modulus (E) = 8750 MPa, Shear supports and are located along the base curves ly- Modulus (G) = 3600 MPa, the density (ρ)= 680.15 ing on the ground. The maximum displacement and

498 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 6 The structural analysis results of continuum shells, irregular and irregular gridshells with equivalent volume of material.

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 499 strain energy is evaluated using linear analysis with E - Elasticity Modulus Karamba3D. For the purpose of this study, the allow- Before analysis of the strain energy and maxi- able displacement is assumed to be ∆ ≤ L/300, mum displacement, all the proposed grids are veri- where L is the maximum span of each shell. The fied to meet the derived above criteria: structural analysis indicates (see figure 6) that the σmax .init. < 30.54 MPa greater the curvature of the shape, the stiffer the shell becomes. Therefore, shells with lower strain energy Figure 7 have higher potential to serve as a base surface for a Strain energy structurally sound gridshell. The geometry of the first results for shell and shell indicates that the shape is vulnerable to deflec- gridshell structures tion and any gridshell with equivalent volume, de- based on three signed on its surface might suffer from stability prob- surfaces. lems. The other surfaces show much lower strain energy parameters and small deflections, therefore might be recognized as more suitable for geodesic gridshell design. Analysis of gridshells: In order to assure that none of the laths breaks during the construction and that there are proper stress reserves to handle exter- nal loads in the structure after assembly, the max- imum initial bending stress (σmax .init.) should be smaller than the design stress (σd): The analysis of the continuum indicated the first sur- σmax .init. < σd (6) face with the lowest curvature richness has the high- The Bending Strength of the material (fr) defines the est strain energy (the lowest stiffness) and the largest bending stress in a material just before it yields, while maximum displacement, which significantly exceeds cracks can appear in the plastic limit phase. Follow- the assumed allowable displacement of 1.67 cm (see ing a conservative approach, the cracking stress is as- figure 7). The analysis of corresponding gridshells sumed to be equal to Bending Strength (fr). In or- based on this surface, shows that the maximum de- der to assure proper stress reserves in the prestressed flection, although lower than in the shell analysis, is structure, the design is defined with safety factor of still considerably too large. This confirms the hypoth- 0.6 and for the selected material equals: esis that the equivalent volume shell analysis can in- dicate the structural feasibility of gridshell based on σd = 0.6fr (7) that shape. From the shell analysis, it can be con- σd = 0.6 * 50.9 MPa = 30.54 MPa cluded that shape 2 and shape 3 have higher po- The maximum initial bending stress was evalu- tential to serve as free-form surfaces for structurally ated using the following formula: sound geodesic gridshells (see figure 6). The results Ezmax show that for both these shapes the deflection pat- σmax .init. = (8) rmin terns are reflected in the discrete equivalent struc- where tures. It can be noticed that for shape 2 and shape zmax - a distance from the neutral axis to the con- 3 the strain energy and maximum displacement of sidered fibre gridshells are higher than in the approximated shell, however still proportional to the shell analysis. In rmin - the smallest inner radius of curvature of all geodesic curves all analysed shapes, the gridshells with regular grids

500 | eCAADe 39 - Structure optimization and material-based design - Volume 2 have lower strain energy and maximum displace- curvature-richness of the shape has an impact on ment (see figures 7, 8). The comprehensive summary the internal strain energy of the shell. The more of the analysis results are summarized in figure 6. curvature-rich surface, the smaller the strain energy of the shell. Despite the fact that both shell and grid- Figure 8 shell structures were considered to be unstrained for The maximum the analysis, the equivalent continuum results out- displacement performed the discrete models in all considered cri- results for shell and teria among all analysed examples. In contrast to gridshell structures some reports in the literature on use of the equiva- based on three lent continuum technique, which showed that there surfaces. was only a 3-10% error between the analysis of shells and gridshells for estimation of the load-bearing ca- pacity (Malek, Wierzbicki & Ochsendorf 2014). The relatively high discrepancy between the analysis of shells and corresponding gridshells might relate to the slenderness of the members, employment of the linear analysis or varying number of support points. In reality, gridshells would be stiffer due to the initial prestress. The findings have a number of possible limita- tions, namely the presented method was applied DISCUSSION on the limited number of free-form surfaces, and This paper is a modest contribution to the ongoing its operation on other shapes remains unexplored. discussions on structurally efficient grid configura- Not only the curvature-richness might influence the tions for geodesic gridshells. We have not only ad- strain energy of the shell, but also the shape and the dressed the problem of generating regular geodesic length of the support curves, therefore the magni- grids on free-form surfaces but also examined if such tude of contribution of the curvature-richness to the configurations have structural advantage over the ir- stiffness of the shell remains unknown. Although in regular grid patterns. The originality of our solution the analysed examples, the geodesic gridshells with lies in the fact that the discrete surface optimization regular grid configurations outperformed those with methods for definition of geodesic patterns are val- irregular patterns of the same density, we cannot idated as a design method for structurally efficient conclude that this is always the case due to the lim- geodesic gridshells. To our knowledge, this is the first ited number of analysed shapes and variations of reg- study to examine the structural performance (stiff- ular and irregular grids per shape. ness and feasibility) of the elastic geodesic gridshells with regular and irregular grids of the same den- CONCLUSIONS sity. The stiffness is interpreted as strain energy of The main concern of the paper was to introduce a the structural system, while maximum initial bending method to generate regular geodesic grid configu- stress and maximum displacement are considered as rations on free-form surfaces. The focus of atten- structural feasibility indicators. tion was on the application of the method for design These results are consistent with other studies of regular bidirectional geodesic patterns for struc- which suggest that the geometry and topology in- turally feasible timber gridshells. In this paper we ex- fluence the stability of gridshells (Malek 2012). An amine the structural performance of regular and ir- important implication of these findings is that the

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 501 regular gridshells of the same density. The findings will be required to investigate how to analyse the of our research are quite convincing, and thus the fol- curvature of surfaces to guarantee that the members lowing conclusions can be drawn: might be arranged in either configuration without breaking before the grid is specified. • The proposed method can successfully generate regularly distributed bidirectional grids on the free-form surfaces. REFERENCES • The surface characteristics e.g. curvature- Adiels, E, Ander, M and Williams, CJK 2017, ’Brick patterns on shells using geodesic coordinates’, Proceedings of richness influence the strain energy of the shell the IASS Annual Symposium 2017 constructed based on it. This study indicates Adiels, E, Bencini, N, Brandt-Olsen, C, Fisher, A, Naslund, I, that the higher the curvature-richness the lower Otani, RK, Poulsen, E, Safari, P and Williams, CJK 2018 the strain energy of the shell structure of the ’Design , fabrication and assembly of a geodesic same maximum span and height. gridshell in a student workshop’, Proceedings of the • The equivalent continuum method can help to IASS Annual Symposium 2018 Cichocka, JM 2020, INFRAME - design and construction determine if the free-form surface can be used of a sequentially erected elastic timber gridshell., Mas- to design a structurally-sound gridshell with ter’s Thesis, MIT any configuration of members (prior to spec- Cuvilliers, P, Douthe, C, du Peloux, L and Le Roy, R 2017, ification of the grid configuration). Displace- ’Hybrid Structural Skin: Prototype of a GFRP Elas- ment analysis patterns occurring on the approxi- tic Gridshell Braced by a Fiber-Reinforced Concrete mated shell analysis are reflected in correspond- Envelope’, Journal of the International Association for Shell and Spatial Structures, 58(1), pp. 65-78 ing gridshells. However, gridshells with regular Harding, J, Pearson, W, Lewis, H and Melville, S 2014, ’The grid seem to better represent the deflection of Ongreening Pavilion’, Advances in Architectural Ge- the shell than the ones with irregular grids. ometry 2014, pp. 295-308 • In all the examined examples the geodesic grid- Lafuente Hernandez, E 2015, Design and Optimisation of shell with regular configurations outperformed Elastic Gridshells, Ph.D. Thesis, UdK Berlin the irregular ones of the same mass in strain en- Kreyszig, E 1991, Differential Geometry, Dover Publica- tions ergy and maximum displacement criteria. Malek, SR 2012, The Effect of Geometry and Topology on The proposed method can be readily used in practice. the Mechanics of Grid Shells, Ph.D. Thesis, MIT This research was concerned with timber geodesic Malek, S, Wierzbicki, T and Ochsendorf, J 2014, ’Buckling of spherical cap gridshells: A numerical and analyt- gridshells; however, the results are applicable to ical study revisiting the concept of the equivalent geodesic gridshells made out of any material (e.g. continuum’, Engineering Structures, 75, pp. 288-298 GFRP planks). Moreover, the regular grid configu- Pillwein, S, Kubert, J, Rist, F and Musialski, P 2020, Design ration has proven to better represent the continu- and Fabrication of Elastic Geodesic Grid Structures, As- ous shell deflection and structurally outperform the sociation for Computing Machinery irregular pattern. Future work will involve the com- Pottmann, H, Deng, B, Schiftner, A, Kilian, M, Guibas, L and Wallner, J 2010, ’Geodesic Patterns’, ACM Trans- parison of the shells and gridshells incorporating the actions on Graphics, 29(4), pp. 43:1-43:10 prestress in the analysis of extended sets of free- Schulitz, M 2017, ’Timber Gridshell Exploration using form surfaces, including an example with the surface Geodesic Segments’, Proceedings of the IASS Annual based on one closed curved beam. The next stage of Symposium 2017 our research will be proposing the method for gener- Soriano, E 2017, ’Low-Tech Geodesic Gridshell: Almond ation of a regular triangular geodesic grid that might Pavilion’, archiDOCT, 4(2)(February), pp. 29-40 Vassallo, M and Malek, S 2017 ’Design and Structural show even better performance than the 2-directional Analysis for a Deployable Gridshell Shelter’, Proceed- grid presented in this paper. Clearly, further research ings of the IASS Annual Symposium 2017

502 | eCAADe 39 - Structure optimization and material-based design - Volume 2 ClayKnit

A composite structure of clay and knitted meshes

Lukas Gosch1, Julian Jauk2, Hana Vašatko3, Elizabeta Šamec4, Milena Stavric5 1,2,3,5Graz University of Technology, Institute of Architecture and Media, Graz, Austria 4University of Zagreb, Faculty of Civil Engineering, Zagreb, Croatia 1,2,3,5{lukas.gosch|julian.jauk|vasatko|mstavric}@tugraz.at 4esamec@grad. hr

In this paper, we will demonstrate a new method of using knitted meshes to act as a formwork and to reinforce thin and hollow spatial clay structures. Currently, ceramic elements in the building industry are formed using extruding, pressing, or casting methods. This new approach can increase the usability of digitally fabricated lightweight elements, by spraying clay onto mass customised knitted meshes. Compared to fabrics that are used to shape concrete as a stay-in-place formwork, knitted meshes are available in various densities and changing patterns. They also offer the possibility to use a non-flexible thread as an elastic mesh. Knitted meshes are formed in a predefined shape by stretching them without the use of elaborate scaffolding. A specific liquid clay mixture is applied by spraying multiple layers onto the mesh by an industrial, six-axis robotic arm to precisely achieve variable wall thicknesses. Due to the complementary qualities of clay, which absorbs compressive forces and the threads, which absorb tensile forces, structures can be designed with a material optimisation scheme. To demonstrate the potential of such composite materials and the building process itself, a 1:1 lightweight module was constructed as an architectural prototype.

Keywords: Ceramics, Knitted Threads, Digital Fabrication, 6-axis Robotic Arm, Spraying

STATE OF THE ART the MARS Pavilion (Sarafian et al. 2017), KnitCrete Fine meshes, which are referred to as fabrics in the (Popescu 2019) and Fabric Forms (Culver et al. 2016), following references, have been used in the past to were carried out to examine the role of fabric form- efficiently cast concrete into organic shapes. Inter- works that are digitally positioned by robotic arms. est has recently emerged regarding the possible in- Two significant differences in clay design and pro- dustrial applications for these meshes (Chandler and cessing as compared with concrete introduce certain Pedreschi 2015). Recent research projects, such as restrictions: 1) concrete can be mixed with acceler-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 503 ators to drastically cut down on drying times (Stark cess was divided into five phases: 1) finding a suit- and Jochen 2000), and 2) the shrinkage factor of con- able thread, 2) obtaining a processable clay mixture, crete is lower than that of clay, because concrete so- 3) testing the material compatibility, 4) setting up the lidifies as a result of a chemical process (Stark and hard- and software for the spraying process and 5) fir- Jochen 2000). ing the produced samples. Once all the small-scale Material mixtures made for thin and light clay experiments were conducted, the gained knowledge structures rely on fibre reinforcement, whereby the fi- was applied to carry out large-scale experiments and, bres are randomly dispersed within the mixture. Ma- ultimately, to demonstrate the architectural applica- terials, such as paper fibres, are used to reinforce tion by constructing an exemplary 1:1 module. clay during the drying process, are flammable and removed during the firing process. However, fibres Thread such as high-temperature glass fibres support the A series of experiments was performed to evaluate drying process and withstand the firing process, be- the most suitable type of thread with regard to its coming a permanent reinforcement structure. Ex- knittability, temperature resistance, availability and amples can be found in the fields of traditional clay- affordability. The availability and the economical based architecture, where hay is used to reinforce feasibility of the reinforcing material played impor- adobe buildings (Nachtigall and Pohl 2013), as well tant roles, as the material value had to correspond as in experimental, contemporary architectural ap- with that of clay. The following fibres in different plications, such as the technique of spraying clay on thicknesses were acquired from the German com- inflated formworks used in the project Bioshotcrete panies Siltex and Engelmann: carbon, glass, basalt (Bravo et al. 2018) developed at IAAC Barcelona. In and stainless steel. These fibres were obtained as the latter, multiple layers of textiles are used to de- corded threads to prevent fraying and weak spots in fine the shape and support the drying process in or- the whole mesh structure. der to create an unfired clay-shell construction. Other Knitability. To manually knit a mesh, the threads research on the reinforcement of fired ceramic ele- need to be flexible enough for processing. Con- ments has been conducted, including studies on the trary to our expectations, knitting threads that have use of two types of stainless steel fibres (Llaudis et al. high friction resulted in the creation of meshes with 2016). Applications in which fibres are utilized as a a more defined geometry. These are also more flat mesh are mostly found as plastic reinforcement likely to stay in place when exposed to the air pres- lattices, which have been in use since the 1980s (Nel- sure needed for spraying. All samples used for the son and Kroll 1996). small-scale experiments were produced with a semi- Our project extends the current research by com- automatic Addi Express Knitting Machine. Conse- bining the concept of fabric formwork for casting and quently, a large number of samples with the same the use of fibres as a means of reinforcement to create dimensions have been efficiently produced. A regu- a new niche: the creation of freeform, lightweight, lar knitting pattern called stocking stitch was used to composite structures consisting of knitted meshes create 10x10 cm test samples (figure 1). Other knit- and sprayed clay. ting patterns could have caused more anisotropic stretching and thereby an uneven distribution of the METHODS stresses. In order to validate the potential of such a compos- Temperature resistance. The threads needed to ite material and to develop methods for shaping it to withstand firing temperatures of at least 800 °C with- building elements, experiments were carried out on out getting brittle afterwards, which would result different scales. The material experimentation pro- in a decreased tensile strength. Firing ensures a

504 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 1 Knitted material samples made of various materials. From left to right: carbon, glass, basalt and stainless steel

durable and weatherproof module and also increases company Goerg & Schneider, type 209, was used. No the structural properties of clay. Certain materials significant differences were noticed when spraying have varying combustion temperatures, depending a mixture containing microsilica as a flux. Viable re- on the amount of oxygen present during the firing sults were obtained while using a 1:1 mixture with- process, according to the manufacturer of the ob- out any additives - referred to as the first liquid clay tained threads. As a consequence, it was neces- mixture (LCM1). To ensure that no clumps remained sary to examine whether spraying a clay coating led in the mixture and to guarantee a constant viscosity to higher heat resistance by shielding the threads throughout the spraying process, it was important to from oxygen during the firing process. Two knitted blend the mixture with an industrial mixer shortly be- meshes were prepared: one lacked any coating, and fore spraying. After the drying phase, the preliminary the other one fully coated with clay. Furthermore, the LCM1 test pieces appeared to be fragile and were elastic modulus is crucial while determining whether more likely to crack during handling. a thread is suitable for reinforcing clay or not. Both of Consequently, a high performance liquid clay the composite parts need to have the same thermal mass called SaphirKeramik, obtained from the Aus- expansion, while tolerating the shrinkage that occurs trian ceramics manufacturer Laufen, was used in fur- during the drying and firing processes. A significant ther experiments, as it has shown low shrinkage in advantage of using knitted meshes as a reinforce- previous work. The second mixture, called LCM2, was ment is that a single thread does not have to be elas- created by adding 20% of water to the obtained liq- tic as long as the entire mesh allows for movement uid clay mixture to achieve a viscosity similar to that while the clay is shrinking during its drying and firing of LCM1. When comparing the shrinking behaviours process. Still, some materials are too stiff or cause too during drying, LCM2 turned out to be more suitable much friction to allow the movement of the threads for large-scale applications, as it was less likely to within the mesh. crack, had a lower drying time and a smoother sur- face quality than LCM1 (figure 2). Clay mixture The viscosity of the sprayed material is strongly re- Spraying lated to the diameter of the spraying nozzle used, the A six-axis ABB IRB 6660 robot was used to carry out number of layers that need to be sprayed to achieve the spraying procedure. Toolpaths were designed a certain material thickness and the overall appear- and simulated in Rhinoceros 3D, Grasshopper and ance of the surface quality. To obtain a viscosity that HAL. The geometry of the test samples and the fi- is suitable for spraying, mixtures were prepared with nal module, was computed by simulating the phys- varying ratios of clay powder and water, ranging be- ical behaviour of knitted meshes when clamped in a tween 2:1 and 1:2. Initially, clay powder from the frame. In order to define the robot’s movement, tar-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 505 Figure 2 Cracking behaviour of different clay mixtures during the drying process. From left to right: LCM1 (picture 1, 2), LCM2 (picture 3,4)

gets were placed at a certain distance and perpendic- ing in the nozzles clogging, while larger openings led ular to the computed geometry of the node (figure 3). to a high-velocity spray beam with insufficient atom- The generated code was then tested in RobotStudio ization. With regard to atomization, no suitable con- to avoid collisions and sent to the console. figuration was found among airless spraying systems. Finally, a low-pressure air-assisted spray system was Figure 3 used (figure 4). This has the benefit that it dispenses Robot targets for the material by an airflow with a pressure als low as spraying procedure 0.5 bar, while the air controls the material atomiza- computed in tion simultaneously. The opening and closing of the Rhinoceros 3D airflow, and thereby the material flow, was controlled by the code. Another pressure regulator was posi- tioned prior to the spraying gun and set manually to not exceed a maximum of 1 bar. During the spraying process, the nozzle had a constant distance of 7 cm to the mesh resulting in Initially, a high pressure airless spray system consist- a spraying width of 2 cm. The speed of the robotic ing of a WIWA Profit 3010 feeding pump with an op- arm was set to 150 mm/s. The first three layers of erating pressure of 8 bar and a WIWA 300 automatic clay were applied, each requiring a consequent dry- spray gun was used. The filter of the spraying sys- ing time of 10 minutes. The following layers included tem was removed, as it tended to separate clay and drying periods of 30 minutes, as the weight of the water. The opening and closing mechanism of the clay would otherwise begin to influence the geom- spraying gun was regulated by another pressure cir- etry. cuit, which was controlled by a separate command While materials with a rougher surface, e.g. car- in the code generated for the robot. Standard noz- bon threads, soak up a large amount of coating, stain- zles that are usually used for spraying paint with a less steel threads are composed in such a way that a diameter of 0.38 mm were too small and clogged liquid clay mass simply rolls off. The more material when used. Consequently, custom-made nozzles that adheres in a single spraying session, the quicker from acrylic glass with diameters ranging from 0.10 a certain material thickness can be achieved. To reach to 1.25 mm were made for spraying larger amounts of a wall thickness of 3 mm, seven layers needed to be clay in a conical area. In addition, nozzles with multi- applied. Since the automated spraying process of- ple holes were tested; the opening-grid of the nozzle fers high precision and reliability, for spraying the fi- consists of 44 holes within an area of 100 mm². Open- nal module, it was possible to apply more material ings between 0.1 and 0.5 mm were too small, result- (5 mm) to areas with a higher structural load, while

506 | eCAADe 39 - Structure optimization and material-based design - Volume 2 Figure 4 Six-axis robotic arm (first, from left), air assisted spraying gun (second), spraying width of a single path (third), spraying procedure (fourth)

other parts remained lighter (2 mm). This would not After all the samples were knitted, sprayed and have been possible by applying material manually. fired (figure 6), stainless steel threads led to a con- sistent cracking pattern throughout the whole test Firing the material sample, due to different thermal expansion of the One design constraint was the inner measurements materials. Furthermore, the threads became brittle. of the ovens that were used to fire the prototypes. Basalt threads with a diameter of 2 mm were iden- A Nabertherm Toplader 60/R was used for the small- tified as the most viable solution when using LCM2, scale experiments, whereas the larger ones were as insignificant changes occurred when compared to fired in a self-made oven with inside dimensions of the other materials fired at 800 °C. However, firing 70x80x105 cm. basalt threads at 1250 °C will drastically change their To gain ideal material properties of clay, the fir- integrity. ing should be carried out at a peak temperature of Figure 5 1250 °C after 11 hours of preheating, according to the Microscopic image company Laufen. However, to achieve sintering of of clay-coated glass the material, a temperature of 800 °C is sufficient. It threads fired at was aimed to keep the temperature and the firing du- 1250 °C ration as low as possible to avoid damaging or weak- ening the threads. Any temperatures lower than 800 °C would not guarantee the formation of a weather- proof surface. To observe the quality of the threads after fir- ing, an Olympus SZX16 light microscope was used to compare unfired and fired threads. No visible dif- ference was detected between clay-coated and un- MODULE PROTOTYPE coated threads, which indicates that a clay coating If we contextualize this research regarding its pos- would not sufficiently detain oxygen. Tests have sibilities for future implementation, we can see that shown that easily flammable threads create voids the developed building principle has potential to be that can be later filled with another reinforcing ma- used in building large-scale freeform structures and, terial, while materials such as glass fibres melt when more precisely, shells and gridshells. A gridshell is fired at 1250 °C within the ceramic and create a solid perceived as the most suitable structural system that filling (figure 5). However, this research was specif- enables spanning large footprints, because of the ce- ically carried out on fibre materials that retain their ramics’ ability to absorb compressive forces. Its stiff- integrity after firing. ness arises from the use of hollow, thinly coated ele-

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 507 Figure 6 Sprayed and fired material samples made of various materials. From left to right: carbon, glass, basalt and stainless steel

ments (i.e. bars). Due to restrictions associated with ular assembly methods, it is easier to design a mod- a robotic arm, such as its reaching capacity, size, and ule as a single knitted piece consisting of the node the difficulties of bringing such heavy apparatus on- together with the connected bars. Furthermore, we site, we concentrated on the development of a sin- presumed that the connection between the mod- gle module. The module can be combined with sim- ules will occur where the bars contact one another ilar ones, enabling an overall modular prefabrication and, in fact, a proper connection needs to be assured process to be carried out. To show the architectural at these points. Computational form-finding is car- potential of the developed method, an exemplary ried out to avoid bending; however, small residual module with a diameter of 100 cm and containing moments can appear at the nodes. Therefore, we three intersecting bars was produced. needed to ensure that the node’s cross section was high enough to prevent the formation of a hinge Design parameters that could potentially lead to structural failure. Addi- To design a gridshell, we needed to perform com- tional stiff elements were needed to ensure that the putational form-finding. Many methods are appli- cross section in the intersecting area was sufficiently cable for this purpose (Veenendaal and Block 2012), high, as well as to ensure that the knitted mesh was such as the linear constrained force density method stretched over the bars under adequate tension to (Šamec et al. 2019), which can be used by apply- prevent the material from sagging under the load of ing a tension-compression analogy. This analogy wet clay. presumes the presence of a hinge connection be- tween the bars, enabling their self-positioning into Construction of the node module the equilibrium position under the assigned load of The knitted mesh structure was manually produced their weight (i.e. a hanging position results due to using 150 m of a basalt thread. The stocking stitch tension-only forces). Once a satisfying geometry has knitting pattern was taken for the module. However, been found (this can be manipulated to achieve a cer- a larger knitting scale was used in an independent tain bar length or internal forces), the connections orientation for the triangular surfaces in the center to between the bars are held in a fixed position. It is allow a uniform pattern transition on all three sides. then turned upside down to form a compression- To clamp the knitted mesh structure prior to spray- only structure. In future large-scale applications, the ing, a wooden frame was built. The supporting struc- geometry of the module will be altered during the ture allowed the mesh to be stretched in a controlled construction to match the one defined by the com- manner to define the geometry in advance. In addi- putational form-finding. tion, to maintain a tube-like geometry, various filling While considering knitted meshes as formwork materials such as styrofoam and expanded clay ag- materials, possible fabrication steps and future mod- gregates were tested. In this research, no filling mate-

508 | eCAADe 39 - Structure optimization and material-based design - Volume 2 rials were used to avoid random surface deformation Therefore, a highly precise production tolerance was in the mesh geometry. possible without any post-processing steps. In addi- The geometry of the triangular intersectional tion, as opposed to materials used in the concrete in- area was defined by circular elements made of acrylic dustry, the excess materials used in this process can glass. These elements were placed inside the mesh be stored and reused. Before firing, the material can and close enough to each other to prevent the mesh also be washed off the mesh with water and reused. from sagging under the weight of the applied clay in Currently, the largest limitation is coordinating the the central area. The same elements were placed at times required for spraying and drying, as well as the the ends of the bars to ensure a precise geometry at availability of ovens with appropriate dimensions. the connector. By loosening or tightening the bot- Figure 7 tom or top screws attaching the outer acrylic glass Spraying procedure pieces, the angle could be adjusted on each side in- of the module dividually to match the positions of the neighbour- ing modules. In this way, it would be possible to respect the geometry defined in the computational form-finding in a future application.

CONCLUSION Knitting a formwork allows the use of non-elastic threads as an elastic mesh. Depending on the scale and the area of application, different fibre materials can be knitted to further develop a wide geometri- FUTURE WORK cal range of building components. This can be ex- During this study, additional research directions and ecuted either by hand or by semi-automatic or fully questions were identified. The next step will be to in- automatic knitting machines. Using temperature- troduce a Brother Electroknit KH-930 automatic knit- resistant fibre materials enables the knitted mesh to ting machine into the workflow. This will enable a reinforce the structure once the elements have been more precise control over the process and ensure re- fired, therefore basalt fibres were chosen for this re- peatability when creating mesh structures. The inclu- search. However, the manual knitting procedure is sion of this machine will allow a larger range of knit- a highly labour-intensive process, which has to be ting patterns, which can achieve different behaviours conducted by a professional. Furthermore, the study once the mesh is clamped and stretched. Varying findings show that not all fibre materials are suitable knitting patterns within one geometry is a possibil- for manual knitting. ity that requires further detailed study as well. Spraying presents an alternative to conventional Once the process is fully automated, it will be ceramic-shaping methods, such as extruding, press- possible to produce a 1:1 mockup of a pavilion-like ing, casting, or dipping methods. Industrial produc- structure consisting of a variety of different mod- tion seems feasible for objects with a large number ules. Therefore, redesigning the support frame is also of sprayed layers, while the production of single el- planned. A circular aluminium frame will allow the ements takes a considerable amount of time due to use of a larger amount of bars positioned at arbi- the drying time required between each iteration. As trary angles. The creation of this prototype mockup the module was fixed in its position during the spray- can pave the way to use this production method ing and drying process, the shrinkage did not de- as a competitive method to create other gridshell form the overall geometry and dimensions (figure 7). constructions, such as the on-site construction of

Structure optimization and material-based design - Volume 2 - eCAADe 39 | 509 lightweight steel elements. Structures, 49(26), pp. 3741-3753 Another potential emerging from this research is Šamec, E, Fresl, K and Gidak, P 2019 ’An Extended Linear shaping unfired clay through the method described Procedure for Constrained Form Finding’, Proceed- ings of the IASS Annual Symposium 2019, Barcelona, in this paper and using it as a formwork for casting pp. 2-4 concrete. Once the concrete is hydrated, the com- posite formwork can be removed by washing the clay off with water.

ACKNOWLEDGEMENT This work was funded by the Austrian Science Fund (FWF) project F77 (SFB “Advanced Computational Design”). We are most grateful to Alfred Mittermaier, factory manager of Laufen Gmunden, for providing SaphirKeramik, to the Institute of Structural Design, Graz University of Technology and to Institute of Bi- ology, University of Graz.

REFERENCES Chandler, A and Pedreschi, R (eds) 2007, FabricFormwork, RIBA, London Culver, R, Koerner, J and Sarafian, J (eds) 2016, Fab- ric Forms: The Robotic Position of Fabric Formwork, Springer Switzerland, Cham Llaudis, AS, Ten, MS, Gonzáles, RN and Díaz, EM 2016 ’Metal Fibre-Reinforced Ceramic Materials’, Qualicer 2016, Castellón Maite, B, Chaltiel, S and Carazas, W 2018, ’Matter-Robotic Calibration for Bioshotcrete’, Temes De Disseny, 34, pp. 80-91 Nachtigall, W and Pohl, G 2013, Bau-Bionik: Natur – Analogien – Technik, Springer Berlin Heidelberg, Berlin Nelson, PE and Kroll, RE (eds) 1996, Exterior Insulation Finish Systems (EIFS): Materials, Properties, and Perfor- mance, ASTM, West Conshohocken Popescu, MA 2019, KnitCrete: Stay-in-place knitted form- works for complex concrete structures, Ph.D. Thesis, ETH Zürich Sarafian, J, Culver, R and Lewis, TS 2017 ’Robotic Form- work in the MARS Pavilion. Towards the Cre- ation Of Programmable Matter’, ACADIA 2017, Mas- sachusettes, pp. 522-533 Stark, J and Wicht, B 2000, Zement und Kalk: Der Baustoff als Werkstoff, Springer Basel AG, Basel Veenendaal, D and Block, P 2012, ’An overview and comparison of structural form finding methods for general networks’, International Journal of Solids and

510 | eCAADe 39 - Structure optimization and material-based design - Volume 2 VR, AR and interactive visualization 512 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 A Motion Vector Visualization Method on a Virtual Reality Screen Preventing virtual reality sickness for architectural investigation

Tomohiro Fukuda1, Hideki Nada2, Hiroyuki Fujii3, Yoann Pencreach4 1Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University 2Sakaiminato Municipal Office 3,4Forum8 Co., Ltd. [email protected] [email protected] 3,4{fujii|yoann}@forum8.co.jp

In the age of mass customization, virtual reality (VR) allows users to virtually visualize architecture from any viewpoint they prefer and to examine the design. It is important to evaluate the movement of the virtual camera to guarantee the quality of VR content in addition to preventing VR sickness. The development of rendering methods to visualize the speed of VR cameras has begun. However, the only absolute velocity values are insufficient because the amount of movement of objects close to the camera is large, and that of distant objects is small. Therefore, this research aims to develop a visualization method of relative velocities known as motion vectors on a VR screen. A prototype of a new rendering technique has been implemented and successfully applied to a VR application for design review of a complex building. There are two rendering methods to display a gradient in RGB colors and to give a motion blur effect. This function allows VR creators to understand where in the virtual world VR sickness is likely to occur.

Keywords: virtual reality, rendering, shader, virtual reality sickness, motion vector, computer-aided architectural design

INTRODUCTION vironments and lower prices. Also, inexpensive and In the age of mass customization, virtual reality high-performance head-mounted displays have be- (VR) allows users to virtually visualize architecture come popular, allowing laymen users to experience from any viewpoint they prefer and to examine immersive VR. the design. The VR applications by using three- However, it is necessary to prevent inappropri- dimensional computer graphics software, building ate camera speed that occurs in VR produced by information modeling, and the game engine has in- users who are not professional VR creators. VR sick- creasingly created in the area of computer-aided ar- ness causes a mismatch between the user’s vision chitectural design owing to improved production en- and vestibular sensation (Dichgans and Brandt 1978,

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 513 Figure 1 Differences in the appearance of objects due to differences in relative velocity (i.e. motion vector): bridge structure Kreutzberg 2014). In a VR session, the user’s body in Therefore, the purpose of this research is to de- (near view) and the real world does not move much, so there is no velop a VR rendering method. We will develop a mountains (far vestibular input, but on the other hand, the field of method for visualizing motion vectors by using a cus- view) when view shifts as the user moves within the virtual world. tomizable rendering technique that calculates the crossing a river, for Also, when a presenter such as a designer demon- relative velocity on the screen for each frame and example strates along a predetermined walk-through or fly- superimposes colors and other elements on the VR through path, the audience is forced to be shown by screen according to the relative velocity values. We VR images with unexpected movement and rotation. develop two rendering methods to display a gradi- Regarding the onset of VR sickness, the sever- ent in RGB colors or to give a motion blur effect. This ity varies depending on the speed and direction of feature allows the creator to understand where in the movement within the VR world (Lo et al. 1999 and virtual world VR sickness is likely to occur. The pro- 2001). Guidelines to prevent VR sickness have been posed method enables a more versatile approach in published [1]. However, the VR camera settings still addition to the conventional VR rendering. Figure 2 depend on the experience and know-how of the VR shows the conceptual diagram of motion vectors on creator. In order to guarantee the quality of VR con- a screen when driving a car, for example. tents after preventing VR sickness, it is necessary to be able to quantitatively visualize the speed of the Figure 2 VR camera in the VR world during the VR production Conceptual process. diagram of motion A rendering method of an absolute camera ve- vectors on a screen locity has been developed using a customized ren- when driving a car, dering technique (Fukuda et al. 2021). It calculates for example the absolute linear and angular velocities of the cam- era in each frame, and superimposes a color corre- sponding to the velocity value on the VR screen. This is a way to visualize the amount of camera movement (XYZ, roll, pitch, and yaw) between two frames. As challenges, a single object, and objects that do not move, will have the same value. Also, the value of XYZ is the same even if the depth value from the camera to the object is different. In the real world, objects close to the camera move relatively faster (see Figure 1). Therefore, it might be insufficient to evaluate VR sickness by the only absolute velocity values. The rel- ative velocity on the screen needs to be studied.

514 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 LITERATURE REVIEW The degree of VR sickness varies from person to As VR systems become more practical and so- person, but by collecting the data of the VR cam- phisticated, concerns about VR sickness increase era and the degree of VR sickness in the user test, it (Kreutzberg 2014). VR sickness, which occurs when is expected that useful information can be obtained experiencing VR applications, has symptoms similar through a statistical approach. In our previous re- to common motion sickness, such as vomiting, cold search, we aimed to develop a function to measure, sensation, numbness of limbs, nausea, sweating, and verify, and visualize the linear and angular velocities headache (LaViola 2000, Groen and Bos 2008). VR of virtual cameras in VR content (Fukuda et al. 2021). camera work greatly affects VR sickness (Lo and So 2001). Previous studies have investigated VR sick- PROPOSED METHOD ness by having the subjects fill out a questionnaire Calculation for simulator sickness as a psychological evaluation The relative velocity on the screen is the frame of ref- and measuring heart rate variability as a physiologi- erence (the image that corresponds to a certain mo- cal evaluation (Kennedy 1993). However, in the pro- ment in the video) and the vector that represents the cess of VR production, measurement and visualiza- motion from the frame of reference. It is also known tion of the virtual camera in the VR space is not com- as a motion vector. The value corresponds to the mon. viewing angle velocity (yaw, pitch). When the depth

Figure 3 General rendering processing flow

Figure 4 The process flow of customized rendering, which overlays colors on the screen according to the speed of movement

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 515 from the camera to an object differs, objects far away Rendering from the camera do not move so much and objects VR software with customizable rendering capabilities near the camera move greatly. Therefore, motion is selected as the VR development platform. The soft- vectors are expected to be an evaluation factor for VR ware allows the developer to implement pre-process sickness. The projection coordinates of the drawing and post-process rendering for the whole or a specific frame are calculated by the formula (1): scene. Furthermore, it is possible to define variables to be passed to the shader when rendering an object, · · · (x, y, z, w) = MP roj MVcurr P osecurr V ertex and to use them on the shader processing side (see (1) Figure 3). The projection coordinates of the previous frame are Figure 4 shows the process flow of customized calculated by the formula (2): segmentation rendering, where colors are overlaid on the screen according to motion vector values. The ’ ’ ’ ’ (x , y , z , w ) = MP roj · MVprev · P oseprev · V ertex Shader Calculation process is processed in the plugin (2) when processing Object Drawing in the Main Unit. In the Shader Calculation process, the position on the MV where curr is the model view matrix of the cur- screen is calculated by the formula (4), the relative ve- rent drawing frame (inverse of the camera matrix), locity on the screen calculated by the formula (5), and P ose curr is the object posture matrix for the current then output to the frame buffer as a texture based on MV drawing frame, prev is the model view matrix in the calculated relative velocity. the previous frame (inverse of the camera matrix) and P oseprev is the object posture matrix for the previ- Figure 5 ous frame. Coordinate system The inverse matrix of the camera posture of the of the overlaid hue, previous frame and the object pose matrix is main- which represents tained and sent to the shader at drawing time. the direction of the Next, the relative velocity on the screen is calcu- motion vector lated. The screen coordinates of the drawing frame are calculated by the formula (3): ( x y ) z (s, t) = , , depth = (3) w w w The screen coordinates of the previous frame are cal- culated by the formula (4): ( x’ y’ ) z’ (s’, t’) = , , depth’ = (4) w’ w’ w’ Display a gradient in RGB colors. The direction and Therefore, the relative velocity on the screen is calcu- intensity of the vectors were defined as hue and sat- lated by the formula (5): uration, respectively, as the color space in which the relative velocity values were mapped. The hues were − ’ − ’ s s t t red (R, G, B) = (1, 0, 0) for the positive x-axis orien- vs = , vt = (5) ∆t ∆t tation, cyan (R, G, B) = (0, 1, 1) for the negative ori- 1 where is equivalent to the frame rate. The results entation, green (R, G, B) = (0.5, 1, 0) for the positive ∆t y-axis orientation, and magenta (R, G, B) = (0.5, 0, 1) of this calculation are sent to the shader. for the negative orientation, as shown in Figure 5. By blending this color space with the original achroma-

516 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 6 Composite of colors representing motion vectors and the original image (monochrome)

tized drawing, the relative velocity on the screen can Design Studio and switched on within the VR plat- be visualized (see Figure 6). form (see Figure 7). Motion blur effect. The computed motion vector is We evaluated the prototype system by applying used to give a motion blur effect. Using the motion it to a building design project for a VR application. vectors, multiple images generated at nearby time This project is a new Sakaiminato civic center com- frames are combined to represent motion blur. The plex with a hall, library, meeting rooms, a Japanese- range of time to be calculated can be set by the user. style room, welfare facilities, disaster prevention fa- cilities (see Figure 8). It is not easy to understand the design contents PROTOTYPE SYSTEM CONSTRUCTION of a 7,000 square meters building complex using AND EXPERIMENTS drawings and perspectives from limited viewpoints. To verify the proposed method in the previous sec- In the case of a building complex, there are multiple tion, we implemented the prototype system. In this managers and operators for different functions, and study, we selected VR Design Studio [2], a VR software the users’ needs are diverse. The new Sakaiminato with customizable rendering capabilities, as our VR civic center complex is also planned to have multi- development platform. The shader programs are im- ple administrators for different sections. Also, it is plemented in the OpenGL Shading Language (GLSL) planned that citizens will participate in the projects and can be customized by the user. These methods and operations of the complex. can be implemented as plug-in software on the VR

Figure 7 Developed plug-in and user interface on VR Design Studio for rendering motion vector on VR

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 517 We loaded the developed plug-in software and Figure 8 VR content into VR Design Studio, ran the devel- Perspective oped rendering method for the scene we were us- drawings of the ing, and observed the results. We used a PC with Intel new Sakaiminato Core i9-9900K processor, 32.0 GB of RAM, an NVIDIA civic center GeForce RTX 2080, and a 1920×1200 display running complex project Microsoft Windows 10 Pro. As shown in Figure 7, a dialog box in the VR screen allows you to switch the rendering method for the camera speed. Also, by setting the maximum and minimum values, the colors on the screen can be au- tomatically changed according to the relative veloc- ity values on the screen. In this case, the maximum value was set to 2.0 and the minimum value was set to 0.0. Typical VR screenshots are shown in Figure 9. The top rows show the ordinary rendering results and the middle rows show the results of displaying a gradient in RGB colors, and the bottom rows show the results of motion blur effects. Table 1 shows the The VR for the new Sakaiminato civic center complex frames per second (fps) of the civic center complex project was created to review the design from the VR when rendered in normal mode, RGB gradient of points of design view by the client, administrators, motion vectors, and motion blur effect. The reason and users, and to discover any problems that might for the decrease in fps in the case of motion blur is appear after the completion of the building, and to that it takes time to combine multiple images. study the solutions. The use of computer graph- The proposed method is more effective in VR ex- ics for design review has been practiced for a long perience scenes such as 360-degree panoramic VR time because it allows the study of a modeled three- movies and fixed paths, where the user’s position is dimensional virtual architectural and urban space not free and only the camera angle can be controlled. from all angles (Sasada 1994). VR makes it possible Detailed verification is needed in the future. to review designs in a real-time walk-through. The VR Table 1 will also use publicizing the building complex project Frames per second to the general public. (fps) of Civic center Architectural VR requires detailed camera work. complex VR For example, when turning down a corridor and go- ing up and down the stairs, the speed of the VR virtual camera needs to be carefully monitored to avoid VR sickness. For example, the entrance space of a library CONCLUSION has a circular shape. We measured and visualized the Using a customizable rendering technique, we de- amount of change in the relative velocity of the VR signed and implemented a method for visualizing camera as the path of the VR walkthrough in the li- motion vectors on a VR screen, and successfully ap- brary moved through a curved corridor with book- plied it to a VR application for design review of com- shelves, desks, and other furniture to help prevent VR plex buildings. The developed rendering method sickness. was observed in VR software and the results were ver- ified. Using the function developed in this research,

518 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 9 VR screenshots of motion vector visualization in the civic center complex, Sakaiminato (Top: normal mode, Middle: Motion blur effect; RGB gradient of motion vectors)

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 519 the camera path can be improved by inspecting the REFERENCES camera path set by the VR creator before the presen- Dichgans, J and Brandt, T 1978, ’Visual-vestibular interac- tation. tion: Effects on self-motion perception and postural As a result, a 360-degree panoramic VR movie control’, in Held, R, Leibowitz, H.W. and Teuber, HL (eds) 1978, Perception. Handbook of Sensory Physiol- was created with VR Design Studio and is available on ogy, vol.8, Springer, Berlin, pp. 756-795 YouTube so that more people can experience the fu- Fukuda, T., Novak, M., Fujii, H. and Pencreach, Y. 2021, ture of the civic center complex project with simple ’Virtual reality rendering methods for training deep operations on their own mobile devices (see Figure learning, analysing landscapes, and preventing vir- 10) [3]. tual reality sickness’, International Journal of Architec- tural Computing, 19(2), pp. 190-207 Groen, E.L. and Bos, J.E. 2008, ’Simulator sickness de- Figure 10 pends on frequency of the simulator motion mis- A 360-degree match: An observation’, PRESENCE: Virtual and Aug- panoramic VR mented Reality, 17(6), pp. 584-593 Kennedy, R.S., Lane, N.E., Berbaum, K.S. and Lilienthal, movie on YouTube M.G. 1993, ’Simulator Sickness Questionnaire (SSQ): [3] a new method for quantifying simulator sickness’, In- ternational Journal of Aviation Psychology, 3(3), pp. 203-220 Kreutzberg, A 2014 ’New Virtual Reality for Architectural Investigations’, Proceedings of eCAADe 2014, vol. 2, In the future, it is necessary to evaluate the valid- Newcastle upon Tyne, pp. 253-260 LaViola, J.J.Jr. 2000 ’A Discussion of Cybersickness in Vir- ity of the proposed method by assuming actual and tual Environments’, ACM SIGCHI Bulletin, 32, p. 47–56 various usage situations in architectural design. In Lo, W.T. and So, R.H.Y. 2001, ’Cybersickness in the pres- the verification of the developed rendering method, ence of scene rotational movements along different the maximum and minimum values of the output VR axes’, Applied Ergonomics, 32(1), pp. 1-14 screen were set so that the difference in color would Sasada, T. 1994 ’Open Design Environment and Collabo- be clear, but the setting of the reasonable maximum rative Design’, Proceedings of eCAADe 1994, Glasgow (Scotland), pp. 3-6 and minimum values to prevent VR sickness needs Schnabel, M.A., Kobayashi, Y., Pencreach, Y., Bennadji, A., to be considered through user testing. Although Choi, D., Fiamma, P., Fukuda, T., Lo, T.T., Narahara, T., there are individual differences in the occurrence of Novak, M., Ron, R., Swarts, M., Terzidis, K., Tucker, T. VR sickness, we expect to obtain useful information and Vital, R. 2021 ’Virtual World16 - Virtual Design by collecting the data and statistically processing it Collaboration for the Intersection of Academia and through the user test. Industry’, Proceedings of CAADRIA 2021, Hong Kong and Online, pp. 203-212 [1] http://brianschrank.com/vrgames/resources/Oculus ACKNOWLEDGMENTS BestPractices.pdf The research was partly supported by joint research [2] https://www.forum8.com/technology/vr-design-stu funding from the Sakaiminato City Office and Osaka dio/ [3] https://www.youtube.com/watch?v=h-3jHmOan54 University. This research was partly carried out as part of the 2020 research activity of the World16 in- ternational research group on virtual reality (Schn- abel et al. 2021). The authors wish to acknowledge the support received from Forum8 Co. Ltd. All im- ages were created by the authors from 2020 to 2021.

520 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Landscape Visualization by Integrating Augmented Reality and Drones with Occlusion Handling to Link Real and Virtual Worlds Towards city digital twin realization

Naoki Kikuchi1,Tomohiro Fukuda2, Nobuyoshi Yabuki3 1,2,3Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering [email protected] 2,3{fukuda|yabuki}@see.eng.osaka-u. ac.jp

In the field of urban architecture and design, augmented reality (AR)-based landscape visualization is useful for building consensus among stakeholders at the design stage. An integrated AR and drone method can visualize future and past landscapes from an aerial perspective but has to address the problem of occlusion, where a 3D virtual model is displayed in front of the real-world objects. In this study, we propose an AR and drone integrated landscape visualization method to handle occlusion by linking the drone's location information in the real world and the camera in the virtual world. The method uses a 3D model of an existing building, which is part of the city model, to represent the 3D model of the design target as if the target were behind the existing building in the real world. Users can use the perspective of the drone, which flies along a set route, to examine the future landscape with high accuracy, as visualized using AR with occlusion handling.

Keywords: Digital twin, Occlusion handling, Landscape visualization, Web-based augmented reality (web AR), Drone, Urban design

INTRODUCTION the before and after appearance of the designed In the field of architectural and urban design, pub- landscape and can help to build consensus among lic involvement is expected during the planning and stakeholders (Goudarznia et al. 2017). When a user design stages to examine landscape changes. The wears a head-mounted display or holds a mobile de- design information provided to participants needs vice such as a smartphone, the AR perspective is lim- to be easy to understand. Augmented reality (AR)- ited to the user’s range of action. based landscape visualization at the design stage ef- Recently, drone research has advanced, and fectively provides users with visual information about drones are used for various applications. In urban

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 521 planning, drones are used for environmental mon- This study proposes an AR and drone integrated itoring and urban space management, providing a landscape visualization method that handles occlu- perspective outside the user’s range of action (Gal- sion by linking the camera’s location information in lacher 2016). To visualize past and future landscapes the real world to the location of a virtual world’s cam- from the air, an integrated AR and drone method was era in cyberspace. First, a design target virtual 3D proposed (Wen and Kang 2014, Unal et al. 2018). model and a virtual 3D model of an existing build- However, the integrated AR and drone method has ing that is part of a city model are placed in the an occlusion problem. A virtual object in front of a virtual world. Next, the drone’s camera and the real object wholly or partially hides the real object. virtual world’s camera are predetermined to move In general, in AR, virtual objects are rendered last in along the same path simultaneously so that the real relation to the real world. Without intervention, a vir- world and the virtual world are linked as digital twins. tual object is displayed in front of real objects. This The method uses 3D models of an existing building, discrepancy between the relative placement of a real which are part of the city model, to represent the ap- object and a virtual object (i.e., determining which pearance of the design target’s 3D model as if it were one is behind the other) is called the occlusion prob- behind the existing building in the real world. Also, to lem (Figure 1). Occlusion handling is a process to integrate AR and a drone with high versatility, mirror- solve this problem. ing and virtual camera technologies are employed. A city model is made of 3D geospatial data that Mirroring allows the same screen to be displayed on reproduce a real-world (physical space) city in a vir- multiple devices. At the same time, a virtual cam- tual world (cyberspace). City models are essential for era treats the PC screen as a webcam video. By em- public participation in reviewing the design of archi- ploying mirroring and a virtual camera, we can easily tectural and urban spaces (Ruohomäki et al. 2018). A use any available drone without regard to a drone- digital twin is defined as “a digital replica of a physi- specific software development kit (SDK). By super- cal asset, process, or system,” where the digital twin imposing the real scene and the occlusion-handled and the physical twin continuously interact (Batty virtual scene, it is possible to study the future land- 2018). The ideal urban digital twin is a city model scape with high accuracy as visualized with AR from that reflects its combined economic, ecological, and the viewpoint of a drone flying along a set route. The demographic conditions and changes (Dembski et al. drone’s camera and the virtual world’s camera move 2020). In the field of urban architecture and design, along the set drone route with synchronized posi- the use of digital twins of cities is at an early stage, tions. and their use remains inconsistent.

Figure 1 Occlusion problem in AR (a) Real world (b) AR with incorrect occlusion (mock-up) (c) AR with correct occlusion (mock-up)

522 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 RELATED WORK tegrates location-based AR and a drone to improve Augmented Reality in Public Participation the accuracy of displaying 3D virtual models. Pre- In Arnstein’s (1969) “Ladder of Participation,” in the vious integration AR and drone methods had the process of democratic empowerment, one step is problem occlusion: the 3D virtual model always ap- providing information to the public. Specifically, pears in front of the real-world object. The problem AR is used as a tool to provide geospatial informa- was that the system configuration depended on the tion to the public. In urban space, AR visualization drone model because the SDK provided for drones effectively provides visual information of landscape has limited functionality. changes before and after the design stage and can help form a consensus among stakeholders, includ- Occlusion Handling Method ing non-specialists such as the public. AR in urban In the occlusion problem, AR users may misunder- space research (Goudarznia et al. 2017, Haynes et al. stand the position of the 3D virtual model when it 2017) is limited to perspectives from within the AR always appears in front of the real-world object. To user’s area of action and does not allow users to ac- provide accurate information, occlusion handling is cess the aerial AR perspective. essential. To solve the occlusion problem, depth- based, foreground object-based, and model-based Integration of AR and Drones methods were proposed. Wen and Kang (2014) integrated AR and a drone to The depth-based method deals with occlusion display 3D virtual models in the real world from an by comparing the depth information of an occlu- aerial perspective. However, their drone was de- sion target obtained from an RGB-D camera with the signed and built from scratch, and their method was depth of a virtual object, as proposed by Du et al. less versatile because only specific drones could be (2020). The depth-based method can handle the oc- used. Unal et al. (2018) proposed a method that in- clusion in real time. However, the depth informa- Figure 2 Conceptual diagram of our proposed method

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 523 tion that the 3D sensing camera can obtain is limited, of the occlusion target (occlusion model) is prede- making it unsuitable for use in wide outdoor spaces. fined in the virtual scene. Occlusion is handled by comparing the depth of the design target 3D vir- tual model with the depth of the occlusion model Figure 3 (Kasperi et al. 2017). Model-based methods need to Flowchart of our create an occlusion model as a pre-processing step. proposed method

Research Contribution In our feasibility study, we handled occlusion using instance segmentation with the foreground object- based method. However, the AR frames occluded by the instance segmentation had a processing speed of only a few frames per second (fps). This paper proposes a model-based method using a 3D virtual model of an existing building that is part of a city model. The 3D virtual model of the existing build- ing is used as an occlusion model. The occlusion problem is solved by comparing the depth from the camera in the virtual world of the 3D virtual model of the design target with the depth from the cam- era in the virtual world of the occlusion model. The proposed method achieved occlusion-handled AR at frame rates of tens of frames per second. We also im- proved the model dependency of the system config- uration with the drone SDK in the integrated AR and drone method (Wen and Kang 2014, Unal et al. 2018).

METHODOLOGY The concept and flow diagram of our method are Figure 4 shown in Figures 2 and 3, respectively. First, 3D vir- The concurrent tual models of the design target and the existing movement of building are placed in the virtual world in an ap- drone’s camera and propriate relative position. The background and 3D virtual world’s model of the existing building are changed from RGB camera values to unrealistic color values and set in the model as emissive material. In order to superimpose the 3D virtual model of the design target on the video Foreground object-based methods use image pro- from the drone’s perspective, the unrealistic colored cessing to extract foreground objects from still im- areas on the video from the drone’s perspective are ages and videos and use the extracted object outline masked. Mask processing is an image processing to handle the occlusions. Roxas et al. (2018) used se- technique that displays certain areas of an image or mantic segmentation to extract the foreground ob- video and hides other areas. By setting the drone’s jects and the occlusion handling. camera and the virtual world’s camera to move along In the model-based method, a 3D virtual model

524 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 the same route simultaneously, the drone’s location the web. A DJI Mavic mini drone was used to realize information in the real world and the camera in the the real-world aerial view. An iPhone 11 (iOS v14.4) virtual world are linked (Figure 4). By superimpos- with an installed drone control application was used ing the video from the drone’s perspective as it flies as a drone controller. An iPad (iOS v14.4) was used as along the predetermined route with the video from a user device to view the AR video uploaded to the the camera’s perspective in the virtual world, AR with web. occlusion from an aerial perspective is realized. The The software of the prototype system was as fol- generated AR content is uploaded to a video distribu- lows. DJI Fly (iOS v1.3.0) drone control software was tion platform using streaming software and output used for the drone’s automatic navigation. Reflec- on the web browser of the AR user’s device. tor3 (v3.2.0) mirroring software was used to dupli- cate the smartphone screen onto a PC, including au- Versatile System Configuration dio, and to transfer the video from the controller to In traditional integration of AR and a drone (Wen and the PC. OBS Studio (v26.1.1) streaming software en- Kang 2014, Unal et al. 2018), the system configura- abled video streaming, and its virtual camera func- tion is limited by the drone SDK. To solve model de- tion was used to treat the PC screen as a webcam pendency, we integrated AR and a drone without us- video. The Unity (2020.1.7f1 64-bit) game engine was ing the drone SDK but instead used mirroring and a used to build the virtual world. Autodesk InfraWorks virtual camera. Mirroring is a technology to duplicate 2019 infrastructure design software was used to cre- the screen to multiple devices. A virtual camera is a ate 3D models of existing buildings as part of the technology to treat a PC screen as a webcam video. city model. The YouTube video distribution platform The video on the controller is displayed on the PC by was used to output the created occlusion-enabled AR mirroring. The video captured from the drone on the onto the web. The Google Chrome (v89.0.4389.82) controller using the virtual camera can be processed. web browser was used to access YouTube. The image processing detects the change in the con- Figure 5 troller’s video when the drone starts to move and syn- Arrangement of chronizes the start of movement of the drone’s cam- existing and era in the real world and the virtual camera in the vir- planned (new) tual world. buildings AR system needs to be user-friendly to encour- age public participation. The video delivery platform enables the use of AR content on web pages. Thus, users can easily experience AR content by accessing a web page using an URL or QR code without installing any specific applications.

EXPERIMENTS AND RESULTS To verify the applicability of the proposed method, we constructed a prototype system. The hardware of the prototype system was as fol- lows. We used a Galleria GCL2060RGF-T PC (Intel Core i7-10875H CPU, 32 GB RAM, NVidia GeForce RTX 2060 GPU, and Windows 10 home v20H2 OS) to generate an AR video with occlusion handling and upload it to

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 525 landscape study. Figure 8 shows the occlusion model Figure 6 of the existing building, created using InfraWorks. Photographs of The occlusion model created using InfraWorks was a experiment setting simple box model with no detailed roof shape. The occlusion model and the background were changed Figure 7 to bright green ((R, G, B) = (0, 255, 0)) (Figure 9). Dimensions of the planned building System Performance Evaluation model To evaluate the performance of the prototype sys- tem, Figure 10 shows the results every 4 s when the drone was flown at altitudes of 20 ± 1 m and 40 ± 1 m for 16 s. In this validation, the occlusion han- dling was possible from the drone camera’s point of view, as it flew along a predetermined route in an outdoor space. The overall system processing speed Figure 8 and delay of the proposed system were measured. Occlusion model of The frame rate of the video captured from the drone the existing was about 30 fps, the frame rate of the video cap- buildings tured from the camera in the virtual world was aver- age about 700 fps on Unity, and the frame rate of the video output on the web was about 30 fps. The delay between displaying the video on the controller and on the web was about 4 s. Figure 9 DISCUSSION Changing settings In the prototype system, the occlusion-handled AR of background and imagery could be visualized in a web browser from occlusion model: the viewpoint of a drone that automatically navi- (a) before changing gated along a predefined route. The overall proto- RGB values and (b) type system frame rate was about 30 fps, and the de- after changing RGB lay was about 4 s. User devices are commonly used values Occlusion Handling Outdoors for transmitting public information. The frame rate To verify whether the proposed system can handle was 30 fps, which is the frame rate of widely used de- occlusion from the viewpoint of a drone camera fly- vices, and the processing speed was sufficient. The ing through an expansive outdoor space, we con- latency was about 4 s, a difference that is notice- ducted an outdoor AR experiment. Figure 5 shows able and needs to be improved. The results in Figure the layout of the planned and existing buildings, and 10 show that the occlusion handling is not accurate the camera route and direction. The drone was auto- around the boundary between the existing building matically navigated in an arc around the center point and the 3D model of the planned building. The accu- in Figure 5, facing the center at 20 ± 1 m and 40 ± 1 m racy of the occlusion handling needs to be improved. altitude, and stopped after 16 s. Figure 6 shows the The occlusion handling’s low accuracy is thought to scene of the outdoor AR experiment. Figure 7 shows be caused by two main factors: the occlusion model’s a 3D virtual model of the planned building for the accuracy and the accuracy of the AR alignment.

526 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 10 Results of occlusion handling

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 527 CONCLUSION D. 2020 ’DepthLab: Real-time 3D Interaction with The conclusions of this study are as follows: Depth Maps for Mobile Augmented Reality’, Proceed- ings of the 33rd Annual ACM Symposium on User Inter- • To realize an AR landscape visualization method face Software and Technology, Virtual Event, USA, p. with occlusion handling from the air, we inte- 829–843 grated AR and drone technologies and handled Gallacher, D. 2017, ’Drone Applications for Environmen- the occlusion based on a model-based method tal Management in Urban Spaces: A Review’, Inter- national Journal of Sustainable Land Use and Urban using 3D virtual models of existing buildings Planning, 3(4), pp. 1-14 that are part of a city model. Goudarznia, T., Pietsch, M. and Krug, R. 2017, ’Testing • To improve the system configuration limitation, the Effectiveness of Augmented Reality in the Pub- we integrated the AR and drone techniques lic Participation Process: A Case Study in the City of without using the drone SDK by using mirroring Bernburg’, Journal of Digital Landscape Architecture, and virtual camera techniques. 2, pp. 244-251 Haynes, P., Hehl-Lange, S. and Lange, E. 2018, ’Mobile • The prototype system frame rate was confirmed Augmented Reality for Flood Visualisation’, Environ- at about 30 fps, and the delay was about 4 s. mental Modelling & Software, 109, pp. 380-389 Kasperi, J., Edwardsson, M.P. and Romero, M. 2017 ’Oc- In future work, to improve the accuracy of occlusion clusion in outdoor augmented reality using geospa- handling, it is necessary to develop a method to cor- tial building data’, Proceedings of the 23rd ACM Sym- rect the position information of the drone camera posium on Virtual Reality Software and Technology, and the camera in the virtual world. We will develop NY, USA, p. Article 30, 1–10 an AR method that includes the digital twin concept, Roxas, M., Hori, T., Fukiage, T., Okamoto, Y. and Oishi, T. which continuously links real-world location informa- 2018 ’Occlusion handling using semantic segmen- tation and visibility-based rendering for mixed real- tion and the virtual world via a drone, and the lo- ity’, Proceedings of the 24th ACM Symposium on Vir- cation information of the drone camera in the real tual Reality Software and Technology, 20, pp. 1-8 world and the camera in the virtual world. Ruohomäki, T., Airaksinen, E., Huuska, P., Kesäniemi, O., Martikka, M. and Suomisto, J. 2018 ’Smart City Plat- form Enabling Digital Twin’, 2018 International Con- ACKNOWLEDGMENT ference on Intelligent Systems, Funchal, Portugal, pp. This research was partly supported by the Japan So- 155-161 ciety for the Promotion of Science, KAKENHI Grant Unal, M., Bostanci, E., Sertalp, E., Guzel, M.S. and Kanwal, Number JP19K12681. All images were created by the N. 2018 ’Geo-location Based Augmented Reality Ap- authors from 2020 to 2021. plication for Cultural Heritage Using Drones’, 2018 2nd International Symposium on Multidisciplinary Studies and Innovative Technologies, Ankara, Turkey, REFERENCES pp. 1-4 Arnstein, S.R. 1969, ’A Ladder of Citizen Participation’, Wen, M.C. and Kang, S.C. 2014, Augmented Reality and Journal of American Institute of Planners, 35, pp. 216- Unmanned Aerial Vehicle Assist in Construction Man- 224 agement, Computing in Civil and Building Engineer- Batty, M. 2018, ’Digital twins’, Environment and Planning ing (2014) (pp. 1570–1577) B: Urban Analytics and City Science, 45(5), pp. 817- 820 Dembski, F., Wössner, U., Letzgus, M., Ruddat, M. and Yamu, C. 2020, ’Urban Digital Twins for Smart Cities and Citizens: The Case Study of Herrenberg, Ger- many’, Sustainability, 12(6), pp. 1-17 Du, R., Turner, E., Dzitsiuk, M., Prasso, L., Duarte, I., Dourgarian, J., Afonso, J., Pascoal, J., Gladstone, J., Cruces, N., Izadi, S., Kowdle, A., Tsotsos, K. and Kim,

528 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Automatic Diminished Reality-Based Virtual Demolition Method using Semantic Segmentation and Generative Adversarial Network for Landscape Assessment

Takuya Kikuchi1, Tomohiro Fukuda2, Nobuyoshi Yabuki3 1,2,3Osaka University [email protected] 2,3{fukuda|yabuki}@see.eng. osaka-u.ac.jp

In redevelopment projects in mature cities, it is important to visualize the future landscape. Diminished reality (DR) based methods have been proposed to represent the future landscape after the structures are removed. However, two issues remain to be addressed in previous studies. (1) the user needs to prepare 3D models of the structure to be removed and the background structure to be rendered after removal as preprocessing, and (2) the user needs to specify the structure to be removed in advance. In this study, we propose a DR method that detects the objects to be removed using semantic segmentation and completes the removal area using generative adversarial networks. With this method, virtual removal can be performed without preparing 3D models in advance and without specifying the removal target in advance. A prototype system was used for verification, and it was confirmed that the method can represent the future landscape after removal and can run at an average speed of about 8.75 fps.

Keywords: landscape visualization, virtual demolition, diminished reality (DR), deep learning, generative adversarial network (GAN), semantic segmentation

INTRODUCTION able to understand plans intuitively, it is necessary When redeveloping a mature city, existing buildings to express how the landscape changes before and and structures (hereafter referred to as existing struc- after the redevelopment in a visually comprehensi- tures) may be demolished to make way for new struc- ble manner (Bishop et al. 2005). In this realm, aug- tures as well as open spaces such as plazas and parks. mented reality (AR) (Caudell et al. 1992) could be In the planning process, meetings are held among helpful for the visualization of the future landscape. stakeholders to decide the details of the redevel- However, the future landscape visualization method opment. In addition to experts such as designers, using AR has a problem in that it cannot correctly government officials, and private developers, nonex- represent the future landscape after the structures pert citizens may also participate in these meetings. are reconstructed if the newly constructed structures These stakeholders have different degrees of partici- (hereafter referred to as new structures) are smaller pation and professional knowledge. Therefore, to be than the existing structures. Another problem is that

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 529 it is not possible to reproduce the remaining land- a discriminator. By training these neural networks scape after existing structures are demolished and re- to compete with each other, it is possible to gener- moved. ate “plausible” images that do not actually exist. It To solve the problems in visualizing the land- also requires manual pre-specification of structures scape after demolition and removal, research and to be removed. It is suitable to consider semantic practical application of diminished reality (DR) (Mann segmentation as one of the techniques to solve this et al. 1994) to future landscape visualization meth- problem. Semantic segmentation is a deep learning ods are in progress. DR is a technology that reduces model that classifies objects in an image. This classifi- the presence of objects in real space by hiding, eras- cation is done for every pixel in the image. Then, each ing, or translucent them in real time (Mori et al. 2017). pixel is colored with a predetermined color based on In the study of landscape DR, the future landscape the classified class. is expressed by superimposing the landscape behind Therefore, the objective of this research is to pro- the target structure onto the surface area of the tar- pose a method that uses GAN and semantic segmen- get structure. However, two issues need to be ad- tation to automatically detect the objects such as ex- dressed in research on the application of DR to land- isting structures to be removed; to generate masks scape studies: how to obtain the hidden background according to the areas to be removed; and to auto- and how to superimpose the hidden background. matically complete the removals without creating 3D Kido et al. (2020) proposed a DR method that models in advance. We also validate our proposed combines structure-from-motion (SfM) technology method in order to assess its usefulness. (Ullman et al. 1979) and deep learning. SfM is a tech- Our proposed method makes it possible to vi- nology that builds a 3D model from multiple photos sualize the future landscape more easily compared by detecting the feature points in each photo and with conventional methods and is expected to fa- estimating the shooting position and posture. This cilitate productive discussions at landscape review method was implemented using the following steps: meetings. (1) photographs of the structure to be removed and its surroundings were taken in advance, and a 3D LITERATURE REVIEW point cloud was acquired using the structure-from- In this section, we summarize relevant previous re- motion technology to create a 3D model, (2) at run- search. time, the system uses deep learning to detect mov- ing objects that have entered the area to be exam- Visualization Methods for Future Land- ined, (3) the structure to be removed and the de- scapes Using DR tected moving object are filled with a single color to DR is capable of diminish, see-through, replace, and create a mask, (4) superimpose a hidden background inpaint functions (Mori et al. 2017). Diminish de- on the masked area. grades visual information by thinning out color infor- However, this presents a challenge because it is mation; see-through covers a real object with a back- necessary to create a 3D model of the area to be ex- ground image so that it is not visible; replace super- amined in advance. There is room to consider the imposes a real object and a virtual object so that the generative adversarial network (GAN) (Goodfellow et object appears to have been replaced; and inpaint al. 2014) as one of the technologies to solve this is- generates and superimposes a plausible background sue. GAN is one of the deep learning models, which image based on the surrounding environment. can generate “plausible” images that do not actu- DR has been used in a variety of fields, including ally exist from random numbers and other inputs. the medical field (Navab et al. 1999) and the indus- It combines two neural networks: a generator and trial field (Zokai et al. 2003). DR has also been used

530 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 to remove people from images provided by services moved can be detected from the acquired image in such as Google Street View for privacy reasons (Flores real time. et al. 2010). To visualize a future landscape, Inoue et al. GAN (2018) proposed a method to represent the land- Pathak et al. (2016) proposed a GAN using a con- scape after the demolition and removal of existing volutional neural network (CNN). In this GAN, miss- structures by virtually removing them using DR. In ing parts in an image can be estimated and comple- this method, the hidden background of the structure mented using CNN to generate a “plausible” image. to be demolished is obtained by SfM by using pho- It is also possible to generate objects that are not in tographs taken in the study area, and the future land- the image. However, the problems are that the im- scape is visualized by superimposing it on the surface age size is fixed, the position, shape, and size of the of the structure to be demolished and removed. missing parts in the image are fixed, the method can- Kido et al. (2020) incorporated object detection not be applied to high-resolution images, and there is by machine learning into the method of Inoue et a high possibility of generating unnatural images in al. (2018). In this method, moving objects were de- which the surrounding region and the complemen- tected at runtime, and objects to be removed, includ- tary region are not contiguous. ing moving objects, were masked, virtually removed, Iizuka et al. (2017) proposed a completion and complemented with 3D surfaces created based method that considers the surrounding area of the on 3D point clouds to visualize the future landscape. missing part. Since the completion is based on the However, a common issue in these two studies features of the surrounding area, when the size of the is the necessity of preparing 3D objects of the struc- missing part in the image is small, a natural image tures to be removed and their hidden backgrounds in is generated in which the surrounding area and the advance. Another issue is that these methods cannot completion area are continuous with high probabil- be applied if the 3D objects cannot be created. ity. Also, the position, shape, and size of the missing parts in the image are not fixed. Since then, research Semantic Segmentation has been conducted to improve the accuracy of com- Semantic segmentation (Long et al. 2015) performs pletion and to apply GAN. In the field of architectural categorization and labeling for every pixel in an im- design, research was also conducted to apply it to the age. This method has been applied to the analysis of creation of floor plans (Huang et al. 2018). medical images and automatic driving of cars. Many However, in the studies by Pathak et al. (2016) algorithms have been developed to speed up the and Iizuka et al. (2017), image completion is per- process and improve segmentation accuracy. In the formed on various types of objects, and it is not field of the urban landscape rendering, it has been known whether it is possible to complete the missing used to detect buildings (Li et al. 2020) and greenery parts in landscape images. Therefore, in this study, from photographs and calculate the greenness ratio we checked if it is possible to use GAN to complete (Ki et al. 2021). the missing parts in a landscape image. If this is pos- Because datasets of the urban landscape are sible, it should be possible to complement the area available, we believe that it is possible to automate after the target structure is removed without prepar- the detection of objects to be removed using seg- ing the landscape hidden by the target structure in mentation techniques, and in this study, we used se- advance. mantic segmentation for the detection of objects to be removed, including buildings. In so doing, it is ex- pected that the area from which the object will be re-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 531 web camera are input to the GAN, which automati- Figure 1 cally completes the masked area based on the feature Workflow of the points around the mask and recognition of the white proposed method area as the mask area. The result of the completion is output as the future landscape. This method can be applied to pre-recorded videos, images, and real- time videos. If it is performed on video, this flow is re- peated every few frames. The system flow is shown in Figure 1. The types of objects to be removed are specified before execution. When a type of object is specified for removal, detected objects of this type are converted to white, and other object types, which are not targeted for removal are converted to black. Therefore, the type of object to be removed can eas- ily be changed. Also, the system can handle multiple types of objects to be removed.

VERIFICATION To clarify the optimal conditions for image comple- tion, we examined the effect of the mask image’s ratio in the total image and the effect of the area around the mask image. Also, to evaluate the usefulness of the proposed method, we examined whether it is possible to remove the objects targeted for removal and how long it takes to complete the process. In this section, we describe the method and re- sults of the accuracy verification of image completion by GAN for the prototype system and the method PROPOSED METHOD and results of the field verification. We propose a method that combines semantic seg- mentation and GAN to automatically detect target Prototype System objects, remove them virtually, and perform these A prototype system was developed to verify the per- tasks without prior preparation. The procedure of formance of the proposed method. The prototype the proposed method is described in this section. system uses FCHarDNet [1] with a HarDNet network First, the current landscape is acquired by a web cam- (Chao et al. 2019) for semantic segmentation and era. Next, we detect the objects to be removed from generative image inpainting with contextual atten- the current landscape using semantic segmentation. tion (Yu et al. 2018) for GAN. To provide network Then, from the detection results, we convert the re- weights, FCHarDNet was trained using the Cityscapes gion of the objects to be removed to white [RGB= dataset [2], and generative image inpainting with (255,255,255)] and the other regions to black [RGB= contextual attention was trained using the ImageNet (0,0,0)] and automatically generate the result as a dataset [3]. The development environment is shown mask image. The automatically generated mask im- in Table 1. age and the current landscape image acquired by a

532 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Table 1 Accuracy Verification of Image Completion Development In this section, we outline the accuracy verification environment of image inpainting and describe the verification method and its results. Overview. We verified whether it is possible to sup- plement the missing parts after virtual removal with- out producing an unnatural image. However, be- cause it is difficult to prepare photographs before and after the removal of structures under different con- ditions, we manually created a mask image to simu- late the state in which the structures to be removed are detected by semantic segmentation. Then, the The implementation was done using Python. Also, completed image automatically generated by GAN when the landscape is visualized outdoors, it is not is compared with the captured image, which is the always possible to use the high-performance PC ground truth, and evaluated. When calculating the needed to perform DR. Therefore, we use a system evaluation index, the masked area is compared and that uses Web Real-Time Communication (WebRTC) evaluated. The peak signal-to-noise ratio (PSNR) was [4] to communicate between high-performance PCs used as an index for quantitative evaluation. The and mobile devices. WebRTC is a technology that higher the value of the PSNR, the better the evalu- enables peer-to-peer communication within a web ation. The minimum value of the PSNR is 0 and the page, making it possible to conduct voice chat, video maximum value is not defined. The calculation for- chat, and file sharing between web browsers. This mula is shown in the equations below. can be done by bringing only a mobile device to the M N site, thereby eliminating the need to bring a high- 1 2 MSE = ∑ ∑(xj,k − yj,k) (1) performance PC to the site. MN j=1k=1 The processing procedure is shown below: 2 MAXI PSNR = 10 · log10 (2) 1. Establish a connection between devices via a MSE web browser using WebRTC. Here, MAXI is the maximum value of the pixel 2. Obtain the current landscape using the camera value. If the pixel value is expressed from 0 to 255, the of the mobile device. value is 255. When calculating the PSNR for an RGB 3. Send the acquired current scenery from the mo- image, the pixel values are compared for each RGB bile device to a high-performance PC. value pixel by pixel, and the average of the squares 4. In the high-performance PC, apply the proposed of the differences is calculated. Then, the values for method to remove the target objects. each RGB value are added together and divided by 3 5. Send the automatically generated post-removal to obtain the mean squared error (MSE). landscape from the high-performance PC to the Because each pixel is compared, the evaluation is mobile device. made without considering the relationship with the 6. Display the received future view on the screen of surrounding area. Also, even for similar images, the the mobile device. evaluation may vary greatly with a shift of just one In this verification, we used a tablet as a mobile de- pixel. In contrast, because there is only one arbitrary vice, but it can also be run on a smartphone or a note- value used for evaluation and the method of deter- book PC having a web camera. mining the arbitrary value is shown, objective evalu- ation is easy.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 533 Method. The verification procedure is as follows: Figure 2 Results of 1. Take a picture. Experiment 2. 2. Create a mask image manually. Relationship 3. Input the photo and mask image into the GAN. between mask ratio 4. Compare the generated image with the cap- and PSNR for each tured image and calculate the PSNR. landscape element The photo used was 3,432 pixels in height and 4,576 included in the pixels in width. In this verification, we conducted two mask area. experiments: Experiment 1, in which the percentage of mask images in the image was fixed and only the background element was changed, and Experiment 2, in which the percentage of mask images in the im- Field Verification age was changed. In Experiment 1, one mask image In this section, we describe the outline of the field ver- was created for each of the eight photos taken, for ification, the verification method, and its results. a total of eight sets of images. In Experiment 2, we Overview. We used the prototype system to verify used six photos and created two or four mask images whether virtual demolition is possible and whether for each photo, for a total of 22 sets. The images were the processing speed is fast enough to be used for verified by resizing them to 480 pixels in height and landscape studies. We also measured the time lag 640 pixels in width when performing image comple- that occurs from the time the real scene acquired by tion. This reduced size was used for the field verifica- the mobile device is displayed on the browser to the tion described in the next section. time it was sent to the server PC, processed, and re- Results. The results of Experiment 1, in which image turned. completion was performed with the same mask per- Method. In the verification of the processing speed, centage, are shown in Table 2 for the relationship be- the time required to execute the processing for one tween background elements and PSNR. frame was measured and the frames per second (fps) was calculated to check whether the processing Table 2 speed was conducive to the landscape study. In the Results of verification of time lag, the lapsed time from trans- Experiment 1 mission of the initial image to reception of the pro- cessed image was calculated by averaging the results of multiple measurements. The verification procedure is shown below. 1. Conduct a landscape study while measuring and recording the time it takes for the server PC to receive, process, and retransmit the landscape data. The area of the mask used in this verification ac- 2. Measure the time lag from the stopwatch dis- counted for 2.6% of the total area. played on each screen by taking a picture of the The relationship between the PSNR and the per- stopwatch count with a mobile device and com- centage of the total mask used in Experiment 2 for paring the transmitted and received images at each background element is shown in Figure 2. the same time.

534 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 3 (a) Verification scene (b) Location of the experiment

Table 3 3. Calculate the average of the measured process- Experimental ing times in fps. conditions 4. Calculate the average of the measured time lags. The experimental conditions are shown in Table 3. The internet speed was averaged five times on a web browser and used as the experimental condition. Fig- ure 3 shows the location and scene of the verification, Table 4 respectively. The time it takes to receive, process, Results. Table 4 shows the results of measuring the and send an image. time that lapsed between receiving the real scene and sending the processed result (processing time) from the server PC, Figure 4 shows the time lag mea- Figure 4 sured on the mobile device side, and Figure 5 shows Change in the time a part of the input real scene and the output result it takes for a when the proposed method is applied to the input. landscape acquired by a mobile device to be processed and displayed.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 535 tern and other display rules do not match. In con- Figure 5 trast, when the background element was only a tree Examples of input or only the sky, there was no sense of unnaturalness scenes and output (Figure 6). results Comparison of the images generated in Exper- iment 2 show that when the background element was a tree, the appearance of the mask did not change significantly as the ratio of the mask to the whole image increased. In contrast, when the back- ground element was not a tree, the appearance of the mask worsened as the ratio of the mask to the whole image increased, resulting in a complemen- tary result that did not match the surrounding ele- ments. Also, regardless of the background element, the PSNR tended to decrease as the percentage of the mask with respect to the total image increased (Fig- ure 7).

Figure 6 Example results of Experiment 1

The difference between the display range of the real scene and that of the output result at the same time is due to the time lag between the transmission and the reception.

DISCUSSION In this section, we use the results of the validation ex- periments to discuss image completion by the pro- posed method. These results indicate that this method is most suit- able when the completion target is only a tree be- Image Completion cause it can be completed more naturally even when When the images generated in Experiment 1 were the mask ratio is increased. Also, if the background evaluated by focusing on the PSNR values, the results element is the sky, the method is suitable when the were highly accurate when there was only one type mask ratio is less than 10%. In contrast, if a retained of background element. However, when the com- structure remains in the background of the object to pletion area contained a structure, the results were be removed, and if there is a pattern on the surface of unnatural compared with the case where the back- this structure, it is difficult to apply this method be- ground element is a tree or the sky. This is due to cause it cannot be complemented properly and re- the presence of patterns on the object, and even if sults in an unnatural image. the color of the complementary part is similar to that in the correct image, it will look strange if the pat-

536 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 7 not be represented correctly. This is because seman- Example results of tic segmentation can detect objects only by type. To Experiment 2 solve this problem, instance segmentation is needed instead of semantic segmentation, which can detect overlapping objects of the same type as different ob- jects. However, at present, this is difficult because no dataset can use instance segmentation to detect structures, which are the main removal targets when visualizing the future landscape. Another issue is that objects that cannot be detected by semantic seg- mentation cannot be targeted for removal.

CONCLUSION When demolishing existing structures and construct- ing new structures and open spaces, the future land- scape should be visualized and a consensus should be reached among project stakeholders. In this study, we proposed a DR method using semantic segmentation and GAN and showed the usefulness of the method through accuracy verification and field tests. Field Verification The conclusions of this study are as follows: The field verification results demonstrated that the proposed method makes it possible to visualize the • By combining semantic segmentation and GAN, future landscape after the demolition and removal it is possible to visualize the future landscape of the target structure without prior preparation. In without any prior preparation. cases where there is no time to prepare images in ad- • It is possible to automatically detect objects to vance, or where 3D objects cannot be created, the be removed, perform virtual removal and auto- proposed method can be used to represent the land- completion, and run the process at an average scape after the removal of the structures. speed of 8.75 fps. However, the problem is that there is a gap be- The future challenge is to make it possible to use this tween the input image and the output image dis- method even when the objects to be removed and played at the same time. This may be due to the the objects not to be removed are of the same type communication between the mobile device and the and overlap. server PC and the processing time on the server PC. It was also confirmed that there were frames where the object to be removed could not be detected, and ACKNOWLEDGEMENTS thus the object to be removed appeared and disap- This research has been partly supported by the Japan peared in different frames. The average frame rate Society for the Promotion of Science, KAKENHI Grant calculated from the processing time was 8.75 fps. Number JP19K12681. Also, because we use semantic segmentation to All images were created by the authors from detect the objects to be removed, if the objects to 2020 to 2021. be removed overlap with objects of the same type that are not to be removed, the future landscape can-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 537 REFERENCES data using a convolutional neural network’, Remote Bishop, I.D. and Lange, E. 2005, Visualization in landscape Sensing, 12, pp. 1-21 and environmental planning, Technology and appli- Long, J., Shelhamer, E. and Darrell, T. 2015 ’Fully convo- cations lutional networks for semantic segmentation’, Pro- Caudell, T.P. and Mizell, D.W. 1992 ’Augmented reality: ceedings of the IEEE Computer Society Conference on an application of heads-up display technology to Computer Vision and Pattern Recognition, pp. 431- manual manufacturing processes’, Proceedingsof the 440 Twenty-Fifth Hawaii International Conference on Sys- Mann, S. (eds) 1994, Mediated Reality, M.I.T. Media tem Sciences, pp. 659-669 Lab Perceptual Computing Section Cambridge, Ma Chao, P., Kao, C.-Y., Ruan, Y., Huang, C.-H. and Lin, Y.-L. Technical Report TR 260 2019 ’HarDNet: A low memory traffic network’, Pro- Mori, S., Ikeda, S. and Saito, H. 2017, ’A survey of di- ceedings of the IEEE International Conference on Com- minished reality: Techniques for visually concealing, puter Vision, pp. 3551-3560 eliminating, and seeing through real objects’, IPSJ Flores, A. and Belongie, S. 2010 ’Removing pedestrians Transactions on Computer Vision and Applications, 9, from Google Street View images’, 2010 IEEE Com- pp. 17- puter Society Conference on Computer Vision and Pat- Navab, N., Bani-Kashemi, A. and Mitschke, M, 1999 tern Recognition - Workshops, CVPRW 2010, pp. 53-58 ’Merging visible and invisible: Two Camera- Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Augmented Mobile C-arm (CAMC) applications’, Pro- Warde-Farley, D., Ozair, S., Courville, A. and Ben- ceedings - 2nd IEEE and ACM International Workshop gio, Y. 2014 ’Generative adversarial nets’, Advances on Augmented Reality, IWAR 1999, pp. 134-141 in Neural Information Processing Systems, pp. 2672- Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T. and 2680 Efros, A.A. 2016 ’Context Encoders: Feature Learning Huang, W. and Zheng, H. 2018 ’Architectural drawings by Inpainting’, Proceedings of the IEEE Computer Soci- recognition and generation through machine learn- etyConferenceonComputerVisionandPatternRecog- ing’, Recalibration on Imprecision and Infidelity - Pro- nition, pp. 2536-2544 ceedings of the 38th Annual Conference of the Associ- Ullman, S. 1979, ’The interpretation of structure from ation for Computer Aided Design in Architecture, ACA- motion’, Proceedings of the Royal Society of London. DIA 2018, pp. 156-165 Series B. Biological Sciences, 203, pp. 405-426 Iizuka, S., Simo-Serra, E. and Ishikawa, H. 2017 ’Glob- Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X. and Huang, T.S. ally and locally consistent image completion’, ACM 2018 ’Generative Image Inpainting with Contextual Transactions on Graphics Attention’, Proceedings of the IEEE Computer Society Inoue, K., Fukuda, T. and Yabuki, N. 2018 ’Tracking Ro- Conference on Computer Vision and Pattern Recogni- bustness and Green View Index Estimation of Aug- tion, pp. 5505-5514 mented and Diminished Reality for Environmental Zokai, S., Esteve, J., Genc, Y. and Navab, N. 2003 ’Mul- Design: PhotoAR+DR 2017 project’, Proceedings of tiview paraperspective projection model for dimin- the 23rd International Conference on Computer-Aided ished reality’, Proceedings - 2nd IEEE and ACM Interna- Architectural Design Research in Asia (CAADRIA 2018), tional Symposium on Mixed and Augmented Reality, pp. 339-348 ISMAR 2003, pp. 217-226 Ki, D. and Lee, S. 2021, ’Analyzing the effects of Green [1] https://github.com/PingoLH/FCHarDNet View Index of neighborhood streets on walking time [2] https://www.cityscapes-dataset.com/ using Google Street View and deep learning’, Land- [3] http://www.image-net.org/ scape and Urban Planning, 205, p. - [4] https://webrtc.org/ Kido, D., Fukuda, T. and Yabuki, N. 2020, ’Diminished re- ality system with real-time object detection using deep learning for onsite landscape simulation dur- ing redevelopment’, Environmental Modelling and Software, 131, p. - Li, Q., Shi, Y., Auer, S., Roschlaub, R., Möst, K., Schmitt, M., Glock, C. and Zhu, X.X. 2020, ’Detection of undoc- umented building constructions from official geo-

538 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Mixed Reality Landscape Visualization Method with Automatic Discrimination Process for Dynamic Occlusion Handling Using Instance Segmentation

Mizuki Nakabayashi1, Tomohiro Fukuda2, Nobuyoshi Yabuki3 1,2,3Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University [email protected] 2,3{fukuda|yabuki}@see.eng.osaka-u. ac.jp

Mixed reality (MR), which blends real and virtual worlds, has attracted attention as a visualization method in landscape design. MR-based landscape visualization enables stakeholders to examine landscape changes at actual scale in real-time at the actual project site. One challenge in MR-based landscape visualization is occlusion, which occurs when virtual objects obscure physical objects that are in the foreground. Previous research proposed an MR-based landscape visualization method with dynamic occlusion by using semantic segmentation of deep learning. However, this method has two problems. The first is that the same kind of objects that are grouped into one or overlapped types are classified as the same object, and the other is that the foreground objects have to be defined in pre-processing. In this study, we developed a system for large-scale MR landscape visualization that enables the recognition of each physical object individually using instance segmentation, and it is possible to accurately represent the positional relationship by comparing the coordinate information of the 3D virtual model and all physical objects.

Keywords: landscape visualization, mixed reality, instance segmentation, dynamic occlusion handling, deep learning

INTRODUCTION users, so it is necessary to visualize the expected land- Background scape appearance upon project completion and ac- To form a high-quality landscape, it is essential to curately communicate the project content and poli- visualize landscape projects and to build a consen- cies. Mixed reality (MR), which blends real and virtual sus among stakeholders for an architectural and ur- worlds (Milgram and Kishino, 1994), has attracted at- ban design (Bishop et al., 2005). Stakeholders include tention as a visualization method in landscape design not only experts such as architects and government (Tamura et al., 2001). MR-based landscape visualiza- officials, but also non-experts such as residents and tion can produce a realistic landscape visualization by

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 539 overlaying a 3D virtual model of an object that does Kido et al. (2021) proposed an MR-based land- not yet exist on a camera view of a given area, and it scape visualization method with dynamic occlu- enables stakeholders to examine the change of the sion using semantic segmentation that classifies and landscape at actual scale in real time at the actual color-codes all pixels in an image by object type project site (Haynes et al., 2018). with deep learning. This system realized MR-based One challenge in MR-based landscape visualiza- landscape visualization even when a physical object tion is occlusion. It is hard to render the relation- changes its shape over time or moves. However, be- ship between real and virtual environments correctly. cause semantic segmentation classifies real environ- Because MR is realized by superimposing a 3D vir- mental elements into physical object types, this sys- tual model on the real scene from the camera, oc- tem has the problem that even multiple physical ob- clusion occurs when virtual objects obscure physical jects of the same type are classified as a single object, objects that exist in the foreground (Figure 1 (b)). In and as a result, it is difficult to handle occlusion when landscape visualization, the relationship with the sur- a 3D virtual model is superimposed between multi- rounding environment is very important. Thus, the ple physical objects of the same type (Figure 1 (c)). disruption of the positional relationship between the Because cities contain many physical objects of the real and virtual environments due to occlusion has same type, it is important to classify these objects in- a significant impact on stakeholders’ intuitive under- dividually in landscape visualization. standing of the results of the visualization. Figure 1 Previous Research Occlusion problem Inoue et al. (2018) developed a landscape visualiza- in MR tion system with dynamic occlusion using a 3D vir- tual model of the surrounding environment created by multiple photos with structure from motion tech- nology in the pre-processing. This system realized occlusion by reproducing the positional relationship between physical objects and a 3D model on a com- puter and making the part of the 3D virtual model that should be hidden by the physical objects in front of it invisible. However, it is necessary to create a 3D virtual model of the surrounding environment in pre- processing, and if a physical object such as vegeta- tion changes its shape over time, it can be difficult to appropriately handle occlusion by changing the oc- clusion model’s shape. Moreover, this method has Objective difficulty handling occlusion if the targets are mov- This paper proposes a method for large-scale MR ing objects such as vehicles and pedestrians. landscape visualization that enables the recognition Methods without pre-processing have been of physical objects of the same type individually and demonstrated for occlusion handling using depth accomplishes visualization when a 3D virtual model information. However, the approach by Zhu et al. is superimposed between multiple physical objects (2010) cannot perform large-scale landscape visual- of the same type. In the previous research (Kido et ization, while that of Roxas et al. (2018) cannot be al., 2021), the foreground objects had to be defined in applied to moving objects. pre-processing. Thus, the purpose of this study was to develop a system that automatically discriminates

540 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 2 Conceptual diagram of proposed method

physical objects that exist in front of or behind the 3D tion with dynamic occlusion can be realized when a virtual model based on the positional information of 3D virtual model is superimposed between multiple the 3D virtual model without pre-processing. physical objects of the same type.

Figure 3 PROPOSED METHOD Flow chart of The terms of the core technologies used in the pro- proposed method posed method are defined below. Deep learning is a part of machine learning, which is a method of learning concepts of a target in a multi-layered struc- ture. Semantic segmentation is a technique that clas- sifies and color-codes all pixels in an image by object type with deep learning. Object detection is a tech- nique that classifies objects in an image by object type and indicated its location by bounding box with deep learning. Figures 2 and 3 respectively show the conceptual diagram and flowchart of our proposed method. In this system, we use an instance segmen- tation technique that adds semantic segmentation to object detection with deep learning. This instance segmentation technique detects each object individ- ually and extracts the region where the object exists for each object, which enables the model to distin- guish the boundaries between objects of the same type. Therefore, large-scale MR landscape visualiza-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 541 Connection between Server and Client PC ject). When an instance segmentation system is built, real- • The camera roll is 0 degrees. time segmentation takes a huge amount of process- • In the case of multiple 3D virtual models or 3D ing and requires a high-end desktop computer. How- virtual models containing cavities, no real ob- ever, landscape visualizations are often conducted jects are inserted between the 3D virtual models outdoors, and a practical system needs to operate (Figure 4). on a laptop computer or tablet that is easy to carry around. Thus, we divided the system into a client Figure 4 PC used for MR landscape simulation and a server Examples where the PC used for instance segmentation. The proposed proposed method system connects instance segmentation with a game cannot be applied engine that provides the MR development environ- ment. A real scene taken by the client PC and the ground position of a 3D virtual model to discriminate the positional relationship between the 3D virtual model and physical objects are sent to the server PC Under these criteria, an object in the foreground from the client PC frame by frame. The server PC cre- always appears lower in the camera image than an ates an instance segmentation image from the real object in the background. In other words, it is pos- scene and also creates an image that extracts only the sible to recognize and maintain the positional rela- physical objects that exist in front of the 3D virtual tionship of objects in the camera image according to model (discrimination image) using the ground posi- their ground coordinates. Using this idea, we discrim- tion of the 3D virtual model and instance segmenta- inated between physical objects that exist in front of tion. The discrimination image is sent to the client PC the 3D virtual model (physical objects under occlu- frame by frame. We connected the instance segmen- sion) and physical objects that exist behind the 3D tation and game engine by this procedure. We used virtual model by comparing the ground coordinates a Python web application framework known as Flask of the 3D virtual model and physical objects. The al- for communication. gorithm is shown in Figure 5. The ground position of the 3D virtual model was set to the position of Algorithm for Automatic Discrimination the bottom edge of the 3D virtual model (the tree Instance segmentation enables us to identify objects roots in Figure 5), which is the closest 3D virtual ob- of the same type individually, but it is not possible ject as seen from the camera. The ground coordi- to visually determine which of the individually iden- nates of the physical objects were set to the coordi- tified physical objects are in front of the 3D model nates of the bottom edge of the bounding box (rect- and which of them are behind the 3D model. There- angle) obtained by instance segmentation. First, a fore, we discriminated the positional relationship be- 3D virtual model was imaged by a virtual camera in tween the 3D virtual model and physical objects by the game engine on the client PC, and the ground comparing the ground position of the 3D model with position of the 3D virtual model was converted from the coordinates of the physical objects. The criteria the world coordinate system to the screen coordinate for this method are as follows: system to obtain the ground position as seen from the camera. This position was sent to the server PC • The ground is almost 0 degrees in a crosswise di- along with the real scene video. On the server PC, rection with respect to the camera. the coordinates of the bottom edge of the bounding • The object under occlusion is in contact with the box (rectangle) of each physical object were obtained ground (not in the air or on top of any other ob- by instance segmentation. These coordinates were

542 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 5 Algorithm for automatic discrimination

Figure 6 compared, and the physical objects with ground co- MR landscape ordinates larger than the 3D model’s ground posi- visualization with tion (positive downward direction) were considered occlusion as the occluded physical objects and masked with green ((R, G, B) = (0, 255, 0)) (discrimination image). The discrimination image was sent to the client PC, and the mask image was generated by changing the green parts of the discrimination image to white ((R, G, B) = (255, 255, 255)) and the other parts to black ((R, G, B) = (0, 0, 0)) in the game engine. By perform- ing this process for each frame, real-time occlusion VALIDATION EXPERIMENT representation is made possible. A prototype system was constructed to verify the ef- fectiveness of the proposed method. In this vali- MR Landscape Visualization with Occlusion dation, we adopted YOLACT++ (Bolya et al., 2019b) An occlusion image is synthesized by the mask image as the instance segmentation model. YOLACT++ is and 3D virtual model. MR landscape visualization is an improved version of YOLACT (Bolya et al., 2019a), performed by overlapping the occlusion image and which is a fast instance segmentation model, and it the input image and visualizing them with a virtual is as accurate as Mask R-CNN (He et al., 2017), which MR camera (Figure 6). is a highly accurate instance segmentation model. The dataset used for instance segmentation was Mi- crosoft COCO (Lin et al., 2014) dataset which is not labeled by color, and detectable cars were targeted for occlusion.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 543 Figure 7 Status of verification

Verification of Occlusion at 34.8 degrees north latitude and 135.5 degrees east Because the prototype system could operate only longitude. The camera was facing north to south, in a LAN environment, we used pre-recorded video but there was no shadow or backlighting at this time to verify the system. The video was taken from the of day. In this verification scenario, installation of a sidewalk to capture the scene shown in Figure 7(a), median divider with trees and plants was planned with cars running on the roadway (Legal speed limit for a four-lane roadway (Figure 7(b)), and cars in the 60km/h). It was taken at 5 p.m. on January 30, 2021, closest two lanes were targeted for occlusion. A 20-s

Figure 8 Results of occlusion handling

544 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 video was used, fixed for 7 s from 0 s to 7 s, rotated up- 15,000, IoU > 0.8. Instance segmentation creates seg- ward in the pitch direction for 7 s from 7 s to 14 s, ro- ments within the detected area because only a small tated downward in the pitch direction for 4 s from 14 portion of the entire physical object is seen. s to 18 s, and rotated leftward in the yaw direction for Figure 9 2 s from 18 s to 20 s. In this visualization, the mask im- IoU application to age was blurred to improve the optical consistency of pixels the boundary between the 3D virtual model and the occlusion target. The visualization results are shown in Figure 8. The processing speed of the whole sys- tem was 7 frames per second.

Accuracy of Instance Segmentation Figure 10 We verified the accuracy of mask images that ex- Accuracy of mask tracted only occlusion targets. For this verification, images we used 145 mask images generated in the visualiza- tion, which were saved frame by frame. Intersection over union (IoU) was adopted as the accuracy eval- uation index. IoU was calculated according to equa- tion (1), and is obtained by dividing the number of pixels in the product set of the true pixels and the predicted pixels by the number of pixels in the union of the true and predicted pixels (Figure 9). In this system, it is important to accurately extract the area where the occlusion objects exist to perform the vi- sualization without any visual discomfort. Because over- or under-extraction of the target area has a sig- nificant impact on the results of the visualization, IoU, which can be evaluated for both over- and under- extraction, was adopted as the evaluation index. TP IoU = (1) TP + FP + F N Figure 11 Figures 10 and 11 show results of comparing the Relationship mask image generated by the prototype with the between number of true image created visually and calculating the IoU. pixels and IoU Of the 145 images, 47 contained occlusion targets, and the average IoU of the 145 images was 0.92. Be- cause IoU is based on a standard value of 0.5 (Jabbar et al., 2017), the accuracy of our system in extract- ing occlusion targets is significantly high. Figure 11 shows the relationship between IoU and the number of occluded pixels in the image. When the number of occluded pixels is less than 15,000, IoU = 0, but when the number of pixels under occlusion is more than

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 545 CONCLUSIONS mented Reality for Flood Visualization’, Environmen- The conclusions of the present study are as follows: tal Modelling & Software, 109, pp. 380-389 He, K., Gkioxari, G., Dollar, P. and Girshick, R. 2017 ’Mask • We developed an MR landscape visualization R-CNN’, Proceedings of the IEEE International Confer- method that uses instance segmentation to rec- ence on Computer Vision (ICCV), pp. 2961-2969 ognize physical objects of the same type individ- Inoue, K., Author, 1, Cao, R. and Author, 2 2018 ually and in situations where multiple physical ’Tracking Robustness and Green View Index Esti- mation of Augmented and Diminished Reality for objects of the same type appear together. Environmental Design: PhotoAR+DR 2017 project’, • The proposed MR landscape visualization Proceedings of the 23rd International Conference on method can automatically draw the positional Computer-Aided Architectural Design Research in Asia relationship between the physical objects and (CAADRIA 2018), pp. 339-348 3D virtual model, without any pre-processing, Jabbar, A., Farrawell, L., Fountain, J. and Chalup, S. 2017 by comparing the coordinates of the physical ’Training Deep Neural Networks for Detecting Drink- ing Glasses Using Synthetic Images’, Proceedings of object obtained by instance segmentation with the 24th International Conference of Neural Informa- the position of the 3D virtual model obtained in tion (ICONIP), pp. 354-363 the game engine. Kido, D., Author, 1 and Author, 2 2021, ’Assessing fu- • The mask images generated by the proposed ture landscapes using enhanced mixed reality with method for correct landscape visualization were semantic segmentation by deep learning’, Advanced evaluated and found to be highly accurate. Engineering Informatics, 48, p. - Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ra- Future work includes migrating the system from the manan, D., Dollar, P. and Zitnick, C. L. 2014, ’Mi- LAN-based communication used in this study to crosoft COCO: Common Objects in Context’, Euro- pean Conference on Computer Vision (ECCV), 8693, Internet-based communication for on-site use and pp. 740-755 reducing the number of adaptive criteria required for Milgram, P. and Kishino, F. 1994, ’A Taxonomy of Mixed visualization. Reality Visual Displays’, IEICE Transactions on Infor- mation Systems, E77-D(12), pp. 1321-1329 Roxas, M., Hori, T., Fukiage, T., Okamoto, Y. and Oishi, T. ACKNOWLEDGMENT 2018 ’Occlusion Handling using Semantic Segmen- This research was partly supported by the Japan So- tation and Visibility-Based Rendering for Mixed Re- ciety for the Promotion of Science, KAKENHI Grant ality’, Proceedings of the 24th ACM Symposium on Vir- Number JP19K12681. tual Reality Software and technology (VRST 2018), pp. All images were created by the authors from 1-8, 20 2019 to 2021. Tamura, H., Yamamoto, H. and Katayama, A. 2001, ’Mixed reality: future dreams seen at the border between real and virtual worlds’, IEEE Computer Graphics and REFERENCES Applications, 21(6), pp. 64-70 Bishop, I. and Lange, E. 2005, Visualization in Landscape Zhu, J., Pan, Z., Sun, C. and Chen, W. 2010, ’Handling Oc- and Environmental Planning: Technology and Appli- clusions in Video-Based Augmented Reality Using cations, Taylor & Francis Depth Information’, Computer Animation and Virtual Bolya, D., Zhou, C., Xioa, F. and Lee, Y. 2019a ’YOLACT Worlds, 21, pp. 509-521 Real-time Instance Segmentation’, Proceedings of the IEEE/CVF International Conference on Computer Vi- sion (ICCV), pp. 9157-9166 Bolya, D., Zhou, C., Xioa, F. and Lee, Y. 2019b ’YOLACT++ Better Real-time Instance Segmentation’, arXiv preprint arXiv:1912.06218 Haynes, P., Lange, S.H. and Lange, E. 2018, ’Mobile Aug-

546 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Development of an Augmented Reality System with Reflection Implementation for Landscape Design Visualization using a Planar Reflection Method in Real- Time Rendering

Hao Chen1, Tomohiro Fukuda2, Nobuyoshi Yabuki3 1,2,3Division of Sustainable Energy and Environmental Engineering, Graduate School of Engineering, Osaka University [email protected] 2,3{fukuda|yabuki}@see.eng.osaka-u. ac.jp

Augmented reality (AR) in landscape design review scenarios has become an important tool that helps designers express their designs and allows stakeholders to more easily understand how the designs will look on the actual site. This study aimed to add the reflection of a virtual design model on the surface of the water in an existing AR system, thereby providing a more complete representation of the waterfront landscape design. First, we constructed an AR system using a smartphone linked to a computer. Then, a virtual surface model was predefined manually according to the water surface area on-site. A planar reflection method was introduced to generate an accurate reflection effect in real-time. Moreover, the reflection was simulated to ripple together with the water surface, providing a visually authentic look. Thus, the virtual model was able to accurately display the real-time reflection effect on the water surface in this realistic environment. Our findings indicate that future tasks could involve the implementation of other interactive optical effects for landscape design visualization, such as refraction simulation for underwater illumination design.

Keywords: augmented reality, reflection, landscape architectural design, waterscape, interactive visualization, computer-aided design in architecture

INTRODUCTION scape design. Existing AR technologies have made AR systems allow experiments to be performed on- it possible to blend virtual design models and realis- site with a 1:1 scale design by displaying virtual data tic environments, but the insufficiency of physical in- onto physical spaces, including walls, floors, desks, teraction between design models and environments and objects (Milovanovic et al. 2017). Thus, AR plays still limits the use of AR in practice. Reflection is a an essential role in the visualization process of land- specific example that is listed as one of the vital char-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 547 acteristics of water in landscape architectural design Reflection in real-time rendering (Booth 1989). However, the reflection of virtual mod- Rendering of virtual models for AR application sce- els cannot be realized in AR systems, which brings narios must reach speeds that allow real-time inter- about difficulties in enabling stakeholders to appre- action with users, and this real-time rendering can ciate the full intent of landscape design. be achieved by the application of many technolo- The aim of this study was to develop an AR sys- gies. Real-time rendering involves rapidly generating tem that can realize real-time reflection effects in images on a computer (Akenine-Möller et al., 2019) landscape design review scenarios. The proposed such that every image is produced according to the system is able to generate reflections of virtual land- viewer’s motion. Real-time rendering techniques are scape models on water surfaces in real-time based widely used in games and other fields that require on predefined water surface models, thus realizing human-computer interaction to meet the expression more realistic design review effects and more effi- of virtual objects. cient communication between designers and stake- In real-time rendering, there are different techni- holders (Fig. 1). cal implementations to realize the reflection of one object on another object. Different methods have different performance needs and expressiveness. Jo- Figure 1 hanson (2004) summarized two methods for simu- AR without and lating reflections on water surfaces: global reflection with reflection and local reflection. The former applies to a case in effect (mock-up) which the object being reflected can be considered infinitely far away, whereas the latter applies to a case in which the object being reflected cannot be con- sidered infinitely far away. In terms of effect, global LITERATURE REVIEW reflection involves a more concise rendering process, AR in the design and planning field but unfortunately, it does not accurately represent This work is based on a series of previous studies on the position of the reflected image and the reflected the application of AR in design and planning review object, so we use local reflection to achieve real-time scenarios. Stintzing et al. (2020) applied AR to the ed- reflection. ucational field to teach students about the profound The limitation of the local reflection method is changes in a cultural landscape. Gordon and Mano- that it requires the reflecting surface to be a plane, sevitch (2011) introduced the concept of augmented but this limitation is acceptable in most landscape deliberation in a pilot project. In their study, a com- design review scenarios because the water surface munity and a designer came together, basing their can be approximated as a plane. There is also a tech- communication on augmented views, which demon- nique called screen-space-reflection, which can pro- strated community engagement in urban planning. vide more accurate and realistic reflections (Beug, Kido et al. (2021) proposed a system capable of ex- 2020), but it was not used in the present study be- pressing the occlusion relationship that occurs when cause it is easy to lose the reflection effect in some virtual objects in the foreground are obscured by viewpoints thus does not fit the landscape design re- physical objects in mixed reality for landscape visual- view scenarios. ization. Broschart and Zeile (2015) gave an overview of the use of AR technologies in architectural design Ripple simulation in real-time rendering and urban planning. Hu et al. (2006) implemented sea rendering in real- time rendering, which included accurate reflection,

548 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 2 System workflow

ripple, Fresnel effects, and refraction. The idea of lo- space and realizes the real-time reflection effect of cal reflection was used in this case. Johanson (2004) the virtual object on the water surface model in the also proposed a distort-based refraction simulation virtual space. The camera in the virtual space and the method. In the present paper, the distort method viewpoint in the actual scene need to maintain the was adopted, but it was applied to simulate water consistency of movement and angle. surface ripples. Proposed system introduction METHODOLOGY We propose a system consisting of a smartphone and Workflow a computer. The smartphone is used to obtain video streams and viewpoint motion information and then The workflow of the system in this study is shown in transmits data to the computer, which is used to in- Fig. 2. First, the system needs to detect the shape tegrate the video streams, viewpoint motion infor- and spatial location of the water surface and produce mation, and real-time planar reflections in order to the corresponding scene in virtual space. This system achieve the expected AR effect. The phone and the currently requires pre-modeling and advance place- tripod are fixed to the site of the design review and ment of the water surface model, which needs to be the viewpoint can be freely rotated, while the com- adjusted by the user at the landscape design review puter is controlled by a presenter. The proposed sys- site because it cannot be automated yet. Then, the tem configuration is shown in Fig. 3. The demon- user imports the virtual design model into the virtual

Figure 3 Proposed system configuration

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 549 strator needs to perform a series of operations on the puter screen to achieve the basic AR effect (i.e., the computer side as follows: overlay of virtual objects and the real scene). When the camera is moved or rotated, the virtual objects 1. Customize the surface model representing wa- and the real scene maintain the correct viewpoint in ter on the computer side in advance and cor- synchronicity. rectly position it. 2. Import the design model to be displayed in ad- Planar reflection vance on the computer side and place it in the Planar reflection is a technique that provides a to- appropriate location. tal reflection effect on a flat surface. This technique 3. Place the surface model representing the water ensures the accuracy of the reflected image position in a suitable location and add appropriate mirror but can only be applied to plane models, which is reflective material to it. A detailed explanation acceptable for the scenes in this study. The prin- of this material is provided below in the Imple- ciple of planar reflection is that the scene of a pla- mentation section (planar reflection implemen- nar reflection from one viewpoint is the same as that tation). from another viewpoint, such that there is symmetry In step 1, it is best to model the plane to match the about the plane. Therefore, the reflection effect can shape of the water, but this process is usually labor- be achieved by generating a symmetric camera view intensive and each model only works for one water in real-time and assigning that view to the plane as surface on a specific site. However, because reflec- mirror reflective material. tions in waterscapes are often complete, designers rarely intentionally cut reflections using the shape of Simulation of ripple the water surface, and the workflow can be simplified Reflections on real water surfaces are not simple mir- by creating a sufficiently large plane model and ig- ror reflections but can be disturbed by the rolling of noring the shape of the water surface. water surfaces due to the wind, interference of the The final AR effect will be visible on both the water’s color, the influence of the light of the envi- smartphone screen and the computer screen. ronment, and so on. This series of complex changes gives the waterfront landscape reflection its beauty Fundamental AR system and makes the reflection different from simple mir- In this paper, a fundamental AR system is defined as a ror reflections, often becoming a highlight in the de- series of functions that integrate the realistic camera sign. To express the impact of the real environment motion and virtual camera motion and overlies the on the water surface as much as possible based on virtual camera’s view onto the realistic camera’s view. the AR system in this paper, complex interfering ele- The camera on the phone captures the video stream ments need to be classified and implemented sepa- and sends it in real-time to the virtual space in the rately. This study focuses on disturbance of the water computer, and the virtual camera in the virtual space surface reflection by ripples generated by wind dis- receives the video stream as the underlying layer for turbance. the camera. The motion trajectory of the virtual cam- We introduce one of the pseudo techniques in era and the phone’s camera can be synchronized us- the field of real-time rendering to simulate the rip- ing the phone’s motion information recorded by the ples. The main idea is to replace the original texture phone itself. Our design model is added to the game (UV) coordinates of each pixel with the color value of engine of the computer and positioned correctly, and the distortion map to achieve texture distortion. In the virtual camera captures the virtual model and dis- this method, the effect of distortion is controlled by plays the virtual model at the bottom of the com- two parameters: one is the selected distortion map, which is produced by Photoshop or other digital con-

550 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 tent creation (DCC) software; the other is the degree erated using the built-in skybox in Unity. In addition, of distortion, which is defined in a shader file and ex- to make the AR effect more realistic, all the pixels of posed in the built-in parameters panel of the game the background should exhibit transparency, which engine (Unity), and can be adjusted by the user to can make the facets in the AR system show only the match the actual environment on-site. reflected image and make the plane itself invisible. To achieve this, we can use alpha channels by set- IMPLEMENTATION ting the camera parameters within Unity to render To balance the formality and portability of the pro- the texture of an object-reflected image pixel with posed system at the landscape design review site an alpha value of 1 (opaque) and a background pixel while minimizing costs and simplifying operational with an alpha value of 0 (transparent). In the shader, processes, this study used a smartphone linked to a we can specify the blend mode as “Blend SrcAlpha laptop computer as hardware configuration. The re- OneMinus SrcAlpha”,which means that once the ren- quired devices were a tripod, a smartphone, and a dering of the reflective plane object starts, the color laptop. Although the proposed system can work with values of all the fully transparent pixels of the object a smartphone without a fixed position, considering will be completely discarded from the existing frame the accuracy of position recording and that multiple to expose the underlying layer, the color values of all viewers will view the scene through the smartphone, fully opaque pixels of the object will directly cover we used a tripod to fix the position of the smart- the underlying layer, and the translucent pixels will phone, while allowing it to rotate freely. be somewhere between completely discarded and completely covered. However, in this study, there Establish an AR system in Game Engine were no translucent pixels. In this study, we captured the video streams using a smartphone, and imported and processed the virtual Reflection disturbance realization model through the game engine within the laptop. A We define two UV values for the distortion sampling: game engine usually works as an integrated platform one is the original UV value of the plane model, and that performs real-time rendering, data transmission, the other is a distorted UV value of the distortion map model processing, and so on. In the present case, the (the red and green channel value). By applying the game engine was able to help us eliminate the com- linear interpolation method to these two values, we plex underlying work of data processing and aggre- obtain the sampling UV for distortion. There is also a gation, which enabled us to focus on the functional- defined value called “degree of distortion” that deter- ity we intended to implement. mines whether the sampling UV is closer to the orig- inal UV or the distorted UV. Planar reflection implementation First, we determined a reflection plane. In the Unity EXPERIMENT AND EVALUATION engine, through scripting, we were able to generate Experiment an additional virtual symmetric camera view to the As described above, we developed an AR system that predefined plane. Every frame captured by the cam- realizes real-time reflection effects in landscape de- era was intended to be added to a material map for sign review scenes. To verify and evaluate the perfor- the reflection plane. The last step was rendering the mance of the system on actual sites, we conducted corresponding reflection material map on the reflec- an experiment. The devices used in the experiment tion plane. are shown in Fig. 4. The surrounding environment of It should be noted that the background of the the experiment site is shown in Fig. 5. The details of image captured from the second viewpoint was gen- the hardware devices and software used in the exper-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 551 iment are shown in Table 1. The experiment parame- camera was facing the design site containing the wa- ters are shown in Table2. ter surface. The smartphone and laptop were con- nected via a USB cable, which came with the sacrifice Figure 4 of some freedom, but this setup was more suitable for Hardware devices this outdoor condition without a wireless network. If component the outdoor network conditions are good, the use of a wireless network for connection is possible. The user rotated the tripod to change the viewpoint of the phone and then confirmed in the Unity engine in the laptop whether the phone’s motion trajectory could be synchronized to the virtual camera. Then, the user manually placed a predefined water surface model in the Unity engine and adjusted the AR view- Figure 5 point presented by the virtual camera so that its po- Experiment site, sition matched the location of the water surface in point of view, and the real environment. The user then moved the pre- the imported virtual prepared virtual design model to the designated lo- model cation in the design site, added the abovementioned mirror reflective material to the water surface model, and reviewed the virtual model in the actual envi- ronment through the screen of the smartphone or laptop. As the camera view was rotated, the virtual model and its reflection were always presented from the correct point of view.

Result Four screenshots from the AR video streams obtained We built the whole system in the Unity engine. in this experiment are shown in Fig. 6. The images In this study, we used the ARCore SDK for Unity above the screenshots show the direction and view from Google as a toolkit to build the fundamental range of each screenshot. AR scenes. ARCore SDK provides a series of gen- eral tools, which made the construction of AR fun- Evaluation damental scenes relatively easy. Most of the pro- The system proposed in this paper enables an AR rep- cesses in the experiment were controlled by a pro- resentation with a real-time reflection effect in land- gram through C sharp scripts. In the rendering mod- scape design review scenarios. When the positions of ule, we controlled the rendering procedures by us- the model and the water surface in the virtual space ing the Unity ShaderLab language, which is a pro- are adjusted in advance, the virtual model and its re- gramming language that integrates HLSL/Cg shader flection will be displayed as correctly overlaid on the language, which is used for programmatic control actual environment on the computer screen. How- of real-time rendering within Unity. We used Visual ever, there are several challenges with this system. Studio as the integrated development environment (IDE). 1. Water ripples in realistic environments are com- Before the start of the experiment, the user set plex and dynamic and cannot be perfectly sim- up the smartphone and tripod so that the phone ulated by simply using a static distort texture.

552 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 2. When moving the camera, sometimes there is phenomenon is related to the method of reflec- a dislocation between reflections and virtual ar- tion implementation, which remains to be re- chitecture. We speculate that the cause of this solved.

Table 1 System specification

Table 2 System parameters

Figure 6 AR result of the proposed system

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 553 3. The system cannot yet detect what area is wa- REFERENCES ter surface; to date, the area is predefined by the Akenine-Möller, T, Haines, E and Hoffman, N 2019, Real- user. Also, the user needs to manually adjust the time rendering, Crc Press position of the water surface model. Beug, AP 2020, Screen Space Reflection Techniques, Ph.D. Thesis, University of Regina Booth, NK 1989, Basic elements of landscape architectural design, Waveland press CONCLUSION AND FUTURE WORK Broschart, D and Zeile, P 2015 ’Augmented reality in ar- This paper presents an AR system that can be imple- chitecture and urban planning’, Peer reviewed pro- mented in landscape design review scenarios with ceedings of digital landscape architecture real-time reflection effects and references the simu- Gordon, E and Manosevitch, E 2010, ’Augmented delib- lation of water ripples in the field of real-time ren- eration: Merging physical and virtual interaction to dering to enhance the representation of water reflec- engage communities in urban planning’, New Media & Society, Vol 13, Issue 1, 2011, pp. 75-95 tions. This AR representation allows stakeholders to Hu, YH, Velho, L, Tong, X, Guo, BN and Shum, H 2006, understand the effect of waterfront landscapes and ’Realistic, real฀time rendering of ocean waves’, Com- will help designers better express their design ideas puter Animation and Virtual Worlds, Volume17, Is- by incorporating reflections into virtual landscapes. sue1, pp. 59-67 The proposed system requires the use of pre- Johanson, C 2004, Real-time water rendering—— defined water surface models and manual adjust- Introducing the projected grid concept, Master’s The- sis, Lund University ment to obtain correct location relationships by the Kido, D, Fukuda, T and Yabuki, N 2021, ’Assessing user, which increases the complexity of the opera- future landscapes using enhanced mixed reality tion. To simplify the operation process, future work with semantic segmentation by deep learning’, Ad- should explore the possibility of automatically iden- vanced Engineering Informatics, Volume 48, April tifying reflective surfaces in real environments and 2021(https://doi.org/10.1016/j.aei.2021.101281), p. generating corresponding surface models. At the 101281 Milovanovic, J, Moreau, G, Siret, D and Miguet, F 2017 same time, the real-time rendering method used in ’Virtual and augmented reality in architectural de- the present system is applicable to water reflective sign and education’, 17th International Conference, surfaces only, and so the possibility of its application CAAD Futures 2017 to non-horizontal reflective surfaces needs to be ex- Stintzing, M, Pietsch, S and Wardenga, U (eds) 2020, plored. The real-time ray-tracing technique may be How to Teach “Landscape” Through Games?, Springer a method to realize the non-horizontal reflection on Fachmedien [1] https://docs.unity3d.com/2018.1/Documentation/ multiple reflective planes, but the limitation is the Manual/PostProcessing-ScreenSpaceReflection.html high demand for hardware performance. This pa- [2] https://developers.google.com/ar/develop per also suggests such future tasks as implementing [3] https://docs.unrealengine.com/en-US/BuildingWorl other interactive optical effects for landscape design ds/LightingAndShadows/PlanarReflections/index.html visualization, including refraction simulation for un- derwater illumination design.

ACKNOWLEDGMENT This research was partly supported by JSPS KAKENHI Grant Number JP19K12681. All images were created by the authors from 2020 to 2021.

554 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Exploring the Possibilities of a Virtual Reality Aided Architectural Design System

Chaohe Lin1, Tiantian Lo2, Xinchuang Hu3 1,2,3Harbin Institute of Technology (SZ) [email protected] [email protected] [email protected]

Creating a visual 3D model is a vital part of the architectural design process. In architectural scenes, immersion is important to perceive the connection between various aspects. The general interaction cannot meet the needs of immersion. The immersion and interaction of virtual reality (VR) allows architects to feel and design spaces better. However, VR is currently mainly used only for visualization and walkthrough of the architectural space. The design process is still done using modeling software. This paper proposes a method of architectural design in virtual reality, allowing designers to experience the model created in real-time and improve the design. The use of visual mesh positioning and ray limits can help users create accurate architectural models. The paper also applied the innovative design method to the participatory design process and showed that architectural design in virtual reality can improve design quality and better meet the needs of users.

Keywords: Digital Architecture Design, Interaction, Virtual Reality, Design tool, Virtual modeling

INTRODUCTION emergence of virtual reality technology has made it Three-dimensional architectural model plays a pos- possible to expand the way of human-computer in- itive role in improving the efficiency of architectural teraction. Virtual reality is a simulation in which com- design. Architects are beginning to use more realistic puter graphics are used to create a realistic-looking visualization mechanisms in their design processes world (Burdea & Coiffet, 2003). At present, with (Shiratuddin & Thabet, 2002). At present, the exist- the progress and development of computer technol- ing design tools cannot completely solve the immer- ogy, virtual reality technology has been applied more sive problem of virtual models. The general human- and more in architecture. Virtual reality technology computer interaction interface is based on WIMP has three characteristics: immersion, interaction and (Windows, Icons, Menus, Pointers). This way has been imagination (Burdea & Coiffet, 2003). Different from relatively mature after years of development. How- the general interaction method, virtual reality can ever, this approach does not allow people to expe- enable users to better experience the design space, rience architectural space. In addition, this human- which is conducive to improving the quality of de- computer interaction is not natural enough. The sign. This paper proposes a method to create house

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 555 type in virtual reality. By using game engine Unity with the building is particularly important. Users’ par- and virtual reality device HTC Vive, a set of virtual ticipation and feedback on the design are valuable reality design platform can be built. The platform for designers. In addition, the human-computer in- allows designers to immerse themselves in the de- teraction of virtual reality technology is an innovation sign of their buildings and interact in a more natural compared with the general way. Using the gamepad way. This platform is conducive to non-professionals and helmet, people can immerse themselves in vir- to participate in the design process and reduces the tual space, which is conducive to the evaluation of cost of communication between non-professionals architectural design. and designers. Application of virtual reality technology in RESEARCH OBJECTIVES architectural design process The existing CAAD software is very unfriendly to non- When virtual reality technology first appeared, it was professionals. Virtual reality technology has broken mainly used by architects to introduce design con- the general form of architectural expression and low- cepts. The characteristics of immersion make virtual ered the threshold for non-professionals to partic- reality technology have great advantages in express- ipate in architectural design. In addition, the cur- ing spatial experience. Virtual reality technology can rent architectural design process mainly uses mouse break through the general form of expression, be- and keyboard as the interaction, and this human- yond reality. This is the main application mode of computer interaction is not natural enough. Vir- virtual reality in architectural design process. In the tual reality technology makes it possible to innovate early stages of design, the use of virtual reality helps human-computer interaction in the process of archi- to understand spatial concepts. VR can be used for tectural design. This study proposes a method to ac- three-dimensional creation and interaction, model- curately create architectural models in virtual reality, ing work (Schnabel, Wang & Seichter, 2008). In the which can expand human-computer interaction in mid-design phase, VR allows designers to experience the architectural design process. General virtual real- the design space and adjust the design according to ity modeling software cannot achieve accurate mod- the effect. At the later stage of design, the combi- eling. This paper proposed using a visual grid to help nation of VR and BIM can lower the user’s operat- users to locate. By limiting the collision point detec- ing threshold, allowing users to experience the de- tion of gamepad ray to the grid intersection point, sign space in real time. The system can also add the the virtual object can be created with accurate val- function of simulating physical dynamics (Yan, Culp ues. This will create a model that can be used in & Graf, 2011). subsequent phases. The user can communicate with the designer and make suggestions on the design Affordances of virtual reality and its impact through this method in the early stage of design. De- on architectural design signers can also use this method to experience the Affordances refer to the fact that in order to interact design space and improve their design. with virtual objects, the user needs to perceive how the object is likely to change (Gibson, 1977). Affor- dances are properties related to the observer’s body, THEORETICAL BASIS sensorimotor ability, and intention, and related to With the development of computer technology, the subject’s perception and activity in the environ- there are not only two-dimensional and three- ment. By providing the observer with the possibil- dimensional ways for the expression of architecture, ity of different perspectives and increasing the con- but also immersive ways. Considering that most tent of the interaction, you can increase affordances buildings are built for users, the users‘ experience (Grabarczyk & Pokropski, 2016). Virtual reality allows

556 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 users to view the environment and objects from vari- a user wearing a helmet mounted display, the user ous angles, increases the interaction content of users, will be completely surrounded by a virtual environ- and satisfies the corresponding relationship between ment, and then experience the immersive space (Suh users and virtual roles, so it has affordances. Based on et al., 2005). Virtual reality provides a high level of this, the use of virtual reality to carry out participatory immersion, and it provides an opportunity for archi- architectural design can meet the user’s experience tects and users to evaluate the space better (Lo & Gao, needs, so that users have a more sense of participa- 2020). In addition, VR provides the possibility for the tion in the design process. Users can map themselves visual interaction of intelligent data, and users can in- to virtual objects, observe the design results in all di- teract with data more intuitively (Al Bondakji et al., rections and modify the content in real time during 2019). In a word, architects can use virtual reality in the design process. the design process to experience the space well and get different feelings from multiple angles. Users are Advantages of using virtual reality in archi- also able to participate in the design process better tectural design process and interact with the data, and the final design of the The general human-computer interaction limits the space can meet the requirements. user’s experience of space, resulting in deviations be- tween results and imagination. Therefore, the appli- Characteristics of the application of VR to cation of virtual reality in the process of architectural participatory residential building design design can better solve the problem of insufficient The interaction of VR is more in line with people’s space experience. The introduction of virtual reality natural habits, so the application of virtual reality in technology into the process of architectural design the process of participatory architectural design is is helpful to improve the quality of design. The in- conducive to the communication between architects teraction of virtual reality shows that people can in- and users. With the development of urbanization, teract well with the virtual world. In order to be im- high-rise residential buildings are gradually increas- mersive, the technology should be inclusive, exten- ing. High-rise residential buildings in high-density sive, surrounding, vivid, compatible, and contain a cities have several characteristics. First, due to the ur- story. Inclusive means that virtual reality prevents ban population concentration, the scale of residen- external stimulation of users. The technology of ex- tial buildings is large. Second, users have a great tensive, surrounding and vividness has the charac- demand for personalization. Third, construction in- teristics of wide vision, multi-sense and high simu- dustrialization leads to the reduction of the produc- lation. Compatible and storyline refers to the fact tion cost of components. Based on the current prob- that the virtual world stays in sync with the user as lems, open building theory can meet the needs, and they move along, and that there is a certain plot it requires users’ participation. The design process theme (Slater & Wilbur, 1997). The virtual reality that of large-scale housing can be broken down into sev- meets these requirements is highly immersive, allow- eral simple tasks can achieve the goal of the user and ing designers and users to better experience archi- the architect to design together (Lo, Schnabel & Mo- tectural space. The immersion of virtual reality al- leta, 2016). Virtual reality is conducive to user par- lows designers and users to better experience archi- ticipation and can achieve personalized customiza- tectural spaces. Imagination allows the virtual scene tion of their own residential space. Architects can to get rid of limitations. Specifically, the immersive design residential buildings according to users’ opin- three-dimensional environment influences multiple ions and improve the quality. Users’ participation in senses, allowing the users to better focus and ab- the design will make the final product adapt to di- sorb more information (Donalek et al., 2014). When verse needs.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 557 Diversityofvirtualrealityhuman-computer support of virtual reality equipment. The Unity game interaction in the process of architectural engine has the ability of fast rendering and real-time design interaction, which can be used for VR system con- There is much room for innovation in the human- tent creation and interactive content programming. computer interaction of virtual reality. The existing Unity and HTC Vive can work well together to create human-computer interactions include virtual three- virtual reality applications. dimensional interaction, gesture and body interac- tion, mobile device interaction, voice interaction, tac- Accurate creation of VR-Aided Design tile interaction, multi-channel interaction and so on. The virtual design modeling platform proposed in Many interactions are being investigated, and using this paper explores the accuracy of virtual reality gestures and tactile feedback to interact with virtual modeling methods. Common virtual reality model- objects can improve the design experience (Cama- ing software realizes the creation and simple interac- cho et al., 2019). Body recognition and interaction tion of virtual objects. Users can directly build sim- can also be connected with virtual reality, making it ple objects in the virtual environment. However, cur- easier for users to interact with the virtual environ- rent modeling software is still unable to accurately ment (Zhang, 2012). The integration of these interac- create architectural models. While the free creation tions can promote the efficiency of using virtual real- approach allows users to be creative, a model that ity in architecture. The use of virtual reality technol- lacks the values cannot meet the needs of construc- ogy in architectural design will innovate the interac- tion. In this paper, the platform uses grid location to tions and become more humanized. solve this problem. We import a 0.5m grid picture into the virtual environment, and then match the size A DESIGN TOOL USING VR TO EXPAND of the grid in Unity (Figure 1). Users can see an ac- HUMAN-COMPUTER INTERACTION curate grid plane after entering the virtual environ- ment. We have optimized the gamepad’s ray loca- VR-Aided Design tion function. When the user’s gamepad points at the This paper introduces VR-Aided Design, which is a grid, the mark cube created is forced to be positioned digital tool for user participation based on virtual re- at the intersection of the grid. All of the user’s cre- ality. This digital tool uses a 3D display device and ation is controlled by the grid, so the exact value of interactive input methods. The tool can import tradi- the object can be obtained. For example, when the tional design data and make precise modifications to user is given a floor plan of the house type, the user the model. This program expands human-computer can position the walls according to the grid plan. In interaction in the process of architectural design, and addition, the grid can help the user determine the reduces the threshold for non-professionals to partic- spatial scale, move and modify the furniture. The ipate in the design. This program realizes accurate two-dimensional grid determines the size of the ob- modeling in the real sense by using the function of ject, while the height information can be adjusted to grid positioning. The development platform adopts the user’s desired value in the software. the game engine Unity, and HTC Vive as the hardware

Figure 1 Grid plane and ray positioning of VR-Aided Design

558 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 2 Creates walls with a specified value on the grid

Implementation Logic and interaction of sign, we preset furniture modules. Users can point VR-Aided Design a ray at the furniture they want. The ray carries out Our platform is developed using the Unity game en- collision detection with the furniture components to gine. It mainly solves several technical points. The obtain the coordinates and transfer them to the po- grid plan is designed by external software, then im- sition of the handle. The user then uses the ray to ports them into Unity and matches the units with position the grid, releasing the button at the speci- Unity. Since it is necessary to use rays to locate and fied location, and the coordinates of the furniture are create large-scale models in the future, we set the transferred to the intersection point of the grid. In mesh accuracy to 0.5m to help users to identify in this way, the layout of the interior furniture can be de- the virtual environment. Then we transferred the in- signed and adjusted. After the furniture and the wall terface of HTC Vive device into Unity and obtained are positioned successfully, we set the MENU button the functions of gamepad buttons and rays. After on the handle. The height of the wall can be changed importing the grid plane, we designed the gamepad by using this key. Users can set the height of the wall ray. The Handle ray design is a black straight line. by themselves. The default value is 3m. After the When the user enters the virtual environment, the walls are formed, the enclosure of the space is deter- ray can be directly used to point to the grid points mined. With the help of HTC Vive hardware devices, that need to be determined. This action allows the users can directly roam the space in the virtual scene. user to estimate the length of the object. We get the For spatial components that do not feel appropriate, point of collision between the ray and the grid plane. the user can use the handle ray to change the posi- When the user’s rays collide with the grid plane, the tion. The user can adjust the spatial layout in real time script reads the coordinates of the position points in while roaming. real time. When the user presses the trigger key, a cube with a width and height of 0.1m is created as Testing and analysis of VR-Aided Design a marker.(Figure 2) We have optimized the position- This paper uses a case to test VR-Aided Design. In ing accuracy to ensure that the marker points are cre- this paper, the design of an urban village youth apart- ated at the grid intersection. When the user points to ment for undergraduate architectural education is another grid intersection with a ray and presses the selected as an experimental project. When college trigger key, a second mark cube is generated based students just graduate, their career development is in on the coordinate of the collision point. The mid- the initial stage and their general income is low. Con- point of the line between the center points of the two sidering the economic problems, their living condi- marked cubes is the coordinate of the generated strip tions will be relatively compact. Therefore, taking col- wall. Because the high wall will block the user’s line of lege students as experimental subjects for research sight, we first use the rectangular cube to locate the is in line with the project user goal. This project site plane, and design the layout of the house on a two- is located in Guimiao New Village, Nanshan District, dimensional scale. When the whole wall position is Shenzhen City, Guangdong Province, China (Figure determined, the user can undertake the arrangement 3). The project is positioned as a youth residence of furniture. To make it easier for users to interior de- in an urban village. The main users are young peo-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 559 ple who have just entered the working stage. Low- filled in a questionnaire to evaluate how well the income users have higher requirements for space uti- house type design meets the needs. The user also lization. The project was designed to meet the indi- gave a subjective rating to the house type design. Us- vidual needs of the user and to achieve a complete ing these data, we can intuitively evaluate the over- residential function in a compact spatial layout. all design quality and analyze the promoting effect of VR-Aided Design on participatory design. Users’ participation in design as a part of the Figure 3 experiment can help architects understand users’ Project site needs. Some users wanted more ventilation, so a ventilation corridor was designed in the house model. Some users wanted to see more views, so more windows were added on the side facing the inner garden. To meet the personalized needs im- proved the users’ satisfaction. Users first used the platform to create a model and expressed their ideas This paper designs a participatory design experiment directly. Due to the lack of professional knowledge to test VR-Aided Design. The experimental process by non-professional users, architects were required is to first let the architect design the overall build- to adjust the design. The architect adjusts the layout ing shape according to the external conditions, and of the house according to the user’s design advice in then let the user participate in the design by using VR-Aided Design, so as to design the final house type the platform (Figure 4). The project is located at the (Figure 7). boundary of the village in the city, surrounded by At the end of the experiment, we asked users to roads, so the overall design adopts a semi-enclosed fill in a questionnaire to evaluate and score the de- way to create a quiet interior space. On this basis, sign. Our questionnaire is divided into three ques- the volume was reduced to reduce the influence of tions. The first question was to ask the user how sunlight in the west. After that, the public space and much VR-Aided Design helped them express their the terrace were added to improve the efficiency of ideas. The second question was to ask how well the the space (Figure 5). This time, 20 students were re- design results meet the users’ needs. The third ques- cruited to participate in the experiment. The stu- tion was to ask users to give subjective ratings to dents are mainly in automation, computer science, the design. The results showed that 85% of users electronic information, etc., and the experiment ar- thought the platform helped them express their per- rangement ensures a multi-disciplinary background sonalization ideas well or very well. By using the plat- for non-professional users. Students directly used the form, they can communicate more directly with the VR-Aided Design platform to enter the virtual reality architect. In addition, 80% of the users felt that the fi- and designed the house type content under the con- nal design was good or very good in meeting their ditions limited by the architect. The youth apartment needs (Figure 8). Through this experiment, it has project is mainly a single apartment of 48 square me- been proved that VR-Aided Design has a good role ters, and the modules are limited to the functional in promoting participatory design. Users had a high scale of 4m units. Students can choose three 4m*4m enthusiasm for this participatory design process and plane modules for combination and arrange the fur- were more willing to provide design advice to archi- niture (Figure 6). Then the architect adjusts and opti- tects. mizes the house type design according to the user’s data, and lets the students experience the design re- sults. After experiencing the scene in VR, students

560 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 4 Design of experimental flow

Figure 5 The architect determined the overall shape according to the constraints

Figure 6 Students participated in building house types and arranging furniture

Figure 7 The house types designed by the architect with the help of the users

APPLICATION sual grid. The user can measure and directly build the Accurate building modeling tool in virtual model that conforms to the real data. This precise reality modeling makes architectural design in virtual real- There have been many examples of virtual reality ity possible. The architects will be able to control the modeling, but a large amount of software has ig- spatial scale more accurately. Users can also directly nored the need for accurate modeling in architecture. use the tool to create and layout interior spaces that Models created by existing products are not available are consistent with the real world. Precise creation for subsequent use due to the lack of precise values. advantages help to truly integrate virtual reality into To solve this problem, this paper proposed using a vi- the architectural design process.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 561 Figure 8 Users’ evaluation analysis diagram

The tool for user participation in architec- ADVANTAGES OF VR-AIDED DESIGN tural design Make it easier for users to participate in de- As a virtual reality modeling platform, VR-Aided De- sign sign can help users to participate in the architec- Based on the features of virtual reality, the platform tural design process, so that the final design can bet- can reduce the user’s operation difficulty. After a long ter meet the needs. The existing 3D modeling soft- time of development, three - dimensional software ware can meet the needs of architects to create mod- has been equipped with powerful modeling func- els. However, these software operations are relatively tions. Existing 3D modeling software, such as Rhino, complex, and can not be well involved in the design SketchUp, Maya, etc., can fulfill the requirements of of non-professionals. VR-Aided Design can be used architectural design very well. However, with the di- as an auxiliary tool in the process of adjusting the de- versification of functions, the complexity of opera- sign scheme. This tool can promote the public par- tion has also increased, and non-professionals need ticipating in the design of personalized high-rise res- a higher learning cost to operate. VR-Aided Design is idential. At present, China’s urbanization process is based on the interactive characteristics of virtual real- accelerating, and the mass construction of high-rise ity, which reduces the difficulty for non-professional housing in big cities leads to the boredom of apart- users to participate in the design. Users can use a few ment types. VR-Aided Design can be used to solve simple keys to complete the design content, and ex- these problems. press their needs directly. Compared with the com- mon 3D modeling method, this kind of participatory The tool for designers to experience space design experience is more interesting, and users are General architectural design tools cannot help archi- more willing to try it. tects to experience space. Architectural design assis- tance method based on VR can help architects to ex- Real-time virtual roaming perience the space while modifying the design. This The virtual reality platform allows real-time viewing method is conducive for architects to adjust the over- of the design results. The operation of 3D software is all layout according to the actual space effect. Virtual mainly presented in the form of perspective, which reality breaks through the general form of architec- can clearly express the spatial relationship. How- tural expression and helps to better reflect the spa- ever, users can only observe the model with a two- tial effect. As a tool for design experience and modi- dimensional screen, and cannot perceive the space. fication, VR-Aided Design can be introduced into the Virtual reality allows users to experience the space architectural design process. more directly and see the results of the design in real time. The users will feel more present.

562 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 The design results are more consistent with create. Since the design required an overall con- reality trol over the building, the platform required a The model created on the platform has a scale closer variety of scale view transitions. The user should to reality, and the results match the reality more be able to create the model on multiple scales closely. The existing 3D modeling software restricts and translate it into real scales in real time. Sub- the user’s perception of the real scale, and the user sequent platforms will add tools for scaling to can only judge whether the scale is appropriate by design the buildings at different scales. experience. The model created in virtual reality lets • The platform can add the function of parametric the user feel the real scale directly, so the design re- modeling. In the future, the current parameter- sults will be more in line with the reality. ized modeling platform can be connected to VR- Aided Design, so that people can interact with FUTURE IMPROVEMENT DIRECTION OF the parameterized virtual model more directly. VR-AIDED DESIGN Designers can create the desired objects in the virtual environment at will. The design process VR-Aided Design uses virtual reality technology to in- will become more efficient. novate the interaction in the process of architectural • The platform can add the process of digital con- design, but there is still much room for improvement struction. The precise design of the model in human-computer interaction. This paper proposes can then be connected to the digital construc- the following directions. tion software in the subsequent process. The • The interaction should be more varied. At solid model can be obtained directly through 3D present, the interaction of VR-Aided Design printing. Due to numerical and scale control, the mainly relies on the handle and helmet display usability of the model is guaranteed, and subse- of HTC Vive. While the immersive feel of virtual quent digital construction is also possible. VR- reality allows users to experience the designed Aided Design has a certain innovation for the de- space, the way controllers interact doesn’t come sign process. The design can be automatically naturally to people. The operation of the con- generated using parameters at the beginning, troller is not friendly to the first user. In addition, then the model can be evaluated and modified most controllers provide fewer interfaces, reduc- by architects and users in virtual reality, and fi- ing the diversity of interactive feedback. There- nally built digitally. This process improves the fore, the addition of new interaction plays a pos- efficiency of digital design. The finished product itive role in improving the design efficiency of of the design can also meet the needs of users the platform and reducing the difficulty of op- better. eration. Existing human-computer interaction methods, including gesture and body recogni- tion, can be connected to improve the usability CONCLUSION of the platform and allow users to design more This paper summarizes the characteristics and advan- freely. In the future, when the platform is inte- tages of virtual reality in architectural design process, grated with other interactions, the architectural and points out the potential of virtual reality in partic- design process based on this platform will be- ipatory residential design. The application of virtual come much easier. reality technology not only helps users and architects • The accuracy of human-computer interaction to better understand the space, but also provides op- needs to be further controlled. Currently, VR- portunities for non-professionals to participate in the Aided Design relies on the grid to position and architectural design process. This paper proposes a virtual reality modeling platform VR-Aided Design.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 563 In view of the problem that the general virtual real- Lo, T T and Gao, X 2020 ’From HCI to HVRI-How differ- ity modeling platform can not achieve accurate cre- ent will the interaction of digital architecture de- ation, a method of using visual grid plane to help sign be?’, Proceedings of the 25th CAADRIA Confer- ence, Chulalongkorn University, Bangkok, Thailand, users determine the value is proposed. In this paper, pp. 253-262 a project is used as a case to test the user participa- Lo, T T, Schnabel, M A and Moleta, T 2016 ’A Sim- tion design of VR-Aided Design. Through question- ple System for Complex Mass Housing Design naire analysis, VR-Aided Design can help to express Collaborations-A system development framework’, ideas. Users can use VR-Aided Design to participate Proceedings of the 34th eCAADe Conference, Univer- in the design process to better meet the needs. VR- sity of Oulu, Oulu, Finland, pp. 137-146 Schnabel, M, Wang, X and Seichter, H 2008 ’Touch- Aided Design can be used as a tool to participate in ing The Untouchables: Virtual-, Augmented-, and architectural design. The platform can also be used Reality’, International Conference on the Association as an auxiliary design tool for architects to experience for Computer-Aided Architectural Design Research in space. In the future, VR-Aided Design can not only Asia, pp. 293-299 use more interactions, but also improve the accuracy Shiratuddin, M F and Thabet, W 2002, ’Virtual office walk- and increase the function of parametric design. through using a 3D game engine’, International Jour- nal of Design Computing, 4, pp. 1-25 Slater, M and Wilbur, S 1997, ’A framework for immersive ACKNOWLEDGEMENTS virtual environments (FIVE): Speculations on the role The authors would like to acknowledge that this pa- of presence in virtual environments’, Presence: Tele- per was financially supported by the National Natural operators & Virtual Environments, 6(6), pp. 603-616 Suh, K S and Lee, Y E 2005, ’The effects of virtual reality on Science Foundation of China (Grant No.51908158). consumer learning: An empirical investigation’, Mis Quarterly, 2005, pp. 673-697 REFERENCES Yan, W, Culp, C and Graf, R 2011, ’Integrating BIM and Al Bondakji, L, Lammich, A L and Werner, L C 2019 ’ViBe gaming for real-time interactive architectural visual- (Virtual Berlin)-Immersive Interactive 3D Urban Data ization’, Automation in Construction, 20(4), pp. 446- Visualization-Immersive interactive 3D urban data 458 visualization’, Proceedings of eCAADe 2019, Portugal, Zhang, Z 2012, ’Microsoft kinect sensor and its effect’, pp. 83-90 IEEE multimedia, 19(2), pp. 4-10 Burdea, G C and Coiffet, P 2003, Virtual Reality Technol- ogy, John Wiley & Sons Camacho, D, Dobbs, T and Fabbri, A 2019 ’Hands On Design-Integrating haptic interaction and feedback in virtual environments for enhanced immersive ex- periences in design practice’, Proceedings of the 24th CAADRIA Conference, Victoria University of Welling- ton, Wellington, New Zealand, pp. 563-572 Donalek, C, Djorgovski, S G and Cioc, A 2014 ’Immersive and collaborative data visualization using virtual re- ality platforms’, 2014 IEEE International Conference on Big Data (Big Data), pp. 609-614 Gibson, JJ 1977, ’The theory of affordances’, Hilldale, 1(2), pp. 67-82 Grabarczyk, P and Pokropski, M 2016, ’Perception of affordances and experience of presence in virtual reality’, Avant. The Journal of the Philosophical- Interdisciplinary Vanguard, 7(2), pp. 25-44

564 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Modeling Inter-dimensional Narratives

Olga Mesa1, John O'Keefe2 1Roger Williams University, Harvard Graduate School of Design 2Roger Williams University [email protected] 2o'[email protected]

The integration of VR in the creative process has caused a profound shift in the use of modeling tools and abstraction. How do instantaneous experiential feedback, body awareness, the triggering of spatial sensations, and traveling in real-time from an object-scale to a habitable-scale affect modeling in VR? This research explores the tensions and exchanges between the physical and the digital relative to spatial perception when designing in VR. The work produced by participants involved in a digital design workshop developed around these topics will be presented. In response to a written provocation, participants modeled three-dimensional dreamscapes in VR using Oculus Medium. Participants explored the connection between the body and its movements to measure, model, and control phenomena when animating virtual scenes. This research contributes to the teaching and implementation of modeling in a virtual environment by exploring the inherent possibilities of VR in relation to the conceptualization of spaces.

Keywords: Virtual Reality, Spatial Perception, Virtual Reality Modeling, Virtual Reality in Architecture

INTRODUCTION AND BACKGROUND cessing units (GPU’s) and related accessibility of con- Immersing a subject in a virtual environment has sumer hardware and software for VR video games been a fascination for decades. From the first concept over the past five years, the inclusion of VR as a developed at MIT in the 1960s, where researchers as- visualization tool has become more pervasive in pired to design a “window” to access another world many disciplines including Architecture and Archi- where people could behave realistically (Sutherland, tecture, Engineering and Construction (AEC) educa- 1965), to pivotal moments in the 1980s where mili- tion (Horne and Hamza, 2006). This has highlighted tary and commercial flight simulations were accom- concerns over its conscientious implementation in plished, to more recent developments of fully immer- design teaching (Horne and Thomson, 2008) and sive video games exemplified by the consumer-ready practice. Oculus Rift and HTC Vive (Cipresso et al., 2018), ad- Within this context, it is worth acknowledging vances in the field of virtual reality have moved into the radical shift from manual representation tech- the mainstream. niques, such as drawing and drafting as primary With the production of affordable graphics pro- modes of investigation of architectural concepts, to-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 565 wards the use of digital modeling, visualization, and als is a primary opportunity in VR because unlike simulation. Orthographic drawings (plans and sec- other traditional modes of architectural representa- tions) offer abstract objective constructs of space, tion that work more asynchronously, the designer re- and while these tools do not offer how a design ceives experiential feedback in real-time; in VR, the would react in relation to gravity, this is comple- fourth dimension is integrated. It is not surprising mented by the use of other tools such as physical then, that much of the work that utilizes VR is aimed models. Similarly, ideas related to habitation and at the visualization of architectural spaces. However, phenomena are explored through the use of sub- there is a latent potential to integrate this capability jective experiential perspective drawings. Although in the early stages of design production via modeling. each of these representation tools stands on its own One of the main goals of this research is to investigate and each is used more appropriately depending on the possibilities of modeling when spatial perception the particular aspect to be explored (objective or sub- is the main entry point in generating space. jective), they feed off of each other. In contrast, in three-dimensional digital models, Figure 1 there is a synchronous confluence of modes of repre- Sketches and initial sentation and points of view when exploring spatial text in response to concepts, where it is possible to switch easily from a the prompt. planimetric to sectional views or perspectives with- out generating each of them individually. In other words, one of the main differences between architec- tural tools used in the representation and conceptu- alization of spaces is that established “paper-based” modes do not offer the instantaneous workflows en- Dorta argues that the “mental workload” required to abled by designing digitally (Oxman, 2008). combine traditional architectural drawings (plans, el- By adding immersivity, virtual reality technolo- evations, sections, even perspectives) into a cohe- gies disrupt the architectural discipline insofar as sive design, is potentially eased when using virtual they redefine the way creative processes and de- reality, allowing designers to participate in a more sign strategies (gestural, abstract, constructive) can fluid creative process (Dorta et al., 1998). However, be taught and learned, but also how the design work there are difficulties associated with this fluidity, such potentially becomes more efficient, how it is devel- as documenting the generative steps within the cre- oped, and ultimately shared (Oxman, 2008). There- ative process and extracting abstract representations fore, it is imperative to critically question the oppor- that facilitate measuring and conversations revolving tunities and shortcomings inherent in this revolu- around scale. tionary technology for future teaching and practice. As designers, we are very familiar with the notion Virtual reality presents significant challenges, as that different scales allow for different kinds of explo- well as opportunities because up to this point, it ap- rations related to the creative process. Some scales proximates but does not fully match reality. The main and viewpoints lend themselves to studying spatial challenge is that within the virtual environment the organizations, while others to examining tectonic re- understanding of gravity and the physics of materi- lationships. In VR, it is possible to make use of sev- als is lacking. The kind of structural and tactile feed- eral scales by expanding or reducing models in rela- back typically gained from a traditional “chipboard tion to the human scale. A VR model can be inhabited and wood” physical model does not occur. and instantly reduced to the scale of a handheld ob- Inhabiting, modeling, and visualizing propos- ject. Thus, another aspect explored by this research is

566 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 how the ability to go back and forth between scales of Washington HITLab in association with the Har- changes the way designers think about spaces and borview Burn Center presents that pain could be al- their modeling, and how this workflow can be sup- leviated in patients who have suffered burns by us- ported seamlessly. ing VR simulation therapies that evoke icy environ- ments, thus reducing the use of opioids (Hoffman et Figure 2 al. al., 2019). In the same way, through VR, one can Dreamscapes being trigger or confront eerie spatial sensations such as developed. Work agoraphobia, claustrophobia, and vertigo. Not sur- by: Orantes & prisingly, the medical industry is applying psycho- Siemssen, Sevinc & logical therapies for people who experience phobias Conner, Lalwani, such as flying phobia, agoraphobia, etc. (Botella et Simpson, & al., 2017). Pleasant sensations associated with the ex- Cinquigranno. perience of space are also possible through VR. For example, in recent research, omnidirectional images of environments such as forests are offered in immer- sive experiences to reduce anxiety, promote mindful- ness and relaxation (Seabrook et al., 2020), and the use of colors and light to create the atmosphere for a space might influence gestures and emotional re- Another aspect to consider is that when using a head- sponses of VR users (Stout, 2002). Additional research mounted display (HMD), stimuli from physical space goals were to investigate how the physicality of body are muted, helping the user to be transported to an awareness and the triggering of spatial sensations in alternate reality. However, even when the body is not real-time informs the way designers model through seen in this environment and its connection to reality VR, and how crafting the ambient color palette of a is only through a rudimentary depiction of hands, the VR scene affects the gestures and thus modeling out- awareness of the body continues. Therefore, in vir- comes. tual reality, the user’s head and arm movements be- This research presents the frictions and reciproc- come the main mode of interaction within an immer- ities between the physical and the digital relative sive scene, instead of being mediated by joysticks or to spatial perception when designing in VR. The re- keyboard and mouse. The eye and head trackers in search questions informed the design of a workshop the HMD, as well as the trackers in the manual con- where participants were asked to develop dream- trols, locate the user within a set space and record the scapes within a virtual environment using the aware- user’s movements in real-time. This provides users ness of their physical bodies and their spatial sensa- with instant environmental feedback that dramati- tions as tools. The objective of the workshop was cally raises the level of emotional investment and dis- to study how the research questions inform teaching connects disturbances from the periphery of the out- workflows related to creative processes. The results side world (Parsons & Rizzo, 2008; Price, Mehta, Tone of the workshop will be presented in this paper. & Anderson, 2011). Although virtual reality is a visually skewed METHODS AND RESULTS framework, there is the potential to evoke other In response to a written provocation, participants senses and explore the connection to bodily ex- modeled three-dimensional dreamscapes using perience. The extent to which the senses can be Oculus Medium with Oculus Rift and Quest head- stimulated through VR is an ongoing research topic. mounted displays (HMD). Recognizing the oppor- For example, the work developed by the University

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 567 Figure 3 Default Oculus Medium interface, 3b-3e. Modified atmospheric scenes for dreamscapes by Mesa & O’Keefe.

tunities that modeling in virtual reality offers, the Oculus Medium and challenging the default interface following sentences attempted to capture a hybrid (Figure 3a). For example, the feeling of awe and tran- between words that evoke physical, haptic, and ethe- quility were sought through the combination of cer- real concepts. Based on phrases like Cyber Spell, tain colors (Figure 3c,3d,3e) or on the contrary, caus- Buoyant Time, Infinite Circus, Entangled Memory, ing a disturbing and groundless feeling was achieved Opaque Space, to name a few, participants wrote within a space where glowing colors were used to about them and sketched, thus building a narrative describe a three-dimensional coordinate system as if (Figure 1). This narrative, although not completely inhabiting the planar ‘grid’ depicted in 1982’s TRON formed, guided the modeling of the dreamscapes as (Figure 3b). participants perceived the space that they created in As mentioned before, when modeling virtually, real time (Figure 2). the HMDs help participants to access a mental space, where stimuli from the physical space are mitigated. For our workshop, having a common reference be- Figure 4 tween these spaces was key to establishing paral- Physical modeling lels between the physical and the virtual. At the boundary and same time, it was important that the virtual space Virtual modeling could be expanded along the z-axis to investigate boundary spatial sensations, such as vertigo or claustrophobia, comparison. which were triggered in the design process. Partici- pants modeled in an 8’wx10’lx10’h physical space but were able to import spatial frameworks measuring Achieving an evocative atmosphere for the vir- 8’wx10l’x30’h (Figure 4). That is, the virtual and phys- tual scene was important in creating the stage for the ical spaces coincided in plan but not in height. The dreamscape. Therefore, the participants were taught three-dimensional frameworks were predetermined to adjust settings to control the light, color, and “Fog” using Rhino and Grasshopper. These featured fields in the space, to craft the ambient qualities of the of varying densities to offer a sense of spatial depth scene and model based on its perception. These (Figure 5a). These frameworks also fostered an un- were obtained by exploring the “World” settings in derstanding of the scale of their dreamscapes in re- lation to their bodies and provided a reference for

568 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 5 Three-dimensional frameworks by Mesa and Norcross.

measuring what was being modeled. Participants to the original habitable size in order to experience were able to turn them off, move them up or down the modifications they had made at the various scales and lock them in place to experience their sensations (Figure 6). from various heights and feel spatial limitations when As participants traveled back and forth between modeling (Figure 5b). these scales, they observed the differences in mod- At a certain point, participants were asked to turn eling perspectives inherent to these scales. In our off the imported 3D frameworks and scale down their conversations, they reflected on the commonalities three-dimensional dreamscape model, such that the and distinctions between small-scale modeling af- scene that they had inhabited a few moments before ter experiencing the scenes at a one-to-one scale. It could now fit in the palm of their virtual hands. They was also recognized how the crux of their projects were asked to continue modeling at this scale and could still be identified even at various scales, akin to after a while, to enlarge the model to fit roughly in observing a Zen garden and being transported to a their arms, to continue modeling, and then to return mountainous landscape.

Figure 6 Various size 3d frameworks (a). Traveling between inhabitable (b) and a hand-held object scales (c). Work by Taylor-Burto, Henderson & Loudon.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 569 Figure 7 VR models based on body movements and ranges. Work by Mendez, Varas & Valenti, Allen & Moran, Holmes & King.

The workshop exercises also aimed to encourage a tactile sensation while visualizing the spatial effect participants to investigate the link between the body that this activity generated even if not physically see- and its movements by modeling and controlling phe- ing the other participant. nomena, when animating the virtual scenes. In terms The awareness of their bodies in motion became of modeling, this connection was explored by using more apparent when they were asked to produce their bodies, arms, and legs to measure space in rela- an experiential video as if they were choreograph- tion to the 3d framework or by using their own body ing a dream. By recording their view from inside the center as an anchoring point. For example, while HMD, they were able to map the experience of walk- stationary at a point relative to the physical model- ing through their dreamscapes, but they were also ing boundary, participants model in VR spaces that able to explore recording with a virtual camera held reflected their own ranges of reach, thus experienc- in their virtual hands. These two ways of recording al- ing the types of intimate enclosures that these move- lowed different movements and as such experiences ments defined and, in doing so, became more aware of the dreamscapes (Figure 8). For instance, some of their own physical bodies (Figure 7). In another developed a “shaky-cam” style, while others quickly example, they were asked to trace with their con- rolled their cameras through space like a camera on trollers the silhouette of another participant’s body a dolly and some used their hands to define sweep- while modeling with virtual clay, thus experiencing ing fly-thrus (Figure 9 & 10). Their videos and percep-

Figure 8 Image from video of Buoyant Time dreamscape. Work by Mendez, Varas & Valenti.

570 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Figure 9 Image from video of dreamscape. Work by Conner & Sevinc and by Howard, Guidry & Crane.

Figure 10 Image from video of dreamscape. Work by Mendez, Varas & Valenti and by Taylor-Burto, Henderson & Loudon.

Figure 11 Mementos of Opaque Space (L) and Entangled Memories (R). Work by: Conner & Sevinc and by Lalwani, Simpson & Cinquigranno.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 571 Figure 12 Exhibition armature.

tion while modeling their scenes were used to finalize virtual space in real-time. The research studied the their written narratives inspired by their initial hybrid tensions and exchanges between the physical and word pairings (Entangled Memory, Buoyant Time, Lu- the digital when using VR in the design of spatial con- minous Romance, etc.). cepts. Through the teaching of a workshop where Participants chose a portion of their virtual participants were asked to design a dreamscape us- dreamscapes to be fabricated using Formlabs 3D ing Oculus Medium, the research questions were in- printers. This artifact was meant to stand as a me- vestigated and a workflow for designers was devel- mento to enable them to access their dreamscapes oped for modeling and designing in VR. Recogniz- through memory, thus connecting a physical object ing the instantaneous experiential feedback granted with the experience of the virtual environment (Fig- by immersing a subject in VR and how this feed- ure 11). back informs the modeling and design of spaces, At the end of the workshop, an exhibition was was central to the research. To that end, the cu- curated where an armature, a sort of cabinet of cu- ration and sequence of the workshop exercises in- riosities, displayed the various artifacts that encapsu- spired participants to explore the physical and spa- lated the dreamscapes (Figure 12). The cabinet held tial sensations experienced in VR and to gradually the 3d printed mementos in front of an image of the model their dreamscapes guided by their perception dreamscape accompanied by the written piece (Fig- of their virtual scenes as they unfolded. The work- ure 13). Visitors had access to the experiential video flow presented participants with the ability to tune and the VR model, giving them a glimpse of the par- atmospheric qualities of space by adjusting the light, ticipants’ creative process and journey back and forth color, and fog to construct the ambient stage for between scales, modes of representation, and inter- their dreamscapes. In the panel discussion, partic- dimensional narratives. ipants reported that inhabiting these atmospheres elicited spatial sensations, emotional reactions, and CONCLUSIONS body movements, which in turn influenced their ges- With the incorporation of VR into the creative pro- tures and the sculptural qualities of Medium’s vir- cess, designers are experiencing a profound change tual clay. In relation to the exercises that were in the use of digital modeling tools and spatial ab- designed to engage body awareness and explored straction. This reconfiguration in creative thinking is the link between the participants’ physical bodies due in part to the ability to inhabit and manipulate and modeling techniques, participants tended to

572 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 construct intimate, personal cocoon-like enclosures, tended to move more physically - walking to the ex- since they mostly explored the movement afforded tents of their real-world boundary and using the ex- by their dimensional ranges relative to a fixed ref- tents of their bodies to model. Materiality was typi- erence. Some participants voiced that experiencing cally added and phenomena adjusted when experi- this space was comforting, while for others it trig- encing the space at one-to-one scale. This became gered claustrophobic sensations, which they miti- especially apparent when producing ”in-model” ex- gated by eroding the enclosures to make them more periential videos of their dreamscapes. In contrast, spacious or invite light in. Participants reported that at the handheld scale, participants were more aware the three-dimensional frameworks preloaded into of the relationship between size of spaces relative to their VR scenes allowed them to better gauge spatial one another, sectional levels, repetition of elements, depth, and gave them a scale reference to their bod- and the geometric order of a particular design. It ies. Some participants observed that these frame- was also observed that even at different scales the works proved helpful when measuring the models essence of a space remained discernable, reinforcing and when they needed to reorient in the expansive the dialogue between scale, perception of space and virtual space. Some voiced that the option to turn the phenomena, and design decisions. Participants ap- preloaded frameworks on and off was useful because peared to choose their tools depending on the na- at times they became either too grounding or visu- ture of their assigned prompt. For instance, in mod- ally distracting. The frameworks also allowed partic- eling ”Entangled Memory” Medium’s ready-made fig- ipants to explore their models at various scales, go- urative ”stamps” (arms, hands, skulls, antlers, etc.) ing back and forth in real-time from the habitable were chosen to produce a three-dimensional ”kit- to the hand-held. At habitable scales, participants bashed” collage. Alternatively, in modeling ”Opaque

Figure 13 Images, narrative, and memento from the students’ dreamscapes.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 573 Space”, participants chose generic brushes and pla- featured in this research. tonic solids (cubes, spheres, cones) and used addi- tive and subtractive processes to build geometric ab- REFERENCES stractions to define a dense space. It is worth high- Botella, C, Gracía-Palacios, A, Villa, H, Baños, R, Quero, lighting that when using figurative stamps, an inher- S, Alcañiz, M and Riva, G 2007, ’Virtual reality expo- ent scale was associated with the recognizable object sure in the treatment of panic disorder and agora- helping participants orient spatially and providing an phobia: A controlled study’, Clinical Psychology and initial scale (a sort of datum) which was not as easy Psychotherapy, 14(3), pp. 164-175 Cipresso, P,Giglioli, I, Raya, M and Riva, G 2018, ’The past, to discern when using abstract solids, even if in both present, and future of virtual and augmented reality cases license was given to change the object sizes. research: A network and cluster analysis of the liter- The synchronicity between modes of modeling and ature’, Frontiers in Psychology, 9(NOV), p. 2086 experience was challenged when participants were Dorta, T and Lalande, P 1998 ’The Impact of Virtual Real- asked to produce a physical artifact from their dream- ity on the Design Process. Digital Design Studios: Do scapes. When preparing the 3d models for 3d print- Computers Make a Difference?’, 1998 ACADIA Confer- ence Proceedings, Quebec City (Canada) , pp. 138- ing, participants addressed physical aspects of their 163 designs unaccounted for in VR. Participants had to Horne, M and Hamza, N 2006, ’Integration of virtual re- choose a portion and determine the scale and level of ality within the built environment curriculum’, Elec- detail of their dreamscapes that embodied the essen- tronic Journal of Information Technology in Construc- tial qualities of their virtual experience once outside tion, 11, pp. 311-324 of it, and adjust modeling if necessary for 3d print- Horne, M and Thompson, E 2008, ’The Role of Virtual Re- ality in Built Environment Education’, Journal for Ed- ability as some designs depended on zero gravity. ucation in the Built Environment, 3(1), pp. 5-24 The spatial narrative was enriched at every step of the Oxman, R 2008, ’Digital architecture as a challenge for offered workflow. Transporting mentally between design pedagogy: theory, knowledge, models and various scales while engaging in the physical and medium’, Design Studies, 29(2), pp. 99-120 spatial sensations inherent in VR. Through the visu- Parsons, T and Rizzo, A 2008, ’Affective outcomes of vir- ally complex, fourth-dimensional dynamic environ- tual reality exposure therapy for anxiety and specific phobias: A meta-analysis’, Journal of Behavior Ther- ment, participants generated spatial concepts that apy and Experimental Psychiatry, 39(3), pp. 250-261 emerged from the journey between dimensions. Price, M, Mehta, N, Tone, E. B and Anderson, P. L. 2011, ’Does engagement with exposure yield better out- ACKNOWLEDGEMENTS comes? Components of presence as a predictor of treatment response for virtual reality exposure ther- The authors wish to thank Steve White, Dean of apy for social phobia.’, Journal of Anxiety Disorders, SAAHP at Roger Williams University because his sup- 25(6), p. 763–770 port made possible this research, to Niloufar Emami Seabrook, E, Kelly, R, Foley, F, Theiler, S, Thomas, N, and Zachary Angles for organizing and inviting Olga Wadley, G and Nedeljkovic, M 2020, ’Understanding Mesa to participate in the 2020 Virtual Frictions Work- how virtual reality can support mindfulness prac- shop at the School of Architecture at the Louisiana tice: Mixed methods study’, Journal of Medical Inter- net Research, 22(3), p. e16106 State University. Special thanks to graduate assistant, Stout, K 2011 ’Emotional responses to color and nonver- Chris Norcross, for his valuable research and for be- bal language: A survey of emotional responses to ing Mesa’s teaching assistant at the workshops, to color swatches and human poses’, Master’s Thesis in Aaron Brode for fabricating the cabinet displayed at Art Education, Texas Tech University the workshop exhibit, and to Josef Horacek for his in- Sutherland, I 1965 ’The ultimate display. Multimedia: sights on written narrative. Thanks to the students From Wagner to virtual reality’, Proceedingsof theIFIP Congress, pp. 506-508 and participants of the workshop and whose work is

574 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Augmented Spaces

If walls could talk

Holly Chan1, Andre Brown2, Tane Moleta3, Marc Aurel Schnabel4 1,2,3,4Victoria University of Wellington, NZ [email protected] 2,3,4{andre.brown|tane.moleta|marcaurel.schnabel}@vuw.ac.nz

This paper explores the development of Augmented Spaces that involve embedding within the built environment, digitally responsive recognition of human presence. Contemporary digital media provides the opportunity to enhance physical space with the property of immediate interaction, which results in a high level of user engagement and responsivenenss. Through the addition of digital media, emotional and reflective value can be added to the built form. If space is designed to be reactive, rather than passive, a dialogue can be established between the user/inhabitant and the environment. We report on the establishment and analysis of a set of prototype digital interventions in urban space that react to human presence. One is in a building threshold space; one an urban street. We describe the development of a digital particle system with two inputs; the first being the geometry that generates the particles and the second being the geometry that displaces the particles. The research goals that we report on are driven by three over-riding response criteria, Visceral, Behavioural and Reflective.

Keywords: augmented space, reactive, synesthetic

PROBLEM STATEMENT opening a window, or rearranging the furniture, we Humans form meaningful connections with the built are affecting and altering our surroundings. These environment through the physical and social interac- daily affordances give us control to create a preferred tions within the spaces they inhabit. The built form level of subjective comfort; creating physical, sensory has been “created for as long as at least three hun- or emotional pleasure in the spaces we occupy. dred thousand years, and strikingly even in the ear- Over the last three decades, our world has be- liest and simplest forms they were interactive and come increasingly augmented by digital technology multi-functional” (Alavi, et al., 2016). On a daily basis, and information. In the past, technology was fixed society is in constant dialogue with the built environ- to a particular location, and its effects were limited ment. This dialogue allows us to alter the parameters to specific spaces: the office, the television, the cin- of the space we occupy; by turning on a light switch, ema. With the advancements of mobile phones, dig-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 575 ital media is accessible throughout the day and in responds to criteria established in research investi- most inhabited locations. Pervceived through the gations in recent years. The term Synesthetic Archi- device of a screen, the virtual world is separate from tecture was coined by the digital media artist Re- the physical environment. fik Anadol (Anadol, 2020). Anadol works with col- This paper explores how virtual and physical in- lecting site-specific data in order to generate digi- teraction can be combined to develop Augmented tal data sculptures. The proposition takes inspiration Space and how it can be used to enhance public en- from ideas on augmented space; combining physi- gagement. An interesting and relevant project is the cal space, and digital information with the enabling Infinity Wall (Nexen Univercity) [1].The research ex- qualities of machine learning. His most well-known amines the implications, benefits and outcomes of work, a projection onto the façade of the Walt Dis- augmented space and aims to develop a system to ney Concert Hall in Los Angeles, uses a collection create a meaningful connection between person and of the LA philharmonics data to create digital media place in the form of a prototype system developed in that presents the building’s past.“Synesthetic archi- response to defined performance targets. tecture, therefore, suggests that architects may en- fold machine learning into building forms, not just AUGMENTED SPACE creating but also redefining space through mixed Augmented space defines the relationship between media and interactive robotics. The age of machine physical environments and virtual information. As intelligence will make our computers, our build- Lev Manovich writes in ‘The Poetics of Augmented ings and our cities more responsive and brain-like” Space’, augmented space is “the physical space (Anadol, 2020). overlaid with dynamically changing information” Equally, Sylvia Lavin uses the term ‘Super Archi- (Manovich, 2016). The concept explores how per- tecture’ to describe the result of adding digital me- sonal experience of a space can be altered and aug- dia to architectural form. Resulting in works that mented through the addition of multimedia informa- “not only superimpose themselves onto architecture tion. but that intensify architectural effect... mingling one Our research consequently investigates how the medium with another so that neither loses its speci- integration of responsive digital media into built ficity.” (Lavin, 2011). form can enhance the level of user immersion and The addition of interaction to augmented space interaction. Biggio notes the key aim which is to de- adds another layer of user engagement that will en- velop “an interface that is able to erase itself, making able the user to become an active participant in the the user experience more immediate” (Biggio, 2020). experience. The participant will become the actor When describing her multimedia video installation at rather than just the spectator, or as Augusto Boal the Museum of Modern Art, Pipilotti Rist stated: “The terms them “Spect-actors” (Boal, 1985). Boal, an ex- basic concept is to try not to destroy or be provoca- pert in performance and theatre, suggests that when tive to the architecture, but to melt in” (Lavin, 2011). the participant becomes a ‘Spec-actor’ they are ap- Our augmented space aims to be a responsive over- plying their own agency to the experience. lay to the physical architecture, becoming an integral Examples of projects that take such approaches and immediately responsive part of it. includes the Nexen University ‘Infinity Wall’ [1]. This visualization, displayed on a 7 metre long LED screen is composed of four parts; Knock, Soaring, Combi- SYNESTHETIC ARCHITECTURE nation and Climbing, each depicting a different vi- The work in this paper is focused on establishing sual language. Knock uses the language of water how Synesthetic Architecture can be implemented and fluidity, depicting a giant wave within a three- by delivering an interactive augmented space that

576 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 dimensional space. The design of the wall was driven design as follows: by three principles; story mode, mood mode and • Visceral Design - The immediate attention grab- message mode. bing impact of designed media. • Behavioural design - The enjoyment and effec- GUIDING PRINCIPLES IN OUR WORK tiveness of the interaction. To test the ideas and principles above, in practice, dif- • Reflective design - The emotional value and ferent systems were developed. We adopted a core speculative discussion the design enables. principle of Emotional Design in generating an ap- propriate response. Emotional Design is explored by These criteria are then set as the Research goals in our Don Norman in his book ‘Emotional Design: Why we work, which was then respond specifically to each of love (or hate) everyday things’. He illustrates how un- the three levels of emotional design. The research derstanding human emotions when designing can was constructed to establish a response and solution result in better user experiences. He describes three for each of the nine component goals shown in Fig- consequent levels of brain processing; the visceral, ure 1. Measuring individual user response is chal- the behavioural and the reflective, each responding lenging in conventional circumstances, but with the to different design interventions in different ways. covid situation prevailing such user feedback proves Visceral design is centered around aesthetics and very difficult or impossible. The approach taken in grabbing the user’s attention. The visceral processing this research was therefore to set the three major level is an instinctive response, it is “about the initial goals (Visceral, Behavioral and Reflective) and have impact of a product, about its appearance, touch and three measures for each of these goals. In turn, this feel” (Norman, 2004). gave us 9 performance criteria. The research aims The behavioural level of the brain regards how a and achievements could then be measured against product is used or experienced. In the specific case of those 9 goals. So Figure 1 has the overarching criteria our research, behavioural design is the experience of and the three goals within each of them, plus a final the interaction taking place. The design must func- column that effectively records the achievement of tion in a satisfying way to create a positive emotional each of the nine components that were established experience. at the outset as the elements that captured the over- The reflective level uses consciousness and all research objective. The table in figure 1 and was thought to create the most effective emotional re- therefore the means of tracking the progress of the sponse. Norman describes the effects of the reflec- research. tive level on design thus: “It is only at the reflec- tive level that consciousness and the highest levels of feeling, emotions and cognition reside. It is only here TESTING ENVIRONMENT that the full impact of both thought and emotions are The conceptual ideas in the design performance experienced” (Norman, 2004). This thought process requirements established above, required specific can be targeted to leave an impression on the viewer tools and processes to achieve the desired outcome. and enable the viewer to engage with the theoretical The tools used explore the research intentions were or thought provoking concepts of the design. chosen to ensure a meaningful augmented experi- ence. The workflow of instruments that we exploited VISCERAL DESIGN - AESTHETICS, APPEAR- is shown in Figure 2. ANCE AND INITIAL REACTION The Instrument implementation took into ac- Consequently the the three levels of emotional brain count good practice research such as that by Nielsen processing can be aligned to an augmented space [2] to ensure novice user interface design was imple-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 577 Figure 1 Design Evaluation Criteria based on Donald Norman’s Emotional Design categories

Figure 2 Workflow of Instruments employed

578 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 mented. The actual instruments can be summarised Other input devices, such as microphones were as: used to collect and manipulate audio data. Display Tools: Different display methods are impor- Animation Software: Animation software is ex- tant to explore as it will alter the overall experience of plored to aid the creation of visualizations which tar- the design. Display tools that were tested include an get the brain’s visceral response. Cinema4D is the iPad, large LED screens and data projectors. primary animation software used in design research. To establish the techniques that could be aggre- This program creates aesthetically pleasing anima- gated to deliver the augmented space that the re- tions that explore scale, weight and materiality. search needed to test, 10 mini-experiments were de- Game Engine Software: Unity3D, a real time game veloped, that each contributed to achieving the nine engine software will be used to explore the spatial research outcome targets mentioned above: dynamics of the animations created. The Unity Aug- mented Reality toolkit is also used to explore the op- 1. Movement and Impact portunities of augmented application design. 2. Virtual Impression Visual Programming Software: Touch Designer is 3. Order and Disorder a visual programming tool which uses creative pro- 4. Fragmentation gramming to manipulate real-time data into visual 5. Surface transformation graphics. The software is used by a variety of creators 6. Weight and materiality to produce animations, interactive installations and 7. Surface Displacement multimedia content. 8. Movement and Impression Input Sensor Devices: Microsoft’s Xbox Kinect de- 9. Sound and Reaction vice is used to collect motion sensor data which is 10. Refelection able to be manipulated in the visual programming software. The Kinect device is available in two ver- REACTIVE AUGMENTED SPACE sions, both of which have different capabilities. Ver- There is not enough space in this paper to show sion one and two have both been used in the design all of the more developed components that deliver research. Figure 3 Displacement and Reaction

Figure 4 Voronoi Fracturing

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 579 Figure 5 Body Displacer driving Network

aspects of the mini-experiments. More advanced most useful successful element of this test. A feed- and composite systems included Flow Emitter, Tube back loop is a method in Touch Designer which cre- Instancing, Body Displacer, Hand Displacer, Body ates a blur effect of the image, it effectively leaves Tracking and Particle Systems. See figure 5 for an il- a trace of the user’s movement on the screen. In lustration of the body displays a technology network. this exploration, the colour gradient and feedback More details on the ones not reported here, and later are aligned. As the user moves their body leaves application of integrated systems can be found in a trace on the screen which is represented through Chan (2020). Here we focus on a particular set of per- colour. The body’s current position is represented in formance interventions that contribute to the over- a blue tone and the previous positions are conveyed all goals. Figure 5 shows the network of enabling el- through purple tones (see Figure 6). ements that drive the system and Figure 6 shows a resulting digital visual trace response on the digital Figure 6 device (e.g. large digital screen). Resulting response The Body displacer is used here as an example of when there is one of these integrated techniques that contributes movement to the whole project. In this exploration, Kinect data detected by the has been input to generate the RGB channels, thus re- Network flecting the interaction in the physical environment. The tx, ty and tz co-ordinates are generated by a grid and noise displacement. The input channels affect the instanced cube geometry; as the user moves the display reflects their silhouette. Effectively the intention and the effect is that the The colour input and feedback loop was the user’s body is leaving a digital trace on the virtual en-

580 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 vironment. This exploration introduces the concepts In the developed system a particle system has of time that is explored in Dan Graham’s project, Pre- two inputs. The first being the geometry that gener- vious Continuous Pasts (Francis and Peltzer, 2002). ates the particles and the second being the geometry Particle systems also proved to be a common that displaces the particles. fundamental core of the integrated systems. Our re- This design development is initiated by the cre- search development on these systems is summarised ation of a mechanism where the user produces the briefly here. particles. The Kinect data is input into the system and A Particle Node is used to create and control the is manipulated to form an outline of the user in three- parameters of a particle system simulation. Particle dimensional world space. This outline forms the ge- systems can be created in Touch Designer and can ometry which is input into the particle system as the have effective aesthetic outcomes. Controllable pa- particle generator. The animated digital image that rameters within the particle system include the num- the system land displays in response is identifiable as ber of particles, particle size, life period and mass. human form but not as a particular individual. So the Forces such as wind and turbulence can be applied person interacting to so anonymously. to displace and control the movement of the parti- The mechanism described above has multiple cles. The system input is composed of two parts; the successful aspects. In practice the outline of the user geometry in which the particles are generated from does not appear instantly and the system captures and the geometry in which they are displaced by. The the outline at timed intervals or when the user has particles generator could be geometry such as a grid, been still for a certain amount of time. The individ- a square or even a noise field. The displacer input is ual’s mark is becoming embedded into virtual space, where Kinect data can be added so that the user can with the visual record available for a short period of displace the movement of particles. time. The particle system has external forces applied,

Figure 7 This shows the script structure that detects presence and responds to multiple persons being detected. The system then responds with colour changes as well as patterns of graphic response.

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 581 as time transgresses the remnants of the user’s im- different levels of brightness. There is a direct rela- print slowly dissolves into digital space. tion between presence and visual output. The wall is reacting to the human atmosphere within the space. DENSITY This development of the idea of ‘density’ is designed to reflect multiple human presence within the space. APPLICATION IN AN URBAN SETTING Density explores how the system can react when The descriptions above show examples of how each multiple users are present. Initially this development of the components in delivering the overall system creates a relationship between the number of people were put into place and evaluated. With the nine to colour (see Figure 7). The code is developed so the research goals that were described in Figure 1 met, simulating colour changes gradient depending on if the next task was to illustrate application in threshold there is one user present or two (see Figure 8) spaces. One of the illustrations that we established was in the setting of the threshold to the building and the other an Urban threshold setting. Figure 8 Urban activation is a topic which is well re- Prototype response searched and explored. Unused urban spaces, such for multiple users as alleyways, car parks or public parks, can encour- age unwanted behaviour and compromise feelings of safety. These spaces often serve limited prag- matic purpose and lack a sense of place. In our case, this can be seen in the alleyway of Lukes Lane in Wellington CBD. The laneway it’s rather anonymous and has no defining characteristic or programme; lack of lighting and sporadic occupation result in the space feeling unwelcoming and indistinct. The re- sponse to this takes its lead from the observations by A script has been added in a similar method to the Wessel and Sauda: “For such spaces to better engage previous sound ripple development (see Figure 8). If people and take advantage of virtual and physical the second player is present then the RGB parameters capabilities, they must be adaptive to the people in in the geometry node are changed to the second gra- them, incorporating not just passive displays but also dients RGB channels. interactivity in a particular place as a form of socially- While this is a successful mechanism and visu- intelligent computing” (Wessel & Sauda, 2012). ally represents a change in presence, the transition There is therefore an opportunity to activate the between colours is static and not conveyed in a po- laneway through the use of a digital interactive in- etic manner. Consequently this was regarded as only stallation. The intervention sketched in Figure 9 pro- partially successful. So subsequently this was refined poses placing the interactive system within the alley- to deliver a more subtle transition has been taken, way using LED screens in the window of the adjacent where user presence is translated to a change in sat- building. The prototype intervention shown in Figure uration. 10 can therefore be extended and applied at a larger In this approach, as illustrated on the following scale in this location. The installation brings light to page, the more users present, the brighter the instal- the space, enticing people to play and interact, re- lation becomes. Using the Kinect CHOP data, up to sulting in the alley becoming populated with human six people can be recognised which correlates to six activity. Through embedded digital media, the alley-

582 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 way can become enlivened with social presence and CONCLUDING COMMENTS light, increasing the sense of occupation. The potential for the application of digital media as a form of materiality was outlined as an opportunity Figure 9 and aim in the research intentions set out above. An Theshold location effective digital system and physical application was and disaggregated explored and evaluated to test the research goals. components of the The design research resulted in two interactive sys- augmented space tems being developed that create visual representa- tions in response to physical actions; as a building threshold intervention and one as an urban interven- tion. The research goals were set by establishing the Figure 10 three core aspects of emotional response (Visceral, Augmented Behavioural, Reflective) that the urban intervention threshold space: system was aimed at triggering. These responses working prototype were then turned into actual performance goals as a in a building way of setting research targets; three for each emo- threshold tional response. The research project as a whole therefore was structured as a set of cumulative and interacting experimental components. Once each component was achieved the next component could be added (and possibly previous elements refined or modified). Achieving the nine component elements was the research target that was met. Unfortunately this paper does not allow us to describe all of the Figure 11 interactive components and accomplishments. Re- Augmented sponse to sound was also a major component; this threshold space: paper focusses on visual response. Actual Urban The interactions are also recorded by the system, threshold but body shape and movement outlines are recorded as pixellated or fragmented images that cannot be recognized. Consequently the system acts some- what like layer of digital archaeology, recoding those who have passed and interacted with the space but recording them with anonymity preserved. So in a The prototype installation for this space (see Figure way the system becomes like a recorder of digital ani- 11) aims to stimulate and enliven the laneway, pro- mated cave paintings, with an author who will always viding a public interface which can give the pub- be anonymous.To the aspect of recording animation lic “the ability to create and communicate meaning we have also added the recording of indistinct sound. through their interaction with the system” (Wessel & The creation of a digital interactive system to the Suada, 2012). set goals has provided proof of a concept that re- sponds to criteria established by researchers such as Don Norman (2004) and reinforced by others over re-

VR, AR and interactive visualization - Volume 2 - eCAADe 39 | 583 cent years. The practical outcomes show how in prin- Rules to Enable Reflection: Creating the Difference:’, ciple a real-world augmented space system, that re- ProceedingoftheChiSparks2014Conference, Nether- spond to a range of research generated goals relating lands, pp. 122-125 Schnabel, M.A. 2009, ’Framing Mixed Realities.’, in Schn- to positive emotional response, can be successfully abel, M and Wang, X (eds) 2009, Reality in Architec- implemented in built environment settings. ture, Design & Construction, Springer, Netehrlands, pp. 3-11 REFERENCES Wessel, G. and Sauda, E. 2012 ’Urban User Interface.’, ACSA International Conference, Barcelona, pp. 180- Alavi, H.S., Churchill, E., Lalanne, D., Kirk, D., Nembrini, J. 185 and Moncur, W. 2016, Future of Human-Building In- [1] http://www.dstrict.com/kr/InfinityWall teraction, CHI, San Jose, CA, USA [2] https://www.nngroup.com10UsabilityHeuristicsfor Alavi, H.S., Churchill, E., Wiberg, M, Lalanne, D., Dals- UserInterfaceDesign-Nielsen gaard, P, Fatah gen Schieck, A and Rogers, Y 2019, ’Introduction to Human-Building Interaction (HBI): Interfacing HCI with Architecture and Urban De- sign’, ACM transactions on computer-human interac- tion, 26, pp. 1-10 Anadol, R 2020, ’Synaesthetic Architecture: A building Dreams’, Architectural Design, 90, pp. 76-85 Biggio, F 2020, ’Guidebook for Mirror Worlds: Poetics of Transparency in Augmented Reality’, Space & Tech- nology Journal, 19, p. 97 Boettger, T 2014, Threshold Spaces: Transitions in Archi- tecture. Analysis and Design Tools, Birkhauser Cannaerts, C 2016, Coding as Creative Practice: Complex- ity and Simplicity, Proceedings of the 34th eCAADe Conference - Volume 1, pp. 397-404, Finland: Uni- versity of Oulu. Chan, H 2020, If Walls could Talk, Master’s Thesis, Victoria University of Wellington, New Zealand Dalton, N, Schnadelbach, H, Wilberg, M and Varoudis, T 2016, Interaction Human Computer Interaction in Space and Place., Springer Francis, M and Pelzer, B Dan Graham. London: Phaidon., Dan Graham, Phaidon, London Gonzalez-Holland, E, Whitmer, D, Morales, H and Mouloua, M 2017 ’Examination of of the Use of Nielsen’, Proceedings of the Human Factors and Er- gonomicsSocietyAnnualMeeting, Los Angeles: SAGE Publications., pp. 1472-1475 Lavin, S 2011, Kissing Architecture, Princeton University Press Manovich, L 2016, The Poetics of Augmented Space., Vi- sual communication, p.219-240, Vol. 5„ London, Matthews, S 2006, ’The Fun Palace as Virtual Architec- ture: Cedric Price and thePractices of Indertermi- nacy.’, Journal of Architectural Education, 1, pp. 39-48 Norman, D.A. 2004, Emotional design why we love (or hate) everyday things, Basic Books, New York Quanjer, A.J. and Lamers, M.H. 2014 ’Breaking Usability

584 | eCAADe 39 - VR, AR and interactive visualization - Volume 2 Index of Authors

A Campos, Tatiana 1-355 Achten, Henri 1-403 Capone, Mara 1-437 Agirbas, Asli 2-391 Carvalho, João 2-483 Agkathidis, Asterios 2-233, 2-303, Chan, Holly 2-575 2-381 Chen, Fukai 2-223 Agraviador, Armand 2-473 Chen, Hanmei 2-233 Aksin, Feyza Nur 2-409 Chen, Hao 2-547 Alassaf, Nancy 2-39 Chen, Yu 1-55 Althoff, Klaus-Dieter 1-45, 2-351 Cheng, Jack 1-563 Amtsberg, Felix 1-303 Chiu, Yu-Hung 2-199 Anishchenko, Maria 1-253 Chiujdea, Antinozzi, Sara 2-211 Ruxandra Stefania 2-171 Apolinarska, Cichocka, Judyta Maria 2-493 Aleksandra Anna 1-497 Clayton, Mark 2-39 Arora, Hardik 1-45, 2-351 Çolakoğlu, Arslan Selçuk, Semra 2-409 Meryem Birgül 2-161 Azadi, Shervin 1-263, 1-285 Coraglia, Ugo Maria 1-295, 2-253, Azambuja Varela, Pedro 1-189 2-341 Correa, David 2-463 B Cortes Perez, Juan Pedro 2-59 Balaban, Özgün 2-273 Costa, Eduardo 2-333 Barba, Salvatore 2-211 Coutinho Quaresma, Bartov, Nitsan 1-55 Filipe 2-313 Basogul, Elif Feyza 2-391 Crompton, Andrew 2-303 Belousov, Boris 1-27 Cruz, Paulo J. S. 1-355, 2-483 Bengisu, Murat 1-115 Cursi, Stefano 1-295, 2-253 Bielski, Jessica 1-45, 1-393, 2-351 D Bitting, Selina 1-263 Dade-Robertson, Martyn 1-77 Blaney, Adam 2-431 Dan, Yuze 1-205 Blay, Georgina 2-463 Dania, Panagiota 2-263 Borgese, Daniela 2-77 De Lorenzo, Andrea 1-537 Borpujari, Nihit 1-55 De Luca, Francesco 2-105, 2-419 von Both, Petra 1-275 Dengel, Andreas 1-45, 2-351 Brown, Andre 2-575 Djuric, Marko 2-49 Burger, Joris 2-199 Dokonal, Wolfgang 2-89 Butler, Andrew 2-9 Dong, Yuebin 1-415 Doumpioti, Christina 1-9 C Duarte, Jose 1-365

Authors - eCAADe 39 | 585 Dzurilla, Dalibor 1-403 2-199 Grasser, Alexander 1-181 E Gratteri, Andrea 2-77 Eisenstadt, Viktor 1-45, 2-351 Guida, George 1-415 Šamec, Elizabeta 2-503 Guo, Xiangmin 2-223 Emo, Beatrix 2-115 Guo, Zhe 1-487 Erioli, Alessio 1-17 Gürsel Dino, İpek 1-555 Eshaghi, Sarvin 2-273 Guterres, Filipe 2-313 Estrina, Tatiana 2-123 Güzelci, Orkan Zeynel 1-455 Evrim, Berfin 1-365 H F Hadighi, Mahyar 2-293, 2-333 Faller, Arnold 1-375 Haeusler, M. Hank 1-325, 2-9 Feringa, Jelle 1-515 Heidari, Farahbod 1-95 Figueiredo, Bruno 1-355, 2-59, Hensel, Michael U. 1-151 2-483 Hilmer, Jacob 2-443 Fioravanti, Antonio 1-295, 2-77, Hirschberg, Urs Leonhard 1-141 2-253, 2-341 Hjelseth, Eilif 1-151 Fiorillo, Fausta 2-211 Hölscher, Christoph 2-115 Foged, Isak 2-443 Holzmann, Thomas 2-21 Frick, Christian 1-515 Hu, Xinchuang 2-555 Fujii, Hiroyuki 2-513 Huang, Alvin 2-123 Fukuda, Tomohiro 2-513, 2-521, Huang, Jeffrey 1-9 2-529, 2-539, Huang, Yurong 2-9 2-547 Hui, Vincent 2-123 Funk, Niklas 1-27 Hussein, Hussein E. M. 2-381 Furtado Lopes, Gonçalo 1-445 Huyeng, Tim-Jonathan 2-89

G I Gao, Tianyi 2-153 Đurić, Isidora 2-243 Garcia del Castillo Lopez, Jose Luis 1-563 J Gardner, Nicole 2-9 Jarzyna, Michał 2-31 Gaston, Elizabeth 2-473 Jauk, Julian 2-503 Gerber, Andri 2-115 Ji, Guohua 1-477 Gero, John S. 1-123 Jiang, Jingjing 1-235 Geropanta, Vasiliki 2-263 Jovanović, Marko 1-505 Gönenç Sorguç, Arzu 1-465 Joyce, Sam Conrad 1-243 Gosch, Lukas 2-503 Junk, Christoph 1-515 Gradisar, Luka 2-59 Gramazio, Fabio 1-497, 1-515, K

586 | eCAADe 39 - Authors Kaiser, Benjamin 1-345 Lloret-Fritschi, Ena 2-199 Karagianni, Anna 2-263 Lo, Tiantian 2-223, 2-555 Khayami, Sima 1-95 Loj, Szymon 2-493 Kholodova, Janna 2-463 Lombardi, Davide 2-233 Khoss, Konstantin 2-67 Lorenz, Wolfgang E. 1-375 Kieferle, Joachim 2-49 Lu, Ming 2-145 Kikuchi, Naoki 2-521 Luis, Orozco 1-303 Kikuchi, Takuya 2-529 Kićanović, Jelena 2-243 M Knippers, Jan 1-303 Ma, Yiwen 1-487 Kocić, Nastasija 1-161 Mahdavinejad, Kohler, Matthias 1-497, 1-515, Mohammadjavad 1-95 2-199 Majzoub, Omar 1-325 Kontovourkis, Odysseas 2-189 Marsillo, Laura 1-437 Körner, Andreas 2-453 Mei, Zihan 1-563 Kotov, Anatolii 1-37 Menges, Achim 1-303, 1-345 Kovacic, Iva 2-67 Mesa, Olga 2-565 Kramberger, Aljaz 2-181 Mintrone, Alessandro 1-17 Krasić, Sonja 1-161 Mitković, Mihailo 1-161 Krnjaić, Aleksandar 1-505 Mitković, Petar 1-161 Kronenburg, Robert 2-381 Moleta, Tane 2-575 Krtschil, Anna 1-303 Mondal, Joy 1-65, 1-427 Krężlik, Adrian 1-455 Morrow, Ruth 1-77 Kuhn, Mathias 1-497 Kunic, Anja 2-181 N Naboni, Roberto 2-181 L Nada, Hideki 2-513 Lacroix, Igor 1-445 Nakabayashi, Mizuki 2-539 Langenhan, Christoph 1-45, 1-393, Naylor, John Osmond 1-315 2-21, 2-351 Nazim, Ibrahim 1-243 Lanzara, Emanuela 1-437 Nejur, Andrei 2-105 Liapi, Katherine 2-323 Newton, David 1-527 Ligler, Heather 2-283 Nicholas, Paul 1-55, 2-171 Lim, Ariel Cheng Sin 1-85 Nikolić, Marko 1-161 Lin, Bian 1-235 Nourian, Pirouz 1-263, 1-285 Lin, Chaohe 2-555 Nováková, Kateřina 2-137 Lin, Yumin 1-205 Nowobilski, Marzena 2-283 Lindner, Clemens 2-21 Liosi, Dimitra 2-323 O Liu, Jie 1-195 O’Keefe, John 2-565 Liu, Yuxi 1-27 Obradović, Miloš 2-243

Authors - eCAADe 39 | 587 Obradović, Ratko 2-243 Sharp, Alexa 2-463 Oral, Hülya 2-161 Shen, Yuhui 1-477 Ozkan, Dilan 1-77 Shen, Zhenjiang 1-205 Silva Dias, Joana 1-189 P Simeone, Davide 1-295, 2-253, Pak, Burak 1-131 2-341 Pan, Yue 1-563 Simon, Bechert 1-303 Paoletti, Ingrid 1-253 Skoury, Lior 1-303 Parger, Alexandra 1-181 Sliwecki, Bartosz 1-385 Parthenios, Panagiotis 2-263 Slotina, Kristine 1-151 Pencreach, Yoann 2-513 Sommer, Simon 1-275 Peters, Jan 1-27 Song, Yanan 1-335 Petzold, Frank 2-21 Sönmez, Ayça 1-465 Pibal, Sophia 2-67 Sonne, Konrad 2-171 Sopher, Hadas 1-123 Q Sousa, José Pedro 1-189, 1-445, Qi, Yue 1-345 1-455 Qiu, Song 1-487, 2-371 Stanković, Jovana 1-161 Stavric, Milena 2-503 R Stojaković, Vesna 2-243 Rath, Anca 2-21 Stojanovic, Djordje 2-401 Refsgaard, Andreas 1-55 Strauss, Jürgen 1-515 Rock, Johanna 1-141 Symeonidou, Ioanna 1-213 Rodríguez Hernández, Szentesi-Nejur, Szende 2-105 José Luis 2-59 Romero, T Rosaura Noemy Hernandez1-131 Tahouni, Yasaman 1-345 Ronchi, Diego 2-211 Tamke, Martin 1-173 Rüppel, Uwe 2-89 Techathuvanun, Chanon 2-199 Rust, Romana 1-515 Teng, Teng 1-105 Tepavčević, Bojan 1-225 S Tessmann, Oliver 1-27 Sabin, Jenny 1-105 Theodoropoulou, Scaffidi, Antonio 2-171 Helena G. 2-263 Scheich, Patrick 2-89 Thomsen, Mette Ramsgaard 1-85 Schnabel, Marc Aurel 2-575 Tian, Runjia 1-415 Schneider-Brachert, Wulf 2-21 Tong, Ziyu 1-477 Scott, Jane 2-473 Trento, Armando 2-77 Sdegno, Alberto 1-537 Tryfonos, George 2-189 Sebestyen, Adam 1-141 Turhan, Gözde Damla 1-115 Sepúlveda, Abel 2-419

588 | eCAADe 39 - Authors V Yu, Zhongsheng 1-235 Vaez Afshar, Sepehr 2-273 Yuan, Chao 1-235, 1-487, Varinlioglu, Guzden 1-115, 2-273 2-371 Varjas, Toivo 2-419 Yuan, Philip F. 1-335, 1-487, Vasiljević, Ivana 2-243 2-145, 2-153 Vazquez, Elena 1-365 Vašatko, Hana 2-503 Z Vele, Jiří 2-137 Zeng, Shaoting 2-371 Verl, Alexander 1-345 Zhan, Qiang 2-153 Vujovic, Milica 2-401 Zhang, Liming 2-145, 2-153 Vukorep, Ilija 1-37 Zhang, Meng 1-77 Vučić, Marko 1-225 Zhang, Ran 1-477 Zhang, Xiao 1-235, 1-487, W 2-371 Wagner, Hans-Jakob 1-303, 1-345 Zhao, Jiangyang 2-233 Wang, Likai 1-477 Zhao, Zhilong 2-371 Wang, Xiang 1-235 Zhong, Ruqing 1-345 Wang, Yuyang 2-303 Zhou, Xinjie 2-145 Wangler, Timothy 2-199 Zhou, Yifan 2-145 Weissenböck, Renate 2-95 Zhu, Weiran 2-145 Werner, Liss C. 1-95 Zhu, Yiyun 1-205 Wibranek, Bastian 1-27 Zhu, Zhelun 1-295, 2-341 Widbiller, Matthias 2-21 Ziegler, Christoph 1-45, 1-393, Wloczyk, Marta Magdalena 2-493 2-351 Woessner, Uwe 2-49 Zolghadrasli, Niloofar 2-333 Wu, Hao 2-153 Wu, Shaoji 1-545, 2-361 Wurzer, Gabriel 1-375

X Xiao, Jianqiang 1-205 Xydis, Achilleas 1-515

Y Yabuki, Nobuyoshi 2-521, 2-529, 2-539, 2-547 Yan, Muchen 1-173 Yang, Liu 1-487, 2-371 Yang, Qingfeng 1-235 Yazıcıoğlu, Gülin 1-555 Yu, Peiran 1-487

Authors - eCAADe 39 | 589 590 | eCAADe 39 - Authors

CIP - Каталогизација у публикацији Библиотеке Матице српске, Нови Сад

72.012:004(082)

ECAADE Conference on Education and Research in Computer Aided Architectural Design in Europe (39 ; 2021 ; Novi Sad) Proceedings : eCAADe 2021 Towards a New, Configurable Architecture : Vol. 2 / 39th eCAADe Conference on Education and Research in Computer Aided Architectural Design in Europe, 8-10th September 2021, Novi Sad ; [editors Bojan Tepavčević, Vesna Stojaković]. - [Bruxelles] : eCAADe [i. e.] Education and research in Computer Aided Architectural Design in Europe ; Novi Sad : Faculty of Technical Sciences, 2021 (Novi Sad : FTN, Grafički centar GRID). - 613 str. : ilustr. ; 20 x 20 cm

Tiraž 150. - Bibliografija uz svaki rad. - Registar.

ISBN 978-86-6022-358-8

а) Архитектонско пројектовање -- Примена рачунара -- Зборници

COBISS.SR-ID 43611913