“Gateway to the Past”: Designing AR-enriched Brno City Guide to Historical Gates
Masaryk University Faculty of Informatics Bachelor’s Thesis
Spring 2020 Júlia Gonová
Declaration
Hereby I declare that this thesis is my original authorial work, which I have worked out on my own. All sources, references, and literature used or excerpted during elaboration of this work are properly cited and listed in complete reference to the due source.
Júlia Gonová
Supervisor: MgA. Helena Lukášová, ArtD. Acknowledgements
Firstly, I would like to thank my supervisor, MgA. Helena Lukášová, ArtD., for her kind guidance, advice and support throughout this whole process.
My thanks also goes to the employees of NPÚ ÚOP in Brno and Muzeum města Brna for providing me with relevant historical information.
Last but definitely not least, I want to express my gratitude to my parents for everything they have given me and for never once wavering in their conviction that I will finish this work, and my brothers, for helping, supporting, and believing in me in their own brotherly way. Abstract
This work focuses on the design and development of an interactive city history guide using augmented reality. To overcome the need for special devices or dedicated applications, the augmented reality aspect of this work is facilitated through a platform-independent web application. The printed guide provides basic information about the main topic, the historical gates of Brno, as well as their likenesses as 2D images. At the same time, the web application enriches the experience by supplying interactive 3D models of the gates in augmented reality.
Index terms augmented reality, web augmented reality, webAR, web application, 3D modelling, historical architecture, history, architecture, design, graphic design, JavaScript, HTML Contents
0 Introduction 1
1 Augmented reality and its uses 3 1.1 Augmented reality 4 1.1.1 Uses of AR 4 1.1.2 Types of AR based on the tracking mode 6 1.1.3 AR mediums and disadvantages 8 1.2 WebAR 8 1.2.1 The AR.js library 9
2 Historical research about the city gates 11 2.1 Měnínská brána 13 2.2 Židovská brána 14 2.3 Ferdinandova brána 15 2.4 Brněnská brána 15 2.5 Veselá brána 16 2.6 Běhounská 16
3 Creation of the 3D models of the gates 19 3.1 Creating blueprints in Adobe Illustrator 20 3.2 Modelling in 3Ds Max 21 3.3 Sculpting in Sculptris 23 3.4 Selecting the model format for WebAR 24 3.5 Texturing in Blender 24 3.5.1 UVW Unwrapping 24 3.5.2 Choosing colours and materials 25 3.5.3 Software and techniques 26 3.5.4 Retouching the models for better texturing 27
4 Designing a city history guide 29 4.1 Design of pictograms used as AR markers 30 4.2 Design and layout of the guide 31 4.3 Typography of the guide 33 4.4 Contents of the guide 34 4.5 Map design 34 4.6 Assembling the final booklet 36
5 Implementation of the webAR application 37 5.1 Implementing a basic webAR application 38 5.2 Implementing the user interface 39 5.3 Implementing controls 39 5.4 Workflow of testing 40 5.5 Encountered problems, issues and bugs 41 5.6 Deployment and use of the web application 43
Conclusion 45
Bibliography 47
Electronic attachments 51 0Introduction For years now, modern technology has been continuously providing us with unique new ways of preserving history and presenting it to the public. In particular, life-like 3D digital models of architecture, whether standing or long-buried, allow for a more personal connection to the past than literature or flat imagery ever could. So many buildings remain no more than a few drawings and descriptions. However, we now have a unique opportunity to breathe life into these forgotten pieces of architecture, thanks to the growing capabilities of not only modelling software, but also smart devices.
Like most medieval cities, Brno had its fortifications and gates leading in and out of it, but nowadays few people know of them. Although the gates had to give way to the city’s expansion long before they could be photographed, and drawings of them are few and not always clear, there is enough material to give us an idea of how they looked and where they stood. They were all unique in their use and design, and all but one of them met the same fate during the 19th century — they were torn down to allow Brno to expand. The last remaining gate, the Měnínská gate, underwent so much change that its current appearance does not say much of its past.
Brno has its fair share of monuments to admire, but the forgotten gates are sadly 2 underrepresented. Thus, the main goal of this work is to use modern means to present them in an accessible, interactive way. To do so, I will design a graphic city guide to present basic information about the gates and hold their simplified representations in the form of a set of pictograms. Each pictogram will serve as a marker for a platform-independent web application that will display a 3D model of the given gate in augmented reality (AR). The app will provide some basic controls over the models, such as zooming and rotation. The 3D models will be created as accurately as possible based on historical drawings, literary mentions, and the few remaining pieces of the gates.
The first chapter introduces AR both in general and in the context of web applications. The second chapter provides an overview of the available historical information about the gates. The third chapter deals with creating 3D models, while the fourth chapter presents the graphic design of the printed guide. The fifth chapter describes the process of implementing and testing the webAR application. Augmented reality and 1 its uses 1.1 Augmented reality
Augmented reality (AR) is the enrichment of a real-world environment by computer-generated data, which can be visual or auditory. In comparison with virtual reality (VR), the real-world environment is not entirely substituted with a digital one. AR alters or adds to real-world objects and places, and thus provides the user with additional information and interaction.
This work describes augmented reality, its uses, types, advantages and disadvantages only briefly, as these topics have been previously covered in other works.1 Instead, the focus of this work will be on WebAR and the circumstances and issues specific to graphic design and 3D modelling in conjunction with WebAR.
1.1.1 Uses of AR
AR can enrich life in many different ways and areas, from entertainment through education to engineering. It is an excellent way of making advertisements more eye-catching and effective, and even cut down on the amount of space they take up in the 4 real world (and consequently lower the price of the ad), since they can be more detailed in the virtual one. Another area in which AR can be helpful is education — it can improve the quality and enjoyment of learning by allowing for quick, simple, and interactive ways to view the study material (such as displaying 3D models of internal organs, for example [1]). Interior designers, architects or engineers can also see the results of their work in a real-world environment before they create the actual physical product. Moreover, AR can be used to complete an artist’s vision and give their art new depth, and in conjunction with that, help to present history and historical architecture to tourists in novel ways.
AR in history and architecture
There have been multiple projects utilising AR to present history and architecture to the public. For example, an ongoing project called SensiMAR [2] uses AR, speakers, and a machine producing
1 For example, see J. Vavrík’s Model of the Historical Building in Augmented Reality, Masaryk University, Faculty of Informatics, 2019. smells to present the historical site of the House of Cantaber in the Conimbriga Ruins in Portugal. The SensiMAR application, developed by Anabela Marto, shows the user a 3D rendering of the building as it once was, overlayed over the current ruins. As a 360-degree experience, it allows the user to walk around the ruins and explore the site. It is enriched with an animation of Romans walking around and the sound of them talking, wagons riding by, and the smell of the fish market that used to be nearby.
Another example is an application [3] developed in 2019 by Ahmed Emara, which uses AR to overlay ancient ruins in Alexandria, Egypt, with a virtual rendering of what the tomb once looked like. It completely replaces the surrounding ruins with the created 3D objects, and it features atmospheric sound and animated fire.
Augtraveler [4] from 2019, is an educational travel application developed for tourists visiting India’s cultural heritage sites, and it is available both for iOS and Android devices. 5 It features many different augmented 3D models, audio, text and pictures. The application presents information about various subjects in the real-world environment, and offers closer looks
Fig. 1.1: Screenshot from at detailed models of buildings or artefacts. Figure 1.1 shows an Augtraveler video [4] example of the application’s features [4].
BAMPFA AR — Augmented Timeline is a project led by architect Luisa Caldas [5]. Her team created an iOS application for interactive storytelling about the history of the new university building. The application uses AR to present the building at different stages of its existence, with digital models superimposed onto physical artifacts or models of the building. Consequently, the experience is limited to the place where the artifacts are exhibited.
AR in graphic design
For a few years now, AR has also been used in educational children’s books. While the books themselves contain pictures and text for the children to learn from, the accompanying AR application brings the material closer to the children by playing audio and displaying 3D models or animations to enrich the experience. Amongst such books is the Animal Kingdom book by Deform Studio, which maps augmented 3D animations and sounds to specific pages and pictures in the book. The augmented content is even interactive — for example, the 3D elephant trumpets when the user taps it on the screen [6]. A similar approach is taken in a prototype educational application by Arloopa, which augments a physics textbook to explain the material better [7].
A most notable example of the use of AR in graphic design is the POST-PRINT project [8] from 2017 by Serafim Mendes. The author developed an app, available both for iOS and Android devices, which augments the author’s graphic design posters. Although originally showcased at the author’s installation, the posters are available online [8]. Thus, anyone can use the application to view the posters, which are subsequently augmented into animated pieces of graphic art. Figure 1.2 shows one of the 2D posters [8], and Figure 1.3 shows the augmented 3D content.
6
Fig. 1.2: POST-PRINT poster [8] Fig. 1.3: POST-PRINT poster in AR
1.1.2 Types of AR based on the tracking mode
There are multiple ways AR can be categorised, but one of the most notable ones is based on tracking — how the software places the objects in the scene. Tracking mode divides AR into three broad categories: marker AR, markerless AR, and location-based AR. Marker AR
In marker AR, the real world is scanned for specific patterns called markers. The detection of a marker triggers the predefined functionality, such as showing a picture, some text, or a 3D object, playing audio, or running any scripted code. Visual AR content is not necessarily displayed over the marker; it can float directly on top of the marker, be entirely outside the viewport, or be anywhere in between. Depending on the specific implementation, a marker can be a simple pictogram, a barcode, a regular 2D image, an animated gif, a 3D object, or a 2D video.
Multiple markers can be used in a single application, and they can work not only individually, but also together as a sort of composite marker. These usually enhance performances, stabilise the display of the AR content [9], and can even be used as a means of Fig. 1.4: Multiple markers for distance measuring [10] gathering data about the real world. For example, 7 two markers can be used to measure the distance between their centres, as is shown in Figure 1.4 [10].
Markerless AR
Markerless AR tracking gathers information about the real-world environment via the camera, and uses the device’s internal sensors to establish its orientation and location in the real‑world environment. It is also capable of recognising walls, horizontal planes, etc. [11]. Many game applications use markerless AR to incorporate 3D elements into the real world, often in conjuction with location-based AR, such as 2019’s Harry Potter: Wizards Unite by Niantic, which drew inspiration from the 2016’s hit game Pokémon Go developed by the same company.
Location-based AR
Location-based AR means the AR functionality is triggered when the device enters the area around predefined GPS coordinates. For example, a 3D model may be shown floating around a city square in AR. 1.1.3 AR mediums and disadvantages
There are multiple ways AR can be experienced, ranging from special devices and tools such as head-mounted displays, special eyeglasses, or projectors, to relatively simple handheld devices such as mobile smartphones or tablets. While there are many sophisticated tools both in development and already available which can provide users with a more immersive experience, the accessibility of smart handheld devices makes them an excellent medium for the enrichment of commerce, tourism, education, art, and more.
Understandably, handheld devices have limitations, the most apparent one being that the size of the screen can never allow for a truly immersive experience. However, a solution to this problem may be a tool such as Google Cardboard [12]. Google Cardboard, or any similar apparatus, is a viewer designed to obscure the user’s peripheral vision, thus narrowing it to only the screen of the handheld device. Additionally, lenses inset into the device and placed between the wearer’s eyes and the screen of the smart device make the displayed images appear more three dimensional.
Although applications for mobile devices open the world to the many possibilities AR 8 offers, they still may be slightly restrictive in practice. They may require purchase or be laden with ads, not to mention they need to be downloaded and take up space on the device. A solution which offers on-demand AR for (not only) mobile devices, assuming a stable enough internet connection, is WebAR.
1.2 WebAR
WebAR is augmented reality facilitated via a webpage. Instead of having to use dedicated tools or applications to experience AR, one simply visits a webpage built for viewing AR directly in the browser. The web application requests access to the device’s camera (and depending on the type of AR, its location and orientation), and it is ready to use. Thus, it can be platform-independent, free, and easy to access. There are a few commercial providers of AR on the web, such as 8th Wall or Blippar. Some of these services can work directly with AR frameworks developed alongside the device’s operational system, such as Apple’s ARKit [13] for iOS, which allows for even better results and smoother experience. A completely free and open-source solution for a platform-independent WebAR application is the library AR.js, which works on top of the A-Frame framework. This work uses the AR.js library for the implementation of the WebAR application.
1.2.1 The AR.js library
The AR.js library [14] is a completely free and open-source project. It offers AR with markers (and image tracking) and location-based AR. It uses JSARToolKit5 for tracking, and the A-Frame framework for displaying 3D content on the web, which in turn uses the Three.js library. The framework makes it easier to use the three.js functionality thanks to custom HTML components. The developer can choose whether to use A-Frame or the Three.js library directly, although the location-based AR feature is available only through A-Frame. 9 The library allows for a relatively smooth and fast experience, achieving up to 60 fps on mobile devices, according to its documentation [14]. It is compatible with any device that has WebGL2 and WebRTC 3 (both are widely used), and requires no installation or additional applications and tools. The main advantages of creating a WebAR application with this library come from its accessibility and simplicity. The developer only has to handle the triggered functionality and implement what happens when a marker is found or when the device enters a specific location. Marker detection, image tracking, location detection, access to the device’s camera, location and orientation, display of 3D models on the web, and everything else that needs to be set up for the display of the AR content is pre-handled by the underlying libraries and frameworks. This allows the developer to focus on the front-end of the application and the final results.
2 WebGL — Web Graphics Library — is a JavaScript API that facilitates rendering and displaying 2D and 3D in a web browser. 3 WebRTC — Web Real-Time Communication — is an open framework for browsers that facilitates peer-to-peer video, audio and generic data communication in real time. The downside of this open-source solution is that it cannot make full use of all of the device’s capabilities, unlike when working with, for example, the ARKit for iOS. It cannot handle larger amounts of data on mobile devices, since the models are not stored on the device and then retrieved when needed (like in phone or tablet applications), but have to be instantly pulled from the cache of the web application. Consequently, this puts restrictions on the level of detail or number of the 3D objects used in the AR experience. Each device may also have its own limits on how much data it can handle, and thus the results may vary across devices.
Nevertheless, the accessibility and simplicity of a WebAR solution make it more suitable for quick enrichment of real-world objects by anyone with a smart device, compared to a solution that requires the acquisition of a dedicated application or the use of special tools. The selected approach uses marker-based WebAR to display the augmented content over the printed guide.
In the early stages of research for this work, location-based AR was also considered for the purpose of showing the gates in the places where they once stood. However, this was not chosen as the final solution for two main reasons. One is the necessity of having an 10 internet connection in the places where the gates stood, which either means having access to public wireless internet, or using cellular data. The heart of Brno and the gates’ past locations are not fully covered with public internet, and cellular data may be costly for some, especially for loading a website with multiple 3D objects. The other issue was the potential danger of viewing gates “in the field”. For example, the place where theŽidovská gate once stood is now a major transportation hub, located next to the train station. The high-frequency car and tram traffic, coupled with dozens of pedestrians, does not create suitable conditions for a comfortable or safe AR experience. Thus, marker-based AR was picked as a reliable, cheap, and safe alternative to location-based AR. Historical research about 2 the city gates The city fortification of Brno was a successful and impressive feat of engineering. During its existence, it gained (and lost) different parts, like towers, bastions, a second wall, or a Zwinger. Additional protection was usually built as the need arose, and the fortification served its purpose admirably, notably during the Swedish attack in 1645.
Gates were understandably integral to the city’s day-to-day life. Depending on the city’s and citizens’ needs, new gates were built and the existing ones rebuilt or fortified. For most of the existence of the city fortification, five main gates led in and out of the city:Brněnská brána (“The Brno Gate”),Veselá brána (“The Cheerful Gate”), Běhounská or Dřevěná brána (“The Runners’” or “Wooden gate”), Měnínská brána (“The Měnín Gate”), Židovská, in its place later Ferdinandova brána (“The Jewish”, later “The Ferdinand’s Gate”). There was a number of smaller, less significant gates, such as Slepá or Hackelova brána (“The Blind”, or “Hackel’s Gate”), or Nová Branka (“The New Small Gate”)4. These were the gates set into the main wall. Some of the suburbs outside the heart of the city had their own walls and gates, but these and the two smaller gates will not be discussed in this work.
My research of the gates consisted of first looking up materials online, then contacting additional sources, such as the National Heritage Institute (Národní památkový ústav, 12 NPÚ) and Brno City Museum (Muzeum města Brna) at the Spilberk castle. Additionally, I also consulted the digital models of medieval Brno [15] created by the company Archaia [16]. Their model of Brno as it was in 1645 served as a template for a physical bronze model of the city that is now a permanent feature of the square in front of the Governor’s Palace (Místodržitelský palác). A rich source of information about the gates was the work of an architect and historian of urbanism, Karel Kuča, especially his book Brno – vývoj města, předměstí a připojených vesnic [17].
From these sources, I have collected letters, news articles, drawings, paintings, and more. Some materials provided more substantial information than others. However, considering all but one of the gates were destroyed before they could be photographed, reconstructing their exact appearances proved to be impossible. As such, in this work, I will try to create models of the gates that are as accurate as possible, while taking
4 In the rest of this work, the gates shall be referred to by their Czech name, e.g. Měnínská gate. some artistic licence based on assumptions we can make about how city gates in central Europe are usually built. For example, I found almost no information about what the gates looked like from inside the city. Thus, I worked with the assumption that they were mostly symmetrical, like the Powder Gate in Prague. I also took into account that some decorative elements, such as city shields, were probably only displayed on the outside part.
Seeing as the guide I will be creating will hold the most important historical information about the gates, the following sections will be mainly concerned with the gates’ appearances and sources of information about this matter.
2.1 Měnínská brána
This gate, being the only one to survive to date, underwent its fair share of changes over the centuries. NPÚ holds some simple sketches showing these changes, along with a document from the 20th century describing the gate’s history. Over the last almost 300 years, the exterior of the gate has not changed significantly. It is a simple building with 13 a large arch opening, big enough for carriages (nowadays for cars) to pass through. The gate’s interior was rebuilt into a residential space some time after the Swedish war in 1645. It became private property and most probably remained so until the 20th century.
The gate changed owners a few times, but ultimately it was repurchased by the city, and rebuilt again between 1978 and 1982. Since then, it has served as a museum and a gallery, changing expositions every few years. It has been closed to the public since January 2020 due to repairs [18]. Figure 2.1 pictures the Měnínská gate in the 20th century [19].
Fig. 2.1: Měnínská brána, beginning of the 20th century [19] 2.2 Židovská brána
Out of all the gates, pictorial sources paint the most detailed image for this one. Heavily ornamental and decorated, it boasted seven consoles5 with human faces, arches, decorative statues, a lintel with an inscription, an embrasure,6 and possibly more. The consoles, lintel and embrasure are preserved at the Spilberk museum. The most detailed picture we have of the gate is a painting from 1820 by František (Franz) Richter, available in the online archives of the Moravian Gallery in Brno (Moravská Galerie v Brně) [20]. It depicts the everyday bustle of the city outside the gate. Figure 2.2 is the black‑and‑white copy of this painting. Fig. 2.2: Židovská brána, beginning of the 19th century, from the archives of NPÚ Brno A small but detailed relief commemorates the gate now. It decorates a building at the end of Masarykova Street, close to where the gate once stood. It is a surprisingly accurate 14 depiction that corresponds to the painting very well. Somewhat incorrectly, it bears the title Das alte Ferdinandstor (“The Old Ferdinand’s Gate”), while depicting the Židovská gate and not Ferdinandova gate. However, this might simply be because at the time of the relief’s creation,7 the torn down gate might have been known under that name.
For a long time during its existence, the gate was also known as Zelená brána, “The Green Gate”. This was due to the green roof, which covered the gate’s tower [21]. The green colour was the result of oxidation of the metal material used on the roof cover — after oxidising, the metal ended up covered in a layer of patina.
The gate was first mentioned in 1328[17, p. 34] and was torn down sometime between 1817 and 1835 [17, p. 94].
5 Or possibly corbels; a corbel is set into the bigger structure, while a console is only superimposed on it. 6 Most probably a horizontal loophole. 7 Presumably created with the building on the verge of the 20th century. 2.3 Ferdinandova brána
The city erected Ferdinandova gate, pictured in Figure 2.3 [22], as a replacement for the Židovská gate in the years 1835–1836 [17, p. 97] to honour the new monarch. It was smaller and much simpler in appearance, and it did not stand in the exact spot where the Židovská gate once was. It was most probably moved inwards to make space for the train station. However, it could not keep up with the quickly growing city Fig. 2.3: Ferdinandova brána, 19th century [22] and met its fate in 1864 [17, p. 102].
2.4 Brněnská brána
While there are several drawings of this gate, they mainly show it from two angles, and 15 appear to show two different versions of the gate. Upon researching the timeline of the gate’s evolution, I found that these two versions seem to have existed at the same time [23]. The only viable explanation I could find is that they show the same gate not in the sense of the same building, but in the sense of the same passage into the city. Thus, one of the pictures probably shows the gate set into the “pre-walls”. Figure 2.4 pictures the (likely outer) Brněnská gate, which was modelled in this work. Figure 2.5 pictures the other gate, likely the inner one.
Fig. 2.4: Brněnská brána, 19th, from the archives of Fig. 2.5: Brněnská brána, 19th century, from the archives of NPÚÚ Brno NPÚ Brno
The gate was first mentioned in 1269[17, p. 34] and stood until 1848–1852 [17, p. 99]. 2.5 Veselá brána
I was only able to find two different drawings of the Veselá gate. The other ones were copies of these two versions, both showing it from outside the fortification. One pictures the gate from a distance, giving us a good idea of where it stood, while the other shows it relatively up close. It is portrayed with people and a carriage passing through it, allowing us to derive its approximate dimensions.
It was a simple gate, mostly inset into the walls, with a smaller passage for people and a bigger one for carriages. The two versions appear to be from around the same time — the gate looks almost the same in both. It was mostly used by the citizens living to the north of the city.
The pictures I found turned out to be ofNová Veselá brána — “The New Cheerful Gate”. In the Fig. 2.6: Nová Veselá brána, 17th century [22] 1670’, the original gate, which stood where the 16 streets Veselá and Česká join, was moved some way further from the city to allow for the construction of additional bastions. The Nová Veselá gate was built between the bastions, to the east of the old gate [17, pp. 67–68]. It is pictured in Figure 2.6 [22].
The gate was first mentioned in 1340[17, p. 34], the old gate was demolished in the 18th century, and the new gate between the years 1859 and 1860 [17, p. 102].
2.6 Běhounská
I have only managed to find two pictures of this gate, one of them shown in Figure 2.7. They are both quite detailed and only differ Fig. 2.7: in some minor elements, one of them being the cover of the roof. Běhounská/Dřevěná brána, from the archives We know the gate was rebuilt a few times, mainly when it became a of NPÚ Brno. part of the fortification of an Augustinian monastery that encompassed the Church of St. Thomas [17, p. 39]. The monastery fortification grew, changed and then disappeared; the church stands to this day.
In the past 20 years, archaeologists discovered the remains of the foundation of this gate. It stood very close to the current passage between the streets Běhounská and Rooseveltova. There were plans to mark the floor plan of the gate into the pavement in the spot where it once stood [24], but I could not find anything in the area.
The gate was first reliably mentioned in 1336[17, p. 34], although it was possibly built sooner, and was torn down sometime before the year 1817 [17, p. 95].
17 18 Creation of the 3D models of 3 the gates There are many modelling programs currently available. They vary in functionality, intuitiveness, and suitability for a given task. I considered using software such as Cinema 4D, Blender, Maya, Mudbox, 3Ds Max, or Sculptris. Sculptris, Maya, and Mudbox were not suitable for the primary modelling, because they are mostly used for sculpting — a more organic kind of modelling. Out of the other three, which focus on the more crisp and precise polygon modelling, I decided to use 3Ds Max, mostly because of my recent experience with it. Although I have used Cinema 4D in the past, I found that getting back into it after using a different software was cumbersome, considering the different user interface, controls, and some heuristics.
During the whole process, I worked iteratively and often simultaneously in the following software: I usedAdobe Illustrator for creating “blueprints”, 3Ds Max for most of the modelling, and Sculptris to create organic sculptures, such as faces on the stone consoles. For some objects, I used Mudbox for mesh checking, which would be important for correct texturing and UVW maps later. I used Blender for texturing and exporting into the final format used in the WebAR application. To create the textures themselves, I used either Adobe Illustrator or Krita,8 and if their final colours were not satisfactory, they were retouched in Adobe Lightroom. 20
All created textures and models can be found according to the Electronic Attachments of this work.
3.1 Creating blueprints in Adobe Illustrator
In order to capture the proportions of the gates and the various arches and shapes they possessed, I decided to create 2D blueprints of the gates in Adobe Illustrator. I imported a picture of the gate I was currently working on, lowered its opacity and locked the picture in place. Then I created vectors representing the different shapes in a separate layer, superimposed on the picture. To have vectors usable in 3Ds Max, I had to make sure they were “clean” — that they had no stray points and that the paths were all correctly closed.
8 Krita is a free and open-source digital painting program. Figure 3.1 illustrates the creation of blueprints from a reference, and Figure 3.2 shows the finished blueprints.
Fig. 3.1: The workflow of creating blueprints for models in Adobe Fig. 3.2: The finished Illustrator blueprints
Once the blueprints were ready, they were saved in the legacy format Adobe Illustrator 8 (AI). I could not use any newer version of the AI format, because 3Ds Max would not import the contents of this file as separate objects, which was crucial for the modelling process (as described in the next section). I could not find the reason why the newer 21 versions were not suitable for 3Ds Max, but saving the file in the legacy format solved this issue [25].
3.2 Modelling in 3Ds Max
3Ds Max imported the contents of the Adobe Illustrator 8 file as shapes, most of which were subsequently converted to editable splines. On the splines, I used either the Extrude modifier or theLathe modifier.
The Extrude modifier filled in the shape and added a third dimension to the so-far 2D object, creating a 3D object with two bases identical to the original 2D vector. The effects of this modfier are illustrated in Figures 3.3 and 3.4. Fig. 3.3: Spline without the Fig. 3.4: Spline with the Subsequently, I was able to use these 3D Extrude modifier Extrude modifier objects directly in the final model, without having to create them from scratch in 3Ds Max. I used this approach when creating more complicated objects, such as arches, shields, and various decorative elements.
However, the bases later needed to be manually subdivided into more polygons, as the vertices on their edges caused texturing issues. For subdividing, I used the Cut tool, creating edges from one side of the base to the other by connecting the loose vertices. Some additional retouching was done later in Blender, as described in Section 3.5.4.
Some parts of the gates, such as some rooftops or decorations, are solids of revolution.9 I created these objects by splitting the spline in half where the vertical axis would be positioned and applying the Lathe modifier. Figure 3.5 shows the half-spline without the Lathe modifier, while Figure 3.6 shows the spline with Fig. 3.5: Spline without the Fig. 3.6: Spline with the Lathe the modifier. Lathe modifier modifier 22
The shapes that I did not directly turn into 3D objects (such as simple rectangles or circles) were used as guidelines for the 3D objects (such as boxes, blocks, and spheres) that I created from scratch in 3Ds Max. Using the front, top, or side view, I was able to position the objects in such way that they overlapped the 2D guidelines and thus ended up in the correct spots, relative to the base of the gate.
Overall, I created ten models of the gates. Five models for the Měnínská gate, one for each of the distinct appearances it had over the centuries. One model was created for each of the following gates: Židovská, Ferdinandova, Brněnská, Dřevěná (Běhounská), and (Nová) Veselá.
9 Solids of revolution are three dimensional objects, which are created by rotating a 2D curve around an axis. 3.3 Sculpting in Sculptris
This software provides a simple and intuitive interface for organic modelling. It allowed me to drag, push, squeeze, or move “mass” as I needed. The Židovská gate features two statues and seven consoles with human heads. TheSymmetry setting allowed for a relatively quick creation of faces and bodies. The whole process was not unlike sculpting with clay in real life, which makes for a realistic model with imperfections. Creating an organic object would be a much more time-consuming process in a tool such as 3Ds Max, which is mostly intended for polygon modelling. Figure 3.7 shows the workflow of sculpting the face in Sculptris with the reference picture Fig. 3.7: Workflow of sculpting organic objects opened on the side.
23 When the sculpting was done, I exported the model to an OBJ10 file and opened it in Mudbox. Upon opening the model, Mudbox automatically checks the mesh for any mistakes or issues, such as high-valence vertices or degenerated faces. With these issues identified and highlighted in Mudbox, I could reopen the model in Sculptris and fix the mesh.
Overall, seven consoles with human faces and two full-body statues were sculpted for the Židovská gate. Figure 3.8 depicts the seven consoles.
Fig. 3.8: The seven consoles sculpted for the Židovská gate
10 The Wavefront OBJ file describes 3D geometry 3.4 Selecting the model format for WebAR
The A-Frame framework, which is responsible for handling 3D objects on the web, supports multiple widely-used file formats of 3D objects, such as OBJ,FBX , or glTF 2.0. After reviewing the available formats and their capabilities, the glTF format turned out to be the best suited for the job. According to the A-Frame documentation, the format offers everything OBJ does, as well as more robust materials, lighting, object hierarchy, and animation. Compared to the COLLADA or FBX formats, it is more suitable for web use, since it “focuses on providing a ‘transmission format’ rather than an editor format, it is more interoperable with web technologies.” [26]
Having correctly exported 3D objects was also part of the reason why I ultimately completed the models in Blender. 3Ds Max does not have a built-in option for exporting a model into the glTF format. Although a plug-in exporter called Babylon.js can be added to provide a few more exporting options, testing this exporter showed it did not work as reliably as was needed. Sometimes, the textures ended up being exported as completely black in colour. I could not find the settings that would correct this issue when exporting using Babylon, but in Blender, which can export to glTF without any additional plug-ins, it was easily 24 fixed by switching off the option to exportVertex colors.
3.5 Texturing in Blender
3.5.1 UVW Unwrapping
UVW mapping is the mapping of the surface of a 3D object to a 2D representation. The 3D surface has to be unwrapped to achieve this. The process is in principle similar to peeling an orange — the surface (the skin) is cut and removed. In the case of UVW unwrapping, a model’s surface is cut along the edges of the polygons, creating so-called seams, and then the individual pieces of the surfaces are laid out flat next to each other. The edges that are to become seams have to be chosen carefully — having seams in visible places where there are no sharp angles would cause undesirable artefacts in the texture. On the other hand, texture on insufficiently unwrapped objects would appear stretched and blurred. Once correctly unwrapped, the surface can be painted, or a texture can be laid over it. Upon rendering, each pixel of the surface from the UVW map carries information about what it should look like, and provides this information to the renderer to create the final textured model.
Unwrapping of UVW maps of the gates and applying textures was done in Blender. Compared to my attempts at unwrapping in 3Ds Max, Blender made the process overall smoother and easier. Part of the reason was that this software is much more optimised for less powerful computers. Given that some of the gates contained multiple intricate objects, 3Ds Max was considerably laggy and slow during the unwrapping process. Necessitating many steps for simple actions, the process of UVW unwrapping is also significantly more complicated in 3Ds Max than it is in Blender. The latter even has a Smart UV Project function, providing automatic UVW unwrapping. Although it may not yield great results when used on more complicated objects, it works reliably on simple cubes or blocks.
Before actually texturing, however, I had to find a balanced solution for two main problems — an artistic and a technical one. 25 3.5.2 Choosing colours and materials
The artistic problem consisted of deciding on the materials and colours to be used on the models. As there is very little of the gates left, and the Měnínská gate has been heavily altered during its numerous reconstructions, it is not possible to identify exactly what kind of building materials were used on each of the gates. The few pieces of the Jewish gate that are on display at the Spilberk museum are made of pale limestone, which came from a location on the outskirts of Brno, called Stránská skála. Like any other building, the gate could have been painted, although the restorers’ reports do not mention this. While it is still possible that the gate or its elements were painted over, the most reliable source of information about the colour of the gate is the painting by František Richter [18], which portrays the gate as pale yellow-grey. The images of the other gates provide very little information about their materials and colours. Although the gates could have been built from the same stone as the Židovská gate, they also could have been constructed from a different material. Ultimately, for both artistic and technical purposes (as described later in Section 3.5.3), I decided to create and use the same limestone texture for all of the gates’ bodies. Some of the roofs seem to be wooden or otherwise tiled, while others were most probably made of copper or bronze, because of their rounded and smooth surface (unlikely to be made from wood) and similarity to church roofs (decorative cylinders, spheres, spikes, etc.) These metals grow a layer of patina in a relatively short time, so for most of their existence, the metal roofs were probably teal and green. The roof of the Židovská gate’s tower was verifiably green, given it was also known as “The Green Gate” (see Section 2.2). Consequently, I decided to create and use two versions of the same texture, a lighter and a darker one. They both depict overlapping oblong shapes of warm brown tones, suitable for shingles or other kinds of roof tiles made from different materials.
3.5.3 Software and techniques
The technical problem consisted of selecting the software to use for texturing, and deciding on the level of detail of the textures which would be best suited for WebAR. After correctly unwrapping the model’s UVW maps, textures are applied as various texture maps. Each type of map provides different information about the object and how it interacts with light. Is it metallic or a dielectric? What colour is it? Is it smooth or rough? How much light does it reflect and absorb? How does it reflect and absorb it? How is light scattered on 26 the surface? Layering these maps (if they are correctly created) can lead to a very realistic looking final texture because these maps simulate characteristics and conditions of real‑life objects and materials interacting with light.
Physically based rendering (PBR) is an approach which uses the properties mentioned above in mathematical equations derived from optics when computing the final image. Consequently, understanding how light behaves is key in achieving realistic textures. A widely-used program which works based on PBR is Substance Painter. It provides complex mechanisms and heuristics for PBR and is thus used for creating highly realistic textures [27]. This solution would be perfect for creating detailed models.
I considered using Blender only for unwrapping the models and then importing them into Substance Painter, which would use the unwrapped UVW maps when applying textures. However, Substance Painter did not turn out to be a suitable solution for texturing the models intended for this WebAR application. Part of the reason was that there is simply not enough information about what the gates were built from. More importantly, a compromise had to be reached between how detailed the gates were and their overall data size. Realistic texturing requires many different maps and large amounts of data, but that interferes with a smooth WebAR experience and correct display of the gates. While a laptop was able to load multiple larger models, testing on a mobile device showed that it simply could not handle models whose total size exceeded a certain limit. Thus, I decided to forgo ultra-realistic textures and focused on the detailed structure of the models and achieving reliable and smooth functionality of the application. I created textures that could be shared amongst the different gates, like the same limestone, patina, wood or roof textures. This significantly improved the loading time of the models in the WebAR application compared to when the gates’ textures were not mostly universal.
The texture files were created either in Adobe Illustrator (for example, the roof tile texture), or in Krita (for example, the patina texture) and exported into JPEG files. Overall, the following textures were created: a limestone texture, a patina texture, a watch face texture, two limestone textures with inscriptions, a roof tile texture, a wood texture, and nine textures with pictures of shields for the Židovská gate. If necessary, some of the textures were retouched (for example lightened, darkened, or saturated) in Adobe Lightroom. 27 In Blender, the textures were assigned under Base Color —> Image Texture. In Texture Shading, some of the textures’ properties were adjusted to better correspond to the material they were supposed to convey. For example, the limestone texture’s Roughness attribute was increased to appear more rugged, while its Specular property was decreased to reflect less light.
3.5.4 Retouching the models for better texturing
When I started unwrapping the models in Blender, I quickly realised that even though the imported model seemed the same as it did in 3Ds Max, it did not retain the exact same geometry. This was especially true for objects that were created with the Extrude modifier and those that had any kind of Boolean subtraction applied. This caused a number of problems — for example, some vertices were duplicated and placed in the same spot, thus disconnecting edges in Blender that were originally connected in 3Ds Max. Figure 3.9 shows the seemingly connected edges, while Figure 3.10 shows the duplicate vertices moved to the side. The duplicate vertices had to be welded together before the object could be correctly unwrapped, which was possible in Blender using the Merge Vertex tool.
The Extrude modifier created many vertices that only had two or three edges leading out of them. This created Fig. 3.9: Seemingly Fig. 3.10: Duplicate vertices polygons that had sometimes even connected edges moved to the sides dozens of vertices along its edges, an issue which presented itself as incorrect shading in Blender, as shown in Figure 3.11. Sometimes this issue was relatively easily fixed by deleting the faces, selecting the leftover edges, and subsequently using the Fill function to create new faces. Blender automatically filled the empty space with multiple faces, not just a single polygon as it was before.
However, sometimes this approach did not work properly, and the issue needed to be fixed manually. In Blender, I used the Knife tool the same way as the Cut tool in 3Ds Max by cutting across the polygon and connecting vertices, as described in 28 Fig. 3.11: Shading issue caused Section 3.2. If this still did not fix the issue, I had to load the by incorrect topology model in 3Ds Max again and add edges connecting the vertices on the circumference of the faulty polygons with the Cut tool. Subsequently only the problematic object was exported to OBJ and then imported into Blender. In both of the programs, I always tried to connect vertices that were roughly opposite each other to preserve the model’s quad‑based topology.11
11 The quad-based topology approach means the 3D creator tries to keep the mesh divided into quadrangles (quads), which makes further subdivision, looping around the edges, and animation much easier and more reliable. Designing a city 4 history guide Although the goal of my work has always been to create a printable guide with historical information and markers, there were many decisions I had to make before I could start designing it. Would the guide hang on a wall? This option would mean the markers would always be in a vertical position, which affects displaying the 3D objects in AR. Would it lay on a flat surface? If so, can it be manipulated by the user? I even had the idea to create a folding book made of hard cardboard, with the pages shaped as the walls and gates of Brno. When laid out, it would be a simplified model of the actual fortification.
Ultimately, I decided to make the guide a foldable booklet that can be spread out and laid flat. I used a folding technique, commonly known as an8-page zine, which makes it possible to create a multi-paged brochure from a single sheet of paper. All pages of the booklet are printed on the same side of the paper, and the other side can be used for something else. Thus, the reader can read the booklet and then unfold it and look at whatever is printed over the other side of the paper. The zine I created was folded from an A3 paper with standard dimensions of 297 mm by 420 mm. When folded according to this technique, the booklet dimensions correspond to an A6 paper.
The pictograms, markers, Czech and English booklets, and map can all be found according 30 to the Electronic Attachments of this work.
4.1 Design of pictograms used as AR markers
When creating the pictograms, I had to be mindful of the restrictions the AR.js library puts on viable markers. These include marker-border ratio, marker symmetry, the level of detail, and the contrast of the colours used. The markers also had to be different enough so that the application would not mistake one for another.
The authors of the AR.js library created an online tool, called AR.jsMarker Training12, to help to create correct markers. To use it, one simply uploads a picture they wish to use as a marker, and then sets up the marker-border ratio and final image size. The tool creates
12 AR.js Marker Training is available at https://ar-js-org.github.io/AR.js/three.js/examples/marker- training/examples/generator.html a downloadable image — the marker itself. It also creates another crucial file — the pattern. A pattern file (.patt) describes the marker in all four orientations in Fig. 4.1: An AR.js Fig. 4.2: A Pattern file for the marker compatible marker a textual form, where numbers represent pixels. The library needs the pattern to recognise the marker when scanned by a camera. Figure 4.1 shows a marker with a 1:9 border ratio and Figure 4.2 illustrates a part of its corresponding pattern file. The shape of the pictogram is recognisable in the pattern file.
The design of the pictograms themselves consisted of making simple yet expressive abstractions of the actual gates. This process was done in Adobe Illustrator. For blueprints, I used front-view renders of the 3D models I have already created. Subsequently, I simplified the outlines of the blueprints and filled in the shapes with solid colour. Finally, I made any adjustments to the shapes that were necessary to make an aesthetically pleasing and consistent set 31 of pictograms. After testing the pictograms as markers, I made changes to some of them to make them less similar, since the application tended to confuse some of the markers. Figure 4.3 shows one of the pictograms in Adobe Illustrator, with the opacity Fig. 4.3: The creation of a marker based on a finished of the pictogram lowered so that the underlying reference render model of the gate is visible.
Overall, five different markers were created, with the intention of placing the models for the Židovská and Ferdinandova gates under the same marker in the webAR application. The reasons for this are their historical connection, as described in Section 2.2 and Section 2.3, as well as the technical difficulties placing two markers so close together would cause.
4.2 Design and layout of the guide
The design and layout of the booklet were done in Adobe InDesign, a tool used specifically for typesetting and page design. The dimensions of each page are those of a standard A6 paper, specifically, 105 mm by 148 mm, which fit the page of the booklet folded according to the technique described at the beginning of this chapter.
Deciding on the content of each page was straightforward. The booklet had to have a front cover, one page had to be reserved for instructions on how to use the web application, and five pages were needed for the five gates that were modelled in this work, with the Ferdinandova gate mentioned on the same page as the Židovská gate. That leaves one page for some general information about the gates and the fortification. More on the content can be found in Section 3.4.
Early in the design process, I knew I wanted to include some pictures of the models of the gates. However interactive 3D models might be, they can only be accessed after the booklet was at least briefly read. The gates had to be quickly presented not only in text but also visually to catch the reader’s attention. Thus, I used rendered pictures of the models with some warm atmospheric lighting and a plain background, and used these pictures as graphical elements in the booklet. Several layout options were created and considered before settling for the final version, with the gates playing a major visual role in the overall design. 32
The front and back covers of the booklet both share a picture of the Židovská gate. The front displays half of a rendered picture of the model with no background. The back bears the other half of the gate. However, instead of a rendered picture, the blueprint for modelling the gate in the form of a black and white line picture is used. The pictures of the two versions of the Fig. 4.4: A mockup illustrating the design of the gate meet at the spine of the booklet. The back booklet's cover cover also contains instructions on how to use the web application, and the QR code13 that redirects the user to the website upon being scanned with a smart device. Figure 4.4 shows a mockup of the booklet’s cover.
13 A QR code is a type of matrix barcode that codes the information it contains into a square made of many smaller ones attached to each other. The QR code used in this work was created by using an online QR code generator. The main part of the booklet consists of the inside pages that contain information about each of the gates. The first page, the preface, is free of any other graphical elements so that it differs from the following pages. Those were designed to give space to the pictures but to still carry enough information and be readable. Since most of the gates are Fig. 4.5: A mockup illustrating the inside pages of the booklet taller than they are wide, the pictures from the 3D models were also rendered as portrait images. This had to be accommodated by the layout of the text. Consequently, I decided to place the text on each page in a single narrow column side‑by‑side with the picture, as is illustrated in Figure 4.5.
4.3 Typography of the guide
The booklet uses two different fonts — a sans-serif font called Spartan in style Light for 33 the main title, the titles of the gates, and the title of the back cover. The rest of the text uses a serif font, Minion Pro, for easier reading. The Spartan font is available online in Google Fonts, and the Minion Pro font is included in the Adobe software package. The sans-serif font is thin, modern, and minimalistic, and contrasts with the historical topic of the brochure, while the serif font underlines it.
The titles and subtitles are left or right-aligned, depending on which side of the page they are located on. Although the columns of text by the pictures of the gates made for a better overall visual when they were fully justified, this alignment caused severerivers . A river is a typographical issue where the white spaces in lines align vertically. This issue is especially prominent in narrow columns of text, since there are often not enough words in each line and the spaces between them may be substantial. These rivers might be distracting to the reader, and for this reason, I chose to align all other bodies of text to the left.
The colour scheme is simple — the background is white all throughout the booklet, and all of the text is black, with some of the subtitles at lower opacities, so they appear grey. If some of the text was colourful, it could distract the viewer from the pictures and be potentially less readable. The grey colour, used on subtitles, helps to maintain information hierarchy. The subtitles are in the serif font, which is by default somewhat thicker even at a lower font-size than the sans-serif font used in the titles. Had both the titles and subtitles been black, the subtitles would be visually heavier and thus contradict the information hierarchy, which is undesirable.
The same typographic decisions were made for both the Czech and the English version of the booklet. The only difference is that the Czech version lacks the English name as a subtitle on the pages about the gates.
4.4 Contents of the guide
The information presented in the guide was put together so that it would be accurate and engaging. The guide is not intended to be heavily textual, and there was not much space for longer texts. Rather, it should serve as something to spark the viewer’s interest in the topic and let them have enough information to look for more on their own. 34 The front cover contains the title and subtitle of the booklet, while the back cover contains a QR code and a URL, both directing to the webpage, as well as instructions on how to use the webAR application. The first page of the booklet is a preface, containing basic information about the gates and the fortification. Each following page describes one of the gates, giving the reader some information such as the time of their existence, their alternative names, or pieces of interesting trivia. The title of each page is the name of the gate in Czech, and the English version of the booklet also has the English translation as the subtitle. The years under the titles represent the approximate time interval in which the gates existed.
4.5 Map design
The other side of the paper from which the booklet contains a simple map of the current heart of Brno. This map includes lines representing streets, some notable locations, and the city walls where they originally stood. The markers are placed where the gates once were, except for the Veselá gate. As mentioned in Section 2.5, the model and pictogram for this gate represent the newer gate, Nová Veselá gate, which stood to the east of the old one. However, the marker is placed where the original Stará Veselá gate once stood.
To create the map, I used Adobe Illustrator. I consulted the 3D models Archaia has created of Brno as it was in 1645 [15]. These models are freely available in the KMZ format, which contains locational data for the 3D objects. The files are easily imported into theGoogle Earth application, where the models are superimposed onto the map. I took a top-view screenshot of the model in Google Earth and a screenshot of the current map of Brno and superimposed them over each other in Adobe Illustrator. I lowered the opacity of the top map and used it as a reference for where to place the 35 walls over the current map of Brno. Fig. 4.6: The created map
To summarise, the map, shown in Figure 4.6, contains the following elements:
• the network of streets of Brno as it is today filled in with white, • labels for some of the bigger streets with a light purple background, • blocks of buildings filled in with a light red colour, • the main walls of the fortification where they once stood as a thick black line, • markers as pictograms of the gates in their respective places, • light brown stylised arrows indicating where the roads from the gates led, • squares and parks filled in with light grey and green colour, respectively, • pictograms of other significant landmarks, such as the main train station or historical churches, and their respective labels.
The pictograms of landmarks were created by outlining photographs of the buildings, or in the case of the main train station Hlavní nádraží, Park Moravské náměstí, and Špilberk, by outlining and filling their shapes from maps. 4.6 Assembling the final booklet
Once the design and layout of the booklet were prepared in Adobe InDesign, the content was moved to Adobe Illustrator. In the latter program, I was able to position and rotate each of the pages from Adobe InDesign onto an A3-sized artboard, as shown in Figure 4.7. Once the content was properly placed, the artboard was exported into a PDF file and printed. On the other of the paper, the map was printed. Lastly, following the instructions on assembling an 8-page zine14, the paper was cut and folded, creating the finished booklet.
36
Fig. 4.7: Layout of the 8-page zine
14 Instructions on how to assemble an 8-page zine can be found online, for example at https:// en.wikibooks.org/wiki/Zine_Making/Putting_pages_together under Single-page options —> An 8-sided zine from 1 sheet with 1 cut. Implementation of the webAR 5 application The AR functionality of my application is based on the JavaScript library AR.js, with underlying functionality from the A-Frame framework (see Section 1.2.1). TheHammer.js library was used to implement control over the 3D model.
5.1 Implementing a basic webAR application
After including the necessary libraries, inserting 3D objects into the scene is straightforward. The structure of the body of the webpage can be described with the scheme in Figure 5.1.
38
Fig. 5.1: Scheme illustrating the structure of the webpage
The scene’s properties set up the webpage for AR. Inside
For the full code, as well as a more detailed commentary, please see the fileindex.html as described in the Electronic Attachments. 5.2 Implementing the user interface
In order to keep the application as simple as possible, I focused on reducing the number of elements on the screen to a minimum, so that they would not interfere with the user experience. However, seeing as the Měnínská gate was rebuilt numerous times, and I had drawings of the different versions available, I decided to create and include all the different versions of the gate under one marker. To achieve this, I implemented five buttons, one for each of the versions. The user can thus scan the marker for the Měnínská gate, have the latest version pop up by default, and then simply change the model to a different version by tapping one of the buttons.
Similarly, I created two buttons for the Židovská and Ferdinandova gates and thus stored them under the same marker, as described in Section 4.1.
5.3 Implementing controls
39 One of my main concerns when designing the controls was to make them as intuitive and straightforward as possible. Ideally, I wanted the user to instinctively know how to control the 3D model without any instructions. To achieve this, I implemented two of the functionalities that I considered to be the most important — rotate and scale — with the two mechanisms most used in virtual object manipulation: pinch to scale and swipe to rotate.
When implementing and testing the application, I ran into a bug which did not allow me to scale the model down, thus leaving me with a giant model outside the viewport and without any way to regain control over it. Whenever this happened, and I wanted to test something else (rotation, for example), I had to reload the page. To remedy this, I implemented another functionality, which I decided to keep in the application — use double-tap to reset the model to its original scale and rotation.
The last way of controlling the object is by pressing on the screen. A long press triggers the rotation by 90 degrees in the X-axis, laying it down horizontally. A subsequent long-press stands the model up again. This functionality was implemented to give the user a better view of the gates, especially if the markers are scanned directly from above. 5.4 Workflow of testing
The AR.js library makes it very easy to implement a basic AR scene in just a few lines of code. However, some parameters need to be set up quite precisely right from the start. For example, the marker is subject to some particular conditions, such as correct dimensions, as described in Section 4.1. Thus, I had to start testing the application early on. Testing can be done by running the web application directly on the computer and using the computer’s webcam to scan for markers.
However, there were shortcomings to only testing like this, one of them being the low quality of a laptop webcam. Another reason was that while it is not impossible to use this application on a computer, it is certainly not preferable to a portable smart device. On the other hand, testing on a smartphone would only provide the information that some feature is or is not working, not why it is so. Thus, I decided to test in two ways simultaneously.
The first one was to run the application on the computer, where I would then use the browser’s Inspect webpage function. Google Chrome’s webpage inspector allowed me to query the code in the console, see whether sources for the models were loaded correctly, get error 40 messages, and more. The A-Frame framework also includes a very useful scene inspector, A-Frame Inspector, shown in Figure 5.2, which is accessible from the web application in the browser by pressing the key combination Ctrl + Alt + I. In case the uploaded model was too scaled up to be visible in the viewport, or its position was incorrect, I was able to zoom out in the scene inspector, and check whether the model was even present and what might be the problem with it. To test the controls which are typically triggered by touch events on smart devices, I included the Touch Emulator debug tool by Hammer. It emulates touch events in the browser of a computer, such as pressing and moving the mouse to simulate swiping on the screen of a smart device. Fig. 5.2: The A-Frame Inspector in use
The second method of testing was on a smartphone, which required creating a tunnel from the local server host. I used the tunnelling service NGRok to create a unique HTTP port from the localhost, which I then opened in the internet browser of my phone. However, the HTTP link had to be changed to HTTPS due to browser restrictions. As stated in the official AR.js documentation,Accessing “ to [sic] the phone camera or to [sic] camera GPS sensors, due to major browsers restrictions, can be done only under https websites.” [14] Once the application was running on my phone, I could then scan the marker (whether printed or displayed on my laptop), and test any functionality I was currently working on.
Testing on the computer allowed me to find technical issues, bugs, or errors in my code. Meanwhile, testing on the phone allowed me to find user experience issues, such as touch response, recognisability of the markers under different angles and light conditions, and the overall “feel” of the application for the user.
5.5 Encountered problems, issues and bugs
Inserting 3D objects into the camera scene was mostly straightforward. However, I ran into a few problems caused by the lack of a detailed documentation of the current version of the 41 AR.js library. Even more problematic were the discrepancies between the recommended solutions and the current versions of the AR.js library and the A-Frame framework. During the implementation of this application, the AR.js library was upgraded from version 2.x to 3.x. The new version included a several new features (such asNFT 15) and bug fixes, and it also somewhat improved the documentation, which helped to solve a few issues. However, many issues encountered during implementation were solved thanks to information from other users of the library. Their insights about their problems and solutions in the Issues sections of both the old and the new GitHub repository of the library were very helpful.
When implementing the controls using the library Hammer.js, I encountered even more issues with finding the correct solutions. The library has not been updated for the past
15 Natural Feature Tracking four years, and its documentation is lacking in both volume and precision. In the end, I was able to write code that allows one to rotate, scale, and reset the model, as described in Section 5.3.
One of the main issues with the performance of the application was the shakiness of the models when viewed on a mobile device. While the augmented objects were relatively stable when viewed on a computer, they were very jittery on phones. After some research, I found the binary parameter smooth (disabled by default) of the
Another encountered issue was the flashing, jittering, or 42 incorrect overlapping of some textures. The solution was to define the scene renderer system in
A slight issue comes up when testing with the markers displayed on a computer screen. Since switching between the markers Fig. 5.4: The model with the takes very little time, the event handler that handles losing the logarithmic depth buffer marker from view does not have time to fire. Since it does not fire, the event handler that handles finding a new marker does not fire; rather, it acts like the old marker is still present. Thus, it recognises that a marker is in the view of the camera, but it does not recognise that it is already a different one. Consequently, the model belonging to the previous marker is displayed over the new one. However, in the typical use case of having the markers printed on paper, the camera has to be moved to locate the next marker. Thus, the event handlers have time to fire properly, first disabling the old model and then displaying the new one.
5.6 Deployment and use of the web application
To publish the app, I used Surge,16 a service which offers free static web publishing for front-end developers right from the command line. This way, the web application can be accessed by anyone with the URL. As previously mentioned in Section 5.4, only HTTPS websites are supported. Thus, when deploying the application using Surge, I forced the page to redirect to HTTPS, as shown in Figure 5.5.
43
Fig. 5.5: Deploying the web application to an HTTPS site As mentioned in Section 4.4, the back cover of the guide contains a QR code which, when scanned, directs the user to the website. Many modern smart devices have a built-in QR reader in their camera app, and those who do not can use a QR scanner app or website of their choice. Alternatively, they can visit the website by typing its URL, which is also written under the QR code.
Upon accessing the website, a dialogue window opens, asking the user for permission to use the device’s camera and orientation and motion sensors. The app successfully launches once the user gives these permissions. The landing screen is simply the view
16 Available at https://surge.sh from the camera. To access the AR content, the user scans any of the viable markers, which triggers the implemented functionality and displays the 3D model of the chosen gate. Depending on the gate, it also shows the buttons that allow the user to display the different versions of the gate.
The application is published to the page https://brno-AR-gates.surge.sh.
44 Conclusion
This work aimed to design a printed city guide, create 3D models of historical buildings, and implement an interactive webAR application connected to the guide. The buildings chosen to be reconstructed as 3D models were Brno’s historical gates, whose likenesses were researched and derived from historical sources. The guide was created in both Czech and English, and thus can be used by international and local tourists, as well as by citizens of Brno who want to know more about the history of their city. WebAR proved to be a suitable solution for displaying 3D models in augmented reality, and the selected approach of using custom markers content incorporated into the guide as triggers for the AR was successfully implemented. The web application displays the models without shakiness and fast enough to ensure a smooth viewing experience.
The web application can be used along with the guide to allow the viewer to inspect the gates more interactively and in greater detail. For instance, the user can take a walk around Brno, follow the map printed on the back of the guide, and use the application everytime they visit one of the spots where a city gate stood. Since the application’s functionality is 45 only tied to the markers, it can be used anywhere where the user has internet connection and the markers available, whether in a digital or printed form.
Some of the issues encountered when designing and developing this application were technical, such as the selected library’s limitations in terms of access to the device’s full computational potential. Furthermore, a compromise also had to be made between the level of detail of the models and ensuring stability of the AR experience, on account of the limited amount of data that can be stored in the cache of the web application. From an artistic point of view, the design of the printed guide, especially of the map, was influenced by the need for reliable and responsive markers. The requirements placed on viable markers were reflected on the features the pictograms could and could not have. Consequently, this also affected the design and layout of the map.
In the future, the possibilities of optimising the 3D models can be explored to allow for greater quality and quantity of the augmented content. The other types of AR, location‑based and markerless AR, can also be tested as possible ways to enrich city tourism and historical landmarks. 46 Bibliography
Kurniawan, Michael, Suharjito, Suharjito, Diana, & Witjaksono, Gunawan. [1] Human Anatomy Learning Systems Using Augmented Reality on Mobile Application [Online]. Procedia Computer Science, vol. 135, pp. 80–88, 2018.
Marto, Anabela. Testing SensiMAR in Conimbriga [Online]. anabelamarto.github.io, [2] 19-Oct-2019. Available at: https://anabelamarto.github.io/post/sensimar-02/. [Accessed: 01-May-2020].
Emara, Ahmed. Augmented Reality in Historical Sites (Alexandria, Egypt) [Online]. [3] YouTube Video, 22-May-2019. Available at: https://www.youtube.com/watch?v=- L8Rk7sn3h0. [Accessed: 01-May-2020].
[4] Augtraveler – AR Based Multimedia App For Heritage Tourism in India [Online]. Augtraveler. Available at: https://www.augtraveler.com/. [Accessed: 01-May-2020]. 47 Caldas, Luisa. BAMPFA AR—Augmented Time [Online]. BAMPFA, 2019. [5] Available at: https://bampfa.org/program/bampfa-ar-augmented-time. [Accessed: 01-May-2020].
Animal Kingdom – Augmented Reality Development | Deform Studios [Online]. [6] Deform Studios. Available at: https://d3form.com/animalkingdom/. [Accessed: 10-May-2020].
ARLOOPA. Augmented Reality Physics Book by ARLOOPA [Online]. ARLOOPA [7] Augmented / Virtual Reality, 06-Jul-2016. Available at: https://www.youtube.com/ watch?v=Qw7HJPol8ZQ. [Accessed: 10-May-2020].
Mendes, Serafim.POST–PRINT — AR & 3D in Graphic Design [Online]. [8] Behance, May-2019. Available at: https://www.behance.net/gallery/79737469/ POSTPRINT-AR-3D-in-Graphic-Design. [Accessed: 01-May-2020]. Etienne, Alexandra. Area Learning with Multi-Markers in AR.js: For a Larger [9] & More Stable Augmented Reality [Online]. Medium, 13-Jun-2017. Available at: https://medium.com/arjs/area-learning-with-multi-markers-in-ar-js- 1ff03a2f9fbe. [Accessed: 1-May-2020].
Etienne, Jerome. Measure distance between two markers [Online]. [10] Twitter, 10-Apr-2017.Available at: https://twitter.com/jerome_etienne/ status/851429567814406144?lang=en. [Accessed: 1-May-2020].
Schechter, Sonia. What is markerless Augmented Reality? | AR Bites [Online]. Marxent 3D Commerce, 9-May-2014, updated: 12-April-2019. [11] Available at: https://www.marxentlabs.com/what-is-markerless-augmented- reality-dead-reckoning/. [Accessed: 1-May-2020].
Google Cardboard [Online]. Website. 48 [12] Available at: https://arvr.google.com/cardboard/. [Accessed: 1-May-2020].
Apple. Augmented Reality - Apple Developer [Online]. Apple Developer, 2019. [13] Available at: https://developer.apple.com/augmented-reality/. [Accessed: 01-May-2020].
Carpignoli, Nicolò & Etienne, Jerome. AR.js Documentation [Online]. AR.js - [14] Augmented Reality on the Web, 2020. Available at: https://ar-js-org.github.io/AR.js-Docs/. [Accessed: 05-May-2020].
Model Brna roku 1645 [Online]. Website. [15] Available at: http://www.brno1645.cz/index.htm. [Accessed: 01-May-2020].
Archaia Brno [Online]. Website. [16] Available: https://www.archaiabrno.org/home_cs/. [Accessed: 01-May-2020]. [17] Kuča, Karel. Brno - vývoj města, předměstí a připojených vesnic. Baset, 2000.
Baumannová, Dagmar. The Měnín Gate – A Part of Brno’s History [Online]. [18] Brno City Museum. Available at: https://www.spilberk.cz/en/brno-city-museum/ the-menin-gate/. [Accessed: 01-May-2020].
Vykoupil, Libor. Měnínská brána [Online]. TIC BRNO, příspěvková organizace. [19] Available at: https://ticbrno.cz/kam-v-brne/brnenske-pruseciky/meninska-brana. [Accessed: 01-Apr-2020].
František (Franz) Richter. Židovská brána v Brně, 1820 [Online]. [20] Moravská Galerie – Sbírka on-line. Available at: http://sbirky.moravska-galerie.cz/ dielo/CZE:MG.B_10125. [Accessed: 01-May-2020].
Steif, Maximilian. O brněnské Židovské bráně. 20-April-2020. Lidové noviny, p. 3., [21] Brno: Vydavatelské družstvo Lidové strany v Brně, 1893–1945.
49 Brněnské městské brány [Online]. Brunn.cz, 09-Mar-2012. [22] Available at: http://www.brunn.cz/brnenske-mestske-brany/ [Accessed: 01-Apr-2020].
Kolařík, Václav. Opevnění města Brna do konce třicetileté války na základě [23] archeologických výzkumů. Master’s Thesis, Masaryk University, Faculty of Arts, 2007. [Accessed: 01-April-2020].
Taušová, Zuzana. Hradby v Běhounské připomene dlažba [Online]. Brněnský [24] deník, 19-Jun-2010. Available at: https://brnensky.denik.cz/zpravy_region/ hradby-v-behounske-pripomene-dlazba20100618.html. [Accessed: 01-05-2020].
Autodesk Support. Cannot import Adobe Illustrator 9 through CC-version .AI files into 3ds Max [Online]. Autodesk Knowledge Network, 12-Oct-2018. [25] Available at: https://knowledge.autodesk.com/support/3ds-max/learn-explore/ caas/sfdcarticles/sfdcarticles/How-to-import-Adobe-Illustrator-files-into-3ds- Max-s.html. [Accessed: 20-Feb-2020]. gltf-model [Online]. A-Frame. Available at: https://aframe.io/docs/1.0.0/ [26] components/gltf-model.html. [Accessed: 01-May-2020].
McDermott, Wes. The PBR Guide. 3rd ed. Allegorithmic, 2018. Available at: https:// [27] academy.substance3d.com/courses/pbrguides. [Accessed: 01-April-2020].
Asset Management System [Online]. A-Frame. Available at: https://aframe.io/ [28] docs/1.0.0/core/asset-management-system.html. [Accessed: 01-May-2020].
renderer – A-Frame [Online]. A-Frame. Available at: https://aframe.io/docs/1.0.0/ [29] components/renderer.html. [Accessed: 01-May-2020].
50 Electronic attachments
The thesis archive of the digital copy of this work in the Masaryk University Information System contains electronic attachments of this work. The three archives and their contents are the following:
• the archive Brno AR Gates, containing all the sources necessary for running the web application, e.g. the 3D objects, the pattern files, and the HTML and JS code files, as well as a README file with instructions on how to run the application, • the archive Textures and objects, containing all of the created textures as JPEG files and 3D models as glTF + BIN files, • the archive Graphic design, containing the markers, all the files created for the design and typography of the booklet, such as pictures, as well as the finished Czech and English booklets and zines and the map as PDF files.
The web application can be run by visiting the site at https://brno-AR-gates.surge.sh on a device 51 of your choice, or on a computer equipped with a web camera directly from the source code by following these instructions:
1. Ensure you have Node.js installed 2. Unzip the Brno AR Gates archive and in command line navigate into its folder 3. Run > npm install > node brnoARgates.js 4. In a browser of your choice, go to the address localhost:8080