An Affect-Based Built Environment Video Analytics T ⁎ A
Total Page:16
File Type:pdf, Size:1020Kb
Automation in Construction 106 (2019) 102888 Contents lists available at ScienceDirect Automation in Construction journal homepage: www.elsevier.com/locate/autcon An Affect-Based Built Environment Video Analytics T ⁎ A. Kaklauskasa, , E.K. Zavadskasa, D. Bardauskieneb, J. Cerkauskasa, I. Ubartea, M. Seniuta, G. Dzemydac, M. Kaklauskaitea, I. Vinogradovaa, A. Velykorusovaa a Vilnius Gediminas Technical University, Vilnius, Lithuania b Vilnius City Municipality, Vilnius, Lithuania c Vilnius University, Vilnius, Lithuania ARTICLE INFO ABSTRACT Keywords: The research described in this article integrated objective and subjective human analyses of the built environ- Built environment ment. These were conducted from two perspectives: that of an individual and that of the built environment. The Research design development of the research design over the course of this study involved 11 phases. A research design using an Integrated BEST method integrated method with intelligent and biometrical technologies served as the basis for developing the Affect- Maintenance and management Based Built Environment Video Analytics (BEST), the application used in this research. BEST supplies stake- Neuro decision tables holders with the required data on the built environment and can assist them in analyzing and making decisions. Intelligent decision support systems Recommender systems The authors of this article attempt to integrate human emotions into the context of the maintenance and management of the built environment. One of the key aims is the indirect participation of inhabitants in maintenance and management procedures relating to the built environment. Consequently, BEST gathers and analyzes data on human affective attitudes and emotional and physiological states in the built environment and delivers these to city planners and stakeholders in maintenance and management. This is performed by applying neuro decision tables and an inhabitant-centered method for mining human emotion data from the built en- vironment under analysis. Finally, this article illustrates the practical capabilities of BEST by employing case studies within Vilnius City as examples. The application of BEST proves that this system is valuable in conducting successful research and practical activities. 1. Introduction humans and their physical environment is a vital constituent of city planning. More than half of the world's inhabitants now reside in towns, Efforts to assess the effect of a city on human emotions, frombotha making the development of healthy city environments a main policy subjective and objective point of view, have involved the development priority [1]. In the opinion of Shoval et al. [2], individual emotional and application of various methods and tools over recent decades. connections with a city's environment have been of vital importance to Analyses of human emotions have been undertaken by an entire array city researchers for decades. A city's physical, social and cultural as- of researchers, and subjective studies include those by Kim et al. [7], pects determine a person's emotional involvement with a city en- Solymosi et al. [8], Resch et al. [9] and Birenboim et al. [10] while vironment. Various research studies [3,4] propose that a city's en- objective studies have been conducted by Sagl et al. [11], Birenboim vironment (its architecture, land use mix, public squares, parks, built [10], Birenboim et al. [12], Zeile et al. [13] and integrated studies by features, polluted areas, rubbish, the quality of the built environment, Birenboim et al. [12,14] and Shoval et al. [15]. manufacturing zones and traffic flow) plays an important role inhuman Subjective studies of human emotions/experiences in an urban emotions, moods and mental health, in conjunction with individual and context have been performed by applying qualitative methods such as social factors. Gong et al. [4] hold the opinion that a city's environment self-reports, interviews, observations, questionnaires and diaries, in affects people on an individual/personal level (based on individual order to register the subjective experiences of individuals in their usual perceptions) and due to the effects of public spaces. Rapoport [5] de- settings. For example, self-reports might encompass methods such as fines the built environment as a form of “nonverbal communication”, the Ecological Momentary Assessment, the and finds that individuals use tools to interpret its meanings. Carmona Pleasure–Arousal–Dominance scale and the Experience Sampling [6] has emphasized that the perception of an association between Method. Russell's circumplex model of emotion [16] can aid in the ⁎ Corresponding author. E-mail address: [email protected] (A. Kaklauskas). https://doi.org/10.1016/j.autcon.2019.102888 Received 12 March 2018; Received in revised form 16 June 2019; Accepted 20 June 2019 Available online 28 June 2019 0926-5805/ © 2019 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/BY-NC-ND/4.0/). A. Kaklauskas, et al. Automation in Construction 106 (2019) 102888 evaluation of a self-report (as per the Pleasure–Arousal–Dominance measurements of the integrated objective and subjective dimensions of scale), and has been widely used in such studies [7]. Loiterton and emotions at two levels (individual perception, and the perception of the Bishop [17] of the Royal Botanic Garden in Melbourne sought to ex- city and built environment). Individual affective, emotional and phy- amine subjective feelings such as boredom, fatigue and hunger, and siological tests [34,35] have also been conducted previously. The au- developed questionnaires focusing on public spaces, applying these thors of this paper have developed an Affect-Based Built-Environment data to make forecasts about people walking in the garden. Video System (BEST) based on the aforementioned literature sources Objective studies of human emotions and/or experiences in a built and their long-term personal experience in this field. Within the fra- environment have involved the application of biometric methods and mework of the H2020 ROCK (Regeneration and Optimization of Cul- systems (blood pressure, body temperature, heart rate, skin con- tural Heritage in Creative and Knowledge Cities) project, the BEST ductance, pupil size, blinking and others). A number of researchers method and system were developed. As part of the ROCK project, stu- have looked at ways to make ambulatory physiological sensing part of dies are being carried out with a broader scope from the perspective of urban planning [10,13,14,18]. In their efforts to integrate ambulatory public spaces in comparing with previous similar studies [7–15,17–25]. sensing within urban planning, Nold [19], Sagl et al. [11], Zeile et al. Significant contact-based biometric investigations are being con- [18] and Resch et al. [20] have relied on fundamental theoretical and ducted in built environments, and have also been carried out across the methodological approaches. Advanced technologies and tools now offer world [8–20]. The term ‘contact-based investigations’ means that sen- more effective ways to focus on residents [18]. Data related to emotions sors have direct contact with the person under analysis. The analysis of can also offer a different way to validate the monitoring of public spaces these important investigations [8–20] involves comparing a small [20]. Many researchers use wearable electronics (smartwatches, Google amount of data according to the number of metrics. Glass, computerized contact lenses, wristband sensors, intelligent tex- Over the course of the ROCK project, BEST (an Affect-Based Built tiles, wearable sensor patches) to conduct unbiased affective, emotional Environment Video Analytics ) has accumulated over 200 million and physiological studies. Many of these devices were first imagined in anonymized data points by remote means, and this number continues to science fiction and mainstream films. grow. Several synonyms are used in the literature for contact-free When we know how individuals respond to environments via their biometric measurements, such as contactless, stand-off, distance or non- emotions, we can begin to understand how people engage and interact contact biometric measurements. BEST expands the important in- with spaces [21]. Based on this, designers, planners and managers of vestigations of the built environment described above [8–20] by ana- public spaces can measure people's responses to various different sti- lyzing the large number of variables for a large number of data col- muli [22] within the built environment [23]. The combined effects of lected by remote means. various environmental stimuli produced through interaction, and their This study is somewhat similar to the research mentioned pre- power to induce the desired emotions and behaviors, need further study viously; however, it introduces three innovative elements, which are before they can be fully understood [24]. Hence, there is a need to described and summarized in more depth in the conclusion. study the emotional reactions of human beings to the everyday stimuli The structure of this article is as follows. Chapter 2 describes the that can affect the senses of people living within and using different design of the BEST research design. Next, Chapter 3 explains the spaces [23]. components of BEST. Chapters 4 and 5 present case studies and a de- Human emotions have also been studied using an integrated