View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Publications at Bielefeld University SoZen – Improving Productivity with a Soundscape Generating Zen Garden Jiajun Yang Thomas Hermann Ambient Intelligence Group, CITEC Ambient Intelligence Group, CITEC Bielefeld University Bielefeld University Bielefeld, Germany Bielefeld, Germany [email protected] [email protected] ABSTRACT This paper presents the SoZen system, an interactive dec- orative artifact that can control features of ambient sound- scapes. The system aims at improving the sonic quality of work/living space. For that, features such as the placement of stones and patterns in the sand are extracted via a web- cam and in turn control directly aspects of the sound play- back/synthesis engine. We evaluated SoZen in a within- subject study to understand the benefit of interactive am- bient soundscapes as participants worked in a sonically simulated office environment. Participants performed sig- nificantly better in terms of error rate in a spreadsheet- inputting task under the ambient soundscape condition com- (a) Overview. (b) Top-down view. pared to the baseline condition, and evaluated the system positively in a subsequent survey and interview. However, Figure 1: Photos of the SoZen. no significant difference in their psychological states be- tween conditions are found based on the PANAS measure, apart from the Excited affect. SoZen serves as an example ringing in a workplace [4]. The sound masking technology for how existing decoration artifacts can be transformed may involve the use of static noise generators [5,6], adap- into multimodal (visual, auditory and tangible) user in- tive sound masking with sound level sensors [7] and fur- terfaces with positive side effects on their inhabitants in niture integrated platform [8]. Although such an approach a home or office environment. provides good results in reducing distraction, it is less aes- thetically pleasant and thus less likely to attract home users 1. INTRODUCTION for instance. Ambient soundscapes, on the other hand, offer a prefer- Urban living and work environments such as open-plan of- able third alternative as they can be less obtrusive than mu- fices are often troubled by noise pollution. Higher level sic and more attractive/pleasant than white noise. There of ambient noise may increase psychosocial job stress [1], are many benefits of listening to natural soundscapes, as blood pressure and heart rate [2]. This motivates com- they can support calmness, relaxation and the emotional panies to seek and researchers to invent new solutions to state [9]. In a recent study, Sayin et. al. found that ambi- improve the sonic environment of workplace in order to ent soundscapes can also improve the perceived safeness provide a healthier experience and to increase productiv- of open spaces [10]. ity. There are many technologies available such as acoustic We believe that the users’ acceptance will be even better panels, noise cancellation, and sound masking, onto which if they can influence features of the ambience. Thus we we focus here. propose a system-guided yet user-driven ambient sound- A common sound masking approach is to simply board- scape artifact for providing calming acoustics content to cast music. While music could mask the surrounding to improve the sonic environment of a noisy office. The sound- some extent and also increases cognitive arousal and pos- scape should contain a backgrounding element that does itive mood [3], music itself can attract attention from the not overtake one’s main attention in most of the daily tasks. main task. This problem is circumvented by using a spec- Concerning the user interface, the interaction should be trally flat signal such as white noise or pink noise to over- effortless to use, without requiring the user to give com- lay environmental noises such as conversations or phone mands such as selecting a song from a playlist or using keyboard or mouse. Copyright: c 2016 Jiajun Yang et al. This is an Based on these criteria, we designed a cross-modal plat- open-access article distributed under the terms of the form combining ambient soundscape control with a home Creative Commons Attribution License 3.0 Unported, which per- decorative artifact. By coupling soundscape generation and mits unrestricted use, distribution, and reproduction in any medium, artifact, it can serve as a control interface but still remains provided the original author and source are credited. visually blended into the environment as a decoration. This concept is in line with Mark Weiser’s definition on ubiqui- and sound engine. At last, the interaction and mapping de- tous and calm computing, which emphasizes the nonintru- sign are presented. sive computing and integration with the physical environ- ment [11]. 3.1 System Architecture The cross-modal interaction we proposed in this project is a vision-to-sound transformation with tangible interaction. The physical structure of SoZen (cf. Fig.1) is very simple: Vision-based sonic interaction and tangible controlled sonic the sand plate is situated on a wooden base with bamboo arts have been widely explored in the interactive music support frames, which are used for holding a webcam or and new musical expression communities such as ICMC, optionally a curtain to assume controlled lighting condi- SMC, NIME etc. Leichsenring et. al. introduced an ambi- tion if, for example, the installation is placed near a win- ent soundscape interface using tangible objects to create a dow. The bamboo as material was selected because it fits decorative interface for the use of multimodal ambiences to the naturalness of the artifact. The hardware structure in the smart bathroom context [12]. [13] presented an in- and software ambient soundscape generation is bridged by teractive music composition platform via paper drawing. using the webcam to capture the image of the arrangement Jo and Nagano developed Monalisa, a cross-modal inter- of the sand and stones. face to transform between image and sound in both direc- Concerning the software, the overall structure of the sig- tions [14]. GrainPlane is a tangible instrument that uses nal flow is depicted in Fig.2. The captured image is pro- actual food grains (e.g. beans and rice) to control parame- cessed in order to extract key features of the arrangement, ters of a granular synthesizer and thus generate electronic which are subsequently sent to the sound engine via Open soundscapes [15]. A famous tangible interface for musi- Sound Control (OSC) as control parameters for rendering cal control is reacTable, a tabletop screen surface with tan- the sound output. Details of the image feature extraction gible objects (cubes) for controlling different sonic mod- and sound generation are presented below. ules [16]. Music can be created by placing and thereby combining cubes representing synthesis modules. 2. CONCEPT OF SOZEN Three key assumptions underlie the project: • A noisy soundscape can be refined by ambient sound- Figure 2: Signal flow of the system. scape in a home or office environment, either to im- prove the user’s perception to the sonic quality of the environment, or to support certain functions such as relaxation, vitalization, cheering up. 3.2 Image Feature Extraction The image feature extraction is written in Python using the • The resulting soundscape should merge into the back- OpenCV library [18]. ground and should not add to the users’ cognitive load so that they stay focussed on any given task. Regarding the stones, there are many details possible to extract, from the stones’ (or rocks’) size, color, texture, • Cross-modal audiovisual interaction can be explored shape, position, orientation in space, etc, to the many de- to create interesting smart artifacts for home and of- tails of how sand is distributed and shaped with the pro- fice applications providing both visual and audio qual- vided tool, a miniature wooden fork. For practical reasons, ities. we decided to reduce the complexity by selecting first the most elementary features, leaving the inclusion of more As decorative artifact we chose a miniature Zen Garden. details to future revisions of the system. In first order, A Zen Garden, originated in Japan, is a visual recreation the setup is just a spatial arrangement of stones which are of mountains and rivers through the special arrangement only characterized by their overall size and location. The of stones and sands. In Japanese Buddhism, Zen Garden rocks’ detection is achieved using a blob detection algo- has various meanings and metaphors, in particular it cre- rithm, which returns the list of rocks including their size 1 ates a sense of calmness, which can be beneficial for med- and coordinates. itation [17]. A miniature Zen Garden is a small sand plate In the previous version of SoZen [19], we used white accompanied with a number of small stones. The user can and black stones for controlling different sound elements. arrange the stones and carve the sand to create a visual pat- However, the white stones were prompt to reflection of tern of the plate. light, leading to inaccurate detection. For that reason we The following section presents the design details of SoZen. decided to solely use black stones. In Section4, a study is presented to investigate how SoZen As for the sand features, since we are only interested in affects productivity and psychological states in a work en- the features from the parts which are carved by the wooden vironment, leading to discussion and summary. fork, Canny Edge detection is applied to only preserve the sand’s image with stronger contrast (carved). Initially we 3. DESIGN OF THE SOZEN SYSTEM applied line and curve detections to detect the lines’ shape, however, when the sand is carved, the contrast between the This section begins with an overview of the system archi- sunken and flatten part is very small.