Construction of an Electronic Timbre

into any system that supports a C language interface. CONSTRUCTION OF AN ELECTRONIC TIMBRE Acknowledgements: Josue´ Moreno’s work has been DICTIONARY IN C++ FOR MULTIPLE PLATFORMS supported by the Sibelius Academy Development Centre.

6. REFERENCES Naotoshi Osaka Hiromichi Tsuda Yoshitada Yamada

[1] C. Goudeseune, “Interpolated mappings for mu- Tokyo Denki University sical instruments,” in Organised Sound, 2002, p. School of Science and Technology for Future life 7(2):8596. Tokyo, Japan [email protected], (tsuda,[email protected]) [2] F. B. R. Muller and N. Schnell., “MnM: a Max/MSP mapping toolbox.” in Proceedings of the Confer- ABSTRACT developed in different languages by different developers ence on New Interfaces for Musical Expression Figure 3. Mapping three disparate parameters to a Spatial the – sound browser mode is written in Adobe Flex and (NIME05), Vancouver, Canada, 2005. Panner We have been building an electronic timbre dictionary as a the 3D timbre display function is written in Adobe Flash, [3] V. Norilo, “Introducing Kronos - A Novel Approach support tool for sound creators, which provides them with while the coordinate mapping of sounds are written in sound materials from environmental sound database. Since in their output signal, describing the range of the signal. In to Signal Processing Languages,” in Proceedings C++. the previous system sub-modules were written independently Resultantly, each module behaves rather independently addition, the Envelope-Follower module receives two time of the Linux Audio Conference, F. Neumann and in multiple languages, we restructured and integrated the and only the intrinsic function in each module has been constants annotated as milliseconds. V. Lazzarini, Eds. Maynooth, Ireland: NUIM, subsystems in C++ to improve the processing speed and Algorithm:Map is used to transform all three param- 2011, pp. 9–16. usability. In order to verify these improvements, we ran tested, and the sequential flow of two related models or eters at once, applying a transformation to a linear range usability test using the search function of the three types, operation between two modules have not been of [ 1,1]. Because the x, y and z components have as- [4] T. B. L. Renato Fernandes Corra, “Dimensionality which resulted in a 96.3% rate of task completion, and a MOS sophisticated. Since the first stage of system goal is − sociated type information, the polymorphic To Linear Reduction of very large document collections by Se- (easiness) of 2.9, which was better than the previous system. almost accomplished, we move to the second stage. − These scores are considered to be satisfactory. This paper function can deduce the appropriate scaling automatically. mantic Mapping,” in Proceedings of the 6th Interna- Our next goal is in specified sound search. These describes the system configuration as well. The transformed values are combined into a single vec- tional Workshop on Self-Organizing Maps, Center independent functions should be integrated and the tor coordinate, which is used as a coordinate for a spatial of Informatics, Federal University of Pernambuco, relevant data should be available via fast and shared 1. INTRODUCTION panner. A simple smoothing is applied by a filter – which Brasil, 2007. access between modules. can maintain the vector coordinate semantics due to the We have been constructing an electronic timbre Previous sub-modules were constructed in different [5] M. W. N. Schnell and J. B. Rovan., “Escher- languages and some modules ran as a different generic approach described in Section 3.4.2. dictionary for the purpose of supporting computer music modeling and performing composed instruments in application. Resultantly, cross-compilation and linking The spatializer can also leverage polymorphism; type composition and multimedia content creation. real-time.” in IEEE Systems, Man, and Cybernetics, of two different languages were not necessarily possible, context can imply whether the control parameter is given For computer music composition and other sound 1998. and sharing parameters and data became complicated. in polar or cartesian form, and if the panning happens in 2 media content creation, finding sound material can be Because of the frequent update of the flash platform or or 3 dimensions. [6] H. C. Steiner, “Towards a catalog and software li- the very first phase of the work sequence. However, brary of mapping methods,” in Proceedings of the recording all the materials on one's own is costly and Flex SDK, its portability was low. Moreover, the International Conference on New Interfaces for Mu- sometimes impossible. resultant application is more difficult to maintain and 5. CONCLUSIONS has slower processing speed than one written in a native sical Expression (NIME06), Paris, France, 2006. We have been constructing an electronic timbre dictionary which provides sound material from an program language. In this paper, the authors have demonstrated some of the [7] H.-C. Steiner, “[hid] toolkit: a unified framework for environmental sound database for such sound creators, Therefore we have implemented the second version to possibilities and advantages of using polymorphism as a instrument design,” in Proceedings of the 2005 In- with various search functions [1][2]. consider strong cooperation among sub-systems. mapping strategy based upon signal semantics. Types are ternational Conference on New Interfaces for Musi- Freesound is a well known system of sound data [3]. Specifically, we aimed to achieve three goals: multi- used as metadata that aid composed mapping monads to cal Expression (NIME05), Vancouver, BC, Canada, The electronic timbre dictionary introduced here has a platform (Windows, Mac OS and Linux), fast processing configure themselves to account for disparate parameter 2005. browsing mode as well as sound registration with an speed, and easiness of linking modules. For these goals ranges. upload function for the users. The major features of our we selected a single lower level language: C++. [8] C. A. C. T. and H. C., “Real-time Gesture Mapping We have the advantage of the generative nature of Kro- system are: a variety of search functions including The system adopts the Creative Commons licence [4] in Pd Environment using Neural Networks,” in Pro- nos programming language. To expand upon this we ap- inverse look up of the sound using onomatopoeia for sound data, which is considered to be suitable for ceedings of the International Conference on New In- ply a type system to mapping. By abstracting the math besides forward search and keyword search, and GUI- sound content creators. For example, it is the license terfaces for Musical Expression (NIME04), Japan, into software objects, mapping can be approached as a based sound exploration: a single stream of a sound is offered by freesound. 2004. system of higher level human logic. defined as a sound object and represented by a sphere in This paper describes the system concepts and system Upon this, the authors are developing a framework for configuration and evaluation test results for various [9] Essl, Georg “UrSound-Live patching of audio and a 3D domain [2]. Acoustic feature types to define the Generative Music Composition and Performance based on sound search method. multimedia using a multi-rate normed single-stream displacement in 3D space are being under study that fits live audio and sensor data input. By making the mapping data-flow engine,” in Proceeding of the International well to our auditory perception. process smarter and quicker to prototype we can enable 2. SYSTEM CONFIGURATION Computer Music Conference 2010 (ICMC 2010), Another aspect is that the system aims to act as a high- another way of interacting and conceiving generative mu- USA, 2010. quality sound database, so that the search results can be sic. used in content creation. 2.1. OpenGL as 3D graphics API The framework along with the Kronos compiler will [10] Bencina, Ross “The Metasurface - Applying Natu- We have built a server-client application and a desktop For 3D timbre display, OpenGL[5] is adopted as a be made freely available to interested parties. It is con- ral Neighbour Interpolation to Two-to-Many Map- (stand-alone) application. For the former type, the cross-platform 3D Graphics API. OpenGL is a portable ceived as a Kronos library, but given its extensive OSC ping,” in Proceedings of the 2005 International database can be shared publicly or limited to a closed programming interface, and does not depend on the OS and MIDI communication capabilities, interaction with Conference on New Interfaces for Musical Expres- group. The latter type is useful for personal and private and hardware types. various software packages is feasible. Finished adapta- sion(NIME05), Canada, 2005. use. Also, we adopted the GLUT (OpenGL Utility Toolkit), tion layers can also be built into object code that can link The purpose of the system construction so far is to which is a class library of C++ designed based on the implement each module and examine each function OpenGL, which reduces system development costs. assigned to the module. Therefore each module has been

470 | 2013 ICMC idea | DEMONSTRATIONS 471 | 2013 ICMC idea | DEMONSTRATIONS

2.2. C language-based GUI toolkits with multi- toolkit automatically. “Audacity”[7] is a prominent Condition No Indication of sound platform application using wxWidgets. Taking the improvement of processing speed and 1 Time signal, “po:n” 2.4. For 3DCG multi-platform into consideration, C++ is adopted as 2 Jet plain, “go:” wxWidgets includes wxGLCanvas, a wrapper class for development language, which is also a native language OpenGL. Therefore, an OpenGL drawing can be 3 Wind chime “shararara:n” of the various OSes. reproduced by wxWidgets. In addition, GLUT is a 4 Wave, “zaza:n” If a program is written in C++, it can run on various library that conforms to OpenGL, and high affinity is 5 Bird “kakko” OSes. However, every OS has a different low-level achieved with wxWidgest. 6 Opening a can, “Kash” GUI (Graphical User Interface) toolkit, and to write a 7 Cymbal, “Ja:n” program corresponding to each GUI toolkit has a 3. NEW SYSTEM CONFIGURATION 8 Casternette “katkatkat” higher development cost. Therefore, in order to achieve 9 Instrument,”jua:n” multi-platform using a single source code, we We have implemented a new electronic timbre 10 Jew’s Harp “boi:n” compared the following five types of high-level GUI C dictionary in Windows, incorporating the results of the Figure 3. 3D timbre display function 11 Opening sodacan,“shwashwashwa” language based toolkits. above discussion. Figure 1 depicts the configuration of 3.2. Sound search 12 Bass drum, “don” 1. Ultimate++ the newly-implemented electronic timbre dictionary. In the previous system sound search was done by Apache 13 Jew’s Harp “byoii:n” 2. Gtk+ The browser has a MyTimbreList function, similar to a Solr [8]. In the new system search is done simply by the 3. Qt playlist in music player software, enabling the user to find function in wxWidgets by entering text such as 14 Cuckoo, ”kakko” 4. OpenFrameworks store and group a series of meaningful sound data. Sound onomatopoeia or keyword. The sound database is 15 Skylark, “pi:chikpa:chik” 5. wxWidgets data in a MyTimbreList is displayed in 3D, built in repewsented in XML format, which has a different 16 Cow,”n:mo:” OpenGL. A user can jump directly from the browser by a hierarchy from sound storage using the OS file structure. 17 Poured water, “tokkun,tokkun” Ultimat GTK Qt OF Wx- single click. This display is aimed at to explore the It is displayed at the left on the browser. A search type 18 Cowbell, “karankaran” e++ + Widgets various sounds before the search phase. following this hierarchy is called “forward lookup GUI original Unix native original native Client search.” Input examples might be birds names or Table 2. Search conditions of sound look and fair bad good fair good Sound Browser instrumental names like ‘bells’. In addition to this, the feel Function 3D Timbre Display system can also search the sound based on keywords 2. For physical evaluation, we measured the search Function OpenGL yes no yes yes yes Sound Registration appeared in a tag, or do “reverse lookup search” from the time in seconds for each condition. We considered a Update recently July recent recently recently onomatopoeia. search time more than 2 minutes as a failure. We 2011 ly Search Wave read Three-dimensional defined “the task completion rate” as the rate at Parameters License BSD LGPL GPL MIT Wx- 4. USABILITY EVALUTION EXPERIMENT which the search was successful – that is, less than license License Windows Acoustic Feature two minutes. MySQL In order to evaluate the usability of the system, we License Extraction Engine 3. Opinion test of satisfaction in five categories ranging prepare two scenes to find the sound materials. Case A is freeware yes yes yes yes yes from 0 (very bad) to 4 (very good). MOS is obtained. 13 conditions of a sound source presented by texts with 4. SUS (System Usability Scale)[9]. Sound Waveform Sound Database onomatopoeia. The other case B is those of similar In the System Usability Scale, ten questions are Table 1. Comparison results of the C-language-based high- Database conditions together with sound explanation and level GUI toolkit Server presented. Even-numbered questions are positive ones onomatopoeia in a storyboard of commercial scenes. Table 1 shows the comparison results for relevant while the odd-numbered are negative. The sum of all the Figure 1. Block diagram of the new electronic timbre Table 2 shows 13 conditions of case A plus 5 conditions frameworks. Based on this examination we decided to dictionary evaluation with consideration of sign, are normalized so of case B. The Japanese is a language known as having adopt wxWidgets [6]. that it ranges from 0 to 100. 3.1. Implementation rich onomatopoeia. More than five thousand lexical As for GUI design, native display is only possible by Qt We have implemented the new system based on the words are said to exist. 4.3. Experimental result from completion rate and and wxWidgets. If these toolkits are used, OS-dependent discussion above. Implementation was done on Windows MOS (easiness) native GUI design is presented to the user, meaning that 4.1. Evaluation experiment OS. Development was carried out on Microsoft Visual Table 3 shows a comparison of task completion rate the system can be used under a familiar GUI. The subjects are requested to find sounds as directed in Studio 2010 Professional. The browser was unified with between the previous and the new system. GTK + does not support OpenGL. In order to the condition sheet from the three types of search 3D timbre display, which became possible under the implement the 3D timbre display function, OpenGL must functions equipped with the system stated in section 3.2. unique language. The new sound browser and 3D timbre Total trial Failed trials the task run on a high-level GUI toolkit. We conducted experiments for the conditions above, display are depicted in Figures 2and 3, respectively. (times) (times) completion rate The wxWindows License, which wxWidgets adopts, is using the newly constructed system in comparison with Previous 126 24 81.0% an improved LGPL license. In this license, there is no previous system. The number of subjects for the previous system duty to disclose the source code of the application, and system is seven, while the number of the subjects for the Present 324 12 96.3% its commercial use is also possible. new system is 18 in twenties, including two females. system 2.3. Features of wxWidgets 4.2. Evaluation method Table 3. Comparison of task completion rate The main features of wxWidgets include the following We adopted the following two evaluations for 18 four points. conditions in Table 2 (items 1 and 2). In addition, the In the previous system, there were many cases that failed 1. High portability subjects evaluated the extra two items, not for the each in the sound search. The new system enables the finding 2. Displayable by using the low-level native GUI but for the overall 18 conditions (items 3 and 4). of the sound, often within two minutes. Figure 4 shows a 3. Achievements of developing commercial applicati- comparison of the average search time between the ons in various fields 1. Opinion test: an instruction to evaluate five previous and the new system for all the search styles. 4. Supports multi-platform categories is given, ranging from 1: difficult to 5: Figure 5 shows a comparison to MOS (easiness) of all easy search. Mean Opinion Score (MOS (easiness)) search functions. In a global sense, both MOS (easiness) As for 1, if a single source code is written, in the building ranging 0 to 4 was derived. and the average amount of time required have been process for each OS, wxWidgets calls its low level GUI improved in comparison with the previous system, with Figure 2. Sound browser function slight exception exhibited by condition No. 5.

472 | 2013 ICMC idea | DEMONSTRATIONS 473 | 2013 ICMC idea | DEMONSTRATIONS

2. There are two reasons for the improvements. One DIPS5 FOR MAX is that OS dependent native GUI can be achieved by using wxWidgets, and the other is that its - Digital Image Processing with Sound, version 5 - operation becomes simpler. 3. From MOS and SUS scores, easy to use evaluation was obtained. Takayuki Rai Shu Matsuda Yota Morimoto The present system is freely downloaded from the site: http://www.srl.im.dendai.ac.jp/ETD/ Lancaster University Kunitachi College of Music University of Birmingham The present version is Japanese based with a few Lancaster Institute for the Sonology Department [email protected] English explanations. Onomatopoeia is totally in Contemporary Arts [email protected] Japanese. [email protected] Takyuto Fukuda 5. CONCLUSION Universität für Musik und Keitaro Takahashi darstellende Kunst Graz Figure 4. Comparison of the average time it takes each We carried out restructuring and integration of Institut für Elektronische Musik condition of all search functions subsystems of electronic timbre dictionary in C++. Due Musik Akademie Basel to the development of a unified language, processing Elektronik Studio Basel und Akustik - IEM speed of the proposed system has been improved. [email protected] [email protected] Moreover, adoption of wxWidgets enabled multi- platform by a single source code. ABSTRACT sensor technology have been implemented. Here, we The usability evaluation experiment was done to find would like to introduce those enhanced features and The DIPS was first presented in the international the sound by using the search function of the three types, demonstrate its ease of programming, power, and which resulted in a 96.3% rate of task completion, and a computer music world at the ICMC 2000 in Berlin as a efficiency in Max programming environment. MOS (easiness) of 2.9, which is better than the previous plug-in software for the legendary Max family system. For overall evaluation, a MOS of 3.0 and 79.8 application ‘jMax’ in order to perform the real-time ! 2. Principal DIPS objects SUS score were acquired. These scores are considered image-processing in Max patches, thus to support the to be satisfactory. creation of interactive multimedia art. In 2005, the DIPS 2.1 DIPS core objects In the future, we will further improve the usability GUI, was ported to Max running on Macintosh computer, and The DIPS programming in Max begins with creating to build on Mac OS and Linux, and to proceed towards in 2007, Dlib (DIPS utility library) and Dfx (DIPS ‘DIPSWindow’ object (see Figure 1), where the result of the release of the full-scale system. Figure 5. Comparison of the MOS (easiness) for each visual effect) objects were introduced to the DIPS to the image-processing calculations will be rendered. The condition of all search functions 6. REFERENCES make image-processing programming much simpler and image-processing and the constructing and handling easier. This version of DIPS ‘DIPS3’ was presented at procedure of 2 and 3 dimensional objects must be [1] Yohei Kobayashi and Naotoshi Osaka, the ICMC 2007 in Copenhagen, and it kept evolving for programmed between ‘DIPSSetCurrentWindow’ and 4.4. Other evaluation aspects Correlation between search time as a physical evaluation “Construction of an electronic timbre dictionary for the last five years. Now, the DIPS consists of more than ‘DIPSSwapBuffer’ objects with specifying target and the MOS (easiness) of 18 conditions for the previous environmental sounds by timbre symbol,” three hundreds Max external objects and abstractions. At ‘DIPSWindow’. Multiple DIPSWindows can be created system was as low as -0.3, while the new system has a Electronic Proceeding of ICMC 2008, Belfast, this occasion of ICMC 2013, we would like to introduce with any window sizes. ‘DlibWindow’ object is the Aug., 2008. correlation of -0.79, which is a sufficiently high the new features of DIPS5 including DIPS OpenCV wrapper object of ‘DIPSWindow’ to simplify the negativecorrelation between the two evaluation values. [2] Naotoshi Osaka, Yoshinori Saito, Shinya Ishitsuka objects and demonstrate how easily the real-time image- creation of ‘DIPSWindow’. All of DIPS objects are This is interpreted as follows: a large amount of time was and Yasuhiro Yoshioka, “An electronic timbre processing can be programmed in Max environment. bang-oriented; therefore, they don’t consume any CPU spent in superficial GUI operation that was not search- dictionary and 3D timbre display,” Proceeding of power, unless they get bang at the leftmost inlet as every related in the previous system, while in the new system, ICMC 2009, pp. 9-12, Montreal, Aug., 2009. 1. INTRODUCTION Max control object does. (see Figure 2) most of the time was spent on search due to the efficiency of the improved GUI, with little time needed [3] Music Technology Group, “The freesound The DIPS was developed for Max/FTS first in 1997 by for GUI operation. Project,” http://freesound.iua.upf.edu/ Shu Matsuda at Sonology Department, Kunitachi The average value of each MOS (easiness) of all 18 [4] Creative Commons Japan , College of Music in Tokyo. It was ported to jMax conditions for the previous system is 2.2. On the other http://creativecommons.org/licenses/ programming environment in 2000, and later to Max hand, the average MOS value over 18 conditions for the [5] OpenGL - The Industry Standard for High running on Macintosh computer. The early versions of new system was improved to 2.9. DIPS consisted of only DIPS core objects and DGL The MOS of the new system was 3.0. Furthermore, the Perform- ance Graphics, http://www.opengl.org/ objects those are wrapper objects of various OpenGL System Usability Scale score is 79.8 points. From these [6] wxWidgets Cross-Platform GUI Library functions. Therefore, to realize 3D image-processing evaluation values a relatively high satisfaction score was http://www.wxwidgets.org/ obtained. using DIPS the knowledge of OpenGL programming [7] Audacity: Free Audio Editor and Recorder, technique was still essential. To resolve this difficulty 4.5. Discussion http://audacity.sourceforge.net/ for ordinary composers and artists, we introduced Dlib We consider the following three points from the (DIPS utility library) and Dfx (DIPS visual effect) experimental results. [8] Apache Lucene - Apache Solr 1. From task completion rate, and comparison of the http://lucene.apache.org/solr/ objects in the third version of DIPS in 2007. Since then further Dlib and Dfx objects have been developed as Figure 1. DIPSWindow object average evaluation and the average search time, [9] Brooke, J.: SUS: A “quick and dirty” usability scale. In: the basic validity of the proposed system was Jordan, P.et al. (eds.) Usability Evaluation in Industry pp. well as new functions such as OpenCV and Kinect confirmed. 189-194. Taylor & Francis, London, UK (1996).

474 | 2013 ICMC idea | DEMONSTRATIONS 475 | 2013 ICMC idea | DEMONSTRATIONS