JUHA ARRASVUORI Playing and Making Music Exploring the Similarities between Video Games and Music-Making Software ACADEMIC DISSERTATION To be presented, with the permission of the Faculty of Humanities of the University of Tampere, for public discussion in the Auditorium Pinni B 1096, Kanslerinrinne 1, Tampere, on September 6th, 2006, at 13 o’clock. UNIVERSITY OF TAMPERE ACADEMIC DISSERTATION University of Tampere Department of Music Anthropology Finland Distribution Tel. +358 3 3551 6055 Bookshop TAJU Fax +358 3 3551 7685 P.O. Box 617 [email protected] 33014 University of Tampere www.uta.fi/taju Finland http://granum.uta.fi Cover design by Juha Siro Printed dissertation Electronic dissertation Acta Universitatis Tamperensis 1165 Acta Electronica Universitatis Tamperensis 543 ISBN 951-44-6688-8 ISBN 951-44-6689-6 ISSN 1455-1616 ISSN 1456-954X http://acta.uta.fi Tampereen Yliopistopaino Oy – Juvenes Print Tampere 2006 Abstract Music-making software titles are special applications for the Sony PlayStation 2 video game console. Music is made with music-making software by selecting, modifying, and arranging digital sound files through their visual representations. This music-making activity is defined as nonlinear multitrack composing. The theory of video games as semiotic domains by the educational linguist James Paul Gee is adapted to define the semiotic domain of multitrack composing, which includes, among other things, the techniques of using multitrack studio equipment in music-making and the meanings given to the resulting music. The purpose of the study is to determine how nonlinear multitrack composing can be combined with play activities, and to define the design foundations of future video games that involve nonlinear multitrack composing. The research material includes three music-making software titles (Magix Music Maker, MTV Music Generator 2, and eJay Clubworld) and seven music-themed video games. The research methods include comparing the three music-making software titles with each other and with various definitions of video games. Nonlinear multitrack composing is compared to play activities to demonstrate the similarities and differences. The three music-making software titles are compared in order to identify their common functionality, and through defining a metaterminology of the functions of multitrack studio equipment, it is shown how the features of music-making software relate to the semiotic domain of multitrack composing. A model of the nonlinear multitrack composing process is defined to contain the phases of selecting, sampling, modifying, arranging, and mixdown. The model is used to structure the analyses. The similarity of music-making software to multitrack studio equipment is demonstrated. The following nonlinear multitrack composing techniques can be realized with music-making software as well as multitrack studio equipment: Sampling, sound design, mixing, programming, sound spatialization, and remixing. The backgrounds of the techniques are defined on the basis of earlier literature. Examples are presented of how the techniques have been used in music-making. Music-making software titles are compared to the following aspects of video games: Formal elements, dramatic elements, video games as systems and simulations, and mimicry in play. These definitions are based on earlier studies and literature. Making music with video games and music-making software is compared to ludus and paidia types of play activities as defined by the game researcher Gonzalo Frasca. Ludus is defined as playing a game with the objective of producing a winner or a record result, whereas paidia is defined as playing without such objectives. The following essential similarities between music-making software and video games are identified. Music-making software titles resemble video games that combine playing with creating and experimenting. As in video games with the objective of construction, so also in music-making software immaterial objects are made by selecting, modifying, and arranging prefabricated components. Music-making software titles are interpreted as open 3 simulations, that is, a kind of “playgrounds” that offer the tools and materials for making music with the approach of paidia play. The following key differences are shown between music-making software and video games. Music-making software titles do not specify explicit goals as games do. There are no predefined end conditions for the music-making. Music-making software titles do not give feedback and motivate music-makers as games motivate players, for instance by giving rewards and adapting the challenge to match the ability. Managing resources is essential in many games. The tools and materials of music-making software are not resources because they can be used unlimitedly. Music-making software titles do not influence the behavior of the music-maker as games influence the behavior of the player by valorizing certain outcomes of the play activity over others. Because there is no competition and the outcomes are not measured through scores or in terms of winning and losing, music-making software titles do not set up a conflict as ludus type of games do. Thus, making music with music-making software is not ludus type of play. The results of the study are design foundations of future video games that involve nonlinear multitrack composing. The design foundations are based on the established similarities and differences between music-making software and video games. Following the definition by the game researcher Jesper Juul, the rules of ludus type of video games that involve nonlinear multitrack composing should define outcomes that are variable, quantifiable, and valued. Such outcomes can be achieved, for example, if the material and tools for music-making are defined as resources and their use is measured. The methods that video games use to facilitate learning should be utilized to assist learning nonlinear multitrack composing. 4 Acknowledgements First, I wish to thank the supervisor of my postgraduate studies, Professor Timo Leisiö for his tolerance towards my diverse ideas and research interests. Thanks to Professor Paul Théberge and Professor Frans Mäyrä, the pre-examiners of this thesis for their constructive comments. In Nokia Research Center, thanks to Jussi Holopainen, Juha Kaario, Kari Lahdensuo, Matti Karlsson, and Professor Jukka Saarinen for giving me opportunities over the years to carry out this research in parallel with my regular work. I am grateful to Game House at the NRC Multimedia Technologies laboratory for funding the language review of the dissertation manuscript. I also thank Jukka Holm, Jussi Holopainen, and Professor Mikko Lehtonen for reading early versions of this thesis and suggesting improvements. Virginia Mattila reviewed the English language of the manuscript. Finally, I warmly thank my mother Leena, my close relatives, and all the others who have given me support and encouragement. Tampere, July 2006 Juha Henrik Arrasvuori 5 6 Table of Contents Abstract 3 Acknowledgements 5 Table of Contents 7 List of Figures 11 List of Tables 11 1. Introduction 13 1.1 Background 13 1.2 Music-Making Software 14 1.3 Remediation and Semiotic Domains 17 1.4 Justifying the Approach 19 1.5 The Framework of the Study 21 1.6 How the Study is Conducted 23 2. The Semiotic Domain of Multitrack Composing 25 2.1 Music and Semiotic Domains 25 2.2 Music as Recorded, Modified, and Reproduced Sound 25 2.3 Composing in the Multitrack Studio 30 2.4 Functions and Techniques 31 2.5 Multitrack Composing as a Music-Making Activity 34 2.6 Multitrack Studio Technology in the Context of New Media 36 2.7 Remediation in the Multitrack Studio Medium 38 2.8 Refashioning the Multitrack Studio Medium 40 2.9 Summary 46 3. Analysis Part 1: Music-Making Software and the Semiotic Domain of Multitrack Composing 49 3.1 Starting Points 49 3.1.1 Method for Part 1 of the Analysis 50 3.2 Analysis Part 1 Stage 1 52 3.2.1 Defining a Model of the Nonlinear Multitrack Composing Process 52 3.2.2 Stage of Systematic Description 54 7 3.3 Magix Music Maker 55 3.3.1 Generic Description 55 3.3.2 GUI 56 3.3.3 Selecting Phase 57 3.3.4 Modifying Phase 58 3.3.5 Arranging Phase 58 3.3.6 Mixdown Phase 59 3.4 MTV Music Generator 2 60 3.4.1 Generic Description 60 3.4.2 GUI 62 3.4.3 Selecting Phase 63 3.4.4 Sampling Phase 63 3.4.5 Modifying Phase 64 3.4.6 Arranging Phase 66 3.4.7 Mixdown Phase 68 3.5 eJay Clubworld 68 3.5.1 Generic Description 68 3.5.2 GUI 69 3.5.3 Selecting Phase 72 3.5.4 Modifying Phase 72 3.5.5 Arranging Phase 73 3.5.6 Mixdown Phase 75 3.6 Analysis Part 1 Stage 2: Defining a Metaterminology of Functions 75 3.6.1 Selecting 75 3.6.2 Sampling 77 3.6.3 Modifying 77 3.6.4 Arranging 81 3.6.5 Mixdown 85 3.7 Comparing Features to Functions 89 3.8 Analysis Part 1 Stage 3: Defining Nonlinear Multitrack Composing Techniques 93 3.8.1 Sampling 94 3.8.2 Sound Design 100 3.8.3 Programming 106 3.8.4 Mixing 112 3.8.5 Sound Spatialization 115 3.8.6 Remixing 120 3.9 Conclusions to Part 1 123 8 4. The Semiotic Domains of Video Games 127 4.1 Video Games as Semiotic Domains 127 4.2 Video Games as Hardware and Software 127 4.3 Defining Play, Games, and Video Games 129 4.4 Video Games as Systems and State Machines 132 4.5 The Theme of Music-Making in Video Games 134 4.5.1 Beatmania 136 4.5.2 Amplitude 137 4.5.3 Gitaroo Man 138 4.5.4 Fluid 139 4.5.5 SingStar 140 4.5.6 Get on da Mic 141 4.5.7 100% Star 142 5.