THE CREATIVE PROCESS OF COMPUTER-ASSISTED COMPOSITION AND MULTIMEDIA COMPOSITION — VISUAL IMAGES AND MUSIC Chi Wai Chen BMus (University of Missouri – Kansas City Conservatory of Music) Dip Ed (Hong Kong Baptist University) MA (Hong Kong Baptist University) A thesis submitted in total fulfillment of the requirements for the degree of Doctor of Philosophy in the School of Education, Portfolio of Design and Social Context Royal Melbourne Institute of Technology (RMIT University) Dec 2006 © 2006 Chi Wai Chen ALL RIGHTS RESERVED 1 DECLARATION To the best of my knowledge and belief, this thesis contains no material previously published or written by any other person, except where due acknowledgment has been made in the text. This thesis has not been submitted previously, in whole or in part, to qualify for any other academic award. The content of this thesis is the result of work which has been carried out since the official commencement date of the approved research program. ………………………………….. Chi Wai Chen i TABLE OF CONTENTS Page Table of Contents ii List of Tables xii List of Figures xvi Terms and Abbreviations xx Glossary xxii Acknowledgement xxiv Abstract xxv Chapters 1 Introduction 1 1.1 THE PURPOSE OF THE RESEARCH 1 1.2 SCOPE OF THE STUDY 1 1.3 RATIONALE 2 1.3.1 Broader elements of the Rationale 4 1.3.2 Definition of the Problem 4 1.4 RESEARCH QUESTIONS 5 1.5 METHODOLOGY 5 1.6 MAIN DEFINITIONS 7 1.6.1 Creative Process and Product 7 1.6.2 Computer-assisted Composition 7 1.6.3 Multimedia 8 1.6.4 Creativity 8 ii 1.6.5 Digital Audio Workstation 8 1.6.6 Music Sequencing and Music Notation 9 1.6.7 Composing Strategies 9 1.7 DELIMITATIONS 9 1.8 OVERVIEW OF CHAPTERS IN THE STUDY 9 2 Background to the Study 13 2.1 HONG KONG INSTITUTE OF EDUCATION 13 2.2 ASSOCIATE OF ARTS (MUSIC) DEGREE 14 PROGRAM 2.3 MUSIC CURRICULUM OF SECONDARY AND 17 PRIMARY SCHOOLS IN HONG KONG 2.3.1 Key Learning Area of the Music Curriculum 17 Guide in Creativity (Primary 1 to Secondary 3) 2.3.2 New Senior Secondary Music Curriculum in 19 Creativity (Secondary 4 to 6) 2.3.3 The Development of Music Technology in 21 Education in Hong Kong 2.4 THE COMPOSITION ENVIRONMENT IN 23 HONG KONG 2.4.1 The Cultural Policy of Hong Kong Arts 24 Development Council 2.4.2 The Composers and Authors Society of Hong 25 Kong Ltd. 2.4.3 Policy for Funding Composition 26 2.4.4 Opportunities for Students from Secondary 29 School to Reach a Professional Level in Composition in Hong Kong 2.5 SIGNIFICANCE DRAWN FROM THE 30 BACKGROUND TO THE STUDY 3 Review of the Literature 32 iii 3.1 INTRODUCTION 32 3.2 MUSIC TECHNOLOGY OVERVIEW 33 3.3 MUSIC TECHNOLOGY FUNCTIONS 38 3.3.1 MIDI 38 3.3.2 MIDI Keyboard 40 3.3.3 Sequencing Software 40 3.3.4 Track 41 3.3.5 Patch 41 3.3.6 Channel 41 3.3.7 Quantizing or Humanizing 42 3.3.8 Sound Modules 42 3.3.9 Digital Audio Recording 42 3.3.10 Mixer 43 3.3.11 Video Capturing and Synchronization with 43 MIDI/Audio Tracks 3.4 COMPUTER-ASSISTED COMPOSITION 44 3.4.1 Notation Software 46 3.4.2 Sequencing Software 46 3.4.3 Accompaniment Software 47 3.4.4 Digital Audio Recording Software 48 3.5 MULTIMEDIA COMPOSITION 49 3.5.1 Judgment and Alignment of Visual Images 53 and Music 3.5.2 Film Scoring and Digital Film Scoring 55 3.5.3 Composing for the Media 60 3.6 CREATIVITY 61 3.6.1 Amabile’s Framework of Creativity 62 iv 3.6.2 Creativity in Children 63 3.6.3 Creativity in Adults 64 3.6.4 Creative People 65 3.7 THE CREATIVE PROCESS IN MUSIC 66 COMPOSITION 3.7.1 Wallas’s Stage Theory 66 3.7.2 Webster’s Model of Creative Thinking in 69 Music 3.7.3 Hickey’s Model of the Creative Process 71 3.7.4 Sloboda’s Model of the Creative Process 73 3.8 THE CREATIVE PRODUCT 74 3.9 COMPOSING AS PROBLEM-SOLVING 76 3.9.1 Composing Strategies 78 3.9.2 Improvisation and Composition 83 3.9.3 Style, Genre and Musical Language 85 3.10 RESEARCH IN COMPOSITION 86 3.10.1 Swanwick’s Model of Musical 87 Development in Children and Adults’ Compositions 3.10.2 Kratus’s Model of Compositional 91 Development in Children 3.10.3 Bennett’s Model of Compositional 92 Development in Professional Composers 3.11 STUDYING COMPOSITION USING A 95 COMPUTER 3.12 IMPLICATIONS FOR THE PRESENT STUDY 105 4 Research Design and Methodology 108 4.1 AIM AND RESEARCH QUESTIONS 108 4.2 RESEARCH DESIGN 109 v 4.2.1 Program 110 4.2.2 Research Plan 110 4.2.3 Freedom to choose time of composition 110 4.2.4 Hardware 111 4.2.5 Software 111 4.3 THE RESEARCH PROCESS 112 4.3.1 Stages 112 4.3.2 Ethics Procedures 113 4.3.3 Sample Selection 114 4.3.4 Subjects 114 4.3.5 Purpose of the Survey 114 4.3.6 Design of the Survey 115 4.3.7 Administration and the Collection of Data 116 4.4 THE TASKS 116 4.4.1 Scheme of Work 116 4.4.2 Task One 118 4.4.3 Task Two 118 4.4.4 Instructions 119 4.4.5 Concepts During Each Session 120 4.5 INSTRUCTIONS ABOUT MULTIMEDIA 121 4.5.1 Developing Visual Skills 121 4.5.2 Image Associated Perception 121 4.5.3 Creating a Storyboard 122 4.5.4 From Story to Markers 122 4.5.5 From Markers to Composing 122 vi 4.5.6 Developing Musical Skills 122 4.5.7 Summary 123 4.6 DATA COLLECTION 125 4.6.1 Interviews 125 4.6.1.1 First Interview 127 4.6.1.2 Second Interview 130 4.6.2 Self-Reflective Journal 131 4.6.3 MIDI File Observation 134 4.6.4 Written Report 136 4.7 DATA ANALYSIS PROCEDURES 137 4.7.1 Transcribing the Data 137 4.7.2 Analyzing Interview Scripts and Profiles 137 4.7.3 Analyzing Written Reports 137 4.7.4 Analyzing the Self-Reflective Journal 137 4.7.5 Analyzing the MIDI File through Observation 140 4.7.6 Triangulation 141 4.8 CONCLUSION 142 5 Presentation of the Data (Task One) 144 5.1 RESULTS OF THE SURVEY 144 5.2 PROFILE OF PARTICIPANTS 147 5.3 RESULTS OF THE ANALYSIS FROM THE 151 WRITTEN REPORTS IN TASK ONE 5.4 RESULTS OF THE ANALYSIS FROM THE 157 INDIVIDUAL INTERVIEWS IN TASK ONE 5.5 RESULTS OF THE ANALYSIS OF THE 177 SELF-REFLECTIVE JOURNAL UNDERTAKEN IN TASK ONE vii 5.6 RESULTS OF THE ANALYSIS OF THE MIDI 181 FILE OBSERVATION IN TASK ONE 5.7 SUMMARY OF FINDINGS FROM MIDI FILE 196 OBSERVATION 6 Presentation of the Data (Task Two) 198 6.1 RESULTS OF THE ANALYSIS FROM THE 199 WRITTEN REPORTS IN TASK TWO 6.2 RESULTS OF THE ANALYSIS OF THE 205 INDIVIDUAL INTERVIEW IN TASK TWO 6.3 RESULTS OF THE ANALYSIS FROM THE 227 SELF-REFLECTIVE JOURNAL IN TASK TWO 6.4 RESULTS OF THE ANALYSIS FROM THE 231 MIDI FILE OBSERVATION IN TASK TWO 6.5 SUMMARY OF FINDINGS FROM MIDI FILE 250 OBSERVATION 7 Analysis of the Data 252 7.1 RESPONDING TO RESEARCH QUESTIONS 1, 252 2, AND 3 7.2 RELATIONSHIPS BETWEEN THE FOUR DATA 268 SOURCES 7.3 TASK ONE 274 7.3.1 Task One - Commonalities among the 10 274 Participants in terms of Interaction with the Musical Elements in Computer-Assisted Composition at the Beginning of the Creative Process 7.3.2 Task One - Commonalities among the 10 276 Participants Applying Music Technology in Computer-Assisted Composition during the Creative Process 7.3.3 Task One - Commonalities among the 10 279 Participants Reflecting on Computer-Assisted Composition at the End of the Creative Process viii 7.4 TASK TWO 281 7.4.1 Task Two - Commonalities among the 10 281 Participants Responding to Visual Images in Multimedia Composition at the Beginning of the Creative Process 7.4.2 Task Two - Commonalities among the 10 283 Participants Applying Music Technology in Multimedia Composition during the Creative Process 7.4.3 Task Two - Commonalities among the 10 286 Participants Reflecting on Multimedia Composition at the End of the Creative Process 7.5 RESPONDING TO RESEARCH QUESTION 4 - 288 PROPOSED MODEL FROM THE DEVELOPMENTAL PERSPECTIVES IN COMPUTER-ASSISTED COMPOSITION AND MULTIMEDIA COMPOSITION 7.5.1 Spiral Development in the Creative Process of 289 Computer-Assisted Composition 7.5.2 Spiral Development in the Creative Process of 291 Multimedia Composition 7.5.3 The Extension of Webster’s Model of Creative 294 Thinking in Computer-Assisted and Multimedia Composition 7.5.4 Developmental Patterns during the Creative 297 Process in Computer-Assisted Composition 7.5.5 Developmental Patterns during the Creative 300 Process in Multimedia Composition 7.6 DISCUSSION 304 7.6.1 Developmental Perspectives 304 7.6.2 Music Technology, Musical Thinking, and 305 Decision-making 7.6.3 Creativity, the Creative Process, and Product 306 7.6.4 Computer-Assisted Composition 308 ix 7.6.5 Multimedia Composition 308 7.6.6 Alignment in Visual Stimulus and Musical 309 Accent 7.6.7 Composing Strategies 310 7.7 COMPARATIVE ANALYSIS BETWEEN TASK 313 ONE - COMPUTER-ASSISTED COMPOSITION AND TASK TWO - MULTIMEDIA COMPOSITION 8 Conclusions and Recommendations 317 8.1 INTRODUCTION 317 8.2 SUMMARY OF THE RESEARCH 319 8.2.1 Task One (Computer-Assisted Composition) 319 8.2.2 Task Two (Multimedia Composition) 320 8.2.3 Findings of Task One 321 8.2.4 Findings of Task Two 323 8.2.5 Limitations of the study 324 8.3 IMPLICATIONS FOR FURTHER RESEARCH 325 8.3.1 Techniques Employed in Data Collection 325 8.3.2 Utilization of Music Technology 326 8.3.3 Understanding of Creative Process 327 8.3.4 Origin of Musical Ideas 327 8.3.5 Suggestions for Further Research 328 8.4 IMPLICATIONS FOR MUSIC EDUCATION 330 8.4.1 Shift Paradigm in Teaching and Learning 330 8.4.2 Model Application - Pedagogy in 332 Computer-Assisted Composition 8.5 CURRENT PRACTICE 336 x 8.5.1 Computer-Based Creative Music 336 Environments 8.6 FUTURE PRACTICE 338 8.6.1 Computer-Based Music Curriculum 338 8.6.2 Hyperinstruments as Musical Interfaces 340 8.6.3 The Evolution from MIDI to Audio Editing 340 8.6.4 Mobile and Laptop Technology 341 8.6.5 The World of Multimedia 341 8.7 CONCLUSION 342 REFERENCES 345 APPENDICES I.