DIGITAL AUDIO WATERMARKING FOR BROADCAST MONITORING AND CONTENT IDENTIFICATION Ron Healy B.Sc. A THESIS SUBMITTED FOR THE QUALIFICATION OF MASTER OF SCIENCE IN COMPUTER SCIENCE National University of Ireland, Maynooth Ollscoil na hÉireann, Má Nuad Department of Computer Science OCTOBER 2009 HEAD OF DEPARTMENT: DR. ADAM WINSTANLEY, PH.D. RESEARCH SUPERVISOR: DR. JOSEPH TIMONEY, PH.D. MR. TOM LYSAGHT. - 1 - Declaration: I hereby certify that this material, which I now submit for assessment on the program of study leading to the award of Master of Science in Computer Science by Research, is entirely my own work and has not been taken from the work of others save and to the extent that such work has been cited and acknowledged within the text of my work. Signed:_____________________________________________________ Dated:______________________________________________________ - 2 - Contents LIST OF FIGURES........................................................................................................ 6 ABSTRACT.................................................................................................................... 8 DEDICATION................................................................................................................ 9 ACKNOWLEDGEMENTS...........................................................................................10 PREVIOUS PUBLICATIONS ......................................................................................11 CHAPTER 1: INTRODUCTION .................................................................................12 1.1 BACKGROUND TO THIS WORK .................................................................................13 1.2 INTRODUCTION TO THE PROBLEM-DOMAIN ..............................................................17 1.2.1 Digital audio fingerprinting ...........................................................................18 1.2.2 Digital audio watermarking ...........................................................................22 1.3 PROBLEM STATEMENT ............................................................................................24 1.4 STRUCTURE OF THE REMAINDER OF THIS DOCUMENT ...............................................25 1.5 SUMMARY .............................................................................................................27 CHAPTER 2: COPYRIGHT, ROYALTIES AND MUSIC ........................................28 2.1 WHAT IS COPYRIGHT AND WHAT CAN BE COPYRIGHTED?.........................................28 2.1.1 Public performance rights and royalties .........................................................31 2.1.2 The need for a royalty collection and distribution process ..............................34 2.2 THE EXISTING ROYALTY COLLECTION AND DISTRIBUTION PROCESS ..........................36 2.2.1 Why is there a need for a new system?............................................................38 2.3 WHAT IS THE ‘COLLECTIVE RIGHTS’ MARKET WORTH? ............................................40 2.3.1 Royalty payments to content owners ...............................................................41 2.4 SUMMARY .............................................................................................................43 CHAPTER 3: THE DEVELOPING ARTIST. .............................................................44 3.1 POPULARITY NETWORKS AND ATTRACTEDNESS .......................................................44 3.1.1 Popularity networks and social marketing ......................................................46 3.2 INDUSTRY INDICATORS ..........................................................................................48 3.2.1 Radio as a referrer ........................................................................................48 3.2.2 Sales charts ...................................................................................................50 3.2.3 Airplay charts ................................................................................................52 3.3 IMPACT OF THE CURRENT SYSTEMS ON DEVELOPING ACTS AND INDUSTRY ................53 3.4 OVERVIEW OF ROYALTY COLLECTION AND DISTRIBUTION IN IRELAND .....................55 3.4.1 Limitations in the current system ....................................................................57 3.5 THE IMPORTANCE OF FAIR AND EQUITABLE RIGHTS ADMINISTRATION ......................60 3.6 IDENTIFICATION OF A UNIQUE PIECE OF AUDIO IN THE MUSIC INDUSTRY ..................62 3.6.1 The ISRC Code ..............................................................................................63 3.7 SUMMARY .............................................................................................................64 CHAPTER 4: SOUND, HEARING AND DATA-HIDING ..........................................65 4.1: THE HUMAN AUDITORY SYSTEM ...........................................................................65 4.1.1 Visualising sound waves ................................................................................66 4.2 PSYCHOACOUSTICS ................................................................................................68 4.2.1 Auditory masking ...........................................................................................69 4.2.2 Threshold of hearing......................................................................................70 4.3 DATA-HIDING CONCEPTS ........................................................................................72 4.3.1 Steganography ...............................................................................................74 4.4 DIGITAL SIGNAL PROCESSING CONCEPTS ................................................................75 4.5 DIGITAL AUDIO FILES .............................................................................................79 4.6 AUTOMATICALLY IDENTIFYING AUDIO IN THE DIGITAL ARENA .................................81 4.3.3 Digital Rights Management ...........................................................................81 4.7 FINGERPRINTING AND WATERMARKING ..................................................................82 4.7.1 Audio fingerprinting and broadcast monitoring ..............................................84 - 3 - 4.7.2 Audio watermarking and broadcast monitoring ..............................................88 4.8 SUMMARY .............................................................................................................90 CHAPTER 5: EXISTING WATERMARKING TECHNIQUES.................................91 5.1 APPLICATION OF WATERMARKING TO BROADCAST MONITORING ..............................91 5.2 LITERATURE REVIEW OF WATERMARKING TECHNIQUES ...........................................94 5.2.1 Least significant bit modification ...................................................................94 5.2.2 Echo-hiding ...................................................................................................96 5.2.3 Amplitude masking ........................................................................................99 5.2.4 Phase coding ............................................................................................... 102 5.2.5 Multiplicative watermarking ........................................................................ 106 5.2.6 Spread spectrum .......................................................................................... 110 5.2.7 Patchwork methods ...................................................................................... 112 5.2.8 Interpolation methods .................................................................................. 112 5.3 ADDITIONAL WATERMARKING SCHEMES ............................................................... 114 5.3.1 Muteness based audio watermarking ............................................................ 114 5.3.2 Sinusoidal pattern watermarking.................................................................. 116 5.3.3 Linear chirp watermarking. ......................................................................... 117 5.3.4 Watermarking with ‘MPEG 1 Layer 3’ compression. .................................... 118 5.3.5 High capacity watermarking schemes........................................................... 120 5.3.6 Audio watermarking for live performances. .................................................. 121 5.3.7 Chaos-based watermarking scheme.............................................................. 122 5.4 SUMMARY ........................................................................................................... 123 CHAPTER 6: LOW COMPLEXITY WATERMARKING SCHEME ..................... 125 6.1 FIRST PHASE OF THE DEVELOPMENT OF THE WATERMARKING SCHEME ................... 127 6.1.1 Identifying components present in the signal ................................................ 132 6.1.2 Synchronising the watermarked bits ............................................................. 133 6.1.3 Problems encountered with pre-existing components .................................... 133 6.1.4 Analysis of representative audio for inherent components ............................. 134 6.1.5 Weighting of frequency pairs ....................................................................... 136 6.1.6 Embedding at higher frequencies ................................................................. 138 6.2 SECOND PHASE OF THE DEVELOPMENT OF THE WATERMARKING SCHEME ............... 140 6.2.1 Application of a notch filter ........................................................................