Advanced Automatic Mixing Tools for Music Perez Gonzalez, Enrique
Total Page:16
File Type:pdf, Size:1020Kb
Advanced automatic mixing tools for music Perez Gonzalez, Enrique The copyright of this thesis rests with the author and no quotation from it or information derived from it may be published without the prior written consent of the author For additional information about this publication click this link. https://qmro.qmul.ac.uk/jspui/handle/123456789/614 Information about this research object was correct at the time of download; we occasionally make corrections to records, please therefore check the published record when citing. For more information contact [email protected] Advanced Automatic Mixing Tools for Music Submitted by Enrique Perez Gonzalez For the Ph.D. degree of Queen Mary University Of London Mile End Road London E1 4NS September 30, 2010 2 I certify that this thesis, and the research to which it refers, are the product of our own work, and that any ideas or quotations from the work of other people, published or otherwise, are fully acknowledged in accordance with the standard referencing practices of the discipline. I acknowledge the helpful guidance and support of our supervisor, Dr. Johua Daniel Reiss. 3 Abstract This thesis presents research on several independent systems that when combined together can generate an automatic sound mix out of an unknown set of multi‐channel inputs. The research explores the possibility of reproducing the mixing decisions of a skilled audio engineer with minimal or no human interaction. The research is restricted to non‐time varying mixes for large room acoustics. This research has applications in dynamic sound music concerts, remote mixing, recording and postproduction as well as live mixing for interactive scenes. Currently, automated mixers are capable of saving a set of static mix scenes that can be loaded for later use, but they lack the ability to adapt to a different room or to a different set of inputs. In other words, they lack the ability to automatically make mixing decisions. The automatic mixer research depicted here distinguishes between the engineering mixing and the subjective mixing contributions. This research aims to automate the technical tasks related to audio mixing while freeing the audio engineer to perform the fine‐tuning involved in generating an aesthetically‐pleasing sound mix. Although the system mainly deals with the technical constraints involved in generating an audio mix, the developed system takes advantage of common practices performed by sound engineers whenever possible. The system also makes use of inter‐dependent channel information for controlling signal processing tasks while aiming to maintain system stability at all times. A working implementation of the system is described and subjective evaluation between a human mix and the automatic mix is used to measure the success of the automatic mixing tools. 4 Acknowledgments Thanks to Dr. Joshua D. Reiss for supervising me during this long journey. Thanks to Professor Mark Sandler for giving me the opportunity to be part of The Centre For Digital Music and for believing in my ideas. Thanks to Xui for great conversations during the early starts of this journey, which ended in great ideas. Thanks for Nagel and Andrew Robertson for allowing me to use some multi‐track recordings that ended being invaluable research data. Thanks to Antonio Zacarias, Mauricio Ramirez, Francisco Miranda, Jorge Urbano, Renato de la Rosa, Oscar Aguilar and Jaime Gonzalez for their kind support toward my research. Thanks to Alice Clifford for her support especially when things seemed impossible to achieve. Thanks to Agnes Doeringer, who played a paramount part in my life during the past 5 years of my life. Upmost thanks to my parents, Enrique Perez Adame and Lucia Gonzalez Iñiguez, without whom none of this would have been possible. Huge thanks for Matthew Davis, Michel Terrell, Martin James Morrell, Steve Welburn, Dan Stowell, Rebecca Stewart, Andrew Nesbit, George Fazekas, and Angi Atmadjaja witch suffered correcting my spelling more than once. Thanks to my examiners Udo Zoelzer and Tony Stockman I truly appreciate you took the time to do so. Big thanks to: Laura Margottini, Youtha Cuypers, Ilya Cuypers, Sabine Altendorf, Leonado Jaso, Adam Stark, Maria Jafari, Larisa and Kurt Jacobson, Louis Martignon, Sylvie Stuiz, Asterios Zacharakis, Vincent Verfaille, Christian Uhle, Tomas Wilmering, Chistopher Harte, Chris Landone, Chris Sutton, Chris Cannam, Matthias Mauch, Mark Pumbley, Simon Dixon, Katy Noland, Heather Andrews, Ben fields, Yves Raimond and Anne‐So Noiret, Juan Pablo Angulo, Rodolfo Rodriguez, Alberto Garcia, Adrian Bisiacchi, Jmmy Robertson, Robert Macrae, all C4DM and to all the people which some how participated in testing or contributed in some form. Finally massive thanks to those which in the process of printing and submitting I forgot to add them to this thanks, but if you should be here you know who you are. 5 Contents Table of contents Abstract .............................................................................................................................. 3 List of figures and tables............................................................................................... 8 List of symbols and abbreviations...........................................................................12 Part I Introduction and background.......................................................................15 Chapter 1 Introduction.............................................................................................................................16 1.1 Justification.......................................................................................................................................... 16 1.2 Scope of the research ...................................................................................................................... 17 1.3 Contributions of this thesis........................................................................................................... 18 1.4 Overview............................................................................................................................................... 19 1.5 Aim and objectives ........................................................................................................................... 21 1.6 Thesis ..................................................................................................................................................... 22 Chapter 2 Background and stateoftheart ......................................................................................23 2.1 The Mixer.............................................................................................................................................. 23 2.1.1 The input channel.................................................................................................................... 24 2.1.2 The master section.................................................................................................................. 27 2.2 State of the art in automatic mixing.......................................................................................... 28 2.2.1 Automatic mixing .................................................................................................................... 29 2.2.2 Automatic mixing classification ........................................................................................ 32 2.2.3 Related work to automatic mixing................................................................................... 34 2.3 Going beyond the state of the art (challenges) .................................................................... 35 2.3.1 Large room and open space mix versus small room mix....................................... 35 2.3.2 Static versus time varying mix........................................................................................... 36 2.4 Summary............................................................................................................................................... 37 Part II Automatic mixing tools for music ..............................................................38 Chapter 3 Automatic mixing building blocks ...................................................................................39 3.1 Adaptive effects ................................................................................................................................. 41 3.2 Cross‐adaptive methods ................................................................................................................ 43 3.3 Side chain processing...................................................................................................................... 44 3.4 Feature extraction processing..................................................................................................... 45 3.5 Feature extraction ............................................................................................................................ 46 3.5.1 Feature extraction with noise............................................................................................ 47 3.6 Cross‐adaptive processing............................................................................................................ 49 3.7 System stability.................................................................................................................................. 50 3.8 Perceptual processing and technical constraints ..............................................................