Department of Computer Science University of Fribourg, Switzerland PHONOGRAPHIC RECORD SOUND EXTRACTION BY IMAGE PROCESSING Thesis Submitted to the Faculty of Science, University of Fribourg (Switzerland) to obtain the degree of Doctor Scientiarum Informaticarum Sylvain STOTZER from Buren¨ an der Aare (BE) and Geneva Thesis N◦ 1534 Imprimerie St-Paul, Fribourg 2006 Accepted by the Faculty of Science of the University of Fribourg (Switzerland), on the proposal of: Prof. B´eat Hirsbrunner, University of Fribourg, Chairman Prof. Rolf Ingold, University of Fribourg, Thesis Supervisor Prof. Martin Vetterli, Swiss Federal Institute of Technology Lausanne (EPFL) Prof. Ottar Johnsen, University of Applied Sciences of Fribourg Prof. Fr´ed´eric Bapst, University of Applied Sciences of Fribourg Fribourg, September 26, 2006 The Thesis Supervisor: The Dean: Prof. Rolf Ingold Prof. Titus Jenny Acknowledgements I would personally like to acknowledge all those who helped me in the development and completion of this research. First, I would like to thank my thesis advisors, Professors Ottar Johnsen, Rolf Ingold and Fr´ed´eric Bapst for their guidance in my studies and my work. They shared with me their expertise and offered fruitful discussions and valuable insights into the different topics required for this research. Their motivation and support was very helpful. I want to thank Professor Martin Vetterli for accepting to be my fourth commit- tee member. I was very pleased that he accepted to read my work. Many famous inventions and discoveries started with a crazy idea. I would like to acknowledge Stefano Cavaglieri and Pio Pelizzari, who got the idea to take pictures of records to archive them and who shared this idea with us. Many people took part in this research project. I am particularly grateful to C´edric Milan and Christoph Sudan for their involvement and their achievements in developing the hardware tools used in this work. Many thanks also to Thierry Fumey for sharing with us his expertise in photography. Finally I would like to thank all the students, assistants and professors who worked on the VisualAudio project for semester work, diploma or bachelor thesis. My colleagues at both the Ecole d’ing´enieurs et d’architectes and the University of Fribourg also offered interesting discussions and helped with practical matters, for which they all deserve thanks. This research would not have been possible without the support of all my family. I particularly thank my wife, Ula, for her patience, understanding and encourage- ment. I would never have been able to achieve this work without her love and unconditional support. And finally, I would like to thank my daughter Sarah who let her father work during long hours and who learned very early to say ”Dad’s gone to work”... Abstract The phonographic record was the only way to store sounds until the introduction of magnetic tape in the early 50’s. Therefore there are huge collections of phonographic records, for example in radio stations and national sound archives. Such archives include pressed discs, which were produced in mass by record companies for com- mercial distribution, as well as direct cut discs obtained by the direct recording with often a great cultural value and available only as single copies. Many records are deteriorating with time. Worse, many records are in an advanced stage of decay and would be destroyed by the movement of the stylus from even the best turnta- bles. Thus, we risk loosing an important cultural heritage, if no alternative playback system is developed. In record players, a needle follows the position of the groove and converts it into an electrical signal corresponding to the sound. This means that the radial displace- ment of the groove contains the sound information. Thus the sound information is visible and it is then possible to extract it by image processing techniques. These observations lead to the VisualAudio project, which proposes an optical system to extract the sound from phonographic records. The VisualAudio concept proposes first to take the record in picture, in order to have a photographic copy of the record and of the sound. This photographic sound copy can be stored for long term archiv- ing. Then the film is digitized using a specially designed rotating scanner, and the image is processed in order to extract the recorded sound. Thus this system can play records without deteriorating them and it is also able to play severely damaged records. This work focuses on the image processing parts of the VisualAudio project. The image acquisition system is thoroughly studied to understand the image formation process as well as all kinds of degradations, which may affect the final sound quality. Based on this analysis, a groove model is proposed, in order to develop dedicated image extraction and signal correction methods. The whole system is then evaluated to point out the strengths and weaknesses of the VisualAudio sound extraction process. R´esum´e Le disque phonographique ´etait le principal support utilis´epourenregistreretcon- server du contenu sonore jusqu’`a l’introduction de la bande magn´etique au d´ebut des ann´ees 50. Ainsi il existe actuellement d’importantes collections de disques phono- graphiques dans les archives nationales, les stations radio et les phonoth`eques. Ces archives poss`edent des disques press´es, produits de fa¸con industrielle pour la dis- tribution commerciale, ainsi que des disques grav´es, obtenus par l’enregistrement en direct de conf´erences ou d’´emissions radios. Les disques grav´es sont ainsi des copies uniques, qui ont souvent une grande valeur culturelle. Mais beaucoup de disques se d´et´eriorent avec le temps, et certains sont tellement d´egrad´es, qu’ils se d´ecomposeraient au seul contact d’une aiguille de tourne-disque. Nous risquons ainsi de perdre une importante partie de notre patrimoine culturel, si nous ne trouvons pas un nouveau syst`eme de lecture pour ces disques. Dans les tourne-disques, une aiguille suit le parcours du sillon le long du disque et convertit le d´eplacement lat´eral en un signal ´electrique correspondant au son. Cela signifie que le d´eplacement lat´eral du sillon contient l’information sonore. Ainsi le contenu sonore est visible et il est donc possible d’extraire le son par des techniques de traitement d’image. Ces observations sonta ` la base du projet VisualAudio, qui propose une m´ethode optique pour extraire le contenu des disques phonographiques. L’id´ee du projet VisualAudio consistea ` prendre tout d’abord une photographie du disque, afin d’en conserver une copie pour un archivagea ` long terme. Ce film est ensuite num´eris´e`a l’aide d’un scanner rotatif con¸cu sp´ecialement dans le cadre de ce projet, et l’image num´erique est analys´ee afin d’en extraire le contenu sonore. Ainsi VisualAudio permet d’extraire le son des disques sans contact avec leur surface. Ce syst`eme est ´egalement capable d’extraire le son de disques gravement endommag´es. Ce travail de th`ese se concentre sur la partie traitement d’image du projet Vi- sualAudio. Le syst`eme d’acquisition d’image est tout d’abord analys´eend´etail, afin de comprendre le processus de formation d’image ainsi que tous les types de d´egradations qui peuvent affecter la qualit´e de l’image et du son extrait. Sur la base de cette analyse, nous proposons un mod`ele de sillon qui est utilis´epourd´evelopper les m´ethodes d’extraction du son et de correction du signal. Le syst`eme est ensuite ´evalu´e afin de comprendre les avantages et inconv´enients du syst`eme de lecture et d’archivage VisualAudio. Contents 1 Introduction 1 1.1 The VisualAudio project ......................... 1 1.2Objectivesofthisthesis......................... 4 1.3Relatedworks............................... 5 1.3.1 Soundonfilm........................... 5 1.3.2 Signalextractionbyimageprocessing.............. 6 1.3.3 Phonographicopticalplaybacksystems............. 8 1.4Overviewofthisdissertation....................... 11 2 Phonographic recording 13 2.1History................................... 13 2.2Mechanicalrecording........................... 14 2.3Recordingprocess............................. 16 2.4Discmanufacturing............................ 16 2.5Recordmaterials............................. 17 2.5.1 Shellac............................... 17 2.5.2 Acetate/lacquer.......................... 17 2.5.3 Vinyl................................ 18 2.6Recordingspeed.............................. 18 2.6.1 78rpm............................... 18 2.6.2 33rpm............................... 19 2.6.3 16rpm............................... 19 2.6.4 45rpm............................... 19 2.7Recordequalization............................ 20 2.7.1 Recordingcharacteristics..................... 20 2.7.2 Playbackcharacteristics..................... 20 2.7.3 Equalization............................ 20 2.7.4 Equalizationtransferfunction:theRIAAcase......... 22 2.8 Reading distortions and undesirable effects ............... 23 2.8.1 Atrecordingandmanufacturing................. 23 2.8.2 Atplayback............................ 24 2.8.3 Recordstorageandmaintenance................. 25 2.9Recordandgroovegeometry....................... 25 2.10Dynamicrangeandsignaltonoiseratio................ 27 VI CONTENTS 3 Acquisition system 29 3.1Films.................................... 29 3.1.1 Film constraints for VisualAudio ................ 29 3.1.2 Filmstructure........................... 30 3.1.3 Sensitometry / response of a photographic film ........ 30 3.1.4 Exposuretime..........................