Telling Stories with Soundtracks: An Empirical Analysis of Music in Film Jon Gillick David Bamman School of Information School of Information University of California, Berkeley University of California, Berkeley
[email protected] [email protected] Abstract (Guha et al., 2015; Kociskˇ y` et al., 2017), natural language understanding (Frermann et al., 2017), Soundtracks play an important role in carry- ing the story of a film. In this work, we col- summarization (Gorinski and Lapata, 2015) and lect a corpus of movies and television shows image captioning (Zhu et al., 2015; Rohrbach matched with subtitles and soundtracks and et al., 2015, 2017; Tapaswi et al., 2015), the analyze the relationship between story, song, modalities examined are almost exclusively lim- and audience reception. We look at the con- ited to text and image. In this work, we present tent of a film through the lens of its latent top- a new perspective on multimodal storytelling by ics and at the content of a song through de- focusing on a so-far neglected aspect of narrative: scriptors of its musical attributes. In two ex- the role of music. periments, we find first that individual topics are strongly associated with musical attributes, We focus specifically on the ways in which 1 and second, that musical attributes of sound- soundtracks contribute to films, presenting a first tracks are predictive of film ratings, even after look from a computational modeling perspective controlling for topic and genre. into soundtracks as storytelling devices. By devel- oping models that connect films with musical pa- 1 Introduction rameters of soundtracks, we can gain insight into The medium of film is often taken to be a canon- musical choices both past and future.