Synchronization and SMPTe Time Code n film and video two different media, visual and auditory, must be experienced together. In order to create sound that works seamlessly with visuals, you need I. two playback systems-one strictly for audio and another strictly for video. These two systems must be capable of playing back together in perfect time. The process of · making two or more distinct audio or visual systems play back together is called synchronization. Synchronization is a basic principle of working with audio for video and film. Any alter­ ation of the audio must be done in the conteJKt of the visual image. It wouldn't be pos­ sible to establish and control the relationship between the two media without synchronization. A thorough mastery of the concepts and practices of synchronization will become invaluable as you get deeper and deeper into audio post-production. This chapter is devoted to synchronization theory, techniques, and real-world practices that you will use every day when creating sound for films and video. Synchronization Basics As I said, synchronization is the process of keeping two or more audio or visual systems running at the same speed and with the same relationship to each other. Keeping audio in sync with video means playing both together from the same point without either one moving faster or slower than the other. It is crucial to do this with a high degree of accuracy and repeatability, as every single edit and mix move you perform must be matched up with a certain action on-screen. Two things are necessary for accurate sync. The first is a positional reference, or the "where-in-time." The second is a playback rate, or "how fast." I like to use the analogy of cars on a two-lane highway. The positional references are the mile markers on the side of the highway that tell you where on the road you are. The playback rate is the speed of the car, for instance 65 mph. If you know what mile marker you're at and how fast you're traveling, it's easy to get another car moving along in sync with your car. Think of the audio as one car and the video as a second car in the next lane. If either car speeds up or slows down, the two cars will no longer be in sync. The drivers of 43 44 Pro Tools for Video, Film, and Multimedia, Second Edition Chapter 3 Synchronization and SMPTE Time Code 45 these two cars must be in constant communication with one another In order to maintain the correct speed and position. With Pro Tools and digital audio, this COlTl.II1U­ nication is handled in two ways-one for positional information and the other forr speed reference. Digital audio is based on a sampling frequency that determines how often the analog audio is digitally sampled and turned into numeric information. The source of thi ~ s sam­ pling frequency is referred to as the digital clock and is required for proper operat ion of any digital audio device. If the sampling frequency or clock speed is changed, the speed at which audio is played back will change accordingly. If the clock speed increas es, digital audio will play back faster and higher in pitch. If the clock speed slows, aud io will play back slower and lower in pitch. Thus, the sampling frequency can control t he speed of the audio. This is how the speed reference for synchronization is communiicated when using Pro Tools. Positional Reference You must assign unique numbers to each segment of audio and video in a projrect in order to know exactly where you are at any given point in either medium. For instance, Figure 3.1 An SMPTE readout displaying 1 hour, 11 minutes, 45 seconds, and 24 frames. bars and beat numbers can be used to identify locations in a piece of music. In wisua l systems, each frame of film or video must be given a unique number. This is aCiCOlTI­ Several standard frame rates for film and video are commonly used. They are as follows: plished through the use of SMPTE (Society of Motion Picture and Television Engirneers) time code. • 24 frames per second for film Every frame of film or video in an AV project may have a specific SMPTE time code • 25 frames per second for European or PAL video number assigned to it. SMPTE time code is the standard method by which you get ]posi­ • 29.97 frames per second for NTSC color video tional references in audio, film, and video (see Figure 3.1). SMPTE time code represents time in an eight-digit number with colons separating each time value for hours, minlUtes, • 30 frames per second for audio only or old B&W television seconds, and frames (OOh:OOm:OOs:OOf). A frame of SMPTE can be as small as 1/60t h of • 23.976 frames per second for HD video that converts to NTSC video a second or as large as 1I24th of a second, depending on how fast the film or vidreo is moving. I'll discuss the different frame rates in more depth in just a bit. For now, Ulllder­ • 59.94 frames per second for HD video that converts to NTSC video (29.97 x 2) stand that SMPTE time code numbers are positional references-they tell you where (often mistakenly referred to as 60fps) you are in a soundtrack or film. • 60 frames per second for HD video at its highest standard speed Playback Rate In digital audio, the rate at which sound is played back is not measured in frames but in The second thing you need to know to achieve accurate synchronization is how fast the samples. Each sample of audio is a unit of information, or slice of time, much in the frames are moving. In my highway analogy, this relates to the speed of the two cars. In same way that each frame of film or video is a single snapshot of time, although an film and video, the playback rate is the frame rate. In digital audio, the playback rate is audio sample is exponentially smaller in duration. The rate at which samples are played the sample rate or sampling frequency. The speed relationship between the audio and back (the sample rate) is usually expressed in kilohertz (abbreviated kHz); multiplying video must remain the same in order to be in sync. Otherwise, the audio will either slip the kilohertz number by 1,000 results in the number of samples per second-for exam­ ahead of or behind the video. ple, 44.1kHz represents a rate of 44,100 samples per second. Chapter 3 Synchronizatio~ and SMPTE Time Code 47 46 Pro Tools for Video, Film, and Multimedia, Second Edition There are several sample rates commonly used in audio production. They are as follows: the edge of the film, placing notes and then numbers to help the editing process. This evolved into the use of "feet and frames" as a mtasuring tool. Each second of film • 44.1kHz: The standard music sample rate. This is used to create the commercial contains 24 individual frames. A foot-long piece Gf 35mm film has 15 frames in it. music CDs people buy everyday. Using feet and frames with edge numbers made com]> licated editing much more feasible. This was the first time code, and it was used while editing film on machines similar to • 48kHz: The video and film standard. Professional digital video formats such as Sony's Digital Betacam use this sample rate. the one shown in Figure 3.2. • 88.2kHz: Hi-res music sample rate for audio CDs. Easily converts to 44.1kHz. • 96kHz: Hi-res sample rate for music and video; it is a multiple of 48kHz and is easier to convert. • 192kHz: Super hi-res sample rate for demanding recordings such as orchestral, jazz, and archival recordings. Currently impractical for film and video due to large file sizes, but may be used in the near future. To sum up, in order to synchronize audio and video, you must know the positional information, in the form of SMPTE time code and playback speed. Audio and video must start from the same position and play back at the same rate in order to maintain perfect sync. This might sound simple, but as you will see, it can get quite complicated once you consider all the variations in SMPTE formats and playback rates. Origin and History of Time Code In order to better understand the variety of SMPTE formats, let's take a look at how time code developed and eventually turned into the SMPTE format. Film In the early days of film, scenes were filmed from beginning to end with one camera and in one complete take. Film scenes paralleled the way plays on stage were viewed. The . only editing that would take place would be to cut all the scenes together in the correct Figure 3.2 A Moviola machine that is still used to edit fil ll1 and magnetic audio film together using order. The high-speed edits and rapid-fire cuts that are the norm today were simply not the system of feet and frames for counting. used in early film. Gradually, as filmmakers began to explore the new medium, new techniques of filming caught on. Soon, scenes were filmed in multiple takes using different camera angles, in NTSC versus PAL order to capture the action in new and exciting ways. These new filming techniques With the advent of television and then videotape lecording, new editing systems devel­ made the editing process much more complicated, as each scene had multiple edits oped that allowed video to be edited from multiple-source tapes onto a final master throughout.