FILE-BASED WORKFLOWS High-qualityVideo processing in the file domain Bruce Devlin AmberFin File-based workflows require more format conversion than workflows in the SDI domain. The effects of interlace and scaling on file-based images is greater than their effects in the SDI domain, yet most file-based processing focuses on raw speed rather than media quality. Better media quality reduces the consumer churn rate and makes the end customers more loyal. This article shows what can be achieved today in the file domain with a little workflow care and some good deinterlacing and scaling. In the middle of 2005, it was a real achievement to make a file-based workflow deliver business benefits. At the time, there was very poor interoperability between encoders, transcoders and edit platforms. Many systems that were built went for a “one-stop shop” approach, sourcing all the components from a single vendor, basing the system around MPEG-2 or QuickTime and working in SD. In the middle of 2005, few pople had heard of YouTube. “Just making the system work” was considered a success. Let’s fast forward now to the middle of 2010. There are a great many file-based workflows delivering business benefits. YouTube, iTunes, NetFlix, DailyMotion and other video portals are doing well. Tape is still the main mechanism for the interchange of programmes, but many content providers are switching to a file-only model. The success of formats like MXF and QuickTime have enabled this new way of working, and the abundance of new distribution platforms – whether they are broadcast, cable, internet, IPTV, web TV or mobile – provide a commercial incentive to create more content in 2010 for less money than was spent in 2009. Interoperability issues are still with us, but “just making the system work” is no longer a good goal for a file-based workflow. Delivering high-quality video at low bitrates has been shown to reduce churn in IPTV and to increase viewers on the web (try Googling “video quality reduces churn” for hundreds of articles). It is also a way in which individual broadcasters and publishers can differentiate themselves. This article looks at the challenges of keeping the quality high, the bitrate down and why it is even more important to process the signal well in the file domain when compared with the traditional SDI domain. Codecs, converters, interlace and standards Before jumping into the signal processing technology, I will first define some terms. Many of these terms are used in a very loose fashion within the industry. This article will use them as follows: EBU TECHNICAL REVIEW – 2010 Q3 1 / 7 B. Devlin FILE-BASED WORKFLOWS Codec: A video or audio or data encoder or decoder. MPEG-2 is a codec whereas MXF is not. H.264 is a codec and so is Dolby D. Wrapper: A multiplexing or interleaving specification that allows different video, audio and data codecs to be synchronized within a single file. MXF, GXF, Quick- Time, Windows Media, AVI, PS and MP4 are all wrappers. Delivery Profile: A combination of codecs and wrappers configured for delivery to a certain specification. Today, this is often determined by the target device: e.g. a particular playout server. As the number of delivery formats increases, this is more likely to become an Application Specification such as AMWA’s AS02 or AS03. Format Converter: A format converter changes the spatial resolution of the video and may modify the frame rate by some integer ratio. For example, a format converter may convert SD (720 x 576 x 50i) to HD (1280 x 720 x 50p). In the USA, a format converter may insert 2:3 cadence to convert 23.98 fps (frames per second) material to 29.97 fps material. Standards Converter: A standards converter changes temporal resolution of the video and may also change resolution. For example a typical conversion is from US mate- rial (29.97 fps at 480i lines) to European video (25 fps at 576i lines). In the file domain, conversion to filmic rates and 1080 50p / 1080 60p is also possible. Interlace: Interlace can be thought of as a very old compression tool. Every other line is displayed during the first half of the frame period. The remaining lines are displayed during the second half of the frame period. Virtually all standard-definition video is interlaced, as is a very large amount of HD material. Like any compression scheme, interlace has it benefits (flicker reduction) and its problems (vertical-temporal aliasing, poor digital compression characteristics). It is worth noting that in 2010, the majority of source content is still interlaced but many display devices that consumers use, including flat-screen TVs, PCs, iPads and mobile devices, are progres- sive. Comparing the SDI domain with the file domain In the SDI domain, video signals required a physical transport and so the number of Format Conver- sions that were needed were limited to the conversions between the formats that could be carried on a SMPTE 292M physical link. During the planning phase of a facility build, the Format Converter devices could be planned and purchased to maximize quality and minimize the number of conver- sions. High quality converters featured good deinterlacers and poor quality converters didn’t. In the file domain, there are hundreds more formats that need to be supported but, until recently, there have not been any good software deinterlacers on the market to keep the picture quality high. Deinterlacing – why you need it A frame captured from a clip of video is shown at the top of the next page. This frame shows the typical “interlaced look” that should be very familiar. To our eye, it is quite clearly a skateboarder who had moved between field 1 being scanned and field 2 being scanned. To a piece of software analysing a portion of the scene, it sees horizontal stripes. The software must do a lot of calculations to work out if this pattern was intended to be stripes (i.e. vertical detail) or whether this pattern was due to motion (i.e. poor temporal sampling). This problem is known as EBU TECHNICAL REVIEW – 2010 Q3 2 / 7 B. Devlin FILE-BASED WORKFLOWS vertical-temporal aliasing and, due to the very low frame rates of video compared to the speed of objects moving across the screen, it is impossible to get it right all the time. Let’s consider the case of compressing an SD video signal. Assuming no scaling takes place, codecs like MPEG-2 and H.264 have knowledge of interlace built into them and they do a reasonable job. MPEG-1 and other 1st generation codecs do not know about interlace and, when used incorrectly, can permanently stamp in the interlace artefacts so that they can (almost) never be removed. What happens when scaling takes place? Well-designed soft- ware will scale each field independently and the pictures will look OK. Poor-quality scaling software will frame blend and then scale the image. From a signal-processing point of view, this is a disaster – but it happens very frequently today. One classic example provided to AmberFin’s engineering department was a clip from a movie that was originally shot at 24p (24 progressive frames per second). By looking at the artefacts, we believe the clip we were asked to process had been converted to 29.97 fps by inserting a 2:3 sequence and converting to a standard-definition MPEG-2 format. The MPEG-2 file had then been frame-blended and badly scaled to 720p (probably 59.94 fps) and encoded as progressive H.264. By performing this scaling and coding, the original interlace had been “stamped into” the picture like a watermark. The poor-quality scaler had not preserved the original interlaced line structure and it was now impossible to reverse the process. Spotting quality clues in a test pattern Many of today’s encoding and trans- coding solutions are geared towards raw throughput rather than keeping the integrity of the original picture. In almost all industries, increasing the quality of a product increases its attractiveness to the end customer. This is true for free-to-air services as well as paid-for services. In order to get better throughput, deinterlacers and scalers are compromised in quality in order to run fast. Let’s look at some test patterns to see the effects of this on images. The humble test pattern was once the mainstay of checking an Test Pattern 1 – poor quality scaling and deinterlacing analogue transmission system. However, it’s an unfortunate recent trend that the testing of equipment before purchase happens less than it did in the past. In fact, modern test patterns are still very useful because they can reveal what is going on behind the scenes when an image gets filtered (a scaler, deinterlacer and a standards converter are all different examples of a filter). We at AmberFin have used a test pattern that has kindly been provided by OmniTek to reveal some obvious and some subtle effects of poor deinterlacing. EBU TECHNICAL REVIEW – 2010 Q3 3 / 7 B. Devlin FILE-BASED WORKFLOWS Hopefully, when you are reading this article, the PDF file has not been over-compressed or scaled with a poor-quality image filter on your screen. [Ed.: the two test patterns are duplicated at higher quality on a single page, inserted at the end of this PDF.] If you look at Test Pattern 1, you will see that there are some obvious artefacts that have been introduced as a result of the poor performance of the deinterlacer.