Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 1 of 21 National Association of Broadcasters
ENGINEERING HANDBOOK 10th Edition
Editor-in-Chief EDMUND A. WILLIAMS
Associate Editors GRAHAM A. JONES DAVID H. LAYER THOMAS G. OSENKOWSKY
National Association of AMSTERDAM • BOSTON • HEIDLEBERG • LONDON NEW YORK • OXFORD • PARIS • SAN DIEGO N;tB· SAN FRANOSCO • SINGAPORE • SYDNEY • TOKYO u BROADCASTERS Focal Pres.5 is an imprint of Elsevier
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 2 of 21 Senior Acquisitions Editor: Angelina Ward Publishing Services Manager: George Morrison Senior Project Manager: Paul Gottehrer Assistant Editor: Doug Shults Marketing Manager: Christine Degon Veroulis Compositor: MuJtiscience Press, lnc.
Focal Press is an imprint of Elsevier 30 Corporate Drive, Suite 400, Burlington, MA 01803, USA Linacre H ouse, Jordan Hill, Oxford OX2 80P, UK
Copyright © 2007, Elsevier. All rights reserved.
Copyright 1935, 1938, 1946, 1960, 1975, 1985, 1992 Nalional Association of Broadcasters. Ali rights reserved. Printed in the United States of America. No part of this book may be reproduced in any form or by any means without permission in writing from the National Association of Broadcasters, 1771 N Street, NW Washington, DC 20036-2891.
This publication is not intended to be a substitute for the professional advice of an engineer or attorney. While the Editors have endeavored to include information current at the time of publication, statutes, rules, regulations and engineering knowledge will undoubtedly change following publication. It is the Reader's responsibility to obtain the most up-to-date information on any topic.
NAB and the contributors to this publication make no representations and disclaims all warranties, express or implied, (except as otherwise provided by law) regarding the information contained herein, including without limitation the implied warranty of fitness for a particular purpose and the implied warranty of merchantability. To the extent permitted by law, neither NAB nor any contributor to this publication shall be liable to any other person or entity for any damages, including without limitation any direct, indirect, special, consequential, exemplary or punitive damages under any theory of law.
For information on other publications available through the National Association of Broadcasters, call 202-429-7424, 800-368-5644 or visit the NAB Store website at http:/ / www.NABstore.com.
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior written permission of the publisher.
Permissions may be sought directly from Elsevier's Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, E-mail: [email protected]. You may also complete your request online via the Elsevier homepage (http:/ / elsevier.com), by selecting "Support & Contact" then "Copyright and Permission" and then "Obtaining Permissions."
@ Recognizing the importance of preserving what has been written, ·'-/ Elsevier prints its books on acid-free paper whenever possible.
Library of Congress Cataloging-in-Publication Data Application submitted British Library Cataloguing-in-Publication Data I A catalogue record for this book is available from the British Library.
ISBN 13: 978-0-24-0-80751-5 ISBN 10: 0-240-80751-0 ISBN 13: 978-0-240-80982-3 (CD-ROM) ISBN 10: 0-240-80982-3 (CO-ROM)
For information on all Focal Press publications visit our website a t www.books.elsevier.com
0708091011 54321
Printed in the United States of America
Working together to grow libraries in developing countries www.dsevier.com I www.bookaid.org I www.sabre.org
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 3 of 21 CHAPTER 5.24
Closed Captioning Systems
JEFF HUTCHINS* ALAN LAMBSHEAD** Accessible Media Industry Coalition Evertz Microsystems Pittsburgh, Pennsylvania Burlington, Ontario, Canada
INTRODUCTION separate data packet accompanying the audio and video of a digital program. Captioning, according to the Accessible Media Indus Captioning is distinct from subtitling in that cap try Coalition, is "the process of converting the audio content of a television broadcast, webcast, film, video, tioning includes aural information other than just dia CD-ROM, DVD, live event, or other productions into logue, such as sound effects, music effects, and text, which is displayed on a screen or monitor." Cap indications of who is speaking, all intended to aid the tions not only display words as the text equivalent of viewer who is unable to hear the soundtrack. The orig spoken dialogue or narration, but also include speaker inal target audience for captioning was primarily peo identification and sound effects. It is important that ple who are deaf or hearing impaired (about 20 the captions be million in 1980). The market has since expanded to include people learning English as a second language; ; ' Synchronized and appear at approximately the those learning to read, especially students with read f sam~ time as the audio is available; ing disabilities; people in noisy places (like bars and 1 ' ~qmvalent and equal in content to that of the audio, airports) or quiet places (like hospitals and spas). Cap mcluding speaker identification and sound effects; tions generally appear in the lower portion of the tele ' Accessible and readily available to those who need vision screen and vary in size in proportion to the size them. of the television screen. The caption characters are sized to be easily visible, typically white letters against . Ca~tions may be displayed on or adjacent to the 11 . deo unage. Open captions are a permanent part of the a black background (for analog television), and usu ~age '.fild cannot be turned off. Closed captions are ally do not obstruct essential parts of the picture. Cap deansm1tted with the audio and video, but require a tions for digital television (DTV) can have a wider thcoder to ~etect, decipher, and display the captions so range of font styles, sizes, and colors than those for 10atonlyv1ewers who wish to see the captions will do NTSC transmissions (see Figure 5.24-1). ~i ~~data for closed captions is transmitted with the Closed captioning may be added in real time to a bl evi~ion_ program, either on line 21 in the vertical live program or may be added before transmission as ~g interval (VBI) of an analog program, or in a part of postproduction for a prerecorded program. lf&,p .. )/f:. ~ ,J~~~Utchins updated the analog captioning portion of this chapter drawing on material first published in the 9th edition of the NAB Engi ''A!anandbook, Chapter 5.13, "Closed Captioning and Extended Services," by Amnon Salomon and Gerald Freda. ~Pro ~~bshead adapted the DTV captioning portion of this chapter from material first published in the NAB Broadcast .Engineering Confer cee mgs 2004, "Implementing Closed Captioning for DTV," by Graham Jones.
NAB ENGINEERING HANDBOOK Copyright© 2007 Focal Press. 1435 All rights of reproduction in any form reserved.
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 4 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY
This act required that, effective July 1, 1993 sion receivers with picture screens 13 inche ' a]] televj. must be equipped to display closed captios or greater sion transmission, and it required the FC~ed televj. rules to implement this requirement. to enact FCC Report and Order 6 of General Docket adopted April 12, 1991, and released to the 91bl.was April 15, 1991. The Order became effectiv~u Uc on 1993. The Order amends Part 15 of the FCC' July 1, adding a new Section 15.119 to set out the F~tles by for captions and caption decoders. The hi hli Rules this section as it relates to NTSC analog tele~i .ghts of tioning are as follows: sion cap- • Effective July 1, 1993, TV receivers 13 inches FIGURE 5.24-1 DTV caption data processed by a larger must have caption decoders; or decoder and displayed as text across the lower portion • Closed caption information to be carried in lin of the picture. of field l; e 21 • Decoders to have user-selectable Caption displ modes. Text display mode is optional; ay A BRIEF HISTORY OF CLOSED • Caption and Text modes may contain data in eith CAPTIONING of two operating channels, usually referred to as er ,.,. Cl, C2, Tl, and T2; The Public Broadcasting Service (PBS) developed closed captioned technology for NTSC analog televi • Receivers must decode at least Cl and C2 captions; sion during the period 1973-1979 with funding sup • Captions to be displayed in "boxes" on the screen port from the federal government (Department of within the Safe Title Area defined by SMPTE 27.3 Health, Education, and Welfare). Field test transmis (now replaced by SMPTE RP218-2002); sions were conducted on all aspects of caption genera tion, encoding, decoding, and display features of the • Caption decoding circuitry must be tolerant of ~ service. cable security systems that alter line numbering, etc. During those years, a small number of programs were open-captioned by The Caption Center at PBS The complete text of the FCC Report and Order station WGBH and carried on PBS. The newly cre (R&O) may be obtained from the U.S. Government ated National Captioning Institute (NCI), in cooper Printing Office. The text was also published in the Fed· ation with the ABC, CBS, NBC, and PBS networks, eral Register, Vol. 56, No. 114, p. 27200. The FCC Rules launched the closed captioning service in March and Regulations are contained in the Code of Federal 1980 with approximately 16 hours per week of cap Regulations, Part 47, Telecommunications. See Chapter tioned programming. The first consumer product 1.2 of this handbook, "Broadcast-Related Organiza· containing the decoding feature, called TeleCaption®, tions and Information," for information on contacting was sold by Sears, Roebuck and Co. and was a sepa tlfe U.S. Government Printing Office and other organi· 1 rate box that worked with a standard television. h~tions mentioned in this chapter. . . . .
FCC Rules and CEA Standards Update Since Passage of Decoder Circuitry Act The line 21 captioning data signal of NTSC analog Since the passage of the Decoder Circuitry. ~ct, th~ television signals is protected from interference from FCC has ruled several additions. These _ad~1tio~~n· documented with other technical details m a any other VBI service, test signal, or spillover from 608 active video under FCC Rules and Regulations, Sec sumer Electronics Association document CEA- th' tion 73.682(a)(22), adopted in 1976. These rules also titled Line 21 Data Services (formerly ~~-608)iJ~oc~ established the transmission standards for captioning time of writing (2006), the current revision oft 5 and list the uses of the data channel. ument is CEA-608-D. . . has Following are highlights from rules associated with The regulation and use of line 21 for cap~?nmf fea the Television Decoder Circuitry Act of 1990 that been expanded to include both fields. Ad~i~o~:i Ian· amended Part 15 of the FCC rules (Radio Frequency tures have been added to deal with a~ditio deliver Devices, which relate to television receivers). guages. Also, within field 2 is a capacity to. addi additional captioning services (C3, C4, T3, T4! ~!udes, tion to extended data services (XDS). X0 5 U1 Television Decoder Circuitry Act of 1990 16 2006, rna~ 1The entire Part 15 regulations, as updated February fc~.gov ;oet1 The U.S. Congress passed the Television Decoder Cir also be downloaded as a PDF file from http:/ 1WWW· cuitry Act of 1990 (Pub. L. 101-431, 104 Stat. 960 (1990)). info/ rules/partl5/ partlS-2-16-06.pdf.
1436 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 5 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS
t limited to, content advisory (V-chip ), Time The roll-up style is most often used for captioning of vut is 110 d source of transmission. live programs. From one to four rows of text display 01DaY:rsc television, the Accessible Media Industry the captions in a scrolling manner. f~r. recommends using Cl (also called CC-1) for The paint-on style is sometimes used in analog sys coalitl.011ary captioning service, and C3 (or CC-3) on tems because it allows characters to be displayed the pr~or any secondary captioning s.ei:rice. ~and immediately, avoiding the transmission lag of pop-on field 2!iJllitations make it extremely difficult, if not style that occurs thanks to the 60 characters per second 11idth 'ble to use Cl and C2 simultaneously for two data rate of line 21. Paint-on can also be used to create . poss1 ' . . an 1 captioning services. special effects, such as a simulated "reveal" of text ;epara e added to a displayed caption. . 'tal Television Captioning 01g1 CEA-608 Captions 1997, the next generation of closed captioning ~!~\al television was adopted. The new Consumer CEA-608 captions (as well as material displayed in 1or trginics Association caption standard, CEA-708, 0 line 21 Text Services) may have any of several 8~d vigital Television (DTV) Closed Captioning, was attributes in addition to block monochrome charac u,:elo ed by an industry group composed of caption ters. Attributes include upper- and lowercase, six dif de . epproviders, receiver manufacturers, broadcast- :ervIC · ferent colors, italics (or slanted text), underline, and · and encoder manufactu-:ers. It en~b~es impr~"'.ed flash. The caption attributes are determined during :~ed captioning to ~e provided for d1g1t~l ~e.lev1s1on caption or text authoring and are communicated to the ansmissions, including both standard defrmtion (SD) decoder using special control codes in the data stream. . ~id high definition (HD) ~ele".ision, while cont~nuing The display of CEA-608 captions consists of 15 rows i to provide~ method of delivering CEA-608 captions to with up to 32 monospace characters per row. Decoder allalog receivers. controls permit the viewer to select Captions or Text. DIV closed captions (known as DTVCC) have In the Captions mode, a maximum of four rows is 11eatly enhanced formatting and display capabilities used onto which each caption pops on when prere ~ompared to line 21 captions, with up to 63 services corded captions are received and rolls up when the f€I program; eight independently controlled display captions are live. The four rows can appear anywhere ~indows; and an extended range of characters and on the 15 displayable rows which occupy most of the multiple fonts, sizes, and background and character screen area. The first row starts at line 43 in each field. colors and edges. The standard specifies how caption Each row occupies 13 lines of a field scan or 26 lines of information is to be coded and processed, minimum the 525 lines. Each row of characters is displayed implementation recommendations for DTV closed within a black surround box to enhance the readability caption decoders, and recommended practices for cap against the normal video background. The box tion encoder and decoder manufacturers. It also extends one character position to the left of the initial . requires that decoders give users control over caption character in each row and one character to the right of font, color, size, and location that may override the . each row. rarameters as transmitted. During the transition from analog to digital broad casting, analog NTSC and DTV transmissions will CEA-708 Captions coexist, as will analog and digital cable distribution ~st~ms. In addition, analog and digital television pro The display of one CEA-708 caption service consists of ~u:~~n and distribution systems may coexist in many up to eight windows of variable dimensions and posi iacili~es and networks, frequently with both SD and tions that contain the text. Caption text may be pro HD video formats. This situation creates added com portional or monospace, with or without serifs, in up ;lexity for closed captioning as well as other aspects to three character sizes. Up to 64 colors are allowed, as ~system implementation. A discussion of how to well as a range of transparency levels, for characters, mplement DTVCC, together with a summary of the character edges, and window backgrounds. Support ~levant FCC rules, is in a later section of this chapter. for these features among decoders varies, however, .,,;, ,. because FCC rules require decoders only to support a subset of the features defined in CEA-708. ·~···~rhr DISPLAY FORMAT . ~alog CEA-608 captions can operate in any of three CAPTION DATA ENCODERS ~rent styles: pop-on, roll-up, and paint-on. DTV Jir ·'.08 captions also allow text to scroll in multiple Encoding is the process of inserting the caption data ections. into the VBI of an analog television signal or into the :ec~ Pap-on style is used when the captions are pre appropriate data packet of a DTV signal. The data ~o ed: Captions are transmitted to a nondisplayed contains the caption text in addition to positional :apti ry m advance of the time they are to appear. The instructions and display attributes (for instance, ~tr on d~ta. stored in this memory is then displayed color and italics). The encoder is placed in the video · ansmiss1on of a single display code. path of the program to be captioned. There are two
1437
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 6 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY types of the line 21 VBI encoders: smart encoder and Prerecorded Captioning simple encoder. Prerecorded captioning (off-line or nonlive . involves the preparation of closed captio captioning) Simple Encoder grams that have been recorded prior to th n~ for pr0- The captions are created and then mer eeir t~lecast. The simple encoder was the first type of line 21 video signal and a closed captioned vide~t d With the encoder. It unconditionally strips any data already of the program is made. A typical block dia ape Version encoded on line 21. It generates the line 21 data signal, prerecorded captioning. process is showfr~m ~f the to be inserted on a video signal, from caption data 5.24-2. Th.e ~ain step_s in theyrocess are as foll Figure received at an RS-232 serial input. This encoder is used A captiorung facility receives a time-cod d ows. exclusively in postproduction applications for insert a program master either on videotape or ae c~py of ing pre-authored captions on program material where video file. A caption author reviews each ss a dig1tal there is no previously recorded caption data. The sim scribes the soundtrack, and formats the trancsen~, ~an- . · · cnpt int ple encoder cannot add captions to a video signal discrete captions usmg a personal computer S 0 already containing line 21 data. quality-control checks are performed, induct: everal ing, grammar, syntax, timing, and screen po~% sp~U of the captions. The captions are saved as a fil 0ni:ig Smart Encoder their corresponding display time codes. The ~~ 1 th 15 The smart encoder is used to insert caption data into then transferred by any of the usual means f fil line 21 of fields 1 or 2 of the VBI. Under user controls, transfer (LAN, WAN, modem, or by e-mail ctf~ e flash memory) to the location where the progra~ .' ~r it can strip all previously encoded data contained in 15 0 have the captions added or "encoded." the video, or intelligently leave one or more data chan nels intact while inserting new data. (Note that all pre The recorded program and caption data are th viously encoded data that is to be passed through merged, using the time c~des to trigger the captio~~ intact will actually be stripped and reinserted typi to produce a closed captioned master video signal cally two video frames later.) which is typically recorded but may be transmitted live to air. Captions that are produced in advance are The smart encoder typically receives new data for displayed on a caption-equipped receiver as pop-on encoding through a serial data port or via telephone captions, each one complete and timed to coincide line. Using a smart encoder, locally produced Caption with the spoken dialogue on the screen. and Text Mode Services may be added to an already closed captioned video program or if noncaption text is contained on line 21. If captions are present on the Live Captioning incoming program, noncaption data may be inter leaved into the gaps between the captions by the Live captioning involves the addition of caption data to smart encoder. the television signal at the time of a live transmission · Smart encoders are used in the process of inserting or broadcast. Examples of live captioned program· real-time captions, live-display captions, and captions ming include news programs, sporting events, lire derived from scripts generated by newsroom comput events (such as meetings, news conferences, and ers (also used to feed teleprompters), and can also be award shows), and special bulletins or reports. There used by off-line captioning systems that control the ff· are three methods of captioning live programs: r~al· timing and preparation of data (including the genera / time, live-display, and newsroom computer. A typKal tion of all control codes needed to manage the caption block diagram of the live captioning process is shown display). in Figure 5.24-3.
Digital Encoders Real-Time Captions Real-time captions are created and transmitted silll:ul· Since the advent of DTV captions, encoder manufac taneously. There are several methods of gene(a~ng turers have produced a new line of encoders to deal real-time captions: (1) by a court reporter -~~ with analog and digital broadcasts. See the section on called a "stenocaptioner") using a stenotype ors~ a implementing DTVCC for more information on vari machine that deciphers phonetic short~and !(Feho0~. ous types of digital encoders. special keyboard; (2) by a "voicecaptioner w eats tens to the soundtrack and simultaneously ~e~ has each word into a speech-recognition syste~ to~ ut; AUTHORING CAPTION DATA been modified to generate a formatted caption tin~ to or (3) by a speech-recognition syste~ attemp 0 Several techniques are employed to create caption transcribe directly from program audio. data based on whether the broadcast is live or prere corded. There are also variations within the live and Steno captioning an prerecorded captioning technologies. . tioner c With a stenotype machine, the real-time rp etic codes key in up to 260 words per minute. The P on
1438 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 7 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS
Videotape Video Recorder Video Monitor
Time-code Authoring Reader Video with Caption Data ""*r ,~1li Personal Character ·~~~ Computer Caption Generator JW. I' £gA Data ;::, · Diskette········-- ~::·~;Modem ------·------
'~
i~_i:f!ft Encoder VTRVideo ·.~ Personal Computer with Time-code Sub master Caption Reader Video with Encoding Data Caption Data
Time-code VTRVideo Monitor with Captions Master Time-code - Displayed Corrected Video Prerecorded captioning system. Top: caption authoring; bottom: encoding.
Live )I Video I Video I )I Monitor Authoring Stenotype Audio and Local Machine Monitor Video with Encoding Caption Data
'.&:' Personal Smart ~ Computer Encoder '1:'~I.t·/ ._, Caption ...,,-.~ Data ·.~ Modem ~ ·······------Modem ------
Smart Off-site Transmitler Encoding Caption Encoder Video with Data Caption Data ',;!£ j ' ~ .. l: ~' Monitor with i~~ Master Video .. Captions l1' Displayed I I FIGURE 5.24-3 Live captioning system. Top: authoring and local encoding; bottom: off-site encoding.
1439
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 8 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY
are translated into English words by a computer that terns will improve greatly in coming year has been programmed with the phonetic codes and a these systems are generally far less ace~' at this tiin dictionary customized to the individual court reporter. ~arther be~d pr?gr~m audio than the s~:te anct la; From the computer, the words and caption control mg and v01cecaphomng systems. nocaption. codes are sent via a data circuit to a caption encoder where they are encoded into the television signal as Live-Display Captions real-time, or roll-up, captions. A typical lag time between the time a word is spo Live-display captioning is used when an ace ken and the same word appears in a caption on a is available in advance 0£ a televised live ev:te Script viewer's screen is two to three seconds. a speech or newscast. The scripted words t such as Errors in real-time stenotype-generated captions verted to captions by an editor and stored are con. are generally the result of one of four factors: (1) the puter disk. When the live event is televised ~ha c~rn captioner may hit the wrong keys; (2) the captioner ~anual~y trigg~rs e~ch c_aption for inserti~n .e editor may be unable to keep pace with the speakers; (3) the video signal. With hve-d1splay captioning th IIlto the captioner may not hear the spoken word correctly; or are timed to appear on the television receivere Words (4) the computer may not have a particular word in its are being spoken. If last-minute changes are :sdth~y . . b uia e lJl "dictionary," which is often the case when unfamiliar the ~cnpt, it may e necessary to switch to real-tirn proper nouns (such as the names of people or places) caphomng. e are spoken. Newsroom Computer Captions Voicecapti oning The third method of live captioning is the generati Captioning by a voicecaptioner is similar to stenocap of captions through the use of newsroom compu~n tioning except that input is by voice rather than key equipment. Many television newsrooms convert nei ~ board. The voicecaptioner trains a customized speech stories from their word processors into data for use~·~ recognition system to understand his/her voice accu a prompting machine for the news reporter to read rately and quickly. Ideally, the speaker uses _a high while on the air. At the same time the reporter reads quality microphone in a controlled, sound-isolated the script from the prompter, the computer controlling environment. The speech recognition software must the teleprompter passes the data through a serial inter ·be "trained" to the voice of the voicecaptioner. A sys- face to the caption encoder that inserts the captions tem that must resolve only one voice has many advan into the video signal (see Figure 5.24-4). tages over a system that must recognize multiJ?le There are some dangers in this process. It is not as voices, especially if those voices are recorded or deliv automatic as might be assumed. Some newsrooms add ered via low-fidelity methods. cues to the prompting text designed for the on-air per Errors in real-time voice-generated captions are sonnel, some of which might be inappropriate to generally the result of one of three factors: (1) the cap appear as captions. Program breaks or changes of tioner may say the wrong word; (2) the captioner may scene may require changes in the prompting operatio~ say the wrong form of a word (failing to distinguish that could interrupt the captioning process. Finally, 1t between conflicts such as "their" and "they're" and may be necessary for an operator to sign~l ~e caption "there"); or (3) the computer may not have a particular ing encoder to add captions at the begmrnng of the word in its "dictionary," which is often the case when )ocal program _and to revert to the pass-thro~gh o~ unfamiliar proper nouns (such as the names of people . , upstream captions at the end so that prev10usly or places) are spoken. f encoded captions (for example, on a ne~or~ newscas~ that follows local news) will be transrmtted mtact. It b Speech-Recognition Captioning desirable for the operators and program pro?ucers to At the time of this writing, speech-recognition systems have an off-air monitor to ensure the integnty of the that attempt to transcribe program audio directly with captioning process. d no human intervention are beginning to appear. Note that FCC rules prohibit ABC, CBS'. ~ox, an Although it is anticipated that such speech-to-text sys- NBC and their affiliates in the top 25 telev1s10n mar-
Video Monitor with )I Captions I Source I Displayed
Smart Prompter )I Transmitter Caption Encoder Video with I I Data Ca plion Data FIGURE 5.24-4 Newsroom captioning with teleprompter. ti!. 1440
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 9 of 21 ~-
CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS
BLANKING LEVEL 7 CYCLES OF 0.503 MHz (CLOCK RUN-IN) 50 - PROGRAM COLOR TWO- 7 BIT+ PARITY ASCII CHARACTERS 25 -- BURST !DATA) 20 - o- ...,__ 12.910uS ___.. 33.764us ------"""' !0.20H) i------{0.53H)
3.972uS l0.06Hl 27.452uS !0.43H)
10.074uS ll""t------51.26BuS !0.16H) I"' ,f 'a 61.342uS {O 965H) ~11, FIGURE 5.24-5 Line 21 data signal waveform specification. ~'
\~ts from counting news programming using elec contains control codes that provide the instructions for i ;onic newsroom techniques toward their captioning the timing, screen position, and attributes of the data. :equirements. Rather, these networks and stations The clock run-in consists of a seven-cycle sinusoidal · ~ust provide real-time captioning using a stenogra burst that is frequency and phase locked to the caption . ;tier to convert the entire audio portion of the live data clock. The frequency of 32 fH (32 x 15,734.26 Hz= ~ugram to captions. Others may use electronic news- 0.503496 MHz) is twice that of the data clock and pro ~ :OOm techniques to meet captioning mandates. vides synchronization for the decoder clock. The clock run-in signal is followed by the equivalent of 2 data ~:~.l.·..· .·.·. rJ!,;. .~. bits at logical zero level and then a logical one start bit. "/" LINE 21 DATA FORMAT ; "'1'f1· .~: The last two cycles of the clock run-in, the 2 logical I zero bits, and the logical one start bit constitute an 8- ~Captions associated with an analog NTSC television yrngram are transmitted as an encoded composite bit frame code signifying the start of data, as shown in ~ta signal during line 21 on field 1 of the video sig Figure 5.24-6. JJ!, as shown in Figure 5.24-5. The signal consists of a The 7-bit character data is coded in a nonreturn-to '.:lockrun-in signal, a start bit, and 16 bits of data corre zero (NRZ) format. An 8th bit is added to each charac ;~nding to 2 bytes of 8 bits each (7-bit character code ter to provide odd parity for error detection. •¢us 1 parity bit). Therefore, transmission of actual The sequence of identification, control, and charac ~tais2bytes every 1/30 of a second, equivalent to 60 ter code transmission is shown in Figure 5.24~7 and i~tes (characters) per second. The data stream also Figure 5.24-8. Each caption transmission is preceded
START ;.lelllllP... --______CL~CK ~UN-: I< BIT -i< DATA....__. 1 2 3 4 5 9 10 ,, 12 13>i ~
I 0 I 1 I 0 I 0 I 0 I 0 I 1 I 1 I BIT 1 I BIT 2 JE FRAME CODE >I
FRAME CODE DET~ FIGURE 5.24-6 Line 21 data timing.
1441 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 10 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY
INITIALIZATION OR MODE SWITCHING +PREAMBLE CONTROL CODE CAPTION TEXT {EACH IS TRANSMITTED TWICE) UP TO 32 CHARACTERS PER ROW ""( ___,A._ ___)
NON-PRINTING PRINTING START PARITY PARITY START 1ST TEXT PARITY 2ND TEXT CONTROL CONTROL PARITY BIT BIT BIT BIT CHARACTER BIT CHARACTER CHARACTER CHARACTER BIT
IDENTIFICATION CODE PLUS BEGINNING OF PRINTED ROW POSITION, INDENT CAPTION & DISPLAY CONDITION INSTRUCTIONS. FIGURE 5.24-7 Beginning of caption sequence. by a preamble code, which consists of a nonprinting after a valid control code are in the range of Dx20 character followed by a printing character to form a through Ox7F. These are interpreted and loaded into row address and display color code. Both characters of the decoder memory as printing characters. Character all control codes are transmitted within the same field codes with bad parity result in the display of a white of line 21 and twice in succession to ensure correct box (the delete symbol) in place of the character. reception of control information. A transmitted cap The data rate is 480 bps or 60 bytes per second (8 tion may be interrupted by a mid-caption control code bits per byte including parity). At an average of 5 between two words in order to change display bytes or letters per word, an average maximum word attributes such as color or italics. At the completion of rate of 12 words per second or 720 words per minute a caption transmission, an end of caption control code can be achieved. In practice, the maximum word rate is sent. is somewhat less than this, about 600 per minute, due Codes are composed of 2 bytes. The first byte is a to the time required for transmission of control codes. reserved code, in the range of OxlO through OxlF. This Because most speech is much slower than this, there is is followed by a printing character, in the range of adequate time for additional data services in each Ox20 through Ox7F. All characters that are received field. p- l TEXT CONTROL CODE TEXT
PRINTING PARtTY START TEXT NON-PRINTING PARITY CONTROL 91T BIT CHARACTER BIT CHARACTER BIT BIT CONTROL BIT CHARACTER CHARACTER
LAST CHARACTERS PRIOR DISPLAY-CONDITION TO MID-ROW DISPLAY CHANGE INSTRUCTION CODE {TRANSMITTED TWICE) FIGURE 5.24-8 End of caption and display sequence. ,
1442
Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 11 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS
,t, LINE 21 TECHNICAL ADVISORY VITS generators and processing amplifiers, and tape delayed broadcast installations should also be rou CC requires that any local TV station or cable tinely checked to ensure passage of the line 21 signal. I ~e/nd that redistributes a signal that contains It is important to note that if the overall signal level is I:ed captions must preserve and forward that cap- allowed to drop substantially below the normal 50 IRE ;~ 10 data. . . . . level, captions could become garbled. Also, the nor :1< ular observation and momtonng are essential mal 7.5 IRE setup is not present on the line 21 signal. I. ReJer to verify the presence and proper location of I~odata signal. Equipment throughout a video facil it.e should be routi1:1ely chec~ed and, !f necessary, Cautionary Notes :0.usted to pass the lme 21 caption data signal. '[~!Maintenance personnel and control room operators Network Origination/Local Origination 'u1d be made aware of the importance of the line 21 10 Network programs and commercials prior to, during, i ~ al and adjust all applicable equipment accord ! A waveform monitor with line-select capability and after locally originated captioned programs may '·~y also be closed captioned. Therefore, at the end of cap ::ge·uired to view the line 21 caption data to deter : it is on the correct line and field. For exam tioning local programming, the local smart encoder ~;that must be sent a command to re-enable passage of : ~le, a Tektronix 1480 series, ~ 700A,. 17~0 Series, :i-goR or Hewlett-Packard eqmvalent with lme selec upstream line 21 data. If both network and local pro easily locate and display the signal. In ad di gramming are passing through the smart encoder, the t\ will line 21 signal should be checked to ensure its integrity. ! ~n caption decoders can be used to verify that the ;;aption content is. being encoded or preserved cor- •rl1:tly on the outgomg feed. · Newsroom Computers i The following video processing equipment may Newsroom computer captioning typically uses the : ~versely affect the line 21 signal: I smart encoder in a newswire/real-time mode. When ~, Video processing amplifiers: Certain types may delete this mode is invoked, incoming line 21 data is auto the line 21 signal,_ move it to anot~er line, exchange matically blocked, preventing any previously I the fields, or partially blank the signal. recorded caption information from being passed I •' Time base correctors (TBCs): Some may blank the line through and displayed. In newswire/real-time mode, i 21 signal or advance or delay line 21 by a field or a the captions are encoded in roll-up style. Other systems may use the smart encoder in the I line. "line 21 direct-entry" mode rather than the newswire/ : • Videotape machines: Certain types contain a TBC that real-time mode. Direct entry offers more complete i may blank or move line 21. control of the encoder, allowing captions to be • 1 Video switchers or digital store devices: May blank or authored in paint-on or pop-on styles. I move the line 21 signal. In either captioning mode, the newsroom computer ·' Frame store or frame synchronizers: May blank or dis- must issue a command to end captioning at the con 1 tort the line 21 signal. clusion of local live captioning to re-enable passage of upstream line 21 data. • VITS generators or inserters: Should be checked to 1 determine whether they are programmed to pass Using Captioned Excerpts I the line 21 signal correctly.
' Digital effects and text generators: May delete or dis- Excerpts of programs received or recorded off-air or 1 1 tort the line 21 data signal. from external satellite or network feeds, for use in ' Digital-to-analog composite encoders: May add setup other programming, may contain line 21 closed cap to the line 21 data signal. . tions. In these cases, the captions should be stripped if the excerpted material audio will be edited or replaced : The line 21, field 1 data signal may be partially, but because the captions may no longer be germane. A il1ldvertently, blanked. This can occur when the active time base corrector or video processing amplifier can ~deo line is advanced or delayed relative to horizon be used to remove or move the caption data signal ~ sync. The decoding system is more tolerant of this from line 21 prior to inserting new captions. ~~~!em when the data signal is early and causes part lie e clock run-in sinusoidal burst to be blanked. On 0 Broadcast Decoder ~ . ther hand, if the data signal is delayed relative to 'nanzontal sync, loss of the last data bit (parity bit) In addition to consumer models, professional broad :a~occur, producing decoding errors that in turn will cast-type decoders are available for stations to use to l!i ethe characters to be displayed as white boxes. In monitor the line 21 data signal on the incoming net ~~Xlreme case, too many errors will cause the work feed, videotape machines, or the broadcast sig Soder to _disable all closed captioning functions. nal. Broadcast models accept a standard baseband 'erti ~e Video processing amplifiers have sensitive video signal with captioning data on line 21 and pro ~o~ c~hphase blanking and horizontal blanking con duce a video output with the captioning data decoded l\ll. ·All e~e may also adversely affect the line 21 sig- and displayed as open captions for viewing on a stan signal paths, backup transmitters and their dard video monitor.
1443 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 12 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY
Emergency Information Messages data services on field 2 have higher priority th The FCC adopted rule 79.2 in 2000 to require broad and can interrupt insertion of an XDS packet Can XDs casters, cable operators, and other multichannel video defines a mechanism to resume XDS packets.th EA.-603 been interrupted by caption data. Howev at have programming distributors to mak~ local _emer~en~y XDS decoders have implemented this not all information accessible to persons with hearmg disabil XD~' feature, and will process only XDS packets thesllrne ities. Emergency information no'. provided through encoded uninterrupted. . at are closed captioning must be provided through some other method of visual presentation, such as open cap- CEA-608 states that a caption waveform h present on field 1 whenever there is a captio s all be tioning or text crawls. . form present on field 2. Consequently, man ~ w~\'e Of particular note to broadcast~rs,. this FCC rule that include an XDS decoder will not enable £t u:ivices requires that the presence of captionmg and emer caption waveform is present on field 1, even if th ess a gency alert services must not disrupt each other. As already noted, text generator devices may delete. or gram is not being captioned. Broadcasters sho~d~ cautioned to always include a valid caption wavef e distort the waveform that carries caption data on lme on field 1 for this reason. 0nn 21. Additionally, captions displayed on a television set may appear over the emergency message text. Either Note that XDS data carried with CEA-608 capti condition would violate the FCC rule. is intended for use only by ancilog NTSC televis~ns Manufacturers of closed caption encoding equip receiver~. While the XDS data. may _be ~anscoded in~~ the section of the DTV caption d~stnbution packets ment responded to these requir~ments by creating fea reserved f?r CE_A-608 ca:ptions, this data is intended tures to reposition existing captions to pre_vent overlap only for reinsertion onto lme 21 by DTV receivers with with emergency alert messages, and to bndge the cap analog NTSC outputs. DTV receivers should use con tion data around emergency alert text inserters and tent advisories, copy protection guidance, and other reinsert it onto line 21. ancillary data from the PSIP tables that are broadcast along with the DTV video content. See the section on DTV captions later in this chapter and also Chapter LINE 21 1EXT AND EXTENDED 5.22 for more details on PSIP. DATA SERVICES A large number of XDS packet types have been defined in CEA-608. A subset of the packet types most Typically, captions· do not use all the available data commonly used by broadcasters and those with cer capacity or "bandwidth" of the line 21 sys~em, ~?the tain caveats are described herein. unused bandwidth may be used to transrmt additional services. These services are time multiplexed between Content Advisory Packet caption data. The content advisory packet is also known as the "V chip" packet. This packet provides a rating !or the Text Mode Services associated video program. Consumer electrorucs t~at support the content advisory packet may be co~g The Text Mode Services consist of four data channels ured to block content that exceeds a particular rating (field 1 includes Tl and T2; field 2 includes T3 ~d T4). level. FCC rules state that the data within the content The text service channels may be used to transmit pro packet shall not change during the course of gram notes, news, weather, sports, farm, and _financial ~dvisory ' a program, which includ~s i:rogr~ s:gments, com reports and other informati~n t?~t may b~ mdepen / mercials, promotions, station identifications, e.tc. . 1 dent of the program with which it is transrmtted. . Four content rating systems have been defined. ~ Text Mode Services are accessed by the user usmg Parental Guidelines (TVPG), MPAA, Canadian the receiver's decoder controls. There are two text English, and Canadian French. These four systems: "channels" for each field. Unlike the captions, the text mutually exclusive, so if one is included, then the 0 service appears as rolling text that obliterates ~e ers shall not be. f ll ew entire image or just the top o: bott~m half.. C~ption FCC Report and Order 98-36 required half o _a n ed data is time sensitive and is given higher pnonty by television models 13 inches or larger encoders, which means that the display of Text mode manutac~u data will pause during caption transmission. after July 1, 1999, and all sets 13 inches or ~ge~ clmol factured after January 1, 2000, to hav_e V- Pf ~iolent. ogy to allow parents to block the disJ;'lay ? harmful 1 Extended Data Services sexual, or other programming they believe : t add· to their children. While support of the con en;,, the Extended Data Services (XDS) is another data channel sory packet by content creators is· vo luntary. es'" that that is time-multiplexed with the caption and text United States, FCC Report and Order 98-36 r~f: rating channels on line 21 of field 2. XDS is used to convey TV stations and cable operators may not m pro ancillary data such as broadc~ste~ identifi~atio:r:, pro information inserted by the network or progra gram description, content adv1sones (V-chip), Time of vider. f ult state of a Day, and copy protection state. FCC rules stipulate that the de a ms but Each data type is encoded in ~ XDS "packet" "':ith receiver should be to not block unrate~ pror~)ockinf a unique identifier and an 8-b1t checksum. Caption users may be provided with the option °
1444 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 13 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS
J11S that are not rated. The wording of the FCC log Source Bit (ASB) specifies whether the source is pil'grdoes not make a distinction between program prerecorded material. ~es with a content rating of NOT RATED and pro CGMS-A data is also specified by IEC 61880 to be t.iJlg that have no content advisory packet encoded transmitted on VBI line 20 of NTSC and PAL transmis ,3111s sions. When it is encoded in this manner, CGMS-A is ~all· Canada, the CRTC mandated that Canadian encoded independently of closed captions. Ind asters must carry the Canadian content rating ~-oa ~s starting in March 2001. Programs originally ~:~ ced for American audiences must have new rat- STANDARDS AND TERMINOLOGY FOR . lllJl'u ncoded in one of the two Canadian rating sys ;,gs ~hen they are broadcast in Canada. Decoders on DTV CAPTIONS :ii'Jl5y older V-chip-capable television sets do not sup- Although broadcasters may not need to know every : the Canadian rating systems, however. . detail of the complex standards involved for DTV cap r tioning, it is necessary to be aware of the terminology rime Packets used. Knowledge of the standards will certainly be useful in troubleshooting and helping determine com , e information can be broadcast using the Time of pliance of installed equipment and systems. :r (TOD) and time zone and daylight saving time -.:[fZ) XDS packets. This data can be used by VCRs ~ d other consumer electronics devices to automati CEA-708 ~y adjust their internal time clocks. Whenever they 1·~encoded, the time packets must always be inserted CEA-708, Digital Television (DTV) Closed Captioning, i ~broadcast time, never subjected to tape delay; other- defines the coding of DTVCC as it is delivered in an iise the data will be delayed, and receiving devices ATSC emission bitstream as specified in ATSC A/53D, 1ilJ~djust their clocks to the incorrect time. and is applicable equally to high definition and stan Every PBS affiliate station has an encoder to insert dard definition video formats, and to both terrestrial . :;me packets. Other br?adc~sters in the United States broadcasting and cable distribution. The captioning <'e discouraged from msertmg TOD and TZ packets. data is carried in the video user bits of the MPEG-2 iJme consumer electronics will behave erratically bitstream. 1hen time packets are available on different channels Captions generated in accordance with CEA-708 nthe broadcast area. standard are generally referred to as "708" captions. The TOD packet data includes the Coordinated At the time of writing, CEA-708-C is the current revi l'niversal Time (UTC) time and date, and a flag to sion of this standard. illdicate whether daylight saving time is presently in ifect. The data in this packet is independent of the ~oadcast area. CEA-608 Legacy Data The TZ packet describes the time zone of the broad In addition to the actual 708 caption data required for :ast area and a flag to indicate whether daylight sav use by a DTV set-top box or integrated receiver to dis ~g time is observed in the broadcast area. The time play DTV captions, the CEA-708 standard requires rone is encoded as an integer number of hours to be carriage of equivalent data for a subset of the captions iilbtr~cted from the UTC time. Consequently, the time coded in the 608 format. This legacy 608 data, also mne msome regions cannot be represented in the TZ known as the 608 compatibility bytes, is required for use ~cket. Examples include Guam and Newfoundland, by DTV set-top boxes so that line 21 data can be Canada. Additionally, the TZ packet must not be inserted in an analog composite video output (if pro ~coded when the intended broadcast area encom vided) to feed a legacy NTSC TV set. Some DTV ~ses multiple time zones, or across regions that do receivers may use the 608 data when 708 data is not md do not observe daylight saving time. In these available for providing closed captioning on the DTV ~-eas, consumers must manually set the time zone and display, but this is not mandatory and is not imple c.iylight saving modes in their VCRs. mented in many receivers.
r0 ' PY Generation Management System SMPTE 334-1 and SMPTE 334-2 ~Copy Generation Management System~ Analog ~.iiMS-A) packet is a copy protection mechanism for SMPTE 334M defines a method of coding which ' 1445 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 14 of 21 ii, B _,,,,,,.. SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY tribution Packet (CDP) Definition, defines the caption tion until native 708 caption authoring is distribution packet (CDP). adopted, but may persist indefinitely for leg generally rial that is already captioned. acy lhate- Caption Distribution Packet There is no standardized algorithm for 60 8 caption translation, but the goal is to produc 10 _708 SMPTE 334-2 defines the caption distribution packet with an appearance as close as possible to th captions (CDP) consisting of a specific sequence of bytes that ance of the original data on a CEA-608 deco~appear can hold the actual 708 DTV caption data, the legacy eral manufacturers have implemented 70S er. Sev 608 caption data, caption service information, and fulfilling this goal. ,, encoders optional time code for synchronization (not needed The resulting DTV captions are usually kno with transport methods where the CDP is directly "translated 708." They are also referred to in th w~ as associated with video frames). The CDP is the basic R&O as "upconverted captions," but this term e CC unit of data that is transported through the profes be deprecated because of the connotation that t~hould sional portion of a DTV caption distribution chain. cess is related to video up-conversion, which it i~ pro The CDP was freviously defined in CEA-708-B. Some manufacturers use the term "transcoded" in: The specification o this data structure was handed off regard, but that should preferably be used onl to SMPTE in 2006 because the CDP is used exclusively refer to the 608 compatibility bytes transcoded l to by professional equipment and is not broadcast in the 608 caption sources. om transmitted DTV bitstream. Transcoded 608 ATSCA/65 Where 708 captions are produced by translation fro ATSC A/ 65, Program and System Information Protocol existing 608 line 21 data, the same data may ~ for Terrestrial Broadcast and Cable, defines information "transcoded" to fill the required 608 compatibi!i; in the Program and System Information Protocol bytes. Because the 608 data itself is unchanged when 1t (PSIP) that describes the contents of an ATSC broad is transcoded into CDPs, it can be subsequently read cast. It defines the caption service descriptor (CSD) to be and transcoded back onto line 21 of an NTSC or SDI carried in the transmitted bitstream to announce the signal later, if so desired. presence and format of captions being carried. The 'E~t~.i,.,,_ •' CSD provides metadata that may be used by the DTV receiver to display information about captioning in an FCC RULES AND REGULATIONS ..... electronic program guide and may be needed to prop FOR DTV CAPTIONS , erly decode and display the closed captions. A/ 65 states that for terrestrial broadcast transport Rules streams, the CSD must be present in the Event Infor Since July 1, 2002, U.S. broadcasters have been mation Table (EIT), and that for transport streams required by FCC 00-259-2000, Report and Order, Closed delivered on cable, the CSD must be present in the Captioning Requirements for Digital Television Receivers, Program Map Table (PMT) for captioned programs to provide closed captioning on at least some of their and, if an EIT is sent, must also be present in the EIT. programs transmitted on DTV channels. At the time of writing, the current revision of this doc The FCC R&O amended Part 15 of the FCC rules, 1. ument is A/65C with Amendment /'-adopting technical standards for the display of cl~sed While CEA-708 captions may use up to 63 caption I captions on digital television receivers. It adopted into services numbers, only 16 services are carried simulta the rules Section 9 of industry standard CEA-!08, neously and the CSD describes only up to 16 services. specifying the decoding and display of closed caphon ing for DTV systems. For informational purpose_s, ''. Native Captions, Translation, and Transcoding also incorporated by reference the remaining se~hOns of the standard, which specify the encoding, dehvef). Native Captions and other aspects of DTVCC. . The R&O amended Part 79 of the rules to require Captions that are encoded and transmitted in the 608 or 708 format in which they were authored may be transmitted closed captioning to reflect the changes~ Part 15 and to require all caption data. !~ be_ pas; . referred to as "native 608" or "native 708" captions. 01 intact through program distribution facilities !Il a . , mat consistent with Part 15, unless such progra]J'llllUlc Translated 708 is recaptioned or captions are reformatted. . din~ 1 It is possible for captions to be encoded in the 708 The R&O also clarified the schedule for me u ' DTVCC format by conversion from 608 legacy cap closed captions in digital programming. ·een tions already encoded on line 21 of an analog NTSC No distinction is made in the FCC rules b~1\0r. video feed. This method can be used when the pro DTVCC for high definition or for standard def~oth gram was created primarily for NTSC transmission programming; the requirements are the same for sition but is being encoded for DTV transmission, either as The R&O makes it clear that during the trandanct an SD program or with up-conversion to HD. For new period to digital television, DTVCC in ac~~A.{I).' : programming this is expected to be an interim solu- with CEA-708 may be derived from legacy 1 tf I 1446 j Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 15 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS tions as well as from native 708 authoring. Equivalent dates for Spanish language programming ,JllaJog) ~hms that to count captioned DTV program are 30% at 1/1/05 and 75% at 1/1/12. riaJso co toward the captioning total for each chan Thus, DTV services now have the same hourly cap 5 otlJlg h0: adcast DTV signal must include both CEA tioning requirements as NTSC services. The program r.el, the ~~A-608 caption data. ming requirements were reconfirmed in an FCC ~~andpCC 04-192Al - 2004, Report and Order, Second Public Notice issued January 6, 2004, titled FCC Public 0~ DTV Review, confirmed the requirement to Notice DA-04-2, Notice to Video Programming Distribu Pe11odtC th 708 caption data and 608 caption data in tors and the Public of the January 1, 2004, Requirements for carrY bo and HD DTV programming. It also confirmed the Closed Captioning of New Nonexempt English and [XJthSD.rement to carry the caption service descriptor Spanish Language Video Programming and Reminder the::~~he pSIP Event Information Table and the Pro- Regarding Other Captioning Requirements. . li1 Map Table. craJJl - Exempt Programming ' ~ i Receivers There are exemptions to the preceding captioning requirements for some types of programs; examples require all DTV set-top boxes and DTV I rules include t ~eivers with a screen height of at least 7.8 inches (the ~~ht of a 13-inch 4:3 displ~y) manufactured a~ter • Most programs shown between 2 a.m. and 6 a.m.; Ihe~ 1 2001, to include a caption decoder complymg • Local non-news programming with no repeat ! ~~Section 9 of CEA-708. ~uch devi~es shall provide value; I th user with control of caption font, size, color, edges, I 'ed background. Decoders must be able to decode the • Commercials no more than five minutes long; ! mstandard services in CEA-708 and allow users to • Some instructional programming that is locally pro ·· ~~seat least one for display. Set-top boxes that have duced by public television stations; manalog NTSC output shall insert 608 caption data • Programs in languages other than English or Span \Jlried in the DTV signal into line 21 of the NTSC ish; rideo output. • Programs shown on new networks for the first four years of the network's operations; · Cable Carriage J • Public service announcements and promotional f Iherulesrequire digital television programming with announcements shorter than 10 minutes, unless ·~ CEA-708 caption data services to be delivered to digi they are federally funded; and ' :alcable,subscribers with captioning intact. • Programming provided by program providers with annual gross revenues under $3 million (although such programmers must pass through video pro CJ rrogramming Requirements and Schedule gramming that is already captioned) . . The FCC rules allow for a phased introduction of cap In addition, a video programming provider or dis IM:ming as described in the following sections (note tributor may file with the FCC a petition for an exemp fut at this time the term "channel" has not been clari tion for specific programming. ::edinrelation to multicast broadcasting). fov Programming DTV CAPTIONING SYSTEM ~ewdigital programming is that "prepared or format- IMPLEMENTATION , al '.or display on digital televisions that was first In accordance with CEA-708, ATSC A/65, ANSI/ 11.1bhshe? or exhibited after July 1, 2002." Each DTV SCTE 43, and ANSI/SCTE 54, DTV signals for broad ~el JS required to have 100% captioned content, 1 cast and cable distribution, both for HD and SD pro ~ some exceptions, for new, nonexempt, English gram material with closed captions, need to carry ~age prog_ramming broadcast after January 1, it.' For Spanish language programming the sched- • 708 DTV caption data; 15somewhat delayed, leading to 100% by 2010. • 608 compatibility bytes; • Caption service descriptor in the EIT and PMT. lreni/e Programming To enable ATSC encoders and PSIP generators to ;ilenonexempt English language digital (and ana generate signals that meet the preceding requirement, ~~:~~amming is defined as that "first published 708 caption encoders should produce the 708 DTV ~req ~t7d before July 1, 2002." The broadcast sched caption data and 608 compatibility bytes. Caption ser lllows~irmg captioning on prerule programming is as vice information is typically inserted into the ATSC video stream by a PSIP generator device, which may l/]/D3to 12/31/07 30% of programming obtain this data from a number of sources. Because of the varying requirements of different A!ten111os 75% of programming networks and production facilities, and the expanding 1447 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 16 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY capabilities of new equipment, it is not practical to Video with source 608 captions describe every possible system implementation in NTSC or 259M detail. The following sections provide an overview of typical captioning applications. Video Video 708 Caption Video + 708 Capu Recorder 1------.i EncoderNANC ons CEA-708 VANC Caption Encoding or Server . 292M or 259M Embedder Where 708 captions need to be carried in a video sig ·Time-code nal, as shown in Figure 5.24-9, the architecture for Computer encoding is similar to the 608 arrangement. This tech Caption data nique is independent of production format and with 708 Captioning Serial link, modem or equally applicable to captioning HD and SD program Software material. The program video is HD-SDI or SDI. LAN For native 708 captioning, the data from the cap FIGURE 5.24-9 Typical 708 caption encodm' tioning computer is processed and inserted as caption g. distribution packets (CDPs) embedded in the VANC ·~ of the video signal in accordance with SMPTE 334-1. Alternatively, as indicated by the dotted line input in the SMPTE 334-1 caption data in VANC. This fun ti the drawing, 608 captions from an NTSC video feed may be integrated into the recording system· in cthon cases it may be by use of external bridges'. Curro er may be used as a source of captions to be translated to . . . ent 708. The two video sources must, of course, have iden record mg eqmpment is genera11 y not capable f tical program content. recording and passing all VANC data from input~ The VANC packets are transported as an integral output, and care is needed to ensure that the relevan~ part of the digital video signal; however, special data packets needed to implement DTVCC are pre arrangements have to be made at any point in the dis served by the selected equipment. 1,. ,, tribution chain where VANC data may be removed. Processing :- Authoring of 708 Captions Various types of video processing equipment in the broadcast plant may delay, repeat, or drop video At this time, work is in progress by U.S. national cap frames and/or delete the VANC part of the video sig tioning organizations for an agreed 708 caption inten nal. Wherever possible, system design should mini· tions format. In the meantime, some 708 caption mize the use of equipment that disrupts the flow of encoders can accept data from files configured with VANC data, or a special provision should be made for data in native 708 format-that is, exactly as defined bridging around it. The bridging equipment should in CEA-708. It is also possible for 708 caption encoders preserve the relative timing between captions and to accept data derived from 608 "caption intention" video as far as possible. , . files, which would then be translated to 708 as described previously. The resulting DTVCC are lim ited to the authoring capabilities of CEA-608 and can Compression Distribution Systems'];; not exercise all features of a 708 caption decoder. ~-the program is distributed through a medi'.1111 ~at For full implementation, it is recommended that . mvolves MPEG compression, such as a satellite link, CEA-708 caption authoring systems should produce I VANC data is not carried directly with the compressed both 708 and the associated 608 caption data types and video; therefore, the CDPs must be bridged from the provide appropriate data for the caption service infor VANC of the input HD-SDI or SDI video signal to a mation. data channel in the MPEG multiplex. In general, there are three different ways to do this: Recording, Processing, and Distribution 1. The VANC data is packaged into the tr~sport . stream multiplex as a private data service on dedi Embedding DTVCC in VANC, whether for HD or SD cated packet IDs (PIDs). program material, allows them to automatically fol low the video through most standard routing and 2. For captions generated direct from a 708 c~ption , switching, and some types of processing and record encoder, a serial representation of the captions can ing equipment, similar to the principle of carrying 608 be packaged into a private data service. captions in line 21 of an analog signal. However, sev 3. The uplink MPEG encoder takes the capti·0 1: datato eral issues must be considered, as discussed in the fol and packs it into the video user data according lowing sections. CEA-708 and ATSC A/53. thods to be It is possible for all three of the above me use- Recording 1 implemented at the uplink system, and th~ m~ll wi!l For recording systems, the integrity of the DTVCC ful one(s) selected for use at the downlink site6os com· data should be ensured by selecting a tape format and equally support 708 captioning data and the equipment or video server technology that preserves patibility bytes at the same time. 1448 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 17 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS All these methods are supported with MPEG trans For the first case, NTSC caption arrangements will .. n at bit rates including, and higher than, the be unchanged from the legacy arrangement. For the 5510 J111 Nfbps ATSC emission standard, such as the 45 second case, 608 caption data will usually need to be J9.39 rate typically used for network distribution. transcoded from VANC CDPs onto line 21. i!b~od 3 is similar to the arrangement used for emis '..fe station captioning a1:1d ~TS~ encoding. It is usu ATSC Station Output ,ion used for network d1stnbuhon at 19.39 Mbps as allY mented by some networks, but may also be used For the ATSC station output there are three main 1 arrangements; it may have iJJlP ~ the bit rate differs from the ATSC nominal, 11 ~e the video outputs from the receiver I decoder 1. A dedicated DTV (possibly HDTV) station master us.~gcaption data in VANC or feeding caption data via control or simple switcher with a (H)DTV network 1\1 !iiik to an MPEG encoder. distribution and local live and recorded sources. ;er~ each case the equipment bridging the caption The DTVCC should already be present on the live d ta through the MPEG process should preserve the network and recorded sources but need to be added :iative timing between captions and video. to the live local sources; 2. A feed (possibly upconverted) from the NTSC net ~· ' work distribution and master control. In this case Local Station Arrangements the 608 line 21 captions will need to be translated to f' e S.24-10 shows a possible configuration of cap 708 as well as transcoded; ti~g-related equipment in a local station with ATSC 3. A bitstream received from the network already andNTSC transmission. This is intended only to illus encoded in ATSC transmission format (at 19.39 trate some of the captioning arrangements discussed Mbps) and ready to send to the DTV transmitter later and is not a full system diagram. with local content spliced in and new PSIP informa The drawing shows a DTV system for HD only. A tion added. In this case, DTVCC should already be similar arrangement could be used for SD DTV using present in the incoming bitstream. sMPTE 259M in place of SMPTE 292M. In that case, up-conversion to and down-conversion from the Some stations may have combinations of these ~SC feeds are not required. It should be noted that arrangements, depending on network DTV distribu ~me equipment manufacturers have combined sev tion arrangements and the number of DTV program eral functions shown in the figure into one unit. services. NTSC Station Output Local Live DTV Captions for the NTSC station output there are two main Live programming may be captioned using a 708 cap . arrangements; it may have tion encoder with VANC CDP embedding, as dis cussed previously but with live video and with I. Alegacy NTSC station master control with legacy caption intentions generated in real time, with author network distribution and local sources; or ing from a stenographer or with electronic newsroom 2. Astandard definition (possibly down-converted) techniques, as mentioned previously (where allowed). feed from one of the DTV program services. This is shown at the upper-left side of Figure 5.24-10. ,;:r \'!t~ Local 292M video source 708 Caption J l 292M SMPTE Local EncoderNANC ~ ~ 292M MPEG caption ___.., Embedder 708 r __. ToATSC data Routing/ Encoder . l?.a.t~ .... Transmitter Master VANC MPEG - ~ Network and locally - Control r---. Disembedder ,_ . Mux r recorded 292M video 292M with captions r PSIP :.... Generator r---. CSD 292M Up- 608 r+ Converter ~ 708 Caption Data 259M ~ EncoderNANC r Line 21 or Downconverted DTV Embedder Down- ~ 4 Caption -r NTSC Converter - ~ Line 21 292M 259M Inserter r lY To NTSC Decoder 608 or Transmitte Data NTSC Direct ~ r FIGURE 5.24-10 Emission station captioning. 1449 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 18 of 21 SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY Video 292M Recorder - MPEG 708 ... ToATSC or Server Encoder i-. .. ~".l!~,. Transmitter Video with 708 Caption MPEG - source······• Encoder or Mux • Time -code 608 captions Translator ·----- ·r :.,. PSIP Generator i--. Computer CSD Caption data with 708 i ·,f~ Captioning Serial link or LAN Software FIGURE 5.24-11 Alternative captioning feed to emission encoder. Alternatively, the 708 caption encoder may feed the In this type of system, instead of inserting ca ti ATSC encoder directly, as shown in Figure 5.24-11. If data into the video at some upstream point for £~n the local programming originates as an NTSC source, bution with the video program, caption files are ges · existing legacy arrangements for live 608 captioning ated and then distributed separately from the vi~er arrangements may be used, with bridging to DTVCC. typically over a computer network. A separate captieo, file for each program is stored on a captioning co~~ NTSC-DTV (Up-Conversion) puter (server) at the emission station that runs soft Some HD DTV programs may be produced by up ware to manage the various caption files and playout conversion from an analog or SDI program intended Software running on the captioning server, or on othe; for NTSC transmission. In this case, the captions must networked computers, allows a program playlist to be be bridged from the analog NTSC or SDI line 21 to established. As programs are played out, time code CDPs in the VANC of the HD-SDI stream. This can be from the video server triggers captions from the cap accomplished by having a decoder extract the 608 data tion server to be fed to the caption encoder and from line 21 and feed it to a 708 caption encoderI inserted into the transmission feed. Information on VANC embedder, as shown at the lower-left side of specific features and arrangements for server-based Figure 5.24-10. Similar techniques can be used to insert captioning is available from the manufacturers. DTVCC in the VANC of SD programs for DTV trans Server-based captioning is applicable to both 6D8 mission, without up-conversion. Some companies captioning for NTSC and 708 captioning for DTV. offer equipment that integrates the line 21 decode, 608 Such systems use basically the same interfaces to 708 caption translation, and VANC CDP embedding between the captioning computers and caption functions in a single device. encoders and between caption encoders and ATSC encoders as systems using captions embedded in DTV-NTSC (Down-Conversion) video streams. }J!I}. There may be a need to derive an NTSC broadcast sig nal from an HD-SDI stream. The CDPs carried in the .fr Caption Input to the ATSC Encoder ·-..~, · VANC in the HD-SDI stream should contain CEA-608' data, which can be embedded into line 21 of the ana The task of embedding the caption data into the ATSC log NTSC or SD-SDI video. This can be achieved by stream in accordance with A/ 53 and CEA-708 falls to having a VANC de-embedder extract the data from the ATSC encoder. Several different methods may be the HD-SDI stream and feed the 608 caption data to an used for getting the caption data into the encoder, analog NTSC or SDI line 21 closed caption inserter, as depending on system configuration and manufacturer shown in the lower-right portion of Figure 5.24-10. and the source of the caption data, as follo~s: Some companies offer equipment that can transcode 608 data from VANC CDPs directly onto line 21 in a 1. As shown in Figure 5.24-10, a VANC CDP de- 'C single device. embedder can extract the CDP stream from V~ t in an HD-SDI or SDI video feed and transform it 0 the data format expected by the ATSC encode:· 1 Server-Based Captioning Most ATSC encoders have a caption input sena port. This port accepts data in one of two docu- a' The techniques described previously for carrying cap mented formats: SMPTE 333M, which is J 1450 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 19 of 21 CHAPTER 5.24: CLOSED CAPTIONING SYSTEMS tively, as shown in Figure 5.24-11, a local 708 manufacturers. A proprietary message format has 1. A]te:na encoder without VANC embedding can be been developed by one manufacturer to allow the · cap~~~ feed the ATSC ~ncoder using a seria.l c~n CSD and also the content advisory descriptor to be use . and taking caption data from a captiorung provided from a caption encoder or VANC de-embed necnonter or live authoring workstation. Figure der to a PSIP generator upon request. ~oIIlP~ does not show the video routing ~nd . However, ATSC standard A/76, Programming Meta :i.24-l ents that may be needed for switching data Communications Protocol, defines an XML-based arraJ_l~g data signals for different sources. open standard for communication of PSIP-related capho metadata between equipment in the facility. This facil hown in Figure 5.24-11, a 608 to 708 translator I itates transfer of both current and future program l. Ass coder unit without VANC embedding can also metadata, including the caption service descriptor, to trans ed to feed the ATSC encoder, with captions the PSIP generator. This is the preferred method of be·~nating from a legacy line 21 caption source. Oflo- PSIP metadata communications. • _ e ATSC encoders can accept analog NTSC or : J. ~f;video and generate the 708 DTVCC captions transmission from 608 legacy captions present 1;;line 21 of the video, without using an external MONITORING OF DTV CAPTIONS !OB caption encoder. Note that some e~rly. ATSC Monitoring of 608 and 708 caption encoding and con encoders inserted only the 608 compatib1hty bytes. tent should be carried out at appropriate points in the Aco~bination of the preceding methods may be distribution chain to check for used. This requires that special switching (not shown • Presence of captioning information; . the drawings) is used to route to the ATSC encoder ~e appropriate caption data for DTV network, local • How captions will be displayed on a consumer sources, or NTSC converted sources. receiver (basic caption functionality); • Video/caption synchronization; • Regulatory compliance. PSIP AND THE CAPTION Caption data should be checked when the captions SERVICE DESCRIPTOR are generated and when programs are delivered. Mas Caption service information is required to be carried ter control facilities should be provided with monitor ing for closed captioning data and functions, both for h1 the DIV bitstream as set out in ATSC A/65, ATSC A/53, CEA-708, and ANSl/SCTE 54. Methods for incoming program material and for transmitted ociginating caption service information are not stan broadcast signals. Points to note for 708 caption moni dardized, and different users and manufacturers may toring include r.ire different arrangements for generating the cap 1. Where 708 captioning data is carried in VANC, a tion service descriptor (CSD). It cannot be assumed VANC de-embedder with appropriate caption iiat all caption encoders generate complete and cor decoders should be used to examine the 708 and lel:t caption service information for the caption ser 608 captions and associated data. Some 708 encod rices as required for insertion in the PSIP tables. ers themselves provide the capability for monitor For systems using CDPs carried in VANC, the CSDs ing and verifying embedded caption data. maybe present in the CDP. SMPTE RP207-2002, Trans 2. Verify correct insertion of the 708 and 608 caption 'TJTI of Program Description Data in Ancillary Data Pack ~· describes another means of encoding CSDs into data in the outgoing DTV bitstream. Presence of the 1A.\JC. In either case, the CSD data may be extracted caption data and the caption service descriptor in •hen the VANC data is disembedded and fed to the the ATSC transport stream should be checked using ~IP generator or, where appropriate, the ATSC an MPEG stream monitor. :oder. That equipment should enter the CSD into 3. Monitor the broadcast 708 captions using an off-air · PMT and current program EIT table entries. DTV receiver with 708 closed captioning decoding )deally, local 708 caption encoders feeding the capability. Professional receivers usually have more '.SC encoder and PSIP generator directly without comprehensive functionality for this than typical :gVANc, as shown in Figure 5.24-11, should also consumer receivers and may also provide a bit ~erate ~e caption service information. Caption ser stream output (typically ASI) derived from the • descnptors to be entered in PSIP tables for future demodulated RF, which can feed an MPEG stream :ts have, of necessity, to be generated and distrib analyzer. In addition, professional receivers may \rr separately from the captioned video service. have a transport stream input that can be used to ~angements for this will vary depending on the monitor the bitstream output from an ATSC ' Pment and system design. encoder prior to sending to the transmitter. This capability will assist in system troubleshooting. Data I nterface to the PSIP Generator 4. DTV caption monitoring should include the associ ~~ ated 608 caption data using the derived NTSC sig ~:~sfor conveying CSD data from the VANC data nal generated by the DTV receiver to feed 608 edder to the PSIP generator vary between monitoring equipment. 1451 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 20 of 21 ¥/ii@""" . / SECTION 5: VIDEO PRODUCTION AND STUDIO TECHNOLOGY 5. Monitoring of cable television captioning will ATSC Standard A/ 65C: Program and System Infonna . require use of a suitable cable television interface (PSIP) for Terrestrial Broadcast and Cable, Revj~~~ Prot0c01 Amendment No. 1, January, 2006, as amended Ma C, With 2 unit or integrated receiver able to select and display Consumer Electronics Association. Documents avail~bl 006 the DTV captions. bal Engineering Documents. See http:/ I global.ihs.c e frorn G!o. CEA-608-D - 2006, Line 21 Data Services orn; CEA-708-C - 2006, Digital Television (DTV) Closed C . Verification Consumer Electronics Association. aptioning. Federal Communications Commission. See http:/ IWww FCC 00-259 - 2000, Report and Order, Closed Captioflin .fee.go\'/ Consideration needs to be given to arrangements for ments for Digital Television Receivers g ReqUire- proving captions were correctly broadcast in the event FCC Public Notice DA-04-2, JanGary 6, 2004, Notice to V that a complaint of noncompliance is received. One gramming Distributors and the Public of the Januar Ideo Pro. Requirements for the Closed Captioning of New Ny l, 2fX4, possible arrangement would be to maintain an off-air 0 recorded log of station output with a bitstream English and Spanish Language Video Progra .nexempt Reminder Regarding Other Captioning Requiremen~g and recorder-disk-based server or tape-based system. FCC 04-192Al - 2004, Report and Order, Second Periodic R . the Commission's Rules and Policies Affecting the Con eviewo1 Digital Television version to National Center for Accessible Media. · See h SUMMARY ncam.wgbh.org/ Up:// Delivering Captions in DTV, DTV Access Brief, October 2002 For nearly three decades, closed captioning has been Society of Motion Picture and Television Engineers. See http· , an important enhancement to television programming smpte.org/ ./: SMPTE 333M-1999, Television-TV Closed-Caption Server for a large number of deaf, hearing-impaired, and Encoder Interface to other television viewers. FCC-mandated requirements SMPTE 334-1-2006, Television-Vertical Ancillary Data Mappin . to caption an increasing amount of material have Caption Data and Other Data got made the service indispensable to many viewers. SMPTE 334-2-2006, Caption Distribution Packet (CDP) Definition Technological developments have made it easier for SMPTE EG 43-2004, System Implementation of CEA-708-B and CEA-608-B Closed Captioning broadcasters to comply with the FCC requirements and improve the service being offered, and such enhancements are still ongoing. Meanwhile, the migration to DTV captioning has created a new set of opportunities and challenges, but it is comparatively recently that standards have been completed that properly address all aspects of the caption production and distribution chain. Equipment manufacturers, program makers, and broadcasters need to work together to implement complete systems that exploit the capabilities of the CEA-708 standard to provide even better captioning services in the years ahead. The reader is referred to the bibliography and other sources of information listed below for more compre hensive data. ff· ACKNOWLEDGMENTS / Thanks are extended to Graham Jones of NAB, who wrote the NAB 2004 Broadcast Engineering Confer ence Paper that forms the basis of the DTV captioning portion of this chapter. The authors extend thanks to SMPTE for permission to quote from and summarize parts of EG 43, System Implementation of CEA-708-B and CEA-608-B Closed Captioning. Bibliography Society of Cable Telecommunications Engineers. See http:/ I scte.org/home.cfm ANSI/SCTE 43 2004, Digital Video Systems Characteristics Stan dard for Cable Television ANSI/SCTE 54 2003, Digital Video Service Multiplex and Transport System Standard for Cable Television Advanced Television Systems Committee. See http:/ I www.atsc.org/ ATSC Standard A/53E: Digital Television Standard, Revision E, with Amendment No. 1, December 2005, as amended April 2006. 1452 Ultratec Exhibit 1008 Ultratec v Sorenson IP Holdings Page 21 of 21