Evaluating Visual Information Provided by Audio Description E. Peli, E.M. Fine, A.T. Labianca
Abstract: Short segments of two TV programs without audio description (AD) were presented to 25 subjects with low vision and 24 subjects with normal vision, and 29 additional subjects heard only the standard audio portions. The subjects then answered questions based on the visual information contained in the AD of the programs. The subjects with normal vision performed the best. followed by those with low vision and those who heard only the audio portion; all performed at better than chance levels. The results indicate that although AD may provide information on visual details to visually impaired audiences, some of the information in the AD may be obtained from the standard audio portion.
Television (TV) and other video outlets (Cronin & King, 1990). Descriptions of de are an important means of obtaining tails concerning such aspects of a scene as information and sharing in the culture of a clothing styles, body language, colors, and society. For people with visual impair landscapes are inserted during pauses in the ments, the visual aspects of these media dialogue or narration of normal programs. are impoverished or absent. Nevertheless, The AD is typically broadcast via the sepa most visually impaired people watch TV rate audio program (SAP) channel that is a with their families and use other video standard feature of most stereo TVs and vid sources, such as videotaped movies and eocassette recorders (VCRs). ADs of pro computer video displays (Berkowitz, Hiatt, grams broadcast on the SAP are available deToledo, Shapiro, Lurie, 1979; Joseph free of charge and do not interfere with.other son, 1968). In fact, visually impaired viewers' enjoyment of the programs. AD is people watch as much or more TV as do also available with VCRs, for which narra sighted people (Cronin & King, 1990). tion is simply added to the videotapes, which Audio description (AD) provides descrip are played on standard equipment. Although tions of the visual elements of TV programs the reviews by visually impaired audiences without interfering with the programs' audio have been positive, AD was not formally evaluated with visually impaired people un This research was supported by NIH Grant EY05957 to E. Peli. The authors wish to thank Julie Irish for til the study presented here. her help in developing the questions used in this Rabbitt and Carmichael (1993) exam study; Barry Cronin and Gerry Field, of WGBH. ined the comprehension by older adults Boston. for providing the program videos, software. with normal vision of short segments of and valuable technical help: and Laurie A. Everett. of WGBH's Descriptive Video Service. for her programs. as well as of an entire 30-minute comments on an earlier draft of the article. program. presented to half the subjects
A378 )ounlal of Visual Impairmenr & Blil1dness. Sept-Oct lQ
1 1 two groups on the basis of the acuity in measured. It should be noted that the AO their better eye. Normal vision (NV) was condition is not a simulation of a truly defined as acuity of 20/40 or better in the blind audience, which may have adapted to better eye and low vision (LV) as acuity long-term vision loss by gathering infor between 20/1 00 and 20/800 in their better mation through other sensory inputs (pri eye. Adults who did not meet the criterion marily auditory). Thus, it is likely that the for either group were excluded. AO group would have been outperformed A total of 24 NV subjects (9 who viewed by a group of blind observers who are only Nature; 9, only Mystery!; and 6, both) presumably more skillful at gaining infor and 25 LV subjects (10 who viewed only mation from nonvisual sources. Nature; 10, only Mystery!; and 5, both) participated. There was no difference in ApPARATUS mean age between the vision groups (NV An Apple Macintosh SE was used to run = 71, SEM 1.63; LV = 75, SEM a HyperCard® stack, deScriptor version 2.10; t(45) -1.51, P = .14). The 1.2.1 (DVS, Boston). Modifications to this median acuities for the NV and LV groups stack allowed the authors both to control were 20/30 and 20/200, respectively. the presentation of the video and to record Twenty-nine additional subjects (2 who subjects' responses. The computer "was heard only Nature; 7, only Mystery!; and 20, interfaced with a Sony U-matic/SP VCR both) were recruited to participate in the au via two V ideo media Serial I/O units that dio-only (AO) condition. The age of this sam synchronized the time code between the ple [mean age = 51, SEM 4.15] was not computer and the VCR. restricted, and the authors continued to col DVS deScriptor stacks were originally de leet data from the AO group beyond the num veloped to add AD to TV programs. Using ber of subjects in the NV and LV groups for this system, a describer can control the VCR another study. Since both the audio portions to navigate precisely through and display par and the ADs of programs rely on hearing, ticular scenes. In addition, a description can and many visually impaired people are el be added by advancing to the appropriate scene derly and may have some hearing loss, the and adding the narration to a script field des authors explored whether age might affect ignated for that scene. The authors added ques performance on this task. There was no sta tion-and-response fields in lieu of the narra tistical difference in performance between tion script field that DVS created. They also the 55-or-older group and the younger group modified the software to export response datil [t(47) < 1.0. n.s.], and so decided to main automatically from the response fields to tain the larger sample size for the analyses. spreadsheet format. These modifications re The AO group neither saw the video nor duced data-entry errors, as well as any risk heard the AD, so their responses were that the experimenter would ask questions or based only on the standard audio of the record responses that did not correspond to program. Since the purpose of this condi the correct scene or question. tion was to verify that questions could not The video segments were displayed on a be answered on the basis of the standard 27-inch Sony Trinitron color TV monitor. audio alone. these subjects' acuity was not Acuity was tested monocularly using a 380 Journal ot Visual Il11l'airl1l(,1If & Blindne...s. Sept·Oct 1996 ©1996 AFB. All Rig.hl~ Reserved 1 I ,i 1£' ,~ , ~ ~ " ~. r:.. ~ : .. : ~ ~;~~: J!.~~",~~~~f;:;!~ .4'..t1'.,~~jo e>>i, Mentor B-VAT II, a system in which sans ity of the episode. The continuous 10 min serif letters are randomly chosen from a utes of film was parsed into 21 and 17 seg limited set and can be displayed individu ments for Nature and Mystery!, respectively. ally on a 12-inch monitor. DVS descriptions were used as a basis for choosing the visual details that are assumed MATERIALS AND DESIGN to be unavailable to persons with visual im Since DVS is available to TV audiences pairments. Although questions were not de across the country both by broadcast media veloped from the entire DVS script, the 10 and by the production of videotapes with minute segments chosen from each episode an AD track, episodes of two TV programs were representative of the descriptions used that had been broadcast with DVS descrip throughout the program. Two-alternative tive narration were chosen: "Seasons in the forced-choice questions (43 for the Nature Sea" from the Nature series and "Poirot: episode and 59 for the Mystery! episode) were The Theft of the Royal Ruby" from the generated on the basis of the DVS descrip Mystery! series. A continuous series of tions of the corresponding scenes (see Box 1 segments of each episode (approximately for examples). These questions were de 10 minutes in length) was presented with signed to test whether a visual detail de only their normal audio (without the DVS scribed by DVS was seen or not seen. After narrations). The addition of AD is nor a third party developed the questions, the mally constrained by the breaks in the first and third authors examined the DVS standard audio of the program. AD can script and the video portions of the programs occur simultaneously with, before, or after to determine whether the questions did, in the visual element described. When possi fact, ask about specific features both con ble, the episode was paused to ask tained in the segment and described by DVS. questions about a segment at the same When they disagreed about the validity of a point that DVS parsed it for inserting question, they decided either to eliminate it narration. When the visual portion did not or to modify it. Because the number of vi coincide with the DVS script, a segment of sual details described in the DVS narration the video that contained the portion for each scene differed, the number of ques corresponding to that DVS narration was tions per segment varied. There was an av presented. Portions of the video that did erage of 2.0 questions per segment for the not directly correspond to the questions Nature episode and 3.5 questions. per seg were also included to maintain the continu ment for the Mystery! episode. Nature Mystery! From the side, the jellyfish's red cap surges grace- A smooth-faced, gray-haired woman. She glances at fully forward, towing ruffled tentacles with rose- the door. She paces hesitanlty across the drawing tinted edges. room. 1. The jellyfish's arms are ... 1. The woman has ... a. smooth a. many wrinkles b. ruffled b. a smooth face 2. The arms have . . . 2. The woman has ... a. greenish blue edges a. gray hair i b. reddish pink edges b. lack hair Bo.\ 1. Examples of AD and corresponding questions. ©1996 AFB. All Rights Reserved Journal or \'isual iml'oirm{,111 & B"I/(IIll'ss. S~pl-Oct 381 ( ! i Acuity testing vision viewers may normally sit closer to Acuity was measured monocularly with their TVs, the authors chose to standardize the same refractive correction used for the viewing distance. viewing the TV programs and with the With the room lights on, the video seg room lights off, using a Mentor B-V AT II. ments were presented without the AD. After For each eye, subjects were asked to name presenting each segment with only the reg letters, beginning at a letter size corre ular program aUdio, the experimenter read the questions and possible answers to the sponding to 20/600, that appeared individ ually on the screen. They stood 10 feet questions that corresponded to the segment just presented and recorded the subject's ver from the monitor, which was adjusted for testing from 20 feet. This combination bal responses. When multiple questions were derived from a single segment, each ques allowed verification of acuity from 20/30 tion and its possible answers were pre to 20/600. The target size was decreased until fewer than four of five letters were sented, and the subject's response was re corded before the next question was read. correctly identified. The size was then The questions always followed the order of increased, one step at a time, until four of events in the DVS narration. When unsure, the five letters were named correctly. This subjects were asked to guess, and all ques target size was recorded as the acuity for tions were answered by all subjects. Several the eye tested. Subjects who were unable subjects completed only the first program to identify letters at the 20/600 size were because of time constraints. Approximately moved closer to the screen (5 feet), thus 45 minutes were required to administer each increasing the relative character size to program. The same procedure was followed 20/1200. If their acuity was better than for the AO group except that the television 20/30, the B-VAT II was adjusted for was turned to face the wall. testing from 10 feet, which allowed verification of acuity down to 20115. Results and discussion Since the purpose of the study was to Procedure determine the amount of visual information Presentation ofprograms obtained from the video by persons with After the acuity testing, each subject low vision, questions that could be an was told that he or she would watch two swered on the basis of the standard audio short portions of TV programs that were of the program and questions whose parsed into several segments and answer correct answer could not be chosen based multiple-choice questions based on each on normal viewing were eliminated from segment. The presentation order of the two the analyses. A strict criterion was used for programs was counterbalanced. The vol exclusion, based on the performance of ume was adjusted to accommodate each both the AO and NV groups, to eliminate a subject. Each subject was seated approxi total of five questions. Four questions (2 mately 6 feet from the TV monitor and from Mystery! and 2 from Nature) were wore his or her usual refraction for eliminated because they were answered distance viewing. Although some low correctly by 90 percem or more of the AO 382 Journal of ViStlal IlIIl'airlll(:,11l &: Blindness, S.:pl·Oct 19% © J996 ..>.FB. All Righl~ R.:serwd I D' ,It.'iit subjects, suggesting that they could be programs (p = < .0005 for all analyses). answered by listening to the standard audio This finding indicates that at least some of alone or on the basis of common knowl the visual information contained in the AD edge. The remaining question (from Mys descriptions can be attained by low vision tery!) was eliminated because it was viewers and that some benefit can be derived answered correctly by 10 percent or fewer from these descriptions. Had all the visual of the NV subjects, suggesting that it could information contained in the AD been avail not be answered on the basis of normal able to the low vision viewers, it is unlikely viewing. that their performance would have been re The proportion of questions answered duced relative to the NV group. correctly by each subject for each episode The AO group was included primarily to (see Table 1) was computed, and a 3 assess the validity of the questions. but their (vision group: NV, LV, or AO) by 2 data can be used to estimate the minimum (program: Mystery! or Nature) ANOVA level of performance that could be expected was used to evaluate the data. Because not from blind audiences engaged in the same all the subjects in each condition saw both task. It is evident from the preceding anal programs, the data from each viewing of ysis that when only the information con each program were treated as independent tained in the audio portion of the programs in the analyses. Although this is an was available to these subjects, their perfor unconventional method of analysis, it mance was reduced relative to both subjects results in an increase in the estimates of with normal vision and with partial sight. variability within each vision group and However, their performance did not drop to thereby decreases the likelihood of finding chance levels (50%), even when questions differences among these groups. with 90 percent or better performance were The results of the ANOV A indicated a eliminated. Although the 4 percent and 5 main effect of vision group [F(2,103) percent differences from chance perfor 130.59, P = .0001] and a main effect of mance for the AO group answering ques program [F(l,103) = 6.77,p = .0107], but tions from Nature and Mystery!, respec no interaction between these variables tively, were small, they were statistically [F(2,103) = 1.84, P = .1639]. Bonferronil greater than chance (1 > 3.0, p < .006 for Dunn's post-hoc analyses showed that the both). Two conclusions can be drawn from NV group performed better than both the LV this finding. First, the AD contains some and AO groups and that the LV group per visual information redundant to the standard formed better than the AO group for both audio or common knowledge, and second, Table 1 Mean percentage of questions answered correctly hy three groups of observers (NV, LV, and AO) in each episode presented (Nature alld Mystery!). '" Nature Mystery! n Mean 87.3 SEM 1.2 * NY 01996 AFB. All Rights Reserwd Journal of Visual Impairment & Blindness. Sept·Oct 383 f blind audiences would be able to acquire at auditory inputs. Again, their performance least this much of the visual detail contained would diminish the difference between in the program without AD. groups and strengthen the conclusion that As was discussed earlier, the questions some portions of the AD are redundant. were developed on the basis of the visual details in the AD narration, portions of Conclusion which described aspects of a scene that This investigation has shown that with were evident without seeing the video out hearing AD, low vision observers and portion. That almost 16 percent of the persons who were exposed only to the questions were answered correctly by 80 audio portion of the programs answered percent or more of the subjects in the AO fewer questions correctly than did nor condition suggests that some of the content mally sighted viewers. This finding sug of the AD could be identified from the gests that the information contained in AD audio portion alone or from common narrations may provide benefits specifi knowledge. For instance, the first sample cally related to the availability of visUal question from Nature, shown in Box I, details to both visually impaired and blind can be answered by listening to the viewers. There are also indications, how standard audio, which states: "A gigantic ever, that some of the AD narrative was pelagic jellyfish . . . has extravagantly redundant with the audio portion of the frilled oral arms trailing some 20 feet from programs. Although the inclusion of re its mouth." The corresponding description dundant and extra information is not regarding the "ruffled tentacles" is redun inherently negative, the time may be better dant here, used to describe elements of programs that Since persons with low vision frequently visually impaired viewers and blind audi sit closer than 6 feet from the TV screen ences are unable to obtain elsewhere. and use optical aids (in addition to normal However, if the extra information provided refraction) to watch TV, it is likely that the by AD is necessary for visually impaired overall performance of the LV group and blind viewers to follow a story or would have improved if each LV subject enjoy a program, AD may be an ideal had been allowed to choose his or her own sensory substitute for viewers who cannot viewing distance. If it had, it would have fully appreciate visual details. reduced the difference between the NV and Totally blind people represent a small LV groups and would more strongly minority of the visually impaired popula suggest that the AD provides information tion. Most visually impaired people have that persons with low vision can obtain partial sight, and the majority are elderly elsewhere, The same argument can be and have some degree of hearing loss in made about the performance of the AO addition to their visual impairment (Rab group relative to a blind audience. That is, bitt & Carmichael, 1993), Thus, many of a blind audience would be likely to the viewers for whom AD is designed outperform the AO group because they may not be able to benefit from it. ha\'e adapted to the loss of sight and are Furthermore, the availability of AD is more adept at gaining information from likely to remain limited (currently, only 384 }u"'"dl of I'is/IOI imf'l1irll1{'ll/ & Blind"<,,,.' , Sepl-Oel 1996 ©1996 ..\FS. All Ri!!hl" Rc"ened about 10 PBS programs per week are broadcast TV, VCR output, and computer broadcast with DVS), and its applicability generated displays. is generally limited to off-line production, although live broadcasts, such as U.S. References President Clinton's inauguration, which Berkowitz, M., Hiatt, L.G., deToledo, P., was broadcast with AD, are possible. In Shapiro, 1., & Lurie, M. (1979). Character addition, the production of AD is time istics, activities and needs of people with consuming and costly relative to other limitation in reading prim. New York: services (such as closed captioning) that American Foundation for the Blind. are designed for audiences with disabili Cronin, B.J. & King, S.R. (1990). The ties, although not in relation to overall development of descriptive video service. Journal of Visual Impairment & Blindness, production costs. For example, the pro 80, 503-506. duction and narration costs of adding Josephson, E. (1968). The spatial life of blind DVS to a PBS documentary are about people (Research Series 19). New York: two-thirds of 1 percent of the total American Foundation for the Blind. production costs, and the costs of adding Peli, E., Fine, E.M., & Pisilno, K. (1994). Video enhancement of text and movies for AD to a videotaped movie are only about the visually impaired. In A.C. Kooijman, one-fiftieth of 1 percent (personal commu P.L. Looijestijn, 1.A. Welling, and G.l. van nication with L.A. Everett, DVS, July der Wildt (Eds.), Low vision: Research and 24, 1995). new developments ill rehabilitation (pp. 191-198). Amsterdam: lOS Press. Despite some limitations, with today's Peli. E., Lee, E .. Trempe, c.L., & Buzney, S. technology, AD may still be the best (1994). Image enhancement for the visually solution for blind users. However, there impaired: The effects of enhancement on may be alternatives for low vision users. face recognition. Journal of the Optical Society of America A. 11(7). 1929-1939. For example, contrast enhancement has Rabbitt, P.M.A. & Carmichael, A. (1993). been shown to improve the recognition of Assessment of the effect of audio description faces by persons with low vision (Peli, on elderly people's comprehension and Lee, Trempe, & Buzney, 1994) and to memory for television programmes. In AUDETEL Project Fillal Report, Vol. 2: increase viewers' appreciation and identifi Optimization of all audio description service cation of details of videotaped movies for visually halldicapped and elderly people: (Peli, Fine, & Pisano, 1994). Using the Humall and technical factors. London: questions developed for this study, the Independent Television Commission. authors will be able to evaluate systemati Eli Peli. M.Sc., O.D., seniol' scielltist, Schepens Eye cally the efficacy of contrast enhancement Research InstiTute. alld associaTe professor. Harvard Medical School. 20 Stalliford S,,·eeT. Bostoll. MA for low vision users. If it is found that 02114: E-mail: © 1996 AFB. All Rights Reserved Journ,ll of Visual 11Il1'(lirmf!nI & Blincin"H. Sept·Oct 385 ARTICLES Aging Visual Function and Visual Acuity in an Urban Adult Population-J. Katz, J .M. Tielsch Audio Evaluating Visual Information Provided by Audio Description Description-E. Peli, E.M. Fine, A.T. Labianca Children Knowledge of Basic Concepts of Young Students with Visual Impairments Who Are Monolingual or Bilingual-M. Milian Meaningful Participation in Early Childhood General Education Using Natural Supports-E.J. Erwin Students with Visual Impairments: Do They Understand Their Disability?-S.Z. Sacks, A.L. Com Blind and Sighted Children with Their Mothers: The Development of Discourse Skills L.S. Kekelis, P.M. Prinz DEPARTMENTS Editor's Page· Comment· Letter· Research Notes· Book Review· Demographics Update· Classified III ~ 111111111111111111111111111111111111111111111111111111I ~145-482X (199609/10J 90, 5,1-4 -.': Contents ARTICLES Aging Visual Function and Visual Acuity in an Urban Adult Population-J. Katz, J.M. Tielsch 367 Audio Description Evaluating Visual Infonnation Provided by Audio Description E. Peli, E.M. Fine, A.T. Labianca 378 Children Knowledge of Basic Concepts of Young Students with Visual Impainnents Who Are Monolingual or Bilingual-M. Milian 386 Meaningful Participation in Early Childhood General Education Using Natural Supports EJ. Erwin 400 Students with Visual Impainnents: Do They Understand Their Disability? J S.Z. Sacks, A.L. Corn 412 r Blind and Sighted Children with Their Mothers: The Development of Discourse Skills LS. Kekelis. P.M. Prinz 423 DEPARTMENTS Editor's Page 363 Comment 364 Letter 438 Research Notes Book Review Demographics Update Classified 456 The Journal of Visual Impairment & Blindness (lSSN: 0145-482X) is published bi-monthly by AFB Press. American Foundation for the Blind. 11 Penn Plaza. Suite 300. New Yorl.:. NY 10001; individual subscriptions, $59 per year, institutional subscriptions, $87 per year. Periodicals postage paid at New York. New York and at additional post offices. POSTMASTER: St:nd address changes and subscription orders to Jour/wi of Visual Impairment & Blindness, 49 Sheridan A'-e. Albany. NY 12210. All .ights reserved. No pan of this work may be reproduced or transmitted in any form or by any means. electronic or mechanical. including photocopying and recording. or by any information storage or AFB retrieval system. except as may be expressly permitted by the 1976 Copyright Act or in writing from the • • ••• publisher. Requests for permission should be addressed in writing to Journal of Visual Impairment & Blindness. II Penn Plaza. Suite 300. New Yorl.:. NY 10001. Printed by Boyd Printing Company. Inc .. PRESS Albany. NY. Copyright © 1996 American Foundation for the Blind.