ARTICLE IN PRESS Computers & Graphics 28 (2004) 157–166 User interface development for interactive television: extending a commercialDTV platformto the virtualchannelAPI Konstantinos Chorianopoulos*, Diomidis Spinellis Athens University of Economics & Business,Patision 76, GR-104 34 Athens, Greece Abstract We explore the generation of interactive computer graphics at digital set-top boxes in place of the fixed graphics that were embedded to the television video before the broadcast. This direction raises new requirements for user interface development, since the graphics are merged with video at each set-top box dynamically, without the traditional quality control from the television producers. Besides the technical issues, interactive computer graphics for television should be evaluated by television viewers. We employ an animated character in an interactive music television application that was evaluated by consumers, and was developed using the Virtual Channel Control Library, a custom high-level API, that was built using Microsoft Windows and TV technologies. r 2004 Elsevier Ltd. All rights reserved. Keywords: Digitalset-top box; User interface; Animated character; Music video clip;TiVo 1. Introduction Up to now, the TV viewers’ interactive experience has been stuck at the Teletext, which is actually an Computer graphics have played an instrumental role information tool and is usually unrelated to the running in delivering the TV experience [1,2]. For example, television content. Recent advances in STB technology computer graphics and animation have been used widely have introduced realtime video capturing and rich in the post-production of television content (Fig. 1) for multimedia at consumers’ homes. Digital STBs like inserting the channellogo,animated intros/endings, and TiVo store television content, while the user controls the displaying various kinds of information (sport statistics, television flow with an on-screen user interface. In quiz show status, stock market ticker, news ticker, etc.). addition to dynamically embedded graphics for ad- In most cases, computer graphics are merged with vanced user interfaces, television content can now be audiovisualcontent and are converted to video at the enriched with rich computer generated content, like production studio or at the broadcast station. The final animated characters and Internet information sources. video is transmitted and displayed from TVs in its fixed For supporting consumer interactivity, we exploit form, without any opportunity for local dynamic update locally stored music video clips and Internet information of the embedded computer graphics. For supporting about artists and songs, to build an easy to use high level development for digital STB applications and interactive music television application. We have devel- local generation of dynamic computer graphics, we have oped the music ITV application using the VCCLib and defined the VirtualChannelmodeland implementedthe we report our experience at two levels: 1) development respective VirtualChannelControlLibrary (VCCLib), and 2) usability evaluation. which was built on top of the Microsoft Windows and TV (MSTV) platforms. 2. Previous research *Corresponding author. Tel.: +30-210-8203663; fax: +30- 210-8203664. With the advent of digitaltelevision(DTV) transmis- E-mail address: [email protected] (K. Chorianopoulos). sion, Internet-enabled set-top boxes (STBs) and digital 0097-8493/$ - see front matter r 2004 Elsevier Ltd. All rights reserved. doi:10.1016/j.cag.2003.12.004 ARTICLE IN PRESS 158 K. Chorianopoulos, D. Spinellis / Computers & Graphics 28 (2004) 157–166 Fig. 1. Computer graphics applications for television post production: News ticker from Fox news (top left), Financial news from Bloomberg (top right), Music information from MTV (bottom left), Channel Mosaic from Disney (bottom right). video recorders (DVRs), consumers are starting to have a medium of entertainment and passive information the need for a multimedia experience that seamlessly discovery [9]. The television experience usually consists integrates diverse sources of information and entertain- of two layers: (1) the background is reserved for video ment content. A few years before the emergence and play-out, while (2) the foreground is used to display widespread adoption of the Internet and the Web, overlaid information. Both layers are traditionally researchers were suggesting an integrated computer– created and controlled at the production studio or at television product [3,4]. Nevertheless, immature technol- the broadcasting station. For future ITV systems, we ogy, technicaldifficulties,reduced consumer demand, propose that the digitalSTB shouldbe imagined as a failed business models [5,6] and the success of the Web virtualchannelprovider, where the perceived TV have been postponing the convergence between the experience is produced from a combination of local computer and the television. Previous research has storage, broadcast transmissions and Internet resources separately addressed the integration of television (a) of audiovisualcontent, applicationsand text data/ with the Internet [7] and (b) with locally stored content metadata. [8], but there is still no integrated approach for both Thus, the name VirtualChannelrefers to the types of content. television channels not being static audiovisual experi- ences that are shared by all TV viewers in the same way, 2.1. The virtual channel model for ITV but a dynamic synthesis of discrete video, graphics, and data controlled at the consumer’s STB. The core idea Digitalbroadcast, persistent localstorageand Inter- behind the VirtualChannelproposal [10], as depicted in net resources should be used to augment the television as Fig. 2, is that the decision-making about media ARTICLE IN PRESS K. Chorianopoulos, D. Spinellis / Computers & Graphics 28 (2004) 157–166 159 Traditional Channel Play-out segm 1 segm 2 segm 3 segm 4 Television programming is experienced the way it has been transmitted info 2 info 4 Virtual Channel Play-out Virtual Television channels are segm 2 segm 4 segm 3 segm 5 created at the STB info 2 info 3 info 1 segm: represents a discrete Storage and media (audiovisual) dynamic programming segment synthesis of programs from info: information that appears the STB overlaid to the audiovisual programming Fig. 2. A generic modelof a system employingthe virtualchannelmetaphor, contrasted to traditionalbroadcasting. programming has shifted from the media source to the will feel unfamiliar. ITV applications should support STB. The television experience is now created and continuous video flow by default, unless consumers controlled at the STB. In the present paper, we are actively select to stop the video. introducing an implementation of the Virtual Channel * Advertising breaks: The cost of TV production is very modelthat supports a few of the model’smost high and it has been traditionally supported by important properties: advertising, at least at some part, which is the case even for subscription schemes. ITV could also * Local storage1: A digitalSTB with a hard disk like enhance the traditionaladvertising modelswith TiVo allows the consumer to time-shift, pause and personalization and new advertising schemes. * control the otherwise linear flow of the television Time-driven user interface: The appearance of an broadcast. interactive element on screen can be triggered by the * Internet resources: Data broadcasting may be used to user, but for the most part it is the application logic provide real-time updates of popular content like and producers’ rules that define when the consumer stock quotes, but the Internet is more flexible for may interact with additionalcontent. * providing personalized information to a diverse Relaxed control: Watching TV content has tradition- audience. ally been a passive and low engagement activity. User * Video overlays: Traditionaltelevisioncontent re- interfaces for augmenting TV content should support mains at the core of ITV services and can be relaxed use. optionally enhanced with interactive elements (user interfaces) and with additionalpersonalizedinforma- 2.2. ITV platforms tion that appear inside unobtrusive semi-transparent windows or at the edges of the screen. The ITV platform market is dominated by simple * Continuous video flow: A TV screen that stays still is digitalSTBs, alsoknown as integrated receivers/ beyond the previous experience of consumers and decoders (IRDs), running each manufacturer’s realtime operating system and offering limited external program- 1 Hereafter, italics will be used to highlight a property of the mability. IRDs’ market dominance is followed by a few VirtualChannelmodelwhenever it is discussed, implemented, competing, incompatible, and proprietary APIs (e.g. or employed in an ITV application. OpenTV, Liberate). In an industry that is driven by ARTICLE IN PRESS 160 K. Chorianopoulos, D. Spinellis / Computers & Graphics 28 (2004) 157–166 sheer volume, application developers have to develop the same application for multiple APIs. There are also a ITV Applications few independent organizations that define standards for ITV application development, like the TV-Anytime TV Producer-Friendly High- Low-level access to forum (http://www.tv-anytime.org), although member Level API (VCCLib) DTV platform organizations are not obligated to conform. Despite the many alternative choices, researchers and engineers with DTV Platform API (e.g. MSTV, OpenTV, MHP) an information technology background will find more flexibility and familiarity