Letter of Transmittal

School of Engineering Science Simon Fraser University Burnaby, BC, V5A 1S6

March 4, 2007 Dr. Lakshman One School of Engineering Science Simon Fraser University Burnaby, BC, V5A 1S6

Re: ENSC 440 Design Specification for Next Generation IPTV Set-Top Box Proof of Concept

Dear Dr. One,

Attached with this letter is Ensight Solution’s Design Specification: Next Generation IPTV Set-Top Box Proof of Concept, which is a document outlining the detailed design specifications as described previously in our project proposal. We are in the process of developing a proof of concept software application for IPTV that will be commercialized in the near future. The objective is to realize a system running on an ARM/DSP CPU based development board in order to display Internet media streams on a standard television set without using a PC. This project is done under the guidance and support of Dr. Farid Azhar of Varietize Technologies.

The purpose of the design specification document is to provide detailed design requirements to be achieved at the end of our proof of concept phase in our targeted project completion date of April 2007. In this document, the reader will be able to discover in depth details as to how each of our functional requirements are implemented as stated in our previous functional specification document and test plans for each module.

Ensight Solutions is comprised of four undergraduate engineering students: Allen Lai, Hugo Kwok, David Shen and Jimmy Jeong. If you require more information, please do not hesitate to contact Allen Lai by phone at (778) 883-3376 or via e-mail at ensc440- [email protected].

Sincerely,

Allen Lai

President and CEO Ensight Solutions

Enclosure: Design Specifications for Next Generation IPTV Set-Top Box Proof of Concept

Design Specification for a NEXT GENERATION IPTV SET-TOP BOX

Project Team: Allen Lai Jimmy Jeong David Shen Pui Kang Kwok

Contact Person: Allen Lai [email protected]

Submitted To: Lakshman One – ENSC440 Steve Whitmore – ENSC305 School of Engineering Science Simon Fraser University

Submission Date: March 4, 2007

“We call it IPTV. And no doubt this is where the world is going." Bill Gates Chairman, Microsoft

Internet Protocol Television (IPTV) is a system where users are able to view TV contents over the publicly available Internet. In essence, IPTV contains two segments: live broadcasts and stored broadcasts or Video On Demand (VOD), in which users are able to stream contents after they are downloaded (ex. to a hard drive). With immense development and availability of broadband technologies, IPTV technology had been steadily increasing in existence and making its way into households worldwide due to its simple compatibility with only an Internet connection and a network detecting device such as a personal computer (PC), or 3G mobile cell phone. Ensight Solutions is determined to bring to the market the next generation IPTV set-top box which will be cross compatible with all operating standards of any IPTV service provider worldwide.

Development of the next generation IPTV set-top box will be performed under the guidance of Farid Azhar from Varietize Technologies. It will occur in two critical phases. Ensight Solutions is currently performing the proof of concept (POC) phase of the set-top box, with goals to achieve completion by April of 2007. In the POC stage, the ARM CPU based system set-top box will be able to:

• Stream real time Real Media codec from a remote server on the Internet • Convert Real Media codec through decode/encode algorithms into MPEG4 format • Play resulting audio/video with a media player (MPlayer) on a Linux development platform via an attached LCD screen on the development board

After the second phase of development, our application will be able to:

• Be extensible for future advanced developments in streaming real time audio/video media applications • Implement streaming of all other input media formats (ex. windows media, VLC, etc), convert and output them into one universal codec format (ex. MPEG4) • Migrating the system from ARM CPU to a C6000 digital signal processing (DSP) CPU for more efficient processing and algorithmic calculations

Upon the end of the allocated time frame, Ensight Solutions will be more equipped to further assess and discuss the feasibility and expected functionalities with Dr. Azhar into realizing the prototype to a commercially available product.

1 Introduction...... 1 1.1 Scope...... 1 1.2 Acronym ...... 1 1.3 Intended Audience ...... 2 2 System Overview...... 2 3 System Hardware Design...... 3 3.1 Evaluation Board ...... 3 3.2 Device Drivers ...... 4 3.2.1 Audio Device Driver...... 5 3.2.2 Video Device Driver...... 5 3.2.3 USB Device Driver...... 5 4 System Application Software Design ...... 5 4.1 Software Block diagram ...... 5 4.1.1 MPlayer...... 6 4.1.2 MEncoder...... 7 4.2 Real Media Codec...... 7 4.3 Media Stream...... 7 4.3.1 Transcoding...... 7 4.3.2 Demuxer...... 8 4.4 Conversion Block...... 9 5 Test Plan...... 10 5.1 Assumptions...... 11 5.2 System Hardware Test Plan...... 11 5.2.1 Component Connection Verification ...... 11 5.2.2 DC Power Verification ...... 11 5.2.3 Device Drivers Verification...... 11 5.2.4 EVEVM Software Verification...... 11 5.3 System Software Test Plan ...... 12 5.3.1 Media Transport Verification ...... 12 5.3.2 WAN Online media Streaming...... 12 5.3.3 LAN Helix Server Simulated Online Media Streaming ...... 12 5.3.4 Local Media Playback...... 12 5.3.5 TV Set Output...... 12 5.4 System Firmware Test Plan ...... 13 5.4.1 On Screen Display ...... 13 5.4.2 IR remote control ...... 13 6 Conclusion ...... 13 7 References...... 14

Figure 2-1: Streaming Media System Overview ...... 2 Figure 3-1: Structural Block Diagram of TMS320DM644x DaVinci Digital Media Processor...... 3 Figure 3-2: DaVinci TMS320DM6446 Architectural Diagram ...... 4 Figure 4-1: Overall System Design...... 6 Figure 4-2: General Block Diagram of MPlayer Functionality...... 6 Figure 4-3: General Block Diagram of MEncoder Functionality...... 7 Figure 4-4: Process of transcoding a media stream ...... 8 Figure 4-5: Conversion Algorithm...... 10

List of Tables

Table 4-1: MPlayer Supported Media...... 6

Revision History

Revision Date Description Name 0.1 March 4, 2007 Initial Version, with David Shen testplans, and other sections 0.2 March 5, 2007 Added Hardware Allen Lai Design 0.3 March 6, 2007 Added Software Jimmy Jeong Design 1.0 March 7, 2007 Added diagrams and Hugo Kwok edited for submission

Ensight Solutions’ next generation set-top box will be capable of handling streaming media from any IPTV service provider. For our application, the input stream will be comprised of the Real Media codec from any remote streaming server on the Internet using the RTP/RTSP protocol. The ARM and C6000 CPU based system will decode it into raw data, then encode it into the MPEG4 codec format and display it on a connected TV set in real time. The project will be developed in two stages comprising of a proof of concept device deliverable for April 2007, followed by further development in preparation for commercial deployment in accordance with Dr. Azhar of Varietize Technologies.

1.1 Scope This document is the design specification that describes Ensight Solution’s functional requirements that must be met in the development process for the next generation IPTV set-top box. A full set of functional requirements are listed for the POC system, however, only a partially list will be available for the production device.

This document acts as a guide to our company’s design and implementation process. The requirements will be traceable in the design documents.

1.2 Acronym AAC A lossy data compression scheme for streaming audio ARM Advanced RISC Machine AV Audio and Video CLI Command Line Interface CPU Central Processing Unit DEVM Digital Video Evaluation Module DP Demultiplexing Packet DS Demultiplexing Stream FAQ Frequently Asked Questions GUI Graphical User Interface IC Integrated Circuit IP Internet Protocol IPTV Internet Protocol Television IR Infrared LAN Local Area Network LCD Liquid crystal display MPEG Moving Picture Experts Group MPEG2 A standard from the Motion Picture Expert Group which define for digital television MPEG4 A standard from the Motion Picture Expert Group which define for multimedia application

PC Personal Computer POC Proof of Concept RM Multimedia container format created by RealNetworks RTP Real Time Protocol RTSP Real Time Streaming Protocol TV Television USB Universal Serial Bus VOD Video On Demand WAN Wide Area Network

1.3 Intended Audience The engineers of Ensight Solutions will use this document as a guideline during the development stages of the IPTV set-top box project; acting as a reference to evaluate progress along the way. The marketing team will use this document in planning attack strategies for sales. All specifications and descriptions contained in this document is the intellectual property of Ensight Solutions. 2 System Overview

Set top Box

Local Streaming Server

Figure 2-1: Streaming Media System Overview

This section is strictly informative. Figure 2-1 depicts the overview of an IP based streaming media system integrated with our proof of concept design. The next generation IPTV set-top box is connected over the Internet to a number of remote streaming servers. Upon user request, the set-top box streams live audio/video media using RTP/RTSP from a requested remote server, decode/encode it into an appropriate codec and displays the resulting media on a normal TV set. In addition to real time streaming, users are also able to store media into a local hard drive if requested, and play it at their convenience. This acts as the Video on Demand functionality of our set-top box.

3 System Hardware Design

3.1 Evaluation Board

A block diagram outlining the major components of our evaluation board is shown below, with suggested peripheral connections:

Figure 3-1: Structural Block Diagram of TMS320DM644x DaVinci Digital Media Processor

The DaVinci TMS320DM6446 is a system-on-chip that contains an ARM9 processor as well as the C6000 digital signal processor. This processor is useful for development of set top boxes as well as streaming media. It is available in two versions, the Digital Video Evaluation Module (DVEM) and the Digital Video Software Development Kit (DVSDK). For the purposes of this proof-of-concept model, we will be using the DVEM of the DM6446, because it provides much of the functionality that we need to develop our system at a fraction of the cost of the DVSDK. The DaVinci TMS320DM6446 is shown in Figure 3-1, and its architectural diagram is depicted in Figure 3-2.

The decoding and encoding of media streams will be developed to be run on the ARM processor. In later phases where this product is ready to be commercialized, the decoding and encoding tasks will be migrated to the C6000 DSP for increased efficiency.

This DVEM meets the following functional requirements for the proof-of-concept model:

• Powered by an external power supply • Accesses the internet through a standard 10/100 Base-T female Ethernet port • Receives video input from a RCA video jack • Outputs video signal through a S-Video jack and a RCA video jack • Outputs audio signal through a pair of RCA audio jacks

Figure 3-2: DaVinci TMS320DM6446 Architectural Diagram

3.2 Device Drivers

Three main device drivers need to be developed in our IPTV systems in order to allow interaction with the ARM development board, namely: audio, video and USB. These device driver interfaces are necessary to establish control and communication between the software and hardware of our application.

3.2.1 Audio Device Driver

The audio device driver is responsible for the interaction of the audio layer in the media. It is implemented in the software source code of the ARM CPU based board after the audio data had been encoded into the MPEG4 codec format from raw audio data in MPlayer. It takes the MPEG4 codec as its input and controls the interaction and communication with the attached TV set via the output port of the development board.

3.2.2 Video Device Driver

Similar to the audio device driver as mentioned previously, the video device driver is responsible for the interaction of the video layer. It is also implemented in the ARM CPU based board after the video data had been encoded into the MPEG4 codec format from raw video data in MPlayer. It takes the MPEG4 codec as its input and controls the interaction and communication with the attached TV set via the output port of the development board.

3.2.3 USB Device Driver

The USB device driver is responsible for establishing the data/port communication between the development board of the IPTV set-top box and other peripherals, such as a camera, for further encoding purposes. It controls and monitors the communication and acts as a bridge between the two devices. 4 System Application Software Design The set-top box shall stream various media format and convert it into MPEG4 video and MP2 audio format for storing and display purposes due to the following two reasons:

1. Easy to manage 2. Further software development focused on MPEG4 format media file

This section describes the software block diagram, major software components and the development based on these components.

4.1 Software Block diagram

The set top box is running on a Linux OS and all the multimedia playing and recording functions are implemented by open source software of Mplayer and MEncoder. Mplayer and MEncoder are cross platform, open source multimedia encoding and decoding programs. Figure 4-1 shows how different software blocks connect to each other in our proposed system:

Figure 4-1: Overall System Design

4.1.1 MPlayer MPlayer supports a wide range of media formats as shown by Table 4-1.

Table 4-1: MPlayer Supported Media

Physical media CDs, DVDs, Video CDs Container formats 3gp, AVI, ASF, FLV, Matroska, MOV (QuickTime), MP4, NUT, Ogg, OGM, RealMedia Video codecs Cinepak, DV, H.263, H.264/MPEG-4 AVC, HuffYUV, Indeo, MJPEG, MPEG-1, MPEG-2, MPEG-4 Part 2, RealVideo, Sorenson, Theora, WMV Audio codecs AAC, AC3, ALAC, AMR, FLAC, MP3, RealAudio, Shorten, Speex, Vorbis, WMA Subtitle formats AQTitle, ASS/SSA, CC, JACOsub, MicroDVD, MPsub, OGM, PJS, RT, Sami, SRT, SubViewer, VOBsub, Vplayer

In addition, the following figure depicts the block diagram of MPlayer’s functionality:

Streaming/Media Demux Decode Video/Audio Driver Display On Local Storage

Figure 4-2: General Block Diagram of MPlayer Functionality

Since the set-top box must contain the capability of displaying different media in various formats and codecs, MPlayer suits our requirements for this project with its large selections of supported media formats as seen in the above table. Also, since it is open source software, we can modify the source such that our proposed functions can be easily implemented without breaching intellectual properties.

4.1.2 MEncoder MEncoder is a transcoder that comes with the MPlayer bundle. It can convert a wide range of multimedia files to a specific file format. Furthermore, it is able to save a video and audio stream separately on to a memory location for post processing. The subsequent block diagram represents the internal operations of MEncoder:

Figure 4-3: General Block Diagram of MEncoder Functionality

4.2 Real Media Codec In the proof of concept device, the real media video codec will be developed such that the set-top box can stream media content in RM format. The codec will be developed based on the source code of RV30 from the Helix Community.

4.3 Media Stream

4.3.1 Transcoding

Internet streams are often available in RealMedia format, with files ending in *.rm extensions. Since the set top box proof-of-concept model is designed to output an Internet stream on a TV screen, it must be able to decode RealMedia streams. However, no codec has been developed that is compatible for the ARM processor to do so. To work around this problem, we must convert all RealMedia streams into a different format so that it can be decoded by the ARM processor. This conversion process is termed transcoding. Figure 4-4 our proposed concept Figure 4-4of transcoding of a live RealMedia stream into an MP2 audio stream and a MPEG4 video stream in our application.

Figure 4-4: Process of transcoding a media stream

The system begins by de-multiplexing an RTSP stream into its respective audio and video components. The audio and video streams are then decoded to its raw format and encoded to MP2 and MPEG4, respectively, which then can be decoded by the ARM processor. Now, the ARM processor treats the streams as MP2 and MPEG4 inputs and decodes it such that it can be output on the on-screen display via the device drivers as described in the next section.

4.3.2 Demuxer

In order to demultiplex the RTSP stream into its audio and video components, we will utilize the functions available within our media player’s code, which for our case is MPlayer.

• The demultiplexing of the input signal to audio and video is performed by demuxer.c in the MPlayer open source code. • All inputs have parsers depending on the type of file format they are encoded in (i.e. avi, real, mpeg, etc.). • The parsers are in demux_*.c files. • Stream header - There are two types: sh_audio_t and sh_video_t. They contain the parameters for decoding such as input/output buffers, the needed codec, fps, etc. • There are stream headers for video, audio respectively (and others that are not related to our problem). • Chosen stream header and its demuxer are connected together • The codecs call demuxer.c/demux_read_data(). It will receive the number of bytes, which memory address we wish to read, and from which DS. • Structure: demux_stream_t. Every channel contains a DS and contains the packets of the stream which can be either audio (d_audio), video (d_video) or DVD subtitle (d_dvdsub). Struct: demux_stream_t - Every channel (a/v/s) has one. This contains the packets for the stream. For now, there can be 3 for each demuxer : - audio (d_audio) - video (d_video) - DVD subtitle (d_dvdsub) • demux_read_data() function call checks if the DS’s buffer contains any information and reads from the buffer if it does. With insufficient data, it calls ds_fill_buffer() • ds_fill_buffer() checks if the connected DS’s buffer has DP, and reads from the oldest location.

The reading of the video frames and the corresponding assembly is performed by video.c in the MPlayer code. Every call to video_read_frame() will read a single video frame and the time length of that frame.

4.4 Conversion Block Figure 4-5 shows the flow chart of the conversion process, it consists of the following steps: 1. At the beginning, the application checks whether the RAW video and audio data from the demuxing layer is ready or not. 2. Once the RAW data is ready, the application checks whether the buffer is full or not. If it is full, the application will go into a wait state. 3. When the buffer is ready, the conversion starts and fills up the buffer. 4. Then, the application will determine the buffer level and set the buffer full or buffer low flag when it is necessary. 5. The video data in MPEG4 format is passed to the decode stage or writes to the hard drive according to the user’s request.

N Y

Encode RAW Fill buffer with RAW Video Buffer Full Flag Y Video Data to N encoded Video Ready? Set? MPEG4 format Data

Set Buffer Full Y Buffer Full? Flag

Reduce Frame Set Buffer Level Buffer Level < Y Rate Per Second Low Flag Requirement?

Go to Decode Block and delete data from buffer

Figure 4-5: Conversion Algorithm

5 Test Plan The goal of the test plan is to verify that every feature in our design has been implemented based on product requirements and to test whether the behaviour of these features is consistent under various conditions and stress scenarios. Each of the following test plans will be carried out independently of each other, with each section emphasizing on different components of our system; mainly hardware, software and firmware testing.

The following test plan is to be executed by a trained test engineer to ensure quality assurance and to prevent failures in our system.

5.1 Assumptions The test plans assume that the test bed, such as a Linux operating system, the media player client MPlayer, its associated packages and various codecs, are installed and operating properly. All source codes are to be compiled successfully in an IDE (ex. Eclipse), with output files (*.so and dll files) with appropriate permissions and be able to be read/write/execute by users of different levels. In addition, all communication components (ex. device drivers, ports numbers) of the development board are assumed to be properly configured.

5.2 System Hardware Test Plan

5.2.1 Component Connection Verification • Verify the connection of IC pins to sockets and development board • Verify all ports are securely connected to appropriate cables/bus • Verify that the development board boots up and is properly initialized, ready for user input • Verify that the integrated hard drive has adequate memory to read/write data

All of these checks must pass before proceeding to the latter stages of the hardware test plan.

5.2.2 DC Power Verification • Apply the correct DC voltages to power connections of the development board • Verify correct DC voltages at all power nodes for different components of the development board

5.2.3 Device Drivers Verification • Verify the audio, video and USB device drivers connected from the evaluation board to the TV set (LCD display) are loaded upon booting • Verify that the communication between these three ports are functional and interactive

5.2.4 EVEVM Software Verification

The Texas Instrument provides the software to be used with the DAVINCI DM 6446 development board. This software is used for functional verification of the ARM CPU.

5.3 System Software Test Plan

5.3.1 Media Transport Verification • Verify that TCP/IP connections to the Internet is established • Verify the connectivity between the evaluation board and a remote server online is established

5.3.2 WAN Online media Streaming • Verify RTP/RTSP sessions are connected and possible via the CLI commands • Verify that both audio and video media streams are retrieved from the remote server through the use of MPlayer • Debug and verify the log of MPlayer to check for potential errors • Verify that the appropriate codecs are being used for both audio and video streams in the decoding and encoding processes • Verify that the speed of network traffic is faster than 300Kbits/s • Verify that the user is able to capture and save the streaming media to the local hard drive

5.3.3 LAN Helix Server Simulated Online Media Streaming • Ensure that the Helix Server is installed and has been started via CLI • Verify contents to be played are located under the ~/HelixServer/Contents directory in local hard drive, and are of appropriate codec • Verify that the video and audio streams playback is acceptable with adequate frame rates and no frame dropping

5.3.4 Local Media Playback • Verify that the contents to be played are under a specified directory in the local hard drive, and are of the appropriate codec • Verify the user is able to view the stored media from the local hard drive • Verify that the video and audio stream playback is acceptable with adequate frame rates and no frame dropping

5.3.5 TV Set Output • Verify that the video and audio stream playback is acceptable with adequate frame rates and no frame dropping • Verify that the output display is in the NTSC standard

5.4 System Firmware Test Plan

5.4.1 On Screen Display

• Verify all the maneuverability (play, fast forward, reverse, pause, stop) of the GUI presented to the user • Verify that the appropriate error message appears when a failure occurs • Verify that the buffer is present

5.4.2 IR remote control • Verify the IR remote control signals can be received and can assist in the navigation of the GUI application 6 Conclusion The requirements outlined in this document will serve as the goal Ensight Solutions will achieve. At the end of the POC stage in April, each of the above requirements will be met. Ensight Solutions is hopeful that the work accomplished in making this application will help push further development of the IPTV set-top box for Varietize Technologies, and the rest of the IPTV market.

7 References 1. L. Allen, J. Jimmy, S. David and K. Hugo, Functional Specification for Next Generation IPTV Set-Top Box, February 2007 2. L. Allen, J. Jimmy, S. David and K. Hugo, Proposal for Next Generation IPTV Set-Top Box, January 2007 3. Texas Instruments, DVEVM Getting Started Guide, August 2006 4. Wikipedia: MPEG. (http://en.wikipedia.org/wiki/MPEG) 5. Wikipedia: IPTV. (http://en.wikipedia.org/wiki/Iptv) 6. Linuxdevices: DaVinci Architecture. (http://www.linuxdevices.com/files/misc/ti_davinci_arch_diag.gif) 7. H. Schulzrinne, A. Rao and R. Lanphier, Real Time Streaming Protocol (RTSP), April 1998 8. H. Schulzrinne, S. Casner, R. Frederick and V. Jacobson, RTP: A Transport Protocol for Real-Time Applications, July 2003