OMAP™: Enabling Multimedia Applications in Third Generation (3G)
Total Page:16
File Type:pdf, Size:1020Kb
Technical White Paper SWPA001 – December 2000 OMAP™: Enabling Multimedia Applications in Third Generation (3G) Wireless Terminals Jamil Chaoui, Ken Cyr, Jean-Pierre Giacalone, OMAP Architecture Team Sebastien de Gregorio, Yves Masse, Yeshwant Muthusamy, Tiemen Spits, Madhukar Budagavi, Jennifer Webb ABSTRACT This paper describes how the Open Multimedia Applications Platform™ software and hardware architecture enables multimedia applications in third-generation (3G) wireless appliances. It provides an overview of OMAP™ strategy, concepts, main milestones, and achievements. It also describes OMAP hardware architecture, explains how multimedia applications can benefit from this advanced architecture, and why a RISC/DSP approach is superior to RISC-only architecture. This paper outlines OMAP software concepts to show how an architecture that combines two heterogeneous processors (RISC and DSP), several operating system (OS) combinations, and applications running on both DSP and RISC can be made accessible seamlessly to third parties. The final parts of the paper outline how key multimedia applications such as speech and video can be integrated in an OMAP solution. Contents 1 Introduction .........................................................................................................................................2 2 OMAP Hardware Architecture ...........................................................................................................3 2.1 Advantages of a Combined RISC/DSP Architecture.....................................................................3 2.2 How the Architecture Works ..........................................................................................................4 2.3 C55x DSP and Multimedia Extensions .........................................................................................5 3 OMAP Software Architecture ............................................................................................................7 4 OMAP Multimedia Applications.........................................................................................................9 4.1 Video...............................................................................................................................................9 4.2 Speech Applications .....................................................................................................................10 5 Conclusion.........................................................................................................................................11 Figures Figure 1. OMAP Hardware Architecture ................................................................................................ 4 Figure 2. How the OMAP Architecture Works ...................................................................................... 8 Figure 3. Hierarchical Organization of Speech APIs.......................................................................... 10 Tables Table 1. Comparative Algorithm Execution......................................................................................... 3 Table 2. Description of the « copr » Opcodes .................................................................................... 5 Table 3. Data Flow Modes ..................................................................................................................... 6 Table 4. Characteristics of Video Hardware Accelerators ................................................................ 6 Table 5. MPEG4 Video Codec Power ................................................................................................... 7 1 SWPA001 1 Introduction Every wireless handset contains two fundamental components: a modem and an applications processing and delivery system. The modem communicates with a network to send and receive data via an air interface. The applications component provides the functions that the user wants in a multimedia appliance. These may include speech, audio playback, image reproduction and streaming video, fax transmission and reception, e-mail, Internet connections, games, personal organizer functions, and a host of other possibilities. The applications component also handles user interface functions such as speech recognition (voice commands), keyboard, and written character recognition. Except for user interface functions, all of these applications depend on the air interface, which provides only limited bandwidth. Managing the volume of data required for applications involving speech, audio, image, and video within the available bandwidth requires heavy signal compression before transmission and decoding or expansion within the receiver, which, for purposes of this paper, is a handheld wireless appliance. Wireless appliances need additional signal processing capabilities for the concurrent noise suppression and echo cancellation algorithms essential for even the most basic functionality, i.e., voice telephony. In many of today’s second generation (2G) wireless handsets, the modem component employs a DSP, which provides the air interface and takes care of such essentials as noise suppression and echo cancellation. The applications component, however, relies on a general purpose processor (GPP), usually a RISC processor, which attempts to provide the signal processing needed at the application level, user interfaces, and overall command and control functions. In 2G appliances, which are, essentially, wireless telephones with extremely limited data features, this processing dichotomy has generally proved to be satisfactory. However, for 2.5G and 3G appliances, which offer such capabilities as full-motion video and high-fidelity audio playback, the currently popular division of tasks between the two processors does not provide reliably robust performance and still maintain the useful battery life that is acceptable to consumers. Next generation appliances, which include numerous applications enabled by high data rates and real-time signal processing, continue to require both a DSP and a GPP. However, the functional wall between the two basic appliance components breaks down as the demand for signal processing within the applications component increases. The DSP, which is optimized for intensive signal processing in real time with extremely low power consumption, becomes the primary processor for both the modem and the applications component. The GPP plays a secondary role, taking care of system management, command, control, and certain user interface activities. The OMAP architecture provides a means to coordinate dual processors across the two basic components of the wireless appliance and to seamlessly take advantage of the unique capabilities of each. Nokia, Ericsson, Sony, Handspring, and others already have selected the OMAP hardware and software architecture as the development platform for their wireless appliances. The first samples of an OMAP-based chipset will be available in 4Q00. In addition to enabling the numerous, media-rich applications made possible by 2.5G and 3G data rates, the OMAP architecture also provides a new capability—the capability to dynamically download applications and application upgrades from the web in the same way that PC users download applications from the Internet today. This dynamic environment requires the programmability offered by DSPs and the application programming interface (API) features built into the OMAP architecture. 2 OMAP: Enabling Multimedia Applications in Third Generation (3G) Wireless Terminals SWPA001 2 OMAP Hardware Architecture 2.1 Advantages of a Combined RISC/DSP Architecture The OMAP architecture is based on a combination of TI’s state-of-the-art TMS320C55x™ DSP core and high performance ARM925T CPU. A RISC architecture, like ARM925T, is well suited for control type code (Operating System (OS), User Interface, OS applications). A DSP is best suited for signal processing applications, such as MPEG4 video, speech recognition, and audio playback. The OMAP architecture combines two processors to gain maximum benefits from each. TI conducted a comparative benchmarking study, based on published data, which shows that a typical signal processing task executed on the latest RISC machine (StrongARM™, ARM9E™) requires three times as many cycles as the same task requires on a C55x™ DSP. See Table 1. In terms of power consumption, tests show that a given signal-processing task executed on such a RISC engine consumes more than twice the power required to execute the same task on a C55x DSP architecture. Battery life, critical for mobile applications, therefore, is much greater when such tasks are executed on a DSP. The OMAP architecture’s use of two processors provides this kind of power consumption benefits. At the same time, it allows the DSP to gain support from the RISC processor. For instance, a single C55x DSP can process, in real-time, a full videoconferencing application (audio and video at 15 images/sec.), using only 40 percent of the available computational capability. Therefore, 60 percent of the capacity can be employed to run other applications concurrently. At the same time, in the OMAP dual-core architecture, the ARM processor stands ready to handle any other application requirements or can be suspended, thus saving battery life. As a result, the mobile user can enjoy access to popular OS applications (Word™, Excel™, etc.) while also engaging a videoconferencing application. With a RISC processor alone, all of the available computing capacity would be needed