This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

 Survey and performance evaluation of real-time operating systems (RTOS) for small

Tran Nguyen Bao Anh*†, Su-Lim Tan†

*Renesas Technology Singapore, Singapore Engineering Centre, Singapore 098632 †School of Computer Engineering, Nanyang Technological University, Singapore 639708

Abstract— RTOS has gained popularity over the years in any OS as it affects how the various softwares are /processor-based embedded system design. In this executed. Most generic OSes are time-sharing systems paper, we will discuss the important differences between RTOS and in which tasks are allocated the same amount of time generic OS, the advantages and disadvantages of using RTOS for slices (e.g. round robin scheduling) for execution. In small microcontroller system development, and the benchmarking RTOS, tasks are often assigned priorities and higher- methods used for RTOS. Several RTOSes are studied and compared priority tasks can preempt lower-priority tasks during based upon numerous selection criteria, and four RTOSes are selected for performance benchmarking on the same microcontroller execution (preemptive scheduling). There are also platform. For the purpose of performance benchmarking, a list of RTOSes that adopt cooperative scheduling. Such benchmarking criteria which is aimed to be simple and representative scheduling technique usually implies that the running of typical RTOS usages are examined. The benchmarking results task has to explicitly invoke the scheduler to perform a show that there is no clear winner and each RTOS performed well on task switch. certain criteria compared to others. x Predictability in task synchronization: For generic Index Terms- kernel; ; real-time system; OS, task synchronization is unpredictable because the RTOS; RTOS benchmarking OS can directly or indirectly introduce delays into the application software. In RTOS, synchronization among I. INTRODUCTION tasks (such as using semaphore, mailbox, message queue, event flag, etc) must be time-predictable. The EAL – time embedded systems are typically designed R for various purposes such as to control or to process data. system services must have known and expected Characteristics of real-time system include meeting certain duration of execution time. deadlines at the right time. To achieve this purpose, real-time x Deterministic behaviors: This can be considered as operating systems (RTOS) are often used. An RTOS is a piece the key difference between generic OS and RTOS. In of software with a set of for users to develop RTOS, task dispatch time, task switch latency, interrupt applications. Using an RTOS does not guarantee that the latency must be time-predictable and be consistent even system will always meet the deadlines, as it also depend on when the number of tasks increases. On the other how the overall system is designed and structured. hand, generic OS (mainly due to its time-sharing While RTOS for embedded systems were predominantly approach) reduces the overall system responsiveness employed in high-end microprocessors or microcontrollers and does not guarantee service call execution within with 32-bit central processing unit (CPU), there is a growing certain amount of time when the number of tasks trend to provide these features in the mid-range (16-bit and 8- increases. Dynamic memory allocation (malloc() in C bit) processor systems. In Section B, the use of RTOS for this language), though being widely supported in generic range of devices will be discussed in details. OS, is not recommended in RTOS because the An operating system (OS) is a piece of software that behavior is unpredictable [1]. Instead, fixed-sized manages the sharing of resources in a computer system. RTOS memory allocation is provided in which only fixed-size is often differentiated from generic OS as it is specifically block of memory is allocated for every request. designed for scheduling to achieve real-time responses. B. RTOS for small scale-embedded systems A. Generic OS versus RTOS There are a number of variants of RTOSes available RTOSes are typically differentiated from generic OSes in nowadays; they ranged from commercial, proprietary, to open- the following: source RTOSes. For small-scaled embedded systems designed x Preemptive, priority-based scheduling: Scheduling using small microcontrollers (i.e. microcontrollers with scheme refers to how the RTOS assigns CPU cycles to maximum ROM of 128Kbytes and maximum RAM of tasks for execution. Scheduling scheme is important in 4Kbytes [2]), there is a common perception that no RTOS is

1 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication. needed. However, there are significant advantages to use an platform employing a different microcontroller with a RTOS for this range of devices [2, 3], such as: different set of peripherals, these timing features need x Optimizing software development: Even in system to be rewritten again. Unless for special and critical development using small microcontrollers, improving timing issue with unique hardware peripheral, using an software productivity is a critical issue due to time-to- RTOS can helps to speed up the development time market pressure as well as shorten development cycle significantly to tackle these timing issues. [4]. Using an RTOS is one of the approaches that has In [3], an example is given to illustrate the importance of gained increasing popularity. As the code complexity RTOS in small system design – a printer system. Without grows, an RTOS is an efficient tool to manage the an RTOS, there is one single chunk of code to manage all software, and to distribute the tasks among developers. the activities, from paper feeding, user-input reading, to Using an RTOS will allow the entire software to be printing control. By having an RTOS, individual task will partitioned into modular tasks that can be taken care of manage each of these activities. Except for passing of status by individual programmer. Moreover, low-level driver information, each task does not need to know much about development can be done by other developers. what other tasks are performing. Hence, having an RTOS in x Better and safer synchronization: In small embedded place can help in partitioning the software in time domain system development without using any RTOS, global (tasks are running concurrently) and in terms of variables are often used for synchronization and functionalities (each task performs a specific operation). communication among modules/functions. However, especially in highly interrupt-driven system, using global variables lead to bugs and software safety issues [5]. These global variables are often shared and accessed among the functions; hence there are high chances of them being corrupted at any time during the program execution. As the code begins to grow, these bugs become hidden deeper and more difficult to be uncovered. Consequently, development time can be lengthened even for such small-scale system development. With an RTOS in place, synchronization Figure 1: RTOS usage, embedded market survey is safely managed and tasks can pass messages or (CMP, EE Times) 2004 [4] synchronize with each other without any corruption problems. RTOS usage is gaining popularity in the past few years as x Resource management: Most RTOSes provide APIs clearly indicated in the CMP, EE Times embedded system for developers to manage the system resources [5]. survey for 2004 - 2006 [4, 5, 6]. Figure 1 shows the embedded These include task management, memory pool market survey conducted by CMP, EE Times in 2004 on the management, time management, interrupt management, number of developers that have used and would consider communication and synchronization methods. These using RTOS in the current and next projects. The number of features provide the abstraction layer for developers to developers who “have used” RTOS takes up more than 49%. freely structure the software, to achieve cleaner code This percentage rises to 80.9% in the 2005 survey, and 71% in and to even quickly port across different hardware the 2006 survey. The number of developers who “would platforms with little code modifications. Especially consider” to use RTOS in the next project in 2004 is 66.6%, with small system development - cost of hardware is of and in 2005 is 86%, which shows a steady trend towards critical constraint, and development time is usually employing RTOS. Definitely, more and more developers and short. designers are adopting RTOS in long run. x Timing can be easily managed by RTOS: With time Table 1 indicates there is another trend in RTOS selection - management functions, software designers can achieve companies are moving towards open-source RTOS - from task delay, timer handling or time-triggered processing 16% in current projects to 19% in the next projects in 2006) or without resorting to understanding the underlying commercial distribution of an open-source RTOS - from 12% hardware mechanisms. As compared with a small in current projects to 17% in the next projects in 2006. system that does not use any RTOS, achieving timing Commercial OS and in-house OS, though currently being related features can be tricky as the software designer extensively used, are declining - from 51% in current projects needs to understand the underlying peripherals (such as to 47% in next projects in 2006 for the case of commercial timers), how to use it, and how to link it with the top- OS, and from 21% to 17% for the case of in-house OS. In the level application code. Any modification, such as to latest 2007 survey [7], the percentage for commercial OS lengthen the delay time would requires the developer to drops further to 41%. Also according to the 2007 survey, the examine the code and peripheral again to make changes key influencing factors in RTOS selection for commercial OS appropriately. When porting the software to another are the quality and the availability of technical support. Hence

2 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication. companies are willing to adopt commercial OS only when the use the same concept for message passing. Some RTOSes technical support provided is adequate. Otherwise, companies pass messages by passing only the memory pointer, and hence may look for other more cost-effective choices. there is no need to use the special memory area managed by the OS. This approach is also more suitable for small Current Next Types of operating systems used project project microcontrollers because there is no extra memory for OS internal use. Interrupt latency time as defined by Rhealstone is Commercial OS 51 % 47 % purely dependent on the CPU architecture and is not Internally developed or in-house OS 21 % 17 % Open-source OS without commercial support 16 % 19 % determined by the RTOS. Rhealstone, in general, are Commercial distribution of an open-source OS 12 % 17 % “somewhat adhoc”, and do not cover other situations commonly found in real-time applications [13]. Table 1: Types of operating system used, embedded market survey In [13], some metrics are proposed (based on frequently (CMP, EE Times) 2006 [5] used system services): response to external event (interrupt), Having detailed about the advantages and trends of using inter-task synchronization and resource sharing, and inter-task RTOS in general, there are certain disadvantages and concerns data transfer (message passing). Inter-task data transfer, as associated with using RTOS for small microcontrollers. explained previously, is also based on data copying into a Firstly, an RTOS takes up additional memory (ROM and memory area managed by the OS, similar to the “datagram RAM), computational resources and consume extra power [8], throughput time” in Rhealstone benchmark. In the test for hence can the system absorb these overheads? For small “response to external event (interrupt)”, the interrupt handler microcontrollers, it is important that the RTOS must be wakes up another task via a semaphore. Using a semaphore in compact in ROM and RAM requirements. There are various this case does not seem to be the best approach. Waking up RTOSes available for this segment of devices, and some are the task directly by using system service call (such as flexible enough to be configured to have only those functions sleep/wakeup service call) instead of going through a and APIs required by the application [2, 9, 10, 11] so that the semaphore is a better approach to reduce the overhead delay. code size can be reduced. In the latter sections, a more In [15], the metrics proposed are (based on frequently used detailed analysis will be done on the possible ROM and RAM system services): tests for measuring the duration of message requirements of RTOSes. Besides memory footprint, an RTOS transfer and the communication through a pipe, tests for also takes up additional CPU resource. Most RTOSes require measuring the speed of task synchronization through proxy a periodic timer (OS ‘tick’) [12] to execute the scheduler and and signal, and tests for measuring the duration of task other relevant system services. RTOS services such as task switching. These metrics are based only on the QNX synchronizations must have known execution time (e.g. how distributed RTOS platform, some concepts such as proxy and much time does it takes for a task switch to occur). Depending signal do not exist on most RTOSes (as illustrated in the on these timing factors and by making use of the relevant RTOSes survey later). RTOS services, the system designer can decide and structure In the next section, a list of RTOSes, including open- the whole system. Hence it is essential to understand the source, commercial and research, will be discussed based on performance measurements and the benchmarking metrics their features and APIs. Those found to be unsuitable for among the RTOSes. small microcontrollers will be eliminated. Finally, among those selected, four of the more popular RTOSes will be C. RTOS benchmarking ported to a MCU (microcontroller) platform for benchmarking There are different approaches towards RTOS (in terms of code size, RAM usage, and execution speed) and benchmarking: based on applications or based on the most evaluated against a list of proposed benchmarking metrics. frequently used system services (fine-grained benchmarking) [13]. As there are various types of applications with each II. RTOS FEATURES AND API COMPARISON having very different requirements, benchmarking against any generic applications will not be reflective of the RTOS A. Criteria for comparison strengths and weaknesses. The objective of this section is to investigate RTOSes There are various research publications related to available (open-source, commercial, and research) and benchmarking method based on frequently used system determine those that are suitable for small microcontrollers services. In [14], the Rhealstone benchmark is proposed with only. Information is mainly based on documentations and the following measurement: task switch time, preemption APIs available on websites. These RTOSes are: μITRON, time, interrupt latency time, semaphore shuffling time, μTKernel, μC-OS/II, EmbOS, FreeRTOS, Salvo, TinyOS, deadlock breaking time, and datagram throughput time. SharcOS, XMK OS, Echidna, eCOS, Erika, Hartik, KeilOS Rhealstone benchmark is not suitable for several reasons. and PortOS. Firstly, very few RTOSes are capable of breaking deadlock As described in [16], criteria used for selecting an RTOS (which we will see later in the RTOSes survey). Datagram includes the following: language support, tool compatibility, throughput time is based on message passing by copying to a system service APIs, memory footprint (ROM and RAM memory area managed by the OS. However, not all RTOSes usage), performance, device drivers, OS-awareness debugging

3 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication. tools, technical support, source/object code distribution, whether it was by a person, an organization, or a licensing scheme and company reputation. Similarly, criteria company. mentioned in [17] are: installation/configuration, RTOS x Scheduling scheme: RTOS scheduling approach will architecture, API richness, documentation and support, and be investigated to determine whether preemptive tools support. Embedded market surveys conducted by CMP, scheduling, cooperative scheduling or other scheduling EE Times in 2005 to 2007 [5, 6, 7] also concluded that the scheme is used. priority of criteria for RTOS selection (see Figure 2), in which x Real-time capability and performance: Real-time real-time capability has taken the highest weighting. capability is generally considered as a system characteristic to describe whether the system is able to meet the timing deadline. Using an RTOS in the system takes up CPU cycles; however the RTOS must not have indeterminist behaviors. The amount of CPU cycles and time consumed by the RTOS for any service call should be measurable and of low or acceptable values to the system designers. Real-time capability and performance information are not available for some RTOSes. Even if these information are available, they might be based on different hardware platforms. In Section III, a selected list of RTOSes will be benchmarked against one another on the same hardware platform so that more comparative results can be obtained. x Memory footprint: Besides CPU cycles, an RTOS also occupies additional ROM and RAM spaces. This could lead to larger ROM and RAM sizes for the entire system. There is always a tradeoff between memory footprint and the functionalities required from the RTOS. To have more robust and reliable APIs, Figure 2: Influential factors in operating system selection, embedded probably more lines of code are needed. On the other market survey, CMP, EE Times 2005, 2006, and 2007 (N=441 is the total number of people surveyed) hand, basic and simple APIs will require only minimum amount of code. Hence, it is important for Based on the above, a list of criteria to compare among the the designers to understand the features offered by the RTOSes is established. In the scope of this paper, it is not RTOS with the corresponding memory footprint feasible to take all the above criteria into considerations. requirement. This criterion will also be compared Criteria, such as “suppliers’ reputation” and “company among the selected RTOSes in Section III. reputation” are subjective to each and individual company’s x Language support: supported judgments. “Overall cost” is project and application by the RTOS. dependent, and “royalty fee” is normally based on quantity, x System call/API richness: This criterion determines even though RTOS vendors may use other business models to how comprehensive the RTOS APIs are as compared charge their customers (such as per application, per product to the rest of the RTOSes. The total number of system model, or per MCU model, etc). “Memory footprint” (ROM calls for each RTOS will be counted. and RAM usage) may not always be available and it is highly x OS-awareness debugging support: This criterion dependent upon the compiler settings as well as RTOS determines if the RTOS is being supported by any of configurations. For this paper, the following criteria will be the Integrated Development Environment (IDE). OS- used for comparison: awareness debugging [3] will ease the development x Design objective: The origins of the RTOSes being work as users can use these RTOS internal information surveyed are different from one another, as some are (e.g. task states, system states, semaphores, event flags) open-source, some are personal hobby-based, and some provided by the IDE. are commercial. It is important to understand the x License type: This is to investigate how the RTOS is history and the background motivation that led to the distributed: free or fee-based for different purposes creation of each RTOS. A personal hobby-based RTOS such as educational or commercial. would be less likely to be as stable compared to a x Documentation: This criterion will focus on what type popular open-source, or to a commercial RTOS. of documentations are available for the RTOS (detail x Author: Similar to design objective, it is essential to APIs, simple tutorial, book or specification). understand the author who originated the RTOS –

4 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

System Design Language OS-awareness Author Scheduling License type Documentation call/API References objective supported support in IDE richness Ken Priority- Open Sakamura & Supported by μITRON Commercial based Fee-based specification and 93 C [2, 18] Tron Renesas IDE preemptive user manual association Free for Commercial/ Priority- Open T-Engine educational Supported by μTKernel Educational/ based specification and 81 C [10] forum and Renesas IDE Research preemptive user manual commercial Commercial/ Priority- Jean Free for μC-OS/II Educational/ based Book by author 42 C Supported by IAR [9] Labrosse educational Research preemptive Priority- Online EmbOS Commercial Segger based Fee-based 56 C Supported by IAR [19] document preemptive Free for Priority- Free- Richard educational Online Hobby based 27 C No support [11] RTOS Barry and document preemptive commercial Pumpkin Online Salvo Commercial Cooperative Fee-based 31 C No support [20, 21] Inc. document Free for Educational/ educational TinyOS UC Berkeley Cooperative Tutorials - nesC No support [22, 23] Research and commercial JDC Priority- SharcOS Commercial Electronics based Fee-based User manual - C No support [24] SA preemptive Free for Priority- Online Educational/ Shift-right educational XMK OS based document - C No support [25] Research Technologies and preemptive (incomplete) commercial Free for Priority- Online Educational/ Maryland educational Echidna based document 18 C No support [26] Research University and preemptive (incomplete) commercial Free for Commercial/ Priority- educational Book and online eCOS Educational/ eCosCentric based - C No support [27] and document Research preemptive commercial Free for Priority- Educational/ Universita di educational Online Erika based 19 C No support [28] Research Siena and document preemptive commercial Free for Priority- Educational/ RETIS Lab educational Online Hartik based 33 C No support [29] Research (Italy) and document preemptive commercial Priority- Online Supported by Keil KeilOS Commercial Keil based Fee-based - C [30] document IDE preemptive Priority- Software Online PortOS Research based Fee-based - C No support [31] Wireless document preemptive Table 2: Basic features comparison of RTOSes for small microcontrollers

x In the case of eCOS [27], a bootloader (known as B. Comparison results Redboot) of at least 64K ROM is required. Table 2 shows the features comparison among the Redboot will boot up first and load programs RTOSes. From the table, it can be seen that: into the RAM via a user terminal (typically over x Priority-based preemptive scheduling has been a serial port). Hence, eCOS requires much more adopted by majority of the RTOSes, except for ROM and RAM spaces. two RTOSes in the list (Salvo and TinyOS) x As far as documentations are concerned, some using cooperative scheduling. RTOSes (KeilOS, PortOS and XMK) do not x Majority of RTOSes support C language, which have details of the APIs available. SharcOS is is the popular choice for embedded system based on μC-OS/II; hence it follows the same programming, especially in small system design APIs of μC-OS/II. For the above reasons, these [32]. RTOSes will not be considered in the following x Only a few RTOSes have OS-awareness support APIs comparison. in IDE: μC-OS/II and EmbOS have plug-in x In the “system call & API richness” column, modules for IAR compiler; KeilOS is supported those RTOSes that do not have details available by Keil compiler; μITRON and μTKernel are will be represented by a “-”. supported by Renesas HEW compiler.

5 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

NNumberumber ofof system API APIs 14 12 10 8 6 4 2 0 μITRONuItron uTkernelμTKernel uC-OS/IIμC-OS/II FreeRTOS FreeRTOS SalvoSalvo EchidnaEchidna EmbOS emBos Erika Erika Hartik Hartik

System management Interrupt management Task management Task-dependent synchronization Semaphore Eventflag Mailbox Data queue Mutex Message buffer Fixed size memory pool Variable size memory pool System time management Trace API

Figure 3: Number of system APIs for various RTOSes

Figure 3 shows a comparison of the number of system except for trace, system time management, system APIs available for each RTOS. Based on the RTOSes management and message buffer. For each category, the common definitions, the APIs in Figure 3 are number of APIs for these four RTOSes also exceed categorized into: other RTOSes because they have been developed and x System management: initialize OS, improved, and have been in the market for a relatively start/shutdown OS, lock/unlock CPU, etc. long period. x Interrupt management: entry/exit function, As far as open-source RTOSes (such as FreeRTOS, begin/end critical section, etc. Echidna, Erika and Hartik) are concerned, most have x Task management: create task, delete task, start minimal implementation. These RTOSes are more task, terminate task, etc. suitable for small system development. However, μC- x Task-dependent synchronization: sleep task, OS/II stands out to have more APIs available. μC-OS/II wakeup task, resume task, etc. originally was an open-source RTOS for personal and x Communication and synchronization: educational purposes. Due to its stability and popularity, semaphore, data queue, event flag, mailbox, it has been commercialized and is being widely used [8]. mutex, message buffer. Among those open-source RTOSes surveyed, μC-OS/II x Memory management: fixed size memory pool, and μTKernel have the most number of APIs available. variable-size memory pool. μTKernel supports all categories, except for data queue. x Time management: get system operating time, It has almost the same number of APIs as the OS timer, etc. commercial RTOS μITRON, as it is backed by T- Engine Forum [33] led by Professor Ken Sakamura who x Trace API: hook routine into certain RTOS is also the designer of μITRON architecture. For the functions such as scheduler. above reasons, these four RTOSes: μITRON, μTKernel, μITRON, μTKernel, μC-OS/II and EmbOS support μC-OS/II and EmbOS will be focused for subsequent comprehensive APIs for all the categories listed above. comparison and benchmarking. Most commercial RTOSes are well-implemented, with Besides those functions mentioned in Figure 3, the μITRON supporting all the categories except for trace following are also supported by some RTOSes: functions. EmbOS also supports most of the categories, x Timeout: timeout is supported in some system

6 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

APIs. For example, a task can wait for a developers can specify whether tasks are queued in semaphore for a maximum number of n FIFO order or in priority order. This flexibility not only milliseconds. RTOSes such as μITRON and applies to semaphore, but is also extended to other APIs μTKernel have mechanism to allow users to (including mailbox, message queue, memory pool and specify the timeout in absolute values (typically event flag). To achieve such features in μITRON and in milliseconds). Other RTOSes such as μC- μTKernel, there are tradeoffs in memory footprint as OS/II, FreeRTOS, Salvo and EmbOS can only well as performance. allow users to specify timeout values in terms of the number of clock ticks. III. PERFORMANCE AND MEMORY FOOTPRINT x Debug API: these are the APIs that allow user’s BENCHMARKING application to retrieve information managed by the kernel. Currently, only μTKernel supports A. Benchmarking methods these APIs (e.g. get task register, set task In this section, these RTOSes will be benchmarked: register). μITRON, μTKernel, μC-OS/II and EmbOS. As x Cyclic handler, alarm handler: cyclic handler discussed in the previous section, the main reasons is a mechanism to indicate to the RTOS to these RTOSes were selected are: execute a function at a periodic interval while x Comprehensive and mature APIs. alarm handler allows RTOS to execute a function x Memory footprint and design concepts are after a certain amount of time. These APIs are suitable for small microcontrollers. currently available in μITRON and μTKernel. x These four RTOSes are made available on the x Rendezvous: μITRON and μTKernel also same platform for the purpose of benchmarking: support rendezvous mechanism (similar to Ada the Renesas M16C/62P starter kit with the HEW 34 language [ ] for real-time) for synchronization IDE together with the NC30 toolchain [35] are and communication between tasks. used. Details of the RTOSes ports on this platform will be described in the next section. For μITRON and μTKernel, several APIs provide For execution time measurements, oscilloscope and better controllability and flexibility. logic analyzer have been used in combination with IO Parameters μC-OS/II EmbOS μTKernel μITRON port toggling to achieve the best accuracy (in terms of Name/ Semaphore Extended information information name micro-seconds). - Tasks can be - Tasks can be queued in queued in order of order of FIFO (1) Ports of the RTOSes on the same M16C/62P FIFO or in order of or in order of priority priority platform Attributes - The first task in This section describes the platform, the IDE and queue has precedence or task with fewer toolchain (compiler, assembler, and linker) used. Also, request has the differences of these RTOSes in implementation and precedence  Initial Initial Initial distribution form will be discussed. Hopefully, this will Initial semaphore InitialCount semaphore semaphore semaphore helps the readers to understand and appreciate the count count count count Maximum comparison results better in the later sections. The Maximum value of following are information on the M16C/62P Count semaphore count microcontroller platform: Table 3: Users controllable parameters for semaphore creation x Microcontroller: Renesas M16C/62P 16-bit. in RTOSes x Operating frequency: 24MHz. Based on Table 3, tasks in μC-OS/II and EmbOS are x ROM: 512Kbytes. queued in a FIFO (first in first out) buffer when waiting x RAM: 31Kbytes (no cache and MMU (memory for a semaphore, and developers are not allowed to management unit)). change the order. However, in μITRON and μTKernel, x Interrupt mask level: 7 levels.

7 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

x IDE: Renesas HEW version 4.03.00.001. of EmbOS because the OS is distributed in the x Toolchain: NC30 toolchain version 5.43.00. form of .lib files. Furthermore, the toolchain and The Renesas M16C/62P has a 16-bit CISC (complex compiler settings that were used when generating instruction set computer) architecture CPU with a total these .lib files might not be totally the same as of 91 instructions available. Most instructions take 2 to μC-OS/II or μTKernel. 3 clock cycles to complete. The MCU is designed with x The amount of stack per task is set to be the a 4-stage instruction queue buffer [41] which is similar same for all the RTOSes. to a simplified pipeline often used in larger 32-bit x If a particular RTOS feature is not used (e.g. processor. semaphore, message queue), that feature is To ensure all the RTOSes operate on the same disabled by C preprocessor for μC-OS/II and platform with the same timer resolution, the following μTKernel, or disable during linking for EmbOS. settings have been used: However, for μITRON, unused features are still x The ‘OS tick’ resolution for all the RTOSes included as the RTOS is provided in library above is taken from timer A0 [36] of the format and system calls are invoked via software microcontroller. Timer A0 is configured as a interrupts. periodic timer at 10ms. Table 4 shows the differences in implementation of x Default settings of NC30 compiler have been system service call and critical section of the RTOSes. used for compiling the workspaces. μC-OS/II μTKernel EmbOS μITRON Service Using software Using software Direct call Direct call x μC-OS/II: the whole original workspace and full call interrupt interrupt Not disable any source code of the OS have been taken from Raise interrupt Raise interrupt Critical Disable all interrupt (based mask level to mask level to Micrium website [37]. Timer A0 was configured section interrupts on internal level 4 level 4 for the OS, and the whole workspace was variable) compiled again with NC30 toolchain version Table 4: System service call implementation and critical 5.43.00. section implementation x μTKernel: the whole original workspace and full In μC-OS/II and EmbOS, whenever a system service source code of the OS have been taken from call is issued, the function is called directly from the superh-tkernel.org website [38], Timer A0 was user task. The advantage is that the function is executed configured at 10ms. immediately with minimal overhead time, while the x EmbOS: the whole original workspace and disadvantage is that the service call will use the current library files (.lib) of the OS have been taken task’s stack for execution. In μITRON and μTKernel, from Segger website [39]. Timer A0 was whenever a system service call is issued, a non- configured for the OS and the whole workspace maskable software interrupt is raised (INT instruction in was compiled with NC30 toolchain version M16C/62P [41]), hence the current execution context is 5.43.00 – except for the .lib files. The .lib files switch to the kernel space (i.e. separate stack) to might have been compiled in older toolchain or execute the service call. The advantage of using this with different optimization settings which is not approach is that the service call will not use the current known to the author of this paper. This means task’s stack for execution, while the disadvantage is that that for the case of EmbOS, the toolchain and there will be an overhead time incur for every system compiler settings might not be exactly the same service call. as μC-OS/II or μTKernel. Table 4 also shows the different methods for critical x μITRON: the whole workspace, library files section implementation. μC-OS/II starts the critical (.lib) of the OS and timer A0 configuration have section by disabling all the interrupts, so no external been generated from the Renesas configuration interrupt can be accepted. The advantage is that the tool for μITRON [40]. The entire workspace was critical section part can execute safely from start to end compiled with NC30 toolchain version 5.43.00 – without any intervention, but this also implies that except for the .lib files. This is similar to the case highly important real-time interrupt will not be accepted

8 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

during this period. In μITRON and μTKernel, critical different terms to describe sleep/inactive and section is implemented by raising the interrupt mask ready/active states (such as μC-OS/II and EmbOS use level (for M16C/62P the interrupt mask level is set to 4, the term suspend, resume while μITRON and μTKernel but can be changed) so that highly critical interrupt can use the term sleep/wakeup). Table 5 shows the system still be accepted, as long as the interrupt handler does calls used in each RTOS. not interfere with the RTOS internal variables. EmbOS (b) Get/Release semaphore time does not disable nor raise the interrupt mask level for Semaphore is commonly used for synchronization critical section. It still allows all interrupts to come in primitive in RTOS [42]. For semaphore benchmarking, but uses some internal variables to control the critical the time taken by get and release semaphore service call section. This has the advantage that any interrupt can be will be measured, and the time required to pass the accepted and handled during the critical section; semaphore from one task to another task will also be however the disadvantage is that it requires additional measured. Figure 5 illustrates the method used to code to handle the internal variables of the critical measure the get and release semaphore time. section. Task1: (2) Benchmarking criteria (A) The proposed benchmarking criteria in this section Get semaphore (B) are aimed to be simple and easy to port to different Release semaphore platforms. For each criterion, execution time (C) measurement together with memory footprint (ROM and RAM) will be collected. Time to get semaphore: (A) to (B) Time to release semaphore: (B) to (C) (a) Task switch time Figure 5: Get/Release semaphore time measurement Task switch time is the time taken by the RTOS to There is only one task (Task1) and one binary transfer the current execution context from one task to semaphore (initialized to 1). Task1 will get the another task. The measurement method is explained in semaphore and then it will release it. Different RTOSes Figure 4. use different terms to describe the get/release of Task1: (higher priority) Task2: semaphore. Table 6 shows the APIs used by each Go to sleep (A) RTOS. (B) Wakeup task1 μC-OS/II μTKernel EmbOS μITRON Get semaphore OSSemPend() tk_wai_sem() OS_WaitCSema() wai_sem() API Release semaphore Task switch time: time from (A) to (B) OSSemPost() tk_sig_sem() OS_SignalCSema() signal_sem() API Figure 4: Task switch time measurement Table 6: APIs used for semaphore benchmark

μC-OS/II μTKernel EmbOS μITRON (c)Semaphore passing time Pass OSTaskSuspend() tk_slp_tsk() OS_Suspend() slp_tsk() message API To measure the performance of semaphore passing, Retrieve OSTaskResume() tk_wup_tsk() OS_Resume() wup_tsk() message API the following measurement method as shown in Figure Table 5: APIs used for task switch time benchmark 6 is used. Task1: (higher priority) Task2: There are two tasks: Task1 and Task2 with Task1 Get semaphore (put into wait list) (A) having higher priority. At the beginning, Task1 is first (B) Release semaphore executed, and it will go into sleep/inactive state. The execution context is then switched over to Task2. Task2 Time to pass semaphore from Task1 to Task2: (A) to (B) will wake up/make active Task1, and right after waking up, the execution context is switched over to Task1 Figure 6: Semaphore passing time measurement because it has higher priority. Different RTOSes use There are two tasks: Task1 and Task2 with Task1

9 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

having higher priority, and a binary semaphore sleep/inactive state, waiting for a new message. The (initialized to 0). At the beginning, Task1 is first current execution context will be switched to Task2, executed and it tries to get the semaphore. Since the which will then put a new message onto the queue. The semaphore value is 0, Task1 will be put into a new message will wake up Task1, and the execution sleep/inactive state, waiting for the semaphore to be context will be switched over to Task1. The difference released. The current execution context will then be between this measurement and the benchmark in the switched to Task2 which will release the semaphore. previous section is that this method includes the The semaphore, once released, will wake up Task1, and overhead time by RTOS to process the message queue the execution context will be switched over to Task1. and to wake up the receiving task.

Task1: (higher priority) Task2: (d) Pass/Receive message time Retrieve message from queue (A) Besides semaphore, message passing has become (will be put into wait list) Put message onto queue more and more popular for synchronization purposes (B) [43]. In this measurement, message passing mechanism Time to pass message from task2 to task1: (A) to (B) based on memory pointer passing is used, i.e. not the copying of message into an internal RTOS area because Figure 8: Message passing time measurement not all RTOS support this approach. The measurement (f) Fixed-size memory acquire/release time method is explained in Figure 7. In RTOS, only fixed-size dynamic memory allocation Task1: should be used. The allocation and de-allocation time (A) must be deterministic. Figure 9 illustrates the method Pass message to the queue (I.e. message is copied into the queue) used to measure the time to acquire and the time to (B) release a fixed-size memory block.

Retrieve message from the same queue Task1: (C) (A) Acquire fixed-size block Time to put message onto queue: (A) to (B) (B) Time to retrieve message from queue: (B) to (C) Release fixed-size block Figure 7: Pass/Retrieve message time measurement (C) There is only one task: Task1. Firstly, the task will Time to acquire memory block: (A) to (B) pass the message pointer (normally to an internal Time to release memory block: (B) to (C) message queue), and after that the task will retrieve the Figure 9: Acquire/Release fixed-size memory time same message pointer. Table 7 shows the APIs used in measurement each RTOS. There is only one task: Task1. Task1 first acquires a μC-OS/II μTKernel EmbOS μITRON fixed-size memory block (128 bytes), and then releases Pass message OSQPost() tk_snd_mbx() OS_Q_Put() snd_mbx() the block. Table 8 shows the APIs used to API Retrieve acquire/release memory block in each RTOS. OSQPend() tk_rcv_mbx() OS_Q_GetPtr() rcv_mbx() message API μC-OS/II μTKernel EmbOS μITRON Table 7: APIs used for message passing benchmark Acquire fixed- OS_MEMF_ OSMemGet() tk_get_mpf() get_mpf() size block API Alloc() Release fixed- OS_MEMF_ OSMemPut() tk_rel_mpf() rel_mpf() (e)Inter-task message passing time size block API Release() Figure 8 illustrates the method used to measure the Table 8: APIs used for fixed-size memory benchmark message passing time between tasks. There are two tasks: Task1 and Task2 with Task1 having higher (g) Task activation from within interrupt handler time priority. Task1 will be first executed, and it will try to An RTOS has to deal with external interrupts that retrieve a message pointer from the queue. As there is may be asserted at any time. Execution of interrupt no message available yet, Task1 will be put into handler is normally kept as short as possible to avoid

10 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

affecting the system response. In the case where long CodeCode size size (ROM) (ROM) processing is required, the handler can activate another 14000 task that will do the necessary processing. The time 12000 from when the interrupt handler resumes the task till the Task switch time 10000 Get & release time when the task is executed is crucial to the system semaphore (1 task) TaskPass switch semaphore time Get & release semaphore (1 task) design. The measurement setup is explained in Figure 8000 (from 1 to another task) PassPass semaphore & retrieve (from 1 to another task) Passmessage & retrieve message to queue to queue

10. bytes PassPass message message (from 1 to another task) 6000 Acquire(from & release 1 to another fixed-size memorytask) block Interrupt Bytes Task1: (higher priority) Task2: TaskAcquire activation &from release interrupt fixed-size memory block Do some processing Go to sleep Do some processing 4000 Task activation from (A) interrupt (B) … Do some processing Resume task1 2000

0 μC-OS/IIucos/ii μTKernel utkernel EmbOS embos μITRON uitron Time from interrupt handler resuming task1 till Task1 is resumed: (A) to (B) Figure 11: Code size comparison for 4 RTOSes Figure 10: Task activation from interrupt handler time measurement Similar to the code size comparison, Figure 12 shows There are two tasks: Task1 and Task2 with Task1 the RAM information for the 4 RTOSes. μTKernel and having higher priority. Besides, an external interrupt μITRON can be seen to have relatively lower RAM with proper handler is also set up. At the beginning, usage, while μC-OS/II and EmbOS are slightly higher. Task1 is first executed. Task1 goes to sleep/inactive Depending on the requirement of each benchmark, we state, and the execution context switches over to Task2. set the number of tasks, stack size and number of RTOS Task2 simply does some processing continuously. objects (e.g. semaphore, event flags) to be the same for When an externally interrupt occurs, the interrupt all RTOSes. The amount of RAM differences among handler will be executed and it will resume Task1. the RTOSes range between 7-10 bytes, which might be Execution context will be switched over to Task1. Table due to internal implementations or due to method of 9 shows the APIs used for each RTOS. designing the APIs. In summary, the ROM and RAM μC-OS/II μTKernel EmbOS μITRON usage of all these RTOSes are well suited for small Go to sleep OSTaskSuspend() tk_slp_tsk() OS_Suspend() slp_tsk() API microcontrollers. Based on Figure 11 and Figure 12, Resume from OSTaskResume() tk_wup_tsk() OS_Resume() iwup_tsk() μITRON has the most optimal usage in terms of both interrupt API ROM and RAM sizes. Table 9: APIs for task activation from within interrupt handler benchmark DataRAM size s ize (RAM) B. Benchmarking results 2500 (1) Memory footprint For each criterion, the benchmarking code is 2000 compiled, and the ROM and RAM usage can be Task switch time obtained from the toolchain report. By averaging the Get & release 1500 Tasksemaphore sw itch time (1 task) ROM information across all the test criteria, the average GetPass & release semaphore semaphore (1 task) Pass(from semaphore 1 to another(from 1 to another task) task) PassPass & retrieve & retrieve message to queue ROM size can be obtained. Figure 11 shows the code bytes Passmessage message (from to queue 1 to another task)

Bytes Bytes AcquirePass & releasemessage fixed-size memory block sizes for the 4 RTOSes when running the 7 benchmarks. 1000 Task(from activation 1 to from another interrupt task) Acquire & release μTKernel can be seen to have a larger code size. This fixed-size memory block Task activation from is due to the flexibility and comprehensiveness support interrupt 500 in the APIs, as explained previously in Table 3. μITRON and EmbOS, which are commercial RTOSes, offer relatively compact code size. Nevertheless, all 0 μC-OS/IIucos/ii μTKernel utkernel EmbOS embos μITRONITRON uitron these RTOSes can fit well into small microcontrollers of Figure 12: Data size comparison for 4 RTOSes limited ROM sizes.

11 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

μC-OS/II μTKernel EmbOS μITRON

Figure 13: Execution time benchmark for four RTOSes

(2)Execution time handler, followed by μC-OS/II, μITRON and EmbOS. Figure 13 shows the measurement of execution time With the benchmarking results shown above, each among the RTOSes for different benchmark criteria. As RTOS stands out to have its own strengths and the only variation in the system is timer interrupt (for weaknesses. As far as open-source RTOS is concerned, OS tick), each benchmark was executed at least twice to for a very small and compact ROM size RTOS, μC- ensure consistent results. Nevertheless, for all OS/II can be used. However, to have a more benchmarks, running once is enough to yield the correct comprehensive APIs support, μTKernel is measurement. For the task activation from within recommended (at the expense of a slightly higher ROM interrupt handler benchmark, there will be another footprint). On the other hand, to select a commercial variation which is an external interrupt (besides the RTOS, either μITRON or EmbOS is recommended with timer interrupt for OS tick). When performing this μITRON having slighter lower RAM footprint. Based benchmark, the external interrupt may or may not be on all these benchmarked results, different performance asserted during the OS critical section (the duration which OS disable interrupts). If it is asserted during the criteria can be compared to ensure that they meet the critical section, the response time of the OS will be time constraint requirements of the system when slightly longer. Hence this measurement may not selecting for a RTOS. include the worst case scenario. μTKernel is shown to have the lowest task switching IV. CONCLUSION time, followed by μITRON, μC-OS/II and EmbOS. On the other hand, μC-OS/II semaphore acquire and RTOS is increasingly popular for deployment with release time are the fastest. The fastest inter-task microcontroller-based embedded systems design. It can semaphore passing is achieved by μITRON, while μC- helps to improve the development cycles, code OS/II and μTKernel have better message passing and reusability, easy of coding as well as maintenance, message retrieval time as compared to μITRON and better resource and timing management even for small EmbOS. As far as fixed-size memory is concerned, μC- microcontrollers. RTOS can be differentiated from OS/II has the best execution time, followed by EmbOS, generic OS in terms of scheduling (priority-based), μITRON and μTKernel. Finally, μTKernel has the best predictability in inter-task synchronization, and performance time for task activation from interrupt deterministic behaviors. In this paper, a list of RTOSes

12 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

has been studied based on various selection criteria report, Sep. 2005. targeting small microcontrollers with less than [18] K. Sakamura, H. Takada, “μITRON 4.0 specifications”, TRON Association, Japan, 2002. 128Kbytes of ROM and 4Kbytes of RAM. [19] Segger, “EmbOS real-time operating system user & reference Subsequently, four RTOSes were selected and guide”, Segger Microcontroller System GmbH, 2006. benchmarked by porting them onto the same [20] Pumpkin Inc, “Salvo user manual”, Pumpkin Inc., 2003. [21] Pumpkin Inc, “Salvo, the RTOS that runs in tiny places”, microcontroller platform. A list of benchmarking http://www.pumpkininc.com/, Oct. 2007. criteria which is aimed to be simple, easy to be ported to [22] P. Levis, “TinyOS programming, revision 1.3”, TinyOS different platforms and representative of typical RTOS Community Forum, 27 Oct 2006. [23] U. Berkeley, “TinyOS, an open-source OS for the networked usages were used. The benchmarking results show that sensor regime”, http://www.tinyos.net/, 2006. each RTOS has different strengths and weaknesses [24] J. Electronic, “SharcOS user guide”, The SharcOS project, 2002. without any clear emerging winner. With these detail [25] S. R. Technologies, “eXtreme Minimal Kernel, a free real-time operating system for microcontrollers”, http://www.shift- performance benchmarks, potential adopters of these right.com/xmk/index.html, 2004. RTOSes can simplify their selection by examining their [26] D. Stewart, “Echidna real-time operating system”, specific application requirements. http://www.glue.umd.edu/dstewart/serts/research/echidna/index.s html. [27] A. J. Massa, “Embedded software development with ECOS?” REFERENCES Prentice Hall, Nov. 2002. [1] D. Kalinsky, “Basic concepts of real-time operating systems”, [28] P. Gai, D. Cantini, M. Cirinei, A. Macina, and A. Colantonio, LinuxDevices magazine, Nov. 2003. “Erika, embedded real-time kernel architecture - education user [2] K. Sakamura, H. Takada, “μITRON for small scale embedded manual”, Realtime System (RETIS) Lab, Scuola Superiore systems”, IEEE Micro, vol. 15, pp. 46–54, Dec. 1995. Sant'Anna, Italy, 2004. [3] J. Ganssle, “The challenges of real-time programming”, [29] G. C. Buttazzo, “Hartik: A hard real-time kernel for programming Embedded System Programming magazine, vol. 11, pp. 20–26, robot tasks with explicit time constraints and guaranteed Jul. 1997. execution”, Proceedings of the 1993 IEEE International [4] CMP Media, “State of embedded market survey”, Embedded Conference on Robotics and Automation, Atlanta, Georgia, USA, System Design Magazine, 2004. pp. 404–409, May 1993. [5] CMP Media, “State of embedded market survey”, Embedded [30] Keil, “Keil real-time kernel and operating system”, System Design Magazine, 2006. http://www.keil.com/rtos/, Mar. 2007. [6] CMP Media, “State of embedded market survey”, Embedded [31] R. Chrabieh, “Operating system with priority functions and System Design Magazine, 2005. priority objects”, TechOnline technical paper, Feb. 2005. [7] CMP Media, “State of embedded market survey”, Embedded [32] R. Bannatyne, G. Viot, “Introduction to microcontrollers, part 2”, System Design Magazine, 2007. Northcon98 Conference Proceedings, pp. 250–254, Oct. 1998. [8] K. Baynes, C. Collins, E. Filterman, B. Ganesh, P. Kohout, [33] T-Engine, “T-Engine forum”, http://www.t-engine.org, Nov. C. Smit, T. Zhang, B. Jacob, “The performance and energy 2007. consumption of embedded real-time operating systems”, IEEE [34] B. Millard, D. Miller, C. Wu, “Support for ADA intertask Transactions on Computers, vol. 52, pp. 1454–1469, 2003. communication in a message-based distributed operating system”, [9] J. J. Labrosse, “μC-OS/II the real-time kernel”, R & D Books, Computers and Communications Conference Proceedings, pp. (Miller Freeman, Inc.), Lawrence KS, 1999. 219–225, Mar. 1991. [10] T.-E. Forum, “μTKernel specification, 1.00.00”, T-Engine Forum, [35] Renesas Technology Corp., “Renesas high-performance Mar. 2007. embedded workshop (HEW)”, [11] R. Barry, “A portable, opensource mini real-time kernel”, http://www.renesas.com/fmwk.jsp?cnt=ide_hew_tools_product_l http://www.freertos.org, Oct. 2007. anding.jsp&fp=/products/tools/ide/ide_hew/, 2007. [12] K. Curtis, “Doing embedded multitasking with small [36] Renesas Technology Corp., “M16c/62P group hardware manual”, microcontrollers, part 2”, Embedded System Design Magazine, http://documentation.renesas.com/eng/products/mpumcu/rej09b01 Dec. 2006. 85_16c62pthm.pdf, Jan. 2006. [13] A. Martinez, J. F. Conde, A. Vina, “A comprehensive approach in [37] Micrium, “μC-OS/II ports for Renesas microcontrollers,” performance evaluation for modern real-time operating systems”, http://www.micrium.com/renesas/index.html, Nov. 2007. Proceedings of the 22nd EUROMICRO Conference, pp. 61, 1996. [38] Renesas Technology Corp., “μTKernel for M16C source code [14] R. P. Kar, K. Porter, “Rhealstone: A real-time benchmarking and documentation”, http://www.superh-tkernel.org/ proposal”, Dr. Dobb's Journal, Feb. 1989. eng/download/misc/index.html, Nov. 2007. [15] K. M. Sacha, “Measuring the real-time operating system [39] Segger, “EmbOS trial version for M16C6X (HEW 4.0 with NC30 performance”, Seventh Euromicro workshop on Real-time version 4.00)”, http://www.segger.com/ systems proceedings, Odense, Denmark, pp. 34–40, Jun. 1995. downloadform_embos_m16c_nc30_v540.html, Nov. 2007. [16] G. Hawley, “Selecting a real-time operating system”, Embedded [40] Renesas Technology Corp., “M3T-MR30/4 RTOS for M16c and System Design Magazine, 1999. R8c families conforming to μITRON 4.0-specification”, [17] M. Timmerman, L. Perneel, “Understanding RTOS technology http://www.renesas.com/fmwk.jsp?cnt=m3t- and markets”, Dedicated Systems RTOS Evaluation project mr30_kernel.htm&fp=/products/tools/os_middleware/u_itron/m3t

13 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE This article has been accepted for publication in IEEE Micro but has not yet been fully edited. Some content may change prior to final publication.

_mr30/child_folder/&title=M3T-MR30%20Kernel, Nov. 2007. [43] D. Kalinsky, “Asynchronous direct message passing rapidly gains [41] Renesas Technology Corp., “M16C/60, M16C/20, M16C/Tiny popularity”, Embedded Control Europe Magazine, Nov. 2004. series software manual”, http://documentation.renesas.com/ eng/products/mpumcu/rej09b0137_m16csm.pdf, Jan. 2004. [42] I. Ripoll, P. Pisa, L. Abeni, P. Gai, A. Lanusse, S. Saez, “RTOS state of the art analysis”, Technical report, Open Components for Embedded Real-time Applications (OCERA) project, 2002.

14 Digital Object Indentifier 10.1109/MM.2009.56 0272-1732/$26.00 © 2009 IEEE