Imaging Application Platform Using MB86S62 Application Processor Atsuhiro Suga Hidetoshi Nakahara Masato Hyodo Shinichi Takimoto Imaging solutions using application processors means providing customers with an application platform that includes software and evaluation boards based on Fujitsu’s extensive image- processing technologies. Customers who manufacture assembled products (set makers) can use this platform to develop and market multi-function products in a shorter period of time. In combination with Fujitsu’s Milbeaut image processor, this platform provides a high-perfor- mance image processing system that can provide customers with a competitive edge over other makers. This paper presents a specific example of this imaging application platform using Fujitsu’s MB86S62 application processor. 1. Introduction the development period and increasing the develop- Today’s smartphones and tablets are evolving at a ment costs compared to existing technologies. As a dramatic pace. Being capable of functions and applica- result, the development of LSI circuits using advanced tions that were inconceivable just a few years ago, they technologies increases risk and becomes a major fac- are now becoming an integral part of business and tor in reducing the customer’s return on investment everyday life. This epic development can be attributed (ROI) when developing individual products. In short, it to the use of application processors on the terminal makes new development projects all the more difficult. side to maintain sufficient processing speeds and to This state of affairs suggests a new business op- stress-free, routine Internet access made possible by portunity in providing new solutions for easing the the creation of a communication infrastructure cen- customer’s load (development costs, software devel- tered around Long Term Evolution (LTE).1) As a result, opment) in relation to 40-nm and beyond process a wide variety of compelling services are coming to be technologies. provided by interlinking terminal-side applications with Against this background, semiconductor manu- the cloud. facturers who develop application processors are, of Looking forward, we can expect advanced user course, targeting the smartphone and tablet markets, interfaces developed for smartphones and tablets to but they are also indicating a desire to participate be applied to a variety of products, which suggests that in the on-vehicle and digital-consumer-electronics image processing using application processors will be- markets. In contrast, customers who are makers of as- come increasingly important. sembled products (set makers) are beginning to move At the same time, a company would need cutting- in the direction of system integration using existing edge process technologies to fabricate such application application processors to reduce development costs processors for achieving novel products in a demanding and risks instead of developing new, proprietary LSI market and to enable it to differentiate itself from its circuits. This trend is providing Fujitsu—which excels in competitors. However, the development of LSI circuits the application-specific integrated circuit (ASIC) busi- using 40-nm and beyond process technologies means ness—an impetus for revolutionizing the way it supports an explosive increase in the difficulty of layout and set makers. the amount of labor for testing, thereby extending In particular, Fujitsu is focusing on applications FUJITSU Sci. Tech. J., Vol. 49, No. 1, pp. 49–58 (January 2013) 49 A. Suga et al.: Imaging Application Platform Using MB86S62 Application Processor that can exploit the company’s extensive expertise over many years and its image processing technology in image processing technology and is providing its centered about its Milbeaut image processor. In addi- customers with an application platform that includes tion to excelling in power consumption, performance, software and evaluation boards. In this way, Fujitsu has and device area (and therefore lowering costs) by in- achieved a solution for providing application-processor- tegrating software and hardware based on advanced based development while enabling the customer to CPU/GPU devices, this platform provides a highly com- incorporate its proprietary logic to achieve original petitive multimedia framework featuring a sequence of functions in much the same way as can be done with image processing functions including sensing, analysis conventional ASICs. Furthermore, by combining this and processing, compression and decompression, and platform with Fujitsu’s Milbeaut image processor, cus- display and output. The platform provides customers tomers can use this application platform as an imaging with value by reducing costs and accelerating delivery platform in a manner different from what can be of- through a platform geared to imaging-related appli- fered by other application processor vendors. cation-specific standard products (ASSPs), by boosting This paper presents a specific example of using existing business and facilitating the creation of high- this imaging application platform in conjunction with value-added products, and by contributing to the Fujitsu’s MB86S62 application processor. creation of new businesses. The configuration of this platform is shown in 2. Imaging application platform using Figure 2. It consists of a system-on-a-chip (SoC) based MB86S62 on application processors, Linux2) and Android3) board The concept of an imaging application platform is support packages (BSPs), and a multimedia framework shown in Figure 1. comprised of Fujitsu image processing functions. This The idea here is to provide a user-friendly plat- platform makes for easy system development since it form that exploits Fujitsu’s ASIC technology developed allows the customer to create original applications on Imaging application platform Reduces costs and Provides competitive Provides APIs for easily accelerates delivery multimedia framework in an open implementing applications by using a platform geared to environment on target system imaging-related ASSPs Linux OS Boosts existing business Has high driver-level efficiency and facilitates creation of high-value-added SoC for imaging application platform products CPU GPU DSP Accelerator Connections Contributes to creation of new businesses High-efficiency architecture (power, performance, area, ease of use) Fujitsu imaging technology Analysis/ Compression/ Sensing processing decompression Display/output Multimedia framework (comprehensive environment including video/audio media) GPU: Graphics processing unit DSP: Digital signal processor Figure 1 Concept of imaging application platform. Figure 1 50 Concept of imaging application platform. FUJITSU Sci. Tech. J., Vol. 49, No. 1 (January 2013) A. Suga et al.: Imaging Application Platform Using MB86S62 Application Processor Android BSP Set-maker development section Options Application Test application Google supplied Expanded Application framework application framework ARM supplied OpenGL ES Expanded Standard library API library library Linux BSP Hardware architecture Multimedia framework FSL ISP section Imaging Computing subsystem High-bandwidth USB CSI SATA Audio Display Standard ISP Other subsystem connectivity HAL HAL HALs HAL HALs GMII/Ether CPU GPU PCIe ISP cluster cluster Cache coherent interconnect System control Clock Codec Reset I2C Generic kernel DSI Post DRAM Power-supply proc. interface UART control HDMI Platform specific (V4) Display GPIO Low-bandwidth UART connectivity Audio I2C High-bandwidth interconnect with QoS with interconnect High-bandwidth SPI Device drivers FSL ISP I2S section Standard Display GPU Other driver driver driver drivers FSL: Fujitsu Semiconductor ISP: Inter-chip connection DRAM HAL: Hardware abstraction layer QoS: Quality of service Figure 2 MB86S6Figure platform 2 confi guration. MB86S6 platform configuration. top of these Linux and Android BSPs. and a secondary cache of 256 KB. The following subsections describe the hardware 3) Powerful and extensive multimedia functions and software confi guration of this platform and associ- The MB86S62 features two camera input ated features. interfaces. The main interface supports up to 8 megapixels (Mpixels) using a MIPI (Mobile Industry 2.1 MB86S62 Processor Interface)4) camera serial interface (CSI) or an A block diagram of the MB86S62 application pro- 8-bit parallel interface. The sub-interface supports up cessor is shown in Figure 3, and its main specifi cations to 3 Mpixels using an 8-bit parallel interface. The video are listed in Table 1. In developing this platform, much codec supports a variety of video standards including attention was paid to achieving greatly reduced power MPEG-2,5) H.264,6) and VC-1.7) It supports, in particular, consumption and high-performance image processing. Full HD 1080p at 30 fps encoding/decoding and 720p The hardware measures taken to achieve these goals dual encoding. The audio codec supports various audio and key MB86S62 features are described below. standards such as MP38) and AAC and various interfaces 1) Low-power process including I2S9) and SPDIF.10) The use of an advanced 45-nm, low-power pro- 4) 3D graphics engine cess in this application processor has reduced power The MB86S62 integrates a 2D/3D graphics hard- consumption and lowered costs. ware accelerator supporting OpenVG1.111) and OpenGL 2) ARM Cortex™-A8 processor ES2.0 and featuring a dual core having a processing The MB86S62 incorporates an ARM core, a ability of 30 MPoly/s and 400