An Introduction to Openvg™ FTF-AUT-F0465

An Introduction to OpenVG™ FTF-AUT-F0465

Oliver Tian | Auto FAE

M A Y . 2 0 1 4

External Use Agenda

• Trend of Graphics in Vehicle • Roadmap of Cluster • Introduction of Rainbow/Vybrid • OpenVG Scenario • Development Ecosystem • Conclusion

TM External Use 1 Trend of Graphics in Vehicle

TM External Use 2 The Connected Vehicle Infotainment + Communication + Security

• Consumer electronics trends are dictating features in the car • Always connected, applications driven, advanced graphics • Infotainment systems becoming battleground for Auto differentiation • As more connected systems get introduced into the vehicle, the need for security is critical − Increasing external communication features (Bluetooth, TPMS, Ethernet, Wi-Fi, etc). − Future interface for vehicle-to-vehicle and vehicle-to-infrastructure.

TM External Use 3 Mobility for Everyone Affordable Solutions for Emerging Markets

• 100M vehicles annually forecasted before 2020, on top of motorcycle & e-bike growth • 80% of quantity growth after 2015 happening in emerging markets • Safety and emissions reduction are key for a sustainable development

Source: IHS Automotive, February 2014

TM External Use 4 More, More, More for Less, Less, Less

More performance, more embedded memory, more safety for less cost, less power and less development effort More

• Electronic complexity

• ECUs per car (50+)

• MCUs per car (100+)

• In-car Wi-Fi ® (7.2Mbps and 3.7Bpcs by 2017) iSuppli

Less Reuse

• Other markets have less critical applications • Some automotive specific challenges

TM External Use 5 Today’s Car

• Complex computerized control − Millions of lines of code, from multiple vendors − Dozens of distinct ECUs, from multiple vendors • Shared internal networking (e.g., CAN, FlexRay) − Increasing external communications features . Telematics, Bluetooth, TPMS, RDS, XM radio, GPS, keyless start/entry, USB ports, Wi-Fi, etc.

TM External Use 6 Tomorrow’s car -> much more of everything

• The Infrastructure − The Intelligent Transportation System (ITS) . V2V/ACAS, V2I, traffic control, autonomous driving, real-time data fusion, pervasive sensing

TM External Use 7 Driver Information Systems Roadmap

Applications st Q3 15 I.MX 8 SOLO 1 Si Jan12 I.MX 6 DUAL ARM Cortex-Dual A7/M4, High-end VSPA, 2D-ACE, GC355 Q1 15 I.MX 6 SLX +VPU I.MX 6 DUALLITE Proposal Best Graphics, ARM Cortex-Dual A7/M4, May12 VSPA, 2D-ACE, GC355 Q1 ‘14 Performance & I.MX 6 SLX I.MX 6 SOLO Planning Integration I.MX53

Execution Jun 14 MACHxx (Halo) ARM Cortex-A5/M4/M0+, 2D- Mid-range ACE, 2D-GPU, GC355 Production Jan 13 SVF5/3R Pin Compatible (Faraday/Vybrid) QFP/BGA’s Optimized ARM Cortex-Dual A5/M4, S08 Core MPC5645S 2D-ACE, GC355 Graphics, High (Rainbow) MACCxx S12 Core Integration Q2 15 (Corona) S12z Core (MagniV) ARM Cortex-/M4/M0+, 2D-ACE, Power Architecture 2D-GPU, GC255 MPC5606S Pin Compatible Cost Effective ARM Cortex Performance (Spectrum) Color Graphics QFP’s Security

Aug 12 S12ZVH (Lumen-4WL) Q1’16 Low-end S12XHY LumenNG512 (Sea Wasp) ARM CortexM0+, 2xCANPhy, MagniV S12HY Nov 12 (Lumen-2W) Lowest System S12ZVHY Cost, High (Jellyfish) Integration S08LG32

2011 2012 2013 2014 2015 2016 2017

TM External Use 8 r6 11-Feb-14 MPC5645S: Rainbow 2M

General Characteritics: System Debug Crossbar Masters • PPC e200z4d Dual Issue core, 5 stage pipeline, 4k I-Cache Integration • 16 entry Memory Management Unit TCON JTAG Z160 • 2M FLASH with ECC PPCTM RSDS DCU • 64k SRAM with ECC VReg Nexus e200z4d 2D VIU DCU Lite • 16 channel DMA Class 3+ Core Oscillator GFX • Memory Protection Unit (16 regions) (4k I-cache) • QuadSPI Serial Flash Interface FMPLLx2 • Voltage Regulator with external ballast transistor

RTC/32Kosc MMU • Real Time Counter + 32kHz crystal oscillator 16ch • Watchdog, Periodic Interrupt Timer, System Timer Interrupt DMA • 4-16MHz XOSC Controller • Frequency Modulated PLL (x2) • Nexus 3+ / JTAG

CROSSBAR SWITCH Graphics Features: Memory Protection Unit (MPU) • 2D Graphics Accelerator: AMD z160 OpenVG • 1M Graphics SRAM • Display Control Unit: 4 planes / 16 layers PIT • Display Conrol Unit –Lite: 2 planes / 4 layers WDOG • DDR DRAM interface (324BGA only) Sys Timer 64k 1M I/O RLE DDR-2 2M Quad • Video input Unit (VIU) SRAM Graphics Bridge Decode DRAM Flash SPI2 • RLE Decoder Boot SRAM Interface

Assist EEE Crossbar General Characteristics: Module • Up to 120MHz operation (BAM) Slaves • Low power modes • -40 to +105C, 3.0V to 5.5V • 176LQFP, 208LQFP, 324BGA package options Communications I/O System Peripherals and Communications: • 6 Stepper Motor Drivers with Stall Detection 20 ch SSD eMIOS0 eMIOS1 3 4 3 4 • Sound Generator Module SGM ATD 6 16ch 16ch FlexCAN LINFlex DSPI I2C • 3xCAN, 3xDSPI, 4xI2C, 4xLIN 10bit SMD • 32 channel eMIOS (PWM+Timer) • 20 channel, 10bit ADC

TM External Use 9 Vybrid F Series System

Key Functional Characteristics: • Cortex-A5 “value“ processor for best MIPs/mW • On-chip SRAM, 2D-ACE, Quad-SPI and RTOS result NEON optimized HMI Tools Royalty-free CODECS/Libraries Comm Stacks RTOS in low system cost (no DRAM) • Flexible memory solution configurable based on application needs (1.5MB SRAM or 1MB SRAM + 512K Power Multimedia Cortex-A5 L2) Management Up to 400MHz Connectivity - single 3.3V supply NEON/FPU USB OTG + Phy • DDR3 and OpenVG support for performance critical - low voltage reset 32KB/32KB L1 applications USB OTG + Phy • Synchronous Audio Interface (SAI) supporting 1.5MB SRAM System eMMC/SD independent I2S, TDM, AC97 and Codec/DSP Connectivity x2 System and interfaces CAN General Purpose x2 Audio I/O • Enhanced Serial Audio Interface (ESAI) with I2S and I2C DMA SAI x4 (i2s x4) x4 ESAI x1 (i2s x6) AC97 modes HAB 4.1 Security UART/LIN Tamper Detect SP/DIF x6 Receiver/Transmitter Key Electrical Characteristics: SPI Watchdog Timer Sample Rate x3 Other Timers (x8) Convertor • A5 at up to 400MHz, and DDR3-800

Media Local Bus Real Time Clock AudioReciever Multiplexer • -40 to +85C (ambient) 3-wire 10/100 Ethernet Pulse Width • 3.0V to 3.6V supply (3.3V nominal) Modulator Display I/O Package: GPIO 2D-ACE x2 External Memory Animation &Comp Engine • 144/176 LQFP; 364MAPBGA 2x 12-bit SAR ADC DRAM (16-bit) with Touchscreen OpenVG GPU LPDDR2, DDR3 (GC355) Initial Samples: Dual Quad-SPI 2x 12-bit DAC Segment Display Flash (SDR and DDR) With Tone Generation Controller (4x40) • Ready Internal Temperature NAND/NOR Flash Monitor Camera Input, 18-bit + 8/16b Composite (4 to 1)) Enablement • Production Software including CODECs, Stacks, RTOS • UI development Tools for 2D-ACE • Radio Reference Design – HW and SW

TM External Use 10 • ARM Core Architecture: - Cortex M4 vehicle processor - Cortex A5 application processor MACHxx Block Diagram - Cortex M0+ I/O processor

• 4MByte ECC flash • 2x 512KB ECC SRAM Power Management Cortex-A5 Cortex-M4 • 1.3MB non-ECC SRAM Single 3.3V supply Up to 320MHz Up to 160MHz Low Voltage Detection NEON / FPU/ MMU FPU • Supports 2 x WVGA displays: Low Power Control 32K / 32K L1 cache 16K / 16K L1 cache - OpenVG 1.1 GPU - 2 x 2D-ACE display interfaces 64 KB TCM System and General Purpose - DigitalRGB, RSDS, LVDS i/f - Hardware HUD warping engine Memory Protection DMA I/O Processor - Digital camera input Security-CSE2 Autonomous RTC Cortex – M0+ Up to 80MHz • Extensive connectivity : Peripherals - Ethernet AVB, MLB50, CAN-FD 32K ECC SRAM CAN(FD) x3 I2C x2 Internal Memory • I/O Processor (Cortex M0+) UART/LIN x3 Watchdog Timers - Supports autonomous operation 4MB ECC Flash SPI x5 IC/OC Timers / PWM Stepper Motor Drivers - Peripheral control and Low power MLB 3-wire SMD / SSD x6 1.3MB Non ECC SRAM operation

12-bit SAR ADC 10/100 Ethernet + AVB 2 x 512KB ECC SRAM • Security (CSE2) Audio / GFx / Video / Display - Meets SHE specification External Memory - Meets GM’s Global B Cybersecurity SGM (includes I2S) 2D-ACE + inline HUD requirements DDR QuadSPI Flash x2 Warping Digital Video In • Functional Safety 16/32-bit DDR2 2D - ACE - Built in support for ASIL-B GC355 OpenVG GPU Open LDI & RSDS 16-bit SDR • Software Support - AutoSAR - GC355 - I/OP Stepper motor driver

• BGA and QFP package options: - 176/208LQFP + 516MAPBGA

TM -40 to +105C TA External Use 11 MPC56xxS vs MACHxx vs Vybrid

Feature MPC56xxS MACHxx Vybrid

Spectrum - 1 x WQVGA MACCxx– 1 x WVGA Target Use-Case Up to 2 x WVGA Rainbow – 2 x WQVGA MACHxx- 2 x WVGA

Spectrum - e200z0h Corona - ARM CM4 Core(s) ARM CM4/CA5 Rainbow - e200z4d Halo - ARM CM4/CA5

Security Censorship only CSE2 (encrypted protection) CAAM

Safety Limited features ASIL-B Limited Features

Flash Spectrum-1M, Rainbow-2M Corona-1-2M, Halo-2-4M None – supports XIP QuadSPI Flash

Corona-256kB Up to 1.5MB on-chip SRAM in total Graphics SRAM Spectum-160kB, Rainbow-1MB Halo-1.3MB (+1MB ECC RAM) (1MB no ECC, 512k ECC)

Stepper Motors SMD/Stall Detect ‘Intelligent’ SMD/SSD None

Head-Up Display None Hardware warping engine on Halo GC355 can support warping

Spectrum - None Corona/Halo - 2D-GPU GPU GC355 OpenVG Rainbow - Z160-OpenVG Halo - GC355 OpenVG

High Speed Serial None MLB50, ENET-AVB, CAN-FD MLB50, ENETx2, USB-HS x2

GRAM Spectrum – None Corona – SDR LPDDR2 / DDR3 Expandability Rainbow – SDR/DDR2 Halo – (SDR)/DDR2

EVB, Basic Compiler tools, “Lab EVBs, Complete High-Performance EVBs, Complete High-Performance Enablement Bench” Demos Demos, Optimized Graphics Tools Demos, Optimized Graphics Tools

TM External Use 12 r0: 23-Sep-13 MPC5645S working scenario LEDs

3x CAN eMIOS 6 Stepper Motor Drivers PWMs with patented stall detection 4x LIN Video Vreg MPC5645S Input Unit 64K SRAM 4x16k 1MB RTC EEPROM Graphics 125MHz RAM sound Power 2MB eMIOS Display TFT DISPLAY PWMs E200z4d core FLASH Control DCU on MPC5645S can drive up to QuadSPI Unit 800x480 24-bit LCD DRAM Serial Flash DCU Lite OpenVG • Optional 2nd Display (DCU-lite) interface Controller GPU OpenVG1.1 GPU (Z160) natively renders

Low-cost • Lines, curves, polygons, textures,etc. Low-cost Quad Optional Quad Serial • Perspective transformation Serial SD / DDR- Flash DRAM Flash • True-type fonts

TM External Use 13 Vybrid working scenario

Serial Serial DRAM Flash Single Flash 3.3V Rail 364BGA

Power 16-bit Dual DDR DRAM Quad-SPI Color LCD Manage- 2D-ACE Up to WVGA ment Unit Interface Interface Vybrid F 24bpp GPIO I2S ARM Cortex-A5 Radio Chipset NEON RTC SPI 400MHz 32K/32K Caches Camera Rearview Interface Camera input ASRC OpenVG GPU with resizing CAN PHY Optional CAN 1.5MByte SRAM 4-ch ADC Touch Screen CAN PHY CAN I2S USB OTG MOST MLB SD/MMC USB I2C I2C UART INIC -50 SD/MMC OTG

iPOD Auth. CD 2xSD Card or drive Managed NAND

TM External Use 14 MACCxx/MACHxx working scenario Common Family Platform

Common GPU Platform WVGA Premium WVGA Common ARM Core Architecture: /HUD - Cortex A5 application processor for high resolution HUD graphics capability* ENET - Cortex M4 vehicle processor common platform for network communications CAN - Cortex M0+ I/O processor for stepper motor control, peripheral control, and low power performance Gauges w/ High Res TFT & HUD SDRAM FLASH Common GPU Platform: - Common 2D-ACE provides low memory footprint WVGA WVGA with advanced graphics capability Advanced - Less requirement for external memory - Scalable vector graphics & font support ENET - Vivante GC355 & GC255

CAN External Components: Gauges w/ - 1x WVGA supported without external RAM High Res TFT’s - 2x WVGA supported with external RAM SDRAM FLASH - High performance DDR Quad SPI interface** - 16/32 Bit DDR2/SDR** WQVGA Regular

* Product dependent (Halo supports A5) QFP Pin Compatible Pin QFP

** Package dependent

CAN Gauges with TFT

Basic LCD Scalablbe

CAN Gauges with LCD Additional Use-cases possible with different package & memory options

TM External Use 15 r0: 23-Sep-13 What is Vector Graphic

Bitmap Graphic Vector Graphic

Vector Graphic is size- independent!

TM External Use 16 Vector Graphic

• Vector graphics are drawn and stored as mathematical vector formulae

• Each vector and fill is assigned color value, instead of assigning color to each separate pixel

• A black circle can be represented as: − x = r cos θ y = r sin θ or: − x2+ y2 = r2 − With color value 0000 for black

• Benefits ‒ Infinitely zoomable ‒ Independent of screen resolution ‒ Saves data memory

TM External Use 17 Vector Graphic

Spline with controls

TM External Use 18 Where is Vector Graphic used today?

Fonts Drawings Page Layout Animations

Formats Postscript

Reader Design Apps Inkscape Adobe Toolsuite Applications - Illustrator - Flash

TM External Use 19 OpenVG Feature Overview

• Design Philosophy − Expands the OpenGL programming model to 2D vector graphics − Provides a low-level hardware acceleration abstraction layer − Uses OpenGL-style syntax where possible − Allows flexibility in the way acceleration can be provided − Enables hardware vendors to use their own preferred internal representations • The VGU Utility Library − Higher-level Geometric Primitives − Image Warping

TM External Use 20 OpenVG Highlights Path (Open path; Closed path) The path is the basis for all vector objects. A path is made up of one or more line segments connected by two or more anchor points. Paths can be made from a combination of straight lines and curves, each of which may be made up of many connecting points. Paths can be open or closed. An open path is one with unconnected end points, while a closed path is one whose start Bézier curve (Curve) and end points meet. A curved segment of a path is known as a Bézier curve (after French mathematician Pierre Bézier). Bézier curves are defined by mathematical equations - essentially, the coordinates of a curve can be calculated Open paths (anchor points and line segments shown in blue) and drawn by knowing the position of two end points and two control points.

Closed paths (anchor points and line segments shown in blue) Fill A fill can be applied to any area within a path. Fills can be single blocks of colour, gradients, patterns or images (raster or vector).

Coloured fill Gradient fill Patterned fill

TM External Use 21 OpenVG Rendering Pipeline

• Path Definition • Stroking Path Definition & Line width, joins & caps, dashing, etc. Setting of API Parameters • Transformation 2 x 3 (paths) and 3 x 3 (images) transformations Stroking • Rasterization Transformation • Clipping & Masking − Scissor rectangles Rasterization • Paint Generation Clipping and Masking Flat color, gradient, or pattern paint • Blending Paint Generation Multiple blend modes Blending • Dithering • Image Filters Dithering

TM External Use 22 Path Definition

• MOVE_TO, LINE_TO, QUAD_TO, CUBIC_TO, CLOSE_PATH

− Elliptical Arcs (px, py) − Absolute / Relative Coordinates − Smooth Curves (sx, sy) − Path Interpolation − Path queries: (ox, oy) . Bounding boxes . Point along path . Tangent along path − Non-Zero and Even-Odd fill rules

TM External Use 23 Stroking

• Stroking takes a path and defines an outline around it based on: − Line width − End cap style (butt, round or square)

− Line join style (bevel, round or miter)

Miter Length

− Miter limit (to convert long miters to bevels) − Dash array and offset Dash array = { 10, 20, 30, 40} / Dash Phase = 35

25 10 30 10 30 10 30 ...

TM External Use 24 Stroking – Dash Array

Dash array = { 10, 20, 30, 40} / Dash Phase = 35

TM External Use 25 Transformation

• Paths use 2 x 3 affine transformations • Images use 3 x 3 perspective transformations • Transformation functions are similar to OpenGL: − vgLoadIdentity − vgLoadMatrix − vgGetMatrix 1.080 0.101 0 − vgMultMatrix 0.209 0.691 0 − vgScale 1.28 10 3 1.19 10 3 1 − vgRotate − vgTranslate

• NOTE: If you want to transform a VG path / object into a perspective, you must do that before sending the points to OpenVG

TM External Use 26 Rasterization

• The goal of rasterization is to determine a filtered alpha value for each pixel, based on the geometry around that pixel • Filters may be up to 3 pixels in diameter • OpenVG handles rasterizing arbitrary shapes 0 0 .1 .4 .5 .2 0 0 0 .3 .8 1 1 .9 .4 0 .4 .8.9 1 1 1 1 .8 0 .6 .4 .3 .7 1 1 1 .3 0 0 0 .2 1 1 1 .5 0 0 0 .1 1 1 1 .5 0 0 0 .1 .9 1 .9 .2 0 0 0 0 .3 .5 .2 0

TM External Use 27 Clipping

• Pixels outside a rectangular viewport are not drawn. • Only pixels inside a set of scissor rectangles are drawn. • Clip-rects help to cut down the amount of pixels rendered in cases that have only a portion of the screen being rendered. This is essential for GUIs and can help with both maximizing performance and minimizing power.

Viewport

Scissor Rectangles

Pixels drawn

Pixels not drawn

TM External Use 28 Masking

• In addition to clipping, a per-pixel mask may be applied • The mask has an alpha value at each pixel that is multiplied by the alpha from the rendering stage • May be used to “cut out” an area, create a transition between areas • Mask values may be modified using image data − Fill, Clear, Set, Add, Subtract, Intersect Generating a Mask is a ‘draw’ action and relatively expensive. Masks should be generated infrequently to minimize performance impact.  

TM External Use 29 Paint Generation

• Paint is generated pixel-by-pixel and applied on top of geometry • The alpha from the previous stage (rendering + masking) is used to determine how much paint to apply • Separate paint objects for stroking, filling

• Paint is applied through an affine transformation Increasing Performance • Four types of paint are supported: − Flat color paint − Linear Gradient paint: Points (x1, y1) and (x2, y2), color ramp Cost

− Radial Gradient paint: center (x, y), focus (x, y), radius, color ramp − Pattern paint based on an image, tiling mode

(x1, y1)

r (cx, cy) (x2, y2)

TM External Use 30 Blending

• Combine masked alpha from path with paint alpha • Blend the result onto the drawing surface • Blending is a function of: − The paint (R, G, B) color − The masked alpha value (path alpha mask alpha paint alpha) − The destination (R, G, B) color − The destination alpha value (1 if no stored alpha) • There are 8 blending functions: − Porter-Duff rules (3) − Lighten (choose lighter of source and destination), Darken − Multiply (black source pixel forces black, white leaves unchanged) − “Screen”(white source pixel forces white, black leaves unchanged) − Additive (add pixel values, add alpha up to 1)

TM External Use 31 Insert Table

• Images are defined using one of 15 pixel formats − Linear or non-linear (sRGB) color spaces (8/8/8, 5/6/5 or 5/5/5 bit depths) − Linear or non-linear grayscale − Pre-multiplied or non-premultiplied alpha − 1-bit Black & White (e.g., for Fax applications) • Images may be stored in accelerated memory • Image filters may be applied: − Color Matrix − Convolve, Separable Convolve, Gaussian Blur − Lookup, LookupSingle − Dither • Images may be drawn in perspective • Image may be used as a stencil to apply paint

TM External Use 32 OpenVG Programming Model

• OpenVG is designed around a State Machine based client-server model − User ‘sets’ and ‘gets’ variables in the machine (enable/disable, bind, etc.)

OpenVG Stack Application vgGet/vgSet Context - State

− OpenVG: handles to paints, paths, gradients, etc., to avoid the re- preprocessing of each frame • Data types in a function are determined by a i, f, or v postfix − Integer, float, or vector • Execution decoupled − Execution is only guaranteed when user blocks (vgFinish) − Or flips the drawing surfaces (eglSwapBuffers)

TM External Use 33 Coordinate System

User Coordinates Screen Coordinates

239 239

0/0 319 0/0 319

1.0 239 319 239

0/0 1.0 0/0 319

TM External Use 34 Graphic Processor, OpenVG1.1

• Full fixed function hardware vector GC355 GPU graphics GPU GC355 GPU Core • Hardware Tessellation AHB AXI Host Interface − Minimum CPU involvement Memory Controller • 16x FSAA Graphics − Photorealistic quality Pipeline Imaging Engine VG Pixel Front Vector Graphics Engine Engine − No performance degradation End • Multiformat rendering − sRGB color transformation • High quality vector font rendering • Standard API OpenVG1.1

TM External Use 35 r0: 23-Sep-13 GC355 Vector Graphics Engine

VGA resolution 16X AA GC355 (Frames/sec) 320MHz (projected)

GC355 VGMark Tiger (rotation) 85 Performance @ 320MHz UI 96 Navigation 28 Flash 42

• Independent 2D Vector GPU Use-cases: − Instrument cluster: 2D engine accelerates needles at 60fps; 2D-ACE renders the rest of the scene • Infotainment: UI acceleration • Native rendering of true-type fonts, with 16x Anti-Aliasing • Additional graphics acceleration for dual display systems

TM External Use 36 r0: 23-Sep-13 Vector Graphic Processor: MACCxx

• Vector graphics − Lines / Bezier curves GC255 GPU − Support for vector fonts • Raster graphics GC255 core AHB AXI

− Raster operations (copy, blend, fill) Host Interface

• Hardware Tessellation Memory Controller − Limited CPU involvement Graphics Size Optimized Pipeline Pixel • Arbitrary Rotation Size-optimized Vector Imaging Engine Front Engine Graphics Engine End • 16x FSAA − Photorealistic quality • Optimize low footprint driver − Also runs on GC355 GPU on MACHxx

TM External Use 37 r0: 23-Sep-13 OpenVG Resources

MPC5645s Z160 GFX 166Mpxl/s，OpenVG 1.1

Vybrid GC355 GPU 300Mpxl/s，OpenVG 1.1

MACHxx GC355 GPU 300Mpxl/s，OpenVG 1.1

MACCxx GC255 GPU 300Mpxl/s，OpenVG 1.1

TM External Use 38 Development Ecosystem

• CodeWarrior • iAR • GCC • PVG illustrator plug-in • PVG Converter tool • Adobe flash • Photoshop • Actia • iKivo • Tiny UI

TM External Use 39 Conclusion

OpenVG in Embedded Graphics System

Elastic

Efficient

Easier

TM External Use 40 Designing with Freescale

Tailored live, hands-on training in a city near you

2014 seminar topics include • QorIQ product family update • Kinetis K, L, E, V series MCU product training

freescale.com/DwF

TM External Use 41 TM