Easy! D3 Intelligent Camera Platform

Intelligent Cameras | Framegrabbers | Made in Germany

WWW.STEMMER-IMAGING.COM Ease of Use The D3 Intelligent Camera

New Intelligent Camera Platform The D3 is the latest, most advanced intelligent camera generation by VRmagic. The D3 platform was designed with usability, flexibility, and performance in mind.

Excellent Usability The D3 platform runs a wide range of embedded software and libraries, such as Common Vision Blox Embedded, EyeVision, or HALCON Embedded. This way you can easily take advantage of the latest, state- of-the-art machine vision algorithms. Additionally, the new Mono compatible .NET interface makes applica- tion development with the VRmagic SDK much easier. Software development for the D3 intelligent camera platform – that’s ease of use.

Built-In Flexibility The D3‘s embedded system provides users with a high level of flexibility by supporting a multitude of interfaces, such as Ethernet, USB, and GPIOs. Choose either a standard OEM interface board for compact systems integration or an interface evaluation board for convenient test and development. A version for industrial environments is also available. Depending on the business case, a custom interface board may also be a viable option.

High Performance The D3 intelligent camera platform features a 1 GHz ARM® Cortex™-A8 Core with floating point unit (FPU) running . A 700 MHz C674x™ DSP with FPU is at your disposal for computationally intensive algo- The D3 Industrial Camera has a rigid aluminum rithms. 2 GB DDR3-800 RAM with 6103 MB/s band- body and industry-standard interfaces such as width and a Gigabit Ethernet interface ensure rapid 24 V power supply and Power over Ethernet. processing and efficient transmission of image data.

WWW.STEMMER-IMAGING.COM Comparison of the D2 and D3 Intelligent Camera Platform

Intelligent Camera Platform D2 D3 Debian Linux Linux ARM Core and Frequency 300 MHz ARM9® 1 GHz ARM® Cortex™-A8 DSP Core and Frequency 600 MHz C64x™ 700 MHz C674x™ ARM and DSP Instruction Type Fixed point Floating point (FPU) and fixed point

Cortex-A8 NEON™ Architecture – z RAM 256 MB DDR2/333 2 GB DDR3/800 Memory Bus Width, Bandwidth 32 bit, 1332 MB/s 2 x 32 bit, 6103 MB/s Flash Memory 512 MB 32 GB Ethernet Fast Ethernet (100 Mbit/s) Gigabit Ethernet (1 Gbit/s) Typical Ethernet Bandwidth 10 MB/s 40 MB/s

SD Card – z USB Host / Device z / – z / z GPIOs up to 13 up to 44

RS232 / RS485 z / – z / z SATA – z CAN Bus – z JTAG z z Real-time Clock (RTC) – z Maximum Video Resolution 720p 1080p

S-Video z z RGB888 z z HDMI { z Audio in/out { z Wake on LAN – z Power over Ethernet – z I2C – z Watchdog – z 24 V operation – z

WWW.STEMMER-IMAGING.COM

D3 Dual-Core Programming

Intelligent Camera with Ubuntu Linux Operating System

Image Processing on the Dual-Core Chip (ARMv7, Thumb2) and its FPU (NEON). Additional compo- The substantial processing power of the ARM processor nents, such as web servers of NFS servers, can be easily is used to run the Linux OS as well as image processing installed using the Linux package repository. USB devices, tasks. The DSP is completely available for image process- such as Wi-Fi dongles, hard disks, or keyboards can also ing. Both processors have a floating-point unit (FPU). be used with the camera. Hardware as well as various commercial codecs and open source algorithms are available for the DSP, for Convenient Application Programming example the TI IMGLIB for basic image processing tasks. With a standard Ubuntu Linux operating system and full A free by VRmagic converts image data into various access to the ARM/DSP core, the D3 provides a conve- target formats, for example RGB565 or YUV. nient platform for programming applications with /C++ or other programming languages. ARM software can be Extendable Linux Operating System developed in standard integrated development environ- The D3 platform runs a customized Linux kernel. The ments under Ubuntu Linux. After cross-compiling, the root file system is taken from Ubuntu 12.04 LTS, which software can be executed on the camera natively supports the instruction set of the Cortex-A8

Image Sensors

Dual-Core Chip DDR3 RAM

Master Slave 400 MHz (DDR3-800) 2 GB ARM DSPDS C674x 2 x 32 Bit Cortex-A88 System Interconnection 64 Bit 1 GHz 700 MHz Linux Interrupt Request Codecs

Ethernet USB 2.0 Video Audio GPIO ... 1 Gbit/S The D3 intelligent camera architecture

WWW.STEMMER-IMAGING.COM

Applications developed on a x86 PC need to be cross- compiled for the D3 platform. Code for the ARM core can ARM Core DSP Core be cross-compiled using the ARM EABI toolchain. Code Control Linux Codecs for the DSP needs to be cross-compiled using the TI DSP compiler. Both compilers and the required tools are avail- able free of charge. Data Data Data Data

Mono™-Compatible .NET Framework Linux Shared DSP Thanks to the Mono compatible .NET interface available Memory Memory Memory on all VRmagic intelligent camera platforms, applications CMEM development has now become easier. Mono is an open source implementation of Microsoft’s .NET Framework RAM designed to facilitate cross-platform applications. Architecture of the dual-core chip: all memory sections Updates via Ethernet can be resized according to your requirements. Updates of your camera applications and the camera firmware are possible via Ethernet. This saves time and money, especially if you cannot access the cameras di- rectly in the field.

Easy Debugging Simple Data Transfer from and to the D3 • Applications can be developed, tested, and debugged • Create a network share (NFS or Samba/Windows) on on a PC and then transferred to the camera (ARM). your PC and mount it on the camera – or vice versa. • Remote debugging via Ethernet is possible, for ex- •Transfer data via USB flash drive or SD card. ample with Eclipse and gdbserver (ARM). •Use the secure copy (scp) command under Linux. • TI TraceUtil enables printf-debugging of DSP code. •A JTAG port connects real-time emulators (XDS510 or XDS560) and enables DSP debugging with CCS.

WWW.STEMMER-IMAGING.COM Machine Vision SDK and Libraries

Software Development for the D3 – It Doesn’t Get Much Easier

Start Developing in 15 Minutes VRmagic SDK The VRmagic Easy Installer automatically sets up a com- The included VRmagic SDK contains an application pro- plete development environment with all required applica- gramming interface (API), which offers full control over the tions and toolchain elements. After only a few minutes, camera and works with all VRmagic cameras and frame you can compile demos and start developing applications grabbers. This makes it possible to exchange devices with- for the D3 intelligent camera platform. out additional programming costs. The SDK also contains a VRmagic codec for the ARM and DSP, which converts CamLab Application image data into different target formats, for example RGB32, YUV, or GRAY. The CamLab application is part of the VRmagic SDK. It offers user full control over all sensor parameters via a graphical user interface. Tool tips indicate the respective Programming Libraries API property for each parameter. The D3’s floating point unit (FPU) allows a wide range of state-of-the-art machine vision libraries and toolkits to run Free Demos with Source Codes on either the ARM or DSP cores. This lets you take advan- tage of numerous image processing algorithms. For exx- The following demos and sources are included in the SDK: ample, develop a HALCON script on your PC and execute Bayer/Grey to RGB565 converter on ARM or DSP, demo it directly on the D3 with HALCON Embedded – develop- for integration of customer specific DSP codecs (JPEG, ment for intelligent cameras has never been easier. Sobel, and more), viewer for S-Video output (DirectFB and SDL), control of general purpose I/Os.

x86 Host ARM Target

Source Binary Code ARM

GCC ARM

Development process for the ARM: code Header (.h) Runtime is developed on an x86 PC using specific Dev Libs ARM development libraries. After cross-compiling Libs ARM Versions must match the code, the binary can run on the D3’s ARM using the corresponding runtime libraries.

WWW.STEMMER-IMAGING.COM Libraries and Toolkits for ARM and DSP

Programming Libraries for ARM Programming Libraries for DSP The following libraries and applications create code or script Programming for the DSP requires advanced program- that executes on the ARM processor of the D3. These ming skills, however, making use of it boosts the speed of tools cover a broad spectrum of project requirements, your applications considerably. The following libraries take from simple to complex, and offer solutions for the full advantage of the DSP on the D3. range of applications in the field of machine vision. • TI Codecs: Audio and video codecs optimized for the • Common Vision Blox Embedded: A fast and DSP, for example H.264, MPEG-2, or JPEG. powerful programming for vision applications. • TI IMGLIB: A collection of more than 70 building block • EyeVision: Software with a graphical user interface for kernels optimized for the DSP that can be used for intuitive development of image processing applications. image and video processing applications. • GStreamer: for playback, • TI VLIB: Video analytics and vision library for video streaming and processing of video and audio. surveillance, automotive vision, consumer electronics, • HALCON Embedded: A leading machine vision library and many more. with more than 1800 operators. • VM_LIB: VRmagic machine vision library for object • OpenCV: An open source computer vision and recognition, blob segmentation, barcode readers, and machine learning software library. many more. Optimized for the DSP, but also runs on the ARM and on Linux or Mircosoft® Windows® PCs. • ®: A cross-platform application and UI framework for developers using C++ or QML.

the power of machine vision

Blob segmentation, one of the machine vision algorithms included in the VM_LIB.

WWW.STEMMER-IMAGING.COM Camera Designs and Image Sensors

Multi-Sensor Camera Up to four external plug and play sensor boards may be connected to the camera base unit using flex-foil cables. This design enables pixel-synchronous images from multiple positions and facilitates 3D applications.

Remote-Sensor Camera A single sensor board is connected to the camera base unit using a flex-foil cable or a round cable, making the camera perfect for narrow installation spaces.

Single-Sensor Camera The sensor board is mounted directly onto the camera base unit, forming one compact unit. This is the most robust and cost-efficient camera design.

Analog Video Converter Analog video converters convert PAL or NTSC cameras into a digital camera with the performance of the D3. This allows a -by-step upgrade from analog to digital cameras.

Left: D3 remote-sensor camera Opposite page: D3 multi-sensor camera

WWW.STEMMER-IMAGING.COM Available Image Sensors

Sensor ID 8 9 12 14 Sensor Type Aptina MT9T031 Aptina MT9M001 Aptina MT9V024 Sony ICX445 Technology CMOS rolling shutter CMOS rolling shutter CMOS global shutter CCD interline transfer

Color z – zz

Monochrome – zzz Sensor Size [inch] 1/2 1/2 1/3 wide 1/3 Resolution [px] 2056 x 1544 1288 x 1032 754 x 480 1296 x 966 Pixel Size [μm] 3.2 x 3.2 5.2 x 5.2 6 x 6 3.75 x 3.75 Bit Depth [bit] 8/10 8/10 8/10 8/10 Available Camera Design Single Remote – Single Remote Multi Single Remote Multi Single Remote Multi Min. Exposure Time [μs] 60 60 – 38 38 38 30 30 30 15 15 15 Pixel Clock [MHz] 5...48 10...48 – 5...48 10...48 26.6 13...27 13...27 26.6 36 36 26.6 Max. Frame Rate [Hz] * 13 13 – 30 30 16 70 70 69 22 22 16

Sensor ID 18 20 22 42 Sensor Manufacturer Aptina MT9M024 Aptina AR0134 CMOSIS CMV2000 CMOSIS CMV4000 Technology CMOS rolling shutter CMOS global shutter CMOS global shutter CMOS global shutter

Color zzzz

Monochrome zzzz Sensor Size [inch] 1/3 1/3 2/3 ultra wide 1 Resolution [px] 1280 x 960 1280 x 960 2048 x 1088 2048 x 2048 Pixel Size [μm] 3.75 x 3.75 3.75 x 3.75 5.5 x 5.5 5.5 x 5.5 Bit Depth [bit] 8/10 8/10 8/10 8/10 Available Camera Design Single Remote Multi Single Remote Multi Single – – Single – – Min. Exposure Time [μs] 22 103 187 56 86.75 156.6 44.3 – – 44.3 – – Pixel Clock [MHz] 74.25 48 26.6 74.25 48 26.6 4 x 25 – – 4 x 25 – – Max. Frame Rate [Hz] * 45 29 16 45 29 16 44 ** – – 24 ** – –

* This is the maximum value at full ROI with minimum exposure time. The actual frame rate depends on the pixel clock, sensor settings, and image format. ** With RLE image format

WWW.STEMMER-IMAGING.COM 24 V Operation JTAG The D3 intelligent camera can be operated J A JTAG interface makes it possible to con- with 24 V DC, which allows easy integration TAG nect real-time emulators (e.g. XDS510) for into industry applications. debugging purposes.

Audio In/Out Power over Ethernet The D3 has various interfaces for audio in- Power over Ethernet allows the camera to put and output: S/PDIF digital out (coaxial/ obtain its power supply over the Ethernet optical), line out, line in, and microphone in. PoE cable, saving you time and money.

CAN Bus Real-Time Clock CAN bus is a multi-master broadcast se- The real-time clock keeps track of the cur- rial bus standard designed for automotive rent date and time while the camera is applications. switched off.

Gigabit Ethernet RGB888 The Gigabit Ethernet interface offers high The RGB888 interface outputs image data GigE data transfer rates, enabling extra fast in parallel 24 bit format. streaming of images.

General Purpose I/Os RS-232 Each general purpose I/O can be configured The UART allows serial communication via to be an input or output. Up to 44 GPIOs two ports: Serial0 for RS232 / console and are available for your application. Serial1 for RS232 / RS485.

HDMI SATA The HDMI output transfers uncompressed The Serial ATA bus interface allows fast digital video data to any compatible moni- connection of mass storage devices such tor, projector, or TV. as hard disks and optical drives.

I2C Bus SD Card Reader The I²C bus is a two-wire serial interface, The SD card interface enables writing and which can be used to connect peripherals reading of SD cards. The VRmEIO3 inter- to the D3 platform. face board features a microSD card slot.

WWW.STEMMER-IMAGING.COM Built-In Flexibility D3 Interface Boards

More Than 20 Interfaces

SPI Bus Choose between a multitude of interfaces on different in- The SPI bus is a synchronous serial data link terface boards to get the most out of your application. Or standard. Devices communicate in master/ order a custom interface board for production quantities. slave mode. VRmEIO3 Status LEDs The VRmEIO3 interface board features all interfaces sup- The status LED indicates the camera sys- ported by the D3. Industry standard connectors and the tem status, for example if the camera is large euroboard form factor allow convenient evaluation LED booting up or ready for operation. of every interface.

S-Video VRmCUEO3 and VRmCUEOS3 OEM SVideo The analog S-Video output transfers stan- The compact OEM interface boards are equipped with dard-definition video signals encoded on miniature connectors, thus making them perfectly suit- two channels. able for OEM applications. VRmCUEO3 features all D3 interfaces except SATA and JTAG. VRmCUEOS3 is identi- cal apart from an additional SATA interface. Trigger and Strobe The camera can be controlled by an external T/S trigger (trigger in) and output strobe signals D3 Industrial Camera IC3 OEM (strobe out). The IC3 OEM has an interface board that is optimized for industry applications and features a 24 V power supply, High-Speed USB Power over Ethernet, and standard industry connectors. The evaluation board features two high- HI-SPEED speed USB ports. Both ports can be con- figured separately as USB device or host.

Watchdog The watchdog timer triggers a certain ac- tion, for example a processor reset, if an- other component fails.

All of the listed interfaces are supported by the D3, however, some restrictions may apply. Please contact us for details. Opposite page: VRmEIO3 interface board. Right: VRmCUEO3 interface board.

WWW.STEMMER-IMAGING.COM Three Steps to Your Individual D3 Camera

1 Choose Design 2 Choose Sensor 3 Choose Interfaces

APTINA evaluation Multi-sensor camera Aptina sensors Evaluation interface board

CMOSIS oem Remote-sensor camera CMOSIS sensors OEM interface board

SONY industrial his document. Single-sensor camera Sony sensors Industrial interfaces

custom Frame grabber Custom interface board technical change without notice. No liability is accepted for errors which may be contained in t be contained which may errors technical change without notice. No liability is accepted for K-VRM16-03/2015 ∙ Subject to K-VRM16-03/2015

Imaging is our passion. STEMMER IMAGING Head Office Gutenbergstraße 9 - 13, D-82178 Puchheim www.stemmer-imaging.com Phone: +49 89 80902-0, [email protected]