Ladybug Player Vista Thursday, April 05, 2007

Ladybug Player Vista

Version 2.2.0

Table of Contents

Chapter 1. Introduction

Technologies …………………………………………………… Features ………………………………………………………….

Chapter 2. Architecture

Diagram …..……………………………………………………… Flat GUI ……...…………………………………………………… Customization …………………………………………………... Keyboard Control ………………………………………………

Chapter 3. Technologies

High-Definition Video …………………………………………. Multi Channel Video Mixing ………………………………….. HLSL Shader ……………………….…………………………… FX Effects ……………………………..………………………… Effects in Ladybug Player Vista ………..………..…………..

Chapter 4. Applications

Digital Signage …………………………………………………. Surveillance …………………………………………………….. VJ Control ……………………………………………………….. Video Communication ………………………………………… Video Remix …………………………………………………….. Mobile Video …………………………………………………….

Appendix

FAQ ……………………………………………………………….. History …………………………………………………………… System Requirements …………………………………………

Chapter 1 Introduction

Ladybug Player Vista is a next-generation media player that can do real-time video mixing and video special effects for standard and high-definition videos. Ladybug Player Vista is suitable for applications such as Digital Signage , Video Surveillance , and Videoconference .

Ladybug Player Vista applied Neatware's real-time mChannel Technology, which includes multithread media engine naturally for multi-core processors, programmable HLSL ( High Level Shading Language ) shader for GPU (Graphics Processing Unit) pipeline, and media codec for MP4 HD , WMV HD , AVI , and DVD/VOB videos. It also supports mobile media applications along with various USB storage devices.

Ladybug Player Vista is simple, compact, and terrific. Based on many years design experience of the popular Ladybug Player, Neatware developed a new generation media player - Ladybug Player Vista - an integration of innovations in media platform, media engine, and media foundation. Ladybug Player Vista explored both the new programmable shader of GPU and the high performance of multi-core CPU processor for advanced media applications.

• The Media Platform of Ladybug Player Vista supports latest DirectX 9.0+

technologies in Windows XP and Vista.

• The Media Engine of Ladybug Player Vista supports video codec, HLSL shader for special visual effects, and multi-core processors.

• The Media Foundation of Ladybug Player Vista works for media control, media management, and media storage.

Ladybug Player Vista consists of Professional Edition, Developer Edition, and Extreme Edition . The Professional Edition is for everybody in video applications. In contrast, the Developer Edition is for deverloers who want do design their own HLSL effects to special video applications such as medical video processing. Finally, the Extreme Edition will explore the hardware capability and performance in extreme for video processing. It includes all the capability of Professional and Developer Editions.

Technologies

Ladybug Player Vista is the first video player that can display quad videos in real-time with visual special effects by GPU in the world. Neatware’s mChannel technology was applied in the Ladybug Player Vista, which implemented real-time video mixing by multi-core CPU along with super fast GPU.

Copyright ©2006-2007 Neatware. All rights reserved. www.neatware.com Ladybug Player Vista Thursday, April 05, 2007 mChannle allows multiple video streams to flow into a multithread pipeline for each channel. Each pipeline applied video splitter and decoders to split video and audio stream and further uncompress the media. Each uncompressed video streams are rendered and mixed in the GPU through a programmable shader. The final result is sent to frame buffer of the graphics card.

In addition to the mChannel technology, the design of Flat, Transparent, and Instant GUI in Ladybug Player Vista makes the use of player much easy.

Features

1. Cutting-edge technologies and new design

2. Small, fast, powerful multithread video engine

3. Real-time FX effects on GPU

4. Rich programmable shader with HLSL language

5. Quad channel video streams for video composition and post-processing

6. Solo, duo, trio, quad zones for independent video display

7. Flat GUI for instant control and ticker

8. Customized player supports multiple windows and screens

9. International language supports

10. Recognize many media formats: WMV, AVI, MPEG, JPEG and HD videos

1. Cutting-edge technologies and new design

Cutting-edge technologies inside Ladybug Player Vista includes multithread video engine, multiple channel video mixing, and special visual effects by GPU. Because of the streaming and pipeline architecture of video processing, a GPU can be 50x faster than a CPU. Ladybug Player Vista applied the high performance of GPU to achieve real-time effect.

2. Small, fast, powerful multithread video engine

The multithread video engine of Ladybug Player Vista can recognize many video formats such as AVI, MPEG, WMV video, JPEG picture, and MP3, WMA, WAV audio. Multi-Core processors can speed up the execution of multithread video engine naturally. There are quad channel in the Ladybug Player Vista. Each channel may need to create threads up to seven. As quad channels are independent in decoding, Quad Core processor is better for Ladybug Player Vista.

3. Real-time FX effects on GPU

Quad video streams can be mixed and processed by a HLSL shader on GPU in real-time. You can switch to a new shader instantly by moving an arrow key or click mouse button. The FX effects may be wipe, fade, zoning, mask, .etc. The effect of black and white video is simple a trivial.

4. Rich programmable shader with HLSL language

Programmable shader is the revolution of GPU. Ladybug Player Vista supports the real- time video shader in advance via FX effects that may includes vertex shader and pixel shader. Shader Model 3.0/2.0x/2.0 , High Level Shader Language ( HLSL ), Assembly, FX Effects are all supported in the Ladybug Player Vista along with DirectX9.0+.

5. Quad channel video streams for video composition and post-processing

Ladybug Player Vista can receive and process four video streams. All of them can be composted on GPU in real-time with special effects. New multiple GPU technology such as SLI and Crossfire can allow more and high resolution video streams to be rendered simultaneously.

6. Solo, duo, trio, quad zones for independent video display

Neatware invented a new technology to display multiple video streams in zoning. So we can display up to four video streams in a quad of window. This function can help users and developers to composite media contents easy for the applications as Digital Signage and Security.

7. Flat GUI for instant control and ticker

All the controls of media playback and settings are showed over the video surface. Users can select shader, play or pause videos, and adjust the volume of sound on each channel. Users can also input a text string as a ticker, which can be displayed while video is

8. Customized player supports multiple windows and screens

Neatware Media File (. NWM) is a new file type, which stores the configuration of current Ladybug Player Vista. Users can apply command line to control the Ladybug Player Vista.

9. International language supports

International languages are supported to help various users in the world. The supported languages include English, French, Chinese, Japanese, Korean, Dutch, Germany, Italian, Spanish, and Portuguese. The snapshot below shows that multiple language characters are displayed in the ticker.

10. Recognize many media formats: WMV, AVI, MPEG, JPEG and HD videos

Ladybug Player Vista supports codec that is compatible with DirectShow in Windows. It also supports super high-resolution images up to 8192x8192 (64MPixel) pixels.

Chapter 2 Architecture

The architecture of Ladybug Player Vista consists of source, processing, and presentation. Below is the diagram of the architecture.

Diagram

In the source part, there are various media types and formats as well as multi-channels. Ladybug Player Vista.0 recognize standard video (SD) and high-definition video (HD) files. JPEG picture can also be a source file. In addition, ticker, a scroll text can be inputted instantly. The current available media types are tagged as purple color in the diagram. Other media sources such as audio, camera, tv, dvd player, and model will be added in the future versions.

Ladybug Player Vista recognizes video formats such as WMV, AVI, MPEG, and others. It can accept four channels simultaneously.

The pink blocks in the middle are processing units of Ladybug Player Vista. There are Multithread Engine; Real-time, Programmable, Instant FX Effects; and Unicode Support.

These green blocks are belonging to presentation. It can support multiple zones, multiple windows, multiple screens, as well as wireless monitors.

Basically the GUI of Ladybug Player Vista is flat. The use of controls is simple and obvious. The GUI is over the video background. Users can control video playback while video are playing.

The top-left area shows the help and output message. On the top-right area, there are toggled fullscreen button, save as .nwm button, help button, as well as language selection combo box. In the middle, there are shader list box, video playback controls, and input of files for quad channels. At the bottom, there is a ticker show/hide check button and a text input box for ticker.

At the bottom is the customized database that stores the configuration data and current states.

At the top is the Flat, Transparent, and Instant GUI module. We are going to discuss it in the following section.

Flat GUI

Below is a snapshot of Ladybug Player Vista GUI. It is a flat, transparent, and instant GUI. Everything was designed to response instantly.

Below is the snapshot in more detail of Ladybug Player Vista.

Vsync on (800x640 ) specifies that the refresh rate is synchronization with the monitor. The video resolution is 800x640. X8R8G8B8 is the format of video buffer. In the next line, HAL (hw vp) specifies that it is a hardware acceleration. ALL-IN-WONDER 9600 SERIES is the name of graphics card. Furthermore, FPS (Frame Per Second) is an indicator of frame update speed.

A help message said that you can press ESC key to exit the player and you can click right mouse button to hide and show whole control. The next line of red text is the output message.

Shaders are listed in a list box in the order of Solo, Duo, Trio, and Quad Channel. You can select an item by mouse clicking and scroll list up or down by mouse wheel.

Tips : you can pick the shader list and hide the control, then you can press the up and down arrow key to shuttle shader instantly. You will be surprised that the effects are

You can click the Play all/Pause all button to start or pause the video playback for all channels. In the similar way, you can click Stop all button to stop all the video playback, rewind and replay from the original position.

CH1 to CH3 is the indication of channel. Each channel has a Play/Pause button, a Stop button, a Loop check, a Volume scale, and a Progress bar. You can move the indicator of the progress bar and stop the player to better snapshot a picture.

Below is more GUI controls. Each channel has an Mute check along with a timer indicator. The time is displayed in the format mm:ss.aa where mm is the minutes, ss is the seconds, and aa is the micro-seconds. The current/duration displayed the current time that a video is played and the duration is the length of a video.

Note: The duration of a JPEG picture is 0.

Toggle Fullscreen allows you to switch the mode between fullscreen and normal window.

You can click the Save as (F11) button to launch a Save File Dialog, then you can save current configuration of Ladybug Player Vista into a Neat Ware Media file .nwm. Later you can launch Ladybug Player Vista with .nwm file in your saving state.

By clicking File item in the drop-down list box you can add a media file into a channel. File Open Dialog Box is displayed after you select the File item. You can also change the folder by clicking the drop-down list button on the top of the dialog box. To select a file you can click the file name in the list box and press the Open button.

Ticker allows you to use dynamic text in animation while playing video. Following snapshot shows the input box of the ticker. You can insert or delete text in the box. The check button on the left allows you to hide or show a ticker when you hide the control.

In the following snapshot, a video and a product picture are displayed on the top zones while a ticker text is displayed at the bottom.

Customization

You can customize player and store the configuration in an .nwm file. Later you can double click the .nwm or launch it in a command line in the following format:

"FullPath \Ladybug Player Vista.exe " [ FullPath\your.nwm] [ -fullscreen ]

If there is no your.nwm specification Ladybug Player Vista will be launched along with the default nwm file stored in the data subdirectory of the installation. The parameter -fullscreen specifies that the player will be launched in fullscreen.

File association allows a media type such as .avi associate to Ladybug Player Vista , so that you can click a media file and launch the player.

NWM Database Manager

There are lots of free and commercial SQLite database managers available. SQLite2007 Pro Enterprise Manager is one of a freeware that can help you to manage NWM File, which is an SQLite database.

Users of Developer Edition can modify the VideoMixerDev.fx file, which is located in the folder of […]\device\effect\

NWM SQL Database Tables

NWM, Neat Ware Media file consists of several SQL tables.

CREATE TABLE tb_control( key INTEGER PRIMAY KEY, channel INTEGER, play INTEGER, stop INTEGER, loop INTEGER, volume INTEGER, mute INTEGER, curfile INTEGER, curpos INTEGER, name TEXT, path TEXT ) tb_control is a table for media playback control of all the channels.

CREATE TABLE tb_logger( logKey INTEGER, key INTEGER, channel INTEGER, play INTEGER, stop INTEGER, loop INTEGER, volume INTEGER, mute INTEGER, curfile INTEGER, curpos REAL, name TEXT, path TEXT, sqlAction VARCHAR(32), DateEnter DATE, TimeEnter TIME ) tb_logger records the date and time of the upgrade in database.

CREATE TABLE tb_language( key INTEGER PRIMARY KEY, name TEXT, selection INTEGER ) tb_language is the table for language selection.

CREATE TABLE tb_others( key INTEGER PRIMARY KEY, visible INTEGER, fullscreen INTEGER, orgx INTEGER, orgy INTEGER, width INTEGER, height INTEGER, title TEXT, REF INTEGER, ChangeDevice INTEGER ) tb_others is the table for window's control.

CREATE TABLE tb_shader( key INTEGER PRIMARY KEY, list_name TEXT, shader_name TEXT, selection INTEGER ) tb_shader is the table for shader control. For Ladybug Player Vista Developer users, you can insert, upgrade, or remove an item of shader in the list. When you designed a new HLSL shader in the .fx file, you need to add a new item with the corresponding shader_name in the .fx file. The list_name is for the display in the shader list box.

CREATE TABLE tb_ticker( key INTEGER PRIMARY KEY, visible INTEGER, script TEXT ) tb_ticker is the table for ticker processing.

Keyboard Control

Ladybug Player Vista also provides keyboard control for video playback. In default, F1 is for help, F2 for play/pause all, F3 for stop all, F4 for ticker show/hide, F11 for .nwm file saving. You can press ESC key to exit program. You can also switch to and back from Fullscreen by pressing ALT + Enter key.

Performance

Multi-Core processors can speed up multithread engine in Ladybug Player Vista. The 64- bit processor is also helpful for fast transmit media from hard disk. Below is a CPU usage snapshot.

Chapter 3 Technologies

High-Definition Video

The resolution of a video can be defined as Low Definition ( LD 240p ), Standard Definition ( SD 480p ), Normal High-Definition ( HD 720p ), Full High-Definition ( HD 1080p ), and Super High-Definition ( SHD 2060p ). Usually, the number of visible (scan) lines, that is the height of a digital picture, is the index of definition.

The resolution of Low Definition ( LD ) is from 128p to 240p where p represents the progressive scanning. In contrast, the low case letter i represents the interlace scanning. LD display usually applied in mobile phones and handheld devices.

The resolution of Standard Definition ( SD ) is from 320p to 480p. Most of TVs and DVD are belong to the SD resolution. Normally, the ratio aspect of SD Video is 4:3.

The snapshot shows the wipe around a circle with two videos.

The resolution of Normal High Definition (HD ) is defined as 1080/i and 720p. The resolution of Full High-Definition is defined as 1080p. The ratio aspect of HD Video is 16:9.

The resolution of Super High-Definition ( SHD ) is beyond 1080p, which can be 1440p, 1800p, 2160p, 2520p, and 2880p. So far there is no standard for SHD.

Today HD technologies have made HD media affordable and available.

• HD Camera

HD digital video camera can shot in 16:9 aspects rate directly.

• HD Display

Large and wide screen monitor from LCD, Plasma, and Digital TV technology.

• HD Processor

Low power and fast processor can be the codec to encode and decode digital video such as MP4.

• HD Sound

Stereo sound evolutes from 2.1 to 7.1 . The MP3 has been widely used to replace

the CD quality music. New Dolby and DTS have been selected to be the HD sound.

• HD Storage

LD media can been stored in CDROM, CDR, or CDRW. Its capacity is 680MB. SD media such as DVD are stored in the DVD-ROM, DVD-R, or DVD-RW. Its capacity is 2GB or 4GB. New generation HD media will be stored in the Blue Ray or HD DVD. Its capacity is 20GB.

• HD Network

HD video can be transmitted through broadband or 3G wireless network in download or streaming method.

• HD TV

High-Definition TV can be transmitted by cable or digital satellites.

• HD Player

Many devices such as DVD, PC, Console, Set-top box can play HD videos.

There are many formats for High-Definition video. Microsoft’s WMV HD is one of them. However, most of standards are based on MPEG standard.

MPEG means Moving Picture Experts Group. This group is working under the direction of International Standards Organization ( ISO ) for the standards of compressed motion pictures and associated audio.

MPEG-1 had been completed on October 1992 known as International Standard ISO-11172. Video CD (VCD) adapted MPEG-1 as its video standard for CD-ROM media. VCD's audio adapted MP2 audio. The normal MPEG-1 video resolution is 352x240 or 240p .

DVD adapted the MPEG-2 as video standard and AC3 as audio standard. The normal MPEG-2 video resolution is 704x480 or 480p . Usually the movie data of DVD is stored in the .vob file.

MPEG-4 or MP4 is the latest MPEG standard for low-bit-rate streaming video. It is a standard that is suitable for wireless multimedia and broadband applications. Its resolution can be 480p . Compare to DVD video, MP4 requires smaller space with the same picture quality.

MP4 HD is a standard to apply MPEG-4 video on High Definition video. Its resolution is defined as 720p or 1080p .

High-Definition video challenges hardware and software on both speed and storage.

Multiple Channel Video Mixing

Today most of real-time video mixing is done on special hardware machines. Those machines are expensive and nonflexible. Unfortunately current general purpose Central Processing Unit ( CPU ) is not fast enough to complete multiple video composition in real-time.

The rapid development of the Graphics Processing Unit ( GPU ) technology had made the fastest GPU to be 10 or 50 times faster than the fastest CPU. The power of GPU with its parallel streaming pipelines makes real-time video mixing on GPU possible.

In theory CPU is good at procedure control and GPU is good at stream computing. Ladybug Player Vista , powered by its multithread media engine and GPU shader can implement multiple channel video composition in real-time.

The media engine of Ladybug Player Vista is benefit from modern technologies of CPU, GPU, and Chipset. The dual-core CPU from Intel and AMD will speed up the processing of multiple video streams in codec. Multi-core processor can speed up the multithread codec in Ladybug Player Vista naturally.

Furthermore, the real-time video effects can be completed on GPU by the shader library of Ladybug Player Vista.

Finally, multiple GPU processors as SLI/Crossfire can greatly improve the performance

Copyright ©2006-2007 Neatware. All rights reserved. www.neatware.com Ladybug Player Vista Thursday, April 05, 2007 of video mixing on HD Media. The fast data channels of DRAM, PCIe Bus, and SATA hard disk are also speed up video mixing.

Video mixing , sometimes called cinematic composition , is to mix multiple video streams, do processing, and generate an output. Video Mixing can be represented as a multiple variable function, R = S ( C0 , C 1 , ..., C n-1) where Ci is the Channel i of video stream, S is the Shader, and R is the Render. In GPU processing, a video stream is set as a texture.

Now let’s look at the video mixing by the number of channels.

• Solo Channel

Similar to image processing, only one single video stream as input can do transform based on each video frame. There are many effects such as black and white, negative, and edge.

Negative is the reverse of the input video colors. Black and White maps color to gray levels. There are Sepia Tone, Edge, Bilinear Blur, Gaussian Blur, Median, and Color Swap

• Duo Channel

Video mixing from two or more channels is often called video composition . Usually, a video channel is treated as foreground video and another video channel is considered as the background video. We are going to call them foreground channel and background channel respectively. A special case is that the background video can be a picture.

Chromic Key Effect

The famous chromic key processing in the traditional video mixing can be easily implemented in a GPU shader.

Suppose the foreground video such as an actor is shot on a background of green screen, and the background video is the scene of a beach. So the value of chromic key is the green color. Duo channels can be composted in the following way: open foreground channel when its pixel value is not equal to the value of chromic key and open background channel otherwise. This concept can be represented as a C conditional expression. result = (color0 == key) ? color1 : color0;

Fade Effect

Another composition is called fade . In this case a control variable alpha in [0.0, 1.0] is applied to generate a linear combination of two channels. Fade is the sum of alpha multiples the pixel value of first channel and 1.0-alpha multiples the pixel value of second channel.

The HLSL code of fade is represented below. color0 and color1 are the pixel values of channels respectively. And the alpha is the transparency value. result = alpha * color0 + ( 1.0 – alpha ) * color1;

Wipe Effect

In addition, the wipe composition is widely used in movie effects. It is based on the position of pixel of a video frame. Wipe transform sweeps frame B over the frame A from left to right.

There are three rectangle regions: frame B only on the left, gradient region on the middle, and frame A only on the right. The gradient region is the rectangle mixing between frame A and B. The boundary between frame A region and the gradient region is called the leading edge . Similarly, the boundary between the frame B region and the gradient region is called the trailing edge .

Finally, the value of alpha can be the function of time. For example, a sin (time) alpha value can display two channels alternatively and smoothly from one channel fade in and fade out to another channel. The HLSL sample code is alpha = saturate( sin( fTime ) + 1 ) / 2.0; result = alpha * color0 + ( 1.0 – alpha ) * color1;

Frame Effect

A headline video is mixed with an illustration video by wipe effect in special position. The headline video can be in the center of the display while the illustration video can be displayed around the headline video like a frame. A signature can be overlay on the bottom of the headline video as a watermark. The videos can be replaced as pictures.

In this example, the headline picture is the large font text of Neatware Digital Signage Solution. The illustration is a high resolution video. The FX effect is circle wipe transform.

Curtain Effect

Curtain effect is the mixing of a picture and a video or another picture while the first picture moves like a curtain to cover the second media gradually. Curtain effect makes a sense of mysterious to audiences who may be attracted by the disclosure of content gradually. And audiences may pay longer time to watch the video.

The procedure to generate a curtain ads is simple in Ladybug Player Vista:

1) Geneate curtain media

Curtain media can be a picture or a video. Usually a picture is used to show a logo or a promotion text, while a video is a decoration to ads.

2) Generate main media

Execute Ladybug Player Vista 2

3) Select curtain and main media

The curtain video is disclosed from left to right. In the end the curtain video disclosed from right to left.

• Trio Channel

A well known example of trio channel is the mask transform. A mask is a black and white image. Two video channels and a mask picture channel are the inputs. The effect is the sum of the first channel multiples the mask and the second channel multiples the negative of mask. Therefore, two channels are outputted in the areas distinguished by the shape of the mask. The HLSL sample code is simple. result = mask * color0 + ( 1.0- mask) * color1;

• Multiple Channels

More video mixing can be considered as multiple layer composition. The lighting and motion effects can make a common video uncommon. For example, a channel that shows a light source from left to right and another video that shows another light source from right to left can be composted along with a channel to generate dynamic lighting effects.

This technology has been widely used in advertising and movie trailers.

HLSL Shader

HLSL language can help you to write a shader easily. Some media players lack of shader support or use a plug-in running on CPU for video mixing. Compare to HLSL just-in time compiler inside the Ladybug Player Vista, a plug-in solution for video mixing is hard to build because you may be required to write C programs for CPU.

High Level Shading Language ( HLSL ), a programming language for Graphic Processing Unit ( GPU ) in DirectX 9.0 and plus, supports the shader construction with C-like syntax, types, expressions, statements, and functions.

Long time ago, Apple's RenderMan was a popular shading language that was used to generate cinematic effects with CPU in render farms. Lately, Microsoft's High-Level Shading Languages ( HLSL ) and OpenGL's OpenGL Shading Language ( GLSL ) shading languages have been developed for real-time shader on GPU. Best integrated into the DirectX 9, HLSL works solely on Windows platform. Similarly, OpenGL 1.5 starts to include OpenGL shading language GLSL as a standard component. These high level languages accelerated the shader development.

To build a complete shader, new shading languages for GPU must work along with a host programming language such as C++. Other languages can also be used to build shader. However, although the procedure is tedious to set large amount of parameters, C++ is fastest. Ladybug Player Vista is a video player that supports DirectX 9.0+, HLSL, and Shader Model 3.0. Developers can design their own HLSL pixel shader for real-time video post-processing with Ladybug Player Vista.

The Simplest Example

A shader consists of vertex shader and pixel shader. The stream of 3D model flows from application to the vertex shader, then to the pixel shader, finally to the frame buffer. a2v struct represents the data structure transferred from an application to a vertex shader, v2p from a vertex shader to a pixel shader, and p2f from a pixel shader to the frame buffer. Below program transforms a vertex's position into the position of clip space by view matrix.

struct a2v { float4 Position : POSITION; };

Inside the HLSL function, struct a2v specifies a vertex structure that represents the data of vertices.

struct v2p { float4 Position : POSITION; }; struct a2v specifies a stream structure from vertex to pixel shader. Position is a four dimensional vector declared by float4. Furthermore, POSITION, called output semantic, indicates the initialized type of Position.

void main( in a2v IN, out v2p OUT, uniform float4x4 ModelViewMatrix ) { OUT.Position = mul( IN.Position, ModelViewMatrix ); }

main is a vertex shader function name. void means that this function will return nothing. The input parameter IN is specified by in modifier and struct a2v. Similarly, the output parameter OUT is specified by out modifier with the type v2p. In addition, float4x4 declares a matrix ModelViewMatrix. The uniform modifier indicates that the value of the matrix is a constant assigned by external program.

Finally, this simplest vertex shader outputs the multiplication of the vector Position and the matrix ModelViewMatrix. While IN.Position is the left parameter of mul, it is considered as a row vector. Otherwise it is considered as a column vector.

HLSL provides scalar data type like float and vector data type like float3. The scalar data types include bool with true or false value, int with 32-bit signed integer value, half with 16-bit floating point value, float with 32-bit floating point value, and double with 64-bit floating point value. A shader emulation will work while a GPU does not support a data type.

A vector data type is declared as vector where size is the dimension and type is the scalar component type. vector is a declaration of 4 dimensional float vector. Usually, we use float2 , float3 , and float4 for two, three, and four dimensional float vectors. float4x4 declares a float matrix type with 4 rows and 4 cols. A general matrix declaration has the form matrix . A variety of dimensional declaration such as float3x4 is also acceptable. To access an element of matrix you can use m[i][j] or zero-based row-column position like _m00 as well as one-based row-column position like _11 .

You can think HLSL as a C language for GPU programming except there are no pointer, union, bitwise operations, and function variables. There are no goto, switch, recursive function in HLSL as well. However HLSL adds vector data type, build-in constructor, swizzling and masking operators. HLSL standard library includes mathematical functions and texture processing functions. The function overloading has been used to unify the operations of different vectors.

vs_1_1 dcl_position v0 m4x4 oPos, v0, c0 mov oD0, c4

This asm program is the compiled code of the simplest HLSL example. First, vs_1_1 specifies the version of vertex shader as 1.1. Second, dcl_position v0 declares that v0 is a position register. The third statement declares a matrix multiply with source variable register v0 and constant register c0 where oPos represents the destination position register. Finally, the last statement moves the value of register c4 to register oD0. Usually, vN represents input register and oXXX represents output register in the assembly vertex shader language.

Add Diffuse Color

In this example we add COLOR component as the diffuse color.

float4x4 ModelViewMatrix;

ModelViewMatrix is declared as a global variable. Its value is assigned by the host program running on CPU. By using global variable declaration instead of the uniform declaration, the HLSL code may become more understandable.

struct a2v { float4 Position : POSITION; float4 Color : COLOR0; };

a2v structure declares the data structure transferred from application vertex shader. Note POSITION and COLOR0 are input semantics that link the vertex buffer of the application to the vertex shader. The input semantics of vertex shader include POSITIONn for Position, BLENDWEIGHTn for Blend weights, BLENDINDICESn for Blend indices, NORMALn for Normal vector, PSIZEn for Point size, COLORn for Color, TEXCOORDn for Texture coordinates, TANGENTn for Tangent, BINORMALn for Binormal, and TESSFACTORn for Tessellation factor.

struct v2p { float4 Position : POSITION; float4 Color : COLOR0; };

v2p structure declares the data structure transferred from vertex shader to pixel shader. Vertex shader output semantics include POSITION for Position, PSIZE for Point size, FOG for Vertex fog, COLORn for Color, and TEXCOORDn for texture coordinates.

void main( in a2v IN, out v2p OUT ) { OUT.Position = mul(IN.Position, ModelViewMatrix); OUT.Color = IN.Color; }

In this example Color component specifies the diffuse color from application. It simply copies the Color from the IN to OUT in the main function.

Diffuse and Specular

This is a little bit more complicated example shown the implementation of diffuse and specular color.

float4x4 ModelViewProj; float4x4 ModelViewIT; float4 LightVec;

These are global variables.

struct a2v { float4 Position : POSITION; float4 Normal : NORMAL; };

Normal item is added for color computing.

struct v2p { float4 Position : POSITION; float4 Color : COLOR0; }; color outputs to pixel shader.

void main( in a2v IN, out v2p OUT ) { input parameters include view project matrix ModelViewProj, view inverse transpose matrix ModelViewIT, and light vector LightVec.

OUT.Position = mul( IN.Position, ModelViewProj ); multiply position with view project matrix

float4 normal = mul( IN.Normal, ModelViewIT ); normal.w = 0.0; normal = normalize( normal ); float4 light = normalize( LightVec ); float4 eye = float4( 1.0, 1.0, 1.0, 0.0 ); float4 vhalf = normalize( light + eye ); transform normal from model-space to view-space, store normalized light vector, and calculate half angle vector. float4 (1.0, 1.0, 1.0, 0.0) is a vector constructor to initialize vector float4 eye.

.xyzz, a swizzle operator, sets the last component w as the z value.

float diffuse = dot( normal, light ); float specular = dot( normal, vhalf ); specular = pow( specular, 32 ); calculate diffuse and specular components with dot product and pow function.

float4 diffuseMaterial = float4( 0.5, 0.5, 1.0, 1.0 ); float4 specularMaterial = float4( 0.5, 0.5, 1.0, 1.0 ); set diffuse and specular material.

OUT.Color = diffuse * diffuseMaterial + specular * specularMaterial; } add diffuse and specular components and output final vertex color.

To better understand a swizzle operator, readers can compare cross definition

float3 cross( float3 a, float3 b ) { return float3( a.y*b.z-a.z*b.y, a.z*b.x-a.x*b.z, a.z*b.y-a.y*b.z ); } and its swizzle implementation

float3 cross( float3 a, float3 b ) { return a.yzx*b.zxy - a.zxy*b.yzx; }

Draw Texture

Now we are going to show how to add texture on a surface

float4x4 ModelViewProj; float4x4 ModelViewIT; float4 LightVec;

These are global variables.

struct a2v { float4 Position : POSITION; float2 Texcoord : TEXCOORD0; float4 Normal : NORMAL; }; add new Texcoord component as texture coordinate with TEXCOORD0.

struct v2p { float4 Position : POSITION; float2 Texcoord : TEXCOORD0; float4 Color : COLOR0; }; add Texcoord in output

void main( in a2v IN, out v2p OUT ) { OUT.Position = mul(IN.Position, ModelViewProj);

... same as the above example set texture coord

OUT.Texcoord = IN.Texcoord; } the code is the same as the above example except that the texture coord copy is added.

Pixel Shader

Pixel shader completes the computing of pixels.

float brightness; sampler2D tex0; sampler2D tex1;

brightness is the value to control the bright of light. Both tex0 and tex1 are the sampler of the textures.

struct v2p { float4 Position : POSITION; float2 Texcoord0 : TEXCOORD0; float2 Texcoord1 : TEXCOORD1; float4 Color : COLOR0; };

v2p declares a struct type that transfers data from vertex shader to pixel shader. It is the same as the struct in vertex shader. The input semantics of pixel shader can be COLOR n for Color or TEXCOORD n for Texture coordinates. Although the struct v2p must be the same as the v2p in the vertex shader, the item Position can not be read in the pixel shader, because it is not bind by the input semantics of the pixel shader.

struct p2f { float4 Color : COLOR0; }; p2f declares output data structure and OUT is the output object. The output semantics of pixel shader can be COLOR n of Color for render target n and/or DEPTH for Depth value.

void main( in v2p IN, out p2f OUT ) {

Constant parameter brightness has a float type. sampler2D specifies a 2D texture unit. When you plan to access a texture you must use sampler with an intrinsic function. A sampler can be used for multiple times.

float4 color = tex2D( tex0, IN.Texcoord0 ); float4 bump = tex2D( tex1, IN.Texcoord1 ); fetch texture color and bump coordinate for further computing of bump effect. tex2D is an texture sampling intrinsic function of HLSL. It generates a vector from a texture sampler and a texture coordinate.

OUT.Color = brightness * IN.Color * color; } the code multiples brightness , IN.Color and color to generate output RGBA color vector.

Image Processing (1): Sobel Edge Filter

This program shows the implementation of Sobel edge filter with a HLSL pixel shader. In the similar way we can implement many image filters in pixel shader.

float Brightness; sampler2D tex0;

global variables

struct v2p { float4 Position : POSITION; float2 Texcoord : TEXCOORD0; float4 Color : COLOR0; };

struct p2f { float4 Color : COLOR0; }; main function includes a vertice struct as input, a float parameter as brightness control, and a 2D texture sampler. The return value has float4 type with the semantic COLOR0.

void main( in v2p IN, out p2f OUT ) { const specifies the constants. The c[NUM] is a float2 constant array. Notes its initialization is convenience like C language. col[NUM] is a variable array of type float3 with NUM elements. int i declares the i as integer. This usage is effective for pixel shader 2.0 or later.

const int NUM = 9; const float threshold = 0.05;

const float2 c[NUM] = { float2(-0.0078125, 0.0078125), float2( 0.00 , 0.0078125), float2( 0.0078125, 0.0078125), float2(-0.0078125, 0.00 ), float2( 0.0, 0.0), float2( 0.0078125, 0.007 ), float2(-0.0078125,-0.0078125), float2( 0.00 , -0.0078125), float2( 0.0078125,-0.0078125), };

float3 col[NUM]; int i; it stores the samples of texture to col array.

for (i=0; i < NUM; i++) {

col[i] = tex2D( tex0, IN.Texcoord.xy + c[i] ); } now we start to compute the luminance with dot product and store them in lum array.

float3 rgb2lum = float3(0.30, 0.59, 0.11);

float lum[NUM]; for (i = 0; i < NUM; i++) { lum[i] = dot(col[i].xyz, rgb2lum); }

Sobel filter computes new value at the central position by sum the weighted neighbors.

float x = lum[2]+ lum[8]+2*lum[5]-lum[0]-2*lum[3]-lum[6]; float y = lum[6]+2*lum[7]+ lum[8]-lum[0]-2*lum[1]-lum[2]; it shows the points which values are over the threshold and hide others. Final result is the product of col[5] and edge detector value. Brightness adjusts the brightness of the image.

float edge =(x*x + y*y < threshold)? 1.0:0.0; final output

OUT.xyz = Brightness * col[5].xyz * edge.xxx; OUT.w = 1.0; }

Image Processing (2): Mask

A well known example of three media streams is the mask transform. A mask is a black and white image. Two media streams and a mask picture are the inputs. The effect is the sum of the first stream multiples the mask and the second stream multiples the negative of mask. Therefore, two streams are showed in the areas distinguished by the shape of the mask.

sampler2D tex[3]; struct v2p { float4 Position : POSITION; float4 Color : COLOR0; float2 Texcoord[3] : TEXCOORD0; };

These are the texture samplers and struct void main(in v2p IN, out float4 OUT : COLOR) { float Brightness = 1.2345; float alpha = 0.96;

float3 color[3]; for (int i=0; i<3; i++) { color[i] = tex2D( tex[i], IN.Texcoord[i] ); }

The HLSL sample code is quite simple for the formula. Here color[3] is the mask texture.

OUT.rgb = Brightness * (color[0] * color[3] + color[1] * (1.0-color[3])); OUT.a = 1.0; }

Video Mixing (1): Vertex Shader

Today the powerful GPU and multithread engine have made real-time video mixing on GPU possible. To watch the real- time video mixing effects you can download and try Ne@tware Player 2005.

In Ladybug Player Vista the sequence of video streams is followed the order in the playlist. You can launch Notepad to modify and save the HLSL source code by double clicking the item in the shader list. Below we describe the default vertex shader of the Ladybug Player Vista.

float4x4 WorldViewProj; float4x4 WorldViewIT; float4 LightPos; float4 EyePos; float fTime;

WorldViewProj is the transform matrix from model space to clip space. WorldViewIT is the transform matrix from model space to view space. LightPos is the light position in model space. EyePos is the eye position in model space. These global variables are reserved for later using. fTime is the application time that can be used to control effects in timeline.

#define NUMTEXCOORD 4

struct a2v { float3 Position : POSITION; float Diffuse : COLOR; float2 TexCoord[NUMTEXCOORD] : TEXCOORD0; };

This is the input structure from application to the vertex program (a2v). Position is the position in model space. Diffuse specifies the diffuse color. TexCoord specifies texture coordinates where video streams are represented as textures.

The number of textures NUMTEXCOORD is dependent on the graphic hardware. Four is the minimum support and some cards can sample upto 16 textures. struct v2p { float4 Position : POSITION; float4 Color : COLOR0; float2 Texcoord[NUMTEXCOORD] : TEXCOORD0; };

This is the structure from vertex shader to pixel shader (v2p). void main( a2v IN, out v2p OUT ) { OUT = (v2p) 0; OUT.Position = mul( float4( IN.Position, 1.0f ), WorldViewProj );

for ( int i=0; i < NUMTEXCOORD; i++ ) { OUT.Texcoord[i] = IN.TexCoord[i]; } }

The main program transforms the position from model space to clip space. Then copy the texture coordinates to output.

Video Mixing (2): Pixel Shader

Basic wipe transform is like to overlay two paper slips. The overlapped area of two pictures in the middle is mixed with alpha values by positions.

Ring wipe defines a ring in the center of the first picture. The area between the inner circle and the outer circle is the mixing of two pictures by alpha values that is decided by the distance to the center. The area inside the inner circle is belong to the first picture. And the area outside the outer circle is belong to the second picture. sampler2D tex[2]; struct v2p { float4 Position : POSITION; float4 Color : COLOR0; float2 Texcoord[2] : TEXCOORD0; }; sampler2D defines texture array that represents the video streams. v2p is the struct of stream from vertex shader to pixel shader. void main(in v2p IN, out float4 OUT : COLOR) { float Brightness = 1.2343; float len = 0.3; float a = 0.2; float b = a+len; float2 center = float2(0.5, 0.5); float3 result;

Brightness is the variable for brightness of light. a and b are the radius of inner and outer circle respectively. len is the width of the ring. And the center is the centric coordinate of the ring. result represents the result of the value of effect.

float3 color0 = tex2D(tex[0], IN.Texcoord[0]); float3 color1 = tex2D(tex[1], IN.Texcoord[1]); float2 point = IN.Texcoord[0]; color0 and color1 are sampled color value from the first picture and the second picture respectively. point is the current coordinate value.

float dist = distance( point, center ); if ( dist < a) { result = color0; } else if (dist > b) { result = color1; } else { result = lerp(color0, color1, saturate((dist-a) / len)); }

This is the kernel of the algorithm. distance is the build-in HLSL function. saturate

Copyright ©2006-2007 Neatware. All rights reserved. www.neatware.com Ladybug Player Vista Thursday, April 05, 2007 function converts dist value from [a, b] to [0, 1]. lerp is the build-in function for linear interpolation.

OUT.rgb = Brightness * result; OUT.a = 1.0; }

OUT is the last result.

Ring wipe transform is one of the most interesting transforms because the position and size of a ring can be variant. Furthermore, many ring wipes can be applied on different center of a shader. At this stage the SM3.0 will be very helpful. Who has said that software was always lagged hardware? FX Effects

An effect consists of multiple rendering techniques which may be one or more passes.

A detailed article about the FX effects and the use of effects in Ladybug Player Vista is here.

FX Effects

Effects Framework

To create a set of visual effects in a logical unit, Microsoft defined effects framework. Effect files are referred to as fx files with the extension .fx . It can be compiled as a .fxo file. The structure of an fx file looks like the following:

Global Variables Effect States Pipeline,Texture,Sampler,Shader

Functions HLSL, Assembly

Effect File Parameters Technique T0 Global Variables Effect States Pass P0

...

The effects framework allows effect developers to group similar graphics techniques in a single file. It also allows the content creators to create a single set of content that will work across multiple PC configurations while maintaining a consistent look and feel. The effects framework allows techniques to range from high-level shader, low-level shader, and fixed-function pipeline.

Here are some features of effects :

• The Global Variables of effects can be set by either the effect itself or by the application.

• The State Management of effects includes pipeline, texture, sampler, shader state management. The pipeline state management includes setting transformations, lighting, materials, and rendering options. The texture and sampler state management contains specifying texture files, initializing texture stages, creating sampler objects, and setting sampler state. The shader state management includes creating and deleting shader, setting shader constants, setting shader state, and rendering with shader.

• The Effects contain multiple rendering options called Techniques . Each technique encapsulates global variables, pipeline state, texture and sampler state, and shader state. A single style is implemented in a rendering pass. One or more passes can be encapsulated in a technique. All of the passes and techniques can be validated to see if the effect code will run on the hardware device. Effects will save and restore state, leaving the device in the same state as before the effect was run.

• The Function is defined by HLSL or Assembly program.

Effect Parameters

Effect parameters are all the non-static variables declared in an effect. This can include global variables and annotations. Effect parameters can be shared between different effects by declaring parameters with the shared keyword and then creating the effect with an effect pool. A global variable with the static keyword in front is not an effect parameter.

Annotations provide a mechanism for adding user information to effect parameters. It is used to exchange data between an application and a GPU program. Annotation declarations are delimited by angle brackets, <>. For example, float3 LightVector < string UILightVector = "Light Vector"; > = {1.0, -1.0, 0.0};

Usually an annotation starts with a data type (string), followed by an assign statement with variable name (UILightVector) and the value ("Light Vector"), and ended by a semicolon. The annotation is declared behind a variable and in front of the assignment of the initial value {1.0, -1.0, 0.0}.

Effect States

Effects simplify managing pipeline state. States of effects specify the render conditions of the pipeline. Effects render a given technique that may contain one or more passes. State is restored each time a technique finished. However there are no state changes between passes. Effect states can be divided into the following functional areas:

Name States Pipeline States Light states Material states Render states Vertex render states Pixel render states

Sampler States Sampler states Sampler stage states

Texture States Texture states Texture stage states

Shader States Shader constant states Vertex shader constant states Pixel shader constant states

Transform States

Notes: State description in details can be found in the DirectX 9.0 SDK documentation.

Effects in Ladybug Player Vista

Note: Programmable HLSL Shader is only available for Developer and Extreme Editions.

Ladybug Player Vista can complete real-time video post-processing through programmable GPU. Users can apply image-processing techniques to video and implement these techniques with HLSL in GPU to achieve 10x performances than CPU in computing.

In GPU a video frame is treated as a texture and is rendered over a scene, then the final result is processed through pixel shader . The pos-processing effects can achieve interesting results such as picture blur, bloom of the bright spots, or depth of field (DOF) that blurs a pixel based on its depth.

An effect file named as VideoMixerDev.fx implemented all the video post-process by GPU in Ladybug Player Vista. The .fx is the source file of fx effects, .fxo file is the compiled .fx file. More than one techniques that may have one or more passes have been defined in the .fxo file.

Declaration

In the effect file, there are four texture objects with samplers from which the post- process pixel shader will sample. They are defined as textures.

#define MAXNUM_CHANNEL 4 texture SamplerTex0; texture SamplerTex1; texture SamplerTex2; texture SamplerTex3; sampler source and scene for color, normal, position, and velocity sampler g_Sampler[MAXNUM_CHANNEL] = { sampler_state { Texture = ; MinFilter = Linear; MagFilter = Linear; MipFilter = None; AddressU = Clamp; AddressV = Clamp; }, 。。。 }

Usually the post-process pixel shader will need to sample a source texture multiple times for a single destination pixel, each at a different location near the destination pixel. When this is the case, the effect file declares a kernel which is an array of texel offsets (float2) from the destination pixel. The effect file then defines another equally-sized array of float2. For example, float2 PixelKernel[KernelSize] = { { -1, 0 },{ 0, 0 }, { +1, 0 } };

float2 TexelKernel[KernelSize] < string ConvertPixelsToTexels = "PixelKernel"; >;

This second array has a string annotation, named ConvertPixelsToTexels, which contains the name of the first kernel array. When the sample loads the effect file and sees this annotation, it will translate the offsets from pixel coordinates to texel coordinates, which are compatible with the texture sampling functions such as tex2D. When the source texels are sampled, the offsets in the kernel are added to the texture coordinates to obtain the texel values near the destination pixel.

Parameters

Some effects use parameters that the user can adjust. Each PostProcess technique can define one or more annotations for this purpose. The maximum number of parameters supported by a single technique is four.

Annotation Name Type Description Parameter0 string Contains the name of the variable that the user can adjust.

Parameter0Def float4 Default value for Parameter0. This is always a float4. If fewer than four floats are needed, . use Parameter0Size to specify it

Parameter0Size Int Specifies how many components of Parameter0 are used. If this is 2, then only the first two components (x and y) are used.

Parameter0Desc string A string that describes this parameter. For example, pass p0 < float fScaleX = 4.0f; float fScaleY = 4.0f; >

Here is the vertex input shader structure struct VS_INPUT { float4 Position : POSITION; float2 TexCoord : TEXCOORD0; };

Here is the vertex output shader structure struct VS_OUTPUT { float4 Position : POSITION; float2 TexCoord0 : TEXCOORD0; float2 TexCoord1 : TEXCOORD1; float2 TexCoord2 : TEXCOORD2; float2 TexCoord3 : TEXCOORD3; };

Timer is a time variable. float Timer : TIME < string UIWidget = "None"; >;

//******************************************************************* // Name VertScene // Description vertex shader // Parameters inputs Pos, Tex0, Tex1, Tex2, Tex3, // outputs oPos, oTex0, oTex1, oTex2, oTex3 // Return void //******************************************************************* void VertScene( input textures

in float4 Pos : POSITION, in float2 Tex0 : TEXCOORD0, in float2 Tex1 : TEXCOORD0, in float2 Tex2 : TEXCOORD0, in float2 Tex3 : TEXCOORD0, output textures

out float4 oPos : POSITION, out float2 oTex0 : TEXCOORD0, out float2 oTex1 : TEXCOORD1, out float2 oTex2 : TEXCOORD2, out float2 oTex3 : TEXCOORD3) { set position and textures

oPos = mul( Pos, WorldViewProj ); oTex0 = Tex0; oTex1 = Tex1; oTex2 = Tex2; oTex3 = Tex3; }

//****************************************************************** // Name PixScene // Description Pixel shader // Parameters input float2 array Tex[] with TEXCOORD0 // Return float4 with COLOR0 //****************************************************************** float4 PixScene(in float2 Tex[MAXNUM_CHANNEL]:TEXCOORD0) : COLOR0 { float4 OUT; get samplers color0 and color1

float4 color0 = tex2D(g_Sampler[0], Tex[0]); float4 color1 = tex2D(g_Sampler[1], Tex[1]); get time’s sin value to fade two frames

saturate((sin( Timer/2.0 )+1.0)/2.0); alpha = 0.5; OUT = lerp(color0, color1, alpha) ; return OUT; }

//******************************************************************* // Name SoloChannel // Description Display chan and Tex // Parameters chan is the channel number // Tex[] is the texture array // Return float4 //******************************************************************* float4 SoloChannel(in int chan, in float2 Tex[MAXNUM_CHANNEL]:TEXCOORD0) { return tex2D( g_Sampler[chan], Tex[chan]); }

//******************************************************************* // Name SoloChannel_1 // Description Display channel 1 only // Parameters Tex[] is the texture array // Return float4 //******************************************************************* float4 SoloChannel_1(in float2 Tex[MAXNUM_CHANNEL]:TEXCOORD0) :COLOR0 { return SoloChannel( 0, Tex ); }

//******************************************************************* // Name SoloChannel_1 // Description Technique SoloChannel_1 transform // Return none //*******************************************************************

Technique SoloChannel_1 { Pass P0 { ignore CullMode and ZEnable

CullMode = None; ZEnable = False; compile VertScene and ScoloChannel_1 by Shader Mode 2.0

VertexShader = compile vs_2_0 VertScene(); PixelShader = compile ps_2_0 SoloChannel_1(); } }

Chapter 4 Applications

Digital Signage

Digital Signage is a technology that replaces traditional signs such as posters by large screen liquid crystal displays (LCDs) and plasma display panels (PDPs) as well as embedded computer systems.

The digital signage system can deliver motion pictures and crystal clean logos to capture consumers' attention instantly. Moreover, advertising can be upgraded, distributed, and expressed effectively through digital signage network.

While other media players can only display single video stream, Ladybug Player Vista is able to show multiple cinematic HD video streams with smooth mixing and zoning. So content creation is simplified and ads become more attractive with special visual effects.

Digital Signage is widely used in retail stores, banks, restaurants, airports, hotels, sport venues, and apartments.

To make a simple standalone Digital Signage , you can do

1. Make a video

• shot a video • cut video with an creator such as Movie Maker 2 • add audio • save video

2. Make a logo sign

• design a logo • place logo onto a black picture with the same size as video • save picture

3. Design a shader

• select a sample HLSL pixel shader such as circle wipe • double click to open the source file • modify parameters for your purpose • save shader

4. Watch effects

• open video in channel 0 • open logo picture in channel • select shader in the pixel shader • launch and watch result

5. Testing

• repeat to achieve best result

Surveillance

Real-time video post-processing technologies are suitable for Surveillance such as Homeland Security . Surveillance requires videos to be captured in multiple sites simultaneously. Then videos are processed and analyzed in real-time. For example, if there is no motion in the video capture, the record switch will not be turned on. When motion is detected the recorded video can be sent to a information center through broadband and raise an alarm.

Let’s investigate a simple application. Suppose two pictures are A and B, we can compare A and B to see if they are the same. A xor (not B) is a simple computing. If B is the same as A its result is 0. This simple computing can be implemented by HLSL. This is a simple motion detection, in which a video is compared to a still picture: a non-black video will be displayed.

The feature of multiple zones in Ladybug Player Vista allows multiple videos from multiple cameras to be display on one screen in zones.

VJ Control

Like DJ for audio mixing and control, VJ is for video mixing and control.

A major function for VJ is the real-time playback control and effects switching. The flat, transparent, and instant GUI of Ladybug Player Vista provides VJ many controls such as play/pause, volume, and effects.

The ticker in the Ladybug Player Vista allows you to input scripts while the video is been playing. Then the text scripts will be scrolled from right to left at the bottom. So a VJ can inform audiences without interrupt the music and video playback.

Video Communication

Videoconference is a well known video communication that has been widely used in business. In video communication video streams are transmitted through wired or wireless network from one site to another. The communication can be peer to peer or multiple clients.

Today broadband Internet and high-speed wireless network provide a platform for video communication. The applications of video communication through IP network can apply Ladybug Player Vista to present multiple channel video streams in real-time.

Video Remix

Video Remix is often referred to video post-processing. As the High-Definition digital cameras become widely adapted in the movie industry, video post-processing becomes more and more important.

In addition to the use of video post-processing in the movie industry, video post- processing has been widely used in the industry such as oil and gas industry for its large video analysis as well as the manufactures for pipeline monitor and product vision.

Mobile Video

Mobile Computing

Mobile Computing is the information processing in portable devices through wire or wireless network connection. Users can store videos in a notebook and display videos on a large flat LCD screen or projector through a cable or wireless.

Mobile Software is the software stored in a portable device and can be carried on everywhere. A Portable Device can be connected to a Host System through a standard interface such as USB or WiFi. The Active Portable Device includes a processor and memory so it can work independently. MP3 Player, iPod, PDA are examples. In contrast the Passive Portable Device does not contain a processor such as USB storage device.

Mobile Software can work for the processor of the portable device or work along with the processor of a Host System. Thus, a Mobile Software can plug and play while the portable device is connected into a Host System. No installation on Host System is required.

Mobile Software and its device can be a token for security access to replace passwords. Many security features can be added on it such as vpn access, network access, web access, email access, boot and file protection, security key and certificate storage, and self-enrollment key.

The configuration database of a Mobile Software provides the information for different Host System. The Mobile Software allows user manually or automatically select the setting data.

Ladybug Player Vista can be a mobile video player that is stored in a U3 smart device, a USB device or a MP3 Player. Ladybug Player Vista can connect to a PC Host with Windows XP/ 2000/ 2003/ Vista operating system.

Mobile Applications

The Mobile Edition of Ladybug Player Vista supports both active and passive portable storage devices. Here are a list:

• U3 Smart Device

A U3 smart drive hosts a truly portable "personal workspace" -- not only a person's data and files, but software programs, personal preferences, passwords and settings, and the means for managing them -- all in a highly secure environment.

Users can carry their personal workspace with them on a small, lightweight, convenient and affordable U3 smart drive (including emails, applications, personal settings, bookmarks and favorites, folders and documents) as they move to and from different locations.

U3 smart device provides a private, secure, and personalized computing platform on host systems for Portable users.

Along with partners like U3 LLC Company, Neatware has ported and optimized Ladybug Player Vista on the U3 platform. Users can store their photos, videos, and music on a U3 smart device and present them on a Host PC with real-time effects for guests and friends.

• iPod for Windows

iPod can be used as a portable hard disk and flash memory. So you can install Ladybug Player Vista on iPod. Then you can watch movies on a PC or a notebook when you insert your iPod into their USB ports.

The capacity of iPod can be in 10GB, 20GB, 40GB, and 60GB. iPod nano with 1GB plus flash memory is also suitable to use with Ladybug Player Vista .

You can install Ladybug Player Vista on iPod as follow

1. Insert iPod into USB 2.0 port on PC. 2. Click My Computer from Start menu. 3. Find the drive referring to iPod such as K: 4. Execute Ladybug Player Vista installation. 5. Select iPod drive as dest folder. 6. Run Ladybug Player Vista.

• USB Flash Drive

Ladybug Player Vista also supports USB Flash Drive from multiple vendors. Users can license and install Ladybug Player Vista on their own USB devices with any capacity. Neatware provides Ne@tware brand USB Flash Drive with Ladybug Player Vista pre-installed in the capacity of 32MB and 128MB as well.

Ladybug Player Vista supports the capacity of USB Flash Drives in 32MB, 64MB, 128MB, 256MB, 512MB, 1GB, 2GB, or 4GB from any vendors. The fast USB Flash Drives are better in performance for mobile content presentation.

There are lots of USB Flash Drive providers in the world. Neatware provides Ne@tware USB Flash Drives along with its mobile software. SanDisk, Lexar, PQI, .etc provide penny-sized USB flash card or Intelligent Stick(TM) small and/or fast USB flash drives.

• USB Mobile Hard Drive

You can install Ladybug Player Vista on a Mobile Hard Drive along with your videos. So you can let your customers to watch your promotion videos in your marketing tutor.

The capacity of USB Portable Hard Drive may be in 1" with 2.5GB and 5GB; 2.5" with 20GB, 40GB, 60GB, and 80GB; 3.5" with 80GB, 120GB, and 160GB.

As a partner of Seagate , Neatware supports Seagate's USB Portable Hard Disk with its Ladybug Player Vista as a mobile video player.

• USB MP3 Player

You can install Ladybug Player Vista on your MP3 Player. When you insert MP3 Player into a PC, you can launch Ladybug Player Vista and watch movies on Windows.

The capacity of USB MP3 Player can in 128MB, 256MB, 512MB, 1GB, and 2GB.

• Wireless Storage

Wireless Hard Drives 802.11g, 54Mbps

Product Activation For Mobile Device

1. Download and install Ladybug Player Vista. 2. Insert an USB device in the host computer. 3. Run GetActivateData application in the host folder. 4. Email the file ActivateData.txt back to [email protected] . 5. Neatware will return an activation binary key file named data.sig. 6. Save the data.sig file to the host folder. 7. Launch Ladybug Player Vista along with the USB device.

Appendix

FAQ

• What is the key functionality of Ladybug Player Vista?

Ladybug Player Vista is a media player to support programmable shader and effects on Windows. Ladybug Player Vista supports video and audio mixing in real-time on True Fullscreen.

Ladybug Player Vista Home Edition allows you to download and try free for 100 uses. Later you can subscribe online to make the full version in activation.

• Can I apply Ladybug Player Vista for video post-processing?

Absolutely, by using the programmable pixel shader, Ladybug Player Vista can complete the video special effects in real-time. In Developer Edition, Users can design their own shader with HLSL or Assembly languages in FX Effects.

• What DirectX version I need to install?

Ladybug Player Vista supports the latest DirectX 9.0c (2004 Summer Upgrade, 7/26/2004). To use Shader Model 3.0 or 2.0x, the DX9.0c is required. Although DX9.0b is supported, we suggest that you upgrade to DX9.0c for better performance and video quality. Windows XP SP2 and Vista is preinstalled DX9.0c.

• What is the difference between True Fullscreen and Frameless Fullscreen?

True Fullscreen is a special display mode with the display exclusively belonging to your application. The performance and image quality are high in True Fullscreen and its resolution dependents on the graphics adapter. Almost all the games use True Fullscreen.

Frameless Fullscreen is a borderless window in screen size. Windows Media Player and many other players used this mode for fullscreen.

At present Ladybug Player Vista supports True Fullscreen that is better for movie watching and video effects.

• How Ladybug Player Vista distinguish from other players?

There are many media players available such as Microsoft's Media Player, Apple's QuickTime, and Real Network’s RealPlayer .etc. All of them did not take the advantage of the GPU. Ladybug Player Vista is a next generation media player that takes advantages of GPU functions and performance.

Ladybug Player Vista adapts true fullscreen and supports programmable shader with

HLSL up to Shader Model 3.0 with FX effects. In addition, it can play multiple videos simultaneously by using the multithread engine inside the Ladybug Player Vista.

3D graphics, programmable shader, and multiple video playbacks all are lacked features in the current media players.

• Where to download WMV codec?

New Windows XP SP2 will include Windows Media 9 Series with WMV codec as default. You can download and install Windows Media 9 Series with WMV/WMA codec in the Microsoft web site.

• Any movie trailers available to test the Ladybug Player Vista?

You may download HD WMV videos on the Microsoft web site.

• Why dual-core processor is suitable for video mixing?

* There are many video streams, graphs, and codecs running in the video mixing. * Each video graph is irrelevant before the mixing. * Each video graph is implemented in independent threads. * Each core can execute a graph in parallel.

• Why GPU is good at video mixing in the post-processing

* High bandwidth between GPU and video memory. * High parallel processing. * Programmable video mixer. * Post-Processing effects in real-time.

• How to change file associations in Windows? http://support.microsoft.com/default.aspx?scid=kb;en-us;307859&product=winxp

History

Ladybug Player Vista (2.2) New Features

• Resume playback • Playlist to play a sequence of videos in an individual channel • Embedded VOB support • NWM management by SQL database • Logger in database • Super High-Resolution Photo support up to 8192x8192 pixels • Programmable FX effects in Developer Edition • International language input and display in ticker

Ladybug Player Vista (2.0) Features

• Cutting-edge technologies and new design • Small, fast, powerful multithread video engine • Real-time FX effects on GPU • Rich programmable shader with HLSL language • Quad channel video streams for video composition and post-processing • Solo, duo, trio, quad zones for independent video display • Flat GUI for instant control and ticker • Customized player supports multiple windows and screens • International language supports • Recognize many media formats: WMV, AVI, MPEG, JPEG and HD videos

Ladybug Player Vista (1.9) New Features

• Frameless window • Download Media by URL

Ladybug Player Vista (1.8) New Features

• Multi-Player: support multiple players, windows, channels, formats. • Quad Videos: view real-time quad videos on single screen. • Programmable Windows: shader supports splitter windows. • Optimization for Dual-Core: multithread on dual-core processors. • Frameless Window and Fullscreen: no board window.

Ladybug Player Vista (1.7) New Features

• URL Support: download video from internet with http access • Neatware Media (NWM) File: new file type for customized playback • Command Line: launch player with a .nwm or a media file as input • Autorun: skip the control pad and watch video directly • Multiple Channels: input media in one of a channel for mixing

Ladybug Player Vista (1.6) New Features • Video Mixing can generate cinematic effects in real-time. • New Graphic User Interface is much convenient to use. • Media Management by SQL Database greatly improved the extensibility and configurability of the player. • Portable Software installed in portable storage devices for multiple computers and users.

Ladybug Player Vista (1.2) New Features

• Video Effects over an 3D model. Ladybug Player Vista is the first media player that allows a movie to show on the surface of an 3D object. • Multiple Render Target (MRT) generates the effects of HDR, blur, glow, and Depth of File • Model Builder supporting such as Wings3D, Maya, 3DS, and other CAD tools allows users to create his own models and render the light maps of video over them. • DVD Codec allows you to watch DVD movie over an object with visual special effects.

Ladybug Player Vista (1.0) Features

• True Fullscreen automatically fits for widescreen notebook, LCD flat screen desktop, and High-Definition TV (HDTV) with clean pictures. • Multithread Engine mixes multiple videos, 3D models, and music to render composted video. • HLSL Shader generates real-time video special effects, executes codes of Shader Model 3.0, and supports the latest DirectX 9.0 technologies. • Format Neutral recognizes any video formats whose codec support Windows like AVI, WMV, and MPEG. • Unicode Support allows world-wide audiences to choose their own languages to control video playback. • USB distribution provides Ne@tware Player in the USB Flash Memory and Mobile Hard Disk. • Technology to protect software and contents.

System Requirements

DirectX 9 Complaint means that all the key functions of DirectX 9 will work especially the Shader Model 2.0. DirectX 9 Support is defined as most functions of DirectX 9 will work as hardware acceleration but some advanced functions such as Shader Model 2.0 may not work. Others are Non-DirectX 9 Support.

Windows 95, NT are Non-DirectX 9 Support. Windows 98, ME, 2000, XP, 2003 can be DirectX 9 Support or DirectX 9 Compliant which is dependent on the video card or chipset.

Intel's GMA 900+ graphics in its chipset 915G and 925G is DirectX 9 Complaint. However Intel's Extreme Graphics 2 in its chipset 865G and 875G are only DirectX 9 Support. Latest Intel 965 chipset for Core Duo processor is Windows Vista Capable.

AMD Radeon 9500, 9600, 9700, 9800, X800, X1300, X1600, and X1900 are DirectX 9 Complaint. The notebook with ATI's X1300+ is DirectX 9 class and Windows Vista Capable. AMD Radeon R600 supports DirectX 10.

NVIDIA GeForce 5200, 5600, 5800, and 6000 are DirectX 9 Complaint. NVIDIA's iForce for AMD64 is DirectX 9 Complaint. In addition, GeForce 6800 supports Shader Model 3.0. GeForce 8800 supports Shader Model 4.0 and DirectX 10.

Via and SiS video cards may be DirectX 9 Support or DirectX 9 Complaint.

Before running Ladybug Player Vista make sure your PC is Windows Vista Capable. Welcome vendors to test and provide their chip information that Neatware's Player can work well.

Ladybug Player Vista supports DirectX 9.0c, the latest DirectX 9.0 Summer 2004 Release and plus. The HLSL and FX compiler in the DirectX9.0c supports Shader Model 3.0. To download DirectX 9.0c from Microsoft site click here (~34MB). Following table briefly described the Shader Model version used in Ladybug Player Vista and the supported video cards,

SM4.0 DX10, Geforce G80+, Radeon R600+ SM3.0 DX9, Geforce 6800+, Radeon X1300+, GMA 965+ SM2.0b DX9, Radeon X800 XT SM2.0a DX9, Geforce 5600+ SM2.0 DX9, Radeon 9600+, Geforce 5200+, Intel Chipset 915G/925G