Google Soc 2009 Proposal | OpenImageIO Gaurav Jain | [email protected]
Google Summer of Code 2009 – Proposal OpenImageIO
By: Gaurav Jain [email protected] | www.gauravjain.com
Name: Gaurav Jain
School: Computer Engineering (4 th Year) Bachelors of Applied Science and Engineering University of Toronto, St. George Campus
Courses: Computer/Software Multimedia /Hardware
Distributed Systems (Java, C) Multimedia Systems
Programming Languages (Python, Prolog, Scheme) Computer Hardware
Operating Systems (C) Computer N etworks
Algorithms and Data Structures Probability and Random Processes
Computer Organization (Assembly) Sensory Communication
Programming Fundamentals (C++) Digital Signal Processing
Expertise (HW, OS, languages, libraries, GUIs, etc.):
Operating System: Linux Red Hat, UNIX, Solaris, Windows Vista/XP
Languages: C, C++, Java, Python, Perl, MATLAB, Flash ActionScript, XML
Libraries: FFmpeg, Basic knowledge of OpenGL and Boost
GUIs: Windows API Framework, MS DirectX 9.0, PyGTK, Adobe AIR
IDEs: VIM, Eclipse, MS Visual Studio, Adobe Flex Builder
Hardware/Network Programming: FPGA Programming (Verilog), TCP/IP, UNIX Sockets, Motorola 68K
Page 1 of 5
Google Soc 2009 Proposal | OpenImageIO Gaurav Jain | [email protected]
Tell us about your programming experience, especially anything to do with graphics or images:
I am a 4 th year Computer Engineering student at the University of Toronto. I am well-versed in C/C++ and possess strong background and knowledge in Image and Video Processing Fundamentals. Some of my current and past work experience in Multimedia and Software is described below. I invite you to visit, www.gauravjain.com , for detailed description of my technical projects and experiences.
Tone Mapping for HDR Images Research Assistant (Professor Konstantinos Plataniotis) BELL CANADA MULTIMEDIA LABORTAORY, University of Toronto, Dec 2008 – May 2009
Implementing adaptive Local and Global Tone Mapping techniques using a model of human retinal processing to render High Dynamic Range (HDR) images on standard displays Researching on techniques to use Microsoft HD Photo codec for HDR image compression Platform: MATLAB, C, Linux
Image Processing Algorithms and DirectDraw Video Player Video Algorithm Design Engineer SIGMA DESIGNS, Burlington, Ontario, May 2007 – August 2008
Designed and implemented a Scene Change Detection feature to prevent pixel interpolation errors in Frame Doubling technique (60->120 fps) for High Definition displays (C++, Linux) Rendered Metadata on the video display for the in-house YUV/RGB video player; Developed on MS Visual Studio using Windows API and MS DirectDraw; Supported 10 /12 bit video formats
Online Business Card Reader using Low Resolution Images CAPSTONE Design Project, Sep 2008 – March 2009
Built an online application to digitize a business card image captured through a 1-2MP webcam Pre-Processing before Optical Character Recognition includes Adaptive Binarization, Denoising, PNG->TIFF trans-coding and Interpolation techniques Development Platform: Adobe Flex; Flash ActionScript; C++; Team Size: 3
Hand Writing Recognition MULTIMEDIA SYSTEMS Course, Sep – Dec 2008
Implemented a Principal Component Analysis based algorithm to distinguish hand written digits Development Platform: MATLAB; Team Size: 2
Why are you interested in this project?
Having worked in the Image Processing industry and undertaken many academic projects in Multimedia Systems, I have a keen interest in Computer Graphics, Image and Video Processing. OpenImageIO has done a commendable job in creating a set of functions for 2D image I/O exhibiting optimal processing time. I would like to contribute to OIIO’s efforts by extending ImageViewer functionality by building OpenGL enhancements such as wipe transition effect between images. I consider GSoc under OIIO’s guidance an excellent opportunity to bring out its some of OpenGL powerful’s functionality to OIIO, and in the process gain further knowledge and understanding of OpenGL. I am highly motivated to learn,
Page 2 of 5
Google Soc 2009 Proposal | OpenImageIO Gaurav Jain | [email protected] design, implement additions to the code-base, and am confident that I can productively contribute to the OIIO library. I also believe that OIIO has a strong platform to build upon, and with some positive contributions can surpass the image IO capabilities offered by other programs such as FFmpeg and ImageMagick.
Project Proposal
I propose to contribute to the OpenGL Enhancements project for the ImageViewer utility in the OpenImageIO library. For the purposes of the proposal, I have a drafted a theoretical and implementation overview of how certain OpenGL enhancements such as Toggling between Images without reloading textures and transition effects such as Wiping between Images can be added to the iv using OpenGL Pixel Buffer Objects and Multiple Texture Objects.
Toggling Between Images without Reloading Textures
Using Multiple Texture Objects in OpenGL memory, we can toggle between images viewed in a sequence (forward and backward) without having to reload pixel data every time from file.
The Algorithm: We can create a circular buffer of n (=5, in our case) texture objects. As the user requests to view a new image (), we fill up the next empty texture object with the requested image and update the display object pointer. In case, a user decides to view the image that has been pre-loaded in OpenGL memory (), we can easily decrement the display object pointer to reflect the texture object holding the previous image in sequence, thereby eliminating the need for a re-load from memory.
In the event where all n texture object buffers are full and a request for a new image comes in, we can offload the first loaded image, and replace it with the latest data.
Pixel Buffer Object Image Source from File Circular Buffer
Texture Objects
OpenGL Memory
Page 3 of 5
Google Soc 2009 Proposal | OpenImageIO Gaurav Jain | [email protected]
Wiping Between Images
OpenGL Memory
Working Row Pixel Buffer Object Image Buffer Source from File Texture Object
Pixel Buffer Object OpenGL Memory
Interaction between CPU and OpenGL Memory for Wiping Transition Effects
Technical Overview: The wiping between images can be simply understood as gradual transition of one image into another. The implementation of the Wiping Effect sweeping left to the right can be divided into the following three steps:
1) Image Regions: The Image sources can be identified into three regions as shown in the following figure. The Gradient region is the combination of the two images to provide the wipe transition effect, whereas Image A and Image B have their own regions. Depending on the parameters provided by the user, the Left Edge and the Right Edge values can be initialized to specify the size of the overlap.
Gradient
Page 4 of 5
Google Soc 2009 Proposal | OpenImageIO Gaurav Jain | [email protected]
2) Setting the Gradient: The Gradient can be set by combining the pixel values from Image A and Image B (weighted average of the two pixel values). In the above example, the we can combine two windows:
[w*(Image B data: Left Edge to End) +(1-w)* (Image A data: Start to Right Edge )]
where, 1.0 < w < 0.0
3) Combining Images: Once we have all the regions identified, the two images can be combined row by row and put into the texture surface. We need a working buffer to hold an entire row of image data, copying parts of Image B, Gradient and Image A based on the left and right edges. Once we have the row buffer ready, it can be transferred to the texture surface.
Parameters: Although the above explanation provides a basic overview, we can include the following parameters to create the wipe effect can be specified, which can then be mapped to set the gradient size and the left and right edges, and the animation time:
Gradient Size: Width of the overlapping window
Animation Time: The amount of time can be specified, in seconds, for the transition to complete.
Direction: Sweep from Left to Right or Top to bottom
Page 5 of 5