De-Anaglyph — Separating the stereoscopic views of anaglyph 3D images Christian Richardt & George Drettakis, REVES – Inria Sophia Antipolis [email protected], [email protected] http://www-sop.inria.fr/reves

Context and research goal With our two eyes, we see the world in stereoscopic 3D with two slightly different images projected on the left and right retinas. Anaglyph 3D is a basic technique for representing these images using colour separation (see the figure to the right). By dividing the spectrum of visible light into two halves, the ‘ half’ can encode the left and the ‘half’ the right image, which can be easily separated using the opposite colour filters. This basic technique is still one of the most practical approaches, as it does not require any special display hardware, but just a pair of red-cyan 3D .

This project aims to invert the anaglyph process by essentially changing the directions of the arrows in the figure on the right. As anaglyph images combine two RGB images into one RGB image, the main difficulty of the inverse direction is to create two full-colour images out of a single anaglyph image. While the initial sepa- ration into red and cyan layers is trivial, reconstructing the original colours requires information from each image to be transferred to the other, for which corres- pondence information between images is needed.

Approach Dense correspondences between the two views of a stereoscopic 3D image can be computed using matching, which is a well-understood field of computer vision [1]. The twist is that virtually all stereo matching techniques assume that corresponding pixels have similar colours, which is not the case for the red/cyan images. Therefore, the core contribution of this project is to extend a state-of-the-art stereo matching technique, for example using graph cuts [2, 3] or belief propagation [4, 5], to the ‘inconsistent’ colours of red/cyan anaglyphs (compare the coloured spheres in the red and cyan images in the above image). If time permits, this project could also be extended to anaglyph 3D , in which case temporal coherence will play a very important role. Requirements The successful candidate should have taken courses in image processing, computer vision and/or computer graphics. For performance reasons and to take advantage of existing libra- ries, this project will most likely have to be implemented using C/C++ or C#. Alternatively, a GPU-based implementation is also possible (and in fact desirable for the extension), in which case relevant experience with CUDA, OpenCL or shading languages such as GLSL is required.

Location The internship will take place at Inria Sophia Antipolis, on the beautiful French Riviera. Inria provides a student stipend/salary to cover living expenses.

The project would start in June 2013, for 2 to 3 months.

References [1] Daniel Scharstein and Richard Szeliski A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms International Journal of Computer Vision, 2002, 47, 7–42 See http://vision.middlebury.edu/stereo/eval/ for the full evaluation table.

[2] Yuri Boykov, Olga Veksler and Ramin Zabih Fast Approximate Energy Minimization via Graph Cuts IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23, 1222–1239

[3] Vladimir Kolmogorov and Ramin Zabih Computing visual correspondence with occlusions using graph cuts Proceedings of the International Conference on Computer Vision (ICCV), 2001, 2, 508–515

[4] Jian Sun, Nan-Ning Zheng and Heung-Yeung Shum Stereo matching using belief propagation IEEE Transactions on Pattern Analysis and Machine Intelligence, 2003, 25, 787–800

[5] Pedro F. Felzenszwalb and Daniel P. Huttenlocher Efficient belief propagation for early vision International Journal of Computer Vision, 2006, 70, 41–54