The Development of Plug-Ins for GIMP

Computer Science Honours Project Proposal The Development of Image Completion and Tutorial Plug-ins for GIMP

Cathy Irwin [email protected] Department of Computer Science, Rhodes University Supervisors: Shaun Bangay and Adele Lobb 8 March 2004

The Problems to be Addressed There are currently many techniques available to manually edit images that have had objects removed from them so that the background region is filled in in a realistic manner. This however requires painstaking and skillful work. Therefore the problem this project aims to address is that of providing a facility to achieve automatic image completion for a common image manipulation package. This involves the development of extra functionality for the GNU Image Manipulation Package (GIMP) in the form of a fragment-based image completion plug-in.

A second problem to be addressed in this regard is that of developing a tutorial for writing general plug-ins for the GIMP.

Intended Approach

First semester Proposed dates: First term Study recommended papers and tutorials related to research 1/03- 12/03 project. Write research proposal. Due 6/03 Prepare first presentation on project. 5/03-8/03 Decide on specific design considerations. 8/03-12/03 Investigate other related research and implementation work in Throughout first

1 order to prepare literature survey. semester Second term Design project website 29/03–8/04 Implement prototype GIMP plug-in to ensure familiarity with 13/04-23/04 environment. Start development of tutorial for writing a general plug-in. 13/04-23/04 Design algorithm for image-completion plug-in. 26/04-30/04 Implement image-completion plug-in. 3/05-14/05 Validate plug-in using original test data and under additional 17/05-28/05 conditions. Hand in literature survey and project abstract. Due 30/05 Second semester Finalize tutorial for development of plug-ins and put on the World 2/08-13/08 Wide Web. Design poster presentation Due 18/08 Write final paper. Due 27/9 Final project write up. Due 8/11

Initial Survey of Resources and Literature Several papers from the proceedings of the ACM SIGGRAPH conferences will be used:

The primary source for this project will be a paper by entitled Fragment-Based Image Completion (Drori, Cohen-Or & Yeshurun, 2003). This paper discusses how the visible parts of an image serve as a training set to infer the unknown parts that become exposed when objects are removed.

A related paper is Image Quilting for Texture Synthesis and Transfer (Efros & Freeman, 2001). This discusses how one can stitch together small blocks of texture to create large, apparently unbroken surfaces. This technique is relevant to this project in that we will need to synthesize background textures.

Another paper is Image Inpainting (Bertalmio, Sapiro, Caselles & Ballester, 2000), which discusses techniques to propagate image information from boundaries inwards in order to

2 modify pictures in a virtually undetectable way. Unlike inpainting however, which is suited to small, smooth, non-textured regions, image completion needs to deal with large, textured and smooth regions.

The following are possible resources to aid the actual development of the plug-in and assist with the generation of a GIMP plug-in tutorial:

‘The Gimp Developers’ website’ (2004) provides an empty template for a generic GIMP plug-in. ‘Script-Fu and Plug-ins for The GIMP’ (Budigs, 2003) provides a detailed sample and explanations for programming plug-ins in C. ‘Writing a GIMP plug-in’ (Turner, 2001), provides a plug-in tutorial as well as samples in C code and explanations for the different elements of a plug-in. Another source is Mile’s O’Neal’s extended annotated randomize plug-in for the GIMP (O’Neal, 1999). I will also refer to Vera Kukic’s Honours thesis (Kukic, 2003) in which she also developed a plug-in for GIMP.

References

 Bertalmio, M., Sapiro, G., Caselles, V. & Ballester, C. “Image Inpainting” in Proceedings of ACM SIGGRAPH (2000): 417-424. ACM Press. 1 Mar 2004.  Budigs, S. “Script-Fu and Plug-ins for The GIMP”. 13 Feb 2003. 1 Mar 2004.

 Drori, I., Cohen-Or, D & Yeshurun, H. “Fragment-Based Image Completion” in Proceedings of ACM SIGGRAPH (2003): 303-312. ACM Press. 1 Mar 2004.  Efros, A & Freeman, W. “Image Quilting for Texture Synthesis and Transfer” in Proceedings of ACM SIGGRAPH (2001): 341-346. ACM Press. 1 Mar 2004.

3  Kukic, V. “Investigating the Development of Colour Interpolation Techniques for GIMP” 13 November 2003. 27 February 2004.  O’Neal, M. Home Page. 26 Jan 1999. 29 Feb 2004.  The Gimp Developers’ Website “GIMP Plug-in template” (2004). 1 Mar 2004.  Turner, K. “Writing a GIMP plug-in”. 29 Jan 2001. 1 Mar 2004.

Preliminary Design Considerations The image completion facility is being developed for the GIMP because the GIMP is an open source package with much documentation regarding its development freely available. The GIMP runs most effectively on a Linux operating system so this will be used as the development environment.

The implementation language for the plug-in could be either Perl or C, but the Perl interface is not very stable. There are also far more resources available and documentation regarding the development of plug-ins using C, so that will be the choice.

Expected Results  The development of a user-friendly fragment-based image completion plug-in for the GIMP.  The development of a general tutorial and template for the development of GIMP plug-ins.

Possible Extensions  The image completion plug-in could be refined to work better over larger, more complicated areas.

4  Implementation of a second plug-in for the GIMP.