BINARY MORPHOLOGY AND RELATED OPERATIONS ON RUN-LENGTH REPRESENTATIONS Thomas M. Breuel DFKI and University of Kaiserslautern, Kaiserslautern, Germany Keywords: Mathematical morphology, binary image processing, document image analysis, layout analysis. Abstract: Binary morphology on large images is compute intensive, in particular for large structuring elements. Run- length encoding is a compact and space-saving technique for representing images. This paper describes how to implement binary morphology directly on run-length encoded binary images for rectangular structuring elements. In addition, it describes efficient algorithm for transposing and rotating run-length encoded im- ages. The paper evaluates and compares run length morphologial processing on page images from the UW3 database with an efficient and mature bit blit-based implementation and shows that the run length approach is several times faster than bit blit-based implementations for large images and masks. The experiments also show that complexity decreases for larger mask sizes. The paper also demonstrates running times on a simple morphology-based layout analysis algorithm on the UW3 database and shows that replacing bit blit morphol- ogy with run length based morphology speeds up performance approximately two-fold. 1 INTRODUCTION phology, using more complex intermediate represen- tations (Droogenbroeck, 2002). Binary morphology is an important and widely used Although some of these algorithms are competi- method in document image analysis, useful for tasks tive for gray scale morphology, they have not been like image cleaning and noise removal, (Ye et al., demonstrated to be competitive with high quality bit 2001) layout analysis, (Wong et al., 1982) skew cor- blit-based implementations for binary morphology rection, (Najman, 2004) and text line finding. (Das (Bloomberg, 2002). It remains to be seen how such and Chanda, 2001) The primary structuring elements algorithms compare to the algorithms in this paper, used in such applications are rectangular. Real-world both in performance and storage; we will not be ad- document analysis systems currently primarily rely dressing that question here. on bitblit-based implementations (Bloomberg, 2002). Bit blit-based implementations at their lowest Practical implementations take advantage of sepa- level take advantage of operations that are highly ef- rability and logarithmic decomposition of rectangu- ficient on current hardware because they are used as lar structuring elements (Bloomberg, 2002; Najman, part of many different algorithms and display opera- 2004). tions: their running time grows quadratically in the A number of other implementations and algorith- resolution of the input image; they do not take advan- mic techniques are noteworthy. Binary mathemat- tage of coherence in the input image–an almost blank ical morphology with convex structuring elements image takes the same amount of time to process as can oomputed using a brushfire-style algorithm (Vin- a highly detailed image; and operations that need to cent, 1992). Another class of algorithms is based on take into account the coordinates of individual pix- loop and chain methods (Vincent, 1992). The van els (e.g., connected component labeling) often need Herk/Gil-Werman algorithms (van Herk, 1992; Gil to decompress (at least on the fly) or use costly pixel and Werman, 1993; Gil and Kimmel, 2002) have con- access functions. stant per-pixel size overhead for grayscale morphol- This paper describes an implementation of mor- ogy, and binary morphology can be viewed as a spe- phological operators directly on run-length encoded cial case. Another class of algorithms is taking advan- binary images. Run length coding has been proposed tage of anchors, (Droogenbroeck and Buckley, 2005). previously for morphological operations(Liang et al., Some authors have looked again at grayscale mor- 1989; van den Boomgaard and van Balen, 1992), but 159 Breuel T. (2008). BINARY MORPHOLOGY AND RELATED OPERATIONS ON RUN-LENGTH REPRESENTATIONS. In Proceedings of the Third International Conference on Computer Vision Theory and Applications, pages 159-166 DOI: 10.5220/0001081501590166 Copyright c SciTePress VISAPP 2008 - International Conference on Computer Vision Theory and Applications not found much use in document image analysis. Our Therefore, an erosion with a rectangular structuring approach was developed independently of that liter- element of size u × v can be written as:2 ature, and we focus on the application of run-length def erode2d(image,u,v): methods to large, complex binary images as found erode1d(image,u) in document image analysis. We give benchmarks transpose(image) and comparisons with the Leptonica library, an open erode1d(image,v) source library for morphological image processing. transpose(image) It has comparatively good performance, uses well- documented algorithms, and is used in several large- 3.1 Within-Line Operations scale document analysis systems. There are four basic morphological operations we consider: erosion, dilation, opening, and closing. 2 RUN LENGTH IMAGE CODING One-dimensional opening and closing are the easiest to understand. Essentially, a one-dimensional open- Run-length image representations have a long history ing with size u simply deletes all runs of pixels that in image processing and analysis. They have been are less than size u large, and leaves all others un- used, for example, for efficient storage of binary and touched: color images and for skeletonization of large images. def close1d(image,u): Consider a 1D boolean array a containing pixel for i in 1,length(image.lines): values 0 and 1 at each location ai. The run line = image.lines[i] length representation r is an array of intervals r1 = filtered = [] [s1;e1];:::;rn = [sn;en] such that ai = 1 iff i 2 r j for for j in 1,length(line.runs): some j and ei < si+1. if runs[j].width() >= u: The 2D run-length representation we are using in filtered.append(runs[j]) this paper is a straight-forward, extension to 2D that image.lines[i] = filtered treats the two coordinates asymmetricaly; in partic- 1 ular, the binary image ai j is represented as a se- A one-dimensional closing with size u deletes all gaps quence of one-dimensional run-length representations that are smaller than size u, joining the neighboring intervals together. It can either be implemented di- ri j, such that for any fixed i0, the 1D array a j = ai0; j is represented by the 1D runlength representation r j = rectly, or it can be implemented in terms of comple- mentation and erosion3 ri0 j. def complement(image): for i in 1,length(image.lines): 3 MORPHOLOGICAL line = image.lines[i] filtered = [] OPERATIONS last = 0 for j in 1,length(line.runs): Because of the asymmetry in the two dimensions of run = line.runs[j] the 2D run-length representation we are using, mor- newrun = make_run(last,run.start) phological operations behave differently in the x and y filtered.append(newrun) direction in run-length representations. An analogous last = run.end asymmetry is found in bit-blit operations, in which filtered.append(make_run(last,maxint)) the bits making up image lines are packed into words, image.lines[i] = filtered and a list of lines represents the entire image. There are multiple possible approaches for dealing with this def open1d(image,u): issue. First, we can implement separate operations for complement(image) horizontal and vertical operations. Second, we can close1d(image) implement only the within-line operations and then complement(image) transform the between-line operations into within-line operations through transposition. For separable oper- 2Our convention is output arguments before input ar- ations, the second approach is often the easier one. guments, and the various procedures modify the image in place. 1This paper and our library uses 3To simplify boundary conditions, we are using the no- PostScript/mathematical conventions, with a0;0 repre- tation exp1 or exp2 to mean means use the value of exp1 senting the bottom left pixel of the image. if it is defined, otherwise use exp2. 160 BINARY MORPHOLOGY AND RELATED OPERATIONS ON RUN-LENGTH REPRESENTATIONS Note that openings and closing are not separable, so 3.2 Efficient Transpose we cannot use these implementations directly for im- plementing true 2D openings and closings; for that, Transposition means that we need to construct runs we have to combine erosions and dilations. How- of pixels in the direction perpendicular to the cur- ever, even as they are, these simple operations are al- rent run-length encoding. A simple way of transpos- ready useful and illustrate the basic idea behind run- ing is to essentially decompress each run individually length morphology: run-length morphology is selec- and then accumulate the decompressed bits in a sec- tive deletion and/or modification of pixel runs. ond run length encoded binary image (Anderson and The most important operation in run-length mor- Michell, 1988; Misra et al., 1999). For this, we main- phology is one-dimensional erosion. Like one- tain an array of currently open runs in each line of dimensional opening, we walk through the list of the output image and iterate through the runs of the runs, but instead of only deleting runs smaller than u, we also shrink runs larger than u by u=2 on current line in the input image. For the range of pix- each side (strictly speaking, for erosions on integer els between the runs of the current line in the input grids, we shrink by floor(u=2) on the left side and image, we finish off the corresponding open runs in u − floor(u=2) on the right side during erosions), and the output image. For the range of pixels overlapping use the opposite convention for dilations). In pseudo- the runs of the current line in the input image, we code, we can write this as follows: start new runs for lines where runs are not currently def erode1d(image,u): open and continue existing open runs for lines where for i in 1,length(image.lines): runs are currently open.