A Clustering-Based Algorithm for Automatic Document Separation Kevyn Collins-Thompson Radoslav Nickolov School of Computer Science Microsoft Corporation Carnegie Mellon University 1 Microsoft Way 5000 Forbes Avenue Redmond, WA USA Pittsburgh, PA USA [email protected] [email protected] ABSTRACT For text, audio, video, and still images, a number of projects have addressed the problem of estimating inter-object similarity and the related problem of finding transition, or ‘segmentation’ points in a stream of objects of the same media type. There has been relatively little work in this area for document images, which are typically text-intensive and contain a mixture of layout, text-based, and image features. Beyond simple partitioning, the problem of clustering related page images is also important, especially for information retrieval problems such as document image searching and browsing. Motivated by this, we describe a model for estimating inter-page similarity in ordered collections of document images, based on a combination of text and layout features. The features are used as input to a discriminative classifier, whose output is used in a constrained clustering criterion. We do a task-based evaluation of our method by applying it the problem of automatic document separation during batch scanning. Using layout and page numbering features, our algorithm achieved a separation accuracy of 95.6% on the test collection. Keywords Document Separation, Image Similarity, Image Classification, Optical Character Recognition contiguous set of pages, but be scattered into several 1. INTRODUCTION disconnected, ordered subsets that we would like to recombine. Such scenarios are not uncommon when The problem of accurately determining similarity between scanning large volumes of paper: for example, one pages or documents arises in a number of settings when document may be accidentally inserted in the middle of building systems for managing document image another in the queue. In other cases, pages may have collections. For example, we are interested in clustering been accidentally omitted and we wish to alert the user to search results for queries on document image collections, this case also. Such examples would normally be difficult or performing near-duplicate detection for indexing and to detect without tedious manual review. other purposes. Our overall approach is most closely related to video and Another such problem is automatically finding document text segmentation methods, which are summarized in separators in an ordered collection of images, such as ones Section 2. We use features based on layout, document coming from scanning or fax transmission. For example, structure, and topic concepts to discriminate between we might wish to scan a large set of technical papers in a related and unrelated pages. This feature set is described continuous “batch” but automatically detect and retain the in Section 3. These steps in turn are used to derive an original boundaries between possibly dozens of papers, overall page-set similarity measure for clustering disjoint saving each as a separate file for ease of browsing, groups of pages, as described in Section 4. We discuss searching and retrieval. Existing imaging applications our error metric and evaluate the quality of our separation require the user to either separate the papers manually and algorithm in Section 5. Lastly, we discuss the strengths start a separate scanning job for each one; physically and weaknesses of our approach and possible areas for insert blank or special separator pages; or mark each further study in Section 6. separator while examining possibly hundreds of thumbnails looking for the first page of each paper. Since these options may be inconvenient or time-consuming, we 2. RELATED WORK seek an automated method to assist the user with reliable document separation. To be clear on terminology: we adopt the term In addition, a single document may not originate as one “document” to signify an ordered collection of images. A single image in such a document is termed a “page”. Two and retrieval. pages are “related” if they come from the same underlying document and are “unrelated” otherwise. We use the term Document image understanding is the broad research area “separation” for the process of identifying the transitions which covers methods for deriving structure from page between documents in an ordered sequence. Related work images. Haralick [Hara94] gives a general survey of work on text or video refers to the same problem as on two major sub-areas: reconstructing geometric page “segmentation” but this has a different meaning in the layout and finding logical page structure. Our application image recognition literature (e.g. “segmenting” a single makes indirect use of such methods to obtain page page into separate layout or texture elements) and so we features, but otherwise document separation can be prefer “separation” instead. considered a higher-level problem. There is a large body of existing work on estimating The problem of finding topic boundaries or transition similarity among images in general, especially for color points in content has been studied in several projects for photographs. Huang and Rui [Huang97] give a text, video, and spoken audio. For example, video comprehensive survey of general image retrieval segmentation, or ‘shot break’ detection, uses audio and techniques. In cases where a document contains very little image features to discriminate between different news text, or for sub-areas on a page, such image similarity stories. The Informedia project [Haupt95] is a notable methods may be applicable, but in this study we focus on example of such work. The TDT Story text-intensive document images. detection/segmentation [Allan98] track has also spawned work on identifying topic boundaries in text and spoken audio. For example, Beeferman et al [Beefer99] use an 3. FEATURE SELECTION exponential model based on topicality and cue-word features to partition text into coherent segments. Earlier The following general types of features form the feature work of Hearst [Hearst94] on TextTiling used a cosine space the model uses to discriminate between related and similarity measure as part of an algorithm to subdivide unrelated pages. These features are currently chosen texts into multi-paragraph subtopics. based on our knowledge of the problem and did not involve a feature selection algorithm. We are unaware of any published work on the related problem for document images: performing automatic document separation. Current document image 3.1 Document Structure management applications accomplish document separation by detecting manually placed separator (or Pages may be related by means of being labeled with “patch”) pages. Separator pages may be blank, or coded meta-information. The three types of meta-information with barcode or other special identifying markers. features we use in this algorithm are headers, footers, and Separator pages take time to print, insert between each page numbers. We identify these features by cross- document, and remove afterwards. They also increase the correlating text elements between pages, with each such volume of paper to be scanned, especially with many short feature being used as a component in the separation documents. Some applications allow the operator to view feature space. a field of thumbnails in order the mark the initial pages by hand. For headers and footers, we define a region of interest at the page top and bottom. Within that region, we look for Also, unlike video, audio, or text segmentation, the pages closely-matching lines of text between pages in that in a document collection may also require re-ordering or region. We form text element correlations spanning recombining, either because the pages were presented out- multiple pages, using these correlations as feature space of-order to the system originally, as with a batch scan, or elements. We detect page number correlations in a similar because the pages are coming from a process such as full- way. text search. In either case, the problem is more difficult than just partitioning an ordered set: we must also do These features require OCR processing. Because the text clustering. The clustering may be made easier because we from OCR may contain errors, we use the approximate can constrain based on the context, and we discuss this is string matching technique described in [Collins01] when Section 4. forming the text element correlations. Related work on document image similarity includes a document image classifier described by Shin and 3.2 Layout Structure Doermann [Shin00]. This assigns a page image to one of 12 general classes of layout structure, such as cover page, For text intensive images, which is the main area of form, and so on, but does not attempt to determine if two interest, we extract and use the text layout information. page images have come from the same document. We aim to reconstruct the word and line structure of the Doermann et al [Doer97] describe a method for detecting original document. In its crudest form, which turns out to duplicate and near-duplicate document images which does be rather effective, this process requires no more than the not rely on OCR, using a robust signature of shape codes word bounding boxes. Our feature space is derived using based on representative lines of text. Doermann [Doer98] probability distributions of the following text elements: gives a survey of techniques for document image indexing 1. Word