Object-Oriented Image Model

William I. Grosky and Peter L. Stanchev

Department of , Wayne State University

Detroit, MI 48202 {grosky, stanchev}@cs.wayne.edu

Abstract: In this paper we analyze some existing framework for describing multimedia presentation, but not tools and approaches to image modeling and we the semantic richness of the images. propose an Object-Oriented Image Data (OOID) model. It can be applied on a wide variety of image collections. THE OOID MODEL DESCRIPTION The model employs multiple logical representations of an image. The logical image representation can be viewed as An Image is a type of image data a multiple level abstraction of the physical image . abstraction that is used to provide a conceptual image The OOID model is based on the analysis of different representation. It is a set of concepts that can be used to image application domains such as: medical images, house describe the structure of an image. The process of image furnishing design plans, electronic schema catalogues and description consists of extracting the global image geographical information systems. The proposed model characteristics, recognizing the image-objects and could be used as a frame for designing and building a assigning a semantic to these objects. Approaches to wide range of image database systems and can be included image can be categorized based on the as a part of the MPEG-7 standard. It can be treated as an views of image data that the specific model supports. extension of the image database models [3, 4]. An example for applying the OOID model to a plant picture Some valuable proposals for image data models are: database is given, as well as the realization of the model in VIMSYS image data model, model where images are 2 the Sofia Image Database System. presented as four plane layers; EMIR - an extended model for image representation and retrieval; and AIR - an Key words: Data models, image adaptive image retrieval model. The proposed OOID model establishes taxonomy Images are becoming an essential part of the based on the systematization on the existing approaches. information systems and multimedia applications. The The main requirement to the proposed model could be image data model is one of the main issues in the design summarized as: and development of any image database management • powerfulness. To be applicable to a wide system. The data model should be extensible and have the variation of image collections; expressive power to present the structure and contents of the image, their objects and the relationships among them. • to consider the characteristics of the images The design of an appropriate image data model will and image-objects as different types of data; ensure smooth navigation among the images in an image • to consider different types of relations among database system. The complexity of the model arises the image-objects; because images are richer in information than text, and • to allow different kind of functions over the because images can be interpreted differently, according physical and logical image description. to the human perception of the application domain. There is a lack of standard model for representing the The data model is object oriented. The image itself semantic richness of an image. Most traditional together with its semantic descriptions is treated as an approaches to image data management use multilevel object in terms of the object oriented approach. The abstraction mechanisms to support content-based retrieval image is presented in two layouts (classes) - logical and [1]. These levels contain: semantic modeling and physical. The image data are defined as a composition of knowledge representation, object recognition and feature the physical view - the image itself, and the logical view - extraction. The most relevant standards for multimedia a description of the image content and additional data models are: HTML, SMIL, MHEG-5, Hy Time [2]. information about the image. The logical view of a given ISO MHEG-5 defines a hierarchy of classes featuring image is defined as the description of the global image attributes, actions, and events. These classes constitute a characteristics and the recognized image-objects and the semantics associated with them. The structured part of

this information can be used in the image indexing and approach assumes that there exists some prior knowledge creating an image retrieval mechanism. There are two (model) about the types and the structure of the image- main approaches to the logical image description: based objects that can be part of the image. In this approach on the global image characteristics and based on the predefined image-objects are extracted from the image image content. In the global view approach the image and the relationships between them are studded. The content is described with the use of a list of attributes. following properties of the image-objects are analyzed: Most of the available image database systems are using color, texture patterns, shape, logical attributes, and this approach for image description. Two kinds of semantic attributes. The following types of relations attributes: meta and semantic attributes are used in this between the image-objects are considered in the OOID approach for global image description. An alternative to model: topological, vector, metric, and spatial. The the global image description is based on the visual image general-purpose view treats the image as a whole and content. The image content-based view describes the measures their global attributes such as color and texture. image-objects properties such as: color, texture patterns, The physical view contains the pixel matrix of the image shapes, image-object attributes and relevant location to and its header. each other of these image-objects. For the content-based A semantic schema of the proposed model is shown in description two approaches: model based and general- Figure 1. Figure 2 shows a possible class description for purpose based approaches are used. The model-based the proposed model.

Image

Logical view Physical view

Global view Content-based view Image Image matrix header

Meta attributes Semantic attributes Model-based view General purpose view

Texture Color

Logical Semantic Objects Image-object relations Spatial relationship

Color Texture Shape Topological Vector Metric relationship relationship relationship

Legend 0,1 to N 1 to 1 1 to N 0,1 to 1 Class inheritance Object class Figure 1. Semantic schema of the OOID model

CLASS PROPERTY NAME DOMAIN NAME Image Header Image name Name # pixels in X direction Integer; values {0 to 255}; format 999 # pixels in Y direction Integer; values {0 to 255}; format 999 FOV in X direction in cm Float FOV in Y direction in cm Float # bytes per pixel Integer; values {1,2}; format 9 Pixel organization Specification Compression schema Specification table Image type Specification table Image Matrix Image name Name Image values Integer Vector; values {0 to 255}; format 999 Meta attributes Name Name Author Person Name … Semantic Name Name attributes Quality Specification table …

Logical Logical Method1 Name Logical Method1 Domain …

Color Color Method1 Name Color Method1 Domain …

Texture Texture Method1 Name Texture Method1 Domain …

Shape Shape Method1 Name Shape Method1 Domain …

Topological Topological Method1 Name Topological Method1 Domain relationship …

Vector Vector Method1 Name Vector Method1 Domain relationship …

Metric Metric Method1 Name Metric Method1 Domain relationship …

Spatial Spatial Method1 Name Spatial Method1 Domain relationship … Figure 2. Class description for the OOID Model

AN EXAMPLE FOR APPLYING THE OOID possible view as a result of applying the OOID model to MODEL the example image is given in Figure 3. At present a realization of the model for Windows 2000 is considered in the Sofia Image Database Management Let’s consider an image of a plant picture. After the System. An example for applying the OOID model segmentation procedure the image is partitioned in the through the Logical Image Definition Language in the following image-objects: blossom, stalk, leaf, root. A

system is shown in Figure 4. It provides an idea about the their associated set of operations. schema constructors in terms of grammar definition, and

PHYSICAL VIEW Global view Class Image header CLASS: META ATTRIBUTES CLASS: SEMANTIC NAME VALUE ATTRIBUTES image name herb 101 Attribute Attribute Attribute Attribute # of pixels in x direction 185 name Value name Value # of pixels in y direction 485 name Rions - tooth use-in- big # of pixels in z direction 1 medicine

FOV in x direction in cm 6 source book 102 OBJECT 1 2 3 4 FOV in y direction in cm 12.5 Class remark page 32 FOV in z direction in cm NULL Vector 1 * NE NE NE distribution grass-lend # bytes per pixel 1 relationship 2 * NE N blossoming [March, period November] pixel organisations RGB 3 * NW compression schema no compression 4 * image type TIFF 400

300 Class Image matrix 200

(237,225,247) … (246,226,237) HU1XPE RI SRLQ W V 100

--- 0 0 50 100 150 200 ,QW HQ VLW \ (24,245,245) … (245,227,238) Class: colour; Method: histogram; Return value: Class: texture; Method: contrast; Return value: 1.38

Class: meta attributes Class: meta attributes Class: meta attributes Class: meta attributes Attribute name: name Attribute name: name Attribute name: name Attribute name: name Attribute value: blossom Attribute value: stalk 1 Attribute value: leaf 1 Attribute value: root Class: colour Class: colour Class: colour Class: colour Method: average Method: average Method: average Method: average: Return value: yellow Return value: dark green Return value: dark green Return value: brawn Class: texture Class: texture Class: texture Class: texture Method: contrast Method: contrast Method: contrast Method: contrast Return value: 2,31 Return value: 0,91 Return value: 3,36 Return value: 3,24 Class: shape Class: shape Class: shape Class: shape Method: centroid Method: centroid Method: centroid Method: centroid Return value: (48,36) Return value: (72,194) Return value: (44,401) Return value: (101,432) Class: shape Class: shape Class: shape Class: shape Method: approximation Method: approximation Method: approximation Method: approximation Return value: circle Return value: rectangle Return value: triangle Return value: rectangle Class: logical attributes Class: logical attributes Class: logical attributes Class: logical attributes Method: length Method: length Method: length Method: length Return value: 2,1 Return value: 1,1 Return value: 2,6 Return value: 1,5 Class: logical attributes Class: logical attributes Class: logical attributes Class: logical attributes Method: with Method: with Method: with Method: with Return value: 1,6 Return value: 8,4 Return value: 1,1 Return value: 1,6 Class: semantic attributes Class: semantic attributes Class: semantic attributes Class: semantic attributes Method: form Method: form Method: form Method: form Return value: basket Return value: smoothly Return value: lanceolate Return value: spindle Class: semantic attributes Class: semantic attributes Class: semantic attributes Class: semantic attributes Attribute name: Attribute name: Attribute name: Attribute name: metamorphosis metamorphosis metamorphosis metamorphosis Attribute value: tongue Attribute value: none Attribute value: none Attribute value: none Figure 3. Example database instance of the plant image

Figure 4. An example for applying the OOID model through the Logical Image Definition Language in the Sofia Image Database Management System CONCLUSION REFERENCES The main advantages of the proposed OOID model could be summarized as follows: [1] W. Al-Khatib, Y. Francis, A. Ghafoor, P. Berra, “Semantic its generality. The model uses the main Modeling and Knowledge Representation in Multimedia • Databases”, IEEE Transactions on Knowledge and Data techniques from the existing image data models Engineering, vol. 11 N. 1, Jan./Feb. pp. 64-80, 1999. and it is applicable to a wide variety of image [2] S. Boll, W. Klas, U. Westermann, “Multimedia document collections; models, Sealed or setting out for new shores?”, IEEE Multimedia Systems ’99, vol. 1, pp. 604-610, 1999. • its practical applicability. The model can be used as a part of image retrieval and image [3] W. Grosky, P. Stanchev, “An Image Data Model”, in Advances in Visual Information Systems, R. Laurini (edt.), database system; Lecture Notes in Computer Science 1929, pp. 14-25, 2000. • its flexibility. The model could be customized [4] P. Stanchev, “General Image Database Model”, in Visual when used with a specific application. Information and Information Systems, Huijsmans, D. Smeulders A., (etd.), Lecture Notes in Computer Science The proposed model could be extended to include the 1614, pp. 29-36, 1999. description of multimedia objects such as voice and video.