Master Thesis Computer Science Thesis no: MCS-2008-14 Image Retrieval Using a Combination of Keywords and Image Features Author(s) Praveen Bandikolla & Keshi Reddy Vishwanath Reddy School of Engineering Blekinge Institute of Technology Box 520 SE – 372 25 Ronneby Sweden i This thesis is submitted to the School of Engineering at Blekinge Institute of Technology in partial fulfillment of the r equirements for the degree of Master of Science in Computer Science. The thesis is equivalent to 20 weeks of full time studies. Contact Information: Author(s): Praveen Bandikolla Address: Lindblomsvagen 96, Room #549, Ronneby, Sweden. E-mail: [email protected] Vishwanath Reddy Keshi Reddy Address: Lindblomsvagen 96, Room #549, Ronneby, Sweden. E-mail: [email protected] University advisor(s): Guohua Bai, Peng Zhang School of Engineering School of Engineering Internet : www.bth.se/tek Blekinge Institute of Technology Phone : +46 457 38 50 00 Box 520 Fax : + 46 457 102 45 SE – 372 25 Ronneby Sweden ii ABSTRACT Information retrieval systems are playing an important role in our day to day life for getting the required information. Many text retrieval systems are available and are working successfully. Even though internet is full of other media like images, audio and video, retrieval systems for these media are rare and have not achieved success as that of text retrieval systems. Image retrieval systems are useful in many applications; there is a high demand for effective and efficient tool for image organization and retrieval as per users need. Images are classified into text based image retrieval and content based image retrieval, we proposed a text based image retrieval system, which makes use of ontology to make the retrieval process intelligent. We worked on Cricket World Cup 2007. We combined text based image retrieval approach with content based image retrieval, which uses color and texture as basic low level features. Keywords : Image Retrieval, Image Features, CBIR, TBIR, Ontology. 1 CONTENTS ABSTRACT ..........................................................................................................................................1 Keywords .......................................................................................................................................1 FIGURES ..............................................................................................................................................1 ABBREVIATIONS...............................................................................................................................1 ACKNOWLEDGEMENTS .................................................................................................................1 1 INTRODUCTION .......................................................................................................................1 1.1 NEED AND APPLICATIONS ...................................................................................................... 1 1.2 QUERY CHARACTERISTICS .....................................................................................................2 1.3 TEXT BASED IMAGE RETRIEVAL ............................................................................................3 1.4 CONTENT BASED IMAGE RETRIEVAL .....................................................................................4 1.5 PROBLEM DOMAIN AND AREA ...............................................................................................6 1.6 AIM AND OBJECTIVES ............................................................................................................ 6 1.7 RESEARCH METHODOLOGY ...................................................................................................7 1.8 SCOPE OF THE THESIS ............................................................................................................ 7 1.9 ORGANIZATION OF THESIS .....................................................................................................7 2 INTRODUCTION TO ONTOLOGY ........................................................................................9 2.1 SEMANTIC WEB .....................................................................................................................9 2.2 ONTOLOGY ............................................................................................................................9 2.2.1 Definition ..........................................................................................................................9 2.2.2 Components of ontology ................................................................................................. 10 2.2.3 Ontology Types and Frameworks ................................................................................... 11 2.2.4 Ontology Languages....................................................................................................... 12 2.2.5 Types of OWL ................................................................................................................. 13 2.3 ONTOLOGIZATION ................................................................................................................ 13 2.4 PROTÉGÉ .............................................................................................................................. 14 3 TEXT BASED IMAGE RETRIEVAL USING ONTOLOGY............................................... 17 3.1 TYPES OF TEXT BASED IMAGE RETRIEVAL .......................................................................... 17 3.1.1 Frequency-of-Occurrence Based Method....................................................................... 17 3.1.2 Field Based Method........................................................................................................ 17 3.1.3 Structure based method .................................................................................................. 18 3.2 AUTOMATIC IMAGE ANNOTATION AND RETRIEVAL USING ONTOLOGY .............................. 18 3.2.1 Document vector............................................................................................................. 18 3.2.2 Query Vector................................................................................................................... 19 3.2.3 Image Retrieval............................................................................................................... 21 3.2.4 Discussion....................................................................................................................... 21 3.2.5 .Ontology Development .................................................................................................. 22 4 COLOR AND TEXTURE BASED IMAGE RETRIEVAL ................................................... 23 4.1 CONTENT BASED IMAGE RETRIEVAL ................................................................................... 23 4.1.1 Definition ........................................................................................................................ 23 4.1.2 Issues .............................................................................................................................. 23 4.2 COLOR ................................................................................................................................. 24 4.2.1 Introduction .................................................................................................................... 24 4.2.2 Color Representation...................................................................................................... 24 4.2.3 Color Histogram............................................................................................................. 25 4.2.4 Image retrieval................................................................................................................ 25 4.3 TEXTURE .............................................................................................................................. 26 4.3.1 Introduction .................................................................................................................... 26 2 4.3.2 Texture Representation................................................................................................... 27 4.3.3 Gray Level Co-occurrence Matrix (GLCM) ................................................................... 28 4.3.4 Texture Filter Functions ................................................................................................. 29 4.3.5 Image Retrieval............................................................................................................... 31 5 INTEGRATION OF TBIR WITH CBIR ................................................................................ 32 5.1 CRICKET WORLD CUP 2007 ONTOLOGY .............................................................................. 32 5.2 IMPLEMENTATION DETAILS ................................................................................................. 32 5.2.1 Crawling ......................................................................................................................... 32 5.2.2 JENA API........................................................................................................................ 32 5.2.3 Offline Work.................................................................................................................... 33 5.2.4 Online Work...................................................................................................................