Digital Ecosystems and Business Intelligence Institute A Customized Semantic Service Retrieval Methodology for the Digital Ecosystems Environment Hai Dong This thesis is presented for the Degree of Doctor of Philosophy of Curtin University of Technology June 2010 Declaration To the best of my knowledge and belief this thesis contains no material previously published by any other person except where due acknowledgment has been made. This thesis contains no material which has been accepted for the award of any other degree or diploma in any university. Signature: …………………………………………. Date: ………………………... TABLE OF CONTENTS TABLE OF CONTENTS ................................................................................................. 1 LIST OF FIGURES .......................................................................................................... 7 LIST OF TABLES ............................................................................................................ 9 THESIS SUMMARY ...................................................................................................... 12 ACKNOWLEDGEMENT .............................................................................................. 15 LIST OF PUBLICATIONS ........................................................................................... 16 CHAPTER 1 - INTRODUCTION ................................................................................. 19 1.1 INTRODUCTION ............................................................................................................................ 19 1.2 DIGITAL ECOSYSTEMS AND SERVICES IN DIGITAL ECOSYSTEMS ................................................. 19 1.2.1 Digital Ecosystems ............................................................................................................. 19 1.2.2 Features of Services in Digital Ecosystems ........................................................................ 20 1.3 ISSUES WITH SERVICE RETRIEVAL IN DIGITAL ECOSYSTEMS ....................................................... 21 1.3.1 Issues with Service Retrieval with Commercial Search Engines ........................................ 22 1.3.2 Issues with Service Retrieval in Research Literature ......................................................... 26 1.3.3 Issues with Service Retrieval in Digital Ecosystems .......................................................... 26 1.4 OBJECTIVE OF THE THESIS ........................................................................................................... 27 1.5 SCOPE OF THE THESIS .................................................................................................................. 28 1.6 SIGNIFICANCE OF THE THESIS ...................................................................................................... 28 1.7 PLAN OF THE THESIS .................................................................................................................... 29 1.8 CONCLUSION ............................................................................................................................... 31 1.9 REFERENCES ................................................................................................................................ 31 CHAPTER 2 – LITERATURE REVIEW .................................................................... 33 2.1 INTRODUCTION ............................................................................................................................ 33 2.2 SEMANTIC CRAWLERS ................................................................................................................. 33 2.2.1 Metadata Abstraction Crawlers ......................................................................................... 36 2.2.2 Semantic Focused Crawlers ............................................................................................... 41 2.2.3 Metadata Harvesting Crawlers .......................................................................................... 46 2.2.4 Metadata Abstraction Focused Crawlers ........................................................................... 47 2.2.5 Metadata Harvesting and Abstraction Crawlers ................................................................ 48 2.3 SEMANTIC SERVICE DISCOVERY/MATCHMAKING ....................................................................... 49 2.3.1 Semantic Service Discovery in Centralized Environments ................................................. 51 2.3.2 Semantic Service Discovery in P2P Environments ............................................................ 60 2.3.3 Semantic Service Discovery in Grid Computing Environments ......................................... 64 2.3.4 Semantic Service Discovery in Ubiquitous Computing Environments ............................... 68 2.4 SEMANTIC SEARCH ENGINES AND RELATED TECHNOLOGIES ...................................................... 71 2.4.1 Related Searches Engines and Technologies ..................................................................... 72 1 | Page 2.4.2 Semantic Search Engines and Technologies for Reference Results ................................... 73 2.4.3 Search Engines and Technologies for Semantically Annotated Results ............................. 74 2.4.4 Full-text Similarity Search Engines and Technologies ...................................................... 75 2.4.5 Search Engines and Technologies on Semantic/Syntactic Annotations ............................. 76 2.4.6 Concept Search Engines and Technologies ....................................................................... 80 2.4.7 Ontology-based Search Engines and Technologies ........................................................... 83 2.4.8 Semantic Web Search Engines and Technologies .............................................................. 88 2.4.9 Faceted Search Engines and Technologies ........................................................................ 95 2.4.10 Clustered Search Engines and Technologies ....................................................................100 2.4.11 Natural Language Search Engines and Technologies.......................................................102 2.5 SEMANTIC SIMILARITY MODELS .................................................................................................104 2.5.1 Edge (Distance)-based Models .........................................................................................105 2.5.2 Node (Information Content)-based Models .......................................................................107 2.5.3 Hybrid Models ...................................................................................................................108 2.6 CRITICAL EVALUATION OF EXISTING TECHNOLOGIES: AN INTEGRATIVE VIEW .........................110 2.6.1 Service Information Discovery in Digital Ecosystems ......................................................111 2.6.2 Service Retrieval in Digital Ecosystems ............................................................................112 2.6.3 Service Recommendation in Digital Ecosystems ...............................................................113 2.6.4 QoS Evaluation and Service Ranking in Digital Ecosystems ............................................114 2.6.5 Service Domain Knowledge Updating in Digital Ecosystems ...........................................115 2.7 CONCLUSION ..............................................................................................................................116 2.8 REFERENCES ...............................................................................................................................117 CHAPTER 3 - PROBLEM DEFINITION ................................................................. 136 3.1 INTRODUCTION ...........................................................................................................................136 3.2 KEY CONCEPTS ...........................................................................................................................136 3.2.1 Digital Ecosystems/Digital Ecosystems environment ........................................................136 3.2.2 Species in Digital Ecosystems ...........................................................................................137 3.2.3 Services in the Digital Ecosystems Environment ..............................................................138 3.2.4 Generic Services in the Digital Ecosystems Environment .................................................138 3.2.5 Service Requester and Service Provider in Digital Ecosystems ........................................138 3.2.6 Digital Ecosystem Service Information .............................................................................138 3.2.7 Service Description Entity (SDE) ......................................................................................138 3.2.8 Ontology ............................................................................................................................139 3.2.9 Ontology Mark-up Language ............................................................................................139 3.2.10 Metadata ...........................................................................................................................139 3.2.11 SDE Metadata ...................................................................................................................139 3.2.12 Service Factory .................................................................................................................139