Semantic Similarity in Owl

SEMANTIC SIMILARITY IN OWL

THIRD YEAR PROJECT REPORT May 2016

Project Supervisor Professor Uli Sattler

By Olumide Mosinmileoluwa Adenmosun BSc.(Hons) Computer Science with Industrial Experience University of Manchester Contents

Abstract v

Declaration vi

Acknowledgements vii

1 Introduction 1 1.1 Semantic Similarity ...... 1 1.1.1 Existing approaches ...... 1 1.2 Why is this tool needed? ...... 2 1.3 Project goals ...... 2 1.4 Report structure ...... 2

2 Background 4 2.1 The Semantic Web ...... 4 2.2 What is an ontology? ...... 4 2.3 Description Logics ...... 5 2.4 OWL ...... 6 2.4.1 Overview ...... 6 2.4.2 The OWL API ...... 7 2.4.3 OWL Reasoners ...... 7 2.5 Protégé ...... 8 2.6 The Similarity Measures ...... 9 2.6.1 Atomic Similarity (AtomicSim)...... 10 2.6.2 Subconcept Similarity (SubSim)...... 10 2.6.3 Grammar Similarity (GrammarSim)...... 10

i CONTENTS ii

3 Design 12 3.1 Design approach ...... 12 3.2 Project Requirements ...... 13 3.2.1 Functional requirements ...... 13 3.2.2 Non-functional requirements ...... 13 3.3 Project Architecture ...... 14 3.4 Development Decisions ...... 14 3.4.1 Protégé 4(P4) ...... 15 3.4.2 OWL API ...... 15 3.5 Plugin Architecture ...... 16 3.6 UI Designs ...... 17 3.6.1 Plugin interface design ...... 17 3.6.2 Debuggr! ...... 18 3.7 Summary ...... 18

4 Implementation 19 4.1 Implementing the Similarity Measures ...... 19 4.1.1 Atomic Similarity ...... 19 4.1.2 Subconcept Similarity ...... 21 4.2 Debuggr! ...... 23 4.3 Developing the Plugin UI ...... 24 4.4 Developing the manager class ...... 25 4.5 Challenges ...... 26

5 Results 27 5.1 Walkthrough and Functional Requirements ...... 27 5.2 Non-functional requirements ...... 30

6 Testing and Evaluation 31 6.1 Manual Testing ...... 31 6.2 Unit Testing ...... 31 6.3 Evaluating the Similarity Measures ...... 32 6.3.1 Results ...... 33

7 Conclusions 36 7.1 Summary ...... 36 7.2 Achievements ...... 36 7.2.1 Things I would have like to do ...... 37 7.3 Future work ...... 37 7.4 Final remarks ...... 37

A NCI Atomic Sim Output 40 List of Figures

2.1 Protégé user interface ...... 8

3.1 Component interactions ...... 14 3.2 Initial Plugin Architecture ...... 16 3.3 The plugin design ...... 17 3.4 The Debuggr! design ...... 18

4.1 The ﬁnished Debuggr! design ...... 23

5.1 Plugin tab view ...... 27 5.2 Reasoner not initialised ...... 28 5.3 Selecting classes using the class selection panel ...... 28 5.4 Classes not correctly selected ...... 28 5.5 Selecting a similarity measure ...... 29 5.6 Results of the similarity calculation ...... 29

6.1 Results from the NCI group ...... 34 6.2 Results from the SNOMED group ...... 35

List of Listings

4.1 Pseudo-code for getting subsuming classes ...... 20 4.2 Pseudo-code for AtomicSim calculations ...... 20 4.3 Approach 1 - Pseudo-code for collecting subsuming class expressions ...... 21 4.4 Approach 1 - Pseudo-code for SubSim calculations ...... 22

iii 4.5 Approach 2 - Pseudo-code for adding subsuming class to the ontology ...... 22 4.6 Pseudo-code for selection listener ...... 24 4.7 Pseudo-code for update selection method ...... 25 4.8 Pseudo-code to parse class expression ...... 25 4.9 Example class expression in functional OWL syntax ...... 25

List of Tables

6.1 Table of Results ...... 33 6.2 Pearson coeﬃcient between Expert values and similarity measures ...... 33

iv Abstract

Diﬀerent disciplines have taken varying views of the concept of similarity. With the increase in data, extension of the principles of the world wide web to the retrieval of data is the intention of the semantic web. There are many possible applications of semantic similarity that make a standard framework for calculation of semantic similarity measurements im- portant. This report details the design and implementation of the aforementioned similarity measurements in a plugin for Protégé as a tool for calculating the aforementioned similarity in OWL ontologies. This tool is built as a plugin in ontology editor, Protégé. An evaluation of the similarity measures is also carried out. Project Title: Semantic Similarity in OWL. Author: Olumide Mosinmileoluwa Adenmosun. Degree: BSc(Hons) in Computer Science with Industrial Experience. Supervisor: Uli Sattler.

v Declaration

No portion of the work referred to in the thesis has been submitted in support of an application for another degree or qualiﬁcation of this or any other university or other institute of learning.

vi Acknowledgements

Thank you Uli for the encouragement, help and for explaining the tricky stuff. I would also like to give huge thanks to Nico for answering my questions and giving advice on implementing said tricky stuff. This has been an interesting academic year. I am so thankful for the help and support I have received from so many these last few months. Thanks to my project mates who made the project sessions fun and enjoyable, offering advice on everything from test cases to LATEXtemplates. Last and definitely not least I would like to say a huge thanks to Tiny, Beaw and M. Your love, support and encouragement got me through the year. P.S. I finished!!!

vii Chapter 1

Introduction

1.1 Semantic Similarity

Semantic Similarity between two terms or sets of documents is deﬁned as the degree of "sameness" between the terms as measured by comparing the information describing their properties.

1.1.1 Existing approaches

Existing semantic similarity measures are classiﬁed into two categories[MPL+14]: Path-based measures based on consideration of the shortest path between the terms in a taxonomy, i.e., a tree or a possibly acyclic directed graph[Als15, MPL+14].

• Rada et al.[RMBB89] is a measure that traverses the shortest path between two concepts. The length is calculated by counting the number of edges between the two concepts through their least common subsumer(LCS), 1 + sim(c1,c2) = minpath(c1,c2) [MPL 14]. The LCS is the closest common ancestor with the the minimum number of IS-A links with concepts c1 and c2), [Sli13]. • Wu and Palmer extends this considering the depth of the LCS in the calculation[MPL+14]. 2∗depthT oRoot(LCS) sim(c1,c2) = depthT oLCS(c1)+depthT oLCS(c2)+2∗depthT oRoot(LCS) [Sli13].

1 CHAPTER 1. INTRODUCTION 2

Information content-based measures based on measurement of probability of a concept occurring concept in a hierarchy. The supposition being the more frequent a concept appears the less specific it is. Therefore, a concept with a high IC value is more specific to a topic than a low IC value. IC is defined as ICc = −logPc, where the Pc is the probability of encountering an instance of it.[Als15, MPL+14] In [Als15], Alsubait identifies the shortcomings of these similarity measures. Path-based approaches are imprecise as they only consider atomic subsumption ignoring any complex subsumers. The other measures impose requirements on the ontology such as low expressivity. A new family of measures is required to precisely measure the semantic similarity of terms in ontologies. These individual measures vary in their accuracy and computational cost based on what features they consider[APS15].

1.2 Why is this tool needed?

1.3 Project goals

The goals of the project are to:

• Implement similarity tool as a Protégé plugin. The objective was to build a protégé plug in that would allow any users to calculate the semantic similarity between classes in an OWL ontology

• A second objective was to test and evaluate the correctness of the results of the similarity measures implemented.

1.4 Report structure

This section gives an overview of the structure of this document and a summary of information in the following chapters of the report Chapter 2 provides relevant background information on tools and technolo- gies used. I also further explain the similarity measures to be implemented as deﬁned in [Als15] CHAPTER 1. INTRODUCTION 3

Chapter 3 gives information about design decisions, requirements(functional and non-functional), and the architecture of the deliverable. Chapter 4 gives details about the technical aspects behind the component development and implementation. Chapter 5 details the achievements of the project. I give a walkthrough of the finished product and compare it against the functional and non-functional requirements defined. Chapter 6 details the testing and evaluation of the finished product. Chapter 7 concludes the report giving my final view on the achievements of the project, possible future work and things I would have done differently. Chapter 2

Background

This chapter provides a high level explanation of the tools and technology used in the development of the Semantic Similarity deliverable to achieve the objectives outlined in Chapter 1. Terms used in the report are deﬁned and explained here.

2.1 The Semantic Web

The intention of the Semantic Web1 is to the extend the principles of the World Wide Web[FH09] and make data easily accessible using general Web technology. The Semantic Web provides a common framework for data to be "shared and reused across application, enterprise, and community bound- aries"[FH09].

2.2 What is an ontology?

In the context of the Semantic Web, an ontology is a "machine-processable en- cyclopaedia"[Sat15]. It allows electronic agents to infer possible relationships between data and to resolve domain-speciﬁc user queries. Researchers and experts have developed specialised ontologies to annotate and share domain- speciﬁc information within communities[NM00, SRH10]. An example is the

1https://www.w3.org/standards/semanticweb/

4 CHAPTER 2. BACKGROUND 5

National Cancer Institute Thesaurus(NCIt)2, a public ontology that contains over 100, 000 textual deﬁnitions of terms and over 400, 000 links between concepts.

2.3 Description Logics

Description Logics(DLs)[KSH12] are "a family of knowledge representation languages that present the knowledge of an application domain in a struc- tured and formally well-understood way"[BHS08]. Most DLs are decidable fragments of ﬁrst-order logic[Bor96]. DLs are equipped with formal, logic based semantics and are widely used in ontological modelling. These semantics, "allow humans and machines (to) exchange DL ontologies without any ambiguity in meaning and also the use of logical deduction to infer additional information from explicitly stated facts"[KSH12]. In DLs there are three kinds of entities:

• Concepts represent the set of individuals. We represent the set of atomic concepts with NC. This includes the Top concept > and sometimes the Bottom concept ⊥[Als15].3

• Individuals represent single individuals within the domain[Bor96]. We represent the set of instances with NI . • Roles represent the relationships between individuals. We represent the set of roles within the domain with NR.

Complex concepts/expressions can be constructed from atomic concepts and possibly some roles with the help of constructors (e.g. conjunction, nega- tions, existential restrictions e.t.c)[Als15]. An example of such is, ¬Doctor, an expression describing the set of individuals that are not doctors. Subconcepts are concepts "that syntactically occur in a possibly complex concept"[Als15]. For example, the concept, Male u Coach, contains the subconcepts: Male,Coach, and Male u Coach.

2https://ncit.nci.nih.gov/ncitbrowser/ 3In OWL, we call > "Thing" and ⊥ "Nothing" CHAPTER 2. BACKGROUND 6

DL ontologies are comprised of a state of statements called axioms. These statements describe the relationships between concepts, individuals and roles. Axioms can be classiﬁed into three groups[KSH12]:

• Assertional axioms(ABox) that describe relationships between named individuals and concepts or between the individuals and roles[Als15]. For example the axiom, Lion(Simba), asserts that the individual named Simba is an instance of the concept Lion.

• Terminological axioms(TBox) that describe the relationships between concepts. For example the axiom, Lion v Carnivores, states that every instance of the concept Lion is also an instance of the concept Carnivore.

• Relational axioms(RBox) that describe the relationships between roles. For example the axiom4, brotherOf ◦parentOf v uncleOf, states that any individual that is a brother to another individual that is a parent is an uncle.

The DL knowledge base K is a tuple K = (T ∪ R,A) that comprises of the three groups where T is a TBox, R is a RBox and A is an ABox[Als15].

2.4 OWL

2.4.1 Overview

The Web Ontology Language(OWL)5 is a web language that brings the expressiveness and reasoning power of Description Logics to the Semantic Web. It was designed to represent complex information about things, collections of things and the relationships that exist within speciﬁc application domains. OWL is part of the World Wide Web Consortium’s(W3C)6 semantic web stack which also include RDF, RDFS, SPARQL. It was designed to be compatible with other W3C standards such as the eXtensible Markup Lan-

4Example taken from [KSH12][p.4] 5https://www.w3.org/2001/sw/wiki/OWL 6The W3C is an international community that maintains open standards for the World Wide Web. CHAPTER 2. BACKGROUND 7 guage(XML) and extends the RDF(Resource Description Framework) and the RDFS(Resource Description Framework Schema)[H+07]. In OWL ontologies:

• Concepts are referred to as classes. • Class Expressions refer to the (possibly complex) concepts that are present within the ontology therefore it follows that classes are also class expressions. • Individuals are also referred to as individuals. • Roles are referred to as properties and can be broken down into two groups. Object properties that describe the relationships between objects and Datatype properties that relate objects to built in datatypes[Als15]

2.4.2 The OWL API

The OWL API[HB11] is used to create and interact with OWL Ontologies. It is open-source project developed at the University of Manchester. It is implemented in Java and is primarily targeted towards OWL DL with a number of design decisions and considerations reﬂecting this[BM14]. It provides data structures that allow users model and manipulate OWL ontologies and also provides a reasoner interface allowing for "a representation that implements/understands the formal semantics of the language"[BM14]. This means that the reasoner is able to "listen" for changes to the ontologies it is reasoning over and will also respond to user queries with respect to the changed ontologies[Hor09].

2.4.3 OWL Reasoners

Reasoners are software tools that infer implicit knowledge from explicitly stated information[HB11]. The OWL API includes[HB11] an OWLReasoner interface to allow access to OWL reasoners. A large number7 of reasoners provide implementation of the OWLReasoner interface. Some examples are Pellet[SPG+07], HermiT[GHM+14], FaCT++[TH06]

7Full list of reasoners available at: http://owl.cs.manchester.ac.uk/tools/ list-of-reasoners/ CHAPTER 2. BACKGROUND 8 and JFact8. Reasoners can be used in ontologies to perform the following[Obi07]:

• Check that an individual is an instance of a class.

• Fetch all individuals that are instances of a class.

• Find all classes that an individual might belong to.

• Determine whether a class C subsumes another class D or not, that is, whether a class is a subset of another class.

• Check that the individuals in the ABox do not violate descriptions of axioms in the TBox, verifying the consistency of the ontology.

• Determine the satisﬁability of classes by determining whether its description is not contradictory, that is, whether an individual can exist that would be an instance of the concept.

These tasks are for the most part semantically similar, for example, satisﬁ- ability can be proved by checking the subsumption of ⊥. A reasoner may however have optimised algorithms for each diﬀerent tasks[Obi07].

2.5 Protégé

Figure 2.1: Protégé user interface

8http://jfact.sourceforge.net/ CHAPTER 2. BACKGROUND 9

Figure 2.1 shows the Protégé User Interface, Protégé9 is an open-source ontology editor developed at Stanford University. It was developed to be modular and use the OSGi framework10 as the basis of its plugin infrastructure to achieve this. Protégé was developed to be compatible with the OWL API and use it for ontology modelling and querying. It has OWL reasoners imple- mentations that are built as plugins. Protégé has a core API that contains many reusable UI elements and utility classes for plugin development. Within Protégé there are two possible ways to view the class hierarchy off an ontology. The first is the asserted class hierarchy which shows the subclass hierarchy that can be obtained directly from the ontology. Child nodes are subsumed by the parent(s) and anything without a parent shows up under the root node(>). The second, the inferred class hierarchy provides a more complete view of this by using the reasoner infer more relationships. In addition, the bottom concept(⊥) is added to this view. This class becomes the parent of every class within the ontology that is unsatisfiable, that is, classes that can have no instances. The current release of Protégé was developed in collaboration with the Uni- versity of Manchester.

2.6 The Similarity Measures

With the basic definition of ontologies established, the rest of this chapter provides a definition of Similarity Measure. In [Als15], Alsubait defines a function that calculates the similarity between two concepts in an ontology by finding the set of common subsumers of the concepts and dividing the cardinality of this set by the set of all subsumers of both concepts. However rather than a count of all subsumers of a concept, we considering the set of (possibly complex) concepts of a concept language L. This is because it is possible for a concept to have an infinite number of subsumers. The equation can be represented as such:

|Com(C,D,O,L)| Sim(C,D,O,L) = |Union(C,D,O,L)|

Where C and D are concepts, O represents an ontology, L(O˜) represents a

9http://protege.stanford.edu/ 10https://www.osgi.org/developer/architecture/ CHAPTER 2. BACKGROUND 10 concept language deﬁned over the signature11 of an ontology O, Com(C,D,O,L) represents the set of common subsumers of concepts C and D, Union(C,D,O,L) represents the set of all subsumers of concepts C or D. The results of the similarity calculation are number values between zero and one, with one in- dicating strong similarity between the concepts.

2.6.1 Atomic Similarity (AtomicSim)

As deﬁned in [Als15], atomic similarity is a taxonomy-based measure where only the atomic concepts of the ontology are captured. It can be presented by the equation[Als15]:

AtomicSim(C,D) = Sim(C,D,O,LAtomic(O˜)), and LAtomic(O˜) = O˜ ∩ NC.

2.6.2 Subconcept Similarity (SubSim)

As deﬁned in [Als15], subconcept similarity allows for more speciﬁc measurements by being more sensitive to the views/focus of the modeller. This focus can be captured in the subconcepts of the ontology. It is presented[Als15]: ˜ ˜ SubSim(C,D) = Sim(C,D,O,LSub(O)), andLSub(O) = Sub(O). where Sub(O) is the set of (possibly complex) concept expressions in O. SubSim(·) provides a cheap, principled way of measuring similarity since the number of candidates is only linear in the size of the ontology[Als15].

2.6.3 Grammar Similarity (GrammarSim)

As deﬁned in [Als15], GrammarSim(·) can be represented by:

GrammarSim(C,D) = Sim(C,D,O,LG(O˜)), and

LG(O˜) = {E | E ∈ Sub(O) or E = ∃r.F,for some r ∈ O˜ ∩ NR and F ∈ Sub(O)}.

This similarity measure is used to capture subsumers that SubSim(·) is not able to. Consider this example12, an ontology O with:

11This is the union of the sets of concepts, individuals and roles 12Taken from Alsubait[Als15, p.117] CHAPTER 2. BACKGROUND 11

T Box = {A v ∃r.B} RBox = {r v s} The set of subconcepts that are identiﬁed from the ontology is: Sub(O) = {>,A,B,∃r.B} The concept expression ∃s.B, is not captured by SubSim(·) but will be considered a relevant subsumer for GrammarSim(·). Chapter 3

Design

This chapter describes the decisions made in regard to the design of the Semantic Similarity plugin. It will cover the approach to development and give a broad overview of the software architecture of the deliverable.

3.1 Design approach

This is the largest personal project I have undertaken to date. I adopted an agile approach to development to ensure that the articulated project objectives were met, milestones were achieved and the overall project was delivered on time. The requirements gathering and implementation phases did not occur in sequential blocks but rather concurrently in regular iterations. An advantage of this approach was flexibility allowed by working at a high level of abstraction. Design decisions could be put off until alternative options and consequences were analysed while development continued. Development occurred in weekly iterations. This allowed that tasks could be broken down into smaller, specific goals and completed within the iteration time-frame. This approach to task decomposition allowed gaps in understanding to be identified, these could then be addressed in a focused manner. Continuous learning led to improvements in domain knowledge which allowed requirement definitions to be reviewed while new functionality was implemented and integrated into the deliverable, problem areas and opportunities for improvement identified.

12 CHAPTER 3. DESIGN 13

3.2 Project Requirements

The functional and non-functional requirements of the ﬁnal product are outlined in this section.

3.2.1 Functional requirements

The functional requirements of the plugin were that:

FR1. The user must be able to navigate the ontology within the plugin. FR2. The user must be able to select classes within the plugin. FR3. The user must only able to select two, diﬀerent classes within the plugin. FR4. The plugin must only make use of the inferred class hierarchy. FR5. The user must be able to select any of the implemented similarity measures in the plugin. FR6. The user must be able to view results of similarity measure.

3.2.2 Non-functional requirements

The non-functional requirements that sought after were:

NFR1. Usability: The plugin should have simple use patterns and intuitive behaviour. NFR2. Scalability: The plugin should have good performance when dealing with very large ontologies. NFR3. Testability: Every component that is part of the plugin should be testable. NFR4. Portability: The plugin should be portable and retain all functionality on compatible versions of Protégé. NFR5. Robustness: The plugin should recover from errors and unpredictable behaviour from the Protégé environment. CHAPTER 3. DESIGN 14

3.3 Project Architecture

Figure 3.1: Component interactions

Figure 3.1 depicts the major components of the system and a high-level view of the interactions that occur between them. The user loads an ontology into Protégé using the UI of the base application.1The base application loads and interacts with the ontology using the OWL API. The user selects a reasoner and classiﬁes the loaded ontology obtaining the inferred class hierarchy. The plugin accesses the loaded, classiﬁed ontology from the base application using the Protégé API and then uses the OWL API to interact with the ontology

3.4 Development Decisions

The OWL and Protege APIs are both written in Java, it was a therefore a logical decision to develop the deliverable in Java. I am familiar with application development in a Java ecosystem from both academic and personal experience and felt comfortable with this.

1The UI is included as part of the core Protégé API http://protege.stanford.edu/ protege/4.3/docs/api/ CHAPTER 3. DESIGN 15

I used a private Git2 repository during the development process for version control, making regular commits during iterations. This repository acted as a redundancy in case of hard drive failure or any other unforeseen events and also allowed me work on multiple features in parallel using branches.

3.4.1 Protégé 4(P4)

I chose to develop my plugin in P4. This was the most appropriate choice as Protégé 5(P5) was and is still under development with no official release.3P4 is stable and has abundance of documentation available for development. In addition, P4 provides a toolkit for plugin development that provides both the base application source code and apache ant4 build files pre-configured to run in development environments allowing the entire application to be built from source. To give users more choice over how they could interact with the plugin, I implement the plugin as both as a standalone workspace tab within Protégé and also as a view component allowing it to be embedded within other plugin workspace tabs. To run a plugin in Protégé it must be packaged as a JAR5. I created a build file to compile and package the source code.

3.4.2 OWL API

To interact with the ontology I used the OWL API. Alternatives to this are the Jena API6 and the Protégé-OWL7 API. The Jena API is primarily used to interact with RDF graphs. While it provides support for OWL, its reasoner interfaces are logically incomplete so are not capable of inferring all information as speciﬁed in the OWL speciﬁcation. The Protégé-OWL API is an extension of the Core API and provides class interfaces for both RDF and OWL. Its Reasoning API was discontinued in favour of the OWL API for all modelling and querying functionality.

2https://git-scm.com/docs 3This is true as of the 30th of April, 2016 4http://ant.apache.org/ 5https://ant.apache.org/manual/Tasks/jar.html 6https://jena.apache.org/ 7http://protegewiki.stanford.edu/wiki/ProtegeReasonerAPI CHAPTER 3. DESIGN 16

3.5 Plugin Architecture

Figure 3.2: Initial Plugin Architecture

Three main component groups emerged from early plugin design(Figure 3.2):

• A UI Class: To allow the user interact with the system and view results of the similarity measures.

• A Manager Class: The manager class would act as an interface with the base Protégé application. It would hold a reference to the loaded, classiﬁed ontology in Protégé. It would also process user queries from the UI Class, pass the formed query to the relevant similarity measure.

• The Similarity Measure Classes: To query the reasoner and perform the similarity calculation and return the result to the Manager Class.

This architecture allowed separation of design logic into speciﬁc classes and simpliﬁed the development process. CHAPTER 3. DESIGN 17

3.6 UI Designs

3.6.1 Plugin interface design

To ensure that the interface integrate well with the Protégé ecosystem, I used elements from the Protégé core API where possible. The API has very robust components for ontology navigation and the design sought to integrate these views into the tab layout.

Figure 3.3: The plugin design

Figure 3.3 shows the initial design of the plugin UI without any core API components. I would use these API components to handle class selection and ontology navigation while the plugin interface would allow the user view selected classes, select the similarity measure to be used and view the results of the similarity calculation. CHAPTER 3. DESIGN 18

3.6.2 Debuggr!

While developing the similarity measures, I realised that I was initially unable to actually test and evaluate their behaviour. I was continuously improving my knowledge of the Protégé and OWL APIs and did not at the initial stage have a user interface to interact with the ontologies. To get around this, I created a simple Java application that allowed me to load ontologies, classify them with a reasoner and then select classes.

Figure 3.4: The Debuggr! design

Figure 3.4 shows the initial design of this application. I named this application "Debuggr!" as its sole purpose was to aid the development of similarity measures and allow detection of errors and unexpected behaviour during the development of the similarity measures.

3.7 Summary

This chapter introduced the iterative agile approach adopted for this project and the reasoning behind this. I also described the initial project architecture and design decisions, the considerations for the selection of speciﬁc tools based on better ﬁt for the design compared with alternatives. Finally, I described the initial plugin architecture, its major components and interaction between them Chapter 4

Implementation

In this chapter I will give a high level overview of the implementation of Similarity Measures and the development of the plugin. I will also discuss some of the challenges faced during implementation.

4.1 Implementing the Similarity Measures

I implemented two of the three similarity measures described in [Als15]. In this section, I will discuss the implementation process.

4.1.1 Atomic Similarity

The ﬁrst similarity measure I implemented was AtomicSim. I created an AtomicSim class, representing all information about the atomic similarity calculation as an object. Calculating the atomic similarity is straight forward. As stated in Chapter 2, the set of subsuming concepts of a class in an ontology is comprised of all the super classes of that class and itself. The OWLReasoner interface provides a convenience method that returns all superclasses of a class in an OWL ontology.

19 CHAPTER 4. IMPLEMENTATION 20 getSubsumingClasses(OWLClass class) resultSet = reasoner.getSuperClasses(class,false ).getFlattened(); resultSet.add(c); return resultSet; Listing 4.1: Pseudo-code for getting subsuming classes

As outlined in listing 4.1, the getSuperClasses() method takes an OWLClass object and a boolean1 as arguments returning the result as a NodeSet.2Using the getFlattened() method, we can get the union of all Nodes in the NodeSet returned as a Java Set. atomicSim(OWLClass class1, OWLClass class2) firstSet = getSubsumingClasses(class1); secondSet = getSubsumingClasses(class2); intersectionSet = firstSet; unionSet = firstSet; intersectionSet = firstSet.retainAll(secondSet); unionSet = firstSet.addAll(secondSet); Double atomicSimValue = intersectionSet.size() / (double) unionSet.size(); Listing 4.2: Pseudo-code for AtomicSim calculations

As indicated in listing 4.2, the atomic similarity between two classes is calculated by:

• Getting the set of subsuming classes for both classes.

• Finding the intersection and union of the sets. Operations on Sets can be carried out using convenience methods from the Java Collections Framework3.

• Dividing the value of the size of the intersection set by the value of the size of the union set. The result of this division is the atomic similarity between the classes. To ensure integer division does not occur, I cast the value of the size of the union set as a double. 1You supply the boolean argument "true" to fetch only the direct superclasses and "false" to fetch all superclasses 2http://owlapi.sourceforge.net/javadoc/org/semanticweb/owlapi/reasoner/NodeSet.html 3https://docs.oracle.com/javase/tutorial/collections/ CHAPTER 4. IMPLEMENTATION 21

4.1.2 Subconcept Similarity

The second similarity measure implemented was SubSim. There are two approaches to calculating the subconcept similarity of two classes however, the ﬁrst step is identical in both. All subconcepts in the OWL ontology are extracted from its signature, the OWL API provides a class for this task. The OWLClassExpression class has a method that collects the nested class expressions within some OWLObject, for example, given the class expression,ObjectUnionOf(C D), the collector would return a Set containing the class expressions C, D and ObjectU- nionOf(C D).

Approach 1 - Iterative approach getSubsumingClassExpr(OWLClass class) setOfAllClassExprs = collector.visit(ontology); for(ClassExpression exp : setOfAllClassExprs) if (exp subsumes class) resultSet.add(exp); return resultSet; Listing 4.3: Approach 1 - Pseudo-code for collecting subsuming class expressions

The listing 4.3 shows the process behind this approach. After the subconcepts in the ontology have been collected, we iterate over the elements in the set checking if the class expressions subsume the class with the reasoner. If the expression subsumes the class it is added to the result set. CHAPTER 4. IMPLEMENTATION 22 iterativeSubSim(OWLClass class1, OWLClass class2) firstSet = getSubsumingClassExpr(class1); secondSet = getSubsumingClassExpr(class2); intersectionSet = firstSet; unionSet = firstSet; intersectionSet = firstSet.retainAll(secondSet); unionSet = firstSet.addAll(secondSet); Double subSimValue = intersectionSet.size() / (double) unionSet.size(); Listing 4.4: Approach 1 - Pseudo-code for SubSim calculations

The listing 4.4 shows the process behind calculating the subconcept similarity. The set of subsuming class expressions for both classes is found, their intersection and union sets obtained and the values used to calculate the SubSim value.

Approach 2 - Reasoned approach

Again, all the subconcepts in the ontology are extracted from its signature. After obtaining the set of subconcepts for the class we calculate the Sub- concept similarity between two classes adopting a process similar to that of AtomicSim. addingClassExpr() setOfAllClassExprs = collector.visit(ontology); for(ClassExpression exp : setOfAllClassExprs) if exp.isAnonymous() create new named axiom equivalent to exp; add axiom to ontology; Listing 4.5: Approach 2 - Pseudo-code for adding subsuming class to the ontology

As seen in listing 4.5, the anonymous4 class expressions are given names and added to the ontology. This adds these axioms to the set NC, changing the state of the ontology and allowing them to be found by the reasoner. The advantage of this approach over the iterative is that once the axioms have

4These are concepts(usually complex) that are not named, that is, they are not members of the set, NC . CHAPTER 4. IMPLEMENTATION 23 been added to the ontology once that step never needs to be repeated. The reasoner is used to obtain the set of superclasses in this approach. As the anonymous classes have now been named, the named complex concepts are identiﬁed by the reasoner and returned in the set. Again, to obtain the set of subsumers the class itself is added to the returned set. To calculate the subconcept similarity, the subsuming class expressions of the two classes are obtained, sets of intersection and union found and the values of these sets used to calculate their subconcept similarity.

4.2 Debuggr!

As explained in the previous chapter, the Debuggr! application enabled me to develop and test my similarity measures while I concurrently developed the user interface for the plugin. The architecture of the Debuggr! mimicked the plugin, with a manager class, a UI class and separate classes for each similarity measure. However as this was a stand alone application, the manager class did not interface with Protégé but directly loaded the ontology using the OWL API.

Figure 4.1: The ﬁnished Debuggr! design

The Debuggr! UI(Fig 4.1) was built with Java Swing5. It was developed as a simple JFrame application. A JTextField allowed the ontology location

5https://docs.oracle.com/javase/tutorial/uiswing/start/index.html CHAPTER 4. IMPLEMENTATION 24 be supplied either as a local path or as a URL. On load, a new instance of the OWLOntologyManager would be instantiated and the supplied path would be passed to the OWLOntologyManager. The instance would then attempt to load an OWLOntology object. If the path did not link to an Ontology Document, the OWLOntologyManager would throw an error.

4.3 Developing the Plugin UI

The SemSimViewComponent was developed using Java Swing6. To fulﬁl ease of use requirements and ensure the interface was familiar to Protégé users, UI elements from the Protégé API were used where possible. In this section, I will go over the implementation of the plugin UI. This component can be divided into two parts:

• The results panel contains a JTextArea which is updated when the similarity measure is run, a radio button group to select similarity measures and the "Calculate" button. The second is the selection panel, this contained an instance of the

• The selection panel contains an instance of the ClassSelectionPanel class. The Protégé API provides an OWLEntitySelectorPanel UI class that allows users select all entities contained within the ontology. I made two modiﬁcations to this class, the ﬁrst was to restrict the displayed entities to classes only. This means that only the classes hierarchy is displayed to users for selection. The second was to create an updateSelection methods. selectionChanged() OWLEntity selection = getOWLWorkspace(). getOWLSelectionModel().getSelectedEntity(); classSelectionPanel.updateSelection(selection); Listing 4.6: Pseudo-code for selection listener

6http://docs.oracle.com/javase/7/docs/technotes/guides/swing/ CHAPTER 4. IMPLEMENTATION 25 updateSelection(OWLEntity entity) if (entity instanceof OWLClass) classSelectionPanel.setSelection((OWLClass) entity ); Listing 4.7: Pseudo-code for update selection method

I added a OWLSelectionModelListener to the selection panel. This meant that when the selected entity changed(listing 4.6) in the workspace, the ClassSelectionPanel would be updated(listing 4.7) if that entity was a class. This let me take advantage of the class hierarchy views and search functionality included in the base Protégé application.

4.4 Developing the manager class

The SemSimManager contains information about the ontology and the reasoner. When an ontology is loaded into the Protégé environment, the reference to that ontology is stored in the manager class also when a reasoner is started a reference to that reasoner object is also stored. The plugin uses the SemSimManager to fetch information about the ontology. It also contains convenience methods for parsing user output. convertToManchesterSyntax(OWLClassExpression e) create new ManchesterOWLSyntaxOWLObjectRenderImpl r; String s = r.render(e); return s; Listing 4.8: Pseudo-code to parse class expression

ObjectIntersectionOf(person ObjectSomeValuesFrom(has_pet animal)) Listing 4.9: Example class expression in functional OWL syntax

Listing 4.8 shows pseudo-code for a method that used to render class expressions in the Manchester Syntax7. The class expression in listing 4.9, for example, would be "person and (has_pet some animal)" in the Manchester

7https://www.w3.org/TR/2009/NOTE-owl2-manchester-syntax-20091027/ CHAPTER 4. IMPLEMENTATION 26

Syntax.

4.5 Challenges

The Protégé 4 development kit is bundled with version 3.4.28 of the OWL API. This is a much older version of the API released in December 2012. This meant I could not take advantage of some of the functionality and improvements present in the newer releases. However, this version of the API allowed me to adequately accomplish the plug in design goals. However, it was surprisingly challenging to find developer documentation for this version of the OWL API. Since the release of v3.4.2, the web home of the OWL API migrated from Sourceforge9 to GitHub. As a result, links to version specific documentation were oft-times broken. For general reference, I could make use of documentation from newer versions however I quickly realised some methods had been deprecated and functionality altered significantly in some cases. Un- derstanding these deprecated methods was time consuming and sometimes very frustrating. There was however a reasonable amount of information that could be gleaned from the mailing lists10 that assisted with this and I was also supported by knowledgeable individuals within the Computer Science department.

8https://sourceforge.net/projects/owlapi/files/OWL%20API%20%28for% 20OWL%202.0%29/3.4.2/ 9https://sourceforge.net/ 10https://sourceforge.net/p/owlapi/mailman/owlapi-developer/ Chapter 5

Results

This chapter will describe the outcome and results achieved by the project deliverable by providing a walk-through of the ﬁnal product and analysing it against functional requirements and the non-functional requirements deﬁned in Chapter 3.

5.1 Walkthrough and Functional Requirements

Figure 5.1: Plugin tab view

27 CHAPTER 5. RESULTS 28

The finished plugin in its workspace tab can be seen in figure 5.1. This section details the user interaction with the different components of the finished product during a potential use case showing expected behaviours. I will also highlight where the application fulfils the functional requirements(FR).

• A user would load an ontology into Protégé using the base application user interface.

• This ontology would need to be classiﬁed with a reasoner of their choice to obtain the inferred class hierarchy using the base application.

Figure 5.2: Reasoner not initialised

• If a user attempted to use the similarity plugin before classiﬁcation, the error box in ﬁgure 5.2 would prompt them to(FR4).

Figure 5.3: Selecting classes using the class selection panel

Figure 5.4: Classes not correctly selected CHAPTER 5. RESULTS 29

• The user would then select two, different classes from the loaded ontology using the class selection panel as seen in figure 5.3. If this was not done, that is, the user selected one class twice or more than two classes, the error box in figure 5.4 would appear(FR3).

Figure 5.5: Selecting a similarity measure

• As shown in ﬁgure 5.5, the user would then select one of the three implemented similarity measures(FR5) and perform a calculation by pressing on the "Calculate!" button.

Figure 5.6: Results of the similarity calculation CHAPTER 5. RESULTS 30

• As shown in ﬁgure 5.6, the semantic similarity between the two selected classes as calculated by the similarity plugin would be visible in the results(FR6) section.

5.2 Non-functional requirements

The non-functional requirements of the project that were identiﬁed as crucial in the design were met:

NFR1. Usability: The plugin was designed to be intuitive to native Protégé users. I believe it achieves this goal

NFR2. Scalability: The plugin is scalable and similarity measures can be run on large ontologies like the NCI thesaurus.

NFR3. Testability: All components of the plugin that I developed were testable. The plugin is portable(NFR4) and can be installed on other instances of Protégé outside the development environment. This is done by copying the plugin JAR artefact to the plugin directory of the Protégé instance. The plugin retained normal behaviour when tested on even newer versions of Protégé application that were equipped with OWL API v3.1.

1Tested on Protégé 5 beta-17 running OWL API 3.5.1 Chapter 6

Testing and Evaluation

This chapter will give an overview of the testing practices adopted over the project lifecycle along with the evaluations made of the ﬁnal product.

6.1 Manual Testing

Testing the similarity measure was done manually. This entailed using the Protégé plugin and the inferred class hierarchy to obtain the set of subsumers of different classes to create a test bed of 10 classes. After this, I created different test pairs with these classes and a count of the sets present in the union and intersections and manual calculation of similarity values was used verify the correctness of results of the tool. This was difficult to do with subconcept similarity as the FaCT++ reasoner oft-times did not pick up some subconcepts leading to tests being failed.

6.2 Unit Testing

I used unit tests to confirm the OWL methods were working as expected during each iteration during development. I evaluated changes in development and confirmed that the tool was functioning where possible. The main unit Tests were to confirm the similarity measures were receiving the right set of subsumers from the reasoner. I added test cases to check what hap- pened when the class "Nothing" was supplied and ensured all classes and

31 CHAPTER 6. TESTING AND EVALUATION 32 subconcepts in the ontology were returned.

6.3 Evaluating the Similarity Measures

The goal of the project was to implement a basic similarity tool in the form of a Protégé plugin. This chapter evaluates the correctness of the tool that has been developed based oﬀ of the similarity measures implemented. To evaluate the implemented similarity measures, the results of the tool was compared with (human) expert-based judgements and two taxonomy based semantic similarity measures. In [PPPC07], thirty pairs of medical terms are scored by human experts. The experts are three physicians that specialise in rheumatology and nine medical coders trained to classify clinical diagnoses using HICDA1, a disease coding system. Both groups of experts, score the pairs on a 4 point scale, practically synonymous(4), related(3), marginally related(2), and unrelated(1) and determine the average within the group. To obtain a value for the expert results, I added the average values of the two groups and divided the addition by two; to obtain the average of the average each group and devided by 2 average of the average value from both groups, divide this value 4 giving me a value between 0 and 1. For example, the pair "Heart" and "Myocardium", has an average score of 3.3 from the physician group and 3.0 from the coder group. Taking the average of these values gives me a result of 3.15. Dividing this result by 4 gives me an expert value 0.79(2 s.f.) for this pair. In [Als15], Alsubait performs the sam test using the SNOMED CT2. This ontology is described as "the most comprehensive and precise health termi- nology in existence for the clinical environment"[CCS+97]. At the time of testing in [Als15], it contained 397,924 classes. I obtained similarity results for this test using the NCIt3. This ontology contains terms relating to general health care research and clinical care. It is a smaller ontology containing 117,257 classes.4 In both tests the measures Rada et. al[RMBB89] and Wu and Palmer[] are used. I calculated the values for these measures manually using the inferred

1Hospital International Classiﬁcation of Diseases Adaptation 2Systematized Nomenclature of Medicine – Clinical Terms http://systems.hscic. gov.uk/data/uktc/snomed 3National Cancer Institute Thesaurus http://nciterms.nci.nih.gov 4NCIt version 16.01d, published 29th January 2016 CHAPTER 6. TESTING AND EVALUATION 33 hierarchy in Protégé to trace the atomic subsumers of the medical terms in the NCIt. Out of the 19 terms tested in [Als15], only 11 were included in the test. This was because some terms in the pairs were not present in the ontology.

6.3.1 Results

Term 1 Term 2 Expert NCI|AS SCT|AS NCI|SS SCT|SS SCT|Rad. NCI|Rad. SCT|Wu. NCI|Wu. Heart Myocardium 0.79 0.20 0.88 0.13 0.87 0.93 0.13 0.93 0.20 Delusion Schizophrenia 0.65 0.71 0.18 0.67 0.14 0.80 0.13 0.30 0.80 Congestive Pulmonary 0.55 0.41 0.41 0.41 0.39 0.71 0.08 0.58 0.33 Heart Failure edema Stomach Diarrhea 0.45 0.33 0.24 0.29 0.17 0.81 0.11 0.38 0.40 cramps Mitral Atrial 0.45 0.12 0.51 0.06 0.51 0.72 0.09 0.68 0.15 Stenosis ﬁbrillation Diabetes Hypertension 0.38 0.10 0.22 0.03 0.16 0.64 0.10 0.36 0.17 Mellitus Multiple Psychosis 0.25 0.17 0.29 0.10 0.14 0.86 0.14 0.44 0.36 sclerosis Appendicitis Osteoporosis 0.25 0.30 0.16 0.18 0.11 0.46 0.13 0.27 0.43 Alcoholic Xerostomia 0.25 0.11 0.25 0.04 0.22 0.52 0.08 0.40 0.14 cirrhosis Peptic Myopia 0.25 0.11 0.30 0.05 0.25 0.77 0.08 0.47 0.13 ulcer disease Table 6.1: Table of Results

Table 6.1 shows the results from the application of the similarity values on the considered medical terms using the diﬀerent similarity measures. Cal- culating Pearson’s coeﬃcient for the relation between the expert values and each pair of measures. I got the following:

Pearson’s coeﬃcient(r) SCT SS 0.69 SCT AS 0.67 SCT Wu & Palmer 0.62 SCT Rada et al. 0.61 NCI SS 0.56 NCI AS 0.52 NCI Wu & Palmer 0.29 NCI Rada et al. 0.04

Table 6.2: Pearson coeﬃcient between Expert values and similarity measures CHAPTER 6. TESTING AND EVALUATION 34

The results of the SubSim were the highest in both groups, followed by the AtomicSim, Wu and Palmer and ﬁnally Rada et. al. This followed the expected pattern as the measures that correlated more strongly with the expert judgements used more information from the ontology. Analysing the groups separately. Comparing the results between both groups, I believe the results from the SNOMED CT show better correlation with the expert judgements because of it is modelled after general medicine and clinical terms, while the NCI might be modelled with a bias towards cancer research.

Figure 6.1: Results from the NCI group

Figure 6.1 shows the results from the NCI group against the expert results, as we can see almost all measures tend to be underestimations of the expert judgement with Rada et. al showing almost no correlation with the expert values. CHAPTER 6. TESTING AND EVALUATION 35

Figure 6.2: Results from the SNOMED group

Figure 6.2 shows the results from the SNOMED group against the expert results, as we can see both SubSim and AtomicSim tend to be underestimations of expert judgement while the other measures tend to be overestimations. Comparing the results here to those from [Als15] the Pearson coeﬃcient is reduced signiﬁcantly. This probably due to the fact that almost half of the medical term pairs from those results were not used in this test. Chapter 7

Conclusions

This final chapter gives a summary of how well the project met the goals set out in Chapter 1. I will look at what has been achieved, possible future paths and conclude the report with my final reflections.

7.1 Summary

7.2 Achievements

This project set out to create a basic tool to calculate semantic similarity between classes in an ontology. At the end, I believe I was able to successfully achieve both the personal and project goals I had set at the beginning of the project while learning about ontologies, OWL and concepts regarding similarity. The key achievements of the project can be summarised as follows. I was able to:

• Build a simple semantic similarity plugin in Protégé that allows users compare the similarity between two diﬀerent classes in an ontology. • Deliver a well thought out project; employing good software development practices along the way. • Learn how to ask productive questions and have meaningful interactions. • Give presentations on non-trivial information.

36 CHAPTER 7. CONCLUSIONS 37

7.2.1 Things I would have like to do

While I regard the project as a success, there are several things I would have liked to have in the ﬁnished deliverable:

• - a progress bar for similarity calculations. The implemented similarity measures take between 0.5 - 5 seconds to be implemented. For calculations in larger ontologies this can more than 20 seconds. For these it would be nice to have a way of informing the user of the measures progress while Protégé is unresponsive.

• - implementation of the GrammarSim measure. The ability to compare textitGrammarSim with the other implemented measures would have allowed for assessment of the computational cost of this against its similarity results.

7.3 Future work

A possible future step identified in[Als15] is that of "weighted similarity". During my project, I identified that the similarity measure could not dis- tinguish between identical similarity results where the set of union of the subsumers had differently cardinalities. For example, a similarity result of 0.5 from the comparison of two classes where the cardinality of the union set is 2 would be exactly the same as the results from a calculation where the cardinality of the union set was 40. Intuitively, the latter result should be "more" similar because there are more concepts in the sets of subsumers. Work is currently being done by a PhD student within the Computer Science department regarding the development of weighted similarity measures as a refinement to the family of measures implemented in this report.

7.4 Final remarks

I spent a lot of time learning about similarity in OWL ontologies while developing this project and have come away with a greater admiration for Computer Science research. I have learnt about the great that come from general-purpose semantic similarity measures and the diﬀerent approaches that are being taken towards their creation. I have also learnt about the CHAPTER 7. CONCLUSIONS 38 beneﬁts of their application to reasearchers and also the public in general, for example, a researcher looking for a paper on dblp1 could also be shown semantically similar papers giving them access to more information and improving the quality of their research. I have greatly enjoyed this project and I hope that is visible from the report.

1http://dblp.uni-trier.de/ Appendix

39 Appendix A

NCI Atomic Sim Output

Out of a combined 10 superclasses Out of a combined 7 superclasses The classes share the following (2) subsumers: The classes share the following Anatomic Structure, System, or (5) subsumers: Substance Psychiatric Disorder Thing Disease or Disorder Disease, Disorder or Finding Schizophrenia and Other Psychotic The class "Myocardium" has Disorders the following (6) unique subsumers: Thing Myocardium Muscle Tissue The class "Delusional Disorder" Striated Muscle Tissue has the following (1) unique sub- Soft Tissue sumers: Tissue Delusional Disorder Connective and Soft Tissue The class "Schizophrenia" has The class "Heart" has the fol- the following (1) unique subsumers: lowing (2) unique subsumers: Schizophrenia Heart Atomic Semantic similarity is: 0.71 Organ Atomic Semantic similarity is: 0.20

40 APPENDIX A. NCI ATOMIC SIM OUTPUT 41

Out of a combined 22 superclasses Out of a combined 15 superclasses

The classes share the following The classes share the following (9) subsumers: (5) subsumers: Disorder by Site Sign or Symptom Respiratory and Thoracic Disorder Finding Disease or Disorder Finding by Site or System Thoracic Disorder Disease, Disorder or Finding Non-Neoplastic Thoracic Disorder Thing Non-Neoplastic Disorder Non-Neoplastic Disorder by Site The class "Stomach Cramp" has the Disease, Disorder or Finding following (6) unique subsumers: Thing Stomach Cramp Musculoskeletal Finding The class "Pulmonary Edema" Abdominal Cramp has the following (7) unique sub- Muscle Cramp sumers: Connective and Soft Tissue Finding Pulmonary Edema Symptom Respiratory System Disorder Respiratory Failure The class "Diarrhea" has the Acute Respiratory Failure following (4) unique subsumers: Non-Neoplastic Lung Disorder Diarrhea Lung Disorder Digestive System Signs and Symp- Non-Neoplastic Respiratory Disor- toms der Digestive System Finding Disturbance in Bowel Function The class "Congestive Heart Atomic Semantic similarity is: 0.33 Failure" has the following (6) unique subsumers: Non-Neoplastic Cardiovascular Disorder Cardiovascular Disorder Heart Disorder Congestive Heart Failure Non-Neoplastic Heart Disorder Heart Failure Atomic Semantic similarity is: 0.41 APPENDIX A. NCI ATOMIC SIM OUTPUT 42

Out of a combined 17 superclasses Out of a combined 21 superclasses

The classes share the following The classes share the following (2) subsumers: (2) subsumers: Disease, Disorder or Finding Disease, Disorder or Finding Thing Thing

The class "Atrial Fibrillation" The class "Diabetes Mellitus" has the following (8) unique sub- has the following (14) unique subsumers: sumers: Disorder by Site Pancreatic Disorder Non-Neoplastic Cardiovascular Endocrine Pancreas Disorder Disorder Disorder by Site Arrhythmia Non-Neoplastic Endocrine Disorder Cardiovascular Disorder Diabetes Mellitus Disease or Disorder Digestive System Disorder Atrial Fibrillation Disease or Disorder Non-Neoplastic Disorder Endocrine System Disorder Non-Neoplastic Disorder by Site Metabolic Disorder Non-Neoplastic Disorder The class "Mitral Valve Steno- Non-Neoplastic Disorder by Site sis" has the following (7) unique Non-Neoplastic Disorder by Special subsumers: Category Finding Glucose Metabolism Disorder Cardiovascular System Finding Carbohydrate Metabolism Disorder Finding by Site or System Other Finding The class "Hypertension" has Mitral Valve Stenosis the following (5) unique subsumers: Stenosis Hypertension Valvular Stenosis Finding Atomic Semantic similarity is: 0.12 Cardiovascular System Finding Finding by Site or System Blood Pressure Finding Atomic Semantic similarity is: 0.10 APPENDIX A. NCI ATOMIC SIM OUTPUT 43

Out of a combined 17 superclasses Out of a combined 19 superclasses

The classes share the following The classes share the following (1) subsumers: (1) subsumers: Thing Thing

The class "Acne" has the fol- The class "Cortisone" has the lowing (10) unique subsumers: following (7) unique subsumers: Acne Physiology-Regulatory Factor Disorder by Site Drug, Food, Chemical or Biomedi- Dermatosis cal Material Non-Neoplastic Skin Disorder Adrenal Hormone Dermatitis Cortisone Disease or Disorder Hormone Skin Disorder Corticosteroid Non-Neoplastic Disorder Glucocorticoid Non-Neoplastic Disorder by Site Disease, Disorder or Finding The class "Total Knee Replace- ment" has the following (11) unique The class "Syringe" has the fol- subsumers: lowing (6) unique subsumers: Surgical Procedure Medical Device Orthopedic Surgical Procedure Diagnostic, Therapeutic, or Re- Clinical or Research Activity search Equipment Intervention or Procedure Package Type Activity Syringe Surgical Procedure by Site or Sys- Device tem Manufactured Object Therapeutic Procedure Atomic Semantic similarity is: 0.06 Arthroplasty of the Knee Total Knee Replacement Arthroplasty Therapeutic Surgical Procedure Atomic Semantic similarity is: 0.05 APPENDIX A. NCI ATOMIC SIM OUTPUT 44

Out of a combined 30 superclasses Out of a combined 34 superclasses

The classes share the following The classes share the following (1) subsumers: (3) subsumers: Thing Disease or Disorder Disease, Disorder or Finding The class "Colonoscopy" has Thing the following (6) unique subsumers: Clinical or Research Activity The class "Stage 0 Laryngeal Colonoscopy Cancer" has the following (29) Endoscopic Procedure unique subsumers: Intervention or Procedure Otolaryngologic Disorder Activity Laryngeal Disorder Gastrointestinal Endoscopy Disorder by Site Head and Neck Disorder The class "Cholangiocarcinoma" Neck Disorder has the following (23) unique sub- Carcinoma sumers: Carcinoma In Situ Disorder by Site Head and Neck Neoplasm Bile Duct Adenocarcinoma Laryngeal Neoplasm Bile Duct Carcinoma Neck Neoplasm Adenocarcinoma Neoplasm Bile Duct Neoplasm Neoplasm by Site Biliary System Disorder Precancerous Condition Carcinoma Throat Cancer Digestive System Disorder Head and Neck Carcinoma Disease or Disorder Epithelial Neoplasm Digestive System Neoplasm Malignant Head and Neck Neo- Neoplasm plasm Neoplasm by Site Neoplasm by Morphology Epithelial Neoplasm Laryngeal Carcinoma Hepatobiliary Disorder Malignant Neck Neoplasm Cholangiocarcinoma Neck Carcinoma Neoplasm by Morphology Neoplasm by Special Category Malignant Digestive System Neo- Common Neoplasm plasm Malignant Laryngeal Neoplasm Disease, Disorder or Finding Common Carcinoma Glandular Cell Neoplasm Cancer-Related Condition Malignant Hepatobiliary Neoplasm Laryngeal Carcinoma by AJCC v6 Hepatobiliary Neoplasm Stage Bile Duct Disorder Stage 0 Laryngeal Cancer Digestive System Carcinoma Laryngeal Carcinoma by AJCC v7 Atomic Semantic similarity is: 0.03 Stage

The class "Lymphoid Hyper- plasia" has the following (2) unique subsumers: Lymphoid Hyperplasia Hyperplasia Atomic Semantic similarity is: 0.09 APPENDIX A. NCI ATOMIC SIM OUTPUT 45

Out of a combined 18 superclasses Out of a combined 20 superclasses

The classes share the following The classes share the following (3) subsumers: (6) subsumers: Disease or Disorder Disorder by Site Disease, Disorder or Finding Disease or Disorder Thing Non-Neoplastic Disorder Non-Neoplastic Disorder by Site The class "Multiple Sclerosis" Disease, Disorder or Finding has the following (12) unique sub- Thing sumers: Nervous System Disorder The class "Osteoporosis" has Immune System and Related Disor- the following (9) unique subsumers: ders Musculoskeletal System Disorder Disorder by Site Non-Neoplastic Connective and Non-Neoplastic Central Nervous Soft Tissue Disorder System Disorder Connective Tissue Disorder Non-Neoplastic Nervous System Non-Neoplastic Bone Disorder Disorder Connective and Soft Tissue Disor- Autoimmune Disease der Central Nervous System Disorder Metabolic Disorder Multiple Sclerosis Osteoporosis Immune System Disorder Non-Neoplastic Disorder by Special Non-Neoplastic Disorder Category Non-Neoplastic Disorder by Site Metabolic Bone Disorder Autoimmune Nervous System Dis- order The class "Appendicitis" has the following (5) unique subsumers: The class "Psychosis" has the Intestinal Disorder following (3) unique subsumers: Non-Neoplastic Digestive System Psychiatric Disorder Disorder Psychosis Digestive System Disorder Schizophrenia and Other Psychotic Appendicitis Disorders Non-Neoplastic Intestinal Disorder Atomic Semantic similarity is: 0.17 Atomic Semantic similarity is: 0.30 APPENDIX A. NCI ATOMIC SIM OUTPUT 46

Out of a combined 15 superclasses Out of a combined 19 superclasses

The classes share the following The classes share the following (1) subsumers: (2) subsumers: Thing Disease, Disorder or Finding Thing The class "Rectal Polyp" has the following (9) unique subsumers: The class "Xerostomia" has the Intestinal Disorder following (4) unique subsumers: Disorder by Site Sign or Symptom Digestive System Disorder Xerostomia Disease or Disorder Finding Polyp Symptom Rectal Polyp Gastrointestinal Polyp The class "Alcoholic Cirrhosis" Colorectal Polyp has the following (13) unique sub- Disease, Disorder or Finding sumers: Non-Neoplastic Digestive System The class "Aorta" has the fol- Disorder lowing (5) unique subsumers: Disorder by Site Anatomic Structure, System, or Cirrhosis Substance Digestive System Disorder Aorta Disease or Disorder Blood Vessel Liver and Intrahepatic Bile Duct Cardiovascular System Part Disorder Body Part Alcoholic Cirrhosis Atomic Semantic similarity is: 0.07 Alcoholic Liver Disease Hepatobiliary Disorder Non-Neoplastic Disorder Non-Neoplastic Disorder by Site Non-Neoplastic Liver and Intrahep- atic Bile Duct Disorder Non-Neoplastic Hepatobiliary Dis- order Atomic Semantic similarity is: 0.11 APPENDIX A. NCI ATOMIC SIM OUTPUT 47

Out of a combined 18 superclasses

The classes share the following (2) subsumers: Disease, Disorder or Finding Thing

The class "Peptic Ulcer" has the following (7) unique subsumers: Non-Neoplastic Digestive System Disorder Disorder by Site Digestive System Disorder Disease or Disorder Peptic Ulcer Non-Neoplastic Disorder Non-Neoplastic Disorder by Site

The class "Myopia" has the following (9) unique subsumers: Sign or Symptom Myopia Finding Finding by Site or System Nervous System Finding Neurological Signs and Symptoms Sensory Manifestations Visual Manifestations Impaired Vision Atomic Semantic similarity is: 0.11 Bibliography

[Als15] T. Alsubait. Ontology-based Multiple-choice Question Genera- tion. PhD thesis, University of Manchester, 2015. (Cited on pages 1, 2, 5, 6, 7, 9, 10, 19, 32, 33, 35, and 37.)

[APS15] Tahani Alsubait, Bijan Parsia, and Ulrike Sattler. Ontology- based multiple choice question generation. KI-Künstliche Intel- ligenz, pages 1–6, 2015. (Cited on page 2.)

[BHS08] Franz Baader, Ian Horrocks, and Ulrike Sattler. Handbook of Knowledge Representation, chapter 3 - Description Logics, pages 135–179. Elsevier, 2008. [Online; last accessed 24-Apr-2016]. (Cited on page 5.)

[BM14] Sean Bechhofer and Nicolas Matentzoglu. The owl api: An introduction. http://studentnet.cs.manchester.ac.uk/pgt/ 2014/COMP60421/slides/Week4-OWLAPI.pdf, 2014. [Online; last accessed 25-Apr-2016]. (Cited on page 7.)

[Bor96] Alex Borgida. On the relative expressiveness of description logics and predicate logics. Artiﬁcial intelligence, 82(1):353–367, 1996. (Cited on page 5.)

[CCS+97] James R Campbell, Paul Carpenter, Charles Sneiderman, Simon Cohn, Christopher G Chute, and Judith Warren. Phase ii evaluation of clinical coding schemes. Journal of the American Medical Informatics Association, 4(3):238–251, 1997. (Cited on page 32.)

[FH09] Lee Feigenbaum and Ivan Herman. W3c semantic web frequently asked questions. https://www.w3.org/RDF/FAQ, 2009. [Online; last accessed 24-Apr-2016]. (Cited on page 4.)

48 BIBLIOGRAPHY 49

[GHM+14] Birte Glimm, Ian Horrocks, Boris Motik, Giorgos Stoilos, and Zhe Wang. Hermit: an owl 2 reasoner. Journal of Automated Reasoning, 53(3):245–269, 2014. (Cited on page 7.)

[H+07] Jeﬀ Heﬂin et al. An introduction to the owl web ontology language. Lehigh University. National Science Foundation (NSF), 2007. (Cited on page 7.)

[HB11] Matthew Horridge and Sean Bechhofer. The owl api: A java api for owl ontologies. Semantic Web, 2(1):11–21, 2011. (Cited on page 7.)

[Hor03] Ian Horrocks. An example owl ontology. http://www.cs.man. ac.uk/~horrocks/ISWC2003/Tutorial/examples.pdf, 2003. [Online; last accessed 01-May-2016]. (Not cited.)

[Hor09] Matthew Horridge. Owlreasoner interface implementation - owl api javadocs. http://owlapi.sourceforge.net/javadoc/ org/semanticweb/owlapi/reasoner/OWLReasoner.html, 2009. [Online; last accessed 25-Apr-2016]. (Cited on page 7.)

[KSH12] Markus Krötzsch, Frantisek Simancik, and Ian Horrocks. A description logic primer. arXiv preprint arXiv:1201.4089, 2012. (Cited on pages 5 and 6.)

[MPL+14] Bridget T McInnes, Ted Pedersen, Ying Liu, Genevieve B Melton, and Serguei V Pakhomov. U-path: An undirected path- based measure of semantic similarity. In AMIA Annual Sym- posium Proceedings, volume 2014, page 882. American Medical Informatics Association, 2014. (Cited on pages 1 and 2.)

[NM00] Natalya F. Noy and Deborah L. McGuinness. Ontol- ogy development 101: A guide to creating your ﬁrst ontology. protege.stanford.edu/publications/ontology_ development/ontology101-noy-mcguinness.html, 2000. [On- line; last accessed 28-Apr-2016]. (Cited on page 4.)

[Obi07] Marek Obitko. Reasoning - introduction to ontologies and semantic web - tutorial. http://www.obitko.com/tutorials/ ontologies-semantic-web/reasoning.html, 2007. [Online; last accessed 25-Apr-2016]. (Cited on page 8.)

[PPPC07] Ted Pedersen, Serguei VS Pakhomov, Siddharth Patwardhan, and Christopher G Chute. Measures of semantic similarity and BIBLIOGRAPHY 50

relatedness in the biomedical domain. Journal of biomedical informatics, 40(3):288–299, 2007. [Online; last accessed 25-Apr- 2016]. (Cited on page 32.)

[RDH+04] Alan Rector, Nick Drummond, Matthew Horridge, Jeremy Rogers, Holger Knublauch, Robert Stevens, Hai Wang, and Chris Wroe. Owl pizzas: Practical experience of teaching owl-dl: Com- mon errors & common patterns. In Engineering Knowledge in the Age of the Semantic Web, pages 63–81. Springer, 2004. [Online; last accessed 26-Apr-2016]. (Not cited.)

[RMBB89] Roy Rada, Hafedh Mili, Ellen Bicknell, and Maria Blettner. De- velopment and application of a metric on semantic nets. Systems, Man and Cybernetics, IEEE Transactions on, 19(1):17–30, 1989. (Cited on pages 1 and 32.)

[Sat15] Ulrike Sattler. Semantic similarity in owl. studentnet.cs.manchester.ac.uk/ugt/year3/project/ projectbookdetails.php?projectid=26357, 2015. [Online; last accessed 19-Apr-2016]. (Cited on page 4.)

[Sli13] Thabet Slimani. Description and evaluation of semantic similarity measures approaches. arXiv preprint arXiv:1310.8059, 2013. (Cited on page 1.)

[SPG+07] Evren Sirin, Bijan Parsia, Bernardo Cuenca Grau, Aditya Kalyanpur, and Yarden Katz. Pellet: A practical owl-dl reasoner. Web Semantics: science, services and agents on the World Wide Web, 5(2):51–53, 2007. (Cited on page 7.)

[SRH10] Robert Stevens, Alan Rector, and Duncan Hull. What is an ontology? http://ontogenesis.knowledgeblog.org/66, Jan- uary 2010. [Online; last accessed 24-Mar-2016]. (Cited on page 4.)

[TH06] Dmitry Tsarkov and Ian Horrocks. Fact++ description logic reasoner: System description. In Automated reasoning, pages 292– 297. Springer, 2006. (Cited on page 7.)

[WP94] Zhibiao Wu and Martha Palmer. Verbs semantics and lexical selection. In Proceedings of the 32nd annual meeting on Associ- ation for Computational Linguistics, pages 133–138. Association for Computational Linguistics, 1994. (Not cited.)