Using Knowledge Anchors to Facilitate User Exploration of Data Graphs
Editor(s): Name Surname, University, Country Solicited review(s): Name Surname, University, Country Open review(s): Name Surname, University, Country
Marwan Al-Tawila,b,*, Vania Dimitrovab, Dhavalkumar Thakkerc a King Abdullah II School of Information Technology, University of Jordan, Amman, Jordan b School of Computing, University of Leeds, Leeds, UK c Faculty of Engineering and Informatics, University of Bradford, Bradford, UK
Abstract. This paper investigates how to facilitate users’ exploration through data graphs for knowledge expansion. Our work focuses on knowledge utility – increasing users’ domain knowledge while exploring a data graph. We introduce a novel explo- ration support mechanism underpinned by the subsumption theory of meaningful learning, which postulates that new knowledge is grasped by starting from familiar concepts in the graph which serve as knowledge anchors from where links to new knowledge are made. A core algorithmic component for operationalising the subsumption theory for meaningful learning to generate explo- ration paths for knowledge expansion is the automatic identification of knowledge anchors in a data graph (KADG). We present several metrics for identifying KADG which are evaluated against familiar concepts in human cognitive structures. A subsumption algorithm that utilises KADG for generating exploration paths for knowledge expansion is presented, and applied in the context of a Semantic data browser in a music domain. The resultant exploration paths are evaluated in a task-driven experimental user study compared to free data graph exploration. The findings show that exploration paths, based on subsumption and using knowledge anchors, lead to significantly higher increase in the users’ conceptual knowledge and better usability than free explo- ration of data graphs. The work opens a new avenue in semantic data exploration which investigates the link between learning and knowledge exploration. This extends the value of exploration and enables broader applications of data graphs in systems where the end users are not experts in the specific domain.
Keywords: Data graphs, knowledge utility, data exploration, meaningful learning, knowledge anchors, exploration paths.
1. Introduction with many options (like exploring job opportunities, travel and accommodation offers, videos, music). Of- In recent years, RDF linked data graphs have be- ten, the users have no (or limited) familiarity with the come widely available on the Web and are being specific domain. When users are novices to a domain, adopted in a range of user-facing applications offering their cognitive structures about that domain are un- search and exploration tasks. In contrast to regular likely to match the complex knowledge structures of a search where the user has a specific need in mind and data graph that represents the domain. This can have an idea of the expected search result [1], exploratory a negative impact on the exploration experience and search is open-ended requiring significant amount of effectiveness, as users may be unable to formulate ap- exploration [2], has an unclear information need [3], propriate knowledge retrieval queries (users do not and is used to conduct learning and investigative tasks know what they do not know [3]). Moreover, users can [4]. There are numerous examples from exploring re- face an overwhelming amount of exploration options sources in a new domain (like in academic research and may not be able to identify which exploration tasks) to browsing through large information spaces
*Corresponding author (E-mail: [email protected]). paths are most useful; this can lead to confusion, high familiar or partially familiar domains, users serendip- cognitive load, frustration and feeling of being lost. itously learn new things that they are unaware of [17– To overcome these challenges, appropriate ways to 19]. However, not all exploration paths can be benefi- facilitate users’ exploration through data graphs are re- cial for knowledge expansion: paths may not bring quired. Research on exploration of data graphs has new knowledge to the user leading to boredom, or may come a long way from initial works on presenting bring too many unfamiliar things so that the user be- linked data in visual or textual forms [5,6]. Recent comes confused and overwhelmed [18]. studies on data graph exploration have brought to- The key contribution of this paper is a novel com- gether research from related areas - Semantic Web, putational approach for generating exploration paths personalisation, adaptive hypermedia, and human- that can lead to expanding users’ domain knowledge. computer interaction - with the aim of reducing users’ Our approach operationalises Ausbel’s subsumption cognitive load and providing support for knowledge theory for meaningful learning [20] which postulates exploration and discovery [7–9]. Several attempts that human cognitive structures are hierarchically or- have developed support for layman users, i.e. novices ganised with respect to levels of abstraction, general- in the domain. Examples include: personalising the ity, and inclusiveness of concepts; hence, familiar and exploration path tailored to the user’s interests [10], inclusive entities are used as knowledge anchors to presenting RDF patterns to give an overview of the subsume new knowledge. Consequently, our approach domain [11], or providing graph visualisations to sup- to generate exploration paths includes: port navigation [12]. However, existing work on facil- computational methods for identifying knowledge itating users’ exploration through data graphs has ad- anchors in a data graph (KADG); and dressed mainly investigative tasks, omitting important algorithms for generating exploration paths by uti- exploratory search tasks linked to supporting learning. lising the identified knowledge anchors. The exploration of a data graph (if properly as- To find possible knowledge anchors in a data graph, sisted) can lead to an increase in the user’s knowledge. we utilise Rosch’s notion of Basic Level Objects This is similar to learning through search - an emerg- (BLO) [21]. According to this notion, familiar cate- ing research area in information retrieval [13,14], gory objects (e.g. the musical instrument Guitar) are which argues that “searching for data on the Web at a level of abstraction called the basic level where should be considered an area in its own right for future the category’s members (e.g. Folk Guitar, Clas- research in the context of search as a learning activity” sical Guitar) share attributes (e.g. both have a [15]. In the context of data graphs, learning while neck and a bridge) that are not shared by members (e.g. searching/exploring has not been studied. The closest Grand Piano, Upright Piano) of another cat- to learning is research on tools for exploration of in- egory at the same level of abstraction such as Piano. terlinked open educational resources [16]. However, We have adapted metrics from formal concept analy- this is a very specific context, and does not consider sis for detecting knowledge anchors in data graphs. the generic context of learning while exploring data The KA metrics are applied on an existing data graphs in any domain. This generic learning context is DG graph, and the output is evaluated against human Basic addressed here. Level Objects in a Data Graph (BLO ) derived via The work presented in this paper opens a new ave- DG free-naming tasks. nue that studies learning through data graph explora- To generate exploration paths, we first identify the tion. It addresses a key challenge: how to support peo- closest knowledge anchor to be used as a starting ple who are not domain experts to explore data graphs point, from where we use subsumption to find a set of in a way that can lead to expanding their domain transition narratives to form a path that can expand the knowledge. We investigate how to build automated user’s knowledge. The effectiveness of our novel ex- ways for navigating through data graphs in order to ploration approach is evaluated in a study with a Se- add a new value to the exploration, which we call mantic data browser in the Music domain. Subsump- ‘knowledge utility’ - expanding one’s domain tion-based exploration paths are compared to free data knowledge while exploring a data graph1. Our earlier graph exploration. The results show that when users work showed that when exploring data graphs in un-
1 We follow definitions of utility (i.e. reducing users’ cognitive load in knowledge retrieval) and usability (i.e. supporting users’ sense making and information exploration and discovery) [103]. have followed the suggested exploration paths, the in- linked data graphs is presented in [22]. Sheiderman’s crease of their knowledge was significantly higher and seminal work on visual information seeking (overview the usability was better. first, zoom and filter, then details-on-demand) [30] is The paper is structured as follows. Section 2 posi- used to evaluate the usability and utility of these ap- tions the work in the relevant literature and presents proaches and focuses on: (i) how well these ap- relevant theories. Section 3 presents experimental and proaches generate summary of data; (ii) how well they theoretical foundations, including the application con- focus on finding relevant and important data; and (iii) text for algorithm validation and the theoretical under- what visualisation techniques are used. In [31], the au- pinning. Section 4 provides preliminaries with key thors utilise a cartographic metaphor and visual infor- definitions, followed by a formal description of met- mation seeking principles to offer overview of data rics for identifying KADG (Section 5). Sections 6 de- based on instance types, and then automatically gen- scribes an experimental study to validate the KADG erating SPARQL queries based on search and interac- metrics against human BLODG. Section 7 describes a tion with a map. The focus of their work is the entry subsumption algorithm for generating exploration point of the vast data graph the users have to explore. paths for knowledge expansion; then Section 8 pre- There are numerous approaches to support visual sents a task-driven user study to evaluate the generated SPARQL query construction and many of these works paths against free exploration. Section 9 discusses the [32–34] have similar target audience, i.e. a layman findings, and Section 10 concludes the paper. user who is unfamiliar with the domain of the data graph. These works use visualisation techniques to help layman users to browse through large data graphs. 2. Related Work For example the work in [32] introduced a graphical interface for semantic query construction which is We will review relevant research on data graph ex- based on the specification of SPARQL query language. ploration to justify the main contributions of our work, The interface allows users to create SPARQL queries and will compare to existing approaches for identify- using a set of graphical notations and editing actions. ing key entities and generating paths in data graphs. The state-of-the-art in approaches to support visual Since our work involves several evaluation steps, we SPARQL query construction is presented in [35]. The review relevant evaluation approaches. focus of these works is in supporting layman users to perform exploratory querying of RDF graphs in space 2.1. Exploration through Data Graphs and time, which themes with interactive visual query construction methods. The authors present a number Semantic data exploration approaches are divided of design principles to counter the challenges and into two broad categories: (i) visualisation approaches evaluate them in a usability study on finding maps in [22–25] and (ii) text-based semantic data browsers a historical map repository [35]. Although these ap- [26–29]. Visualisation provides an important tool for proaches hide the complexity of graph terminologies, exploration that leverages the human perception and they primarily focus on helping layman users to gen- analytical abilities to offer exploration trajectories. erate SPARQL queries instead of focusing on the These approaches, in addition to intuitiveness, focus properties of data graphs to guide users’ exploration. on the need for managing the dimensions in semantic [36] presents work to manipulate data graph properties data represented as properties, similarity and related- to guide exploration by helping users to focus on the ness of concepts. The text-based browser approaches most important bit of information about an entity first operate on semantically augmented data (e.g. tagged and then explore other related information. The ap- content) with layout browsing trajectories using rela- proach utilises encyclopaedic knowledge patterns as tionships in the underpinning ontologies. These ap- relevance criteria for selecting, organising, and visual- proaches adopt techniques from learning, human- ising knowledge. The patterns are discovered by min- computer interaction and personalisation to enhance ing the link structure of Wikipedia to build entity-cen- the data exploration experience of users. tric summaries that can be exploited to help users in exploratory search tasks. However, this approach is 2.1.1. Visualisation Approaches for Data Graph feasible in multi-knowledge domains that are built by Exploration humans (e.g. Wikipedia) and may not be feasible in A state-of-the-art review of approaches that harness specific domains with complex structures. Also, the visualisation for exploratory discovery and analysis of approach considers one level below the root, and does not cover entities at different abstraction levels. These visualization efforts are geared towards help- the facets provide many options (i.e. multiple links) ing layman users to explore complex graph structures for the users to explore. The authors in [11] proposed by hiding the complexity of semantic terminology. Sview, a browser that utilises a link pattern-based However, the effectiveness of any visualization de- mechanism for entity-centric exploration over Linked pends on the user’s ability to make sense of the graph- Data. Link patterns describe explicit and implicit rela- ical representation which in many cases can be rather tionships between entities and are used to categorise complex. Users who are new to the domain may strug- linked entities. A link pattern hierarchy is constructed gle to grasp the complexity of the knowledge pre- using Formal Concept Analysis (FCA), and three sented in the visualisation. Our approach to automati- measures are used to select the top-k patterns from the cally identify entities that are close to the human cog- hierarchy. The approach does not consider the user nitive structures can be used as complementary to vis- perspective when identifying link patterns to support ualisation approaches to simplify the data graph by exploratory search, which would make browsing chal- pointing at entities that layman users can be familiar lenging especially in unfamiliar domains. with. The prime focus of our approach is providing ex- Personalisation approaches consider the user’s pro- ploration paths to augment users’ interaction in text- file and interests to adapt the exploration to the user’s based data browsers by, which are reviewed next. needs. An approach that explicitly targets personalisa- tion in semantic data exploration using users’ interests 2.1.2. Text-based Semantic Data Browsers is presented in [46]. Recent approaches aim to im- Two types of semantic data browsers had emerged prove search efficiency over Linked Data graphs by since the early days– (i) pivoting (or set-oriented considering user interests [47] or to diversify the user browsing) browsers and (ii) multi-pivoting browsers. exploration paths with recommendations based on the In a pivoting browser, a many-to-many graph brows- browsing history [48]. A method for personalised ac- ing technique is used to help a user navigate from a set cess to Linked Data has been suggested in [49] based of instances in the graph through common links [26]. on collaborative filtering that estimates the similarity Exploration is often restricted to a single starting point between users, and produces resource recommenda- and uses ‘a resource at a time’ to navigate anywhere tions from users with similar tastes. A graph-based in the data graph [27]. This form of browsing is re- recommendation methodology based on a personal- ferred to as uni-focal browsing. Another type of ised PageRank algorithm has been proposed in [50]. browsing is multi-pivoting where the user starts from The approach in [51] allows the user to rate semantic multiple points of interest, e.g. [28], [29], [34], [37]. associations represented as chains of relations to re- A noteworthy variation of the pivoting approach is veal interesting and unknown connections between the use of facets for text-based data browsing of linked entities for personalised recommendations. datasets. Faceted browsing is the main approach for The above approaches stress the importance of tai- exploratory search in many applications. The ap- loring the exploration to users. None of them investi- proach employs classification and properties features gates users’ familiarity with the domain, which is the from linked datasets as a mean to offer facets and con- main focus of the approach we present here, where fa- text of exploration. Facet Graphs [38], gFacet [39], miliarity is related to domain understanding and and tFacet [40], are early efforts in this area. More re- knowledge expansion. Moreover, conventional per- cent attempts include Rhizomer [41], which combines sonalisation approaches suffer from the ‘cold start’ navigation menus and maps to provide flexible explo- problem – for a reliable user model to be obtained, the ration between different classes; Facete [42], a visual- users have to spend time interacting with the system to ization-based exploration tool that offers faceted fil- provide sufficient information about their interests. In- tering functionalities; Hippalus [43], which allows us- stead, we exploit the structure of a data graph to iden- ers to rank the facets according to their preferences; tify entities that are likely to be familiar to the users, Voyager [44], which couples faceted browsing with which overcomes the cold start problem. Strictly con- visualization recommendation to support users explo- sidered our approach is not personalisation, because ration; and SynopsViz [45], which provides faceted we do not dynamically adapt to the user’s knowledge browsing and filtering RDF over classes and proper- as it expands while the user browses through the data ties. Although these approaches provide support for graph. However, knowledge anchors are a way to ap- user exploration, layman users who are performing ex- proximate what entities in a domain can be familiar to ploratory search tasks to learn or investigate a new the users, and are used for generating navigation paths. topic, can be cognitively overloaded, especially when 2.2. Identifying Key Entities in Data Graphs sub-concepts and a concept has all instances of its de- scendants. However, these approaches focus on the The most recent statistics2 show that there are 3360 experimental side, and do not adopt the formal defini- interlinked, heterogeneous datasets containing ap- tions of BLO and cue validity described in [21,61] in proximately 3.9 billion facts. The volume and hetero- the context of a data graph. Our work operationalises geneity of such datasets makes their processing for the these formal definitions by developing several metrics purpose of exploration a daunting challenge. Utilisa- for identifying knowledge anchors in a data graph. tion of various computational models makes it possi- Formal Concept Analysis (FCA) is a method for ble to handle this challenge in order to offer fruitful analysis of object-attribute data tables [53], where data exploration of semantic data. Finding key entities in a is represented as a table describing objects (i.e. taxo- data graph is an important aspect of such computa- nomical concepts), attributes and their relationships. tional models and is generally implemented using on- FCA has been applied in different application areas tology summarization [52] and Formal Concept Anal- [62,63] such as Web mining and ontology engineering. ysis (FCA) [53] techniques. In Web mining, FCA based approaches have been used Ontology summarisation has been seen as an im- to improve the quality of search results presented to portant method to help ontology engineers to make the end users. For example, the work in [64] has de- sense of an ontology, in order to understand, reuse and veloped a personalised domain-specific search system build new ontologies [23,54,55]. Summarising an on- that uses logs of keywords and Web pages previously tology involves identifying the key concepts in an on- entered and visited by other persons to build a concept tology [56]. An ontology summary should be concise, lattice. More recently, FCA has been applied to con- yet it needs to convey enough information to enable struct a link pattern hierarchy to organise semantic ontology understanding and to provide sufficient cov- links between entities in a data graph [11]. In ontology erage of the entire ontology [57]. Centrality measures engineering, FCA has been used in two topics: ontol- have been used in [52] to identify key concepts and ogy construction and ontology refinement. The work produce RDF summaries. The notion of relevance in [65] uses FCA to construct ad hoc ontologies to help based on the relative cardinality and the in/out degree the user to better understand the research domain. In centrality of a node has been used in [58] to produce [66] the authors present OntoComp, an approach for graph summaries. The approach presented in [57] ex- supporting ontology engineers to check whether an ploits the structure and the semantic relationships of a OWL ontology covers all relevant concepts in a do- data graph to identify the most important entities using main, and supports the engineers to refine (extend) the the notion of relevance, which is based on relative car- ontology with missing concepts. The psychological dinality (i.e. judging the importance of an entity from approaches to basic level concepts have been formally the instances it contains) and the in/out degree central- defined for selecting important formal concepts in a ity (i.e. the number and type of the incoming and out- concept lattice by considering the cohesion of a formal going edges) of an entity. concept [67]. This measures the pair-wise similarity The closest ontology summarisation approach to between the concept’s objects based on common at- the context of our work deals with extracting key con- tributes. More recently, the work in [68] has reviewed cepts in an ontology [59,60]. It highlights the value of and formalised the main existing psychological ap- cognitive natural categories for identifying key con- proaches to basic level concepts. Five approaches to cepts to aid ontology engineers to better understand basic level objects have been formalised with FCA the ontology and quickly judge the suitability of an on- [68]. The approaches utilise the validity of formal con- tology in a knowledge engineering project. The au- cepts to produce informative concepts capable of re- thors applies a name simplicity approach, which is in- ducing the user’s overload from a large number of spired by the cognitive science notion of Basic Level concepts supplied to the user. Objects (BLO) [21] as a way to filter entities with Existing studies in ontology summarisation and lengthy labels for the ontology summary. The work in FCA utilise BLO to identify key concepts in an ontol- [60] has utilised BLO to extract ontologies from col- ogy in order to help experts to examine or reengineer laborative tags. A metric based on the category utility the ontology. They have been evaluated with domain is proposed to identify basic concepts from collabora- experts, and are applicable in tasks where the users tive tags, where tags of a concept are inherited by its have a good understanding of the domain. In contrast, we apply the notion of BLO in a data graph to identify
2 http://stats.lod2.eu/ concepts which are likely to be familiar to users who and type-property paths have been used in [74] to sup- are not domain experts. Focusing on layman users, we port querying RDF datasets. Central types and proper- provide unique contribution that adopts Rosch’s sem- ties in paths are extracted based on their centrality in inal cognitive science work [21] to devise algorithms the RDF graph and used to construct a knowledge ar- that identify (i) KADG that represent familiar graph en- chitecture of the graph. More recently, the work in tities, and (ii) BLODG which correspond to human cog- [75] proposes a serendipity model to extract paths be- nitive structures over a data graph. Crucially, these al- tween items in the graph based on novelty of items, gorithms are validated with layman users who are not which is used for serendipitous recommendations. domain experts. While several approaches address the problem of supporting users’ exploration through data graphs, 2.3. Generating Paths in Data Graphs none of them aims at supporting layman users who are not domain experts. Many of the existing approaches In data graphs, the notion of path queries uses reg- may not be suitable for layman users, who may be- ular expressions to indicate start and end entities of come confused or overloaded with too much unfamil- paths in data graphs [69]. For example, in a geograph- iar entities. None of the existing approaches offers ex- ical graph database representing neighborhoods (i.e. ploration paths to help users to expand their domain places) as entities and transport facilities (e.g. Bus, knowledge. Several approaches generate paths that Tram) as edges, the user writes a simple query such as link graph entities specified by the user, and are there- “I need to go from Place a to Place b”, and the user is fore suitable for tasks where the users are familiar with then provided with different transportations facilities the domain. Instead, we provide paths for uni-focal ex- going through different routes (paths) starting from ploration where the user starts from a single entry Place a to reach the destination Place b [69]. Another point and explores the data graph. The unique feature used notion is property paths which specify the possi- of our work is the explicit consideration of knowledge ble routes between two entities in a data graph. Prop- utility of exploration paths. We are finding entities that erty paths are used to capture associations between en- are likely to be familiar to the user and using them as tities in data graphs where an association from entity knowledge anchors to gradually introduce unfamiliar a to entity b comprises entity labels and edges [70]. entities and facilitate learning. This can enhance the However, in data graphs there are usually high num- usability of semantic data exploration systems, espe- bers of associations (i.e. possible property paths) be- cially when the users are not domain experts. There- tween the entities and ways to refine and filter the pos- fore, our work can facilitate further adoption of linked sible paths, are required. To tackle this challenge, the data exploration in the learning domain. It can also be work in [7] presented Explass3 for recommending pat- useful in other applications to facilitate the exploration terns (i.e. paths) between entities in a data graph. A by users who are not familiar with the domain pre- pattern represents a sequence of classes and relation- sented in the graph. ships (edges). Explass uses frequency of a pattern to reflect its relevance to the query. It also uses informa- 2.4. Data Exploration Evaluation Approaches tiveness of classes and relationships in the pattern to indicate its informativeness by adding the informa- In the context of ontology summarisation, there are tiveness of all classes and relationships. Relfinder 4 two main approaches for evaluating a user-driven on- [71] provides an approach for helping users to get an tology summary [54]: gold standard evaluation, where overview of how two entities are associated together the quality of the summary is expressed by its similar- by showing all possible paths between these entities in ity to a manually built ontology by domain experts, or the data graph. Discovery Hub [72] is another ap- corpus coverage evaluation, in which the quality of the proach that offers faceted browsing and multiple re- ontology is represented by its appropriateness to cover sults explanations features to drive the user in unex- the topic of a corpus. The evaluation approach used in pected browsing paths. The work in [73] presents a [59] included identifying a gold standard by asking linked data based exploratory search feature for re- ontology engineers to select a number of concepts they trieving topic suggestions based on the user’s query considered the most representative for summarising an and a set of heuristics, such as frequency of entities, ontology. In this paper, we evaluate algorithms for events and places. The notions of knowledge patterns identifying KADG by comparing the algorithms’ out- puts versus a benchmarking set of BLO identified by
3 http://ws.nju.edu.cn/explass/ 4 http://relfinder.dbpedia.org humans. To the best of our knowledge, there are no DBTune.org we utilise: (i) Jamendo which is a large evaluation approaches that consider key concepts in repository of Creative Commons licensed music; (ii) data graphs which correspond to cognitive structures Megatune is an independent music label; and (iii) Mu- of users who are not domain experts. Our evaluation sicBrainz is a community-maintained open source en- approach that identifies BLODG through an experi- cyclopaedia of music information. The dataset coming mental method adapting Cognitive Science methods is from DBTune.org (such as MusicBrainz, Jamendo and novel and can be applied to a range of domains. Megatunes) already contains the “sameAs” links be- Evaluation of data exploration applications usually tween them for linking same entities. We utilise the considers the exploration utility from a user’s point of “sameAs” links provided by DBpedia to link Mu- view or analyses the application’s usability and per- sicBrainz and DBpedia datasets. In this way, DBpedia formance (e.g. precision, recall, speed etc.) [22]. The is linked to the rest of the datasets from DBtune.org, prime focus is assessing the usability of semantic Web enabling exploration via rich interconnected datasets. applications, while assessing how well the applica- The MusicPinta dataset has 2.4M entities and 19M tions help the users with their data exploration tasks is triple statements, taking 2GB physical space, includ- still a key challenge [76]. Task driven user studies ing 876 musical instruments entities, 71k (perfor- have been utilised to assess whether a data exploration mances, albums, records, tracks), and 188k music art- application provides useful recommendations for ac- ists. The dataset is made available on sourceforge7. All complishing users exploration tasks [36]. A task datasets in MusicPinta are available as a linked RDF driven benchmark for evaluating semantic data explo- data graph and the Music ontology8 is the ontology ration has been presented in [76]. The benchmark pre- used as the schema to interlink them. sents a set of information-seeking tasks and metrics Table 1. Main characteristics of MusicPinta data graph. The for measuring the effectiveness of completing the data graph includes five class hierarchies. Each class hierarchy has tasks. The evaluation approach in [77] aims to identify number of classes linked vie the subsumption relationship whether the simulated exploration paths information rdfs:subClassOf . DBpedia categories are linked to classes networks are similar to those produced by human ex- via the dcterms:subject relationship, and classes are linked via domain-specific relationship ploration. We will adopts the established task-based MusicOntology:instrum ent to musical performances. The depth of a class hierarchy is approach and will utilise an educational taxonomy for the maximum depth value for entities in the class hierarchy. assessing conceptual knowledge to assess knowledge Instrument class No. of Depth No. of DBpe- No. of music utility and usability of the generated exploration paths. hierarchy classes dia categories performances String 151 7 255 348 Wind 108 7 161 1539 3. Experimental and Theoretical Foundation Percussion 82 5 182 127 Electronic 16 1 7 11 Other 7 1 0 2 3.1. Application Context The MusicPinta dataset provides an adequate setup Our novel data graph exploration approach includes since it is fairly large and diverse, yet of manageable several algorithms which are formally defined and are size for experimentation. The music ontology provides independent from the domain and the data graph used. sufficient class hierarchy for experimentation. For in- In order to validate the approach and evaluate the al- stance, the class hierarchies for the String and gorithms, we need a concrete application context. We Wind musical instruments have depths of 7, which is will utilise MusicPinta - a semantic data browser in the considered ideal for applying the cognitive science no- music domain [18]. MusicPinta provides a uni-focal tion of basic level objects [21] on data graphs, as this interface for users to navigate through musical instru- notion states that objects within a hierarchy are classi- ment information extracted from various linked da- fied at least three different levels of abstraction (super- tasets. The MusicPinta dataset includes several ordinate, basic, subordinate). Figures 1 - 3 show ex- sources, including DBpedia5 for musical instruments amples of the user interface in the MusicPinta seman- and artists, extracted using SPARQL CONSTRUCT tic data browser. queries. The DBTune6 dataset is utilised for music-re- lated structured data. Among the datasets on
5 http://dbpedia.org/About. 8 http://musicontology.com/. 6 http://dbtune.org/. 7 http://sourceforge.net/p/pinta/code/38/tree/ which is used for assessing conceptual knowledge. The taxonomy identifies a set of progressively com- plex learning objectives that can be used to assess learning experiences over information seeking and search tasks [79]. It suggests linking knowledge to six cognitive processes: remember, understand, apply, an- alyze, evaluate, and create. Among these, remember and understand are directly related to browsing and exploration activities. The remaining processes re-
quire deeper learning activities, which usually happen Fig. 1. Semantic search interface in MusicPinta where a user in- outside a browsing tool, in our case Semantic data serts a name of a musical instrument (e.g. ). Xylophone browser, and hence will not be considered. The pro- cess remember is about retrieving relevant knowledge from the long-term memory, and includes recognition (locating knowledge) and recall (retrieving it from memory) [78]. The process understand is about con- structing meaning; the most relevant to our context are categorise (determine entity membership) and com- pare (detect similarities) [78]. To approximate the knowledge utility of an explo- ration path, we employ schema activation - it was ap- plied for assessing user knowledge expansion when reading text [80]. To assess the user’s knowledge of a Fig. 2. Description page of the entity 'Xylophone' in Mu- target domain concept (X), the user is asked to name sicPinta, extracted from DBpedia using CONSTRUCT queries. concepts that belongs to and are similar to the target concept X. A schema activation test is conducted be- fore an exploration and after an exploration, using three questions related to the cognitive processes re- member, categories, and compare: - Q1 [remember] What comes in your mind when you hear the word X?; - Q2 [categorise] What musical instrument categories does X belong to?; - Q3 [compare] What musical instruments are similar to X? The number of accurate concepts named (e.g. nam- ing an entity with its exact name, or with a parent or with a member of the entity) by the user before and Fig. 3. Semantic Links (i.e. predicates) related to entity Xylo- after exploration is counted, and the difference indi- phone presented in Features and Relevant Information. Fea- cates the knowledge utility of the exploration. For ex- tures include semantic relationships rdf:type (e.g. Xylo- phone is an instrument), rdfs:subclassOf (e.g. subClass ample, if a user could name correctly two musical in- Xylophone belongs to superClass Tuned Percussion) struments similar to the musical instrument Biwa and dcterms:subject (e.g. Xylophone belongs to (Q3) before an exploration and then the user could DBpedia category Greek loanwords). Relevant Information name correctly six names of musical instruments sim- include rdfs:subclassOf(e.g.Celesta is subClassOf ilar to the instrument Biwa after the exploration, then Xylophone) the effect of the exploration on the cognitive process compare is indicated as 4 (i.e. as a result of the explo- 3.2. Knowledge Utility of an Exploration Path ration the user learned 4 new similar musical instru- ments to the musical instrument Biwa). If the user To approximate the knowledge utility of an explo- named only one instrument after the exploration, the ration path, we need a systematic approach. For this, user knowledge did not increase, and the knowledge we adapt the well-known taxonomy by Bloom [78] utility will be counted as zero. 3.3. Subsumption Theory Underpinning Exploration Basic Level Objects (BLO) notion which was intro- duced by Cognitive science research. It states that do- In a scoping user study, we examined user explora- mains of concrete objects include familiar categories tion of musical instruments in MusicPinta to identify that exist at an inclusive level of abstraction in human what strategies would lead to paths with high cognitive structures (called the basic level). Most peo- knowledge utility (details of the study are given in ple are likely to recognise and identify objects at the [81]). We examined two dimensions - the user’s famil- basic level. An example from the experimental studies iarity with the domain and the density of entities in the from Rosch et al. [21] of a BLO in the music domain data graph. Paths which included familiar and dense is Guitar. Guitar represents a familiar category entities and brought unfamiliar entities led to increas- that is neither too generic (e.g. musical instru- ing the users’ knowledge. For example, when a partic- ment) nor too specific (e.g. Folk Guitar – sub- ipant was directed to explore the entity Guitar class of the category Guitar). which he/she was familiar with, the participant could Rosch, et al [21], define BLO as: categories that see unfamiliar entities linked to Guitar such as “carry the most information, possess the highest cate- Resonator Guitar and Dobro. The entity gory cue validity, and are, thus, the most differentiated Guitar served as an anchor from where the user from one another”. Crucial for identifying basic level made links to new concepts (Resonator Guitar categories is calculating cue validity: “the validity of a and Dobro). We also noted that the dense entities, given cue x as a predictor of a given category y (the which had many subclasses and were well-connected conditional probability of y/x) increases as the fre- in the graph, provided good potential anchors that quency with which cue x is associated with category y could serve as bridges to learn new concepts. increases and decreases as the frequency with which While the scoping study provided us with useful in- cue x is associated with categories other than y in- sights, it did not give solid theoretical model for de- creases” [21]. A members of a BLO share many fea- veloping an approach that generalises across domains tures (attributes) together, and hence they have high and data graphs. The study findings directed us to Au- similarity values in terms of the feature the BLO mem- subel’s subsumption theory for meaningful learning bers share. Consequently, two approaches can be ap- [20] as a possible theoretical underpinning model for plied to identify BLO in a domain taxonomy: generating exploration paths. This theory [20,82–84] Distinctiveness (highest cue validity). This fol- has been based on the premise that a human cognitive lows the formal definition of cue validity (given structure (i.e. individual’s organisation, stability, and above). It identifies most differentiated category ob- clarity of knowledge in a particular subject matter jects in a domain. A differentiated category object has field) is the main factor that influences the learning most (or all) of its cues (i.e. attributes) linked to its and retention of new knowledge [84]. In relation to members (i.e. subclasses of the category object) only, meaningful learning, the subsumption process postu- and not linked to other category objects in the taxon- lates that a human cognitive structure is hierarchically omy. Each entity linked to one (or more) members of organised with respect to levels of abstraction, gener- the category object will have a single validity value ality, and inclusiveness of concepts. Highly inclusive used as a predictor for the distinctiveness of the cate- concepts in the cognitive structure can be used as gory among other category objects in the taxonomy. knowledge anchors to subsume and learn new, less in- For example, the category object v2 in Figure 4 has clusive, sub-concepts through meaningful relation- four entities (u3,u4,u5,u6) linked to its members ships [20,83,85,86]. Once the knowledge anchors are (v21,v22,v23,v24). The validity of entity u4 as predictor of identified, attention can be directed towards identify- category v2 is higher than entity u3, since u4 is only ing the presentation and sequential arrangement of the linked to members of the category v2 whereas entity u3 new subsumed content [84]. Hence, to subsume new is linked to members of the categories v1 and v2. knowledge, anchoring concepts are first introduced to the user, and then used to introduce new concepts. Homogeneity (highest commonality between cat- egory members). This identifies category objects 3.4. Basic Level Objects whose members have high similarity values. The higher the similarity between category members, the To identifying knowledge anchors in data graphs, more likely it is that the category object is at the basic we need to find entities that can be highly inclusive level of abstraction. This is complementary with the and familiar to the users. For this, we will adopt the distinctiveness feature described above. A category object with high cue validity will usually have high using the rdfs:subclassOf subsumption rela- number of entities shared by its members. The homo- tionship denoted as ) and following its transitivity geneity value for a category object considers the pair- inference. This includes: wise similarity values between the category’s mem- - Root entity ( r ) which is superclass for all entities in bers. For example in Figure 4, the category entity v2 the domain; considers the pairwise similarities between its mem- - Category entities ( C V ) which are the set of all bers (e.g. similarity between [v21,v22], [v21, v23], inner entities (other than the root entity r ), that have [v21,v24]). The higher the similarity between members at least one subclass, and may also include some in- of that category, the more likely that the category is at dividual objects; the basic level. - Leaf entities ( L V ) which are the set of entities In the following sections, we will utilise Ausbel’s that have no subclasses, and may have one or more subsumption theory to generate exploration paths individuals. through data graphs based on knowledge anchors. We Starting from the root entity r , the class hierarchy will split this into two stages: (i) identifying in a data graph is the set of all entities linked via the knowledge anchors in data graphs, and (ii) using the subsumption relationship rdfs:subClassOf. The knowledge anchors to subsume new knowledge. set of entities in the class hierarchy include the root entity r , Category entities C and Leaf entities L . The set of edge labels E is divided further consid- 4. Preliminaries ering two relationship categories: - Hierarchical relationships (H ) is a set of subsump- We provide here the main definitions that will be tion relationships between the Subject and Object used in the formal description of the algorithms. entities in the corresponding triples. RDF describes entities and attributes (edges) in the - Domain-specific relationships represent rele- data graph, represented as RDF statements. Each state- (D) ment is a triple of the form
5. Identifying Knowledge Anchors in Data Graphs |{ ve ,e,v : v v}| AV (v,e) (2) |{ v,e,v : v V}| In section 3 we highlighted and justified the need for ve W (v,e) e a a two approaches to identify knowledge anchors in a An example is provided in Figure 4. The AV value data graph: distinctiveness and homogeneity. We for category entity v with regard the domain-specific adopt metrics from FCA to define such distinctiveness 2 and homogeneity matrices. relationship D is the aggregation of the AV values of the subject entities u3,u4,u5,u6 linked to members of the category entity (i.e. objects ) via the 5.1. Distinctiveness Metrics v2 v21, v22 , v23 , v24 edge label or predicate D . This group of metrics aims to identify differentiated categories whose members are linked to distinctive en- tities that are shared amongst the categories’ members but not with other categories. Each category entity v V that is linked through an edge label e to mem- bers v v of the category entity vC will have a sin- gle validity value to distinguish v from the other cate- gory entities. We follow the definition of cue validity provided by Rosch et al [21] in identifying basic level objects. Ac- cording to Rosch et al, “the cue validity of an entire category may be defined as the summation of the cue validities for that category of each of the attributes of the category”. This definition is similar to the ap- proach used to identify key concepts in formal concept Fig. 4. A data graph showing entities and relationship types analysis [68] where they summed up the validity of between entities. objects of a formal concept. Three distinctiveness met- The AV value for the entity u3 equals the number rics were developed and presented in [90]. of triples between the subject entity u3 and members of
Attribute Validity (AV). The attribute validity def- the category v2 (the object entities v21, v22) via the re- inition corresponds to the cue validity definition in lationship D (i.e. 2 triples), divided by the number of triples between the subject entity u3 and all the object members. We adapt the formula in [68] for a data entities in the graph (i.e. v12 , v21 , v22 ) via the relation- graph: ship D (i.e. 3 triples). Hence the AV value for u3 |V| |{ve,e,v :v v}| 2 |{ve,e,va :va V}| 2 (4) equals 2/3 = 0.66. The aggregation of the individual CU(v,e) ( ) ( ) |V| veW(v,e) |V | |V| AV values for entities u3,u4,u5,u6 will identify the AV value for the category entity v2 . Continuing the previous example from Figure 4 for Category Attribute Collocation (CAC). This ap- calculating the AV and the CAC values for entity u3; proach was used in [88] to improve the cue validity in addition to the category-feature collocation measure metric by adding a homogeneity weight called cate- used by the CAC value, the CU will also include the gory-feature collocation measure which takes into ac- proportion of all triples between the subject entity u3 count the frequency of the attribute within the mem- and all the object entities in the graph (i.e. entities v12, bers of the category. This gives preference to ‘good’ v , v ) linked via relationship D (i.e. 3 triples) over categories that have many attributes shared by their 21 22 the number of entities linked via subsumption relation- members (i.e. high similarity between members of the ships (e.g. and ) category). In our case, a good category will be an en- rdfs:subClassOf rdf: type in the graph (i.e. total 11 entities). Hence the CU tity v C with a high number of relationships of type 2 2 value for u3 will be: (2/4) - (3/11) = 0.177. The aggre- e between v′e and the subclasses v : v v , relative gation of CU values for entities u3,u4,u5,u6, multiplied to the number of its subclasses. Formula 3 defines the by the total number of members of category v , di- category-attribute collocation metric for a given entity 2 v with regard to a relationship type e. vided by the total number of entities in the graph linked via the susbsumption relationships will result |{ v,e,v :vv}| |{ v,e,v :vv}| in the CU value for v . CAC(v,e) e e (3) 2 |{ v,e,v :v V}| |V| veW(v,e) e a a 5.2. Homogeneity Metrics Considering the example in Figure 4 for identifying the AV value for the entity 푢 , and considering the re- These metrics aim to identify categories whose lationship D, the CAC adds a weight of (the number members share many entities among each other. In this of the triples between the subject u3 and the members work, we have utilised three set-based similarity met- 9 of v via the relationship D divided by the number of rics [90]: Common Neighbours (CN), Jaccard (Jac), 2 and Cosine (Cos). We have selected these similarity members of the category entity v . Hence the CAC 2 metrics, since they are well-known and have been pre- of u3 will be the AV value of u3 (i.e., 2/3) multiplied viously used for measuring similarity between entities by (2/4), equal to 0.33. The aggregation of individual in linked data graphs (e.g. Jaccard similarity was used CAC values for entities u3,u4,u5,u6 will identify the in [7] to measure the similarity between two patterns
CAC value for v2 . in a linked data graph, and cosine similarity was used Category Utility (CU). This approach was pre- in [50] to measure the similarity between pairs of sented in [89] as an alternative metric for obtaining items in recommendation lists). In homogeneity met- categories at the basic level object. The metric takes rics we normalise the values of the pair-wise similarity into account that a category is useful if it can improve between every two members of a category objects and the ability to predict the attributes for members of the then take the average value. This is similar to the ap- category, i.e. a good category will have many attrib- proach used in [67] for calculating the similarity be- utes shared by its members (as mentioned in the cate- tween members of a category in formal concept anal- gory-attribute collocation metric), and at the same ysis. For instance (see Figure 4), the Jaccard similar- time, it possess ‘unique’ attributes that are not related ity between the pair-wise members (v21,v22)of the en- to many other categories in the data graph. In other tity v2 considering the edge label D is equal to the words, CU gives preference to a category that have number of intersected subject entities (in this example unique attributes associated only with the category’s one entity, i,e., u3) linked to them via edge label D , divided by the number of union subject entities (in this
9 The implementation algorithm can be found in [90]. example two entities, i.e., u3 ,and u4) linked to them entity, we collected a representative image from the via edge label D . Musical Instrument Museums Online (MIMO) 10 ar- chive to ensure that pictures of high quality were shown to participants11. We ran ten online surveys12: 6. Evaluation of Knowledge Anchor Algorithms (i) eight surveys presented 256 leaf entities, each showed 32 leaves; and (ii) two surveys presented 108 We adapt the Cognitive Science experimental ap- category entities (54 categories each). proach of free-naming tasks to identify human basic Each image was shown for 10 seconds on the par- level objects over a data graph (BLODG) which are ticipant's screen, and the participant was asked to type compared to the knowledge anchors KADG obtained the name of the given object (for leaf entities) or the by applying the metrics from section 5. category of objects (for category entities). The image allocation in the surveys was random. Every survey
6.1. Obtaining Human BLODG had four respondents from the study participants (i.e. each image was named by four different participants - The set of entities in a data graph (as described in overall there were 1456 answers in the surveys). Each the preliminaries section) can be divided into two: (i) participant was allocated to one survey (either leaf en- category entities, representing the inner entities in a tities or category entities). Figures 5-8 show example class taxonomy that have at least one member (e.g. one images and participants’ answers (Figure 5 from Strat- subclass member), and (ii) leaf entities, representing egy 1; Figures 6-8 from Strategy 2). Processing the an- the set of entities that have no subclasses, and may swers, we identify two sets of human BLODG (see [91] have one or more individuals. Therefore, we divide the for detailed algorithm how the two sets were obtained). free-naming task into two strategies that correspond to Set1 [resulting from Strategy1]. We consider accu- these two types of entities: rate naming of a category entity (parent) when leaf en- Strategy 1: the participants were shown an image of tity that belongs to this category is seen. For example, a leaf entity, and were asked to type its name. in Figure 5 a participant has named Violotta, a leaf Strategy 2: the participants were shown a group of entity, with its parent category Violin. This will be images presenting the entities of a category, and were counted as an accurate naming and will increase the asked to type the name of the category. count for Violin. The overall count for Violin To obtain a set of human BLODG used to benchmark will include all cases when participants named Vio- the KADG metrics, we conducted a user study with the lin while seeing any of its leaf members. MusicPinta data graph. Participants. The study involved 40 participants recruited on a voluntary basis, varied in Gender (28 male and 12 female), cultural background (1 Belgian, 10 British, 5 Bulgarian, 1 French, 1 German, 5 Greek, 1 Indian, 2 Italian, 6 Jordanian, 1 Libyan , 2 Malaysian, 1 Nigerian, 1 Polish, and 3 Saudi Arabian), and age (18 – 55, mean = 25). None of the participants had any expertise in music. Method. The participants were asked to freely name objects that were shown in image stimuli, under limited response time (10 seconds) for each image. Overall, 364 taxonomical musical instruments were extracted from the MusicPinta dataset by running SPARQL queries over the triple-store hosting the da- Fig. 5. An image of Violotta (a leaf entity in the data graph) was shown to a user, who named it as . taset to get all musical instrument entities linked via Violin Set2 [resulting from Strategy 2]. We consider nam- rdfs:subclassOf relationship. There were 256 ing a category entity with its exact name or a name of leaf entities and 108 category entities. For each leaf
10 http://www.mimo-international.com/MIMO/ 12 The study was conducted with Qualtrics (www.qualtrics.com). 11 MIMO provided pictures for most musical instruments. In the Examples from the surveys are in: https://login.qualtrics.com/ rare occasions when an image did not exist in MIMO, Wikipedia jfe/preview/SV_cHhHPPthBFO5r6d?Q_CHL=preview images were used instead. its parent or subclass member. For example (see Fig- In each of the two sets, entities with frequency equal ure 6), a participant saw the category Fiddle and or above two (i.e. named by at least two different par- named its parent category Violin; this will increase ticipants) were identified as human BLODG. The union the count for Violin. In Figure 7 a participant saw of Set1 and Set2 gives human BLODG (we identified the image of category Violin and named it with its 24 human BLODG). The full list of human BLODG ob- exact name; this will increase the count for Violin. tained from MusicPinta is available in [92]. In Figure 8 a participant saw the category Bowed 6.2. Evaluating KA Against Human BLO String Instrument and named it as its member DG DG Violin; this will increase the count for Violin. We used the human BLODG to examine the perfor- mance of the KADG metrics. For each metric, we ag- gregated (using union) the KADG entities identified us- ing the hierarchical relationships (H). We noticed that the three homogeneity metrics have the same values. Therefore, we chose one metric when reporting the re- sults, namely Jaccard similarity13. A cut-off threshold point for the result lists with po- tential KADG entities was identified by normalizing the output values to a range between 0 and 1 from each metric and taking the mean value for the 60th percen- 14 Fig. 6. An image of Fiddle (a category entity with two leaf tile of the normalised lists . The KADG metrics evalu- entities) was shown to a user, who named it as Violin. ated included the three distinctiveness metrics plus the Jaccard homogeneity metric; each metric was applied over both families of relationships – hierarchical (H) and domain-specific (D). As in ontology summarisation approaches [59], a name simplicity strategy based on data graphs was ap- plied to reduce noise when calculating key concepts (usually, basic level objects have relatively simple la- bels, such as chair or dog). The name simplicity ap- proach we use is solely based on the data graph. We identify the weighted median for the length of the la- bels of all data graph entities v V and filter out all Fig. 7. An image of Violin (a category entity) was shown to entities whose label length is higher than the median. a user, who named it as Violin For the MusicPinta data graph, the weighted median is 1.2, and hence we only included entities which consist of one word. Table 2 illustrates precision and recall values comparing human BLODG and KADG derived using hierarchical and domain specific relationships. We used Precision, Recall and F scores since we are dealing with a binary classification problem where all knowledge anchors identified by our KADG metrics are considered of same rank. All class entities above the 60th percentile are identified as knowledge anchors of same rank (i.e. knowledge anchors above the 60th per- centile are given the value of 1).
Fig. 8. An image of Bowed String Instrument (a cate- gory entity) was shown to a user, who named it as Violin
13 14 th th th th The Jaccard similarity metric is widely used, and was used in We experimented the KADG metrics at the 60 , 70 , 80 and 90 identifying basic formal concepts in the context of formal concept percentiles on the metrics normalised output lists, and the metrics analysis [67]. performed best using at the 60th percentile. Table 2. MusicPinta: performance of the KADG algorithms Missing basic level entities due to unpopulated ar- compared to human BLODG. eas in the data graph. We noticed that none of the met- Meas- Relationship rics picked FN entities (such as AV CAC CU Jac Harmonica, ure Type Banjo or Cello) that belonged to the bottom H Preci- 0.60 0.62 0.62 0.60 quartile of the class hierarchy and had a small number sion D 0.55 0.53 0.55 0.62 of subclasses (e.g. Harmonica, Banjo and H 0.68 0.73 0.73 0.55 Recall Cello each have only one subclass and there are no D 0.50 0.45 0.50 0.36 domain-specific relationships with their members). H 0.64 0.67 0.67 0.57 F-score Similarly, none of the metrics picked Trombone D 0.52 0.49 0.52 0.46 (which is false negative) - although Trombone has three subclasses, it is linked only to one performance Hybridisation of Metrics. Further analysis of the and is not linked to any DBpedia categories. While False Positive (FP) and False Negative (FN) entities these entities belong to the cognitive structures of hu- indicated that the metrics had different performance mans and were therefore added in the benchmarking on different taxonomical levels in the graph. This led sets, one could doubt whether such entities would be to the following hybridisation heuristics. useful exploration anchors because they are not suffi- Heuristic 1: Use Jaccard metric with hierarchical ciently presented in the data graph. These entities relationships for the most specific categories in the would take the user to ’dead-ends’ with unpopulated graph (i.e. the categories at the bottom quartile of the areas which may be confusing for exploration. We taxonomical level). There were FP entities (e.g. therefore argue that such FN cases could be seen as and ) returned by distinctiveness metrics Shawm Oboe ‘good misses’ of algorithms. using the domain-specific relationship MusicOn- Selecting entities that are superordinate of basic tology:Performance because these entities are level entities. The FP included entities, such as Tam- highly associated with musical performances (e.g. bura, Reeds, Bass, Brass, Castanets, and is linked to 99 performances and is Shawm Oboe Woodwind, which are well presented in the graph linked to 27 performance). Such entities may not be hierarchy (e.g. Reeds has 36 subclasses linked to 60 good knowledge anchors for exploration, as their hier- DBpedia categories, Brass has 26 subclasses linked archical structure is flat. The best performing metric at to 22 DBpedia categories, has 72 sub- the specific level was Jaccard for hierarchical attrib- Woodwind utes - it excluded entities which had no (or a very small classes linked to 82 DBpedia categories). Also, their number of) hierarchical attributes. members participate in many domain-specific rela- Heuristic 2: Take the majority voting for all the tionships (e.g. Reeds members are linked to 606 other taxonomical levels. Most of the entities at the performances, Brass - 33, and Woodwind - 853). middle and top taxonomical level will be well repre- Although, these entities are not close to the human sented in the graph hierarchy and may include do- cognitive structures, they provide direct links KADG main-specific relationships. Hence, combining the entities from the benchmarking sets (e.g. Reeds links values of all algorithms is sensible. Each algorithm to Accordion, Brass links to Trumpet and represents a voter and provides two lists of votes, each Woodwind links to Flute). We therefore argue list corresponding to hierarchical or domain-specific that such FP cases could be seen as ‘good picks’ of the associated attributes (H, D). At least half of the voters algorithms because they can provide exploration should vote for an entity for it to be identified in KADG. bridges to reach BLO. Examples from the list of KADG identified by applying The generated KADG after applying hybridisation is the above hybridisation heuristics included Accor- used for generating data graph exploration paths, as dion, Guitar and Xylophone. The full KADG presented in the next section. list is available here [92]. Applying the hybridisation heuristics presented above improved Precision value to 0.65 (average Precision in Table 2 = 0.59), Recall 7. Exploration Strategies Based on Subsumption value to 0.68 (average Recall in Table 2 = 0.56), and F score to 0.66 (average F score in Table 2 = 0.57). In this section we describe how we use KADG to Examining the FP and FN entities for the hybridi- generate navigation paths following on the subsump- zation algorithm, led to the following observations tion theory for meaningful learning (see section 3.3). about the possible use of KADG as exploration anchors. To do this, we have to address two challenges: Challenge 1: how to find the closest knowledge an- both entities). Due to the fact that class hierarchies ex- chor to the first entity of an exploration path? In uni- ist in most data graphs, we adopt the semantic similar- focal browsing (pivoting), a user starts his/her explo- ity metric from [93] and used in [94] and apply it in ration from a single entity in the graph, also referred the context of a data graph, where semantic similarity as a first entity (vs) of an exploration path. For example, is based on the lengths between the entities in the class a user who wants to explore information about the mu- hierarchy. The semantic similarity between vs and a sical instrument Xylophone is directed to use Mu- knowledge anchor vi is calculated as: sicPinta semantic data browser. The user starts his/her 2.depth(lca(v ,v )) exploration by entering the name of Xylophone in sim(v ,v ) : s i (5) s i depth(v ) depth(v ) MusicPinta Semantic search interface (see Figure 1). s i Xylophone is the first entity of an exploration path. where, lca(vs , vi) is the least common ancestor of vs To start the subsumption process, the user has to be and vi, and depth(v) is a function for identifying the directed from this first entity to a suitable knowledge depth of the entity v in the class hierarchy. anchor in the data graph from where links to new en- Algorithm II describes how the semantic similarity tities can be made. However, there can be several metric is applied to identify vKA. The algorithm takes a knowledge anchors in a data graph; hence, we need a data graph, the first entity vs of an exploration path and mechanism to identify the closest knowledge anchor a set of knowledge anchors KADG as an input, and vKA to the first entity vs. We propose an algorithm to identifies the closest knowledge anchor vKA KADG find the closest and most relevant knowledge anchor, with highest semantic similarity value to vs. presented in section 7.1. Challenge 2: how to use the closest knowledge an- Algorithm II: Identifying Closest KADG chor to subsume new class entities for generating an Input: DG V , E,T, v V , KA {v ,v ,...,v } exploration path? The closest knowledge anchor usu- s DG 1 2 i Output: vKA – closest knowledge anchor with highest semantic ally can have many subclass entities at different levels similarity to vs of abstractions. It is important to identify which sub- 1. if then // v is a knowledge anchor vs KADG s classes to subsume and in what order while generating 2. v : v ; an exploration path for the user. Furthermore, we also KA s 3. else // v is NOT a knowledge anchor need to identify appropriate narrative scripts between s 4. S : {} ; //list for storing similarity values the entities in the exploration path to help layman us- 5. for all do //for all knowledge anchors ers to create meaningful relationships between famil- vi KADG iar entities they already know. Our algorithm for gen- 6. CA:{}; //list to store common ancestors of vs and vi erating an exploration path and the corresponding 7. L : {} ; //list for storing trajectory lengths 8. ; transition narratives is presented in section 7.2. CA common _ ancestors(vs ,vi ) 9. for all //for all common ancestors vca CA 7.1. Finding the Closest KADG 10. ; //length between vca and vi L length(vca ,vi ) 11. end for; Let vs be the first entity of an exploration path. It 12. vlca : vca with least length in L ; can be any class entity in the class hierarchy. If vs is a 13. 2 depth (v lca ) ; knowledge anchor (vs KADG), then there is no need S depth (v ) depth (v ) to identify the closest knowledge anchor, and the sub- s i 14. end for; sumption process can start immediately from vs. How- 15. with maximum similarity value in list S; vKA : vi ever, if vs is not a knowledge anchor (vs KADG), 16. end if; then vs can be superordinate, subordinate, or sibling of one or more knowledge anchors. Hence, an auto- matic approach for identifying the closest knowledge If the first entity vs belongs to the set of knowledge anchors v KADG (line 1), then the first entity vs is anchor vKA to vs is required. For this, we calculate the s semantic similarity between vs and every knowledge identified as the closest knowledge anchor vKA (line 2). anchor vi KADG. The semantic similarity between However, if the first entity vs does not belong to the set two entities in the class hierarchy is based on their dis- of knowledge anchors (line 3), then the following tances (i.e. length of the data graph trajectory between steps are conducted: - The algorithm initialises a list S to store semantic Table 3. Narrative types of transitions between entities in an exploration path. similarity values between vs and every knowledge Output script anchor vi KADG (line 4). Narrative From To en- Description (From_ entity, Narrative type, Type entity tity - For every knowledge anchor vi KADG (line 5), the To_entity) algorithm initiates two lists: list CA for storing the “You may find it useful to common ancestors (i.e. common superclasses) of vs vs is know that vs belongs to a fa- N1 vs vKA subclass of vKA miliar and well-known class– and v (line 6), and list for storing the trajectory i L vKA. Let's explore vKA.” lengths between the common ancestors in list CA “You may find it useful to know that there is a well- and the knowledge anchor vi (line 7). vs is superclass N 2 vs vKA known class – vKA that be- of vKA - The algorithm in (line 8) uses a function com- longs to vs . mon_ancestors(vs,vi) which retrieves all common Let's explore vKA” “You may find it useful to ancestors of vs and vi in the class hierarchy via the vs and vKA know that vs is similar to a N 3 vs vKA following SPARQL, and stores them in list CA: Are siblings well-known class – vKA. SELECT distinct ?common_ancestor Let's explore vKA.” WHERE { “You may find it useful to v′KA is subclass know that v′KA belongs to vKA, vs rdfs:subClassOf ?common_ancestor. N 4 vKA v′KA of vKA and su- and vs belongs to v′KA. perclass of vs vi rdfs:subClassOf ?common_ancestor}. Let's explore v′KA.” “You may find it useful to - For every common ancestor in list CA (line 9), the vʺKA is subclass N v vʺ of vKA and not know that vʺKA belongs to algorithm identifies the length of the data graph tra- 5 KA KA superclass of v s . Let's explore KA.” jectories between vi and each of the common ances- vKA vʺ tors vca in list CA, via the following SPARQL: The algorithm takes a data graph DG, the first entity SELECT(count(?intermediate)-1 as ?length) v , the closest knowledge anchor vKA, the length of an WHERE { s exploration path (m), and the edge label e = vi rdfs:subClassOf ?intermediate. rdfs:subClassOf as an input, and generates an v ?intermediate rdfs:subClassOf ca.} exploration path P of length m. The algorithm starts - The common ancestor vca with least trajectory length by initialising an empty exploration path P (line 1) to vi in list L is identifies as the least common ances- used to store m transition narratives. Then the algo- tor vlca (line 12). rithm starts identifying the relationship type between
- Then, the semantic similarity metric (Formula 5) is vs and vKA. If vs is a subclass of vKA (lines 2), then the applied (line 13). The metric includes identifying following steps are conducted: depths of vs, vi , and vlca. The depth of an entity v is - The transition narrative from vs vs , script (N1), vKA identified using the following SPARQL: to vKA is inserted into P (line 3) where the function SELECT (count(?intermediate)-1 as ?depth) WHERE { script(N1) retrieves the script output for narrative type N1 from Table 3. The length of exploration path v rdfs:subClassOf ?intermediate. ?intermediate rdfs:subClassOf r .} m is decreased by one (line 4). where r is the root entity in the data graph. - The algorithm in (line 5) identifies the set of inter- mediate class entities (V´KA) between vs and vKA, us- The semantic similarity value is then inserted into the ing the following SPARQL query: list S (line 13), and the knowledge anchor with the SELECT distinct ?intermediate highest similarity value to the first entity vs will be - WHERE { identified as closest knowledge anchor vKA (line 15). vs rdfs:subClassOf ?intermediate.
?intermediate rdfs:subClassOf vKA .} 7.2. Subsumption Using Closest Knowledge Anchor - A list Q´ is created in (line 6), and the function
sortDepth() sorts the class entities vKA VKA based The closest knowledge anchor vKA is used to sub- on their depths starting from the least depth class en- sume new class entities and to generate transition nar- tity (i.e. direct subclass of vKA) to highest depth in as- ratives in the exploration path. Table 3, describes the cending order, and inserts the sorted class entities different narrative types used between entities while into list Q (line 7). The function sortDepth() iden- generating an exploration path. Algorithm III de- scribes our approach for generating exploration paths tifies the depth of a class entity v´KA via the following using the subsumption theory for meaningful learning. SPARQL query: SELECT (count(?intermediate)-1 as If vs is superclass of vKA (lines 12), then the transition ?depth) narrative v , script(N ),v from vs to vKA is inserted WHERE { s 2 KA into P (line 13) where the function script(N2) retrieves v rdfs:subClassOf ?intermediate. KA the script output for narrative type N from Table 3. ?intermediate rdfs:subClassOf r .} 2 Length m of P is decreased by one (line 14). If vs and where r is the root entity in the data graph. vKA are siblings (line 15), then the transition narrative - The algorithm in (lines 8 – 11) uses the closest vKA to v , script(N ),v from vs to vKA is inserted into P subsume the class entities in Q´. The transition nar- s 3 KA (line 16) where the function script(N3) retrieves the rative v , script(N ),Q[i] from vKA to Q[i] (i.e. KA 4 script output for narrative type N from Table 3. v ) is inserted into P (line 9) where the function 3 KA Length m pf P is decreased by one (line 17). script(N4) retrieves the script output for narrative After identifying the relationship between vs and type N4 from Table 3. The length of the path m is vKA, the following steps are conducted: decreased by one (line 10). - Identify the set of subclass entities (VKA ) of vKA which Algorithm III: Subsumption Using Closest KADG do not belong to Q(line 19). A list Q is created in Input: , , , = length of DGV,E,T vs V vKAKADG m (line 20), and the function sortDepthDensity() sorts exploration path, e = rdfs:subClassOf the class entities vKA V KA based on two their Output: an exploration path P of length m. depths and density, as the following steps: 1. P : {}; //empty exploration path P (i) Identify the depth of the class entities (similar to 2. if v , e, v then vs is subclass of vKA s KA function sortDepth() described above). 3. //insert narra- P vs , script (N1 ), vKA ; (ii) Identify the density (using degree centrality) of tive the class entities based on number of subclasses. 4. m ; //reduce length of P by one (iii) Sort the class entity starting from the least depth 5. is all ; VKA vKA :vs ,e,vKA vKA,e,vKA (i.e. direct subclasses of vKA) to highest depth in 6. Q : {} ; ascending order, and from highest density to 7. Q sortDepth(V ) ; least density (i.e. first sort using depth, if two or KA more entities are at the same depth, then sort 8. for (i:1 ; i |Q| m0 ; i) these entities based on their density from highest 9. //insert narrative P vKA , script(N4 ),Q[i]; to lowest). The sorted class entities are inserted 10. m; //reduce length of P by one into list Q (line 21). 11. end for; - The algorithm in (lines 22–25) uses vKA to subsume
12. else if then vs is superclass of vKA vKA ,e,vs the class entities in Q . The transition narrative 13. P vs , script(N 2 ),vKA ; vKA , script (N 5 ),Q[ j] from vKA to Q[ j] (i.e. vKA ) 14. m ; //reduce length of P by is inserted into P (line 23) where the function one script(N5) retrieves the script output for narrative 15. else if v ,e,v v ,e,v then //vs and vKA are siblings s i KA i type N5 from Table 3. The length of exploration path 16. //insert narrative P v s , script (N 3 ), v KA ; m is decreased by one (line 24). 17. m ; //reduce length of P by one 18. end if; 19. is all ; V KA vKA :vKA ,e,vKA vKA Q 8. Evaluation of the Subsumption Algorithm 20. Q : {} ; To evaluate the subsumption algorithm, we con- 21. ; Q sortDepthDensity(VKA ) ducted an experimental study to examine the 22. for ( j :1 ; j | Q| m 0 ; j ) do knowledge utility and usability of the generated explo- 23. P v , script (N ),Q[ j]; //insert narrative ration paths. We will compare two conditions: KA 5 Experimental condition (EC): where users follow 24. m; //reduce length of P by one 25. end for; exploration paths automatically generated using the 26. end if; subsumption algorithm presented in section 7; Control condition (CC): where users perform free exploration and are free to select entities to explore. A controlled task-driven user study is conducted to The second step in designing data exploration task examine the following hypotheses: was to identify unfamiliar entities in the domain of this H1. Users who follow EC expand their domain user study. For this we ran a questionnaire with users knowledge. to identify the unfamiliar entities in the String In- H2. The expansion in the users’ knowledge when fol- strument and Wind Instrument class hierar- lowing EC is higher than when following CC. chies in the MusicPinta data graph. These two class H3. The usability when EC is followed is higher than hierarchies have the richest class representation in when CC is followed. terms of the number of classes and the hierarchy depth as discussed in Section 3.1, and have the highest num- 8.1. Data Graph Exploration Task ber of knowledge anchors (9 anchors in the String Instrument class hierarchy and 10 anchors in the Designing exploration tasks for users is considered Wind Instrument class hierarchy – out of 24 an- an important requirement for evaluating data explora- chors in MusicPinta data graph). We have extracted tion approaches [95]. A typical exploration task has to class entities at the bottom quartile of the two class hi- be generic (i.e. the scope of the task is broad and the erarchies (note that the depth of the two class hierar- user don’t have specific information needs), realistic chies is 7 – see Table 1, and entities of depth 6 or 7 are (i.e. real-life task that set in a familiar situation), dis- considered to be at the bottom quartile of the data covery-oriented (i.e. users travel beyond what they graph). This is based on earlier Cognitive science stud- know), open-ended (i.e. requires a significant amount ies acknowledging that layman users are not familiar of exploration, where open-endedness relates to uncer- with specific objects in a domain [97]. Overall 61 class tainty over the information available, or incomplete entities from the String Instrument and Wind information on the nature of the search task), and set Instrument class hierarchies were used in the sur- in an unfamiliar domain for the user [2,95,96]. In this vey. The selected classes were randomised and distrib- work, we follow a two-step approach (similar to [95]) uted among twelve participants who are not experts in to design a data exploration task for the study partici- the musical instruments (the participants have limited pants. The approach involves: (i) Designing a task knowledge about musical instruments and may have template that places the participant in a familiar situa- seen the instrument, and none of the participants had tion which involves exploring multiple entities in an played any musical instruments. unfamiliar domain or topic (e.g. a researcher at a uni- The most unfamiliar instrument from each class hi- versity that wants to write a research paper about a erarchy was Biwa (class hierarchy: String In- new topic), and (ii) Identifying unfamiliar candidate , origin: Japanese) and (class entities (e.g. find new research topic) in the domain strument Bansuri that could be plugged into the task template. hierarchy: Wind Instrument, origin: Indian). Our aim was to design a generic task template that encourages layman users to seek knowledge in a do- 8.2. Experimental Setup main unfamiliar to them. Therefore, we designed the task template in the context of a general knowledge Participants. 32 participants, including university quiz show where layman users need to acquire as students and professionals (24 students and 8 profes- 15 much knowledge as they can. Inspired by the task tem- sionals ), were recruited on a voluntary basis (a com- plates in [95], our task template in Table 4 was de- pensation of £5 Amazon voucher was offered). Partic- signed to suit the musical instrument domain. ipants varied in age 18–45 (mean age is 30), and cul- tural background (1 Austrian, 9 British, 1 Chinese, 3 Table 4. Task template used in the experimental user study Greek, 1 Italian, 5 Jordanian, 1 Libyan, 2 Malaysian, Task template 6 Nigerian, 1 Polish, 1 Romanian and 1 Saudi). Method. We ran four online surveys16, each survey “Imagine that you are a member of a team which will take part in a general knowledge quiz show. You have had 8 participants and each participant was allocated been asked to explore two musical instruments for 20 one survey. Figure 9 shows the overall structure of the minutes in order to prepare a short presentation to de- user study. Each participant explored both musical in- scribe to your team what you have learned about these struments (i.e. Biwa, Bansuri), where each instru- instruments”. ment is allocated to an exploration strategy (EC or CC).
15 Academics and private Sector employees (Banking and Airlines). 16 The study was conducted with Qualtrics (www.qualtrics.com). of Lute other than class entities that have been sub- sumed using N4 – line 25 in Algorithm V).
Fig. 9. Structure of user study to examine EC against CC in terms of knowledge utility, usability and cognitive load. In both the strategies, the participants explored the same information including categories from a seman- tic data browser (i.e. participants’ explored 'Descrip- Fig. 10. Extract from the String Instrument class hierarchy tion', 'Features' and 'Relevant information' of musical in MusicPinta showing exploration path of Biwa. This path was instruments – See Figures 2, and 3). The order of EC followed in the experimental condition EC. and CC was randomised to counter balance the impact Table 5 lists the transition narratives that were fol- on the results. Every participant session was con- lowed in generating the exploration path for Biwa. ducted separately and observed by the authors. All participants were asked to provide feedback before, Table 5. Transition narratives used for generating the exploration during, and after the interaction with MusicPinta. path for Biwa The different stages of the study are explained below. Transition Narratives Narrative Script Task presentation [1 min] – utilise the task template for path of Biwa You may find it useful to know (described in Section 8.1) to present the data explora- v s , script (N 1 ), v KA tion task for the users at the beginning of their explo- that ‘Biwa’ belongs to a familiar and well-known instrument called ration session. We used the task template in Table 4. ‘Lute’. Let's explore ‘Lute’. Pre-study questionnaire [2 min] - collected infor- You may also find it useful to know v KA , script(N 4 ), Q[1] mation about the participants’ profiles, and their fa- that ‘Oud’ belongs to ‘Lute’ , and miliarity with the music domain, focusing on the two ‘Biwa’ belongs to ‘Oud’. Let's musical instrument class hierarchies which would be explore ‘Oud’. You may also find it useful to vKA , script (N 5 ), Q[1] explored – String Instrument and Wind In- know that ‘Tambura’ belongs to strument. The participants’ familiarity with the two ‘Lute’. Let's explore ‘Tambura’ You may also find it useful to class hierarchies varied from low to medium (the fig- v KA , script (N 5 ), Q[2] know that ‘Pipa’ belongs to ‘Lute’. ures were 63% and 78% for low familiarity with Let's explore ‘Pipa’ String Instrument and Wind Instrument, respectively). Table 6 shows examples of the entities that were freely Graph exploration [20 min] – each user explored visited in the control condition for Biwa. the two strategies where each strategy corresponds to Table 6. Examples of entities the participants have visited during one of the two unfamiliar instruments (Biwa or their free exploration of Biwa Bansuri). Figure 10 shows an example Example 1 Example 2 Example 3 for generating an exploration path under EC for the Biwa Biwa Biwa instrument Biwa (the first entity of the exploration Bouzouki String Japanese Musi- Instruments cal Instruments path) using the closest knowledge anchor Lute. Xalam ‘Guitar’ Lute The first transition narrative in the exploration path Banjitar Acoustic Moon Lute is between the Biwa and the closest knowledge an- Guitar Plucked String Classical Bouzouki chor Lute (using N1 in Table 3). After that, the closest instruments Guitar knowledge anchor Lute is used to subsume new class entities and generate transition narratives in the explo- Figure 11 shows an example for generating the explo- ration path under EC for the instruments Bansuri ration path using the narrative types N4 (subsume in- termediate class entities between Lute and Biwa – (the first entity vs in the exploration path) using the closest knowledge anchor Flute. line 10 in Algorithm V) and N5 (subsume subclasses Table 8 shows examples of the entities that were freely visited in control condition CC for Bansuri Table 8. Examples of entities the participants have visited during their free exploration of Bansuri Example 1 Example 2 Example 3 Bansuri Bansuri Bansuri Transverse Fipple Flute Bamboo Musi- Flute cal Instru- ments Saw Truck Contrabass Side-blown Recorder Flute Fipple Flute Recorder Concert Flute Flute D’amour Great bass Fipple Flute recorder Both, EC and the CC had the same length (EC had four
transition narratives, and CC had four edges)17. We an- Fig. 11. Extract from the Wind Instrument class hierarchy in alysed knowledge utility and user exploration experi- MusicPinta showing exploration path of Bansuri. This path was ence using usability aspects, associated with the user’s followed in experimental condition EC. exploration settings under the experimental and con- The exploration path for Bansuri was generated us- trol conditions. ing the closest knowledge anchor Flute and narra- 8.3. Results tive types N1, N4 given in Table 3. The first transition narrative is between the first entity Bansuri and the Approximating Knowledge Utility. The partici- closest knowledge anchor Flute (using N1 in Table pants’ knowledge was measured before and after each 3). After that, the closest knowledge anchor Flute is exploration using the three questions of the schema ac- used to subsume new class entities and to generate tivation test related to the entities Biwa and transition narratives in the exploration path using nar- Bansuri (see section 3.2). Before exploration, none rative type N4 (subsume intermediate class entities be- of the users were able to articulate any item linked to tween Flute and Bansuri). Transition narratives the two musical instruments ( and ) that were followed in generating the exploration path Biwa Bansuri using the three cognitive processes. The knowledge for Bansuri are listed in Table 7. utility for the three cognitive process before and after Table 7. Narrative scripts used for generating the experimental con- exploration of EC and CC is shown in Figure 12. dition EC (i.e. exploration path) for instrument Bansuri Transition Narratives Narrative Script for path of Bansuri You may find it useful to know that v s , script (N1 ), v KA 'Bansuri' belongs to a familiar and well-known instrument called 'Flute'. Let's explore 'Flute'.
v KA , script(N 4 ), Q[1] You may also find it useful to know that 'Fipple Flute' belongs to 'Flute' , and 'Bansuri' belongs to 'Fipple Flute'. Let's explore 'Fipple Flute'.
vKA , script (N 4 ), Q[2] You may also find it useful to know that ‘Transverse Flute' belongs to 'Flute' , and 'Bansuri' belongs to 'Transverse Fig. 12. Knowledge utility of the two strategies (EC and CC) of Flute'. Let's explore 'Transverse Flute'. the user cognitive processes (median of the knowledge utility of You may also find it useful to know that vKA, script(N4 ),Q[3] exploration for all users). ‘Indian Bamboo Flutes' belongs to The knowledge utility of the exploration under EC 'Flute' , and 'Bansuri' belongs to 'In- dian Bamboo Flutes'. Let's explore 'In- in the three cognitive processes was higher than the dian Bamboo Flutes'. CC; and this difference is significant (See Table 9).
17 The length of four edges (5 entities) is based on Miller's Law [104], which indicates the number of objects that an average human can hold in working memory is 7 ± 2. The results showed that all participants were able to and noticed that 78% of the participants had low fa- remember and categorise entities with EC (only 5 par- miliarity (i.e. participants have limited knowledge and ticipants couldn’t compare new entities). Whereas not they may have seen some instruments) with Wind all participant could remember, categorise or compare Instrument, whereas 65% of the participants had new entities after they have finished their exploration low familiarity with the String Instrument with CC (there were 2 participants that could not re- class hierarchy. Being more familiar with the String member or categorise new entities; 13 participants that Instrument class hierarchy than the Wind In- could not compare between entities). strument class hierarchy, participants may have Table 9. Statistically significant differences of the values in Fig- found it easier to name entities for comparison. Also, ure 12 (Mann-Whitney, 1-tail, Na=Nb=32) entities in the String Instrument class hierar- Difference in Knowledge Cognitive Z-value P chy are associated with more DBpedia categories Utility between P and F Process compared to entities in the Wind Instrument class Remember 3.6 P<0.01 hierarchy ( has 255 and EC > CC Categorise 5.1 P<0.0001 String Instrument Compare 2.7 P<0.01 Wind Instrument has 161 DBpedia categories). Notably, for the cognitive process categorise the Table 10. Statistically significant differences of the values in Fig- bigger effect on exploration of the subsumption explo- ure 13 (Mann-Whitney, 1-tail, Na=Nb=16) ration strategy over the free exploration strategy is Difference in Instrument Cognitive Z-value P highly significant (p<0.0001). The difference between Knowledge Utility (class Hier- Process between EC and archy) the median values for categorise under EC and CC CC was higher than remember and compare. Furthermore, Biwa Remember 1.658 P<0.05 we examined the knowledge utility for the three cog- EC > CC (String) Categorise 3.373 P<0.001 nitive processes for each instrument in its correspond- Compare 2.449 P<0.05 ing class hierarchies, as shown in Figure 13. Bansuri Remember 3.467 P<0.001 EC > CC (Wind) Categorise 3.900 P<0.001 Compare 1.280 P<0.5
User Exploration Experience. After each explora- tion strategy, the participants’ feedback on the explo- ration experience was collected including exploration usability and exploration complexity, adapted from NASA-TLX [98] (Table 11). Furthermore, the partic- ipants were asked to think aloud and notes of all com- ments were kept.
Table 11. Questions to gather feedback on user exploration ex- perience, adapted from NASA-TLX (mental demand, effort, per- Fig. 13. Knowledge utility of the two strategies (EC and CC) of formance). the user cognitive processes (median of the knowledge utility of exploration for all users). Subjective Question text process The knowledge utility of the exploration under EC Knowledge How much the exploration expanded your in the three cognitive processes was higher than the Expansion knowledge? effect of free exploration under CC for Biwa and was Content How diverse was the content you have ex- higher in the cognitive processes compare and catego- Diversity plored? Mental How mentally demanding was this explora- rise for Bansuri (See Figure 13). This difference in Demand tion? EC and CC is significant except for the cognitive pro- Effort How hard did you have to work in this explo- cess compare for instrument Bansuri (See Table 10). ration? To further inspect what caused the low knowledge Performance How successful do you think you were in this exploration? utility for the cognitive process compare for instru- ment Bansuri, we looked into the participants’ fa- Figures 14 and 15 summarise the users’ feedback. miliarity with the class hierar- Wind Instrument As shown in figure 14, the exploration experience with chy (the class hierarchy that Bansuri belongs to) EC was the most informative (all 32 participants iden- tified their exploration experience with the exploration paths under EC as informative, whereas 20 partici- Table 12. Statistically significant differences of users’ subjec- pants indicated their exploration with CC as informa- tive perception of cognitive process (Mann-Whitney, 1-tail, Na=Nb=32) tive). The participants also found their exploration un- der EC to be slightly more interesting and enjoyable Difference in the us- Subjective Z p than CC. Furthermore, the participants found the ex- ers’ experience Process value Knowledge Expansion 3.98 P<0.0001 ploration paths under EC to be the least boring and Content Diversity 0.32 P<0.5 least confusing – only 3% (one participant) and 16% EC > CC Mental Demand 0.14 P<0.5 (four participants) of the participants founded their ex- Effort 0.14 P<0.5 ploration with EC to be boring or confusing, respec- Performance 2.15 P<0.05 tively. For instance, one participant indicated “Narra- Notably, for the subjective process knowledge ex- tives in paths allows me to explore entities in a hierar- pansion the bigger effect on exploration of EC over chical fashion, and I would like to freely explore other CC is highly significant (p<0.0001). types of relationships”. Another participant indicated his exploration experience with EC as confusing: “I saw the same instruments several times during my ex- 9. Discussion ploration”. The paper presented a novel computational ap- proach to facilitate users’ exploration of data graphs leading to knowledge expansion. In this section, we will summarise the benefits of the approach, will re- visit its main parts to discuss key features of the algo- rithms, and will point at the generality and future ap- plications for semantic data exploration.
9.1. Benefit of the Exploration Approach
Fig. 14. Users’ exploration experience of the two exploration strat- Overall, the evaluation results have supported our egies (EC and CC). Values show number of user paths rated with hypotheses, which were set out in section 8. the corresponding characteristics. H1: When following the subsumption exploration paths, the users expanded their domain knowledge. When users followed the experimental condition, i.e. the paths generated by the subsumption algorithm us- ing knowledge anchors, all of them expanded their do- main knowledge. All participants indicated that their exploration in the experimental condition was in- formative (in other words, they felt that while follow- ing the path they were able to find useful information); in contrast, only 62% of the participants founded their free exploration trajectories to be informative. H2: The expansion in the users’ knowledge when following the subsumption exploration paths was higher than knowledge expansion when following free Fig. 15. Users' subjective perception of the two exploration strate- exploration. The participants were able to remember, gies (EC and CC), based on an adapted NASA-TLX questionnaire categorise and compare significantly more entities. [99] (median values for all users in the range 1-10). The results also showed that the cognitive process cat- The effect of the exploration path under EC on the egorise had most effect on expanding the participants’ subjective processes knowledge expansion and perfor- knowledge. This was caused because: (i) the subsump- mance was higher than the effect of the free explora- tion hierarchical relationship (rdfs:subclassOf) tion strategy; and this difference was significant (See was used to create the narrative scripts between enti- Table 12 and Fig 15). ties of the generated exploration paths, which helped the users to categorise new entities at different levels of abstraction at their cognitive structures; and (ii) the subsumption process uses knowledge anchors to sub- this was done through experimentation – the best per- sume and learn new sub-categories similarly to the formance was achieved when using the 60th percentile. way humans learn new concepts. The same percentile was identified as best for the ca- H3: The usability when the subsumption explora- reer domain (see [91]). We expect that when applied tion paths were followed was higher than the free ex- to a range of data graphs, the 60th percentile will give ploration cases. The results showed that participants reasonable performance. However, the best cut-off found the generated exploration paths to be more en- point for a specific data graph would require experi- joyable and less confusing than free exploration, and mentation comparing different percentiles. their assessment of performance was higher. One par- Sensitivity to data graph structure. The output of ticipant thought that the experience with hierarchical the KADG algorithms is sensitive to the data graph narrative scrips was boring; and suggested that the structure in terms of the richness of the class entities system should diversify the types of narratives used and the hierarchy depth. Specifically, the algorithms between entities in the exploration path. tend to pick more anchoring entities when a data graph has many classes and high depth. For instance, the al- 9.2. Identifying Knowledge Anchors in Data Graphs gorithms identified 9 anchors in the String In- strument class hierarchy and 10 anchors in the To identify knowledge anchors in a data graph, we Wind Instrument class hierarchy – out of 24 an- have utilised Rosch’s definitions of basic level objects. chors (String Instrument and Wind In- This required operationalising cue validity, using two strument are the richest class hierarchies in Mu- groups of metrics: distinctiveness (to identify the most sicPinta– See Table 1). The algorithms did not pro- differentiated categories whose attributes are shared duce any anchor in the Electronic Instrument amongst the category members but not with members class hierarchy (only 15 classes with a depth of one). of other categories), and homogeneity (to identify cat- The same was observed in the career domain [91]. It egories whose members share many attributes). should be noted that KADG metrics may not produce We have adapted existing methods in FCA to define KADG in shallow class hierarchies (depth 1 or 2). metrics for KADG, following Rosch’s definitions of Possible high number of KADG. Applying the BLO. The algorithms have been applied over two data KADG algorithms over large data graphs may produce graphs – in music (presented here) and in careers (pre- high number of KADG. Further filtering would be re- sented in [91]); both data graphs had different size and quired to reduce the number of KADG. One possible hierarchy structure. Although this paper focuses only way to address this is to use crowdsourcing – showing on the music domain (so that we can show a holistic all derived knowledge anchors and asking a large approach illustrated with a concrete application), in number of users (crowd) to identify the most familiar the discussion below we identify key features of the entities, which will then form the refined KADG list. algorithms which have been confirmed in broader evaluation (see [91] for further detail). 9.3. Generating Exploration Paths Hybridisation. The analysis indicated that hybrid- isation of the metrics notably improved performance. We have developed a novel computational ap- The same was observed in the careers domain ([91]). proach to generate exploration paths, which is the first Appropriate hybridisation heuristics for the upper attempt to address users’ domain knowledge expan- level of the data graph is to combine the KA metrics DG sion during exploration. We have provided an original using majority voting. The hybridisation heuristics for way to link learning and exploration by operationalis- the bottom level of the hierarchy are dependent on the ing Ausubel’s subsumption theory for meaningful domain-specific relationships in the data graph. Hence, learning. Two algorithms have been formally defined: to derive appropriate hybridisation heuristics that give (i) identifying the closest knowledge anchor; and (ii) good performance for categories at the bottom level, generating exploration paths and transition narratives. further experimentation will be required. This will in- Several possible knowledge anchors. It is possible clude comparing the KA derived using the various DG that several knowledge anchors in a data graph have domain-specific relationships against human BLO . DG the same semantic similarity value with the first entity Cut-off point in the evaluation. An important step (the entity selected as a starting point for the path). For in the evaluation is to identify a suitable cut-off point example, there were two knowledge anchors (Flute for the KA metrics. In the current implementation, DG and Reeds) with the same semantic similarity value with the first entity Bansuri. One way to address this is to consider the density of each knowledge an- 9.4. Generality and Further Applications chor entities (e.g. using degree centrality in the graph). Then, select knowledge anchor with the highest den- The proposed exploration approach includes formal sity, as it will include many subclass members to sub- definitions for the algorithms used in both parts – iden- sume while generating the exploration path. tifying KADG and generating a path and transition nar- Transition to the closest knowledge anchor. In ratives. Both parts are generic and can be applied over some cases, the semantic similarity metric can identify different data graphs and adopted in any domain rep- closest knowledge anchors which are not superclass resented with a data graph. The algorithms will per- (i.e. N1 in Table 3), subclass (i.e. N2 in Table 3) nor form well when the taxonomy of the data graph has sibling (i.e. N3 in Table 3) to the first entity, which depth higher than 2. means that the closest knowledge anchor cannot be The formal description of the KADG provides a ge- reached directly from the first entity using one of the neric solution for identifying familiar entities over suggested narrative transitions. For instance, although data graphs, which make such entities useful in differ- the anchors Flute and Reeds have the same seman- ent ways. We have shown here how KADG enable op- tic similarity value with the first entity Bansuri, erationalising the subsumption theory for meaningful Flute is a superclass of Bansuri that can be learning to generate exploration paths for knowledge reached directly using the narrative type (N1 in Table expansion. There are other possible applications be- 3), whereas Reeds can’t be reached directly from yond data graph exploration. For example, our ap- Bansuri. Our solution to address this issue was to proach for identifying KADG can be applied to ontol- apply the semantic similarity measure proposed in ogy summarisation where KADG allow capturing lay- [93] to find knowledge anchors which could be man users’ view of the domain. Furthermore, KADG reached directly from the first entity. Future work can can be applied to solve the key problem of ‘cold start’ consider other semantic similarity algorithms, such as in personalisation and adaptation. One of the popular the measure proposed in [100] where calculating se- choices for addressing the cold start problem is a dia- mantic similarity between two entities is based on the logue system with the user. The data graphs can pro- shortest path and maximum depth of the class hierar- vide a large knowledge pool to implement such prob- chy, or the semantic similarity model presented in ing dialogue, however, one needs to select entities [101] where each class entity is given a probability from the vast amount of possibilities for probing to value used to calculate the semantic similarity be- avoid too long interactions with the user. KADG can be tween two entities. Another solution for reaching the such entities, e.g. we have proposed an approach to closes knowledge anchor is to add a narrative type in detect user domain familiarity by exploiting KADG for Table 3 that indicates that the first entity and the clos- probing interactions over data graph concepts [102]. est knowledge anchor simply belong to the same do- The subsumption algorithms for generating an ex- main but there is no direct trajectory between them. ploration path are generic and can be applied in differ- Exploration path length. The path generation al- ent domains that are represented as data graphs. This gorithm uses a knowledge anchor to subsume subordi- opens a broad spectrum of applications where users nate entities (i.e. subclasses of the closest knowledge not familiar with the domain explore semantic data, anchor) while generating transition narratives of a pre- such as: (i) researching a new domain (e.g. digital li- defined length m (given as input to the algorithm). The braries); (ii) exploring information content in a do- algorithm is dependent on the subclasses of the se- main where the user is not an expert (e.g. cultural her- lected knowledge anchor, and it may not always be itage or news); (iii) browsing through large graphs possible to generate m transition narratives in the ex- with many options which the user may not be familiar ploration path. Our implementation uses only the with (e.g. job opportunities, health information, travel knowledge anchor, and can result in paths whose information); (iv) exploring content for learning (e.g. length of less than m. Another way to address this videos or presentations). would be to continue the path by selecting other knowledge anchors. It is also possible to use superor- dinate categories to the knowledge anchor to extend 10. Conclusion an exploration path. This will be suitable for cases when the user has gained knowledge at the subordi- Exploration of data to carry out an open-ended task nate level and is ready to generalise to a more abstract is becoming a key daily life activity. It usually in- level. In such cases, a user model will be required. volves a journey through large datasets or search sys- (Eds.), 3rd Int. Semant. Web User Interact. Work., Citeseer, tems that starts with an entry point, often an initial 2006. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.9 query, and then exploring a large amount of data while 7.950&rep=rep1&type=pdf. constantly making decisions about which data to ex- [6] C. Bizer, J. Lehmann, G. Kobilarov, S. Auer, C. Becker, R. plore next. Users who are unfamiliar with the domain Cyganiak, and S. Hellamnn, DBpedia - A cystallization point they are exploring can face high cognitive load and us- for the Web of Data, Web Semant. Sci. Serv. Agents World Wide Web. 7 (2009) 154–165. ability challenges when exploring such large amount [7] G. Cheng, Y. Zhang, and Y. Qu, Explass: Exploring of data. Our work investigates how to support such us- Associations between Entities via Top-K Ontological ers’ exploration through a data graph in a way that Patterns and Facets, in: ISWC ’13, 2014: pp. 422–437. leads to expansion of the user’s domain knowledge. doi:10.1007/978-3-319-11915-1_27. [8] G.C. and Y. Qu, Searching linked objects with falcons: We introduced a novel exploration support mecha- Approach, implementation and evaluation, Int. J. Semant. nism underpinned by the subsumption theory of mean- Web Inf. Syst. 5 (2009) 49–70. ingful learning, which postulates that new knowledge [9] L. Zheng, Y. Qu, J. Jiang, and G. Cheng, Facilitating entity is grasped by starting from familiar concepts in the navigation through top-K link patterns, in: Proc. Int. Semant. Web Conf., 2015: pp. 163–179. doi:10.1007/978-3-319- graph which serve as knowledge anchors from where 25007-6_10. links to new knowledge are made. A task-driven ex- [10] M. Sah, and V. Wade, Personalized concept-based search on perimental user study was conducted to evaluate the the Linked Open Data, Web Semant. Sci. Serv. Agents World exploration paths generated from the subsumption al- Wide Web. 36 (2016) 32–57. doi:10.1016/j.websem.2015.11.004. gorithm as compared to free exploration. The findings [11] L. Zheng, Y. Qu, and G. Cheng, Leveraging link pattern for from the evaluation showed that the generated explo- entity-centric exploration over Linked Data, in: Proc. WWW, ration paths using the subsumption algorithm lead to World Wide Web, 2017. doi:10.1007/s11280-017-0464-y. significantly higher increase in the users’ knowledge [12] R. Pienta, G. Tech, J. Vreeken, G. Tech, and J. Abello, FACETS : Adaptive Local Exploration of Large Graphs, in: compared to free exploration, which enables its adop- IEEE SDM’17, 2017. tion over different domains and contexts. [13] P. Vakkari, Searching as learning: A systematization based A possible further extension would be to maintain a on literature, J. Inf. Sci. 42 (2016) 7–18. user model and personalise the path to the user’s do- doi:10.1177/0165551515615833. [14] J. Gwizdka, P. Hansen, C. Hauff, J. He, and N. Kando, Search main knowledge. This can be extended further with a As Learning (SAL) Workshop 2016, in: Proc. 39th Int. ACM diversification strategy to suggest interesting concepts SIGIR Conf. Res. Dev. Inf. Retr., ACM, New York, NY, that are more likely to attract people to engage in ex- USA, 2016: pp. 1249–1250. doi:10.1145/2911451.2917766. ploration, and hence learn more about the domain. [15] L. Koesten, E. Kacprzak, and J. Tennison, Learning When Searching for Web Data., in: Sal@SigirSearch as Learn., 2016: pp. 2–3. http://dblp.uni- Acknowledgements. The MusicPinta semantic data trier.de/db/conf/sigir/sal2016.html#KoestenKT16. browser was developed as part of the the EU/FP7 pro- [16] S. Dietze, and E. Kaldoudi, Socio-semantic Integration of ject Dicode. We are grateful to the participants in the Educational Resources - the Case of the mEducator Project, Univers. Comput. Sci. 19 (2013) 1543–1569. experimental studies. [17] V. Dimitrova, L. Lau, D. Thakker, F. Yang-Turner, and D. Despotakis, Exploring exploratory search: a user study with linked semantic data, Proc. 2nd Int. Work. Intell. Explor. Semant. Data - IESD ’13. (2013) 1–8. doi:978-1-4503-2006- References 1. [18] D. Thakker, V. Dimitrova, L. Lau, F. Yang-Turner, and D. [1] E. Palagi, F. Gandon, A. Giboin, R. Troncy, I.S. Antipolis, F. Despotakis, Assisting user browsing over linked data: Gandon, I.S. Antipolis, A. Giboin, and I.S. Antipolis, A Requirements elicitation with a user study, in: ICWE’13 Int. survey of definitions and models of exploratory search, in: Conf. Web Eng., 2013: pp. 376–383. doi:10.1007/978-3-642- ACM Work. Explor. Search Interact. Data Anal. - ESIDA ’17, 39200-9_31. 2017: pp. 3–8. doi:10.1145/3038462.3038465. [19] V. Maccatrozzo, Burst the filter bubble: using semantic web [2] R.W.W. and R.A. Roth, Exploratory Search Beyond the to enable serendipity, in: ISWC ’11, 2012. doi:10.1007/978- Query–Response Paradigm, Morgan & Claypool, 2009. 3-642-35173-0. doi:10.1287/orsc.1110.0676. [20] D.P. Ausubel, A subsumption theory of meaningful verbal [3] N. Belkin, Anomalous States of Knowledge as the Basis of learning and retention, J. Gen. Psychol. 66 (1962) 213–224. Information Retrieval., Can. J. Inf. Sci. 5 (1980) 133–143. doi:10.1080/00221309.1962.9711837. [4] G. Marchionini, Exploratory search: from finding to [21] E. Rosch, C.B. Mervis, W.D. Gray, D.M. Johnson, and understanding, in: Commun. ACM, 2006: p. 41. P.Boyes-Braem, Basic Objects in Neutral categories, Cogn. doi:10.1145/1121949.1121979. Psychol. 8 (1976) 382–439. [5] T. Berners-lee, Y. Chen, L. Chilton, D. Connolly, R. [22] S. Mazumdar, D. Petrelli, K. Elbedweihy, V. Lanfranchi, and Dhanaraj, J. Hollenbach, A. Lerer, and D. Sheets, Tabulator : F. Ciravegna, Affective graphs: The visual appeal of Linked Exploring and Analyzing linked data on the Semantic Web, Data, Semant. Web. 6 (2015) 277–312. doi:10.3233/SW- in: L. Rutledge, M C Schraefel, A. Bernstein, and D. Degler 140162. [23] B. Fu, N.F. Noy, and M. Storey, Eye Tracking the User concepts, in: DCI@ INTERACT, 2011: pp. 31–36. Experience - An Evaluation of Ontology Visualization [41] J.M. Brunetti, R. García, and S. Auer, From Overview to Techniques, Semant. Web J. 8 (2017) 23–41. Facets and Pivoting for Interactive Exploration of Semantic http://www.semantic-web- Web Data, Int. J. Semant. Web Inf. Syst. 9 (2013) 1–20. journal.net/system/files/swj770.pdf. http://www.igi-global.com/article/overview-facets-pivoting- [24] A.S. Dadzie, and E. Pietriga, Visualisation of Linked Data - interactive-exploration/77822 (accessed January 13, 2014). Reprise, Semant. Web. 8 (2017) 1–21. doi:10.3233/SW- [42] C. Stadler, M. Martin, and S. Auer, Exploring the web of 160249. spatial data with facete, Proc. 23rd Int. Conf. World Wide [25] M. Javed, S. Payette, J. Blake, and T. Worrall, VIZ – VIVO : Web - WWW ’14 Companion. (2014) 175–178. Towards Visualizations-driven Linked Data Navigation, in: doi:10.1145/2567948.2577022. VOILA@ISWC, 2016. [43] P. Papadakos, and Y. Tzitzikas, Hippalus: Preference- [26] I.O. Popov, M.C. Schraefel, W. Hall, and N. Shadbolt, enriched faceted exploration, in: EDBT@ICDT, 2014. Connecting the Dots: A Multi-pivot Approach to Data [44] K. Wongsuphasawat, D. Moritz, A. Anand, J. Mackinlay, B. Exploration, in: ISWC’10, 2011: pp. 553–568. Howe, and J. Heer, Voyager: Exploratory Analysis via doi:http://dx.doi.org/10.1007/978-3-642-25073-6_35. Faceted Browsing of Visualization Recommendations, IEEE [27] S. Araújo, D. Schwabe, and S.D.J. Barbosa, Experimenting Trans. Vis. Comput. Graph. 22 (2016) 649–658. with Explorator: A direct manipulation generic RDF browser doi:10.1109/TVCG.2015.2467191. and querying tool, in: VISSW, 2009. [45] N. Bikakis, G. Papastefanatos, M. Skourla, and T. Sellis, A [28] D. Huynh, and D. Karger, Parallax and companion: Set-based Hierarchical Framework for Efficient Multilevel Visual browsing for the data web, WWW Conf. (2009) 2005–2008. Exploration and Analysis, SWJ. 8 (2017) 139–179. http://davidhuynh.net/media/papers/2009/www2009- http://arxiv.org/abs/1511.04750. parallax.pdf. [46] M. Sah, and V. Wade, Personalized Concept-Based Search [29] G. Vega-Gorgojo, M. Giese, S. Heggestøyl, A. Soylu, and A. and Exploration on the Web of Data Using Results Waaler, PepeSearch: semantic data for the masses, PLoS One. Categorization, in: Proc. Ext. Semant. Web Conf., 2013: pp. 11 (2016). 532–547. [30] B. Shneiderman, The eyes have it: a task by data type [47] O. Rossel, Implemention of a “ search and browse ” scenario taxonomy for information visualizations, in: Proc. 1996 IEEE for the LinkedData, in: IESD@ISWC, 2014. Symp. Vis. Lang., IEEE, 1996: pp. 336--343. [48] L. De Vocht, A. Dimou, J. Breuer, M. Van Compernolle, R. http://reference.kfupm.edu.sa/content/e/y/the_eyes_have_it_ Verborgh, E. Mannens, P. Mechant, and R. Van De Walle, A _a_task_by_data_type_ta_43453.pdf. visual exploration workflow as enabler for the exploitation of [31] A. Valsecchi, M. Abrate, C. Bacciu, M. Tesconi, and A. linked open data, in: IESD@ISWC, 2014. Marchetti, Linked Data Maps: Providing a Visual Entry Point [49] M. Dojchinovski, and T. Vitvar, Personalised Access to for the Exploration of Datasets, IESD@ISWC. 1472 (2015). Linked Data, in: Knowl. Eng. Knowl. Manag., 2014: pp. [32] P.R. Smart, A. Russell, D. Braines, Y. Kalfoglou, J. Bao, and 121–136. doi:10.1007/978-3-319-13704-9_10. N. Shadbolt, A Visual Approach to Semantic Query Design [50] C. Musto, P. Lops, P. Basile, M. de Gemmis, and G. Using a Web-Based Graphical Query Designer, in: Proc. 16th Semeraro, Semantics-aware Graph-based Recommender Int. Conf. Knowl. Eng. Knowl. Manag., 2008: pp. 275–291. Systems Exploiting Linked Open Data, in: UMAP ’16, 2016: doi:10.1007/978-3-540-87696-0_25. pp. 229–237. doi:10.1145/2930238.2930249. [33] M. Zviedris, and G. Barzdins, Viziquer: A tool to explore and [51] F. Bianchi, M. Palmonari, M. Cremaschi, and E. Fersini, query SPARQL endpoints, in: Proc. 8th ESWC, 2011: pp. Actively Learning to Rank Semantic Associations for 441–445. doi:10.1007/978-3-642-21064-8. Personalized Contextual Exploration of Knowledge Graphs, [34] W. Javed, S. Ghani, and N. Elmqvist, PolyZoom : Multiscale in: Proc. 14th Int. Conf. ESWC 2017, 2017: pp. 120–135. and Multifocus Exploration in 2D Visual Spaces, in: CHI’12, [52] X. Zhang, G. Cheng, and Y. Qu, Ontology summarization ACM, 2012. http://dl.acm.org/citation.cfm?id=2207716. based on rdf sentence graph, in: Proc. 16th Int. Conf. World [35] S. Scheider, A. Degbelo, R. Lemmens, C. Van Elzakker, P. Wide Web WWW 07, ACM Press, 2007. Zimmerhof, N. Kostic, J. Jones, and G. Banhatti, Exploratory http://portal.acm.org/citation.cfm?doid=1242572.1242668. querying of SPARQL endpoints in space and time, Semant. [53] R. Wille, Formal Concept Analysis as Mathematical Theory Web. 8 (2017) 65–86. doi:10.3233/SW-150211. of Concepts and Concept Hierarchies, Form. Concept Anal. [36] A.G. Nuzzolese, V. Presutti, A. Gangemi, S. Peroni, and P. (2005) 1–33. doi:10.1007/11528784_1. Ciancarini, Aemoo: Linked Data exploration based on [54] N. Li, and E. Motta, Evaluations of user-driven ontology Knowledge Patterns, Semant. Web. 8 (2017) 87–112. summarization, Lect. Notes Comput. Sci. (Including Subser. doi:10.3233/SW-160222. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 6317 [37] A. Harth, Visinav: Visual web data search and navigation, in: (2010) 544–553. doi:10.1007/978-3-642-16438-5. DEXA, 2009: pp. 214–228. [55] N. Li, and E. Motta, Evaluations of user-driven ontology [38] P. Heim, T. Ertl, and J. Ziegler, Facet graphs: complex summarization, EKAW 17th Int. Conf. Knowl. Eng. Manag. semantic querying made easy, in: L. Aroyo, G. Antoniou, E. by Masses. 6317 (2011) 544–553. doi:10.1007/978-3-642- Hyvönen, A. Teije, H. Stuckenschmidt, L. Cabral, and T. 16438-5. Tudorache (Eds.), ESWC’7th, Springer Berlin Heidelberg, [56] K. Wang, Z. Wang, R. Topor, J.Z. Pan, and G. Antoniou, Berlin, Heidelberg, 2010: pp. 288–302. Eliminating concepts and roles from ontologies in expressive http://dl.acm.org/citation.cfm?id=2176871.2176894 descriptive logics, Comput. Intell. 30 (2014) 205–232. (accessed February 26, 2013). doi:10.1111/j.1467-8640.2012.00442.x. [39] P. Heim, J. Ziegler, and S. Lohmann, GFacet: A browser for [57] G. Troullinou, H. Kondylakis, E. Daskalaki, D. Plexousakis, the web of data, in: IMC-SSW’08, 2008: pp. 49–58. F. Gandon, M. Sabou, and H. Sack, Ontology understanding doi:10.1.1.143.1026. without tears: The-summarization approach, Semant. Web. 8 [40] S. Brunk, and P. Heim, Tfacet: Hierarchical faceted (2017) 797–815. doi:10.3233/SW-170264. exploration of semantic data using well-known interaction [58] G. Troullinou, H. Kondylakis, E. Daskalaki, and D. Plexousakis, RDF Digest : Efficient Summarization of RDF hal-01057056 Discovery Hub : on-the-fly linked data / S KBs, in: Gandon F., Sabou M., Sack H., d’Amato C., exploratory search, (2014). Cudré-Mauroux P., Zimmermann A. Semant. Web. Latest [73] J. Waitelonis, and H. Sack, Towards exploratory video search Adv. New Domains. ESWC 2015. Lect. Notes Comput. Sci. using linked data, Multimed. Tools Appl. 59 (2012) 645–672. Vol 9088. Springer, Cham, 2015. doi:10.1007/978-3-319- doi:10.1007/s11042-011-0733-1. 18818-8. [74] V. Presutti, L. Aroyo, A. Adamou, A. Gangemi, and G. [59] S. Peroni, E. Motta, and M. Aquin, Identifying key concepts Schreiber, Extracting core knowledge from Linked Data, in: in an ontology , through the integration of cognitive COLD, 2011. principles with statistical and topological measures, in: [75] V. Maccatrozzo, M. Terstall, L. Aroyo, and G. Schreiber, ASWC ’08, 2008. SIRUP : Serendipity In Recommendations via User [60] Y. Cai, W.H. Chen, H.F. Leung, Q. Li, H. Xie, R.Y.K. Lau, Perceptions, in: IUI’22, 2017. H. Min, and F.L. Wang, Context-aware ontologies generation [76] R. García, R. Gil, J.M. Gimeno, E. Bakke, and D.R. Karger, with basic level concepts from collaborative tags, BESDUI: A benchmark for end-user structured data user Neurocomputing. 208 (2016) 25–38. interfaces, in: ISWC ’16th, 2016: pp. 65–79. doi:10.1016/j.neucom.2016.02.070. doi:10.1007/978-3-319-46547-0_8. [61] E. Rosch, and B.B. Lloyd, Cognition and Categorization, [77] D. Lamprecht, M. Strohmaier, D. Helic, C. Nyulas, T. Lloydia Cincinnati. pp (1978) 27–48. Tudorache, N.F. Noy, and M.A. Musen, Using ontologies to http://www.worldcat.org/title/cognition-and- model human navigation behavior in information networks: categorization/oclc/3844382&referer=brief_results. A study based on Wikipedia, Semant. Web. 6 (2015) 403–422. [62] J. Poelmans, D.I. Ignatov, S.O. Kuznetsov, and G. Dedene, doi:10.3233/SW-140143. Formal concept analysis in knowledge processing: A survey [78] D.R. Krathwohl, A Revision of Bloom’s Taxonomy : An on applications, Expert Syst. Appl. 40 (2013) 6538–6560. Overview, Theory Pract. 41 (2002). doi:10.1016/j.eswa.2013.05.009. [79] L. Freund, S. Dodson, and R. Kopak, On measuring learning [63] P. Cimiano, A. Hotho, and S. Staab, Learning Concept in search: A position paper, in: Search as Learn. Work., 2016: Hierarchies from Text Corpora Using Formal Concept pp. 1–2. Analysis, J. Artif. Int. Res. 24 (2005) 305–339. [80] S.C. Carr, and B. Thompson, The effects of prior knowledge http://dl.acm.org/citation.cfm?id=1622519.1622528. and schema activation strategies on the inferential reading [64] W.C. Cho, and D. Richards, Improvement of precision and comprehension of children with and without learning recall for information retrieval in a narrow domain: Reuse of disabilities, Learn. Disabil. Q. 19 (1996) 48–61. concepts by formal concept analysis, in: Proc. - doi:10.2307/1511053. IEEE/WIC/ACM Int. Conf. Web Intell. WI 2004, 2004: pp. [81] M. Al-tawil, D. Thakker, and V. Dimitrova, Nudging to 370–376. doi:10.1109/WI.2004.10082. Expand User ’ s Domain Knowledge while Exploring Linked [65] D. Richards, Ad-Hoc and Personal Ontologies: A Data, in: IESD@ISWC, 2014. Prototyping Approach to Ontology Engineering, in: PKAW, [82] D.P. Ausubel, and D. Fitzgerald, ANTECEDENT 2006: pp. 13–24. LEARNING VARIABLES IN SEQUENTIAL VERBAL [66] B. Sertkaya, OntoComP: A Protégé Plugin for Completing LEARNING, (1962). OWL Ontologies, in: L. Aroyo, P. Traverso, F. Ciravegna, P. [83] D.P. Ausubel, The use of advance organizers in the learning Cimiano, T. Heath, E. Hyvönen, R. Mizoguchi, E. Oren, M. and retention of meaningful verbal material, J. Educ. Psychol. Sabou, and E. Simperl (Eds.), Semant. Web Res. Appl. 6th 51 (1960) 267–272. doi:10.1037/h0046669. Eur. Semant. Web Conf. ESWC 2009 Heraklion, Crete, [84] D.P. Ausubel, Cognitive structure and the facilitation of Greece, May 31--June 4, 2009 Proc., Springer Berlin meaningful verbal learning, J. Teach. Educ. 14 (1963) 217– Heidelberg, Berlin, Heidelberg, 2009: pp. 898–902. 222. doi:10.1177/002248716301400220. doi:10.1007/978-3-642-02121-3_78. [85] D.P. Ausubel, J.D. Novak, and Helen Hanesian, Education [67] R. Belohlavek, and M. Trnecka, Basic level of concepts in Psychology: A cognitive View, 1968. formal concept analysis, ICFCA’10. 7278 LNAI (2012) 28– [86] D.P. Ausubel, In Defense of Advance Organizers: A Reply 44. doi:10.1007/978-3-642-29892-9_9. to the Critics*, Rev. Educ. Res. Rev. Educ. Res. Spring. 48 [68] R. Belohlavek, and M. Trnecka, Basic level in formal concept (1978) 251–257. doi:10.3102/00346543048002251. analysis: Interesting concepts and psychological [87] C. Bizer, T. Heath, and T. Berners-Lee, Linked data-the story ramifications, IJCAI Int. Jt. Conf. Artif. Intell. (2013) 1233– so far, Int. J. Semant. Web Inf. Syst. (2009). 1239. doi:10.4018/jswis.2009081901. [69] A. Bonifati, R. Ciucanu, and A. Lemay, Learning Path [88] G. V Jones, Identifying Basic Categories, Psychol. Bull. 94 Queries on Graph Databases, in: Proc. 18th Int. Conf. (1983) 423–428. doi:10.1037//0033-2909.94.3.423. Extending Database Technol., 2015. [89] M.A. Corter, James E.; Gluck, Explaining Basic Categories: doi:10.5441/002/edbt.2015.11. Feature Predictability and Information, (1992). [70] K. Anyanwu, and A. Sheth, ρ-Queries Enabling Querying for [90] M. Al-Tawil, V. Dimitrova, D. Thakker, and B. Bennett, Semantic Associationson the Semantic Web.pdf, in: Proc. Identifying knowledge anchors in a data graph, in: HT 2016 12th Int. Conf. World Wide Web, 2003: pp. 690–699. - Proc. 27th ACM Conf. Hypertext Soc. Media, 2016. doi:10.1145/775152.775249. doi:10.1145/2914586.2914637. [71] P. Heim, S. Hellmann, J. Lehmann, and S. Lohmann, [91] M. Al-Tawil, V. Dimitrova, D. Thakker, and Alexandra RelFinder : Revealing Relationships in RDF Knowledge Poulovassilis, Evaluating Knowledge Anchors in Data Bases, (n.d.). Graphs against Basic Level Objects, in: ICWE’17 Int. Conf. [72] N. Marie, F. Gandon, R. Myriam, F. Rodio, N. Marie, F. Web Eng., 2017. Gandon, R. Myriam, F. Rodio, D. Hub, N. Marie, I. Sophia- [92] M. Al-Tawil, Intelligent Support for Exploration of Data antipolis, F. Gandon, I. Sophia-antipolis, S. Antipolis, M. Graphs, University of Leeds., 2017. Ribière, F. Rodio, and A.B. Labs, Discovery Hub : on-the-fly [93] Z. Wu, and M. Palmer, Verbs semantics and lexical selection, linked data exploratory search To cite this version : HAL Id : Proc. 32nd Annu. Meet. Assoc. Comput. Linguist. -. (1994) 133–138. doi:10.3115/981732.981751. [99] Sandra G. HartLowell E. Staveland, Development of NASA- [94] G.C. Liang Zheng, Jiang Xu, Jidong Jiang, Yuzhong Qu, TLX (Task Load Index): Results of Empirical and Iterative Entity Navigation via Co-clustering Semantic Links Theoretical Research, Adv. Psychol. 52 (1988) 139–183. and Entity Classes, in: 13th Int. Conf. ESWC, 2016. [100] C. Leacock, and M. Chodorow, Combining local context and [95] B. Kules, R. Capra, M. Banta, and T. Sierra, What Do WordNet similarity for word sense identification, WordNet Exploratory Searchers Look at in a Faceted Search Interface?, An Electron. Lex. Database. 49 (1998) 265–283. JCDL ’09 Proc. 9th ACM/IEEE-CS Jt. Conf. Digit. Libr. . [101] et al Dekang Lin, An information-theoretic definition of (2009) 313–322. doi:10.1145/1555400.1555452. similarity, Icml. 98 (1998) 296–304. [96] T. Nunes, and D. Schwabe, Frameworks of Information [102] M. Al-tawil, V. Dimitrova, and D. Thakker, Using Basic Exploration - Towards the Evaluation of Exploration Level Concepts in a Linked Data Graph to Detect User’s Systems Conceptual View of an Exploration Framework, in: Domain Familiarity, in: UMAP, Dublin, Ireland, 2015. IESD@ISWC, 2016. [103] A.S. Dadzie, and M. Rowe, Approaches to visualising Linked [97] J.W. Tanaka, and M. Taylor, Object categories and expertise: Data: A survey, Semant. Web. 2 (2011) 89–124. Is the basic-level in the eye of the beholder?, Cogn. Psychol. doi:10.3233/SW-2011-0037. 23 (1991) 457–482. doi:10.1016/0010-0285(91)90016-H. [104] G.A. Miller, The magical number seven, plus or minus two: [98] S.G. Hart, and L.E. Staveland, Development of NASA-TLX some limits on our capacity for processing information, (Task Load Index): Results of Empirical and Theoretical Psychol. Rev. 63 (1956). Research, Adv. Psychol. 52 (1988) 139–183. doi:10.1016/S0166-4115(08)62386-9.