Skosprovider Documentation Release 0.7.0

Koen Van Daele

Feb 19, 2020

Contents

1 Introduction 1 1.1 Other providers...... 1 1.2 Deviations from SKOS...... 2 1.3 Support...... 2

2 Usage 3

3 Development 7

4 API Documentation 9 4.1 Skos module...... 9 4.2 Providers module...... 14 4.3 Registry module...... 24 4.4 Uri module...... 27 4.5 JsonLd module...... 28 4.6 Exceptions module...... 29 4.7 Utils module...... 29

5 History 31 5.1 0.7.0 (2020-01-19)...... 31 5.2 0.6.1 (2017-07-16)...... 31 5.3 0.6.0 (2016-08-09)...... 32 5.4 0.5.3 (2015-06-24)...... 32 5.5 0.5.2 (2015-03-02)...... 32 5.6 0.5.1 (2015-03-02)...... 32 5.7 0.5.0 (2014-12-18)...... 33 5.8 0.4.2 (2014-10-16)...... 33 5.9 0.4.1 (2014-10-15)...... 33 5.10 0.4.0 (2014-10-02)...... 33 5.11 0.3.0 (2014-05-14)...... 34 5.12 0.2.1 (2013-12-06)...... 34 5.13 0.2.0 (2013-05-16)...... 34 5.14 0.1.3 (2013-03-22)...... 34 5.15 0.1.2 (2013-02-07)...... 35 5.16 0.1.1 (2012-12-11)...... 35 5.17 0.1.0...... 35

i 6 Glossary 37

7 Indices and tables 39

Python Module Index 41

Index 43

ii CHAPTER 1

Introduction

Skosprovider provides an interface that can be included in an application to allow it to talk to different SKOS vocabularies. These vocabularies could be deﬁned locally or accessed remotely through webservices. Adhering to this interface in you application decouples your application and the actual thesaurus. This makes unit testing easy because it allows you to swap a remote and a local implementation. It also makes it easy to switch from a simple, static implementation based on a csv ﬁle to a more complete implementation using your relation database of choice. One of the main goals of this project is to be able to build an application that can use thesauri or vocabularies without knowing upfront what these might be or where they might come from. This could be for an application that allows cataloguing things, but where it can be expected that different instances will require different thesauri or would need to be able to talk to existing vocabulary systems. Some sample providers are present in this package. The skosprovider.providers.DictionaryProvider uses a simple python dict as the datastore. It can be considered the reference implementation for the skosprovider.providers.VocabularyProvider interface. Most likely you will want to implement a provider for your own SKOS, vocabulary or thesaurus system.

1.1 Other providers

Currently the following other providers exist: • Skosprovider_sqlalchemy: An implementation of the VocabularyProvider interface with a SQLAlchemy backend. This allows using a RDBMS for reading, but also writing, SKOS concepts. • Skosprovider_rdf: An implementation of the VocabularyProvider interface with a RDFLib backend. This allows using a SKOS RDF ﬁle as the source for a provider, but also dumping a skosprovider to a SKOS RDF ﬁle. • Skosprovider_atramhasis: The AtramhasisProvider lets you interact with an Atramhasis instance. • Skosprovider_getty: An implemenation of the VocabularyProvider against the Linked Open Data vocabularies published by the Getty Research Institute at http://vocab.getty.edu such as the Art and Architecture Thesaurus (AAT) and the Thesaurus of Geographic Names (TGN).

1 Skosprovider Documentation, Release 0.7.0

• Skosprovider_heritagedata: An implementation of the VocabularyProvider against the vocabularies published by EH, RCAHMS and RCAHMW at heritagedata.org. There also exists a library to integrate Skosprovider with Pyramid at pyramid_skosprovider. This allows you to embed a set of REST services in a Pyramid application that expose SKOSproviders as JSON services that can be consumed by eg. Javascript clients or other clients. For those who are looking to build a vocabulary, there’s also Atramhasis, an online SKOS vocabulary editor that builds upon this library and others. Atramhasis can function as the central SKOS registry for an organisation looking to manage its own thesauri and other controlled vocabularies. It provides a public website that allows people to browse you vocabularies and a private interface that allows vocabulary editors to create, edit and delete concepts and collections. By using other Skosproviders Atramhasis can import concepts and collections from other thesauri, saving you the trouble of having to write your own controlled vocabulary from scratch.

1.2 Deviations from SKOS

In a few places we’ve deviated a bit from the SKOS standard: • While SKOS technically allows for things like a broader/narrower relation between concepts in different conceptschemes, Skosprovider assumes that all hierarchical or associative relations should be between concepts in the same conceptscheme. For relations between concepts in different schemes, the SKOS mapping properties (skos:mappingRelation, skos:closeMatch, skos:exactmatch, . . . ) should be used. These are supported by Skosprovider since version 0.4.0. • The SKOS standard allows a concept that is marked as a topConcept to have a broader concept. Skosprovider expects that the concepts returned by the skosprovider.providers.VocabularyProvider. get_top_concepts() do not have any broader concepts. • The SKOS ontology ony describes a SKOS:member predicate to indicate that a collection has certain members. There’s an implicit reverse side to this relation (eg. a concept is a member of a collection). We’ve standardised this on the member_of property that’s available on a skosprovider.skos.Concept and a skosprovider.skos.Collection. • SKOS provides no way for specifying where in a hierarchy a skosprovider.skos.Collection should be placed. Since this is a fairly standard requirement for most thesauri, we have implemented this by looking at the SKOS-THES speciﬁcation. We have borrowed the skosprovider.skos.Concept. subordinate_arrays and skosprovider.skos.Collection.superordinates properties from this speciﬁcation. In effect, it turns a SKOS Collection that has one or more superordinates into a The- saurusArray. Since 0.7.0 it’s possible to explicitly state if the member of a collection that has a superordinate concept should be seen as narrower concepts of that superordinate concept with the skosprovider.skos. Collection.infer_concept_relations. By default this is set to True. If you want to model a collection that does not contain narrower concepts of it’s superordinate, set it to False. This will mainly stop search expansion using the skosprovider.providers.VocabularyProvider.expand() method.

1.3 Support

If you have questions regarding Skosprovider, feel free to contact us. Any bugs you ﬁnd or feature requests you have, you can add to our issue tracker. If you’re unsure if something is a bug or intentional, or you just want to have a chat about this library or SKOS in general, feel free to join the Atramhasis discussion forum. While these are separate software projects, they are being run by the same people and they integrate rather tightly.

2 Chapter 1. Introduction CHAPTER 2

Usage

The basic idea of Skosprovider is very much about the idea of loose coupling, allowing two pieces of software to talk to each other while having as little knowledge of each other as possible. In this case, software A (eg. a webapp) can make use of 1 or more thesauri, controlled vocabularies, authority files and other vocabularies without have to no the details of the vocabularies. This has been achieved by defining an interface of the most common operations needed to integrate an vocabulary in an application. It’s complex and rich enough to allow for most operations that an application might need. And of course, in special cases you might consider expanding the interface. If you do so, please let us know. Your needs might be similar to someone else’s needs and might warrant an extension to the interface. While the interface is very clear on the operations supported and the data it shoudl receive from the provider, it makes not statement whatsoever about the format the data should be in when the provider receives it. So, while a client knows that each provider has a method to find a concept by an id and what this method will return, the client does not need to know where the provider goes to find this concept. One provider might look this up in a CSV file, another one in a database and yet another one through a webservice. In a way, the provider transforms the data from the source format to a common view of the SKOS data model. Because of this, the only part where the different providers are different is when instantiating the provider, since they need to be configured for their data source. Eg., the skosprovider.providers.DictionaryProvider is a very simple provider that requires a list of dictionaries to operate on. Apart from this one very specific data element, there are a few configuration parameters that are passed to every existing provider. Every provider requires that a parameter metadata is passed to it. This is a dictionary that has one required parameter, id. This is an id that is assigned to the provider. It’s used when registering the provider with a skosprovider.registry.Registry and for a few other things. Other than this required parameter, a metadata object can also set a default_language for the provider and register a number of subjects for the provider. Upon instantiation, a provider can also be passed a skosprovider.uri.UriGenerator. This is an object that allows the provider to generate URI’s for it’s concepts and collections. If you do not pass in a uri generator, the provider will set one up for you. Finally, you can also pass in a skosprovider.skos.ConceptScheme. This is the concept scheme the providers represents. Again, if you do no pass in a concept scheme, the provider will create a default scheme.

provider= DictionaryProvider( { (continues on next page)

3 Skosprovider Documentation, Release 0.7.0

(continued from previous page) 'id':'TREES', 'default_language':'nl', 'subject':['biology'] }, [larch, chestnut, species], uri_generator=UriPatternGenerator('http://id.trees.org/types/%s'), concept_scheme=ConceptScheme('http://id.trees.org') )

Providing you have gotten hold of a Skosprovider, you now have an object with several methods. Largely these can mainly be grouped into a two different categories: First there are methods that return a single skosprovider.skos.Concept or skosprovider.skos. Collection. These methods allow you to retrieve an individual item based on one of two possible identiﬁers. The ﬁrst method, get_by_id(), retrieves an item based on an id that is known internally by the provider. This id is not necessarily globally unique. It’s only required to be unique to a certain provider. The second method, get_by_uri(), retrieves an item based on it’s URI. Since a URI is guaranteed to be globally unique. Quite often the URI wil also contain the id, but this is not mandatory.

# Get a concept or collection by id provider.get_by_id(1)

# Get a concept or collection by uri provider.get_by_uri('http://id.trees.org/types/1')

A second group of methods return a list of concepts or collections. In this case the concepts or collections are only partially output. For each concept or collection an id, uri, type (concept or collection) and a label are returned. Each of these methods also takes an optional keyword language that detemines in what language a label is rendered. A second optional keyword, sort allows the client to specify how the list should be sorted. Options here are id, label and sortlabel. This last parameter allows sorting on a special sortLabel that can be assigned to concepts and collections. This way, arbitrary sorting order can be created by the editor of a scheme. When sorting, the sort_order keyword can be used to set the sort order. One method, get_all() returns all concept and collections in a certain provider. It’s rarely used and might possibly not make sense in very large providers. It’s mainly there as a convenience method for small providers and in testing situations. More useful is get_top_concepts(). This methods returns all top concepts (not collections) in the provider. These are the concepts that have no broader concepts. There are also two related methods that can help in building a display hierarchy. As opposed to get_top_concepts(), these do return both concepts and collections and can thus actually be used to create a sensible display hierarchy for a vocabulary. The ﬁrst one, get_top_display(), returns the top of a display hierarchy. These would be the concepts and collections that form the top of this hierarchy. To descend the hierachy, you would call get_children_display(). A ﬁnal method in this group of methods is actually the most important one in this group. By calling skosprovider. providers.VocabularyProvider.find(), you can search the provider for concepts or collections matching your criteria. You do this by passing in a query parameter to this method. This way you can ask the provider to search for certain labels (eg. churches), to only search certain types (concept or collection) or to only search within a cerain collection. As always in this category of methods, you can control in what languages labels should be returned using the language keyword.

# Get all concepts and collections in a provider # If possible, show a Dutch(as spoken in Belgium) label provider.get_all(language='nl-BE')

(continues on next page)

4 Chapter 2. Usage Skosprovider Documentation, Release 0.7.0

(continued from previous page) # Get all concepts and collections in a provider # If possible, show a Dutch(as spoken in Belgium) label # and order them by this label provider.get_all(language='nl-BE', sort='label', sort_order='asc')

# Get the top concepts in a provider provider.get_top_concepts()

# Find anything that has a label of church. provider.find({'label':'church'})

# Get the top of a display hierarchy provider.get_top_display(sort='id', sort_order='desc')

# Get the children to display in a hierarchy concept 1 # If possible, show a French(as spoken in Belgium) label provider.get_children_display(1, language='fr-BE')

Apart from the two categories, there are a couple more miscellaneous methods. The most interesting one of these is skosprovider.providers.VocabularyProvider.expand(). This methods take a certain concept or collection id as argument and returns a list of all concept ids that are “underneath” this concept or collection. This is mainly intended to be used when querying datasets. It allows a client to broaden the scope of a search. Eg. when searching for churches, the expand method might return the ids for both churches and cathedrals.

# Get all concepts underneath a concept or collection provider.expand(1)

5 Skosprovider Documentation, Release 0.7.0

6 Chapter 2. Usage CHAPTER 3

Development

While skosprovider is still in development, the basic premise is fairly stable and the API changes have been fairly minor from version to version. We do continue to reﬁne the project and make the providers more expressive. We try to cover as much code as we can with unit tests. You can run them using tox or directly through nose. When providing a pull request, please run the unit tests ﬁrst and make sure they all pass. Please provide new unit tests to maintain 100% coverage.

$ tox # No coverage $ py.test # Coverage $ py.test --cov skosprovider --cov-report term-missing # Only run a subset of the tests $ py.test skosprovider/tests/test_registry.py

7 Skosprovider Documentation, Release 0.7.0

8 Chapter 3. Development CHAPTER 4

API Documentation

4.1 Skos module

This module contains a read-only model of the SKOS specification. To complement the SKOS specification, some elements were borrowed from the SKOS-THES specification (eg. superordinate and subordinate array). New in version 0.2.0. class skosprovider.skos.Collection(id, uri=None, concept_scheme=None, labels=[], notes=[], sources=[], members=[], member_of=[], superordinates=[], infer_concept_relations=True) A SKOS Collection. concept_scheme = None The ConceptScheme this Collection is a part of. id = None An id for this Collection within a vocabulary infer_concept_relations = True Should member concepts of this collection be seen as narrower concept of a superordinate of the collection? label(language=’any’) Provide a single label for this collection. This uses the label() function to determine which label to return. Parameters language (string) – The preferred language to receive the label in. This should be a valid IANA language tag. Return type skosprovider.skos.Label or False if no labels were found. labels = [] A lst of skosprovider.skos.label instances.

9 Skosprovider Documentation, Release 0.7.0

member_of = [] A lst of collection ids. members = [] A lst of concept or collection ids. notes = [] A lst of skosprovider.skos.Note instances. sources = [] A lst of skosprovider.skos.Source instances. superordinates = [] A lst of concept ids. type = 'collection' The type of this concept or collection. eg. ‘collection’ uri = None A proper uri for this Collection class skosprovider.skos.Concept(id, uri=None, concept_scheme=None, labels=[], notes=[], sources=[], broader=[], narrower=[], related=[], mem- ber_of=[], subordinate_arrays=[], matches={}) A SKOS Concept. broader = [] A lst of concept ids. concept_scheme = None The ConceptScheme this Concept is a part of. id = None An id for this Concept within a vocabulary eg. 12345 label(language=’any’) Provide a single label for this concept. This uses the label() function to determine which label to return. Parameters language (string) – The preferred language to receive the label in. This should be a valid IANA language tag. Return type skosprovider.skos.Label or False if no labels were found. labels = [] A lst of Label instances. matches = ({},) A dictionary. Each key is a matchtype and contains a list of URI’s. matchtypes = ['close', 'exact', 'related', 'broad', 'narrow'] Matches with Concepts in other ConceptSchemes. This dictionary contains a key for each type of Match (close, exact, related, broad, narrow). Attached to each key is a list of URI’s. member_of = [] A lst of collection ids.

10 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

narrower = [] A lst of concept ids. notes = [] A lst of Note instances. related = [] A lst of concept ids. sources = [] A lst of skosprovider.skos.Source instances. subordinate_arrays = [] A list of collection ids. type = 'concept' The type of this concept or collection. eg. ‘concept’ uri = None A proper uri for this Concept eg. http://id.example.com/skos/trees/1 class skosprovider.skos.ConceptScheme(uri, labels=[], notes=[], sources=[], languages=[]) A SKOS ConceptScheme. Parameters • uri (string)–A URI for this conceptscheme. • labels (list) – A list of skosprovider.skos.Label instances. • notes (list) – A list of skosprovider.skos.Note instances. label(language=’any’) Provide a single label for this conceptscheme. This uses the label() function to determine which label to return. Parameters language (string) – The preferred language to receive the label in. This should be a valid IANA language tag. Return type skosprovider.skos.Label or False if no labels were found. labels = [] A lst of skosprovider.skos.label instances. languages = [] A lst of languages that are being used in the ConceptScheme. There’s no guarantuee that labels or notes in other languages do not exist. notes = [] A lst of skosprovider.skos.Note instances. sources = [] A lst of skosprovider.skos.Source instances. uri = None A URI for this conceptscheme. class skosprovider.skos.Label(label, type=’prefLabel’, language=’und’) A SKOS Label.

4.1. Skos module 11 Skosprovider Documentation, Release 0.7.0

static is_valid_type(type) Check if the argument is a valid SKOS label type. Parameters type (string) – The type to be checked. label = None The label itself (eg. churches, trees, Spitﬁres,...) language = 'und' The language the label is in (eg. en, en-US, nl, nl-BE). type = 'prefLabel' The type of this label (prefLabel, altLabel, hiddenLabel, ‘sortLabel’). valid_types = ['prefLabel', 'altLabel', 'hiddenLabel', 'sortLabel'] The valid types for a label class skosprovider.skos.Note(note, type=’note’, language=’und’, markup=None) A SKOS Note. static is_valid_markup(markup) Check the argument is a valid type of markup. Parameters markup (string) – The type to be checked. static is_valid_type(type) Check if the argument is a valid SKOS note type. Parameters type (string) – The type to be checked. language = 'und' The language the label is in (eg. en, en-US, nl, nl-BE). markup = None What kind of markup does the note contain? If not None, the note should be treated as a certain type of markup. Currently only HTML is allowed. note = None The note itself type = 'note' The type of this note ( note, deﬁnition, scopeNote, . . . ). valid_types = ['note', 'changeNote', 'definition', 'editorialNote', 'example', 'historyNote', 'scopeNote'] The valid types for a note. class skosprovider.skos.Source(citation, markup=None) A Source for a concept, collection or scheme. citation = None A bibliographic citation for this source. static is_valid_markup(markup) Check the argument is a valid type of markup. Parameters markup (string) – The type to be checked. markup = None What kind of markup does the source contain? If not None, the source should be treated as a certain type of markup. Currently only HTML is allowed. skosprovider.skos.dict_to_label(dict) Transform a dict with keys label, type and language into a Label.

12 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

Only the label key is mandatory. If type is not present, it will default to prefLabel. If language is not present, it will default to und. If the argument passed is not a dict, this method just returns the argument. skosprovider.skos.dict_to_note(dict) Transform a dict with keys note, type and language into a Note. Only the note key is mandatory. If type is not present, it will default to note. If language is not present, it will default to und. If markup is not present it will default to None. If the argument passed is already a Note, this method just returns the argument. skosprovider.skos.dict_to_source(dict) Transform a dict with key ‘citation’ into a Source. If the argument passed is already a Source, this method just returns the argument. skosprovider.skos.filter_labels_by_language(labels, language, broader=False) Filter a list of labels, leaving only labels of a certain language. Parameters • labels (list) – A list of Label. • language (str) – An IANA language string, eg. nl or nl-BE. • broader (boolean) – When true, will also match nl-BE when filtering on nl. When false, only exact matches are considered. skosprovider.skos.find_best_label_for_type(labels, language, labeltype) Find the best label for a certain labeltype. Parameters • labels (list) – A list of Label. • language (str) – An IANA language string, eg. nl or nl-BE. • labeltype (str) – Type of label to look for, eg. prefLabel. skosprovider.skos.label(labels=[], language=’any’, sortLabel=False) Provide a label for a list of labels. The items in the list of labels are assumed to be either instances of Label, or dicts with at least the key label in them. These will be passed to the dict_to_label() function. This method tries to find a label by looking if there’s a pref label for the specified language. If there’s no pref label, it looks for an alt label. It disregards hidden labels. While matching languages, preference will be given to exact matches. But, if no exact match is present, an inexact match will be attempted. This might be because a label in language nl-BE is being requested, but only nl or even nl-NL is present. Similarly, when requesting nl, a label with language nl-NL or even nl-Latn-NL will also be considered, providing no label is present that has an exact match with the requested language. If language ‘any’ was specified, all labels will be considered, regardless of language. To find a label without a specified language, pass None as language. If a language or None was specified, and no label could be found, this method will automatically try to find a label in some other language. Finally, if no label could be found, None is returned. Parameters

4.1. Skos module 13 Skosprovider Documentation, Release 0.7.0

• language (string) – The preferred language to receive the label in. This should be a valid IANA language tag. • sortLabel (boolean) – Should sortLabels be considered or not? If True, sortLabels will be preferred over prefLabels. Bear in mind that these are still language dependent. So, it’s possible to have a different sortLabel per language. Return type A Label or None if no label could be found. skosprovider.skos.valid_markup = [None, 'HTML'] Valid types of markup for a note or a source.

4.2 Providers module

This module provides an abstraction of controlled vocabularies. This abstraction allows our application to work with both local and remote vocabs (be they SOAP, REST, XML-RPC or something else). The basic idea is that we have skos providers. Each provider is an instance of a VocabularyProvider. The same class can thus be reused with different configurations to handle different vocabs. Generally speaking, every instance of a certain VocabularyProvider will deal with concepts and collections from a single conceptscheme. class skosprovider.providers.VocabularyProvider(metadata, **kwargs) An interface that all vocabulary providers must follow. __init__(metadata, **kwargs) Create a new provider and register some metadata. Parameters • uri_generator – An object that implements the skosprovider.uri. UriGenerator interface. • concept_scheme –A ConceptScheme. If not present, a default ConceptScheme will be created with a uri generated by the DefaultConceptSchemeUrnGenerator in combination with the provider id. • allowed_instance_scopes – A list of instance_scope keywords that can be us by the registry to check if this provider is compatible. • metadata (dict) – Metadata essential to this provider. Possible metadata: – id: A unique identifier for the vocabulary. Required. – default_language: Used to determine what language to use when returning labels if no language is specified. Will default to en if not specified. – subject: A list of subjects or tags that define what the provider is about or what the provider can handle. This information can then be used when querying a Registry for providers. – dataset:A dict detailing the dataset the conceptscheme and all concepts and collections are part of. Currently the contents of the dictionary are undefined except for a uri attribute that must be present. allowed_instance_scopes = None Indicates what instance_scopes this provider can safely accomodate. This will be checked by the registry upon registering a provider.

14 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

concept_scheme = None The ConceptScheme this provider serves. expand(id) Expand a concept or collection to all it’s narrower concepts. This method should recurse and also return narrower concepts of narrower concepts. If the id passed belongs to a skosprovider.skos.Concept, the id of the concept itself should be include in the return value. If the id passed belongs to a skosprovider.skos.Collection, the id of the collection itself must not be present in the return value In this case the return value includes all the member concepts and their narrower concepts. Parameters id – A concept or collection id. Return type A list of id’s or False if the concept or collection doesn’t exist. find(query, **kwargs) Find concepts that match a certain query. Currently query is expected to be a dict, so that complex queries can be passed. You can use this dict to search for concepts or collections with a certain label, with a certain type and for concepts that belong to a certain collection.

# Find anything that has a label of church. provider.find({'label':'church'})

# Find all concepts that are a part of collection 5. provider.find({'type':'concept','collection':{'id':5})

# Find all concepts, collections or children of these # that belong to collection 5. provider.find({'collection':{'id':5,'depth':'all'})

# Find anything that has a label of church. # Preferentially display a label in Dutch. provider.find({'label':'church'}, language='nl')

# Find anything that has a match with an external concept # Preferentially display a label in Dutch. provider.find({ 'matches':{ 'uri':'http://id.python.org/different/types/of/trees/nr/1/the/larch' }}, language='nl')

# Find anything that has a label of lariks with a close match to an external

˓→concept # Preferentially display a label in Dutch. provider.find({ 'label':'lariks', 'matches':{ 'type':'close', 'uri':'http://id.python.org/different/types/of/trees/nr/1/the/larch' }}, language='nl')

Parameters • query – A dict that can be used to express a query. The following keys are permitted:

4.2. Providers module 15 Skosprovider Documentation, Release 0.7.0

– label: Search for something with this label value. An empty label is equal to searching for all concepts. – type: Limit the search to certain SKOS elements. If not present or None, all is assumed:

* concept: Only return skosprovider.skos.Concept instances. * collection: Only return skosprovider.skos.Collection instances. * all: Return both skosprovider.skos.Concept and skosprovider. skos.Collection instances. – collection: Search only for concepts belonging to a certain collection. This argument should be a dict with two keys:

* id: The id of a collection. Required. * depth: Can be members or all. Optional. If not present, members is assumed, meaning only concepts or collections that are a direct member of the collection should be considered. When set to all, this method should return concepts and collections that are a member of the collection or are a narrower concept of a member of the collection. – matches: Search only for concepts having a match to a certain external concept. Since collections can’t have matches, this automatically excludes collections. The argument with two keys:

* uri: The uri of the concept to match. Required. * type: The type of match, see matchtypes for the full list of options. • language (string) – Optional. If present, it should be a language-tag. This language- tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en.

16 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

• sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and finally falls back to en. get_by_id(id) Get all information on a concept or collection, based on id. Providers should assume that all id’s passed are strings. If a provider knows that internally it uses numeric identifiers, it’s up to the provider to do the typecasting. Generally, this should not be done by changing the id’s themselves (eg. from int to str), but by doing the id comparisons in a type agnostic way. Since this method could be used to find both concepts and collections, it’s assumed that there are no id collisions between concepts and collections. Return type skosprovider.skos.Concept or skosprovider.skos. Collection or False if the concept or collection is unknown to the provider. get_by_uri(uri) Get all information on a concept or collection, based on a URI. Return type skosprovider.skos.Concept or skosprovider.skos. Collection or False if the concept or collection is unknown to the provider. get_children_display(id, **kwargs) Return a list of concepts or collections that should be displayed under this concept or collection. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc • id (str) – A concept or collection id. Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection

4.2. Providers module 17 Skosprovider Documentation, Release 0.7.0

• type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en. get_metadata() Get some metadata on the provider or the vocab it represents. Return type Dict. get_top_concepts(**kwargs) Returns all top-level concepts in this provider. Top-level concepts are concepts that have no broader concepts themselves. They might have narrower concepts, but this is not mandatory. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts, NOT collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en. get_top_display(**kwargs) Returns all concepts or collections that form the top-level of a display hierarchy. As opposed to the get_top_concepts(), this method can possibly return both concepts and collections. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc

18 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en. get_vocabulary_id() Get an identiﬁer for the vocabulary. Return type String or number. uri_generator = None The UriGenerator responsible for generating URIs for this provider. class skosprovider.providers.MemoryProvider(metadata, list, **kwargs) A provider that keeps everything in memory. The data is passed in the constructor of this provider as a lst of skosprovider.skos.Concept and skosprovider.skos.Collection instances. __init__(metadata, list, **kwargs) Parameters • metadata (dict) – A dictionary with keywords like language. • list (list) – A list of skosprovider.skos.Concept and skosprovider.skos.Collection instances. • case_insensitive (Boolean) – Should searching for labels be done case- insensitive? case_insensitive = True Is searching for labels case insensitive? By default a search for a label is done case insensitive. Older versions of this provider were case sen- sitive. If this behaviour is desired, this can be triggered by providing a case_insensitive keyword to the constructor. expand(id) Expand a concept or collection to all it’s narrower concepts. This method should recurse and also return narrower concepts of narrower concepts. If the id passed belongs to a skosprovider.skos.Concept, the id of the concept itself should be include in the return value. If the id passed belongs to a skosprovider.skos.Collection, the id of the collection itself must not be present in the return value In this case the return value includes all the member concepts and their narrower concepts. Parameters id – A concept or collection id. Return type A list of id’s or False if the concept or collection doesn’t exist. find(query, **kwargs) Find concepts that match a certain query.

4.2. Providers module 19 Skosprovider Documentation, Release 0.7.0

Currently query is expected to be a dict, so that complex queries can be passed. You can use this dict to search for concepts or collections with a certain label, with a certain type and for concepts that belong to a certain collection.

# Find anything that has a label of church. provider.find({'label':'church'})

# Find all concepts that are a part of collection 5. provider.find({'type':'concept','collection':{'id':5})

# Find all concepts, collections or children of these # that belong to collection 5. provider.find({'collection':{'id':5,'depth':'all'})

# Find anything that has a label of church. # Preferentially display a label in Dutch. provider.find({'label':'church'}, language='nl')

# Find anything that has a label of lariks with a close match to an external

Parameters • query – A dict that can be used to express a query. The following keys are permitted: – label: Search for something with this label value. An empty label is equal to searching for all concepts. – type: Limit the search to certain SKOS elements. If not present or None, all is assumed:

* concept: Only return skosprovider.skos.Concept instances. * collection: Only return skosprovider.skos.Collection instances. * all: Return both skosprovider.skos.Concept and skosprovider.skos.Collection instances. – collection: Search only for concepts belonging to a certain collection. This argument should be a dict with two keys:

* id: The id of a collection. Required. * depth: Can be members or all. Optional. If not present, members is assumed, meaning only concepts or collections that are a direct member of the collection should be considered. When set to all, this method should

20 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

return concepts and collections that are a member of the collection or are a narrower concept of a member of the collection. – matches: Search only for concepts having a match to a certain external concept. Since collections can’t have matches, this automatically excludes collections. The argument with two keys:

* uri: The uri of the concept to match. Required. * type: The type of match, see matchtypes for the full list of options. • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en.

get_all(**kwargs) Returns all concepts and collections in this provider. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en.

4.2. Providers module 21 Skosprovider Documentation, Release 0.7.0

get_by_id(id) Get all information on a concept or collection, based on id. Providers should assume that all id’s passed are strings. If a provider knows that internally it uses numeric identifiers, it’s up to the provider to do the typecasting. Generally, this should not be done by changing the id’s themselves (eg. from int to str), but by doing the id comparisons in a type agnostic way. Since this method could be used to find both concepts and collections, it’s assumed that there are no id collisions between concepts and collections. Return type skosprovider.skos.Concept or skosprovider.skos. Collection or False if the concept or collection is unknown to the provider. get_by_uri(uri) Get all information on a concept or collection, based on a URI. Return type skosprovider.skos.Concept or skosprovider.skos. Collection or False if the concept or collection is unknown to the provider. get_children_display(id, **kwargs) Return a list of concepts or collections that should be displayed under this concept or collection. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc • id (str) – A concept or collection id. Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and finally falls back to en. get_top_concepts(**kwargs) Returns all top-level concepts in this provider. Top-level concepts are concepts that have no broader concepts themselves. They might have narrower concepts, but this is not mandatory. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept.

22 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

• sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts, NOT collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en. get_top_display(**kwargs) Returns all concepts or collections that form the top-level of a display hierarchy. As opposed to the get_top_concepts(), this method can possibly return both concepts and collections. Parameters • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. • sort (string) – Optional. If present, it should either be id, label or sortlabel. The sortlabel option means the providers should take into account any sortLabel if present, if not it will fallback to a regular label to sort on. • sort_order (string) – Optional. What order to sort in: asc or desc. Defaults to asc Returns A lst of concepts and collections. Each of these is a dict with the following keys: • id: id within the conceptscheme • uri: uri of the concept or collection • type: concept or collection • label: A label to represent the concept or collection. It is determined by looking at the language parameter, the default language of the provider and ﬁnally falls back to en. class skosprovider.providers.DictionaryProvider(metadata, list, **kwargs) A simple vocab provider that use a python list of dicts. The provider expects a list with elements that are dicts that represent the concepts. __init__(metadata, list, **kwargs) Parameters • metadata (dict) – A dictionary with keywords like language.

4.2. Providers module 23 Skosprovider Documentation, Release 0.7.0

• list (list) – A list of skosprovider.skos.Concept and skosprovider.skos.Collection instances. • case_insensitive (Boolean) – Should searching for labels be done case- insensitive? class skosprovider.providers.SimpleCsvProvider(metadata, reader, **kwargs) A provider that reads a simple csv format into memory. The supported csv format looks like this: ,,, This provider essentialy provides a ﬂat list of concepts. This is commonly associated with short lookup-lists. New in version 0.2.0. __init__(metadata, reader, **kwargs) Parameters • metadata – A metadata dictionary. • reader – A csv reader.

4.3 Registry module

This module provides a registry for skos providers. This registry helps us ﬁnd providers during runtime. We can also apply some operations to all or several providers at the same time. class skosprovider.registry.Registry(instance_scope=’single’, metadata={}) This registry collects all skos providers. concept_scheme_uri_map = {} Dictionary mapping concept scheme uri’s to vocabulary id’s. find(query, **kwargs) Launch a query across all or a selection of providers.

# Find anything that has a label of church in any provider. registry.find({'label':'church'})

# Find anything that has a label of church with the BUILDINGS provider. # Attention, this syntax was deprecated in version 0.3.0 registry.find({'label':'church'}, providers=['BUILDINGS'])

# Find anything that has a label of church with the BUILDINGS provider. registry.find({'label':'church'}, providers={'ids':['BUILDINGS']})

# Find anything that has a label of church with a provider # marked with the subject 'architecture'. registry.find({'label':'church'}, providers={'subject':'architecture'})

# Find anything that has a label of church in any provider. # If possible, display the results with a Dutch label. registry.find({'label':'church'}, language='nl')

# Find anything that has a match with an external concept # If possible, display the results with a Dutch label. registry.find({ (continues on next page)

24 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

(continued from previous page) 'matches':{ 'uri':'http://id.python.org/different/types/of/trees/nr/1/the/larch' }}, language='nl')

# Find anything that has a label of lariks with a close match to an external

˓→concept # If possible, display the results with a Dutch label. provider.find({ 'matches':{ 'label':'lariks', 'type':'close', 'uri':'http://id.python.org/different/types/of/trees/nr/1/the/larch' }}, language='nl')

Parameters • query (dict) – The query parameters that will be passed on to each find() method of the selected. providers. • providers (dict) – Optional. If present, it should be a dictionary. This dictionary can contain any of the keyword arguments available to the get_providers() method. The query will then only be passed to the providers conﬁrming to these arguments. • language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. Returns a list of dict. Each dict has two keys: id and concepts.

get_all(**kwargs) Get all concepts from all providers.

# get all concepts in all providers. registry.get_all()

# get all concepts in all providers. # If possible, display the results with a Dutch label. registry.get_all(language='nl')

Parameters language (string) – Optional. If present, it should be a language-tag. This language-tag is passed on to the underlying providers and used when selecting the label to display for each concept. Returns a list of dict. Each dict has two keys: id and concepts.

get_by_uri(uri) Get a concept or collection by its uri. Returns a single concept or collection if one exists with this uri. Returns False otherwise. Parameters uri (string) – The uri to ﬁnd a concept or collection for. Raises ValueError – The uri is invalid. Return type skosprovider.skos.Concept or skosprovider.skos. Collection

4.3. Registry module 25 Skosprovider Documentation, Release 0.7.0

get_metadata() Get some metadata on the registry it represents. Return type Dict. get_provider(id) Get a provider by id or uri. Parameters id (str) – The identiﬁer for the provider. This can either be the id with which it was registered or the uri of the conceptscheme that the provider services. Returns A skosprovider.providers.VocabularyProvider or False if the id or uri is unknown. get_providers(**kwargs) Get all providers registered. If keyword ids is present, get only the providers with these ids. If keys subject is present, get only the providers that have this subject.

# Get all providers with subject 'biology' registry.get_providers(subject='biology')

# Get all providers with id 1 or 2 registry.get_providers(ids=[1,2])

# Get all providers with id 1 or 2 and subject 'biology' registry.get_providers(ids=[1,2], subject='biology']

Parameters • ids (list) – Only return providers with one of the Ids or URIs. • subject (str) – Only return providers with this subject. Returns A list of providers

instance_scope = 'single' Indicates how the registry is being used. Options: • single: The registry is part of a script or a single process. It can be assumed to be operational for the entire duration of the process and there are no threads involved. • threaded_global: The registry is part of a program that uses threads, such as a typical web application. It’s attached to the global process and duplicated to threads, making it not thread safe. Proceed carefully with certain providers. Should generally only be used with appli- cations that only use read-only providers that load all data in memory at startup and use no database connections or other kinds of sessions. • threaded_thread: The registry is part of a program that uses threads, such as a typical web application. It’s attached to a thread, such as a web request. The registry is instantiated for this thread/request and dies with this thread/request. This is needed for providers such as the SQLAlchemyProvider. Providers that use database connections or other session handling code generally require this. metadata = {} Dictionary containing metadata about this registry. providers = {} Dictionary containing all providers, keyed by id.

26 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

register_provider(provider) Register a skosprovider.providers.VocabularyProvider. Parameters provider (skosprovider.providers.VocabularyProvider)– The provider to register. Raises RegistryException – A provider with this id or uri has already been registered. remove_provider(id) Remove the provider with the given id or URI. Parameters id (str) – The identiﬁer for the provider. Returns A skosprovider.providers.VocabularyProvider or False if the id is unknown. exception skosprovider.registry.RegistryException

4.4 Uri module

This module provides utilities for working with URIs. New in version 0.3.0. class skosprovider.uri.DefaultConceptSchemeUrnGenerator Generate a URN for a conceptscheme specific to skosprovider. Used for generating default URI for providers that do not have an explicit conceptscheme. generate(**kwargs) Generate a URI based on parameters passed. Parameters id – The id of the conceptscheme. Return type string class skosprovider.uri.DefaultUrnGenerator(vocabulary_id) Generate a URN specific to skosprovider. Used for providers that do not implement a specific UriGenerator. Parameters vocabulary_id – An identifier for the vocabulary we’re generating URIs for. generate(**kwargs) Generate a URI based on parameters passed. Parameters id – The id of the concept or collection. Return type string class skosprovider.uri.TypedUrnGenerator(vocabulary_id) Generate a URN specific to skosprovider that contains a type. Parameters vocabulary_id – An identifier for the vocabulary we’re generating URIs for. generate(**kwargs) Generate a URI based on parameters passed. Parameters • id – The id of the concept or collection. • type – What we’re generating a URI for: concept or collection. Return type string

4.4. Uri module 27 Skosprovider Documentation, Release 0.7.0

class skosprovider.uri.UriGenerator An abstract class for generating URIs. generate(**kwargs) Generate a URI based on parameters passed. class skosprovider.uri.UriPatternGenerator(pattern) Generate a URI based on a simple pattern. generate(**kwargs) Generate a URI based on parameters passed. Parameters id – The id of the concept or collection. Return type string skosprovider.uri.is_uri(uri) Check if a string is a valid URI according to rfc3987 Parameters uri (string)– Return type boolean

4.5 JsonLd module

This module contains functions dealing with jsonld reading and writing. New in version 0.7.0. skosprovider.jsonld.jsonld_c_dumper(provider, id, context=None, relations_profile=’partial’, language=’en’) Dump a concept or collection to a JSON-LD serialisable dictionary. Parameters • provider (skosprovider.providers.VocabularyProvider) – The provider that contains the concept or collection. • or int id (str) – Identifier of the concept to dump. • or dict context (str) – Context as a dict or link to context file. • relations_profile (str) – Either partial or uri to render links to other resources with some information or just a URI. • language (string) – Language to render a single label in. Return type A dict skosprovider.jsonld.jsonld_conceptscheme_dumper(provider, context=None, re- lations_profile=’partial’, language=’en’) Dump a conceptscheme to a JSON-LD serialisable dictionary. Parameters • provider (skosprovider.providers.VocabularyProvider) – The provider that contains the conceptscheme. • or dict context (str) – Context as a dict or link to context file. • relations_profile (str) – Either partial or uri to render links to other resources with some information or just a URI. • language (string) – Language to render a single label in.

28 Chapter 4. API Documentation Skosprovider Documentation, Release 0.7.0

Return type A dict skosprovider.jsonld.jsonld_dumper(provider, context=None, language=None) Dump a provider to a JSON-LD serialisable dictionary. Parameters • provider (skosprovider.providers.VocabularyProvider) – The provider that wil be turned into a JSON-LD dict. • or dict context (str) – Context as a dict or link to context ﬁle. • language (string) – Language to render a single label in. Return type A dict

4.6 Exceptions module

This module provides custom exceptions for skos providers. New in version 0.5.0. exception skosprovider.exceptions.ProviderUnavailableException(message) This exception can be raised by a provider if it’s unable to provide the thesaurus. This can occur when an underlying resource is unavailable (database connection, webservice, . . . ). The message should contain some more information about the problem.

4.7 Utils module

This module contains utility functions for dealing with skos providers. skosprovider.utils.add_lang_to_html(htmltext, lang) Take a piece of HTML and add an xml:lang attribute to it. New in version 0.7.0. skosprovider.utils.dict_dumper(provider) Dump a provider to a format that can be passed to a skosprovider.providers. DictionaryProvider. Parameters provider (skosprovider.providers.VocabularyProvider) – The provider that wil be turned into a dict. Return type A list of dicts. New in version 0.2.0. skosprovider.utils.extract_language(lang) Turn a language in our domain model into a IANA tag. New in version 0.7.0. skosprovider.utils.text_(s, encoding=’latin-1’, errors=’strict’) If s is an instance of binary_type, return s.decode(encoding, errors), otherwise return s

4.6. Exceptions module 29 Skosprovider Documentation, Release 0.7.0

30 Chapter 4. API Documentation CHAPTER 5

History

5.1 0.7.0 (2020-01-19)

• Add dumpers to transform a provider, a conceptscheme, a concept or collection in to a dictionary compatible with a JSON-LD context that has been added. This makes it possible to transform a provider into Linked Data. • Make handling of the hierarchy involving collections as thesaurus arrays more logical. A collection now has an attribute skosprovider.skos.Collection.infer_concept_relations that indicates if the members of a collection should be seen as narrower concepts of a superordinate concept. This is generally important when expanding a concept to all it’s narrower concepts for searching. (#57) • Add a new query option for querying matches with concepts from external conceptschemes to skosprovider.providers.VocabularyProvider.find(). (#58) • A registry can now carry metadata just like a provider. • A registry now has an attribute skosprovider.registry.Registry.instance_scope that indicates how the registry is managed in the application process. All providers need to indicate what kinds of instance_scope they’re compatible with. Especially important for SQLAlchemyProvider run in a web application. (#63, #66) • Fix a bug that made it impossible for a SimpleCsvProvider to read sources. (#36) • Drop support for Python 3.3, 3.4 and 3.5. Adds support for 3.8. This is the last version that will support Python 2. Version 0.8.0 will drop support for Python 2.7.

5.2 0.6.1 (2017-07-16)

• A provider can now receive a dataset keyword containing a dict. If present, this dict needs to have a uri attribute. • Update some requirements.

31 Skosprovider Documentation, Release 0.7.0

5.3 0.6.0 (2016-08-09)

• Allow marking a note as containing HTML. (#17) • Add languages attribute to skosprovider.skos.ConceptScheme to make it possible to track what languages are being used in a thesaurus. (#19) • Add a sources attribute to ConceptScheme, Collection and Concept. Every source is an object that currently only has one attribute, a citation. This looks like a good universal common denominator. Just as with notes, a citation may contain HTML. (#20, #24) • Add sorting to skosprovider.providers.VocabularyProvider.get_all(), skosprovider.providers.VocabularyProvider.find(), skosprovider. providers.VocabularyProvider.get_top_concepts(), skosprovider. providers.VocabularyProvider.get_top_display(), skosprovider.providers. VocabularyProvider.get_children_display(). Sorting can be done on id, label or sortlabel. The last option makes it possible to introduce arbitrary sorting for concepts, eg. to sort periods chronologically. The sort order can be speciﬁed with the sort_order parameter. (#21) • Remove skosprovider.providers.VocabularyProvider.expand_concept() that was deprecated since 0.2.0. • Fixed a bug with skosprovider.skos.dict_to_label() and skosprovider.skos. dict_to_note() that would assign None instead of und as the language for labels and notes that have no language. • Improved checking for valid URIs with e.g. skosprovider.skos.ConceptScheme. This was causing weird issues with registering a provider to the skosprovider.registry.Registry. (#27)

5.4 0.5.3 (2015-06-24)

• When a skosprovider.providers.DictionaryProvider reads a dictionary containing a URI and that URI’s None, generate a URI. (#12) • Upgrade to the newest version of language-tags, this fixes a showstopping bug on Windows machines. (#16) • Added an examples folder with a script that demonstrates the API using a DictionaryProvider. • Added a wheel config file.

5.5 0.5.2 (2015-03-02)

Release 0.5.1 was a brown-paper-bag release due to some mucking about with pypi.

5.6 0.5.1 (2015-03-02)

• Make it possible to pass a language tag to skosprovider.registry.Registry.find() that will be passed on to all relevant registered providers. This determines in what language the labels of the returned concepts will displayed. (#10) [dieuska] • Make it possible to pass a language tag to skosprovider.registry.Registry.get_all() that will be passed to all registered providers. This determines in what language the labels of the returned concepts will displayed.

32 Chapter 5. History Skosprovider Documentation, Release 0.7.0

• Fixed some errors with the skosprovider.utils.dict_dumper(). It didn’t dump the matches or the subordinate_arrays of concepts. • Wrote some new documentation on what a provider is and how to use it. Some other documentation work as well such as documenting the language parameter in the API better.

5.7 0.5.0 (2014-12-18)

• Changed the default language from None to the ofﬁcial IANA language code und (undetermined). This is a minor BC break for users dealing with labels that have not been assigned a language. • Added a ProviderUnavailableException to let a provider signal that an underlying backend is not available.

5.8 0.4.2 (2014-10-16)

• Fix a problem with SKOS matches. • BC compatibilty break with 0.4.0 and 0.4.1: renamed the matchtypes broader to broad and narrower to narrow to be more inline with the SKOS standard.

5.9 0.4.1 (2014-10-15)

• Made the DictionaryProvider return Collection objects with Note objects attached if available. • Fix a problem in ﬁnd operations when a concept or collection had no label attached to it. (#6) [dieuska]

5.10 0.4.0 (2014-10-02)

• Dropped support for Python 2.6 and 3.2. • Added ability to add Note to Collection and ConceptScheme. • Added a ConceptScheme to every provider. This ConceptScheme can then be passed on to Concepts and Collections. This allows Concepts and Collections that have left the context of their provider, to still refer back to the ConceptScheme and thus the skosprovider.providers.VocabularyProvider where they originated. • When querying the Registry for providers, a URI of an accompanying ConceptScheme can now also be used. • Added subordinate_arrays attribute to Concept and superordinates to Collection. These attributes are based on the SKOS-THES speciﬁcation. They allow linking Concepts and Collections for the purpose of displaying a hierarchy. • Expanded support for languages with language-tags library. When generating a label, the language speciﬁcation handles inexact language matches better. Eg. when asking for a label with language nl for a concept that only has nl-BE labels, these will now be returned while in the past this was not guaranteed. • Added subject to the metadata of a providers. This is a list of subjects or tags that help describe or type the provider. The Registry can now be searched for providers with a certain subject through the get_providers() method.

5.7. 0.5.0 (2014-12-18) 33 Skosprovider Documentation, Release 0.7.0

5.11 0.3.0 (2014-05-14)

• Added support for URI.A skosprovider.skos.Concept, skosprovider.skos.Collection or skosprovider.skos.ConceptScheme can now have a URI. • Query a skosprovider.providers.VocabularyProvider or the skosprovider.registry. Registry by URI. • Added skosprovider.uri module to handle generating of URIS. • Added a get_top_concepts() method to skosprovider.providers.VocabularyProvider. This method returns the Top Concepts in a ConceptScheme (the concepts that don’t have a broader concept). • Added the get_top_display() and get_children_display() methods to handle generating a display hierarchy for a certain provider. • A method that used to return a list of dicts containing an id and a label, now also returns a uri and a type (concept/collection) for each dict. (#2) • Provide list of valid noteTypes and labelTypes as attributes of Note and Label so they can be used externally. (#4) • Reworking tests. Now using pytest in stead of nose. • Adding code coverage based on Coveralls.

5.12 0.2.1 (2013-12-06)

• Make the skosprovider.providers.MemoryProvider forward compatible by constructing skosprovider.skos.Concept and skosprovider.skos.Collection objects with keywords. • Soms minor ﬁxes in documentation. • Added an extra unit test.

5.13 0.2.0 (2013-05-16)

• Major rewrite and refactoring. Tried to keep BC in place as much as possible, but did change some stuff. • Added a read only SKOS domain model in the skosprovider.skos module. • Providers no longer return dicts as concepts, but instances of skosprovider.skos.Concept. • Added support for skos collections with a skosprovider.skos.Collection object. • Expanded concept query syntax. Now allows for querying on type (concept or collection) and on collection membership. See skosprovider.providers.VocabularyProvider.find(). • Added skosprovider.utils.dict_dumper().

5.14 0.1.3 (2013-03-22)

• Find empty label now returns no results • Find without a label now calls get_all

34 Chapter 5. History Skosprovider Documentation, Release 0.7.0

5.15 0.1.2 (2013-02-07)

• Providers can be removed from the registry • Added the ability to get a single provider from the registry • No longer possible to register the same provider twice

5.16 0.1.1 (2012-12-11)

• Some pep8 ﬁxes • Add support for tox • Now tested for python 3.2 • Added skos:notes as an example to the unit tests.

5.17 0.1.0

• Initial version

5.15. 0.1.2 (2013-02-07) 35 Skosprovider Documentation, Release 0.7.0

36 Chapter 5. History CHAPTER 6

Glossary

language-tag A valid tag from the IANA language subtag registry. Eg. nl-BE, en-Latn-GB, i-klingon, lb, zh-latn- pinyin-x-notone, . . . Skosprovider uses the language-tags library to handle the complexities of these tags. RDF Resource Description Framework. A very flexible model for data definition organised around triples. These triples forms a directed, labeled graph, where the edges represent the named link between two resources, repre- sented by the graph nodes. SKOS Simple Knowledge Organization System. A general specification for Knowledge Organisation Systems (thesauri, word lists, authority files, . . . ) that is commonly serialised as RDF. SKOS-THES The ISO 25964 SKOS extension defines mappings between the ISO 25964 standard and the SKOS specification. URI A Uniform Resource Identifier. URN A URN is a specific form of a URI.

37 Skosprovider Documentation, Release 0.7.0

38 Chapter 6. Glossary CHAPTER 7

Indices and tables

• genindex • modindex • search

39 Skosprovider Documentation, Release 0.7.0

40 Chapter 7. Indices and tables Python Module Index

s skosprovider.exceptions, 29 skosprovider.jsonld, 28 skosprovider.providers, 14 skosprovider.registry, 24 skosprovider.skos,9 skosprovider.uri, 27 skosprovider.utils, 29

41 Skosprovider Documentation, Release 0.7.0

42 Python Module Index Index

Symbols DefaultUrnGenerator (class in skosprovider.uri), __init__() (skosprovider.providers.DictionaryProvider 27 method), 23 dict_dumper() (in module skosprovider.utils), 29 __init__() (skosprovider.providers.MemoryProvider dict_to_label() (in module skosprovider.skos), 12 method), 19 dict_to_note() (in module skosprovider.skos), 13 __init__() (skosprovider.providers.SimpleCsvProvider dict_to_source() (in module skosprovider.skos), 13 method), 24 DictionaryProvider (class in __init__() (skosprovider.providers.VocabularyProvider skosprovider.providers), 23 method), 14 E A expand() (skosprovider.providers.MemoryProvider add_lang_to_html() (in module skosprovider.utils), method), 19 29 expand() (skosprovider.providers.VocabularyProvider allowed_instance_scopes method), 15 (skosprovider.providers.VocabularyProvider extract_language() (in module skosprovider.utils), attribute), 14 29 B F broader (skosprovider.skos.Concept attribute), 10 filter_labels_by_language() (in module skosprovider.skos), 13 C find() (skosprovider.providers.MemoryProvider case_insensitive (skosprovider.providers.MemoryProvider method), 19 attribute), 19 find() (skosprovider.providers.VocabularyProvider citation (skosprovider.skos.Source attribute), 12 method), 15 Collection (class in skosprovider.skos),9 find() (skosprovider.registry.Registry method), 24 Concept (class in skosprovider.skos), 10 find_best_label_for_type() (in module concept_scheme (skosprovider.providers.VocabularyProvider skosprovider.skos), 13 attribute), 14 concept_scheme (skosprovider.skos.Collection at- G tribute),9 generate() (skosprovider.uri.DefaultConceptSchemeUrnGenerator concept_scheme (skosprovider.skos.Concept at- method), 27 tribute), 10 generate() (skosprovider.uri.DefaultUrnGenerator concept_scheme_uri_map method), 27 (skosprovider.registry.Registry attribute), generate() (skosprovider.uri.TypedUrnGenerator 24 method), 27 ConceptScheme (class in skosprovider.skos), 11 generate() (skosprovider.uri.UriGenerator method), D 28 generate() (skosprovider.uri.UriPatternGenerator DefaultConceptSchemeUrnGenerator (class in method), 28 skosprovider.uri), 27

43 Skosprovider Documentation, Release 0.7.0 get_all() (skosprovider.providers.MemoryProvider is_valid_markup() (skosprovider.skos.Note static method), 21 method), 12 get_all() (skosprovider.providers.VocabularyProvider is_valid_markup() (skosprovider.skos.Source method), 16 static method), 12 get_all() (skosprovider.registry.Registry method), 25 is_valid_type() (skosprovider.skos.Label static get_by_id() (skosprovider.providers.MemoryProvider method), 11 method), 21 is_valid_type() (skosprovider.skos.Note static get_by_id() (skosprovider.providers.VocabularyProvider method), 12 method), 17 get_by_uri() (skosprovider.providers.MemoryProvider J method), 22 jsonld_c_dumper() (in module get_by_uri() (skosprovider.providers.VocabularyProvider skosprovider.jsonld), 28 method), 17 jsonld_conceptscheme_dumper() (in module get_by_uri() (skosprovider.registry.Registry skosprovider.jsonld), 28 method), 25 jsonld_dumper() (in module skosprovider.jsonld), get_children_display() 29 (skosprovider.providers.MemoryProvider method), 22 L get_children_display() Label (class in skosprovider.skos), 11 (skosprovider.providers.VocabularyProvider label (skosprovider.skos.Label attribute), 12 method), 17 label() (in module skosprovider.skos), 13 get_metadata() (skosprovider.providers.VocabularyProviderlabel() (skosprovider.skos.Collection method),9 method), 18 label() (skosprovider.skos.Concept method), 10 get_metadata() (skosprovider.registry.Registry label() (skosprovider.skos.ConceptScheme method), method), 25 11 get_provider() (skosprovider.registry.Registry labels (skosprovider.skos.Collection attribute),9 method), 26 labels (skosprovider.skos.Concept attribute), 10 get_providers() (skosprovider.registry.Registry labels (skosprovider.skos.ConceptScheme attribute), method), 26 11 get_top_concepts() language (skosprovider.skos.Label attribute), 12 (skosprovider.providers.MemoryProvider language (skosprovider.skos.Note attribute), 12 method), 22 language-tag, 37 get_top_concepts() languages (skosprovider.skos.ConceptScheme at- (skosprovider.providers.VocabularyProvider tribute), 11 method), 18 get_top_display() M (skosprovider.providers.MemoryProvider markup (skosprovider.skos.Note attribute), 12 method), 23 markup (skosprovider.skos.Source attribute), 12 get_top_display() matches (skosprovider.skos.Concept attribute), 10 (skosprovider.providers.VocabularyProvider matchtypes (skosprovider.skos.Concept attribute), 10 method), 18 member_of (skosprovider.skos.Collection attribute),9 get_vocabulary_id() member_of (skosprovider.skos.Concept attribute), 10 (skosprovider.providers.VocabularyProvider members (skosprovider.skos.Collection attribute), 10 method), 19 MemoryProvider (class in skosprovider.providers), 19 I metadata (skosprovider.registry.Registry attribute), 26 id (skosprovider.skos.Collection attribute),9 N id skosprovider.skos.Concept attribute ( ), 10 narrower (skosprovider.skos.Concept attribute), 10 infer_concept_relations Note (class in skosprovider.skos), 12 skosprovider.skos.Collection attribute ( ), note (skosprovider.skos.Note attribute), 12 9 notes (skosprovider.skos.Collection attribute), 10 instance_scope skosprovider.registry.Registry at- ( notes (skosprovider.skos.Concept attribute), 11 tribute ), 26 notes (skosprovider.skos.ConceptScheme attribute), 11 is_uri() (in module skosprovider.uri), 28

44 Index Skosprovider Documentation, Release 0.7.0

P UriPatternGenerator (class in skosprovider.uri), providers (skosprovider.registry.Registry attribute), 28 26 URN, 37 ProviderUnavailableException, 29 V R valid_markup (in module skosprovider.skos), 14 RDF, 37 valid_types (skosprovider.skos.Label attribute), 12 register_provider() valid_types (skosprovider.skos.Note attribute), 12 (skosprovider.registry.Registry method), 26 VocabularyProvider (class in Registry (class in skosprovider.registry), 24 skosprovider.providers), 14 RegistryException, 27 related (skosprovider.skos.Concept attribute), 11 remove_provider() (skosprovider.registry.Registry method), 27 S SimpleCsvProvider (class in skosprovider.providers), 24 SKOS, 37 SKOS-THES, 37 skosprovider.exceptions (module), 29 skosprovider.jsonld (module), 28 skosprovider.providers (module), 14 skosprovider.registry (module), 24 skosprovider.skos (module),9 skosprovider.uri (module), 27 skosprovider.utils (module), 29 Source (class in skosprovider.skos), 12 sources (skosprovider.skos.Collection attribute), 10 sources (skosprovider.skos.Concept attribute), 11 sources (skosprovider.skos.ConceptScheme attribute), 11 subordinate_arrays (skosprovider.skos.Concept attribute), 11 superordinates (skosprovider.skos.Collection attribute), 10 T text_() (in module skosprovider.utils), 29 type (skosprovider.skos.Collection attribute), 10 type (skosprovider.skos.Concept attribute), 11 type (skosprovider.skos.Label attribute), 12 type (skosprovider.skos.Note attribute), 12 TypedUrnGenerator (class in skosprovider.uri), 27 U URI, 37 uri (skosprovider.skos.Collection attribute), 10 uri (skosprovider.skos.Concept attribute), 11 uri (skosprovider.skos.ConceptScheme attribute), 11 uri_generator (skosprovider.providers.VocabularyProvider attribute), 19 UriGenerator (class in skosprovider.uri), 27

Index 45