ODJ
1/ Technique
2/ Planification
3/ Point contractuel ● Geocat.ch 3.10 / Point sur le merge ○ Nouveautés https://geocat-dev.dev.bgdi.ch/geonetwork/ ODJ ○ Modification du schema che ● Geocat.ch 4 ? Etat des lieux sur GN4, démonstration & dashboard 1/ Technique ● Disponibilité de Geocat ○ How-to cluster ? ElasticSearch & GN ○ Amélioration de la robustesse de l’application i. eg. pb RDF Search, coupe-circuits, découper l’application 2/ Planification ii. Allègement des fichiers de log iii. Traitement des fuites mémoire
3/ Point contractuel ● Roadmap geocat.ch: ○ Qualité des métadonnées (urls, mots-clés, batch, …) ○ Opendata.swiss (conditions d’utilisation, moissonage, dépublication, Handbuch, …) ○ GM03 Anpassungen (réflexions sur le modèle actuel – FIG Metadaten) ○ Connexion avec le portail européen INSPIRE ○ Réflexions sur le versionnement / l’historisation des métadonnées ● Communauté des utilisateurs geoNetwork (existant, contacts, FOSS4G, …) ● Alertes Pingdom (pas envoyées le 14.01.2020) Updating geocat.ch to GeoNetwork 3.10.0
Some of the changes Who is using 3.10.x in production? Who is using 3.10.x in testing? https://geocat-dev.dev.bgdi.ch/geonetwork/
● Liste des filtres actifs ● Recherches utilisateurs ● Admin ○ Analyse des liens ○ Portails ● Questions ○ Landing page https://jira.swisstopo.ch/browse/MGEO_SB-525 ■ JSON-LD & Google Dataset Search pour une meilleur indexation ■ Eg. https://geocat-dev.dev.bgdi.ch/geonetwork/srv/api/records/d2ab7e4e -d135-4442-b0af-2f8892d87843 ■ https://datasetsearch.research.google.com/ Questions? Actions? geocat.ch 4? GeoNetwork 2020 Challenges INSPIRE TG2 compliant
GDPR
User search experience (GN4)
Large catalogues (eg. clustering, tasking, cursors, microservices)
KISS & clean
Angularjs 1.x End-Of-Life
Java 11
Community
Multilingual documentation GeoNetwork 4 ? Why?
Speed
Search experience
… Clustering capabilities
https://github.com/geonetwork/core-geonetwork/pull/2830
GeoNetwork 4 Architecture & Setup Not a single easy app to install anymore
DB
Webapp
Elasticsearch
(Kibana)
GeoNetwork 4 ? Speed 3.4 min indexing 1K records in metawal Search is fast
Scenario
● Load default search ● 3 facets ○ Datasets ○ Then BDinfraSIG ○ Then Grid Search is fast (and response light)
● Lucene
● ES Search is fast … but rendering is still “slow”
● Related records ● Angular one time binding ● Watchers GeoNetwork 4 What about search? More like this Suggest a field (eg. title) but search on all content
In current GeoNetwork service, search value in a field and return values found. If nothing found, try to search for title Suggest a field (eg. title and keyword) but search on all content Phrase suggestion (ngrams in action) Suggestions based on score
GN suggestion issue / can display information not taking into account privileges. This is fixed in ES. Suggestions based on score Scoring
API Suggestions / Typo proof / TODO Search / Plurals ie. Analyzer per language & racinisation
Search / Plurals ie. Analyzer per language & racinisation
Search / Synonyms Aggregations with OR Aggregations exclude Nested aggs
Based on thesaurus hierarchy
Or by nesting fields Aggregations with filter Aggregations with paging
In GeoNetwork only a fixed number of values is returned Histogram aggregations … … heatmaps … ... All is configured in the admin
And is based on Elasticsearch API
Eg.
● Facet list ● Autocomplete mode Search API
Search service is based on Elasticsearch API
+ privileges handling
Main UI is using the main search API.
Then CSW search is built on top of ES search API. Search response content
The client request the fields they need.
Eg. only the title for making a list of record Application “run” if index is down
Joins https://github.com/geonetwork/core-geonetwork/pull/2830
Inner_hits, nested object Convinced ? Roadmap & Sponsors
● https://docs.google.com/spreadsheets/d/1HPEfZETFVD2E6yFKmmMBt8KPmxlR-IpX2jDmXVhyhjo/edit#gid=1521480254 ● 2016 ○ Solr POC / camptocamp, titellus ● 2018 ○ Bolsena sprint / camptocamp, geocat.bv, titellus ● 2019 ○ Bolsena sprint / camptocamp, geocat.bv, titellus ○ July Sprint / EEA / 20 days ○ December Sprint / camptocamp, titellus ● 2020 ○ Metawal 4 use ES 4 - starting February 2020 ○ .... GeoNetwork 4 for What is missing? Migrate search UI Missing / Subtemplate support Missing / Multilingual support Missing / ISO19139.che
Indexing Missing / Packaging, doc & security setup
Docker compose with
● GeoNetwork ● ES ● Kibana
Documentation on how to install, configure and secure ES. Missing but “invisible” for geocat.ch
Map viewer search (eg. layers, services, maps)
ATOM feed search
RDF search
RSS search
OAI-PMH
... Iso-fonctionnel High Availability? Clustering issues
Geonetwork 1 Geonetwork 2
Indexing Index Indexing Lucene Lucene index lock
Thesaurus Thesaurus Data dir Thesaurus synch Search Search
Editeur DB Editeur Solve the index lock
Geonetwork 1 Geonetwork 2 ES Indexing Indexing
Thesaurus Thesaurus Data dir Thesaurus synch Search Search
Editeur DB Editeur ES 1 ES 2 Geonetwork 1 Geonetwork 2 ES Indexing 3 Indexing
Thesaurus Thesaurus Data dir Thesaurus synch Search Search
Editeur DB Editeur Dedicated instances
Gn Editor Gn CSW server ES Indexing Indexing
Thesaurus Data dir Search
Editeur
DB GeoNetwork “Microservices”
ES Search app 1
ES 2 ES Harvesters & 3 background tasks
Data dir Editing app
Thesaurus app
DB GeoNetwork “Microservices” on demand
ES Search app 1 Search app Search app ES
2 ES Harvesters & 3 background tasks
Data dir Editing app
Thesaurus app
DB Proposal for a roadmap Roadmap - proposal
1. Geocat.ch 4 based on ES - ISO functional 2. Kibana setup 3. Clustering ES with one webapp 4. Split webapp in 2 : a. Search service b. The rest 5. Cluster Search services for better availability Roadmap geocat.ch:
· Qualité des métadonnées (urls, mots-clés, batch, …)
· Opendata.swiss (conditions d’utilisation, moissonage, dépublication, Handbuch, …)
· GM03 Anpassungen (réflexions sur le modèle actuel – FIG Metadaten)
· Connexion avec le portail européen INSPIRE
· Réflexions sur le versionnement / l’historisation des métadonnées
Communauté des utilisateurs GeoNetwork (existant, contacts, FOSS4G, Conf INSPIRE 2020 …)
Alertes Pingdom (pas envoyées le 14.01.2020)