HMA for Science - Evolution
Final Presentation
Y. Coene, Spacebel s.a. Tri Tran Minh, Spacebel s.a. Minh Nguyen Quang, Spacebel s.a.
16 October 2014 ESRIN, Frascati
Page 1 Outline
• Introduction • Science Services Integration • OpenSearch • Other • SSE Portal • Conclusion and Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 2 Introduction - What is HMA
A collaborative project started in Europe and Canada by the Ground Segment Coordination Body (GSCB) in 2005 with the objective to: •Guarantee a seamless and harmonised access to heterogeneous EO datasets from multiple mission ground segments, including national missions and ESA missions. •Standardise the ground segment interfaces of the satellite missions for easier access to EO data. •Provide interoperability for coordinated data access enabling the interactions with services or Value Adders and EO Contributing Missions.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 3 Introduction
• HMA for Science – Contributions to standardisation • Identity Management specification • Metadata specification • Product discovery specification • Collection discovery specification • Ordering specifications • Processing specifications • Feasibility Analysis specification • FedEO • FedEO is a system providing a brokered discovery (and access) capability to European (and Canadian) EO missions data based on HMA interfaces
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 4 Architecture
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 5 Architecture
• Main results: – Service endpoints for external clients • Option 1: – OpenSearch Dataset Series, Dataset Discovery and download: OpenSearch Gateway • Option 2: – OGC 13-084 Dataset Series Discovery – OGC 06-131 Dataset Discovery – OGC 06-141 Ordering (incl. OGC 07-118) – Portal clients
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 6 Outline
• Introduction • Science Services Integration • OpenSearch • Other • SSE Portal • Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 7 Architecture – OpenSearch Interface
• Dataset series, dataset (and services) discovery • Applied for heterogeneous metadata: – EOP O&M (OGC 10-157r3 and r4) metadata – DC or ISO metadata – RDF metadata – ISO19139 metadata (as per OGC 11-035r1 and OGC 08- 167r2) • Interface : OpenSearch – OGC 10-032r8 Geo and Time Extension – OGC 13-026r4 Extension for Earth Observation – Aligned with OASIS searchRetrieve 1.0 conventions – OpenSearch SRU Extension – RESTful interface being prepared.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 8
OASIS Search Web Services (SWS): SearchRetrieve
searchRetrieve
<
<
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 9 OpenSearch Specification Tree
searchRetrieve
<
<
SRU Extension Parameter Extensiion <
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 10 Architecture - OSDD
• Recursive or multi-step search: – Step 1: return dataset series metadata and OSDD to be used for dataset search inside the dataset series – Step 2: return dataset (i.e. product/granule) metadata • Approach: – RFC 6573, The Item and Collection Link Relations. –
HTTP GET http://fedeo.esa.int/opensearch/description.xml?parentIdentifier={eo:paren tIdentifier?} OpenSearch Description HTTP Response Document ID ID
Spatial Extent Spatial Extent HTTP GET Request Temporal Temporal Extent Extent parentIdentifier = … Metadata Link Metadata Link
name = frascati [Search Link] [Search Link] startDate = 2000-10- 31T00:00:00Z [Data Link] [Data Link]
HTTP GET http://fedeo.esa.int/opensearch/request/? parentIdentifier=EOP%3AEGEOS%3AEGEOS%23IKONOS&startDate=2000-10- 31T00:00:00Z&cloudCover=10]&endDate=2013-10- 31T00:00:00Z&name=frascati&radius=1000
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 12 Use Case 2 : Two-step search
HTTP GET http://fedeo.esa.int/opensearch/descript ion.xml
Parent OSDD Child OSDD
HTTP GET Request HTTP GET Request
HTTP Response HTTP Response
Parent ATOM Feed Child ATOM Feed
Atom Entry 1 Atom Entry 2 Atom Entry 1 Atom Entry 2
Metadata Link Metadata Link Metadata Link Metadata Link Search Link Search Link [Data Link] [Data Link] Spatial extent Spatial extent Spatial extent Spatial extent Temporal extent Temporal extent Temporal extent Temporal extent
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 13 FedEO Two-step Search
Partner Mission GS Search Client FedEO CollectionPartnerMission Mission CatalogsGS GS
Get OSDD Partner Mission GS Create OSDD based on backend GranulePartnerMission Mission Catalogs GS GS capabilities. OSDD for collection search
Perform collection search Return matching collections and number of hits Atom response with collections Collect responses and # of results Search Collection 1: Step and link to OSDD for each from backend collection catalog(s). Compute first/next/previous/last links.
Extract location of OSDD from Atom response Get OSDD @atom:link Create OSDD based on backend capabilities. Note that atom:link may Return contain non-FEDEO address. matching OSDD for granule search granules and number Perform granule search of hits
Step 2: Granule Search Search Granule 2: Step Atom response with granules and link to thumbnail, browse, product
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 14 Two-Step Search
• Dataset Series (Collections) – Described with Collection Metadata – Searchable Collections
Collections Collections that with metadata are searchable
Collection Collection supporting Searchable Collection with 2-step Search not yet described with collection metadata but collection metadata not searchable (e.g. Copernicus, ASF, …) (e.g. GSCDA)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 15 Architecture - Dataset Series
• Dataset Series hierarchy: • Top-level: parallel search in 4 (TBC) catalogs • Search inside a selected dataset series (eo:parentIdentifier).
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 16 EOP:ASF ATS_TOA_1P EOP:VITO:VGT_S10 MSG_SEVIRI Dataset Series metadata
EOP:ESA:FEDEO:COLLECTIONS EOP:ESA:GPOD-EO
20130901:391673
20130821:391238
20130811:390830 200411010615 20130801:390463 200411020430 Dataset metadata
EOP:VITO:VGT_S10 MSG_SEVIRI
Metadata Legend: ISO 19139 OGC 10-157 RDF
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 17 Architecture – Dataset Series
• Search target: {eo:parentIdentifier} – Can be used in dataset series search and in dataset search. – Default value is EOP:ESA:FEDEO – Can be logical name for one or more endpoints – No new naming conventions: – CDS EO-DAIL collection identifiers kept. “urn:ogc:” optional – VA4, GPOD, M2CS and CWIC collection identifiers kept. – Allows for same collection identifier in two catalogs (use prefix) – parentIdentifier = EOP:ESA:GPOD-EO:ASA_IM__0P
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 18 Architecture – Dataset Series
EOP:ESA:FEDEO
EOP:ESA:FEDEO:COLLECTIONS EOP:ESA:GPOD-EO EOP:ESA:EO-VIRTUAL-ARCHIVE4
EOP:VITO:VGT_S10 EOP:ASF … EOP:SPOT:ALL MSG_SEVIRI … ATS_TOA_1P ASA_IM__0P
• Queryables in OSDD differ per dataset series: – http://fedeo.esa.int/opensearch/description.xml?parentIdenti fier={eo:parentIdentifier} – http://fedeo.esa.int/opensearch/description.xml?parentIdenti fier=EOP:ESA:FEDEO:COLLECTIONS – http://fedeo.esa.int/opensearch/description.xml?parentIdenti fier=EOP:DLR:EOWEB:Geohazard.Supersite.TerraSAR-X_SSC HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 19 Parameter Extension (*)
• Additional constraints on input parameters (in OSDD): – For example (OGC 06-131 Connector): http://fedeo.esa.int/opensearch/description.xml?parentIdenti fier=EOP:DLR:EOWEB:Geohazard.Supersite.TerraSAR-X_SSC
(*) Draft with known issues (See conclusion) and competing with WADL (http://www.w3.org/Submission/wadl/) and RAML (RESTful API Modeling Language) etc. and partly overlapping with searchRetrieve Explain (OASIS) capabilities.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 20 Architecture - Queryables
• Queryables differ per dataset series: – «time» and «geo» namespace OGC 10-032r8 – «eo» namespace OGC 13-026r4 • Mapping to dataset series metadata ISO19139 (step 1) – {eo:parentIdentifier}, {eo:organisationName}, {eo:productType}, {eo:instrument}, {eo:platform} • Mapping to dataset metadata OGC 10-157r4 (step 2) – 35 queryables e.g. {eo:instrument}, {eo:cloudCover}, … – «dc» namespace • E.g. {dc:title}, {dc:type} (service, collection, ..), {dc:publisher}, .. – «semantic» namespace • {semantic:classifiedAs} : search dataset series with (SKOS) concept URI
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 21 Architecture - Queryables
• dc:type allows for searches for dataset series, datasets or services e.g. in ISO AP, I15 EP catalogs. • Both dc:type and eo:parentIdentifier have expected defaults.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 22 Architecture - Parameters
• url=“http://fedeo.esa.int/opensearch/request? httpAccept=application/atom%2Bxml& query={searchTerms?}& recordSchema={sru:recordSchema?}& startRecord={startIndex?}& maximumRecords={count?}& bbox={geo:box}&…” • Parameter conventions: – searchRetrieve: Part 3. searchRetrieve Operation: APD Binding for SRU 2.0, Version 1.0, OASIS Standard, 30 January 2013 – searchRetrieve: Part 4. APD Binding for OpenSearch Version 1.0, OASIS Standard, 30 January 2013 – OpenSearch Extensions SRU 1.0 Draft 1 – OGC 10-032r8, OpenSearch Geo and Time Extensions
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 23 OpenSearch (Example with Gazetteer)
• Parameters: – Media type: Atom httpAccept – Collection: IKONOS {eo:parentIdentifier} – Less than 10% cloud cover {eo:cloudCover} – Area of interest: 1 km from Frascati {geo:name},{geo:radius} – Time frame: 2000 – 2013 {time:*} • Request: – http://fedeo.esa.int/opensearch/request/?httpAccept=application/atom%2Bxml&parentIdentifi er=EOP%3AEGEOS%3AEGEOS%23IKONOS&startDate=2000-10- 31T00:00:00Z&cloudCover=10]&endDate=2013-10- 31T00:00:00Z&name=frascati&radius=1000 – Use https://www.google.com/maps?output=classic as client.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 24 OpenSearch (Example – Google Maps)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 25 OpenSearch (Example - Firefox)
• Same result visualised directly by Firefox Web Browser
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 26 Architecture - Responses
• OGC 13-026r4 is flexible: – Metadata can be embedded or be external (atom:link). • FedEO Client can select recordSchema (and Media Type): – {sru:recordSchema} : allows client to decide which metadata is to be embedded as foreign markup (iso, om, om11, dc). – Give me the best metadata you have: «server-choice» – By default, atom:link are included instead.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 27 Architecture - Responses
Access to other media types and metadata formats (atom:summary)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 28 Responses - HATEOAS
• Hypermedia as the Engine of Application State – Standardised Media types – Standardised Link Relations (RFC 5988) • ‘first’, ‘previous’, ‘self’, ‘next’ and ‘last’ : Navigation • ‘search’ : Two step search • ‘enclosure’ : Product download • ‘payment’ : Product order page (future work - TBC)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 29 Responses – Media RSS
• Use of Media RSS as per OGC 13-026r4
Access to cloud mask, thumbnail, browse image
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 30 Responses – Media RSS
• FedEO Media RSS understood by third party applications …
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 31 Architecture - Responses
• Improved Atom response in case of ISO collection metadata with semantic annotations (OGC 08-167r2)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 32 Architecture - Exceptions
• Exception handling: – HTTP error codes as per OGC 10-032r8 – ows:ExceptionReport as per OGC 06-121r9 (OGC Web Services Common Standard)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 33 Architecture - Explain
• Explain – OASIS SearchRetrieve (Part 7) – Provides additional metadata about endpoint • E.g. response schemas • Possible values for parameters e.g. {eo:parentIdentifier} • Default values for search parameters etc. – Available at http://fedeo.esa.int/opensearch/request
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 34 Documents - Client Partner Guide
– Audience: Client developers, service consumers – General description of the FedEO environment and its relation to other systems – Details about the FedEO query interface based on OpenSearch (e.g., principles, the search parameters, search response format, etc ...) – Description of two cases: one starting from a dataset series catalogue, and a second one accessing immediately the dataset catalogue – Details on the catalogue connectors and how their corresponding dataset series and dataset metadata can be discovered
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 35 Documents – Software Configuration File
– Software Configuration File – Target audience: administrator OpenSearch Gateway.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 36 Software Configuration
• Endpoint configuration done via configuration file(s)
CSW Capabilities File (CWIC) …
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 37 Documents – Software Configuration
• Endpoint configuration done via configuration file: – dc:identifier, accessURL, type (protocol).
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 38 Documents – Software Configuration
Note: File format to change in next release
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 39 Deployment
(*) (*) (*) include series HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 40 metadata Deployment Architecture
• Connectors implemented supporting: – OGC 06-131 (CSW EOP EP) – OGC 10-032 (OpenSearch) – OGC 07-045 (CSW ISO AP) – OGC 07-038 (CSW CIM EP) – OGC 13-084 (CSW I15 EP) – CWIC (CSW) – ASF (API)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 41 Science Services Integrated (Oct. 2014)
Dataset Series / Collection Search
ISO OSDD RDF
FedEO OpenSearch
OSDD OSDD OSDD OSDD OSDD OSDD
ISO ISO ISO I15 REF GPOD VA4 (2) (3) (4) (5)
synch (gscda) GSCDA
ISO ISO @ESA
synch (cwic) ECHO IDN (6) synch/convert (fedeo)
OSDD is collection-specific.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 42 Science Services Integrated (Oct. 2014)
Dataset / Product Search EOP O&M ISO DC OSDD RDF
FedEO OpenSearch
EOP O&M
EOP DISS O&M SMOS M2CS GPOD VA4 (tar) CDS DAIL HARM
FedEO OpenSearch ISO
DC @ESA
CWIC VITO DLR EUM CNES ASF ECHO CCM CCM SPIRIT … SPOT CHARTER
… …
OSDD is collection-specific.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 43 Science Services - examples
• Alaska Satellite Facility – http://fedeo.esa.int/opensearch/request/?httpAccept=application/ato m%2Bxml&parentIdentifier=EOP:ASF&startDate=2011-01- 30T11:59:59Z&endDate=2011-03- 30T11:59:59Z&maximumRecords=10&frame=[290,310]
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 44 Science Services - examples
• NASA ECHO – http://fedeo.esa.int/opensearch/request/?httpAccept=applica tion/atom%2Bxml&parentIdentifier=EOP:NASA:ECHO&query =vegetation – http://fedeo.esa.int/opensearch/request/?parentIdentifier=E 1_STD_FRAME_L0&bbox=-80.0,- 90.0,80.0,90.0&startPage=1&maximumRecords=10
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 45 Science Services - examples
• CNES SPIRIT, Disasters Charter catalogs – http://fedeo.esa.int/opensearch/request/?parentIdentifier=E OP:CNES:SPIRIT&query=Russia – http://fedeo.esa.int/opensearch/request/?parentIdentifier=E OP:CNES:CHARTER&query=Germany • ESA DISSHARM – Landsat 5 etc. – http://geo.spacebel.be/opensearch/request/?parentIdentifier =LANDSAT_5_TM&startDate=2000-01- 01T00:00:00Z&endDate=2014-06-06T00:00:00Z
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 46 Science Service - DISSHARM
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 47 Science Services - Metrics
• Collections accessible through FedEO (26/09/2014)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 48 Outline
• Overall Architecture • Science Services Integration • OpenSearch • Other • SSE Portal • Conclusion and Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 49 Discovery and Ordering Gateway
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 50 Discovery and Ordering Gateway
• OGC 06-131 and 06-141 integration: – VITO • 1 VITO collection added to OGC 06-131 and OpenSearch – DLR • 23 DLR collections added to OGC 06-131 and OpenSearch • OGC 06-141 integration (DLR test server). – EUMETSAT • 13 Collections added to OGC 06-131 and OpenSearch
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 51 Outline
• Introduction • Science Services Integration • OpenSearch • Other • SSE Portal • Conclusion and Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 52 SSE Portal
• Instance of DAIL/SSE Portal with additional « clients » – Output of Evolution framecontract
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 53 SSE Portal
• OGC 06-131 Client • Pre-existing client (DAIL/SSE) • HMA-SE collections added to CDS EO-DAIL collection list...
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 54 SSE Portal - Ordering
Page 55 SSE Portal
• OGC 13-026 OpenSearch Client • New HMA-SE client. • Main purpose: test client for FEDEO Gateway
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 56 SSE Portal
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 57 SSE Portal
• CWIC product search
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 58 SSE Portal
• ASF Product Search
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 59 Outline
• Introduction • Science Services Integration • OpenSearch • Other • SSE Portal • Conclusion and Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 60 Conclusion
• Synergies with HMA-S – FedEO OpenSearch implementation pre-dated HMA-S specifications and provided inputs/feedback to HMA-S Tasks 3 and 4 (generalisation, ISO19139, MediaRSS, etc.). – FedEO OpenSearch implementation more general than HMA-S OpenSearch specification. – FedEO implementation exploits OGC 10-157r4 EOP O&M specification (Task 3): • CDS EO-DAIL, M2CS, ASF product metadata in EOP O&M available now. G-POD, SMOS, VA4 available on test platform. – FedEO implementation relies on software components from HMA-S and raised NCR against these components. • I15 EP Catalog Implementation (Task 5) • OGC 10-032r8 Catalog Implementation (Task 3)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 61 FedEO Components
• OpenSearch Gateway – Being reused in ESA CDS v3 (OpenSearch to OGC 06-131).
• Open-Source Components available at: – https://code.google.com/hosting/search?q=label=HMA – Documentation available on ESA HMA Wiki.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 62 Conclusion
• I/F Architecture exploits OGC 13-026r4 flexibility: – Mapping to EOP O&M Metadata (OGC 10-157r4) – Mapping to ISO19139 Metadata – Two-step and/or one step approach – Heterogeneous metadata types either: • Embedded – also ngEO approach • By reference (atom:link) – CEOS Recommendation. – Complemented with existing OpenSearch extensions: • "sru" • "semantics" – Can be extended with additional media types (RSS, SRU, RDF, HTML, JSON…) – Can be extended with additional link types (HATEOAS), e.g. for ordering, metadata editing, ... HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 63 Conclusion - Issues
• Implementation/Specification Issues –
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 64 Conclusion - Issues
• Implementation/Specification Issues – OGC 13-026r4 has all its queryables under eo: namespace. Some should be “dc:” instead. • Conformance Testing – Conformance Tests from HMA-S for OpenSearch • Not Tester-friendly: unsupported optional parameters flagged as errors. • Interpretation of empty BBOX should be a point. • Flag error if EOP O&M not embedded inside response: fails if a derived type (opt:, sar: etc.) or if included as atom:links or if other metadata type (ISO, …) … • Procedural – Who should contact whom when backend-systems not operational or failing.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 65
Future work
• Outline – Metadata completion – Proposed interface changes – CNES RESTo query analyzer integration – Interface harmonisation (homogeneous metadata formats) – RESTful and Linked Data interfaces
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 66 Future work – Collection Metadata
• Much remaining work to populate Dataset Series metadata catalogs. E.g. additional metadata in EOP:ESA:FEDEO:COLLECTIONS – http://smaad.spacebel.be/opensearch/request/?publisher=DL R&parentIdentifier=EOP:ESA:FEDEO:COLLECTIONS&type=col lection – Has positive effect on #collections that support 2-STEP search in FedEO.
511
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 67 Future work
• Interface changes: – Implement additional CEOS OpenSearch recommendations – Improve compliance with INSPIRE Download Service Technical Guidance. – Harmonise Atom response messages from different backends • Have 2nd-step OSDD link point to FEDEO in all cases. • Start counting results from 1 in all cases. • Support ISO and EOP O&M metadata format for more backends (GPOD, VA4 etc.)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 68
Future work
• Interface changes (cont’d): – Replace dc:subject by eo:parentIdentifier consistently. – Reduce supported parameters for dataset search in principal OSDD to those required by CEOS. – Refine OSDD generated for OGC 06-131 Connector: e.g. different parameters in case of Optical or Radar collection. Requires this to be discoverable on CDS side. – Support « virtual collections » and efficient parallel search with correct paging for product search. • Integration with Ordering via HATEOAS – Could more seamlessly integrate OpenSearch Gateway with HMA OGC 06-141 ordering…
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 69
Query Analyzer Integration
• Integration with CNES RESTo Query Analyzer. • Simplied OSDD understood by Web Browser search I/F
…
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 70 Query Analyzer Integration
• Gateway obtains platform, instrument and the other parameters from the CNES RESTo Query Analyzer
Search Terms FedEO Gateway RESTo Query Resolver (P3) List of OpenSearch parameters (P7)
• Example: – Search Terms: “Images acquired today by Spot HRV” – List of OpenSearch parameters returned by RESTo Query Analyzer: • "eo:platformShortName":"SPOT%” • "eo:instrument":"HRV%" • "time:start":"2014-06-17T00:00:00” • "time:end":"2014-06-17T23:59:59"
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 71 Query Analyzer Integration
• Integration with CNES RESTo Query Analyzer: – http://vm-ds-portal.spacebel.be/opensearch/request/? type=dataset&query=Images acquired today by Spot HRV – http://vm-ds- portal.spacebel.be/opensearch/request/?type=dataset&query=Images acquired yesterday by radarsat sar
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 72 Future work - Harmonisation
• Harmonisation Collection Metadata format (OGC 11-035r1)
• Harmonisation Product Metadata format (OGC 10-157r4)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 73 Future work - Harmonisation
• Harmonise Collection Metadata Format – http://smaad.spacebel.be/opensearch/request/?httpAccept=applicat ion/atom%2Bxml&query=MER_FRS_1P
CURRENT NEW
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 74 Future work - Harmonisation
• Harmonise Product Metadata Format – http://smaad.spacebel.be/opensearch/request/?httpAccept=application/atom%2Bxml&pare ntIdentifier=MER_FRS_1P&startDate=2004-05-16T00:00:00Z&endDate=2005-05- 16T00:00:00Z
NEW
CURRENT
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 75 Future Work – Faceted Search
• Faceted Search – “allowing users to explore a collection of information by applying multiple filters” (*) – E.g. provide links to (dataset series) results per group (e.g. FedEO, CWIC, ECHO, VA4, GPOD, CSCDA etc.) or other classification. – In roadmap of CNES and NASA implementations (Source: WGISS #38 Moscow) – Can be integrated in Atom-based responses using rel=« »: • Advertise number of items (if known): thr:count=123 (http://www.ietf.org/rfc/rfc4685.txt) • Other approaches at http://www.feedbooks.com/api/facets and http://opds- spec.org/specs/opds-catalog-1-1-20110627/#Facets. – Addressed in OASIS searchRetrieve specifications.
(*) http://en.wikipedia.org/wiki/Faceted_search
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 76 Future Work – FedEO Dataset Series Server / ISO19115-3 support
• Current implementation – HMA-S (Task 5) I15 EP Implementation with OpenSearch front-end. • Possible alternative – Derived from HMA-S (Task 3) EOP O&M Catalog Server. • Advantages: – Lightweight/fast. More native OpenSearch and scalable. – Based on SolR/Lucene -> generate Parameter extension / facet information - from index. – Less dependent on metadata format, e.g. future support for ISO19115-1 and -3.
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 77 Future work - Semantic Annotations
• HTML Microdata: – Recommended by CEOS. May be indexed by major search engines: Google, Yahoo, Bing – Support https://schema.org/Dataset in HTML response of FedEO OpenSearch Gateway
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 78 Future work – Additional RESTful binding
• RESTful requests: Resources Description ../series - GET Dataset series search ../series/{id} - GET Dataset series with given ID ../series/{id}/datasets - GET Dataset search ../ series/{id}/datasets/{id} - GET Dataset with given ID
– http://geo.spacebel.be/series/EOP:VITO:VGT_P • Response type controlled by: – Explicit parameter: httpAccept (current implementation) – Resource extension: .atom, .rss, .rdf (CEOS Best Practice) – Content negotiation: • Atom or RSS or RDF or HTML etc... (Linked Data requirement)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 79 Future work – Linked Data
• Open Linked Data: – Additional media types: • RDF media type for dataset series. – Make available FedEO metadata to the Linked Open Data (LOD) community. • W3C RDF, DCAT, LDP 1.0
ESA FedEO
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 80 Future work – Linked Data isni.org? organisation URI CEOS? GEMET eo:instrument keyword eo:platform URI URI
ESA FedEO dataset series loc.gov dataset URI Language URI NASA KMS Keyword Mgt System
ESA RARE loc.gov Multidomain Country thesaurus URI
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 81 Outline
• Introduction • Science Services Integration • OpenSearch • Other • SSE Portal • Conclusion and Future Work • Demo
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 82 FedEO Demo
• http://fedeo.esa.int/opensearch/readme.html
Examples of two-step approach: collections and granule search
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 83 FedEO Demo
• http://services-test.eoportal.org/web/guest/fedeo-demo- services
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 84 Additional information
• http://fedeo.esa.int/opensearch/readme.html • http://geo.spacebel.be/opensearch/readme.html • [email protected] • [email protected]
• Further reading: – http://www.esa.int/About_Us/ESA_Publications/ ESA_TM-21_Heterogeneous_Missions_Accessibility
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 85 Spare slides
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 86 FedEO Client Applications
• News reader: SharpReader (http://www.sharpreader.net/) – Test file fedeo-public.opml
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 87 FedEO Client Applications
• CNES Mapshup Client – GSCB Federation Pilot #3 (June 2014)
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 88 FedEO Client Applications
• Android App – SmartHMA
– https://code.google.com/hosting/search?q=label:HMA
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 89 FedEO Client Applications
• Android App – Proof of Concept
HMA-SE | Final Presentation | Frascati | 16 Oct 2014 Page 90 ISO Metadata Landscape
Page 91