EUROPEAN COMMISSION DIGIT Connecting Europe Facility CEF Context Broker ​ ​ Conformance Testing Version 1.0 Service Offering Description CEF Context Broker Conformance Testing - SOD - v1.0 page. 1 Document Status: Status Draft Document Approver(s): Name Role Document Reviewer(s): Name Role Summary of Changes: Version Date Created by Short Description of Changes 1.0 19/04/2019 Stefano De Panfilis Document creation and first complete release based on already delivered content on the matter CEF Context Broker Conformance Testing - SOD - v1.0 page. 2 CEF Context Broker Conformance Testing - SOD - v1.0 page. 3 Table of Contents 1. Approach and purpose of the document 5 1.1 Glossary 5 2. Introduction 6 2.1 Purpose of the service 7 2.2 Users 8 2.3 Scope 8 2.4 Benefits 9 3. Roles and Responsibilities 9 3.1 Software / Service Provider (aka Developer) 10 3.2 CEF CB Support Team 10 3.3 CEF CB Testing Team 11 3.4 CEF Stakeholder Management Office (SMO) 11 4. Orion Conformance Testing service step by step 12 4.1 Introduction 12 4.2 Step 0: Set-up of the Conformance Testing environment 13 4.2.1 Orion Context Broker set-up 13 4.2.2 Orion Context Broker second instance set-up 14 4.2.3 JMeter set up 14 4.3 Step 1: Execute the test 15 4.4 Test Cases 16 5. Cygnus Conformance Test Process 19 5.1 Introduction 20 5.2 Step 0: Set-up of the Conformance Testing environment 21 5.2.1 Cygnus set-up 21 5.2.2 JMeter set-up 21 5.3 Step 1: Execute the test 22 5.4 Testing with Storage 23 5.5 Test Cases 23 6. Terms and conditions 27 7. APPENDIX 28 7.1 Cygnus install.sh file 28 7.2 Cygnus start.sh file 31 CEF Context Broker Conformance Testing - SOD - v1.0 page. 4 1. APPROACH AND PURPOSE OF THE DOCUMENT The present document is the Service Offering Description (SOD) of the CEF Context Broker Conformance Testing service. Key content includes an explanation of the roles and responsibilities and the process description of Context Broker Conformance Testing service. This document is intended for any developer intending to develop their own implementation of Context Broker based on the FIWARE NGSI API specifications base of the current CEF Context Broker technologies and verify it is conformant to CEF Context Broker specifications. 1.1 Glossary The key terms used in this Service Offering Description are defined in the CEF Definitions section on the CEF Digital Single Web Portal: https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/CEF+Definitions The key acronyms used in this Service Offering Description are defined in the CEF Glossary on the CEF Digital Single Web Portal: https://ec.europa.eu/cefdigital/wiki/pages/viewpage.action?spaceKey=CEFDIGITAL&title=CEF+Glossa ry CEF Context Broker Conformance Testing - SOD - v1.0 page. 5 2. INTRODUCTION The CEF Context Broker enables organizations (including but not limited to public administrations) to manage and share data in real-time describing “what is currently happening” within their organizations or in the real world they manage or where they run their daily business processes. The CEF Context Broker provides the FIWARE NGSI API, which is a RESTful API enabling applications to provide updates and get access to context information. More concretely: ● Update context information, e.g. send updates on air quality data for a given district of the city, weather forecast for a given region or the administrative record created for handling a request issued by a given citizen ● Query context information. The Context Broker stores context information updated from applications, so queries are resolved based on that information. ● Being notified when changes on context information take place (e.g. the air quality in a given street has changed) or with a given frequency (e.g. get measures of the traffic in a street each minute) ● Register context provider systems which can be queried by the Context Broker to get the latest status of context, e.g. a system provided by the national meteorology agency which provides updated weather forecasts upon request. In this context the CEF Context Broker Conformance Testing service is offered to developers intending to develop their own implementation of Context Broker based on the FIWARE NGSI API specifications base of the current CEF Context Broker technologies and verify it is conformant to CEF Context Broker specifications. This document describes the Conformance Testing service provided as part of the Context Broker building block service offering. It introduces the purpose of the service, its users, its scope, its benefits, the concerned role and responsibilities as well as the overall process. Note that CEF Context Broker offers two different kind of testing services: ● Conformance Testing service: https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Conformance+testing The goal of the CEF Context Broker Conformance Testing service is to verify that an implementation of the CEF Context Broker specifications, being a software package either commercial or Open Source, conforms to the FIWARE NGSI v2 specifications. ● Connectivity Testing service: https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Connectivity+testing CEF Context Broker Conformance Testing - SOD - v1.0 page. 6 The goal of the CEF Context Broker Connectivity Testing service is to test if a newly deployed instance of the Context Broker, conformant with the FIWARE NGSI v2 specifications, can respond to sample requests. If successful, such a test confirms that the relevant component is in all likelihood correctly deployed and configured. This document only describes the Conformance Testing service. Please note that since the CEF Context Broker is composed by two different software components, namely Orion Context Broker and Cygnus, this document provides two separate sections for executing the conformance testing of the two components: these are section 4 for what concerns Orion and section 5 for what concerns Cygnus. 2.1 Purpose of the service The goal of the CEF Context Broker Conformance Testing service is to verify that an implementation of the CEF Context Broker specifications, a software package either commercial or Open Source, conforms to the FIWARE NGSI v2 specifications. The service is based on a set of ready to use test cases, guidelines on how to build a dedicated testing platform. The FIWARE NGSI v2 specifications are translated in test assertions (a measurable statement for evaluating the conformance of an implementation). These test assertions are the main unit for testing and reporting the conformance of an implementation. Based on the test assertions a set of test cases are derived which are implemented for the test platform in the form of executable test scripts. The result of the test execution is documented in a test report. In the positive case that the test report shows that all mandatory test cases have been passed, the software implementation becomes conformant. An implementation of the CEF Context Broker is marked FIWARE NGSI v2 Conformant if and only if all mandatory test cases have been successfully completed. This document describes the ready to use test cases and provides step-by-step guidelines through which build a dedicates test platform. The CEF Context Broker team supports, on demand, the users of the CEF Context Broker Conformance Testing service during the whole testing process from test planning, trough testing environment building, till test execution. A list of key terminology related to the CEF Context Broker Conformance Testing service is provided in the ​ table below. Terminology Definition CEF Context Broker Conformance Testing - SOD - v1.0 page. 7 [Terminology] [definition] [Terminology] [definition] Table 1 - CEF Context Conformance Testing Key Terminology ​ 2.2 Users The CEF Context Broker Conformance Testing service is intended for the following type of users: ● Software providers, aka Developers (Build) - to confirm that their software ​ product is conformant to the FIWARE NGSI v2 specifications. ● Service Providers, aka Developers (Adopt) - to confirm that their deployment of ​ the CEF Context Broker sample software or one of its stand-alone services is conformant to the FIWARE NGSI v2 specifications. 2.3 Scope The scope of Conformance services provided by the CEF Context Broker Support team is as follows: ● to test own implementations of FIWARE NGSI v2 specifications by Software Providers who do not aim at using the Sample Software provided by the CEF Context Broker team with unique aim at verifying the new implementations are conformant with FIWARE NGSI v2 specifications. The specifications are available at the following link: http://telefonicaid.github.io/fiware-orion/api/v2/stable/ ● to verify that a newly deployment made by a Service Provider of the Sample Software provided by the CEF Context Broker Team performs according the FIWARE NGSI v2 specifications. The CEF Context Broker Conformance Testing service is not intended for: ​ ​ ● superseding professional functional and non-functional test procedures on own implementation of the CEF Context Broker which are solely responsibility of the concerned Software Provider. CEF Context Broker Conformance Testing - SOD - v1.0 page. 8 2.4 Benefits The CEF Context Broker Conformance Testing service has been designed to generate a list of benefits to the users of the service: ● Confirm and assure your users/customers that your software package or implementation of the CEF Context Broker is conformant to the FIWARE NGSI v2 specifications ● Testing supported by professional staff of the European Commission ● Quick testing cycle with reduced cost and time ● Testing anywhere at anytime ● Tests are domain independent ● Ready to use test cases ● Easy set-up of testing platform ● Automatic test report generation. 3. ROLES AND RESPONSIBILITIES This section describes the main roles in the CEF Context Broker Conformance Testing service and their responsibilities. The following table summarises the split of roles and responsibilities between the different actors in the CEF Context Broker Conformance Testing process in the form of a RACI matrix where: ● Responsible (R): indicates the entities that performs the process-step.