CEF Context Broker Conformance Testing

EUROPEAN COMMISSION

DIGIT Connecting Europe Facility

Version 1.0

Service Offering Description

CEF Context Broker Conformance Testing - SOD - v1.0 page. 1

Document Status:

Status

Draft

Document Approver(s):

Name Role

Document Reviewer(s):

Name Role

Summary of Changes:

Version Date Created by Short Description of Changes

1.0 19/04/2019 Stefano De Panfilis Document creation and first complete release based on already delivered content on the matter

CEF Context Broker Conformance Testing - SOD - v1.0 page. 2

CEF Context Broker Conformance Testing - SOD - v1.0 page. 3

Table of Contents

1. Approach and purpose of the document 5 1.1 Glossary 5

2. Introduction 6 2.1 Purpose of the service 7 2.2 Users 8 2.3 Scope 8 2.4 Benefits 9

3. Roles and Responsibilities 9 3.1 Software / Service Provider (aka Developer) 10 3.2 CEF CB Support Team 10 3.3 CEF CB Testing Team 11 3.4 CEF Stakeholder Management Office (SMO) 11

4. Orion Conformance Testing service step by step 12 4.1 Introduction 12 4.2 Step 0: Set-up of the Conformance Testing environment 13 4.2.1 Orion Context Broker set-up 13 4.2.2 Orion Context Broker second instance set-up 14 4.2.3 JMeter set up 14 4.3 Step 1: Execute the test 15 4.4 Test Cases 16

5. Cygnus Conformance Test Process 19 5.1 Introduction 20 5.2 Step 0: Set-up of the Conformance Testing environment 21 5.2.1 Cygnus set-up 21 5.2.2 JMeter set-up 21 5.3 Step 1: Execute the test 22 5.4 Testing with Storage 23 5.5 Test Cases 23

6. Terms and conditions 27

7. APPENDIX 28 7.1 Cygnus install.sh file 28 7.2 Cygnus start.sh file 31

CEF Context Broker Conformance Testing - SOD - v1.0 page. 4

1. APPROACH AND PURPOSE OF THE DOCUMENT

The present document is the Service Offering Description (SOD) of the CEF Context Broker Conformance Testing service. Key content includes an explanation of the roles and responsibilities and the process description of Context Broker Conformance Testing service.

This document is intended for any developer intending to develop their own implementation of Context Broker based on the FIWARE NGSI API specifications base of the current CEF Context Broker technologies and verify it is conformant to CEF Context Broker specifications.

1.1 Glossary

The key terms used in this Service Offering Description are defined in the CEF Definitions section on the CEF Digital Single Web Portal:

https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/CEF+Definitions

The key acronyms used in this Service Offering Description are defined in the CEF Glossary on the CEF Digital Single Web Portal: https://ec.europa.eu/cefdigital/wiki/pages/viewpage.action?spaceKey=CEFDIGITAL&title=CEF+Glossa ry

CEF Context Broker Conformance Testing - SOD - v1.0 page. 5

2. INTRODUCTION

The CEF Context Broker enables organizations (including but not limited to public administrations) to manage and share data in real-time describing “what is currently happening” within their organizations or in the real world they manage or where they run their daily business processes.

The CEF Context Broker provides the FIWARE NGSI API, which is a RESTful API enabling applications to provide updates and get access to context information. More concretely:

● Update context information, e.g. send updates on air quality data for a given district of the city, weather forecast for a given region or the administrative record created for handling a request issued by a given citizen

● Query context information. The Context Broker stores context information updated from applications, so queries are resolved based on that information.

● Being notified when changes on context information take place (e.g. the air quality in a given street has changed) or with a given frequency (e.g. get measures of the traffic in a street each minute)

● Register context provider systems which can be queried by the Context Broker to get the latest status of context, e.g. a system provided by the national meteorology agency which provides updated weather forecasts upon request.

In this context the CEF Context Broker Conformance Testing service is offered to developers intending to develop their own implementation of Context Broker based on the FIWARE NGSI API specifications base of the current CEF Context Broker technologies and verify it is conformant to CEF Context Broker specifications.

This document describes the Conformance Testing service provided as part of the Context Broker building block service offering. It introduces the purpose of the service, its users, its scope, its benefits, the concerned role and responsibilities as well as the overall process.

Note that CEF Context Broker offers two different kind of testing services:

● Conformance Testing service: https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Conformance+testing

The goal of the CEF Context Broker Conformance Testing service is to verify that an implementation of the CEF Context Broker specifications, being a software package either commercial or Open Source, conforms to the FIWARE NGSI v2 specifications.

● Connectivity Testing service: https://ec.europa.eu/cefdigital/wiki/display/CEFDIGITAL/Connectivity+testing

CEF Context Broker Conformance Testing - SOD - v1.0 page. 6

The goal of the CEF Context Broker Connectivity Testing service is to test if a newly deployed instance of the Context Broker, conformant with the FIWARE NGSI v2 specifications, can respond to sample requests. If successful, such a test confirms that the relevant component is in all likelihood correctly deployed and configured.

This document only describes the Conformance Testing service.

Please note that since the CEF Context Broker is composed by two different software components, namely Orion Context Broker and Cygnus, this document provides two separate sections for executing the conformance testing of the two components: these are section 4 for what concerns Orion and section 5 for what concerns Cygnus.

2.1 Purpose of the service

The goal of the CEF Context Broker Conformance Testing service is to verify that an implementation of the CEF Context Broker specifications, a software package either commercial or Open Source, conforms to the FIWARE NGSI v2 specifications.

The service is based on a set of ready to use test cases, guidelines on how to build a dedicated testing platform.

The FIWARE NGSI v2 specifications are translated in test assertions (a measurable statement for evaluating the conformance of an implementation). These test assertions are the main unit for testing and reporting the conformance of an implementation.

Based on the test assertions a set of test cases are derived which are implemented for the test platform in the form of executable test scripts. The result of the test execution is documented in a test report. In the positive case that the test report shows that all mandatory test cases have been passed, the software implementation becomes conformant.

An implementation of the CEF Context Broker is marked FIWARE NGSI v2 Conformant if and only if all mandatory test cases have been successfully completed.

This document describes the ready to use test cases and provides step-by-step guidelines through which build a dedicates test platform. The CEF Context Broker team supports, on demand, the users of the CEF Context Broker Conformance Testing service during the whole testing process from test planning, trough testing environment building, till test execution.

A list of key terminology related to the CEF Context Broker Conformance Testing service is provided in the table below. Terminology Definition

CEF Context Broker Conformance Testing - SOD - v1.0 page. 7

[Terminology] [definition]

Table 1 - CEF Context Conformance Testing Key Terminology

2.2 Users

The CEF Context Broker Conformance Testing service is intended for the following type of users:

● Software providers, aka Developers (Build) - to confirm that their software product is conformant to the FIWARE NGSI v2 specifications. ● Service Providers, aka Developers (Adopt) - to confirm that their deployment of the CEF Context Broker sample software or one of its stand-alone services is conformant to the FIWARE NGSI v2 specifications.

2.3 Scope

The scope of Conformance services provided by the CEF Context Broker Support team is as follows:

● to test own implementations of FIWARE NGSI v2 specifications by Software Providers who do not aim at using the Sample Software provided by the CEF Context Broker team with unique aim at verifying the new implementations are conformant with FIWARE NGSI v2 specifications. The specifications are available at the following link: http://telefonicaid.github.io/fiware-orion/api/v2/stable/ ● to verify that a newly deployment made by a Service Provider of the Sample Software provided by the CEF Context Broker Team performs according the FIWARE NGSI v2 specifications.

The CEF Context Broker Conformance Testing service is not intended for: ● superseding professional functional and non-functional test procedures on own implementation of the CEF Context Broker which are solely responsibility of the concerned Software Provider.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 8

2.4 Benefits

The CEF Context Broker Conformance Testing service has been designed to generate a list of benefits to the users of the service:

● Confirm and assure your users/customers that your software package or implementation of the CEF Context Broker is conformant to the FIWARE NGSI v2 specifications ● Testing supported by professional staff of the European Commission ● Quick testing cycle with reduced cost and time ● Testing anywhere at anytime ● Tests are domain independent ● Ready to use test cases ● Easy set-up of testing platform ● Automatic test report generation.

3. ROLES AND RESPONSIBILITIES

This section describes the main roles in the CEF Context Broker Conformance Testing service and their responsibilities.

The following table summarises the split of roles and responsibilities between the different actors in the CEF Context Broker Conformance Testing process in the form of a RACI matrix where:

● Responsible (R): indicates the entities that performs the process-step. Every process-step has at least one responsible entity. Responsibilities can also be shared. ● Accountable (A): indicates the entity that is ultimately accountable for the process-step. Every process-step has only one accountable entity. ● Consulted (C): indicates the entities that give feedback or are consulted during the process-step. This is a two-way process. Not every process-step has an entity that is being consulted. ● Informed (I): indicates the entities that needs to be informed on the results of the process-step. This is a one-way process. Not every process-step has an entity that is being informed.

The process is described in detail in the following Section 4.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 9

Process/Service Responsible Entity

Developer CEF CB CEF CB CEF Support Testing SMO Team Team

Step 0: Set-up of the RI CI RC I Conformance Testing Environment

Step 1: Execute the test RA C C I

Table 2 – CEF Context Broker Conformance Testing Environment service Roles and Responsibilities

3.1 Software / Service Provider (aka Developer)

Role: user (of the CEF Context Broker Conformance Testing service) Responsibilities: ● Requests the service and provide the necessary information at registration ● Integrates the software product or implementation with the testing platform ● Tests the software product or implementation based on the specifications and related test assertions ● If needed, provides the CEF Context Broker Testing Team with a version of the software product or implementation that is ready for conformance testing.

3.2 CEF CB Support Team

Role: First line support. Responsibilities: ● Registers new requests from software or service providers applying for the CEF Context Broker Conformance Testing service

● Acts as first line support to coordinate and follow-up the work between the software / service provider and the CEF Context Broker Testing Team

● Provides the software or service provider with information about the specifications, related test assertion, and the testing tool

● Signs off the final test report for submission to the software or service provider.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 10

3.3 CEF CB Testing Team

Role: second line & technical support Responsibilities: ● Acts as the technical single point of contact to the software or service provider and provides support during the set-up of the test platform and the test execution ● Conducts a final test run and generate the test report.

3.4 CEF Stakeholder Management Office (SMO)

Role: communication and stakeholder management Responsibilities: ● Publishes and maintains an overview of conformant software products or implementations and the related test reports on the CEF Digital Single Web Portal.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 11

4. ORION CONFORMANCE TESTING SERVICE STEP BY STEP

This section describes the process, part of the overall CEF Context Broker Conformance Testing service, concerning the Orion Context Broker conformance test.

4.1 Introduction

Purpose: Purpose of this step is to create a suitable and easy to use testing environment. It can be easily set up either in your own environment or using the CEF Context Broker Sandbox, which is based on the cloud operating system OpenStack. The testing environment is designed in a such a way that test cases will be executed automatically so that the test can be repeated any time assuring the test results are uniform and comparable.

Actors: ● Developer

● CEF CB Support Team

● CEF CB Testing Team

Process: In order to test Orion, three Virtual Machines are needed. They are:

1. Orion Context Broker - follow the instruction to deploy a dedicated Orion instance 2. Orion Context Broker GE - deploy a second instance of Orion. This will be used as context provider 3. JMeter – create a an ubuntu 16.04 virtual machine. If you are using the CEF Sandbox environment you need to simply select the "base_ubuntu_16.04" image and run it. This machine will be used to install JMeter on an Ubuntu Virtual Machine.

The process itself is composed by the following two main steps:

● Step 0: Set-up of the Conformance Testing environment

○ Context Broker 1

○ Context Broker 2

○ JMeter

CEF Context Broker Conformance Testing - SOD - v1.0 page. 12

● Step 1: Execute the test.

4.2 Step 0: Set-up of the Conformance Testing environment

4.2.1 Orion Context Broker set-up

Deploy the instance of Orion Context Broker you want to test in the testing environment. Follow the following instructions if you want to install Orion from scratch:

1. deploy a Centos 7 VM and connect to it in SSH: 2. copy the contextBroker-2.0.0-1.x86_64.rpm file in the /home/centos older or use the wget command:

wget -O contextBroker-2.0.0-1.x86_64.rpm https://nexus.lab.fiware.org/repository/el/7/x86_64/release/contextBrok er-2.0.0-1.x86_64.rpm

3. install the needed dependencies:

sudo yum install boost boost-devel boost-doc openssl-libs

4. install and start mongodb: 4.1. create a mongodb-org.repo file: sudo vi /etc/yum.repos.d/mongodb-org.repo

and add these lines:

[mongodb-org-3.4]

name=MongoDB Repository

baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-o rg/3.4/x86_64/

gpgcheck=1

enabled=1

gpgkey=https://www.mongodb.org/static/pgp/server-3.4.asc

4.2. then start mongodb:

sudo yum repolist

sudo yum install mongodb-org

sudo systemctl start mongod

5. install and start orion:

CEF Context Broker Conformance Testing - SOD - v1.0 page. 13

sudo rpm -i contextBroker-2.0.0-1.x86_64.rpm

sudo service contextBroker start

6. and check that the version of Orion is the one you want to test

curl http://localhost:1026/version

4.2.2 Orion Context Broker second instance set-up

As for the previous step, please note that you can use any Orion Context Broker image in the CEF Sandbox or you can follow the step 1 to install from scratch an Orion instance. This Orion instance will be only the context provider for the previous.

Then open the /etc/hosts file by using this command:

sudo vi /etc/hosts and add Orion IP of the second Orion VM with provider alias (the second instance of Orion must be used as context provider for testing this Orion instance), for instance:

192.168.111.175 provider and check the version

curl http://provider:1026/version

4.2.3 JMeter set up

Open the /etc/hosts file by using this command: sudo vi /etc/hosts and verify that Orion IPs of previous step VMs created with orion and provider aliases according to your instance are properly included, for instance:

192.168.111.174 orion

192.168.111.175 provider

Copy in the /tmp/ folder the OrionContextBroker-2.0.0.jmx file. This file is the file that contains the scripts for automatically running the test cases.

At this point you can Install JMeter 4.0 on Ubuntu 16.04 as described in the following steps:

1. sudo add-apt-repository ppa:webupd8team/java - add Java in the repository 2. sudo apt-get update - to refresh packages metadata

CEF Context Broker Conformance Testing - SOD - v1.0 page. 14

3. sudo apt-get install oracle-java8-installer - Java 8 is pre-requisite for JMeter 4.0 4. sudo wget -c http://ftp.ps.pl/pub/apache/jmeter/binaries/apache-jmeter-4.0.tgz - download JMeter 4.0 5. sudo tar -xf apache-jmeter-4.0.tgz - unpack JMeter

4.3 Step 1: Execute the test

As said above in this document, the test will be executed automatically. To run the various test cases composing the test it is enough to execute the follow command:

./apache-jmeter-4.0/bin/jmeter -n -t /tmp/OrionContextBroker-2.0.0.jmx

Once the JMeter test session is ended it is possible to retrieve the test results. They are collected in a csv formatted file which is placed in the same folder where you are using the jmx file and named as following: orion_context_broker-2.0.0_yyyy-MM-dd HHmmss.csv where “yyyy-MM-dd HHmmss” is the date-time of the test.

An excerpt of a test results file is provided in the following picture (it can be different according to the visualization software you are using, this case it is visualized embedded in the Github environment).

CEF Context Broker Conformance Testing - SOD - v1.0 page. 15

Figure 1 – Excerpt of the test results file for orion

4.4 Test Cases

The following table lists all the test cases which can be executed through the OrionContextBroker-2.0.0.jmx file.

# Case Name Case Description Method API Path 1 getAPIResources Retrieve API GET /v2 Resources 2 setLogLevel Set log level PUT /admin/log?level=${LOG_LEVEL} 3 getLogLevel Retrieve log level GET /admin/log 4 setLogTrace Set log trace PUT /log/trace/${TRACE_LEVEL} 5 getLogTrace Retrieve log trace GET /log/trace 6 deleteLogTrace Delete log trace DELETE /log/trace/${TRACE_LEVEL} 7 setLogTraces Set log traces PUT /log/trace/${TRACE_LEVEL1},${TRACE_LEVEL2} 8 getLogTraces Retrieve log GET /log/trace traces 9 deleteLogTraces Delete log traces DELETE /log/trace/${TRACE_LEVEL1},${TRACE_LEVEL2}

CEF Context Broker Conformance Testing - SOD - v1.0 page. 16

10 getSemaphore Retrieve GET /admin/sem semaphore 11 getMetrics Retrieve metrics GET 12 getAndResetMetrics Retrieve and GET /admin/metrics?reset=true reset metrics 13 resetMetrics Reset metrics DELETE /admin/metrics 14 getStatistics Retrieve statistics GET /statistics 15 resetStatistics Reset statistics DELETE /statistics 16 getCacheStatistics Retrieve cache GET /cache/statistics statistics 17 resetCacheStatistics Reset cache DELETE /cache/statistics statistics 18 createFirstEntity Create the first POST /v2/entities entity. This entity is needed for the following test cases from 20 to 38. 19 createSecondEntity Create a second POST /v2/entities entity 20 countEntities Retrieve the total GET /v2/entities?options=count number of entities in the context 21 queryKeyValues Retrieve values GET /v2/entities/${ENTITY_ID}/attrs?options=keyValues of all attributes of identified entities with query option = keyValues 22 queryType Retrieve values GET /v2/entities?type=${ENTITY_TYPE}&options=keyValues of all attributes of entities of a specified type with query option = keyValues 23 queryFilter Retrieve values GET /v2/entities?q=${ATTRIBUTE_NAME}%3E${__Random(1,9 of all attributes 9)} of entities if a given attribute value is > 10 24 getTypes Retrieve the list GET /v2/types of all entity types 25 getType Retrieve detailed GET /v2/types/${ENTITY_TYPE} information of a given specified type 26 getAttribute Retrieve a given GET /v2/entities/${ENTITY_ID}/attrs/${ATTRIBUTE_NAME} attribute of a given entity 27 getAttributeValue Retrieve the GET /v2/entities/${ENTITY_ID}/attrs/${ATTRIBUTE_NAME}/val value of a given ue attribute of a given entity 28 retrieveAllAttrs Retrieve all GET /v2/entities/${ENTITY_ID}/attrs attributes of a given entity

CEF Context Broker Conformance Testing - SOD - v1.0 page. 17

29 retrieveEntities Retrieve a GET /v2/entities/${ENTITY_ID} specific Entity 30 allEntities List all entities GET /v2/entities 31 geEntitiesOrderByAttr List entities order GET /v2/entities?orderBy=speed by a specified attribute (this case “speed”) 32 geEntitiesOrderById List entities order GET /v2/entities?orderBy=id by entity Id 33 updateAttribute Update the value PUT /v2/entities/${ENTITY_ID}/attrs/${ATTRIBUTE_NAME}/val of a given ue attribute of a given entity 34 checkAttribute Check the GET /v2/entities/${ENTITY_ID}/attrs?options=keyValues attribute value of test case 33 is correctly updated 35 replaceAttribute Replace attribute PATCH /v2/entities/${ENTITY_ID}/attrs value 36 deleteAttribute Remove a single DELETE /v2/entities/${ENTITY_ID}/attrs/${ATTRIBUTE_NAME} attribute from a given entity 37 deleteFirstEntity Delete the entity DELETE /v2/entities/${ENTITY_ID} created in test case 18 38 deleteSecondEntity Delete the entity DELETE /v2/entities/${ENTITY_ID2} created in test case 19 39 insertEntities Batch operation: POST /v2/op/update insert two entities 40 updateEntities Batch operation: POST /v2/op/update update two entities 41 getEntities Batch operation: POST /v2/op/query query 42 deleteEntities Batch operation: POST /v2/op/update delete two entities 43 addSubscription Create a new POST /v2/subscriptions subscription 44 getSubscriptionByID Retrieve GET /v2/subscriptions/${subscriptionId} subscription By ID 45 getSubscriptions Retrieve all GET /v2/subscriptions subscriptions 46 updateSubscriptionByID Update PATCH /v2/subscriptions/${subscriptionId} subscription by ID 47 updateStatus Update status of PATCH /v2/subscriptions/${subscriptionId} subscription 48 getStatus Retrieve GET /v2/subscriptions/${subscriptionId} subscription status 49 deleteSubscription Delete DELETE /v2/subscriptions/${subscriptionId} Subscription

CEF Context Broker Conformance Testing - SOD - v1.0 page. 18

50 updateAttributeProvider Update provider: POST /v2/op/update another entity 51 createRegistration Create POST /v2/registrations Registration 52 getRegistrationByID Retrieve GET /v2/registrations/${registrationId} registration by Id 53 getRegistrations Retrieve all GET /v2/registrations?limit=15&offset=0&options=count Registrations 54 updateRegistrationByID Update PATCH /v2/registrations/${registrationId} Registration by ID - [PATCH /v2/registration/ is not implemented] 55 retrieveEntities Retrieve Entity as GET /v2/entities/${ENTITY_ID_PROVIDER} Provider 56 updateAttribute Update attribute PUT /v2/entities/${ENTITY_ID_PROVIDER}/attrs/${ATTRIBUTE_ value NAME}?type=Car 57 getAttributeProvider Retrieve GET /v2/entities/${ENTITY_ID_PROVIDER}/attrs/${ATTRIBUTE_ attribute value NAME} from Context Provider 58 deleteRegistrationByID Delete DELETE /v2/registrations/${registrationId} Registration By ID 59 addGeoPoint Add three geo POST /v2/op/update points: Madrid, Leganés and Alcobendas. These geo points are used in the following test cases: from 60 to 62. 60 maxDistance Retrieve cities GET /v2/entities?idPattern=.*&type=City&georel=near;maxDis within max tance:13500&geometry=point&coords=40.418889,-3.691 distance 944 61 minDistance Retrieve cities GET /v2/entities?idPattern=.*&type=City&georel=near;minDis within min tance:13500&geometry=point&coords=40.418889,-3.691 distance 944 62 deleteAllPoint Batch operation: POST /v2/op/update delete the three geo points created in test case 59

Table 3 - CEF Context Broker: Orion test cases

5. CYGNUS CONFORMANCE TEST PROCESS

This section describes the process, part of the overall CEF Context Broker Conformance Testing service, concerning the Cygnus conformance test.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 19

Cygnus is the component of the CEF Context Broker building block in charge of persisting certain sources of data in certain configured third-party storages, creating a historical view of such data. Cygnus (more specifically, cygnus-ngsi agent) plays the role of a connector between Orion Context Broker (which is a NGSI source of data) and many storages such as CKAN, Cosmos Big Data (Hadoop) and STH Comet, so it is recommended to use/install Cygnus adapter in the same Orion Context Broker Virtual Machine.

5.1 Introduction

Actors: ● Developer

● CEF CB Support Team

● CEF CB Testing Team

Process: In order to test Cygnus, you need to set-up two Virtual Machines, which are:

● Cygnus - follow the instruction in the next step to install Cygnus via source. ● JMeter – create a an ubuntu 16.04 virtual machine. If you are using the CEF Sandbox environment you need simply to select the "base_ubuntu_16.04" image and run it. This machine will be used to install JMeter on an Ubuntu Virtual Machine.

The process itself is composed by the following two main steps:

● Step 0: Set-up of the Conformance Testing environment

○ Cygnus set-up

○ JMeter set-up

● Step 1: Execute the test.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 20

5.2 Step 0: Set-up of the Conformance Testing environment

5.2.1 Cygnus set-up

In order to install Cygnus via source to test APIs with JMeter, you have to deploy a Centos 7 virtual machine (VM). After connected on VM via SSH, you have to run the install.sh, script provided in appendix of this document, to install the software. Please note that before to run the script, you must change the permissions of the install.sh file (i.e. chmod +x install.sh) and run it as root user (sudo -i). At the end of the installation process you have to start the Cygnus server using the script start.sh, script provided in appendix of this document, after having changed the permissions for this file too.

Summarising the commands to be executed are: sudo -i

Copy install.sh and start.sh files on VM, and change permissions: chmod +x install.sh chmod +x start.sh

Run the installation:

./install.sh

Please note that for this test (API) it is not necessary to configure any storage systems. The start.sh script creates for you two default files (agent_ngsi_api.conf and cygnus_instance_api.conf) to test the Cygnus APIs.

Run the server:

./start.sh

5.2.2 JMeter set-up

Open the /etc/hosts file by using this command: sudo vi /etc/hosts and add Cygnus IP as Cygnus alias according to your instance, for instance: 192.168.111.71 cygnus

Copy in the /tmp/ folder the Cygnus-1.9.0_API.jmx and file.properties files. Edit the file.properties file with your credentials (in particular email and password). This step

CEF Context Broker Conformance Testing - SOD - v1.0 page. 21

depends where you have installed your VMs: if they are in the CEF Sandbox environment you can use your Sandbox credentials.

At this point you can Install JMeter 4.0 on Ubuntu 16.04 as described in the following steps:

1. sudo add-apt-repository ppa:webupd8team/java - add Java in the repository 2. sudo apt-get update - to refresh metadata packages 3. sudo apt-get install oracle-java8-installer - Java 8 is pre-requisite for JMeter 4.0 4. sudo wget -c http://ftp.ps.pl/pub/apache/jmeter/binaries/apache-jmeter-4.0.tgz - download JMeter 4.0 5. sudo tar -xf apache-jmeter-4.0.tgz - unpack JMeter

5.3 Step 1: Execute the test

As said above, the tests will be executed automatically. To run the tests it is enough to execute the follow command:

./apache-jmeter-4.0/bin/jmeter -n -t /tmp/Cygnus-1.9.0_API.jmx

Once the JMeter test session is ended it is possible to retrieve the test results. They are collected in a csv formatted file which is placed in the same folder where you are using the jmx file and named as following: cygnus-1.9.0_api_yyyy-MM-dd HHmmss.csv where “yyyy-MM-dd HHmmss” is the date-time of the test.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 22

Figure 2 – Excerpt of the test results file for Cygnus

5.4 Testing with Storage

You can also run the test to check how Cygnus stores the context information by using the related documentation available at https://github.com/Fiware/test.Functional/blob/master/API.test/data.Cygnus/1.9.0/cygnu s.storage.

5.5 Test Cases

The following table lists all the test cases which can be executed through the Cygnus-1.9.0_API.jmx file.

# Name Description Method Path 1 getStat Retrieve all the GET /v1/stats information of the flume component 2 resetStat Reset by a PUT PUT /v1/stats request 3 addGroupingRules A grouping rule can POST /v1/groupingrules be added by POST method. 4 getGroupingRules Grouping rules can GET /v1/groupingrules be retrieved by GET methods.

CEF Context Broker Conformance Testing - SOD - v1.0 page. 23

5 updateGroupingRules A grouping rule can PUT /v1/groupingrules?id=${id} be updated by PUT method. 6 deleteGroupingRules A grouping rule can DELETE /v1/groupingrules?id=${id} be deleted by DELETE method. 7 getTokenFiwareLab Get Token from POST /v3/auth/tokens FIWARE LAB 8 createSubscriptionsV1 Create POST /v1/subscriptions?ngsi_version=1 subscriptions ngsi_version=1 9 createSubscriptionsV2 Create POST /v1/subscriptions?ngsi_version=2 subscriptions ngsi_version=2 10 getAllSubscriptionsV2 Get subscriptions GET /v1/subscriptions?ngsi_version=2 ngsi_version=2 11 getSubscriptionV2 Get subscription GET /v1/subscriptions?ngsi_version=2&subscription_id=${ ngsi_version=2 by subscriptionId2} subscriptionId 12 deleteSubscriptionsV2 Subscriptions can DELETE /v1/groupingrules?ngsi_version=2&subscription_id=${ be deleted by subscriptionId2} DELETE methods, deleting a subscription by its ID for ngsi_version=2. 13 addAppenders Posts an appender POST /v1/admin/log/appenders?transient=false in a running Cygnus given a JSON with the information about the name and class of the appender and its layout and ConversionPattern of its pattern

14 getAppender Gets an existent GET /v1/admin/log/appenders?name=${LOG_DAILY}&tran appender from a sient=false running logger given its name 15 setAppender Puts an appender PUT /v1/admin/log/appenders?transient=false in a running Cygnus given a JSON with the information about the name and class of the appender and its layout and ConversionPattern of its pattern

16 getAllAppenders Gets all existent GET /v1/admin/log/appenders?transient=false appenders from a running logger 17 deleteAppender Deletes an existent DELETE /v1/admin/log/appenders?name=${LOG_DAILY}&tran appender from a sient=false running logger given its name

CEF Context Broker Conformance Testing - SOD - v1.0 page. 24

18 deleteAppenders Deletes all existent DELETE /v1/admin/log/appenders?transient=true appenders from a running logger

19 addLoggers Posts an logger on POST /v1/admin/log/loggers?transient=false a running Cygnus

20 getLogger Gets an existent GET /v1/admin/log/loggers?name=${LOGGER_NAME}&tr logger from a ansient=false running Cygnus given its name 21 setLogger Puts an logger in a PUT /v1/admin/log/loggers?transient=false running Cygnus given a JSON with the information about the name and level of the logger 22 getAllLoggers Gets all existent GET /v1/admin/log/loggers?transient=true loggers from a running Cygnus 23 deleteLogger Deletes an existent DELETE /v1/admin/log/loggers?name=${LOGGER_NAME}&tr logger from a ansient=false running Cygnus given its name

24 deleteLoggers Deletes all existent DELETE /v1/admin/log/loggers?transient=true loggers from a running Cygnus

25 getMetrics Gets metrics for a GET /v1/admin/metrics whole Cygnus agent 26 deleteMetrics Deletes metrics, DELETE /v1/admin/metrics putting counters to zero 27 getLogging Logging level can GET /admin/log be retrieved by GET method. 28 updateLogging Updates the PUT /admin/log?level=DEBUG logging level of Cygnus, given the logging level as a query parameter 29 getAllParamsFromAgent Gets all the GET /admin/configuration/agent/usr/cygnus/conf/${AGENT parameters from an _NAME} agent given the path to the configuration file as the URI within the URL 30 addParamInTheAgent Posts a single POST /admin/configuration/agent/usr/cygnus/conf/${AGENT parameter if it _NAME}? doesn't exist in the agent given

CEF Context Broker Conformance Testing - SOD - v1.0 page. 25

31 getParamFromAgent Gets a single GET /admin/configuration/agent/usr/cygnus/conf/${AGENT parameter from an _NAME}?param=${AGENT_NAME_PARAM}.${AGE agent given the NT_NAME_PARAM_VALUE} path to the configuration file as the URI within the URL and the name of the parameter as a query parameter

32 updateParamInTheAgent Puts a single PUT /admin/configuration/agent/usr/cygnus/conf/${AGENT parameter if it _NAME}?param=${AGENT_NAME_PARAM}.${AGE doesn't exist or NT_NAME_PARAM_VALUE}&value=${AGENT_VAL update it if already UE_PARAM} exists in the agent given 33 deleteParamInTheAgent Deletes a single DELETE /admin/configuration/agent/usr/cygnus/conf/${AGENT parameter if it _NAME}?param=${AGENT_NAME_PARAM}.${AGE exists in the agent NT_NAME_PARAM_VALUE} given 34 getAllParamsFromInstance Gets all the GET /admin/configuration/instance/usr/cygnus/conf/${INST parameters from an ANCE_NAME} instance given the path 35 addParamInTheInstance Posts a single POST /admin/configuration/instance/usr/cygnus/conf/${INST parameter if it ANCE_NAME}?param=${INSTANCE_NAME_PARA doesn't exist in the M}&value=value instance given 36 getParamFromInstance Gets a single GET /admin/configuration/instance/usr/cygnus/conf/${INST parameter from an ANCE_NAME}?param=AGENT_NAME instance given the path 37 updateParamInTheAgent Puts a single PUT /admin/configuration/instance/usr/cygnus/conf/${INST parameter if it ANCE_NAME}?param=${INSTANCE_NAME_PARA doesn't exist or M}&value=value1 update it if already exists in the instance given 38 deleteParamInTheAgent Deletes a single DELETE /admin/configuration/instance/usr/cygnus/conf/${INST parameter in the ANCE_NAME}?param=${INSTANCE_NAME_PARA instance given the M} path

Table 4 - CEF Context Broker: Cygnus test cases

CEF Context Broker Conformance Testing - SOD - v1.0 page. 26

6. TERMS AND CONDITIONS

The general terms and conditions of CEF Building Blocks can be consulted in the Master Service Arrangement, available on the CEF Digital Single Web Portal:

[Link to the SWP page]

The terms and conditions specific to the CEF Context Broker Conformance Testing service are described in the table below.

Term / Condition Description

To this service apply the term and conditions as described in the following page CEF Context Broker Sandbox Conformance Testing service http://...

Table 5 - CEF Context Broker Conformance Testing service Terms and Conditions

CONTACT INFORMATION

CEF Support Team

By email: [email protected]

By phone: +32 2 299 09 09

● Standard Service: 8am to 6pm (Normal EC working Days)

● Standby Service*: 6pm to 8am (Commission and Public Holidays, Weekends)

* Only for critical and urgent incidents and only by phone

CEF Context Broker Conformance Testing - SOD - v1.0 page. 27

7. APPENDIX

7.1 Cygnus install.sh file

#!/bin/bash

# please change permission to this file (chmod +x install.sh) and run it as root user

# Install JDK

echo "Install JDK" yum -y install tar nc which git java-1.8.0-openjdk-devel

yum clean all

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk # Install Maven

echo "Install Maven" curl --remote-name --location --insecure --silent --show-error "http://www.eu.apache.org/dist/maven/maven-3/3.5.3/binaries/apache-maven-3. 5.3-bin.tar.gz"

tar xzvf apache-maven-3.5.3-bin.tar.gz

export MVN_HOME=/opt/apache-maven mv apache-maven-3.5.3/ $MVN_HOME

rm -f apache-maven-3.5.3-bin.tar.gz

# Install Flume

echo "Install Flume" curl --remote-name --location --insecure --silent --show-error "http://archive.apache.org/dist/flume/1.4.0/apache-flume-1.4.0-bin.tar.gz"

tar zxf apache-flume-1.4.0-bin.tar.gz

export FLUME_HOME=/usr/cygnus mv apache-flume-1.4.0-bin $FLUME_HOME

CEF Context Broker Conformance Testing - SOD - v1.0 page. 28

rm -f apache-flume-1.4.0-bin.tar.gz

mkdir -p $FLUME_HOME/plugins.d/cygnus

mkdir -p $FLUME_HOME/plugins.d/cygnus/lib

mkdir -p $FLUME_HOME/plugins.d/cygnus/libext

# Downlaod Cygnus

echo "Downlaod Cygnus" export CYGNUS_HOME=/opt/fiware-cygnus git clone https://github.com/telefonicaid/fiware-cygnus.git $CYGNUS_HOME

cd $CYGNUS_HOME git checkout release/1.9.0

# Cygnus-common

echo "Compile Cygnus-common" cd $CYGNUS_HOME/cygnus-common $MVN_HOME/bin/mvn clean compile exec:exec assembly:single

cp target/cygnus-common-1.9.0-jar-with-dependencies.jar $FLUME_HOME/plugins.d/cygnus/libext/

$MVN_HOME/bin/mvn install:install-file -Dfile=$FLUME_HOME/plugins.d/cygnus/libext/cygnus-common-1.9.0-jar-with-dep endencies.jar -DgroupId=com.telefonica.iot -DartifactId=cygnus-common -Dversion=1.9.0 -Dpackaging=jar -DgeneratePom=false

# Cygnus-ngsi

echo "Compile Cygnus-ngsi" cd $CYGNUS_HOME/cygnus-ngsi ${MVN_HOME}/bin/mvn clean compile exec:exec assembly:single

cp target/cygnus-ngsi-1.9.0-jar-with-dependencies.jar $FLUME_HOME/plugins.d/cygnus/lib/

# Install Cygnus Application script

echo "Install Cygnus Application script"

CEF Context Broker Conformance Testing - SOD - v1.0 page. 29

cp $CYGNUS_HOME/cygnus-common/target/classes/cygnus-flume-ng $FLUME_HOME/bin/

chmod +x $FLUME_HOME/bin/cygnus-flume-ng

echo "Add cygnus user" adduser cygnus

chown -R cygnus:cygnus $FLUME_HOME

# Create Cygnus log folder

echo "Create Cygnus log folder" mkdir /var/log/cygnus

chown cygnus:cygnus /var/log/cygnus

# Cleanup to thin the final image

echo "Cleanup to thin the final image" cd $CYGNUS_HOME/cygnus-common ${MVN_HOME}/bin/mvn clean

cd $CYGNUS_HOME/cygnus-ngsi ${MVN_HOME}/bin/mvn clean

# Copy files

echo "Copy files" cp $CYGNUS_HOME/cygnus-common/conf/*.* $FLUME_HOME/conf/ cp $CYGNUS_HOME/cygnus-ngsi/conf/*.* $FLUME_HOME/conf/ # Configure files

echo "Rename template files" cd $FLUME_HOME/conf cp flume-env.sh.template flume-env.sh

cp grouping_rules.conf.template grouping_rules.conf

cp name_mappings.conf.template name_mappings.conf

cp krb5.conf.template krb5.conf

CEF Context Broker Conformance Testing - SOD - v1.0 page. 30

cp -ru log4j.properties.template log4j.properties

cp agent_ngsi.conf.template agent_ngsi_api.conf

cp cygnus_instance.conf.template cygnus_instance_api.conf

7.2 Cygnus start.sh file

#!/bin/bash

echo "Start Cygnus"

export FLUME_HOME=/usr/cygnus

$FLUME_HOME/bin/cygnus-flume-ng agent --conf $FLUME_HOME/conf -f $FLUME_HOME/conf/agent_ngsi_api.conf -n cygnus-ngsi -p 5080 -Dflume.root.logger=INFO,console,LOGFILE -Duser.timezone=UTC

CEF Context Broker Conformance Testing - SOD - v1.0 page. 31

CEF​ ​Context Broker Conformance Testing

CEF Context Broker Conformance Testing