RESTful CIM using the Open Data Protocol
Alan McMorran B.Eng Ph.D
OGOpen Grid Systems !1 CIM Exchanges
!2 OGOpen Grid Systems CIM Data Exchange
• The exchange of data using CIM has traditionally been used in two primary ways: • Full Model exchange as CIM RDF XML • Message based exchange as CIM XSD • CIM RDF XML and CIM XSD cover two common use-cases: • CIM RDF XML supports Network Model Management (NMM) exchanges both within and between companies and organisations • CIM XSD exchanges support intra-system exchanges using an Enterprise Service Bus and/or web services
OGOpen Grid Systems !3 www.opengrid.com Hierarchical vs Graph
OGOpen Grid Systems !4 www.opengrid.com Hierarchical vs Graph
• RDF is very well suited to non-directed graph data • XSD is better suited for hierarchical data • Electrical network data rarely fits neatly into a hierarchy
OGOpen Grid Systems !5 www.opengrid.com CIM XSD Standards
• The IEC 61968 series of standards is primarily focussed on defining CIM XSD based messages • These messages are explicitly defined hierarchies that can be used by Web Services • Examples include: • IEC 61968-3 Network Operations • IEC 61968-6 Maintenance & Construction • IEC 61968-9 Meter Reading and Control
OGOpen Grid Systems !6 www.opengrid.com Message Payloads
• These standards define fixed message payloads of hierarchical CIM data • These payloads are defined within the standard and implemented by vendors • They are combined with standard verbs (CREATE, UPDATE, DELETE etc.)
OGOpen Grid Systems !7 www.opengrid.com Services
• This means that every payload must be defined and agreed between all parties • This approach has scalability issues • As more parties connect; more services are interconnected; and more CIM data is being shared a more flexible approach is needed • Systems should be capable of sharing CIM data without requiring a full system export or all parties to agree a fixed subset and hierarchy for every possible service
OGOpen Grid Systems !8 www.opengrid.com RESTful Services
!9 OGOpen Grid Systems What is RESTful?
• “Representational State Transfer (REST) is an architectural style that defines a set of constraints and properties based on HyperText Transfer Protocol (HTTP)" • “REST-compliant web services allow the requesting systems to access and manipulate textual representations of web resources by using a uniform and predefined set of stateless operations”
— Wikipedia
OGOpen Grid Systems !10 www.opengrid.com RESTful Principles • All resources are uniquely addressable, usually through Uniform Resource Identifiers (URI) • All resources can be manipulated through a constrained set of well-known actions, usually CRUD (Create, Read, Update, Delete) • The data is transferred through any of a constrained number of well-known representations (e.g. HTML, XML, JSON) • The communications between client and application are stateless allowing for multiple layers of intermediaries, caching etc.
OGOpen Grid Systems !11 www.opengrid.com RESTFUL Example • Read (HTTP GET) HTTP GET https://opengrid.com/myservice/001
{ Result "type":"example", "name":"get", "id": "001", "company":"Open Grid Systems", "web":"https://www.opengrid.com" }
OGOpen Grid Systems !12 www.opengrid.com RESTful Example
• Create (HTTP POST) HTTP POST https://opengrid.com/myservice
{ Payload "type":"example", "name":"post", "company":"Statnett", “web":"http://www.statnett.no" }
{ Result "id": "002", }
OGOpen Grid Systems !13 www.opengrid.com RESTful Example
• Update (HTTP POST) HTTP POST https://opengrid.com/myservice/002
{ Payload “web”:”http://www.statnett.no/en/" }
Status: HTTP 204 No Content Result
OGOpen Grid Systems !14 www.opengrid.com RESTful Example
• Delete (HTTP DELETE) HTTP DELETE https://opengrid.com/myservice/002
Status: HTTP 204 No Content Result
OGOpen Grid Systems !15 www.opengrid.com RESTful Benefits
• All communications go through a single uniform interface • With HTTP protocols and JSON/XML payloads the services can be accessed by any modern language • Simple, easy to use architecture • Web Browsers use RESTful services over HTTP • Standard verbs, response codes, data types etc. • Flexible service endpoint structures
OGOpen Grid Systems !16 www.opengrid.com Challenges
• This flexibility allows for different but valid URI patterns: https://opengrid.com/myservice/002 https://opengrid.com/myservice?id=002 https://opengrid.com/myservice(id='002') • All valid URI structures • The payload formats are commonly JSON or XML but there is no restrictions • The contents of the payloads can follow any structure • For interoperability, these must be restricted and standardised
OGOpen Grid Systems !17 www.opengrid.com OData
!18 OGOpen Grid Systems OData Background • The Open Data Protocol (OData) is a protocol standard that defines a set of best practices for building and consuming RESTful APIs • OData was initiated by Microsoft in 2007 and since version 4.0 has been under the OASIS OData Technical Committee • This was submitted to the ISO/IEC as an international standard • Allows the creation and consumption of REST APIs using meta-data driven URLs and and standard HTTP protocols and messages
OGOpen Grid Systems !19 www.opengrid.com OData Enhancements • Standard URI structure for RESTful service endpoints • Standard Query format • Supports client-selectable JSON or XML formats for payloads/results • Supported by existing BI and office applications (including Excel and Sharepoint) • OData libraries for .NET, Java, Javascript, PHP, C++ (but can be access as plain HTTP RESTful service without a library in any language that supports HTTP) • Fully meta-data driven with an Entity Data Model (EDM)
OGOpen Grid Systems !20 www.opengrid.com Entity Data Model • Standard UML components map one-to-one to the Entity Data Model UML EDM
Package Schema
Class EntityType
SuperClass BaseClass
Attribute Property
Association NavigationProperty
• Full support for data typing (e.g. String, Boolean, Floating Point, Integer) that map to XSD types
OGOpen Grid Systems !21 www.opengrid.com RESTful with MetaData
• This provides a fully meta-model driven RESTful endpoint for managing data • The CIM can be fully mapped into the EDM including all inheritance, associations, attributes and data types
OGOpen Grid Systems !22 www.opengrid.com Extended Payloads • Synchronous Machine as plain JSON { "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "operatingMode":1, "type":1, } • Synchronous Machine as JSON with extended meta-data { "@odata.context":"http://localhost:8080/cimphony/concert/db/ resources.svc$metadata#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine/$entity", "@odata.type":"#Cimphony.Concert.CIM16A.IEC61970.Base.Wires.SynchronousMachine", "@odata.id":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/ SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "@odata.editLink":"ConcertResource(ID='6f4c8710-6995-11e8-b5cf-96b18064619b')/Contents/Cimphony.Concert.Resource.CIM16A/ SynchronousMachine('_f4852cc6-ca12-11e7-b499-4ee79879b30e')", "UUID":"_f4852cc6-ca12-11e7-b499-4ee79879b30e", "name":"Turner V1 SM", "p":0.0, "q":0.8, "maxQ":50.0, "minQ":-17.0, "[email protected]":"#Int32", "operatingMode":1, "[email protected]":"#Int32", "type":1, }
OGOpen Grid Systems !23 www.opengrid.com Meta-Data Defined Services
• The EDM is used by OData aware clients and servers to validate communications • Clients can request the EDM from the endpoint in a standard format to perform auto-discovery and client- side validation • Servers will validate all endpoint URIs and service payloads against the EDM • This allows for meta-data driven validation of data at the interface before it reaches the application itself
OGOpen Grid Systems !24 www.opengrid.com OData Example • OData abstracts the interface from a particular format so can provide XML or JSON representations • Endpoints can also be configured to returning other formats e.g. SVG, PNG or RDF XML through an OData interface • The standard RESTful URL structure is all based on the meta-model: http(s)://
OGOpen Grid Systems !25 www.opengrid.com { "@odata.context":"http://localhost:8080/cimphony/concert/db/management.svc$metadata#ConcertResource", "value":[ http(s)://
], "DiagramObjects":[
], "ModelingAuthoritySet":null, "Substations":[ { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Substation('_58F637D8B03A4B12A67DF2E5797F9B6A')", "UUID":"_58F637D8B03A4B12A67DF2E5797F9B6A", "mRID":null, "aliasName":null, "name":"CROSSPLAINS" }, { "@odata.type":"#Cimphony.Concert.CIM15.IEC61970.Core.Substation", "@odata.id":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "@odata.editLink":"ConcertResource(ID='d8815f90-41db-11e7-b565-96b6cd723f39')/Contents/ Cimphony.Concert.Resource.CIM15/Substation('_7324D6723635494784A4D8A9578FCE8A')", "UUID":"_7324D6723635494784A4D8A9578FCE8A", !29 "mRID":null, "aliasName":null, OData in Action for DERMS
!30 OGOpen Grid Systems DERMS Project • Open Grid Systems is providing the CIM data management repository for an ongoing DERMS innovation project at a large UK distribution company • Multiple internal and external systems all provide and consume CIM data (including real-time data)
OGOpen Grid Systems !31 www.opengrid.com CIM Data • The distribution network model from the DMS • Geographical data integrated with the DMS model • DER market data including availability/pricing • TSO market queries and requests • Outputs from market/system simulation engines • Real-time SCADA data
OGOpen Grid Systems !31 www.opengrid.com Expanding Interfaces
• The number of requested interfaces is constantly growing • Requests for access to different streams of data based on application requirements • The CIM repository provides an OData interface to the static as-built network data and the real-time data • Role Based Access Control to allow different users to view/update different sets of data • OData allows most data access requests to be met with the generic service
OGOpen Grid Systems !32 www.opengrid.com Example Queries
• “We need a list of the current real and reactive power set points for all the DER on the network” • The CIM class is PowerElectronicsConnection • Only need the p and q attributes values for each http(s)://
OGOpen Grid Systems !33 www.opengrid.com Example Queries
• “Oh but can we only get DER that are solar?” • We need to check the PowerElectronicsUnit association of the device is of type PhotoVoltaicUnit http(s)://
OGOpen Grid Systems !34 www.opengrid.com Example Queries • “We need an endpoint to return all the SCADA measurement points attached to circuit breakers that are monitoring the switch status” • The CIM class is Discrete (subclass of Measurement) • Need all instances that associate with a Breaker via the PowerSystemResource association and have a measurementType of SwitchStatus http(s)://
OGOpen Grid Systems !35 www.opengrid.com Services as Queries
• The same approach is used for updates • Applications can run a CIM query for specific CIM object(s) then use the returned data to run subsequent queries to update the data (or run more queries) • e.g. Updating a SCADA measurement involves querying for the CIM Analog with the desire ICCP name • The query includes an instruction to also include the associated AnalogValue ($expand=AnalogValue) • A subsequent update changes the value of the AnalogValue with the returned ID
OGOpen Grid Systems !36 www.opengrid.com Conclusion
!37 OGOpen Grid Systems Summary • Increasing numbers of systems, applications, and services want to access and update CIM data • A generic, platform-independent, metadata-driven service provides flexibility for users to use CIM data • OData solves many of the problems by defining standard best practices and libraries for RESTful HTTP services • Existing support in common applications, programming languages and platforms • Can still be used as a light-weight plain RESTful service for clients that are not OData-aware • A single endpoint to support multiple services
OGOpen Grid Systems !38 www.opengrid.com Questions
www.opengrid.com
OGOpen Grid Systems !39