USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
UNITED STATES AGENCY FOR INTERNATIONAL DEVELOPMENT
Energy Investment Activity (USAID EIA)
Contract Number AID-168-C-14-00002
Guidelines for Electronic Data Interchange
Author: EIA Project Team
January, 2017
Implemented by: Advanced Engineering Associates International, Inc. (AEAI) USAID COR: Ankica Gavrilovic
This document is made possible by support from the American People sponsored by United States Agency for International Development (USAID). The contents of this document were prepared by and are the sole responsibility of Advanced Engineering Associates International, Inc., and do not necessarily reflect the views of USAID or the United States Government.
Contract Number AID-168-C-14-00002 1
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Table of Contents
Abbreviations...... 3 1. Preface ...... 5 2. General project approach for BiH ...... 6 3. Objectives of this report ...... 7 4. EDI Framework ...... 8 4.1 Introduction ...... 8 4.2 Identification of roles and domains ...... 9 4.2.1 DSO Identification directly issued by FERC, RSERC and SERC ...... 9 4.2.2 Supplier Identification ...... 11 4.2.3 Metering point Identification ...... 12 4.2.4 BRP Identifications ...... 14 4.2.5 Check digit ...... 14 4.3 Communication means and file names ...... 14 4.3.1 File Transfer Protocol (FTP) ...... 14 4.3.2 Web services ...... 14 4.3.3 Simple Mail Transfer Protocol (SMTP) ...... 14 4.3.4 Recommendation ...... 15 4.3.5 File names ...... 15 4.4 Data exchange ...... 15 4.4.1 Acknowledgement and error messages ...... 17 4.4.2 Message types ...... 17 4.4.3 Message time, date and time period ...... 17 4.4.4 Message structure and content...... 18 4.5 DSO Metering Point Database ...... 23 4.5.1 Brief functional description of the database ...... 23 5. Roadmap for development of EDI in BiH ...... 26
Contract Number AID-168-C-14-00002 2
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Abbreviations
BiH Bosnia and Herzegovina
BRP Balance Responsible Party
CET Central European Time
CEST Central European Summer Time
CSV Comma-Separated Values
DSO Distribution System Operator
EBIX European forum for energy Business Information eXchange
EDI Electronic Data Interchange
EIC Energy Identification Coding
ENTSO-E European Network of Transmission System Operators
EIA Energy Investment Activity
F-GCS General Conditions for Electricity Supply
FERC Regulatory Commission for Energy in the Federation of Bosnia and Herzegovina
FTP File Transfer Protocol
FTPS FTP Secure
GIAI Global Individual Asset Identifier
GLN Global Location Number
GSRN Global Service Relation Number
ID Identifier
INVOIC Invoice message
ISO Independent System Operator
IT Information Technology
LIO Local issuing office kW Kilowatt kWh Kilowatt Hour
Contract Number AID-168-C-14-00002 3
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
MIME Multipurpose Internet Mail Extensions
MSCONS Metered Services Consumption Report
MWG Market Working Group
OBIS OBject Identification System
R-GCS General Conditions for Delivery and Supply of Electricity
RSERC Republic of Srpska Regulatory Commission for Energy
SERC State Electricity Regulatory Commission
SFTP Secure (or SSH) File Transfer Protocol
SMTP Simple Mail Transfer Protocol
USAID U.S. Agency for International Development
UN/EDIFACT United Nations Electronic Data Interchange for Administration, Commerce and Transport
UTC Coordinated Universal Time
UTILMD Utility master data message
XML EXtensible Markup Language
Contract Number AID-168-C-14-00002 4
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
1. Preface
USAID’s Energy Investment Activity project is supporting Bosnia and Herzegovina (BiH) in its efforts to increase investments in the energy sector and to advance the accession process related to energy sector requirements. In particular, the Energy Investment Activity project aims to address impediments to increased investments in the energy sector, to address retail market deficiencies, to achieve energy savings in BiH using regulatory incentives and to advance EU accession requirements in the energy sector.
In order to assist the electricity sector in BiH to make further necessary steps toward the establishment of a fully transparent retail electricity market, USAID funded the technical assistance project, Energy Investment Activity (EIA), with component 2 “Address retail Market Deficiencies in BiH.” EIA proposed the constitution of a Market Working Group (MWG) and pertinent subgroups:
• Subgroup 1 – deals with activities related to development of DSO rules and processes • Subgroup 2 – deals with activities related to the development of Electronic Data Interchange
Assistance to work of Subgroup 1 resulted in the report “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator.” This report dealt with development of an information model and outline of business processes linked with the role of the DSO as independent market facilitator; the report is expected to serve as a guideline for development of the rulebook1 and other legal documents that should cover business processes associated with DSO market roles.
In continuation of the work, as a result of assistance to the efforts of Subgroup 2, this report – Guidelines for Electronic Data Interchange (hereinafter EDI) report has been prepared. EDI can be described as computer to computer exchange of business documents in standard electronic format within a business process between different parties. Using as an input international best practices from one side and a developed information model and business processes2 from the other side, EDI infrastructure is reviewed and recommendations specified.
This report considers the actual situation in BiH3 related to existing EDI between key electricity retail market players. As a result, the report suggests options and specifies recommendations that should facilitate development of technical standards for EDI. The review and analysis EDI for the traditional role of the DSO (for example: network operation, maintenance, development, etc.) is not a subject of this document, as these processes are already well covered with existing material.
1 This rulebook will be developed by the working group and shall contain rules with business processes that relate to DSO market roles. 2 Based on “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator” document. 3 Key interested parties in retail market have unanimous opinion that development of EDI and related IT systems is required immediately.
Contract Number AID-168-C-14-00002 5
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
2. General project approach for BiH
The main objective of Subgroup 2 is development of standards for electronic data interchange (EDI) in the retail market, which implies a sequence of messages between market participants (in our case mainly DSO and Supplier, but also the BRP and ISO4), either of whom may serve as the originator or recipient. Implementation of these standards should enable the transmission of messages (formatted data messages, representing the documents) from sender to recipient via telecommunications facilities with a high degree of automation.5
The automation enables efficient processing of a large amount of data with minimal human interaction. Human intervention in the processing of received messages is typically intended only for error conditions, for quality review, and in special situations.
As a precondition for the development of the technical standards, the work of Subgroup 1 related to an information model and business processes development needed to be finished and information model and business processes defined. An information model, as a common language for all participants, shall be used for clear and unambiguous definition of EDI guidelines. These guidelines shall take into consideration business processes.
The impact of the current situation in EDI at key retail market players shall be assessed and taken into consideration.
Information Business model processes
International Current EDI best practice situation EDI Guidelines
Figure 1 Influence on development of EDI Guidelines
4 Regardless the fact that ISO is not a part of retail electricity market, there is significant data exchange between the ISO and DSO related to imbalance settlement process. 5 Technology that provides automation includes computers and other communications devices that can gather, store, manipulate, prepare and distribute data.
Contract Number AID-168-C-14-00002 6
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Options for issues of particular interest shall be specified with elaborated recommendation based on a best practice and current situation. Guidelines for technical standards shall provide guiding principles and recommendations for messages/file formats and content, communication means/infrastructure, security measures, etc.
3. Objectives of this report
As full retail market opening6 occurred in BiH on the 1st of January 2015, the issue of establishment of the DSOs as independent retail market facilitators with a suitable infrastructure for data exchange becomes urgent.
The objective of this report is to provide support to the work of the MWG and Subgroup 2 in Development of technical standards for data exchange in the market by specifying these Guidelines for EDI. The support is necessary for the setup of the EDI in a competitive market.
Based on the Guidelines for EDI, the MWG and Subgroup 2 will be able to develop a detailed and sound set of rules for EDI, which comply with today’s best practices and are accommodated to the situation in BiH. The mentioned set of rules will additionally improve transparency and contribute to the efficiency of electricity retail market in BiH.
As Guidelines for EDI are based, inter alia, on the current situation in BiH, review and evaluation of current practices shall be included in document. This development will, as expected, have as an additional result capacity strengthening of the MWG and Subgroup 2 participants in the EDI domain.
6 Introduction of free choice of supplier for all customers.
Contract Number AID-168-C-14-00002 7
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4. EDI Framework
4.1 Introduction Quick, reliable and secure data exchange is a priority in modern electricity markets. Business processes that are implemented in retail electricity markets are usually composed of a number of steps, of which the majority falls into the category of data exchange. One party sends data, and another party (or parties) receives data. Therefore, the definition of Electronic Data Interchange (EDI) framework for data exchange of the relevant business services (existing and missing) is of utter importance. Such framework shall be based on a modern IT infrastructure for data exchange, as well as on the definition of messages (and their structures, contents and formats) that will be exchanged. Content of presented EDI framework shall constitute key elements of future EDI Standard.
Additionally, it is necessary to specify a simple and efficient pattern/system for the identification of roles (actors, market participants) and domains7 (metering points) in the electricity market. By assigning unique and unambiguous identifiers to each object (market participant, metering point, and the like), the exchanged messages coming to the “proper” addresses, related to the “proper” objects is assured.
The transfer of messages is usually realized using standardized Internet protocols. There are several typical solutions that will be presented in this document. The right solution for communication protocols depends on the parties involved in the business process data exchange. The structure and content of messages will be explained and illustrated in this chapter.
Although not directly linked, an adequate database (metering, market participant) has significant influence on the usability of EDI. A database is the collection of data that is organized in a suitable way that enables quick data access and management. An example of a logical database table structure will be presented in this chapter as well.
The general recommendations for development of the EDI framework are:
• Delivery of proxies -To enable a highly automated process, the delivery of proxies (physical copy, etc.) should be avoided as a rule, and the existence of proxies should be guaranteed in a contract. Only in justified individual cases, transmission of proxies is required. • Deadlines - In case of a requirement of an authorization or declaration for which the requesting party has deadlines, this cannot be in question in the process. The requesting party may only cancel the running process if the requested party does not respond immediately. • Communication between Supplier and Customer - The communication between supplier and customer is not the primary subject of the processes, and no guidelines regarding EDI will be developed / described for it. This communication may be freely negotiated (according the legislation).
7 Roles and domains are already defined as a part of information model in “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator” document.
Contract Number AID-168-C-14-00002 8
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4.2 Identification of roles and domains The unique identification of market participants (roles) and domains (e.g., metering point) in the electricity market is crucial for market functioning. Without the possibility to uniquely and unambiguously identify the DSO, Suppliers or metering points, data exchanged within business process between sender and receiver would be unrecognizable and automation not possible. Since there is no uniform European standard8 for identification, we suggest the assignment of identifiers as described below.
Pease note that identification is related to a role. This implies, for example, if during a transitional period the DSO and the Supplier are in one company (not unbundled), nevertheless they need to have two IDs – one for the role as DSO and one for the role as Supplier.
In the following, we will consider identifications that need to be unique during their life time and centrally assigned. Other identifications that may change several times (e. g., customer ID) are not part of these considerations. Once assigned, the DSO identification may not be changed during the validity period of the license. Options for identification schemes are considered in the continuation of this chapter.
4.2.1 DSO Identification directly issued by FERC, RSERC and SERC
Each DSO must have a unique Identification. Since the DSO must be licensed, and the regulators (FERC, RSERC and SERC) are issuing licenses, they can be responsible for issuing a unique DSO identification also.
4.2.1.1 Custom identification scheme
Using a custom identification scheme, the DSO ID can be assigned as follows:
• Digit 1 and 2 contain the country identification (fix “BA”) • Digit 3 contains a “D” for DSO • Digit 4 to 8 contain a consecutive number (filled with zero from the left)
Example: Country Consecutive No.
B A D 0 0 1 2 3
Digits 1 2 3 4 5 6 7 8
8 Further recommendations for identification, see ebIX “Recommended identification schemes for the European energy industry” http://www.ebix.org/content.aspx?ContentId=990&SelectedMenu=3.
Contract Number AID-168-C-14-00002 9
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4.2.1.2 GS1-GLN identification scheme
The GS1 system9 provides standard numbering structures for different applications, which guarantees worldwide uniqueness within the relevant area of application. The main idea is that the identification of parties and locations are unique and as stable as possible over time. This system encompasses a number of codes that are used for different purposes.
Parties in the electricity market (and the DSO as well) are identified via Global Location Number (GLN) codes. These codes are composed of thirteen (13) digits:
• GS1 company prefix – has variable size and is administered by GS1. The first three (3) digits is the GS1 prefix of national organization (which is for BiH 387) • Location reference – has variable size and is assigned by company. Usually, it is a sequential number. • Last digit – thirteenth digit is the check digit and is an integral part of the location number.
Example 3 8 7 1 2 3 4 5 6 7 8 9 1
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3
In total, the GS1 company prefix and Location reference together have 12 digits. In the example above, the GS1 company prefix is presented in blue, the location reference in orange, and the check digit in grey. Companies may apply for different “capacities” (containing 10, 100, 1000 or more unique codes), depending on their needs and requirements.
4.2.1.3 EIC identification scheme
An alternative assignment of the DSO ID can be done according the energy identification coding scheme (EIC) issued by ENTSO E, consisting of:
• 2 Digits: Local issuing office (LIO) number, for BiH the ISO is registered as LIO for BiH by ENTSO E (the LIO number 36 was assigned to NOS BiH). • 1 Digit: Code type, in this case EIC object type X (Party). • 12 digits: uppercase characters or minus signs allocated by the LIO in compliance with general and local rules to identify the object. • 1 digit: check character based on the 15 previous characters used to ensure the validity of the EIC code. The check digit algorithm is described in the EIC implementation guide document.
9 Global Standards http://www.gs1.org/.
Contract Number AID-168-C-14-00002 10
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Example 3 6 X D ------0 0 1 2 3 7
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
4.2.1.4 Recommendation
As DSOs in BiH do not have assigned identifications, our recommendation is to assign EIC codes for them.
4.2.2 Supplier Identification
Each Supplier must have a unique Identification. Since Suppliers must be licensed, and the regulators (FERC, RSERC and SERC) are issuing these licenses, we suggest that they be responsible for issuing a unique Supplier identification also.
Once assigned, the supplier identification may not be changed during validity of the license. Also, this identification should not change if the supplier company merges or splits.
4.2.2.1 Custom identification scheme
Using a custom identification scheme, the Supplier ID could be assigned as follows:
• Digit 1 and 2 contain the country identification (fix “BA”) • Digit 3 contains a “S” for supplier • Digit 4 to 8 contain a consecutive number (filled with zero from the left)
Example: Country Consecutive No.
B A S 0 0 1 2 3
Digits 1 2 3 4 5 6 7 8
4.2.2.2 GS1-GLN identification scheme
The same identifier scheme which is used for a DSO - Global Location Number (GLN) - could be applied for identification of the suppliers that are participating in the electricity market. As described above, the code is composed of 13-digit number with last digit in the structure check digit.
4.2.2.3 EIC identification scheme
Again, an alternative assignment of the Supplier ID can be done according the energy identification coding scheme (EIC).
Contract Number AID-168-C-14-00002 11
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Example 3 6 X S ------0 0 1 2 3 6
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
The ISO in BiH has already issued a number of EIC codes,10 and it administers a registry of market participants and balance responsible parties.
4.2.2.4 Recommendation
Numerous EIC-X11 codes have been already issued by ISO for participants in the wholesale electricity market. This assignment is also part of the Market Rules requirements related to registration of market participants at the ISO. Since this scheme is already successfully implemented, we suggest its extension to participants in the retail market, under responsibility of authorized regulators.
Regulators could coordinate implementation of this activity with the ISO.
4.2.3 Metering point Identification
Each Metering point must have a unique Identification. Since the DSO is responsible for a building connection point and supply point and for metering at the metering point, also the DSO can be responsible12 for assigning a unique metering point ID to a metering point.
Once assigned, the Metering point identification may not be changed during lifetime of the metering point (normally equal to the lifetime of the building).
4.2.3.1 Custom identification scheme
The Metering Point ID could be assigned as follows:
• Digit 1 to 8 contain the DSO identification • Digit 9 to 28 contain a 20-digit consecutive number
Exampl DSO identification Consecutive number e:
B A D 0 0 1 2 3 0 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
10 ISO registry is available at http://www.nosbih.ba/en/partneri/registers/46. 11 EIC codes for market participants. 12 FERC Article 61 (1).
Contract Number AID-168-C-14-00002 12
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4.2.3.2 GS1-GSRN identification scheme
For identification of metering (active, passive and virtual) and accounting points, a Global Service Relation Number (GSRN) may be used, as it provides a unique and unambiguous identification number. A GSRN code13 is defined with the code that has 18 digits:
• GS1 company prefix – has a variable size and is administered by GS1. First three (3) digits is the GS1 prefix of a national organization (which is for BiH 387). • Service reference – has a variable size. • Last digit – eighteenth digit is the check digit and is integral part of location number.
Example 1 7 3 5 0 0 5 3 8 5 0 0 0 0 0 0 1 6
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8
In total, the GS1 company prefix and Service reference together have 17 digits. The GSRN can be encoded in a barcode.
4.2.3.3 EIC identification scheme
Again, an alternative assignment of the metering point ID can be done according to the energy identification coding scheme (EIC).
In this case the code type “Z” will be used.
Example 3 6 Z 1 2 3 - 1 2 3 4 5 6 7 8 2
Digits 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6
4.2.3.4 Recommendation
For metering points, each DSO already has an existing (one, or even more) identification scheme, which may be different from the identification schemes adopted by other DSOs. DSOs in BiH should have unique and unified identification scheme on the country level. It is recommended that DSOs solve this issue (not in the scope of this project), together as participants of a responsible workgroup.
13 For identification of meters as physical equipment (i.e., not as metering points) in the network, another identifier Global Individual Asset Identifier (GIAI) may be used. It may be up to 30 characters long, encompassing alphanumeric characters.
Contract Number AID-168-C-14-00002 13
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4.2.4 BRP Identifications
This process is already in place in BiH. The ISO, as the local issuing office, issues a unique identifier for BRP, using an EIC identification scheme. The same EIC-X code which is used for a particular market participant is also used for a corresponding BRP if it accepts balance responsibility for a particular balance group.
4.2.5 Check digit
A checksum is a small-sized datum from a block of digital data for the purpose of detecting errors which may have been introduced during its transmission or storage. Check digits are special cases of checksums.
If the IDs are not assigned according to EIC or GS1, it must be decided whether the ID’s described above should have a check digit (created by means of a common checksum algorithm) as a last digit or not.
4.3 Communication means and file names There are many options for the exchange of messages between market participants. Internet-based standards, allowing automation of message transmission, receipt, validation and data storage, are used. For security reasons, it is recommended to use data encryption14.
4.3.1 File Transfer Protocol (FTP)
A standard network protocol is used to transfer files between market participants via a computer network. FTP is widespread and reliable; but since data traffic is not encrypted, this protocol is today considered unsecure. FTP derivatives (SFTP and FTPS) which encrypt passwords and sensitive information from being transmitted openly over the network are popular for this purpose.
4.3.2 Web services
There is an open XML standard for data interchange from application to application over the Internet. This approach allows market participants to exchange data using web services. When a message is sent to the web service, its syntax is validated and the user is informed if the message is valid and correctly received (accepted by the IT system). Messages are received by calling the web service, which then returns a message indicating it is ready for transmission. Access to web services is commonly subject to strict security rules.
4.3.3 Simple Mail Transfer Protocol (SMTP)
For the transmission of messages SMTP (email) could be used efficiently. When this protocol is used, the message must be in the Multipurpose Internet Mail Extensions (MIME) format, which extends SMTP- enabling additional functionality. The actual message must be sent as an attached file, not as a part of the body text. There will only be one attachment in one SMTP-exchange, the attachment containing only one message type.
14 Encryption is the process of encoding messages or information in such a way that only authorized parties can read it.
Contract Number AID-168-C-14-00002 14
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
The usage of SMTP is illustrated by the following Figure 2. The Sender (computer) prepares and sends an e- mail message with the address, subject, body and attached file in “proper” format. The e-mail message is received at the destination by Receiver (computer), which validates the e-mail message and attached files and data, and performs further processing.15
Data file
Sender Receiver E-mail message
Figure 2. E-mail message with attached data file, sent from one market participant to other
All described activities can be achieved in an automated manner.
4.3.4 Recommendation
SMTP could be used efficiently for the transmission of messages in the first set of rules, for the following reasons:
• Simple to use • Straightforward and easy implementation • Widespread and available solution • Simple monitoring of messages exchange
4.3.5 File names
The name of the attached file (attached to an email using SMTP) shall be according the following
• Creation date (CCYYMMDDHHMMSS) • Sender (e.g., BAD00123) • Receiver (e.g., BAS00123) • Message type (e.g., Meter Reading)
An example of a complete file name: 20150531181501BAD00123BAS00123MeterReading. Such file names may be automatically generated and parsed by IT systems involved in data exchange, and they are human- readable.
4.4 Data exchange When data is interchanged between different parties via telecommunication - transmission methods, a “common language" will be used with an agreed mode of expressing it, i.e., common protocols, message identification, agreed abbreviations or codes, etc. Besides using compatible systems, interchange partners
15 For example, enters data into a database or sends acknowledgement or confirmation messages.
Contract Number AID-168-C-14-00002 15
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
should follow uniform rules with respect to communication procedures that include the types of messages acceptable, identification of parties, reference to previously agreed protocols or agreements on character set, language, transliteration, and interchange structure.16
As mentioned before, our proposals reference electronic data exchange between the following retail market partners:
• Supplier • Distribution system operator • Balance responsible party
Data exchange is described in the relevant business processes. The exchange of information between customer and supplier or DSO will be explained in the description of the business processes but is not included in our considerations regarding electronic data exchange.
All existing and future business processes in the BiH retail market (defined and elaborated in the report “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator”) shall be supported by EDI framework. Detailed definition of non-standard messages 17 that are transferred between market participants within these processes should be defined in separate document.
General overview of EDI interchanges within the retail market business processes is presented in the below table:
Business process group Parties in retail market EDI Exchanged data
Customer switch Customers, Suppliers (including Applications, requests, notifications, Supplier of Last Resort and Public certificates, proposals, contracts, metering Supplier), DSOs, BRP data reads…
Connection contract Network users, DSOs Applications, requests, power consent, (initial) power permits, denials, complaints, decisions, contracts, declarations Imbalance settlement DSOs, Suppliers, BRPs, ISO. Aggregated metering data in retail market Table 1 Overview of EDI interchanges
The column “Exchanged data” in the table specifies the types of documents/messages that are exchanged between retail market participants within one business process group.
16 The principles mentioned above led to the development of the United Nations Electronic Data Interchange for Administration, Commerce and Transport (UN/EDIFACT) syntax rules and standard messages. 17 These are messages different from standard messages, which are described in chapter 4.4.4 Message structure and content.
Contract Number AID-168-C-14-00002 16
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
4.4.1 Acknowledgement and error messages
Messages which contain payload data are composed and sent by one market participant and received and accepted (or rejected, if invalid) by another market participant. The flow of these (business) messages may be accompanied by other ancillary messages that contribute to the successful data interchange and facilitate error handling:
• Acknowledgement18 or confirmation messages are used for tracking of the main (business) messages in an end-to-end delivery process. They are sent by receiver in order to confirm that message is received and/or validated and accepted by the system. • Error or rejection messages are used to assist a sender and receiver in case of error. They are sent by the receiver when error is registered. These messages contain information that should assist the sender to identify and remove error.
The implementation of acknowledgement and error messages is not mandatory for the functioning of EDI, but is recommended.
4.4.2 Message types
The following messages will be exchanged between the market partners:
Type Content EDIFACT equivalent
Meter Reading Contains the meter readings, i.e., single meter reading or MSCONS19 load profiles.
Master Data Contains master data for the customer and contract UTILMD20 metering point, e.g., begin supply, end supply.
Invoice Data Shall contain the data for billing grid fees. INVOIC21
Table 2 Message Types
4.4.3 Message time, date and time period
Local time in BiH is depends on the season and may be either standard or daylight savings time.
18 These messages are used for identification if sent message is successfully delivered to the recipient and/or if message is successfully verified and accepted by the system. 19 MSCONS Metered Services Consumption Report. 20 UTILMD Utility master data message. 21 INVOIC invoice message.
Contract Number AID-168-C-14-00002 17
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
• Standard time in BiH is Central European Time (CET), which it is one hour ahead UTC22 (i.e., UTC+1, where +1 indicates time offset in hours from UTC), and it lasts from the last Sunday in October to the last Sunday in March of the next year. • Daylight saving time in BiH is Central European Summer Time (CEST), which is two hours ahead UTC (i.e. UTC+2), and it lasts from the last Sunday in March till the last Sunday in October of the same year.
This time transition complicates the time presentation in messages, as the last Sunday in March has 23 and the last Sunday in October 25 hours. All other days in year have 24 hours. IT systems that are handling messages commonly take these circumstances into account by keeping UTC as their basic time-keeping standard, with the provision of services for automatic time conversion from UTC to local time. This approach provides the exact registration of the time in the exchanged messages, regardless of daylight- saving time changes.
Time Content Number of hours
Standard time (CET) UTC from 23:00 to 23:00 on the following day 24
Last Sunday in March UTC from 23:00 to 22:00 on the following day 23
Daylight savings time (CEST) UTC from 22:00 to 22:00 on the following day 24
Last Sunday in October UTC from 22:00 to 23:00 on the following day 25
Table 3 Calendar days in BiH in UTC Time
It is also important that IT systems that are generating and exchanging messages between themselves are synchronized as much as possible within a certain range (for example +10 seconds) around local time.
Although IT systems may be available for non-stop processing of EDI messages, certain business processes cannot happen in a certain time period. Usually, business days (Monday to Friday), weekends and holidays are defined with strictly limited work time. Depending on the defined limitations, IT systems may (or may not) process the messages, provide IT support, perform error handling, etc.
In case a system is not available due to down time, the involved player should notify affected parties promptly. System maintenance should be performed in a manner so as not to disturb market operations.
4.4.4 Message structure and content
In Europe there are some common message formats based on the EDIFACT (Electronic Data Interchange for Administration, Commerce and Transport) standard. In any case, this standard has been adapted to the needs of the specific region. Same procedure needs to be done for BiH requirements taking into account
22 UTC is the same time as GMT (Greenwich Mean Time).
Contract Number AID-168-C-14-00002 18
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
that in consideration of the actual small number of switching customers in BiH, it may be reasonable to start with a “light” version. This means that instead of using complex EDIFACT standard rules as a basis for the creation of message formats for EDI, it may be just as sufficient to use some simpler file format, such as CSV (Comma-Separated Values) or XML (Extensible Markup Language). A decision on the implemented message format for EDI will have influence on the time and resources for implementation of the necessary software modifications.
The selection of message format may depend on several criteria:
• Message format complexity. Not all message formats that are in use for EDI are of the same complexity. o The CSV file format is the simplest, where data (number and text) are delimited with a comma (“,”) and stored in plain text form. These files are common message types and are supported by various systems and applications. o XML file format is more complex than CSV. It is a language that defines a set of rules for encoding documents into a format readable by machines and humans. As it is cheap and designed for communication via the Internet, today it is quite popular in EDI. o EDIFACT is widely accepted as an international standard. An EDIFACT document is based on strict rules which manage the position of data within a file. The structure of a file is hierarchical with a lot of elements. • Estimated number data exchanges. If it is predicted that a modest number of messages will be exchanged within EDI, it may not be recommended to develop and implement a complex message format such as EDIFACT, as this is not cost-efficient. • In-house (in-country) experience and expertise. The experience with data exchange, in particular the EDI format, may have an impact on the selection.
For the sake of convenience, we will use the names of the EDIFACT equivalents in the subsequent texts.
Messages in general consist of a Message Header and a Message Body. The message header contains information regarding control of the message flow, the processing of the message body and has a unique structure valid for all message types. The message body contains the real application information and is different from one message type to another.
4.4.4.1 Message Header
Information Description Example / Values
Message type Identifies the type of the message (body) UTILMD
Version Actual version of the format description itself V1.1
Contract Number AID-168-C-14-00002 19
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Information Description Example / Values
Sender Identification of the sender of the message DSO ID
Receiver Identification of the receiver of the message Supplier ID
Date Date when the message was created CCYYMMDDHHMMSS
Number Unique number created by the sender of the Serial number message
Response type Code specifying the type of acknowledgment Not Applicable (No code required or transmitted acknowledgement needed)
Table 4 Message Header
Please note: it must be insured that all market participants use the same time base. We suggest using UTC as time base. For BiH this is UTC+1, respectively UTC+2.
4.4.4.2 Message Body: Meter Reading
The meter reading message is a message sent between parties in the power industry, providing consumption and/or associated technical information at locations for products or services, where the supply is metered.
Regarding meter readings, we must distinguish between the following occurrences of meter readings
• A counter reading in kWh monthly drawn from a conventional meter, or • A consumption in kWh calculated as difference of two counter readings, or • A time series containing power values (kW) or consumption (kWh) measured in a period (e.g., every quarter of an hour) as counter readings.
Information Description Example / Values
Metering point Unique ID of metering point BAD0012300000000000123456789 ID
Address Address of the metering point Additionally for clarification
Meter-ID Unique ID of meter Usually the serial number of the meter
Date of meter The time the meter is read 20150531180000 (31st of May 2015) reading (CCYYMMDDHHMMSS)
Begin period of Period for meter reading 20150501000000 (from 1st of May 2015)
Contract Number AID-168-C-14-00002 20
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Information Description Example / Values meter reading (CCYYMMDDHHMMSS)
End period of Period for meter reading 20150531240000 (until 31st of May 2015) meter reading (CCYYMMDDHHMMSS)
Reason Reason for reading the meter COM (change of meter)
IOM (installation of meter)
ROM (removal of meter)
COS (change of supplier)
COB (change of balancing area)
PMR (periodic meter reading)
COT (Intermediate meter reading
Status Status of meter reading SMV (start measure value) e.g., change of meter
EMV (end measure value) e.g., change of meter
MRV (meter reading value) e.g., periodical reading
Product Identifies the measured product OBIS23-Value e. g., 1-b:1.8.0 (active energy Identification import (+))
Quantity Indicates how the quantity has been True Value (relevant for billing) Qualifier measured Estimated Value (relevant for billing)
Preliminary Value (not relevant for billing)
Not usable value
Unit Unit of measurement KWT (kilowatt)
KWH (kilowatt hours)
23 The Object Identification System (OBIS) defines the identification codes (ID-codes) for commonly used data items in electricity metering equipment (IEC 62056-61).
Contract Number AID-168-C-14-00002 21
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Information Description Example / Values
Quality Quality of measurement Single counter reading
Consumption
Number of The number of values transmitted in 1 (for a single reading at residential values this message customer)
96 (a load profile with ¼ hours values)24
Quantity Quantity read from the meter Measured quantity “123460” (with fixed decimal places)
Table 5 Message Body Meter Reading
4.4.4.3 Message body: Master Data
The message can be used for submission of master data regarding metering points
• request for supplier switch (DSO to old supplier) • confirmation of supplier switch (old supplier to DSO) • request for submitting master data (supplier requests of DSO as authorized by the customer)
Information Description Example / Values
Transaction Coding of the reason of this request Request for master data of the metering point
Request for supplier switch
Request for confirmation of the supplier switch
Status The status of the answer to a request Request for supplier switching rejected
Metering point The ID of the metering point on which BAD0012300000000000123456789 ID the request refers. See chapter 5.3
Customer data Data of the customer on which the Name, Address request is related
24 In the Nordic countries UTILTS is used for transmitting time series.
Contract Number AID-168-C-14-00002 22
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Information Description Example / Values
Metering point Data related to the meter point Address, consumption data
Table 6 Message Body: Master Data
4.4.4.4 Message body: Invoice Data
This message is used for the transmission of invoices for services and products. In our case, it is used for invoicing grid fees.
Since billing of grid fees is not yet clarified, the message structure and content shall not be presented within this document.
4.5 DSO Metering Point Database In our analysis we assume that the model for data exchange is a bilateral model. This means that the exchange of information while executing the business processes is direct and bilateral; the market parties directly send one another standardized messages.25
According to its role as market facilitator, the DSO needs to have a database26 containing all metering points, the customers consuming energy at these points and the supplier which is supplying the metering point. In Figure 3 DSO Metering Point Database, there is a schematic diagram of the data model concept.
Keeping a database with this structure enables the DSO to support the business processes (switching, etc.) deliver accurate data (i.e., meter readings) to the authorized parties and to meet its obligations within the legal framework.
4.5.1 Brief functional description of the database
The DSO will register all metering points in its network in the database. Once registered, a metering point may not be modified or deleted during its lifetime. The only reason to change a metering point is when the physical condition changes (e.g., when the building is demolished).
During the lifetime of the metering point, several customers may be linked to the metering point (e.g.m when a customer moves into the building). The link between customer and metering point has always a start date and an end date. The supplier is linked to the customer and the metering point when the customer concludes a contract of supply with the supplier. Again, this link has a start date (commencement
25 The bilateral model is not explicitly stated in the regulations of FERC and RSERC; but a reading of them indicates into this direction. 26 According F-GCS Art. 5 (1), F-GCS Art. 61 (2) and R-GCS Art. 9.
Contract Number AID-168-C-14-00002 23
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
of contract) and an end date (termination of contract). The meter installed at the metering point is linked to the metering point. Whenever a meter is read, the meter reading (valid for a certain period) will be recorded.
Following this concept, the DSO will be able to know the actual status of the metering point and its actual and past relations at any time.
Contract Number AID-168-C-14-00002 24
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Balance Reponsible Party
PK BPR ID
Is balanced by
Customer Supplier
Has a conttract on supply PK Metering Point ID PK Supplier ID
Adress Adress
Metering Point supplies Consumption metered via
PK Metering Point ID
Adress
Is installed at Meter
Serial No.
Is part of
Register
Register No.
Is part of
Meter Reading
Date from
Date to
Value
Figure 3 DSO Metering Point Database
Contract Number AID-168-C-14-00002 25
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
5. Roadmap for development of EDI in BiH
This section presents summary recommendations, which are structured in the form of a roadmap for development of EDI for the retail market in BiH. The roadmap is composed from a set of steps (or activities), which are sorted in chronological order below in list.
Steps describe the work to be done by the retail market participants, responsible actors, the estimated duration and expected outcome. The DSOs shall have central role, with necessary involvement of other retail market participants (Suppliers and BRPs), which are the DSO’s counterparties in EDI. Suppliers and BRPs will have an opportunity to participate in activities of concern and to influence decisions and development processes. In addition, they will also have responsibility for implementation of the agreed requirements.
The Roadmap is composed of the following steps:
1. Determine communication means – In the beginning, the implementation of SMTP for the exchange of messages is recommended, since this approach is most cost efficient for bilateral data exchange. However, it is necessary to review alternatives. A means for backup communication (phone call or fax) should be determined and should be used in case the main communication means becomes unavailable. o Responsible: Group of experts (IT and power engineering) composed of representatives of retail market participants and regulators. o Duration: 1-2 months. o Outcome: Decision on implementation of SMTP (or other) communication means, and details on implementation of this approach.
2. Unique and unambiguous market participant identification – This process is already implemented in BiH for wholesale market and suppliers, and BRPs are identified using EIC X codes by the ISO, which acts as the Local Issuing Office. This approach is legally binding and stated in the Market Rules. It is necessary to extend this process to cover the retail market as well. Still, it is necessary to review other possibilities and their applicability in this case. o Responsible: Group of experts composed of ISO representatives (as Local Issuing Office for BiH should have a pivotal role), DSOs, and regulators. o Duration: 1-2 month. o Outcome: Decision on identification method for retail market participants.
3. Unique and unambiguous identification of metering point scheme –DSOs in BiH have their own internal schemes (in-house solution on a company level), which are used for identifying metering points. Therefore, it is necessary to jointly (all DSOs in BiH) review the existing identification schemes and agree about a unified identification scheme for the whole country. In case an agreed
Contract Number AID-168-C-14-00002 26
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
identification scheme is different (most likely) than the internal coding scheme of a particular DSO, the DSO will be responsible to implement the agreed scheme on external data exchanges.27 o Responsible: Group of experts composed of representatives of DSOs and regulators. o Duration: 1-2 months. o Outcome: Adopted/agreed coding scheme for metering points on the level of country.
4. Definition of messages that are exchanged – The messages are exchanged within the business processes where the DSO acts as “independent market facilitator” and are described in document “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator”. This document shall be a starting point for the work on this step. Each and every message should be reviewed, having in mind the sender and receiver and context of business process in which the message is exchanged. o Responsible: Group of experts (power engineering) composed of representatives of market participants. For the purpose of reducing the process duration, the Group workload may be divided into a few subgroups, each dealing with set of co-related processes. o Duration: 2-3 month. o Outcome: Adopt/agree on list of messages for all data exchanges, within business processes described in the retail market.
5. Definition of content (data, list of information) or messages – This step continues the previous step, going into more detail about message structure. Each message that is exchanged within the retail market business processes (defined in the previous step) should have its contents defined28 (i.e. data set/list of info). The message content (data) must be sufficient to provide business process implementation. o Responsible: Group of experts (power engineering) composed of representatives of market participants. For the purpose of reducing the process duration, the Group workload may be divided into a few subgroups, each dealing with a set of co-related processes. o Duration: 3-5 months. o Outcome: Define structured data for each message from the list defined in the previous step. Each message shall be presented on a conceptual level with a list of info (data set).
6. Definition of file formats for messages – Once the content of all messages is defined (in the previous step), it is further necessary to define/develop data file formats in which the data set/list of info shall be “packed” or “embedded.” Initially, a decision on the file format that shall be used shall be adopted. Afterwards, it will be necessary to develop all messages with their contents into a workable file format.
27 And do the mapping of external and internal coding scheme where necessary. 28 For some messages/business processes, contents (a list of info that shall be exchanged) are already defined in detail in the document “Draft Guidelines for Development of the Business Processes for DSO as Independent Market Facilitator”; for others this task should be performed within this step.
Contract Number AID-168-C-14-00002 27
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
o Responsible: Group of experts (IT, power engineering) composed of representatives of market participants. For the purpose of reducing process duration, the Group workload may be divided into a few subgroups, each dealing with set of co-related processes. o Duration: 3-5 months. o Outcome: Decision on the data file type; development and creation of the file formats and templates for EDI exchange as well as a document (manual) for usage of the developed format. Each message shall be presented on paper with the concrete data file, format (EDIFACT, CSV, EXCEL or XML) and values.
7. Definition of necessary software modifications and/or procurement for EDI implementation – It is necessary to identify possibilities of existing systems to implement requirements29 stemming from the above-mentioned activities. Hence, a “gap analysis” should be performed, the result of which will be a functional design of missing/required functionality that will be achieved either with an upgrade of existing or procurement/development of new software platforms as in the next step. o Responsible: Each market participant shall form a group of IT and power engineering experts dedicated for this task. o Duration: 2-3 month (estimate for DSO, depending on the situation). o Outcome: List of basic functional requirements that are needed for exchange messages defined (content, formats, templates) in previous steps.
8. Technical specification, procurement, development, commissioning – Based on the functional design from the previous point, a detailed technical specification for software upgrade/new solution must be developed. Procurement, development and the commissioning of this upgrade/solution needs to be done, selecting one of following three approaches: o In-house development, where a party’s own IT personnel develop the requested software upgrade/new solution. o Development by a local independent software company, where market participant contracts with a software company in BiH to do the job of development and commissioning. o Buying an “off the shelf” solution for the market (if applicable) and adapt it to the situation in BiH.
Each market participant shall make a decision in accordance to its own situation and company circumstances, with some kind of cost-benefit analysis in place.
o Responsible: Each market participant shall form a group of experts (IT, power engineering, financial, decision makers) dedicated to this task. o Duration: 6-9 months (estimate for DSO, depending on the situation). o Outcome: Decision on scope and method for software modification/new solution; technical specifications of upgrade/new solution; development and commissioning of software solution based on specifications; working software solution.
29 For example, implementation and translation of identification schemes (new and existing) for metering points, data exchanges in specific file formats, other requirements related to business processes, etc.
Contract Number AID-168-C-14-00002 28
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
9. Test operation – Once a software solution is developed and commissioned, a testing operation of proper work during a specific period is required. During this period, users shall be able to get to use the software, and developers will have opportunity to correct errors and unpredicted behavior. o Responsible: Each market participant shall form a group of experts (IT and power engineering) dedicated for this task. o Duration: 3-6 months (estimate for DSO, depending on the situation). o Outcome: Software solution tested and emerging errors and dysfunctionalities corrected.
10. Regular-commercial operation – After a successful testing period, the software solution is ready for regular, day-to-day operation. o Responsible: Each market participant shall form a group of experts (IT, power engineering, and decision making) dedicated for this task. Software operation shall be monitored and the group will suggest potential improvements on a regular basis. o Duration: During software lifetime o Outcome: Stable and reliable operation of a software solution for EDI data exchanges in the retail market in BiH.
First six steps are related to development of EDI Standard. Steps from 7 to 10 are related to application of EDI Standard to existing IT Systems of stakeholders Development of first three steps (1 – 3) could start at the same time and may work concurrently (in sync), as they relate to basic EDI fundamentals (communication means and identification schemes). Decisions that are adopted through these activities should be issued in a legal document. The next three steps (4 – 6) should be developed sequentially, as the start of next step depends on the completion of the previous step. Development of step 4 may start at the same time as the first three steps, as these processes are not intertwined. EIA will take a lead in coordination of activities described in steps 1 – 6, as these steps require a common understanding and agreement between stakeholders and will result in setting up mandatory standards.
Last four steps (7 – 10) should be also developed sequentially, but by each market participant (DSOs, Suppliers and BRPs) individually. The precondition for development of step 7 (and later steps) is the successful implementation of all previous steps. The concerned market participants are responsible for the activities described in steps 7 – 10.
The duration of steps is roughly estimated and is an example. It depends on a number of issues, such as the actual situation, organization, efficiency of participants, etc. Before realization of the roadmap activities, the estimated duration of each step should be reviewed in this sense.
The estimated summary duration period of all processes in the roadmap ranges from 19 (minimum) to 31 (maximum) months. For illustration purposes, in Figure 4, roadmap is presented as a Gantt chart for both the maximum and minimum periods. September 1st was arbitrarily selected as a starting date. The starting date and duration of steps shall be reviewed and updated before beginning of the development process.
Contract Number AID-168-C-14-00002 29
USAID Energy Investment Activity Project (EIA) Guidelines for Electronic Data Interchange, January 2017
Figure 4 Gantt chart – minimum period
Figure 5 Gantt chart – maximum period
Contract Number AID-168-C-14-00002 30