Ground Rules for the Live Webinar Welcome to the Live Please mute your microphone Webinar #8: XML validation before the webinar starts mechanisms To chat with other participants and submit questions, please use the Chat function to the right of your screen
To connect your audio go to the Start 10:00 (CET) Quick Start tab and connect your audio (dial-in, dial-out or connect via computer)
Live Webinar #8 XML validation mechanisms
13 February 2018 Today's speakers
Martin Forsberg Thomas Fillis Martin Forsberg works as an expert in Thomas Fillis is a Communications the area of electronic business, customs Consultant in DIGIT working on CEF and financial processes. Martin was Digital. Prior to joining DIGIT, Thomas involved in the PEPPOL and eSENS advised a Member of the European Large Scale Pilots. He is active in Parliament, and worked in the private standardization committees such as and not-for-profit sectors. A native of CEN TC434 and OASIS UBL. Liverpool, and has certifications from Universities in the UK, Germany and the Netherlands. Agenda
10 00 Welcome Thomas Fillis, DIGIT
10 05 CEF eInvoicing services & upcoming webinars Thomas Fillis, DIGIT
10 15 Interoperability and validation
10 30 XML Schema validation
10 40 Schematron introduction Martin Forsberg, DIGIT
10 50 Xpath & XSLT
11 00 Executation of schematron
11 15 Q & A All
11 30 Close Thomas Fillis, DIGIT Highlights of the webinar
DURING ?
Ask questions Download the presentation
AFTER
Download the Interact with our webinar recording online community 5 Objectives of this module Audience for this webinar
• This webinar will provide information • Public authorities on the XML validation • Private entities mechanisms used in relation to the European Norm for eInvoicing • Policy makers in the Member • It will give an understanding of the States vocabulary used in relation to • Members of standardization bodies validation • Describe typical approaches on how • eInvoicing implementers for… to deploy testing • Software services • Solution providers CEF eInvoicing services & upcoming webinars
Thomas Fillis DIGIT CEF Digital eInvoicing Readiness Checker eInvoicing User Community CEF eInvoicing Trainings
Implementation workshops
• Typically at least one full or one half-day workshop;
• Possibly in combination with bilateral meetings b/w EC and MS;
• So far workshops in Cyprus, Finland, Estonia, Poland and Greece;
• Planned workshops in Croatia, Ireland, Malta and EESPA
• Apply here: [email protected]
Remote trainings • Live sessions on a focused eInvoicing topic for a specialised target audience; • 1-3 hour-long sessions provided on-line; • Focused training sessions on key areas derived from the on-site workshops.
Webinars • About 1 hour-long sessions with core elements from on-site and remote trainings to gain expertise in key areas. CEF eInvoicing Service offering
OPERATIONS SERVICES STAKEHOLDER MANAGEMENT SERVICES
Stakeholders Community management Testing services Supporting services follow-up services
Awareness raising & Service Desk Knowledge base A P stakeholder follow-up A P Conformance testing A P eInvoicing A P Training A P Community animation Readiness Checker A P A P
STANDARDS OF EUROPEAN STANDARDS ORGANIZATIONS (ESOs) AUDIENCE
A Public Administrations Available CEN semantic standard for eInvoicing in public procurement P Solution Providers Coming soon A P
Service offering Service Level eLearning, videos, CEF Digital platform Description (SoD) Arrangements (SLA) success stories CEF eInvoicing service offering, and All services are described in an SoD Documents that describe Service Some services feature multimedia such more about the building block, can describing its purpose, the users for Level Targets to be reached when as eLearnings, instructional videos or be found online which it is for, its benefits and the delivering Building Block Services. success stories to help grasp what the process to use it service is about CEF Digital > More Webinars Related to the Standard and the Directive
September Webinar #3 The European norm and its content (eInvoicing Directive)
October Webinar #4 Infrastructure based on CEF eDelivery DSI
November Webinar #5 eInvoicing from a user's perspective (incl. ordering & payments)
December Webinar #6 Examples of Early Adopters of large scale eInvoicing
January Technical webinars Webinars #7, 8 & 9 Basic XML + XML Validation mechanisms + OASIS UBL 2.1 and UN/CEFACT CII D16B February
13 More information on the events can be found here: https://ec.europa.eu/cefdigital/wiki/x/MQHpAQ Next Webinar # 9: Audience for next webinar: Understanding OASIS UBL 2.1 and UN/CEFACT Cross Industry Invoice D16B • Public authorities
• March • Private entities • Give understanding of how to find • Policy makers in the Member specifications can be found and how States to navigate the specification package • Members of standardization bodies • Insights into the XML schemas and technical artefacts • eInvoicing implementers for… • Where to learn more • Software services • Solution providers XML validation mechanisms
Martin Forsberg DIGIT Interoperability and validation Y1 XML Schema and Schematron – a good combination
XML Schema • Vocabularies and document models • Datatypes and formats • Structures and order • Everything which isn’t explicitly allowed is forbidden! Schematron • Rules-oriented • Conditions and relation between elements • Advanced logic • Everything which isn’t explicitly forbidden is allowed!
Interoperability and validation technology
Schematron Validation
XML Schema Validation
XML Well formedness Well Formed XML
• When a document follows the XML syntax-rules, it is Well formed Well-Formed XML Not Well-Formed XML
• A document which is not Well Formed is not considered XML
XML Schema validation Y2 XSD Schema
• DTD (originally from SGML) often considered too complex and at the same time to limited • XSD was developed to be more fit-for-purpose and is a W3C-standard • XSD describes the ”grammar” of a document model including types • It is possible to test if a XML document is valid in relation to a schema Benefits from using XML Schemas
• Structured validation • Find errors •Integration • Document interfaces • All integrating actors can validate against the same artefact • Standardized way of exchanging a document model • Compliance testing • Version control • Widely spread and support in many tools • Contracts in service oriented solutions Describe and validate structure and content
• Define message structures • Define cardinality, if an element is • Mandatory • Optional • Repetable
• Simple Types • Defines elements and data types • Complex Types • Use to define complex structures
Notation (tool dependent)
Mandatory element with underlying elements
Mandatory element with text content
Indicates that all underlying elements must be stated in a specific ordered sequence
Indicates that there is a choice between one of the underlying elements
Optional element with text content
Optional and repeatable element (unlimited repetitions allowed) Datatypes
Simple Type Example Description anySimpleType Basic type definition on which all other primitive types are based anyURI http://www.example.com/ A URI boolean true, false,1, 0 True or false date 1999-05-31 Year-month-day dateTime 1999-05-31T13:20:00.000 Represents a point in time. In the example, 1:20 p.m. on May 31, 1999. The ‘T’ is a separator character. decimal -1.23, 0, 123.4, 1000.00 Represents a real number with any precision. double -INF, -1E4, -0, 0, 12.78E-2, Represents a 64-bit floating-point number 12, INF with double precision. duration P1Y2M3DT10H30M12.3S Represents a length of time. In the example, 1 year, 2 months, 3 days, 10 hours, 30 minutes and 12.3 seconds. float -INF, -1E4, -0, 0, 12.78E-2, Represents a 32-bit floating-point number 12, INF with simple precision. gDay 31 A day of the month in the Gregorian calender. gMonthDay 31-maj A month and day in the Gregorian calender. gMonth 5 A month in the Gregorian calender. Datatypes
Simple Type Example Description gYear 1999 A year in the Gregorian calender. gYearMonth feb-99 A year and month in the Gregorian calender. integer -126789, -1, 0, 1, 126789 An integer. int -1, 126789675 A 32-bit integer. language en-GB, en-US, fr The designation of a language. Permissible values of XML::lang are specified in XML 1.0. Name Shipping address The XML 1.0 name type. NCName Address Non-colonized name: in other words, a QName without the prefix and colon. QName cac:Address A qualified name within a specified namespace. string Packaging included A character string with no other constraints. time 13:20:00.000, 13:20:00.000 Hour/minute/second/fraction AnyType Use of code lists in XML Schemas
• Built in ”enumerations” of code values is a common way of defining allowed value domains • Code lists must then be published as an integrated part of the XML Schemas • New versions of XML Schemas must be used to get access to new code values • Potential compatibility issues between publications
Enumeration
Facets
• Element and attribute value restrictions
Facet Description xs:length Specifies the exact length of a string. xs:maxLength Specifies the maximum length of a string. xs:minLength Specifies the minimum length of a string. xs:pattern Specifies a regular expression which the string must match. xs:maxExclusive Upper limit for numeric values. The value must be less than that specified here. xs:minExclusive Lower limit for numeric values. The value must be greater than that specified here. xs:minInclusive Lower bound for numeric values. The value must be greater than or equal to that specified here. xs:maxInclusive Upper bound for numeric values. The value must be less than or equal to that specified here. xs:enumeration Specifies a list of permissible values. xs:whiteSpace Specifies how “whitespace” characters should be treated. xs:totalDigits Specifies the exact number of digits. xs:fractionDigits Specifies the maximum number of decimal places permitted. Validation with XSD-scheman
Well Formed
Well-Formed XML Not Well-Formed XML
XML Processor Error!
Application
Well Formed •Well Formed och Valid
Well-Formed XML Not Well-Formed XML Valid XML In valid XML
XML Processor Error! XML Processor
Application Validation Error!
Application
• Rule based validation language for XML documents • ISO-standard • Can test dependencies between elements/attributes such as: • If Invoice contains a VAT amount, then the Supplier VAT-number must be stated • Advanced functions • Invoice Total Amount = Sum of (Invoice Line Amount + charges – Allowances) • Check-sum validation of GLNs, GTINs… • Gives meaningful error messages • Can be used to decouple code list values from XML-schema
The structure of a typical validation rule in Schematron
• Context – The element to be tested • Assertion – The logical statement/rule to evaluate • If true – everything is ok, continue to next test • If false – rule violation! • Message • Normally the rule text in natural language • Flag • Often used flags: • fatal, warning och information • Fatal – violations against “SHALL/MUST”-rules • Warning – violations against SHOULD-regler • Information – highlights something which is not necessarly an error • “Element ABC should only be used if a bilateral agreement exists”
Schematron example
Type of error
Typical validation steps with schematron
Valid XML Invalid XML
XML Processor Well formed?
Validation Schema describes types and Valid? structures
Schematron Rules – dependencies, Rules? calculations…validation Error!
Application
Valid XML In valid XML
XML Processor Automatically by the XML parser
Validation Schema describes types and XML-schema (XSD) structures
Schematron-files or rules transformed into XSLT-files Schematron Rules – dependencies, calculations…validation Error!
Application
Valid XML Invalid XML
XML Processor
Validation Schema describes types and structures
Schematron Rules – dependencies, calculations…validation Error!
Application
XPATH - XML Path Language
XSLT - EXtensible Stylesheet Language
SVRL - Schematron Validation Report Language Xpath & XSLT Y4 XPath
• Path/query language for XML • Used to access/point out elements relative a given position
Xpath Meaning
Elementnamn Return the child element with the given name
/ Starting at the root element
// Return the matching element regardless its position within the document . Returns the value of the given element
.. Returns the parent element
@ Returns an attribut XPath
Operators Functions Arithmetic (+, -, * and div) concat Relational ( =, !=, <, >, <= och >=) contains Boolean (and, or, not() ) format-number starts-with substring string-length count sum
http://www.w3schools.com/xsl/xsl_functions.asp Logical assertions in XPath
Invoice/TotalAmount = sum(Invoice/Line/LineAmount) Invoice Total amount must equal the sum of the invoice line amounts concat(ElementA,ElementB) = ‘SEDK’ The concatenated value of ElementA and ElementB must equal SEDK
XSL Family
• XSL Transformations (XSLT) transforms an XML document into • Another XML-document • Other formats, such as HTML • XSL-FO • XSL-FO – XML Formatting Objects • Used as intermediate when creating PDF, PNG and PostScript • XPath • Actually part of the XSL standard XSL-transformation
• An XSLT processor takes an XML-document an transforms it, according to the XSLT, into a result document
Schematron execution Y5 Execution of schematron
• Direct (Native) implementation • Application uses schematron formatted files directly • Good performance • Few implementations
• By first creating XSLT from Schematron • Schematron is transformed into a stylesheet • XML document to be tested is then transformed by this stylesheet • Result in XML (Standard Validation Result Language)
Native execution of schematron
Schematron Native execution of schematron
Schematron
Schematron
Schematron
QUESTIONS?