XML in Electronic Data Interchange, Application Integration and Databases

"Modern" XML applications XML in electronic data interchange, application integration and databases

Patryk Czarnik

Institute of Informatics University of Warsaw

XML and Modern Techniques of Content Management – 2010/11

Electronic data interchange Introduction Pre-XML solutions XML for EDI

Application integration Idea Web Services

XML in security XML Signature XML Encryption

XML and databases XML support in relational databases XML databases Electronic data interchange (EDI) — motivation

How to interchange data between How to establish EDI companies/institutions (B2B)? protocol?

I paper I customer receives (or I electronic data interchange buys) a tool from provider

Standard deployment levels I smaller partner complies to bigger software developed according to I parter standard from beginning ad-hoc created interface added to legacy system I I conversion tools

I standard

EDI standardisation prior to XML introduction

ANSI Accredited Standards Committee X12 sub-group

I USA national standard

I used mainly in America

EDIFACT

I international standard (UN/CEFACT and ISO)

I used mainly in Europe and Asia EDIFACT characteristic

Format

I text

I hardly readable

I tree structure

Predeﬁned dictionaries

I 193 message types

I 279 segments

I 186 elements (version 08a, 2008)

EDIFACT

EDIFACT message example

UNB+IATB:1+6XPPC+LHPPC+940101:0950+1’ UNH+1+PAORES:93:1:IA’ MSG+1:45’ IFT+3+XYZCOMPANY AVAILABILITY’ ERC+A7V:1:AMD’ IFT+3+NO MORE FLIGHTS’ ODI’ TVL+240493:1000::1220+FRA+JFK+DL+400+C’ PDI++C:3+Y::3+F::1’ APD+74C:0:::6++++++6X’ TVL+240493:1740::2030+JFK+MIA+DL+081+C’ PDI++C:4’ APD+EM2:0:1630::6+++++++DA’ UNT+13+1’ UNZ+1+1’

cite: Wikipedia EDIFACT structure

Wymiana (interchange)

Wiadomość (message)

Grupa (segment group)

Segment MEA+WT+AAD+KGM:690+X5'

Złożenie (composite) +KGM:690+

Element (data element) :690

XML EDI Idea: use XML as data format for EDI

Traditional EDI XML EDI

I Documents unreadable I „Self-descriptioning” documents without speciﬁcation format

I Compact messages I Verbose messages I Centralised standard I “Pluggable”, ﬂexible standards maintenance I Well written software ready to I Changes in format requires format extensions software change I XML-format layer handled by I Specialised tools needed general XML libraries XML EDI ﬂexibility

Format ﬂexibility

I Structures: choosing, repeating, nesting, optionality I Format extensions and mixing via namespaces

Applications

I Data interchange between partners’ systems I Web interface (easy transformation via XSLT) I Web Services integration

XML EDI standardisation Framework level

I general rules for all kinds of data

I data of the same kind should be represented in the same way (not to deﬁne the same twice)

I example: Electronic Business XML (ebXML).

Industry standards

I SWIFT — banking

I RosettaNet — trade and logistic

I Automotive Industry Action Group — motor industry (mainly American)

I Health Level Seven — health care

I Open Travel Alliance — (people) transport and tourist services

I ... ebXML

ebXML

I set of speciﬁcations deﬁning concepts and methodologies for conducting electronic business via Internet (2001)

I XML used as data format

Electronic Business XML Working Group

I founded in 1999

I more than hundred specialists

I OASIS and UN/CEFACT patronage

ebXML standardisation

I Meta-model:

I zbiór podstawowych schematów, elementów XML oraz procesów biznesowych, I sposób deﬁniowania słowników danych, I nie deﬁniuje konkretnych, docelowych komunikatów – mog ˛aone zaleze˙ c´ od konkretnego zastosowania.

I Metainformacje:

I informacje o wersjach, I metadane odpowiadaj ˛acenagłówkom z istniej ˛acych systemów EDI.

I Ramy architektury technicznej:

I sposoby implementacji repozytoriów, serwisów, itp., I integracja z istniej ˛acymi technologiami EDI. XML for application integration

I Goal — data interchange between applications

I applications/modules/components with different internal formats I XML as interface

I Usage:

I client/server communication I distributed system nodes I components integration I conﬁguration of application or components I ...

Local and global applications

“Local” integration

I within single project or related projects of single institution

I communication between components

I possibly in distributed architecture

I ad-hoc solutions for given problems

I possibility of using standard

“Global” integration

I services available in Internet for any party

I different parts cooperation

I standardisation required

I most popular standard — Web Services Web Services

Idea Web Service — a website for programs (instead of people) Practice

I high-level network protocols (HTTP)

I services described (WSDL)

I structural messages (XML, SOAP)

I possibility of services registration and searching (UDDI)

Web Services — typical applications

I Providing data (free or paid):

I timetables I weather I stock and currency notes

I Services:

I searching I software updates

I Business operation between partners

I booking tickets or hotel rooms I ordering (and tracing order status) I electronic data interchange Web Services standardisation

I SOAP (initially “Simple Object Access Protocol”: I beginnings: 1998 I v1.2: W3C Recommendation, June 2003 I Web Services Description Language: I v1.1: W3C Note, 2001 I v2.0: W3C Recommendation, June 2007 I Universal Description Discovery and Integration: I OASIS project I part of WS-I Basic Profile I WS-* standards: I various standards, usually not W3C: I Web Services Interoperability — levels of WS compliance: WS-I Basic Profile, Simple Soap Binding Profile, . . . , I WS-Eventing, WS-Addressing, WS-Routing, . . . — IBM documents

I Business Process Execution Language (OASIS) — WS semantics description, programming using WS as building blocks

SOAP — communication protocol

I Underlying transport protocol (HTTP or other)

I Message format (XML) I Differences to RPC, CORBA, DCOM etc.:

I data represented in extensible, structural format (XML) I data types independent of platform (XML Schema) I lower efﬁciency SOAP message — general form

SOAP message

I XML document for a single message

I namespace http://www.w3.org/2001/12/soap-envelope, I main element: Envelope.

I Main parts: header optional body required I Restrictions:

I no DTD (and external entity references) I no processing instructions SOAP header

I actor — header receiver identiﬁer (URI), optional

I mustUnderstand — must header be understood? (0/1)

W3Schools example

234

...

SOAP body

I remote procedure call

I parameters

I encodingStyle — data encoding style (URI)

Request — altered W3Schools example

Apples PLN

SOAP body

I procedure result

I output parameters

Response — altered W3Schools example

1.90 PLN

SOAP — failure message

I standard error code

I short text description

I additional data (XML)

Response with failure message

Receiver Data missing Found no student identified with 123

WSDL — service description

I XML document describing service(s)

I namespace: http://schemas.xmlsoap.org/wsdl/ I main element: definitions

I Splitting into parts available

WSDL document components types type deﬁnitions (XML Schema) message message type deﬁnitions portType set of operations, which have input and output messages serviceType consists of portType-s binding service type bound to concrete transport protocol service concrete service available somewhere

WSDL — messages, operations, port types

W3Schools example

WSDL — SOAP binging style rpc or document transport transport protocol (URI) soapAction SOAP action corresponding to WSDL operation

W3Schools example

Service registration and discovery

Idea

I service provider registers service

I user searches for service and ﬁnds it in registry

Universal Description Discovery and Integration (UDDI)

I available as service (SOAP)

I business category-based directory (“yellow pages”)

I searching basing on service name, description (“white pages”)

I registration and updates for service providers UDDI — issues

Main issue — who can register?

I anybody — chaos and low reliability

I accepted partners — institution responsible for access policy needed, no such (widely accepted) institution exists

Reality

I UDDI rarely used

I if ever — for “local” SOA-based solutions (intranets)

Service Oriented Architecture

Idea

I services built basing on other services

I even addition deﬁned as a Web Service :)

I software split into components and layers with WS interfaces between them

I precise speciﬁcation required (interesting research ﬁeld. . . )

Critique

I modular, ﬂexible, and scalable solutions

I by the cost of (sometimes) irrational inefﬁciency and complexity

I use reasonably XML in security

Main (cryptographic) security aspects conﬁdence assured by encryption authentication assured by cryptographic signatures

XML standards related to security

I Signing — XML Signature

I Encrypting — XML Encryption

XML Signature

I Element Signature in appropriate namespace

I cryptographic signature I additional information (algorithm etc.)

I Signature added to XML documents

I externally (detached signature) I internally (enveloped signature) I in container enveloping signed data (enveloping signature)

I XML Signature capabilities

I signing fragments of documents I signing external resources (identiﬁed with URL) I multiple signatures in single document XML Signature — example 1 (detached)

60NvZvtdTB+7UnlLp/H24p7h4bs= OsH9A1jTNL...

imup6lm...

xDve3j7...NlugAf... W7dOmH/v...

Cite: Kazienko, P., Co tam panie w XML-u?, Software 2.0, 6/2003

XML Signature — example 2 (enveloped)

... .... XML Encryption

I Goal: guarantee XML data conﬁdence

I Encryption of whole documents or parts of them

book 123-958-74598 12 123654-8988889-9996874 visa 12-10-2004

XML Encryption — example 1 (fragment encrypted)

book 123-958-74598 12 A23B45C564587 visa 12-10-2004 XML Signature — example 2 (whole document encrypted)

John Smith

A23B45C56...56F47345

XML support in databases — categorisation

Classic (usually relational) database with XML support

I logical structure — relations and references

I additional XML-related features

I used for application integration or storing XML data as part of larger data structures

XML database

I logical structure — XML document tree

I XQuery (or XPath) as native query language

I natural XML-related features

I used for storing XML data (or structural data easily mapped to XML tree) XML support in relational databases

Possible functionalities

I data import and export in XML format I special treatment of XML data stored in ﬁelds

I XML validation as part of integrity constraints checking I XPath or XQuery for querying ﬁelds content

I XSLT applied to query results

How to store XML data

I whole document (fragment) stored in single ﬁeld I split into prima factors

I each XML node in separate ﬁeld I tables structure reﬂects tree structure of XML

Example — XML support in Oracle database

I Since Oracle 8i (http://www.oracle.com/xml). I XML parsers

I for database programming (PL/SQL) I or middleware programming (Java, C++)

I XML-SQL Utility

I XML data import and export

I XMLType data type XML-SQL Utility getXML() function — XML data export

SELECT xmlgen.getXML(’select * from emp’) FROM dual;

10 Scott Tiger specialist ...

XML support in database engines

Substantial support

I DB2, IBM (since version 9 — pureXML)

I Oracle (since 8i)

I Microsoft SQL Server (od wersji 2000)

I Sybase ASE 12.5

Minimal support

I MySQL (XPath queries over text ﬁelds containing XML)

I PostgreSQL??? “XML datbase”

I Logical layer

I XML document as basic data entity I XML schema (or equivalent) as structure deﬁnition I XQuery (or XPath) as “native” query language I document collections

I Physical layer

I not necessarily ﬁles with XML documents

I More than just XML ﬁles:

I updates interface I transactions and concurrent access I security

XML:DB

I Initiative for XML database interfaces speciﬁcation I XML Database API (XAPI)

I accessing XML databases from programs I resource collections (resource = XML document) I reading and writing documents via DOM or SAX I pluggable “services”; speciﬁed: XPath, transactions, operations on collections I last version: 2001

I XML Update Language (XUpdate)

I XML application for updating XML databases I inserting, updating and removing nodes I XPath used for node addressing I last version: 2000 XUpdate — example

Example (from standard documentation)

2 Lars Martin Leizig Germany

XML databases — overview product licence queries XML:DB API Apache XIndice open source XPath yes BaseX open source XPath, XQuery yes eXist open source XPath, XQuery part Sedna open source XPath, XQuery yes Tamino paid XQuery, XPath part Gemﬁre Enterprise paid XQuery, OQL yes

cite: Wikipedia and providers’ websites