The Mapping Editor

Transcription

The Mapping Editor

Woche der Anwendungsentwicklung, Ehningen, 27.3.2014
Carsten Mai
IBM Integration Bus V9
© 2013 IBM Corporation
Systeme sehen oft so aus …..
Instabile Punkt-zu-Punkt-Verbindungen
Viel Redundanz in den
Verbindungen
Enge Abhängigkeiten zwischen Systemen
Wenig Fexibilität – langsame Reaktion auf
Änderungen der Geschäftsabläufe
Hoher Wartungsaufwand
Schwierige Problemanalyse und - lösung
SOA addressiert diese Herausforderungen
Cloud
Services
Event
Processing
APIs
OLTP
B2B
Rules
Device
Interaction
Analytics
Process
Automation
Batch
Web
Application
“The beauty of SOA…is that we can change our components as needed,
seamlessly…it might be a business process or a whole new business model.”
-Phil Mumford, CEO, Queensland Motorways
“Make SOA a prerequisite architecture. It's time to breathe new life into your SOA
initiative, this time by focusing on architecture instead of technology.”
-Gartner Application and Integration Platforms Key Initiative Overview July 22,
2011
…SOA is simply good design
Social, Mobile, Cloud und Big Data bringen neue
Herausforderungen
Reach
Big Data, Cloud,
Mobile, Social
Cloud
Web, e-business
and SOA
Big
Data
Mobile
Social
Transaction
Systems
Mainframe,
IMS and CICS
WebSphere,
Information Management
1960-
1990-
New Era Platform
Time
2010-
Entwicklung und Wartung vereinfachen!

Entkoppeln von Anwendungen
und Schnittstellen

Kommunikation zwischen
Anwendungen unabhängig von:

Programmiersprache

Systemplattform

Programmiermodellen

Protokollen

Datenformaten.
Lösung: Der Enterprise Service Bus
Das Herz eines 'interconnected' Unternehmens: der Enterprise Service Bus
(ESB)
Was ist ein ESB?
Ein ESB ist eine flexible Verbindungs-Infrastruktur zur Integration von Anwendungen, Systemen und Services.
Ein ESB leistet zwischen
Requestor und Service
ROUTET
Kommunications-Flüsse zwischen Services
KONVERTIERT
zwischen verschiedenen Transportprotokollen
TRANSFORMIERT
zwischen verschiedenen Datenformaten
IDENTIFIZIERT und VERTEILT
Business Events
Shape = Transport protocol
Color = Data format
6
WebSphere Message Broker
Product Roadmap
IBM Integration Bus V9 Q2 2013
Policy-based Workload Management and Flow Management
Web-based Visualization and Performance Analysis
MQ and Database Service Discovery
Business Rules and BPM Integration
.NET Input, Dynamics, MSMQ
Message Broker
V8 FP2 Q1 2013
Message Broker
V8 FP1 Q2 2012
Message Broker
V8 Q4 2011
Healthcare Connectivity Pack
V8 Q1 2013
V7 FP2 Q1 2012
7
DICOM Medical Imaging
Web-based Operational Monitoring
CDA Analytics and Reporting
Message Broker V8 Exploitation
V8 FP1 Q3 2013
V9 FP1 Q4 2013
IBM Integration Bus - Industry Packs Roadmap
WMB Healthcare Pack V8 - Q1
2013
DICOM Medical Imaging
Web-based Operational Monitoring
CDA Analytics and Reporting
Message Broker V8 Exploitation
IIB Healthcare Pack 1.0
Q1 2014
IIB V9
Q2 2013
IIB Retail Pack 1.0 - Q4 2013
IBM's plans, directions,
and intent are subject
to change or withdrawal
8
In-store processing
Store to HQ
Web Operational Monitoring
More
industries
Architektur - IBM Integration Bus
Integration
Toolkit
Integration Node
Integration
Server
Integration
Server
Integration
Explorer
 Integration Toolkit
– Entwicklung und Test
– Basiert auf Rational Application Developer
 Integration Explorer
– Administration Tool
– Plug-In im MQ Explorer
9
 Integration Node
– Laufzeitumgebung in der die Message
Flows ausgeführt werden
– Integration Server - Isolation und
Scalierbarkeit
– Viele verschiedene Plattformen
– Setzt auf MQ Queue Manager auf
Runtime Komponenten
 Eine oder mehrere ‘Integration Server’,
welche jeweils mehrere Message Flows
beinhalten
– Isolation und Skalierbarkeit
– Effektive Ausnutzung von Multi-core
Prozessoren
Integration Node
Integration Server
Integration Server
Integration server
 auf Windows, z/OS, AIX, HP, Solaris, Linux
(Intel, zSeries, PPC)
 Pro Server ist ein lokaler MQ Queue
Manager erforderlich
Queue Manager
Demo …..Anlegen Integration Server,
Integration Toolkit,
Integration Explorer
A Broad Range of Supported Platforms and Environments
Broad range of operating system and hardware platforms supported
– AIX, Windows, z/OS, HP-UX, Linux on xSeries, pSeries, zSeries, Solaris (x86-64 & SPARC), Ubuntu
– Optimized 64-bit support on all platforms; 32-bit option for development platforms (Windows &
x/Linux)
– New support for Windows 8 and Windows Server 2012; .NET CLR V4.5 included on Windows
Virtual images for efficient utilization & simple provisioning
– Extensive support for virtualized environments, e.g. VMWare, AIX Hypervisor… any!
– IBM Workload Deployer for x/Linux & AIX
– Support for Pure on POWER hardware to complement xLinux
– SmartCloud and IBM Workload Deployer images for simplified solution provisioning
Includes access to full range of industry standard databases and ERP systems
– DB2, Oracle, Sybase, SQL Server, Informix, solidDB
– Open Driver Manager support enables new ODBC databases to be accessed
– JDBC Type 4 for popular databases
– SAP, Siebel, Peoplesoft, JDEdwards at no additional cost
Traditional
OS
Public Cloud
Technology components and pre-requisites
– Java 7 on all platforms
– MQ 7.5 prerequisite (7.1 on z/OS)
Private
Cloud
IBM Workload
Detailed System Requirements
Deployer
http://www.ibm.com/software/integration/wbimessagebroker/requirements/#V90
12
IBM Pure
WebSphere Message Broker –
Protokolle und Anwendungen
Industry and Vendor Protocols
IBM Protocols
WebSphere MQ (+ File Transfer Edition + C:D)
Any 3rd-party JMS
(Enterprise applications (+ managed file transfer))
(TIBCO EMS, Sonic MQ, BEA JMS,
webMethods, See Beyond, Vitria)
WebSphere MQ Everyplace
TIBCO Rendezvous
(Mobile device applications)
(plug-in component)
WebSphere MQ Telemetry
HTTP and HTTP(S)
(RFID, sensors & actuators)
FTP and File
WebSphere MQ Real-time
(Very low latency over WANs, and
the Internet)
TCP/IP Sockets
WebSphere MQ Multicast
SMTP
(Reliable Multicast Messaging (RMM))
(Very low latency for LANs)
SAP
CICS
Oracle
Siebel
JDEdwards
Peoplesoft
Enterprise Applications
13
Custom
WebSphere Adapters
Existierende Anwendungen am ESB anschliessen
Wiederverwendung von Anwendungsdaten und -logik
End-to-end Interaktion
mitgelieferte Adapter und Entwicklung eigener Adapter per Toolkit
ESB
ROUTING
MATCHING
CONVERTING
messages between
services
granularity of services
exposed to applications
TRANSFORMING
HANDLING
transport protocols between message formats between
requestor and service
requestor and service
business events from
disparate sources
z/OS and
Linux on
System z
Adapter
z/OS and
Linux on
System z
Adapter
Linux on
System
z
Adapter
Adapter
Adapter
z/OS and
Linux on
System z
E-Business Suite
14
Adapter
CICS,IMS
Carsten Mai
IBM Integration Bus
Message Modeling
The Challenge
Unterschiedlichste Datenformate müssen verarbeitet werden ...
BINARY
00011110010010011010010
10010010010010010000011
11010100101010110010010
10010010010010001010010
10010010101010101000100
11100010100010010001001
00100100100100101010010
01010100100010010010010
01001001110001010010101
01010101010010010101001
00100010010100101000010
10101010001001010001001
01001010100101010100101
01100101000000000001111
10010010010101110010010
01010101001010101101111
STREAM
TABLE
COPYBOOK
Make
Model
PKG
Extended_Fea
tures
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
Ford
Prefect
34890
2984782q,
93847920,
3438084
PROPRIETARY
01
03
05
TP-API-CB.
TP-AGAPI-CB.
TP-AGAPI-REQUEST
PIC X(40).
88
TP-AGAPI-INITIALIZE-REQUEST
VALUE 'INITIALIZE-MAPPING'.
88
TP-AGAPI-PERFORM-MAPPING
VALUE 'PERFORM-MAPPING'.
88
TP-AGAPI-FINISH-MAPPING
VALUE 'FINISH-MAPPING'.
05
TP-AGAPI-VERSION
PIC X(04).
88
TP-AGAPI-VERSION-VALID
VALUES ARE '0100' '0200'.
88
TP-AGAPI-VERSION-0100
VALUE
'0100'.
88
TP-AGAPI-VERSION-0200
VALUE
'0200'.
05
TP-AGAPI-RESPONSE.
10
TP-AGAPI-RESPONSE-CODE
PIC 9(04) COMP.
88 TP-AGAPI-ALL-OKAY
VALUE 0.
88 TP-AGAPI-REQUEST-ERROR
VALUE 1.
88 TP-AGAPI-INITIALIZE-ERROR
VALUE 2.
88 TP-AGAPI-MAP-ERROR
VALUE 3.
88 TP-AGAPI-FINISH-ERROR
VALUE 4.
88 TP-AGAPI-UNKNOWN-LOOP-ID
VALUE 5.
88 TP-AGAPI-NO-ALGORITHM
VALUE 6.
88 TP-AGAPI-NO-PARTNER
VALUES 7 15.
88 TP-AGAPI-NO-APPLICATION
VALUE 8.
88 TP-AGAPI-ALGORITHM-IO-ERROR VALUE 9.
88 TP-AGAPI-FATAL-GATEWAY-ERROR VALUE 10.
88 TP-AGAPI-GATEWAY-WRITE-ERROR VALUE 11.
88 TP-AGAPI-PARTNER-IO-ERROR
VALUE 12.
88 TP-AGAPI-BAD-VERSION
VALUE 13.
88 TP-AGAPI-BAD-NUMERIC-TYPE
VALUE 14.
88 TP-AGAPI-NO-ALIAS
VALUE 15.
88 TP-AGAPI-ACCESS-ERROR
VALUE 16.
10
TP-AGAPI-RESPONSE-MESSAGE
PIC X(80).
05
TP-AGAPI-APPLICATION-ID
PIC X(10).
03
01
CASH RECONCILATION
10029847 - $100,000,000.00
13948589 - $679,495,094.98
13950967 - $588,345,058.00
13950968 - $000,000,000.00
14001321 - $098,957,038.12
AAA: HT4459
AAA: B33566
AAA: C4058G
AAB: 948409
AAB: 874931
27 files
EBCDIC
DDA Application Updates
OH,257*IN,142*MI,1
54*WI,80*MT,5*ID,8
*WY,3*CO,21*NM,8*
AZ,15*UT,13*NV,4*
MN,48*MO,67*ND,9
*SD,9*KS,27
DUNS 0123
F046000INV
for 2 months 120799
12 718-339-1700I43989D-2
120799999-b
003000010000
DUNS 4445
P55590
120799499QR
000004004000
ãÖ@Ô…™ƒ£–
™z@Ã–•¥…
™£@”…@£–
@ÅÂÃÄÉÃ
<MSG 19934749>
<ACCT BAL RPT>
<ACCTNUM><“14001321”>
<ENDBALANCE><” $098,957,038.12”>
<DEBIT><“103048382$394,394.00”>
<DEBIT><“103048383$001,293.65”>
<CREDIT><“987463921$928,943.67”>
</ACCTNUM>
</MSG>
413 msgs
Parser
Model
Input Message Bit-stream
F r e d
S m i t h , G r a p h i c s
Parser converts
bit-stream to
logical structure
Parser converts
logical structure
to bit-stream
< o r d e r > < n a m e > M r . S m i t h < / n …
Output Message Bit-stream
Model
17
C a r d …
Message Modeling
Physical
Logical
<order>
<name>
<first>John</first>
<last>Smith</last>
</name>
<item>Graphics Card</item>
<quantity>32</quantity>
<price>200</price>
<date>07/11/09</date>
</order>
Order
Name
John,Smith,Graphics Card,
32,200,07/11/09
John Smith............
Graphics Card.........
3220020071109.........
18
First
String
Last
String
Item
Qty
Price
Date
String
Integer
Integer
Date
Definieren von Message Modellen
C Header
COBOL
WSDL
Copybook
Pre-built
XML
Schema
DTD
Enterprise
Information
System
(SAP, Siebel,
PeopleSoft)
File Import
DFDL/Message Set
Parsers
Message Broker
19
SOAP, MIME,
CSV, IDOC,
SWIFT,
EDIFACT, X12,
FIX, HL7,
etc
Define
your own
using the
Eclipse-based
Tooling
Unterstützt alle Datentypen

High Performance XML Parsing
–
–
–

Standardkonformer XML Parser (XMLNSC Domain) mit komplettem Funktionsumfang
High Performance XML Schema Validierung
Erweitertes 'Opaque Parsing' für optimale XML Subtree Verarbeitung
Unterstützt die gebräuchlichen binären Datentypen in Text-Messages (TDS
domain)
–
–
hexadezimale Inhalte in Daten-Strukturen; Wiederholungsreferenzen
CSV Standard – vorgefertigtes Datenmodel im Toolkit
Datenmodellierung mit DFDL
Einfacher und leistungsfähiger Standard für Datenmodellierung
Für binäre, Text- & Industriestandard Datenformate
• Logische Struktur mit Beschreibung der physischen Formate
• z.B. Endian, ASCII/EBCDIC, Padding, Ausrichung…
Data Format Description Language (DFDL)
• in IBM und nicht-IBM Produkten
• forge.gridforum.org/projects/dfdl-wg
Werkzeuge für leichtes Erstellen der Datenmodelle
'Quick wizards' für Beispieldaten (z.B.) CSV, record oriented data
Importer (z.B.) COBOL copybooks
DFDL editor für 'power users'
• Erstellen des logischen Modells & Definieren der physischen Ausprägungen
Test Parsen und Testdaten-Generierung
Test, ob DFDL-Definition den Testdaten entpricht
• Parser-Trace mit Erfolg-/Fehlermeldungen
Auto-Generierung von Testdaten
DFDL-Parser kann von allen Broker-Nodes genutzt werden
Hervorragende Performance
• Unterstützt Streaming, partial Parsing …
21
Agenda
 DFDL – a new standard for modeling text and binary data
 DFDL support in IIB
22
Data modeling – why DFDL?
 Across organisations, the ability to exchange data is of vital importance
 But there has been no universal standard for modeling general text and binary data
– XML -> use XML Schema
– RDBMS -> use database schema
– Text/binary -> ??
 Existing standards are too prescriptive: “Put your data in this format!”
 IBM products evolved their own way of modeling text and binary data based on
customer need. Examples…
– WebSphere Message Broker: MRM message set
– WebSphere ESB, IBM Process Server: Data Handlers
– Transformation Extender: Type Trees
– DataPower: FFD
– Cast Iron: Flat File Schema
– Sterling Integrator: DDF and IDF files
 DFDL:
a universal, shareable, non-prescriptive description for
general text & binary data formats
23
Data Format Description Language (DFDL)
 A new open standard
– From the Open Grid Forum (OGF)
– Version 1.0
– ‘Proposed Recommendation’ status
 A way of describing data…
– It is NOT a data format itself!
 That can describe any data format …
– Textual and binary
– Commercial record-oriented
– Scientific and numeric
– Modern and legacy
– Industry standards
 While allowing high performance …
– Choose the right data format for the
job
 Leverage XML technology and concepts
– Use W3C XML Schema subset & type
system to describe the logical format of
the data
– Use annotations within the XSD to
describe the physical representation of
the data
– Use XPath when referencing fields within
the data
 Support round-tripping
– Read and write data in described format
from same description
 Keep simple cases simple
– Simple descriptions should be human
readable
 Generality
– Think “Type Tree + MRM + more”
24
Example – Delimited text data
Separator
Initiator
Initiator
intval=5;fltval=-7.1E8
ASCII text
integer
ASCII text
floating point
Separators, initiators (aka tags), & terminators are all examples in DFDL of delimiters
25
Example – DFDL schema
<xs:complexType name=“myNumbers">
<xs:sequence>
<xs:annotation>
<xs:appinfo source="http://www.ogf.org/dfdl/v1.0">
<dfdl:sequence separator=“;” encoding=“us-ascii”/>
</xs:appinfo>
</xs:annotation>
<xs:element name=“myInt" type=“xs:int”>
<xs:annotation>
<dfdl:element representation="text"
encoding=“us-ascii" textNumberPattern=“###0”
lengthKind="delimited" initiator="intval=" .../>
</xs:appinfo>
</xs:annotation>
</xs:element>
<xs:element name=“myFloat" type=“xs:float”>
<xs:annotation>
<dfdl:element representation="text"
encoding=“us-ascii" textNumberPattern=“##0.0#E0”
lengthKind="delimited" initiator="fltval=" .../>
</xs:appinfo>
</xs:annotation>
</xs:element>
DFDL
DFDL
</xs:sequence>
properties
annotation
</xs:complexType>
26
;
intval=5
fltval=-7.1E8
Example – DFDL schema (short form)
<xs:sequence dfdl:separator=“;” dfdl:encoding=“us-ascii” >
<xs:element name=“myInt" type=“xs:int”
dfdl:representation="text"
dfdl:encoding=“us-ascii" textNumberPattern=“###0”
dfdl:lengthKind="delimited" dfdl:initiator="intval=" … />
<xs:element name=“myFloat" type=“xs:float”
dfdl:encoding=“us-ascii" textNumberPattern=“##0.0#E0”
dfdl:lengthKind="delimited" dfdl:initiator="fltval=" … />
</xs:sequence>
</xs:complexType>
DFDL
properties
27
DFDL processor
 A DFDL processor uses a DFDL schema to understand a data stream
 It consists of a DFDL parser and a DFDL serializer
 The DFDL parser reads a data stream and creates a DFDL ‘infoset’
 The DFDL serializer takes a DFDL ‘infoset’ and writes a data stream
28
DFDL
DFDL
Processor
Processor
<Document>
<Document>
<Element name=“myNumbers”/>
<Element name=“myInt” dataType=“xs:int”
<ElementdataValue=“5”/>
name=“myInt” dataType=“xs:int”
dataValue=“5”/>
<Element name=“myFloat” dataType=“xs:float”
<ElementdataValue=“-7.1E08”/>
name=“myFloat” dataType=“xs:float”
dataValue=“-7.1E08”/>
</Element>
</Element>
</Document>
</Document>
<xs:complexType
name=“myNumbers">dfdl:encoding=“ascii” ... >
<xs:sequence dfdl:separator=“;”
<xs:sequence
dfdl:separator=“;”
dfdl:encoding=“ascii” ... >
<xs:element name=“myInt"
type=“xs:int”
<xs:elementdfdl:representation="text"
name=“myInt" type=“xs:int”
dfdl:encoding="ascii“
dfdl:textNumberPattern=“###0”
dfdl:encoding="ascii“ dfdl:textNumberPattern=“###0”
dfdl:lengthKind="delimited"
dfdl:initiator="intval=“
... />
dfdl:initiator="intval=“
...type=“xs:float”
/>
<xs:element name=“myFloat"
<xs:element
name=“myFloat"
type=“xs:float”
dfdl:encoding="ascii“
dfdl:textNumberPattern=“##0.0#E0”
dfdl:encoding="ascii“ dfdl:textNumberPattern=“##0.0#E0”
dfdl:initiator="fltval=“
... />
dfdl:initiator="fltval=“ ... />
</xs:sequence>
</xs:sequence>
</xs:complexType>
</xs:complexType>
DFDL features
 Text data types such as strings, numbers, zoned decimals, calendars, booleans
 Binary data types such as integers, floats, BCD, packed decimals, calendars, booleans
 Fixed length data and data delimited by text or binary markup
 Language data structures found in COBOL, C and PL/1
 Industry standards such as SWIFT, HL7, FIX, HIPAA, X12, EDIFACT, ISO8583
 Bi-directional text
 Bit data of arbitrary length
 Pattern languages for text numbers and calendars
 Ordered, unordered and floating content
 Default values on parsing and serializing
 Nil values for handling out-of-band data
 Fixed and variable arrays
 XPath 2.0 expression language including variables to model dynamic data
 Speculative parsing to resolve choices and optional content
 Validation to XML Schema 1.0 rules
 Scoping mechanism to allow common property values to be applied at multiple points
 Hide elements in the data
 Calculate element values
29 © 2013 IBM Corporation
DFDL language - objects
type
element
*
*
sequence
group
complexType
choice
DFDL properties are placed on yellow objects only
30
simpleType
DFDL language – basic annotations
31
Annotation
Used on Component
Purpose
dfdl:element
xs:element
xs:element reference
Contains the DFDL properties of an xs:element and xs:element
reference
dfdl:choice
xs:choice
Contains the DFDL properties of an xs:choice.
dfdl:sequence
xs:sequence
Contains the DFDL properties of an xs:sequence.
dfdl:group
xs:group reference
Contains the DFDL properties of an xs:group reference to a
group definition containing an xs:sequence or xs:choice.
dfdl:simpleType
xs:simpleType
Contains the DFDL properties of an xs:simpleType
dfdl:format
xs:schema
dfdl:defineFormat
Contains a set of DFDL properties that can be used by multiple
DFDL schema components. When used directly on xs:schema,
the property values act as defaults for all components in the
DFDL schema.
dfdl:defineFormat
xs:schema
Defines a reusable data format by associating a name with a set
of DFDL properties contained within a child dfdl:format
annotation. The name can be referenced from DFDL annotations
on multiple DFDL schema components, using dfdl:ref.
DFDL language – more advanced annotations
32
Annotation
Used on Component
Purpose
dfdl:assert
xs:element, xs:choice
xs:sequence, xs:group
Defines a test to be used to ensure the data are well formed.
Used only when parsing data.
dfdl:discriminator
Defines a test to be used when resolving a point of
uncertainty such as choice branches or optional elements.
Used only when parsing.
dfdl:escapeScheme
dfdl:defineEscapeScheme
Defines a scheme by which quotation marks and escape
characters can be specified. This is for use with delimited
text formats.
dfdl:defineEscapeScheme
xs:schema
Defines a named, reusable escape scheme. The name can
be referenced from DFDL annotations on multiple DFDL
schema components.
dfdl:hidden
xs:sequence
Defines a hidden element that appears in the schema for
use by the DFDL processor, but is not part of the infoset.
dfdl:defineVariable
xs:schema
Defines a variable that can be referenced elsewhere. This
can be used to communicate a parameter from one part of
processing to another part.
dfdl:newVariableInstance
Creates a new instance of a variable
dfdl:setVariable
Sets the value of a variable whose declaration is in scope
Links
 OGF DFDL home page: http://www.ogf.org/dfdl/
 DFDL 1.0 specification (pdf): http://www.ogf.org/documents/GFD.174.pdf
 DFDL 1.0 specification (html): http://www.ogf.org/dfdl/spec.php
 DFDL tutorial:
http://forge.gridforum.org/sf/docman/do/listDocuments/projects.dfdl-wg/docman.root.curr
 DFDL-WG GridForge project: https://forge.ogf.org/sf/projects/dfdl-wg
 DFDL Wikipedia page: http://en.wikipedia.org/wiki/DFDL
33
Agenda
 DFDL – a new standard for modeling text and binary data
 DFDL support in IIB
34
DFDL support in runtime and toolkit
 DFDL models
– Schema files reside in libraries
– Not in Message Sets
 DFDL domain and high-performance parser
– Available in nodes, ESQL, Java, …
– On-demand, streaming parser
– Use instead of MRM CWF/TDS
 Tooling for creating DFDL models in ITK
– Guided authoring wizards
– Language importer wizards
– DFDL graphical editor
 DFDL model debugger
– Debug parsing & writing of data in ITK
– No deploy necessary!
 DFDL schema deployed to broker in BAR file
– No dictionary file!
35
How does Message Broker use DFDL?
Message models
Schema files in Message Broker libraries
NOT Message Set projects
DFDL domain and model-driven parser
All nodes, just like other domains and parsers
Use instead of MRM CWF/TDS
• More capable and higher performing than MRM
Creating DFDL models
Guided authoring wizards
Language importer wizards
DFDL editor
Model debugger
Debug parsing / writing of data within MB toolkit
No deploy necessary!
Models deployed to broker in BAR file
No dictionary file!
36
Creating a DFDL model
New launcher
for creating
Message
Broker project
artifacts
 Or drop existing DFDL schema onto Message Broker library
37
Select the
type of data
you are
working with
Wizards simplify Model Creation
Guided authoring
Using the DFDL editor
Importing from other metadata
Already have DFDL schema
Select how you
want to create
the DFDL
model
38
Creating Model using Guided Authoring
Make
selection
Generated
model
39
Creating Model using the Editor
Outline
view
Logical
structure
view
DFDL
properties
view
Problem
s view
40
Testing a model within the editor
Parsed
‘infoset’
Run
parser
Sample
Parsed
data
data
41
No
deploy
needed!
Delimiters
highlighted
Hex view
Debugging a Model Test Failure
Parsed
‘infoset’
up to error
Object in
error
Error
messag
e
Model
and data
linked
Parsed
data up to
error
42
Trace
console
DFDL domain and parser
DFDL
domain
Specify
schema &
message
On Demand
or Complete
parsing
Streaming
capability
Validation
43
The DFDL parser is clever!

Recursive-descent parser with look-ahead to resolve uncertainty:
– Choice
– Optional element
– Variable array of elements

Speculatively attempts to parse data
– Until object is either ‘known to exist or not exist’
– Processing error causes the parser to
• Suppress error, back track and make another attempt

Example:
– Tries 1st branch of a choice, but gets a processing error
– Back tracks and tries the 2nd branch, which succeeds

Discriminator annotation can assert that object is ‘known to exist’
– Prevents incorrect back tracking

44
Initiator (tag) can assert ‘known to exist’
Using IBM DFDL in other environments
 Allow the Java IBM DFDL classes to be used outside MB
 Update to MB license to permit IBM DFDL classes to be copied to remote
locations and used from there by application programs
– Similar idea to CMP API classes
 IBM DFDL classes would be fully supported in remote location
Java
Java
Application
Application
45
IBM
IBMDFDL
DFDL
for
forJava
Java
<Document>
<Document>
<Element name=“myInt” dataType=“xs:int”
<ElementdataValue=“5”/>
name=“myInt” dataType=“xs:int”
dataValue=“5”/>
<Element name=“myFloat” dataType=“xs:float”
<ElementdataValue=“-7.1E08”/>
name=“myFloat” dataType=“xs:float”
dataValue=“-7.1E08”/>
</Element>
</Element>
</Document>
</Document>
Demo ….. Import COBOL → DFL
Carsten Mai
IBM Integration Bus
Message Flows
Message Flows
Output target
(Failure)
Input source
Output target
 Reusable
 Scalable
 Transactional
Transform
48
Output target
Message Flows
Transaktional
enthält die Verarbeitungs- und Datenmanipulationsoperationen
produziert alle Ausgaben (und konsumiert die Eingabe) oder keine – abhängig vom
Verarbeitungserfolg
Multithreaded
Verarbeitung eines Flows in einem Thread.
Additional threads können definiert werden → erhöhter Durchsatz
bei Spitzenlast können additional Threads genutzt werden, die bei Inaktivität an
Pool zurückgegeben werden.
Verschachtelung und Verkettung
Wiederverwendbarkeit durch Subflows und Verketten individueller Flows
49
Applications und Libraries
developing, deploying and managing your integration solutions
Application
• means of encapsulating resources to solve a specific
connectivity problem
• application can reference one or more libraries
Library
• a logical grouping of related routines and/or data
• libraries help with reuse and ease of resource management
• library can reference one or more libraries
 These concepts span all aspects of the Toolkit and broker runtime, and are
designed to make the development and management of WMB solutions easier.
Services
 developing, deploying and managing your service-oriented integration
solutions.
Service
• An application with a well-defined interface
 Service interface is expressed via WSDL with a port type
 Service interface gives *service* application a structure!
• Services promote encapsulation and isolation for serviceoriented integration solutions
• Service can reference one or more libraries
• Binding must be specified for successful deployment
 default binding is created out of the box
 Services span all aspects of the Toolkit and broker runtime, and are designed to
make the development and management of service-oriented solutions easier in
WMB.
Nodes
 Bausteine der Message Flows
 Jeder Node Type erfüllt eine bestimmte Aktion
(Input, Output, Verarbeitung,...)
 Viele unterschiedliche Typen
– gruppiert in Kategorien im Message Flow Editor
– ca. 100 Typen out-of-the-box
52
Node Terminology
node
error
terminal
output
connectors
input
connector
input
message
tree
53
input
terminal
Action
output
terminals
output
message
trees
Lots of Nodes are Built in [1]
54
Lots of Nodes are Built in [2]
 weitere Node-Typen mit Produkt-Erweiterungen und SupportPac erhältlich
– z.B. WebSphere TX, Tibco RV, VSAM, QSAM
 Sie können Ihre eigenen User-Defined Nodes entwicklen (in C oder Java)!
55
Demo …..eine Art 'Hello World'
Mediation Patterns - Examples
Interaction
Patterns
Transformation
& Routing
Patterns
Composite
Patterns
Examples
57
One-way
Protocol
transform
Pub/Sub Event Prop.
Transform
+
Canonical
Adapter
Request /
Response
Enrich
Gateway
Dispersion /
Aggregation
Log/Monitor
+
Route
Distribute
+
Transform – Log - Route
Correlate
Transformation Pattern - Transform
Enterprise Service Bus
Service
Provider
Service
Requestor
Transformation Patterns
Transform
58
Enrich
Log/Monitor
Routing Patterns
Route
Distribute
Correlate
Message Transformation

Konvertieren eines (logischen) Message-Formates in ein anders

Grafisch, easy
to use

Drag&Drop
Felder und
Folder,
Funktionen
59


Konvertieren von
XML in anderes
Format
Nutzt standard
XSL Style sheets

ESQL

Java

Leistungsfähige
Transformation

XPath-Anweisungen

Integration von .NETProgrammen

PHP

XPath-Anweisungen
Grafischer Mapping Node
IBM Graphical Data Mapper (GDM)
Visuelles Mappen und Transformieren der Source- in die Targetdaten
• 'Code-free', hoch performant und skalierbar
GDM für das gesamte IBM Produkt-Portfolio designed
• z.B. Vollständige Kompatibilität der Map mit MDM Server V9
• Mapping Script Language (MSL) Format
grafisches Mapping – einfach und leistungsfähig
Transformationen auf Feldebene oder mit komplexen Strukturen
• Conditionals (if then else),
• Loops (for each),
• Functions (target = f(source))
• & more!
Datenbanken als Mapping Source und Target in Routing und Enrichment Scenarien)
Ergänzt und unterstützt vorhandene Transformations-Sprachen
• 'user defined transformation' in Java, SQL, XPath, .NET…
einfaches Deployment
Maps innerhalb einer BAR-File oder separat
JIT compile
• Source deploy + runtime compilation = enhanced performance
60
Deep Integration for .NET
 New node for native .NET program invocation
– Call .NET programs directly via CLR V4; includes app domains for isolation
– C#, VB .NET (COM), JScript & F# programming available natively in MB
• Extensive range of .NET data types supported for easy integration
 Integrated Visual Studio Development
– Create .NET nodes in Visual Studio; Native MB assemblies simplifies process
• e.g. packages, templates, #using, debug, content assist etc
– Visual studio compiled resources available without redeploy
• Broker .NET watcher loads App Domain with most recent assembly
 Call .NET programs from new and existing MB nodes
– Toolkit can introspect .NET assemblies to dynamically discover available methods
• Automatically create appropriate language signatures to simplify invocation
– (e.g.) ESQL can now directly invoke .NET programs using simple procedure call
 .NET performance compares favourably with native ESQL & Java transformations
– .NET resource statistics show behaviour partitioned by app domain (calls, storage…)
61
Transformation Pattern - Enrich and Augment
Service
Provider
Service
Requestor
Database, File,
or Static data
Transform
62
Enric
h
Log/Monitor
Routing Patterns
Route
Distribute
Correlate
Enrich and Augment
IF Body.Order.Date < ‘2008/01/01’ THEN
INSERT INTO Database.OldOrders (LastName,Item,Quantity)
VALUES (Body.Order.Name.Last,
Body.Order.Item,
Body.Order.Quantity);
ENDIF;
Output
message
Look up
employee
00002
(Key not
found)
Look up
employee
00002
Output
message
DB Select
DB Insert
/Update
Database Input Node
 Database Input node
– Triggered & polled Queries um Message Flow zu starten
– Supports Database Update Technologien:
• Changes (Insert Update, Delete) in Database Tables --> Database Trigger Records in Event
Table
Event
Table
1.Database
trigger
1.Get
Event
1.Notification
1.Table
update
User
Table
1.Get
Data
Database
Processor
1.Propagate
Data
 Nutzt Data Tools für 'Code free' oder SQL basierte Queries
– RAD Tools erlauben das graphische Erstellen der Database Input Node Properties
– Power user können Notification, GetEvent, RetrieveData, PresentData & EndEvent SQL routines
spezifizieren
 Operational Control
– Unterstützt alle Message Broker databases
– MB Explorer Resource Statistics pro Database
64
Transformation Pattern – Log or Monitor
Service
Provider
Service
Requestor
Database, File,
or Static data
Transform
65
Enrich
Log/Monitor
Routing Patterns
Route
Distribute
Correlate
Monitoring und Auditing
 Erzeugen von Monitoring und Audit Events im Message Flow
– Einsicht in Anwendungs- und Service-Verbindungen
– Business Monitoring & Intelligence und Auditing
Monitor
Analyze
 Administration und Entwicklungszeit-Konfiguration
– jeder MB Node hat einen ‘Monitor’-Tab um Events zu konfigurieren
– Transaction: Start, End, Rollback ausgelöst durch Input Nodes
– Terminal: von jedem Terminal an jedem Node
– Konfigurierbar: Payload Data, Inhalt, Idendität, Korrelations- & Sequencing Daten
– Non-invasive: Monitoring-Profile können vorhandenen Flows zugefügt werden
 Steuerung im Betrieb
– Ein-, Ausschalten und Ändern der Event-Erzeugung:
•
mqsichangeflowmonitoring Kommando
– Events werden zu Topics per MQ-Transport publiziert
– excellente Performance-Charakteristik
 Integration mit anderen Produkten
– WebSphere Business Monitor
– Überwachen und Analysieren von KPIs
– Ausführliches Beispiel enthalten
– WBMTM
– Audit, Repair, Replay
66
Routing Pattern - Route
Service
Provider
Service
Requestor
Transform
67
Enrich
Log/Monitor
Routing Patterns
Route
Distribute
Correlate
Routing and Distribution Patterns
Mediation Patterns – Common Composite Pattern
Example
Service
Provider
Service
Requestor
Database, File,
or Static data
+
Canonical Adapter
Protocol+Data
Transform
69
+
+
Transform – Log - Route
Managed File Transfer
 Unterstütze MFT Produkte
– MQ Managed File Transfer (MQ FTE)
– Sterling Connect:Direct
Business
Partners
Sterling
Enterprise
Apps
Message
Broker
 Input nodes
– Message Flow startet Verarbeitung sobald der FTE Agent einen abgeschlossenen Filetranfer anzeigt
– Metadata im LocalEnvironment erlaubt intelligente Verarbeitung
– File kann als ganzes oder als einzelne Sätze verarbeitet werden (wie File Node)
 Output nodes
– Erzeugt File und initiiert Transfer
– LocalEnvironment ermöglicht dynamisches Setzen der Transferparameter und Metadaten
 FTE & C:D Agents werden automatisch Installiert, Konfiguriert und Gemanaged
70
IMS Node
 IMS Request node
– Message Flows können IMS Transactionen aufrufen und Antworten verarbeiten
– Typische Scenarien: Web Service->IMS, File->IMS, SAP->IMS…
– Hochperformant, synchron, Multi-platform Zugriff auf IMS
– ergänzt MQ IMS Bridge & IMS Web Services
 Synchroner Aufruf von IMS Transactions und Command
– Unterstützt viele IMS Facilities
•MPP, BMP und Fast Path Transaction Regions
•Commit mode 0, 1
•SyncLevel NONE, CONFIRM
•Single & multi segment IMS messages
 nutzt IMS TM Resource Adapter
– Integraler Bestandteil von Message Broker
•IMS Connect ist erforderlich
71
CICS Request Node
 CICS Request Node
–
–
–
–
Message Flows können CICS Programme/Transaktionen aufrufen und Antworten verarbeiten
Typische Scenarien: WebService->CICS, File->CICS, SAP->CICS, IMS->CICS
Hochperformanter, synchroner, Multi-platform Zugriff auf CICS
ergänzt MQ CICS Bridge und Web Services
 Synchroner Aufruf von CICS Programmen
– Specification von COMMAREA Request & Response bzw. Channel Container
– Timeouts, APPLIDs, SYNCONRETURN ebenso verfügbar
 Security
– Link level via SSL, und Message Level Identity via security profiles
 Flexibler Topology Support
– 2 Tier: MB->CICS
– 3 Tier: MB->CTG->CICS
• CTG ermöglicht zusätzliche HA und WLM Optionen
72
Leistungsfähige Anwendungsentwicklung
 Umfassende AD Perspektive
– unterstützt alle AD life cycle Schritte
– Visuell konsistent mit Integration
Explorer
 Create, modify, delete Nodes&Servers
– Create local Node & remote connect
– Create local & remote Servers
– Start & stop Nodes & Servers
– Autokonfigurieren/Konfigurieren der
debug Ports
– Automatisches Deployment beim Unit
test (.mbtest)
 Schnelles Compile, Debug, Test (CDT)
– Debug local & remote
– Drag & drop Message Flows direkt
– Test message flow direkt oder .mbtest
– Drag & drop BARs vom FileSystem
– Synchrones Deployment Status Log
73
IIB Patterns
74
Programmierbare Message Flows
Message Flows per Programm generierbar
Kompletter Flow-Lebenszyklus per Java API verfübar
• Create flow, add/remove node, change properties
• erweitert CMP API um Deployment & Konfiguration
Ein API für MBTK, MBX, commands, end users & 3rd Party
Message Broker API
Ein API ermöglicht Erzeugen und Managen aller IIB Ressourcen
z.B. Umbenennen eines Message Flow
File msgFlow = new File("main.msgflow");
MessageFlow mf1 = FlowRendererMSGFLOW.read(msgFlow);
mf1.setName(mf1.getName()+"Generated");
75
Simplify Management mit REST
CMP
HTTP
MQ
REST based management API
– Complements & compatible with existing CMP interface
– HTTP client can manage MB independent of CMP
• Includes interface for message record & replay
REST
Broker
GET /admin/eg/MYEGNAME HTTP/1.1
From: machine@ibm.com
User-Agent: MyApp/1.0
URI for all MB Resources
– ATOM data format for payload describes MB resources & related entities
– Intuitively mapped to MB artefacts
– Natural navigation of MB resources
• e.g. Execution group document contains EG properties & per-message flow ATOM feed
HTTP/1.1 200 OK
Date: Sun, 1 Oct 2011 21:46:59 GMT
Content-Type: text/html
Content-Length: 426
<?xml version="1.0" encoding='utf-8'?>
<service xmlns=http://www.w3.org/2007/app xmlns:atom="http://www.w3.org/2005/Atom">
<workspace>
<atom:title>Execution group feeds</atom:title>
<collection href="http://my.broker.com/admin/eg/MYEGNAME/resources" >
</collection>
</workspace>
<executiongroup description.long="" description.short="" ... >
</executiongroup>
</service>
Fully open interface can be exploited by 3 rd party tools
– HTTP REST/ATOM formats published & maintained for use by external users
– Enables widgets, mash-ups & other situational applications
76
Web Administration for Universal Access
77
Record & Replay
 Record, Edit und Replay von In-flight Daten
record
edit
– umfassendes Audit von Messages, Web, ERP, File,..
– Flexible Topologie: ein oder mehrere Nodes für recording, capture & replay
replay
 Data Recording, Capture & Store
– Grafisches Konfigurieren der Aufzeichnung inclusive der kompletten oder Teilen der Daten
– Datenquellen momentan beschränkt auf MB-Flows (incl. MB6.1, MB7 & MB8)
– Aufzeichnen von Events auf *jedem Node* lokal oder remote
• jeder Server kann als 'capture agent' fungieren
• 'Configurable service' definiert topic und Target-Database
– Agent speichert Daten in einer der unterstützten Datenbanken
• Oracle, DB2, SQL Server, Sybase, Informix…
 Web Tooling für View, Query data
– Editors für View, Query Payloads
– Web Administration & Capture
 Replay - Redelivery oder Flow Reprocessing
– Replay ausgewählter Daten in Flows oder Applicationen
– MB admin konfiguriert logische Ziele
• Zuordung zu physischem Protocol, z.B. MQ: {Qmgr, Q}
– Nutzer wählt Ziel aus Drop-down Liste
78
Cached Data – WebSphere eXtreme Scale
 Built-in facility to share data between multiple brokers
 Supports highly availability and scalable data sharing
 Improves mediation response times for reused data
 No extra moving part – runs within IIB itself
Cache shared between flows, integration servers &
nodes
– Seamless access to global cache from all
message flows
“elastic scaling”
Typical scenarios include multi-broker request-reply
and multi-broker aggregation
– Improve mediation response times and
dramatically reduce application load
Fully integrated with IIB Monitoring and Reporting tools
– shows cache interactions and other relevant
statistics
– Activity log shows cache agent operations for
write and read
No additional WXS license required for IIB use
“elastic caching”
79
Worklight mobile services patterns
 Mobile enable any enterprise service in 2 clicks !
 Build robust solutions with integrated caching and security
 Push data to mobile users from enterprise applications
 Create end-to-end mobile solutions for Microsoft .NET

●

Four new IIB development patterns
– Fully integrated in Integration toolkit
– Generate Worklight adapter, test application and
supporting IIB flows
Simple Service to mobile – WSDL based
1. Resource access from mobile - Controlled access
to enterprise data as a resource: methods for
read create, update and delete
2. Microsoft .Net service pattern
3. Queue based Push Notification
Patterns exploits value add IIB capability including
policy decision point for access authorisation and
WebSphere eXtreme Scale for performance
+
80
Why run IIB on z/OS? The golden rules…
• IIB on z/OS takes full advantage of the z/OS Sysplex
– The only platform supporting MQ Shared Queues for high-availability and
optimized “pull” load-balancing with the very best in scalability
• IIB on z/OS utilizes many of the other inherent features of z/OS






RACF security
WLM resource optimisation
SMP/E installation control
SMF performance recording
RRS transaction management for speed and reliability
ARM restart
• IIB on z/OS offers several specific interfaces to the z/OS platform
– QSAM and VSAM nodes
– Using WMB on z/OS will typically be much more efficient if some of the
interfaced applications are co-located on z/OS
• Running on the z/OS and System z platform brings all the security
and robustness of the the mainframe platform
 Looking for the highest scalability and high-availability for a strategic corporate
ESB
81
Demo … Web Service Facade für CICS Programm
82
IBM Integration Bus
 Universal Connectivity
– Simplify application connectivity to provide a flexible and dynamic infrastructure
 Routes and transforms messages FROM anywhere, TO anywhere
– Supports a wide range of protocols
• MQ, JMS 1.1, HTTP(S), Web Services, File, EIS (SAP,SEBL…), TCP/IP, User Defined
– Supports a broad range of data formats
• Binary (C/COBOL), XML, Industry (SWIFT, EDI, HIPAA…), User Defined
– Interactions and Operations
• Route, Filter, Transform, Enrich, Monitor, Distribute, Decompose, Correlate, Detect…
 Simple programming
– Patterns based for top-down, parameterized connectivity of common use cases
• Web Service façades, message oriented processing, queue to file…
– Construction based for bottom-up assembly of bespoke connectivity logic
• Message flows to describe application connectivity comprising…
• Message nodes which encapsulate required integration logic which operate on…
• Message tree which describes the data in a format independent manner
– Transformation options include graphical mapping, PHP, Java, ESQL, XSL and WTX
 Operational Management and Performance
– Extensive Administration and Systems Management facilities for developed solutions
– Wide range of operating system and hardware platforms supported
– Offers performance of traditional transaction processing environments
83
Carsten Mai
IBM Integration Bus
Mapping Node
Agenda
 Mapping Node overview
 Structured and functional mapping
 Array processing
 XPath 2.0
 Message assembly processing
 Database mapping
 User-defined functions
85
New Mapping Node for Message Broker Version 8
 Improved usability
– Simple learning curve
– Simple transformations are simple to create
– Complex transformations build on concepts previously mastered
– Visual debugger support with breakpoints, single step, variables view
 Improved capability
– Builds upon ‘best of breed’ mapping technologies in IBM products
– Full XPath 2.0 scripting support
• Standards-based language designed for hierarchical data structures
 Improved performance
– Dedicated runtime engine
86
Broker overview – flows, nodes, message tree
Parser
associated with
bit-stream
Parser invoked
(on demand)
bit-stream  tree
Parser invoked
(in reverse)
tree  bit-stream
Input node reads
bit-stream
Node accesses logical
message tree
Output node writes
bit-stream
Transform
Output
Input
87
Creating a new map
 New maps are created by double clicking on a
mapping node, or from the project view
 Dialog box launched to select source and
target definitions
– DFDL, XSD, Message Set
 New map is opened in mapping editor
88
Specify source and target types
 Select root elements
 Supports:
– Message Sets
– XSD schemas
– DFDL schemas
89
The Mapping Editor
90
The Mapping Editor
Input / Source tree
91
The Mapping Editor
Output / Target tree
92
The Mapping Editor
Properties editor
93
The Mapping Editor
Transform
94
The Mapping Editor
Toolbar
95
The Mapping Editor
Name
+ Expand
- Collapse
96
Type
Occurrence
The Mapping Editor
Find elements
anywhere in the tree
Click <Show all>
To remove filter
97
The Mapping Editor
Help system
- progressive disclosure
98
The Mapping Editor
 Source and target trees
– Shows structure of the message using an expandable tree
– Each element shows name, repeatability (minOccurs..maxOccurs), and type
– If the list of elements is too long, then the user can ‘filter’ the list of displayed elements
(useful also for searching a large schema)
 Transform
– Defines what to do with the data between the source and the target
– Drop down list contains available transforms appropriate to source and target types
 Properties view
– For customising the nature of the transforms
99
Mapping the Message Broker tree
 The mapping node understands the WMB ‘Message Assembly’ structure
 A new map provides default mappings for the ‘Properties’ header
– Message ‘Set’, ‘Type’ & ‘Format’ fields are automatically assigned values
– All other fields are copied from the input message Properties header
 The target ‘domain’ (parser) is set in the properties page
 Full support for
– All transport headers (MQMD, RFH2, JMS, HTTP, etc)
– LocalEnvironment tree
Creating mappings
 Drag and drop source to target (can also drag target to source)
 Choose a transform (how to map)
– Defaults to most commonly used transform appropriate to the source and target
 Basic transforms
– Move
• Simple copy of source value to target value
• Can be used if target is same type (simple or complex) as source
• If source and target are repeating element, then there is an implicit loop
• If target is optional (according to schema), then element is only created if the source
exists
– Convert
• For casting values between different source and target simple types
– Assign
• For giving a target a fixed value (no source)
• Type the value in the Properties view
Creating mappings
Map navigation
Breadcrumb trail
Up one level
Down one level
Function Transforms
 The target value can be computed by applying a function to one or more inputs
 Large function library from XPath 2.0
 String manipulation
– E.g. concatenation, sub-string, matching, find/replace, regex
 Numeric calculation
– E.g. counting, summing, rounding, min/max etc
 Date/time processing
– E.g. creating timestamps, extracting components of date/time, duration processing
Structured Mapping
 Maps can be broken down into a hierarchy of ‘nested’ maps using one of the structural
transforms
– Local
• Map a complex source to a complex target in a separate ‘page’
• Reduces clutter in the mapping editor
– For each
• Iterate over a repeating source element (array)
– Append
• For appending one array to the end of another
– Join
• For merging the data from two or more arrays
– Submap
• Similar to local, but the mapping is defined in a separate map file and is reusable
 These transforms create a structural container between complex elements
– Drill down to create the mappings between the children of these elements
– Good practice to use ‘Local’ to avoid clutter in large maps
Conditional Mapping
 Individual transforms can be configured to occur only if a condition is met
– ‘Condition’ tab on properties page
• User enters an XPath 2.0 predicate
• Content assist available
• XPath 2.0 builder available
• Mapping is only performed if predicate evaluates to ‘true’
Conditional Mapping
 If/Else transform
– Allows ‘else’ condition to be mapped as well
– ‘Grouped transform’ containing as many else-if conditions as necessary
– This is a structural mapping
• Each clause contains a nested map
Each condition is individually mapped
XPath predicates are
set in the properties view
Navigate into nested map
To configure transformations
Array Processing – Cardinality property
 Cardinality
– If the source is an array (repeating element – simple or complex), can specify which
elements in the array (by index, starting at 1) are mapped
Array Processing - Looping
 ‘For each’ transform – loops over each element source element in the repeating structure
and produces the same number of target elements
– How the structure gets mapped is determined by the nested map – click on the transform
to drill down into the nested map
 If you don’t want to process all of the element in the input array
– Use the ‘Cardinality’ tab in the properties page to specify which elements to include.
• More than one index can be specified using a comma separated list – 1, 3, 5, 8
• A range of indices can be specified with a colon – 1:5, 8:* (* - means end)
– Use the ‘Filter inputs’ tab to specify an XPath 2.0 filter expression
 Sorting the elements in the array
– Use the ‘Sort’ tab in the properties page to specify how to sort the input array
Advanced Array Processing
 Append transform – concatenates the contents of one array to another
– Has two (or more inputs), one output. The order in which the inputs are processed
(appended) is specified in the ‘Order’ tab in the properties page.
 Join transform – merges contents of two (or more) arrays together.
– Arrays are correlated by specifying a join condition (XPath 2.0 predicate)
– Analogous to the ‘INNER JOIN’ of a relational database
Automap
 Automates the task of producing
transformations between source
and target elements of the same
(or similar) names
 Useful for working with large
schemas
XPath 2.0
 The expression syntax used by properties in the mapping node
 Specifies location paths for a tree structure
– E.g. $Order/Item[1]/Description
 Type system based on XML Schema (and hence DFDL)
 Rich function library
 Expressive syntax supporting complex calculations with minimal code
 Read-only ‘functional’ language – no side effects
 ‘Variables’ refer to elements in the input tree
 High level programming constructs including ‘for’ loops, ‘if-then-else’ conditions
 http://www.w3.org/TR/xpath20/
XPath Predicates
 Any XPath expression that returns a Boolean result
 Used by many of the transform properties, e.g.:
– The ‘condition’ in an ‘If’ or ‘Move’ transform
• count($Item) > 5
– The join criteria in a ‘Join’ transform
• $Book/ISBN = $Author/ISBN
Custom XPath Transform
 Transform the input to the output using any legal XPath 2.0 expression
1.6*sum($Item/(Price * Quantity))
Custom XPath Transform – Array Processing
 In XPath 2.0, the array (sequence) is a first class type
 (‘Message’, ‘Broker’, ‘Mapping’, ‘Node’)
– is an array of 4 strings
– This could be assigned to a repeating target element
(‘Message’, ‘Broker’, ‘Mapping’, ‘Node’)
<string-val>Message</string-val>
<string-val>Broker</string-val>
<string-val>Mapping</string-val>
<string-val>Node</string-val>
Custom XPath Transform – Array Processing
 for $var in $array return <expr>
for $item at $index in $string-val
return concat($index, ': ', $item)
<string-val>aaa</string-val>
<string-val>bbb</string-val>
<string-val>ccc</string-val>
<string-val>ddd</string-val>
<string-val>1:
<string-val>2:
<string-val>3:
<string-val>4:
aaa</string-val>
bbb</string-val>
ccc</string-val>
ddd</string-val>
Inputs – Primary and Supplementary
 Primary inputs are directly involved in the transformation – they contain the data used to
produce the output
 They can also be used in the properties of the transform
– E.g. The ‘condition’ property – whether the mapping should occur
– The input is assigned to a variable which is referenced in the XPath predicate
• e.g. $Price
 Supplementary inputs can be added to a transform
– Not used directly in the transformation
– But can be referenced in the transform properties
– E.g. The Move transform can copy its data from one (primary) input element to the
target, if a predicate based on another (supplementary) input evaluates to true.
– Can have as many supplementary inputs as required
Supplementary Inputs on Structural Transforms
 Supplementary inputs are used to pass elements into a transform without affecting the
primary purpose of that mapping
 They can be used to pass extra data into a nested map
 ‘For each’ transform – iterates over the Primary input
– Supplementary inputs (both repeating and non-repeating) can be passed into the ‘For
each’.
– However, they don’t affect the ‘looping’ behaviour of the ‘For each’ transform.
– But the nested map displays these inputs on the LHS where they can be used as primary
inputs to mappings
Message Assembly – Headers and LocalEnvironment
 By default, the Properties header and message body are available to map
– All other headers are copied from input to output unchanged
– LocalEnvironment, ExceptionList and Environment trees are passed through unchanged
 All transport headers (MQ, JMS, HTTP, etc) can be added to the map and transformed.
 Headers can also be added and removed
 LocalEnvironment tree can be added to the map and transformed
Message Assembly – Adding and Deleting Headers
Message Assembly – Adding and Deleting Headers
 Selected headers are added into the map
editor.
 Headers in the input message that don’t
appear in the target map are copied to the
output message.
 Headers not in the input message can be
added to the output message by adding them
to the target map and mapping to them.
 Headers in the input message can be deleted
by adding them to the target map and not
mapping to them.
Casting elements
 xsd:any elements can be cast to any simple
or complex type
 Other elements can be cast to a derived
type
 Use to map variables in LocalEnvironment
 Use to map header and body in SOAP
message
Database Mapping
 Data can be extracted from a relational database
– Currently support DB ‘SELECT’
 Can select fields from DB tables based on data in the input message
 Graphical SQL ‘WHERE’ clause generator
– Can contain parameter placeholders to reference the input message tree
– Parameter values specified using XPath
 The result set of the SELECT is presented as a source tree in the map editor
Database Mapping – JDBC configuration
 Create JDBC Configurable
Service
– Using MQ Explorer or
command line
 Set security credentials
– mqsisetdbparms
 Toolkit and Runtime both
connect to the DB using JDBC
Database SELECT
Custom ‘User-Defined’ Transforms
 As well as writing custom transformation logic using XPath 2.0, other languages will be
supported for creating user-defined functions
– Java
• Ubiquitous object oriented programming language
• Large skills base
• Extensive function library
– ESQL
• Broker’s domain-specific language
• Based on SQL
• Familiar to many Broker users
Invoking Java - configuration
 Open the ‘Java’ perspective
 Create a new Java project in Package Explorer
 (Optional) Create a new Java package
 Create a new Class
 Import Java class(es) into map…
Invoking Java methods – Simple Types
public static String formatPrice(BigDecimal price) {
NumberFormat format =
NumberFormat.getCurrencyInstance();
String result = format.format(price.doubleValue());
return result;
}
XPath 2.0 expression
Invoking Java methods – Arrays and Complex Types
public static double total(MbElement[] items, String[] initials)
throws MbException {
double amount = 0;
// some algorithm...
return amount;
}
Invoking ESQL functions
CREATE FUNCTION stringToBlob (IN field character) RETURNS BLOB
BEGIN
RETURN CAST(field AS BLOB CCSID 1208);
END;
Invoking Java/ESQL methods – from an XPath
expression
 All user-defined functions are callable from XPath expressions
– Including all map properties that use XPath
Summary
 New v8 Mapping Node
– Improved usability, capability and performance
 Hierarchical mapping editor sub-divides complex transformations
– Drag and drop between source and target
– Select appropriate transform type (defaults to most commonly used)
– Configure advanced options in property editor
 XPath 2.0 allows complex transformation logic
– Provides large built-in function library
– Extensible via user-defined functions – Java, ESQL
– W3C open standard
 Interact with Databases

The Mapping Editor

Transcription

Similar documents