IBM z Analytics

IBM z Analytics

ANY Data for ANY ApplicationExploring IBM Data Virtualization Manager for z/OS

in the era of API Economy

Francesco Borrello | francesco.borrello@it.ibm.com

IBM z Analytics Data Virtualization Manager

Traditional Data Integration Inadequate

No longer Viable to Move Data(using extract, transform, load ETL)

• Risk to data security

• Data inconsistency

• Rigid, limits business agility

• High cost and latency

IBM z Analytics Data Virtualization Manager

A New Approach is Needed – Data Virtualization

a data services layer that integrates data and content on demand from disparate sources in real time

provides a layer of abstraction that shields developers from unique data implementation

provides a metadata catalog to keeps track of data, location, availability, and state

supports transactions that write back to the original data sources regardless of location

doesn’t mandate a API or schema - supports “many to many”

Unlike traditional data integration�

data virtualization creates virtualized and integrated views of disparate

data in-memory for immediate read/write access rather than waiting

on moving the data and holding the integrated views in an EDW

IBM z Analytics Data Virtualization Manager

What is expected of Data Virtualization?

IBM z Analytics Data Virtualization Manager

What is expected of Data Virtualization?

Provide ANY data for ANY application

Cloud Mobile

Analytics/Search ESB, ETL

Transactional DataData Virtualization

IBM z Analytics

Data Virtualization on IBM Z

IBM z Analytics Data Virtualization Manager

� accelerate mainframe modernization initiatives involving Big Data

� gain real-time business insights across z/OS and enterprise data

� eliminate coding of complex z/OS apps via built-in APIs and

interfaces

• ensures data is secure and in-place with real-time data virtualization

• supports Hybrid architectures (on-premises, Cloud, Hadoop and MF)

• broad API support via SQL, NoSQL, SOAP, and REST via z/OS Connect EE

• z/OS resident optimization for improved performance and TCO

• abstraction layer for improved productivity and business agility

IBM Data Virtualization Manager for z/OS

The industry’s only Z-resident data

virtualization solution!

Virtualize z/OS data with other enterprise data sources in

real-time without data movement to provide

comprehensive information that is readily consumable by

analytics, cloud and cognitive applications

Take action now!

IBM z Analytics Data Virtualization Manager

IBM Data Virtualization Manager for z/OS

IBM z Analytics Data Virtualization Manager

Runs Without A Processing Penalty

WLM

Enclave

WLM

Enclave

Network I/O

Hybrid SRB/TCB

SRB/TCB

Mode Switch

API

WLM

Enclave

Network I/O

Hybrid SRB/TCB

SRB/TCB

Mode Switch

API

WLM

Enclave

WLM

Enclave

Data I/O

Hybrid SRB/TCB

SRB/TCB

Mode Switch

API

WLM

Enclave

Data I/O

Hybrid SRB/TCB

SRB/TCB

Mode Switch

API

zIIP

Ability to route up to

99% of data

processing time to

zIIP

Ability to route up to

99% of data

processing time to

zIIP

Ability to integrate

mainframe data in

place with

transformation

running on zIIP

Ability to integrate

mainframe data in

place with

transformation

running on zIIP

IBM z Analytics Data Virtualization Manager

IBM DVM Address Space

Pro

vid

er I/O

Hyb

rid

SR

B/T

CB

Pro

vid

er I/O

Hyb

rid

SR

B/T

CB

Pro

vid

er I/O

Hyb

rid

SR

B/T

CB

Co

ns

um

er I/O

Hyb

rid

SR

B/T

CB

Co

ns

um

er I/O

Hyb

rid

SR

B/T

CB

Co

ns

um

er I/O

Hyb

rid

SR

B/T

CB

zIIP

Map/Reduce Implemented on z/OS

Data Consumer

SQL

ODBC/JDBC

NoSQL

(JSON)

Web

Services

(SOAP/REST)

Data Provider

Db2, ADABAS, IMS

IDMS, VSAM

Physical

Sequential, SMF,

Sys Logs , MQ

Db2 LUW, SQL

Server, Oracle

Write to ClientWrite to Client

Request for DataRequest for Data

MapReduce LDU

MapReduce LDU

MapReduce LDU

MapReduce LDU

MapReduce LDU

MapReduce LDUBuffer

IBM z Analytics

Adabas Db2 for z/OS

IMS DB IDMS

SEQ files VSAM

SMF records Log Streams

Virtual Tape zFS

Db2 WarehouseDb2 Warehouse

on Cloud

Db2 Informix

MAINFRAME

z/OS Systems

Distributed

More than just data access )

JDBC/ODBC

REST AP/SOAPI

NoSQL

Access to all data from all applications

without

need to know which format the data it is in

using the API of your choice

IBM Data Virtualization

Manager for z/OS

IBM z/OS Connect EEIBM z/OS Connect EE

IBM z Analytics

Selective direct Distributed Relational Database Architecture support

IBM Data Virtualization

Manager for z/OS

Oracle

Data

Provider for DRDA

(Linux)

Oracle

SQL

ServerMicrosoft

Service for DRDA

Apache

Derby

DRDA

IBM z Analytics Data Virtualization Manager

Enabling APIs for Data on IBM Z

CICS

IMS

WAS

MQ

DB2

� Agile web, mobile, cloud access via REST to

mainframe applications such as CICS, IMS, WAS*

CICS

IMS

WAS

MQ

DB2

z/OS Connect

Enterprise Edition

Trusted zone

Load b

ala

ncer

Inte

lligent

Conte

nt

routing

Gate

way /

Data

Pow

er

Security

layer

DMZ Inner APIsOuter APIs

Discovery Services

REST Services

REST API

Consumers

IBM z Analytics Data Virtualization Manager

Enabling APIs for Data on IBM Z

CICS

IMS

WAS

MQ

DB2

Data

Virtualization

Manager for z/OS

Adabas

IMS DB

IDMS

SMF

VSAM

� Agile web, mobile, cloud access via REST to

mainframe data such as Adabas, IMS, VSAM*

CICS

IMS

WAS

MQ

DB2

z/OS Connect

Enterprise Edition

Trusted zone

Load b

ala

ncer

Inte

lligent

Conte

nt

routing

Gate

way /

Data

Pow

er

Security

layer

DMZ Inner APIsOuter APIs

Discovery Services

REST Services

REST API

Consumers

DVM Non z/OS

IBM z Analytics

Automated generation of code to access data from Spark &

Python

� Reduce coding of data access via automated generation of Scala code

(Python in progress)

� Automatically optimized based on available parallelism to generate

best performance results without manual coding

IBM z Analytics

16