INTEGRATING Z/OS DATA WITH LINUX AND HADOOP · INTEGRATING Z/OS DATA WITH LINUX AND HADOOP . Big...

Mike Combs, VP of Marketing 978-996-3580 • mcombs@veristorm.com

INTEGRATING Z/OS DATA WITH LINUX AND HADOOP

Big Data and Hadoop §  Volume, Velocity, Variety (3 Vs) §  Challenge traditional solutions §  Enables discovery, decision-making,

and process optimization

§  Discovery §  Exploratory, needs short iterations §  Traditional reporting: Often an

lengthy IT project that restructures data for periodic reports

2

Hadoop SQL

Scale-Out via parallel compute, I/O

Scale-Up

Key/value pairs (text docs, XML)

Relational tables

Functional programming Declarative queries

Offline Batch (write once, read many)

Online Transactions

MapReduce: Distributes jobs Hive: Access via SQL subset, not transactional HDFS: Distributes files

Increasing Needs for Detailed Analytics §  Baselining & Experimenting §  Parkland Hospital analyzed records

to find and extend best practices

§  Segmentation §  Dannon uses predictive analytics to

adapt to changing tastes in yogurt

§  Data Sharing §  US Gov Fraud Prevention shared

data across departments

3

§  Decision-making §  Lake George ecosystem project

uses sensor data to protect $1B in tourism

§  New Business Models §  Social media, location-based

services, mobile apps

http://www.veristorm.com/content/big-advantages-big-data

The Veristorm Solution 4

§  Mainframes process 60% of commercial transactions and contain 70% of all enterprise data

§  Proprietary data may contain insights unavailable to your competition

§  Logs provide visibility into behavior before and beyond transactions

Apps

Mobile

Things

Web Customers Transactions Proprietary App logs

System logs

Near real-time access to mainframe data provides a 360° view for better insights and decisions.

Long Services Engagements

Technical Challenges

COBOL copy books Compressed data SQL usage Staging build-out Aggregation

Staging

Transform

Load

Hadoop

Security? Governance? Agility? TCO?

System z EAL 5

IMS

VSAM

DB2

5

vStorm Enterprise 6

§  Point-and-click software data copy §  No programming required. COBOL

copy books, compressed data handled automatically

§  Target: File system or HDFS/Hive §  No staging build-out: 33% storage

savings §  No ETL: Avoid MIPS usage charges §  Near real-time access §  For security and compliance sensitive

data, data can be kept on mainframe

z/OS Linux

Logs

IMS

VSAM

System z Mainframe

DB2

Linux

vStorm Enterprise

zDoop Big Insights, Mongo DB,

Cloudera, Splunk, Pentaho,

Hortonworks, Red Hat

Hadoop

7

8

Authenticate for z/OS source

9

Browse tables and contents on z/OS

10

Authenticate for Linux HDFS destination

11

Select destination for copy

12

13

Copybooks are no problem

14

Direct to HIVE for SQL manipulation or

analytics

Unlock New Insight and Reduce Cost §  Do More §  Analyze large amount of information

in minutes §  Offload batch processes to IFLs to

free up batch window

§  Reduce Cost §  Take advantage of IFL pricing and

Linux ecosystem

§  Application extensibility achieved through newly available skillset §  Linux, Java, Python, Hadoop §  Bring dev teams together

15

System z Workloads

Batch Real-time

Stop by our booth: #106 Ask for a free trial: http://www.veristorm.com/content/free-trial-

vstorm-enterprise

Thank you! 16