17
IBM Analytics © 2015 IBM Corporation 1 © 2015 IBM Corporation DB2 Analytics Accelerator Version 5.1 New Functions and Capabilities Patric Becker, Frank Neumann, Ute Baumbach Analytics on z Systems CoE, Boeblingen Laboratory November 2015 © 2015 IBM Corporation 2 Legal Disclaimer © IBM Corporation 2015. All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. © 2015 IBM Corporation 3 DB2 Analytics Accelerator V5.1 – Functional Highlights Version 5.1 of DB2 Analytics Accelerator opens up a new dimension of analytical processing by introducing accelerator-only tables and in-database analytics . Accelerator-only tables can benefit statistics and analytics tools that use temporary data for reports. The high velocity of execution enables these tools to quickly gather all required data. Accelerator-only tables enable acceleration of data transformations implemented via SQL statements. Storing interim results in accelerator-only tables enables subsequent queries or data transformations to process all relevant data on the accelerator with high speed In-database analytics capabilities enable acceleration of predictive analytics applications. This enables SPSS/Netezza Analytics data mining and in-database modeling to be processed within the Accelerator.

DB2 Analytics Accelerator V5.1 Technical Update.pdf

Embed Size (px)

Citation preview

Page 1: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 1

© 2015 IBM Corporation

DB2 Analytics Accelerator Version 5.1New Functions and Capabilities

Patric Becker, Frank Neumann, Ute Baumbach

Analytics on z Systems CoE, Boeblingen Laboratory

November 2015

© 2015 IBM Corporation2

Legal Disclaimer

• © IBM Corporation 2015. All Rights Reserved.

• The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM’s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

• References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

© 2015 IBM Corporation3

DB2 Analytics Accelerator V5.1 – Functional Highlights

� Version 5.1 of DB2 Analytics Accelerator opens up a new dimension of analytical processing by introducing accelerator-only tables and in-database analytics .

� Accelerator-only tables can benefit statistics and analytics tools that use temporary data

for reports. The high velocity of execution enables these tools to quickly gather all required data.

� Accelerator-only tables enable acceleration of data transformations implemented via SQL statements. Storing interim results in accelerator-only tables enables subsequent queries or data transformations to process all relevant data on the accelerator with high

speed

� In-database analytics capabilities enable acceleration of predictive analytics

applications. This enables SPSS/Netezza Analytics data mining and in-database modeling to be processed within the Accelerator.

Page 2: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 2

© 2015 IBM Corporation4

DB2 Analytics Accelerator V5.1 – Functional Highlights (contd.)

� System Temporal and Bitemporal tables support (DB2 11 only)

� Active and history table loaded to Accelerator

� Timestamp(12) data type now supported

� Truncation to timestamp(6), new options for ZPARM QUERY_ACCEL_OPTIONS

� Queries using temporal SQL expressions can be routed to Accelerator

� Encryption of data-in-motion

� Encrypt network traffic between z Systems and the Accelerator using Internet Protocol Security (IPsec).

� Network traffic includes:

� DRDA traffic (e.g. Queries, Table Loads)

� Configuration Console traffic

� Incremental Update traffic

� Requires configuration and enablement on z Systems and on the Accelerator

� Incremental update operation enhancements

� Ability to disable query acceleration automatically for suspended tables

© 2015 IBM Corporation5

DB2 Analytics Accelerator V5.1 – Functional Highlights (contd.)

� Improved installability and maintainability

� Extended compatility between stored procedure version level and accelerator version level

� Enhanced product packaging

� Further supported SQL functions: Datetime arithmetic functions, Inlined scalar UDFs (DB2 11 only)

© 2015 IBM Corporation6

Details of new functions - Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

Page 3: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 3

© 2015 IBM Corporation7

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation8

Introducing Accelerator-only table type in DB2 for z/OSCreation (DDL) and access remains through DB2 for z/OS in all cases

Non-accelerator DB2 table

• Data in DB2 only

Accelerator-shadow table

• Data in DB2 and the Accelerator

Accelerator-archived table / partition

• Empty read-only partition in DB2

• Partition data is in Accelerator only

Accelerator-only table (AOT)

• “Proxy table” in DB2

• Data is in Accelerator only

Table 1

Table 4

Table 3

Table 2Table 2

Table 4

Table 3

© 2015 IBM Corporation9

In-database transformation and multi-step processing

� DB2 Analytics Accelerator introduces Accelerator-Only tables to store intermediate or final results of data transformation or multi-step processes

� Accelerator-Only tables enable complex data transformations taking place on DB2 Analytics Accelerator, called “In-database transformation”

� Accelerator-Only tables enable intermediate and final results of reporting or other processes to be generated and stored on DB2 Analytics Accelerator, called “In-database multi-step processing”

� Advantages:

� Accelerate in-database data transformations and data movement processes

� Reduced need of data movement processes to other platforms for data transformation purposes

� Enables multi-step reporting on the Accelerator

� Saves disk space and CPU cost on z Systems currently used for transformations and reporting steps

� Allow data preparation steps for data mining and other advanced analytics to execute on the Accelerator

� Store a set of data in DB2 Analytics Accelerator only, not on DB2 for z/OS, without

using the High Performance Storage Saver functionality

Page 4: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 4

© 2015 IBM Corporation10

Accelerator-only tables – Technical basics

� AOTs are created and dropped using DB2 DDL statements (CREATE; DROP)

• Accelerator must be started

• QUERY ACCELERATION behavior may have any value during CREATE/DROP

• Syntax:

� CREATE TABLE MYTABLE (...) CCSID ccsid IN ACCELERATOR <ACCEL1>;

� DROP TABLE MYTABLE;

� Recommended to create a database in DB2 to be used for the AOTs

• CREATE TABLE MYTABLE (...) CCSID ccsid IN ACCELERATOR <ACCEL1> IN

DATABASE MYDB;

• Usual authorization necessary to create objects in database

� SELECT and INSERT/UPDATE/DELETE operations using AOTs can only run on the Accelerator

• QUERY ACCELERATION behavior must be set to ENABLE/ELIGIBLE/ALL

• Accelerator-shadow tables, Accelerator-archived tables and other AOTs can be used in the same statement

� Dynamic and static SQL can be used with AOTs

© 2015 IBM Corporation11

Accelerator-only tables – Technical basics

� There are some restrictions for using certain DDL against AOTs

� For example: ALTER TABLE, RENAME TABLE

� Restrictions on clauses/keywords used in DDL statements

� For example: Generated, Partition By, WITH DEFAULT

� Restrictions on data types, column types, usage and code pages

� LOB columns

� LOCK TABLE, reference to MQTs, TRUNCATE

� In general, the same data type restrictions as found in normal accelerated shadow DB2 tables

� AOT counters in IFCIDs 2 (Statistics) and 3 (Accounting)

� See the IBM DB2 Analytics Accelerator for z/OS documentation for a full list

© 2015 IBM Corporation12

Accelerator-only tables – Technical basics

� Queries using AOTs can only run on the Accelerator

� QUERY ACCELERATION behavior must be set to ENABLE/ELIGIBLE/ALL

� INSERTS, UPDATES and DELETES to AOTs can only run in the Accelerator

� QUERY ACCELERATION behavior must be set to ENABLE/ELIGIBLE/ALL

� INSERT FROM SELECT into an AOT

� Can only run on the accelerator

� All tables in the SELECT must reside in the accelerator

� Accelerated shadow DB2 tables

� Archive tables

� AOTs

� QUERY ACCELERATION behavior must be set to ENABLE/ELIGIBLE/ALL

� Cross-Loader to load data from AOTs into DB2 for z/OS

� Different table names� PI49378 may be needed: WHEN USING DB2 CROSSLOADER WITH SET CURRENT QUERY

ACCELERATION, THE SPECIAL REGISTER IS NOT APPLIED TO THE QUERIES THAT ARE RUN

Page 5: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 5

© 2015 IBM Corporation13

Multi-step reporting applications with DB2 for z/OS BEFORE Accelerator-only tables: Report processing on DB2, source data might reside on the Accelerator already

Temporary results

2

Temporary results

1Temporary results

Reports and Dashboards

n

Multi-Step Report

n

1

2

Reporting Application

Credit Card

Transaction History

Customer

Summary Mart

Credit CardTransaction History

CustomerSummary Mart

© 2015 IBM Corporation14

Multi-step reporting applications with DB2 for z/OSWith Accelerator-only tables: Temporary objects and processing on the Accelerator

Temporary results

2Temporary results

1Temporary results

Reports and Dashboards

n

Multi-Step Report

n

1

2

Reporting Application

Credit CardTransaction History

CustomerSummary Mart

Credit CardTransaction History

CustomerSummary Mart

Data for transactional and analytical processing

© 2015 IBM Corporation15

In-database transformationUsing Accelerator-only tables and ELT logic in the Accelerator

Transaction Processing Systems (OLTP)

Analytics

Advantages: • Simpler to manage

• Better performance and reduced latency Data for transactional and analytical processing

Customer Transactions

CustomerData

Customer Transaction Summary and History AOTs

Customer Summary Mart AOTs

Customer Transactions

CustomerData

ELT logic

Page 6: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 6

© 2015 IBM Corporation16

Data scientist work areaUsing Accelerator-only tables for ad-hoc analysis

Transaction Processing Systems (OLTP)

Data for transactional and analytical processing

Customer Transactions

CustomerData

Customer Transactions

CustomerData

Work database John

Work Area AOTs

Work database Bob

Work Area AOTs

Data Scientist John

Data Scientist Bob

© 2015 IBM Corporation17

Integrate more data sources for analyticsLoad external data to the accelerator and save combined analytic results in accelerator-only tables

Transaction Processing Systems (OLTP)

AnalyticsData for transactional and analytical processing

Customer Transactions

CustomerData

Combined result in AOTs

Customer Transactions

CustomerData

Related data from

other sourcesExternal

Files

Imagecopy

DB2 Analytics Accelerator Loader

© 2015 IBM Corporation18

• Deeper insight into operational status through faster reportingSupport multistep reporting applications. Third party BI reporting suites, QMF, home grown applications, etc.

• Accelerated campaign tuning for IBM Campaign (Unica)Improved performance for iterative campaign tuning.

• Simplifying data-transformation processesDelivering in-database transformation within DB2 Analytics Accelerator

• Data mart consolidation through flexible data infrastructure Host data marts on z Systems, where the data originates

• Deeper insight into customers and marketsData scientist work area

• Simpler data integration with DB2 Analytics Accelerator Loader for z/OS (load non-DB2 for z/OS data)Assimilate more data sources for analytics to shorten development cycles and speed integration efforts

• In-database analytics to accelerate predictive analyticsImproves the quality of models, speeds calculations and delivers real-time, actionable business processes

What can I do with AOTs?

Page 7: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 7

© 2015 IBM Corporation19

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation20

In-database analytics

Enable acceleration of predictive analytics applications

In-database analytics enables SPSS/NetezzaAnalytics (INZA) data mining and in-database

modeling to be processed within the Accelerator

� Accelerates SPSS/Netezza Analytics (INZA) data mining and in-database modeling

� Allows frequent model refreshes to enable adequate scoring

� Reduced need of data movement processes to other platforms for predictive analytics purposes

� Supports the full lifecycle of a real-time analytics solution on a single,

integrated system, combining transactional data, historical data, and predictive analytics

© 2015 IBM Corporation21

In-database analytics – Technical basics

� Set of stored procedures of IBM Netezza In-Database Analytics Package (INZA) are available on the Accelerator for modeling purposes, including:

� Decision Tree

� Regression Tree

� Naive Bayes

� K-means Clustering and TwoStep Clustering

� Stored procedures use accelerator-shadow tables or accelerator-only tables as input and create accelerator-only tables as output

� DB2 for z/OS contains stored procedure wrappers to enable invocation of the

stored procedures from SPSS Modeler 17

� Actual stored procedure execution takes place on Accelerator

� Separate install of the IBM Netezza Analytics stored procedures required on the Accelerator:

� http://www-01.ibm.com/support/knowledgecenter/SS4LQ8_5.1.0/com.ibm.datatools.aqt.doc/installmanual/concept/c_idaa_inst_analytics.html?lang=en

Page 8: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 8

© 2015 IBM Corporation22

In-database analytics – Technical basics

� SPSS Modeler 17 uses the following INZA stored procedures that now run on the Accelerator:

� dectree - Builds a Decision Tree model by growing and pruning a tree

� grow_dectree - Builds a Decision Tree model

� predict_dectree - Applies a Decision Tree model to generate classification predictions

� prune_dectree - Prunes a previously built Decision Tree model

� regtree - Builds a Regression Tree model by growing and pruning a tree

� grow_regtree - Builds a Regression Tree model

� prune_regtree - Prunes a previously built Regression Tree model

� predict_regtree - Applies a Regression Tree model to generate regression predictions for

a dataset

� naivebayes - Builds a Naive Bayes model

� predict_naivebayes - Applies a Naive Bayes model to generate classification predictions

for a dataset

� kmeans - Builds a Clustering model that clusters the input data into k centers. The

centers are calculated as the mean value of the nearest input data records

� predict_kmeans - Applies a K-means Clustering model to cluster records of a dataset

© 2015 IBM Corporation23

In-database analytics – Technical basics

� two_step - Builds a TwoStep Clustering model that first distributes the input data into a

hierarchical tree structure according to the distance between the data records, then

reduces the tree into k clusters. A second pass over the data associates the input data

records to the next cluster

� predict_twostep - Applies a TwoStep Clustering model to score records of a dataset

� split_data - Randomly splits the input data into two separated subsets

� pmml_model - Stores the given analytics model as PMML document to a table

� export_pmml - Exports the given analytics model as PMML document to a file, or it

exports a model from a PMML table to a file. If no PMML table exists containing the

PMML document for this model, one can be created automatically when requested.

Optionally, instead of writing to a file, the result can be returned by the procedure.

� model_exists - Checks if the given model exists. The model can be searched in the current

or in another given database.

� drop_model - Drops the given model. All managed tables of this model are also dropped.

© 2015 IBM Corporation24

In-database analyticsData preparation (using AOTs) and SPSS modeling in the Accelerator

Transaction Processing Systems (OLTP)

With embedded scoring

Advantages: • Allows fast model refreshes

• Ensures adequate scoring

• Better performance and reduced latency

Data for transactional and analytical processing

Customer Transactions

CustomerData

Customer TxnData Prep AOTs

Customer Transactions

CustomerData

Modeling

ModelModelSPSS Modeler

Page 9: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 9

© 2015 IBM Corporation25

source(select)

process process typenode

modeling nugget score output

Step 1 DB2 Analytics Accelerator Query Acceleration

Step 2 Step 3 In-database transformationand analytics

Without Accelerator: The SPSS source node selects the input data from DB2 z/OS without any acceleration, then all data preparation is done in SPSS Modeler, followed by the modeling algorithm in

SPSS modeler, then the model is scored and scoring results are inserted into a DB2 z/OS result table

Source tables are accelerator-shadow tables and the accelerator processes the (complex)select statement

Step 1

Data preparation is pushed down into the accelerator producing an accelerator-only table for thetype node

Step 2

INZA modeling algorithm is called and executed in the accelerator on an accelerator-onlytable

Step 3

Data Source Access and Preparation Modelling Scoring

Source datasource process process t

(select) (select) Result data

In-database analyticsData preparation (using AOTs) and SPSS modeling in the Accelerator

With Accelerator:

© 2015 IBM Corporation26

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation27

Temporal Table support in DB2 Analytics Accelerator

� With V4 PTF 5

� Application(Business)-period temporal tables

� With V5.1, but for DB2 11 only

� System-period temporal tables

• Active table

• History table: contains the historical versions of the rows in the active

table

• Uses TIMESTAMP(12) for Begin and End columns

� Bitemporal tables

• Contains both Business Time temporal and System time temporal

definition

Temporal Table Concept: DB2 supports time-based data management that allows you to insert, update, delete, and query data in the past, the present, and future while keeping a complete history of what you knew and when you knew it.

Page 10: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 10

© 2015 IBM Corporation28

Timestamp(12) support in DB2 Analytics Accelerator

� With DB2 Analytics Accelerator V5.1 tables with timestamp(12) columns can be added

and loaded to the Accelerator with the truncation of timestamp(12) to timestamp(6).

• Impact: Queries running on the Accelerator return different results than on DB2 for z/OS for these columns

� Introduction of two new options 5 and 6 for ZPARM QUERY_ACCEL_OPTIONS to

control the truncation of timestamp precision > 6.

• Option 5: queries referencing system temporal tables are eligible.

• Option 6: queries referencing an expression or regular (non system temporal) table

columns with a TIMESTAMP(12) data type are eligible.

Netezza NPS doesn't support timestamp(12). Workaround: MAPPING timestamp precision 12 to timestamp precision 6 on the Accelerator.

© 2015 IBM Corporation29

System Time Temporal Query Routing

� Both active and history tables with Timestamp(12) can be loaded to the Accelerator

� Queries on system temporal tables are routed to the Accelerator when ZPARM

QUERY_ACCEL_OPTIONS is set to 5 or 6

• Special query rewrite is applied for the following 3 temporal SQL:

• FOR SYSTEM_TIME AS OF expr

• FOR SYSTEM_TIME FROM expr1 TO expr2

• FOR SYSTEM_TIME BETWEEN expr1 AND expr2

� For Bitemporal tables the support is straightforward since both System Time and Business Time temporal tables anc clauses are supported

© 2015 IBM Corporation30

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

Page 11: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 11

© 2015 IBM Corporation31

Encryption of data …

� Encryption of data “at rest”

� Already available by default with N3001 self-encrypting disks (SEDs)

� Addresses risk of stolen disks

� Encryption of “data-in-motion”

� Encrypt network traffic between z Systems and the Accelerator using Internet Protocol Security (IPsec)

� Network traffic includes:

� DRDA traffic (e.g. queries and results, table loads)

� Configuration Console traffic (passwords)

� Incremental Update traffic

� Requires configuration and enablement on z Systems and on the

Accelerator

� Addresses network sniffing (interception of TCP/IP packets)

© 2015 IBM Corporation32

Do you need encryption of data-in-motion ?

� Recommended setup is private data network, also see http://www-01.ibm.com/support/docview.wss?uid=swg27038935

• This is still the best and recommended alternative to ensure security

• Private data network has no performance impact (> 2 TB/hour load thruput)

� Sample domains for encryption of data-in-motion:

• Some customers have multiple data centers and connect the remote IDAA over a non-

private network (routed, accessible from intra-net)

• Some customer have compliance rules (security policy) to encrypt all network traffic

� However Ipsec

• Is complex to administer/setup

• Reduces network bandwidth

• All incoming traffic to one accelerator combined is currently limited to approx. 700-900 GB/hour

� Get help from z/OS Communications Server L2/L3 if you are not a network expert!

© 2015 IBM Corporation33

Encryption of data in DataStudio

Icon Description

Disks are not encrypted.Network traffic is not encrypted

Disks are not encrypted.Network traffic is encrypted for at least one z/OS system.

Disks are encrypted.Network traffic is not encrypted.

Disks are encrypted.Network traffic is encrypted for at least one z/OS system.

One of the certificates used for network encryption is expiring in 17 days.orOne of the certificates used for network encryption has expired.

Click on details link

shows details about certificates and keys

Click on details link

shows details about certificates and keys

Page 12: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 12

© 2015 IBM Corporation34

Encryption, keys, certificates

It is more complex than this• Security Association (SA) with lifetime

• Authentication phase is implemented by a component called "IPsec Key Exchange" (IKE), needs to be encrypted as well

• Key and certificate management• Configuration restart required for an initially

unencrypted connection

Public/private keys are used to encrypt and decrypt dataCertified by trusted certificate authority (CA)

© 2015 IBM Corporation35

Prereqs for z/OS LPAR and Accelerator

� z/OS

• Communication server V 1.13 or higher or V2.1 or higher supporting IKEV2

• Set up and configure the TCP/IP stack of communication server with IKED (=Internet

Key Exchange daemon), NSSD (=Network security services daemon), policy agent (=manages policy definitions and dependent services), TRMD (=Traffic Regulation Management daemon), SYSLOGD (=system logging daemon)

• Implementing IPsec in the z/OS communications server - depending on the prior experience with that setup - can be a lengthy, complex process that requires planning and preparation. The system programmer performing the configuration steps on the

z/OS side should be familiar with z/OS communications server

� Accelerator (only N200x (Striper) and N3001 (Mako) models supported)

• Host management version 5.4.1.0 or higher incl. libreswan (installed by IBM service for

existing systems)

� Network

• For existing firewalls in the private data network: configure IKED port 500 the protocols

50 and 51.

© 2015 IBM Corporation36

Configuration sequence

1. Generate a key pair and certificate for the z/OS side and activate it in the keyring owned by NSSD

2. Generate a key pair and certificate for the accelerator peer and export it into a PKCS#12 binary file. Store the file in the USS directory used for IDAA software updates

3. Transfer the PKCS#12 certificate file to the accelerator: IDAA Studio Accelerator View -> Transfer Update

4. Import the transferred certificate into the accelerator NSS database: IDAA configuration console -> (8) -(Menu) Manage Encryption of Data in Motion -> (1) - Import PKCS12 file with RSA keys and certificates

5. Restart the IPsec service to activate an imported certificate: IDAA configuration console ->(8) - (Menu) Manage Encryption of Data in Motion -> (7) - Restart the IPsec service

6. z/OS: Prepare the IPsec policy agent configuration for the LPAR

7. Enable an LPAR for IPsec on the accelerator: IDAA configuration console ->(8) - (Menu) Manage

Encryption of Data in Motion -> (3) - Enable encryption of data in motion

8. z/OS: Activate the prepared IPsec policy on the LPAR (Refresh policy agent)

9. Test your connection and diagnose problems

10. Confirm the IPsec configuration: IDAA configuration console -> (8) - (Menu) Manage Encryption of Data in Motion -> (4) - Confirm enablement of encryption

11. Verify that the network traffic is encrypted: IDAA configuration console -> (8) - (Menu) Manage Encryption of Data in Motion -> (6) - Display the status of encrypted connections

Configuration Details: http://www-

01.ibm.com/support/docview.wss?uid=swg27047011&myns=swgimgmt&mynp=OCSS4LQ8&mync=E&cm_sp=swgimgmt-_-

OCSS4LQ8-_-E

Known Issues: http://www-01.ibm.com/support/docview.wss?uid=swg27047150

Page 13: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 13

© 2015 IBM Corporation37

Console options for encryption of data and performance implications

Because current RedHat 6 Linux kernel allows only serial de/encryption (single threaded), throughput of Accelerator operations is currently limited to ~700 GB/h

for encrypted traffic. (Compared to 2-4 TB/h for unencrypted traffic)

Because current RedHat 6 Linux kernel allows only serial de/encryption (single threaded), throughput of Accelerator operations is currently limited to ~700 GB/h

for encrypted traffic. (Compared to 2-4 TB/h for unencrypted traffic)

© 2015 IBM Corporation38

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation39

Incremental update improvements in 2015

• V4.1 PTF 5 (May 2015)

• Suspending faulty tables from replication

• Improved incremental update performance

– Parallel Apply

– Automatic definition of distribution and organization keys

• V4.1 PTF 6 (4Q 2015)

• Continuous operation of replication

• V5.1 (4Q 2015)

• Disable query acceleration for suspended faulty tables

Page 14: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 14

© 2015 IBM Corporation40

Disable query acceleration for suspended faulty tables

� Available since V4.1 PTF5: “Suspend faulty table”

• Replication of tables may be suspended & mirroring continuous w/out table

• By default: “Suspend Faulty Tables” is on

• Issue: queries still accelerated while faulty table is not replicated anymore

� New capability

• Automatically disable query acceleration for faulty tables

• Generate DSNX message: notify query acceleration is disabled for table

• Configuration

• Off per default (to avoid breaking existing automation)

• Turn on in configuration console per subsystem / database system

© 2015 IBM Corporation41

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation42

Product packaging and installation enhancements and changes

Keep only applied software package files in HFS (more like SMP/E), automatic clean-up of older versions when SMP/E updates are applied.

Option to switch back to the old behavior so that old files are not deleted.

Keep only applied software package files in HFS (more like SMP/E), automatic clean-up of older versions when SMP/E updates are applied.

Option to switch back to the old behavior so that old files are not deleted.

For DB2 object dropping, re-creation and procedure bind:

New installation program AQTTRIN, invoked from job AQTTIJSP.

Performs required steps without step customization.

For DB2 object dropping, re-creation and procedure bind:

New installation program AQTTRIN, invoked from job AQTTIJSP.

Performs required steps without step customization.

V5.1 available as fully working GA version.

Before, we always requested to install PTF1 at GA time.

Release Notes: http://www-1.ibm.com/support/docview.wss?uid=swg27047096&myns=swgimgmt&mynp=OCSS4LQ8

V5.1 available as fully working GA version.

Before, we always requested to install PTF1 at GA time.

Release Notes: http://www-1.ibm.com/support/docview.wss?uid=swg27047096&myns=swgimgmt&mynp=OCSS4LQ8

AQT_INSTALL_PREFIX environment variable has been removed. Specify an absolute path in the environment variable AQT_HOST_PACKAGE_DIRECTORY. AQT_INSTALL_PREFIX environment variable has been removed. Specify an absolute path in the environment variable AQT_HOST_PACKAGE_DIRECTORY.

Page 15: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 15

© 2015 IBM Corporation43

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

© 2015 IBM Corporation44

Installation dependencies

� Sequence of applying updates is crucial

� New flexibility

• Software upgrade of server components when stored procedure level > Accelerator server level(starting with 5.1 stored procedures and accelerator server 4.1)

© 2015 IBM Corporation45

Agenda

� Accelerator-only tables

• Introduced as foundation in V4.1 PTF 5 for DB2 for z/OS 10 only

• Now available for DB2 for z/OS 10 and DB2 for z/OS 11

� Analytics Server, SPSS, INZA

� Temporal Tables

� Encryption of Data-in-motion

� Incremental Update

� Product Packaging Enhancements

� Stored Procedure and Component Dependencies

� Further SQL support

Page 16: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 16

© 2015 IBM Corporation46

Further SQL support for query acceleration

� For DB2 11 only, enabled with DB2 APAR PI42662 and NPS 7.2.1

• Datetime arithmetic functions support – adding and subtracting date, time, timestamp types

• Motivation: Different date, time representations within DB2 and NPS lead to datetime arithmetic incompatibilities

• DB2 uses yymmdd representation for data, NPS uses interval representation

• E.g. 115 means 1 month and 15 days on DB2, but 115 days in NPS

• Inlined scalar UDF support

• Definition: An inlined SQL scalar returns a single value each time it is invoked. The returned value is specified in the RETURN statement of the

body of the function.

© 2015 IBM Corporation47

Supported Datetime Arithmetic Examples

Arithmetic (addition and subtraction) with date, time, timestamp types

� SELECT DATEC1 - DATEC2 FROM IDAA.TIMETB FOR FETCH ONLY;

Arithmetic (addition and subtraction) with datetime constants:

� SELECT DATEC1 - '2013-02-01' FROM IDAA.TIMETB FOR FETCH ONLY;

Arithmethic with labeled durations (were already supported before) � SELECT DATEC - 1 MONTH FROM IDAA.TIMETB FOR FETCH ONLY;

© 2015 IBM Corporation48

DDL

CREATE FUNCTION

HCM.GMT_DATE (p_date DATE, p_time TIME, p_gmt_off_set BIGINT)

RETURNS DATE LANGUAGE SQL

DETERMINISTIC READS SQL DATA NO EXTERNAL ACTION

RETURN(

DATE (TIMESTAMP(p_date , p_time) + p_gmt_off_set seconds)

);

Query Reference

WHERE HCM.GMT_DATE( VIEW.DATE, VIEW.TIME, 0) BETWEEN DATE('2014-08-02') AND DATE('2014-09-01')

Inlined Scalar UDF Example

Page 17: DB2 Analytics Accelerator V5.1 Technical Update.pdf

IBM Analytics© 2015 IBM Corporation 17

© 2015 IBM Corporation49

Inlined Scalar UDF Restriction for Acceleration

� Inline scalar UDF DDL cannot be nested. Nested inline scalar UDF in a query is allowed though.

Not allowed

CREATE FUNCTION INSCUDF1(arg1 date)…;

CREATE FUNCTION INSCUDF2(arg1 integer)

RETURN arg1*DATE(INSCUDF1(CURRENT_DATE));

Allowed

SELECT C1

WHERE INSCUDF3(COL1,INSCUDF4(COL2)) > 12

© 2015 IBM Corporation50