SAP IQ Roadmap Webinar February 27 2014

© 2013 SAP AG. All rights reserved. 1Public


Agenda

Welcome

Agenda• Introduction to Dobler Consulting• SAP IQ Roadmap – What to Expect• Q&A

Presenters• Courtney Claussen - SAP IQ Product Management • Peter Dobler - CEO Dobler Consulting

Closing


Introduction to Dobler Consulting

Dobler Consulting is a leading information technology and database services company, offering a broad spectrum of services to their clients; acting as your Trusted Adviser, Provide License Sales, Architectural Review and Design Consulting, Optimization Services, High Availability review and enablement, Training and Cross Training, and lastly ongoing support and preventative maintenance. Founded in 2000, the Tampa consulting firm specializes in SAP/Sybase, Microsoft SQL Server, and Oracle.

Visit us online at www.doblerconsulting.com, or contact us at 813 322 3240, or [email protected].

http://www.doblerconsulting.com/

mailto:[email protected]


Your Data is Your DNA, Dobler Consulting Focus Areas

Strategic Database Consulting

DBA Managed Services

Database Training Programs

SAP VAR D&T

Cross-Platform Expertise SAP Sybase® SQL Server® Oracle®


What’s Ahead

ISUG-TECH Annual Conference April 14-17, Atlanta

• Register at http://my.isug.com/conference/registration

• Early Bird ending 2/28/14 (free hotel room with registration)

SAPPHIRENOW Annual Conference June 3-5, Orlando

• Come visit our kiosk in the exhibition hall

http://my.isug.com/conference/registration

Courtney Claussen / SAP IQ Product Management

February 27, 2014

SAP IQ Roadmap Dobler Events Webinar

© 2014 SAP AG or an SAP affiliate company. All rights reserved. 2

Disclaimer

This presentation outlines our general product direction and should not be relied on in making a

purchase decision. This presentation is not subject to your license agreement or any other agreement

with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to

develop or release any functionality mentioned in this presentation. This presentation and SAP's

strategy and possible future developments are subject to change and may be changed by SAP at any

time for any reason without notice. This document is provided without a warranty of any kind, either

express or implied, including but not limited to, the implied warranties of merchantability, fitness for a

particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this

document, except if such damages were caused by SAP intentionally or grossly negligent.


Table of contents

Product Overview

SAP IQ product description

Strategic asset in SAP’s database portfolio

SAP IQ architecture

Product Road Map

Shared nothing

Extreme data volume

Enhanced administration

Summary

Product Overview Capabilities and positioning


SAP IQ product description High performance analytics with low TCO

10x – 1000x faster queries over

traditional DBMS

Supports heavy ad-hoc loads with

1000s of users concurrently

Store & analyze terabytes to

petabytes of structured &

unstructured data

Performance & scalability

Best-of-breed partners certified

with SAP IQ

Tighter integration with SAP

HANA, BusinessObjects BI

Supports 3rd party BI tools and

analytic applications

Integrated ecosystem

#1 disk-backed column DB for

reporting and analytics

Platform of algorithms, services &

APIs for big data analysis

Highly flexible deployment options;

supports all major hardware & OS

platforms

Trusted open analytics

server

See Appendix for abbreviations


SAP IQ in SAP’s database portfolio Strategic asset

SAP HANA Data Platform

SAP HANA

SAP

SQL Anywhere

Mobile

and embedded

database

Transactional

database with low

TCO

SAP ASE SAP IQ

Petabyte scale

analytics database

with low TCO

SAP ESP,

Replication Server,

PowerDesigner & EIM

Unified

EIM platform for

real-time

Open for

partners



Ecosystem

SAP IQ architecture A comprehensive analytics platform

Most mature

column store

Comprehensive

lifecycle tiering

MPP queries, virtual marts,

and user scaling

High-speed

loads

Structured and

unstructured store

Comprehensive ANSI

SQL with OLAP

Built-in full-text

search

In-database analytics with

MapReduce and simulator Web 2.0 APIs

Big Data open

source APIs

Optimized BI and EIM to

model and replicate

Development and

administration tools Predictive analytics

Packaged ILM

applications

Bradmark,

Symantec,

Whitesands, Quest,

and ZEND

SAS, SPSS, KXEN,

Fuzzy Logix,

Zementis

BMMSoft,

SOLIX, and PBS

SAP PowerDesigner,

SAP Replication Server,

SAP Data Services,

Panopticon

Application

services

DBMS Hadoop and

R

Product Road Map Improvements planned through the end of 2014


2009

Very large database (VLDB) platform foundation

2011

MapReduce API

2011

PlexQ MPP Foundation

2010

Full text search; Web 2.0 API

2009

In-database analytics

2013

• Extended

storage for

SAP HANA

• Shared

nothing

architecture

SAP Sybase IQ 16 SP08 and SP09 Path to Extreme Scale

2014

New generation

column store


SAP IQ 16.0 SP08 and SP09 Planned improvements through 2014

Shared Nothing

• DAS (Direct Attached Storage) Cache

• DAS Files

• DAS Files elasticity

Enhanced Administration

• Point in time recovery

• Integrated system management

Extreme Data Volume

• Extended storage for HANA

• Distributed load

• RLV (row level version) store on Multiplex

SAP IQ

Enhanced XLDB


SAP IQ 16 Key building blocks

SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine

N-bit and tiered indexing

Text search

Web-enabled analytics

Info

rma

tion

lifecycle

ma

na

ge

me

nt

Communications and security

Schema

Query engine

Column store

Hash partitioned tables

Distributed queries with data affinity

Database, column, and wire encryption

LDAP, Kerberos authentication, and RBAC

Re-engineered petascale disk store

In-memory, write-optimized delta store

Fully parallel

bulk loading


SAP IQ 16 SP08 and SP09 Key building blocks – new features highlighted in green boxes

SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading




SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading





SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading


• HANA extended tables • PITR – Point in Time Recovery




• DAS Cache extends data affinity

• DAS Files “shared nothing” architecture with elasticity

SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading








SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading








SAP IQ 16: engine

Re

silie

nt M

ultip

lex g

rid a

rch

itectu

re

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

Shared store

Loading engine


Text search


Info

rma

tion

lifecycle

ma

na

ge

me

nt


Schema

Query engine

Column store







Fully parallel

bulk loading



• RLV on Multiplex


Shared Nothing • DAS (Direct Attached Storage) Cache

• DAS Files

• DAS Files elasticity


High Speed Interconnect to Shared Store

Scale out Scale out

Elastic Logical Server

IQ Server

Data affinity

IQ Server

Data affinity

IQ Server

Data affinity

IQ Server

Data affinity

IQ Server

Data affinity

Elastic Logical Server

Scale out Scale out

IQ 16.0 – “Shared Everything” Multiplex with Data Affinity

SAP IQ Multiplex


IQ Server

Data affinity

IQ Server

Data affinity

IQ Server

Data affinity

Shared Store

DAS cache extends data affinity

• DAS (direct attached storage) Cache is a

mechanism for maintaining one or more

copies of primary data in direct attached

storage on different physical servers.

• The DAS cache is populated in a pull model

as new versions of data are accessed

• The DAS cache can offload bandwidth from

the SAN for subsequent data access

DAS cache DAS cache DAS cache

SAP IQ Multiplex


Multiplex with high speed interconnect

IQ Server

DAS Files – IQ adopts a “shared nothing” architecture

• DAS (direct attached storage) files are primary

direct attached storage devices that are being

shared across the cluster using the high-

speed interconnect.

• Primary data may exist in only one DAS file.

• These can be used in place of the SAN for

primary storage

• When data is loaded, it will be automatically

written to the appropriate DAS file that the

data has affinity to.

• With DAS files, DAS cache will still have value

for dealing with data access that cannot be

affinitized.

DAS Files DAS Files DAS Files

DAS

cache DAS

cache

DAS

cache

IQ Server IQ Server


DAS Files – mirrored for write safety

• DAS files are managed within a logical server

• Primary files are mirrored on a different server for data redundancy

• Optionally, user may create a DAS file without a redundant copy

Logical Server (Server_1, Server_2, Server_3, Server_4)

Server_1 Server_2 Server_3 Server_4

L1 M4 L2 M2 L3 M3 L4 M1

Primary Mirror


High speed interconnect

DAS Files elasticity

The IQ server redistributes data when nodes or files are added or removed

IQ

Server_3

NEW

IQ

Server_4

IQ

Server_2

IQ

Server_1

Empty


NEW

IQ

Server_4

IQ

Server_3

IQ

Server_1



DAS Files – does not preclude a shared store

• SAP Sybase IQ continues its flexible

architecture.

• An IQ database may be comprised of both

DAS DBSpaces and shared DBSpaces.

• Shared store could be used for data that is

needed within each node, without duplication.

Optional shared store

DAS Files DAS Files DAS Files

DBFile DBFile DBFile

DAS

cache

DAS

cache

DAS

cache

IQ Server IQ Server IQ Server

Extreme Data • Extended storage for SAP HANA


• RLV (row level version) store on Multiplex


SAP IQ is the preferred solution for warm and cold data archiving

capabilities for SAP BW, Business Suite, and HANA

NLS (Near Line Store) for SAP BW

• Optimized NLS data transfer throughput using IQ Loader functionality

• SAP HANA and IQ share the same columnar paradigm, and similar data compression rates

• Ready for large data volumes

• Suitable for ad-hoc queries with long history

• Minimum administration effort

ILM for SAP Business Suite

• Fast archive index read w/o additional secondary DB indexes

• Increased search capabilities

• Faster archive I/O – fewer layers (software, network, storage hardware)

• ERP archive files as well as archive indexes stored in SAP IQ

Warm archive for HANA using smart data access (SDA)

• Using SDA, HANA customers can access IQ as a federated store

• Store warm data in IQ, and real time data in HANA

• Push query processing down to IQ


HANA extended storage brings integrated warm data management to

SAP HANA

NLS Store for SAP BW

ILM for SAP Business Suite

Warm archive for HANA, federated via SDA

Extended storage for SAP HANA

• Handle all data seamlessly via HANA tables with Calc View support

• Transformative ROI – enable new insights from data processing at speed and scale not possible before

• Steady performance across data volumes, variety, velocity with scalable/elastic capacity

• Practical for both on-premise and hosted models

• Native Big Data solution allows customers to gain real-time insights by cost-effectively managing and analyzing ALL enterprise data

Cold/warm data archival

Warm Data Management for SAP HANA


HANA extended storage Requirements from our customers

• Manage data cost effectively, yet with desired performance based on SLAs

• Application defines which data is “hot”, and which data is “warm”

• Handle very large data sets – terabytes to petabytes

• Update and query all data seamlessly via HANA extended tables – IQ store is transparent to

applications

• Provide a native Big Data solution that covers most enterprise use cases without Hadoop

Table

Table

Queries and

updates initiated

from HANA

SAP HANA

SAP IQ

Extended table (access via HANA)

29 © 2014 SAP AG or an SAP affiliate company. All rights reserved.

BW Process and Data Management

SAP HANA

Master Data, InfoCubes, DataStore Objects (DSO)

SAP IQ Near Line

Store (NLS) SAP Extractors, Data Services, SLT

Replication

SAP and non-SAP data

sources

NL

S

inte

rfac

e

Persistent

Staging Area

(PSA)

Corporate

Memory

BW on HANA with extended storage in IQ

SAP IQ

Reads

Bulk load

Smart Data Access (SDA) SDA SDA


DISTRIBUTED LOADS SCALED OUT DATA LOADING

Value proposition

Optimize performance of massive bulk loads

Exploit large memory and core footprints

Architectural considerations

SAP IQ base table data ingestion is distributed.

Bulk loads scale out for the best possible load

performance.

Load speed should scale almost linearly when more

cores / nodes are added.

User may control whether a load gets distributed or not.

Graphical plans for loads will be enhanced for

distributed loads.

Multiplex

Shared Store

Client

IQ Server IQ Server IQ Server IQ Server

Logical server

Bulk load

to IQ table


ROW LEVEL VERSION DELTA STORE FOR MULTIPLEX HIGH VELOCITY DATA LOADING ON IQ CLUSTER

Multiplex

IQ Writer

Server

IQ Store

Memory RLV

store

Clients

High velocity,

concurrent writes

IQ Writer

Server

Memory RLV

store

IQ

Reader

Server

periodic

merge

periodic

merge

Clients

High velocity,

concurrent writes

Value proposition

Continuous analytics over operational data

High velocity, concurrent data modifications

across all writer nodes in a Multiplex

Exploit large memory and core footprints in a

clustered deployment

Architectural considerations

Each writer node in a Multiplex may manage its

own RLV store

All RLV stores are fully recoverable with

dedicated transaction logs on each writer node

Query engine sees a consistent view of data

across RLV stores and main IQ store

Enhanced Administration

• Point in time recovery



POINT IN TIME RECOVERY

Point in time recovery:

• Database backup performed at 6:00pm on Monday

• Disk failure occurs on Thursday at 3:00pm

• User wants to recover database state on Thursday at noon: • Restore Monday backup

• Replay transaction log from time of Monday backup until Thursday at noon – apply committed transactions, and roll back uncommitted transactions

Backup at

midnight on

Monday

Monday Tuesday Wednesday Thursday

3:00pm Sunday

Server crashed

and database is

corrupt

Thursday

Database transaction log U = Update B = Backup

D = Delete C = Commit

I = Insert K = Checkpoint

Op I D U C I D K B I U K D C I K I C I D C K U I C Crash

2a 6a 11a 4p 3a 2p 5p 6p 11p 1a 8a 3p 9p 3a 10a 7p 11p 4a 10a 1p 8p 9a 11a 1p 3p Time

Recover to noon


INTEGRATED SYSTEMS MANAGEMENT

SAP Control Center (SCC) supports

up to 100 servers of:

SAP ASE

SAP IQ

SAP Replication Server (SRS)

SAP Event Stream Processor (ESP)

SAP Mobile Platform (SMP)

* Emerging in future:

SAP HANA

SAP Real-Time Data Platform (RTDP)

SAP SQL Anywhere (SA)

Support up to 250 servers

SCC >Admin >Monitor

>Alert

HANA*

RTDP*

SA*

SMP

ESP

SRS

IQ

ASE


SAP CONTROL CENTER

PLANNED ARCHITECTURAL IMPROVEMENTS

• Creation of a new datacenter management

feature which will easily allow an

administration group to identify the overall

health of the servers within the datacenter

and quickly drill into those servers.

• Provide integration of the datacenter features

with existing landscape components

registered to the LMDB

• Modify the existing Flex based management

console to UI5 to enable a better user

experience for ‘on board’ features.


Landscape view

Show nodes or server statuses, for example SCC Heat Chart:


RS HANA ASE

RS IQ ASE

ESP HANA

SA ASE

Topology view

Show connectivity or relationship, for example SCC Topology View (future and current):

Summary


Summary

SAP IQ is a strategic technology asset

for SAP’s database portfolio, and

continues its innovative roadmap

with:

Shared Nothing MPP – Optional shared

nothing MPP configuration supports cloud

deployments with more elastic

architecture.

Extreme Data Volume – New extended

store feature for HANA, distributed loads,

and RLV store on Multiplex address the

extreme data volume and velocity

requirements of big data.

Enhanced Administration – Improved

management features help today’s

database administrators perform their jobs

more efficiently and confidently.

This is the current state of planning and may be changed by SAP at any time.



SAP IQ Resources Where to get information on SAP IQ

• SAP IQ SCN Developer Site:

http://scn.sap.com/community/developer-center/analytic-server

• Free Express and Enterprise Edition Downloads

https://www.sap.com/iq-downloads

• “How to” Tutorials

http://scn.sap.com/docs/DOC-43300

• SAP IQ 16 (video):

https://www.youtube.com/watch?v=Bbcl8uY3tj4

• SAP IQ for Big Data(video):

https://www.youtube.com/watch?v=XRNgVPxiyVw

• SAP IQ as Near-line Storage (video):

https://www.youtube.com/watch?v=FdtdAhTQYSg

• SAP IQ for BusinessObjects (video):

http://www.youtube.com/watch?v=A4hR0n2y8zQ













https://www.youtube.com/watch?v=Bbcl8uY3tj4


https://www.youtube.com/watch?v=XRNgVPxiyVw

https://www.youtube.com/watch?v=FdtdAhTQYSg

http://www.youtube.com/watch?v=A4hR0n2y8zQ

© 2014 SAP AG or an SAP affiliate company. All rights reserved.

Thank you

Courtney Claussen, Director of Product Management, SAP IQ

[email protected]


Acronym Full Text

API Application Programming Interface

ANSI American National Standards Institute

BI Business Intelligence

DAS Direct Attached Storage

EDW Enterprise Data Warehouse

EIM Enterprise Information Management

XLDB Extreme Large Database

ILM Information Lifecycle Management

LDAP Lightweight Directory Access Protocol

PAM Linux Pluggable Authentication Module

MPP Massively Parallel Processing

NLS Near Line Storage

Acronym glossary (1 of 2)


Acronym Full Text

OS Operating System

RTDP Real-Time Data Platform

RDBMS Relational Database Management System

RBAC Role Based Access Control

RLV Row Level Versioned

ASE SAP Adaptive Server Enterprise

ERP SAP Enterprise Resource Planning

ESP SAP Event Stream Processor

SAN Storage Area Network

SMP Symmetric multiprocessing (slide 21); SMP is also the acronym for ‘Service Marketplace’

TCO Total Cost of Ownership

Acronym glossary (2 of 2)

Software

SAP IQ Roadmap Webinar February 27 2014