Short Presentation Title - Dobler Consulting · 2016-08-05 · © 2013 SAP AG. All rights reserved. Public 4 Your Data is Your DNA, Dobler Consulting Focus Areas Strategic Database

© 2013 SAP AG. All rights reserved. 1Public


Agenda

Welcome

Agenda

• Introduction to Dobler Consulting

• Introduction to SAP Sybase IQ, Challenges and Trends, Key Capabilities

• What’s New In Version 16

• Q&A

Presenters

• Dan Lahl - Vice President of Product Marketing, SAP

• Joydeep Das - Vice President Product Management, SAP Database Management Portfolio

• Peter Dobler - CEO Dobler Consulting

• Allan Roeder - VP of Sales Dobler Consulting

Closing


Introduction to Dobler Consulting

Dobler Consulting is a leading information technology and

database services company, offering a broad spectrum of

services to their clients; acting as your Trusted Adviser,

Provide License Sales, Architectural Review and Design

Consulting, Optimization Services, High Availability review

and enablement, Training and Cross Training, and lastly

ongoing support and preventative maintenance. Founded in

2000, the Tampa consulting firm specializes in SAP/Sybase,

Microsoft SQL Server, and Oracle.

Visit us online at www.doblerconsulting.com, or contact us

at 813 322 3240, or [email protected].

http://www.doblerconsulting.com/

mailto:[email protected]


Your Data is Your DNA, Dobler Consulting Focus Areas

Strategic

Database

Consulting

DBA

Managed

Services

Database

Training

Programs

SAP VAR D&T

Cross-Platform Expertise SAP Sybase® SQL Server® Oracle®

Dan Lahl

9/19/2013

SAP Sybase IQ 16

Tap Into Big Data at the Speed of Business


Today’s topics

Introduction to SAP Sybase IQ

Challenges and Trends

Key Capabilities:

– Exploit the value of Big Data

– Transform businesses through deeper insights

– Extend the power of analytics across the entire enterprise

Customer and Ecosystem Success


SAP Sybase IQ: smarter answers now

Up to 1,000x faster than

traditional DBMS

Handles volumes of ad

hoc queries in real time

Simple and complex

analytics on structured

and unstructured data

Speed & Flexibility

2x as many customers as

the next leading provider

96%+ customer

satisfaction rates

Consistently ranked as

EDW market leader by

leading analysts*

Mature and Proven

Compression reduces data

volume by up to 70%

Less storage and

maintenance required

Easy to lean and use –

requires minimal training

Lower TCO

Source: Magic Quadrant for Data Warehouse Data Management System, Gartner, January 31, 2013

Business Challenges and Trends


What’s happening in the marketplace…

Exploding Data

Volumes

The Need for

Speed

Rising IT Costs

and Complexity


Today’s challenges

Lost Revenues;

Lack of Insight

High Costs &

Complexities

Data Management

Challenges

Security

Slow Performance


• Marketing analytics

Digital channelsTrack visits, discover best channel mix: e-mail, social media, and search

• Sales analytics

Deep correlationsPredict risks based on deal DNA (e-mails and meetings) and pattern match

• Operational analytics

Atomic machine dataAnalyze RFID, blogs, Short Message Service (SMS), and sensors — for continuous

operational inefficiency

• Financial analytics

Detailed simulationsLiquidity and portfolio simulations — for stress tests and error margins

Competitive advantage is now data-driven


The data explosion continues

“Petabyte is the new Terabyte” - Forbes

Data volumes and applications in analytics environments are “growing up”

to Big Data

Source: Wikibon, Feb 2013, Big Data and Data Warehousing, www.wintercorp.com

$4.6B

Big Data

Apps

$6.9B

http://www.wintercorp.com/

Key Capabilities


SAP Sybase IQ 16

SAP Sybase IQ transforms the way

companies compete and win

through actionable intelligence

delivered at the speed of business

to more people and processes.


Key CapabilitiesThe Value of SAP Sybase IQ 16

Extend the power of analytics across the entire

enterprise

Exploit the value of Big Data

Transform businesses through deeper insights

1

2

3


Exploit the value of Big Data

Unconstrained, unlimited, fast

access

Flexible, expandable grid

architecture

Integral native frameworks

Cost-effective compression


Transform business through deeper insights

Unified platform for all data

Low latency for immediate updates

Comprehensive answers using

massive volumes


Extend the power of analytics across the entire enterprise

User driven self-service at “Google

speed”

Highly available enterprise data

Fast and efficient MPP and query

optimization

Secure data and systems

Customer & Partner Success


SAP Sybase IQGlobal Presence in a Range of Data-Driven Industries

Since adding Sybase IQ, the demand

for reports has risen dramatically as

users realize just how much more

helpful the information is that we can

now provide. We receive requests for

10-20,000 more reports per month,

and ad hoc queries have grown from

5,000 to 10,000 per month.

Marc Guillard, DBA , BNP Paribas

Securities Services

“

”

“

”

“

”

SAP Sybase technology added a new

dimension to our business – we can

now report on any data our executives

need within minutes. Previously,

reports were impossible or took a full

day to generate.

Bonjan Sovilj, Research and Development

Manager, Mozzart

…in North America …in EMEA …in EMEA

I would venture to say SAP Sybase IQ

is the fastest in the industry. We load

almost 10 billion rows a month, a

terabyte a quarter. With SAP Sybase

IQ, it just flies.

Craig Silver, Senior Database Architect,

Nielsen Media Research


Measurable benefits for our customers

75% 75% 800%Reduced daily DB

administration by

75%

Faster data loads -

increase productivity

during maintenance

window

Reduced BI storage

requirements by 75%

a ratio of 4 to 1

http://www.digiturk.com.tr/



2,500 + Customers

96% Customer

satisfaction rating

4,500 Installations

SAP Sybase IQ Market-Leading RDBMS Platform for EDW

http://www.comscore.com/

http://www.comscore.com/



Transforming companies through new insights.

SAP Sybase IQ 16: What’s New

Joydeep DasVice President, Product Management,

SAP DBMS Portfolio


2009

Very large database (VLDB) platform foundation

2011

MapReduce API

2011

PlexQ MPP Foundation

2010

Full text search; Web 2.0 API

2009

In-database analytics

1H 2013

XLDB

Analytics

SAP SYBASE IQ 16

PATH TO ACTIONABLE INTELLIGENCE


Ecosystem

SAP SYBASE IQ

A COMPREHENSIVE ANALYTICS PLATFORM

Most mature

column store

Comprehensive

lifecycle tiering

MPP queries, virtual

marts, and user scaling

High-speed

loads

Structured and

unstructured store

Comprehensive SQL

with OLAP

Built-in full-

text search


with MapReduce

Web 2.0

APIs

Big Data open

source APIs

Optimized BI and

EIM

Development and

administration toolsPredictive analytics

Packaged ILM

applications

Bradmark,

Symantec,

Whitesands,

Quest, and

ZEND

SAS, SPSS,

KXEN, Fuzzy

Logix, Zementis,

and Visual

Numerics

BMMSoft,

SOLIX, and PBS

Sybase

PowerDesigner,

Sybase Replication

Server,

SAP and Panopticon

Application

services

DBMSHadoop

and R

SAP Sybase IQ 16

Architectural Details


SAP Sybase IQ 16 Engine

Multip

lex g

rid a

rchite

ctu

re

Storage area network

Loadingengine


Text search

Web-enabled analytics

Info

rma

tion life

cycle

ma

na

ge

me

nt

Low latency, write optimized store

Role-based access control

Communications and security

N-bit and tiered indexing

Column indexing

subsystem

Query engine

Column store

Aggressive scale out

Fullyparallel

Re

silie

nt

We

b b

ase

d a

dm

inis

tratio

n a

nd

mo

nito

ring

LDAP authentication

Hash partitioned tables

and data affinity

New Generation

PETABYTE SCALE store

New indexing technology for increased compression and fast, incremental batch loads

SAP SYBASE IQ 16

WHAT’S NEW!

High performance, fully parallel bulk loading

Enterprise-grade RBAC authorization and LDAP authentication

Comprehensive web-based monitoring and administration

Low latency, concurrent write optimized delta store

Re-engineered column store for extreme data volumes

Resilient Multiplex grid withstands network interrupts and server failures

Aggressively scaled out query engine

Intelligent query engine with data affinity for “shared nothing” performance on a flexible “shared everything” architecture


SAP SYBASE IQ 16

INNOVATIONS FOR EXTREMELY LARGE DATABASES (XLDB)

Storage Architecture

• New generation column store

• New partitioning and compression

Query Processing

• Data affinity

• Aggressively parallel and distributed

Loading Engine

• Fully parallel bulk loading

• Real-time loading into delta store

System Reliability

• Grid resiliency

• LDAP and role-based security

SAP Sybase IQ

XLDB

Analytics

Petabytes Real-time


SAP SYBASE IQ 16

NEW COLUMN STORE ARCHITECTURE

Value proposition

Enhanced compression

Storage savings

Improved I/O bandwidth

Architectural considerations

Support variable number of cells per page

Support various page formats within a column

High performance access paths

Even with variable length data, insert/update/delete

efficiently into an existing page

Richer metadata

Logical

page of

variable

sized cells

cell values

directory

Logical

page of

fixed sized

cells

Before

NOW…


SAP SYBASE IQ 16

N-BIT DICTIONARY COMPRESSION

Value proposition

Reduced memory footprint

Improved effective I/O rates

More efficient table scans

Reduced disk space


N-bit Fast Project (FP) Indexes, instead of 1, 2 and 3-byte FP Indexes

Different data pages for same column can have different values of “N” for N-bit

No more requirement to rollover FP format for all column data

Column is N-bit by default, unless otherwise specified to be flat

Options provided to set threshold for rollover to flat (to prevent large dictionaries)

Options provided to prevent rollover to flat (to prevent long rollover time)

Compatibility mode allows the database to mimic IQ 15 rollover behavior

Raw Data = 400 MB; 1 Billion 4-byte integer values fn (N)

N Token Size Savings

2 (1B * 2 ) / 8 = 250MB 93.75%

3 (1B * 3 ) / 8 = 375MB 90.6%

4 (1B * 4 ) / 8 = 500MB 87.5%

…….

24 (1B * 24 ) / 8 = 3000MB 25%

2->3

3->4

4->5

5->6

6->8

8->10

10->12

12->16

16->21

21->24


SAP SYBASE IQ 16

FULLY PARALLEL BULK LOAD

Value proposition

Improved performance

Maximize use of existing cores on the

machine

Dynamic load balancing


Load an index/column concurrently with

multiple threads

Remove all bottlenecks which contribute to

inefficient use of CPU and storage

Dynamically scale up and down degree of

parallelism depending on the workload

Before…

NOW

…


SAP SYBASE IQ 16

SMALL BATCH LOAD PERFORMANCE

Value proposition

Improved performance of frequent, small batch

loads

Predictable performance of small batch loads:

performance is proportional to the size of the

data being loaded, not the table being

updated


Inserting into a large High Group (HG) b-tree

index is costly

HG index will have a tiered structure with a small

tree component and a large tree component

Small, batch loads into the HG index are written

to the small tree component quickly and

synchronously

The small tree component is periodically merged

into the large tree component as a background

task

Small tree

component

Btree-

based

index

Trickle

insert

Periodic

Merge


SAP SYBASE IQ 16

HIGH VELOCITY DATA LOADING

Value proposition

Continuous analytics over operational data

High velocity, concurrent data modifications

Exploit large memory and core footprints


Write optimized in-memory In-memory RLV (Row-

level versioned) store

Row level locking, and statement snapshot

isolation

Low latency micro operations

In-memory RLV store has reduced compression,

no sorting, no indexing

Fully recoverable with dedicated transaction log

Asynchronous data transfer from In-memory RLV

store to IQ main store

Users choose which tables are In-memory RLV

tables

In-memory

RLV Store

RLV log on shared store

IQ main on shared store

Hybrid Table

Storage

Write ahead/commit

periodic merge

IQ query engine

IQ DML engine

RLV rows

main rows

Transaction Manager

Consistent View Manager

Version Manager

Lock Manager

Insert/Update/Delete

Co

nsisten

t View

global row

space


SAP SYBASE IQ 16

QUERY SCALE OUT – HASH PARTITIONING

Value proposition

Gives best of both worlds of shared everything and

shared nothing

Decreases hardware needs and localizes processing


Data is automatically partitioned during loading with built-

in hash algorithms

Data is divided into persistent subsets

Reduces results sharing

More efficient CPU usage

Reduces instantaneous temp usage

Optimizer will use hash partitions for join and group by

when available


SAP SYBASE IQ 16

QUERY SCALE OUT – DATA AFFINITY

Value proposition

Provides efficient utilization of cluster-wide

cache resources such as shared temp

Achieves ultra low-latency data access while

preserving elastic multiplex capabilities

Group by and Order by benefit


Affinity is automatically configured and used in

multiplex

Adapts to query workloads and self manages

Data must be hash or logically partitioned

Each partition is assigned to a specific node

Fact Dim

Cache Cache


SAP SYBASE IQ 16

QUERY SCALE OUT – Query Runtime and DQP Optimization

Value proposition

Eliminates SMP (single node) and DQP

(distributed query processing) bottlenecks

Leverages large memory and scale out

Lowers shared temp and interconnect bandwidth


Takes place automatically as optimizer selects

best plan based on cost

For non-partitioned data, new Join and Group

algorithms reduce the amount of intermediate

results exchanged

Group ByQuery operator specialized for med cardinality

processing

Eliminate single-threaded hash

group-by bottlenecks

Improve DQP scale-out

Join New scale-out DQP merge join

PredicateOptimize in-

memory/cached processing

Eliminate intermediate

result bottlenecks

Improve DQP scale-out

SubqueryNew query

operators for improved

efficiency and scale-out


SAP SYBASE IQ 16

LDAP AUTHENTICATION

Value proposition: Reduced TCO and improved

security

Enable customers to hook into existing enterprise infrastructures for

managing users and passwords

Enable central management of password complexity policies

Multiple domains and multiple LDAP servers

Architectural considerations:

Secure communication with LDAP server using TLS

Deployable across various vendor’s Directory Service that supports

Lightweight Directory Access Protocol (LDAP)

Support 24x7 operation: Automatic failover and failback

Efficient design for frequent, short-lived connections

No client side changes needed

SQL Anywhere, Sybase IQ, and ASE can share common user

repository

WebServer


SAP SYBASE IQ 16

ROLE BASED ACCESS CONTROL

Value Proposition

Support separation of duties and principle of least privilege

Breakdown privileged operations into fine grained sets that can be individually

granted

Control over propagation of privileges

Who can grant which privileges

Complete backwards compatibility and clean migration

Stay competitive

Architectural Considerations

Support ANSI SQL role semantics, system defined roles and user defined roles

Minimum number of role administrators

Grantable system privileges for privileged database operations

Secure system stored procedures with SQL SECURITY INVOKER

Minimize performance impact by adding connection level cache mechanism

Same in-memory representation of RBAC structures: upgraded and non-

upgraded DBs

Restrict impersonation through SET USER to adhere to RBAC model


SAP SYBASE IQ 16

SCALE OUT CLUSTER (MULTIPLEX) ENHANCEMENTS

• Shared System Temp- Reduces the size of local temporary store

- Simplifies sizing requirements for temp stores

- Opens up support for DQP on temp tables in future

• Logical Server – Login redirection- Zero changes on client side on dynamic changes to logical servers

- Single point to connection “redirector”

- HA with multiple servers in the connection string to prevent a single point of failure

• Cache Ejection Policy improvements for better cache hits

- Better infrastructure to track on disk changes and maximize cache hits to increase performance

• Global Transaction Resiliency

- Suspend/resume global DML transactions, with a timeout, during INC disconnect/reconnect, coordinator downtime


SAP SYBASE IQ 16

SCALE OUT CLUSTER (MULTIPLEX) – GBL TRANSACTION RESILIENCY

Offline Coordinator

Writer1

Heartbeat

Client Reconnect

Reader2

Global R/W

transactions

suspend until

INC connections

reestablished

or timed out

Global R/W INC Connections

automatically reconnect and resume transaction

after a coordinator failover.

In most cases long-running loads will transparently resume.

Storag

e

Failover node restarts as Coordinator


Summary

• Market-Leading product with tremendous momentum • 96%+ customer satisfaction rates• Pioneering Column-store with 10+ patents• World record holder in data loading• SAP Sybase IQ is used by twice as many companies as

the next leading pure column store DBMS provider• Focused sales and support teams• SAP commitment to product leadership

SAP Sybase IQ in the Context of RTDP


Requirements for Real-Time Data-Driven ApplicationsDefining a Big Data-enabled Logical Data Warehouse

Business Applications

Endlessly expanding

repository for all kinds of data

Deep reference and

trend analysis

Instant classification of

streaming data

Real-time processing,

correlation, transactions, and

analysis

Data movement, cleansing, and

placement throughout the

architecture

Business Events

Metadata Management,

MDM, and Governance


Enabling The Logical Data Warehouse With RTDPEngineered to Work Together – even with Big Data

HANA

IQ

Business Applications

ESP fast driver for HANA +

integration of ESP and

Hana Studios

Real-time Replication + ETL

to HANA and IQ

SQOOP to HANA and IQ

from/to Hadoop

Data Virtualization:

SAP Sybase IQ to

Hadoop to HANA

Query Integration: IQ

to Hadoop

SAP BW

Tightly integrated EDW

management with SAP

HANA and SAP Sybase IQ

Business Events

Da

ta S

erv

ice

s

ES

P

High Performance data

processing

with tight integration with SAP

BW, SAP Sybase IQ, ESP and

Data Services


Isn’t it time to rethink your EDW strategy?Benefits of a complete Big Data Architecture

Accelerate AnalyzeAcquire

Effective, Rapidly

and Efficiently

acquire and

consolidate massive

amounts of diverse

and arbitrary data

Real-time response

time regardless of

data volume or data

location manage and

integrate massive

volumes of data

Achieve real results to

get the insights you

need with a variety of

means alone or in

combination


Empowering you to Deliver Data Management

for the New Real-time World

Open APIs and Protocols

SAP Real-time Data Platform

Data Services Information ServicesCommon Services

TransactionReal-time

EDW

Integration Quality

MDM ILM

Mobile

Stream

Model

Manage

Metadata

Engineered to Work Together

Custom Mobile Analytics/BI Cloud Big Data

Thank you


Offer from Dobler Consulting

For Qualified IQ installed base customers we will provide 4 hours of health

check / upgrade consulting

For Qualified New Customers we will provide 4 days of design and

optimization consulting

Please contact Allan Roeder at Dobler Consulting for more details.

Call (813) 322 3240 or email [email protected]


Q & A

Please submit your questions through the questions panel

on the webinar control menu.

For more information on SAP Sybase IQ

Call 813 322 3240 or email:

[email protected]

Documents

Short Presentation Title - Dobler Consulting · 2016-08-05 · © 2013 SAP AG. All rights reserved. Public 4 Your Data is Your DNA, Dobler Consulting Focus Areas Strategic Database