Real-time Data Distribution: When Tomorrow is Too Late

Twitter Tag: #briefr

[email protected]

9/4/12


!  Reveal the essential characteristics of enterprise software, good and bad

!  Provide a forum for detailed analysis of today’s innovative technologies

!  Give vendors a chance to explain their product to savvy analysts

!  Allow audience members to pose serious questions... and get answers!


!  September: Integration

!  October: Database

!  November: Cloud

!  December: Innovators

!  January: Architecture


!  Data integration involves combining heterogeneous data sources and providing one unified view of said data.

!  It is a necessity for all IT sites, increasingly becoming a problem

area in the era of remorseless data growth (average about 55% per year) which is swiftly becoming an era of Big Data.

!  Data integration involves many competing technologies, each

with its nuances, upside and downside. But which is best for you?

!  The costs of data integration are high and rising. This calls for

strategy and effective technology.


Robin Bloor is Chief Analyst at

The Bloor Group.

[email protected]


!   Sybase, an SAP company, provides enterprise and mobile infrastructure, development and integration solutions.

!   It offers a suite of database management technologies designed to increase performance and time to insight.

!   Its Replication Server product allows for real-time reporting with minimal performance impacts across heterogeneous database environments.


Bill Zhang is a veteran at Sybase, an SAP Company. As Director of product management, Mr. Zhang is responsible for the complete product strategy for Replication Server. He interacts with strategic customers and partners as well as industry analysts to formulate product strategies. He defines product roadmaps for engineering groups. Prior to his current role, Mr. Zhang held several customer-facing positions at Sybase in Sales and Professional Services. Mr. Zhang has an MBA degree from the Leonard N. Stern School of Business, New York University, a master’s degree in electrical engineering from Columbia University, and a bachelor’s degree in electrical engineering from the University of Rhode Island.

Tom Traubitz is a Director of Analytics Product Marketing with SAP/Sybase’s Data Management and Tools Group, specializing in enterprise-class transaction processing and data analytics. He has spent the past 25 years designing, engineering, testing, and marketing large scale, networked information management systems for a wealth of clients throughout the United States and the world.

August 2012

SAP Sybase Replication Server

©  2012 SAP AG. All rights reserved. 10 This presentation and SAP‘s strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement

Replication Server: WHAT DOES IT DO?

High Availability

Disaster Recovery

Real-Time Business Reporting

Load Balancing

Data Integration

Data Assurance

Replication Server


Sybase Replication Server Use Case Scenarios

Data distribution and migration §  Distribute: move centralized data to operational applications §  Share: share data between operational applications §  Synchronize: maintain consistency in overlapping data values §  Migrate: move from older version of database platform to newer one

Real-time Decision Support §  Create ODS (copy of OLTP production systems for daily reporting) §  Real-time loading of data warehouses (Sybase IQ, ASE, Oracle, Microsoft,

IBM), aka, Change Data Capture

High availability/disaster recovery §  Enable business continuity in event of site-wide disaster §  Maintain application availability during planned/unplanned downtime


Philadelphia Operations

Denver Operations

Warm Standby

Sybase Replication High Availability

PRIMARY DATACENTER

ASE Replication

Server

SECONDARY DATACENTER

ASE Replication

Server

OFF LINE

• Minimize/eliminate user impact • Protect against unplanned outages

� Software, hardware, application failure � Unforeseen circumstances like data corruption

• Protect against planned outages � Software, hardware, application upgrades � Enable ops to perform maintenance activities

• Recover from natural disaster � Without geographic restrictions


OLTP DSS

Sybase Replication Replication and Live Decision Support

DB Rep Server DB Rep Server

� Maintain a complete copy of the primary OLTP database � Run operational reports and queries against this copy (ODS) � Preserve transactional system processing performance � Enable more robust and responsive reporting environment � Sources can be ASE, Oracle, Microsoft, and IBM � Targets can be ASE, Oracle, Microsoft, IBM, and Sybase IQ � HA/DR warm standby can also be ODS


Sybase Replication Data Distribution

New York (sales department)

San Francisco (order processing)

San Francisco (finance department)

Dallas (manufacturing department)

§  Continuous replication of changed data

§  One source to many targets §  Guaranteed delivery. Publish and

subscribe architecture §  Propagate order info to related

downstream applications §  Can also have bi-directional

scenarios §  Can also have many – one and

many – many topologies

One example, many permutations

Order Entry Application

ASE Rep Server

Rep Option for Microsoft

Sales Support Application

Financial Reporting Application

Rep Option for Oracle

Rep Option for IBM

Manufacturing Planning Application

WAN

WAN

LAN


Replication Server – In a Nutshell

Replication Server (RS) Primary DB

Replication Server •  Replicates “transactions” from primary to secondary site(s), non-intrusively

•  Near real time, bi-directional data movement

•  Guaranteed delivery with store and forward mechanism

•  Flexible filtering / transformation of data

•  DML, Schema (DDL) changes, Stored Procedures replication •  Database Integrity is guaranteed and protects against corruptions

Secondary DB


Flexible Replication landscape Data movement across heterogeneous databases

Message Bus – MQ, Tibco, JMS,

Replication Agent

Replication Server

Express Connect & ECDA

MS SQL

IBM UDB

Sybase ASE

Sybase IQ

Oracle, MS SQL, IBM UDB

Sybase ASE Staging Database

RepConnector

Oracle

Ø  Multiple Database vendors

Ø  Many to one, one to many, any to any

Ø  Geographically dispersed

Sybase IQ


New Feature Highlight: Multi-Path Replication

Mul$ple RepAgent Senders

Dedicated Route Paths Mul$ple DSI

Mul$ple RS from Same Source

Single RepAgent per PDB

Single Route between PRS & RRS

Single DSI connec$on to RDB

Single-Path M

ulti-Path


The Orchestration of Replication


We need to duplicate data.

We have no choice.

So the question is not whether we do it, but how best to do it.


!   Database Logging !   We duplicate for the sake of recovery

!   Database Back-ups/Snapshots !  We duplicate for the sake of a recovery start-point

!   Data Warehouse !  We duplicate for the sake of data consolidation

!   Data Staging !  We duplicate for the sake of data flow

!   Database Subsetting (Data Marts) !  We duplicate for the sake of performance

!   Operational Data Store !  We duplicate for the sake of timeliness



!   Of course, it isn’t just performance, but performance is the major driver for the way we build the data layer.

!   Because we cannot have a single coherent distributed data store, we have no option but to think in terms of data flows.

!   This means database plus middleware. !   Middleware is a lousy word with many meanings: ETL, ESB,

data governance, data virtualization, etc. !   The truth is that data flow service levels and database

service levels are strongly interrelated. One hand washes the other (and both hands wash the face).

!   Database replication is a critical capability in this primarily because of its performance characteristics.


!   Disaster Recovery (An extreme service level and often an expensive one)

!   High Availability (A service level thing)

!   Real-time Business Reporting (A data flow and service level thing)

!   Load Balancing (A service level thing)

!   Data Integration (A data flow and service level thing)

!   Data Assurance (A security thing)


!  What are the costs likely to be in situations where replication replaces other data flow strategies? Does it reduce storage costs or increase them?

!  Where is there a performance advantage when replication replaces other data flow strategies?

!   Is the replication server used for “software modernization” rather than just to build new data flows? Can you provide use cases?

!   How frequently is it used in that way (roughly)?

!   Can you please provide a description of the most extensive use of this capability by one of your customers?


!   How difficult is it to use? In other words, what are the labor overheads compared to alternative approaches?

!  What situations (in respect of data flow) do you think it does not apply to (i.e., where not to use it)?

!  What do you think it competes with? Which other products do you actually meet in competition?

!   Does it play well with others (i.e., other databases, other data flow tools)?

!  Where does it sit in the spectrum of strategy --> tactics?



!  September: Integration

!  October: Database

!  November: Cloud

!  December: Innovators

!  January: Architecture


Technology

Real-time Data Distribution: When Tomorrow is Too Late