Accelerate Migration to - static.rainfocus.com availability and zero downtime migrations ... Datacenter Migration TUCSON –M6 Oracle Net NON-PROD PROD ... Accelerate Migration to

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Accelerate Migration to Cloud Infrastructure with Oracle Data Integration PlatformSUN6896

Thomas VengalDirector of Product Management, Oracle Data Integration @thomasvengal

Kalyan VilluriSenior Manager, Data Services, Veritas Technologies LLC

October 01, 2017

Confidential – Oracle Internal/Restricted/Highly Restricted


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Confidential – Oracle Internal/Restricted/Highly Restricted 3


Program Agenda

Cloud Journey -Challenges

Introducing Data Integration Platform Cloud

Customer Spotlight– Veritas Cloud Journey

Question & Answers

1

2

3


4

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Cloud JourneyChallenges & Requirements


Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 6

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 7

Business Demand for Integration is Everywhere

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 7


Capabilities a Data-DrivenBusiness Needs

1. Make Data AlwaysAvailable – No Outages

2. Get Data to Where it isNeeded, at Right Time

3. Access Data in Any Format

4. Govern Data so that itcan be Trusted

Oracle Confidential


Oracle’s Strategy is to Enable Your Organization’s Journey to the Cloud

Oracle Confidential--Internal Use Only 9Copyright © 2017 Oracle and/or its affiliates. All rights reserved. | 9

To Date,Oracle has

helped

Customers Transition

to the Cloud25,000


CloudPlatform

`

On-PremOperations

Move

Workloads

Embrace

SaaS

Modernize

AppDev

Our Most Innovative Customers are on a Journey to Cloud…

Copyright © 2017 Oracle and/or its affiliates. All rights reserved. |

Insights from

Analytics

Optimize IT

Operations


Database record level

sharding

Data High

Availability

Multi-Region Cloud

Availability (Oracle or Amazon)

Active-Active Databases

Migrate from Amazon RDS to

Oracle Cloud

Data

Migrations

PeopleSoft or Workday into Fusion HCM

Oracle Database Migrations into

12c

Customer 360 from Salesforce or Sales Cloud

DW/Mart

Automation

Marketing Analytics on Big

Data Cloud

Move a Data Warehouse into

the Cloud

Streaming ETL for Data Pipelines

Streaming

Integration

3 Kinds of Data Lineage for LoB

and IT Users

Serving Layer for Raw Data

Access

Prepared Data Subscriptions

for LoB

Data

Governance

Business Catalog and Data Policies

Advanced Profiling and

Cleansing

Customer Needs

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Oracle Data Integration Platform CloudTechnology and Detailed Use-Cases

12

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |01/10/17 13Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Application and Data Integration

Complete

Simplified

Open

DATA

GOVERNANCE

PROCESS

AUTOMATION

STREAM

ANALYTICS

API

MANAGEMENT

APPLICATION

INTEGRATION

DATA

QUALITY

BULK DATA

TRANSFORMATION

REAL TIME DATA

STREAMING AND DATA

REPLICATION

Oracle Cloud Platform for Integration

13


Enterprise Grade and Broad Data Integration Capabilities

ORACLE DATA INTEGRATION

PLATFORM

ORACLEDATA INTEGRATOR

CLOUD SERVICE

ORACLE DATA

INTEGRATION PLATFORM CLOUD

Architects and IT Ops

ORACLE ENTERPRISEMETADATA

MANAGEMENT

ORACLEDATA INTEGRATOR

ORACLE GOLDENGATE

ORACLEGOLDENGATE CLOUD

SERVICE

ORACLE ENTERPRISE

DATA QUALITY

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. 15Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

NEW: Oracle Data Integration PlatformIntegrate Cloud and On-Premises Data Lakes and Data Warehouses

…a Unified solution …that’s Easy to use …for Powerful data-driven solutions

15

Key Capabilities

1. Data High Availability

2. Data Migrations

3. Data WarehouseAutomation

4. Databus & StreamIntegration

5. Data Governance


Open & Secure Platform

Business DataOLTP/Applications

Oracle Data Integration Platform CloudAnalytics / OLAP / Data

Science and Serving Layers

Runs on Oracle Cloud

Runs on Premise(cloud machine or remote agent…)

Runs on Amazon(remote agent on any cloud…)

3rd PartyData Sets

Access >125 Sources Integrate >100 Targets

SecurityAES-128, SAML 2.0, SSH,OAUTH 2.0 , MFA, KMS

Data Science

Workbench


Example #1: On-Premises Integration

Example #2: Oracle Cloud Integration

Example #3: Third-party Cloud Integration

Optional: Cloud Machine @ Customer Site

SaaS / PaaS / DaaS

Customer Site

Oracle Cloud

StudioApp

Data Integration Platform Host

Data Integration Platform Agent

Any Data Center(Customer On-Premise or 3rd Party PaaS)

HT

TP

(me

tad

ata

on

ly)

HTTP(metadata only)

Other DBs

Apps/SaaS

Big Data Messaging

PlatformServices

An

y D

ata

Hybrid & Modular Footprint


Oracle Cloud

On Premise

RDS

DBCSExadataCloud Service

DB on Compute

• Seamless data migration from older databases or poorly optimized Cloud services

• No data loss for phased / online migrations with reliable and proven core GoldenGate technology

• Migrate older Amazon RDS instances to Oracle or data delivery from on-premises to 3rd party Clouds

• Maximize Cloud infrastructure usage for complex transformations

Real-Time Data Migrations

Data Integration Platform


Provides non-intrusive data

capture from on-premises

databases

Provides automated and real-

time data feed into the

reporting environment in the

cloud

Allows maintaining high level

service agreement for query

applications

Real-Time Operational Reporting

Oracle Cloud


Europe

DIPC Asia

DIPC

OGG

SaaSApps


• Lower the costs of your EDW running on PaaS or IaaS offering

• Fastest data movement in real-time or batch

• Maximize performance for your Data Integration and Data Warehouse processes with cloud computing

• Consolidate all your data into one place for cross LOB analytics

• Innovate with the latest technologies in database and big data for faster and greater insight

BI & DataVisualization

SaaSApps

Real-Time Data Warehouse

Oracle Cloud

DBCSExadataCloud Service

DB on Compute


Big Data CS

Real-time

Real-time

Real-time

ML & PredictiveAnalytics


• Fully realize a multi-cloud vendor strategy with Data Integration Platform Cloud Service

• Move data easily between Oracle Cloud and other clouds

• Move data in real-time to ensure high availability and zero downtime migrations

• Supports data integration across heterogeneous environments with core GoldenGate

• Integrated with HA solutions

Business Continuity & Heterogeneous Cloud

Oracle Cloud


DIPC

DIPC

DIPC


90

Easily Move to Cloud with Oracle Data Integration Platform

22

Same Architecture Same Software Same Skills

Complete: Everything

You Need for Data Integration

and Data Governance

100% Compatible

with existing on-premises

deployments

Full portability across

Hybrid Clouds

22Copyright © 2017, Oracle and/or its affiliates. All rights reserved.

Oracle Data Integration PlatformOracle Data Integration Platform

Copyright © 2017 Veritas Technologies.23

Migration To Cloud Infrastructure

Kalyan Villuri

Copyright © 2017 Veritas Technologies25

1 Introduction

2 GoldenGate deployment in Veritas

3 GG use cases


1 Introduction


3 GG use cases

• Who we are :Veritas Technologies LLC is a data management company with more than 8200 employees, partners with the largest technology leaders, including Amazon, Cisco, Fujitsu, Google, Hitachi, HP, IBM, Microsoft, NetApp, Open Stack, Symantec, and many more

• What we do:Veritas Technologies empowers businesses of all sizes to discover the truth in information—their most important digital asset. Using the Veritas platform, customers can accelerate their digital transformation and solve pressing IT and business challenges including multi-cloud data management, data protection, storage optimization, compliance readiness and workload portability—with no cloud vendor lock-in. 86% percent of Fortune 500 companies rely on Veritas today to reveal data insights that drive competitive advantage.


Introduction



1 Introduction


3 GG use cases

Copyright © 2017 Veritas Technologies LLC30

Veritas Ecosystem

Rev ProVirtual TraderSabrix

30

Oracle ERP

DataHUB

PublicCloud Vendors

GG

OBIA

BI/OBIEE

ODI

Replicated Stg.

Staging

GG

GG

OBAW

OBAW Schema

STAGI

NG

REPORTS

Licensing GMS

GG

GG

GG

Replicated Stg.

Replicated Stg.

80+ Replication

pairs

6000+ Tables in replication

500GB+ data replication

daily

Customer DB/TCA

GG

GG


Accomplishments

Datacenter Migration

• Data Center Migration

• 129 DBs/200+ servers

• 45+ GG replication pairs between cloud databases

• DBs migrated to new Data center using Dataguard and GoldenGate

GG Upgrade

• 80+ GG pairs

• 6000+ tables

• Data flow 500+ GB per day

• Used integrated PARALLELISM to Improve GG performance

Bi directional replication

• GG replications from Licensing to ERP

• GG Bi directional replication setup

GG Heartbeat Implementation

• Configured GG Heartbeat table setup for all environments

• Developed real time Dashboard Displaying Latency with most accurate timestamp along with “Data As Of” and Hosted on WebLogic server

• Upgraded Heartbeat table configuration to GoldenGate 12.2

Veridata Installation

• Configured GoldenGate Veridata 12.1.3 for Out of sync Data comparison and repair.

• Automated OOS Report in HTML notification

• Configured 6000+ tables in 144 Veridata Jobs

• Veridata monitors 80+ GG pairs

GoldenGate Event Capture

• Golden Gate delete event capture

• Golden Gate replication from EBS to OBIEE


1 Introduction


3 GG use cases


VERITAS GoldenGate Replication - Current Setup• Oracle GoldenGate Version 12.2.0.1 along with Oracle

12c and 11.2.0.4 • Separate GoldenGate homes for oracle 11g and 12c

databases• All GoldenGate processes are using integrated mode

with parallelism enabled• Total 80 GoldenGate pairs and 6000+ tables in

replication between cloud databases.• Approximately 500+ GB real time being replicated

between cloud databases• Minus Query based monitoring scripts were used

earlier for comparing tables on source and target databases.

• Oracle GoldenGate Veridata tool replaced the manual scripts for the tables in replication comparison and fixing out of sync issues.

• GG Heartbeat table for GoldenGate process status and lag monitoring

• All GoldenGate pairs configured using Conflict Detection and Resolution

GG

GG

GG

GG

DataHubLicensing GMS M7

Supercluster

OBIEE

GG

ERP

GG

TCA

GG

GG


GG Replication Setup between cloud databasesWhy its important:As per the recent data survey, more than 50% of total data will be stored in Public and Private cloud systems by the year 2020

Minimum Requirements:

• Systems - Do ensure the source and target systems are defined• Network - it can be either P2P, internet or MPLS circuit, do ensure sufficient bandwidth is in place for the volume

of the data being replicated• Security - Do ensure the source and target system IP Addresses are Natted/Masked to ensure the security• Channel Encryption - Do ensure GG data being replicated is encrypted


Datacenter MigrationTUCSON – M6

Oracle Net

NON-PROD

PROD

DALLAS – M7

NON-PROD

PROD

OracleData Guard

DEV DEV

QA QA

UAT UAT

PROD PROD

Migration to cloud Infrastructure

• 129 DBs/200+ servers

• 45+ GG replication pairs

• Total 129 Applications

• 800+ total application and database servers

• DBs migrated to new Data center using Dataguard

• Without the need for GG rebuild

• Not a single transaction was lost


Primary Database Standby Database

TUCSON Data Center– M6 Supercluster

Dallas Data Center– M7 Supercluster

OMCS Cloud

Test Table

Switch Over The DatabaseSwitch Over The Database

Test Table

Data Guard Is Running Data Guard Is Running

Standby Database Primary Database

GG GG

Migration to Cloud

Oracle GoldenGate


IBM Cloud

GG Setup

Licensing Database

ORACLE Cloud

GG Setup

ERPDatabase

• Bi-Directional replication is set between IBM cloud and ORACLE cloud environment in VERITAS

• Bi-Directional Replication also known as Active-Active Replication

• Applications perform DML/DDL activities on both Databases

Licensing GG replication

OMCS EnvironmentOMCS Environment


Revpro Delete Event Capture Using GG

OBIEE Target DatabaseERP Source Database

Source TableSource Table Target TableTarget Table

GG Setup GG Setup

Event Capture TableEvent Capture Table

Extract

DML

Replicat

Event Capture Extract Parameters.

TABLE REVPRO.RPRO_RI_PS_WF_DET,TOKENS(CSCN = @GETENV('TRANSACTION','CSN'),

RSID = @GETENV('RECORD','RSN')), GETBEFORECOLS(ON UPDATE ALL);

Event Capture Replicat Parameters.

MAP REVPRO.RPRO_RI_PS_WF_DET, TARGET STAGE_EBSR12.RPRO_RI_PS_WF_DET_HIST ,&

COLMAP (LINK_TRANS_ID=LINK_TRANS_ID,PERIOD_NAME=PERIOD_NAME,RUN_SEQ=RUN_SEQ,

LAST_UPDATE = @GETENV ('GGHEADER', 'COMMITTIMESTAMP'), &TRAN_TYPE=@GETENV ('GGHEADER', 'OPTYPE'), &

BEFORE_AFTER=@GETENV('GGHEADER','BEFOREAFTERINDICATOR'), &RSID=@TOKEN('RSID'), &

CSCN=@TOKEN('CSCN')),INSERTALLRECORDS,FILTER (@STREQ (@GETENV ('GGHEADER', 'OPTYPE'), 'DELETE'));

Automatic Heartbeat Table• Intelligent Functionality

• Automatically discovers replication topology

• Unidirectional, bi-directional, N-way...

• Automatically propagates heartbeats along replication paths

• Shows incoming and outgoing lags in replication paths in each database for active scenarios

• Easy to configure and create dashboard

• Execute GGSCI command ADD HEARTBEATTABLE at each database


Single Click Golden Gate Latency Dashboard Using Heartbeat Table


Goldengate Veridata Implementation

• Goldengate Veridata 12.1.3 is a high-speed data comparison solution that identifies and reports data discrepancies between two databases

• Out-Of-Sync Data:• Facilitates the repair of out-of sync data in live

database environments • Ability to select which tables and rows should

be repaired and when• No disruption to business systems while repair

operations.• Architecture consist of Veridata Server along with

Veridata Repository database for storing configuration information

• Veridata agents needs to be deployed on the targets those we want to monitor

• Veridata Web User Interface is browser based graphical user interface which will allows us to review status and output of comparisons

• Customized the Out-Of-Sync report to html file email notification

WebLogicServer Domain

Veridata Repository

DB

Veridata Command-line tool

Veridata Veridata Web Client

Veridata Agent

Veridata Agent

Veridata Server

Source Database

Target Database

GG Replication

Source DB Server

Target DB Server


Veridata Configuration

Veridata Configuration Summary

Veridata Agents 28

GG Pairs 80

GG Tables 6000+

Veridata Group Configurations 148

Veridata Jobs Configuration 144

Daily Data replication 500+ GB


GoldenGate Version Upgrade to 12.2

OGG 12.2 Home

Take Extract information with Below command-INFO EXTRACT <ext_name>,SHOWCH and -info <ext_name>,detail

Take Replicate information with Below Command-lag <Replicat Name>-stop <Replicat Name>

OGG 12.1 Home

Stop MGR

Once All processes are stopped -ddl_disable.sql,ddl_remove.sql,marker_remove.sql

Stop Extract

Stop Replicat

Stop Pump

Copy all parameter files and dircrd (Credential store) file to new 12.2 home

& Run below scripts from New OGG 12.2 home-marker_setup.sql,ddl_setup.sql,role_setup,ddl_enable,ulg.sql,ddl_pin

start manager process and add extract/pump process in new 12.2 home.

ADD extract with SCN (Lower SCN which we took it from Old GG info)ADD EXTRACT E01OBIEP, INTEGRATED TRANLOG,SCN <take scn from step 3(lower scn)>

ADD EXTTRAIL ./dirdat/MP, EXTRACT E01OBIEP, megabytes 2000

Start MGR

Start Extract

Start Pump

Start Replicat

Q & AQ & A


Thank you!

Copyright © 2017 Veritas Technologies. All rights reserved. Veritas and the Veritas Logo are trademarks or registered trademarks of Veritas Technologies or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners.

This document is provided for informational purposes only and is not intended as advertising. All warranties relating to the information in this document, either express or implied, are disclaimed to the maximum extent allowed by law. The information in this document is subject to change without notice.

Kalyan Villuri

[email protected]


Get a sneak peek at cutting-edge data integration designs and receive a free gift!

• Oracle is constantly developing new software and features that will make your work easier, and Oracle's User Experience team would love to get your feedback on new data integration designs.

• Feedback sessions will take place at a date and time of your own choice.

• You can take part via webconference, from the comfort and convenience of your own office.

• If you’re interested, please fill out the 1-page form at http://bit.ly/2vIHlSg uppercase I lowercase l

• To show our appreciation, we will post all participants their choice from a wide selection of thank-you gifts.

Oracle Confidential – Highly Restricted 45


Presen-tationson:

Oracle Confidential 46

Data Integration Programming – FOCUS ON DOC LINK

DemoStations:

Hands-on Labs:

OracleEnterprise

Data Quality

OracleGoldenGate

Oracle Data Integrator

OracleData Integration Platform Cloud

OracleEnterprise Metadata

Management

Oracle GoldenGateReal-Time Data Replication

in the CloudHOL7715

Oracle Enterprise Data Quality

HOL7653

ODI and OGGfor Big Data

HOL7708

Oracle Data Integration Platform Cloud

HOL7673

The EXchangeIntegration Area- Moscone West

The EXchangeAnalytics & Big Data Area

- Moscone West

The EXchangeData Management Area

- Moscone West


Data Integration Programming – FOCUS ON DOC LINK

Sunday, October 1• Lift and Shift Workloads to Cloud with Oracle Data Integration Platform

Cloud [SUN6653]

• Data Movement between On-Prem, Fusion ERP Cloud, Fusion HCM

Cloud and Salesforce [SUN7286]

• Accelerate Migration to Cloud Infrastructure with Data Integration

Platform [SUN6896]

Monday, October 2• Oracle Data Integration Platform Strategy and Roadmap [CON6646]

• Filling Your Data Lake with Potable Data, Using Data Integration

[CON5465]

• GoldenGate : Deep Dive into Automating OGG using the new

Microservices [CON6569]

• Oracle Data Integration Platform: Foundation for Cloud Integration

[CON6650]

• Oracle Data Integration Platform Empowers Enterprise Grade Big Data

Solutions [CON6893]

• Oracle Data Integration Platform Cloud Deep Dive [CON6651]

• Oracle GoldenGate Cloud Service: Real-Time Data Replication in the

Cloud [HOL7715]

Tuesday, October 3• Oracle Data Integrator Product Update and Strategy [CON6654]

• Oracle Enterprise Data Quality: Product Overview and Roadmap [CON6656]

• Accelerate Cloud On-Boarding Using Oracle GoldenGate Cloud Service

[CON6894]

• Oracle Enterprise Data Quality for All Types of Data [HOL7653]

• Oracle Data Integration Platform: a Cornerstone for Big Data [CON6655]

• GoldenGate: MAA and Best Practices for Oracle GoldenGate Microservices

[CON6570]

• Oracle GoldenGate Product Update and Strategy [CON6897]

Wednesday, October 4• A Practical Path to Enterprise Data Governance at Energy Australia

[CON6657]

• Oracle Data Integrator and Oracle GoldenGate for Big Data [HOL7708]

• Introduction to Oracle Data Integration Platform Cloud [HOL7673]

• An Enterprise Databus: GoldenGate in the Cloud Working with Kafka and

Spark (CON6895]

• GoldenGate: Best Practices & Deep Dive on OGG 12.3 Microservices at

Cloud [CON6568]

• Oracle GoldenGate for Big Data [CON6898]

• Oracle Data Integration Platform Cloud Service Governance Edition

[CON6652]


Connect with Oracle Integration

@OracleDI

Blogs.oracle.com/DataIntegration/

Oracle Data Integration

Oracle Data Integration

Oracle FMW

@OracleIntegrate

Blogs.oracle.com/Integration/

Oracle SOA


Safe Harbor Statement

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

Oracle Confidential 50

Documents

Accelerate Migration to - static.rainfocus.com availability and zero downtime migrations ... Datacenter Migration TUCSON –M6 Oracle Net NON-PROD PROD ... Accelerate Migration to