58
Thank you for joining us, the webinar will start at: 08:00 Pacific / 11:00 Central / 12:00 East / 16:00 UK Time How to Enable Risk-Free Database Experiments Mind-meld with the illustrious members of the OakTable 32 talks over 2 days, right next door to Oracle OpenWorld More info. and registration: www.kylehailey.com/oaktable-world OakTable World: Sept 23 & 24, San Francisco

How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Embed Size (px)

Citation preview

Page 1: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Thank you for joining us, the webinar will start at:

08:00 Pacific / 11:00 Central / 12:00 East / 16:00 UK Time

How to Enable Risk-Free Database Experiments

• Mind-meld with the illustrious members of the OakTable

• 32 talks over 2 days, right next door to Oracle OpenWorld

More info. and registration: www.kylehailey.com/oaktable-world

OakTable World: Sept 23 & 24, San Francisco

Page 2: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Presenters

Kyle Hailey @kylehhailey • Technical Evangelist at Delphix

• Oracle ACE, member of the OakTable Network

James Murtagh @allthingsoracle • Oracle tools team at Red Gate

• Creator and editor of All Things Oracle.com

Page 3: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Before we start

• You will be on mute for the duration of the event

• We are now talking so please type a message in the Questions box in the Control Panel if you can’t hear us (please check your speakers and GoToWebinar audio settings first)

• There will be a Q+A session at the end but please feel free to type your questions in the Questions box in the Control Panel in advance

• A recording of the full webinar will be put up online

Page 4: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

About Delphix

• Founded in 2008, launched in 2010

• CEO Jedidiah Yueh (founder of Avamar: >$1B revenue))

• Based in Silicon Valley, Global Operations

• 10% of Fortune 500

Page 5: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

About Red Gate

• Over 600,000 technology professionals worldwide use Red Gate’s software, including 93% of Fortune 100 companies.

• Our philosophy is to design highly usable, reliable tools that solve problems commonly faced by DBAs and developers.

www.red-gate.com/oracle

Page 6: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

How to Enable Risk-Free

Database Experiments

Kyle Hailey & James Murtagh

Page 7: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Tradeoff: Speed, Quality, Cost

Page 8: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Good, Cheap, Fast : choose two

Fast

Good Cheap

Page 9: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

What We’ve Seen

1. Inefficient QA: Higher costs of QA 2. QA Delays : Greater re-work of code 3. Sharing DB Environments : Bottlenecks 4. Using DB Subsets: More bugs in Prod 5. Slow Environment Builds: Delays

Page 10: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

1. Inefficient QA: Long Build times

Build Time

QA Test

96% of QA time was building environment

$.04/$1.00 actual testing vs. setup

Build

Page 11: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

2. QA Delays: bugs found late require more code re-work

Build QA Env QA Build QA Env QA

Sprint 1 Sprint 2 Sprint 3

Bug Code X

010203040506070

1 2 3 4 5 6 7

Delay in Fixing the bug

Cost To Correct

Software Engineering Economics – Barry Boehm (1981)

Page 12: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

3. Full Copy Shared : Bottlenecks

Frustration Waiting

Old Unrepresentative Data

Page 13: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

4. Subsets : cause bugs

Page 14: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

3. Subsets : cause bugs

Classic problem is that queries that run fast on subsets hit the wall in production. Developers are unable to test against all data

The Production ‘Wall’

Page 15: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

5. Slow Environment Builds: 3-6 Months to Deliver Data

Management

DBA

System Admin

Storage Admin

Developers Submit Request

Disk Capacity?

Approve Request $$ (2 Weeks)

Approve Request $$

(1 Week)

Request Additional Storage?

Provision Capacity

File System Configured?

Configure LUNS & Build File System

Coordinate Replication w/ Infrastructure

Re-Parameterize &

Configure DB

Mount Recovery DB

to Specific PIT

Begin Work

Approve Request $$ (2 Weeks)

(3 Days)

(3 Days)

(2 Days)

(3 Days)

(3 Days)

…….1-2 Weeks of Approvals, Delays, and Provisioning……

15

Page 16: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

5. Slow Environment Builds: culture of no

Page 17: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

What We’ve Seen

1. Inefficient QA: Higher costs 2. QA Delays : Increased re-work 3. Sharing DB : Bottlenecks 4. Subset DB : Bugs 5. Slow Environment Builds: Delays

Page 18: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

60% Projects Over Schedule and Budget

CIO Magazine Survey:

Data is the problem

Solve the data problem.

TODAY.

Page 19: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

UNLOCK YOUR DATA

Page 20: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Clone 1 Clone 3 Clone 2

99% of blocks are identical

Page 21: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Clone 1 Clone 2 Clone 3

Thin Clone

Page 22: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Virtualization Layer

Virtualization

Page 23: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Three Physical Copies Three Virtual Copies

Page 24: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source
Page 25: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Install Delphix on x86 hardware

Intel hardware

Page 26: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Allocate Any Storage to Delphix

Allocate Storage Any type

Page 27: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

One time backup of source database

Database

Production

Instance

File system File system

Page 28: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

DxFS (Delphix) Compress Data

Database

Production

Instance

Data is compressed typically 1/3 size

File system

Page 29: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Incremental forever change collection

Database

Production

Instance

File system

Changes

• Collected incrementally forever • Old data purged

Time Window

File system

Page 30: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Typical Architecture

Production

Instance

Development

Instance

QA

Instance

UAT

Instance

Database

File system File system

Database

File system File system

Database

File system File system

Database

File system File system

Page 31: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

With Delphix

Database

File system

Production

Instance

Database

Development

Instance

Database

QA

Instance

Database

UAT

Instance

Page 32: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source
Page 33: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Three Core Parts

Production

Instance

Time Window

Instance Virtual Database

1

2

3

1. Source Syncing

2. Storage (DxFS)

3. Self Service

Development

Page 34: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Fast, Fresh, Full

Database

Production

Instance

File system Time Window

Instance Virtual Database

Page 35: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Free

Database

Production

Instance

File system Time Window

Instance Virtual Database

Instance

Instance

Virtual Database

Virtual Database

Page 36: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Branching to QA

Database

Production

Instance

File system Time Window

Instance

Virtual Database

Instance Virtual Database Dev

QA

Page 37: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Self Service

Page 38: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

What We’ve Seen With Delphix

1. Efficient QA: Low cost, high utilization 2. Quick QA : Fast Bug Fix 3. Every Dev gets DB: Parallelized Dev 4. Full DB : Less Bugs 5. Fast Builds: Fast Dev, Culture of Yes

Page 39: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

1. Efficient QA: Lower cost

Build Time

QA Test

1% of QA time was building environment

$.99/$1.00 actual testing vs. setup

Build Time

QA Test

Build

Page 40: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

2. QA Immediate: bugs found fast and fixed

Sprint 1 Sprint 2 Sprint 3

Bug Code X

QA QA

Build QA Env QA Build QA Env QA

Sprint 1 Sprint 2 Sprint 3

Bug Code X

Page 41: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

3. Private Copies: Parallelize

Page 42: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

4. Full Size DB : Eliminate bugs

Page 43: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

5. Self Service: Fast, Efficient. Culture of Yes!

Management

DBA

System Admin

Storage Admin

Developers Submit Request

Disk Capacity?

Approve Request $$ (2 Weeks)

Approve Request $$

(1 Week)

Request Additional Storage?

Provision Capacity

File System Configured?

Configure LUNS & Build File System

Coordinate Replication w/ Infrastructure

Re-Parameterize &

Configure DB

Mount Recovery DB

to Specific PIT

Begin Work

Approve Request $$ (2 Weeks)

(3 Days)

(3 Days)

(2 Days)

(3 Days)

(3 Days)

…….1-2 Weeks of Approvals, Delays, and Provisioning……

Page 44: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

What We’ve Seen With Delphix

1. Efficient QA: Low cost, high utilization 2. Quick QA : Fast Bug Fix 3. Every Dev gets DB: Parallelized Dev 4. Full DB : Less Bugs 5. Fast Builds: Fast Dev, Culture of Yes

Page 45: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

DEPLOY OFTEN, REDUCE ERRORS

Page 46: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Source control reduces deployment errors

• See who changed what, why and when

• Gives context to the development of the db

• Collate changes in sandboxed development

Page 47: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Best practices

• Develop in a sandboxed environment (not usually an issue for app development)

• Get latest often (and test changes) – before checking in

• Check in often – small logical units

• Check in messages should be descriptive

Page 48: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Reducing errors at deployment

• Deploy changes from source control

• Compare source and target schemas before deploying changes

Page 49: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

No need to write update scripts by hand

Page 50: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Benefits of Red Gate tools – shared development db

SVN /

TFS Dev Test UAT, Production etc.

• Removes manual processes to checking in changes to source control • Lock objects on a shared development database while you work on them • Fast and accurate comparison of environments (compare source control to the database) • Generate update scripts to deploy changes accurately

Page 51: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Benefits of Red Gate tools – sandboxed dbs

SVN /

TFS Test UAT, Production etc.

• Removes manual processes to checking in changes to source control • Get latest changes from source control • Fast and accurate comparison of environments (compare source control to the database) • Generate update scripts to deploy changes accurately

Page 52: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Live Demo

• Delphix: Create 3 Virtual Databases

1. Dev1

2. Dev2

3. Trunk

• Have 2 Developers

– Developer 1 modifies “Dev1” and source controls changes

– Developer 2 modifies “Dev2” and source controls changes

• Red Gate Schema Compare for Oracle Merges:

– “Dev1” changes into “Trunk”

– “Dev2” changes into “Trunk”

Page 53: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Time Window

Instance

Instance

Instance

Virtual Database Dev1

Dev2

Trunk

Virtual Database

Virtual Database

Developer 1 modify

Developer 2 modify

Page 54: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Modify

Merge

QA if fail rollback

Merge dev1

Merge dev2

Modify

Merge

Trunk

QA if fail rollback

Page 55: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Fork

Merge to Fork

Merge to dev2 Modify Fork

Modify

If pass QA, merge trunk

Merge to dev1

If pass QA, merge trunk

Merge to Fork

Page 57: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Q&A

Kyle Hailey @kylehhailey

Delphix: delphix.com

James Murtagh @allthingsoracle

Red Gate: red-gate.com/oracle

Page 58: How to Enable Risk-Free Database Experiments to Enable Risk-Free Database Experiments ... • A recording of the full webinar will be put up online . ... One time backup of source

Thanks!

Kyle Hailey @kylehhailey

Delphix: delphix.com

James Murtagh @allthingsoracle

Red Gate: red-gate.com/oracle