NEW: Oracle Real Application Clusters (RAC) and Oracle Clusterware 11g Release 2

NEW: Oracle Real Application Clusters (RAC) and Oracle Clusterware 11g Release 2

Markus MichalewiczProduct Manager Oracle Clusterware

The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions.The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

<Insert Picture Here>

Agenda

• Overview

• Easier Installation– SSH Setup, prerequisite checks, and FixUp-scripts– Automatic cluster time synchronization configuration– OCR & Voting Files can be stored in Oracle ASM

• Easier Management – Policy-based and Role-separated Cluster Management– Oracle EM-based Resource and Cluster Management– Grid Plug and Play (GPnP) and Grid Naming Service – Single Client Access Name (SCAN)

• Summary

• Dedicated silos are inefficient

• Sized for peak load

• Constrained performance

• Difficult to scale

• Expensive to manage

Dedicated Stacks

The Traditional Data CenterExpensive and Inefficient

Grid ComputingVirtualize Pools and Resources

• A virtualized single instance database

• Delivers value of server virtualization to databases on physical servers– Server consolidation– Online upgrade to RAC– Standardized deployment

across all Oracle databases– Built-in cluster failover

for high availability– Live migration of instances across servers– Rolling patches for single instance databases

Oracle RAC One NodeBetter Virtualization for Databases

Oracle Grid InfrastructureThe universal grid foundation

DB Datafiles

DB Datafiles

OCR &Voting Files

OCR &Voting Files

BinariesBinaries

FileSystem

FileSystem

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

• Standardize infrastructure software– Eliminates the need for 3rd-party solutions– Combines Oracle Automatic Storage

Management (ASM) & Oracle Clusterware– Typically used by System Administrators

• Includes:

– Oracle ASM– ASM Cluster File System (ACFS)– ACFS Snapshots– Oracle Clusterware– Cluster Health Manager

CAPEX

OPEX

Servers

Software

Storage

Data Center

Admin

Low cost serversConsolidation

Full Oracle Stack

Fewer disks/LUNs

Consolidation

Easy ProvisioningEasy Management

Oracle Database 11g Release 2 Lowering CapEx and OpEx using Oracle RAC

Oracle RAC

Oracle ASM

Oracle Grid Infrastructure

Easier Installation

• New intelligent installer– 40% fewer steps to install Oracle Real

Application Clusters and Oracle Grid Infra.– Integrated Validation and Automation

• Nodes can be easily repurposed– Nodes can be dynamically added

or removed from the cluster– Network and storage information are read

from profile and configured automatically– No need to manually prepare a node

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

Oracle Database 11g Release 2 Easier Grid Installation and Provisioning

OPEX

Data Center

Admin

Consolidation


Typical and Advanced Installation

Software Only Installation for Oracle Grid Infrastructure

Grid Naming Service (GNS) and Auto assignment of VIPs

SSH Setup, prerequisite checks, and FixUp-scripts

Automatic cluster time synchronization configuration

OCR & Voting Files can be stored in Oracle ASM

1

2

4

5

3

Easier Grid Installation

6

Secure Shell (SSH) Setup …

… CVU-based Prerequisite Checks …

…and FixUp-Scripts

• Time synchronization between cluster nodes is crucial

• Typically, a central time server, accessed by NTP, is used to synchronize the time in the data center

• Oracle provides the Oracle CTSS as an alternative for cluster time synchronization

• CTSS runs in 2 ways:– Observer mode: whenever NTP is installed

on the system, CTSS only observes– Active mode: time in cluster is synchronized

against the CTSS master (node)

Automatic Cluster Time SynchronizationOracle Cluster Time Syncronization Service (CTSS)

Ora

cle

Gri

d In

fras

tru

ctu

re

OCR / Voting Files stored in Oracle ASM

OCR / Voting Files stored in Oracle ASM

Create ASM Disk Group

• The OCR is managed like a datafile in ASM (new type)– It adheres completely to the redundancy settings for the DG

The OCR Managed in Oracle ASM

Voting Files Managed in Oracle ASM

• Unlike the OCR, Voting Files are– Stored on distinguished ASM disks

• ASM auto creates 1/3/5 Voting Files – Based on Ext/Normal/High redundancy

and on Failure Groups in the Disk Group– Per default there is one failure group per disk– ASM will enforce the required number of disks– New failure group type: Quorum Failgroup

[GRID]> crsctl query css votedisk

1. 2 1212f9d6e85c4ff7bf80cc9e3f533cc1 (/dev/sdd5) [DATA]

2. 2 aafab95f9ef84f03bf6e26adc2a3b0e8 (/dev/sde5) [DATA]

3. 2 28dd4128f4a74f73bf8653dabd88c737 (/dev/sdd6) [DATA]

Located 3 voting disk(s).

Easier Management

• Oracle Enterprise Manager (EM) is able to manage the full stack, including Oracle Clusterware– Manage and monitor

clusterware components– Manage and monitor

application resources

• New Grid Concepts:– Server Pools – Grid Plug and Play (GPnP)– Grid Naming Service (GNS)– Auto-Virtual IP assignment– Single Client Access Name

(SCAN)

Oracle Database 11g Release 2 Easier Grid Management

OPEX

Data Center

Admin

Consolidation


OCR & Voting Files can be stored in Oracle ASM

Clusterized Commands

Policy-based and Role-separated Cluster Management

Oracle EM-based Resource and Cluster Management

Grid Plug and Play (GPnP) and Grid Naming Service

Single Client Access Name (SCAN)

2

3

4

5

1

Easier Grid Management

6

New Grid Concept: Server Pools Foundation for a Dynamic Cluster Partitioning

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One

Ora

cle

RA

C D

Bs

FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

• Logical division of a cluster into pools of servers.

• Hosts applications (which could be databases or applications)

Why Use Server Pools?

• Easy allocation of resources to workload

• Easy management of Oracle RAC – Just define instance requirements

(# of nodes – no fixed assignment)

• Facilitates Consolidation of Applications and Databases on Clusters

• Policy-based management uses server pools to– Enable dynamic capacity assignment when needed– Ensure isolation

where necessary (“dedicated servers in a cluster”)

• In order to guarantee:– Applications get the required minimum resources

(whenever possible)– Applications do not “take” resources from

more important applications

Resource management without policies

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

FREEFREE

RAC DB1

RAC DB2

RAC DB1

RAC DB2

RAC

One

RAC

One

Policy-based Cluster Management Ensure Isolation based on Server Pools

Resource management without policies

Resource management with policies

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One

Ora

cle

RA

C D

Bs

FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

Policy-based Cluster Management Ensure Isolation based on Server Pools

• Policy-based management uses server pools to– Enable dynamic capacity assignment when needed– Ensure isolation

where necessary (“dedicated servers in a cluster”)

• In order to guarantee:– Applications get the required minimum resources

(whenever possible)– Applications do not “take” resources from

more important applications

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One

Ora

cle

RA

C D

Bs

FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

• A Server Pool is defined by 4 attributes:

1. Server Pool Name

2. Min – specifies the “minimum” number of servers that should run in the server pool

3. Max – states the “maximum” number of servers that can run in the server pool.

4. Imp – “importance” specifies the relative importance between server pools. This parameter is of relevance at the time of the server assignment to server pools or when servers need to be re-shuffled in the cluster due to failures.

Enable Policy-based Cluster Management Define Server Pools using the appropriate Definition

Role-separated Cluster Management

• Addresses organizations with strict separation of duty

• Role-separated management is implemented in 2 ways:1. Vertically: Use a different user (groups) for each layer in the stack

2. Horizontally: ACLs on server pools for policy-managed DBs / Apps.

• The default installation assumes no separation of duty

Oracle RAC DBs DBAn User

RAC DB1RAC DB1 Siebel Siebel RAC DB2RAC DB2PSFT PSFT

Oracle Grid Infrastructure

OS User

Grid User

DBA1 DBA2 Adm1

Oracle EM – Integrated Server Pool Mgmt

• GPnP eliminates the need for a per node configuration– It is an underlying grid concept that enables

the automation of operations in the cluster– Allows nodes to be dynamically added or removed from the cluster– Provides an easier management to build large clusters– It is the basis for the Grid Naming Service (GNS)

• Technically, GPnP is based on an XML profile– Defining node personality (e.g. cluster name, network classification)– Created during installation– Updated with every relevant change (using oifcfg, crsctl)– Stored in local files per home and in the OCR– Wallet protected

• GPnP is apparent in things that you do not see and that you are not asked for (anymore).

Grid Plug and Play (GPnP) Foundation for a Dynamic Cluster Management

OUI does not ask for a private node name anymore.

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

Grid Naming Service (GNS) Dynamic Virtual IP and Naming

• The Grid Naming Service (GNS) allows dynamic name resolution in the cluster

• The Cluster manages its own virtual IPs– Removes hard coded node information– No VIPs need to be requested, if cluster changes

• Enables nodes to be dynamically added or removed from the cluster

• Defined in the DNS as a “delegated domain”– Mycluster.myco.com– DHCP provides IPs inside delegated domain

• Benefit: Reduced configuration for VIPs in the cluster

• Defined in the DNS as a “delegated domain” – DNS delegates request to mycluster.myco.com to GNS

• Needs its own IP address (the GNS VIP)– This is the only “NAME IP” assignment required in DNS

• All other VIPs, and SCAN-VIPs are defined in the GNS for a cluster– DHCP is used for dynamic IP assignment

• Optional way of resolving addresses– Requires novel configuration by DNS administrator

Grid Naming Service (GNS) Steps to set up GNS

DNS

client

delegated cluster domain

SCAN listeners

Local listeners

Oracle RAC cluster - GRIDA

GNSDHCP server

dynamic VIP assignment

Grid Naming Service Client Connect

corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


1


corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


2

1


corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


3

1

2


corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


4

1

2

3


corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


5

1

2

3

4


corporate domain

DNS

client


SCAN listeners

Local listeners


GNSDHCP server


1

6

2

3

4

5


corporate domain

• Used by clients to connect to any database in the cluster

• Removes the requirement to change the client connection if cluster changes

• Load balances across the instances providing a service

• Provides failover between “moved instances”

Clu

ster

SC

AN

na

me

Siebel

PSFT

Siebel

PSFT

Ora

cle

Gri

d In

fras

tru

ctu

re

RAC

One

RAC

One FR

EE

FR

EE

RAC

DB2

RAC

DB2

RAC

DB1

RAC

DB1

Single Client Access Name (SCAN) The New Database Cluster Alias

• Requires a DNS entry or GNS to be used• In DNS, SCAN is a single name defined to resolve to 3 IP-addresses:

• Each cluster will have 3 SCAN-Listeners,combined with a SCAN-VIP defined as cluster resources

• The SCAN VIP/LISTENER combination will failover to another node in the cluster, if the current node fails

Cluster Resources--------------------------------------------ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE node1

ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE node2

ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE node3

Single Client Access Name Network Configuration for SCAN

clusterSCANname.example.com IN A 133.22.67.194

IN A 133.22.67.193

IN A 133.22.67.192

PMRAC =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = node1)(PORT = 1521)) …

(ADDRESS = (PROTOCOL = TCP)(HOST = nodeN)(PORT = 1521))

(CONNECT_DATA =

… ))

Single Client Access Name Easier Client Configuration

PMRAC =

(DESCRIPTION =

(ADDRESS = (PROTOCOL = TCP)(HOST = clusterSCANname)(PORT = 1521))

(CONNECT_DATA =

… ))

• Without SCAN (pre-11g Rel. 2) TNSNAMES has 1 entry per node• With every cluster change, all client TNSNAMES need to be changed

• With SCAN only 1 entry per cluster is used, regardless of the # of nodes:

Application Server

Connection Load Balancing using SCAN

Clients

SCANListeners

Local Listeners

Oracle RAC Database Cluster

Application Server

Connection Load Balancing using SCAN

Clients

SCANListeners

Oracle RAC Database Cluster

Local Listeners

Summary

RAC RAC

RAC RAC RAC RAC

Shared Infrastructure

Dedicated Infrastructure

Oracle RAC Oracle RAC for HAfor HA

Lower the cost Lower the cost of HAof HA

Lower the Lower the cost scalabilitycost scalability

Oracle RAC for Oracle RAC for

Scale out Scale out

Shared Cluster Shared DatabaseShared Cluster Shared DatabaseShared StorageShared Storage

Lower the infrastructure costs Lower the infrastructure costs Improved utilizationImproved utilization Storage consolidationStorage consolidation Management efficiency (Shared DB)Management efficiency (Shared DB)

Lower the cost of Lower the cost of deploymentsdeploymentsLower CAPEXLower CAPEXLower OPEXLower OPEX

RAC

RAC

RAC RACRAC

RAC RAC

EM

RAC RAC RACRAC RACRAC RAC

ASAS

ASAS

ASAS AS

EM

EM

EMRAC

RAC

RAC RACRAC

RAC RAC

EM

StandardizedInfrastructure

Datacenter Datacenter

GridGridGRID for DBGRID for DB

The Evolution of the Grid Lowering the Cost of Database Deployments

<Insert Picture Here>

Questionsand

Answers

Documents

NEW: Oracle Real Application Clusters (RAC) and Oracle Clusterware 11g Release 2