Embed Size (px)
Citation preview
Real Application Clusters
Trevor Crljenko
Emilija Knezevic
Objectives
• Background Information
• A Working Database
• Database Creation and Maintenance
• Summary
• Q & A
Real Application Clusters
• Emilija and Trevor working on the Linux implementation of the RAC
Architectural Characteristics
• Each instance has its own:– SGA and background processes– Redo Logs– Undo Tablespaces
• Data files and control files are shared
• Cache Fusion guarantees cache coherency among cluster nodes
Basic RAC Layout
Instance A
Comm Layer
Shared Disk Driver
Node 2
Cluster Interconnect
Shared Disk Subsystem
Oracle Cluster
Manager
Node 1
Shared Disk Driver
Comm Layer
Oracle Cluster
Manager
Instance B
Oracle Cluster Manager
• Operating System dependant
• Provides a cluster-wide view of node membership– discovers new nodes– tracks the state of existing nodes– coordinates session tracking when a node fails
Cache Fusion
• Enables sharing of database caches among multiple instances to avoid I/O calls
• Synchronizes access to the database
• Implemented through GCS (Global Cache Services)
The Cluster Interconnect
• Supports the cluster and Cache Fusion
• Can use:– TCP/IP over Ethernet– User Datagram Protocol (UDP) over Ethernet– Virtual Interface Architecture over a Gigabit network
Limitations?
• The use of Transparent Application Failover (also called TAF) has a few limitations:– Uncommitted transactions must be rolled back– Global temporary tables are lost– PL/SQL package states are lost– The effect of ALTER SESSION is lost
A working RAC on the Linux OS
A demonstration of a Real Application Cluster Database
The Environment
• Node1: TPOL instance: RAC1• Node2: SCOTTY instance: RAC2• Database name: RAC1• Clients:
– Referee - runs a heavy query– Scott
• Both clients on machine TORRES• The query takes approx. 4 min. to execute by one
uninterrupted instance
Scripts Used – gvinstance
SELECT inst_id
, instance_number inst#
, instance_name name
, host_name host
, startup_time startup
, status
, parallel
, thread#
, logins
, database_status dbstat
, instance_role
, active_state state
FROM gv$instance
Scripts Used – gvsession
SELECT inst_id , sid , username , user# , server , machine , program , logon_time FROM gv$session WHERE username IS NOT NULL
Scripts Used – gvtaf
SELECT inst_id , sid , username , machine , failover_type , failover_method , failed_over FROM gv$session WHERE username IS NOT NULL AND username != 'SYS'
Scripts Used – gvbeen_here
SELECT bh.inst_id
, o.owner
, o.object_name
, o.object_type
, count(distinct bh.block#) "Num. Buffers"
FROM dba_objects o
, gv$bh bh
WHERE o.object_id = bh.objd
AND owner != 'SYS'
AND owner != 'SYSTEM'
GROUP BY o.owner, o.object_name, o.object_type, bh.inst_id
ORDER BY 2, 4, 5 DESC
The Demo
• Our RAC
TEST CASE
Start UpTwo Instances
ConnectClient
Start The Query
Shut DownA Instance
Start Up TheAborted Inst.
ConnectAnother Client
Demonstrate The
Load Balancing
Stage 1 Setup
Stage 2: Availability
Stage 3: Load Balance
Server SCOTTY
ServerTPOL
User REFEREE
Real Application ClusterTESTING SETUP
Step 1: Start Up Both INSTANCES - TPOL & SCOTTY
SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sql
SYSDATE
-------------------
2004-01-14:12:24:21
INST SYS HOST USER USER# SERVER PROGRAM
ID ID NAME NAME
------- ------ --------- --------- ------------ ------------------- ------------------
1 22 tpol SYS 0 DEDICATED sqlplus@tpol (TNS V1-V3)
1 18 tpol SYS 0 DEDICATED oracle@tpol (P000)
2 18 scotty SYS 0 DEDICATED sqlplus@scotty (TNS V1-V3)
2 23 scotty SYS 0 DEDICATED oracle@scotty (P000)
SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sqlINST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATEID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- --------- 2 2 RAC2 scotty 2004-01-14:10:01:17 OPEN PRIMARY_INSTANCE NORMAL 1 1 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL
2 23 scotty REFEREE 27 DEDICATED oracle@torres (TNS V1-V3)
Step 2: Connect User REFEREE to INSTANCE SCOTTY
REFEREE@TORRES> select * 2 from tab 3 /TNAME TABTYPE CLUSTERID------------------------------------------------------------------------------------------ ---------------CERTIFICATIONS TABLEPERSONS TABLEPERSON_CERTIFICATIONS TABLEREF_ASSIGNMENT_REQUEST_QUEUE TABLEROLE_APPLICANTS TABLERUN_TIMES TABLESTATS_CERTS_1 TABLE STATS_CERTS_2 TABLE8 rows selected
REFEREE@TORRES> @ $RAC_SCRIPTS/sysdateSYSDATE-------------------2004-01-14:12:37:44
Step 3: Start the Query by User REFEREE
Step 4: Shut Down INSTANCE SCOTTY Transparent Application Failover
User REFEREE
Server SCOTTY
Server TPOL
BEFORE Shut Down INSTANCE SCOTTY
SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql
INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATEID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- ---------
1 1 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL
SYS@SCOTTY> @ $RAC_SCRIPTS/inst_no
INSTANCE_NUMBER--------------- 2
SYS@SCOTTY> @ $RAC_SCRIPTS/sysdate
SYSDATE-------------------2004-01-14:12:20:07
2 2 RAC2 scotty 2004-01-14:10:01:17 OPEN PRIMARY_INSTANCE NORMAL
AFTER Shut Down INSTANCE SCOTTY
Server SCOTTY
Server TPOL
Transparent Application Failover
SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sqlINST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATEID # NAME NAME ------- ------ --------- --------- --------------------------- --------- ------------------------- ---------1 1 RAC1 tpol 2004-01-14:11:21:57 OPEN PRIMARY_INSTANCE NORMAL
SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sqlSYSDATE-------------------2004-01-14:12:24:21
INST SYS HOST USER USER# SERVER PROGRAM ID ID NAME NAME ------- ------ --------- --------- ------------ ------------------- ------------------1 18 tpol REFEREE 27 DEDICATED sqlplus@torres (TNS V1-V3) 1 22 tpol SYS 0 DEDICATED sqlplus@tpol (TNS V1-V3)
Referee
SYS@SCOTTY> @ $RAC_SCRIPTS/gvinstance.sql
INST INST INST HOST STARTUP STATUS INSTANCE_ROLE STATE
ID # NAME NAME
------- ------ --------- --------- --------------------------- --------- ------------------------- ---------
2 2 RAC2 scotty 2004-01-14:12:30:41 OPEN PRIMARY_INSTANCE NORMAL
SYS@SCOTTY> @ $RAC_SCRIPTS/gvsessiont.sql
INST SYS HOST USER USER# SERVER PROGRAM
ID ID NAME NAME
------- ------ ---------- --------- --------- -------------- -------------------------------------
2 18 scotty SYS 0 DEDICATED sqlplus@scotty (TNS V1-V3)
STEP 6: Start Up the Aborted INSTANCE SCOTTY
Step 7: Connect User SCOTT Load Balancing
Server SCOTTY
ServerTPOL
User REFEREEUser SCOTT
SYS@TPOL> @ $RAC_SCRIPTS/gvsessiont.sql
SYSDATE
-------------------
2004-01-14:12:24:21
INST SYS HOST USER USER# SERVER PROGRAM
ID ID NAME NAME
------- ------ --------- --------- ------------ ------------------- ------------------
1 18 tpol REFEREE 27 DEDICATED sqlplus@torres (TNS V1-V3)
1 22 tpol SYS 0 DEDICATED sqlplus@tpol (TNS V1-V3)
1 25 tpol SYS 0 DEDICATED oracle@tpol (P000) 2 18 scotty SYS 0 DEDICATED sqlplus@scotty (TNS V1-V3)
2 19 scotty SYS 0 DEDICATED oracle@scotty (P000)
2 24 scotty SCOTT 26 DEDICATED sqlplus@torres (TNS V1-V3)
Step 7: Connect User SCOTT Load Balancing
RAC Database Creation
• Verify that the shared disks have been configured correctly
• Configure the Oracle network services
• Start the listeners and test the configuration
• Create the database
• Add and set up additional instances
Database Creation and Maintenance
An example of a Real Application Cluster installed on 3 PC’s using SuSE
Linux 8.1
RAC Specific Parameters• In the INIT.ORA file
*.db_name=RAC1*.cluster_database=true*.cluster_database_instances=2rac1.instance_name='RAC1'rac1.instance_number=1rac1.thread=1rac1.undo_tablespace='UNDOTBS1'rac2.instance_name='RAC2'rac2.instance_number=2rac2.thread=2rac2.undo_tablespace='UNDOTBS2'rac1.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))'rac1.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))'rac2.local_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = scotty)(PORT = 1521))'rac2.remote_listener='(ADDRESS=(PROTOCOL = TCP)(HOST = tpol)(PORT = 1521))’
RAC Specific Issues
• Preconditions for running the CREATE DATABASE script:– GSD should be running on both nodes– Listeners should be running on both nodes
•tnsping was used to verify the network configuration• An UNDO tablespace should be created for each node in the cluster• Cluster redo logs:
– Each instance has its own thread of redo log files– There are an equal number of redo log groups for each node– All log files must be placed on the shared storage
• Run the catalog scripts for the RAC database:@ $ORACLE_HOME/rdbms/admin/catclust.sql
Add Another Instance
• The database should be started using the PFILE• Create a password file, and a PFILE for the next
instance• Start the instance RAC2 from the PFILE• Create an SPFILE from the PFILE of RAC1• In both PFILEs, remove everything but the pointer to the
SPFILE• Each instance has its own password file, but they MUST
be synchronized across all instances
Configuring Transparent Application Failover (TAF)
• TAF instructs Oracle Net to transfer a failed connection to a different listener
• A user can continue to work using the new connection• TAF configuration:
– tnsnames.ora - on both server and client side– set up the 'Local Listener' and 'Remote Listener' in the init file
• Configuration that works:(CONNECT_DATA =
(SERVICE_NAME=RAC1.eis.ca) (FAILOVER_MODE = (TYPE = SELECT) (METHOD = BASIC) (RETRIES = 64) (DELAY = 4))))
REDO Logs and ThreadsSYS@TPOL> @ $RAC_SCRIPTS/gvlog.sql
INST_ID GROUP# THREAD# SEQ# BYTES MEMBERS ARC STATUS FIRST_CHAN FIRST_TIME
------- ------ ------- ------ ---------- ---------- --- -------- ---------- ---------------
2 1 1 61 20971520 2 YES INACTIVE 571934296 04-01-14:12:22
2 2 1 62 20971520 2 NO CURRENT 571975370 04-01-14:12:35
2 3 1 60 20971520 2 YES INACTIVE 571810001 04-01-13:06:33
2 4 2 49 20971520 2 YES INACTIVE 571934293 04-01-14:12:22
2 5 2 50 20971520 2 NO CURRENT 571954935 04-01-14:12:31
2 6 2 48 20971520 2 YES INACTIVE 571905460 04-01-14:10:01
1 1 1 61 20971520 2 YES INACTIVE 571934296 04-01-14:12:22
1 2 1 62 20971520 2 NO CURRENT 571975370 04-01-14:12:35
1 3 1 60 20971520 2 YES INACTIVE 571810001 04-01-13:06:33
1 4 2 49 20971520 2 YES INACTIVE 571934293 04-01-14:12:22
1 5 2 50 20971520 2 NO CURRENT 571954935 04-01-14:12:31
1 6 2 48 20971520 2 YES INACTIVE 571905460 04-01-14:10:01
12 rows selected.
Switching from NOARCHIVELOG to ARCHIVELOG mode
• There should be only one instance running, and the instance should be mounted in EXCLUSIVE mode
• The only way to mount the instance in EXCLUSIVE mode is to set the cluster_database parameter to FALSE)
Archivelog ListSYS@TPOL> archive log listDatabase log mode Archive ModeAutomatic archival EnabledArchive destination /var/opt/oracle/RAC1/archiveOldest online log sequence 58Next log sequence to archive 60Current log sequence 60
SYS@SCOTTY> archive log listDatabase log mode Archive ModeAutomatic archival EnabledArchive destination /var/opt/oracle/RAC1/archiveOldest online log sequence 45Next log sequence to archive 47Current log sequence 47
Backups with RMAN
connected to target database: RAC1 (DBID=410737804)
Starting backup at 2004-01-13:14:25:44channel t1: starting full datafile backupsetchannel t1: specifying datafile(s) in backupsetincluding current controlfile in backupsetinput datafile fno=00001 name=/var/opt/oracle/RAC1/system/system01.dbfinput datafile fno=00002 name=/var/opt/oracle/RAC1/undo/undotbs1.dbfinput datafile fno=00005 name=/var/opt/oracle/RAC1/undo/undotbs2.dbfinput datafile fno=00004 name=/var/opt/oracle/RAC1/data/users01.dbfchannel t1: starting piece 1 at 2004-01-13:14:26:20channel t1: finished piece 1 at 2004-01-13:15:01:48piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_28_1_515341546.bak comment=NONEchannel t1: backup set complete, elapsed time: 00:36:02channel t1: starting full datafile backupsetchannel t1: specifying datafile(s) in backupsetinput datafile fno=00003 name=/var/opt/oracle/RAC1/index/index01.dbfchannel t1: starting piece 1 at 2004-01-13:15:01:49channel t1: finished piece 1 at 2004-01-13:15:02:04piece handle=/var/opt/oracle/RAC1/backup/rman/RAC1_29_1_515343709.bak comment=NONEchannel t1: backup set complete, elapsed time: 00:00:15Finished backup at 2004-01-13:15:02:04
Summary
• Architecture components
• How does it work in practice?
• Creating and Maintaining a Real Application Cluster Database
Additional Sources of Information
• Oracle Online Documentation: Real Application Clusters Concepts
• Oracle Metalink document # 184821.1: Step-By-Step Installation of 9.2.0.4 RAC on Linux
• Linux SuSE 8.1 documentation (http://sdb.suse.de/en/sdb/html/)
• Bill Garner, Wally Pereira, Gary Dodge: "Oracle9i Real Application Clusters (RAC) with Red Hat Linux Advanced Server* 2.1 Powered by Intel Itanium 2 Processors", Oracle World, San Francisco, 2003.
Our Thanks
• Enterprise Information Systems• OOUG
• Contact Information– Emilija Knezevic
– Trevor Crljenko• [email protected]
Questions&Answers
