Installing Oracle RAC 10g Release 2 on Linux x86

8/18/2019 Installing Oracle RAC 10g Release 2 on Linux x86

1/24

alling Oracle RAC 10g on Linux x86

//www.oracle.com/technetwork/articles/smiley-rac10g-install-082032.html[22/09/2011 20:09:35]

DBA: LinuxInstalling Oracle RAC 10 g Release 2 on Linux x86

by John Smiley

Learn the basics of installing Oracle RAC 10 g Release 2 on Red Hat Enterprise Linux or Novell SUSE Enterprise Linux, fromthe bare metal up (for evaluation purposes only)

Contents

Overview Background Part I: Install Linux Part II: Configure Linux for Oracle Part III: Prepare the Shared DisksPart IV: Install Oracle RAC SoftwareConclusion

OverviewThis guide provides a walkthrough of installing an Oracle Database 10 g Release 2 RAC database on commodity hardwarefor the purpose of evaluation . If you are new to Linux and/or Oracle, this guide is for you. It starts with the basics and walksyou through an installation of Oracle Database 10 g Release 2 RAC from the bare metal up.

This guide will take the approach of offering the easiest paths, with the fewest number of steps, for accomplishing a task.This approach often means making configuration choices that would be inappropriate for anything other than an evaluation.

For that reason, this guide is not appropriate for building production-quality environments, nor does it reflect best practices.

The three Linux distributions certified for Oracle 10 g Release 2 RAC are:

Red Hat Enterprise Linux 4 (RHEL4)


Novell SUSE Linux Enterprise Server 9

We will cover both of the Linux 2.6 kernel-based distributions: RHEL4 and SLES9. RHEL3 is not covered here.This guide is divided into four parts: Part I covers the installation of the Linux operating system, Part II covers configuringLinux for Oracle, Part III discusses the essentials of partitioning shared disk, and Part IV covers installation of the Oraclesoftware.

A

Release 1 version of this guide is also available.

BackgroundThe illustration below shows the major components of an Oracle RAC 10 g Release 2 configuration. Nodes in the cluster aretypically separate servers (hosts).

Hardware At the hardware level, each node in a RAC cluster shares three things:

1. Access to shared disk storage

2. Connection to a private network

http://www.oracle.com/technetwork/articles/smiley-rac10gr1-install-098533.htmlhttp://www.oracle.com/technetwork/articles/smiley-rac10gr1-install-098533.html


2/24



3. Access to a public network.

Shared Disk StorageOracle RAC relies on a shared disk architecture. The database files, online redo logs, and control files for the database mustbe accessible to each node in the cluster. The shared disks also store the Oracle Cluster Registry and Voting Disk(discussed later). There are a variety of ways to configure shared storage including direct attached disks (typically SCSI over copper or fiber), Storage Area Networks (SAN), and Network Attached Storage (NAS).

Private NetworkEach cluster node is connected to all other nodes via a private high-speed network, also known as the cluster interconnect or high-speed interconnect (HSI). This network is used by Oracle's Cache Fusion technology to effectively combine the physicalmemory (RAM) in each host into a single cache. Oracle Cache Fusion allows data stored in the cache of one Oracle instance

to be accessed by any other instance by transferring it across the private network. It also preserves data integrity and cachecoherency by transmitting locking and other synchronization information across cluster nodes.

The private network is typically built with Gigabit Ethernet, but for high-volume environments, many vendors offer proprietarylow-latency, high-bandwidth solutions specifically designed for Oracle RAC. Linux also offers a means of bonding multiplephysical NICs into a single virtual NIC (not covered here) to provide increased bandwidth and availability.

Public NetworkTo maintain high availability, each cluster node is assigned a virtual IP address (VIP). In the event of node failure, the failednode's IP address can be reassigned to a surviving node to allow applications to continue accessing the database throughthe same IP address.

Configuring the Cluster HardwareThere are many different ways to configure the hardware for an Oracle RAC cluster. Our configuration here uses two serverswith two CPUs, 1GB RAM, two Gigabit Ethernet NICs, a dual channel SCSI host bus adapter (HBA), and eight SCSI disksconnected via copper to each host (four disks per channel). The disks were configured as Just a Bunch Of Disks (JBOD)— that is, with no hardware RAID controller.

Software At the software level, each node in a RAC cluster needs:

1. An operating system

2. Oracle Clusterware

3. Oracle RAC software

4. An Oracle Automatic Storage Management (ASM) instance (optional).

Operating SystemOracle RAC is supported on many different operating systems. This guide focuses on Linux. The operating system must beproperly configured for the OS--including installing the necessary software packages, setting kernel parameters, configuring

the network, establishing an account with the proper security, configuring disk devices, and creating directory structures. All these tasks are described in this guide.

Oracle Cluster Ready Services becomes Oracle ClusterwareOracle RAC 10 g Release 1 introduced Oracle Cluster Ready Services (CRS), a platform-independent set of system servicesfor cluster environments. In Release 2, Oracle has renamed this product to Oracle Clusterware.

Clusterware maintains two files: the Oracle Cluster Registry (OCR) and the Voting Disk. The OCR and the Voting Disk mustreside on shared disks as either raw partitions or files in a cluster filesystem. This guide describes creating the OCR andVoting Disks using a cluster filesystem (OCFS2) and walks through the CRS installation.

Oracle RAC SoftwareOracle RAC 10 g Release 2 software is the heart of the RAC database and must be installed on each cluster node.Fortunately, the Oracle Universal Installer (OUI) does most of the work of installing the RAC software on each node. Youonly have to install RAC on one node—OUI does the rest.

Oracle Automatic Storage Management (ASM) ASM is a new feature in Oracle Database 10 g that provides the services of a filesystem, logical volume manager, andsoftware RAID in a platform-independent manner. Oracle ASM can stripe and mirror your disks, allow disks to be added or removed while the database is under load, and automatically balance I/O to remove "hot spots." It also supports direct andasynchronous I/O and implements the Oracle Data Manager API (simplified I/O system call interface) introduced in Oracle9 i.

Oracle ASM is not a general-purpose filesystem and can be used only for Oracle data files, redo logs, control files, and theRMAN Flash Recovery Area. Files in ASM can be created and named automatically by the database (by use of the OracleManaged Files feature) or manually by the DBA. Because the files stored in ASM are not accessible to the operating system,the only way to perform backup and recovery operations on databases that use ASM files is through Recovery Manager (RMAN).

ASM is implemented as a separate Oracle instance that must be up if other databases are to be able to access it. Memoryrequirements for ASM are light: only 64MB for most systems. In Oracle RAC environments, an ASM instance must berunning on each cluster node.

Part I: Installing LinuxInstall and Configure Linux as described in the

first guide in this series. You will need three IP addresses for each server: one for the private network, one for the publicnetwork, and one for the virtual IP address. Use the operating system's network configuration tools to assign the private andpublic network addresses. Do not assign the virtual IP address using the operating system's network configuration tools; this

http://www.oracle.com/technetwork/articles/smiley-10gdb-install-092939.htmlhttp://www.oracle.com/technetwork/articles/smiley-10gdb-install-092939.html


3/24



will be done by the Oracle Virtual IP Configuration Assistant (VIPCA) during Oracle RAC software installation.


Required Kernel:2.6.9-11.EL or higher

Verify kernel version:

# uname - r

2. 6. 9- 22. ELsmp

Other required package versions (or higher):bi nut i l s - 2. 15. 92. 0. 2 - 10. EL4

compat - db- 4.1. 25- 9

cont r ol - cente r - 2. 8. 0 - 12

gcc-3 .4 . 3 - 9. EL4

gcc- c++- 3. 4. 3- 9. EL4

gl i bc - 2. 3. 4 - 2

gl i bc- common- 2. 3. 4- 2

gnome- l i bs- 1. 4. 1. 2. 90- 44. 1

l i bst dc++- 3. 4. 3- 9. EL4

l i bst dc++- devel - 3. 4. 3- 9. EL4

make- 3. 80- 5

pdksh- 5. 2.14- 30

s ys st a t - 5. 0. 5 - 1

xscreensaver - 4. 18-5. rhel 4. 2 Verify installed packages:

# r pm - q bi nuti l s compat - db cont r ol - cent er gcc gcc- c++ gl i bc gl i bc- common \

gnome- l i bs l i bstdc++ l i bstdc++- devel make pdksh sysst at xscreensaver

bi nut i l s - 2. 15. 92. 0. 2 - 1 5

compat - db- 4.1. 25- 9

cont r ol - cente r - 2. 8. 0 - 12. rhel 4. 2

gcc-3 .4 . 4 - 2

gcc- c++- 3. 4. 4- 2

gl i bc - 2. 3. 4 - 2 . 1 3

gl i bc- common- 2. 3. 4- 2. 13

gnome- l i bs- 1. 4. 1. 2. 90- 44. 1

l i bst dc++- 3. 4. 4- 2

l i bst dc++- devel - 3. 4. 4- 2

make- 3. 80- 5


4/24



pdksh- 5. 2. 14- 30. 3

s ys st a t - 5. 0. 5 - 1

xscreensaver - 4. 18-5. rhel 4. 9

SUSE Linux Enterprise Server 9 (SLES9)Required Package Sets:Basis Runtime SystemYaSTGraphical Base SystemLinux ToolsKDE Desktop EnvironmentC/C++ Compiler and Tools (not selected by default)

Do not install: Authentication Server (NIS, LDAP, Kerberos)

Required Kernel:2.6.5-7.97 or higher

Verify kernel version:

# uname - r

2. 6. 5- 7. 97- smp

Other required package versions (or higher):gcc-3 .3

gcc- c++- 3. 3. 3- 43

gl i bc - 2. 3. 3 - 9 8. 28

l i bai o - 0 . 3. 98 - 1 8

l i bai o - d evel - 0. 3. 98- 18

make- 3. 80

openmot i f - l i bs - 2. 2. 2 - 519. 1 Verify installed packages:

# r pm - q gcc gcc- c++ gl i bc l i bai o l i bai o- devel make openmoti f - l i bs

gcc- 3. 3. 3- 43. 24

gcc- c++- 3. 3. 3- 43. 24

l i bai o - 0 . 3. 98 - 1 8. 3

l i bai o - d evel - 0. 3. 98- 18. 3

make- 3. 80- 184. 1

openmot i f - l i bs - 2. 2. 2 - 519. 1

Part II: Configure Linux for OracleCreate the Oracle Groups and User Account

Next we'll create the Linux groups and user account that will be used to install and maintain the Oracle 10 g Release 2software. The user account will be called 'oracle' and the groups will be 'oinstall' and 'dba.' Execute the following commandsas root on one cluster node only:

/ usr / sbi n/ groupadd oi nst al l

/ usr/ sbi n/ groupadd dba


5/24



/ us r / sbi n/ useradd -m - g oi ns ta l l - G dba oracl e

i d or a cl e

Ex:# / usr / sbi n/ gr oupadd oi nst al l

# / usr/ sbi n/ groupadd dba

# / us r / sbi n/ useradd - m - g oi ns ta l l - G dba oracl e

# i d or acl e

ui d=501( oracl e) gi d=501( oi nst al l ) gr oups=501( oi nstal l ) , 502( dba)

The User ID and Group IDs must be the same on all cluster nodes. Using the information from the i d or a cl e command,create the Oracle Groups and User Account on the remaining cluster nodes:/ usr / sbi n/ groupadd - g 501 oi nst al l

/ usr/ sbi n/gr oupadd - g 502 dba

/ us r / sbi n/ useradd - m - u 501 - g oi ns ta l l - G dba oracl e

Ex:# / usr / sbi n/ gr oupadd - g 501 oi nst al l

# / usr/ sbi n/ groupadd - g 502 dba

# / usr/ sbi n/ user add - m - u 501 - g oi nst al l - G dba or acl e

# i d or acl e

ui d=501( oracl e) gi d=501( oi nst al l ) gr oups=501( oi nstal l ) , 502( dba)

Set the password on the oracle account:

# passwd or acl e

Changi ng password f or user oracl e.

New passwor d:

Retype new passwor d:

passwd: al l aut hent i cat i on t okens updated successf ul l y.

Create Mount Points

Now create mount points to store the Oracle 10 g Release 2 software. This guide will adhere to the Optimal Flexible

Architecture (OFA) for the naming conventions used in creating the directory structure. For more information on OFAstandards, see Appendix D of the Oracle Database 10 g Release 2 Installation Guide .Issue the following commands as root:

mkdi r - p /u01/app/ oracl e

chown - R oracl e: oi nstal l / u01/ app/ or acl e

chmod - R 775 / u01/ app/ oracl e

Ex:# mkdi r - p / u01/ app/ oracl e

# chown - R oracl e:oi nst al l / u01/app/ oracl e

# chmod - R 775 / u01/ app/ oracl e

Configure Kernel Parameters

http://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/install.102/b15660/toc.htm


6/24



Login as root and configure the Linux kernel parameters on each node.cat >> / etc/ sysct l . conf > / etc/ securi t y/l i mi t s. conf > / et c/ pam. d/ l ogi n > / etc/ prof i l e


7/24



ul i mi t - u 16384 - n 65536

f i

umask 022

f i

EOF

cat >> / et c/ csh. l ogi n > / etc/ prof i l e. l ocal > / etc/ csh. l ogi n. l ocal


8/24



modpr obe hangcheck- t i mer hangcheck_ t i ck=30 hangcheck_mar gi n=180

cat >> / etc/ r c. d/r c. l ocal > / etc/ i ni t . d/ boot . l ocal


9/24



$ /us r / bi n/ s sh- keygen - t r sa

Gener at i ng publ i c/pri vat e rsa key pai r.

Ent er f i l e i n whi ch to save t he key ( / home/oracl e/ . ssh/ i d_r sa) :

Ent er passphr ase ( empty f or no passphr ase) :

Ent er same passphr ase agai n:

Your i dent i f i cat i on has been saved i n / home/ or acl e/ . ss h/ i d_r sa.

Your publ i c key has been saved i n / home/ or acl e/ . ss h/ i d_r sa. pub.

The key f i ngerpr i nt i s:

4b:df : 76:77: 72:ba: 31: cd: c4: e2: 0c: e6: ef: 30: f c: 37 oracl e@ds1. orademo.or g

$ / usr/ bi n/ ssh- keygen - t dsa

Generati ng publ i c/ pri vat e dsa key pai r .

Ent er f i l e i n whi ch to save the key ( / home/oracl e/ . ssh/ i d_dsa) :

Ent er passphr ase ( empty f or no passphr ase) :

Ent er same passphr ase agai n:

Your i dent i f i cat i on has been saved i n / home/ or acl e/ . ss h/ i d_ds a.

Your publ i c key has been saved i n / home/ or acl e/ . ss h/ i d_dsa. pub.

The key f i ngerpr i nt i s:

af : 37: ca: 69: 3c:a0: 08: 97: cb: 9c: 0b: b0: 20: 70: e3: 4a oracl e@ds1. orademo. org

Now the contents of the public key files id_rsa.pub and id_dsa.pub on each node must be copied to the~/.ssh/authorized_keys file on every other node. Use ssh to copy the contents of each file to the ~/.ssh/authorized_keys file.Note that the first time you access a remote node with ssh its RSA key will be unknown and you will be prompted to confirmthat you wish to connect to the node. SSH will record the RSA key for the remote nodes and will not prompt for this onsubsequent connections to that node.

From the first node ONLY, logged in as oracle (copy the local account's keys so that ssh to the local node will work):cat ~/ . ssh/i d_r sa. pub >> ~/ . ssh/aut hori zed_keys

cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

Now copy the keys to the other node so that we can ssh to the remote node without being prompted for a password.ssh oracl e@ds2 cat ~/ . ssh/ i d_rs a. pub >> ~/ . ssh/ aut hor i zed_keys

(If you are cut and pasting these commands, run each of them separately. SSH will prompt for the oracle password eachtime and if the commands are pasted at the same time, the other commands will be lost when the first one flushes the inputbuffer prior to prompting for the password.)ssh oracl e@ds2 cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

chmod 644 ~/ . ssh/ aut hor i zed_keys

Ex:$ cat ~/ . ssh/i d_r sa. pub >> ~/ . ssh/aut hori zed_keys

$ cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

$ ssh or acl e@ds2 cat ~/ . ssh/ i d_rs a. pub >> ~/ . ssh/ aut hor i zed_keys

The aut hent i ci t y of host ' ds2 ( 192. 168. 200. 52) ' can' t be est abl i shed.


10/24



RSA key f i nger pri nt i s d1: 23: a7: df: c5: f c: 4e: 10: d2: 83:60: 49:25: e8: eb: 11.

Ar e you sure you want t o cont i nue connect i ng (yes/ no) ? yes

Warni ng: Per manent l y added ' ds2, 192. 168. 200. 52' ( RSA) t o t he l i st of known host s.

or acl e@ds2' s password:

$ ssh oracl e@ds2 cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

or acl e@ds2' s password:

$ chmod 644 ~/ . ssh/ aut hor i zed_keys

Now do the same for the second node. Notice that this time SSH will prompt for the passphrase you used when creating thekeys rather than the oracle password. This is because the first node (ds1) now knows the public keys for the second nodeand SSH is now using a different authentication protocol. Note, if you didn't enter a passphrase when creating the keys withssh-keygen, you will not be prompted for one here.cat ~/ . ssh/i d_r sa. pub >> ~/ . ssh/aut hori zed_keys

cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

ssh oracl e@ds1 cat ~/ . ssh/ i d_rs a. pub >> ~/ . ssh/ aut hor i zed_keys

ssh oracl e@ds1 cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

chmod 644 ~/ . ssh/ aut hor i zed_keys

Ex:$ cat ~/ . ssh/i d_r sa. pub >> ~/ . ssh/aut hori zed_keys

$ cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

$ ssh or acl e@ds1 cat ~/ . ssh/ i d_rs a. pub >> ~/ . ssh/ aut hor i zed_keys

The aut hent i ci t y of host ' ds1 ( 192. 168. 200. 51) ' can' t be est abl i shed.

RSA key f i nger pri nt i s bd: 0e: 39: 2a: 23: 2d: ca: f 9: ea: 71:f 5:3d: d3:dd: 3b:65.

Ar e you sure you want t o cont i nue connect i ng (yes/ no) ? yes

Warni ng: Per manent l y added ' ds1, 192. 168. 200. 51' ( RSA) t o t he l i st of known host s.

Ent er passphr ase f or key ' / home/oracl e/ . ssh/ i d_r sa' :

$ ssh oracl e@ds1 cat ~/ . ssh/ i d_dsa. pub >> ~/ . ssh/ aut hor i zed_keys

Ent er passphr ase f or key ' / home/oracl e/ . ssh/ i d_r sa' :

$ chmod 644 ~/ . ssh/ aut hor i zed_keys

Establish User Equivalence

Finally, after all of the generating of keys, copying of files, and repeatedly entering passwords and passphrases (isn'tsecurity fun?), you're ready to establish user equivalence. When user equivalence is established, you won't be prompted for a password again.

As oracle on the node where the Oracle 10 g Release 2 software will be installed (ds1):exec / usr / bi n/ ssh- agent $SHELL

/ us r / bi n/ s sh- add

Ex:


11/24


12/24



Bui l di ng a new DOS di skl abel . Changes wi l l r emai n i n memory onl y,

unt i l you deci de to wr i t e t hem. Af t er t hat , of cour se, t he previ ous

cont ent won' t be r ecover abl e.

The number of cyl i nders f or t hi s di sk i s set t o 4427.

Ther e i s not hi ng wr ong wi t h t hat , but t hi s i s l ar ger t han 1024,

and coul d i n cer t ai n setups cause pr obl ems wi t h:

1) sof t ware that runs at boot t i me (e. g. , ol d ver si ons of LI LO)

2) boot i ng and part i t i oni ng soft ware fr omot her OSs

( e. g. , DOS FDI SK, OS/2 FDI SK)

Command ( m f or hel p): p

Di sk / dev/ sdb: 255 heads, 63 sector s, 4427 cyl i nders

Uni t s = cyl i nders of 16065 * 512 byt es

Devi ce Boot St art End Bl ocks I d System

Command ( m f or hel p): n

Command act i on

e ext ended

p pri mar y part i t i on ( 1- 4)

p

Part i t i on number ( 1- 4) : 1

F i r s t cyl i nder ( 1 -4427, def aul t 1) :

Usi ng defaul t val ue 1

Last cyl i nder or +si ze or +si zeM or +si zeK ( 1- 4427, defaul t 4427):

Usi ng def aul t val ue 4427

Command ( m f or hel p): w

The par t i t i on t abl e has been al t er ed!

Cal l i ng i oc t l ( ) t o r e - r e ad par t i t i on t a bl e.

WARNI NG: I f you have cr eat ed or modi f i ed any DOS 6. x


13/24



part i t i ons, pl ease see the f di sk manual page f or addi t i onal

i nfor mati on.

Synci ng di sks.

Now verify the new partition:

Ex:

# f di sk - l / dev/ sdb

Di sk / dev/ sdb: 36. 4 GB, 36420075008 byt es

255 heads, 63 sect ors/ t r ack, 4427 cyl i nders

Uni t s = cyl i nder s of 16065 * 512 = 8225280 byt es

Devi ce Boot St ar t End Bl ocks I d System

/ dev/ sdb1 * 1 4427 35559846 83 Li nux

Repeat the above steps for each disk to be partitioned. Disk partitioning should be done from one node only. When finishedpartitioning, run the 'partprobe' command as root on each of the remaining cluster nodes in order to assure that the newpartitions are configured.

Ex:

# partprobe

Oracle Cluster File System (OCFS) Release 2

OCFS2 is a general-purpose cluster file system that can be used to store Oracle Clusterware files, Oracle RAC databasefiles, Oracle software, or any other types of files normally stored on a standard filesystem such as ext3. This is a significantchange from OCFS Release 1, which only supported Oracle Clusterware files and Oracle RAC database files.

Obtain OCFS2

OCFS2 is available free of charge from Oracle as a set of three RPMs: a kernel module, support tools, and a console. Thereare different kernel module RPMs for each supported Linux kernel so be sure to get the OCFS2 kernel module for your Linuxkernel. OCFS2 kernel modules may be downloaded from http://oss.oracle.com/projects/ocfs2/files/ and the tools andconsole may be downloaded from http://oss.oracle.com/projects/ocfs2-tools/files/.

To determine the kernel-specific module that you need, use uname -r.# uname - r

2. 6. 9- 22. ELsmp

For this example I downloaded:ocf s2consol e- 1. 0. 3- 1. i 386. r pm

oc f s 2 - t ool s - 1. 0. 3 - 1 . i 386. r pm

ocf s2- 2. 6. 9- 22. ELsmp- 1. 0. 7- 1. i 686. rpm Install OCFS2 as root on each cluster node

# rpm- i vh ocf s2consol e - 1. 0. 3 - 1. i 386. rpm\

http://oss.oracle.com/projects/ocfs2/files/http://oss.oracle.com/projects/ocfs2-tools/files/http://oss.oracle.com/projects/ocfs2-tools/files/http://oss.oracle.com/projects/ocfs2/files/


14/24



oc f s 2 - t ool s - 1. 0. 3 - 1 . i 386. r pm \

ocf s2- 2. 6. 9- 22. ELsmp- 1. 0. 7- 1. i 686. rpm

Pr epar i ng. . . ########################################### [ 100%]

1: ocf s2- t ool s ########################################### [ 33%]

2: ocf s2consol e ########################################### [ 67%]

3: ocf s2- 2. 6. 9- 22. EL smp ########################################### [ 100%]

Configure OCFS2

Run ocfs2console as root:# ocf s2consol e

Select Cluster → Configure NodesClick on Add and enter the Name and IP Address of each node in the cluster

Once all of the nodes have been added, click on Cluster --> Propagate Configuration. This will copy the OCFS2configuration file to each node in the cluster. You may be prompted for root passwords as ocfs2console uses ssh topropagate the configuration file. Leave the OCFS2 console by clicking on File --> Quit. It is possible to format and mountthe OCFS2 partitions using the ocfs2console GUI; however, this guide will use the command line utilities.

Enable OCFS2 to start at system boot: As root, execute the following command on each cluster node to allow the OCFS2 cluster stack to load at boot time:

/etc/init.d/o2cb enable

Ex:

# / et c/ i ni t . d/ o2cb enabl e

Wr i t i ng O2CB conf i gur ati on: OK Loadi ng modul e "conf i gf s": OK Mount i ng conf i gf s f i l esystem at / conf i g: OK Loadi ng modul e " ocf s2_nodemanager" : OK Loadi ng modul e " ocf s2_dl m" : OK Loadi ng modul e "ocf s2_dl mf s" : OK Mount i ng ocf s2_dl mf s f i l esyst em at / dl m: OK

St ar t i ng cl ust er ocfs2: OK Create a mount point for the OCFS filesystem

As root on each of the cluster nodes, create the mount point directory for the OCFS2 filesystemEx:


15/24



# mkdi r / u03

Create the OCFS2 filesystem on the unused disk partition

The example below creates an OCFS2 filesystem on the unused /dev/sdc1 partition with a volume label of "/u03" (-L /u03), ablock size of 4K (-b 4K) and a cluster size of 32K (-C 32K) with 4 node slots (-N 4). See the OCFS2 Users Guide for moreinformation on mkfs.ocfs2 command line options.Ex:# mkf s. ocf s2 - b 4K - C 32K - N 4 - L / u03 / dev/ sdc1

mkf s. ocf s2 1. 0. 3

Fi l esyst em l abel =/ u03

Bl ock si ze=4096 ( bi t s=12)

Cl uster si ze=32768 ( bi t s=15)

Vol ume si ze=36413280256 ( 1111245 cl ust er s) ( 8889960 bl ocks)

35 cl uster gr oups ( t ai l covers 14541 cl ust er s , rest cover 32256 cl ust ers)

J our nal si ze=33554432

I ni t i al number of node sl ot s: 4

Cr eat i ng bi t maps: done

I ni t i al i z i ng superb l ock: done

Wr i t i ng sys temf i l es : done

Wr i t i ng superbl ock: done

Wr i t i ng l ost+f ound: done

mkf s. ocf s2 successf ul

Mount the OCFS2 filesystem

Since this filesystem will contain the Oracle Clusterware files and Oracle RAC database files, we must ensure that all I/O tothese files uses direct I/O (O_DIRECT). Use the "datavolume" option whenever mounting the OCFS2 filesystem to enabledirect I/O. Failure to do this can lead to data loss in the event of system failure.Ex:

# mount -t ocfs2 -L /u03 -o datavolume /u03

Notice that the mount command uses the filesystem label (-L u03) used during the creation of thefilesystem. This is a handy way to refer to the filesystem without having to remember the device

name.

To verify that the OCFS2 filesystem is mounted, issue the mount command or run df:# mount -t ocfs2

/dev/sdc1 on /u03 type ocfs2 (rw,_netdev,datavolume)

# df /u03

Filesystem 1K-blocks Used Available Use% Mounted on

/dev/sdc1 35559840 138432 35421408 1% /u03

http://oss.oracle.com/projects/ocfs2/dist/documentation/ocfs2_users_guide.pdfhttp://oss.oracle.com/projects/ocfs2/dist/documentation/ocfs2_users_guide.pdf


16/24



The OCFS2 filesystem can now be mounted on the other cluster nodes.

To automatically mount the OCFS2 filesystem at system boot, add a line similar to the one below to/etc/fstab on each cluster node:LABEL=/u03 /u03 ocfs2 _netdev,datavolume,nointr 0 0

Create the directories for shared filesCRS files

mkdir /u03/oracrs

chown oracle:oinstall /u03/oracrs

chmod 775 /u03/oracrs

Database files

mkdir /u03/oradata

chown oracle:oinstall /u03/oradata

chmod 775 /u03/oradata

Automatic Storage Management (ASM)ASM was a new storage option introduced with Oracle Database 10gR1 that provides the services of a filesystem, logical volume manager, and software RAID in a platform-independent manner. ASMcan stripe and mirror your disks, allow disks to be added or removed while the database is under load, and automatically balance I/O to remove "hot spots." It also supports direct and asynchronousI/O and implements the Oracle Data Manager API (simplified I/O system call interface) introduced inOracle9 i .

ASM is not a general-purpose filesystem and can be used only for Oracle data files, redo logs,control files, and flash recovery area. Files in ASM can be created and named automatically by thedatabase (by use of the Oracle Managed Files feature) or manually by the DBA. Because the filesstored in ASM are not accessible to the operating system, the only way to perform backup andrecovery operations on databases that use ASM files is through Recovery Manager (RMAN).

ASM is implemented as a separate Oracle instance that must be up if other databases are to beable to access it. Memory requirements for ASM are light: only 64 MB for most systems.

Installing ASM

On Linux platforms, ASM can use raw devices or devices managed via the ASMLib interface. Oraclerecommends ASMLib over raw devices for ease-of-use and performance reasons. ASMLib 2.0 isavailable for free download from OTN. This section walks through the process of configuring asimple ASM instance by using ASMLib 2.0 and building a database that uses ASM for disk storage.

Determine Which Version of ASMLib You Need

ASMLib 2.0 is delivered as a set of three Linux packages:

oracleasmlib-2.0 - the ASM libraries

oracleasm-support-2.0 - utilities needed to administer ASMLib

oracleasm - a kernel module for the ASM library

Each Linux distribution has its own set of ASMLib 2.0 packages, and within each distribution, eachkernel version has a corresponding oracleasm package. The following paragraphs describe how todetermine which set of packages you need.First, determine which kernel you are using by logging in as root and running the followingcommand:

uname -rm

http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.htmlhttp://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html


17/24



Ex:

# uname -rm

2.6.9-22.ELsmp i686

The example shows that this is a 2.6.9-22 kernel for an SMP (multiprocessor) box using Intel i686

CPUs.

Use this information to find the correct ASMLib packages on OTN:

1. Point your Web browser to http://www.oracle.com/technology/tech/linux/asmlib/index.html

2. Select the link for your version of Linux.

3. Download the oracleasmlib and oracleasm-support packages for your version of Linux

4. Download the oracleasm package corresponding to your kernel. In the example above, theoracleasm-2.6.9-22.ELsmp-2.0.0-1.i686.rpm package was used.Next, install the packages by executing the following command as root:

rpm -Uvh oracleasm-kernel_version-asmlib_version.cpu_type.rpm \

oracleasmlib-asmlib_version.cpu_type.rpm \

oracleasm-support-asmlib_version.cpu_type.rpm

Ex:

# rpm -Uvh \

> oracleasm-2.6.9-22.ELsmp-2.0.0-1.i686.rpm \

> oracleasmlib-2.0.1-1.i386.rpm \

> oracleasm-support-2.0.1-1.i386.rpm

Preparing... ########################################### [100%]

1:oracleasm-support ########################################### [

33%]

2:oracleasm-2.6.9-22.ELsm########################################### [67%]

3:oracleasmlib ###########################################[100%]

Configuring ASMLib

Before using ASMLib, you must run a configuration script to prepare the driver. Run the followingcommand as root, and answer the prompts as shown in the example below. Run this on each nodein the cluster.

# /etc/init.d/oracleasm configure

Configuring the Oracle ASM library driver.

http://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.htmlhttp://www.oracle.com/technetwork/topics/linux/asmlib/index-101839.html


18/24



This will configure the on- boot properties of the Oracle ASM library

driver. The following questions will determine whether the driver is

loaded on boot and what permissions it will have. The current values

will be shown in brackets ('[]'). Hitting without typing an

answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface []: oracle

Default group to own the driver interface []: dba

Start Oracle ASM library driver on boot (y/n) [n]: y

Fix permissions of Oracle ASM disks on boot (y/n) [y]: y

Writing Oracle ASM library driver configuration: [ OK ]

Creating /dev/oracleasm mount point: [ OK ]

Loading module "oracleasm": [ OK ]

Mounting ASMlib driver filesystem: [ OK ]

Scanning system for ASM disks: [ OK ]

Next you tell the ASM driver which disks you want it to use. Oracle recommends that each diskcontain a single partition for the entire disk. See Partitioning the Disks at the beginning of thissection for an example of creating disk partitions.

You mark disks for use by ASMLib by running the following command as root from one of the cluster nodes:

/etc/init.d/oracleasm createdisk DISK_NAME device_name

Tip: Enter the DISK_NAME in UPPERCASE letters.

Ex:

# /etc/init.d/oracleasm createdisk VOL1 /dev/sdb1

Marking disk "/dev/sdb1" as an ASM disk: [ OK ]

# /etc/init.d/oracleasm createdisk VOL1 /dev/sdc1

Marking disk "/dev/sdc1" as an ASM disk: [ OK ]


19/24



# /etc/init.d/oracleasm createdisk VOL1 /dev/sdd1

Marking disk "/dev/sdd1" as an ASM disk: [ OK ]

Verify that ASMLib has marked the disks:

# /etc/init.d/oracleasm listdisks

VOL1

VOL2

VOL3 On all other cluster nodes, run the following command as root to scan for configured ASMLib disks:/etc/init.d/oracleasm scandisks

Part IV: Install Oracle Software

Oracle Database 10 g Release 2 can be downloaded from OTN. Oracle offers a development and

testing license free of charge. However, no support is provided and the license does not permitproduction use. A full description of the license agreement is available on OTN.The easiest way to make the Oracle Database 10 g Release 2 distribution media available on your server is to download them directly to the server.

Use the graphical login to log in as oracle.

Create a directory to contain the Oracle Database 10 g Release 2 distribution:

mkdir 10gR2

To download Oracle Database 10 g Release 2 from OTN, point your browser (Firefox works well) to

http://www.oracle.com/technology/software/products/database/oracle10g/htdocs/10201linuxsoft.html.Fill out the Eligibility Export Restrictions page, and read the OTN License agreement. If you agreewith the restrictions and the license agreement, click on I Accept.

Click on the 10201_database_linux32.zip link, and save the file in the directory you created for thispurpose —if you have not already logged in to OTN, you may be prompted to do so at this point.

Since you will be creating a RAC database, you will also need to download and install OracleClusterware Release 2. Click on the 10201_clusterware_linux32.zip link and save the file.

Unzip and extract the files:

cd 10gR2

unzip 10201_database_linux32.zip

unzip 10201_clusterware_linux32.zip

Establish User Equivalency and Set Environment Variables

If you have not already done so, login as oracle and establish user equivalency between nodes:exec /usr/bin/ssh-agent $SHELL

/usr/bin/ssh-add

Enter passphrase for /home/oracle/.ssh/id_rsa:

Identity added: /home/oracle/.ssh/id_rsa (/home/oracle/.ssh/id_rsa)

Identity added: /home/oracle/.ssh/id_dsa (/home/oracle/.ssh/id_dsa)

http://www.oracle.com/technetwork/database/10201linuxsoft-097986.htmlhttp://www.oracle.com/technetwork/database/10201linuxsoft-097986.html


20/24



Set the ORACLE_BASE environment variable:

ORACLE_BASE=/u01/app/oracle; export ORACLE_BASE

Install Oracle Clusterware

Before installing the Oracle RAC 10 g Release 2 database software, you must first install OracleClusterware. Oracle Clusterware requires two files to be shared among all of the nodes in thecluster: the Oracle Cluster Registry (100MB) and the Voting Disk (20MB). These files may be storedon raw devices or on a cluster filesystem. (NFS is also supported for certified NAS systems, but thatis beyond the scope of this guide.) Oracle ASM may not be used for these files because ASM is

dependent upon services provided by Clusterware. This guide will use OCFS2 as a cluster filesystem to store the Oracle Cluster Registry and Voting Disk files.

Start the installation using "runInstaller" from the "clusterware" directory:1. Welcome

2.

Click on Next

3. Specify Inventory Directory and Credentials

4.

The defaults should be correct

Click on Next

5. Specify Home Details6.

Name: OraCRS_Home

Path: /u01/app/oracle/product/crs

a.

7. Product-Specific Prerequisite Checks

8.

Correct any problems found before proceeding.

Click on Next

9. Specify Cluster Configuration

10.Enter the cluster name (or accept the default of "crs");

11. Specify Network Interface Usage - Specify the Interface Type (public, private, or "do no use") for each interface


21/24



12. Specify Oracle Cluster Registry (OCR) Location

13.

Choose External Redundancy and enter the full pathname of the OCR file (ex:/u03/oracrs/ocr.crs).

14. Specify Voting Disk Location

15.

Choose External Redundancy and enter the full pathname of the voting disk file (ex:/u03/oracrs/vote.crs)

16. Summary

17.

Click on Install

18. Execute Configuration Scripts

19.

Execute the scripts as root on each node, one at a time, starting with the installation node.

Do not run the scripts simultaneously. Wait for one to finish before starting another.

Click on OK to dismiss the window when done.

End of Clusterware InstallationVerify that the installation succeeded by running olsnodes from the $ORACLE_BASE/product/crs/bindirectory; for example:

$ /u01/app/oracle/product/crs/bin/olsnodes

ds1

ds2

Once Oracle Clusterware is installed and operating, it's time to install the rest of the Oracle RACsoftware.

Create the ASM Instance

If you are planning to use OCFS2 for database storage, skip this section and continue with Createthe RAC Database. If you plan to use Automatic Storage Management (ASM) for database storage,follow the instructions below to create an ASM instance on each cluster node. Be sure you haveinstalled the ASMLib software as described earlier in this guide before proceeding.


22/24



Start the installation using "runInstaller" from the "database" directory:

1. Welcome

2.

Click on Next

3. Select Installation Type

4.

Select Enterprise Edition

Click on Next

5. Specify Home Details

6.

Name: Ora10gASM

Path: /u01/app/oracle/product/10.2.0/asmNote:Oracle recommends using a different ORACLE_HOME for ASM than the ORACLE_HOMEused for the database for ease of administration.

Click on Next

7. Specify Hardware Cluster Installation Mode

8.

Select Cluster Installation

Click on Select All

Click on Next

9. Product-specific Prerequisite Checks

If you've been following the steps in this guide, all the checks should pass without difficulty. If one or more checks fail, correct the problem before proceeding.

Click on Next

10. Select Configuration Option

11.

Select Configure Automatic Storage Management (ASM)

Enter the ASM SYS password and confirm

Click on Next

12. Configure Automatic Storage Management

13.

Disk Group Name: DATA

Redundancy- High mirrors data twice.- Normal mirrors data once. This is the default.- External does not mirror data within ASM. This is typically used if an external RAID array isproviding redundancy.

Add DisksThe disks you configured for use with ASMLib are listed as Candidate Disks. Select each diskyou wish to include in the disk group.

Click on Next

14. Summary

A summary of the products being installed is presented.

Click on Install.


16.

At the end of the installation, a pop up window will appear indicating scripts that need to berun as root. Login as root and run the indicated scripts.

Click on OK when finished.

17. End of Installation

Make note of the URLs presented in the summary, and click on Exit when ready.

18. Congratulations! Your new Oracle ASM Instance is up and ready for use.

Create the RAC DatabaseStart the installation using "runInstaller" from the "database" directory:

1.

2. Welcome


23/24



3.

Click on Next

4. Select Installation Type

5.

Select Enterprise Edition

Click on Next

6. Specify Home Details

7.

Name: OraDb10g_home1

Path: /u01/app/oracle/product/10.2.0/db_1Note:Oracle recommends using a different ORACLE_HOME for the database than theORACLE_HOME used for ASM.

Click on Next

8. Specify Hardware Cluster Installation Mode

9.

Select Cluster Installation

Click on Select All

Click on Next

10. Product-specific Prerequisite Checks

If you've been following the steps in this guide, all the checks should pass without difficulty. If one or more checks fail, correct the problem before proceeding.

Click on Next

11. Select Configuration Option

12.

Select Create a Database

Click on Next

13. Select Databse Configuration

14.

Select General Purpose

Click on Next

15. Specify Database Configuration Options

16.

Database Naming: Enter the Global Database Name and SID

Database Character Set: Accept the default

Database Examples: Select Create database with sample schemas

Click on Next

17. Select Database Management Option

18.

Select Use Database Control for Database Management

Click on Next

19. Specify Database Storage Option

20.

If you are using OCFS2 for database storage

Select File System

Specify Database fle location: Enter the path name to the OCFS2 filesystem directory youwish to use.ex: /u03/oradata/racdemo

If you are using ASM for database storage

Select Automatic Storage Management (ASM)

Click on Next

21. Specify Backup and Recovery Options

22.

Select Do not enable Automated backups

Click on Next


24/24


23. For ASM Installations Only:

24.

Select ASM Disk Group

Select the DATA disk group created in the previous section

Click on Next

25. Specify Database Schema Passwords

26.

Select Use the same password for all the accounts

Enter the password and confirm

Click on Next

27. Summary

A summary of the products being installed is presented.

Click on Install.

28. Configuration Assistants

29.

The Oracle Net, Oracle Database, and iSQL*Plus configuration assistants will run automatically


31.

At the end of the installation, a pop up window will appear indicating scripts that need to berun as root. Login as root and run the indicated scripts.

Click on OK when finished.

32. End of Installation

Make note of the URLs presented in the summary, and click on Exit when ready.

33. Congratulations! Your new Oracle Database is up and ready for use.

ConclusionNow that your database is up and running, you can begin exploring the many new features offeredin Oracle Database 10 g Release 2. A great place to start is Oracle Enterprise Manager, which hasbeen completely re-written with a crisp new Web-based interface. If you're unsure where to begin,the

Oracle Database Concepts 10g Release 2 and the 2-Day DBA Guide will help familiarize you with your newdatabase. OTN also has a number of guides designed to help you get the most out of OracleDatabase 10 g Release 2.

John Smiley [ [email protected] ] works as a senior database engineer for a major online retailer and is an Oracle Certified Master DBA with over 19 years of experience with Oracle databasesrunning on all major platforms. He specializes in engineering high-volume Oracle databases,advanced performance tuning methods, and RAC, and enjoys developing with PL/SQL, C, and Perl.
http://download.oracle.com/docs/cd/B19306_01/server.102/b14220/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/server.102/b14196/toc.htmmailto:[email protected]:[email protected]://download.oracle.com/docs/cd/B19306_01/server.102/b14196/toc.htmhttp://download.oracle.com/docs/cd/B19306_01/server.102/b14220/toc.htm

Documents

Installing Oracle RAC 10g Release 2 on Linux x86