OVS Dom0 Upgrade Livemigration SOP v1.52

7/23/2019 OVS Dom0 Upgrade Livemigration SOP v1.52

1/17

Doc : Dom0 Upgrade Execution Process

Version : 1.52

---------------------------------------------------------------------------------------------------------------------

-----

Objective :-

This Document describes the steps to be folloed to perform the !"#$ upgrade or Dom0

upgrade.

Pre check to be performed 1 hr before the schedule of Execution

1.Ensure : D%& has ta'en the () for the &ffected Dom0 ping channel o*ms-up-pro+ect

,mig-os - script ill auto mount

3.i!-os mount source det"ils :-

m'dir ,mig-os mount -o noacl rm02stor2-nas/,export,roh2ahmig,rmdcdom0 ,mig-os

m'dir ,mig-os mount -o noacl adc0ntap1-b'p/,*ol,aoh1ahmig,adcdom0 ,mig-os

m'dir ,mig-os mount -o noacl sl05stor02-nas/,export,sloh02ahmig,sldcdom0 ,mig-os

m'dir ,mig-os mount -o noacl t*p01stor05-nas/,export,t*poh05ahmig,t*pdcdom0 ,mig-os

m'dir ,mig-os mount -o noacl epc002oodstor02-nas/,export,epcoh02ahmig,epdcdom0 ,mig-os

m'dir ,mig-os mount -o noacl s3c002oodstor01-nas/,export,s3coh01ahmig,s3dcdom0 ,mig-os

m'dir ,mig-os mount -o noacl llg02stor02-b'p/,export,llgoh0ahmig,llgdom0 ,mig-os

#.V"lid"te Prep D"t" "nd $urrent Environment

44444444444444 TE#P&TE 44444444444444

$67 /

$ource Dom0 /

Target Dom0 /

$ource Dom0 "# list /

8os of "#s from P6EP /

%os of Vs from Port"l:

&re the "# names isted in Prep the same as current "# ist 9:,8; /

44444444444444 TE#P&TE 44444444444444

Use This as reference through out acti*it3


2/17

&.'o '"ke:(ource Dom) * Pre +e"lth checks "nd p"ste the output in the (,

6un/

" pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o source -o pre -rfc 3-

'E('

&bo*e script co*ers/

(=!# *ersionunamecat ,etc,enterprise-releasemig-os is mountedreset ilom passord$hos Total *cpus of all running "#s >$hos Total #emor3 of all running "#s)hec'ing the *m.cfg for *cpumax parameter)hec' if *ms are on localdis'=dentif3 the container to snapE# agent status >!*s-agent statusdap status=fconfig>bond>#TU>route details

c?. Ensure 3ou are able to access the ilom console of the Dom0

d?. Ensure that ,etc,o*ms-pro*ision.conf exists. =f it does not> perform the folloing step/

cat > /etc/ovms-provision.conf


3/17

4. (et 5O "nd Dis"ble Pin!er

E# / https/,,omcsem.oracle.com,apex,fCp4200/5/1025201//8!/// locate 3our E# then set %!

Ai*e the %! name as / $67-$er*er-%oucne

Pinger /https/,,globaldc.oracle.com,host,detail,*mfsnchpg02

On 'emp(t"!e Dom)

6. 'o '"ke:'"r!et Dom) * V"lid"tions

,un:

pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o t"r!et -o v"lid"te -rfc

3-'E('

- This ill chec' same %ridges from $ource are a*ailable on the temp Dom0

- Ensure the same container is a*ailable.

$& (as to /-

- Ensure V07%s t"!!ed 8ith the s"me V07%( "s (ource Dom) *check the rel"ted (, to confirm th

V07% is "lre"d2 t"!!ed.

$earch for $6 based on FsubG chec' the $6 is closed and included the "&8s in use on the

current Dom0 H based on output captured in the abo*e prep script.

999Proceed to live mi!r"te if 2our "ns8er is 2es for the belo8 conditions 9999

- the s"me brid!es "re on 'emp domo

- V07% "re t"!!ed

- (ource cont"iner is mounted on temp Dom)

- (ource (8"p cont"iner is mounted on the temp Dom)
https://omcsem.oracle.com/apex/f?p=200:35:1039424757297901::NOhttps://globaldc.oracle.com/host/detail/vmfsnchpg029https://globaldc.oracle.com/host/detail/vmfsnchpg029https://omcsem.oracle.com/apex/f?p=200:35:1039424757297901::NO


4/17

Execution Ph"se :-

1. Perform 0ive mi!r"tion usin! the script

$"ses 8here 8e don;t h"ve "n2 dependenc2 on other (,( for t"r!et Dom) proceed next steps

Else if dependenc2 !o 8ith the pl"n from 0e"d.

source pre step h"s !ener"ted the : mig"#s.lst file. --- this contains our source *m names

Ex/ sample output

Creatin+ t4e 5 mi+V%s.lst file for t4is 6-$$7$89:8:$#

--------------------------------------------

8;ee;3a;a86a;;6a3$3#eec8c6cc97 :;3it V%

f#e6f96#8;6ca8f3c$f3:6:96 :;3it V%

--------------------------------------------

/mig-os/logs/OEL-DOM0_UPG/3-11215965617/migVMs.lst -- was g!"at#

Plas "$iw it% a!# mo#i&' i& !(ssa"') ma* s+" it ,as t, a(t sam !+m" o& ,osts &"om m li

'o st"rt mi!r"tion run

'"ke confirm"tion from (0 on +$ b2 this time.

niti"te " pin! to the Vs "nd t"ke ptime

Ensure df *hP is not h"n!in! on "n2 mounts

,un:

E1&?4&416mi!Vs.lst@t"r!et Dom)A

Bcont


5/17

E


6/17

2.Ugrade / Hthis steps performs Disable uptrac' and upgrade ?

6un/

'ime pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o source -o

up!r"de -rfc 3-'E('

0o! loc"tion for the up!r"de: Ex ,*ar,log,od-pro*ision,2015-0-0-20M/0M/1.log

3. 5ounce the Dom)

$heck !rub.conf * e should see the ne 'ernel

Po8er off the Dom)

se: pts"dminos/mi!r"tionbins2stem/rest"rt.pl -s--

Crom console

Ta'e current poer status of host / sho ,$:$ N if should of off

$tart Dom0 from )onsole / start ,$:$

--

#.Once the Dom) is up run the belo8

,ptsadmin,ateam,bin,fixfirmare --- Hgo ith defaults ?

if it su!!ests : Cirm8"re up!r"de needed .."llo8 it proceed "nd perform belo8 steps

b? =8O!6# & TE E&D Hthis item is being trac'ed?

c? $hutdon E# &gent

d? $hutdon Dom0

e? Poer off Dom0 host

f? 6eboot =!# b3 executing reset ,$P in an =!# ssh session

g? Poer on Dom0 host and aait boot-up

h? rerun / ,ptsadmin,ateam,bin,fixfirmare ... it should not sho firmare upgrade needed


7/17

Post checks on (ource Dom)

To Ta'e/ (ource Dom) * Post checks

Ensure to ount : mi!-os

1.

,un:

pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o source -o post -rfc 3-

'E('

7bove script covers : ,ptsadmin,ateam,bin,o*ms-goli*echec'.pl as ell

>.Ensure th"t OV discover2 completes successfull2 throu!h OV

- :ou can identif3 the !"# manager for a Dom0 b3 running the folloing as root on the Dom0 Hno *alue mean

that the Dom0 is not currentl3 managed b3 !"##?/

o*s-agent-db readitem ser*er managercoreapiurl M cut -f2 -dI cut -f1 -d/

Ex/

9rootIauc02Loodhost1 pchandru;7 o*s-agent-db readitem ser*er managercoreapiurl cut -f2 -dI cut -f

-d/

10.22..21

)onnect to the !"## ser*er and follo steps gi*en in the belo doc

OVMM-Dom0-discovery.docx

3.f the up!r"de h"s !one correctl2 "nd "ll verific"tions p"ssed then remove uptr"ck ,Ps

3um erase -3 uptrac'-lib3aml uptrac'-p3thon-p3curl uptrac'-P3: uptrac'


8/17

#.5rin!b"ck the Vs to ori!in"l Dom)Clip them b"ck

%ote:

On the (ource* O'D if 8e find " note "s belo8 do not move b"ck the Vs to (,$

----------------------------------------------------------------------------------------------------------

PO,'7%' %O'E

----------------------------------------------------------------------------------------------------------

'+( (E,VE, 00 5E (ED E


9/17

ogin to $ource Dom0 / ta'e xm list

ogin to $tage,Temp Dom0 / ta'e xm list

ove the Vs b"ck Crom 'emp'"r!et Dom) to (ource

-ogin to Temp ,Target dom0

Ensure df *hP is not h"n!in! on "n2 mounts

6un/ use the same and mig"#s.lst that got created in / source Q pre Q phase

Ex / 6eplace according to 3our $6

pts"dminos/mi!r"tionOE0mi!V.sh/custom/pch"ndru/v1.1 mi!-oslo!sOE0-DO)/P=3-

11>1&?4&416mi!Vs.lst@(ource Dom)A

Ta'e xm list on the temp,stage dom0 / 8o "#s should be running no

Ta'e xm list on the $ource dom0 / "#s should be running no

ogin to each "# and ta'e uptime

On (ource Dom):

!nce &ll "#s are up and mo*ed bac' Q chec' ,etc,xen,auto!"#$UPA%BP.6O)8U#%E6 bac'up folder isauto mo*ed

as ,etc,xen,auto b3 the script hen e executed Qsource-post.

nform (hift 0e"d to t"ke Post +e"lth checks


10/17

(, fin"l pd"te 'empl"te,ele"se note

The scheduled maintenance acti*it3 has been completed successfull3.should 3ou encounter an3 issue> Please contact/

$&1 )ontact/------------8ame / De*esh Bumar#obile / R1 L115200Email =D / de*esh.d.'umarIoracle.com

#anagement Escalation /---------------------8ame / 'aruppiah ramaEmail =D / 'aruppiah.ramaIoracle.com#obile / R1 5L 25

8ame / (ari :ala*arth3Email =D / hari.3ala*arth3Ioracle.com#obile / R1 5025L

6egards>

SSSSSSE8DSSSSSS


11/17

Doc : Dom0 Upgrade P6EP Process

Version : 1.52

--------------------------------------------------------------------------------------------------------------------------

!"#$ Dom0 Upgrade Prep Procedure

1.$"pture the Det"ils in to the 'empl"te

44444444444444 TE#P&TE 44444444444444

$ource Dom0 /

Target Dom0 /

$ource Dom0 "# list /

%os of Vs from Port"l :

8os of "#s currentl3 running /

&re the "# names isted in $6 the same as current "# ist 9:,8; /

7ddition"l Vs Cound in Prep :nst"nce n"mes on 7ddn hosts :

8ET

44444444444444 TE#P&TE 44444444444444

se the belo8 code to !et V n"mes:-

--for i in xm list a' -O KV print W1 XK egrep -i* 8ameDomain do cat

,!"$,6epositories,S,"irtual#achines,Wi,*m.cfg grep -i !"#simplename a' -O KV print W XK done

C there is " mism"tch from port"l "nd current kindl2 upd"te port"l "nd send (, to 5iIop

"s 8ell


12/17

ount the mi!-os on the (ource "nd '"r!et Dom)s

6#D) /m'dir ,mig-os mount -o noacl rm02stor2-nas/,export,roh2ahmig,rmdcdom0 ,mig-os

&D) /m'dir ,mig-os mount -o noacl adc0ntap1-b'p/,*ol,aoh1ahmig,adcdom0 ,mig-os

$D) /m'dir ,mig-os mount -o noacl sl05stor02-nas/,export,sloh02ahmig,sldcdom0 ,mig-os

T"P /m'dir ,mig-os mount -o noacl t*p01stor05-nas/,export,t*poh05ahmig,t*pdcdom0 ,mig-os

EPD) /m'dir ,mig-os mount -o noacl epc002oodstor02-nas/,export,epcoh02ahmig,epdcdom0 ,mig-os

$:D) /m'dir ,mig-os mount -o noacl s3c002oodstor01-nas/,export,s3coh01ahmig,s3dcdom0 ,mig-os

A / m'dir ,mig-os mount -o noacl llg02stor02-b'p/,export,llgoh0ahmig,llgdom0 ,mig-os

On (ource Dom) :-

#ount / ,mig-os

,%:

pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o source -o prep8ork

-rfc 3-'E('

This script ill co*er belo steps

-Ta'es a ist of the "#s running on the $ource Dom0

-Ta'es the Total #emor3 and ")PU used,assigned to the "#$ on $ource Dom0

-=dentif3 the bridges of the "#s running on the $ource Dom0

- =dentif3 the $torage )ontainer and sap container on the $ource dom0- 6eset =!# Passord on $ource Dom0

- chec' if "# is on local dis'

On '"r!et Dom) :-

#ount / ,mig-os

pts"dminos/mi!r"tionOE0Dom)/p!r"de/s2sinfo/pre/up!/post/checks.sh -o t"r!et -o prepv"lid"te

-rfc 3-'E('

'his 8ill comp"re:-

$ame $torage and sap container is &*ailable on Target dom0

$ho if enough free memor3 is a*ailable for "# migration

$ame bridges of source Dom0 "ms are a*ailable on the Target Dom0


13/17

- =n case there are an3 missing bridges create them and ma'e them up

"&8 Tagging/-

1.$heck in port"l for vl"n t"!!in! (, "nd confirm the reHuired vl"n

)onfirm it is completed.

update the *lan tagging page

>.if vl"n t"!!in! is not there then identif2 "ll the VJs in " !iven server pool "nd !et the vl"n det"ils of

e"ch V "nd r"ise net8ork vl"n t"!!in! (,

steps

ssh -l admin localhost -p 10000

!"#G sho ser*erPool name4rmc002oodpool001

!"#G sho ser*erPool name4rmc002oodpool001

update the *lan tagging page

3.C 8e find more th"n '8o 5rid!es for " V * check it h"s *r"c interf"ce

n th"t c"se 8e need to t"ke .!etnet for *r"c "lso "nd !et it t"!!ed

6un for /

-rac

-*ip

-Orontend

ogin / hYsun1 - ta'e the *lan id details of the "#s

EZ/

bash-2.0W ,home,**arughe,getnet *mohsad*g501

6eYuest/ *mohsad*g501


14/17

O[D8/ *mohsad*g501.oracleoutsourcing.com

=P/ 1..205.1L

8etmas'/ 255.255.255.20

vl"n id: rmdc-I>-"dv!-v14)

vl"n note: %7,-D5-7dv"nt"!e-("les-"nd-"rketin!-Ksr3-&6&4L&41?#

8etor'/ 1..205.1L0,2

Oirst =P/ 1..205.1L1

ast =P/ 1..205.1

%roadcast =P/ 1..205.15

'o =et the (8itch port det"ils "nd : PCE "nd (CE

se port"l : http:pnvc"pp)1.or"cle.comc!i-bins8itches.c!i

("mple V07% t"!!in! (, 'empl"te

(ubject: !"#$ Pro+ect/auc055oodhost0/"&8 tagging reYuest

(u!!ested (, reHuest

'his (, is p"rt of the OV( project for t"!!in! "ll servers 8hich belon!s to rmc))>oodpool))4

Ple"se V07% '"! the follo8in! Dom)+O('%7E: rmc002oodhost052"c "ddress : 00/10/E0/5/)/O

(1: cosprings1\2-si-10.oracle.com(>: cosprings1\2-si-10.oracle.comPO,': 12

V07%:

rmdc-I>-isuv-v>1? 1).>>>.44.1?>>L %7%,-,7$-llinois-(t"te-niversit2-Ksr3-?1))3>)

6asie $6 to / 8etor' Team

$ample $67-11151

Oollo up till completion of the "&8 tagging $6

"alidate all "&8$ from the list gi*en are tagged.
http://pnvcapp01.oracle.com/cgi-bin/switches.cgihttp://pnvcapp01.oracle.com/cgi-bin/switches.cgi


15/17

#. Durin! Prep st"!e for : cit-em-"!ent-1>-,#

cop3 the / ,mig-os,pchandru,cit-em-agent-12-6.remo*e file

as / ,usr,local,git,etc,softare,cit-em-agent-12-6.remo*e

&. ,un : usrloc"l!itbinhost-!roup -f "ll m"ke sure service/code none

f it is sho8in! : ebso pod etc upd"te '7(

go to =T&$ -- G &sset Details -- G change ser*icecode to 8one

it ill ta'e 15 min to pushup &P$

4. $heck the 5usiness/"re" of the host must be : commerci"l "n2 other inform 0e"d

EZ/ 9rootIrmc002oodhost J;7 ,usr,local,git,bin,host-group -f all grep -i businessarea

business/"re": commerci"l

6. check 0O D%( is resolvin! fine 8ith correct form"t

,efer belo8:-

=!#8E4WVD&T&)E8TE6X-mgmt-WV$E6=&X.WVD!#&=88EX

7 &D) @one

7 &D) @one

7 &D) @one

7 &D) @one 15

7 &D) @one 2L

7 &D) @one 1

7 &D) @one 27 &D) @one

7 &D) @one

7 &D) @one L

7 &D) @one

7 &D) @one

7 &D) @one 0


16/17

7 6#D) @one 2

7 6#D) @one 5

7 6#D) @one L

7 A @one 1

7 A @one 2

7 A @one L

7 A @one

7 A @one 7 T"P @one 1

7 T"P @one 2

7 $D) @one

7 $D) @one 5

=!#8E4WV(!$T8EX-ilom.WVD!#&=88EX

7 &D) @one 2

7 &D) @one

7 &D) @one

7 &D) @one 5

7 &D) @one L

7 &D) @one

7 &D) @one

7 &D) @one 7 &D) @one 50

7 &D) @one 51

7 &D) @one 52

7 &D) @one 5

7 &D) @one 5

7 &D) @one 55

7 &D) @one 5L

7 &D) @one 5

7 &D) @one 5

7 &D) @one 57 &D) @one L0

7 &D) @one 0

7 A @one 10

7 A @one 11

7 $D) @one L

7 $D) @one


17/17

7 $D) @one

7 $D) @one

7 $D) @one 10

7 $D) @one 11

7 $D) @one 12

7 $D) @one 1

7 $D) @one 1

7 $D) @one 07 $:D) @one 1 ] 2

7 $:D) @one

7 EPD) @one 2

7 T6D) @one 1

7 T6D) @one 2

Documents

OVS Dom0 Upgrade Livemigration SOP v1.52