.
SFRAC 操作手册Jul, 2011
Symantec Consulting Services
Symantec Consulting Service, Great China Region
文件信息
版本
关于作者该文件可以与 Symantec Consultant 取得联系
修改历史
日期 作者 版本 修改内容
2
Subject Symantec 大中国区顾问服务
Version
Author
Comments
Symantec Consulting Service, Great China Region
目录
1 Storage Foundation for RAC/CFS 简单操作.............................61.1 命令行操作简介...................................................................................................................6
1.1.0 如何重启主机............................................................................................................................ 6
1.1.1 检查 cluster 状态....................................................................................................................... 6
1.1.2 启动服务组............................................................................................................................... 8
1.1.3 停止服务组............................................................................................................................... 8
1.1.4 切换服务组............................................................................................................................... 8
1.1.5 停止某个资源............................................................................................................................ 8
1.1.6 启动某个资源............................................................................................................................ 8
1.1.7 发现某个资源出错后清除错误...................................................................................................9
1.1.8 如果资源无法启动或停止需要 flush 状态...................................................................................9
1.1.9 查看心跳信息............................................................................................................................ 9
1.1.10 查看 cluster 支持 RAC/CFS 的状态..........................................................................................9
1.1.11 查看磁盘状态.......................................................................................................................... 9
1.1.12 查看 disk group 状态.............................................................................................................10
1.1.13 查看 disk group 剩余空间......................................................................................................10
1.1.14 查看 volume 的状态..............................................................................................................10
1.1.15 添加一个 disk group..............................................................................................................12
1.1.16 在 disk group 中添加一块盘...................................................................................................12
1.1.17 创建 stripe 的 volume.............................................................................................................12
1.1.18 创建文件系统........................................................................................................................ 12
1.1.19 mount 文件系统....................................................................................................................12
1.1.20 查看 cluster 的日志................................................................................................................13
1.1.21 查看 volume manager 的日志................................................................................................13
1.1.22 删除一个 volume...................................................................................................................13
1.1.23 删除一个 disk group..............................................................................................................13
1.1.24 如何扩大或缩小文件系统......................................................................................................13
1.1.25 如何禁止 fencing...................................................................................................................14
1.1.26 shutdown 说明...................................................................................................................... 14
1.1.27 检查磁盘的 serial number.....................................................................................................14
1.1.28 如何查看 license key.............................................................................................................15
1.1.29 如何添加 license key.............................................................................................................15
1.1.30 如何查看多路径信息..............................................................................................................15
1.1.31 如何让 cluster 两主机都停止后启动一台主机提供服务..........................................................15
1.1.32 如何停止一个 HBA 卡上的 I/O(用于升级微码)...................................................................16
1.1.33 如何更换一个 HBA 卡............................................................................................................16
1.1.34 如何添加一个 oracle 使用的裸设备........................................................................................16
1.1.35 如何停止 cluster 软件............................................................................................................16
1.1.36 如何手工 import disk group 并启动 volume............................................................................16
1.1.37 如何手工 deport disk group...................................................................................................17
2 图形界面简介......................................................................182.1 cluster 图形界面................................................................................................................18
2.2 Volume Manager 图形界面................................................................................................32
3 高级操作............................................................................33
3
Symantec Consulting Service, Great China Region
3.1 dmp 操作...........................................................................................................................333.1.1 如何禁止多路径....................................................................................................................... 33
3.1.2 查看磁盘路径的常用命令.........................................................................................................33
3.1.3 查看 dmp 路径的常用命令........................................................................................................33
3.1.4 查看 controller 的常用命令.......................................................................................................33
3.1.5 查看阵列的常用命令................................................................................................................33
3.1.6 查看 dmp 的实时 I/O 状态........................................................................................................34
3.1.7 如何升级阵列的微码................................................................................................................34
3.1.8 对磁盘 I/O 错误的处理..............................................................................................................35
3.1.9 查看 asl 和 apm 信息................................................................................................................36
3.1.10 添加对不支持阵列的简单支持................................................................................................37
3.1.11 磁盘名称的显示方式..............................................................................................................37
3.1.12 重新生成磁盘名.....................................................................................................................37
3.1.13 cvm 的操作............................................................................................................................ 38
3.2 fencing key 操作................................................................................................................383.2.1 查看 key 的方法........................................................................................................................ 38
3.2.2 清除 key 的方法 1..................................................................................................................... 38
3.2.3 清除 key 的方法 2..................................................................................................................... 39
3.2.4 清除 key 的方法 3..................................................................................................................... 40
3.3 snapshot 操作....................................................................................................................403.3.1 说明......................................................................................................................................... 40
3.3.2 创建 dco version 20................................................................................................................. 41
3.3.3 创建 full-size instant snapshot.................................................................................................41
3.3.4 创建 space-optimized instant snapshot....................................................................................42
3.3.5 创建 emulation of third-mirror break-off snapshot....................................................................43
3.3.6 其它操作.................................................................................................................................. 45
3.3.7 snapshot 的性能调优...............................................................................................................46
3.4 mirror 操作.........................................................................................................................463.4.1 注意事项.................................................................................................................................. 46
3.4.2 常见操作.................................................................................................................................. 47
3.4.3 性能调优.................................................................................................................................. 49
3.4.4 对 ssb 情况的处理....................................................................................................................503.5 site mirror..........................................................................................................................51
3.5.1 注意事项.................................................................................................................................. 51
3.5.2 配置 site mirror......................................................................................................................... 51
3.5.3 对存在的 disk group 配置 site mirror........................................................................................52
3.5.4 日常操作.................................................................................................................................. 53
4
Symantec Consulting Service, Great China Region
1 Storage Foundation for RAC/CFS 简单操作Storage Foundation for RAC/CFS 的操作可以采用命令行、java 图形界面和 web 界面进行。
1.1 命令行操作简介
1.1.0 如何重启主机
SFRAC/SFCFS for AIX 环境下要使用 shutdown –ry 0 命令来重启主机,不能使用 reboot 命令
shutdown –ry 0 命令会执行/etc/rc.d 下的 K 脚本,会正常地停止 cluster 软件,而 reboot 不会。
如果使用 reboot 命令重启后,需要查看 errpt,如果有 fencing 的报错,并且无法 import disk group, 就要使用 vxfenclearpre 命令后重启主机才能恢复正常。
1.1.1 检查 cluster 状态
rp84db1:/#hastatus –sum 查看总体的 cluster 状态
-- SYSTEM STATE
-- System State Frozen
A rp84db1 RUNNING 0
A rp84db2 RUNNING 0
-- GROUP STATE
-- Group System Probed AutoDisabled State
B Oradb rp84db1 Y N ONLINE
B Oradb rp84db2 Y N ONLINE
B ccolap_sg rp84db1 Y N OFFLINE
B ccolap_sg rp84db2 Y N ONLINE
B cvm rp84db1 Y N ONLINE
B cvm rp84db2 Y N ONLINE
Rp84db1:/#hastatus 用于查看动态的 cluster 信息
attempting to connect....connected
group resource system message
5
Symantec Consulting Service, Great China Region
--------------- -------------------- -------------------- --------------------
rp84db1 RUNNING
rp84db2 RUNNING
Oradb rp84db1 ONLINE
Oradb rp84db2 ONLINE
-------------------------------------------------------------------------
cvm rp84db1 ONLINE
cvm rp84db2 ONLINE
ccolap_sg rp84db2 ONLINE
ccolap_sg rp84db1 OFFLINE
CFSocrvote rp84db1 ONLINE
-------------------------------------------------------------------------
CFSocrvote rp84db2 ONLINE
CFSoradb rp84db1 ONLINE
CFSoradb rp84db2 ONLINE
CFSorafb rp84db1 ONLINE
CFSorafb rp84db2 ONLINE
-------------------------------------------------------------------------
DGocrvote rp84db1 ONLINE
DGocrvote rp84db2 ONLINE
DGora rp84db1 ONLINE
DGora rp84db2 ONLINE
vxfsckd rp84db1 ONLINE
-------------------------------------------------------------------------
vxfsckd rp84db2 ONLINE
cvm_clus rp84db1 ONLINE
cvm_clus rp84db2 ONLINE
cvm_vxconfigd rp84db1 ONLINE
cvm_vxconfigd rp84db2 ONLINE
-------------------------------------------------------------------------
ccolap_dg rp84db2 ONLINE
ccolap_dg rp84db1 OFFLINE
ccolap_oradb_vol rp84db2 ONLINE
ccolap_oradb_vol rp84db1 OFFLINE
ccolap_orafb_vol rp84db2 ONLINE
-------------------------------------------------------------------------
ccolap_orafb_vol rp84db1 OFFLINE
ccolap_etl_vol rp84db2 ONLINE
ccolap_etl_vol rp84db1 OFFLINE
6
Symantec Consulting Service, Great China Region
ccolap_oradb_mnt rp84db2 ONLINE
ccolap_oradb_mnt rp84db1 OFFLINE
-------------------------------------------------------------------------
ccolap_orafb_mnt rp84db2 ONLINE
ccolap_orafb_mnt rp84db1 OFFLINE
ccolap_etl_mnt rp84db2 ONLINE
ccolap_etl_mnt rp84db1 OFFLINE
ccolap_ip rp84db2 ONLINE
-------------------------------------------------------------------------
ccolap_ip rp84db1 OFFLINE
ccolap_nic rp84db2 ONLINE
ccolap_nic rp84db1 ONLINE
ccolap_oracle rp84db2 ONLINE
ccolap_oracle rp84db1 OFFLINE
-------------------------------------------------------------------------
ccolap_listener rp84db2 ONLINE
ccolap_listener rp84db1 OFFLINE
1.1.2 启动服务组
#hagrp –online <service group name> -sys <host name>
1.1.3 停止服务组
#hagrp –offline <service group name> -sys <host name>
1.1.4 切换服务组
这个操作相当于在一边停止服务组,在另一边驱动服务组
#hagrp –switch <service group name> -to <host name>
1.1.5 停止某个资源
#hares –offline <resource name> -sys <host name>
1.1.6 启动某个资源
#hares –online <resource name> -sys <host name>
7
Symantec Consulting Service, Great China Region
1.1.7 发现某个资源出错后清除错误
#hares –clear <resource name> -sys <host name>
1.1.8 如果资源无法启动或停止需要 flush 状态
#hagrp –flush <service group name> -sys <host name>
1.1.9 查看心跳信息
rp84db1:/#gabconfig -a
GAB Port Memberships
===============================================================
Port a gen 705d01 membership 01
Port b gen 705d07 membership 01
Port d gen 705d06 membership 01
Port f gen 705d0f membership 01
Port h gen 705d05 membership 01
Port o gen 705d04 membership 01
Port v gen 705d0b membership 01
Port w gen 705d0d membership 01
1.1.10 查看 cluster 支持 RAC/CFS 的状态
rp84db1:/#vxdctl -c mode
mode: enabled: cluster active - SLAVE
master: rp84db2
相当与 oracle 中的 lsnodes
1.1.11 查看磁盘状态
#vxdisk –o alldgs list
DEVICE TYPE DISK GROUP STATUS
DEVICE TYPE DISK GROUP STATUS
c0t6d0 auto:LVM - - LVM
c2t6d0 auto:LVM - - LVM
c25t0d1 auto:cdsdisk - (vxfencoorddg) online
c25t0d3 auto:cdsdisk - (vxfencoorddg) online
c25t0d4 auto:LVM - - LVM
8
Symantec Consulting Service, Great China Region
c25t0d6 auto:cdsdisk - (vxfencoorddg) online
c25t1d0 auto:cdsdisk - (ccolapdg) online
c25t1d1 auto:cdsdisk - (ccolapdg) online
c31t0d2 auto:cdsdisk ocrvotedg01 ocrvotedg online shared
c31t0d5 auto:cdsdisk oradg02 oradg online shared
c31t0d7 auto:cdsdisk oradg01 oradg online shared
c31t1d2 auto:cdsdisk - (ccolapdg) online
1.1.12 查看 disk group 状态
#vxdg list
NAME STATE ID
oradg enabled,shared,cds 1237977903.46.rp84db1
ocrvotedg enabled,shared,cds 1237979570.48.rp84db1
1.1.13 查看 disk group 剩余空间
#vxdg free
GROUP DISK DEVICE TAG OFFSET LENGTH FLAGS
oradg oradg01 c31t0d7 c31t0d7 398508160 20872960 -
ocrvotedg ocrvotedg01 c31t0d2 c31t0d2 972800 26496 -
1.1.14 查看 volume 的状态
Disk group: oradg
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME RVG KSTATE STATE NVOLUME
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
9
Symantec Consulting Service, Great China Region
EX NAME ASSOC VC PERMS MODE STATE
SR NAME KSTATE
dg oradg default default 45000 1237977903.46.rp84db1
dm oradg01 c31t0d7 auto 32768 419381120 -
dm oradg02 c31t0d5 auto 32768 52379520 -
v oradbvol - ENABLED ACTIVE 398458880 SELECT - fsgen
pl oradbvol-01 oradbvol ENABLED ACTIVE 398458880 CONCAT - RW
sd oradg01-01 oradbvol-01 oradg01 0 398458880 0 c31t0d7 ENA
v orafbvol - ENABLED ACTIVE 52428800 SELECT - fsgen
pl orafbvol-01 orafbvol ENABLED ACTIVE 52428800 CONCAT - RW
sd oradg02-01 orafbvol-01 oradg02 0 52379520 0 c31t0d5 ENA
sd oradg01-02 orafbvol-01 oradg01 398458880 49280 52379520 c31t0d7 ENA
Disk group: ocrvotedg
DG NAME NCONFIG NLOG MINORS GROUP-ID
ST NAME STATE DM_CNT SPARE_CNT APPVOL_CNT
DM NAME DEVICE TYPE PRIVLEN PUBLEN STATE
RV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRL
RL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNK
CO NAME CACHEVOL KSTATE STATE
VT NAME RVG KSTATE STATE NVOLUME
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
EX NAME ASSOC VC PERMS MODE STATE
SR NAME KSTATE
dg ocrvotedg default default 35000 1237979570.48.rp84db1
dm ocrvotedg01 c31t0d2 auto 32768 999296 -
10
Symantec Consulting Service, Great China Region
v ocrvotevol - ENABLED ACTIVE 972800 SELECT - fsgen
pl ocrvotevol-01 ocrvotevol ENABLED ACTIVE 972800 CONCAT - RW
sd ocrvotedg01-01 ocrvotevol-01 ocrvotedg01 0 972800 0 c31t0d2 ENA
1.1.15 添加一个 disk group
添加 cluster 的 disk group
vxdg -s init <disk group name> <disk name>
添加普通的 disk group
vxdg init <disk group name> <disk name>
1.1.16 在 disk group 中添加一块盘
#vxdg –g <disk group name> adddisk <disk name>
1.1.17 创建 stripe 的 volume
#vxassist –g < disk group name> make <volume name> 500g layout=stripe stripeunit=64k ncol=2
1.1.18 创建文件系统
#mkfs –V vxfs –o largefiles /dev/vx/rdsk/<disk group name>/<volume name>
1.1.19 mount 文件系统
mount cluster 的文件系统:
#mount –V vxfs –o cluster,largefiles /dev/vx/dsk/<disk group name>/<volume name> /<mount point name>
mount 文件系统:
#mount –V vxfs –o largefiles /dev/vx/dsk/<disk group name>/<volume name> /<mount point name>
1.1.20 查看 cluster 的日志
#tail –f /var/VRTSvcs/log/engine_A.log
11
Symantec Consulting Service, Great China Region
2006/10/22 19:44:25 VCS NOTICE V-16-1-10447 Group ysdb_sg is online on system rp84db1
2006/10/22 19:44:25 VCS INFO V-16-6-15004 (rp84db1) hatrigger:Failed to send trigger for nfs_restart; script doesn't exist
2006/10/22 19:44:25 VCS INFO V-16-6-15004 (rp84db1) hatrigger:Failed to send trigger for postonline; script doesn't exist
2006/10/22 20:55:53 VCS INFO V-16-1-10077 Received new cluster membership
2006/10/22 20:55:53 VCS NOTICE V-16-1-10080 System (rp84db1) - Membership: 0x3, Jeopardy: 0x2
2006/10/22 20:55:53 VCS ERROR V-16-1-10087 System rp84db2 (Node '1') is in Regardy Membership - Membership: 0x3, Jeopardy: 0x2
2006/10/22 20:55:53 VCS INFO V-16-6-15004 (rp84db2) hatrigger:Failed to send trigger for injeopardy; script doesn't exist
2006/10/22 20:56:46 VCS INFO V-16-1-10077 Received new cluster membership
2006/10/22 20:56:46 VCS NOTICE V-16-1-10080 System (rp84db1) - Membership: 0x3, Jeopardy: 0x0
2006/10/22 20:56:46 VCS NOTICE V-16-1-10086 System rp84db2 (Node '1') is in Regular Membership - Membership: 0x3
1.1.21 查看 volume manager 的日志
AIX 使用#errpt –a
HP-UX 查看/var/log/syslog/syslog.log
1.1.22 删除一个 volume
#vxedit –g <disk group name> -rf rm <volume name>
1.1.23 删除一个 disk group
#vxdg destroy <disk group name>
1.1.24 如何扩大或缩小文件系统
To resize a file system and a volume that contains it, the vxresize command can be
used. The command should be run from the cvm master node. If resizing the volume or
the file system independent of each other, then the command should be run from the cvm
master node or the cfs primary node respectively.
To determine the primary node for a file system in a cluster, type:
# fsclustadm –v showprimary mount_point
To determine if the current node is the master CVM node, type:
12
Symantec Consulting Service, Great China Region
# vxdctl -c mode
mode: enabled: cluster active - SLAVE
master: star33
如果 cvm 的 master 和 cfs 的 primary node 在一边,可以使用如下命令
# vxresize -g <disk group name> -b -F vxfs <volume name> 10g (此命令只能用于mount起来的文件系统)
1.1.25 如何禁止 fencing
You may have to disable fencing in the following cases:
◆ The cluster has been upgraded to the latest SFCFS stack and the storage does not
support SCSI-3 PGR feature.
◆ During installation fencing was turned on but later you want to turn it off.
By default, the VxFEN driver operates with I/O fencing enabled. To disable this feature
without removing the coordinator disks, you must create the file /etc/vxfenmode and
include a string within the file to notify the VxFEN driver, then stop and restart the driver,
as instructed below:
# echo "vxfen_mode=disabled" > /etc/vxfenmode
# /etc/rc.d/rc2.d/S97vxfen stop
# /etc/rc.d/rc2.d/S97vxfen start
1.1.26 shutdown 说明
不要使用 reboot 进行停机
要使用 shutdown –Fr 或 shutdown –r –y now 进行
shutdown –Fr 会自动将上面的服务组停止后才重启
1.1.27 检查磁盘的 serial number
# vxfenadm -i /dev/rhdisk75Vendor id : EMCProduct id : SYMMETRIXRevision : 5567Serial Number : 42031000a
#vxdmpinq /dev/rhdisk75
1.1.28 如何查看 license key
#vxlicrep
13
Symantec Consulting Service, Great China Region
1.1.29 如何添加 license key
#vxlicinst
可以在线添加,立刻生效
1.1.30 如何查看多路径信息
方法 1:
vxdisk list <disk name>
如 vxdisk list GENESIS0_0 输出的最后五行
numpaths: 4
hdisk39 state=enabled
hdisk77 state=enabled
hdisk115 state=enabled
hdisk153 state=enabled
如果线路出错,状态会变成 disabled
同时要注意如果一个盘上没有任何 I/O,磁盘的状态会一直是 enabled
当盘上有 I/O 后并发现线路上有错误,磁盘的状态会变成 disabled
方法 2:
列出主机所有的控制器# vxdmpadm listctlr all列出主机一条控制器上的所有磁盘路径# vxdmpadm getsubpaths ctlr=<ctlr name>
方法 3:#vxdisk path
1.1.31 如何让 cluster 两主机都停止后启动一台主机提供服务
在启动后运行/sbin/gabconfig –c –x
可以适用于 RAC/CFS 环境
1.1.32 如何停止一个 HBA 卡上的 I/O(用于升级微码)
列出主机所有的控制器# vxdmpadm listctlr all
14
Symantec Consulting Service, Great China Region
To disable a controller, use the following command:# vxdmpadm <-f> disable ctlr=<ctlr name>
1.1.33 如何更换一个 HBA 卡
列出主机所有的控制器# vxdmpadm listctlr all
To disable a controller, use the following command:# vxdmpadm <-f> disable ctlr=<ctlr name>rmdev –dl删除HBA卡的磁盘
cfgmgr识别磁盘
vxdmpadm enable ctlr=<ctlr name>
1.1.34 如何添加一个 oracle 使用的裸设备
# vxassist -g <disk group name> make <volume name> <size># vxedit -g <disk group name> set user=oracle group=dba mode=660 <volume name>
例如
# vxassist -g hbora1dg make orasrvvol 300m# vxedit -g hbora1dg set user=oracle group=dba mode=660 orasrvvol
1.1.35 如何停止 cluster 软件
由于当前配置中 oracle 服务和 listener 服务都不由 cluster 软件控制,当需要停止 cluster 软件时,需要手工先停止 oracle 服务和 listener 服务,然后才能使用 hastop –all 命令停止 cluster 软件,同时archive log 的目录会被停止,disk group 会被 deport。
1.1.36 如何手工 import disk group 并启动 volume
注:缺省情况下建议使用 cluster 软件来启动和停止 disk group,cluster 软件自动启动时会自动启动 disk group,停止时会自动停 disk group。
可以先使用 vxdisk –o alldgs list 查看所有磁盘和 disk group,其中带()的 disk group 表明是没有import 进来的 disk group,这些 disk group 可以使用如下命令手工 import
启动 cluster disk group
vxdg –s import <disk group>
vxrecover –sb
15
Symantec Consulting Service, Great China Region
然后如果有 cluster 文件系统就启动 cluster 文件系统
mount –V vxfs –o cluster /dev/vx/dsk/<disk group>/<vol_name> <mount_point>
启动普通的 disk group
vxdg import <disk group>
vxrecover –sb
然后如果有普通文件系统就启动文件系统
mount –V vxfs /dev/vx/dsk/<disk group>/<vol_name> <mount_point>
1.1.37 如何手工 deport disk group
确定 disk group 上的 mount 点都 umount 掉后,可以使用如下命令:
对 cluster 和普通的文件系统,都是如下命令:
umount <mount_point>
确认应用不在,然后使用如下命令
对 cluster disk group 和普通 disk group 都一致:
vxdg deport <disk group>
16
Symantec Consulting Service, Great China Region
2 图形界面简介
以下以操作示例进行介绍
2.1 cluster 图形界面cluster 的操作界面是:
xwindows 下可以使用 hagui
windows 下可以使用安装的 java cluster administrator
两个界面一样
选择要连接的 cluster 主机
17
Symantec Consulting Service, Great China Region
选择用户名和密码,缺省是 admin 和 password
18
Symantec Consulting Service, Great China Region
先看到整个 cluster 的状态,可以看到哪个服务组在哪台主机上启动
19
Symantec Consulting Service, Great China Region
点击 service group 可以看到服务组的依赖状态,图中表示 RAC数据库的服务组依赖底层 cvm 的服务组
20
Symantec Consulting Service, Great China Region
点击具体的服务组可以看到资源的状态
21
Symantec Consulting Service, Great China Region
点击Resource 可以看到资源的依赖状态
蓝色表明资源在线,灰色表示资源停止
22
Symantec Consulting Service, Great China Region
如上图中显示了 ysdb_lstener依赖于 ysdb_oracle 服务;当服务组启动时,oracle 会先启动,然后再启动 listener;当服务组停止时,listener 会先停止,然后再停止 oracle
23
Symantec Consulting Service, Great China Region
在服务组上点击右键可以进行 online(启动),offline(停止),switch(切换),clear fault(清除错误), freeze(冻结),unfreeze(解冻),flush(hang 在启动或停止时清除状态)等
24
Symantec Consulting Service, Great China Region
也可以点在资源上点击右键可以 online(启动),offline(停止),clear fault(清除错误),enabled(启用),critical(重要资源,出错后会进行切换),delete(删除)
25
Symantec Consulting Service, Great China Region
点在资源上可以修改资源的属性
26
Symantec Consulting Service, Great China Region
当修改了属性时,cluster 的配置信息变成读写
27
Symantec Consulting Service, Great China Region
需要点第三个 icon关闭配置文件
28
Symantec Consulting Service, Great China Region
如果一个资源出错就会有红X
29
Symantec Consulting Service, Great China Region
可以选择 clear fault 清除错误
30
Symantec Consulting Service, Great China Region
然后就会变正常
2.2 Volume Manager 图形界面volume manager 使用的操作界面是 vea
31
Symantec Consulting Service, Great China Region
3 高级操作
3.1 dmp 操作
3.1.1 如何禁止多路径
vxdiskadm中选择Prevent multipathing/Suppress devices from VxVM’s view
在sun cluster配置时可能需要
3.1.2 查看磁盘路径的常用命令
#vxdisk list
#vxdisk –o alldgs list
#vxdisk path
#vxdisk –e list
#vxdisk –p list
3.1.3 查看 dmp 路径的常用命令
#vxdmpadm list dmpnode all
# vxdmpadm getsubpaths ctlr=scsi2
# vxdmpadm getsubpaths enclosure=HDS9500V0
3.1.4 查看 controller 的常用命令
# vxdmpadm listctlr all
# vxdmpadm getctlr c5
# vxdmpadm [-c|-f] disable ctlr=ctlr_name
# vxdmpadm enable ctlr=ctlr_name
3.1.5 查看阵列的常用命令
使用 vxdmpadm listenclosure all 可以看出这个盘阵上看到了多少个 lun,同时可以看到是什么类型的盘
阵。如 A/A,A/P,A/P-C,A/A-A等
32
Symantec Consulting Service, Great China Region
# vxdmpadm getattr enclosure Disk iopolicy
# vxdmpadm setattr enclosure enc0 iopolicy=balanced
minimumq
balanced
round-robin
adaptive
3.1.6 查看 dmp 的实时 I/O 状态
vxdmpadm [options] iostat show groupby=enclosure all interval=5 count=10
3.1.7 如何升级阵列的微码
To upgrade the disk controller firmware
1 Disable the plex that is associated with the disk device:
# /opt/VRTS/bin/vxplex -g diskgroup det plex
(The example is a volume mirrored across 2 controllers on one HBA.)
2 Stop I/O to all disks through one controller of the HBA:
# /opt/VRTS/bin/vxdmpadm disable ctlr=first_cntlr
For the other controller on the HBA, enter:
# /opt/VRTS/bin/vxdmpadm -f disable ctlr=second_cntlr
3 Upgrade the firmware on those disks for which the controllers have been
disabled using the procedures that you obtained from the disk drive vendor.
在升级过程中注意:
a. 升级微码的过程中很可能丢失scsi3的key!!!
b. 可能设备号会发生变化,需要重新使用操作系统的命令重新扫描磁盘,然后再使用vxdctl enable
重新扫描磁盘
4 After doing the upgrade, re-enable all the controllers:
# /opt/VRTS/bin/vxdmpadm enable ctlr=first_cntlr
# /opt/VRTS/bin/vxdmpadm enable ctlr=second_cntlr
5 Re-enable the plex associated with the device:
# /opt/VRTS/bin/vxplex -g diskgroup att volume plex
This command takes some time depending upon the size of the mirror set.
33
Symantec Consulting Service, Great China Region
3.1.8 对磁盘 I/O 错误的处理
如何控制dmp在发生I/O错误后如何处理错误
# vxdmpadm setattr \
{enclosure enc-name|arrayname name|arraytype type} \
recoveryoption=fixedretry retrycount=n
# vxdmpadm setattr \
{enclosure enc-name|arrayname name|arraytype type} \
recoveryoption=timebound iotimeout=seconds
# vxdmpadm setattr \
{enclosure enc-name|arrayname name|arraytype type} \
recoveryoption=nothrottle
# vxdmpadm setattr \
{enclosure enc-name|arrayname name|arraytype type}\
recoveryoption=throttle {iotimeout=seconds|queuedepth=n}
# vxdmpadm gettune
Tunable Current Value Default Value
------------------------------ ------------- -------------
dmp_failed_io_threshold 57600 28800.
>>>
dmp_retry_count 5 5.
>>>
dmp_pathswitch_blks_shift 11 9.
>>>
dmp_queue_depth 32 32.
>>>
dmp_cache_open on on.
>>>
dmp_daemon_count 10 10.
dmp_scsi_timeout 30 30.
>>>
dmp_delayq_interval 15 15.
dmp_path_age 300 300.
34
Symantec Consulting Service, Great China Region
dmp_stat_interval 1 1.
dmp_health_time 60 60.
dmp_probe_idle_lun on on.
dmp_log_level 1 1.
dmp_fast_recovery on on.
>>>
dmp_enable_restore on on.
dmp_restore_policy check_disabled check_disabled.
dmp_restore_interval 300 300.
dmp_restore_cycles 10 10.
dmp_monitor_fabric off on.
3.1.9 查看 asl 和 apm 信息
# vxdmpadm listapm all
# vxdmpadm listapm module_name
To add and configure an APM, use the following command:
# vxdmpadm -a cfgapm module_name [attr1=value1 \
[attr2=value2 ...]]
# vxddladm list
# vxddladm list devices
# vxddladm list hbas
# vxddladm list ports
To list all of the targets, use the following command:
# vxddladm list targets
# vxddladm listsupport all
# vxddladm listsupport libname=library_name.so
# vxddladm excludearray libname=libvxenc.so
# vxddladm includearray libname=libvxenc.so
35
Symantec Consulting Service, Great China Region
3.1.10添加对不支持阵列的简单支持
如果阵列不支持,可以按照如下方法添加 A/A 或 A/P 的 jbod 阵列
# vxddladm addjbod vid=vendorid [pid=productid] \
[serialnum=opcode/pagecode/offset/length]
[cabinetnum=opcode/pagecode/offset/length] policy={aa|ap}]
# /etc/vx/diag.d/vxdmpinq /dev/hdisk10
#vxdctl enable
# vxddladm listjbod
3.1.11磁盘名称的显示方式
# vxddladm get namingscheme
# vxddladm set namingscheme=ebn [persistence={yes|no}] \
[use_avid=yes|no] [lowercase=yes|no]
# vxddladm set namingscheme=osn [persistence={yes|no}] \
[lowercase=yes|no]
hp-ux 11.31 上多了mode, new 表示是 diskX 的方式,但这个只是显示,实际上 dmp 并不使用这个名
字
# vxddladm set namingscheme=osn [mode={default|legacy|new}] \
[persistence={yes|no}]
# vxdmpadm setattr enclosure enclosure tpdmode=native|pseudo
3.1.12重新生成磁盘名
对 SF5MP3即今后版本
To regenerate the persistent names repository, use the following command:
# vxddladm [-c] assign names
标准做法
36
Symantec Consulting Service, Great China Region
To update the disk names so that they correspond to the new path names
1 Remove the file that contains the existing persistent device name database:
# rm /etc/vx/disk.info
# rm /dev/vx/rdmp/*
# rm /dev/vx/dmp/*
2 Restart the VxVM configuration demon:
# vxconfigd -k
This regenerates the persistent name database.
3.1.13 cvm 的操作
# vxdg -g diskgroup set diskdetpolicy=local dgfailpolicy=leave
# /etc/vx/bin/vxclustadm nodestate
# /etc/vx/bin/vxclustadm nidmap
3.2 fencing key 操作
3.2.1 查看 key 的方法
#vxfenadm –g /dev/rhdisk1
#vxfenadm –G /dev/rhdisk1
本命令可以看出是哪台主机注册的 key
#vxfenadm –r /dev/rhdisk1
#vxfenadm –R /dev/rhdisk1
本命令可以看出是哪台主机保留的 key
3.2.2 清除 key 的方法 1
停止所有主机的 HA
#hastop –all
停止所有主机的 vxfen
/etc/rc.d/rc2.d/K98vxfen stop
37
Symantec Consulting Service, Great China Region
在一台主机上执行
vxfenclearpre
重启所有主机
3.2.3 清除 key 的方法 2
这种清除 key 的原理是替换 key 的方法,这个脚本是从 vxfenclearpre演变出来的
VXFENADM=/sbin/vxfenadm
GREP=/usr/bin/grep
AWK=/usr/bin/awk
for i in `lsdev -Ccdisk|grep EMC|awk '{ print $1 }'`
do
echo ---------------
echo checking vxfenadm /dev/r$i
key=`$VXFENADM -g /dev/r$i | $GREP Numeric | $AWK '{print $5}'`
if [ ! -z "$key" ]
then
echo "/dev/r$i" > /tmp/disk
for z in $key
do
#
# First make sure that we are not the
# owner of the key by deleting it.
#
$VXFENADM -x -K$z -f /tmp/disk > /dev/null 2>&1
done
for z in $key
do
#
# Even though it may have been our key,
# also do the register and preempt abort
# in case another node has the same key.
#
$VXFENADM -a -k"VERITASP" -f /tmp/disk > /dev/null 2>&1
$VXFENADM -p -V$z -k"VERITASP" -f /tmp/disk > /dev/null 2>&1
38
Symantec Consulting Service, Great China Region
$VXFENADM -x -k"VERITASP" -f /tmp/disk
done
fi
done
3.2.4 清除 key 的方法 3
使用未公开的参数,这种方法不需要查到原来的 key
vxfenadm -a -k TMP -f /tmp/data_disks
vxfenadm -c -k TMP -f /tmp/data_disks
3.3 snapshot 操作
3.3.1 说明
Storage Foundation 5版本支持如下 5种 snapshot 方式:
1. traditional third-mirror break-off snapshot
这种方法用的是 vxassist 命令,dco版本是 0,现在一般不使用,所以本文不描述。在 nbu 的server-free 备份中使用还较多。
2. full-size instant snapshot
instant 的意思是做了快照立刻就可以使用,不需要等待长时间的同步结束。实际的同步可以在后台慢慢完成
这种方法用的是 vxsnap 命令,dco版本是 20,现在使用较多
第一次初始化同步和 vxsnap refresh 使用 copy-on-write技术,对性能有一定影响。
3. space-optimized instant snapshot
只需要 20%-30%的空间,只可以用于本机,可以用于文件系统在 fsck 前做个备份
4. Emulation of third-mirror break-off snapshot
这种方法用的是 vxsnap 命令,dco版本是 20,现在使用较多
第一次初始化同步不使用 copy-on-write技术,不使用 vxsnap refresh,而是 reattach,对性能影响小。
5. linked break-off snapshot
由于新出来,暂不使用
如下部分主要介绍 2、3、4 这三种 instant snapshot
39
Symantec Consulting Service, Great China Region
3.3.2 创建 dco version 20
检查是否有 dco version 20 和启用了 fastresync
# vxprint -g volumedg -F%instant volume
# vxprint -g volumedg -F%fastresync volume
创建 dco version 20
# vxsnap [-g diskgroup] prepare volume [regionsize=size] [ndcomirs=number]
[alloc=storage_attributes]
regionsize缺省值是64KB
ndcomirs 控制建出来的有几份,缺省两份
vxsnap 无法使用 mirror=enclr,所以需要用 alloc=xx,xx 来指定磁盘名
3.3.3 创建 full-size instant snapshot
Full-sized instant snapshots are not suitable for write-intensive volumes (such
as for database redo logs) because the copy-on-write mechanism may degrade the
performance of the volume.
Copy-on-write 只用在第一次同步和 refresh 的时候,平时没有用
1.确认原volume的长度
# LEN=`vxprint [-g diskgroup] -F%len volume`
也可以使用 vxprint –v 查看
2.查看原 dco 的名字
# DCONAME=`vxprint [-g diskgroup] -F%dco_name volume`
3.查看原 regionsize
# RSZ=`vxprint [-g diskgroup] -F%regionsz $DCONAME`
4.创建snap_vol(同时创建了dco)
# vxassist [-g diskgroup] make snapvol $LEN \
llayout=mirror nmirror=number] logtype=dco drl=off \
dcoversion=20 [ndcomirror=number] regionsz=$RSZ \
init=active [storage_attributes]
40
Symantec Consulting Service, Great China Region
5. 创建snapshot
# vxsnap [-g diskgroup] make source=volume/snapvol=snapvol\
[/snapdg=snapdiskgroup] [/syncing=off]
缺省会自动在后台做同步syncing=on
也可以定义成无须做同步,就象space-optimized的那样。
如果需要把snapshot拆分到其它disk group,必须等待同步结束
使用如下命令等待同步结束:
# vxsnap [-g diskgroup] syncwait snapvol
也可以用如下命令查看同步是否结束:
# vxprint [-g diskgroup] -F%incomplete snapvol
6.不管是syncing=on或syncing=off,都可以立刻使用
fsck –V vxfs /dev/vx/dsk/diskgroup/snapvol
7.可以refresh把生产卷上的内容重新同步到snapvol上,然后立刻可以使用,无须等待同步结束。
Snapvol的文件系统必须umount
8.危险!可以restore把snapvol上的内容反向同步到生产卷, 然后立刻可以使用,无须等待同步结束。
生产卷的文件系统必须umount
3.3.4 创建 space-optimized instant snapshot
1.To create a space-optimized instant snapshot, snapvol, and also create a
cache object for it to use:
# vxsnap [-g diskgroup] make source=vol/newvol=snapvol\
[/cachesize=size][/autogrow=yes][/ncachemirror=number]\
[alloc=storage_attributes]
2.立刻就可以使用fsck对snapvol做操作,无须等待同步结束(实际上就不会有同步)
# fsck -V vxfs /dev/vx/dsk/diskgroup/snapvol
3.可以refresh把生产卷上的内容重新同步到snapvol上,然后立刻可以使用,无须等待同步结束(实际上
就不会有同步)。
41
Symantec Consulting Service, Great China Region
Snapvol的文件系统必须umount
4.危险!可以restore把snapvol上的内容反向同步到生产卷, 然后立刻可以使用,无须等待同步结束(实际
上就不会有同步)。
生产卷的文件系统必须umount
5.删除snapshot
# vxedit -g diskgroup -r rm snapvol
# vxcache -g diskgroup stop cache_object
# vxedit -g diskgroup -r rm cache_object
3.3.5 创建 emulation of third-mirror break-off snapshot
Break-off snapshots are suitable for write-intensive volumes, such asdatabase redo logs.
确认原volume 的长度
# LEN=`vxprint [-g diskgroup] -F%len volume`
也可以使用 vxprint –v 查看
查看原 dco 的名字
# DCONAME=`vxprint [-g diskgroup] -F%dco_name volume`
查看原 regionsize
# RSZ=`vxprint [-g diskgroup] -F%regionsz $DCONAME`
4.创建mirror的plex,同时状态是SNAPDONE
可以使用如下两种方法创建
a. # vxsnap [-b] [-g diskgroup] addmir volume [nmirror=N] [alloc=storage_attributes]
这个命令会自动做镜像,同时plex做完镜像状态是SNAPDONE
使用如下命令查看状态
# vxsnap -g mydg snapwait vol1 nmirror=2
b. 创建mirror
vxassist mirror
把mirror的plex变成SNAPDONE,看3.3.6.3
42
Symantec Consulting Service, Great China Region
5.创建快照,由于前面是mirror,所以不需要同步
# vxsnap [-g diskgroup] make source=volume[/newvol=snapvol]\
{/plex=plex1[,plex2,...]|/nmirror=number}}
6.可以立刻使用
fsck –V vxfs /dev/vx/dsk/diskgroup/snapvol
7.可以reattach再做snapshot把生产卷上的内容重新同步到snapvol上,然后立刻可以使用,无须等待
同步结束。
Snapvol的文件系统必须umount
# vxsnap [-g diskgroup] reattach snapvolume|snapvolume_set \
source=volume|volume_set [nmirror=number]
等待同步结束
# vxsnap -g mydg snapwait myvol nmirror=1
然后再做snapshot
# vxsnap [-g diskgroup] make source=volume[/newvol=snapvol]\
{/plex=plex1[,plex2,...]|/nmirror=number}}
8.危险!可以restore把snapvol上的内容反向同步到生产卷, 然后立刻可以使用,无须等待同步结束。
生产卷的文件系统必须umount
9.refresh是否能做不确定
3.3.6 其它操作
3.3.6.1snapshot 的两种等待vxsnap make 后等待 copy-on-write 同步结束
# vxsnap [-g diskgroup] syncwait snapvol
vxsnap refresh后等待copy-on-write同步结束
43
Symantec Consulting Service, Great China Region
# vxsnap [-g diskgroup] syncwait snapvol
vxsnap addmir 后等待mirror结束
# vxsnap -g mydg snapwait vol1 nmirror=2
vxsnap reattach 后等待增量mirror结束
# vxsnap -g mydg snapwait myvol nmirror=1
3.3.6.2查看 snapshot 状态# vxsnap -g mydg print
# vxsnap [-g diskgroup] -n [-l] [-v] [-x] print [vol]
3.3.6.3把 mirror plex 变成 snapshot plex(无须同步)
dco可以是0或20# vxplex [-g diskgroup] -o dcoplex=dcologplex convert \state=SNAPDONE plex
3.3.6.4把 snapshot plex 变成 mirror plex(无须同步)
原状态应该是SNAPDONE# vxplex [-g diskgroup] convert state=ACTIVE plex
3.3.6.5删除 snapdone 的 plex
# vxsnap -g mydg rmmir vol1
3.3.6.6拆分快照把 snapshot volume 变成独立的 volume
# vxsnap [-f] [-g diskgroup] dis snapvolume
3.3.6.7删除 snapshot
disassociate volume后删除# vxsnap [-f] [-g diskgroup] dis snapvolume# vxedit -g mydg -r rm snap2myvol
3.3.7 snapshot 的性能调优
a.regionsize必须小于voliomem_maxpool_sz的一半# vxsnap -g mydg -f unprepare vol1# vxsnap -g mydg prepare vol1 regionsize=1M
44
Symantec Consulting Service, Great China Region
regionsize可以控制同步的速度Use the vxprint command on the DCO to discover its region size (in blocks):# RSZ=`vxprint [-g diskgroup] -F%regionsz $DCONAME`缺省值是16K,推荐使用64K
b.vxsnap或vxassist参数中间带iosize和slow
同时vxtask命令中可以通过set设置slow来减低同步速度#vxtask –l list
#vxtask set slow=x tag
c.volpagemod_max_memsz
对比较大的如 1TB 以上的 volume,建议修改这个值
1. Change "volpagemod_max_memsz" online temporarily with the following command (notice the :value is followed by a "k"):
# vxtune volpagemod_max_memsz 65536k
2. To make the change permanent across reboots, add the following entry to the /etc/vx/vxvm_tunables file by running the following command (notice the value is NOT followed by a "k"):
# vxvoltune volpagemod_max_memsz 65536
aix 使用 smitty 进行修改
solaris 修改/kernel/drv/vxio.conf
3.4 mirror 操作
3.4.1 注意事项
1.必须使用非 layer 的 volume, layer 的 volume经测试发现会做全同步
因为 layer 的 volume 的 dco 是加在最外面的 volume 上的,而不是加在 subvolume 上的
2.vxreattach 问题
# As a part of incident 108818, it was decided that we will decide
# whether to use FMR for sync'ing plexes while reattach'ing the disks,
# depending upon the default file. Fmr will not be used if the default
# file does not exist
#
p_opt="-o plex:nofmr"
default_file="/etc/default/vxreattach"
所以必须创建/etc/default/vxreattach 文件,否则 vxreattach做的是全同步
45
Symantec Consulting Service, Great China Region
3.mirror=enclosure 的 volume 扩大时可能不保持 mirror=enclosure 的属性
必须使用如下两种方法之一:
a.echo "mirror=enclosure" >> /etc/default/vxassist
b.vxresize 后必须加 mirror=enclr
4. 可以 disable vxrelocd
修改/etc/init.d/vxvm-recover 中把 vxrelocd 前加注释号
5.vxsize 后原来非 layered 的 volume 变成 layered volume
When a non-ISP volume is grown, its layout may be converted as a side effect if vxassist determines that the new volume is too large for the original layout. The values of the stripe-mirror-col-trigger-pt and stripe-mirror-col-split-trigger-pt attributes (by default, 1 gigabyte) control whether a new layout will be applied. A mirror-stripe volume that is larger than the value of stripe-mirror-col-trigger-pt is converted to a stripe-mirror volume. If each column of a stripe-mirror-col volume is larger than the value of stripe-mirror-col-split-trigger-pt, the volume is converted to a stripe-mirror-sd volume where the individual subdisks, rather than the columns, are mirrored. A mirror-concat volume that is larger than the value of stripe-mirror-col-split-trigger-pt is converted to a concat-mirror volume where the individual subdisks, rather than the plexes, are mirrored.
# cat /etc/default/vxassist
stripe-mirror-col-trigger-pt=10g
stripe-mirror-col-split-trigger-pt=10g
3.4.2 常见操作
1.如何确认已经打开了 fastresync
# vxprint -g <dg-name> -l <volume-name> | egrep '(Volume|flags)'
Volume: <volume-name>
flags: open writeback fastresync
或者
# vxprint [-g diskgroup] -F%fastresync volume
# vxprint [-g diskgroup] -F%hasdcolog volume
或者
# vxprint -g <dg-name> -m <volume-name> | grep fastresync
46
Symantec Consulting Service, Great China Region
fastresync=on
2.确认 dco 的版本
# vxprint -g cvmdg -m volfmr3_dco | grep version
version=20
或者
Use the vxprint command on the DCO to determine its version number:
# vxprint [-g diskgroup] -F%version $DCONAME
3.查看是否打开drl logging
To determine if DRL is enabled on the volume, use the following command
with the volume’s DCO:
# vxprint [-g diskgroup] -F%drl $DCONAME
Use the vxprint command on the DCO volume to find out if DRL logging is
active:
# vxprint [-g diskgroup] -F%drllogging $DCOVOL
4.添加 mirror
# vxassist [-b] [-g diskgroup] mirror volume [storage_attribute]
Another way to mirror an existing volume is by first creating a plex, and then
attaching it to a volume, using the following commands:
# vxmake [-g diskgroup] plex plex sd=subdisk ...
# vxplex [-g diskgroup] att volume plex
5.删除 mirror
# vxplex -g mydg dis vol01-02
# vxedit -g mydg -r rm vol01-02
6.如何启动 disable 的单个卷
vxvol –g diskgroup –f start vol_name
这个步骤等于
a. vxmend -o force off <DisabledRecoverPlex>
b. vxmend on <DisabledRecoverPlex>
c. vxmend fix clean <DisabledRecoverPlex>
d. vxinfo <Volumename>
The volume should now be startable.
e. vxvol -g <diskgroupname> start <NewVolumeName>
47
Symantec Consulting Service, Great China Region
7.如何启动 disable 的 mirror卷
#vxmend -g <diskgroup name> -o force off testvol-01
#vxmend -g <diskgroup name> fix clean testvol-01
vxvol -g <diskgroup name> start <volume>
#vxmend -g <diskgroup name> on testvol-02
#vxplex -g <diskgroup name> att <volume name> testvol-02
8.detach 一个 plex
# vxplex [-g diskgroup] det plex
9.attach 一个 plex
会自动做同步,有 dco 会自动做增量同步
# vxplex [-g diskgroup] att volume plex
3.4.3 性能调优
1.regionsize必须小于voliomem_maxpool_sz的一半
# vxsnap -g mydg -f unprepare vol1
# vxsnap -g mydg prepare vol1 regionsize=1M
regionsize可以控制同步的速度
Use the vxprint command on the DCO to discover its region size (in blocks):
# vxprint [-g diskgroup] -F%regionsz $DCONAME
缺省值是16K,推荐使用64K
2.vxsnap或vxassist参数中间带iosize和slow
iosize可以设置为8M,可以提高速度
同时vxtask命令中可以通过set设置slow来减低同步速度
#vxtask –l list
#vxtask set slow=x tag
3.volpagemod_max_memsz
对比较大的如 1TB 以上的 volume,建议修改这个值
a. Change "volpagemod_max_memsz" online temporarily with the following command
(notice the :value is followed by a "k"):
48
Symantec Consulting Service, Great China Region
# vxtune volpagemod_max_memsz 65536k
b. To make the change permanent across reboots, add the following entry to the
/etc/vx/vxvm_tunables file by running the following command (notice the value is NOT
followed by a "k"):
# vxvoltune volpagemod_max_memsz 65536
aix 使用 smitty 进行修改
solaris 修改/kernel/drv/vxio.conf
4.需要调整 read policy,确认从本地读
# vxvol [-g diskgroup] rdpol round volume
round
prefer
select
siteread
# vxvol [-g diskgroup] rdpol prefer volume preferred_plex
5.mirror远端的盘阵尽量要做 striping
可以适当提高性能
3.4.4 对 ssb 情况的处理
处理方法 1:
使用 vxsplitlines,这个命令会告诉你需要如何执行 vxdg –o selectcp=<disk_id> import xxdg
但这个命令比较慢
处理方法 2:
如果最近没有 volume metadata 的修改,应该使用任何一份 configuration copy 都可以。所以可以直接使用 vxdg –o selectcp=<disk_id> import xxdg
这个 disk_id 是用 vxdisk list disk 看出来的 id
需要预先确认这个盘有 configuration copy,可以查看原来的 vxdg list xxdg 的信息
也可以使用 vxprivutil scan/list/dumpconfig disk 来查看这个盘是否有 configuration copy
处理方法 3:
提前修改 disk group 的 ssb属性
vxdg –g xxdg set ssb=off
49
Symantec Consulting Service, Great China Region
处理方法 4:
使用 vxprivutil 命令进行修改,这种方法使用起来比较困难
使用 #vxprivutil set /dev/rdsk/c1t12d0s2 ssbid=0.2 设置 ssbid
使用/etc/vx/diag.d/vxprivutil dumpconfig /dev/vx/dmp/Disk_2s2 查看 ssbid
3.5 site mirror
3.5.1 注意事项
1.必须关闭 sf5.0mp3 for aix 上的 autotagging属性(是缺省值)
否则有问题
3.vxsited 是脚本,会自动进行 reattachsite 和 vxrecover 的工作,发现有
vxrelocd 和没有 vxrelocd 的行为模式不一样
同时 vxsited 是不会自动去做-o overridessb 的操作,需要手工执行
3.5.2 配置 site mirror
1.对主机设置 site
# vxdctl set site=sitename
# vxdctl list | grep siteid
# vxdctl [-F] unset site
2.关闭 autotagging
# vxdg [-g diskgroup] set autotagging=off
3.对所有磁盘添加 tag
# vxdisk [-g diskgroup] settag disk site=sitename
# vxdisk [-g diskgroup] listtag
# vxdisk rmtag disk site=sitename
4.对 disk group 添加 site 信息
# vxdg -g diskgroup [-f] addsite sitename
# vxdg -g diskgroup [-f] rmsite sitename
5.打开site consistent
# vxdg -g diskgroup set siteconsistent=on
50
Symantec Consulting Service, Great China Region
# vxdg list diskgroup | grep siteconsistent
# vxdg -g diskgroup set siteconsistent=off
新加的volume自动会带disk group中siteconsistent=on属性。只有对已经存在的volume才需要按
照如下方式修改
# vxvol [-g diskgroup] set siteconsistent=on volume
# vxvol [-g diskgroup] set siteconsistent=off volume
缺省自动会配置siteread属性,也可以通过如下命令手工配置
# vxvol [-g diskgroup] rdpol siteread volume
allsite属性表示创建任何 volume,在所有 site 都必须有一份
siteconsistent属性表示添加了 dco 日志
3.5.3 对存在的 disk group 配置 site mirror
1 Ensure that the disk group is updated to at least version 140, by running the
vxdg upgrade command on it:
# vxdg upgrade diskgroup
2 On each host that can access the disk group, define the site name:
# vxdctl set site=sitename
关闭 autotagging
# vxdg [-g diskgroup] set autotagging=off
3 Tag all the disks in the disk group with the appropriate site name:
# vxdisk [-g diskgroup] settag disk site=sitename
4 Use the vxdg move command to move any unsupported RAID-5 volumes to
another disk group. Alternatively, use the vxassist convert command to
convert the volumes to a supported layout such as mirror or
mirror-stripe. You can use the site and mirror=site storage allocation
attribute to ensure that the plexes are created on the correct storage.
5 Use the vxevac command to ensue that the volumes have equal number of
plexes at each site. You can use the site and mirror=site storage allocation
attribute to ensure that the plexes are created on the correct storage.
6 Register a site record for each site with the disk group:
# vxdg -g diskgroup addsite sitename
7 Turn on site consistency for the disk group:
# vxdg -g diskgroup set siteconsistent=on
51
Symantec Consulting Service, Great China Region
8 Turn on site consistency for each volume in the disk group:
# vxvol [-g diskgroup] set siteconsistent=on volume ...
3.5.4 日常操作
1.演习时 detach 一个 site
# vxdg -g diskgroup [-f] detachsite sitename
2.只 import 单边的 site
vxdg -o site=sitename import dgname
3.恢复
# vxdg -g diskgroup [-o overridessb] reattachsite sitename# vxrecover -g diskgroup
如果是手工 detach 的 site,vxsited 后台进程不会自动恢复 site
如果是由于故障 detach 的 site,故障修复后,vxsited 会自动执行 vxdg –g diskgroup reattachsite sitename; vxrecover –g diskgroup。但如果发生 ssb,vxsited 就不会进行处理,等待手工干预。所以需要手工使用 vxdg –g diskgroup –o overridesb reattachsite sitename 和 vxrecover –g diskgroup
52