本帖最后由 mahan 于 2024-5-19 17:55 编辑
一、需求
由于ORAARCH 和 ORADATA 这两个ASM磁盘组空间紧张，从存储上划分出了两块盘分别为600G 和 2T，需要将这两块盘分别添加到这两个ASM磁盘组，从而对这两个磁盘组的空间进行扩容。
二、在线扫盘
–在线扫盘，识别新增磁盘，扫盘不用担心，不会影响集群和数据库，所有的rac的节点都要做
# for host in /sys/class/scsi_host/host*;do echo "- - -" > $host/scan;done
三、修改磁盘配置
3.1 磁盘配置说明
• 在ORACLE RAC中一般有四层磁盘
• 第一层：裸盘（由存储划分过来的）
• 第二层：使用多路径绑定后的磁盘
• 第三层：UDEV绑定多路径ID后的盘
• 第四层：ASM 磁盘
3.2 查看裸盘
• 这里重点要看扫盘后的磁盘哪些是跟客户给的ID一致的磁盘
• 这里的sdai、sdaw、sdg是同一个WWN，说明是同一个磁盘的三条路径
# for i in `cat /proc/partitions | awk {'print $4'} |grep sd`; do echo "### $i: `/usr/lib/udev/scsi_id --whitelist /dev/$i`"; done
### sda: 3600508e000000000399f1f6da560e00d
### sda1: 3600508e000000000399f1f6da560e00d
### sda2: 3600508e000000000399f1f6da560e00d
### sdai: 3600b3424ad60e33def22d24ead0000d9
### sdak: 3600b34287abddc4dad20d9d56d2900d6
### sdam: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdao: 3600b342225bddbed42a3d27c2d5900d6
### sdar: 3600b34242a9fee6de689d7731d5900d9
### sdat: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdau: 3600b342fdd3354dde1d4dd132d4800d9
### sdaw: 3600b3424ad60e33def22d24ead0000d9
### sday: 3600b34287abddc4dad20d9d56d2900d6
### sdba: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdbd: 3600b342225bddbed42a3d27c2d5900d6
### sdbf: 3600b34242a9fee6de689d7731d5900d9
### sdbh: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdbi: 3600b342fdd3354dde1d4dd132d4800d9
### sdg: 3600b3424ad60e33def22d24ead0000d9
### sdh: 3600b34287abddc4dad20d9d56d2900d6
### sdi: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdj: 3600b342225bddbed42a3d27c2d5900d6
### sdk: 3600b34242a9fee6de689d7731d5900d9
### sdl: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdm: 3600b342fdd3354dde1d4dd132d4800d9
### sdn: 3600b3424c37e15cda19ede154d7100d6
### sdo: 3600b342df65ee3adaf40d84ffd9d00d6
### sdp: 3600b3424c37e15cda19ede154d7100d6
### sdq: 3600b342df65ee3adaf40d84ffd9d00d6
### sdr: 3600b3424c37e15cda19ede154d7100d6
### sds: 3600b342df65ee3adaf40d84ffd9d00d6
### sdt: 3600b3424c37e15cda19ede154d7100d6
### sdu: 3600b342df65ee3adaf40d84ffd9d00d6
3.3 修改多路径配置（所有的rac的节点都要修改）
–查看当前多路径情况
# multipath -ll
ora-archive-0002 (3600b342df65ee3adaf40d84ffd9d00d6) dm-11 Sugon ,LU
size=600G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:8 sdq 65:0 active ready running
| `- 20:0:1:8 sds 65:32 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:2:8 sdo 8:224 active ready running
`- 20:0:2:8 sdu 65:64 active ready running
ora-archive-0001 (3600b3424ad60e33def22d24ead0000d9) dm-4 Sugon ,LU
size=600G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:0 sdai 66:32 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:3:0 sdaw 67:0 active ready running
`- 20:0:1:0 sdg 8:96 active ready running
ora-vote-0003 (3600b342fdd3354dde1d4dd132d4800d9) dm-2 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:6 sdau 66:224 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:3:6 sdbi 67:192 active ready running
`- 20:0:1:6 sdm 8:192 active ready running
ora-vote-0002 (3600b34293c07bbdd3d02d2b24dcc00d6) dm-8 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:5 sdbh 67:176 active ready running
| `- 20:0:1:5 sdl 8:176 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 15:0:2:5 sdat 66:208 active ready running
ora-vote-0001 (3600b34242a9fee6de689d7731d5900d9) dm-7 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:4 sdar 66:176 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:3:4 sdbf 67:144 active ready running
`- 20:0:1:4 sdk 8:160 active ready running
ora-data-0004 (3600b3424c37e15cda19ede154d7100d6) dm-10 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:2:7 sdn 8:208 active ready running
| `- 20:0:2:7 sdt 65:48 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:3:7 sdp 8:240 active ready running
`- 20:0:1:7 sdr 65:16 active ready running
ora-data-0003 (3600b342225bddbed42a3d27c2d5900d6) dm-5 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:3 sdbd 67:112 active ready running
| `- 20:0:1:3 sdj 8:144 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 15:0:2:3 sdao 66:128 active ready running
ora-data-0002 (3600b3428a8d6156d1e7cddfb3dd900d9) dm-6 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:2 sdam 66:96 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
|- 15:0:3:2 sdba 67:64 active ready running
`- 20:0:1:2 sdi 8:128 active ready running
ora-data-0001 (3600b34287abddc4dad20d9d56d2900d6) dm-3 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:1 sday 67:32 active ready running
| `- 20:0:1:1 sdh 8:112 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
`- 15:0:2:1 sdak 66:64 active ready running
–修改配置文件
# vi /etc/multipath.conf
checker_timeout 120
}
devices{
device {
vendor "Sugon"
product "LU"
path_grouping_policy group_by_prio
no_path_retry 30
prio alua
path_checker tur
path_selector "round-robin 0"
hardware_handler "0"
failback 15
}
}
multipaths {
multipath {
wwid 3600b3424ad60e33def22d24ead0000d9
alias ora-archive-0001
}
multipath {
wwid 3600b342df65ee3adaf40d84ffd9d00d6
alias ora-archive-0002 <<<<新增
}
multipath {
wwid 3600b34287abddc4dad20d9d56d2900d6
alias ora-data-0001
}
multipath {
wwid 3600b3428a8d6156d1e7cddfb3dd900d9
alias ora-data-0002
}
multipath {
wwid 3600b342225bddbed42a3d27c2d5900d6
alias ora-data-0003
}
multipath {
wwid 3600b3424c37e15cda19ede154d7100d6
alias ora-data-0004 <<<<新增
}
multipath {
wwid 3600b34242a9fee6de689d7731d5900d9
alias ora-vote-0001
}
multipath {
wwid 3600b34293c07bbdd3d02d2b24dcc00d6
alias ora-vote-0002
}
multipath {
wwid 3600b342fdd3354dde1d4dd132d4800d9
alias ora-vote-0003
}
}
###刷新multipath
# multipath -v3
###查看新增磁盘
# multipath -ll | grep -i 100d6
ora-data-0004 (3600b3424c37e15cda19ede154d7100d6) dm-10 Sugon ,LU
# multipath -ll | grep -i d00d6
ora-archive-0002 (3600b342df65ee3adaf40d84ffd9d00d6) dm-11 Sugon ,LU
# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 557.9G 0 disk
├─sda1 8:1 0 500M 0 part /boot
└─sda2 8:2 0 557.4G 0 part
├─centos-root 253:0 0 293.4G 0 lvm /
├─centos-swap 253:1 0 64G 0 lvm [SWAP]
└─centos-u01 253:9 0 200G 0 lvm /u01
sdg 8:96 0 600G 0 disk
└─ora-archive-0001 253:4 0 600G 0 mpath
sdh 8:112 0 2T 0 disk
└─ora-data-0001 253:3 0 2T 0 mpath
sdi 8:128 0 2T 0 disk
└─ora-data-0002 253:6 0 2T 0 mpath
sdj 8:144 0 2T 0 disk
└─ora-data-0003 253:5 0 2T 0 mpath
sdk 8:160 0 10G 0 disk
└─ora-vote-0001 253:7 0 10G 0 mpath
sdl 8:176 0 10G 0 disk
└─ora-vote-0002 253:8 0 10G 0 mpath
sdm 8:192 0 10G 0 disk
└─ora-vote-0003 253:2 0 10G 0 mpath
sdn 8:208 0 2T 0 disk
└─ora-data-0004 253:10 0 2T 0 mpath
sdo 8:224 0 600G 0 disk
└─ora-archive-0002 253:11 0 600G 0 mpath
3.4 修改UDEV配置（所有的rac的节点都要修改）
--生成udev配置相关信息
# cd /dev/mapper
for i in ora-*; do
printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/"$i" | grep -i dm_uuid)"
done
ora-archive-0001 E: DM_UUID=mpath-3600b3424ad60e33def22d24ead0000d9
ora-archive-0002 E: DM_UUID=mpath-3600b342df65ee3adaf40d84ffd9d00d6 <<<<<新增磁盘
ora-data-0001 E: DM_UUID=mpath-3600b34287abddc4dad20d9d56d2900d6
ora-data-0002 E: DM_UUID=mpath-3600b3428a8d6156d1e7cddfb3dd900d9
ora-data-0003 E: DM_UUID=mpath-3600b342225bddbed42a3d27c2d5900d6
ora-data-0004 E: DM_UUID=mpath-3600b3424c37e15cda19ede154d7100d6 <<<<<新增磁盘
ora-vote-0001 E: DM_UUID=mpath-3600b34242a9fee6de689d7731d5900d9
ora-vote-0002 E: DM_UUID=mpath-3600b34293c07bbdd3d02d2b24dcc00d6
ora-vote-0003 E: DM_UUID=mpath-3600b342fdd3354dde1d4dd132d4800d9
--修改udev配置
# vi /etc/udev/rules.d/99-oracle-asmdevices.rules
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3424ad60e33def22d24ead0000d9",SYMLINK+="asm_arch0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342df65ee3adaf40d84ffd9d00d6",SYMLINK+="asm_arch0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34287abddc4dad20d9d56d2900d6",SYMLINK+="asm_data0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3428a8d6156d1e7cddfb3dd900d9",SYMLINK+="asm_data0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342225bddbed42a3d27c2d5900d6",SYMLINK+="asm_data0003",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3424c37e15cda19ede154d7100d6",SYMLINK+="asm_data0004",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34242a9fee6de689d7731d5900d9",SYMLINK+="asm_vote0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34293c07bbdd3d02d2b24dcc00d6",SYMLINK+="asm_vote0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342fdd3354dde1d4dd132d4800d9",SYMLINK+="asm_vote0003",OWNER="grid",GROUP="asmadmin",MODE="0660"
--重载udev
# udevadm control --reload-rules
# udevadm trigger --type=devices
# ls -lrt /dev/asm*
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0003 -> dm-5
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0002 -> dm-8
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0003 -> dm-3
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0002 -> dm-6
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0001 -> dm-7
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_arch0001 -> dm-2
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0001 -> dm-4
lrwxrwxrwx 1 root root 5 May 15 14:57 /dev/asm_arch0002 -> dm-11 <<<<新增磁盘
lrwxrwxrwx 1 root root 5 May 15 14:57 /dev/asm_data0004 -> dm-10 <<<<新增磁盘
四、扩容ASM磁盘组
4.1 当前asm磁盘情况查询
# su - grid
Last login: Wed May 15 14:37:22 CST 2024 on pts/1
$ sqlplus / as sysasm
SQL*Plus: Release 11.2.0.4.0 Production on Wed May 15 15:01:24 2024
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options
SQL>
col path for a30;
col name for a20;
col mount_status format a10
col header_status format a10
col mode_status format a10
set linesize 400;
select a.group_number, a.name,b.mount_status,b.header_status,b.mode_status, b.path,b.total_mb/1024 gb,b.name
from v$asm_diskgroup a, v$asm_disk b
where a.group_number(+) = b.group_number;
GROUP_NUMBER NAME MOUNT_STAT HEADER_STA MODE_STATU PATH GB NAME
------------ -------------------- ---------- ---------- ---------- ------------------------------ ---------- --------------------
CLOSED CANDIDATE ONLINE /dev/asm_data0004 0
CLOSED CANDIDATE ONLINE /dev/asm_arch0002 0
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0001 10 ORACRS_0000
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0002 10 ORACRS_0001
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0003 10 ORACRS_0002
2 ORAARCH CACHED MEMBER ONLINE /dev/asm_arch0001 600 ORAARCH_0000
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0002 2048 ORADATA_0001
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0001 2048 ORADATA_0000
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0003 2048 ORADATA_0002
9 rows selected.
4.2 扩容磁盘
alter diskgroup ORAARCH add disk '/dev/asm_arch0002' rebalance power 11;
alter diskgroup ORADATA add disk '/dev/asm_data0004' rebalance power 11;
select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation;
SQL> alter diskgroup ORAARCH add disk '/dev/asm_arch0002' rebalance power 11;
Diskgroup altered.
SQL> alter diskgroup ORADATA add disk '/dev/asm_data0004' rebalance power 11;
Diskgroup altered.
SQL> col path for a30;
col name for a20;
col mount_status format a10
col header_status format a10
col mode_status format a10
set linesize 400;
select a.group_number, a.name,b.mount_status,b.header_status,b.mode_status, b.path,b.total_mb/1024 gb,b.name
from v$asm_diskgroupSQL> a, v$asm_disk b
where a.group_number(+) = b.group_number;SQL> SQL> SQL> SQL> SQL> 2 3
GROUP_NUMBER NAME MOUNT_STAT HEADER_STA MODE_STATU PATH GB NAME
------------ -------------------- ---------- ---------- ---------- ------------------------------ ---------- --------------------
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0001 10 ORACRS_0000
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0003 10 ORACRS_0002
1 ORACRS CACHED MEMBER ONLINE /dev/asm_vote0002 10 ORACRS_0001
2 ORAARCH CACHED MEMBER ONLINE /dev/asm_arch0001 600 ORAARCH_0000
2 ORAARCH CACHED MEMBER ONLINE /dev/asm_arch0002 600 ORAARCH_0001
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0004 2048 ORADATA_0003
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0002 2048 ORADATA_0001
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0003 2048 ORADATA_0002
3 ORADATA CACHED MEMBER ONLINE /dev/asm_data0001 2048 ORADATA_0000
9 rows selected.
4.3 查看重平衡状态
SQL> select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation;
INST_ID OPERATION STATE POWER SOFAR EST_WORK EST_RATE EST_MINUTES
---------- ---------- -------- ---------- ---------- ---------- ---------- -----------
1 REBAL RUN 11 128425 128425 0 0
1 REBAL WAIT 11
2 REBAL WAIT 11
2 REBAL RUN 11 3030 1175452 27089 43
|
|手机版|小黑屋|重庆思庄Oracle、Redhat认证学习论坛
( 渝ICP备12004239号-4 )
发表于 2024-5-19 17:49:23
QQ好友和群
QQ空间
腾讯微博
腾讯朋友
收藏
支持
反对