重庆思庄Oracle、Redhat认证学习论坛

标题: ORACLE RAC 扩容ASM磁盘组空间详细步骤 [打印本页]

作者: mahan 时间: 2024-5-19 17:49
标题: ORACLE RAC 扩容ASM磁盘组空间详细步骤
本帖最后由 mahan 于 2024-5-19 17:55 编辑

一、需求
由于ORAARCH 和 ORADATA 这两个ASM磁盘组空间紧张，从存储上划分出了两块盘分别为600G 和 2T，需要将这两块盘分别添加到这两个ASM磁盘组，从而对这两个磁盘组的空间进行扩容。

二、在线扫盘
–在线扫盘，识别新增磁盘，扫盘不用担心，不会影响集群和数据库，所有的rac的节点都要做

# for host in /sys/class/scsi_host/host*;do echo "- - -" > $host/scan;done
三、修改磁盘配置
3.1 磁盘配置说明
• 在ORACLE RAC中一般有四层磁盘
• 第一层：裸盘（由存储划分过来的）
• 第二层：使用多路径绑定后的磁盘
• 第三层：UDEV绑定多路径ID后的盘
• 第四层：ASM 磁盘

3.2 查看裸盘
• 这里重点要看扫盘后的磁盘哪些是跟客户给的ID一致的磁盘
• 这里的sdai、sdaw、sdg是同一个WWN，说明是同一个磁盘的三条路径

# for i in `cat /proc/partitions | awk {'print $4'} |grep sd`; do echo "### $i: `/usr/lib/udev/scsi_id --whitelist /dev/$i`"; done

### sda: 3600508e000000000399f1f6da560e00d
### sda1: 3600508e000000000399f1f6da560e00d
### sda2: 3600508e000000000399f1f6da560e00d
### sdai: 3600b3424ad60e33def22d24ead0000d9
### sdak: 3600b34287abddc4dad20d9d56d2900d6
### sdam: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdao: 3600b342225bddbed42a3d27c2d5900d6
### sdar: 3600b34242a9fee6de689d7731d5900d9
### sdat: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdau: 3600b342fdd3354dde1d4dd132d4800d9
### sdaw: 3600b3424ad60e33def22d24ead0000d9
### sday: 3600b34287abddc4dad20d9d56d2900d6
### sdba: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdbd: 3600b342225bddbed42a3d27c2d5900d6
### sdbf: 3600b34242a9fee6de689d7731d5900d9
### sdbh: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdbi: 3600b342fdd3354dde1d4dd132d4800d9
### sdg: 3600b3424ad60e33def22d24ead0000d9
### sdh: 3600b34287abddc4dad20d9d56d2900d6
### sdi: 3600b3428a8d6156d1e7cddfb3dd900d9
### sdj: 3600b342225bddbed42a3d27c2d5900d6
### sdk: 3600b34242a9fee6de689d7731d5900d9
### sdl: 3600b34293c07bbdd3d02d2b24dcc00d6
### sdm: 3600b342fdd3354dde1d4dd132d4800d9
### sdn: 3600b3424c37e15cda19ede154d7100d6
### sdo: 3600b342df65ee3adaf40d84ffd9d00d6
### sdp: 3600b3424c37e15cda19ede154d7100d6
### sdq: 3600b342df65ee3adaf40d84ffd9d00d6
### sdr: 3600b3424c37e15cda19ede154d7100d6
### sds: 3600b342df65ee3adaf40d84ffd9d00d6
### sdt: 3600b3424c37e15cda19ede154d7100d6
### sdu: 3600b342df65ee3adaf40d84ffd9d00d6
3.3 修改多路径配置（所有的rac的节点都要修改）
–查看当前多路径情况

# multipath -ll
ora-archive-0002 (3600b342df65ee3adaf40d84ffd9d00d6) dm-11 Sugon ,LU
size=600G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:8 sdq  65:0 active ready running
| `- 20:0:1:8 sds  65:32  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:2:8 sdo  8:224  active ready running
  `- 20:0:2:8 sdu  65:64  active ready running
ora-archive-0001 (3600b3424ad60e33def22d24ead0000d9) dm-4 Sugon ,LU
size=600G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:0 sdai 66:32  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:3:0 sdaw 67:0 active ready running
  `- 20:0:1:0 sdg  8:96 active ready running
ora-vote-0003 (3600b342fdd3354dde1d4dd132d4800d9) dm-2 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:6 sdau 66:224 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:3:6 sdbi 67:192 active ready running
  `- 20:0:1:6 sdm  8:192  active ready running
ora-vote-0002 (3600b34293c07bbdd3d02d2b24dcc00d6) dm-8 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:5 sdbh 67:176 active ready running
| `- 20:0:1:5 sdl  8:176  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 15:0:2:5 sdat 66:208 active ready running
ora-vote-0001 (3600b34242a9fee6de689d7731d5900d9) dm-7 Sugon ,LU
size=10G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:4 sdar 66:176 active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:3:4 sdbf 67:144 active ready running
  `- 20:0:1:4 sdk  8:160  active ready running
ora-data-0004 (3600b3424c37e15cda19ede154d7100d6) dm-10 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:2:7 sdn  8:208  active ready running
| `- 20:0:2:7 sdt  65:48  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:3:7 sdp  8:240  active ready running
  `- 20:0:1:7 sdr  65:16  active ready running
ora-data-0003 (3600b342225bddbed42a3d27c2d5900d6) dm-5 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:3 sdbd 67:112 active ready running
| `- 20:0:1:3 sdj  8:144  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 15:0:2:3 sdao 66:128 active ready running
ora-data-0002 (3600b3428a8d6156d1e7cddfb3dd900d9) dm-6 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| `- 15:0:2:2 sdam 66:96  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  |- 15:0:3:2 sdba 67:64  active ready running
  `- 20:0:1:2 sdi  8:128  active ready running
ora-data-0001 (3600b34287abddc4dad20d9d56d2900d6) dm-3 Sugon ,LU
size=2.0T features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='round-robin 0' prio=50 status=active
| |- 15:0:3:1 sday 67:32  active ready running
| `- 20:0:1:1 sdh  8:112  active ready running
`-+- policy='round-robin 0' prio=10 status=enabled
  `- 15:0:2:1 sdak 66:64  active ready running
–修改配置文件

# vi /etc/multipath.conf

      checker_timeout       120
}
devices{
      device {
            vendor                "Sugon"
            product                "LU"
            path_grouping_policy group_by_prio
            no_path_retry          30
            prio                   alua
            path_checker          tur
            path_selector          "round-robin 0"
            hardware_handler       "0"
            failback             15
      }
}
multipaths {
      multipath {
            wwid 3600b3424ad60e33def22d24ead0000d9
            alias ora-archive-0001
      }
      multipath {
            wwid 3600b342df65ee3adaf40d84ffd9d00d6
            alias ora-archive-0002                                     <<<<新增
      }
      multipath {
            wwid 3600b34287abddc4dad20d9d56d2900d6
            alias ora-data-0001
      }
      multipath {
            wwid 3600b3428a8d6156d1e7cddfb3dd900d9
            alias ora-data-0002
      }
      multipath {
            wwid 3600b342225bddbed42a3d27c2d5900d6
            alias ora-data-0003
      }
      multipath {
            wwid 3600b3424c37e15cda19ede154d7100d6
            alias ora-data-0004                                        <<<<新增
      }
      multipath {
            wwid 3600b34242a9fee6de689d7731d5900d9
            alias ora-vote-0001
      }
      multipath {
            wwid 3600b34293c07bbdd3d02d2b24dcc00d6
            alias ora-vote-0002
      }
      multipath {
            wwid 3600b342fdd3354dde1d4dd132d4800d9
            alias ora-vote-0003
      }
}
###刷新multipath
# multipath -v3

###查看新增磁盘
# multipath -ll | grep -i 100d6
ora-data-0004 (3600b3424c37e15cda19ede154d7100d6) dm-10 Sugon ,LU
# multipath -ll | grep -i d00d6
ora-archive-0002 (3600b342df65ee3adaf40d84ffd9d00d6) dm-11 Sugon ,LU

# lsblk
NAME             MAJ:MIN RM SIZE RO TYPE  MOUNTPOINT
sda                8:0 0 557.9G  0 disk
├─sda1             8:1 0 500M  0 part  /boot
└─sda2             8:2 0 557.4G  0 part
  ├─centos-root 253:0 0 293.4G  0 lvm /
  ├─centos-swap 253:1 0 64G  0 lvm [SWAP]
  └─centos-u01    253:9 0 200G  0 lvm /u01
sdg                8:96 0 600G  0 disk
└─ora-archive-0001 253:4 0 600G  0 mpath
sdh                8:112  0    2T  0 disk
└─ora-data-0001 253:3 0    2T  0 mpath
sdi                8:128  0    2T  0 disk
└─ora-data-0002 253:6 0    2T  0 mpath
sdj                8:144  0    2T  0 disk
└─ora-data-0003 253:5 0    2T  0 mpath
sdk                8:160  0 10G  0 disk
└─ora-vote-0001 253:7 0 10G  0 mpath
sdl                8:176  0 10G  0 disk
└─ora-vote-0002 253:8 0 10G  0 mpath
sdm                8:192  0 10G  0 disk
└─ora-vote-0003 253:2 0 10G  0 mpath
sdn                8:208  0    2T  0 disk
└─ora-data-0004 253:10 0    2T  0 mpath
sdo                8:224  0 600G  0 disk
└─ora-archive-0002 253:11 0 600G  0 mpath
3.4 修改UDEV配置（所有的rac的节点都要修改）
--生成udev配置相关信息
# cd /dev/mapper
for i in ora-*; do
  printf "%s %s\n" "$i" "$(udevadm info --query=all --name=/dev/mapper/"$i" | grep -i dm_uuid)"
done

ora-archive-0001 E: DM_UUID=mpath-3600b3424ad60e33def22d24ead0000d9
ora-archive-0002 E: DM_UUID=mpath-3600b342df65ee3adaf40d84ffd9d00d6 <<<<<新增磁盘
ora-data-0001 E: DM_UUID=mpath-3600b34287abddc4dad20d9d56d2900d6
ora-data-0002 E: DM_UUID=mpath-3600b3428a8d6156d1e7cddfb3dd900d9
ora-data-0003 E: DM_UUID=mpath-3600b342225bddbed42a3d27c2d5900d6
ora-data-0004 E: DM_UUID=mpath-3600b3424c37e15cda19ede154d7100d6    <<<<<新增磁盘
ora-vote-0001 E: DM_UUID=mpath-3600b34242a9fee6de689d7731d5900d9
ora-vote-0002 E: DM_UUID=mpath-3600b34293c07bbdd3d02d2b24dcc00d6
ora-vote-0003 E: DM_UUID=mpath-3600b342fdd3354dde1d4dd132d4800d9

--修改udev配置
# vi /etc/udev/rules.d/99-oracle-asmdevices.rules

KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3424ad60e33def22d24ead0000d9",SYMLINK+="asm_arch0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342df65ee3adaf40d84ffd9d00d6",SYMLINK+="asm_arch0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34287abddc4dad20d9d56d2900d6",SYMLINK+="asm_data0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3428a8d6156d1e7cddfb3dd900d9",SYMLINK+="asm_data0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342225bddbed42a3d27c2d5900d6",SYMLINK+="asm_data0003",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b3424c37e15cda19ede154d7100d6",SYMLINK+="asm_data0004",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34242a9fee6de689d7731d5900d9",SYMLINK+="asm_vote0001",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b34293c07bbdd3d02d2b24dcc00d6",SYMLINK+="asm_vote0002",OWNER="grid",GROUP="asmadmin",MODE="0660"
KERNEL=="dm-*",ENV{DM_UUID}=="mpath-3600b342fdd3354dde1d4dd132d4800d9",SYMLINK+="asm_vote0003",OWNER="grid",GROUP="asmadmin",MODE="0660"

--重载udev
# udevadm control --reload-rules
# udevadm trigger --type=devices

# ls -lrt /dev/asm*
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0003 -> dm-5
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0002 -> dm-8
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0003 -> dm-3
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0002 -> dm-6
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_data0001 -> dm-7
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_arch0001 -> dm-2
lrwxrwxrwx 1 root root 4 May 15 14:57 /dev/asm_vote0001 -> dm-4
lrwxrwxrwx 1 root root 5 May 15 14:57 /dev/asm_arch0002 -> dm-11 <<<<新增磁盘
lrwxrwxrwx 1 root root 5 May 15 14:57 /dev/asm_data0004 -> dm-10 <<<<新增磁盘
四、扩容ASM磁盘组
4.1 当前asm磁盘情况查询
# su - grid
Last login: Wed May 15 14:37:22 CST 2024 on pts/1

$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.4.0 Production on Wed May 15 15:01:24 2024

Copyright (c) 1982, 2013, Oracle.  All rights reserved.

Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Real Application Clusters and Automatic Storage Management options

SQL>
col path for a30;
col name for a20;
col mount_status format a10
col header_status format a10
col mode_status format a10
set linesize 400;
select a.group_number, a.name,b.mount_status,b.header_status,b.mode_status, b.path,b.total_mb/1024 gb,b.name
  from v$asm_diskgroup a, v$asm_disk b
where a.group_number(+) = b.group_number;

GROUP_NUMBER NAME                MOUNT_STAT HEADER_STA MODE_STATU PATH                                  GB NAME
------------ -------------------- ---------- ---------- ---------- ------------------------------ ---------- --------------------
                              CLOSED    CANDIDATE  ONLINE    /dev/asm_data0004                      0
                              CLOSED    CANDIDATE  ONLINE    /dev/asm_arch0002                      0
         1 ORACRS             CACHED    MEMBER    ONLINE    /dev/asm_vote0001                   10 ORACRS_0000
         1 ORACRS             CACHED    MEMBER    ONLINE    /dev/asm_vote0002                   10 ORACRS_0001
         1 ORACRS             CACHED    MEMBER    ONLINE    /dev/asm_vote0003                   10 ORACRS_0002
         2 ORAARCH             CACHED    MEMBER    ONLINE    /dev/asm_arch0001                   600 ORAARCH_0000
         3 ORADATA             CACHED    MEMBER    ONLINE    /dev/asm_data0002                   2048 ORADATA_0001
         3 ORADATA             CACHED    MEMBER    ONLINE    /dev/asm_data0001                   2048 ORADATA_0000
         3 ORADATA             CACHED    MEMBER    ONLINE    /dev/asm_data0003                   2048 ORADATA_0002

9 rows selected.
4.2 扩容磁盘
alter diskgroup ORAARCH add disk '/dev/asm_arch0002' rebalance  power 11;
alter diskgroup ORADATA add disk '/dev/asm_data0004' rebalance  power 11;

select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation;

SQL> alter diskgroup ORAARCH add disk '/dev/asm_arch0002' rebalance  power 11;

Diskgroup altered.

SQL> alter diskgroup ORADATA add disk '/dev/asm_data0004' rebalance  power 11;

Diskgroup altered.

SQL> col path for a30;
col name for a20;
col mount_status format a10
col header_status format a10
col mode_status format a10
set linesize 400;
select a.group_number, a.name,b.mount_status,b.header_status,b.mode_status, b.path,b.total_mb/1024 gb,b.name
  from v$asm_diskgroupSQL>  a, v$asm_disk b
where a.group_number(+) = b.group_number;SQL> SQL> SQL> SQL> SQL> 2 3

GROUP_NUMBER NAME    MOUNT_STAT HEADER_STA MODE_STATU PATH    GB NAME
------------ -------------------- ---------- ---------- ---------- ------------------------------ ---------- --------------------
   1 ORACRS    CACHED    MEMBER ONLINE    /dev/asm_vote0001    10 ORACRS_0000
   1 ORACRS    CACHED    MEMBER ONLINE    /dev/asm_vote0003    10 ORACRS_0002
   1 ORACRS    CACHED    MEMBER ONLINE    /dev/asm_vote0002    10 ORACRS_0001
   2 ORAARCH    CACHED    MEMBER ONLINE    /dev/asm_arch0001    600 ORAARCH_0000
   2 ORAARCH    CACHED    MEMBER ONLINE    /dev/asm_arch0002    600 ORAARCH_0001
   3 ORADATA    CACHED    MEMBER ONLINE    /dev/asm_data0004 2048 ORADATA_0003
   3 ORADATA    CACHED    MEMBER ONLINE    /dev/asm_data0002 2048 ORADATA_0001
   3 ORADATA    CACHED    MEMBER ONLINE    /dev/asm_data0003 2048 ORADATA_0002
   3 ORADATA    CACHED    MEMBER ONLINE    /dev/asm_data0001 2048 ORADATA_0000

9 rows selected.
4.3 查看重平衡状态

SQL> select inst_id, operation, state, power, sofar, est_work, est_rate, est_minutes from gv$asm_operation;

INST_ID OPERATION  STATE       POWER    SOFAR EST_WORK EST_RATE EST_MINUTES
---------- ---------- -------- ---------- ---------- ---------- ---------- -----------
      1 REBAL    RUN             11    128425    128425       0          0
      1 REBAL    WAIT          11
      2 REBAL    WAIT          11
      2 REBAL    RUN             11    3030 1175452    27089       43

欢迎光临重庆思庄Oracle、Redhat认证学习论坛 (http://bbs.cqsztech.com/)