[转帖]在11gR2 RAC中修改ASM DISK Path磁盘路径

郑全 · 发表于 2012-8-31 13:02:38

在11gR2 RAC中修改ASM DISK Path磁盘路径

具体问题如下：

aix 6.1，grid 11.2.0.3+asm11.2.0.3+rac

建数据库的时候使用的是aix自带的多路径软件mpio，建了diskgroup

现在改造成veritas dmp多路径，已经修改了asm的disk_strings＝/dev/vx/rdmp/*，crs/asm启动的时候已经可以识别到磁盘/dev/vx/rdmp/开头的磁盘，但是读取不回原来的diskgroup信息。

crs启动的报错日志：
2012-07-13 15:07:29.748: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2108 clsgpnp_profileCallUrlInt] get-profile call to url “ipc://GPNPD_ggtest1″ disco “” [f=0 claimed- host: cname: seq: auth:]
2012-07-13 15:07:29.762: [ GPNP][1286]clsgpnp_profileCallUrlInt: [at clsgpnp.c:2236 clsgpnp_profileCallUrlInt] Result: (0) CLSGPNP_OK. Successful get-profile CALL to remote “ipc://GPNPD_ggtest1″ disco “”
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmReadDiscoveryProfile: voting file discovery string(/dev/vx/rdmp/*)
2012-07-13 15:07:29.762: [ CSSD][1286]clssnmvDDiscThread: using discovery string /dev/vx/rdmp/* for initial discovery
2012-07-13 15:07:29.762: [ SKGFD][1286]Discovery with str:/dev/vx/rdmp/*:

2012-07-13 15:07:29.762: [ SKGFD][1286]UFS discovery with :/dev/vx/rdmp/*:

2012-07-13 15:07:29.769: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_919:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_212:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_211:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_210:

2012-07-13 15:07:29.770: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_209:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_181:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/v_df8000_180:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_3:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_2:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_1:

2012-07-13 15:07:29.771: [ SKGFD][1286]Fetching UFS disk :/dev/vx/rdmp/disk_0:

2012-07-13 15:07:29.771: [ SKGFD][1286]OSS discovery with :/dev/vx/rdmp/*:

2012-07-13 15:07:29.771: [ SKGFD][1286]Handle 1115e7510 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_916:

2012-07-13 15:07:29.772: [ SKGFD][1286]Handle 1118758b0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_912:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118d9cf0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_908:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118da450 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_904:

2012-07-13 15:07:29.773: [ SKGFD][1286]Handle 1118dad70 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_903:

2012-07-13 15:07:29.802: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_916:
2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1115e7510 for disk :/dev/vx/rdmp/v_df8000_916:

2012-07-13 15:07:29.803: [ SKGFD][1286]Lib :UFS:: closing handle 1118758b0 for disk :/dev/vx/rdmp/v_df8000_912:

2012-07-13 15:07:29.804: [ SKGFD][1286]Handle 1115e6710 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_202:

2012-07-13 15:07:29.808: [ SKGFD][1286]Handle 1115e7030 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_201:

2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1115e7ad0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_200:

2012-07-13 15:07:29.809: [ SKGFD][1286]Handle 1118733f0 from lib :UFS:: for disk :/dev/vx/rdmp/v_df8000_199:

2012-07-13 15:07:29.816: [ CLSF][1286]checksum failed for disk:/dev/vx/rdmp/v_df8000_186:
2012-07-13 15:07:29.816: [ SKGFD][1286]Lib :UFS:: closing handle 1118de5d0 for disk :/dev/vx/rdmp/v_df8000_186:

2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvDiskVerify: Successful discovery of 0 disks
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery
2012-07-13 15:07:29.816: [ CSSD][1286]clssnmvFindInitialConfigs: No voting files found
2012-07-13 15:07:29.816: [ CSSD][1286](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found. Retrying discovery in 15 seconds
2012-07-13 15:07:30.169: [ CSSD][1029]clssgmExecuteClientRequest(): type(37) size(80) only connect and exit messages are allowed before lease acquisition proc(1115e4870) client(0)

其需求为修改ASM DISK PATH的磁盘设备路径，但是由于11gR2 RAC+ASM的特殊性，导致CRS无法正常启动，虽然使用crsctl start crs -excl -nocrs的方式可以启动CSS服务和ASM实例，

但是最后还是报(clssnmCompleteInitVFDiscovery: Voting file not found)，这是由于Voteing file投票磁盘的位置没有合理更新导致的。

这里我们来学习一下，如何正确的修改11gR2 RAC+ASM下的ASM DISK路径：

1.我们来营造一个修改ASM DISK路径的环境，这里我们使用UDEV的设备名绑定服务，并利用UDEV将原本的ASM DISK从/dev/asm-disk* 修改为 /dev/rasm-disk*的形式，这只需要修改udev rule文件即可实现：

[grid@maclean1 ~]$ export ORACLE_HOME=/g01/grid/app/11.2.0/grid

[grid@maclean1 ~]$ /g01/grid/app/11.2.0/grid/bin/sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:09:28 2012

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> show parameter diskstri

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

asm_diskstring string /dev/asm*

可以看到当前ASM实例使用的asm_diskstring 为/dev/asm*，切换到root用户修改UDEV RULE文件：

[root@maclean1 rules.d]# cp 99-oracle-asmdevices.rules 99-oracle-asmdevices.rules.bak

[root@maclean1 rules.d]# vi 99-oracle-asmdevices.rules

[root@maclean1 rules.d]# cat 99-oracle-asmdevices.rules

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB09cadb31-cfbea255_", NAME="rasm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB5f097069-59efb82f_", NAME="rasm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB4e1a81c0-20478bc4_", NAME="rasm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBdcce9285-b13c5a27_", NAME="rasm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB82effe1a-dbca7dff_", NAME="rasm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB950d279f-c581cb51_", NAME="rasm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB14400d81-651672d7_", NAME="rasm-diskh", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB31b1237b-78aa22bb_", NAME="rasm-diski", OWNER="grid", GROUP="asmadmin", MODE="0660"

以上修改了原始99-oracle-asmdevices.rules的UDEV RULE规则文件，生成的设备名被修改为/dev/rasm-disk*的形式，不同于之前的ASM DISK设备名，这要求我们后续的一系列操作来保证RAC CRS可以正常启动。

当前运行时的votedisk和ocr 存放位置:

[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/crsctl query css votedisk

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 6896bfc3d1464f9fbf0ea9df87e023ad (/dev/asm-diskb) [SYSTEMDG]

2. ONLINE 58eb81b656084ff2bfd315d9badd08b7 (/dev/asm-diskc) [SYSTEMDG]

3. ONLINE 6bf7324625c54f3abf2c942b1e7f70d9 (/dev/asm-diskd) [SYSTEMDG]

4. ONLINE 43ad8ae20c354f5ebf7083bc30bf94cc (/dev/asm-diske) [SYSTEMDG]

5. ONLINE 4c225359d51b4f93bfba01080664b3d7 (/dev/asm-diskf) [SYSTEMDG]

Located 5 voting disk(s).

[root@maclean1 rules.d]# /g01/grid/app/11.2.0/grid/bin/ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 2844

Available space (kbytes) : 259276

ID : 879001605

Device/File Name : +SYSTEMDG

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

因为votedisk file的位置具体指向一个ASM DISK，所以后续我们会需要crsctl replace votedisk，我们现在重启LINUX OS:

[root@maclean1 rules.d]# init 6

rebooting ............

[root@maclean1 dev]# ls -l *asm*

brw-rw---- 1 grid asmadmin 8, 16 Jul 15 04:15 rasm-diskb

brw-rw---- 1 grid asmadmin 8, 32 Jul 15 04:15 rasm-diskc

brw-rw---- 1 grid asmadmin 8, 48 Jul 15 04:15 rasm-diskd

brw-rw---- 1 grid asmadmin 8, 64 Jul 15 04:15 rasm-diske

brw-rw---- 1 grid asmadmin 8, 80 Jul 15 04:15 rasm-diskf

brw-rw---- 1 grid asmadmin 8, 96 Jul 15 04:15 rasm-diskg

brw-rw---- 1 grid asmadmin 8, 112 Jul 15 04:15 rasm-diskh

brw-rw---- 1 grid asmadmin 8, 128 Jul 15 04:15 rasm-diski

重启后自动获得了形如/dev/rasm-disk*的ASM DISK,查阅ASM日志可以发现css服务仍搜索/dev/asm*路径来获得ASM DISK，但是这将导致找不到任何有效的ASM DISK:

more /g01/grid/app/11.2.0/grid/log/maclean1/cssd/ocssd.log

2012-07-15 04:17:45.208: [ SKGFD][1099548992]Discovery with str:/dev/asm*:

2012-07-15 04:17:45.208: [ SKGFD][1099548992]UFS discovery with :/dev/asm*:

2012-07-15 04:17:45.208: [ SKGFD][1099548992]OSS discovery with :/dev/asm*:

2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmvDiskVerify: Successful discovery of 0 disks

2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmCompleteInitVFDiscovery: Completing initial voting file discovery

2012-07-15 04:17:45.208: [ CSSD][1099548992]clssnmvFindInitialConfigs: No voting files found

2012-07-15 04:17:45.208: [ CSSD][1099548992](:CSSNM00070:)clssnmCompleteInitVFDiscovery: Voting file not found.

Retrying discovery in 15 seconds

2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmExecuteClientRequest(): type(37) size(80) only connect and

exit messages are allowed before lease acquisition proc(0x26a8ba0) client((nil))

2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmDeadProc: proc 0x26a8ba0

2012-07-15 04:17:45.251: [ CSSD][1096661312]clssgmDestroyProc: cleaning up proc(0x26a8ba0) con(0xfe6) skgpid

ospid 3751 with 0 clients, refcount 0

2012-07-15 04:17:45.252: [ CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0xfe6

2012-07-15 04:17:45.829: [ CSSD][1096661312]clssscSelect: cookie accept request 0x2318ea0

2012-07-15 04:17:45.829: [ CSSD][1096661312]clssgmAllocProc: (0x2659480) allocated

2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: properties of cmProc 0x2659480 - 1,2,3,4,5

2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: Connect from con(0x114e) proc(0x2659480) pid(3751)

version 11:2:1:4, properties: 1,2,3,4,5

2012-07-15 04:17:45.830: [ CSSD][1096661312]clssgmClientConnectMsg: msg flags 0x0000

2012-07-15 04:17:45.939: [ CSSD][1096661312]clssscSelect: cookie accept request 0x253ddd0

2012-07-15 04:17:45.939: [ CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x253ddd0

2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmRegisterClient: proc(3/0x253ddd0), client(61/0x26877b0)

2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmExecuteClientRequest(): type(6) size(684) only connect and exit messages are

allowed before lease acquisition proc(0x253ddd0) client(0x26877b0)

2012-07-15 04:17:45.939: [ CSSD][1096661312]clssgmDiscEndpcl: gipcDestroy 0x1174

2012-07-15 04:17:46.070: [ CSSD][1096661312]clssscSelect: cookie accept request 0x26368a0

2012-07-15 04:17:46.070: [ CSSD][1096661312]clssscevtypSHRCON: getting client with cmproc 0x26368a0

2012-07-15 04:17:46.070: [ CSSD][1096661312]clssgmRegisterClient: proc(5/0x26368a0), client(50/0x26877b0)

由于11gR2中CRS服务依赖于ASM，因为ocr存放在ASM中，所以ASM若无法有效启动，这导致CRS服务也无法正常工作:

[root@maclean1 ~]# crsctl check has

CRS-4638: Oracle High Availability Services is online

[root@maclean1 ~]# crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4535: Cannot communicate with Cluster Ready Services

CRS-4530: Communications failure contacting Cluster Synchronization Services daemon

CRS-4534: Cannot communicate with Event Manager

2. 以上是修改ASM DISK PATH磁盘路径的现状，我们需要通过以下操作来恢复CRS:

首先彻底关闭OHASD服务：

[root@maclean1 ~]# crsctl stop has -f

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'

CRS-2673: Attempting to stop 'ora.crf' on 'maclean1'

CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded

CRS-2677: Stop of 'ora.crf' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'

CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'

CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

3. 以-excl -nocrs方式启动CRS，这将仅启动ASM 实例而不会启动CRS服务：

[root@maclean1 ~]# crsctl start crs -excl -nocrs

CRS-4123: Oracle High Availability Services has been started.

CRS-2672: Attempting to start 'ora.mdnsd' on 'maclean1'

CRS-2676: Start of 'ora.mdnsd' on 'maclean1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'maclean1'

CRS-2676: Start of 'ora.gpnpd' on 'maclean1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'maclean1'

CRS-2672: Attempting to start 'ora.gipcd' on 'maclean1'

CRS-2676: Start of 'ora.cssdmonitor' on 'maclean1' succeeded

CRS-2676: Start of 'ora.gipcd' on 'maclean1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'maclean1'

CRS-2672: Attempting to start 'ora.diskmon' on 'maclean1'

CRS-2676: Start of 'ora.diskmon' on 'maclean1' succeeded

CRS-2676: Start of 'ora.cssd' on 'maclean1' succeeded

CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'maclean1'

CRS-2672: Attempting to start 'ora.ctssd' on 'maclean1'

CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded

CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'maclean1'

CRS-2676: Start of 'ora.ctssd' on 'maclean1' succeeded

CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'maclean1'

CRS-2676: Start of 'ora.asm' on 'maclean1' succeeded

#建议同时修改CRS_HOME所在的ORACLE_BASE为777权限，避免可能的问题

[root@maclean1 ~]# chmod 777 /g01

4.修改ASM实例的disk_strings为当前的ASM DISK PATH信息:

[root@maclean1 ~]# su - grid

[grid@maclean1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 11.2.0.3.0 Production on Sun Jul 15 04:40:40 2012

Connected to:

Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options

SQL> alter system set asm_diskstring='/dev/rasm*';

System altered.

SQL> alter diskgroup systemdg mount;

Diskgroup altered.

SQL> create spfile from memory;

File created.

SQL> startup force mount;

ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance

ASM instance started

Total System Global Area 283930624 bytes

Fixed Size 2227664 bytes

Variable Size 256537136 bytes

ASM Cache 25165824 bytes

ASM diskgroups mounted

SQL> show parameter spfile

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

spfile string /g01/grid/app/11.2.0/grid/dbs/

spfile+ASM1.ora

SQL> show parameter disk

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

asm_diskgroups string SYSTEMDG

asm_diskstring string /dev/rasm*

SQL> create pfile from spfile;

File created.

SQL> create spfile='+SYSTEMDG' from pfile;

File created.

SQL> startup force;

ORA-32004: obsolete or deprecated parameter(s) specified for ASM instance

ASM instance started

Total System Global Area 283930624 bytes

Fixed Size 2227664 bytes

Variable Size 256537136 bytes

ASM Cache 25165824 bytes

ASM diskgroups mounted

SQL> show parameter spfile

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

spfile string +SYSTEMDG/maclean-cluster/asmp

arameterfile/registry.253.7886

82933

以上成功修改了asm_diskstring ，且更新了ASM DISKGROUP上的SPFILE ，由于ASM使用共享的SPFILE所以其他节点上一般无需在做其他操作。

5. crsctl replace votedisk 命令将votedisk重置位置：

[root@maclean1 ~]# crsctl replace votedisk +systemdg

Successful addition of voting disk 864a00efcfbe4f42bfd0f4f6b60472a0.

Successful addition of voting disk ab14d6e727614f29bf53b9870052a5c8.

Successful addition of voting disk 754c03c168854f46bf2daee7287bf260.

Successful addition of voting disk 9ed58f37f3e84f28bfcd9b101f2af9f3.

Successful addition of voting disk 4ce7b7c682364f12bf4df5ce1fb7814e.

Successfully replaced voting disk group with +systemdg.

CRS-4266: Voting file(s) successfully replaced

[root@maclean1 ~]# crsctl query css votedisk

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 864a00efcfbe4f42bfd0f4f6b60472a0 (/dev/rasm-diskb) [SYSTEMDG]

2. ONLINE ab14d6e727614f29bf53b9870052a5c8 (/dev/rasm-diskc) [SYSTEMDG]

3. ONLINE 754c03c168854f46bf2daee7287bf260 (/dev/rasm-diskd) [SYSTEMDG]

4. ONLINE 9ed58f37f3e84f28bfcd9b101f2af9f3 (/dev/rasm-diske) [SYSTEMDG]

5. ONLINE 4ce7b7c682364f12bf4df5ce1fb7814e (/dev/rasm-diskf) [SYSTEMDG]

Located 5 voting disk(s).

[root@maclean1 ~]# ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 2844

Available space (kbytes) : 259276

ID : 879001605

Device/File Name : +SYSTEMDG

Device/File integrity check succeeded

Device/File not configured

Cluster registry integrity check succeeded

Logical corruption check succeeded

以上replace了votedisk到新的 ASM DISK上，并确认votedisk和OCR均为可用状态。

6.重启CRS服务：

[root@maclean1 ~]# crsctl stop crs

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'maclean1'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'maclean1'

CRS-2673: Attempting to stop 'ora.ctssd' on 'maclean1'

CRS-2673: Attempting to stop 'ora.asm' on 'maclean1'

CRS-2677: Stop of 'ora.mdnsd' on 'maclean1' succeeded

CRS-2677: Stop of 'ora.asm' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'maclean1'

CRS-2677: Stop of 'ora.ctssd' on 'maclean1' succeeded

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'maclean1'

CRS-2677: Stop of 'ora.cssd' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'maclean1'

CRS-2677: Stop of 'ora.gipcd' on 'maclean1' succeeded

CRS-2673: Attempting to stop 'ora.gpnpd' on 'maclean1'

CRS-2677: Stop of 'ora.gpnpd' on 'maclean1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'maclean1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

[root@maclean1 ~]# crsctl stat res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.BACKUPDG.dg

ONLINE ONLINE maclean1

ora.DATA.dg

ONLINE ONLINE maclean1

ora.LISTENER.lsnr

ONLINE ONLINE maclean1

ora.SYSTEMDG.dg

ONLINE ONLINE maclean1

ora.asm

ONLINE ONLINE maclean1 Started

ora.gsd

OFFLINE OFFLINE maclean1

ora.net1.network

ONLINE ONLINE maclean1

ora.ons

ONLINE ONLINE maclean1

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE maclean1

ora.cvu

1 ONLINE ONLINE maclean1

ora.maclean1.vip

1 ONLINE ONLINE maclean1

ora.maclean2.vip

1 ONLINE INTERMEDIATE maclean1 FAILED OVER

ora.oc4j

1 ONLINE OFFLINE STARTING

ora.prod.db

1 ONLINE OFFLINE Instance Shutdown,S

TARTING

2 ONLINE OFFLINE

ora.scan1.vip

1 ONLINE ONLINE maclean1

因为上面更新了ASM共享使用的SPFILE，所以其他节点上一般不会存在问题，直接重启后CRS即可正常工作。

帐号		自动登录	找回密码
密码			注册