重庆思庄Oracle、Redhat认证学习论坛

 找回密码
 注册

QQ登录

只需一步,快速开始

搜索
查看: 1770|回复: 1
打印 上一主题 下一主题

[转载] CR / Vote disk Maintenance Operations: (ADD/REMOVE/REPLACE/MOVE) (文档 ID 428...

[复制链接]
跳转到指定楼层
楼主
发表于 2016-7-6 13:56:49 | 只看该作者 回帖奖励 |倒序浏览 |阅读模式
Applies to:

Oracle Database - Enterprise Edition -Version 10.2.0.1 to 11.2.0.1.0 [Release 10.2 to 11.2]
Oracle Database - Enterprise Edition - Version 11.2.0.4 to 11.2.0.4 [Release11.2]
Oracle Database - Enterprise Edition - Version 12.1.0.2 to 12.1.0.2 [Release12.1]
Information in this document applies to any platform.
Goal

The goal of this note is to provide steps to add, remove, replace ormove an Oracle Cluster Repository (OCR) and/or Voting Disk in OracleClusterware 10gR2, 11gR1 and 11gR2 environment. It will also provide steps tomove OCR / voting and ASM devices from raw device to block device. For OracleClusterware 12c, please refer to Document 1558920.1 Software Patch Level and 12cGrid Infrastructure OCR Backup/Restore.

This article is intended for DBA and Support Engineers who need to modify, ormove OCR and voting disks files, customers who have an existing clusteredenvironment deployed on a storage array and might want to migrate to a newstorage array with minimal downtime.

Typically, one would simply cp or dd the files once the new storage has beenpresented to the hosts. In this case, it is a little more difficult because:

1. The Oracle Clusterware has the OCR and voting disks open and is activelyusing them. (Both primary and mirrors)
2. There is an API provided for this function (ocrconfig and crsctl), which isthe appropriate interface than typical cp and/or dd commands.

It is highly recommended to take a backup of the voting disk, and OCR devicebefore making any changes.
Note: while the OCR and Voting disk files may bestored together, such as in OCFS (for example in pre-11.2 Clusterwareenvironments) or in the same ASM diskgroup (for example in 11.2 OracleClusterware environments), OCR and Voting disk files are in fact two separatefiles or entities and so if the intention is to modify or move both OCR andVoting disk files, then one must follow steps provided for both of these typesof files.
Solution

Prepare the disks

For OCR or voting disk addition orreplacement, new disks need to be prepared. Please refer to Clusteware/GirdInfrastructure installation guide for different platform for the diskrequirement and preparation.
1. Size
For 10.1:
OCR device minimum size (each): 100M
Voting disk minimum size (each): 20M
For 10.2:
OCR device minimum size (each): 256M
Voting disk minimum size (each): 256M
For 11.1:
OCR device minimum size (each): 280M
Voting disk minimum size (each): 280M
For 11.2:
OCR device minimum size (each): 300M
Voting disk minimum size (each): 300M
2. For rawor block device (pre 11.2)
Please refer to Clusterware installation guide ondifferent platform for more details.
On windows platform the new raw device link is created via$CRS_HOME\bin\GUIOracleOBJManager.exe, for example:
\\.\VOTEDSK2
\\.\OCR2
3. For ASMdisks (11.2+)
On Windows platform, please refer to Document 331796.1 How to setup ASM on Windows
On Linux platform, please refer to Document 580153.1 How To Setup ASM on Linux UsingASMLIB Disks, Raw Devices or Block Devices?
For other platform, please refer to Clusterware/Gird Infrastructureinstallation guide on OTN(Chapter: Oracle Automatic Storage Management Storage Configuration).
4. Forcluster file system
If OCR is on cluster file system, the new OCR orOCRMIRROR file must be touched before add/replace command can be issued.Otherwise PROT-21: Invalid parameter (10.2/11.) or PROT-30 The OracleCluster Registry location to be added is not accessible (for 11.2) will occur.
As root user
# touch /cluster_fs/ocrdisk.dat
# touch /cluster_fs/ocrmirror.dat
# chown root:oinstall /cluster_fs/ocrdisk.dat  /cluster_fs/ocrmirror.dat
# chmod 640 /cluster_fs/ocrdisk.dat  /cluster_fs/ocrmirror.dat
It is not required to pre-touch voting disk file oncluster file system.
After delete command is issued, the ocr/voting files onthe cluster file system require to be removed manually.
5. Permissions
For OCR device:
chown root:oinstall <OCR device>
chmod 640 <OCR device>
For Voting device:
chown <crs/grid>:oinstall <Voting device>
chmod 644 <Voting device>
For ASM disks used for OCR/Voting disk:
chown griduser:asmadmin <asm disks>
chmod 660 <asm disks>
6.Redundancy
For Voting disks (never use even number of voting disks):
External redundancy requires minimum of 1 voting disk (or 1failure group)
Normal redundancy requires minimum of 3 voting disks (or 3failure group)
High redundancy requires minimum of 5 voting disks (or 5failure group)

Insufficient failure group in respect of redundancy requirement could causevoting disk creation failure. For example: ORA-15274: Not enough failgroups (3)to create voting files
For OCR:
10.2 and 11.1, maximum 2 OCR devices: OCR and OCRMIRROR
11.2+, upto 5 OCR devices can be added.
For more information, please refer to platform specific Oracle&reg;Grid Infrastructure Installation Guide.
ADD/REMOVE/REPLACE/MOVE OCR Device
Note: You must be logged in as the root user,because root owns the OCR files. "ocrconfig -replace" command canonly be issued when CRS is running, otherwise "PROT-1: Failed toinitialize ocrconfig" will occur.

Please ensure CRS is running on ALL cluster nodes during this operation,otherwise the change will not reflect in the CRS down node, CRS will haveproblem to startup from this down node. "ocrconfig -repair" optionwill be required to fix the ocr.loc file on the CRS down node.

For 11.2+ with OCR on ASM diskgroup, due to unpublished Bug 8604794 - FAIL TOCHANGE OCR LOCATION TO DG WITH 'OCRCONFIG -REPAIR -REPLACE', "ocrconfig-repair" to change OCR location to different ASM diskgroup does not workcurrently. Workaround is to manually edit /etc/oracle/ocr.loc or/var/opt/oracle/ocr.loc or Windows registryHYKEY_LOCAL_MACHINE\SOFTWARE\Oracle\ocr,  point to desired diskgroup.

If there is any issue with OLR, please refer to How to restore OLR in 11.2 GridInfrastructure Note 1193643.1.
Make sure there is a recent copy of the OCR file beforemaking any changes:
ocrconfig -showbackup
If there is not a recent backup copy of the OCR file, anexport can be taken for the current OCR file. Use the following command togenerate an export of the online OCR file:

In 10.2
# ocrconfig -export <OCRexport_filename> -s online
In 11.1 and 11.2
# ocrconfig -manualbackup
node1 2008/08/06 06:11:58 /crs/cdata/crs/backup_20080807_003158.ocr
To recover using this file, the following command can beused:
# ocrconfig -import <OCRexport_filename>

From 11.2+, please also refer How to restore ASM based OCR after complete lossof the CRS diskgroup on Linux/Unix systems Document 1062983.1

To see whether OCR is healthy, run an ocrcheck, which should return with likebelow.
# ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 2
Total space (kbytes) : 497928
Used space (kbytes) : 312
Available space (kbytes) : 497616
ID : 576761409
Device/File Name : /dev/raw/raw1
Device/File integrity check succeeded
Device/File Name : /dev/raw/raw2
Device/File integrity check succeeded

Cluster registry integrity check succeeded

For 11.1+, ocrcheck as root user should also show:
Logical corruption check succeeded
1. To addan OCRMIRROR device when only OCR device is defined:
To add an OCR mirror device, provide the full pathincluding file name.
10.2 and 11.1:
# ocrconfig -replace ocrmirror<filename>
eg:
# ocrconfig -replace ocrmirror /dev/raw/raw2
# ocrconfig -replace ocrmirror /dev/sdc1
# ocrconfig -replace ocrmirror /cluster_fs/ocrdisk.dat
> ocrconfig -replace ocrmirror \\.\OCRMIRROR2  - for Windows
11.2+: From11.2 onwards, upto 4 ocrmirrors can be added
# ocrconfig -add <filename>
eg:
# ocrconfig -add +OCRVOTE2
# ocrconfig -add /cluster_fs/ocrdisk.dat
2. Toremove an OCR device
To remove an OCR device:
10.2 and 11.1:
# ocrconfig -replace ocr
11.2+:
# ocrconfig -delete <filename>
eg:
# ocrconfig -delete +OCRVOTE1
* Once an OCR deviceis removed, ocrmirror device automatically changes to be OCR device.
* It is not allowed to remove OCR device if only 1 OCR device is defined, thecommand will return PROT-16.

To remove an OCR mirror device:
10.2 and 11.1:
# ocrconfig -replace ocrmirror
11.2+:
# ocrconfig -delete <ocrmirrorfilename>
eg:
# ocrconfig -delete +OCRVOTE2
After removal, the old OCR/OCRMIRROR can be deleted ifthey are on cluster filesystem.
3. Toreplace or move the location of an OCR device
Note. 1. An ocrmirror must be in place beforetrying to replace the OCR device. The ocrconfig will fail with PROT-16, ifthere is no ocrmirror exists.
2. If an OCR device is replaced with a device of a different size, the size ofthe new device will not be reflected until the clusterware is restarted.

10.2 and 11.1:
To replace the OCR device with <filename>, provide the full pathincluding file name.
# ocrconfig -replace ocr <filename>
eg:
# ocrconfig -replace ocr /dev/sdd1
$ ocrconfig -replace ocr \\.\OCR2 - for Windows
To replace the OCR mirror device with <filename>,provide the full path including file name.
# ocrconfig -replace ocrmirror<filename>
eg:
# ocrconfig -replace ocrmirror /dev/raw/raw4
# ocrconfig -replace ocrmirror \\.\OCRMIRROR2  - for Windows
11.2:
The command is same for replace either OCR or OCRMIRRORs (at least 2 OCR existfor replace command to work):
# ocrconfig -replace <current filename>-replacement <new filename>
eg:
# ocrconfig -replace /cluster_file/ocr.dat -replacement +OCRVOTE
# ocrconfig -replace +CRS -replacement +OCRVOTE
4. Torestore an OCR when clusterware is down
When OCR is not accessible, CRSD process will not start,hence the clusterware stack will not start completely. A restore of OCR deviceaccess and good OCR content is required.
To view the automatic OCR backup:
# ocrconfig -showbackup
To restore the OCR backup:
# ocrconfig -restore <path/filename of OCRbackup>
For 11.2: If OCR is located in ASM disk and ASM disk is also lost, please checkout:
How to restore ASM based OCR after complete loss of the CRS diskgroup onLinux/Unix systems Document 1062983.1
How to Restore OCR After the 1st ASM Diskgroup is Lost on Windows Document 1294915.1

If there is no valid backup of OCR presented, reinitialize OCR and Voting isrequired.
For 10.2 and 11.1:
Please refer to How to Recreate OCR/Voting Disk Accidentally Deleted Document 399482.1

For 11.2:
Deconfig the clusterware stack and rerun root.sh on all nodes is required.

ADD/DELETE/MOVE Voting Disk
Note: 1. crsctl votedisk commands must be run asroot for 10.2 and 11.1, but can be run as grid user for 11.2+
2. For 11.2, when using ASM disks for OCR and voting, the command is same forWindows and Unix platform.
For pre 11.2, to take a backup of voting disk:
$ dd if=voting_disk_name of=backup_file_name
For Windows:
ocopy \\.\votedsk1 o:\backup\votedsk1.bak
For 11.2+, it is no longer required to back up the votingdisk. The voting disk data is automatically backed up in OCR as part of anyconfiguration change. The voting disk files are backed up automatically byOracle Clusterware if the contents of the files have changed in the followingways:

  • Configuration parameters, for example misscount, have been added or modified
  • After performing voting disk add or delete operations
The voting disk contents are restored from a backupautomatically when a new voting disk is added or replaced.
For 10gR2release
Shutdown the Oracle Clusterware (crsctl stop crs as root)on all nodes before making any modification to the voting disk. Determine thecurrent voting disk location using:
crsctl query css votedisk

1. To add a Voting Disk, provide the full path including file name:
# crsctl add css votedisk<VOTEDISK_LOCATION> -force
eg:
# crsctl add css votedisk /dev/raw/raw1 -force
# crsctl add css votedisk /cluster_fs/votedisk.dat -force
> crsctl add css votedisk \\.\VOTEDSK2 -force   - for windows
2. To delete a Voting Disk, provide the full pathincluding file name:
# crsctl delete css votedisk<VOTEDISK_LOCATION> -force
eg:
# crsctl delete css votedisk /dev/raw/raw1 -force
# crsctl delete css votedisk /cluster_fs/votedisk.dat -force
> crsctl delete css votedisk \\.\VOTEDSK1 -force   - for windows
3. To move a Voting Disk, provide the full path includingfile name, add a device first before deleting the old one:
# crsctl add css votedisk<NEW_LOCATION> -force
# crsctl delete css votedisk <OLD_LOCATION> -force
eg:
# crsctl add css votedisk /dev/raw/raw4 -force
# crsctl delete css votedisk /dev/raw/raw1 -force
After modifying the voting disk, start the OracleClusterware stack on all nodes
# crsctl start crs
Verify the voting disk location using
# crsctl query css votedisk
For 11gR1release
Starting with 11.1.0.6, the below commands can beperformed online (CRS is up and running).

1. To add a Voting Disk, provide the full path including file name:
# crsctl add css votedisk<VOTEDISK_LOCATION>
eg:
# crsctl add css votedisk /dev/raw/raw1
# crsctl add css votedisk /cluster_fs/votedisk.dat
> crsctl add css votedisk\\.\VOTEDSK2        - for windows
2. To delete a Voting Disk, provide the full pathincluding file name:
# crsctl delete css votedisk <VOTEDISK_LOCATION>
eg:
# crsctl delete css votedisk /dev/raw/raw1 -force
# crsctl delete css votedisk /cluster_fs/votedisk.dat
> crsctl delete css votedisk \\.\VOTEDSK1     - forwindows
3. To move a Voting Disk, provide the full path includingfile name:
# crsctl add css votedisk<NEW_LOCATION>
# crsctl delete css votedisk <OLD_LOCATION>
eg:
# crsctl add css votedisk /dev/raw/raw4
# crsctl delete css votedisk /dev/raw/raw1
Verify the voting disk location:
# crsctl query css votedisk
For 11gR2release
From 11.2, votedisk can be stored on either ASM diskgroupor cluster file systems. The following commands can only be executed when GridInfrastructure is running. As grid user:

1. To add a Voting Disk
a. When votedisk is on cluster file system:
$ crsctl add css votedisk<cluster_fs/filename>
b. When votedisk is on ASM diskgroup, no add optionavailable.
The number of votedisk is determined by the diskgroup redundancy. If morecopies of votedisks are desired, one can move votedisk to a diskgroup withhigher redundancy. See step 4.
If a votedisk is removed from a normal or high redundancy diskgroup forabnormal reason, it can be added back using:
alter diskgroup <vote diskgroup name>add disk '</path/name>' force;

2. To delete a Voting Disk
a. When votedisk is on cluster file system:
$ crsctl delete css votedisk<cluster_fs/filename>
or
$ crsctl delete css votedisk <vdiskGUID>    (vdiskGUID is the File Universal Id from 'crsctl query css votedisk')
b. When votedisk is on ASM, no delete option available,one can only replace the existing votedisk group with another ASM diskgroup

3. To move a Voting Disk on cluster file system
$ crsctl add css votedisk<new_cluster_fs/filename>
$ crsctl delete css votedisk <old_cluster_fs/filename>
or
$ crsctl delete css votedisk <vdiskGUID>
4. To move voting disk on ASM from one diskgroup toanother diskgroup due to redundancy change or disk location change
$ crsctl replace votedisk<+diskgroup>|<vdisk>
Example here is moving from external redundancy +OCRVOTEdiskgroup to normal redundancy +CRS diskgroup
1. create new diskgroup +CRS as desired

2. $ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----   -----------------               --------- ---------
1. ONLINE   5e391d339a594fc7bf11f726f9375095 (ORCL:ASMDG02)[+OCRVOTE]
Located 1 voting disk(s).

3. $ crsctl replace votedisk +CRS
Successful addition of voting disk 941236c324454fc0bfe182bd6ebbcbff.
Successful addition of voting disk 07d2464674ac4fabbf27f3132d8448b0.
Successful addition of voting disk 9761ccf221524f66bff0766ad5721239.
Successful deletion of voting disk 5e391d339a594fc7bf11f726f9375095.
Successfully replaced voting disk group with +CRS.
CRS-4266: Voting file(s) successfully replaced

4. $ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----   -----------------               --------- ---------
1. ONLINE   941236c324454fc0bfe182bd6ebbcbff (ORCL:CRSD1) [CRS]
2. ONLINE   07d2464674ac4fabbf27f3132d8448b0 (ORCL:CRSD2) [CRS]
3. ONLINE   9761ccf221524f66bff0766ad5721239 (ORCL:CRSD3) [CRS]
Located 3 voting disk(s).
5. To move voting disk between ASM diskgroup and clusterfile system
a. Move from ASM diskgroup to cluster file system:
$ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----   -----------------               --------- ---------
1. ONLINE   6e5850d12c7a4f62bf6e693084460fd9 (ORCL:CRSD1) [CRS]
2. ONLINE   56ab5c385ce34f37bf59580232ea815f (ORCL:CRSD2) [CRS]
3. ONLINE   4f4446a59eeb4f75bfdfc4be2e3d5f90 (ORCL:CRSD3) [CRS]
Located 3 voting disk(s).

$ crsctl replace votedisk /rac_shared/oradata/vote.test3
Now formatting voting disk: /rac_shared/oradata/vote.test3.
CRS-4256: Updating the profile
Successful addition of voting disk 61c4347805b64fd5bf98bf32ca046d6c.
Successful deletion of voting disk 6e5850d12c7a4f62bf6e693084460fd9.
Successful deletion of voting disk 56ab5c385ce34f37bf59580232ea815f.
Successful deletion of voting disk 4f4446a59eeb4f75bfdfc4be2e3d5f90.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced

$ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----   -----------------               --------- ---------
1. ONLINE   61c4347805b64fd5bf98bf32ca046d6c(/rac_shared/oradata/vote.disk) []
Located 1 voting disk(s).
b. Move from cluster file system to ASM diskgroup
$ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----    -----------------               --------- ---------
1. ONLINE   61c4347805b64fd5bf98bf32ca046d6c(/rac_shared/oradata/vote.disk) []
Located 1 voting disk(s).

$ crsctl replace votedisk +CRS
CRS-4256: Updating the profile
Successful addition of voting disk 41806377ff804fc1bf1d3f0ec9751ceb.
Successful addition of voting disk 94896394e50d4f8abf753752baaa5d27.
Successful addition of voting disk 8e933621e2264f06bfbb2d23559ba635.
Successful deletion of voting disk 61c4347805b64fd5bf98bf32ca046d6c.
Successfully replaced voting disk group with +CRS.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced

[oragrid@auw2k4 crsconfig]$ crsctl query css votedisk
##  STATE    File UniversalId               File Name Disk group
--  -----    -----------------               --------- ---------
1. ONLINE   41806377ff804fc1bf1d3f0ec9751ceb (ORCL:CRSD1) [CRS]
2. ONLINE   94896394e50d4f8abf753752baaa5d27 (ORCL:CRSD2) [CRS]
3. ONLINE   8e933621e2264f06bfbb2d23559ba635 (ORCL:CRSD3) [CRS]
Located 3 voting disk(s).
6. To verify:
$ crsctl query css votedisk
For Votingdisk maintenance in Extended Cluster
If there is any issue using asmca tool, please refer toHow to Manually Add NFS voting disk to an Extended Cluster using ASM in 11.2 Note 1421588.1 for detailed commands.
CommunityDiscussions
Still have questions? Use the communities window below tosearch for similar discussions or start a new discussion on this subject.(Window is the live community not a screenshot)
Click here to open in main browser window
References

NOTE:1558920.1 - Software Patch Level and 12cGrid Infrastructure OCR Backup/Restore
NOTE:1062983.1 - How to restore ASM based OCRafter complete loss of the CRS diskgroup on Linux/Unix systems
NOTE:866102.1 - Renaming OCR Using"ocrconfig -overwrite" Fails
NOTE:390880.1 - OCR Corruption afterAdding/Removing voting disk to a cluster when CRS stack is running
NOTE:1573574.1 - OCR Placement and SizeSpecification
分享到:  QQ好友和群QQ好友和群 QQ空间QQ空间 腾讯微博腾讯微博 腾讯朋友腾讯朋友
收藏收藏 支持支持 反对反对
回复

使用道具 举报

沙发
 楼主| 发表于 2016-7-6 13:59:50 | 只看该作者
1. To add a Voting Disk

If a votedisk is removed from a normal or high redundancy diskgroup forabnormal reason, it can be added back using:
alter diskgroup <vote diskgroup name>add disk '</path/name>' force;

这次就遇到这个问题了,用户的votdisk冗余为normal,但发现只有两个盘在用,另一个盘没有加上去,听说是上次出现问题后,没有处理,我直接使用这个把这个盘加上去即可.
alter disgroup dgcrs add disk 'orcl:ocr_vote01' force;


这个很
回复 支持 反对

使用道具 举报

您需要登录后才可以回帖 登录 | 注册

本版积分规则

QQ|手机版|小黑屋|重庆思庄Oracle、Redhat认证学习论坛 ( 渝ICP备12004239号-4 )

GMT+8, 2024-4-29 00:51 , Processed in 0.097361 second(s), 19 queries .

重庆思庄学习中心论坛-重庆思庄科技有限公司论坛

© 2001-2020

快速回复 返回顶部 返回列表