标签：CRS RAC o11gr21 故障模拟 grid Oracle OCR root asm
测试目的： 模拟整个CRS盘损坏后，如何处理
处理过程： 重新创建一个同名的磁盘组给OCR使用。restore OCR信息，重新创建voting file。 即可。
RDBMS 11.2.0.4
参考文档： Linux/Unix 平台，在CRS 磁盘组完全丢失后，如何恢复基于 ASM 的 OCR (Doc ID 2331776.1)
步骤
1 查看当前集群状态、OCRCHECK 、VOTEDISK
2 查看当前的OCR的备份 ，如有必要，模拟前手工备份一次
3 使用DD命令进行模拟破坏
4 确认所有节点上的GI已经关闭
5 以排他模式启动CRS（在拥有最近OCR备份的节点上，仅在该节点上排他启动CRS）
6 创建ASM磁盘给CRS盘使用（注意，要和原来的CRS盘的名字一样，冗余模式可以不一样）
7 使用最近的OCR备份进行restore
8 当前节点启动CRS daemon （仅仅适用于11.2.0.1.当前版本不适用，略）
9 重建ASM的spfile，否则重建voting file，会找不到asm盘 （主要原因是asm string参数，可以先设置这个参数，重建了voting file后，再次重新创建asm pfile）。
10 重建voting file
11 关闭CRS
12 在各个节点上启动CRS
13 检验CRS、OCRCHCK、VOTEDISK
++++++++++以下为详细模拟过程
1 检查集群状态
[root@o11gr21 ~]# su - grid
[grid@o11gr21 ~]$ crsctl check cluster -all
**************************************************************
o11gr21:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr22:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr23:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[grid@o11gr21 ~]$ -- 查看当前的ocr情况和votedisk
[grid@o11gr21 o11gr2-cluster]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3324
Available space (kbytes) : 258796
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
[grid@o11gr21 o11gr2-cluster]$ crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 2cb64f80fd974f61bf9ce6fa0f0bfbde (/dev/asm-ocr1) [OCR]
2. ONLINE 8a44f55e4f8f4f8dbf3dbe890877a857 (/dev/asm-ocr2) [OCR]
3. ONLINE 81ffe4523d5b4f24bf14083103925201 (/dev/asm-ocr3) [OCR]
Located 3 voting disk(s).
[grid@o11gr21 o11gr2-cluster]$ 2 查看集群OCR的备份情况
[grid@o11gr21 ~]$ ocrconfig -showbackup
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup00.ocr
o11gr21 2019/08/10 13:55:54 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup01.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/day.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/week.ocr
o11gr22 2022/03/18 16:45:11 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup_20220318_164511.ocr
[grid@o11gr21 ~]$ -- 手工备份一次后，再次查看备份情况
[grid@o11gr21 o11gr2-cluster]$ ocrconfig -showbackup
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup00.ocr
o11gr21 2019/08/10 13:55:54 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup01.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/day.ocr
o11gr21 2019/08/24 13:50:40 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/week.ocr
o11gr21 2022/03/19 09:19:17 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup_20220319_091917.ocr
o11gr22 2022/03/18 16:45:11 /u01/app/11.2.0/grid/cdata/o11gr2-cluster/backup_20220318_164511.ocr
[grid@o11gr21 o11gr2-cluster]$ 3 查看当前ocr所在的盘，进行模拟破坏
SQL> select group_number ,name from v$asm_diskgroup;
GROUP_NUMBER NAME
------------ ------------------------------
1 DATA
2 OCR
SQL>
SQL> select path from v$asm_disk where GROUP_NUMBER =2;
PATH
--------------------------------------------------------------------------------
/dev/asm-ocr3
/dev/asm-ocr2
/dev/asm-ocr1
SQL> -- 使用dd命令，破坏掉/dev/asm-ocr1,/dev/asm-ocr2,/dev/asm-ocr3盘
dd if=/dev/zero of=/dev/asm-ocr1 bs=1M count=10
dd if=/dev/zero of=/dev/asm-ocr2 bs=1M count=10
dd if=/dev/zero of=/dev/asm-ocr3 bs=1M count=10[root@o11gr21 ~]# dd if=/dev/zero of=/dev/asm-ocr1 bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0615155 s, 170 MB/s
[root@o11gr21 ~]# dd if=/dev/zero of=/dev/asm-ocr2 bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0584999 s, 179 MB/s
[root@o11gr21 ~]# dd if=/dev/zero of=/dev/asm-ocr3 bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.0442179 s, 237 MB/s
[root@o11gr21 ~]# -- 使用asmca后，会发现找不到ocr盘，其中alert log如下 （alerto11gr21.log 和alert_+ASM1.log ）
2022-03-19 09:28:11.498:
[/u01/app/11.2.0/grid/bin/oraagent.bin(4383)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/oraagent_grid' disconnected from server. Details at (:CRSAGF00117:) {0:1:9} in /u01/app/11.2.0/grid/log/o11gr21/agent/crsd/oraagent_grid/oraagent_grid.log.
2022-03-19 09:28:11.499:
[/u01/app/11.2.0/grid/bin/orarootagent.bin(4387)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/orarootagent_root' disconnected from server. Details at (:CRSAGF00117:) {0:3:62} in /u01/app/11.2.0/grid/log/o11gr21/agent/crsd/orarootagent_root/orarootagent_root.log.
2022-03-19 09:28:11.501:
[/u01/app/11.2.0/grid/bin/scriptagent.bin(4521)]CRS-5822:Agent '/u01/app/11.2.0/grid/bin/scriptagent_grid' disconnected from server. Details at (:CRSAGF00117:) {0:9:9} in /u01/app/11.2.0/grid/log/o11gr21/agent/crsd/scriptagent_grid/scriptagent_grid.log.
2022-03-19 09:28:12.848:
[crsd(9211)]CRS-1013:The OCR location in an ASM disk group is inaccessible. Details in /u01/app/11.2.0/grid/log/o11gr21/crsd/crsd.log.
2022-03-19 09:28:12.856:
[crsd(9211)]CRS-0804:Cluster Ready Service aborted due to Oracle Cluster Registry error [PROC-26: Error while accessing the physical storage
]. Details at (:CRSD00111:) in /u01/app/11.2.0/grid/log/o11gr21/crsd/crsd.log.
2022-03-19 09:28:17.906:
[ohasd(2734)]CRS-2765:Resource 'ora.crsd' has failed on server 'o11gr21'.
[client(9325)]CRS-10001:19-Mar-22 09:28 ACFS-9203: true
[client(9341)]CRS-10001:19-Mar-22 09:28 ACFS-9203: true
SQL> alter diskgroup OCR dismount force /* ASM SERVER:4059585829 */
GMON dismounting group 2 at 11 for pid 32, osid 8717
NOTE: Disk OCR_0000 in mode 0x7f marked for de-assignment
NOTE: Disk OCR_0001 in mode 0x7f marked for de-assignment
NOTE: Disk OCR_0002 in mode 0x7f marked for de-assignment
SUCCESS: diskgroup OCR was dismounted
SUCCESS: alter diskgroup OCR dismount force /* ASM SERVER:4059585829 */
Sat Mar 19 09:28:08 2022
NOTE: diskgroup resource ora.OCR.dg is offline
SUCCESS: ASM-initiated MANDATORY DISMOUNT of group OCR
Sat Mar 19 09:28:08 2022
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_4284.trc:
ORA-15078: ASM diskgroup was forcibly dismounted
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_4284.trc:
ORA-15078: ASM diskgroup was forcibly dismounted
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_4284.trc:
ORA-15078: ASM diskgroup was forcibly dismounted
WARNING: requested mirror side 1 of virtual extent 5 logical extent 0 offset 708608 is not allocated; I/O request failed
WARNING: requested mirror side 2 of virtual extent 5 logical extent 1 offset 708608 is not allocated; I/O request failed
Errors in file /u01/app/grid/diag/asm/+asm/+ASM1/trace/+ASM1_ora_4284.trc:
ORA-15078: ASM diskgroup was forcibly dismounted
ORA-15078: ASM diskgroup was forcibly dismounted
Sat Mar 19 09:28:08 2022
SQL> alter diskgroup OCR check /* proxy */
ORA-15032: not all alterations performed
ORA-15001: diskgroup "OCR" does not exist or is not mounted
ERROR: alter diskgroup OCR check /* proxy */
Received dirty detach msg from inst 3 for dom 2
List of instances:
1 2 3
Dirty detach reconfiguration started (new ddet inc 3, cluster inc 6)
Global Resource Directory partially frozen for dirty detach
* dirty detach - domain 2 invalid = TRUE
0 GCS resources traversed, 0 cancelled
freeing rdom 2
Dirty Detach Reconfiguration complete
NOTE: client exited [4273]
Sat Mar 19 09:28:12 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9211] opening OCR file
Sat Mar 19 09:28:19 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9315] opening OCR file
Sat Mar 19 09:28:25 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9576] opening OCR file
Sat Mar 19 09:28:32 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9626] opening OCR file
Sat Mar 19 09:28:38 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9642] opening OCR file
Sat Mar 19 09:28:46 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9673] opening OCR file
Sat Mar 19 09:28:52 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9716] opening OCR file
Sat Mar 19 09:28:59 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9730] opening OCR file
Sat Mar 19 09:29:05 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9784] opening OCR file
Sat Mar 19 09:29:11 2022
NOTE: [crsd.bin@o11gr21 (TNS V1-V3) 9797] opening OCR file-- 检查集群，会发现crs没有启动
[grid@o11gr23 ~]$ crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[grid@o11gr23 ~]$ crsctl check cluster -all
**************************************************************
o11gr21:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr22:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr23:
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[grid@o11gr23 ~]$
4 在所有的节点上关闭掉GI
crsctl stop crs -f 5 以exclusive模式启动CRS，（在拥有最近的 OCR 备份的节点上，最近备份的ocr的节点是o11gr21）
crsctl start crs -excl -- 11.2.0.1版本
crsctl start crs -excl -nocrs -- 11.2.0.2及以上版本[root@o11gr21 bin]# ./crsctl start crs -excl -nocrs
CRS-4123: Oracle High Availability Services has been started.
CRS-2672: Attempting to start 'ora.mdnsd' on 'o11gr21'
CRS-2676: Start of 'ora.mdnsd' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.gpnpd' on 'o11gr21'
CRS-2676: Start of 'ora.gpnpd' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.cssdmonitor' on 'o11gr21'
CRS-2672: Attempting to start 'ora.gipcd' on 'o11gr21'
CRS-2676: Start of 'ora.cssdmonitor' on 'o11gr21' succeeded
CRS-2676: Start of 'ora.gipcd' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.cssd' on 'o11gr21'
CRS-2672: Attempting to start 'ora.diskmon' on 'o11gr21'
CRS-2676: Start of 'ora.diskmon' on 'o11gr21' succeeded
CRS-2676: Start of 'ora.cssd' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.drivers.acfs' on 'o11gr21'
CRS-2679: Attempting to clean 'ora.cluster_interconnect.haip' on 'o11gr21'
CRS-2672: Attempting to start 'ora.ctssd' on 'o11gr21'
CRS-2681: Clean of 'ora.cluster_interconnect.haip' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'o11gr21'
CRS-2676: Start of 'ora.drivers.acfs' on 'o11gr21' succeeded
CRS-2676: Start of 'ora.ctssd' on 'o11gr21' succeeded
CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'o11gr21' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'o11gr21'
CRS-2676: Start of 'ora.asm' on 'o11gr21' succeeded
[root@o11gr21 bin]# --这个时候，检查GI的状态，是看不到的，但是asm实例是启动的
[root@o11gr21 bin]# ./crsctl status resource -t
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4000: Command Status failed, or completed with errors.
[root@o11gr21 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4529: Cluster Synchronization Services is online
CRS-4534: Cannot communicate with Event Manager
[root@o11gr21 bin]# ps -ef | grep pmon
grid 11924 1 0 09:41 ? 00:00:00 asm_pmon_+ASM1
root 13358 6219 0 09:47 pts/0 00:00:00 grep pmon
[root@o11gr21 bin]# 6 通过SQL PLUS创建磁盘组 ,这里创建了CRS，和原来的不一样，最后发现restore ocr的时候有问题，又重新创建成了和原来一样的OCR
create diskgroup CRS external redundancy disk '/dev/asm-test3' attribute 'COMPATIBLE.ASM' = '11.2';
[grid@o11gr21 o11gr2-cluster]$ sqlplus /nolog
SQL*Plus: Release 11.2.0.4.0 Production on Sat Mar 19 09:48:47 2022
Copyright (c) 1982, 2013, Oracle. All rights reserved.
SQL> conn / as sysasm
Connected.
SQL> create diskgroup CRS external redundancy disk '/dev/asm-test3' attribute 'COMPATIBLE.ASM' = '11.2';
Diskgroup created.
SQL> [grid@o11gr21 o11gr2-cluster]$ asmcmd
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 2047 1995 0 1995 0 N CRS/
ASMCMD>
-- restore最近的一次的ocr备份 ，发现出错，主要原因是，现有的OCR盘的名称是CRS，和备份中的OCR盘的名称不一样。创建和原来一样的OCR asm盘名称一样即可（原来的盘名称是OCR）。
[root@o11gr21 bin]# cd /u01/app/11.2.0/grid/cdata/o11gr2-cluster/
[root@o11gr21 o11gr2-cluster]# ls -l
total 37148
-rw-------. 1 root root 7602176 Aug 24 2019 backup00.ocr
-rw-------. 1 root root 7602176 Aug 10 2019 backup01.ocr
-rw-------. 1 root root 7630848 Mar 19 09:19 backup_20220319_091917.ocr
-rw-------. 1 root root 7602176 Aug 24 2019 day.ocr
-rw-------. 1 root root 7602176 Aug 24 2019 week.ocr
[root@o11gr21 o11gr2-cluster]# ocrconfig -restore backup_20220319_091917.ocr
[root@o11gr21 o11gr2-cluster]# /u01/app/11.2.0/grid/bin/ocrconfig -restore backup_20220319_091917.ocr
PROT-35: The configured OCR locations are not accessible.
[root@o11gr21 o11gr2-cluster]#-- 在恢复的时候报错，初步怀疑，原来的OCR盘叫做OCR，现在的盘叫做CRS。名字不一样。更改后试试。
create diskgroup OCR external redundancy disk '/dev/asm-test3' attribute 'COMPATIBLE.ASM' = '11.2';
SQL> alter diskgroup CRS mount;
Diskgroup altered.
SQL> drop diskgroup CRS;
Diskgroup dropped.
SQL> create diskgroup OCR external redundancy disk '/dev/asm-test3' attribute 'COMPATIBLE.ASM' = '11.2';
Diskgroup created.
SQL> ７ 重新进行restore OCR的备份，这次可以正常restore，看来OCR盘的名字要和原来一样
[root@o11gr21 ~]# cd /u01/app/11.2.0/grid/cdata/o11gr2-cluster/
[root@o11gr21 o11gr2-cluster]# /u01/app/11.2.0/grid/bin/ocrconfig -restore backup_20220319_091917.ocr
[root@o11gr21 o11gr2-cluster]# ８ 在当前节点上启动CRS （只适用于11.2.0.1. 11.2.0.4略过）
crsctl start res ora.crsd -init９ 重建voteing file ，出现问题。查看log，主要原因是asm_diskstring这个是ｎｕｌｌ
crsctl replace votedisk +OCR
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl replace votedisk +OCR
CRS-4602: Failed 27 to add voting file aee3a63cd40b4fc6bf2f334b0738088f.
Failed to replace voting disk group with +OCR.
CRS-4000: Command Replace failed, or completed with errors.
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
Located 0 voting disk(s).
[root@o11gr21 ~]# -- 查看alter log
[client(5938)]CRS-1002:The OCR was restored from file backup_20220319_091917.ocr.
2022-03-19 10:44:13.513:
[cssd(3002)]CRS-1638:Unable to locate voting file with ID aee3a63c-d40b4fc6-bf2f334b-0738088f that is being added to the list of configured voting files; details at (:CSSNM00022:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
2022-03-19 10:44:13.513:
[cssd(3002)]CRS-1638:Unable to locate voting file with ID aee3a63c-d40b4fc6-bf2f334b-0738088f that is being added to the list of configured voting files; details at (:CSSNM00027:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
2022-03-19 10:44:13.513:
[cssd(3002)]CRS-1630:A configuration change request failed because not all the new voting files were discovered; Details at (:CSSNM00012:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
2022-03-19 10:44:15.786:
[cssd(3002)]CRS-1638:Unable to locate voting file with ID aee3a63c-d40b4fc6-bf2f334b-0738088f that is being added to the list of configured voting files; details at (:CSSNM00022:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
2022-03-19 10:44:15.786:
[cssd(3002)]CRS-1638:Unable to locate voting file with ID aee3a63c-d40b4fc6-bf2f334b-0738088f that is being added to the list of configured voting files; details at (:CSSNM00027:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
2022-03-19 10:44:15.786:
[cssd(3002)]CRS-1630:A configuration change request failed because not all the new voting files were discovered; Details at (:CSSNM00012:) in /u01/app/11.2.0/grid/log/o11gr21/cssd/ocssd.log
--继续查看ocssd.log
2022-03-19 10:44:13.512: [ CSSD][3005736704]clssnmReadDiscoveryProfile: voting file discovery string()
2022-03-19 10:44:13.512: [ CSSD][3005736704]clssnmvDDiscThread: using discovery string for voting file add
2022-03-19 10:44:13.512: [ SKGFD][3005736704]Discovery with str::
2022-03-19 10:44:13.512: [ SKGFD][3005736704]UFS discovery with ::
2022-03-19 10:44:13.512: [ SKGFD][3005736704]Execute glob on the string /dev/raw/*
2022-03-19 10:44:13.513: [ SKGFD][3005736704]running stat on disk:/dev/raw/rawctl
2022-03-19 10:44:13.513: [ SKGFD][3005736704]Fetching UFS disk :/dev/raw/rawctl:
2022-03-19 10:44:13.513: [ SKGFD][3005736704]OSS discovery with ::-- 修改asm string ，可以看到asm_diskstring 是null ，修改为合适的值
SQL> show parameter string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring string
SQL>
SQL> alter system set asm_diskstring ='/dev/asm*';
System altered.
SQL> 10 修改后，再次重建voting disk ，可以了
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl replace votedisk +OCR
Successful addition of voting disk 837c32bd1b864f42bfb55750cb0babca.
Successfully replaced voting disk group with +OCR.
CRS-4266: Voting file(s) successfully replaced
[root@o11gr21 ~]# [root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 837c32bd1b864f42bfb55750cb0babca (/dev/asm-test3) [OCR]
Located 1 voting disk(s).
[root@o11gr21 ~]# [root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 837c32bd1b864f42bfb55750cb0babca (/dev/asm-test3) [OCR]
Located 1 voting disk(s).
[root@o11gr21 ~]# /u01/app/11.2.0/grid/bin/ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3304
Available space (kbytes) : 258816
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@o11gr21 ~]# -- 为ASM重新创建spfile
create pfile='/home/grid/asm_pfile' from memory ;
SQL> create pfile='/home/grid/asm_pfile' from memory ;
File created.
SQL> -- 编辑asm_pfile后，重新创建spfile
create spfile='+OCR' from pfile='/home/grid/asm_pfile';
SQL> create spfile='+OCR' from pfile='/home/grid/asm_pfile';
File created.
SQL>
SQL> shutdown immediate
ASM diskgroups volume disabled
ASM diskgroups dismounted
ASM instance shutdown
SQL> startup
ASM instance started
Total System Global Area 1135747072 bytes
Fixed Size 2260728 bytes
Variable Size 1108320520 bytes
ASM Cache 25165824 bytes
ASM diskgroups mounted
ASM diskgroups volume enabled
SQL>
SQL> show parameter spfile
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
spfile string +OCR/o11gr2-cluster/asmparamet
erfile/registry.253.1099739169
SQL> show parameter string
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
asm_diskstring string /dev/asm*
SQL> -- 或者，使用下面基本的参数来创建 (本次没有使用这个)
vi /home/grid/pfile_tmp
*.asm_power_limit=1
*.diagnostic_dest='/u01/app/oragrid'
*.instance_type='asm'
*.large_pool_size=12M
*.remote_login_passwordfile='EXCLUSIVE'11 关闭CRS
crsctl stop crs -f
[root@o11gr21 bin]# ./crsctl stop crs -f
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'o11gr21'
CRS-2673: Attempting to stop 'ora.ctssd' on 'o11gr21'
CRS-2673: Attempting to stop 'ora.asm' on 'o11gr21'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'o11gr21'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'o11gr21'
CRS-2677: Stop of 'ora.mdnsd' on 'o11gr21' succeeded
CRS-2677: Stop of 'ora.ctssd' on 'o11gr21' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'o11gr21' succeeded
CRS-2677: Stop of 'ora.asm' on 'o11gr21' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'o11gr21'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'o11gr21' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'o11gr21'
CRS-2677: Stop of 'ora.cssd' on 'o11gr21' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'o11gr21'
CRS-2677: Stop of 'ora.gipcd' on 'o11gr21' succeeded
CRS-2673: Attempting to stop 'ora.gpnpd' on 'o11gr21'
CRS-2677: Stop of 'ora.gpnpd' on 'o11gr21' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'o11gr21' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@o11gr21 bin]# 12 集群中所有节点启动crs
[root@o11gr21 bin]# ./crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@o11gr21 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4535: Cannot communicate with Cluster Ready Services
CRS-4530: Communications failure contacting Cluster Synchronization Services daemon
CRS-4534: Cannot communicate with Event Manager
[root@o11gr21 bin]#
[root@o11gr21 bin]#
[root@o11gr21 bin]# ./crsctl check crs
CRS-4638: Oracle High Availability Services is online
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
[root@o11gr21 bin]#
[root@o11gr21 bin]# ./crsctl check cluster -all
**************************************************************
o11gr21:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr22:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
o11gr23:
CRS-4537: Cluster Ready Services is online
CRS-4529: Cluster Synchronization Services is online
CRS-4533: Event Manager is online
**************************************************************
[root@o11gr21 bin]# 13 检查，ocrcheck，votedisk
[root@o11gr21 bin]# ./crsctl query css votedisk
## STATE File Universal Id File Name Disk group
-- ----- ----------------- --------- ---------
1. ONLINE 837c32bd1b864f42bfb55750cb0babca (/dev/asm-test3) [OCR]
Located 1 voting disk(s).
[root@o11gr21 bin]# ./ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 3
Total space (kbytes) : 262120
Used space (kbytes) : 3304
Available space (kbytes) : 258816
ID : 953786240
Device/File Name : +OCR
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check succeeded
[root@o11gr21 bin]# 到此，CRS磁盘组故障模拟，修复完毕。
END
标签：CRS,RAC,o11gr21,故障模拟,grid,Oracle,OCR,root,asm 来源： https://blog.csdn.net/xxzhaobb/article/details/123591657
本站声明： 1. iCode9 技术分享网（下文简称本站）提供的所有内容，仅供技术学习、探讨和分享； 2. 关于本站的所有留言、评论、转载及引用，纯属内容发起人的个人观点，与本站观点和立场无关； 3. 关于本站的所有言论和文字，纯属内容发起人的个人观点，与本站观点和立场无关； 4. 本站文章均是网友提供，不完全保证技术分享内容的完整性、准确性、时效性、风险性和版权归属；如您发现该文章侵犯了您的权益，可联系我们第一时间进行删除； 5. 本站为非盈利性的个人网站，所有内容不会用来进行牟利，也不会利用任何形式的广告来间接获益，纯粹是为了广大技术爱好者提供技术内容和技术思想的分享性交流网站。