dg備庫加入集羣和ORA-00304: requested INSTANCE_NUMBER is busy

我在搭建dg時，用

srvctl add database -d docare5pri -n docare5 -o /oracle/app/oracle/product/11.2.0/dbhome_1 -c RAC -p +DATA/icupri/parameter/spfileicu.ora -r physical_standby -s OPEN -t IMMEDIATE -a DATA,ARCH
srvctl add instance -d docare5pri -i docare51 -n smdb01
srvctl add instance -d docare5pri -i docare52 -n smdb02

添加數據庫到ocr中，讓集羣管理，但是srvctl start 報錯

 

srvctl啓動rac數據庫報錯：

[oracle@smdb01 ~]$ srvctl start database -d docare5pri
PRCR-1079 : Failed to start resource ora.docare5pri.db
CRS-5017: The resource action "ora.docare5pri.db start" encountered the following error: 
ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0
. For details refer to "(:CLSN00107:)" in "/oracle/app/11.2.0/grid/log/smdb02/agent/crsd/oraagent_oracle//oraagent_oracle.log".

CRS-2674: Start of 'ora.docare5pri.db' on 'smdb02' failed
CRS-2632: There are no more servers to try to place resource 'ora.docare5pri.db' on that would satisfy its placement policy

看到shared memeory我還以爲是內存問題，看了內存又free很多，內核參數也設置正確，大頁都正確加載了

 

集羣狀態：

ora.LISTENER_SCAN1.lsnr
      1        ONLINE  ONLINE       smdb02                                       
ora.cvu
      1        ONLINE  ONLINE       smdb02                                       
ora.docare5pri.db
      1        ONLINE  ONLINE       smdb01                   Open,Readonly       
      2        ONLINE  OFFLINE                               Instance Shutdown   
ora.icu.db
      1        OFFLINE OFFLINE                                                   
      2        OFFLINE OFFLINE                                                   
ora.icupri.db
      1        ONLINE  ONLINE       smdb01                   Open,Readonly       
      2        ONLINE  OFFLINE                               Instance Shutdown   
ora.oc4j
      1        ONLINE  ONLINE       smdb02                                       
ora.scan1.vip
      1        ONLINE  ONLINE       smdb02                                       
ora.smdb01.vip
      1        ONLINE  ONLINE       smdb01                                       
ora.smdb02.vip
      1        ONLINE  ONLINE       smdb02  

 

oraagent_oracle.log：

2020-06-11 22:26:35.380: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstConnection::connectInt: server not attached
2020-06-11 22:26:35.417: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstConnection:connectInt connected
2020-06-11 22:26:35.417: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstAgent::startup
2020-06-11 22:27:01.871: [ora.icupri.db][1215178496]{2:50963:52532} [start] ORA-00304: requested INSTANCE_NUMBER is busy

2020-06-11 22:27:01.871: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstAgent::start: 1 errcode 304
2020-06-11 22:27:01.871: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstAgent::start: ORA-304 instance busy  m_instanceType 1 m_lastOCIError 304
2020-06-11 22:27:01.872: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstAgent::start: ORA-304 instance busy  dbType RAC
2020-06-11 22:27:01.873: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstConnection::connectInt: server not attached
2020-06-11 22:27:01.916: [ora.icupri.db][1215178496]{2:50963:52532} [start] ORA-01034: ORACLE not available
ORA-27101: shared memory realm does not exist
Linux-x86_64 Error: 2: No such file or directory
Process ID: 0
Session ID: 0 Serial number: 0

2020-06-11 22:27:01.917: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstConnection::connectInt (2) Exception OCIException
2020-06-11 22:27:01.917: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstConnection:connect:excp OCIException OCI error 1034
2020-06-11 22:27:01.917: [ora.icupri.db][1215178496]{2:50963:52532} [start] InstAgent::start: 2 errcode 1034
2020-06-11 22:27:01.917: [ USRTHRD][1215178496]{2:50963:52532} InstConnection:~InstConnection: this 3000e590

 

從log日誌中仔細查看發現，啓動前面都沒有報錯，到requested INSTANCE_NUMBER is busy時報錯

 

mos：

OERR: ORA-304 "requested INSTANCE_NUMBER is busy" Reference Note (Doc ID 18363.1)

   Bug 27731019 - Starting a RAC Database may Fail With Error ORA-304 (Doc ID 27731019.8)

解決辦法：

Stop the clusterware and started it with 'crsctl stop crs' and 'crsctl start crs'

Bug 27731019 說是12c和19c纔有這個bug，我的環境是11g！！！

 

爲了徹底解決，把CRS中的數據庫資源刪除了

srvctl remove instance -d icupri -i icu1 
srvctl remove instance -d icupri -i icu2
srvctl remove database -d icupri

然後再root中重啓集羣，2個節點都需要執行

crsctl stop crs

crsctl start crs

檢查啓動好後加入數據庫資源到CRS

srvctl add database -d docare5pri -n docare5 -o /oracle/app/oracle/product/11.2.0/dbhome_1 -c RAC -p +DATA/icupri/parameter/spfileicu.ora -r physical_standby -s OPEN -t IMMEDIATE -a DATA,ARCH
srvctl add instance -d docare5pri -i docare51 -n smdb01
srvctl add instance -d docare5pri -i docare52 -n smdb02

 

最後啓動成功

srvctl start database -d icupri