問題現象：
集羣好像不漂移IP了，全部會話連接數量只集中在某個節點上，這個節點斷掉也不會自動連接到另外一個節點上；
問題排查：
在節點RAC1上執行集羣狀態檢查命令（注意看紅色字體部分）：
grid@rac01:[/home/grid]crsctl stat res -t
--------------------------------------------------------------------------------
NAME TARGET STATE SERVER STATE_DETAILS
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.ARDATA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.CLDATA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.LISTENER.lsnr
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.USDATA.dg
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.USDATA01.dg
ONLINE OFFLINE rac01
ONLINE ONLINE rac02
ora.asm
ONLINE ONLINE rac01 Started
ONLINE ONLINE rac02 Started
ora.gsd
OFFLINE OFFLINE rac01
OFFLINE OFFLINE rac02
ora.net1.network
ONLINE ONLINE rac01
ONLINE ONLINE rac02
ora.ons
ONLINE ONLINE rac01
ONLINE ONLINE rac02
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE rac02
ora.cvu
1 ONLINE ONLINE rac02
ora.oc4j
1 ONLINE ONLINE rac02
ora.rac01.vip
1 ONLINE ONLINE rac01
ora.rac02.vip
1 ONLINE ONLINE rac02
ora.racdb.db
1 ONLINE OFFLINE Instance Shutdown
2 ONLINE ONLINE rac02 Open
ora.scan1.vip
1 ONLINE ONLINE rac02
由此發現可能USDATA01卷組不能正常工作，嘗試啓動USDATA01卷組，發現繼續報錯。
grid@rac01:[/home/grid]srvctl start diskgroup -g USDATA01
PRCR-1079 : Failed to start resource ora.USDATA01.dg
CRS-5017: The resource action "ora.USDATA01.dg start" encountered the following error:
ORA-15032: not all alterations performed
ORA-15017: diskgroup "USDATA01" cannot be mounted
ORA-15063: ASM discovered an insufficient number of disks for diskgroup "USDATA01"
. For details refer to "(:CLSN00107:)" in "/u01/app/11.2.0/grid/log/rac01/agent/crsd/oraagent_grid/oraagent_grid.log".
CRS-2674: Start of 'ora.USDATA01.dg' on 'rac01' failed
查看ASM磁盤卷組，發現USDATA01卷組無法正常識別，所以節點RAC1的狀態不正常是因爲無法正常識別該卷組的問題。
grid@rac01:[/home/grid]asmcmd
ASMCMD> lsdg
State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 4096 1048576 666603 332494 0 332494 0 N ARDATA/
MOUNTED EXTERN N 512 4096 1048576 51207 50811 0 50811 0 Y CLDATA/
MOUNTED EXTERN N 512 4096 1048576 512009 124302 0 124302 0 N USDATA/