在做Corosync+DRBD的高可用MySQL集群实验中，意外发现各个节点无法识别对方，连接为StandAlone则主从节点无法通信，显示如下：
[root@node1 ~]# cat /proc/drbd version: 8.3.15 (api:88/proto:86-97) GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by [email protected], 2013-03-27 16:04:08 0: cs:StandAlone ro:Primary/Unknown ds:UpToDate/DUnknown r----- ns:0 nr:0 dw:36 dr:285 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:24 [root@node2 ~]# cat /proc/drbd version: 8.3.15 (api:88/proto:86-97) GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by [email protected], 2013-03-27 16:04:08 0: cs:StandAlone ro:Secondary/Unknown ds:UpToDate/DUnknown r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:8
查看Node1节点的日志发现脑裂：
[root@node1 ~]# tail -n 12 /var/log/messages Sep 21 11:19:53 lab4 kernel: block drbd0: Split-Brain detected but unresolved, dropping connection! Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 Sep 21 11:19:53 lab4 kernel: block drbd0: helper command: /sbin/drbdadm split-brain minor-0 exit code 0 (0x0) Sep 21 11:19:53 lab4 kernel: block drbd0: conn( WFReportParams -> Disconnecting ) Sep 21 11:19:53 lab4 kernel: block drbd0: error receiving ReportState, l: 4! Sep 21 11:19:53 lab4 kernel: block drbd0: meta connection shut down by peer. Sep 21 11:19:53 lab4 kernel: block drbd0: asender terminated Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_asender Sep 21 11:19:53 lab4 kernel: block drbd0: Connection closed Sep 21 11:19:53 lab4 kernel: block drbd0: conn( Disconnecting -> StandAlone ) Sep 21 11:19:53 lab4 kernel: block drbd0: receiver terminated Sep 21 11:19:53 lab4 kernel: block drbd0: Terminating drbd0_receiver
以下为DRBD脑裂手动恢复过程：
将Node1设置为主节点并挂载测试
[root@node1 ~]# drbdadm primary mydrbd [root@node1 ~]# mount /dev/drbd0 /mydata [root@node1 ~]# ll /mydata/ total 20 -rw-r--r-- 1 root root 1666 Sep 20 18:18 inittab drwx------ 2 root root 16384 Sep 20 18:15 lost+found
将Node2设置为从节点并丢弃资源数据
[root@node2 ~]# drbdadm secondary mydrbd [root@node2 ~]# drbdadm -- --discard-my-data connect mydrbd
在Node1主节点上手动连接资源
[root@node1 ~]# drbdadm connect mydrbd
最后查看各个节点状态，连接已恢复正常
[root@node1 ~]# cat /proc/drbd version: 8.3.15 (api:88/proto:86-97) GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by [email protected], 2013-03-27 16:04:08 0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r----- ns:36 nr:0 dw:24 dr:185 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
[root@node2 ~]# cat /proc/drbd version: 8.3.15 (api:88/proto:86-97) GIT-hash: 0ce4d235fc02b5c53c1c52c53433d11a694eab8c build by [email protected], 2013-03-27 16:04:08 0: cs:Connected ro:Secondary/Primary ds:UpToDate/UpToDate C r----- ns:0 nr:36 dw:36 dr:0 al:0 bm:1 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
在线帮助文档参考：(Manual split brain recovery)
http://www.drbd.org/users-guide/s-resolve-split-brain.html