實驗環境：
vm1-ning:172.16.3.1/16
vm2-hong:172.16.3.2/16
drbd操作步驟：
1、創建mysql的drbd資源磁盤(兩節點都需要創建)
[root@hong ~]# fdisk /dev/sda
n
p
+5G
w
[root@hong ~]# kpartx -af /dev/sda
[root@hong ~]# partx -a /dev/sda
2、drbd設置（全局配置文件這裏和web高可用中的一樣沒有改變，不再次說明。）
[root@hong ~]# cd /etc/drbd.d
[root@hong drbd.d]# ls
global_common.conf global_common.conf.bak global_common.conf.rpmsave web.res
[root@hong drbd.d]# cp web.res mydata.res
[root@hong drbd.d]# vim mydata.res 定義資源名稱
[root@hong drbd.d]# cat mydata.res
resource mydata { 定義資源名稱也是唯一的，不能相同
on ning {
device /dev/drbd1; 指定drdb設備名稱爲drbd1
disk /dev/sda4; 定義的分區磁盤
address 172.16.3.1:7790; 兩個資源之間不能使用相同的端口號
meta-disk internal;
}
on hong {
device /dev/drbd1;
disk /dev/sda4;
address 172.16.3.2:7790;
meta-disk internal;
}
}
[root@hong drbd.d]# scp mydata.res 172.16.3.1:/etc/drbd.d/ 雙方節點上都需要配置這樣複製過去即可
[email protected]'s password:
mydata.res
100% 278 0.3KB/s 00:00
3、初始化drbd在兩節點上都需要初始化，並啓動drbd
[root@hong drbd.d]# drbdadm create-md mydata
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
[root@ning ~]# drbdadm create-md mydata
Writing meta data...
initializing activity log
NOT initializing bitmap
New drbd meta data block successfully created.
[root@hong drbd.d]# service drbd start
[root@ning ~]# service drbd start
4、提升一個節點爲主節點進行分區數據同步和格式化分區
[root@hong drbd.d]# drbd-overview （查看當前狀態）
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 Connected Secondary/Secondary Inconsistent/Inconsistent C r----- （分區還沒有進行同步，數據不一致狀態）
[root@hong drbd.d]# drbdadm primary --force mydata （提升hong節點爲主節點，進行數據同步，並進行格式化）
[root@hong drbd.d]# drbd-overview
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 SyncSource Primary/Secondary UpToDate/Inconsistent C r---n-
[>....................] sync'ed: 4.3% (4912/5128)M （正在同步塊按照位在對齊）
[root@hong drbd.d]# drbd-overview
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 Connected Primary/Secondary UpToDate/UpToDate C r----- （分區同步完成）
[root@hong ~]# mke2fs -t ext4 /dev/drbd1 （對分區進行格式化）
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe blocks
328656 inodes, 1313263 blocks
65663 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1346371584
41 block groups
32768 blocks per group, 32768 fragments per group
8016 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
mysql配置：
1、創建mysql用戶（注意兩節點的mysql的ID號要一致）
hong節點：
[root@hong drbd.d]# groupadd -r -g 306 mysql
[root@hong drbd.d]# useradd -r -g 306 -u 306 mysql
[root@hong drbd.d]# id mysql
uid=306(mysql) gid=306(mysql) groups=306(mysql)
[root@hong ~]# mkdir /mydata 創建數據的存儲目錄
ning節點：
[root@ning ~]# groupadd -r -g 306 mysql
[root@ning ~]# useradd -r -g 306 -u 306 mysql
[root@ning ~]# id mysql
uid=306(mysql) gid=306(mysql) groups=306(mysql)
[root@ning ~]# mkdir /mydata 創建數據的存儲目錄（磁盤的掛載點）
2、安裝mysql
（1）在主節點hong上掛載/dev/drbd1到數據的存儲目錄/mydata
[root@hong ~]# mount /dev/drbd1 /mydata/
（2）在掛載目錄/mydata下創建數據存儲目錄爲data，並設置屬主組
[root@hong ~]# cd /mydata/
[root@hong mydata]# mkdir data
[root@hong mydata]# chown -R mysql.mysql data
[root@hong mydata]# ll
total 20
drwxr-xr-x 2 mysql mysql 4096 Aug 16 17:21 data
drwx------ 2 root root 16384 Aug 16 17:16 lost+found
驗證下是否兩節點都是有data目錄的
[root@hong ~]# umount /mydata
[root@hong ~]# drbdadm secondary mydata 在hong上降級drbd爲從節點
[root@hong ~]# drbd-overview
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
[root@ning ~]# drbdadm primary mydata 在ning節點上提升爲主節點
[root@ning ~]# drbd-overview
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 Connected Primary/Secondary UpToDate/UpToDate C r-----
[root@ning ~]# mount /dev/drbd1 /mydata/
[root@ning ~]# cd /mydata/
[root@ning mydata]# ll
total 20
drwxr-xr-x 2 mysql mysql 4096 Aug 16 17:21 data （查看是否data目錄爲mysql屬主組）
drwx------ 2 root root 16384 Aug 16 17:16 lost+found
（3）安裝mysql軟件
包： mariadb-5.5.36-linux-x86_64.tar.gz
ning節點上：
[root@ning ~]# tar xf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
[root@ning ~]# cd /usr/local
[root@ning local]# ln -sv mariadb-5.5.36-linux-x86_64/ mysql
`mysql' -> `mariadb-5.5.36-linux-x86_64/'
[root@ning local]# chown -R root.mysql mysql/*
[root@ning local]# cd mysql
[root@ning mysql]# scripts/mysql_install_db --datadir=/mydata/data/ --user=mysql
[root@ning mysql]# mkdir /etc/mysql
[root@ning mysql]# cp support-files/my-large.cnf /etc/mysql/my.cnf
[root@ning mysql]# vim /etc/mysql/my.cnf
[mysqld]
datadir = /mydata/data
[root@ning mysql]# cp support-files/my-large.cnf /etc/mysql/my.cnf
[root@ning mysql]# vim /etc/mysql/my.cnf
[root@ning mysql]# vim /etc/mysql/my.cnf
[root@ning mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
[root@ning mysql]# chmod +x /etc/rc.d/init.d/mysqld
[root@ning mysql]# chkconfig --add mysqld
[root@ning mysql]# chkconfig mysqld off
[root@ning mysql]# service mysqld start
Starting MySQL.... [ OK ]
[root@ning mysql]# ls /mydata/data
aria_log.00000001 ibdata1 ib_logfile1 mysql-bin.000001 ning.pid test
aria_log_control ib_logfile0 mysql mysql-bin.index performance_schema
[root@ning mysql]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.36-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> create database mysdb;
Query OK, 1 row affected (0.00 sec)
MariaDB [(none)]> \q
Bye
到這裏mariadb數據庫已經安裝完成
[root@ning mysql]# service mysqld stop
Shutting down MySQL. [ OK ]
[root@ning mysql]# cd
[root@ning ~]# umount /dev/drbd1 卸載掛載點
[root@ning ~]# drbdadm secondary mydata 降級爲從節點
[root@ning ~]# drbd-overview
0:web/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
1:mydata/0 Connected Secondary/Secondary UpToDate/UpToDate C r-----
[root@ning ~]# scp mariadb-5.5.36-linux-x86_64.tar.gz 172.16.3.2:/root 複製包到ning節點，進行安裝
[email protected]'s password:
mariadb-5.5.36-linux-x86_64.tar.gz 100% 212MB 26.5MB/s 00:08
hong節點：
[root@hong ~]# drbdadm primary mydata 提升hong節點爲主節點
[root@hong ~]# mount /dev/drbd1 /mydata 掛載分區
[root@hong ~]# scp -r 172.16.3.1:/etc/mysql /etc/ 並複製ning節點上的mysqld配置文件，（因爲配置文件是一樣的）
[email protected]'s password:
my.cnf 100% 4924 4.8KB/s 00:00
[root@hong ~]# tar xf mariadb-5.5.36-linux-x86_64.tar.gz -C /usr/local
[root@hong ~]# cd /usr/local
[root@hong local]# ln -sv mariadb-5.5.36-linux-x86_64/ mysql
`mysql' -> `mariadb-5.5.36-linux-x86_64/'
[root@hong local]# cd mysql
[root@hong mysql]# chown -R root.mysql ./*
這裏說明下，不需要對mariadb做初始化了，因爲數據已經有了，在ning節點上做的。在這裏只需要提供服務啓動腳本即可
[root@hong mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
[root@hong mysql]# chmod +x /etc/rc.d/init.d/mysqld
[root@hong mysql]# chkconfig --add mysqld
[root@hong mysql]# service mysqld start
Starting MySQL.... [ OK ]
[root@hong mysql]# /usr/local/mysql/bin/mysql
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 2
Server version: 5.5.36-MariaDB-log MariaDB Server
Copyright (c) 2000, 2014, Oracle, Monty Program Ab and others.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
MariaDB [(none)]> show databases; 查看在ning節點上創建的數據庫是存在的
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysdb | 看這裏呵呵
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.05 sec)
MariaDB [(none)]> grant all on *.* to root@'172.16.%.%' identified by 'ning'; 授權遠程連接用戶
Query OK, 0 rows affected (0.03 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
測試下是否能夠連接上數據庫：（隨便找個節點上測試的（這節點上需要有mysql客戶端））
[root@liang ~]# mysql -uroot -h 172.16.3.2 -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.36-MariaDB-log MariaDB Server
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysdb |
| mysql | 這裏可以看到創建的數據庫
| performance_schema |
| test |
+--------------------+
5 rows in set (0.00 sec)
mysql> \q
Bye
（4）測試完成後，在兩節點上關閉mysqld和drbd開始配置corosync
[root@hong mysql]# service mysqld stop
Shutting down MySQL. [ OK ]
[root@hong mysql]# umount /mydata
[root@hong mysql]# drbdadm secondary mydata
[root@hong mysql]# service drbd stop
Stopping all DRBD resources: .
[root@ning ~]# service drbd stop
Stopping all DRBD resources: .
配置corosync集羣設置
1、定義drbd
[root@ning ~]# crm
crm(live)# configure
定義drbd資源
crm(live)configure# primitive mydrbd ocf:linbit:drbd params drbd_resource=mydata op monitor role=Master interval=10s timeout=20s op monitor role=Slave interval=20s timeout=30s op start timeout=240s op stop timeout=100s
注意下定義時間間隔的時候主從不能一樣。
crm(live)configure# verify
定義drbd克隆資源
crm(live)configure# master ms_mydrbd mydrbd meta master-max=1 master-node-max=1 clone-max=2 clone-node-max=1 notify=True
crm(live)configure# verify
定義掛載磁盤資源
crm(live)configure# primitive myfs ocf:heartbeat:Filesystem params device=/dev/drbd1 directory=/mydata fstype=ext4 op monitor interval=20s timeout=60s op start timeout=60s op stop timeout=60s
crm(live)configure# verify
定義mysqlIP資源
crm(live)configure# primitive myip ocf:heartbeat:IPaddr params ip=172.16.3.101 op monitor interval=30s timeout=20s
crm(live)configure# verify
定義mysql服務啓動資源
crm(live)configure# primitive myserver lsb:mysqld op monitor interval=30s timeout=20s
crm(live)configure# verify
定義規則
crm(live)configure# group myservice myip myfs myserver 定義組名稱爲myservice資源名稱爲myip\myfs\myserver
crm(live)configure# colocation myfs_with_ms_mydrbd_master inf: myfs ms_mydrbd:Master 定義myfs掛載磁盤資源一定和ms_mydrbd克隆資源在一起
crm(live)configure# verify
crm(live)configure# order myfs_after_ms_mydrbd_master mandatory: ms_mydrbd:promote myfs:start 定義啓動順序掛載磁盤my_fs一定在克隆資源（成爲主節點）ms_mydrbd之後啓動
crm(live)configure# colocation myserver_with_myfs inf: myserver myfs 定義mysql服務器啓動一定要和掛載磁盤my_fs在一起
crm(live)configure# verify
crm(live)configure# commit
crm(live)# status (查看狀態信息)
Last updated: Fri Sep 19 13:16:37 2014
Last change: Fri Sep 19 12:26:20 2014 via cibadmin on hong
Stack: classic openais (with plugin)
Current DC: ning - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
5 Resources configured
Online: [ hong ning ] （兩個節點都在線）
Master/Slave Set: ms_mydrbd [mydrbd]
Masters: [ ning ]
Slaves: [ hong ]
Resource Group: myservice
myip (ocf::heartbeat:IPaddr): Started ning
myfs (ocf::heartbeat:Filesystem): Started ning
myserver (lsb:mysqld): Started ning
測試corosync+mariadb+drbd
1、測試是否ning節點下線，資源將被切換到hong節點上去
[root@ning ~]# crm node standby 讓ning節點下線
[root@ning ~]# crm status
Last updated: Fri Sep 19 13:24:33 2014
Last change: Fri Sep 19 13:24:25 2014 via crm_attribute on ning
Stack: classic openais (with plugin)
Current DC: ning - partition with quorum
Version: 1.1.10-14.el6-368c726
2 Nodes configured, 2 expected votes
5 Resources configured
Node ning: standby
Online: [ hong ]
Master/Slave Set: ms_mydrbd [mydrbd]
Masters: [ hong ] drbd將自動切換到hong節點上
Stopped: [ ning ]
Resource Group: myservice
myip (ocf::heartbeat:IPaddr): Started hong
myfs (ocf::heartbeat:Filesystem): Started hong
myserver (lsb:mysqld): Started hong
在hong 節點查看下mysqlIP；
[root@hong ~]# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:d7:f7:9c brd ff:ff:ff:ff:ff:ff
inet 172.16.3.2/16 brd 172.16.255.255 scope global eth0
inet 172.16.3.101/16 brd 172.16.255.255 scope global secondary eth0 這裏是有mysqlIP
inet6 fe80::20c:29ff:fed7:f79c/64 scope link
valid_lft forever preferred_lft forever
在liang節點上嘗試連接下mariadb
[root@liang ~]# mysql -uroot -h 172.16.3.101 -p 這裏連接的mysqlIP的地址
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.5.36-MariaDB-log MariaDB Server
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysdb |這裏有我們在上面測試用到的mysdb
| mysql |
| performance_schema |
| test |
+--------------------+
5 rows in set (0.10 sec)
2、測試mysqld手動當掉，是否會自動啓動？
# service mysqld stop
# ss -tnl 觀察3306端口將會看到30s後會自動上線