1、實驗說明：
node1主機名：node1.abc.com，地址192.168.1.10；node2主機名：node2.abc.com，地址192.168.1.20；vip地址192.168.1.50；操作系統linux企業版5.4；內核版本號：2.6.18-164.el5
實驗用到的軟件包：
由於drbd內核模塊代碼只在linux內核2.6.3.33以後的版本中才有，所以我們要同時安裝內核模塊和管理工具：
drbd83-8.3.8-1.el5.centos.i386.rpm #drbd的管理包
kmod-drbd83-8.3.8-1.el5.centos.i686.rpm #drbd的內核模塊
cluster-glue-1.0.6-1.6.el5.i386.rpm #在羣集中增加對更多節點的支持
cluster-glue-libs-1.0.6-1.6.el5.i386.rpm #庫文件
corosync-1.2.7-1.1.el5.i386.rpm corosync #的主配置文件
corosynclib-1.2.7-1.1.el5.i386.rpm #corosync的庫文件
heartbeat-3.0.3-2.3.el5.i386.rpm #做heartbeat四層的資源代理
heartbeat-libs-3.0.3-2.3.el5.i386.rpm #heartbeat的庫文件
ldirectord-1.0.1-1.el5.i386.rpm #在高可用性羣集中實驗對後面realserver的探測
libesmtp-1.0.4-5.el5.i386.rpm
openais-1.1.3-1.6.el5.i386.rpm #做豐富pacemake的內容
openaislib-1.1.3-1.6.el5.i386.rpm #openais 的庫文件
pacemaker-1.1.5-1.1.el5.i386.rpm # pacemake的主配置文檔
pacemaker-libs-1.1.5-1.1.el5.i386.rpm #pacemaker的庫文件
pacemaker-cts-1.1.5-1.1.el5.i386.rpm
perl-TimeDate-1.16-5.el5.noarch.rpm
resource-agents-1.0.4-1.1.el5.i386.rpm # 開啓資源代理
mysql-5.5.15-linux2.6-i686.tar.gz # mysql的綠色安裝包
2、實驗步驟：
1、同步時間：
node1：
[root@node1 ~]# hwclock –s
node2：
[root@node2 ~]# hwclock –s
2、修改hosts文件：
[root@node1 ~]# vim /etc/hosts
# Do not remove the following line, or various programs
# that require network functionality will fail.
127.0.0.1 localhost.localdomain localhost
::1 localhost6.localdomain6 localhost6
192.168.1.10 node1.abc.com node1
192.168.1.20 node2.abc.com node2
將hosts文件拷貝都node2中：
[root@node1 ~]# scp /etc/hosts 192.168.1.20:/etc/hosts
The authenticity of host '192.168.1.20 (192.168.1.20)' can't be established.
RSA key fingerprint is d4:f1:06:3b:a0:81:fd:85:65:20:9e:a1:ee:46:a6:8b.
Are you sure you want to continue connecting (yes/no)? yes #覆蓋原hosts文件
Warning: Permanently added '192.168.1.20' (RSA) to the list of known hosts.
[email protected]'s password: #輸入node2的管理員密碼
hosts
3、在兩個節點上生成密鑰，實現無密碼的方式通訊：
node1：
[root@node1 ~]# ssh-keygen -t rsa #產生一個非對稱加密的私鑰對
enerating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): #默認，直接回車
Enter passphrase (empty for no passphrase): #默認，直接回車
Enter same passphrase again: #默認，直接回車
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
61:b1:4a:c8:88:19:31:5d:cb:8f:91:0c:fe:38:bd:c3 [email protected]
[root@node1 ~]# ssh-copy-id -i .ssh/id_rsa.pub [email protected] #拷貝到node2上
15
The authenticity of host 'node2.abc.com (192.168.1.20)' can't be established.
RSA key fingerprint is d4:f1:06:3b:a0:81:fd:85:65:20:9e:a1:ee:46:a6:8b.
Are you sure you want to continue connecting (yes/no)? yes #輸入yes
Warning: Permanently added 'node2.abc.com' (RSA) to the list of known hosts.
[email protected]'s password: #node2的管理員密碼
node2：
[root@node2 ~]# ssh-keygen -t rsa #產生一個非對稱加密的私鑰對
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): #默認，直接回車
Enter passphrase (empty for no passphrase): #默認，直接回車
Enter same passphrase again: #默認，直接回車
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
3f:0b:27:14:8a:ba:b1:c6:4d:02:2b:22:86:a3:46:0a [email protected]
[root@node2 ~]# ssh-copy-id -i .ssh/id_rsa.pub [email protected] #拷貝到node1節點上
15
The authenticity of host 'node1.abc.com (192.168.1.10)' can't be established.
RSA key fingerprint is d4:f1:06:3b:a0:81:fd:85:65:20:9e:a1:ee:46:a6:8b.
Are you sure you want to continue connecting (yes/no)? yes #輸入yes
Warning: Permanently added 'node1.abc.com,192.168.1.10' (RSA) to the list of known hosts.
[email protected]'s password: #node1的管理員密碼
至此我們可以再兩個節點之間實現無密碼的通信了。
4、編輯yum客戶端
node1：
[root@node1 ~]# mkdir /mnt/cdrom
[root@node1 ~]# mount /dev/cdrom /mnt/cdrom/
[root@node1 ~]# vim /etc/yum.repos.d/rhel-debuginfo.repo
[rhel-server]
name=Red Hat Enterprise Linux server
baseurl=file:///mnt/cdrom/Server
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
[rhel-cluster]
name=Red Hat Enterprise Linux cluster
baseurl=file:///mnt/cdrom/Cluster
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
[rhel-clusterstorage]
name=Red Hat Enterprise Linux clusterstorage
baseurl=file:///mnt/cdrom/ClusterStorage
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
[rhel-vt]
name=Red Hat Enterprise Linux vt
baseurl=file:///mnt/cdrom/VT
enabled=1
gpgcheck=1
gpgkey=file:///mnt/cdrom/RPM-GPG-KEY-redhat-release
將yum客戶端文件拷貝到node2的/etc/ yum.repos.d/ 目錄下
[root@node1 ~]# scp /etc/yum.repos.d/rhel-debuginfo.repo node2.abc.com:/etc/yum.repos.d/
node2：
[root@node2 ~]# mkdir /mnt/cdrom
[root@node2 ~]# mount /dev/cdrom /mnt/cdrom/
5、將下載好的rpm包上傳到linux上的各個節點
安裝drbd：
node1：
[root@node1 ~]# yum localinstall drbd83-8.3.8-1.el5.centos.i386.rpm kmod-drbd83-8.3.8-1.el5.centos.i686.rpm -y –nogpgcheck
node2:
[root@node2 ~]# yum localinstall drbd83-8.3.8-1.el5.centos.i386.rpm kmod-drbd83-8.3.8-1.el5.centos.i686.rpm -y --nogpgcheck
6、在各節點上增加一個大小類型都相關的drbd設備（sda4）：
node1：
[root@node1 ~]# fdisk /dev/sda
The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n #增加一分區
Command action
e extended
p primary partition (1-4)
p #主分區
Selected partition 4
First cylinder (1580-2610, default 1580): #默認值，回車
Using default value 1580
Last cylinder or +size or +sizeM or +sizeK (1580-2610, default 2610): +1G #大小爲1G
Command (m for help): w #保存並推出
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@node1 ~]# partprobe /dev/sda #重新加載內核模塊
[root@node1 ~]# cat /proc/partitions
major minor #blocks name
8 0 20971520 sda
8 1 104391 sda1
8 2 10482412 sda2
8 3 2096482 sda3
8 4 987997 sda4
node2：
[root@node2 ~]# fdisk /dev/sda
The number of cylinders for this disk is set to 2610.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n n #增加一分區
Command action
e extended
p primary partition (1-4)
p #主分區
Selected partition 4
First cylinder (1580-2610, default 1580): #默認值，回車
Using default value 1580
Last cylinder or +size or +sizeM or +sizeK (1580-2610, default 2610): +1G #大小爲1G
Command (m for help): w #保存並推出
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table.
The new table will be used at the next reboot.
Syncing disks.
[root@node2 ~]# partprobe /dev/sda #重新加載內核模塊
[root@node2 ~]# cat /proc/partitions
major minor #blocks name
8 0 20971520 sda
8 1 104391 sda1
8 2 10482412 sda2
8 3 2096482 sda3
8 4 987997 sda4
7、配置drbd：
node1：
複製配置文件drbd.conf
[root@node1 ~]# cp /usr/share/doc/drbd83-8.3.8/drbd.conf /etc/
備份global_common.conf
[root@node1 ~]# cd /etc/drbd.d/
[root@node1 drbd.d]# cp global_common.conf global_common.conf.bak
編輯global_common.conf
[root@node1 drbd.d]# vim global_common.conf
global {
usage-count no; #不開啓統計
}
common {
protocol C;
handlers {
pri-on-incon-degr "/usr/lib/drbd/notify-pri-on-incon-degr.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
pri-lost-after-sb "/usr/lib/drbd/notify-pri-lost-after-sb.sh; /usr/lib/drbd/notify-emergency-reboot.sh; echo b > /proc/sysrq-trigger ; reboot -f";
local-io-error "/usr/lib/drbd/notify-io-error.sh; /usr/lib/drbd/notify-emergency-shutdown.sh; echo o > /proc/sysrq-trigger ; halt -f";
}
startup {
wfc-timeout 120; #等待連接的超時時間
degr-wfc-timeout 100; #等待降級的節點連接的超時時間
}
disk {
on-io-error detach; #當出現I/O錯誤，節點要拆掉drbd設備
}
net {
cram-hmac-alg "sha1"; #使用sha1加密算法實現節點認證
shared-secret "mydrbdlab"; #認證碼，兩個節點內容要相同
}
syncer {
rate 100M; #定義同步數據時的速率
}
}
定義mysql的資源：
[root@node1 drbd.d]# vim mysql.res
resource mysql {
on node1.abc.com {
device /dev/drbd0;
disk /dev/sda4;
address 192.168.1.10:7898;
meta-disk internal;
}
on node2.abc.com {
device /dev/drbd0;
disk /dev/sda4;
address 192.168.1.20:7898;
meta-disk internal;
}
}
8、將global_common.conf、mysql.res、drbd.conf拷貝到node2中：
[root@node1 drbd.d]# scp global_common.conf node2.abc.com:/etc/drbd.d/
[root@node1 drbd.d]# scp mysql.res node2.abc.com:/etc/drbd.d/
[root@node1 drbd.d]# scp /etc/drbd.conf node2.abc.com:/etc/
9、分別在node1和node2上初始化定義的mysql的資源並啓動相應的服務
node1：
[root@node1 drbd.d]# drbdadm create-md mysql
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
[root@node1 drbd.d]# service drbd start
node2：
[root@node2 drbd.d]# drbdadm create-md mysql
Writing meta data...
initializing activity log
NOT initialized bitmap
New drbd meta data block successfully created.
[root@node2 drbd.d]# service drbd start
查看drbd的狀態：
node1：
[root@node1 drbd.d]# drbd-overview
0:mysql Connected Secondary/Secondary Inconsistent/Inconsistent C r----
node2：
[root@node2 drbd.d]# drbd-overview
0:mysql Connected Secondary/Secondary Inconsistent/Inconsistent C r----
由此可知此時兩個節點均爲Secondary狀態。
現將node1設爲主節點，在node1上執行：
node1：
[root@node1 drbd.d]# drbdadm -- --overwrite-data-of-peer primary mysql
再次查看：
node1：
[root@node1 drbd.d]# drbd-overview
0:mysql SyncSource Primary/Secondary UpToDate/Inconsistent C r----
[========>...........] sync'ed: 45.9% (538072/987928)K delay_probe: 40
node2：
[root@node2 drbd.d]# drbd-overview
0:mysql Connected Secondary/Primary UpToDate/UpToDate C r----
此時node1爲主節點，node2爲備份節點。
10、格式化、創建掛載點（只在主節點，即在node1上執行）：
[root@node1 ~]# mkfs -t ext3 /dev/drbd0
[root@node1 ~]# mkdir /mysqldata
[root@node1 ~]# mount /dev/drbd0 /mysqldata/
[root@node1 ~]# cd /mysqldata
[root@node1 mysql]# touch node1 #創建名爲node1的文件
[root@node1 mysql]# ll
total 16
drwx------ 2 root root 16384 Mar 14 19:10 lost+found
-rw-r--r-- 1 root root 0 Mar 14 19:19 node1
11、卸載drbd設備
node1：
[root@node1 mysql]# cd
[root@node1 ~]# umount /mysqldata
將node1設置爲secondary節點：
[root@node1 ~]# drbdadm secondary mysql
[root@node1 ~]# drbd-overview
0:mysql Connected Secondary/Secondary UpToDate/UpToDate C r----
12、將node2設置爲primary節點
node2：
[root@node2 drbd.d]# cd
[root@node2 ~]# drbdadm primary mysql
[root@node2 ~]# drbd-overview
0:mysql Connected Primary/Secondary UpToDate/UpToDate C r----
[root@node2 ~]# mkdir /mysqldata
[root@node2 ~]# mount /dev/drbd0 /mysqldata
[root@node2 ~]# cd /mysqldata
[root@node2 mysql]# ll
total 16
drwx------ 2 root root 16384 Mar 14 19:10 lost+found
-rw-r--r-- 1 root root 0 Mar 14 19:19 node1
卸載：
[root@node2 mysql]# cd
[root@node2 ~]# umount /mnt/mysql/
至此/dev/drbd0已同步，drbd已經正常安裝完成。
13、安裝並配置mysql：
node1：
添加用戶和組：
[root@node1 ~]# groupadd -r mysql
[root@node1 ~]# useradd -g mysql -r mysql
由於主設備才能讀寫，掛載，故我們還要設置node1爲主設備，node2爲從設備：
node2：
[root@node2 ~]# drbdadm secondary mysql
node1：
[root@node1 ~]# drbdadm primary mysql
掛載drbd設備：
[root@node1 ~]# mount /dev/drbd0 /mysqldata
[root@node1 ~]# mkdir /mysqldata/data
data目錄要用存放mysql的數據，故改變其屬主屬組：
[root@node1 ~]# chown -R mysql.mysql /mysqldata/data/
安裝mysql：
[root@node1 ~]# tar -zxvf mysql-5.5.15-linux2.6-i686.tar.gz -C /usr/local/
[root@node1 ~]# cd /usr/local/
[root@node1 local]# ln -sv mysql-5.5.15-linux2.6-i686 mysql
[root@node1 local]# cd mysql
[root@node1 mysql]# chown -R mysql:mysql . #修改當前目錄寫文件的權限
初始化mysql數據庫：
[root@node1 mysql]# scripts/mysql_install_db --user=mysql --datadir=/mysqldata/data
[root@node1 mysql]# chown -R root .
爲mysql提供主配置文件：
[root@node1 mysql]# cp support-files/my-large.cnf /etc/my.cnf
編輯my.cnf：
[root@node1 mysql]# vim /etc/my.cnf
39行 thread_concurrency = 2
添加如下行指定mysql數據文件的存放位置：
datadir = /mysqldata/data
爲mysql提供sysv服務腳本，使其能使用service命令：
[root@node1 mysql]# cp support-files/mysql.server /etc/rc.d/init.d/mysqld
node2上的配置文件，sysv服務腳本和此相同，故直接複製過去：
[root@node1 mysql]# scp /etc/my.cnf node2.abc.com:/etc/
[root@node1 mysql]# scp /etc/rc.d/init.d/mysqld node2.abc.com:/etc/rc.d/init.d/
添加至服務列表：
[root@node1 mysql]# chkconfig --add mysqld
確保開機不能自動啓動，我們要用CRM控制：
[root@node1 mysql]# chkconfig mysqld off
啓動服務：
[root@node1 mysql]# service mysqld start
測試之後關閉服務：
查看data是否有文件
[root@node1 mysql]# ll /mnt/mysql/data/
total 29756
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:17 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:17 ib_logfile1
-rw-rw---- 1 mysql mysql 18874368 Mar 14 20:17 ibdata1
drwx------ 2 mysql root 4096 Mar 14 19:52 mysql
-rw-rw---- 1 mysql mysql 27017 Mar 14 20:16 mysql-bin.000001
-rw-rw---- 1 mysql mysql 996460 Mar 14 20:16 mysql-bin.000002
-rw-rw---- 1 mysql mysql 107 Mar 14 20:17 mysql-bin.000003
-rw-rw---- 1 mysql mysql 57 Mar 14 20:17 mysql-bin.index
-rw-rw---- 1 mysql root 1699 Mar 14 20:17 node1.abc.com.err
-rw-rw---- 1 mysql mysql 5 Mar 14 20:17 node1.abc.com.pid
drwx------ 2 mysql mysql 4096 Mar 14 20:16 performance_schema
drwx------ 2 mysql root 4096 Mar 14 19:51 test
[root@node1 mysql]# service mysqld stop
爲了使用mysql的安裝符合系統使用規範，並將其開發組件導出給系統使用，這裏還需要進行如下步驟：
輸出mysql的man手冊至man命令的查找路徑：
[root@node1 mysql]# vim /etc/man.config
48行 MANPATH /usr/local/mysql/man
輸出mysql的頭文件至系統頭文件路徑/usr/include，這可以通過簡單的創建鏈接實現：
[root@node1 mysql]# ln -sv /usr/local/mysql/include/ /usr/include/mysql
輸出mysql的庫文件給系統庫查找路徑：（文件只要是在/etc/ld.so.conf.d/下並且後綴是.conf就可以）
[root@node1 mysql]# echo '/usr/local/mysql/lib/' > /etc/ld.so.conf.d/mysql.conf
而後讓系統重新載入系統庫：
[root@node1 mysql]# ldconfig
修改PATH環境變量，讓系統所有用戶可以直接使用mysql的相關命令:
[root@node1 mysql]# vim /etc/profile
PATH=$PATH:/usr/local/mysql/bin
重新讀取環境變量
[root@node1 mysql]# . /etc/profile
[root@node1 mysql]# echo $PATH
/usr/kerberos/sbin:/usr/kerberos/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin:/usr/local/mysql/bin
卸載drbd設備：
[root@node1 mysql]# umount /mysqldata
node2：
添加用戶和組：
[root@node2 ~]# groupadd -r mysql
[root@node2 ~]# useradd -g mysql -r mysql
由於主設備才能讀寫，掛載，故我們還要設置node2爲主設備，node1爲從設備：
node1上操作：
[root@node1 mysql]# drbdadm secondary mysql
node2上操作：
[root@node2 ~]# drbdadm primary mysql
掛載drbd設備：
[root@node2 ~]# mount /dev/drbd0 /mysqldata
查看：
[root@node2 ~]# ll /mnt/mysql/data/
total 29752
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:20 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:17 ib_logfile1
-rw-rw---- 1 mysql mysql 18874368 Mar 14 20:20 ibdata1
drwx------ 2 mysql root 4096 Mar 14 19:52 mysql
-rw-rw---- 1 mysql mysql 27017 Mar 14 20:16 mysql-bin.000001
-rw-rw---- 1 mysql mysql 996460 Mar 14 20:16 mysql-bin.000002
-rw-rw---- 1 mysql mysql 126 Mar 14 20:20 mysql-bin.000003
-rw-rw---- 1 mysql mysql 57 Mar 14 20:17 mysql-bin.index
-rw-rw---- 1 mysql root 2116 Mar 14 20:20 node1.abc.com.err
drwx------ 2 mysql mysql 4096 Mar 14 20:16 performance_schema
drwx------ 2 mysql root 4096 Mar 14 19:51 test
安裝mysql：
[root@node2 ~]# tar -zxvf mysql-5.5.15-linux2.6-i686.tar.gz -C /usr/local/
[root@node2 ~]# cd /usr/local/
[root@node2 local]# ln -sv mysql-5.5.15-linux2.6-i686 mysql
[root@node2 local]# cd mysql
一定不能對數據庫進行初始化，因爲我們在node1上已經初始化了：
[root@node2 mysql]# chown -R root:mysql .
mysql主配置文件和sysc服務腳本已經從node1複製過來了，不用在添加。
添加至服務列表：
[root@node2 mysql]# chkconfig --add mysqld
確保開機不能自動啓動，我們要用CRM控制：
[root@node2 mysql]# chkconfig mysqld off
而後就可以啓動服務測試使用了：（確保node1的mysql服務停止）
[root@node2 mysql]# service mysqld start
測試之後關閉服務：
查看其中是否有文件
[root@node2 mysql]# ll /mysqldata/data/
total 29764
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:48 ib_logfile0
-rw-rw---- 1 mysql mysql 5242880 Mar 14 20:17 ib_logfile1
-rw-rw---- 1 mysql mysql 18874368 Mar 14 20:20 ibdata1
drwx------ 2 mysql root 4096 Mar 14 19:52 mysql
-rw-rw---- 1 mysql mysql 27017 Mar 14 20:16 mysql-bin.000001
-rw-rw---- 1 mysql mysql 996460 Mar 14 20:16 mysql-bin.000002
-rw-rw---- 1 mysql mysql 126 Mar 14 20:20 mysql-bin.000003
-rw-rw---- 1 mysql mysql 107 Mar 14 20:48 mysql-bin.000004
-rw-rw---- 1 mysql mysql 76 Mar 14 20:48 mysql-bin.index
-rw-rw---- 1 mysql root 2116 Mar 14 20:20 node1.abc.com.err
-rw-rw---- 1 mysql root 937 Mar 14 20:48 node2.abc.com.err
-rw-rw---- 1 mysql mysql 5 Mar 14 20:48 node2.abc.com.pid
drwx------ 2 mysql mysql 4096 Mar 14 20:16 performance_schema
drwx------ 2 mysql root 4096 Mar 14 19:51 test
[root@node2 mysql]# service mysqld stop
爲了使用mysql的安裝符合系統使用規範，並將其開發組件導出給系統使用，這裏還需要進行與node1上相同的操作，這裏不再闡述。
卸載設備：
[root@node2 mysql]# umount /dev/drbd0
14、corosync+pacemaker的安裝和配置
node1：
[root@node1 ~]# yum localinstall *.rpm -y –nogpgcheck
此處不需要安裝ldirectord
node:2：
[root@node2 ~]# yum localinstall *.rpm -y –nogpgcheck
此處不需要安裝ldirectord
對各個節點進行相應的配置：
node1：
[root@node1 ~]# cd /etc/corosync/
[root@node1 corosync]# cp corosync.conf.example corosync.conf
[root@node1 corosync]# vim corosync.conf
# Please read the corosync.conf.5 manual page
compatibility: whitetank
totem {
version: 2
secauth: off
threads: 0
interface {
ringnumber: 0
bindnetaddr: 192.168.1.0 #只需改動這裏
mcastaddr: 226.94.1.1
mcastport: 5405
}
}
logging {
fileline: off
to_stderr: no #是否發送標準出錯
to_logfile: yes
to_syslog: yes #系統日誌 （建議關掉一個），會降低性能
logfile: /var/log/cluster/corosync.log #需要手動創建目錄cluster
debug: off #排除時可以啓動
timestamp: on #日誌中是否記錄時間
以下是openais的東西
logger_subsys {
subsys: AMF
debug: off
}
}
amf {
mode: disabled
}
補充一些東西，前面只是底層的東西，因爲要用pacemaker
service {
ver: 0
name: pacemaker
use_mgmtd: yes
}
雖然用不到openais ，但是會用到一些子選項
aisexec {
user: root
group: root
}
創建cluster目錄
[root@node1 corosync]# mkdir /var/log/cluster
爲了便面其他主機加入該集羣，需要認證，生成一authkey
[root@node1 corosync]# corosync-keygen
Corosync Cluster Engine Authentication key generator.
Gathering 1024 bits for key from /dev/random.
Press keys on your keyboard to generate entropy.
Writing corosync key to /etc/corosync/authkey.
[root@node1 corosync]# ll
total 28
-rw-r--r-- 1 root root 5384 Jul 28 2010 amf.conf.example
-r-------- 1 root root 128 Mar 14 21:13 authkey
-rw-r--r-- 1 root root 563 Mar 14 21:08 corosync.conf
-rw-r--r-- 1 root root 436 Jul 28 2010 corosync.conf.example
drwxr-xr-x 2 root root 4096 Jul 28 2010 service.d
drwxr-xr-x 2 root root 4096 Jul 28 2010 uidgid.d
[root@node1 corosync]# ssh node2.abc.com 'mkdir /var/log/cluster'
將node1節點上的文件拷貝到節點node2上面（記住要帶-p)
[root@node1 corosync]# scp -p authkey corosync.conf node2.abc.com:/etc/corosync/
在node1和node2節點上面啓動 corosync 的服務
node1：
[root@node1 corosync]# service corosync start
node2：
[root@node2 corosync]# service corosync start
驗證corosync引擎是否正常啓動了：
node1（注意此時node1是備份節點）：
[root@node1 corosync]# grep -i -e "corosync cluster engine" -e "configuration file" /var/log/messages
Mar 14 21:16:54 node1 corosync[6081]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Mar 14 21:16:54 node1 corosync[6081]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
node2：
[root@node2 corosync]# grep -i -e "corosync cluster engine" -e "configuration file" /var/log/messages
Mar 14 21:17:03 node2 corosync[5876]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Mar 14 21:17:03 node2 corosync[5876]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
Mar 14 21:17:03 node2 corosync[5876]: [MAIN ] Corosync Cluster Engine exiting with status 8 at main.c:1397.
Mar 14 21:17:53 node2 corosync[5913]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Mar 14 21:17:53 node2 corosync[5913]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
Mar 14 21:17:53 node2 corosync[5913]: [MAIN ] Corosync Cluster Engine exiting with status 8 at main.c:1397.
Mar 14 21:19:53 node2 corosync[5978]: [MAIN ] Corosync Cluster Engine ('1.2.7'): started and ready to provide service.
Mar 14 21:19:53 node2 corosync[5978]: [MAIN ] Successfully read main configuration file '/etc/corosync/corosync.conf'.
查看初始化成員節點通知是否發出：
node1：
[root@node1 corosync]# grep -i totem /var/log/messages
Apr 3 14:13:16 node1 corosync[387]: [TOTEM ] Initializing transport (UDP/IP).
Apr 3 14:13:16 node1 corosync[387]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Apr 3 14:13:16 node1 corosync[387]: [TOTEM ] The network interface [192.168.1.10] is now up.
Apr 3 14:13:17 node1 corosync[387]: [TOTEM ] Process pause detected for 565 ms, flushing membership messages.
Apr 3 14:13:17 node1 corosync[387]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
Apr 3 14:13:19 node1 corosync[387]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
node2：
[root@node2 ~]# grep -i totem /var/log/messages
Apr 3 14:13:19 node2 corosync[32438]: [TOTEM ] Initializing transport (UDP/IP).
Apr 3 14:13:19 node2 corosync[32438]: [TOTEM ] Initializing transmit/receive security: libtomcrypt SOBER128/SHA1HMAC (mode 0).
Apr 3 14:13:19 node2 corosync[32438]: [TOTEM ] The network interface [192.168.1.20] is now up.
Apr 3 14:13:21 node2 corosync[32438]: [TOTEM ] A processor joined or left the membership and a new membership was formed.
檢查過程中是否有錯誤產生：
[root@node1 ~]# grep -i error: /var/log/messages |grep -v unpack_resources
[root@node2 ~]# grep -i error: /var/log/messages |grep -v unpack_resources
檢查pacemaker時候已經啓動了：
node1：
[root@node1 ~]# grep -i pcmk_startup /var/log/messages
Mar 14 21:16:55 node1 corosync[6081]: [pcmk ] info: pcmk_startup: CRM: Initialized
Mar 14 21:16:55 node1 corosync[6081]: [pcmk ] Logging: Initialized pcmk_startup
Mar 14 21:16:55 node1 corosync[6081]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Mar 14 21:16:55 node1 corosync[6081]: [pcmk ] info: pcmk_startup: Service: 9
Mar 14 21:16:55 node1 corosync[6081]: [pcmk ] info: pcmk_startup: Local hostname: node1.abc.com
Mar 14 22:13:15 node1 corosync[3179]: [pcmk ] info: pcmk_startup: CRM: Initialized
Mar 14 22:13:15 node1 corosync[3179]: [pcmk ] Logging: Initialized pcmk_startup
Mar 14 22:13:15 node1 corosync[3179]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Mar 14 22:13:15 node1 corosync[3179]: [pcmk ] info: pcmk_startup: Service: 9
Mar 14 22:13:15 node1 corosync[3179]: [pcmk ] info: pcmk_startup: Local hostname: node1.abc.com
node2：
[root@node2 ~]# grep -i pcmk_startup /var/log/messages
Mar 14 21:19:55 node2 corosync[5978]: [pcmk ] info: pcmk_startup: CRM: Initialized
Mar 14 21:19:55 node2 corosync[5978]: [pcmk ] Logging: Initialized pcmk_startup
Mar 14 21:19:55 node2 corosync[5978]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Mar 14 21:19:55 node2 corosync[5978]: [pcmk ] info: pcmk_startup: Service: 9
Mar 14 21:19:55 node2 corosync[5978]: [pcmk ] info: pcmk_startup: Local hostname: node2.abc.com
Mar 14 22:13:20 node2 corosync[3174]: [pcmk ] info: pcmk_startup: CRM: Initialized
Mar 14 22:13:20 node2 corosync[3174]: [pcmk ] Logging: Initialized pcmk_startup
Mar 14 22:13:20 node2 corosync[3174]: [pcmk ] info: pcmk_startup: Maximum core file size is: 4294967295
Mar 14 22:13:20 node2 corosync[3174]: [pcmk ] info: pcmk_startup: Service: 9
Mar 14 22:13:21 node2 corosync[3174]: [pcmk ] info: pcmk_startup: Local hostname: node2.abc.com
在node2（主節點）上查看羣集的狀態
[root@node2 corosync]# crm status
============
Last updated: Tue Apr 3 15:26:56 2012
Stack: openais
Current DC: node1.abc.com - partition with quorum
Version: 1.1.5-1.1.el5-01e86afaaa6d4a8c4836f68df80ababd6ca3902f
2 Nodes configured, 2 expected votes
0 Resources configured.
============
Online: [ node1.abc.com node2.abc.com ]
15、配置羣集的工作屬性
corosync默認啓用了stonith，而當前集羣並沒有相應的stonith設備，因此此默認配置目前尚不可用，這可以通過如下命令先禁用stonith：
node1：
[root@node1 ~]# crm configure property stonith-enabled=false
node2：
[root@node2 ~]# crm configure property stonith-enabled=false
對於雙節點的集羣來說，我們要配置此選項來忽略quorum，即這時候票數不起作用，一個節點也能正常運行：
node1：
[root@node1 ~]# crm configure property no-quorum-policy=ignore
node2：
[root@node2 ~]# crm configure property no-quorum-policy=ignore
定義資源的粘性值，使資源不能再節點之間隨意的切換，因爲這樣是非常浪費系統的資源的。
資源黏性值範圍及其作用：
0：這是默認選項。資源放置在系統中的最適合位置。這意味着當負載能力“較好”或較差的節點變得可用時才轉移資源。此選項的作用基本等同於自動故障回覆，只是資源可能會轉移到非之前活動的節點上；
大於0：資源更願意留在當前位置，但是如果有更合適的節點可用時會移動。值越高表示資源越願意留在當前位置；
小於0：資源更願意移離當前位置。絕對值越高表示資源越願意離開當前位置；
INFINITY：如果不是因節點不適合運行資源（節點關機、節點待機、達到migration-threshold 或配置更改）而強制資源轉移，資源總是留在當前位置。此選項的作用幾乎等同於完全禁用自動故障回覆；
-INFINITY：資源總是移離當前位置；
我們這裏可以通過以下方式爲資源指定默認黏性值：
node1：
[root@node1 ~]# crm configure rsc_defaults resource-stickiness=100
node2：
[root@node2 ~]# crm configure rsc_defaults resource-stickiness=100
因字數限制，所以分上下兩篇，請看“下”