RHEL6-红帽HA（RHCS）ricci+luci+fence
1.整体架构：
2实验环境：
luci管理机：192.168.122.1
ricci节点：192.168.122.34
192.168.122.33
192.168.122.82
yum仓库:
[rhel-source]
name=RedHat Enterprise Linux $releasever - $basearch - Source
baseurl=ftp://192.168.122.1/pub/rhel6.3
gpgcheck=0
[HighAvailability]
name=InstructorServer Repository
baseurl=ftp://192.168.122.1/pub/rhel6.3/HighAvailability
gpgcheck=0
[LoadBalancer]
name=InstructorServer Repository
baseurl=ftp://192.168.122.1/pub/rhel6.3/LoadBalancer
gpgcheck=0
[ResilientStorage]
name=InstructorServer Repository
baseurl=ftp://192.168.122.1/pub/rhel6.3/ResilientStorage
gpgcheck=0
[ScalableFileSystem]
name=InstructorServer Repository
baseurl=ftp://192.168.122.1/pub/rhel6.3/ScalableFileSystem
gpgcheck=0
要将红色部分加入yum'仓库；
说明：
不支持在集群节点中使用NetworkManager。如果您已经在集群节点中安装了NetworkManager,您应该 删除或者禁用该程序。
2.环境配置：
以下的操作在所有的HA节点上进行：
在所有的ha节点上安装ricci 在客户端（要有web浏览器）上安装luci
# yum-y install ricci
[root@desk82~]# chkconfig ricci on
[root@desk82~]# /etc/init.d/ricci start
[root@desk82~]# passwd ricci #一定要有否则将认证失败：
启动luci：
[root@wangzirhel6cluster]# /etc/init.d/luci start
Pointyour web browser to https://localhost.localdomain:8084(or equivalent) to access luci
网页访问：
https://localhost.localdomain:8084
并使用root登录
创建集群：
此时luci管理端正在为ricci的HA节点上自动安装所需要的包：
在HA节点上可以看见有yum的进程正在运行：
完成后：
3.fence设备配置：
采用虚拟机fence设备：虚拟机与主机名的对应关系：
hostnameipkvmdomain name
desk34192.168.122.34ha1
desk33192.168.122.33ha2
desk82192.168.122.82desk82
luci主机上：
[root@wangzidocs]# yum -y install fence-virt fence-virtd fence-virtd-libvirtfence-virtd-multicast
[root@wangzidocs]# fence_virtd -c
Modulesearch path [/usr/lib64/fence-virt]:
Availablebackends:
libvirt 0.1
Availablelisteners:
multicast 1.1
Listenermodules are responsible for accepting requests
fromfencing clients.
Listenermodule [multicast]:
Themulticast listener module is designed for use environments
wherethe guests and hosts may communicate over a network using
multicast.
Themulticast address is the address that a client will use to
sendfencing requests to fence_virtd.
MulticastIP Address [225.0.0.12]:
Usingipv4 as family.
MulticastIP Port [1229]:
Settinga preferred interface causes fence_virtd to listen only
on thatinterface. Normally, it listens on the default network
interface. In environments where the virtual machines are
usingthe host machine as a gateway, this *must* be set
(typicallyto virbr0).
Set to'none' for no interface.
Interface[none]: virbr0 #根据主机网卡配置而定或是br0
如果虚拟机与真机之间使用NAT选择virbr0，使用的桥接选择br0
The keyfile is the shared key information which is used to
authenticatefencing requests. The contents of this file must
bedistributed to each physical host and virtual machine within
acluster.
KeyFile [/etc/cluster/fence_xvm.key]:
Backendmodules are responsible for routing requests to
theappropriate hypervisor or management layer.
Backendmodule [checkpoint]: libvirt
Thelibvirt backend module is designed for single desktops or
servers. Do not use in environments where virtual machines
may bemigrated between hosts.
LibvirtURI [qemu:///system]:
Configurationcomplete.
===Begin Configuration ===
fence_virtd{
listener= "multicast";
backend= "libvirt";
module_path= "/usr/lib64/fence-virt";
}
listeners{
multicast{
key_file= "/etc/cluster/fence_xvm.key";
address= "225.0.0.12";
family= "ipv4";
port= "1229";
interface= "virbr0";
}
}
backends{
libvirt{
uri ="qemu:///system";
}
}
=== EndConfiguration ===
Replace/etc/fence_virt.conf with the above [y/N]? y
luci主机上的cluster配置文件位置：/etc/fence_virt.conf
[root@wangzidocs]# dd if=/dev/urandom of=/etc/cluster/fence_xvm.key bs=128count=1
[root@wangzicluster]# scp /etc/cluster/fence_xvm.key desk33:/etc/cluster/
[root@wangzicluster]# scp /etc/cluster/fence_xvm.key desk34:/etc/cluster/
[root@wangzicluster]# scp /etc/cluster/fence_xvm.key desk82:/etc/cluster/
[root@wangzicluster]# /etc/init.d/fence_virtd start
[root@wangzicluster]# netstat -anplu | grep fence
udp 0 0 0.0.0.0:1229 0.0.0.0:* 10601/fence_virtd
[root@wangzicluster]# fence_xvm -H vm4 -o reboot #检查fence是否可以控制虚拟机如果可以，对应的虚拟机将会重启；
添加fence设备：
在网页上所做的一切操作都会写在各个节点的/etc/cluster/cluster.conf中：
[root@desk82cluster]# cat cluster.conf
<?xmlversion="1.0"?>
<clusterconfig_version="2" name="wangzi_1">
<clusternodes>
<clusternodename="192.168.122.34" nodeid="1"/>
<clusternodename="192.168.122.33" nodeid="2"/>
<clusternodename="192.168.122.82" nodeid="3"/>
</clusternodes>
<fencedevices>
<fencedeviceagent="fence_xvm" name="vmfence"/>
</fencedevices>
</cluster>
在个Node上：
添加错误域：
“Priority”为优先级，越小优先级越高：
NoFailback 为服务不回切（默认为回切）
添加资源：
此ip为一个虚拟的浮动ip，用于外界访问。将会浮动出现在后端提供服务的HA节点上；
最后一行的数字越小，浮动ip切换的速度越快；
httpd服务必须是自己在HA节点上提前安装，但不要启动
添加服务组：
在服务组apsche下添加资源刚才添加的资源ipadress 与httpd
可看见集群已经自动将192.168.122.34上httpd启动了。
[root@desk34~]# /etc/init.d/httpd status
httpd(pid 14453) is running...
测试：
[root@desk34~]# clustat
ClusterStatus for wangzi_1 @ Sat Sep 7 02:52:18 2013
MemberStatus: Quorate
MemberName
ID Status
---------- ---- ------
192.168.122.34 1 Online, Local,rgmanager
192.168.122.33 2 Online, rgmanager
192.168.122.82 3 Online, rgmanager
Service Name Owner (Last) State
------- ---- ----- ------ -----
service:apsche 192.168.122.34 started
（1.）关闭httpd服务
[root@desk34~]# /etc/init.d/httpd stop
在desk33上将会出现192.168.122.122的浮动ip：
[root@desk33~]# ip addr show
2:eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdiscpfifo_fast state UP qlen 1000
link/ether 52:54:00:d0:fe:21 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.33/24 brd 192.168.122.255 scope global eth0
inet 192.168.122.122/24 scope global secondaryeth0
如果再将desk33上的httpd关掉，将会切换到desk82上；
如果将desk34的httpd启动，浮动将会回到desk34上，因为desk34的优先级最高，并且设置了服务回切
（2）断网模拟：
[root@desk34~]# ifconfig eth0 down
desk34会重启服务将会切换到desk33上；
当desk34重启完毕后，服务会回切到desk34上。
（3）内核崩溃：
[root@desk34~]# echo c > /proc/sysrq-trigger
主机重启，服务切换到desk33
西安石油大学
王兹银