All demonstrations are performed in VMware virtual machines.
Operating System: CentOS 6.5 64-bit
High Availability Software: Heartbeat 3.0.4
Web Application Server: Apache HTTPD 2.2.15
Network Configuration and Resource Planning
| Hostname | Interface | IP Address | Purpose |
|---|---|---|---|
| ha-primary | eth0 | 172.18.10.227 | External management IP for WAN traffic |
| eth1 | 172.18.0.227 | Heartbeat interface for server communication (direct connection) | |
| 172.18.10.200 | Virtual IP for application service A | ||
| ha-secondary | eth0 | 172.18.10.228 | External management IP for WAN traffic |
| eth1 | 172.18.0.228 | Heartbeat interface for server communication (direct connection) | |
| 172.18.10.201 | Virtual IP for application service B |
Installation and Configuration (Primary and Secondary Servers)
1. Configure Hostanme and Hosts File
1.1 Update hostname to ha-primary
[root@Base ~]# sed -i 's#HOSTNAME=Base#HOSTNAME=ha-primary#g' /etc/sysconfig/network
[root@Base ~]# hostname ha-primary
Logout and login again (ctrl+d)
[root@ha-primary ~]#
1.2 Configure hosts file
[root@ha-primary ~]# cat >>/etc/hosts<<eof> 172.18.0.227 ha-primary
> 172.18.0.228 ha-secondary
> eof
[root@ha-primary ~]# tail -2 /etc/hosts
172.18.0.227 ha-primary
172.18.0.228 ha-secondary
</eof>
1.3 Verify configuration
[root@ha-primary ~]# ping ha-primary
[root@ha-primary ~]# ping ha-secondary
2. Configure Host Routing
2.1 On ha-primary server:
[root@ha-primary ~]# route add -host 172.18.0.228 dev eth1
[root@ha-primary ~]# echo "/sbin/route add -host 172.18.0.228 dev eth1" >> /etc/rc.local
[root@ha-primary ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.18.0.228 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
172.18.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.18.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
2.2 On ha-secondary server:
[root@ha-secondary ~]# route add -host 172.18.0.227 dev eth1
[root@ha-secondary ~]# echo "/sbin/route add -host 172.18.0.227 dev eth1" >> /etc/rc.local
[root@ha-secondary ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.18.0.227 0.0.0.0 255.255.255.255 UH 0 0 0 eth1
172.18.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
172.18.10.0 0.0.0.0 255.255.255.0 U 0 0 0 eth0
Installing Heartbeat Software
Note: CentOS 5.8 includes Heartbeat in default repositories, but CentOS 6+ requires EPEL repository.
1. Create Software Directory
[root@ha-primary ~]# mkdir -p /opt/packages
[root@ha-primary ~]# cd /opt/packages/
2. Download and Install EPEL Repository
[root@ha-primary packages]# wget http://mirrors.ustc.edu.cn/fedora/epel/6/x86_64/epel-release-6-8.noarch.rpm
[root@ha-primary packages]# rpm -ivh epel-release-6-8.noarch.rpm
[root@ha-primary packages]# rpm -qa|grep epel
epel-release-6-8.noarch
3. Install Heartbeat via YUM
[root@ha-primary packages]# yum install heartbeat -y
Installing Apache HTTPD
1. Install HTTPD via YUM
[root@ha-primary ~]# yum install httpd -y
Configuring High Availability Services
1. Configure Heartbeat
1.1 Copy configuration files from documentation directory
[root@ha-primary ~]# cp /usr/share/doc/heartbeat-3.0.4/{authkeys,ha.cf,haresources} /etc/ha.d/
[root@ha-primary ~]# cd /etc/ha.d/
[root@ha-primary ha.d]# ls -l
-rw-r--r--. 1 root root 645 authkeys
-rw-r--r--. 1 root root 10502 ha.cf
-rwxr-xr-x. 1 root root 745 harc
-rw-r--r--. 1 root root 5905 haresources
1.2 Configure authkeys file
[root@ha-primary ha.d]# cat authkeys
auth 1
1 sha1 f93j4k5l2m8n7p1q9r3s6t8u2v5w7x9z
[root@ha-primary ha.d]# chmod 600 authkeys
1.3 Configure haresources file
[root@ha-primary ha.d]# cat haresources
ha-primary IPaddr::172.18.10.200/24/eth0 httpd
ha-secondary IPaddr::172.18.10.201/24/eth0
Configuration notes:
a. 172.18.10.200 is initially bound to ha-primary, 172.18.10.201 to ha-secondary
b. VIPs are taken over only when heartbeat communication fails
c. httpd parameter delegates Apache service control to Heartbeat
1.4 Configure ha.cf file
[root@ha-primary ha.d]# cat ha.cf
debugfile /var/log/ha-debug.log
logfile /var/log/ha.log
logfacility local1
keepalive 2
deadtime 30
warntime 10
initdead 90
mcast eth1 225.0.0.100 694 1 0
auto_failback on
node ha-primary
node ha-secondary
crm no
Parameter explanations:
| debugfile | Heartbeat debug log file path |
|---|---|
| logfile | Heartbeat operational log file path |
| logfacility | Syslog facility for log messages |
| keepalive | Heartbeat interval in seconds (2s) |
| deadtime | Time without heartbeat before failover (30s) |
| warntime | Delay before warning about missed heartbeat (10s) |
| initdead | Initial delay before starting resources (90s) |
| mcast | Multicast configuration on eth1 interface |
| auto_failback | Automatic resource return after recvoery |
| node | Cluster node names (must match uname -n) |
| crm | Disable Cluster Resource Manager |
1.5 Disable firewall for testing
[root@ha-primary ~]# chkconfig iptables off
[root@ha-primary ~]# chkconfig|grep iptables
iptables 0:off 1:off 2:off 3:off 4:off 5:off 6:off
1.6 Copy configuration files to secondary server
[root@ha-primary ha.d]# scp authkeys ha.cf haresources root@172.18.10.228:/etc/ha.d/
1.7 Start Heartbeat and verify VIP binding
# Primary server ha-primary
[root@ha-primary ha.d]# service heartbeat start
[root@ha-primary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.227/24 brd 172.18.10.255 scope global eth0
inet 172.18.10.200/24 brd 172.18.10.255 scope global secondary eth0
# Secondary server ha-secondary
[root@ha-secondary ha.d]# service heartbeat start
[root@ha-secondary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.228/24 brd 172.18.10.255 scope global eth0
inet 172.18.10.201/24 brd 172.18.10.255 scope global secondary eth0
1.8 Simulate primary server failure
# Primary server ha-primary
[root@ha-primary ha.d]# service heartbeat stop
[root@ha-primary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.227/24 brd 172.18.10.255 scope global eth0
# Secondary server ha-secondary
[root@ha-secondary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.228/24 brd 172.18.10.255 scope global eth0
inet 172.18.10.201/24 brd 172.18.10.255 scope global secondary eth0
inet 172.18.10.200/24 brd 172.18.10.255 scope global secondary eth0
The secondary server takes over VIP 172.18.10.200 and httpd service.
1.9 Simulate primary server recovery
# Primary server ha-primary
[root@ha-primary ha.d]# service heartbeat start
[root@ha-primary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.227/24 brd 172.18.10.255 scope global eth0
inet 172.18.10.200/24 brd 172.18.10.255 scope global secondary eth0
# Secondary server ha-secondary
[root@ha-secondary ha.d]# ip addr|grep 172.18.10
inet 172.18.10.228/24 brd 172.18.10.255 scope global eth0
inet 172.18.10.201/24 brd 172.18.10.255 scope global secondary eth0
Primary server regains control of VIP 172.18.10.200 and httpd service.
2. Configure HTTPD for Testing
2.1 Create test pages
# Primary server ha-primary
[root@ha-primary ~]# echo "Service running on: 172.18.10.227" > /var/www/html/index.html
[root@ha-primary ~]# curl 172.18.10.227
Service running on: 172.18.10.227
# Secondary server ha-secondary
[root@ha-secondary ~]# echo "Service running on: 172.18.10.228" > /var/www/html/index.html
[root@ha-secondary ~]# curl 172.18.10.228
Service running on: 172.18.10.228
2.2 Test web application access via VIP
Access 172.18.10.200 in browser - shows primary server content
Simulate primary failure - browser shows secondary server content
Recover primary server - browser shows primary server content again