Full series of PHP video tutorials: Detailed PHP – http://www.xishuophp.com/
In production applications, a certain “Nginx+PHP+MySQL” interface data server plays a very important role. If the server is hard
software or Nginx, MySQL failure, and a short time
If it cannot be recovered internally, the consequences will be very serious. In order to avoid a single point of failure, I designed this solution, wrote the failover.sh script, and realized
Dual-machine mutual backup, fully automatic switching, and the failover time only takes a few seconds
ten seconds.
1. Two-machine mutual backup and fully automatic switching scheme:
1. Topology map:
2. Explanation:
(1), assuming that the external network domain name blog.zyan.cc is resolved to the external network virtual IP 72.249.146.214, and the internal network hosts are set to db10 corresponding to the internal network
Virtual IP 192.168.146.214
(2) By default, the host is bound to the internal and external network virtual IPs, and the standby machine is used as a backup. When MySQL, Nginx or the server of the host fails
In case of inaccessibility due to failure, the standby machine will automatically take over the internal and external network
Virtual IP. Both servers start the daemon process /usr/bin/nohup /bin/sh responsible for monitoring and automatically switching virtual IPs
/usr/local/webserver/failover/failover.sh 2>&1 > /dev/null
&
(3) The MySQL servers on the master and backup machines are master-slave and synchronize with each other. When the host is active (that is, the virtual IP is bound by the host
), read and write the host’s
MySQL, the data written to the main machine will be synchronized to the standby machine; when the standby machine is active, the MySQL of the standby machine will be read and written, and the data written to the standby machine will be synchronized
Step to the host (if the MySQL on the host dies, there is no
Synchronization method, after the recovery of MySQL on the host, the data will be automatically synchronized from the backup machine, and vice versa).
(4). When the host is active, it will set the
/data0/htdocs/ (webpage, program, image storage directory), /usr/local/webserver/php/etc/ (php.ini and other configuration files
head
directory), /usr/local/webserver/nginx/conf/ (Nginx configuration file directory) files are pushed to the backup through rsync
machine server
The corresponding directory (incremental push, the same file on the two servers will not be pushed repeatedly), on the contrary, if the standby machine is active, every 20
Seconds will try to push the file to the host. The configuration file of rsync see two
The /etc/rsyncd.conf of the server, the startup command of the rsync daemon is rsync –daemon
3. Automatic switching process
(1) The host is bound to the internal and external network virtual IPs by default. When the host’s MySQL and Nginx cannot be accessed or the server is down, the host’s
The failover.sh daemon automatically removes itself
Bound internal and external virtual IPs (if the failover.sh on the host is dead, it doesn’t matter if you can’t remove the virtual IP bound to you), on the backup machine
The failover.sh daemon will automatically
Automatically take over the internal and external network virtual IP originally bound to the standby machine, and send an ARPing packet to the internal and external network gateways to update the MAC, and take over forcibly.
(2) After the standby machine is bound to the virtual IP, it will send an ARPing packet to the internal and external network gateways to notify the gateway to update the MAC address of the virtual IP to that of the standby machine
MAC address, so as to ensure that the standby machine can be accessed in time through the virtual IP after the switchover.
(3) If the host’s MySQL and Nginx start up and normal access is restored, the failover.sh daemon process on the host will detect the
Whether the MySQL data on the machine has been
Completely synchronized from the backup machine. If the synchronization delay time is 0, the host will automatically take over the virtual IPs of the internal and external networks, and send ARPing packets to the internal and external networks.
The external network gateway, and the standby machine will automatically remove the internal and external network virtual
IP.
(4) The entire switching process is automatically completed by failover.sh without manual processing.
4. Precautions (very important):
(1). The files in the crontab are not automatically synchronized. If they are modified, they need to be modified manually on both servers.
(2), any soft connection established with ln -s in the /data0/htdocs/ directory, rsync will not automatically synchronize, if it is established on a server
If you set up a soft connection, you need to manually build the same soft connection on another server.
(3), if you want to delete some files or directories in the /data0/htdocs/ directory, you need to delete the active state (that is, bind the virtual
IP) file or directory on the server, and then delete the file or directory on the standby server.
(4), except /data0/htdocs/ (webpage, program, image storage directory), /usr/local/webserver/php/etc
/ (php.ini and other configuration file directories), /usr/local/webserver/nginx/conf/ (Nginx configuration file directory) outside the three directories
other of
Configuration modification needs to be modified on both servers.
2. Configuration documents and scripts:
1. The rsync configuration of the host and backup servers (the configuration is the same)
(1), rsync configuration file
vi /etc/rsyncd.conf
Enter some content and save:
quote
uid = root
gid = root
use chroot = no
max cOnnecions= 20
pid file = /var/run/rsyncd.pid
lock file= /var/run/rsync.lock
log file = /var/log/rsyncd.log
[data0_htdocs]
path = /data0/htdocs/
ignore errors
read Only= no
hosts allow = 192.168.146.0/24
hosts deny = 0.0.0.0/32
[php_etc]
path = /usr/local/webserver/php/etc/
ignore errors
read Only= no
hosts allow = 192.168.146.0/24
hosts deny = 0.0.0.0/32
[nginx_conf]
path = /usr/local/webserver/nginx/conf/
ignore errors
read Only= no
hosts allow = 192.168.146.0/24
hosts deny = 0.0.0.0/32
(2), start the rsync daemon process
/usr/bin/rsync –daemon
2. Two MySQL master-slave configuration
Here I won’t describe the configuration process of mutual master and slave in detail. If you don’t understand, you can search it on Google. one thing to point out
, please add the skip-name-resolve parameter to the my.cnf configuration file, and use the IP to verify the MySQL account.
3. The failover.sh daemon process for load monitoring and virtual IP automatic switching between the host and backup servers
(1) Start the failover.sh daemon process (in order to run automatically after booting, please add the following statement to the /etc/rc.local file
):
/usr/bin/nohup /bin/sh /usr/local/webserver/failover/failover.sh 2>&1>
/dev/null &
(2), stop the failover.sh daemon process:
ps -ef | grep failover.sh
The following information will be displayed:
root 15428 1 0 Nov17 ? 00:00:03 /bin/sh
/usr/local/webserver/failover/failover.sh
root 20123 6878 0 16:16 pts/2 00:00:00 grep failover.sh
Then kill the failover.sh process:
kill -9 15428
(3), failover.sh code content (please pay attention to the type setting in it, the master is set to master, and the standby is set to slave):
#!/bin/sh
LANG=C
date=$(date -d “today” +”%Y-%m-%d %H:%M:%S”)
#—————Configuration Information (Start)—————
#Type: host is set to master, standby is set to slave
type=”master”
#Host, backup switch log path
logfile=”/var/log/failover.log”
#MySQL executable file address, such as /usr/local/mysql/bin/mysql; MySQL username; password; port
mysql_bin=”/usr/local/webserver/mysql/bin/mysql”
mysql_username=”root”
mysql_password=”123456″
mysql_port=”3306″
#Intranet Gateway
gateway_eth0=”192.168.146.1″
#The real IP of the host intranet
rip_eth0_master=”192.168.146.213″
#The real IP of the internal network of the standby machine
rip_eth0_slave=”192.168.146.215″
#Virtual IP shared by the main machine and standby machine intranet
vip_eth0_share=”192.168.113.214″
#External network gateway
gateway_eth1=”72.249.146.193″
#Host external network real IP
rip_eth1_master=”72.249.146.213″
#The real IP of the external network of the standby machine
rip_eth1_slave=”72.249.146.215″
#Virtual IP shared by the host and the standby machine on the external network
vip_eth1_share=”72.249.146.214″
#—————Configuration information (end)—————
#Bind internal and external network virtual IP
function_bind_vip()
{
/sbin/ifconfig eth0:vip ${vip_eth0_share} broadcast ${vip_eth0_share} netmask 255.255.255.255
up
/sbin/route add -host ${vip_eth0_share} dev eth0:vip
/sbin/ifconfig eth1:vip ${vip_eth1_share} broadcast ${vip_eth1_share} netmask 255.255.255.255
up
/sbin/route add -host ${vip_eth1_share} dev eth1:vip
/usr/local/webserver/php/sbin/php-fpm reload
kill -USR1 `cat /usr/local/webserver/nginx/logs/nginx.pid`
/sbin/service crond start
}
#Release internal and external network virtual IP
function_remove_vip()
{
/sbin/ifconfig eth0:vip ${vip_eth0_share} broadcast ${vip_eth0_share} netmask 255.255.255.255
down
/sbin/ifconfig eth1:vip ${vip_eth1_share} broadcast ${vip_eth1_share} netmask 255.255.255.255
down
/sbin/service crond stop
}
#The function of pushing files from the main machine to the standby machine
function_rsync_master_to_slave()
{
/usr/bin/rsync -zrtuog /data0/htdocs/ ${rip_eth0_slave}::data0_htdocs/ > /dev/null
2>&1
/usr/bin/rsync -zrtuog /usr/local/webserver/php/etc/ ${rip_eth0_slave}::php_etc/ >
/dev/null 2>&1
/usr/bin/rsync -zrtuog /usr/local/webserver/nginx/conf/ ${rip_eth0_slave}::nginx_conf/ >
/dev/null 2>&1
}
#The function of pushing files from the standby machine to the host
function_rsync_slave_to_master()
{
/usr/bin/rsync -zrtuog /data0/htdocs/ ${rip_eth0_master}::data0_htdocs/ > /dev/null
2>&1
/usr/bin/rsync -zrtuog /usr/local/webserver/php/etc/ ${rip_eth0_master}::php_etc/ >
/dev/null 2>&1
/usr/bin/rsync -zrtuog /usr/local/webserver/nginx/conf/ ${rip_eth0_master}::nginx_conf/ >
/dev/null 2>&1
}
#Virtual IP ARPing
function_vip_arping()
{
/sbin/arping -I eth0 -c 3 -s ${vip_eth0_share} ${gateway_eth0} > /dev/null 2>&1
/sbin/arping -I eth1 -c 3 -s ${vip_eth1_share} ${gateway_eth1} > /dev/null 2>&1
}
while true
do
#Check virtual IP with HTTP protocol
if
(curl -m 30 -G http://${vip_eth1_share}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${vip_eth0_share}” -e “show slave status\G” >
/dev/null 2>&1)
then
#Get the server intranet IP bound to the intranet VIP
eth0_active_server=$(${mysql_bin}
-u “${mysql_username}” -p “${mysql_password}” -P “${mysql_port}”
-h “${vip_eth0_share}” -e “show slave status\G” | grep
“Master_Host” | awk -F ‘: ‘ ‘{printf $2}’)
#If intranet VIP=host intranet IP (Master_Host in the host MySQL shows the domain name or IP of the standby machine), and this machine is the host
if [ “${eth0_active_server}” = “${rip_eth0_slave}” ] && [ “${type}” = “master” ]
then
function_rsync_master_to_slave
function_vip_arping
#If Intranet VIP=Intranet IP of the standby machine (Master_Host in the MySQL of the standby machine shows the domain name or IP of the host)
elif [ “${eth0_active_server}” = “${rip_eth0_master}” ]
then
if
(curl -m 30 -G http://${rip_eth1_master}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P”${mysql_port}” -h”${rip_eth0_master}” -e”show slave status\G”
| grep “Seconds_Behind_Master: 0” > /dev/null 2>&1)
then
#If the host can be accessed, there is no delay in database synchronization, and this machine is the host, then the virtual IP is bound by this machine
if [ “${type}” = “master” ]
then
#If this machine is the host
function_bind_vip
function_vip_arping
echo “${date} host has bound virtual IP!(Type:1)” >> ${logfile}
else
#If this machine is a backup machine
function_remove_vip
echo “${date} standby machine has removed the virtual IP!(Type:2)” >> ${logfile}
fi
else
if [ “${type}” = “slave” ]
then
#If this machine is a backup machine
function_rsync_slave_to_master
function_vip_arping
fi
fi
fi
else
#When the virtual IP cannot be accessed, determine whether the host can access it
if
(curl -m 30 -G http://${rip_eth1_master}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${rip_eth0_master}” -e “show slave status\G” >
/dev/null 2>&1)
then
#If the host can be accessed, and this machine is the host, then the virtual IP is bound by this machine
if [ “${type}” = “master” ]
then
function_bind_vip
function_vip_arping
echo “${date} host has bound virtual IP!(Type:3)” >> ${logfile}
else
function_remove_vip
echo “${date} standby machine has removed the virtual IP!(Type:4)” >> ${logfile}
fi
elif
(curl -m 30 -G http://${rip_eth1_slave}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${rip_eth0_slave}” -e “show slave status\G” >
/dev/null 2>&1)
then
#If the main machine cannot be accessed but the standby machine can be accessed, and this machine is the standby machine, then the standby machine will bind the virtual IP
if [ “${type}” = “slave” ]
then
function_bind_vip
function_vip_arping
echo “${date} standby machine has bound virtual IP!(Type:5)” >> ${logfile}
else
function_remove_vip
echo “${date} host has removed virtual IP!(Type:6)” >> ${logfile}
fi
else
echo “${date} master and backup are all inaccessible!(Type:7)” >> ${logfile}
fi
fi
#Pause each cycle for 20 seconds (that is, detect once every 20 seconds)
sleep 20
done
eth1 -c 3 -s ${vip_eth1_share} ${gateway_eth1} > /dev/null 2>&1
}
while true
do
#Check virtual IP with HTTP protocol
if
(curl -m 30 -G http://${vip_eth1_share}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${vip_eth0_share}” -e “show slave status\G” >
/dev/null 2>&1)
then
#Get the server intranet IP bound to the intranet VIP
eth0_active_server=$(${mysql_bin}
-u “${mysql_username}” -p “${mysql_password}” -P “${mysql_port}”
-h “${vip_eth0_share}” -e “show slave status\G” | grep
“Master_Host” | awk -F ‘: ‘ ‘{printf $2}’)
#If intranet VIP=host intranet IP (Master_Host in the host MySQL shows the domain name or IP of the standby machine), and this machine is the host
if [ “${eth0_active_server}” = “${rip_eth0_slave}” ] && [ “${type}” = “master” ]
then
function_rsync_master_to_slave
function_vip_arping
#If Intranet VIP=Intranet IP of the standby machine (Master_Host in the MySQL of the standby machine shows the domain name or IP of the host)
elif [ “${eth0_active_server}” = “${rip_eth0_master}” ]
then
if
(curl -m 30 -G http://${rip_eth1_master}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P”${mysql_port}” -h”${rip_eth0_master}” -e”show slave status\G”
| grep “Seconds_Behind_Master: 0” > /dev/null 2>&1)
then
#If the host can be accessed, there is no delay in database synchronization, and this machine is the host, then the virtual IP is bound by this machine
if [ “${type}” = “master” ]
then
#If this machine is the host
function_bind_vip
function_vip_arping
echo “${date} host has bound virtual IP!(Type:1)” >> ${logfile}
else
#If this machine is a backup machine
function_remove_vip
echo “${date} standby machine has removed the virtual IP!(Type:2)” >> ${logfile}
fi
else
if [ “${type}” = “slave” ]
then
#If this machine is a backup machine
function_rsync_slave_to_master
function_vip_arping
fi
fi
fi
else
#When the virtual IP cannot be accessed, determine whether the host can access it
if
(curl -m 30 -G http://${rip_eth1_master}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${rip_eth0_master}” -e “show slave status\G” >
/dev/null 2>&1)
then
#If the host can be accessed, and this machine is the host, then the virtual IP is bound by this machine
if [ “${type}” = “master” ]
then
function_bind_vip
function_vip_arping
echo “${date} host has bound virtual IP!(Type:3)” >> ${logfile}
else
function_remove_vip
echo “${date} standby machine has removed the virtual IP!(Type:4)” >> ${logfile}
fi
elif
(curl -m 30 -G http://${rip_eth1_slave}/ > /dev/null 2>&1)
&& (${mysql_bin} -u”${mysql_username}” -p “${mysql_password}”
-P “${mysql_port}” -h “${rip_eth0_slave}” -e “show slave status\G” >
/dev/null 2>&1)
then
#If the main machine cannot be accessed but the standby machine can be accessed, and this machine is the standby machine, then the standby machine will bind the virtual IP
if [ “${type}” = “slave” ]
then
function_bind_vip
function_vip_arping
echo “${date} standby machine has bound virtual IP!(Type:5)” >> ${logfile}
else
function_remove_vip
echo “${date} host has removed virtual IP!(Type:6)” >> ${logfile}
fi
else
echo “${date} master and backup are all inaccessible!(Type:7)” >> ${logfile}
fi
fi
#Pause each cycle for 20 seconds (that is, detect once every 20 seconds)
sleep 20
done