FreeBSD用CARP实现负载均衡

1. 准备4台新装的系统,每台都有2个网卡。

2. 重新编译内核,加入对carp等的支持

fb01# cd /usr/src/sys/i386/conf

fb01# cp GENERIC UNIXNOTES

3. 加下面的内容到新的UNIXNOTES内核编辑文件

#new add
device          carp            #Common Address Redundancy Protocol
device          pf              #PF OpenBSD packet-filter firewall
device          pflog           #logging support interface for PF
device          pfsync          #synchronization interface for PF
options         ALTQ
options         ALTQ_CBQ
options         ALTQ_RED
options         ALTQ_RIO
options         ALTQ_HFSC
options         ALTQ_PRIQ
options         ALTQ_NOPCC

4. Build 和install 新的itkylin.com内核参考

5. 在一台机器上编译好内核之后直接scp到别的机器上启用即可.

fb01# cd /boot/kernel

6. 在每台机器上用port安装apache,可以只在一台机器上编译,然后scp到其他上直接pkgadd.

7. 编辑每台机器的rc.conf 文件

fb01# cat /etc/rc.conf
sshd_enable=”YES”
ifconfig_le0=”inet 172.20.6.201  netmask 255.255.255.0″
ifconfig_le1=”inet 192.168.1.201  netmask 255.255.255.0″
defaultrouter=”172.20.6.2″
hostname=”fb01.abc.com”
cloned_interfaces=”carp0 carp1 carp2 carp3″
ifconfig_carp0=”vhid 1 advbase 5 advskew 10 pass aaa123 172.20.6.205/24″
ifconfig_carp1=”vhid 2 advbase 5 advskew 20 pass aaa123 172.20.6.206/24″
ifconfig_carp2=”vhid 3 advbase 5 advskew 30 pass aaa123 172.20.6.207/24″
ifconfig_carp3=”vhid 4 advbase 5 advskew 40 pass aaa123 172.20.6.208/24″
ifconfig_pfsync0=”up syncif le1″
apache22_enable=”YES”

fb02# cat /etc/rc.conf
sshd_enable=”YES”
ifconfig_le0=”inet 172.20.6.202  netmask 255.255.255.0″
ifconfig_le1=”inet 192.168.1.202  netmask 255.255.255.0″
defaultrouter=”172.20.6.2″
hostname=”fb02.abc.com”
cloned_interfaces=”carp0 carp1 carp2 carp3″
ifconfig_carp0=”vhid 1 advbase 5 advskew 40 pass aaa123 172.20.6.205/24″
ifconfig_carp1=”vhid 2 advbase 5 advskew 10 pass aaa123 172.20.6.206/24″
ifconfig_carp2=”vhid 3 advbase 5 advskew 20 pass aaa123 172.20.6.207/24″
ifconfig_carp3=”vhid 4 advbase 5 advskew 30 pass aaa123 172.20.6.208/24″
ifconfig_pfsync0=”up syncif le1″

fb03# cat /etc/rc.conf
gateway_enable=”YES”
hostname=”fb01.abc.com”
sshd_enable=”YES”
ifconfig_le0=”inet 172.20.6.203  netmask 255.255.255.0″
ifconfig_le1=”inet 192.168.1.203  netmask 255.255.255.0″
defaultrouter=”172.20.6.2″
hostname=”fb03.abc.com”
apache22_enable=”YES”
cloned_interfaces=”carp0 carp1 carp2 carp3″
ifconfig_carp0=”vhid 1 advbase 5 advskew 30 pass aaa123 172.20.6.205/24″
ifconfig_carp1=”vhid 2 advbase 5 advskew 40 pass aaa123 172.20.6.206/24″
ifconfig_carp2=”vhid 3 advbase 5 advskew 10 pass aaa123 172.20.6.207/24″
ifconfig_carp3=”vhid 4 advbase 5 advskew 20 pass aaa123 172.20.6.208/24″
ifconfig_pfsync0=”up syncif le1″

fb04# cat /etc/rc.conf
gateway_enable=”YES”
hostname=”fb01.abc.com”
sshd_enable=”YES”
ifconfig_le0=”inet 172.20.6.204  netmask 255.255.255.0″
ifconfig_le1=”inet 192.168.1.204  netmask 255.255.255.0″
defaultrouter=”172.20.6.2″
hostname=”fb04.abc.com”
apache22_enable=”YES”
cloned_interfaces=”carp0 carp1 carp2 carp3″
ifconfig_carp0=”vhid 1 advbase 5 advskew 20 pass aaa123 172.20.6.205/24″
ifconfig_carp1=”vhid 2 advbase 5 advskew 30 pass aaa123 172.20.6.206/24″
ifconfig_carp2=”vhid 3 advbase 5 advskew 40 pass aaa123 172.20.6.207/24″
ifconfig_carp3=”vhid 4 advbase 5 advskew 10 pass aaa123 172.20.6.208/24″
ifconfig_pfsync0=”up syncif le1″

8. 为每个apache编写主页文件,类似下面

fb01# cat /usr/local/www/apache22/data/index.html
<html><body><h1>201 It works!</h1></body></html>

9. 为每个机器加入下面语句,以是carp级别高会自动拿回master

fb04# cat /etc/sysctl.conf
net.inet.carp.preempt=1
net.inet.carp.arpbalance=1

10. 各个机器重启之后的网卡状态,只看carp部分

fb01# ifconfig

carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.205 netmask 0xffffff00
carp: MASTER vhid 1 advbase 5 advskew 10
carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.206 netmask 0xffffff00
carp: BACKUP vhid 2 advbase 5 advskew 20
carp2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.207 netmask 0xffffff00
carp: BACKUP vhid 3 advbase 5 advskew 30
carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.208 netmask 0xffffff00
carp: BACKUP vhid 4 advbase 5 advskew 40

fb02# ifconfig

carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.205 netmask 0xffffff00
carp: BACKUP vhid 1 advbase 5 advskew 40
carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.206 netmask 0xffffff00
carp: MASTER vhid 2 advbase 5 advskew 10
carp2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.207 netmask 0xffffff00
carp: BACKUP vhid 3 advbase 5 advskew 20
carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.208 netmask 0xffffff00
carp: BACKUP vhid 4 advbase 5 advskew 30

fb03# ifconfig

carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.205 netmask 0xffffff00
carp: BACKUP vhid 1 advbase 5 advskew 30
carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.206 netmask 0xffffff00
carp: BACKUP vhid 2 advbase 5 advskew 40
carp2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.207 netmask 0xffffff00
carp: MASTER vhid 3 advbase 5 advskew 10
carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.208 netmask 0xffffff00
carp: BACKUP vhid 4 advbase 5 advskew 20

fb04# ifconfig

carp0: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.205 netmask 0xffffff00
carp: BACKUP vhid 1 advbase 5 advskew 20
carp1: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.206 netmask 0xffffff00
carp: BACKUP vhid 2 advbase 5 advskew 30
carp2: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.207 netmask 0xffffff00
carp: BACKUP vhid 3 advbase 5 advskew 40
carp3: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
inet 172.20.6.208 netmask 0xffffff00
carp: MASTER vhid 4 advbase 5 advskew 10

11. 通过别的机器来检查carp工作情况

分别访问http://172.20.6.205 ;http://172.20.6.206; http://172.20.6.207;http://172.20.6.208

得到返回页分别是fb01,fb02,fb03,fb04 上的apache页面,因为4台同时在线,即访问每carp优先级最高的。

12. 重启fb01之后,在访问http://172.20.6.205,将有别的机器接管master