Basic Concepts

Load balancing solutions encompass various approaches including DNS-based domain rotation, client-side load distribution, application-layer load balancing, and IP address-based scheduling. This document focuses on LVS, a layer-4 load balancer built upon transport protocols like TCP/UDP. LVS stands for Linux Virtual Server, representing a 4-layer load balancer in modern terminology. Originally initiated by Dr. Zhang WenSong, it's now integrated into standard Linux kernels since version 2.4. Prior to this, kernel recompilation was required for LVS functionality. LVS aims to create high-performance, highly available server clusters with excellent reliability, scalability, and manageability, delivering optimal service performance at minimal cost. Since its inception in 1998, LVS has matured into a robust solution supporting scalable, highly available network services such as WWW, caching, DNS, FTP, email, and multimedia streaming. Major websites like Linux.com, Real.com, and SourceForge.net utilize LVS clusters. ### Main Features

LVS implements three IP load balancing techniques (VS/NAT, VS/TUN, VS/DR) and ten scheduling algorithms (rr, wrr, lc, wlc, lblc, lblcr, dh, sh, sed, nq). ### Advantages

Functional Characteristics

1. High load handling capability as it operates at Layer 4 without generating traffic 2. Minimal configuration complexity reducing human error probability 3. Stable operation with complete failover solutions 4. IO performance unaffected by traffic volume 5. Broad application compatibility for all services #### High Availability

LVS leverages kernel-level operations providing exceptional processing capabilities. Individual node failures don't impact overall system functionality while maintaining balanced load distribution. Systems support millions of concurrent connections. With 100Mbps NICs and VS/TUN/DR techniques, throughput can reach 1Gbps; with 1Gbps NICs, up to 10Gbps is achievable. #### High Reliability

Extensively adopted in enterprise and academic environments, LVS has proven stability through practical applications. Many systems operate continuously without restarts, demonstrating superior stability and reliability. #### Deployment Environment

Director servers support Linux and FreeBSD, while real servers can run on any TCP/IP supported OS including Linux, Unix variants, macOS, and Windows. LVS supports most TCP and UDP protocols including HTTP, HTTPS, FTP, SMTP, DNS, NTP, and media streaming protocols. ### Limitations

Requires network infrastructure support Limited protocol analysis capabilities Hardware implementation would offer better performance ### Similar Solutions

Commercial load balancers like NetScaler, F5, Radware, and Array provide comprehensive four-to-seven layer capabilities but at significant cost. Open-source alternatives include LVS, Nginx, and HAProxy. Nginx operates at Layer 7 with HTTP-based load balancing features but lacks URL-based health checks and has limited session management. HAProxy also functions at Layer 7 with enhanced session persistence, cookie handling, and more sophisticated load balancing strategies compared to Nginx. LVS Architecture

Operational Modes

LVS implements IP load balancing through the IPVS module. IPVS serves as the core component installed on Director Servers, creating a virtual IP address (VIP) that users access. Requests first reach the VIP before being distributed among Real Servers. Three primary load balancing mechanisms exist: NAT, TUN, and DR. #### Forwarding Mechanisms

VS/NAT (Network Address Translation)

Requests are modified to redirect target addresses to Real Servers while preserving source addresses. Both request and response packets traverse the Director, creating potential bottlenecks with increasing load. ##### VS/TUN (IP Tunneling)

Uses IP tunneling to forward requests with out requiring response packet redirection through the Director. Real Servers respond directly to clients, improving throughput for large-scale deployments. ##### VS/DR (Direct Routing)

Modifies MAC addresses instead of IP addresses for direct communication between Director and Real Servers, eliminating IP tunnel overhead. Requires shared network segments between Director and Real Servers. ### Scheduling Algorithms

LVS employs multiple scheduling methods: 1. Round Robin (rr) - Simple sequential distribution 2. Weighted Round Robin (wrr) - Considers server capacity weights 3. Destination Hashing (dh) - Static mapping based on destination IP 4. Source Hashing (sh) - Static mapping based on source IP 5. Least Connections (lc) - Distributes to server with fewest active connections 6. Weighted Least Connections (wlc) - Considers both connections and weights 7. Locality-Based Least Connections (lblc) - Optimizes cache locality 8. Locality-Based Least Connections with Replication (lblcr) - Manages hot content replication 9. Shortest Expected Delay (sed) - Predictive load balancing 10. Never Queue (nq) - Direct allocation to idle servers LVS Configuration

Installation Requirements

Linux kernel versions 2.6 and later include IPVS functionality. Older kernels require manual integration and recompilation. The ipvsadm tool manages IPVS operations. ### Configuration Commands

Key commands include: - Service management (-A, -E, -D) - Real server management (-a, -e, -d) - Table operations (-L, -C, -R, -S) - Load balancing modes (-g, -i, -m) - Weight settings (-w) - Persistence options (-p) ### NAT Configuration Example

# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# Clear existing rules
ipvsadm -C

# Add virtual service
ipvsadm -A -t 192.168.1.75:81 -s rr

# Add real servers
ipvsadm -a -t 192.168.1.75:81 -r 192.168.1.100:8080 -m
ipvsadm -a -t 192.168.1.75:81 -r 192.168.1.76:8080 -m

IP Tunnel Configuration

# Configure tunnel interface
ifconfig tunl0 192.168.1.240 netmask 255.255.255.0 broadcast 192.168.1.255

# Add services
ipvsadm -A -t 192.168.1.240:8080 -s rr
ipvsadm -a -t 192.168.1.240:8080 -r 192.168.1.76:8080 -i
ipvsadm -a -t 192.168.1.240:8080 -r 192.168.1.100:8080 -i

Direct Routing Configuration

# Configure virtual IP on Director
ifconfig eth0:0 192.168.1.240 netmask 255.255.255.0 broadcast 192.168.1.255

# Add services
ipvsadm -A -t 192.168.1.240:8080 -s rr
ipvsadm -a -t 192.168.1.240:8080 -r 192.168.1.76:8080 -g
ipvsadm -a -t 192.168.1.240:8080 -r 192.168.1.100:8080 -g

Performance Optimization

Hardware Considerations

Optimized server hardware featuring network-focused configurations enhances LVS performance. The LVS author provides reference hardware specifications. ### Kernel Tuning

The Taobao kernel offers substantial LVS performance improvements through optimized configurations, including adjusting CONFIG_IP_VS_TAB_BITS from 12 to 20 for enhanced hash table sizing. High Availability Implementation

Single Point of Failure

LVS single points of failure can be addressed using dual-server setups with Keepalived for automatic failover. Keepalived monitors health status via VRRP protocol and automatically transfers VIP to standby servers upon detection of failures. ### Keepalived Configuration Example

vrrp_instance VI_1 {
    state MASTER
    interface eth0
    virtual_router_id 51
    priority 100
    advert_int 1
    
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    
    virtual_ipaddress {
        192.168.1.240
    }
}

virtual_server 192.168.1.240 8080 {
    delay_loop 6
    lb_algo wrr
    lb_kind DR
    persistence_timeout 50
    
    protocol TCP
    
    real_server 192.168.1.100 8080 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
    
    real_server 192.168.1.76 8080 {
        weight 1
        TCP_CHECK {
            connect_timeout 3
        }
    }
}

Advanced Deployment Scenarios

Scalability Solutions

For horizontal scaling, DNS round-robin can distribute traffic across multiple LVS instances. OSPF integration with LVS enables dynamic routing adjustments. ### Monitoring and Troubleshooting

Effective troubleshooting involves monitoring: - LVS status with watch ipvsadm -ln - Real server availability - Packet capture analysis - System logs (tail -f /var/log/messages) References

Comprehensive documentation includes IBM developerWorks tutorials, official LVS documentation, and community resources covering installation, configuration, and advanced deployment scenarios.