3.4 Packet Capture in Linux

ARP Resolution Process: Wireshark packet capture workflow

# Install packet capture software
yum install -y wireshark

# Capture ARP packets on interface eth1
tshark -i eth1 -f arp

# Sample output:
# 1 0.000000000 Vmware_2e:aa:48 -> Broadcast    ARP 42 Who has 172.16.1.51? Tell 172.16.1.61
# 2 0.000263929 Vmware_c7:5d:dd -> Vmware_2e:aa:48 ARP 60 172.16.1.51 is at 00:0c:29:c7:5d:dd

# Check ARP table
arp -n
# Sample output:
# Address                  HWtype  HWaddress           Flags Mask            Iface
# 172.16.1.51              ether   00:0c:29:c7:5d:dd   C                     eth1
# 10.0.0.1                 ether   00:50:56:c0:00:08   C                     eth0
# 10.0.0.51                ether   00:0c:29:c7:5d:d3   C                     eth0
# 10.0.0.254               ether   00:50:56:e1:eb:34   C                     eth0

tshark (Wireshark) options:

• -i: Specify network interface (defaults to first non-loopback interface)
• -f: Apply capture filter
• -w: Save captured packets to file

For filter syntax, refer to: man wireshark-filter

3.5 ARP Protocol Layer Analysis

ARP (Address Resolution Protocol) operates at Layer 2 (Data Link Layer) of the OSI model, despite dealing with IP addresses (Layer 3). It resolves IP addresses to MAC addresses for local network communication.

Further reading:

• TCP/IP详解: https://blog.csdn.net/zhangliangzi/article/details/52551089
• arp图解1: https://www.cnblogs.com/csguo/p/7527303.html#
• arp图解2: https://www.cnblogs.com/csguo/p/7527073.html

3.6 ARP Spoofing

ARP spoofing involves sending falsified ARP messages over a local network to link an attacker's MAC address with the IP address of a legitimate computer or server on the network.

Prevention methods:

1. Static MAC-IP address binding
2. Enterprise security software implementation

4. LVS Introduction

LVS (Linux Virtual Server), created by Wensong Zhang, is a high-performance virtual server cluster system that implements request forwarding based on destination address and port. It doesn't generate traffic itself but only forwards user requests, making it currently the best-performing load balancing cluster system.

Related resources:

• https://www.cnblogs.com/aubin/p/7718739.html
• Four working mode principles: https://blog.csdn.net/Running_free/article/details/77981201

LVS has been integrated in to the Linux kernel as ip_vs and can be controlled via keepalived (configuration file) or ipvsadm (command).

4.1 LVS Terminology

• CIP: Client IP
• VIP: Virtual IP - external service IP, typically bound to a domain name
• DIP: Director IP - communicates with internal host RIP
• RIP: Real Server IP - actual service-providing host
• Director: Load balancer host
• VS: Virtual Server
• RS: Real Server

4.2 LVS Cluster Modes

• LVS-NAT: Modifies destination IP of request packets (DNAT)
• LVS-DR: Manipulates MAC address encapsulation
• LVS-TUN: Adds new IP header to original request packet
• LVS-FULLNAT: Modifies both source and destination IP of request packets

4.3 LVS-DR Mode (Layer 2)

Principle Overview

When a client sends a request to VIP, the packet [source CIP; destination VIP] is routed to the Director. The Director doesn't modify source or destination IP but changes the destination MAC to the RS's MAC after scheduling. The RS receives the packet, identifies the destination IP as its local interface, processes it, and returns the response directly to the client with source IP as its local interface and destination IP as the client's IP.

Data Flow:

• CIP → VIP: Source IP (client) → Director (modifies destination MAC to RS) → Destination IP (RS local interface)
• VIP → CIP: Source IP (RS local interface) → via public network → Destination IP (client)

Suppressing ARP on RS

LVS-DR Mode Characteristics:

1. In DR mode, LVS modifies the destination MAC address to the backend RS server's MAC
2. Responses are sent directly from RS servers to users
3. LVS and backend RS must be in the same LAN segment
4. RS servers require public IP addresses
5. RS node VIP binding (lo:vip/32) and ARP suppression are necessary
6. DR mode operates at Layer 2 (MAC), while NAT mode operates at Layer 3 (IP) and Layer 4 (port)

4.4 LVS-NAT Mode (Layers 3 and 4)

Principle Overview

The client sends a request to VIP, and the Director selects an RS after scheduling, performing port or IP mapping between the client and RS. The RS returns data to the Director, which then forwards it to the client.

Data Flow:

• CIP → VIP → RIP: Source IP (client) → Director (maps client IP to RS) → Destination IP (RS IP)
• RIP → VIP → CIP: Source IP (RS IP) → Director (maps RS IP to VIP) → Destination IP (client IP)

Characteristics:

1. NAT (Network Address Translation)
2. Requires kernel forwarding: net.ipv4.ip_forward = 1
3. Supports port forwarding (e.g., 80 → 8080)
4. LVS and RS can be in different networks

4.5 LVS-TUNNEL Mode

Principle Overview

Client sends request to VIP. Director selects an RS and encapsulates the packet [source DIP; destination RIP [source CIP; destination VIP]] using tunneling. The RS decapsulates the packet, identifies the destination IP as its local interface, processes the request, and returns the response directly to the client.

Characteristics:

1. Both LVS and RS require VIP
2. Doesn't become a bottleneck
3. Limited by packet size

Advantages/Disadvantages:

• Fast performance
• Less secure, vulnerable to DOS attacks

4.6 FULL NAT Forwarding Mode

FULLNAT Principle:

Similar to NAT mode, both inbound and outbound packets pass through LVS. The main difference is that no special configuration is needed on RS or switches. FULLNAT introduces a local address, converting CIP-VIP to LIP→RIP, where both LIP and RIP are internal IDC IPs that can communicate across VLANs.

Origin:

For large-scale networks like Taobao, standard LVS couldn't meet requirements due to:

1. High deployment costs of traditional modes
2. Lack of DDOS defense compared to commercial load balancers
3. Performance limitations in active-standby deployment

NAT vs FULLNAT Comparison:

FULLNAT performs both source and destination IP modification (SNAT+DNAT), unlike NAT which only performs two address translations.

FULLNAT Advantages/Disadvantages:

The main issue with FULLNAT is that RS cannot obtain the client IP. TOA (TCP Option Address) was developed to solve this by placing the client address in TCP Options.

5. LVS-DR Mode Setup

5.1 Environment Preparation

Install required software on lb01 and lb02:

yum install -y ipvsadm

Disable nginx and keepalived services:

systemctl stop nginx.service keepalived.service
systemctl disable nginx.service keepalived.service

Configure web01 and web02:

# Add static pages
curl 10.0.0.[7-8]/index.html

# Nginx configuration (identical on both servers)
cat /etc/nginx/nginx.conf
user nginx;
worker_processes 1;
events {
    worker_connections 1024;
}
http {
    include /etc/nginx/mime.types;
    default_type application/octet-stream;
    include /etc/nginx/conf.d/*.conf;
}

cat /etc/nginx/conf.d/01-blog.conf
server {
    listen 80;
    server_name blog.example.com;
    root /var/www/blog;
    location / {
        index index.html;
    }
}

nginx -t
systemctl restart nginx

Add static web pages:

# On web01
cat /var/www/blog/index.html
web01
<h1>Nginx server 10.0.0.7 web01</h1>

# On web02
cat /var/www/blog/index.html
web02
<h1>Nginx server 10.0.0.8 web02</h1>

Verify web server accessibility:

curl 10.0.0.[7-8]

5.2 LVS-DR Mode Deployment

Load kernel module:

modprobe ip_vs
ipvsadm -ln
lsmod | grep ip_vs

5.3 LVS Configuration Commands

# Add virtual IP
ip addr add 10.0.0.3/24 dev eth0 label eth0:0

# Clear existing rules
ipvsadm -C

# Set TCP timeout
ipvsadm --set 30 5 60

# Create virtual service with weighted round-robin scheduling
ipvsadm -A -t 10.0.0.3:80 -s wrr -p 20

# Add real servers with DR mode
ipvsadm -a -t 10.0.0.3:80 -r 10.0.0.7:80 -g -w 1
ipvsadm -a -t 10.0.0.3:80 -r 10.0.0.8:80 -g -w 1

# View LVS configuration
ipvsadm -ln