Load Balancing Fundamentals

Global Server Load Balancing (GSLB)

GSLB operates across geographically distributed infrastructure with the following components:

• Global Coordinator: Oversees the entire traffic distribution system
• Regional Dispatcher: Manages traffic within a specific geographic area
• Application Controller: Routes requests to appropriate service endpoints
• Application Nodes: Actual servers hosting the application

The global coordinator supervises regional dispatchers, while the application controller manages individual application nodes.

Request Flow Example:

1. Client A sends a request to the regional dispatcher
2. The regional dispatcher responds with the optimal server address
3. Client A directly accesses the assigned application node
4. The application node processes and returns the response

Server Load Balancing (SLB)

In SLB configurations, dispatcher and service nodes exist within the same logical unit, providing excellent latency and responsiveness for time-sensitive operations. Nginx implements SLB at its core.

Layer 4 vs Layer 7 Load Balancing

Layer 4 (Transport Layer) Load Balancing:

Operating at the transport layer of the OSI model, L4 load balancing handles TCP/IP packet forwarding. Since it works at the network level without examining application content, it offers superior performance with minimal processing overhead.

Layer 7 (Application Layer) Load Balancing:

L7 load balancing operates at the application layer, enabling deep inspection and manipulation of HTTP headers, request bodies, and protocol-specific content. This allows for advanced routing decisions based on URL patterns, cookies, and application-specific rules. Nginx excels as a Layer 7 load balancer.

Nginx Load Balancer Configuration

Basic Upstream Directive

The upstream block defines a virtual server pool that Nginx uses for distributing incoming requests.

Dircetive Syntax:

upstream name {
    # server definitions
}

• Valid context: http

Configuratino Examples

Backend Server Definitions

Web Server One:

server {
    listen       8001;
    server_name  localhost;

    location / {
        root   /var/www/app1;
        index  index.html index.htm;
    }
}

Web Server Two:

server {
    listen       8002;
    server_name  localhost;

    location / {
        root   /var/www/app2;
        index  index.html index.htm;
    }
}

Web Server Three:

server {
    listen       8003;
    server_name  localhost;

    location / {
        root   /var/www/app3;
        index  index.html index.htm;
    }
}

Main Proxy Configuration

upstream backend_pool {
    server 10.0.0.101:8001;
    server 10.0.0.101:8002;
    server 10.0.0.101:8003;
}

server {
    listen       80;
    server_name  api.example.com;

    resolver  8.8.8.8;

    location / {
        proxy_pass http://backend_pool;
        proxy_redirect default;
    }
}

Advanced Upstream Configuration

upstream application_servers {
    server primary.example.com weight=5;
    server secondary.example.com:8080;
    server unix:/var/run/backend.sock;
    
    server standby1.example.com:8080 backup;
    server standby2.example.com:8080 backup;
}

Server Health and Failover Settings

upstream production_pool {
    server 10.0.0.101:8001 down;
    server 10.0.0.101:8002 backup;
    server 10.0.0.101:8003 max_fails=1 fail_timeout=10s;
}

server {
    listen       80;
    server_name  api.example.com;

    access_log  /var/log/nginx/api.access.log  main;
    resolver  8.8.8.8;

    location / {
        proxy_pass http://production_pool;
        include proxy_params;
    }
}

Parameter Definitions:

• down: Marks server as permanently unavailable
• backup: Designates server as standby; only activates when all primary servers fail
• max_fails: Number of failed attempts before server is considered unavailable
• fail_timeout: Time window for counting failures

Load Balancing Algorithms

Weighted Round Robin

upstream weighted_pool {
    server 10.0.0.101:8001;
    server 10.0.0.101:8002;
    server 10.0.0.101:8003 weight=5;
}

server {
    listen       80;
    server_name  api.example.com;

    access_log  /var/log/nginx/api.access.log  main;
    resolver  8.8.8.8;

    location / {
        proxy_pass http://weighted_pool;
        include proxy_params;
    }
}

With this configuration, out of 7 requests, 5 go to port 8003, while 1 each goes to ports 8001 and 8002.

IP Hash Algorithm

upstream iphash_pool {
    ip_hash;
    server 10.0.0.101:8001;
    server 10.0.0.101:8002;
    server 10.0.0.101:8003;
}

server {
    listen       80;
    server_name  api.example.com;

    access_log  /var/log/nginx/api.access.log  main;
    resolver  8.8.8.8;

    location / {
        proxy_pass http://iphash_pool;
        include proxy_params;
    }
}

Behavior: The ip_hash directive ensures requests from the same client IP consistently route to the same backend server.

URL Hash Algorithm

upstream urlhash_pool {
    hash $request_uri;
    server 10.0.0.101:8001;
    server 10.0.0.101:8002;
    server 10.0.0.101:8003;
}

server {
    listen       80;
    server_name  api.example.com;

    access_log  /var/log/nginx/api.access.log  main;
    resolver  8.8.8.8;

    location / {
        proxy_pass http://urlhash_pool;
        include proxy_params;
    }
}

Behavior: The hash directive distributes requests based on the request URI, ensuring identical URLs from any client IP consistently reach the same backend server.

Example Routing: