Background

As digital platforms expand and user bases grow exponentially, systems encounter escalating pressure from concurrent requests. A single server can quick become a bottleneck, unable to process incoming traffic efficiently. This challenge necessitates architectural approaches that can distribute workload across multiple machines while maintaining responsiveness and reliability.

Core Concept

Load balancing acts as an intermediary layer between users and backend servers. By intelligently routing incoming requests, it ensures no single server bears excessive burden. Modern load balancers like LVS, Nginx, and HAProxy monitor server health and exclude unresponsive nodes from the routing pool.

The fundamental routing strategies include:

• Random Selection
• Round Robin
• Weighted Random
• Weighted Round Robin
• Consistent Hashing (IP-based)
• Least Connections

Implementation Examples

Data Model

First, define the core entities representing backend nodes and their configuration:

package com.example.lb.core;

public class BackendNode {
    private String nodeAddress;
    private int capacity;
    private int currentConnections;

    public BackendNode(String nodeAddress, int capacity) {
        this.nodeAddress = nodeAddress;
        this.capacity = capacity;
        this.currentConnections = 0;
    }

    public String getNodeAddress() {
        return nodeAddress;
    }

    public int getCapacity() {
        return capacity;
    }

    public int getCurrentConnections() {
        return currentConnections;
    }

    public void incrementConnections() {
        this.currentConnections++;
    }

    public void decrementConnections() {
        if (this.currentConnections > 0) {
            this.currentConnections--;
        }
    }

    @Override
    public String toString() {
        return "BackendNode{address='" + nodeAddress + "', capacity=" + capacity + ", connections=" + currentConnections + "}";
    }
}

package com.example.lb.core;

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;

public class NodeRegistry {
    public static final Map<String, BackendNode> nodeMap = new ConcurrentHashMap<>();

    static {
        nodeMap.put("10.0.1.101", new BackendNode("10.0.1.101", 1));
        nodeMap.put("10.0.1.102", new BackendNode("10.0.1.102", 2));
        nodeMap.put("10.0.1.103", new BackendNode("10.0.1.103", 3));
        nodeMap.put("10.0.1.104", new BackendNode("10.0.1.104", 4));
    }
}

Random Selection

This approach selects a server purely by chance. With four availlable nodes, the algorithm generates a random index and returns the corresponding server address:

package com.example.lb.strategies;

import com.example.lb.core.BackendNode;
import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;

public class RandomSelection {

    public static String selectServer() {
        List<String> addresses = new ArrayList<>(NodeRegistry.nodeMap.keySet());
        int targetIndex = new Random().nextInt(addresses.size());
        return addresses.get(targetIndex);
    }

    public static void main(String[] args) {
        System.out.println("=== Random Selection Test ===");
        for (int i = 0; i < 10; i++) {
            System.out.println("Selected: " + selectServer());
        }
    }
}

Round Robin

Sequential distribution cycles through servers in a fixed order. A counter increments and wraps around using modulo arithmetic to stay within bounds:

package com.example.lb.strategies;

import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;

public class RoundRobinSelection {

    private static int counter = 0;

    public static String selectServer() {
        List<String> addresses = new ArrayList<>(NodeRegistry.nodeMap.keySet());
        String selected = addresses.get(counter);
        counter = (counter + 1) % addresses.size();
        return selected;
    }

    public static void main(String[] args) {
        System.out.println("=== Round Robin Test ===");
        for (int i = 0; i < 10; i++) {
            System.out.println("Selected: " + selectServer());
        }
    }
}

Weighted Random Selection

Servers receive different selection probabilities based on their capacity. Higher capacity nodes appear multiple times in the candidate pool:

package com.example.lb.strategies;

import com.example.lb.core.BackendNode;
import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.Random;

public class WeightedRandomSelection {

    public static String selectServer() {
        List<String> candidates = new ArrayList<>();
        
        for (Map.Entry<String, BackendNode> entry : NodeRegistry.nodeMap.entrySet()) {
            BackendNode node = entry.getValue();
            for (int i = 0; i < node.getCapacity(); i++) {
                candidates.add(entry.getKey());
            }
        }
        
        return candidates.get(new Random().nextInt(candidates.size()));
    }

    public static void main(String[] args) {
        System.out.println("=== Weighted Random Test ===");
        for (int i = 0; i < 15; i++) {
            System.out.println("Selected: " + selectServer());
        }
    }
}

Weighted Round Robin

Combining rotation with weights ensures proportional distribution while maintaining predictability:

package com.example.lb.strategies;

import com.example.lb.core.BackendNode;
import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;
import java.util.Map;

public class WeightedRoundRobinSelection {

    private static int position = 0;

    public static String selectServer() {
        List<String> candidates = new ArrayList<>();
        
        for (Map.Entry<String, BackendNode> entry : NodeRegistry.nodeMap.entrySet()) {
            BackendNode node = entry.getValue();
            for (int i = 0; i < node.getCapacity(); i++) {
                candidates.add(entry.getKey());
            }
        }
        
        String selected = candidates.get(position);
        position = (position + 1) % candidates.size();
        return selected;
    }

    public static void main(String[] args) {
        System.out.println("=== Weighted Round Robin Test ===");
        for (int i = 0; i < 15; i++) {
            System.out.println("Selected: " + selectServer());
        }
    }
}

Consistent Hashing

This technique maps client IPs to specific servers, providing session persistence. The same source address always routes to the same backend:

package com.example.lb.strategies;

import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;

public class ConsistentHashSelection {

    public static String selectServer(String clientIp) {
        List<String> addresses = new ArrayList<>(NodeRegistry.nodeMap.keySet());
        int hash = clientIp.hashCode();
        int targetIndex = Math.abs(hash) % addresses.size();
        return addresses.get(targetIndex);
    }

    public static void main(String[] args) {
        String client1 = "192.168.1.50";
        String client2 = "10.10.10.25";
        
        System.out.println("=== Consistent Hash Test ===");
        System.out.println("\nClient 1 (" + client1 + "):");
        for (int i = 0; i < 5; i++) {
            System.out.println("  Request " + (i + 1) + " -> " + selectServer(client1));
        }
        
        System.out.println("\nClient 2 (" + client2 + "):");
        for (int i = 0; i < 5; i++) {
            System.out.println("  Request " + (i + 1) + " -> " + selectServer(client2));
        }
    }
}

Least Connections

Routing to the server with the fewest active connections allows dynamic load distribution based on current utilization:

package com.example.lb.strategies;

import com.example.lb.core.BackendNode;
import com.example.lb.core.NodeRegistry;

import java.util.ArrayList;
import java.util.List;

public class LeastConnectionsSelection {

    public static String selectServer() {
        List<String> addresses = new ArrayList<>(NodeRegistry.nodeMap.keySet());
        
        BackendNode bestCandidate = null;
        int lowestCount = Integer.MAX_VALUE;

        for (String address : addresses) {
            BackendNode node = NodeRegistry.nodeMap.get(address);
            if (node.getCurrentConnections() < lowestCount) {
                lowestCount = node.getCurrentConnections();
                bestCandidate = node;
            }
        }

        if (bestCandidate != null) {
            bestCandidate.incrementConnections();
            System.out.println("Pool status: " + NodeRegistry.nodeMap);
            return bestCandidate.getNodeAddress();
        }
        
        return null;
    }

    public static void main(String[] args) {
        System.out.println("=== Least Connections Test ===");
        for (int i = 0; i < 8; i++) {
            String selected = selectServer();
            System.out.println("Request " + (i + 1) + " -> " + selected);
        }
    }
}

Summary

Each algorithm addresses different operational requirements. Rendom and round robin work well for homogeneous server pools with equal processing power. Weighted variants suit heterogeneous environments where machines have varying capacities. Consistent hashing maintains client-server affinity for stateful applications. Least connections excel in scenarios with varying request processing times.