Understanding Load Balancers

A load balancer is a network device or software mechanism that distributes incoming network traffic across multiple backend servers, enabling optimal resource utilization while enhancing system availability and performance.

Load Balancer Types

Load balancing can be categorized into two main types:

• Server-side load balancing: Implemented on the server side using tools like Nginx, HAProxy, or F5
• Client-side load balancing: Embedded within the client application, such as Spring Cloud LoadBalancer

Common Load Balancing Strategies

Both server-side and client-side load balancers employ similar distribution strategies:

• Round Robin: Distributes requests sequentially across available servers
• Random Selection: Randomly selects backend servers for request processing
• Least Connections: Routes requests to servers with the fewest active connections
• IP Hash: Uses client IP address hashing to ensure consistent server selection
• Weighted Round Robin: Distributes requests based on server capacity weights
• Weighted Random: Random selection weighted by server capacity
• Least Response Time: Routes to servers with the fastest resposne times

Spring Cloud LoadBalancer Overview

Spring Cloud LoadBalancer serves as the official replacement for Netflix Ribbon in the Spring Cloud ecosystem. It provides client-side load balancing capabilities when integrated with Spring Cloud OpenFeign and service discovery mechanisms like Nacos.

Default Round Robin Strategy

Spring Cloud LoadBalancer employs round robin as its default strategy, implemented in the LoadBalancerClientConfiguration class:

@Configuration
public class LoadBalancerClientConfiguration {
    
    @Bean
    @ConditionalOnMissingBean
    public ReactorLoadBalancer<ServiceInstance> 
    reactorServiceInstanceLoadBalancer(Environment env,
                                      LoadBalancerClientFactory factory) {
        String serviceName = env.getProperty(LoadBalancerClientFactory.PROPERTY_NAME);
        return new RoundRobinLoadBalancer(
            factory.getLazyProvider(serviceName, ServiceInstanceListSupplier.class),
            serviceName
        );
    }
}

The core round robin implementation:

private Response<ServiceInstance> getInstanceResponse(List<ServiceInstance> instances) {
    if (instances.isEmpty()) {
        log.warn("No available servers for service: " + serviceId);
        return new EmptyResponse();
    }
    
    if (instances.size() == 1) {
        return new DefaultResponse(instances.get(0));
    }
    
    int position = this.counter.incrementAndGet() & Integer.MAX_VALUE;
    ServiceInstance selectedInstance = instances.get(position % instances.size());
    
    return new DefaultResponse(selectedInstance);
}

Random Load Balancing Configuration

Spring Cloud LoadBalancer supports two built-in strategies: round robin and rendom selection.

Creating Random Load Balancer

@Configuration
public class RandomBalancerConfiguration {
    
    @Bean
    public ReactorLoadBalancer<ServiceInstance> randomBalancer(
            Environment env, LoadBalancerClientFactory factory) {
        String serviceName = env.getProperty("loadbalancer.client.name");
        return new RandomLoadBalancer(
            factory.getLazyProvider(serviceName, ServiceInstanceListSupplier.class),
            serviceName
        );
    }
}

Service-Specific Configuration

@Service
@FeignClient("user-service")
@LoadBalancerClient(name = "user-service", 
                   configuration = RandomBalancerConfiguration.class)
public interface UserClient {
    @RequestMapping("/user/details")
    String getUserDetails(@RequestParam("id") Integer userId);
}

Global Configuration

@SpringBootApplication
@EnableFeignClients
@LoadBalancerClients(defaultConfiguration = RandomBalancerConfiguration.class)
public class Application {
    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }
}

Nacos Weight-Based Load Balancing

Nacos supports weight-based load balancing and geographic routing. Configure Spring Cloud LoadBalancer to use Nacos strategies:

Nacos Load Balancer Configuration

@Configuration
@LoadBalancerClients(defaultConfiguration = NacosBalancerConfig.class)
public class NacosBalancerConfig {
    
    @Autowired
    private NacosDiscoveryProperties discoveryProps;
    
    @Bean
    public ReactorLoadBalancer<ServiceInstance> nacosBalancer(
            Environment env, LoadBalancerClientFactory factory) {
        String serviceName = env.getProperty("loadbalancer.client.name");
        return new NacosLoadBalancer(
            factory.getLazyProvider(serviceName, ServiceInstanceListSupplier.class),
            serviceName,
            discoveryProps
        );
    }
}

Custom Load Balancer Implementation

Implement custom load balancing in three steps:

Custom Load Balancer Class

public class CustomIPHashBalancer implements ReactorServiceInstanceLoadBalancer {
    
    private final String serviceIdentifier;
    private ObjectProvider<ServiceInstanceListSupplier> supplierProvider;
    
    public CustomIPHashBalancer(ObjectProvider<ServiceInstanceListSupplier> supplier, 
                               String serviceId) {
        this.serviceIdentifier = serviceId;
        this.supplierProvider = supplier;
    }
    
    public Mono<Response<ServiceInstance>> choose(Request request) {
        ServiceInstanceListSupplier supplier = supplierProvider
            .getIfAvailable(NoopServiceInstanceListSupplier::new);
        return supplier.get(request).next()
            .map(instances -> processSelection(supplier, instances));
    }
    
    private Response<ServiceInstance> processSelection(
            ServiceInstanceListSupplier supplier, List<ServiceInstance> instances) {
        Response<ServiceInstance> response = selectInstance(instances);
        if (supplier instanceof SelectedInstanceCallback && response.hasServer()) {
            ((SelectedInstanceCallback)supplier)
                .selectedServiceInstance(response.getServer());
        }
        return response;
    }
    
    private Response<ServiceInstance> selectInstance(List<ServiceInstance> instances) {
        if (instances.isEmpty()) {
            log.warn("No available servers for: " + serviceIdentifier);
            return new EmptyResponse();
        }
        
        ServletRequestAttributes attributes = (ServletRequestAttributes)
            RequestContextHolder.getRequestAttributes();
        HttpServletRequest httpRequest = attributes.getRequest();
        String clientIP = httpRequest.getRemoteAddr();
        
        int hashValue = clientIP.hashCode();
        int selectedIndex = Math.abs(hashValue % instances.size());
        ServiceInstance chosenInstance = instances.get(selectedIndex);
        
        return new DefaultResponse(chosenInstance);
    }
}

Custom Load Balancer Configuration

@Configuration
public class CustomBalancerConfig {
    
    @Bean
    public ReactorLoadBalancer<ServiceInstance> customBalancer(
            Environment env, LoadBalancerClientFactory factory) {
        String serviceName = env.getProperty("loadbalancer.client.name");
        return new CustomIPHashBalancer(
            factory.getLazyProvider(serviceName, ServiceInstanceListSupplier.class),
            serviceName
        );
    }
}

Caching Configuration

Spring Cloud LoadBalancer provides instance caching with default settings:

• Cache TTL: 35 seconds
• Cache capacity: 256 entries

Custom cache configuration:

loadbalancer:
  cache:
    enabled: true
    ttl: 10
    capacity: 1000

Technical Implementation

Spring Cloud LoadBalancer operates through core components:

ServiceInstanceListSupplier

public interface ServiceInstanceListSupplier {
    Flux<List<ServiceInstance>> get();
}

LoadBalancer Interface

public interface LoadBalancer<T> {
    Mono<Response<T>> choose(Request request);
}

Execution Flow

1. ServiceInstanceListSupplier retrieves available instances from registry
2. LoadBalancer applies selection strategy to choose instance
3. LoadBalancerClient executes request using selected instance