Circuit breakers serve as a protective mechanism to prevent system overload during partial failures. Degradation strategies generally fall into two categories: active degradation, where non-core services are intentionally scaled down during high-load events like promotions, and passive degradation, which includes fallbacks triggered by circuit breaking or rate limiting.

Circuit Breaker Triggered Fallback

When a circuit breaker is activated, it prevents subsequent requests from reaching the downstream service, allowing the system to recover. The mechanism relies on specific thresholds: for instance, if the failure rate exceeds 50% over a rolling window of 20 requests within 10 seconds, the breaker trips. Once tripped, the circuit remains open for a defined period (e.g., 5 seconds), during which all incoming requests are immediately routed to the fallback logic instead of the remote endpoint.

@HystrixCommand(
    commandProperties = {
        @HystrixProperty(name = "circuitBreaker.enabled", value = "true"),
        @HystrixProperty(name = "circuitBreaker.requestVolumeThreshold", value = "5"),
        @HystrixProperty(name = "circuitBreaker.sleepWindowInMilliseconds", value = "5000"),
        @HystrixProperty(name = "circuitBreaker.errorThresholdPercentage", value = "50")
    },
    fallbackMethod = "getSystemHealthStatus",
    threadPoolKey = "fulfillment-service-pool"
)
@GetMapping("/api/v1/orders/{id}")
public String retrieveOrderDetails(@PathVariable("id") int identifier) {
    if (identifier % 2 == 0) {
        return "Operation Successful";
    }
    // Simulating a remote call that may hang
    return restTemplate.getForObject("http://remote-service:8082/inventory", String.class);
}

public String getSystemHealthStatus(int identifier) {
    return "Service Unavailable: Please try again later";
}

Timeout Triggered Fallback

To prevent threads from being blocked indefinitely by slow responses, Hystrix allows configuring execution timeouts. If the operation does not complete within the specified duration, the thread is interrupted, and the fallback method is invoked.

@HystrixCommand(
    fallbackMethod = "handleLatency",
    commandProperties = {
        @HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "3000")
    }
)
public String performRemoteOperation() {
    // Logic susceptible to high latency
    return externalServiceClient.getData();
}

public String handleLatency() {
    return "Request timed out";
}

Resource Isolation Strategies

Hystrix provides two primary isolation modes: Thread Pool Isolation and Semaphore Isolation. These strategies limit the impact of failing dependencies on the rest of the application.

Semaphore Isolation

Semaphore isolation acts as a concurrency limiter, essentially functioning as a rate limiter for the container's threads (e.g., Tomcat threads). It uses a counter to track concurrent requests; once the limit is reached, new requests are rejected and trigger the fallback. This approach avoids the overhead of thread context switching but cannot interrupt a blocked request if the downstream service hangs without returning.

@HystrixCommand(
    fallbackMethod = "semaphoreFallback",
    commandProperties = {
        @HystrixProperty(
            name = "execution.isolation.strategy", 
            value = "SEMAPHORE"
        ),
        @HystrixProperty(
            name = "execution.isolation.semaphore.maxConcurrentRequests", 
            value = "100"
        )
    }
)
public String processRequestWithSemaphore() {
    return "Processing";
}

public String semaphoreFallback() {
    return "Concurrency limit reached";
}

Thread Pool Isolation

Thread pool isolation provides a higher degree of protection by executing commands in a separate thread pool dedicated to a specific dependency. This allows the system to time out and reject requests even if the downstream service is unresponsive. Although this introduces overhead due to thread management and context switching, it ensures that a single slow dependency cannot consume all resources of the calling application.

@HystrixCommand(
    groupKey = "fulfillment-service",
    commandKey = "fetchInventory",
    threadPoolKey = "inventory-pool",
    threadPoolProperties = {
        @HystrixProperty(name = "coreSize", value = "30"),
        @HystrixProperty(name = "maxQueueSize", value = "100"),
        @HystrixProperty(name = "keepAliveTimeMinutes", value = "2"),
        @HystrixProperty(name = "queueSizeRejectionThreshold", value = "15")
    },
    fallbackMethod = "isolationFallback"
)
public String fetchInventoryData() {
    return restTemplate.getForObject("http://inventory-service/api/items", String.class);
}

public String isolationFallback() {
    return "Thread pool resource exhausted";
}

Request Collapsing

Purpose

Request collapsing reduces the load on backend systems by batching multiple individual requests into a single aggregated request. Instead of executing single-row SQL queries or multiple Redis calls, the system can combine them into a batch operation or utilize Redis pipelines.

Implementation Approaches

In Spring Cloud, this is achieved using @HystrixCollapser in conjunction with @HystrixCommand. Alternatively, one can implement custom batching logic using JDK queues and scheduled thread pools when a service governance framework is not present.

Example

public class UserBatchCommand {

    @HystrixCollapser(
        batchMethod = "retrieveUsersBatch",
        collapserProperties = {
            @HystrixProperty(name = "timerDelayInMilliseconds", value = "100")
        }
    )
    public Future<User> findUser(String userId) {
        return null; // Implementation handled by the batch method
    }

    @HystrixCommand
    public List<User> retrieveUsersBatch(List<String> userIds) {
        List<User> userList = new ArrayList<>();
        for (String id : userIds) {
            userList.add(new User(id, "User Name: " + id));
        }
        return userList;
    }

}