Implementing Circuit Breaking and Degradation with Sentinel in Microservices

Circuit breaking is a protective mechanism that triggers specific actions when certain thresholds are exceeded. This concept appears in various domains:

  1. Stock market circuit breakers: For example, U.S. equities halt trading for 15 minutes at 7%, 13%, and 20% declines—giving investors time to pause and assess.

  2. Electrical fuses: When current exceeds the maximum capacity of a wire, the fuse blows, cutting off all power to prevent fires. This protects property and life.

In software architecture, similar problems emerge. In microservices, a single request may traverse multiple services, creating a long call chain. If one node experiences network failure and responds slowly, it can block the entire chain's response. Under high load, delayed responses from one dependency can saturate all server resources within seconds, rapidly consuming system resources, leading to outages and potentially a service avalanche.

To prevent this, software systems adopt circuit breaking: if a target service responds slowly or times out excessively, subsequent requests are immediately blocked and instead return a fallback response, freeing resources. Once the target service recovers, the circuit closes dynamically.

Sentinel Circuit Breaking Basics

Sentinel's circuit breaker degrades a resource when it becomes unstable—e.g., high latency or elevated error rates. Requests are then fast-failed to avoid cascading failures. Degraded resources are automatically blocked for a defined time window.

How is instability determined?

  1. Slow call ratio (SLOW_REQUEST_RATIO)
  2. Error ratio (ERROR_RATIO)
  3. Error count (ERROR_COUNT)

Sentinel provides the DegradeRule class for configuring these strategies. Key properties:

  • resource: Resource name
  • count: Threshold (for error ratio/count modes: the threshold value; for slow call ratio: the maximum acceptable RT in milliseconds)
  • grade: Circuit breaker mode (SLOW_REQUEST_RATIO, ERROR_RATIO, or ERROR_COUNT)
  • timeWindow: Duration (seconds) the circuit stays open before entering half-open state

Slow Call Ratio (SLOW_REQUEST_RATIO)

If, within a certain number of requests over a period, a specified proportion exceeds a response time threshold, the circuit opens. After the time window, the breaker enters a half-open probing state. If the next request completes within the RT threshold, the circuit closes; otherwise, it reopens.

Example: Trigger circuit break if within 1 minute, atleast 10 requests per second have 20% exceeding 3 seconds, with a 5-second break window.

DegradeRule rule = new DegradeRule(RESOURCE_KEY)
    .setGrade(CircuitBreakerStrategy.SLOW_REQUEST_RATIO.getType())
    .setCount(3000)                     // Max allowed RT (ms)
    .setTimeWindow(5)                   // Retry timeout (s)
    .setSlowRatioThreshold(0.2)         // 20% slow ratio
    .setMinRequestAmount(10)            // Minimum requests to trigger evaluation
    .setStatIntervalMs(60000);          // Statistical interval (ms)

Error Ratio (ERROR_RATIO)

When requests per second >= 5 and the ratio of errors exceeds the threshold, the breaker opens.

  • grade: CircuitBreakerStrategy.ERROR_RATIO
  • count: Error ratio threshold (0.0–1.0, i.e., 0%–100%)
  • timeWindow: Break duration (seconds)
  • minRequestAmount: Minimum request count before triggering (default 5)

Error Count (ERROR_COUNT)

When the number of errors in a statistical interval exceeds the threshold, the circuit opens. After the time window, a successful request closes the breaker; otherwise, it reopens.

Note: The statistical interval is minute-level. If timeWindow < 60s, the circuit may re-enter the open state after closing.

  • grade: CircuitBreakerStrategy.ERROR_COUNT
  • count: Error count threshold
  • timeWindow: Break duration (seconds)
  • statIntervalMs: Statistical interval in ms (e.g., 60 × 1000)

Integrating Sentinel with Spring Cloud Alibaba for Circuit Breaking

public class DegradeExample {
    private static final String RESOURCE = "hello";

    public static void main(String[] args) {
        initDegradeRule();

        for (int i = 0; i < 2000; i++) {
            try (Entry entry = SphU.entry(RESOURCE)) {
                TimeUnit.MILLISECONDS.sleep(ThreadLocalRandom.current().nextInt(10, 100));
                System.out.println("current" + i + " request success");
            } catch (BlockException | InterruptedException e) {
                System.out.println("current" + i + ", Occur BlockException: " + e.getMessage() + "");
            }
        }
    }

    private static void initDegradeRule() {
        List<DegradeRule> rules = new ArrayList<>();
        DegradeRule rule = new DegradeRule(RESOURCE)
                .setGrade(CircuitBreakerStrategy.SLOW_REQUEST_RATIO.getType())
                .setCount(20)
                .setTimeWindow(5)
                .setSlowRatioThreshold(0.2)
                .setMinRequestAmount(10)
                .setStatIntervalMs(1000);

        rules.add(rule);
        DegradeRuleManager.loadRules(rules);
        EventObserverRegistry.getInstance().addStateChangeObserver("logging",
            (prevState, newState, rule1, snapshotValue) -> {
                if (newState == CircuitBreaker.State.OPEN) {
                    System.out.println(prevState.name() + " enters OPEN state at " + TimeUtil.currentTimeMillis() + ", snapshot: " + snapshotValue);
                } else {
                    System.err.println("PrevState: " + prevState.name() + ", NewState: " + newState.name() + " at " + TimeUtil.currentTimeMillis());
                }
            });
    }
}

Dynamic Rule Configuration with Nacos

In the earlier examples, rules are loaded locally via SPI extension points. However, in a Spring Cloud Alibaba ecosystem, we can store rules in a configuration center like Nacos.

Sentinel supports two ways to modify rules:

  • Direct API calls (e.g., loadRules())
  • DataSource adapters for different data sources (recommended for production)

Manual API modification is straightforward but typically used only for testing:

FlowRuleManager.loadRules(List<FlowRule> rules);
DegradeRuleManager.loadRules(List<DegradeRule> rules);

For production, use dynamic rule sources. The architecture:

  • Sentinel Dashboard or Config Center pushes rules to a centralized config center (Nacos, Zookeeper, etc.).
  • Clients implement ReadableDataSource to listen for changes and update rules in real-time.

DataSource Types

Pull mode: Clients periodically poll a rule store (database, file, etc.). Simple but less timely.

Push mode: Clients subscribe to changes via listeners (Nacos, Zookeeper, etc.). Better real-time and consistency.

Sentinel supports:

  • Pull: Dynamic file data source, Consul, Eureka
  • Push: ZooKeeper, Redis, Nacos, Apollo, etcd

Implementing pull mode: Subclass AutoRefreshDataSource and implement readSource().

Implementing push mode: Subclass AbstractDataSource, add listeners in the constructor, and implement readSource(). For example, Nacos-based data source.

To push application-level rules to a config center, you may need to customize the Sentinel dashboard.

Dynamic Rate Limiting via Configuration in Spring Cloud Alibaba

Instead of SPI-based InitFunc, you can configure rules directly in application.properties:

spring.cloud.sentinel.datasource.ds1.nacos.server-addr=127.0.0.1:8848
spring.cloud.sentinel.datasource.ds1.nacos.data-id=com.example.userserviceapi.IHelloService
spring.cloud.sentinel.datasource.ds1.nacos.group-id=SENTINEL_GROUP
spring.cloud.sentinel.datasource.ds1.nacos.data-type=json
spring.cloud.sentinel.datasource.ds1.nacos.rule-type=flow
spring.cloud.sentinel.datasource.ds1.nacos.username=nacos
spring.cloud.sentinel.datasource.ds1.nacos.password=nacos

After adding these configurations, you can remove the custom InitFunc. Load testing the interface yields the same results.

Note: By default, rules configured in Sentinel Dashboard are not persisted. To persist them, you need to modify the Dashboard to back rules to a store like Nacos.

Tags: Sentinel circuit-breaker degradation spring-cloud-alibaba Nacos

Posted on Wed, 22 Jul 2026 16:51:07 +0000 by Leadwing