Circuit breaking is a protective mechanism that triggers specific actions when certain thresholds are exceeded. This concept appears in various domains:
-
Stock market circuit breakers: For example, U.S. equities halt trading for 15 minutes at 7%, 13%, and 20% declines—giving investors time to pause and assess.
-
Electrical fuses: When current exceeds the maximum capacity of a wire, the fuse blows, cutting off all power to prevent fires. This protects property and life.
In software architecture, similar problems emerge. In microservices, a single request may traverse multiple services, creating a long call chain. If one node experiences network failure and responds slowly, it can block the entire chain's response. Under high load, delayed responses from one dependency can saturate all server resources within seconds, rapidly consuming system resources, leading to outages and potentially a service avalanche.
To prevent this, software systems adopt circuit breaking: if a target service responds slowly or times out excessively, subsequent requests are immediately blocked and instead return a fallback response, freeing resources. Once the target service recovers, the circuit closes dynamically.
Sentinel Circuit Breaking Basics
Sentinel's circuit breaker degrades a resource when it becomes unstable—e.g., high latency or elevated error rates. Requests are then fast-failed to avoid cascading failures. Degraded resources are automatically blocked for a defined time window.
How is instability determined?
- Slow call ratio (
SLOW_REQUEST_RATIO) - Error ratio (
ERROR_RATIO) - Error count (
ERROR_COUNT)
Sentinel provides the DegradeRule class for configuring these strategies. Key properties:
resource: Resource namecount: Threshold (for error ratio/count modes: the threshold value; for slow call ratio: the maximum acceptable RT in milliseconds)grade: Circuit breaker mode (SLOW_REQUEST_RATIO,ERROR_RATIO, orERROR_COUNT)timeWindow: Duration (seconds) the circuit stays open before entering half-open state
Slow Call Ratio (SLOW_REQUEST_RATIO)
If, within a certain number of requests over a period, a specified proportion exceeds a response time threshold, the circuit opens. After the time window, the breaker enters a half-open probing state. If the next request completes within the RT threshold, the circuit closes; otherwise, it reopens.
Example: Trigger circuit break if within 1 minute, atleast 10 requests per second have 20% exceeding 3 seconds, with a 5-second break window.
DegradeRule rule = new DegradeRule(RESOURCE_KEY)
.setGrade(CircuitBreakerStrategy.SLOW_REQUEST_RATIO.getType())
.setCount(3000) // Max allowed RT (ms)
.setTimeWindow(5) // Retry timeout (s)
.setSlowRatioThreshold(0.2) // 20% slow ratio
.setMinRequestAmount(10) // Minimum requests to trigger evaluation
.setStatIntervalMs(60000); // Statistical interval (ms)
Error Ratio (ERROR_RATIO)
When requests per second >= 5 and the ratio of errors exceeds the threshold, the breaker opens.
grade:CircuitBreakerStrategy.ERROR_RATIOcount: Error ratio threshold (0.0–1.0, i.e., 0%–100%)timeWindow: Break duration (seconds)minRequestAmount: Minimum request count before triggering (default 5)
Error Count (ERROR_COUNT)
When the number of errors in a statistical interval exceeds the threshold, the circuit opens. After the time window, a successful request closes the breaker; otherwise, it reopens.
Note: The statistical interval is minute-level. If timeWindow < 60s, the circuit may re-enter the open state after closing.
grade:CircuitBreakerStrategy.ERROR_COUNTcount: Error count thresholdtimeWindow: Break duration (seconds)statIntervalMs: Statistical interval in ms (e.g., 60 × 1000)
Integrating Sentinel with Spring Cloud Alibaba for Circuit Breaking
public class DegradeExample {
private static final String RESOURCE = "hello";
public static void main(String[] args) {
initDegradeRule();
for (int i = 0; i < 2000; i++) {
try (Entry entry = SphU.entry(RESOURCE)) {
TimeUnit.MILLISECONDS.sleep(ThreadLocalRandom.current().nextInt(10, 100));
System.out.println("current" + i + " request success");
} catch (BlockException | InterruptedException e) {
System.out.println("current" + i + ", Occur BlockException: " + e.getMessage() + "");
}
}
}
private static void initDegradeRule() {
List<DegradeRule> rules = new ArrayList<>();
DegradeRule rule = new DegradeRule(RESOURCE)
.setGrade(CircuitBreakerStrategy.SLOW_REQUEST_RATIO.getType())
.setCount(20)
.setTimeWindow(5)
.setSlowRatioThreshold(0.2)
.setMinRequestAmount(10)
.setStatIntervalMs(1000);
rules.add(rule);
DegradeRuleManager.loadRules(rules);
EventObserverRegistry.getInstance().addStateChangeObserver("logging",
(prevState, newState, rule1, snapshotValue) -> {
if (newState == CircuitBreaker.State.OPEN) {
System.out.println(prevState.name() + " enters OPEN state at " + TimeUtil.currentTimeMillis() + ", snapshot: " + snapshotValue);
} else {
System.err.println("PrevState: " + prevState.name() + ", NewState: " + newState.name() + " at " + TimeUtil.currentTimeMillis());
}
});
}
}
Dynamic Rule Configuration with Nacos
In the earlier examples, rules are loaded locally via SPI extension points. However, in a Spring Cloud Alibaba ecosystem, we can store rules in a configuration center like Nacos.
Sentinel supports two ways to modify rules:
- Direct API calls (e.g.,
loadRules()) - DataSource adapters for different data sources (recommended for production)
Manual API modification is straightforward but typically used only for testing:
FlowRuleManager.loadRules(List<FlowRule> rules);
DegradeRuleManager.loadRules(List<DegradeRule> rules);
For production, use dynamic rule sources. The architecture:
- Sentinel Dashboard or Config Center pushes rules to a centralized config center (Nacos, Zookeeper, etc.).
- Clients implement
ReadableDataSourceto listen for changes and update rules in real-time.
DataSource Types
Pull mode: Clients periodically poll a rule store (database, file, etc.). Simple but less timely.
Push mode: Clients subscribe to changes via listeners (Nacos, Zookeeper, etc.). Better real-time and consistency.
Sentinel supports:
- Pull: Dynamic file data source, Consul, Eureka
- Push: ZooKeeper, Redis, Nacos, Apollo, etcd
Implementing pull mode: Subclass AutoRefreshDataSource and implement readSource().
Implementing push mode: Subclass AbstractDataSource, add listeners in the constructor, and implement readSource(). For example, Nacos-based data source.
To push application-level rules to a config center, you may need to customize the Sentinel dashboard.
Dynamic Rate Limiting via Configuration in Spring Cloud Alibaba
Instead of SPI-based InitFunc, you can configure rules directly in application.properties:
spring.cloud.sentinel.datasource.ds1.nacos.server-addr=127.0.0.1:8848
spring.cloud.sentinel.datasource.ds1.nacos.data-id=com.example.userserviceapi.IHelloService
spring.cloud.sentinel.datasource.ds1.nacos.group-id=SENTINEL_GROUP
spring.cloud.sentinel.datasource.ds1.nacos.data-type=json
spring.cloud.sentinel.datasource.ds1.nacos.rule-type=flow
spring.cloud.sentinel.datasource.ds1.nacos.username=nacos
spring.cloud.sentinel.datasource.ds1.nacos.password=nacos
After adding these configurations, you can remove the custom InitFunc. Load testing the interface yields the same results.
Note: By default, rules configured in Sentinel Dashboard are not persisted. To persist them, you need to modify the Dashboard to back rules to a store like Nacos.