Testing Default Request Handling Capacity

A standard Spring Boot project created with minimal configuration will be analyzed to determine its concurrent request handling capabilities. The test setup uses Spring Boot 2.7.13 with only essential dependencies included.

The test controller accepts requests and holds the thread for an extended period to identify how many threads are available for processing:

@RestController
public class TestController {

    @GetMapping("/process")
    public void getTest(int num) throws Exception {
        log.info("Request received on thread: {} with num={}", Thread.currentThread().getName(), num);
        TimeUnit.HOURS.sleep(1);
    }
}

The application.properties file remains empty, representing a vanilla Spring Boot installation.

To identify the maximum concurrent requests, a client test launches multiple threads simultaneously:

public class LoadTest {
    public static void main(String[] args) {
        for (int i = 0; i < 1000; i++) {
            int requestId = i;
            new Thread(() -> {
                HttpUtil.get("127.0.0.1:8080/process?num=" + requestId);
            }).start();
        }
        Thread.yield();
    }
}

Analyzing the Results

Running the test against the default Spring Boot application shows exactly 200 requests being processed simultaneously. This raises the question: where does this 200 come from?

The Spring Boot starter includes Tomcat as its default embedded contaienr. Thread dump analysis reveals threads in sleep state with call stacks containing:

at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1791)
at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(TaskThread.java:61)

This confirms Tomcat manages the request processing threads.

Tomcat Thread Pool Configuration

Debugging the Tomcat thread pool reveals the following key parameters:

• corePoolSize: 10
• maximumPoolSize: 200
• queue capacity: Integer.MAX_VALUE

The maximum pool size of 200 is hardcoded in Tomcat's default configuration.

Why 200 Instead of Queue Behavior?

Standard JDK thread pool behavior follows this sequence: core threads → queue → max threads. However, Tomcat implements a different algorithm through its custom TaskQueue class.

The TaskQueue.offer() method contains the logic that changes this behavior. When the parent (the Tomcat thread pool) exists, the queue checks whether the current pool size equals the maximum pool size. If true, tasks are added to the queue rather than triggering new thread creation.

The critical logic appears in the queue's offer method: if the current pool size is less than the maximum, the method returns false, forcing the thread pool to create new threads instead of queuing tasks.

JDK Thread Pool Logic: core threads → queue → max threads

Tomcat Thread Pool Logic: core threads → max threads → queue

This explains why 200 requests are processed immediately rather than 10 concurrent plus 990 queued.

import org.apache.tomcat.util.threads.TaskQueue;
import org.apache.tomcat.util.threads.TaskThreadFactory;
import org.apache.tomcat.util.threads.ThreadPoolExecutor;

import java.util.concurrent.TimeUnit;

public class ThreadPoolComparison {


    public static void main(String[] args) throws InterruptedException {
        String threadPrefix = "worker-";
        boolean daemonThread = true;
        TaskQueue taskQueue = new TaskQueue(300);
        TaskThreadFactory factory = new TaskThreadFactory(threadPrefix, daemonThread, Thread.NORM_PRIORITY);
        
        ThreadPoolExecutor executor = new ThreadPoolExecutor(5,
                150, 60000, TimeUnit.MILLISECONDS, taskQueue, factory);
        taskQueue.setParent(executor);
        
        for (int i = 0; i < 300; i++) {
            try {
                executor.execute(() -> {
                    printStatus(executor, "Task Created");
                    try {
                        TimeUnit.SECONDS.sleep(2);
                    } catch (InterruptedException e) {
                        e.printStackTrace();
                    }
                });
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        Thread.currentThread().join();
    }

    private static void printStatus(ThreadPoolExecutor executor, String label) {
        TaskQueue queue = (TaskQueue) executor.getQueue();
        System.out.println(Thread.currentThread().getName() + "-" + label + "-:" +
                "Core:" + executor.getCorePoolSize() +
                "\tActive:" + executor.getActiveCount() +
                "\tMax:" + executor.getMaximumPoolSize() +
                "\tTotal:" + executor.getTaskCount() +
                "\tQueued:" + queue.size() +
                "\tRemaining:" + queue.remainingCapacity());
    }
}

Removing the taskQueue.setParent(executor) line changes behavior back to standard JDK thread pool logic.

Other Tomcat Parameters

The max-connections parameter (default: 8192) controls the maximum number of connections held in the accept queue. Setting it to 10 limits concurrent connections to 10.

The accept-count parameter (default: 100) determines how many connections can wait in the OS queue when all accept threads are busy.

Switching to Undertow

Replacing Tomcat with Undertow requires only changing Maven dependencies:

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
    <exclusions>
        <exclusion>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-tomcat</artifactId>
        </exclusion>
    </exclusions>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-undertow</artifactId>
</dependency>

With Undertow, the same test reveals only 48 concurrent requests being processed. This value equals CPU core count multiplied by 8, which Undertow uses as its default worker thread calculation.

Impact of @Async

Adding @Async to an endpoint dramatically changes the concurrency model:

@Async
@GetMapping("/process")
public void getTest(int num) throws Exception {
    // processing logic
}

The async executor has a core pool size of 8, reducing concurrent processing capacity from 200 to 8.

Key Takeaways

The answer to "how many requests can a Spring Boot app handle" depends on multiple factors:

1. Web container type (Tomcat, Jetty, Undertow)
2. Thread pool configuration (core size, max size, queue capacity)
3. Network settings (max-connections, accept-count)
4. Async processing (separate thread pools)

For Tomcat with default settings: 200 concurrent requests.

For Undertow with default settings: CPU × 8 concurrent requests.

Understanding these container-specific behaviors is essential when discussing request handling capacity in Spring Boot applications.