Most developers use Tomcat daily, but few understand the execution strategy behind its thread pool. This article explores how Tomcat's thread pool differs from the standard JDK implementation, diving into the mechanism that allows Tomcat to utilize maximum threads before queuing tasks.

The JDK ThreadPoolExecutor Behavior

Before examining Tomcat's approach, let's recall how the standard JDK ThreadPoolExecutor operates. When you submit tasks to a JDK thread pool with core size 10 and maximum size 30, the pool first fills all core threads before placing additional tasks into the queue. Only when the queue is full does the pool create new threads up to the maximum.

This behavior might seem counterintuitive for scenarios where quick task execution matters more then queue management. Tomcat addresses this by implementing a different strategy—one that creates threads up to the maximum before using the queue.

Tomcat's StandardThreadExecutor

When configuring Tomcat's server.xml, you may have noticed the Executor configuration section. The default implementation uses org.apache.catalina.core.StandardThreadExecutor.

Looking at the startInternal() method in Tomcat 10.0.0-M4, the thread pool construction occurs as follows:

// Line 123
taskqueue = new TaskQueue(maxQueueSize);

// Line 124
TaskThreadFactory tf = new TaskThreadFactory(namePrefix, daemon, getThreadPriority());

// Line 125
executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 
    maxIdleTime, TimeUnit.MILLISECONDS, taskqueue, tf);

// Line 126
executor.setThreadRenewalDelay(threadRenewalDelay);

// Line 127-129
if (prestartminSpareThreads) {
    executor.prestartAllCoreThreads();
}

// Line 130 - Critical line
taskqueue.setParent(executor);

The TaskQueue extends LinkedBlockingQueue and includes specific comments indicating it was designed specifically for thread pool integration, behaving differently from standard queues when combined with executors.

Note that prestartminSpareThreads defaults to false. However, prestarting occurs regardless of this setting because the ThreadPoolExecutor constructor calls prestartAllCoreThreads() internally. The conditional check in lines 127-129 appears redundant.

The Critical Difference: Line 130

Line 130 proves essential: taskqueue.setParent(executor). Without this line, Tomcat's thread pool behaves identically to the standard JDK implementation—creating only core threads before queuing. With this line present, the pool creates threads up to the maximum before using the queue.

Consider a custom thread pool with core size 5, max size 150, and queue size 300. Without setParent(executor), only 5 threads run. With setParent(executor), the pool creates up to 150 threads before queuing remaining tasks.

public class TomcatThreadPoolTest {
    public static void main(String[] args) throws InterruptedException {
        String namePrefix = "worker-thread-";
        boolean daemon = true;
        TaskQueue taskqueue = new TaskQueue(300);
        TaskThreadFactory tf = new TaskThreadFactory(namePrefix, daemon, Thread.NORM_PRIORITY);
        ThreadPoolExecutor executor = new ThreadPoolExecutor(5, 150, 60000, 
            TimeUnit.MILLISECONDS, taskqueue, tf);
        // taskqueue.setParent(executor); // Uncomment to see different behavior
        
        for (int i = 0; i < 300; i++) {
            executor.execute(() -> {
                logStatus(executor, "task started");
                try {
                    TimeUnit.SECONDS.sleep(2);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            });
        }
        Thread.currentThread().join();
    }

    private static void logStatus(ThreadPoolExecutor executor, String marker) {
        TaskQueue queue = (TaskQueue) executor.getQueue();
        System.out.println("[" + marker + "] Core:" + executor.getCorePoolSize() + 
            " Active:" + executor.getActiveCount() + " Max:" + executor.getMaximumPoolSize() + 
            " Total:" + executor.getTaskCount() + " Queued:" + queue.size() + 
            " Remaining:" + queue.remainingCapacity());
    }
}

Source Code Analysis: TaskQueue.offer()

Understanding org.apache.tomcat.util.threads.TaskQueue#offer reveals how Tomcat modifies thread pool behavior:

public boolean offer(Runnable o) {
    if (parent == null) {
        return super.offer(o);
    }
    
    int poolSize = parent.getPoolSize();
    if (poolSize < parent.getMaximumPoolSize()) {
        return false; // Forces thread creation
    }
    
    int submitted = parent.getSubmittedCount();
    if (submitted < poolSize) {
        return super.offer(o);
    }
    
    return false;
}

The offer method contains four conditional checks:

First Condition: Parent Check

If parent is null, the method delegates to the standard queue's offer. Only when setParent(executor) is called does custom logic apply.

Second Condition: Pool Capacity Check

When currrent pool size is less than maximum pool size, the method returns false. This return value triggers the JDK ThreadPoolExecutor's logic for creating new threads (line 1378 in the JDK source). The pool continues creating threads until reaching the maximum.

Third Condition: Submitted Count vs Pool Size

getSubmittedCount() equals queue size plus running thread count. When submitted count is less than pool size (indicating idle threads exist), tasks are added to the queue for idle threads to consume. This avoids immediate execution while idle capacity remains.

Fourth Condition: Final Decision

If all previous conditions fail, the method returns false, triggering thread creation.

Rejection Strategy

Tomcat's ThreadPoolExecutor includes a custom execute() method handling rejection scenarios:

public void execute(Runnable command, long timeout, TimeUnit unit) {
    // Attempt to add to queue first
    if (!queue.offer(command, timeout, unit)) {
        if (isRunning()) {
            // Try once more with timeout
            if (!queue.offer(command, timeout, unit)) {
                throw new RejectedExecutionException("Queue full");
            }
        } else {
            throw new RejectedExecutionException("Pool shutting down");
        }
    }
}

When the queue is full, the executor waits for the specified timeout before attempting to add the task again. If still unsuccessful, it throws a RejectedExecutionException.

Tomcat's TaskQueue provides a force() method for forced insertion:

public boolean force(Runnable o, long timeout, TimeUnit unit) throws InterruptedException {
    if (parent == null || parent.isShutdown()) {
        throw new RejectedExecutionException("TaskQueue not running");
    }
    return super.offer(o, timeout, unit);
}

When examining rejection behavior, submitting more tasks than the queue capacity triggers the exception path. For example, submitting 500 tasks to a pool supporting 450 total tasks causes the rejection logic to activate.

A Note on contextStopping

When Tomcat's context stops, the forcedRemainingCapacity parameter is set to 0 in the TaskQueue. This relates to JDK's ThreadPoolExecutor.setCorePoolSize checking remaining capacity. While this appears unnecessary in JDK versions after 1.6, Tomcat preserves this behavior for compatibility.

If you monitor Tomcat's thread pool queue, be aware that after contextStopping(), the reported remaining capacity may not accurately reflect actual queue space (queue size minus queued tasks).

Comparison: Dubbo's Approach

Dubbo's EagerThreadPoolExecutor follows similar principles but with differences in execution:

public void execute(Runnable command) {
    int poolSize = getPoolSize();
    if (poolSize < getMaximumPoolSize()) {
        addWorker(command);
        return;
    }
    
    if (!getQueue().offer(command)) {
        // Retry immediately without timeout
        if (!getQueue().offer(command)) {
            throw new RejectedExecutionException("Queue is full");
        }
    }
}

Dubbo retries the offer() immediately without delay between attempts. Its rejection handler logs extensively and performs thread dumps to preserve debugging information:

// In rejection handler
log.warn("Thread pool is EXHAUSTED!");
dumpJStack();
shutdownThreadPool();

After dumping stack information, the thread pool executes shutdown() to prevent potential memory leaks. Using shutdown() rather than shutdownNow() allows submitted tasks to complete while preventing new submissions.

Why Tomcat Uses This Strategy

Tomcat primarily handles IO-bound tasks where response latency matters significantly. The standard JDK approach of queuing requests while idle threads exist contradicts user experience expectations. By maximizing thread usage before queuing, Tomcat ensures faster task processing for IO-bound workloads.

Understanding these implementation details helps when configuring thread pools and diagnosing performance issues. The design choice reflects the fundamental difference between CPU-bound and IO-bound task handling strategies.