The Concept of Thread Reuse

In standard Java programming, when a task needs to be executed asynchronously, a developer typically creates a Thread object and passes a Runnable target to it. The execution is triggered by the native start0 method, which eventually calls the run() method of the Thread class. The standard implementation of Thread.run() simply executes the run() method of the associated Runnable target, if one exists.

This design pattern implies a 1:1 relationship between a thread instance and a specific task. Once the run() method completes, the thread transitions to the TERMINATED state and cannot be restarted. In high-load systems, creating and destroying threads for every task introduces significant performance overhead due to context switching and memory allocation. Furthermore, uncontrolled thread creation can exhaust system resources, leading to instability.

Thread pools address this by decoupling task execution from the thread lifecycle. The core principle of thread reuse involves maintaining a set of persistent worker threads that can execute multiple tasks sequentially. Instead of terminating after a single task, a worker thread retrieves the next task from a blocking queue and executes it. This section explores the internal source code of Java's ThreadPoolExecutor to demonstrate exactly how this reuse is implemented.

Execution Flow Overview

Before diving into the source code, its helpful to visualize how a thread pool handles a submitted task. When a task is submitted, the pool checks the number of active threads. If the count is below the core pool size, a new thread is started immediately. If the core pool is full, the task is placed into a work queue. Only if the queue is full and the thread count is below the maximum pool size are additional non-core threads created. The mechanism that allows these threads to stay alive and process tasks from the queue is the focus of the analysis below.

Analyzing the Reuse Implementation

Thread Pool Usage Example

To set the context, consider the following implementation of a custom thread pool used to simulate handling server requests. This example uses ThreadPoolExecutor directly.

import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;

public class ServerRequestHandler {

    private static final AtomicInteger threadCounter = new AtomicInteger(0);
    
    private static final ThreadFactory namedThreadFactory = r -> {
        Thread t = new Thread(r, "worker-pool-" + threadCounter.incrementAndGet());
        t.setDaemon(false);
        return t;
    };

    private static final ThreadPoolExecutor executor = new ThreadPoolExecutor(
            2, // core pool size
            5, // max pool size
            30L, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(10),
            namedThreadFactory,
            new ThreadPoolExecutor.CallerRunsPolicy());

    public static void main(String[] args) {
        Runnable requestTask = () -> {
            System.out.println("Processing request on " + Thread.currentThread().getName());
            try {
                TimeUnit.MILLISECONDS.sleep(500);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        };

        for (int i = 0; i < 20; i++) {
            executor.submit(requestTask);
        }
        
        executor.shutdown();
    }
}

Task Submission and Execution Logic

The entry point for task submission is the submit method. This method wraps the Runnable into a RunnableFuture and delegates the heavy lifting to the execute(Runnable command) method.

The execute method contains the logic that determines whether a new thread is needed or if the task should be queued. The critical step for reuse occurs in the addWorker method. This method is responsible for creating a new Worker instance and starting the underlying thread. The Worker class is the internal implementation that encapsulates the thread and the logic for running tasks.

The Worker Class Implementation

The Worker class extends AbstractQueuedSynchronizer to implement a simple non-reentrant lock, which is used to manage interruptions during task execution. Crucially, Worker implements Runnable.

In the Worker constructor, a new Thread is created using the pool's ThreadFactory, with the Worker instance itself passed as the Runnable target. When addWorker calls thread.start(), it invokes the run() method of the Worker class.

private final class Worker extends AbstractQueuedSynchronizer implements Runnable {
    
    final Thread thread;
    Runnable firstTask;
    volatile long completedTasks;

    Worker(Runnable firstTask) {
        setState(-1); // inhibit interrupts until runWorker
        this.firstTask = firstTask;
        this.thread = getThreadFactory().newThread(this);
    }

    public void run() {
        runWorker(this);
    }
    
    // Lock management methods omitted for brevity
}

The runWorker Method: The Heart of Reuse

The runWorker(Worker w) method is where the thread reuse logic actually resides. This method does not execute a single task and exit. Instead, it enters a loop designed to fetch and execute tasks continuously.

final void runWorker(Worker w) {
    Thread wt = Thread.currentThread();
    Runnable task = w.firstTask;
    w.firstTask = null;
    w.unlock(); 
    boolean completedAbruptly = true;
    try {
        // The core loop: execute firstTask, then poll from queue
        while (task != null || (task = getTask()) != null) {
            w.lock();
            
            // Interrupt logic checks omitted for clarity
            
            try {
                beforeExecute(wt, task);
                Throwable thrown = null;
                try {
                    task.run();
                } catch (RuntimeException x) {
                    thrown = x; throw x;
                } catch (Error x) {
                    thrown = x; throw x;
                } catch (Throwable x) {
                    thrown = x; throw new Error(x);
                } finally {
                    afterExecute(task, thrown);
                }
            } finally {
                task = null;
                w.completedTasks++;
                w.unlock();
            }
        }
        completedAbruptly = false;
    } finally {
        processWorkerExit(w, completedAbruptly);
    }
}

The logic is straightforward yet powerful:

1. The method first attempts to run the firstTask (the task provided when the worker was created).
2. It then enters a while loop.
3. Inside the loop, it calls getTask() to retrieve the next unit of work from the work queue.
4. If getTask() returns a task, the run() method of that task is invoked directly by the current worker thread.
5. After execution, the loop repeats, calling getTask() again.

This loop ensures that the thread does not terminate. It simply blocks on getTask() when the queue is empty and wakes up when a new task arrives, thereby reusing the same thread object for multiple Runnable instances.

Fetching Tasks: The getTask Method

The getTask() method handles the blocking behavior that keeps the thread alive without consuming CPU cycles. It interacts with the blocking work queue.

private Runnable getTask() {
    boolean timedOut = false; 

    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);

        // Check if pool is shutting down or empty
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            decrementWorkerCount();
            return null;
        }

        int wc = workerCountOf(c);

        // Determine if the worker should time out (allow core thread timeout or count > core size)
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;

        if ((wc > maximumPoolSize || (timed && timedOut))
            && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            continue;
        }

        try {
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) : // Wait with timeout
                workQueue.take();                                      // Wait indefinitely
            if (r != null)
                return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

Depending on the pool configuration (e.g., whether the current thread is a core thread or if allowCoreThreadTimeOut is set), getTask() either calls workQueue.take() (blocking indefinitely until a task is available) or workQueue.poll() (waiting for a specified duration). If the queue is empty and the conditions are met for the thread to die (e.g., timeout for a non-core thread), it returns null, causing the runWorker loop to break and the thread to terminate. Otherwise, it blocks, preserving the thread for future tasks.