Recently, a friend asked me a question on WeChat. He said: "WhyBro, are you there?"

I replied: "What's up?"

He quickly threw out a question.

I took a casual glance and thought this question was very simple.

But it turned out there was a hidden article behind it.

The story starts with this question:

The thread pool configuration in the image above is:

ExecutorService executorService = new ThreadPoolExecutor(40, 80, 1, TimeUnit.MINUTES,
    new LinkedBlockingQueue<>(100), 
    new DefaultThreadFactory("test"),
    new ThreadPoolExecutor.DiscardPolicy());

I won't explain the parameters or execution flow of this thread pool again, as I already covered them in a previous article. The question above is actually a very standard interview question:

When are non-core threads recycled?

If a thread exceeding the core pool size does not receive a new task within keepAliveTime, it will be recycled.

This is the textbook answer, perfectly correct.

Now let's simulate a simple scenario. For clarity, let's adjust the thread pool parameters:

ExecutorService executorService = new ThreadPoolExecutor(2, 3, 30, TimeUnit.SECONDS,
    new LinkedBlockingQueue<>(2), 
    new DefaultThreadFactory("test"),
    new ThreadPoolExecutor.DiscardPolicy());

Then the question becomes:

1. What is the maximum number of tasks this thread pool can hold? 5? (since core=2, max=3, queue=2)
2. If each task takes 1 second, and I submit 5 tasks sequentially within 1 second, will the number of active threads be 3?
3. If no task is submitted for the next 30 seconds, will the active thread count drop to 2 after 30 seconds?

The answer to all three is yes. If you don't understand why, you should review thread pool basics first.

Now the next question:

If the pool currently has 3 active threads (2 core + 1 non-core), and all have completed their tasks and are waiting, and then I submit one task (lasting 1 second) every 3 seconds, what will be the active thread count after 30 seconds?

The answer: still 3.

My initial intuition: a core thread is idle, and submitting a 1-second task every 3 seconds only needs one core thread. Therefore, the non-core thread remains idle for more than 30 seconds and should be recycled.

But the actual behavior is different. After 30 seconds, the non-core thread is not recycled. The active count remains 3.

If you already know the answer and the reason, you can stop here. If not, read on.

Demo for Verification

Here's the code for the scenario described above:

public class ThreadTest {

    @Test
    public void test() throws InterruptedException {

        ThreadPoolExecutor executorService = new ThreadPoolExecutor(2, 3, 30, TimeUnit.SECONDS,
            new LinkedBlockingQueue<>(2), new DefaultThreadFactory("test"),
            new ThreadPoolExecutor.DiscardPolicy());

        // Print thread pool info every 2 seconds
        ScheduledExecutorService scheduledExecutorService = Executors.newScheduledThreadPool(1);
        scheduledExecutorService.scheduleAtFixedRate(() -> {
            System.out.println("=====================================thread-pool-info:" + new Date() + "=====================================");
            System.out.println("CorePoolSize:" + executorService.getCorePoolSize());
            System.out.println("PoolSize:" + executorService.getPoolSize());
            System.out.println("ActiveCount:" + executorService.getActiveCount());
            System.out.println("KeepAliveTime:" + executorService.getKeepAliveTime(TimeUnit.SECONDS));
            System.out.println("QueueSize:" + executorService.getQueue().size());
        }, 0, 2, TimeUnit.SECONDS);

        try {
            // Submit 5 tasks at once to simulate reaching max
            for (int i = 0; i < 5; i++) {
                executorService.execute(new Task());
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
        // Sleep 10 seconds to observe pool state
        Thread.sleep(10000);

        // Submit a task every 3 seconds
        while (true) {
            Thread.sleep(3000);
            executorService.submit(new Task());
        }
    }

    static class Task implements Runnable {
        @Override
        public void run() {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
            System.out.println(Thread.currentThread() + "-执行任务");
        }
    }
}

You can run this code directly. The output looks like:

There are five tasks total. Let's analyze:

• Part ①: Three threads are executing tasks. Thread 2 and Thread 1 finish first, then pickup tasks from the queue (part ②).
• According to the program, after that, one task (1-second duration) is submitted every 3 seconds. At this moment, all three threads are idle.

Now the question: which thread gets the new task? Is it random?

From the log, we can predict that the execution order will be:

Thread[test-1-3,5,main] - 执行任务 Thread[test-1-2,5,main] - 执行任务 Thread[test-1-1,5,main] - 执行任务 Thread[test-1-3,5,main] - 执行任务 ...

It's round-robin, not random. Because of this round-robin, the non-core thread never stays idle for more than 9 seconds, so it never exceeds the 30-second keepAliveTime. Therefore, the active thread count remains 3.

This demo explains the phenomenon, but what about the underlying mechanism?

Why Round-Robin?

To understand the internal mechanism, we need to look at the code. When threads are idle, they wait for new tasks. This is essentially a producer-consumer problem.

I dumped the thread stack and found that the threads were waiting in an AQS ConditionObject's waiting queue.

The relevant source code is in:

java.util.concurrent.locks.AbstractQueuedSynchronizer.ConditionObject#awaitNanos

When a new task arrives, the thread pool calls signalNotEmpty(), which eventually calls doSignal and then transferForSignal. This method uses LockSupport.unpark(node.thred) to wake up threads in the order of the waiting queue.

The order in the waiting queue is determined when threads call await:

So the execution order is determined by the order in the Condition's waiting queue — it's not random.

If you are not familiar with Condition, you can refer to the book "The Art of Java Concurrent Programming" (Section 5.6.2) for a detailed explanation with diagrams.

How Are Non-Core Threads Recycled?

If a non-core thread is idle for more than 30 seconds, how is it recycled?

The key is in the getTask method of ThreadPoolExecutor:

private Runnable getTask() {
    boolean timedOut = false; // Did the last poll() time out?

    for (;;) {
        int c = ctl.get();
        int rs = runStateOf(c);

        // Check if queue empty only if necessary.
        if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
            decrementWorkerCount();
            return null;
        }

        int wc = workerCountOf(c);

        // Are workers subject to culling?
        boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;

        if ((wc > maximumPoolSize || (timed && timedOut))
            && (wc > 1 || workQueue.isEmpty())) {
            if (compareAndDecrementWorkerCount(c))
                return null;
            continue;
        }

        try {
            Runnable r = timed ?
                workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
                workQueue.take();
            if (r != null)
                return r;
            timedOut = true;
        } catch (InterruptedException retry) {
            timedOut = false;
        }
    }
}

When timed is true (which happens when allowCoreThreadTimeOut is true or when worker count exceeds core pool size), the method calls workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS).

If the queue remains empty for keepAliveTime, poll returns null, setting timedOut = true. In the next iteration, it returns null, which causes the worker to exit.

The worker exit process goes to processWorkerExit:

private void processWorkerExit(Worker w, boolean completedAbruptly) {
    if (completedAbruptly)
        decrementWorkerCount();

    final ReentrantLock mainLock = this.mainLock;
    mainLock.lock();
    try {
        completedTaskCount += w.completedTasks;
        workers.remove(w);
    } finally {
        mainLock.unlock();
    }

    tryTerminate();

    int c = ctl.get();
    if (runStateLessThan(c, STOP)) {
        if (!completedAbruptly) {
            int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
            if (min == 0 && !workQueue.isEmpty())
                min = 1;
            if (workerCountOf(c) >= min)
                return; // replacement not needed
        }
        addWorker(null, false);
    }
}

The thread is removed from the workers set via workers.remove(w), effectively recycling it.

A common question: is the recycled thread necessarily a non-core thread? The answer is no. The thread pool does not distinguish between core and non-core threads at runtime. Once wc > corePoolSize, all threads are subject to polling and potential recycling. The first thread that happens to time out gets recycled, regardless of whether it was originally a "core" or "non-core" thread.

This is also evident in the addWorker method:

private boolean addWorker(Runnable firstTask, boolean core) {
    // ...
    int wc = workerCountOf(c);
    if (wc >= CAPACITY ||
        wc >= (core ? corePoolSize : maximumPoolSize))
        return false;
    // ...
}

The core parameter only determines whether to compare against corePoolSize or maximumPoolSize. The actual worker itself is not tagged.

So, if you want to distinguish core from non-core threads, you'd have to extend ThreadPoolExecutor and add custom logic (e.g., by naming threads differently). But in practice, such a distinction is rarely needed.

Conclusion

Understanding the round-robin idle thread behavior and the recycling mechanism deepens your grasp of thread pool internals. The key is to recognize that idle threads wait in a Condition's waiting queue, and new tasks wake them in order. The keepAliveTime recycling applies to any thread once the pool size exceeds core size, without distinguishing between core and non-core roles.