java线程池ThreadPoolExecutor类核心方法理解
李老爷子的注释其实已经非常详细了,这里主要是将逐句意义和我自己的理解贴出来.
本文不涉及ctl状态的二进制算法的理解(在此无意义).
完整构造线程池
/**
* Creates a new {@code ThreadPoolExecutor} with the given initial
* parameters.
*
* @param corePoolSize the number of threads to keep in the pool, even
* if they are idle, unless {@code allowCoreThreadTimeOut} is set
* @param maximumPoolSize the maximum number of threads to allow in the
* pool
* @param keepAliveTime when the number of threads is greater than
* the core, this is the maximum time that excess idle threads
* will wait for new tasks before terminating.
* @param unit the time unit for the {@code keepAliveTime} argument
* @param workQueue the queue to use for holding tasks before they are
* executed. This queue will hold only the {@code Runnable}
* tasks submitted by the {@code execute} method.
* @param threadFactory the factory to use when the executor
* creates a new thread
* @param handler the handler to use when execution is blocked
* because the thread bounds and queue capacities are reached
* @throws IllegalArgumentException if one of the following holds:<br>
* {@code corePoolSize < 0}<br>
* {@code keepAliveTime < 0}<br>
* {@code maximumPoolSize <= 0}<br>
* {@code maximumPoolSize < corePoolSize}
* @throws NullPointerException if {@code workQueue}
* or {@code threadFactory} or {@code handler} is null
*/
public ThreadPoolExecutor(int corePoolSize,//核心线程数(核心线程不会被销毁)
int maximumPoolSize,//最大线程数
long keepAliveTime,//超过核心线程数的线程的最大空闲生存时间,其后将可能被销毁
TimeUnit unit,//keepAliveTime的单位
BlockingQueue<Runnable> workQueue,//线程队列,当线程数超过核心线程数时入队
ThreadFactory threadFactory,//线程工厂
RejectedExecutionHandler handler//当线程数满,队列满时的拒绝策略
) {
if (corePoolSize < 0 ||
maximumPoolSize <= 0 ||
maximumPoolSize < corePoolSize ||
keepAliveTime < 0)
throw new IllegalArgumentException();
if (workQueue == null || threadFactory == null || handler == null)
throw new NullPointerException();
this.corePoolSize = corePoolSize;
this.maximumPoolSize = maximumPoolSize;
this.workQueue = workQueue;
this.keepAliveTime = unit.toNanos(keepAliveTime);
this.threadFactory = threadFactory;
this.handler = handler;
}
ThreadPoolExecutor::submit
//提交一个允许有返回值的任务,Future::get获取返回值.
public Future<?> submit(Runnable task) {
if (task == null) throw new NullPointerException();
//RunnableFuture自己就是一个Runnable且同时是一个Future可以用来接收返回值
RunnableFuture<Void> ftask = newTaskFor(task, null);
//执行execute,添加woker运行指定task
execute(ftask);
return ftask;
}
ThreadPoolExecutor::execute
/**
* Executes the given task sometime in the future. The task
* may execute in a new thread or in an existing pooled thread.
*
* If the task cannot be submitted for execution, either because this
* executor has been shutdown or because its capacity has been reached,
* the task is handled by the current {@link RejectedExecutionHandler}.
*
* @param command the task to execute
* @throws RejectedExecutionException at discretion of
* {@code RejectedExecutionHandler}, if the task
* cannot be accepted for execution
* @throws NullPointerException if {@code command} is null
*/
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
/*
* 分三步进行:
* 1. 如果运行的线程少于corePoolSize,
* 尝试以command作为第一个task开启一个一个新核心线程.
* 2. 如果成功将command入队workQueue,
* 双重检测确保线程池正RUNNING,
* (可能有其他线程执行了shutdown).
* 如果线程池已经shutdown,则回滚入队操作,
* 并执行拒绝策略
* 3. 如果无法入队,直接添加新的工作线程并执行command,
* 如果操作失败了,则说明线程池可能已经shutdown或饱和了,
* 则执行拒绝策略
*/
// 也就是说:1.如果核心线程没满开核心线程,否则将任务加入任务队列,
// 2.如果此时线程池关了,出队任务并执行拒绝策略
// 3.如果核心线程设定为且工作线程为0,则开非核心线程并执行队列中的任务,
// 4.如果队列满了开非核心线程,如果失败了执行拒绝策略
// 即:1.如果核心线程数设定大于0,只要任务队列没满就最多只会有核心线程.
// 2.如果核心线程数设定等于0,只要任务队列没满就最多只有一个非核心线程.
//获取ctl快照
int c = ctl.get();
//第一步
//判断工作线程数是否少于设定的核心线程数值
if (workerCountOf(c) < corePoolSize) {
//添加核心工作线程
if (addWorker(command, true))
return;
//重新获取ctl快照(ctl可能已被其他线程修改)
c = ctl.get();
}
//第二部
//如果线程池正RUNNING,将command加入workQueue
if (isRunning(c) && workQueue.offer(command)) {
//重新获取ctl快照
int recheck = ctl.get();
//双重检测,确保线程池没有shutdown,如果shutdown了则将command出队workQueue
if (! isRunning(recheck) && remove(command))
//执行拒绝策略
reject(command);
//判断此时线程池正RUNNING,且工作线程为0(corePoolSize可被设定为0)
else if (workerCountOf(recheck) == 0)
//添加非核心线程,并从workQueue中取出首个command运行
addWorker(null, false);
}
//队列可能已满从而失败的情况下,直接添加非核心工作线程,并将command作为task运行
else if (!addWorker(command, false))
//执行addWorker失败(线程池关闭或饱和)则执行拒绝策略
reject(command);
}
ThreadPoolExecutor::addWorker
/*
* Methods for creating, running and cleaning up after workers
*/
/**
* Checks if a new worker can be added with respect to current
* pool state and the given bound (either core or maximum). If so,
* the worker count is adjusted accordingly, and, if possible, a
* new worker is created and started, running firstTask as its
* first task. This method returns false if the pool is stopped or
* eligible to shut down. It also returns false if the thread
* factory fails to create a thread when asked. If the thread
* creation fails, either due to the thread factory returning
* null, or due to an exception (typically OutOfMemoryError in
* Thread.start()), we roll back cleanly.
*
* @param firstTask the task the new thread should run first (or
* null if none). Workers are created with an initial first task
* (in method execute()) to bypass queuing when there are fewer
* than corePoolSize threads (in which case we always start one),
* or when the queue is full (in which case we must bypass queue).
* Initially idle threads are usually created via
* prestartCoreThread or to replace other dying workers.
*
* @param core if true use corePoolSize as bound, else
* maximumPoolSize. (A boolean indicator is used here rather than a
* value to ensure reads of fresh values after checking other pool
* state).
* @return true if successful
*/
private boolean addWorker(Runnable firstTask, boolean core) {
retry:
for (int c = ctl.get();;) {//死循环,每次循环获取ctl最新快照
// Check if queue empty only if necessary.
// 必要时检测workQueue是否为空.(这里利用与或行为的短路一层一层判断)
// 什么是必要条件:当且仅当线程池被SHUTDOWN的时候,且不再有新任务.
// 即:addWorker时,如果线程池已经SHUTDOWN就不再接受新任务,但继续消费workQueue中的任务.
if (
//1.检测线程是否已经被SHUTDOWN,如果此时还是RUNNING就直接执内循环,否则如果至少是SHUTDOWN则进入下个与(进入下一个与线程池至少SHUTDOWN,甚至是STOP)
runStateAtLeast(c, SHUTDOWN)
&& (
//2.1.检查线程是否已经被STOP,如果被STOP了就不再消费workQueue,返回false,如果小于STOP则进入下一个或(进入下一个或线程池必然处在SHUTDOWN)
runStateAtLeast(c, STOP)
//2.2.如果有指定要执行的任务,由于此时线程池已经SHUTDOWN,不接收新任务,直接返回false,如果没给定新任务则进入下一个或
|| firstTask != null
//2.3. 如果任务队列为空,此时线程池也正处在SHUTDOWN,同时也没有新任务,则返回false,否则需要进入内循环消费workQueue剩余任务
|| workQueue.isEmpty()
)
)
//执行失败(三种情况:1.线程池已经STOP,2.线城池是SHUTDOWN但指定了新任务,3.线城池是SHUTDOWN且workQueue为空)
return false;
for (;;) {
//当前线程数:1.如果是add核心线程,判断是否大于等于核心线程数,否则判断是否大于等于最大线程数
if (workerCountOf(c)
>= ((core ? corePoolSize : maximumPoolSize) & COUNT_MASK))
//线程池饱和,执行失败
return false;
//上面判断都过了,说明此时可以添加任务,CAS先将线程数加一(如果后面实际添加worker执行失败再回退),CAS执行成功则跳出外循环,执行下面的添加worker
if (compareAndIncrementWorkerCount(c))
break retry;
//重新获取ctl快照,确保获取到的是最新的值(值传递)
c = ctl.get(); // Re-read ctl
//如果此时状态至少是SHUTDOWN,则重新执行外循环
if (runStateAtLeast(c, SHUTDOWN))
continue retry;
// else CAS failed due to workerCount change; retry inner loop
// 否则,重新执行内循环将线程数加一
}
}
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
//新建worker,并将firstTask丢进入,以保证如果有firstTask的情况下它会最先执行
//内部线程的run方法会runWorker方法,runWorker会循环从workQueue取任务执行
w = new Worker(firstTask);
//拿到worker内部新建的线程快照
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
//这里的操作需要加锁主要是因为workers是HashSet,线程不安全
mainLock.lock();
try {
// Recheck while holding lock.
// Back out on ThreadFactory failure or if
// shut down before lock acquired.
// 在获得锁后重新检测,以确保线程池正处在正常运行状态
// 重新获取最新快照
int c = ctl.get();
//如果正RUNNING,则直接添加worker到集合中
if (isRunning(c) ||
//否则如果线程池是SHUTDOWN且没有新任务的情况下才添加worker到集合中
(runStateLessThan(c, STOP) && firstTask == null)) {
//如果线程不是处与新建状态,抛出异常(因为后面会执行start)
if (t.getState() != Thread.State.NEW)
throw new IllegalThreadStateException();
//添加worker到集合中
workers.add(w);
//修改worker添加状态
workerAdded = true;
//修改总worker数量
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
}
} finally {
//解锁
mainLock.unlock();
}
//如果已经添加了worker,说明此时worker创建成功,且内部的线程没有开始运行,则使其运行
if (workerAdded) {
t.start();
//修改worker启动状态
workerStarted = true;
}
}
} finally {
//如果worker线程被启动失败
if (! workerStarted)
//回退上面的工作线程数加一操作,并将worker从集合中移除(如果worker已经被加入了集合的话),并执行tryTerminate内部的terminated钩子
addWorkerFailed(w);
}
return workerStarted;
}
ThreadPoolExecutor::runWorker
/**
* Main worker run loop. Repeatedly gets tasks from queue and
* executes them, while coping with a number of issues:
*
* 1. We may start out with an initial task, in which case we
* don't need to get the first one. Otherwise, as long as pool is
* running, we get tasks from getTask. If it returns null then the
* worker exits due to changed pool state or configuration
* parameters. Other exits result from exception throws in
* external code, in which case completedAbruptly holds, which
* usually leads processWorkerExit to replace this thread.
*
* 2. Before running any task, the lock is acquired to prevent
* other pool interrupts while the task is executing, and then we
* ensure that unless pool is stopping, this thread does not have
* its interrupt set.
*
* 3. Each task run is preceded by a call to beforeExecute, which
* might throw an exception, in which case we cause thread to die
* (breaking loop with completedAbruptly true) without processing
* the task.
*
* 4. Assuming beforeExecute completes normally, we run the task,
* gathering any of its thrown exceptions to send to afterExecute.
* We separately handle RuntimeException, Error (both of which the
* specs guarantee that we trap) and arbitrary Throwables.
* Because we cannot rethrow Throwables within Runnable.run, we
* wrap them within Errors on the way out (to the thread's
* UncaughtExceptionHandler). Any thrown exception also
* conservatively causes thread to die.
*
* 5. After task.run completes, we call afterExecute, which may
* also throw an exception, which will also cause thread to
* die. According to JLS Sec 14.20, this exception is the one that
* will be in effect even if task.run throws.
*
* The net effect of the exception mechanics is that afterExecute
* and the thread's UncaughtExceptionHandler have as accurate
* information as we can provide about any problems encountered by
* user code.
*
* @param w the worker
*/
/*
这个线程的核心是while循环不停的从队列中拿取任务并在独占锁的情况下执行(不会被中断)
线程被终止的情况:
1. 当执行的任务抛出异常线程将终止(线程池还在运行时将有新线程替代它);
2. 非核心线程在等待任务到超时后被通过中断终止;
3. 线程池停止时中断自己或被中断.
线程自己中断的条件:线程池已经至少是STOP,且当前线程没被中断过
判单当前线程是否空闲:在while循环中是会上锁的,而在getTask等待的过程中没有,
此时其他线程通过tryLock如果获取到了锁则说明线程当前处于空闲状态,
这种方式还可以保证其他线程在判断到当前线程是空闲时,当前线程不会被getTask从阻塞中唤醒并执行task
*/
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
//拿到指定的firstTask先执行并清空firstTask
Runnable task = w.firstTask;
w.firstTask = null;
w.unlock(); // allow interrupts 允许中断
boolean completedAbruptly = true;
try {
//当没有firstTask的时候阻塞地等待获取task
while (task != null || (task = getTask()) != null) {
//拿到task了就上锁,主要目的用来在其他线程判断当前线程是否还在阻塞等待获取task
w.lock();
// If pool is stopping, ensure thread is interrupted;
// if not, ensure thread is not interrupted. This
// requires a recheck in second case to deal with
// shutdownNow race while clearing interrupt
// 如果线程池处在执行停止方法,确保线程已被中断;
// 否则,确保线程没有被中断.
// 这种情况下需要在shutdownNow方法中双重检测以清理(处理)中断.
// 这里的if需要完成的任务: 如果线程池的状态至少是STOP状态,则要保证当前线程已经被执行过中断,
// 没有的话就立刻执行中断,后面的task将不会执行,这种情况可能时执行了shutdownNow方法.
if (
(
//1.1.当线程池已经至少时STOP状态 或
runStateAtLeast(ctl.get(), STOP) ||
(
//2.1.当前线程线程已经被中断(如果中断这个方法内部执行clearInterruptEvent) 且
Thread.interrupted() &&
//2.2.线程池已经至少时STOP状态
runStateAtLeast(ctl.get(), STOP)
)
) &&
//1.2.线程未被中断
!wt.isInterrupted()
)
//中断线程,下面的try将不再被执行
wt.interrupt();
try {
//执行钩子方法beforeExecute
beforeExecute(wt, task);
try {
task.run();
////执行钩子方法afterExecute(无异常)
afterExecute(task, null);
} catch (Throwable ex) {
//执行钩子方法afterExecute(异常)
afterExecute(task, ex);
//将异常抛出,这意味这个线程将消亡,如果线程池还在运行,将有新的线程替代它
throw ex;
}
} finally {
//清除已经执行的task引用
task = null;
//当前线程完成任务数+1
w.completedTasks++;
//解锁,则说明线程又进入空闲状态
w.unlock();
}
}
//如果循环内抛出了异常,这里将不会执行
completedAbruptly = false;
} finally {
//执行线程退出的方法,当执行到这里线程说明线程即将消亡(非核心线程超时或线程任务执行抛出了异常或线程池将停止)
processWorkerExit(w, completedAbruptly);
}
}
ThreadPoolExecutor::beforeExecute和afterExecute模板方法(钩子方法)
//钩子方法只在runWorker中执行
//继承ThreadPoolExecutor以实现钩子方法
class MyThreadPoolExecutor extends ThreadPoolExecutor {
public MyThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, RejectedExecutionHandler handler) {
super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
}
@Override
//runWorker内部执行task.run()前执行这个钩子方法
protected void beforeExecute(Thread t, Runnable r) {
super.beforeExecute(t, r);
System.out.println("执行任务前的钩子,已执行"+this.getTaskCount());
}
@Override
//runWorker内部执行task.run()后执行这个钩子方法,如果run抛出了异常可以在此处理
protected void afterExecute(Runnable r, Throwable t) {
super.afterExecute(r, t);
System.out.println("执行任务后的钩子,完成执行"+this.getTaskCount());
}
}
ThreadPoolExecutor::processWorkerExit
/**
* Performs cleanup and bookkeeping for a dying worker. Called
* only from worker threads. Unless completedAbruptly is set,
* assumes that workerCount has already been adjusted to account
* for exit. This method removes thread from worker set, and
* possibly terminates the pool or replaces the worker if either
* it exited due to user task exception or if fewer than
* corePoolSize workers are running or queue is non-empty but
* there are no workers.
*
* @param w the worker
* @param completedAbruptly if the worker died due to user exception
*/
/*
线程退方法只在runWorker最后执行
主要完成一些清理工作(将当前线程从池中移除)和记录工作(当前线程的)
如果线程池还在运行,将可能添加新线程替代这个线程(此时这个线程可能在执行中抛出了异常
*/
private void processWorkerExit(Worker w, boolean completedAbruptly) {
//如果completedAbruptly为true,则说明runWorker中task.run没执行(此时这个方法是在中断时执行)或runWorker的while循环中抛出了异常
if (completedAbruptly) // If abrupt, then workerCount wasn't adjusted
decrementWorkerCount();//这种情况工作线程数还没被减一,在此减一
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();//上锁以线性执行,保障workers和completedTaskCount不会混乱
try {
completedTaskCount += w.completedTasks;//将当前线程的任务完成数加到线程池
workers.remove(w);//把当前线程从works中移除
} finally {
mainLock.unlock();
}
//执行tryTerminate清理线程并在其内部执行terminated钩子
tryTerminate();
int c = ctl.get();//获取ctl快照
//如线程还没进入STOP(处在RUNNING或SHUTDOWN)
if (runStateLessThan(c, STOP)) {
//如果不是被突然终止(即task.run及其钩子执行成功,没抛出异常的情况下)
if (!completedAbruptly) {
//拿到最小线程数
int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
//如果最小线程数为0,但此时工作队列内还有task,将最小线程数设置为1保证有线程执行队列中的task
if (min == 0 && ! workQueue.isEmpty())
min = 1;
//如果当前存在的工作线程数大于等于最小线程数就不用添加新线程,否则就添加一个线程执行队列中的task
if (workerCountOf(c) >= min)
return; // replacement not needed
}
//否则(即task.run及其钩子没执行或抛出异常的情况下)
addWorker(null, false);//添加一个新的工作线程替代此线程
}
}
ThreadPoolExecutor::tryTerminate
/**
* Transitions to TERMINATED state if either (SHUTDOWN and pool
* and queue empty) or (STOP and pool empty). If otherwise
* eligible to terminate but workerCount is nonzero, interrupts an
* idle worker to ensure that shutdown signals propagate. This
* method must be called following any action that might make
* termination possible -- reducing worker count or removing tasks
* from the queue during shutdown. The method is non-private to
* allow access from ScheduledThreadPoolExecutor.
*/
/*
这种方法被多个方法调用,主要是保证清理工作的正常进行,空闲的线程能被正常清除,
同时又至少保留一个线程以保证有线程能执行清理.
保证线程池的ctl被正确设置.
为什么在清理空闲线程时只清理一个?
因为需要保证至少留有一个线程善后,一个一个的清理能保证最后一个线程一定可以善后
(如过中断全部,由于此时没有上锁,其他线程也可以中断当前线程,可能就没线程了)
*/
final void tryTerminate() {
//CAS死循环,不执行成功誓不罢休
for (;;) {
int c = ctl.get();//ctl快照
//如果 线程正在运行 或
if (isRunning(c) ||
//线程已经是TIDYING或TERMINATED 或
//(这种情况下清理工作已经做完了)
runStateAtLeast(c, TIDYING) ||
//线程是RUNNING或SHUTDOWN 且 工作线程不为空
//(这个状态下要保证任务能得到执行,无需清理)
(runStateLessThan(c, STOP) && ! workQueue.isEmpty()))
return;//则退出
//如果还有工作线程存活,则清理一个空闲线程
if (workerCountOf(c) != 0) { // Eligible to terminate
interruptIdleWorkers(ONLY_ONE);
return;
}
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();//上锁,保证terminated钩子和ctl正常
try {
//CAS的方式将状态设置为TIDYING
if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))) {
try {
terminated();//执行钩子
} finally {
//无论怎样,此时线程池都彻底结束了,状态设置为TERMINATED
ctl.set(ctlOf(TERMINATED, 0));
//唤醒所有执行了awaitTermination方法并阻塞的线程,让他们继续执行
termination.signalAll();
}
return;//一切都结束了,线程池完成了它的使命
}
} finally {
mainLock.unlock();
}
// else retry on failed CAS
}
}