BlockingQueue,什麼鬼,剛開始接觸到這個數據結構的時候,從字面意義上根本沒看出這個的意思,怎麼會有個block冠在前邊,感覺是個不太好的詞。但是又發現其在某些服務任務框架上經常使用。今天我們來探祕下這個數據結構的用法。
BlockingQueue源碼分析
首先,打開JDK的源碼,找到BlockingQueue這個數據結構。(android開發要安裝的jdk 1.8,在其安裝目錄即有源碼)。
package java.util.concurrent;
import java.util.Collection;
import java.util.Queue;
* @since 1.5
* @author Doug Lea
* @param <E> the type of elements held in this collection
*/
public interface BlockingQueue<E> extends Queue<E> {
首先,一個數據結構,包名是java.util.concurrent,很顯然,這是java爲併發操作設計的數據結構。並且還import了Collection和Queue。 所以其當然也有這兩個數據結構的特性。而BlockingQueue是個interface接口類型,那這就不是一個真實的可以對象化的類。如同List 是ArrayList<>()的父類,我們初始化時經常用new ArrayList<>()創建對象並賦值給List<>類型的變量一樣。他們都是泛型類,數據類型E可以在使用時具體制定。
接下來我們分析這個接口的主要方法。
/**
* Inserts the specified element into this queue if it is possible to do
* so immediately without violating capacity restrictions, returning
* {@code true} upon success and throwing an
* {@code IllegalStateException} if no space is currently available.
* ……
* /
boolean add(E e);
從註釋看出,add方法可以插入數據到隊列中,如果隊列滿了,則拋出IllegalStateException異常。
之後的方法是offer(E e),其和add方法的不同之處在於,前者在隊列滿時是異常,後者是返回false,而add只是拋出異常。
而下一個方法,就可以看出BlockingQueue的精髓所在,即阻塞。
/**
* Inserts the specified element into this queue, waiting if necessary
* for space to become available.
*
* @param e the element to add
* @throws InterruptedException if interrupted while waiting
* @throws ClassCastException if the class of the specified element
* prevents it from being added to this queue
* @throws NullPointerException if the specified element is null
* @throws IllegalArgumentException if some property of the specified
* element prevents it from being added to this queue
*/
void put(E e) throws InterruptedException;
put的精髓就在於,當隊列滿的時候,會阻塞住,等有空間時插入。
/**
* Retrieves and removes the head of this queue, waiting if necessary
* until an element becomes available.
*
* @return the head of this queue
* @throws InterruptedException if interrupted while waiting
*/
E take() throws InterruptedException;
反之, 隊列的取元素也有阻塞方法,如果隊列中有元素,則取出處理,否則隊列爲空時則阻塞等待。
這裏只看了上述方法的註釋,其他的都是些輔助方法。
由於BlockingQueue只是個接口,只有定義的方法,但是沒有實際的實現,即如何實現阻塞,是在實現類中實現的。
其實現類有ArrayBlockingQueue、BlockingDeque等,詳細的如下圖所示。
ArrayBlockingQueue :一個由數組結構組成的有界阻塞隊列。
LinkedBlockingQueue :一個由鏈表結構組成的有界阻塞隊列。
PriorityBlockingQueue :一個支持優先級排序的無界阻塞隊列。
DelayQueue:一個使用優先級隊列實現的無界阻塞隊列。
SynchronousQueue:一個不存儲元素的阻塞隊列。
LinkedTransferQueue:一個由鏈表結構組成的無界阻塞隊列。
LinkedBlockingDeque:一個由鏈表結構組成的雙向阻塞隊列。
我們分別分析ArrayBlockingQueue和BlockingDeque的源碼。
ArrayBlockingQueue源碼分析
這裏不會分析所有的方法,只撿重要的變量和方法進行分析。
首先是類的聲明和變量。
public class ArrayBlockingQueue<E> extends AbstractQueue<E>
implements BlockingQueue<E>, java.io.Serializable {
/** The queued items */
final Object[] items;
/** items index for next take, poll, peek or remove */
int takeIndex;
/** items index for next put, offer, or add */
int putIndex;
/** Number of elements in the queue */
int count;
/** Main lock guarding all access */
final ReentrantLock lock;
/** Condition for waiting takes */
private final Condition notEmpty;
/** Condition for waiting puts */
private final Condition notFull;
/**
* Shared state for currently active iterators, or null if there
* are known not to be any. Allows queue operations to update
* iterator state.
*/
transient Itrs itrs = null;
其中的items數組,即要添加的隊列數據,有取數據的takeIndex和添加數據的putIndex。還有可重入鎖lock。
其構造函數顯示,隊列的長度是外部傳入的,即這個類的對象創建的時候,其大小就確定了。同時確定了lock爲非公平鎖。
/**
* Creates an {@code ArrayBlockingQueue} with the given (fixed)
* capacity and default access policy.
*
* @param capacity the capacity of this queue
* @throws IllegalArgumentException if {@code capacity < 1}
*/
public ArrayBlockingQueue(int capacity) {
this(capacity, false);
}
……
public ArrayBlockingQueue(int capacity, boolean fair) {
if (capacity <= 0)
throw new IllegalArgumentException();
this.items = new Object[capacity];
lock = new ReentrantLock(fair);
notEmpty = lock.newCondition();
notFull = lock.newCondition();
}
然後分析其不會阻塞的的offer方法。首先獲取lock的引用,並上鎖,之後對數據進行操作,如果能正常插入數據,則返回true,否則返回false,最後再finally中unlock。
/**
* Inserts the specified element at the tail of this queue if it is
* possible to do so immediately without exceeding the queue's capacity,
* returning {@code true} upon success and {@code false} if this queue
* is full. This method is generally preferable to method {@link #add},
* which can fail to insert an element only by throwing an exception.
*
* @throws NullPointerException if the specified element is null
*/
public boolean offer(E e) {
checkNotNull(e);
final ReentrantLock lock = this.lock;
lock.lock();
try {
if (count == items.length)
return false;
else {
enqueue(e);
return true;
}
} finally {
lock.unlock();
}
}
最主要我們要分析的方法是,這個BlockingQueue如何實現阻塞的。接下來查看put方法的實現。
/**
* Inserts the specified element at the tail of this queue, waiting
* for space to become available if the queue is full.
*
* @throws InterruptedException {@inheritDoc}
* @throws NullPointerException {@inheritDoc}
*/
public void put(E e) throws InterruptedException {
checkNotNull(e);
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
while (count == items.length)
notFull.await();
enqueue(e);
} finally {
lock.unlock();
}
}
這裏的lockInterruptibly什麼意思呢。我們在源碼的註釋中加入中文註釋。
Acquires the lock unless the current thread is
* {@linkplain Thread#interrupt interrupted}.
* 如果當前線程被打斷,則立即獲取鎖。
* <p>Acquires the lock if it is not held by another thread and returns
* immediately, setting the lock hold count to one.
*如果當前鎖未被其他線程持有,則立即獲取鎖,並設置鎖持有數爲1
* <p>If the current thread already holds this lock then the hold count
* is incremented by one and the method returns immediately.
*如果當前線程已持有這個鎖,則持有鎖的數目加1,並立即返回。
* <p>If the lock is held by another thread then the
* current thread becomes disabled for thread scheduling
* purposes and lies dormant until one of two things happens:
*如果當前鎖被另外線程持有,然後當前線程被線程調度爲不可運行,則當前線程處於休眠狀態,直到 1,鎖被當前線程持有 2,其他線程打斷了此線程。
* <ul>
*
* <li>The lock is acquired by the current thread; or
*
* <li>Some other thread {@linkplain Thread#interrupt interrupts} the
* current thread.
*
* </ul>
*
* <p>If the lock is acquired by the current thread then the lock hold
* count is set to one.
* 如果當前線程獲取鎖,則將鎖持有數設爲1
* …其他註釋省略
這裏我們瞭解了這個鎖的特性,然後代碼
while (count == items.length)
notFull.await();
表示如果隊列滿了,則等待。一個鎖對象可以有一個或多個相關的條件對象,用newCondition方法獲取一個條件對象。notFull就是這樣一個對象,可以回頭看ArrayBlockingQueue的構造方法中,notFull = lock.newCondition();
**如果這個條件變量notFull調用了await()方法,則當前線程阻塞,並且釋放鎖。**這樣,另外的線程就可以拿到鎖,並取走隊列中的數據,並通知當前線程有了剩餘空間,線程被喚醒並添加數據到隊列。究竟是不是這樣呢,接下來我們找一個取數據的方法 take進行分析。
public E take() throws InterruptedException {
final ReentrantLock lock = this.lock;
lock.lockInterruptibly();
try {
while (count == 0)
notEmpty.await();
return dequeue();
} finally {
lock.unlock();
}
}
/**
* Extracts element at current take position, advances, and signals.
* Call only when holding lock.
*/
private E dequeue() {
// assert lock.getHoldCount() == 1;
// assert items[takeIndex] != null;
final Object[] items = this.items;
@SuppressWarnings("unchecked")
E x = (E) items[takeIndex];
items[takeIndex] = null;
if (++takeIndex == items.length)
takeIndex = 0;
count--;
if (itrs != null)
itrs.elementDequeued();
notFull.signal();
return x;
}
從上面的代碼看出,首先我們不分析取數據時數據爲空導致的阻塞,其和數據滿時的鎖操作時一致的,我們還是假設剛纔數據已滿,獲取鎖,並且取出一個數據,則調用dequeue方法。取出鎖之後,**notFull.signal();這個方法的意思是隨機解除等待集中某個線程的阻塞狀態。**和signal對應的方法是signalAll,意思是解除等待線程的阻塞,即通知所有等待線程,並通過競爭獲得鎖。 和signal,signalAll對應的有synchronized的notify和notifyAll的區別。可以想見,LinkedBlockingQueue也是類似原理實現阻塞,只不過不是array數組,而是單鏈表,且由於其可同時頭部取數據,尾部添加數據,所以其數據結構中有兩把鎖,和兩個條件變量。還有個原子類型的AtomicInteger類型的count,保證長度數據的原子性。這裏不再詳細分析,有需要的可以自己查看源碼學習。
系統中有很多使用阻塞隊列的例子,如系統Tts播放音頻時的播放隊列。
class AudioPlaybackHandler {
private static final String TAG = "TTS.AudioPlaybackHandler";
private static final boolean DBG = false;
private final LinkedBlockingQueue<PlaybackQueueItem> mQueue =
new LinkedBlockingQueue<PlaybackQueueItem>();
經過上邊的分析我們已經知道了BlockingQueue的實現阻塞的原理。
BlockingDeque的源碼分析
deque,雙端隊列,和queue的區別是。。。。看源碼吧
Deque的源碼註釋是:一個線性組合,支持在兩頭插入或者移除。**deque是double end queue的簡稱,原來如此,就是隊列queue只能first in last out,即頭部出,尾部進。 但是deque即可以頭部插入和移除,尾部也可以插入或移除。
/
- A linear collection that supports element insertion and removal at
- both ends. The name deque is short for “double ended queue”
- and is usually pronounced “deck”. Most {@code Deque}
- implementations place no fixed limits on the number of elements
- they may contain, but this interface supports capacity-restricted
- deques as well as those with no fixed size limit.
由於其也是個接口,deque的接口方法有
void addFirst(E e);
void addLast(E e);
boolean offerFirst(E e);
boolean offerLast(E e);
E removeFirst();
E removeLast();
E pollFirst();
E pollLast();
。。。。。
相當於queue有的方法,他都有兩份,即first的操作和last的操作。廢話少說,開始BlockingDeque的源碼分析,由於
public interface BlockingDeque extends BlockingQueue, Deque {
其繼承了BlockingQueue和Deque,所以其有這兩個數據結構的共同特性。
而再查找BlockingDeque的實現類時,我在SDK中只找到一個實現類,即
LinkedBlockingDeque。剛開始還想不分析LinkedBlockingQueue,這裏就來了 LinkedBlockingDeque。
首先其內部泛型類Node,即節點的聲明。其中包含一個數據項item和prev方法和next方法。其他爲LinkedBlockingDeque的內部變量和鎖的聲明。
static final class Node<E> {
/**
* The item, or null if this node has been removed.
*/
E item;
/**
* One of:
* - the real predecessor Node
* - this Node, meaning the predecessor is tail
* - null, meaning there is no predecessor
*/
Node<E> prev;
/**
* One of:
* - the real successor Node
* - this Node, meaning the successor is head
* - null, meaning there is no successor
*/
Node<E> next;
Node(E x) {
item = x;
}
}
/**
* Pointer to first node.
* Invariant: (first == null && last == null) ||
* (first.prev == null && first.item != null)
*/
transient Node<E> first;
/**
* Pointer to last node.
* Invariant: (first == null && last == null) ||
* (last.next == null && last.item != null)
*/
transient Node<E> last;
/** Number of items in the deque */
private transient int count;
/** Maximum number of items in the deque */
private final int capacity;
/** Main lock guarding all access */
final ReentrantLock lock = new ReentrantLock();
/** Condition for waiting takes */
private final Condition notEmpty = lock.newCondition();
/** Condition for waiting puts */
private final Condition notFull = lock.newCondition();
既然是鏈表實現,那麼其中方法也就是鏈表數據項的的添加和刪除
/**
* Links node as first element, or returns false if full.
*/
private boolean linkFirst(Node<E> node) {
// assert lock.isHeldByCurrentThread();
if (count >= capacity)
return false;
//下邊三步即是把新添加的node添加到隊頭,並且將first賦值爲對頭的node
Node<E> f = first;
node.next = f;
first = node;
if (last == null)
last = node;
else
f.prev = node;
++count;
notEmpty.signal();
return true;
}
/**
* @throws NullPointerException {@inheritDoc}
* @throws InterruptedException {@inheritDoc}
*/
public void putFirst(E e) throws InterruptedException {
if (e == null) throw new NullPointerException();
Node<E> node = new Node<E>(e);
final ReentrantLock lock = this.lock;
//添加數據到隊頭,並獲取鎖
lock.lock();
try {
//如果隊列滿了,則返回false,就會發生阻塞,並釋放鎖。
while (!linkFirst(node))
notFull.await();
} finally {
lock.unlock();
}
}
public E takeFirst() throws InterruptedException {
//從對頭取走元素,並獲取鎖
final ReentrantLock lock = this.lock;
lock.lock();
try {
E x;
//如果對頭取出數據爲空,則發生阻塞,否則,返回取出的元素
while ( (x = unlinkFirst()) == null)
notEmpty.await();
return x;
} finally {
lock.unlock();
}
}
從上述源碼可以看出,**如果由於隊列滿導致put數據阻塞,則會釋放鎖,然後等有takefirst或其他從對頭取數據的方法調用後,會在unlinkFirst方法中調用notFull.signal(); 則通知上次putFirst的notFull.await()喚醒,在while方法中判斷,插入數據到隊頭。**隊尾操作也是相同的模式。
總結
使用阻塞隊列,多線程操作共同的隊列時不需要額外的同步。在經典的生產者-消費者模型中,隊列會自動平衡負載,即任意一邊(生產與消費)處理快了就會被阻塞掉,從而減少兩邊的處理速度差距,自動平衡負載這個特性就造成它能被用於多生產者隊列,因爲你生成多了(隊列滿了)你就要阻塞等着,直到消費者消費使隊列不滿你纔可以繼續生產。