BlockingQueue阻塞隊列原理解析

BlockingQueue,什麼鬼,剛開始接觸到這個數據結構的時候,從字面意義上根本沒看出這個的意思,怎麼會有個block冠在前邊,感覺是個不太好的詞。但是又發現其在某些服務任務框架上經常使用。今天我們來探祕下這個數據結構的用法。

BlockingQueue源碼分析

首先,打開JDK的源碼,找到BlockingQueue這個數據結構。(android開發要安裝的jdk 1.8,在其安裝目錄即有源碼)。

package java.util.concurrent;

import java.util.Collection;
import java.util.Queue;

 * @since 1.5
 * @author Doug Lea
 * @param <E> the type of elements held in this collection
 */
public interface BlockingQueue<E> extends Queue<E> {

首先,一個數據結構,包名是java.util.concurrent,很顯然,這是java爲併發操作設計的數據結構。並且還import了Collection和Queue。 所以其當然也有這兩個數據結構的特性。而BlockingQueue是個interface接口類型,那這就不是一個真實的可以對象化的類。如同List 是ArrayList<>()的父類,我們初始化時經常用new ArrayList<>()創建對象並賦值給List<>類型的變量一樣。他們都是泛型類,數據類型E可以在使用時具體制定。
接下來我們分析這個接口的主要方法。

/**
     * Inserts the specified element into this queue if it is possible to do
     * so immediately without violating capacity restrictions, returning
     * {@code true} upon success and throwing an
     * {@code IllegalStateException} if no space is currently available.
     * ……
     * /
     boolean add(E e);

從註釋看出,add方法可以插入數據到隊列中,如果隊列滿了,則拋出IllegalStateException異常。
之後的方法是offer(E e),其和add方法的不同之處在於,前者在隊列滿時是異常,後者是返回false,而add只是拋出異常。
而下一個方法,就可以看出BlockingQueue的精髓所在,即阻塞。

   /**
     * Inserts the specified element into this queue, waiting if necessary
     * for space to become available.
     *
     * @param e the element to add
     * @throws InterruptedException if interrupted while waiting
     * @throws ClassCastException if the class of the specified element
     *         prevents it from being added to this queue
     * @throws NullPointerException if the specified element is null
     * @throws IllegalArgumentException if some property of the specified
     *         element prevents it from being added to this queue
     */
    void put(E e) throws InterruptedException;

put的精髓就在於,當隊列滿的時候,會阻塞住,等有空間時插入。

/**
     * Retrieves and removes the head of this queue, waiting if necessary
     * until an element becomes available.
     *
     * @return the head of this queue
     * @throws InterruptedException if interrupted while waiting
     */
    E take() throws InterruptedException;

反之, 隊列的取元素也有阻塞方法,如果隊列中有元素,則取出處理,否則隊列爲空時則阻塞等待。
這裏只看了上述方法的註釋,其他的都是些輔助方法。
由於BlockingQueue只是個接口,只有定義的方法,但是沒有實際的實現,即如何實現阻塞,是在實現類中實現的。

其實現類有ArrayBlockingQueue、BlockingDeque等,詳細的如下圖所示。
BlockingQueue實現類
ArrayBlockingQueue :一個由數組結構組成的有界阻塞隊列。
LinkedBlockingQueue :一個由鏈表結構組成的有界阻塞隊列。
PriorityBlockingQueue :一個支持優先級排序的無界阻塞隊列。
DelayQueue:一個使用優先級隊列實現的無界阻塞隊列。
SynchronousQueue:一個不存儲元素的阻塞隊列。
LinkedTransferQueue:一個由鏈表結構組成的無界阻塞隊列。
LinkedBlockingDeque:一個由鏈表結構組成的雙向阻塞隊列。

我們分別分析ArrayBlockingQueue和BlockingDeque的源碼。

ArrayBlockingQueue源碼分析

這裏不會分析所有的方法,只撿重要的變量和方法進行分析。
首先是類的聲明和變量。

public class ArrayBlockingQueue<E> extends AbstractQueue<E>
        implements BlockingQueue<E>, java.io.Serializable {
    /** The queued items */
    final Object[] items;

    /** items index for next take, poll, peek or remove */
    int takeIndex;

    /** items index for next put, offer, or add */
    int putIndex;

    /** Number of elements in the queue */
    int count;
    /** Main lock guarding all access */
    final ReentrantLock lock;

    /** Condition for waiting takes */
    private final Condition notEmpty;

    /** Condition for waiting puts */
    private final Condition notFull;

    /**
     * Shared state for currently active iterators, or null if there
     * are known not to be any.  Allows queue operations to update
     * iterator state.
     */
    transient Itrs itrs = null;

其中的items數組,即要添加的隊列數據,有取數據的takeIndex和添加數據的putIndex。還有可重入鎖lock。
其構造函數顯示,隊列的長度是外部傳入的,即這個類的對象創建的時候,其大小就確定了。同時確定了lock爲非公平鎖。

 /**
     * Creates an {@code ArrayBlockingQueue} with the given (fixed)
     * capacity and default access policy.
     *
     * @param capacity the capacity of this queue
     * @throws IllegalArgumentException if {@code capacity < 1}
     */
    public ArrayBlockingQueue(int capacity) {
        this(capacity, false);
    }
    ……
    public ArrayBlockingQueue(int capacity, boolean fair) {
    if (capacity <= 0)
        throw new IllegalArgumentException();
    this.items = new Object[capacity];
    lock = new ReentrantLock(fair);
    notEmpty = lock.newCondition();
    notFull =  lock.newCondition();
    }

然後分析其不會阻塞的的offer方法。首先獲取lock的引用,並上鎖,之後對數據進行操作,如果能正常插入數據,則返回true,否則返回false,最後再finally中unlock。

 /**
     * Inserts the specified element at the tail of this queue if it is
     * possible to do so immediately without exceeding the queue's capacity,
     * returning {@code true} upon success and {@code false} if this queue
     * is full.  This method is generally preferable to method {@link #add},
     * which can fail to insert an element only by throwing an exception.
     *
     * @throws NullPointerException if the specified element is null
     */
    public boolean offer(E e) {
        checkNotNull(e);
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
            if (count == items.length)
                return false;
            else {
                enqueue(e);
                return true;
            }
        } finally {
            lock.unlock();
        }
    }

最主要我們要分析的方法是,這個BlockingQueue如何實現阻塞的。接下來查看put方法的實現。

/**
     * Inserts the specified element at the tail of this queue, waiting
     * for space to become available if the queue is full.
     *
     * @throws InterruptedException {@inheritDoc}
     * @throws NullPointerException {@inheritDoc}
     */
    public void put(E e) throws InterruptedException {
        checkNotNull(e);
        final ReentrantLock lock = this.lock;
        lock.lockInterruptibly();
        try {
            while (count == items.length)
                notFull.await();
            enqueue(e);
        } finally {
            lock.unlock();
        }
    }

這裏的lockInterruptibly什麼意思呢。我們在源碼的註釋中加入中文註釋。

Acquires the lock unless the current thread is
     * {@linkplain Thread#interrupt interrupted}.
     * 如果當前線程被打斷,則立即獲取鎖。
     * <p>Acquires the lock if it is not held by another thread and returns
     * immediately, setting the lock hold count to one.
     *如果當前鎖未被其他線程持有,則立即獲取鎖,並設置鎖持有數爲1
     * <p>If the current thread already holds this lock then the hold count
     * is incremented by one and the method returns immediately.
     *如果當前線程已持有這個鎖,則持有鎖的數目加1,並立即返回。
     * <p>If the lock is held by another thread then the
     * current thread becomes disabled for thread scheduling
     * purposes and lies dormant until one of two things happens:
     *如果當前鎖被另外線程持有,然後當前線程被線程調度爲不可運行,則當前線程處於休眠狀態,直到  1,鎖被當前線程持有  2,其他線程打斷了此線程。
     * <ul>
     *
     * <li>The lock is acquired by the current thread; or
     *
     * <li>Some other thread {@linkplain Thread#interrupt interrupts} the
     * current thread.
     *
     * </ul>
     *
     * <p>If the lock is acquired by the current thread then the lock hold
     * count is set to one.
     * 如果當前線程獲取鎖,則將鎖持有數設爲1
     * …其他註釋省略

這裏我們瞭解了這個鎖的特性,然後代碼
while (count == items.length)
notFull.await();
表示如果隊列滿了,則等待。一個鎖對象可以有一個或多個相關的條件對象,用newCondition方法獲取一個條件對象。notFull就是這樣一個對象,可以回頭看ArrayBlockingQueue的構造方法中,notFull = lock.newCondition();
**如果這個條件變量notFull調用了await()方法,則當前線程阻塞,並且釋放鎖。**這樣,另外的線程就可以拿到鎖,並取走隊列中的數據,並通知當前線程有了剩餘空間,線程被喚醒並添加數據到隊列。究竟是不是這樣呢,接下來我們找一個取數據的方法 take進行分析。

public E take() throws InterruptedException {
        final ReentrantLock lock = this.lock;
        lock.lockInterruptibly();
        try {
            while (count == 0)
                notEmpty.await();
            return dequeue();
        } finally {
            lock.unlock();
        }
    }
    /**
     * Extracts element at current take position, advances, and signals.
     * Call only when holding lock.
     */
    private E dequeue() {
        // assert lock.getHoldCount() == 1;
        // assert items[takeIndex] != null;
        final Object[] items = this.items;
        @SuppressWarnings("unchecked")
        E x = (E) items[takeIndex];
        items[takeIndex] = null;
        if (++takeIndex == items.length)
            takeIndex = 0;
        count--;
        if (itrs != null)
            itrs.elementDequeued();
        notFull.signal();
        return x;
    }

從上面的代碼看出,首先我們不分析取數據時數據爲空導致的阻塞,其和數據滿時的鎖操作時一致的,我們還是假設剛纔數據已滿,獲取鎖,並且取出一個數據,則調用dequeue方法。取出鎖之後,**notFull.signal();這個方法的意思是隨機解除等待集中某個線程的阻塞狀態。**和signal對應的方法是signalAll,意思是解除等待線程的阻塞,即通知所有等待線程,並通過競爭獲得鎖。 和signal,signalAll對應的有synchronized的notify和notifyAll的區別。可以想見,LinkedBlockingQueue也是類似原理實現阻塞,只不過不是array數組,而是單鏈表,且由於其可同時頭部取數據,尾部添加數據,所以其數據結構中有兩把鎖,和兩個條件變量。還有個原子類型的AtomicInteger類型的count,保證長度數據的原子性。這裏不再詳細分析,有需要的可以自己查看源碼學習。
系統中有很多使用阻塞隊列的例子,如系統Tts播放音頻時的播放隊列。

class AudioPlaybackHandler {
    private static final String TAG = "TTS.AudioPlaybackHandler";
    private static final boolean DBG = false;

    private final LinkedBlockingQueue<PlaybackQueueItem> mQueue =
            new LinkedBlockingQueue<PlaybackQueueItem>();

經過上邊的分析我們已經知道了BlockingQueue的實現阻塞的原理。

BlockingDeque的源碼分析

deque,雙端隊列,和queue的區別是。。。。看源碼吧
Deque的源碼註釋是:一個線性組合,支持在兩頭插入或者移除。**deque是double end queue的簡稱,原來如此,就是隊列queue只能first in last out,即頭部出,尾部進。 但是deque即可以頭部插入和移除,尾部也可以插入或移除。
/

  • A linear collection that supports element insertion and removal at
  • both ends. The name deque is short for “double ended queue”
  • and is usually pronounced “deck”. Most {@code Deque}
  • implementations place no fixed limits on the number of elements
  • they may contain, but this interface supports capacity-restricted
  • deques as well as those with no fixed size limit.

由於其也是個接口,deque的接口方法有
void addFirst(E e);
void addLast(E e);
boolean offerFirst(E e);
boolean offerLast(E e);
E removeFirst();
E removeLast();
E pollFirst();
E pollLast();
。。。。。
相當於queue有的方法,他都有兩份,即first的操作和last的操作。廢話少說,開始BlockingDeque的源碼分析,由於
public interface BlockingDeque extends BlockingQueue, Deque {
其繼承了BlockingQueue和Deque,所以其有這兩個數據結構的共同特性。

而再查找BlockingDeque的實現類時,我在SDK中只找到一個實現類,即
LinkedBlockingDeque。剛開始還想不分析LinkedBlockingQueue,這裏就來了 LinkedBlockingDeque。
首先其內部泛型類Node,即節點的聲明。其中包含一個數據項item和prev方法和next方法。其他爲LinkedBlockingDeque的內部變量和鎖的聲明。

static final class Node<E> {
        /**
         * The item, or null if this node has been removed.
         */
        E item;

        /**
         * One of:
         * - the real predecessor Node
         * - this Node, meaning the predecessor is tail
         * - null, meaning there is no predecessor
         */
        Node<E> prev;

        /**
         * One of:
         * - the real successor Node
         * - this Node, meaning the successor is head
         * - null, meaning there is no successor
         */
        Node<E> next;

        Node(E x) {
            item = x;
        }
    }
 /**
     * Pointer to first node.
     * Invariant: (first == null && last == null) ||
     *            (first.prev == null && first.item != null)
     */
    transient Node<E> first;

    /**
     * Pointer to last node.
     * Invariant: (first == null && last == null) ||
     *            (last.next == null && last.item != null)
     */
    transient Node<E> last;

    /** Number of items in the deque */
    private transient int count;

    /** Maximum number of items in the deque */
    private final int capacity;

    /** Main lock guarding all access */
    final ReentrantLock lock = new ReentrantLock();

    /** Condition for waiting takes */
    private final Condition notEmpty = lock.newCondition();

    /** Condition for waiting puts */
    private final Condition notFull = lock.newCondition();

既然是鏈表實現,那麼其中方法也就是鏈表數據項的的添加和刪除

 /**
     * Links node as first element, or returns false if full.
     */
    private boolean linkFirst(Node<E> node) {
        // assert lock.isHeldByCurrentThread();
        if (count >= capacity)
            return false;
            //下邊三步即是把新添加的node添加到隊頭,並且將first賦值爲對頭的node
        Node<E> f = first;
        node.next = f;
        first = node;
        if (last == null)
            last = node;
        else
            f.prev = node;
        ++count;
        notEmpty.signal();
        return true;
    }
    /**
     * @throws NullPointerException {@inheritDoc}
     * @throws InterruptedException {@inheritDoc}
     */
    public void putFirst(E e) throws InterruptedException {
        if (e == null) throw new NullPointerException();
        Node<E> node = new Node<E>(e);
        final ReentrantLock lock = this.lock;
        //添加數據到隊頭,並獲取鎖
        lock.lock();
        try {
        //如果隊列滿了,則返回false,就會發生阻塞,並釋放鎖。
            while (!linkFirst(node))
                notFull.await();
        } finally {
            lock.unlock();
        }
    }
     public E takeFirst() throws InterruptedException {
     //從對頭取走元素,並獲取鎖
        final ReentrantLock lock = this.lock;
        lock.lock();
        try {
            E x;
            //如果對頭取出數據爲空,則發生阻塞,否則,返回取出的元素
            while ( (x = unlinkFirst()) == null)
                notEmpty.await();
            return x;
        } finally {
            lock.unlock();
        }
    }

從上述源碼可以看出,**如果由於隊列滿導致put數據阻塞,則會釋放鎖,然後等有takefirst或其他從對頭取數據的方法調用後,會在unlinkFirst方法中調用notFull.signal(); 則通知上次putFirst的notFull.await()喚醒,在while方法中判斷,插入數據到隊頭。**隊尾操作也是相同的模式。

總結

使用阻塞隊列,多線程操作共同的隊列時不需要額外的同步。在經典的生產者-消費者模型中,隊列會自動平衡負載,即任意一邊(生產與消費)處理快了就會被阻塞掉,從而減少兩邊的處理速度差距,自動平衡負載這個特性就造成它能被用於多生產者隊列,因爲你生成多了(隊列滿了)你就要阻塞等着,直到消費者消費使隊列不滿你纔可以繼續生產。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章