ElasticSearch之PrioritizedEsThreadPoolExecutor

PrioritizedEsThreadPoolExecutor優先級線程池,顧名思義,根據線程任務的優先級來提供線程,每次提供優先級最高的線程任務,還沒看實現前我們可以來思考一下如何做到,普通線程池提供的是線程隊列,那麼每次僅需要得到優先級最高的線程任務,那麼可以改成使用優先隊列,優先隊列本質不過是堆。

先看下PrioritizedEsThreadPoolExecutor的構造

PrioritizedEsThreadPoolExecutor(String name, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit,
                                ThreadFactory threadFactory, ThreadContext contextHolder, ScheduledExecutorService timer) {
    super(name, corePoolSize, maximumPoolSize, keepAliveTime, unit, new PriorityBlockingQueue<>(), threadFactory, contextHolder);
    this.timer = timer;
}

這裏可以看到直接調用了父類EsThreadPoolExecutor的構造,但是隊列這裏採用的是PriorityBlockingQueue隊列。

    EsThreadPoolExecutor(String name, int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit,
            BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory, ThreadContext contextHolder) {
        this(name, corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, new EsAbortPolicy(), contextHolder);
    }

EsThreadPoolExecutor繼承自ThreadPoolExecutor,已經到jdk層面,這裏無非將參數傳遞。

看其execute()方法

    public void execute(Runnable command, final TimeValue timeout, final Runnable timeoutCallback) {
        command = wrapRunnable(command);
        doExecute(command);
        if (timeout.nanos() >= 0) {
            if (command instanceof TieBreakingPrioritizedRunnable) {
                ((TieBreakingPrioritizedRunnable) command).scheduleTimeout(timer, timeoutCallback, timeout);
            } else {
                // We really shouldn't be here. The only way we can get here if somebody created PrioritizedFutureTask
                // and passed it to execute, which doesn't make much sense
                throw new UnsupportedOperationException("Execute with timeout is not supported for future tasks");
            }
        }
    }

    @Override
    protected Runnable wrapRunnable(Runnable command) {
        if (command instanceof PrioritizedRunnable) {
            if (command instanceof TieBreakingPrioritizedRunnable) {
                return command;
            }
            Priority priority = ((PrioritizedRunnable) command).priority();
            return new TieBreakingPrioritizedRunnable(super.wrapRunnable(command), priority, insertionOrder.incrementAndGet());
        } else if (command instanceof PrioritizedFutureTask) {
            return command;
        } else { // it might be a callable wrapper...
            return new TieBreakingPrioritizedRunnable(super.wrapRunnable(command), Priority.NORMAL, insertionOrder.incrementAndGet());
        }
    }

可以看到將傳進來的runnable任務通過wrapRunnable,獲取其優先級並封裝成TiBreakingPriortizedRunable類型。TieBreakingPrioritizedRunnable是其內部類,繼承自PrioritizedRunnable,我們看其構造

        TieBreakingPrioritizedRunnable(Runnable runnable, Priority priority, long insertionOrder) {
            super(priority);
            this.runnable = runnable;
            this.insertionOrder = insertionOrder;
        }

可以看到,TieBreakingPrioritizedRunnable封裝了runnable跟insertionOrder,其中insertionOrder傳入時,是PrioritizedEsThreadPoolExecutor的內部類insertionOrder,且是atomicLong類型,每次傳入都incrementAndGet,說明整體上來看一個PrioritizedEsThreadPoolExecutor中的所有的任務封裝的insertionOrder是不同的且是遞增的,先要被執行的任務insertionOrder小。

然後調用父類的doExecute()

    protected void doExecute(final Runnable command) {
        try {
            super.execute(command);
        } catch (EsRejectedExecutionException ex) {
            if (command instanceof AbstractRunnable) {
                // If we are an abstract runnable we can handle the rejection
                // directly and don't need to rethrow it.
                try {
                    ((AbstractRunnable) command).onRejection(ex);
                } finally {
                    ((AbstractRunnable) command).onAfter();

                }
            } else {
                throw ex;
            }
        }
    }

很明顯扔到父類jdk層面的線程池去。然後無非走線程池策略,從workQueue中取線程執行。

繼續回到TieBreakingPrioritizedRunnable分析。TieBreakingPrioritizedRunnable繼承自PrioritizedRunnable,並且PrioritizedRunnable實現了runnable跟Comparable接口,說明他是一個可以比較的runnable。看TieBreakingPrioritizedRunnable的compareTo方法

        @Override
        public int compareTo(PrioritizedRunnable pr) {
            int res = super.compareTo(pr);
            if (res != 0 || !(pr instanceof TieBreakingPrioritizedRunnable)) {
                return res;
            }
            return insertionOrder < ((TieBreakingPrioritizedRunnable) pr).insertionOrder ? -1 : 1;
        }

先調用父類PrioritizedRunnable的compareTo方法,如果相同那麼繼續通過insertionOrder來比較的出結果。

父類PrioritizedRunnable的compareTo的方法自然很簡單,通過傳入的優先級比較

    @Override
    public int compareTo(PrioritizedRunnable pr) {
        return priority.compareTo(pr.priority);
    }

那麼這個線程池的任務優先級比較規則可以確定了,先是按照任務的優先級,優先級相同則按照傳入的先後順序。

分析到這接下來重點來看下PriorityBlockingQueue是如何實現。

    public boolean offer(E e) {
        if (e == null)
            throw new NullPointerException();
        final ReentrantLock lock = this.lock;
        lock.lock();
        int n, cap;
        Object[] array;
        while ((n = size) >= (cap = (array = queue).length))
            tryGrow(array, cap);
        try {
            Comparator<? super E> cmp = comparator;
            if (cmp == null)
                siftUpComparable(n, e, array);
            else
                siftUpUsingComparator(n, e, array, cmp);
            size = n + 1;
            notEmpty.signal();
        } finally {
            lock.unlock();
        }
        return true;
    }

先粗看一下,一開始如果傳入任務爲null自然報異常,然後加鎖。把queue賦值給array,如果長度不夠嘗試擴容。因爲我們採用的是無參構造方法,那麼自然走的是siftUpComparable方法,然後長度加1,非空的condition條件隊列元素喚醒。

    private static <T> void siftUpComparable(int k, T x, Object[] array) {
        Comparable<? super T> key = (Comparable<? super T>) x;
        while (k > 0) {
            int parent = (k - 1) >>> 1;
            Object e = array[parent];
            if (key.compareTo((T) e) >= 0)
                break;
            array[k] = e;
            k = parent;
        }
        array[k] = key;
    }

儼然一看,就是堆排序的算法,每次跟當前位置的父節點比較,如果優先級更高,那麼當前位置換成父節點,父節點位置留給當前元素繼續循環,時間複雜度logN,最壞樹的高度。

當然對應的優先級隊列元素出隊操作

    private static <T> void siftDownComparable(int k, T x, Object[] array,
                                               int n) {
        if (n > 0) {
            Comparable<? super T> key = (Comparable<? super T>)x;
            int half = n >>> 1;           // loop while a non-leaf
            while (k < half) {
                int child = (k << 1) + 1; // assume left child is least
                Object c = array[child];
                int right = child + 1;
                if (right < n &&
                    ((Comparable<? super T>) c).compareTo((T) array[right]) > 0)
                    c = array[child = right];
                if (key.compareTo((T) c) <= 0)
                    break;
                array[k] = c;
                k = child;
            }
            array[k] = key;
        }
    }

從跟下標爲0(已空)開始,先找到其左右孩子的下標,然後取出左右孩子中優先級較高的那個,填到空的位置,然後繼續從其左右孩子中找一直循環,到最後空出一個(葉子結點或者非葉子節點但是優先級低於最後一個元素)的位置,給最後一個元素安置,這麼來說,給最後一個元素安置在前面,那麼最後一個位置也就空了出來。正好最大的被取走,整體還是保持着堆的特性。

分析到最後也沒怎麼涉及線程池,反而是一個用堆實現優先隊列的經典案例。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章