如何高效的使用並行流

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在Java7之前想要並行處理大量數據是很困難的，首先把數據拆分成很多個部分，然後把這這些子部分放入到每個線程中去執行計算邏輯，最後在把每個線程返回的計算結果進行合併操作；在Java7中提供了一個處理大數據的fork/join框架，屏蔽掉了線程之間交互的處理，更加專注於數據的處理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule","attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"Fork/Join框架","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Fork/Join框架採用的是思想就是分而治之，把大的任務拆分成小的任務，然後放入到獨立的線程中去計算，同時爲了最大限度的利用多核CPU，採用了一個種","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"工作竊取","attrs":{}}],"attrs":{}},{"type":"text","text":"的算法來運行任務，也就是說當某個線程處理完自己工作隊列中的任務後，嘗試當其他線程的工作隊列中竊取一個任務來執行，直到所有任務處理完畢。所以爲了減少線程之間的競爭，通常會使用雙端隊列，被竊取任務線程永遠從雙端隊列的頭部拿任務執行，而竊取任務的線程永遠從雙端隊列的尾部拿任務執行；在百度找了一張圖","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/6e/6ebd3bdeb6ffeec79d871eaa011048ab.png","alt":"image","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"RecursiveTask","attrs":{}}],"attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用Fork/Join框架首先需要創建自己的任務，需要繼承","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"RecursiveTask","attrs":{}}],"attrs":{}},{"type":"text","text":"，實現抽象方法","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"protected abstract V compute();","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實現類需要在該方法中實現任務的拆分，計算，合併；僞代碼可以表示成這樣：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"if(任務已經不可拆分){\n return 順序計算結果;\n} else {\n 1.任務拆分成兩個子任務\n 2.遞歸調用本方法，拆分子任務\n 3.等待子任務執行完成\n 4.合併子任務的結果\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Fork/Join實戰","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"任務：完成對一億個自然數求和","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們先使用串行的方式實現，代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"long result = LongStream.rangeClosed(1, 100000000)\n .reduce(0, Long::sum);\nSystem.out.println(\"result：\" + result);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用Fork/Join框架實現，代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"public class SumRecursiveTask extends RecursiveTask {\n private long[] numbers;\n private int start;\n private int end;\n\n public SumRecursiveTask(long[] numbers) {\n this.numbers = numbers;\n this.start = 0;\n this.end = numbers.length;\n }\n\n public SumRecursiveTask(long[] numbers, int start, int end) {\n this.numbers = numbers;\n this.start = start;\n this.end = end;\n }\n\n @Override\n protected Long compute() {\n int length = end - start;\n if (length < 20000) { //小於20000個就不在進行拆分\n return sum();\n }\n SumRecursiveTask leftTask = new SumRecursiveTask(numbers, start, start + length / 2); //進行任務拆分\n SumRecursiveTask rightTask = new SumRecursiveTask(numbers, start + (length / 2), end); //進行任務拆分\n leftTask.fork(); //把該子任務交友ForkJoinPoll線程池去執行\n rightTask.fork(); //把該子任務交友ForkJoinPoll線程池去執行\n return leftTask.join() + rightTask.join(); //把子任務的結果相加\n }\n\n\n private long sum() {\n int sum = 0;\n for (int i = start; i < end; i++) {\n sum += numbers[i];\n }\n return sum;\n }\n\n\n public static void main(String[] args) {\n long[] numbers = LongStream.rangeClosed(1, 100000000).toArray();\n\n Long result = new ForkJoinPool().invoke(new SumRecursiveTask(numbers));\n System.out.println(\"result：\" +result);\n }\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Fork/Join默認的線程數量就是你的處理器數量，這個值是由","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Runtime.getRuntime().available- Processors()","attrs":{}}],"attrs":{}},{"type":"text","text":"得到的。 但是你可以通過系統屬性","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"java.util.concurrent.ForkJoinPool.common. parallelism","attrs":{}}],"attrs":{}},{"type":"text","text":"來改變線程池大小，如下所示： ","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"System.setProperty(\"java.util.concurrent.ForkJoinPool.common.parallelism\",\"12\");","attrs":{}}],"attrs":{}},{"type":"text","text":" 這是一個全局設置，因此它將影響代碼中所有的並行流。目前還無法專爲某個 並行流指定這個值。因爲會影響到所有的並行流，所以在任務中經歷避免網絡/IO操作，否則可能會拖慢其他並行流的運行速度","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule","attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"parallelStream","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以上我們說到的都是在Java7中使用並行流的操作，Java8並沒有止步於此，爲我們提供更加便利的方式，那就是","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"；","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"底層還是通過Fork/Join框架來實現的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"常見的使用方式","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1.串行流轉化成並行流","attrs":{}}]},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"LongStream.rangeClosed(1,1000)\n .parallel()\n .forEach(System.out::println);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2.直接生成並行流","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":" List values = new ArrayList<>();\n for (int i = 0; i < 10000; i++) {\n values.add(i);\n }\n values.parallelStream()\n .forEach(System.out::println);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正確的使用parallelStream","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"來實現上面的累加例子看看效果，代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"public static void main(String[] args) {\n Summer summer = new Summer();\n LongStream.rangeClosed(1, 100000000)\n .parallel()\n .forEach(summer::add);\n System.out.println(\"result：\" + summer.sum);\n\n}\n\nstatic class Summer {\n public long sum = 0;\n\n public void add(long value) {\n sum += value;\n }\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運行結果如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/67/6756af09b85de9f4aa46ce4bae7f1e6f.png","alt":"result","title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運行之後，我們發現運行的結果不正確，並且每次運行的結果都不一樣，這是爲什麼呢？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這裏其實就是錯用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"常見的情況，","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"是非線程安全的，在這個裏面中使用多個線程去修改了共享變量sum, 執行了","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"sum += value","attrs":{}}],"attrs":{}},{"type":"text","text":"操作，這個操作本身是非原子性的，所以在使用並行流時應該避免去修改共享變量。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"修改上面的例子，正確使用","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"來實現，代碼如下：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"long result = LongStream.rangeClosed(1, 100000000)\n .parallel()\n .reduce(0, Long::sum);\nSystem.out.println(\"result：\" + result);","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在前面我們已經說過了fork/join的操作流程是：拆子部分，計算，合併結果；因爲","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"底層使用的也是fork/join框架，所以這些步驟也是需要做的，但是從上面的代碼，我們看到","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Long::sum","attrs":{}}],"attrs":{}},{"type":"text","text":"做了計算，","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"reduce","attrs":{}}],"attrs":{}},{"type":"text","text":"做了合併結果，我們並沒有去做任務的拆分，所以這個過程肯定是","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"parallelStream","attrs":{}}],"attrs":{}},{"type":"text","text":"已經幫我們實現了，這個時候就必須的說說","attrs":{}},{"type":"codeinline","content":[{"type":"text","text":"Spliterator","attrs":{}}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"codeinline","content":[{"type":"text","text":"Spliterator","attrs":{}}],"attrs":{}},{"type":"text","text":"是Java8加入的新接口，是爲了並行執行任務而設計的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":""},"content":[{"type":"text","text":"public interface Spliterator {\n boolean tryAdvance(Consumer super T> action);\n\n Spliterator trySplit();\n\n long estimateSize();\n\n int characteristics();\n}","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"tryAdvance: 遍歷所有的元素，如果還有可以遍歷的就返回ture，否則返回false","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"trySplit: 對所有的元素進行拆分成小的子部分，如果已經不能拆分就返回null","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"estimateSize: 當前拆分裏面還剩餘多少個元素","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"characteristics: 返回當前Spliterator特性集的編碼","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule","attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"總結","attrs":{}}]},{"type":"numberedlist","attrs":{"start":"1","normalizeStart":1},"content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"要證明並行處理比順序處理效率高，只能通過測試，不能靠猜測（本文累加的例子在多臺電腦上運行了多次，也並不能證明採用並行來處理累加就一定比串行的快多少，所以只能通過多測試，環境不同可能結果就會不同）","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"數據量較少，並且計算邏輯簡單，通常不建議使用並行流","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"需要考慮流的操作時間消耗","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"在有些情況下需要自己去實現拆分的邏輯，並行流才能高效","attrs":{}}]}],"attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"horizontalrule","attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"感謝大家可以耐心地讀到這裏。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，文中或許會存在或多或少的不足、錯誤之處，有建議或者意見也非常歡迎大家在評論交流。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後，希望朋友們可以點贊評論關注三連，因爲這些就是我分享的全部動力來源🙏","attrs":{}}]}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}