深入淺出Spark(三):Spark調度系統之“權力的遊戲”

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"專題介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2009 年,Spark 誕生於加州大學伯克利分校的 AMP 實驗室(the Algorithms, Machines and People lab),並於 2010 年開源。2013 年,Spark 捐獻給阿帕奇軟件基金會(Apache Software Foundation),並於 2014 年成爲 Apache 頂級項目。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如今,十年光景已過,Spark 成爲了大大小小企業與研究機構的常用工具之一,依舊深受不少開發人員的喜愛。如果你是初入江湖且希望瞭解、學習 Spark 的“小蝦米”,那麼 InfoQ 與 FreeWheel 技術專家吳磊合作的專題系列文章——"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.cn\/theme\/84","title":"xxx","type":null},"content":[{"type":"text","text":"《深入淺出 Spark:原理詳解與開發實踐》"}]},{"type":"text","text":"一定適合你!"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文系專題系列第三篇。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"令狐沖:“我要退出江湖,從此不問江湖之事。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"任我行:“這個世界有人的地方就有恩怨,有恩怨就有江湖,人就是江湖,你怎麼退出?”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"感謝各位看官在百忙之中來聽我說書,真是太給面子啦!前文書《"},{"type":"link","attrs":{"href":"https:\/\/www.infoq.cn\/article\/LBzKJPoaFAre5c0cI4ur","title":"xxx","type":null},"content":[{"type":"text","text":"內存計算的由來 ——DAG"}]},{"type":"text","text":"》咱們說到DAGScheduler以首尾倒置的方式從後向前回溯DAG計算圖,沿途以Shuffle爲邊界劃分Stages。那麼,這些Stages劃分出來之後有什麼用呢?DAGScheduler如何將DAG劃分出的Stages轉化爲可執行的分佈式任務?本期“權力的遊戲”將帶您走進Spark調度系統,筆者將竭盡全力與您一起揭開Spark調度系統的神祕面紗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本段書正式開始之前,咱們先得鋪墊鋪墊,畢竟保不齊有剛入座的看官頭一次來咱們書棚,咱們都得照顧到不是。在講Spark調度系統之前,咱們先來簡單回顧一下Spark分佈式系統架構和重要概念。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章