Flink1.11+Hive批流一體數倉

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"導讀"},{"type":"text","text":"： Flink從1.9.0開始提供與Hive集成的功能，隨着幾個版本的迭代，在最新的Flink 1.11中，與Hive集成的功能進一步深化，並且開始嘗試將流計算場景與Hive進行整合。本文主要分享在Flink 1.11中對接Hive的新特性，以及如何利用Flink對Hive數倉進行實時化改造，從而實現批流一體的目標。主要內容包括："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flink與Hive集成的背景介紹"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Flink 1.11中的新特性"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"打造Hive批流一體數倉"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"01 Flink與Hive集成的背景介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲什麼要做Flink和Hive集成的功能呢？最早的初衷是我們希望挖掘Flink在批處理方面的能力。衆所周知，Flink在流計算方面已經是成功的引擎了，使用的用戶也非常多。在Flink的設計理念當中，批計算是流處理中的一個特例。也就意味着，如果Flink在流計算方面做好，其實它的架構也能很好的支持批計算的場景。在批計算的場景中，SQL是一個很重要的切入點。因爲做數據分析的同學，他們更習慣使用SQL進行開發，而不是去寫DataStream或者DataSet這樣的程序。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}