Kylin 在攜程的實踐（上）

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"攜程在 2016 年左右開始應用 Kylin 的解決方案。在 2018 年的 5、6 月份，我作爲小白接手了 Kylin，逐漸琢磨、踩坑，折騰折騰就過來了。我將介紹 Kylin 在攜程這一年的發展歷程，碰到的挑戰，以及解決的問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1 早期架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下圖是攜程早期的 OLAP 結構，比較簡單。有兩個應用，一個是 BI 分析報表工具，另一個是自助分析的 Adhoc 平臺，下層主要是 Hive，技術比較單一。Hive 是比較慢的運行引擎，但是很穩定。期間我們也使用過 Shark，但 Shark 維護成本比較高，所以後面也被替換掉了。文件存儲用的是 HDFS。整個架構是比較簡單的，搭建過程中成本也比較低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ab\/ff\/ab851db04ef8d2bb27283154a0ba67ff.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"早期架構的特點：一個字 "},{"type":"text","marks":[{"type":"strong"}],"text":"慢！"},{"type":"text","text":" 兩字 "},{"type":"text","marks":[{"type":"strong"}],"text":"很慢！"},{"type":"text","text":" 三個字 "},{"type":"text","marks":[{"type":"strong"}],"text":"非常慢！！！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2 技術選型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着業務需求的多樣化發展，我們團隊引入了許多 OLAP 引擎，其中也包括了 Kylin。這裏我們重點介紹下選擇 Kylin 所考慮的幾個方面："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/de\/e8\/deb34d35f3b9da9e670978a61de808e8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"百億數據集支持"},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先對我們來說，海量數據的支持必不可少的。因爲很多的用戶向我們抱怨，由於攜程早期都是採用微軟的解決方案，幾乎沒辦法支撐百億級的數據分析，即便使用 Hive，也需要等待很長時間才能得到結果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}