Kylin 在携程的实践（上）

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"携程在 2016 年左右开始应用 Kylin 的解决方案。在 2018 年的 5、6 月份，我作为小白接手了 Kylin，逐渐琢磨、踩坑，折腾折腾就过来了。我将介绍 Kylin 在携程这一年的发展历程，碰到的挑战，以及解决的问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1 早期架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下图是携程早期的 OLAP 结构，比较简单。有两个应用，一个是 BI 分析报表工具，另一个是自助分析的 Adhoc 平台，下层主要是 Hive，技术比较单一。Hive 是比较慢的运行引擎，但是很稳定。期间我们也使用过 Shark，但 Shark 维护成本比较高，所以后面也被替换掉了。文件存储用的是 HDFS。整个架构是比较简单的，搭建过程中成本也比较低。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ab\/ff\/ab851db04ef8d2bb27283154a0ba67ff.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"早期架构的特点：一个字 "},{"type":"text","marks":[{"type":"strong"}],"text":"慢！"},{"type":"text","text":" 两字 "},{"type":"text","marks":[{"type":"strong"}],"text":"很慢！"},{"type":"text","text":" 三个字 "},{"type":"text","marks":[{"type":"strong"}],"text":"非常慢！！！"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2 技术选型"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着业务需求的多样化发展，我们团队引入了许多 OLAP 引擎，其中也包括了 Kylin。这里我们重点介绍下选择 Kylin 所考虑的几个方面："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/de\/e8\/deb34d35f3b9da9e670978a61de808e8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"百亿数据集支持"},{"type":"text","text":"："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先对我们来说，海量数据的支持必不可少的。因为很多的用户向我们抱怨，由于携程早期都是采用微软的解决方案，几乎没办法支撑百亿级的数据分析，即便使用 Hive，也需要等待很长时间才能得到结果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}