ClickHouse在手淘流量分析业务实践

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"导读：本文主要介绍手淘流量分析业务发展过程中，实时性业务分析需求的产生，实时分析目标的设定，如何进行技术的选型，以及如何基于ClickHouse构建系统架构和未来的业务预期。主要内容包括："}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量分析与业务背景：什么是流量分析，以及我们的业务背景"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"\"大数据\"带来的难题：当你的数据量是守恒的时候，需要怎么处理你的数据"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"技术选型与产品考虑：在以上背景下，我们在技术选择和产品考虑时，都做了哪些考虑，以及为什么最终选择ClickHouse，并给大家介绍一些技术解决方案"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"流量分析与业务背景"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 流量分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/f6\/09\/f65b4c292805e7998713cba46f78be09.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，流量分析到底是什么？ 从最基本的角度来说流量分析就是底层的数据模型加上指标体系。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"底层数据模型："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"底层数据模型是把不同的用户行为数据，先放到一个最基本的叫做“事件”的数据模型中，这是一个单事件的数据模型。与此单个事件数据模型的上一层，形成一个路径的实现模型，可以把一些数据，比如一些流量数据或者一些业务内部数据同交易数据做关联。在此基础上，可以做规定的分析，后续也可以做更多的不同分析。既可以从企业整体来看，也可以从单个业务着手，例如：淘宝有很多个行业，可以从行业视角来分析数据；淘宝有许多新用户和老用户，可以从用户角度来分析数据。所以，一旦有了这个底层数据后， 我们用很多不同的方法来分析这些数据，每一种分析方法产出的指标其实是一样的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"指标体系："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我们通常用以下四种指标来分析数据："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量规模是多少，有多少UV，PV。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"参与度，比如说停留时长，浏览深度。以目前火爆的直播为例，我们要看下直播的参与度，例如：在一次直播中，交互多少次，点击多少次等一系列操作。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"转化，行业对转化的理解就是让用户做你想让他做的事情，比如说转发、收藏、购买。此外，还有一些其他类型的转化：对于视频产品， 转化就是电视剧的完播率；对与社交产品，转化是用户注册或者分享页面；以及根据业务场景定义的转化。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"粘性，就是你花了多长时间把用户拉过来，让用户完成一件事情，并且了解用户对此具体业务有没有粘性。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于业务的复杂度，我们会理解这些不同的数据，并且按照不同的维度来做切分和汇总。在大数据背景下，很多东西和ClickHouse自有技术是密切相关的，这也是为什么最终选择了ClickHouse做我们的技术方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}