ClickHouse在手淘流量分析業務實踐

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"導讀：本文主要介紹手淘流量分析業務發展過程中，實時性業務分析需求的產生，實時分析目標的設定，如何進行技術的選型，以及如何基於ClickHouse構建系統架構和未來的業務預期。主要內容包括："}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量分析與業務背景：什麼是流量分析，以及我們的業務背景"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"\"大數據\"帶來的難題：當你的數據量是守恆的時候，需要怎麼處理你的數據"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"技術選型與產品考慮：在以上背景下，我們在技術選擇和產品考慮時，都做了哪些考慮，以及爲什麼最終選擇ClickHouse，並給大家介紹一些技術解決方案"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"流量分析與業務背景"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 流量分析"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/f6\/09\/f65b4c292805e7998713cba46f78be09.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先，流量分析到底是什麼？ 從最基本的角度來說流量分析就是底層的數據模型加上指標體系。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"底層數據模型："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"底層數據模型是把不同的用戶行爲數據，先放到一個最基本的叫做“事件”的數據模型中，這是一個單事件的數據模型。與此單個事件數據模型的上一層，形成一個路徑的實現模型，可以把一些數據，比如一些流量數據或者一些業務內部數據同交易數據做關聯。在此基礎上，可以做規定的分析，後續也可以做更多的不同分析。既可以從企業整體來看，也可以從單個業務着手，例如：淘寶有很多個行業，可以從行業視角來分析數據；淘寶有許多新用戶和老用戶，可以從用戶角度來分析數據。所以，一旦有了這個底層數據後， 我們用很多不同的方法來分析這些數據，每一種分析方法產出的指標其實是一樣的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"指標體系："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們通常用以下四種指標來分析數據："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量規模是多少，有多少UV，PV。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"參與度，比如說停留時長，瀏覽深度。以目前火爆的直播爲例，我們要看下直播的參與度，例如：在一次直播中，交互多少次，點擊多少次等一系列操作。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"轉化，行業對轉化的理解就是讓用戶做你想讓他做的事情，比如說轉發、收藏、購買。此外，還有一些其他類型的轉化：對於視頻產品， 轉化就是電視劇的完播率；對與社交產品，轉化是用戶註冊或者分享頁面；以及根據業務場景定義的轉化。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"粘性，就是你花了多長時間把用戶拉過來，讓用戶完成一件事情，並且瞭解用戶對此具體業務有沒有粘性。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於業務的複雜度，我們會理解這些不同的數據，並且按照不同的維度來做切分和彙總。在大數據背景下，很多東西和ClickHouse自有技術是密切相關的，這也是爲什麼最終選擇了ClickHouse做我們的技術方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}