閒魚是如何實踐一套完整的埋點自動化驗證方案的？

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作爲一款國民級二手交易App，閒魚每天都有成千上萬的二手閒置商品發佈，精準的個性化的商品推薦是促進閒魚用戶快速成交的祕訣。搜索推薦算法的精準和埋點數據的準確性息息相關。一旦埋點數據出現問題，用戶側就會出現推薦商品不準確、過度推薦等問題，同時宏觀的交易大盤數據的統計也會有偏差，進而影響整個商品運營策略，因此採取有效的手段來保障埋點質量就成爲了閒魚客戶端質量保障的關鍵的一環。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"過去的一年閒魚客戶端在首頁、搜索、商品發佈等核心場景下進行App體驗優化與升級，在客戶端快速迭代過程中經常會出現UI改版之後某些關鍵埋點沒有上報、埋點關鍵字段缺失、埋點字段值不正確等問題，而這些問題在線下測試的時候由於不影響用戶體感而被忽略，往往客戶端版本發佈之後算法或者數據同學察覺到數據異常纔會回過頭來定位埋點問題，問題修復代價很高，通常會追加客戶端版本或者開關降級來解決埋點異常的問題。通過對迭代過程中出現的問題覆盤得出來主要有以下的急需解決的問題"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•哪些是我們需要重點保障的核心埋點"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•如何開展有效的線下埋點測試"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•如何提高埋點問題的排除效率"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對以上問題，閒魚技術質量團隊結合自身業務特點提供了一套低侵入埋點質量保障方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋點質量保障方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚端上承載着數以萬計的埋點，且隨着業務的增長埋點個數也在不停地增多，而這些埋點由於歷史原因都沒有沉澱相關的說明文檔，依賴開發、數據、產品同學主動梳理埋點數據顯然是一件耗時費力又容易出錯的事情，所以針對端上埋點質量保障我們的核心思想是：通過歷史數據的分析和人工干預生成埋點畫像（校驗規則、值特徵），優先保障核心埋點，並提供自動化測試和埋點版本對比來提升埋點數據交付的信心。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/3b\/b9\/3bd0d7b37dee34557d0f771e3556ffb9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如圖所示爲閒魚端內埋點質量保障方案，埋點數據hook後進行數據的抄送，抄送的埋點數據會分爲兩部分，一部分樣本會交給埋點分析服務提取埋點的Key\/Value特徵，進而生產埋點的校驗規則；另一部分核心埋點會交給驗證服務處理，參考已生成的校驗規則和人工干預的校驗條件會對這部分數據進行逐個校驗，最後在版本灰度發佈前也會提供埋點的版本比對功能來確定核心埋點是否漏報。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"核心埋點梳理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先需要從成千上萬的埋點中圈選出重點保障的埋點，圈選的原則是"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋點數據缺失\/異常會導致搜索推薦算法精準性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋點數據缺失\/異常會導致大盤統計數據偏差"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋點數據缺失\/異常會導致運營投放策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"滿足上述條件的都會被標記爲閒魚客戶端核心埋點。前期我們梳理了閒魚首頁、同城、關注、搜索、詳情等場景，並通過埋點所屬頁面（PAGE）、事件標識（ARG1）、事件類型（EVENTID）進行核心埋點的標記，最終在後續的測試中也更側重於這部分核心埋點，而這部分的埋點的校驗規則則是由樣本數據分析和人工規則干預得出的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋點數據上報"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圈選出核心埋點之後，接着需要解決的問題就是如何獲取客戶端埋點數據。閒魚端上集成的埋點SDK是通過實時上報通道對埋點數據進行上報，上報後數據經過處理後會最終落到數倉中，所以說要拿到埋點數據可以從數倉中取數據，也可以在端上下功夫。由於數倉取數存在數據實時性不高、數據量大、調用鏈路長等問題，最終選取了客戶端埋點抄送的方案。通過對端上埋點上報通道的hook來實現每一個埋點數據抄送。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如在Android端我們採用的是AOP切面攔截的方式對開發包的埋點數據進行截獲然後通過HttpAPI抄送到數據接收服務。當然採用這種直接攔截代碼的方式做數據抄送需要熟悉代碼邏輯，做最小化的侵入。除了AOP切面攔截，類似的通用技術方案Frida[1]也是不錯的選擇。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋點校驗規則"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了埋點數據之後接下來就是補全核心埋點的特徵，例如埋點上報哪些字段、字段是必須上報的、字段的值是離散的還是可枚舉的、字段在上下文場景中值的特點。後端的數據處理就會根據核心埋點的分佈和版本上報數據進行樣本的提取，對每個樣本逐字段進行檢查，並統計Key的分佈、Value的分佈，當樣本數達到閾值之後根據歷史Key\/Value分佈數據就能得出以下的基礎校驗規則"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Key非空"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Value非空"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Value取值範"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的基礎規則得出之後進一步對數據進行聚類分析，就可以得出以下的場景校驗規則"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•組合Key條件下Value的特徵"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：通過樣本的聚類分析可以推斷出類似於“同一次搜索過程中，rn參數必須保持一致”這樣的規則"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋點自動化測試"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了數據和規則接下來就需要自動化測試腳本大顯身手了。通過手工操作閒魚App能知道對應埋點觸發的時機、頁面等信息，因此只要編寫自動化測試代碼替代人工的點擊、滑動、瀏覽行爲就可以做到埋點的自動化驗證。以搜索點擊核心埋點爲例整體的自動化驗證過程示意如下"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2b\/34\/2bd3fd8fbf64e5ec5537f7174de3d934.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"埋點的自動化測試可以大幅提高埋點回歸的效率，測試同學只需按照版本維護核心埋點自動化用例，就可以在分鐘級別完成閒魚核心埋點的自動化驗證。埋點自動化測試解決了大部分的核心埋點的精準驗證問題，版本比對則是在自動化之上實現埋點Diff的功能，通過不同版本埋點數據的比對快速檢測出新版本中哪些埋點丟了，哪些埋點埋點格式發生了變化，進一步降低人工排除的成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/48\/80\/48c0743011979607e43ac10b5fcf5380.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"閒魚端內埋點質量保障方案對端上侵入較少，通過歷史樣本數據分析免去了人工主動梳理埋點的工作量，目前梳理出閒魚端內重要場景（首頁\/關注\/同城\/搜索\/發佈\/詳情）下的核心埋點共計100+，UI自動化驗證整體迴歸時間由手工測試耗時0.5天下降爲到分鐘級別，快速的版本比對和可視化的埋點數據展示和篩選也讓埋點問題排查變得更加方便。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在未來我們將繼續從以下兩點來進行方案的整體優化："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋點自動化將納入端上集成卡點之中，即開發同學集成後就立即觸發埋點的自動化驗證和比對，在集成階段就提早發現埋點問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•自動化深度和用例的可維護性方便也是我們需要努力的方向。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"希望我們的自動化手段能讓更多技術小二從重複勞動中釋放出來，提升數據質量的同時也爲每一個閒魚的用戶提供貼心的個性化推薦服務，爲每一個閒魚用戶提供更好的購物體驗。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章轉載自： 閒魚技術（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/jzFZUuj8C5mTUTZjjJ63GQ","title":"xxx","type":null},"content":[{"type":"text","text":"閒魚是如何實踐一套完整的埋點自動化驗證方案的？"}]}]}]}