闲鱼是如何实践一套完整的埋点自动化验证方案的？

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作为一款国民级二手交易App，闲鱼每天都有成千上万的二手闲置商品发布，精准的个性化的商品推荐是促进闲鱼用户快速成交的秘诀。搜索推荐算法的精准和埋点数据的准确性息息相关。一旦埋点数据出现问题，用户侧就会出现推荐商品不准确、过度推荐等问题，同时宏观的交易大盘数据的统计也会有偏差，进而影响整个商品运营策略，因此采取有效的手段来保障埋点质量就成为了闲鱼客户端质量保障的关键的一环。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"问题"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"过去的一年闲鱼客户端在首页、搜索、商品发布等核心场景下进行App体验优化与升级，在客户端快速迭代过程中经常会出现UI改版之后某些关键埋点没有上报、埋点关键字段缺失、埋点字段值不正确等问题，而这些问题在线下测试的时候由于不影响用户体感而被忽略，往往客户端版本发布之后算法或者数据同学察觉到数据异常才会回过头来定位埋点问题，问题修复代价很高，通常会追加客户端版本或者开关降级来解决埋点异常的问题。通过对迭代过程中出现的问题覆盘得出来主要有以下的急需解决的问题"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•哪些是我们需要重点保障的核心埋点"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•如何开展有效的线下埋点测试"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•如何提高埋点问题的排除效率"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对以上问题，闲鱼技术质量团队结合自身业务特点提供了一套低侵入埋点质量保障方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋点质量保障方案"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"闲鱼端上承载着数以万计的埋点，且随着业务的增长埋点个数也在不停地增多，而这些埋点由于历史原因都没有沉淀相关的说明文档，依赖开发、数据、产品同学主动梳理埋点数据显然是一件耗时费力又容易出错的事情，所以针对端上埋点质量保障我们的核心思想是：通过历史数据的分析和人工干预生成埋点画像（校验规则、值特征），优先保障核心埋点，并提供自动化测试和埋点版本对比来提升埋点数据交付的信心。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/3b\/b9\/3bd0d7b37dee34557d0f771e3556ffb9.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如图所示为闲鱼端内埋点质量保障方案，埋点数据hook后进行数据的抄送，抄送的埋点数据会分为两部分，一部分样本会交给埋点分析服务提取埋点的Key\/Value特征，进而生产埋点的校验规则；另一部分核心埋点会交给验证服务处理，参考已生成的校验规则和人工干预的校验条件会对这部分数据进行逐个校验，最后在版本灰度发布前也会提供埋点的版本比对功能来确定核心埋点是否漏报。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"核心埋点梳理"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"首先需要从成千上万的埋点中圈选出重点保障的埋点，圈选的原则是"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋点数据缺失\/异常会导致搜索推荐算法精准性"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋点数据缺失\/异常会导致大盘统计数据偏差"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋点数据缺失\/异常会导致运营投放策略"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"满足上述条件的都会被标记为闲鱼客户端核心埋点。前期我们梳理了闲鱼首页、同城、关注、搜索、详情等场景，并通过埋点所属页面（PAGE）、事件标识（ARG1）、事件类型（EVENTID）进行核心埋点的标记，最终在后续的测试中也更侧重于这部分核心埋点，而这部分的埋点的校验规则则是由样本数据分析和人工规则干预得出的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋点数据上报"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"圈选出核心埋点之后，接着需要解决的问题就是如何获取客户端埋点数据。闲鱼端上集成的埋点SDK是通过实时上报通道对埋点数据进行上报，上报后数据经过处理后会最终落到数仓中，所以说要拿到埋点数据可以从数仓中取数据，也可以在端上下功夫。由于数仓取数存在数据实时性不高、数据量大、调用链路长等问题，最终选取了客户端埋点抄送的方案。通过对端上埋点上报通道的hook来实现每一个埋点数据抄送。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如在Android端我们采用的是AOP切面拦截的方式对开发包的埋点数据进行截获然后通过HttpAPI抄送到数据接收服务。当然采用这种直接拦截代码的方式做数据抄送需要熟悉代码逻辑，做最小化的侵入。除了AOP切面拦截，类似的通用技术方案Frida[1]也是不错的选择。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋点校验规则"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了埋点数据之后接下来就是补全核心埋点的特征，例如埋点上报哪些字段、字段是必须上报的、字段的值是离散的还是可枚举的、字段在上下文场景中值的特点。后端的数据处理就会根据核心埋点的分布和版本上报数据进行样本的提取，对每个样本逐字段进行检查，并统计Key的分布、Value的分布，当样本数达到阈值之后根据历史Key\/Value分布数据就能得出以下的基础校验规则"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Key非空"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Value非空"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•Value取值范"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上面的基础规则得出之后进一步对数据进行聚类分析，就可以得出以下的场景校验规则"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•组合Key条件下Value的特征"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如：通过样本的聚类分析可以推断出类似于“同一次搜索过程中，rn参数必须保持一致”这样的规则"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"埋点自动化测试"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了数据和规则接下来就需要自动化测试脚本大显身手了。通过手工操作闲鱼App能知道对应埋点触发的时机、页面等信息，因此只要编写自动化测试代码替代人工的点击、滑动、浏览行为就可以做到埋点的自动化验证。以搜索点击核心埋点为例整体的自动化验证过程示意如下"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/2b\/34\/2bd3fd8fbf64e5ec5537f7174de3d934.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"埋点的自动化测试可以大幅提高埋点回归的效率，测试同学只需按照版本维护核心埋点自动化用例，就可以在分钟级别完成闲鱼核心埋点的自动化验证。埋点自动化测试解决了大部分的核心埋点的精准验证问题，版本比对则是在自动化之上实现埋点Diff的功能，通过不同版本埋点数据的比对快速检测出新版本中哪些埋点丢了，哪些埋点埋点格式发生了变化，进一步降低人工排除的成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/48\/80\/48c0743011979607e43ac10b5fcf5380.png","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"总结"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"闲鱼端内埋点质量保障方案对端上侵入较少，通过历史样本数据分析免去了人工主动梳理埋点的工作量，目前梳理出闲鱼端内重要场景（首页\/关注\/同城\/搜索\/发布\/详情）下的核心埋点共计100+，UI自动化验证整体回归时间由手工测试耗时0.5天下降为到分钟级别，快速的版本比对和可视化的埋点数据展示和筛选也让埋点问题排查变得更加方便。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在未来我们将继续从以下两点来进行方案的整体优化："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•埋点自动化将纳入端上集成卡点之中，即开发同学集成后就立即触发埋点的自动化验证和比对，在集成阶段就提早发现埋点问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"•自动化深度和用例的可维护性方便也是我们需要努力的方向。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"希望我们的自动化手段能让更多技术小二从重复劳动中释放出来，提升数据质量的同时也为每一个闲鱼的用户提供贴心的个性化推荐服务，为每一个闲鱼用户提供更好的购物体验。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"文章转载自： 闲鱼技术（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/jzFZUuj8C5mTUTZjjJ63GQ","title":"xxx","type":null},"content":[{"type":"text","text":"闲鱼是如何实践一套完整的埋点自动化验证方案的？"}]}]}]}