闲鱼同城第三方对接架构设计

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"闲鱼同城作为闲鱼本地生活的主战场，维护闲鱼基本盘"},{"type":"text","marks":[{"type":"strong"}],"text":"闲置物品"},{"type":"text","text":"的同时，还针对"},{"type":"text","marks":[{"type":"strong"}],"text":"闲置时间"},{"type":"text","text":"和"},{"type":"text","marks":[{"type":"strong"}],"text":"闲置空间"},{"type":"text","text":"场景进行相关的孵化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"于买家而言：淘到经济实惠的闲置物品（二手数码），打发闲置时间（兼职，服务）去挣钱。于卖家而言：闲置物品（二手数码）卖钱，闲置空间（二手房租卖）换钱。"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"闲置时间（兼职）和闲置空间（租房）区别于同城传统的闲置物品，闲置物品为传统的C2C的商品，也就是买卖双方均为C端的用户。但是对于兼职、租房等业务来说，需要供应商入驻提供供给。因此一旦涉及到第三方提供的供给，就不得不面临以下问题："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着业务的不断发展，必将有越来越多的供应商入驻。为了能让供应商快速接入，除了必备的接入文档之外在技术侧应该能有一套动态响应机制，防止在供应商接入的过程中被问题频繁的打断（双方数量对不齐、同步失败原因等）。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"每个供应商的供给质量和技术水平存在差异，如何控制好供给质量的同时保证服务的稳定性成为另一大关键因素。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"技术方案"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"整体架构"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/7a\/7a985058519c7a02764117894f5b43a2.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整体架构设计会复用中台的一些基础能力，比如用户、商品、交易等。抽象出三个领域，商家域、审核域和独立业务域（每个业务可以单独划分）。安全生产层面，为了保证系统的稳定性，围绕限流（高并发下限流重试）+ 监控（对异常的状态出对应的报警）+ 数据安全（重试下情况下保证幂等性）展开。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"数据对账"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/88\/88a9fdceb32a1787a333ac178753cb51.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"审核：复用了审核中心的能力（机审+人审）。机审：预置过滤规则，不满足过滤规则的判定为审核失败。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"开放接口能力：提供查询商品审核、校验操作日志。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"异常回调"}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"收到同步消息后会对数据进行校验，包括不限于数据字段合规性校验（长度、枚举值等）、夹带违禁词、状态更新异常（已经下架的宝贝执行下架动作）等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"针对上述初审通过，会进入审核中心二次审核，审核主要内容为语义违规。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"两种回调场景都复用了异常流转的能力如下图："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/1e\/1e4d05d2de0a02d432ff077354aa21fa.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"通过catch(LocalBizException e)的方式，将e.getCode()**和e.getMessage()**封装为response进行返回，不需要对不同的异常进行单独catch让异常逻辑在业务回调侧闭环。供应商获取到错误信息后根据错误信息修改信息进行二次同步。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"定时播报"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"采集"},{"type":"text","marks":[{"type":"strong"}],"text":"状态变更日志表"},{"type":"text","text":"和"},{"type":"text","marks":[{"type":"strong"}],"text":"业务商品表"},{"type":"text","text":"将对应的一个小时内发生状态转化的商品数量（上架、下架、编辑、审核不通过等）最后以钉钉消息播报到钉钉群中（按照钉钉的机器人api）。兼职中主要关心的指标项为上下架成功与否、是否审核失败等指标，兼职播报架构如下图所示："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/25\/256660efd4669317c408d3134323d364.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"开放接口"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"与此同时开放的接口能力提供查询商品审核、校验操作日志。接口定义如下，提供时间范围、同步id、类型、分页参数等信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/f9\/f93b435aec37b40eb9c2d5ede999e629.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：涉及图中状态日志表和商品表的插入部分参见下一章节“"},{"type":"text","marks":[{"type":"strong"}],"text":"稳定性"},{"type":"text","text":"”。"}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"稳定性（安全生产）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/a0\/a09aef32b76b32657bb8c61737f0b9d6.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"稳定性治理一直是一个系统绕不开的话题，在这个场景中涉及到第三方之间的交互，谁都无法确定对方是如何调用己方的系统。这种情况下，稳定性的重要性更加不言而喻。"}]},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"流量控制"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在数据同步时，将请求打入队列对于第三方的同步请求使用异步返回。打入队列的好处就是可以利用队列实现流量控制，削峰填谷。限流这部分依赖于阿里开源的Sentinel框架，网上对于Sentinel的分析很多这里不多加赘述。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/9d\/9da1dd15b96b9382684c195dab31cf88.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"数据一致性保证"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于存在重试操作，所以必然需要在重试过程中保证数据的正确性。"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"状态变更日志表"},{"type":"text","text":"：数据库采用的是nosql的数据库，这边会根据参数生成唯一id，进行覆盖插入保证数据的唯一性。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"业务商品表"},{"type":"text","text":"：采用先查后插的方式，同时利用分布式锁+itemId唯一键冲突保证数据的唯一性。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"拿兼职插入一致性举个栗子：ic表：底层商品表，包括一些商品的基本信息。岗位表：扩展信息的存储，工作地点、工作时间等。当一条兼职同步消息来了之后这边会涉及到两张表的维护。这里采用的方式是以下架的方式插入ic表，如果业务表成功后去更新上架ic表中状态（如下图）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/76\/7622852077fa046447cc1710dcafcef1.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"异常监控告警"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"攻击流量通常会伪装成正常流量进入。在这种情况下，系统会一直无法消费此异常消息，所以这边设置一个消费重试阈值，如果达到上限后对消息进行丢弃，同时进行系统告警（有的场景是需要强一致性保障，此时报警后需人工接入排查）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"兼职业务的告警场景包括："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"限流触发报警（持续时间超过10分钟）：限流期间被限制的消息业务会主动进行重试，控制重试n次整体持续时间不会超过10分钟，如果限流超过10分钟认定为异常情况会进行告警。（通常来说是供应商大批量上下架岗位导致，未通知前提下认定为他们系统问题）"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"状态更新失败（持续时间超过5分钟，每分钟数量大于n）：小批量的更新失败可以理解为是垃圾数据，持续时间过于长可以理解为供应商系统异常。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"总结和展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本章节主要介绍了闲鱼同城业务在“闲置时间”和\"闲置空间“场景下针对与第三方系统对接的过程中开发资源和稳定性问题展开。通过上述方案也解决了在开篇提到的2个问题："}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"动态响应机制：商品同步时通过实时回调和异步回调的方式将商品的每个异常状态返回给供应商；提供了小时时间维度的统计播报，最后以钉钉消息通知至钉钉群中，如若发现异常也可根据开放接口去查询商品历史变更状态。这样就能很大程度上解放开发，不会因为对账的问题被频繁的打断。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"稳定性治理：通过接口限流保证异常流量打满线程池进而影响系统；通过接口幂等保证数据的安全唯一性；通过监控（搭配合适的报警规则）去监控异常场景，如若出现问题人为介入。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"随着业务越来越复杂，对应的独立业务域也将会越来越多，在独立业务域上的开发精力也会越来越多。能否根据大量复杂业务场景的输入找到共同点抽象出较为理想的架构将是努力的目标。"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"抽象出独立业务域中的共同点，推动业务完善路径：完善租房的订单和履约路径，统一抽取出订单域和履约域。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对不同业务的不同商家统一商家管理平台，现阶段每个业务都有自己的一套接入方式。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文转载自：闲鱼技术（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/kSKWk09Sr7kfC8_NDB0Ovw","title":"xxx","type":null},"content":[{"type":"text","text":"闲鱼同城第三方对接架构设计"}]}]}]}