58同城风控平台演进

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"导读：","attrs":{}},{"type":"text","text":"58同城的风控业务从2010年开始搭建平台，到2012年正式成立信息质量部，经历过了10年的发展，三代系统变革，目前已经是全集团最重要的中台之一，肩负了每天百亿级别数据的风险控制。本次分享将主要解析在不同业态、不同年代下，风控治理的不同思路，进而打造出不同平台的过程。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"58做风控的起因","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一个分类信息平台，其主要是以内容和流量为核心，包括上下游两大类用户：B端用户（供应端）——生产内容以及C端（消费端）用户——使用内容。在信息的产生和消费两端，都会面临许多问题，诸如诈骗、羊毛党、刷单、刻章、办证、发票、微信吸粉、广告、水贴等一系列非正常流量行为。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"面临的问题：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"诈骗是最为严重的一种行为，是红线以下，对个体伤害巨大。从2014-2019年中国网络诈骗举报数量以及中国网络诈骗人均损失趋势两个统计表可以看到，随着国家的管控力度逐步增大，诈骗数量也呈现逐年下降的趋势，但对于个体伤害而言，却是逐年增加的。我们能从中看出，一方面人们的收入水平逐年上升，另一方面，诈骗的手段逐渐更新迭代，隐蔽性增强，更多高知分子也逐步被卷入骗局。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"羊毛党主要指的是借助各类营销活动薅羊毛，诸如之前的某多多被曝出现重大Bug，用户可领100元无门槛券，吸引了众多羊毛党“薅羊毛”，网传“一夜损失了200亿”；更有甚者，羊毛党自发组织形成团体，可以搜到很多以“羊毛”为关键词的QQ群，仅在2019年，58利用技术手段挽回的各类羊毛党损失就超过千万级别。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/66/66585288c8863ef1add72dc715a5b971.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刷单是电商平台面临的头疼难题，这里会分为两类情况，一种是为各电商平台商家刷量，赚取收益，另一类则回归到诈骗渠道，诈骗贪小便宜的各种“兼职”人员。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/e2/e2421a11129cdf710645a981919c4981.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"刻章、办证、发票为平台的灰色地带，这其中良莠不齐，正规的诸如招聘类目下的财务人员会有开具发票的情形，但类似代开发票这种行为已经触碰红线了。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"微信吸粉是各类恶意行为中套路最丰富的，诸如房产类目会存在假冒房东发假房源信息，吸引房客咨询，实际为了吸粉的营销活动。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"广告水贴是各类UGC平台面临的普遍问题，行为上也和正常用户趋紧，治理难度较高。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"58风控发展历程","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"58风控发展历程主要包括4个阶段：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8e/8e9027e685a88974cf9d6bbb516fe4c6.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第1阶段：雏形阶段——微量业务，封闭系统","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第2阶段：发展阶段——少量业务，增强运营","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第3阶段：中台转变——能力复用，业务自治","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第4阶段：生态建设——服务思维","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第一阶段：微量机审+人工审核","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/75/75a207298081a018c3194de3a573cc88.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第一阶段平台业务较少，主要为58信息、企业发布以及简历发布，采用人工审核配合部分机器审核的手段。此阶段主要规范了系统的边际，包括UGC产生后的一系列流程操作。经过在线检测后，分成了两个分支，其一经过离线检测，其二经过人工审核，并行操作，两者的结果最终通过判断执行模块进行审核执行操作。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为了加强对外部情势的感知，主要主动采用“卧底”黑产群，通过购买各类黑产软件，探索平台漏洞，对系统进行更新迭代。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"暴露问题：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"该阶段对于系统中的“变”和“不变”部分缺乏抽象，主要是采用硬编码的方式，这使得开发频次及成本较高，策略上线周期上，开发速度远远无法跟上黑产破解的速度。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第二阶段：配置化机审+人工审核","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/8d/8d486f3b52b679c9e46d3f625a67191d.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第二阶段主要是在前一阶段的基础上，明确了策略层的运营能力，以特征为基础，规则和条例为抓手，同时也增加了部分算法的能力，诸如图像算法、文本算法以及行为聚类等。在此基础上形成了三大模块，包括简化特征开发的平台用于风险发现、可运营的策略管理用于风险评估以及集中化的风险处理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"暴露问题：","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"并购：58和赶集风控系统同时存在，人员重叠，维护代价高；","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"业务增长：风控场景快速增加，技术和运营成本的不断增加；","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"新思想：友商风控平台的快速崛起，学习借鉴友商思路；","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"中台：部门定位的变化，逐步考虑中台化。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第三阶段：机审人审融合、分场景治理","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/68/687b4e5e5f181b788af7e380ae9a4473.jpeg","alt":"图片","title":null,"style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第三阶段目前运行至今，主要包括三方面，包括","attrs":{}}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"自助式开发：改变原有仅风控部门独立开发的模式，提供业务侧参与风控工具开发的能力，同时能够进行整个开发后的测试、部署、上线工作。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一站式运营：提供了一整套的运营方案，运营人员可以快速完成业务注册。同时积极提供自助接入场景的能力，在新场景接入时提供基础的通用能力，助力业务快速落地构建防御体系；同时，也能够提供一些专家经验，输出前期总结经验，避免反复造轮子。","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"业务隔离：改变了前期多个业务共用一套风控模型的方案，第三阶段实现了业务间进程级别的隔离以及微服务的拆分，避免业务间问题的相互影响；同时增加了熔断降级等各类服务治理能力，避免上下游数据服务造成的影响。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第三阶段整体实现了毫秒级的响应速度、目前已支撑千级别业务场景、全场景内容服务覆盖、上万节点集群规模、百亿量级离线分析能力以及全自助平台管理。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"第四阶段：专家指挥、智能审核","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"第四阶段为正在实现中以及对未来的规划方面，包括：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隔离方面，目前业务间的隔离还没有做到极致，后期会对数据库、人工审核、配置中心等进行隔离；","attrs":{}}]}],"attrs":{}},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"以算法为中心，逐步提升系统自动运营水平。","attrs":{}}]}],"attrs":{}}],"attrs":{}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"整体而言，风控的能力提升有很大的空间，这对技术储备要求很高，需要多职能跨部门、跨组织合作，从而实现高并发、高可用需求下的快速风控能力。当然，风控其实是一个不断对抗的过程，版本的迭代需要足够快，以交付为目标，避免过度设计导致长时间的对抗真空期。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"后续答疑","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"① 能否举一个58同城黑产攻防案例？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"答：常见的攻防案例一般是敌方攻击、我方防守，对抗的过程。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"当时有遇到一个案例，房产业务因为流量大，关注度很高，有利用工具批量发布信息的行为。开始阶段能够从中发现，其发布内容行为与正常人发布有一定的偏差，常见的手段就是通过模拟批量Post请求提交数据，但这种情况下上报的数据中缺乏正常用户的行为数据，很容易可以识别到为非真人发布而进行拦截。后续，对方也会逐渐升级手段，寻找可突破点，诸如采用按键精灵，通过浏览器进行提交，用模拟点击的方式。我们解决方案则是通过寻找相似点以及用户行为时间序列来发现非真人操作，之后采用动作随机化，这时从单一用户的角度已经很难着手，我们开始从单一用户转向群体用户行为的分析，挖掘某时间段内存在大量异常行为的账户，通过诸如验证码、认证等手段增加对抗的成本，黑产也在升级过程中采用包括打码平台等方式提高刷帖成功率，我们则增加验证的方式，用更复杂的人机交互等，可以看出这过程是不断对抗，提升作恶成本的过程。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"② 详细介绍下当前风控架构？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"答：最上层为业务层，包括58信息、企业发布、简历发布、同镇信息、交友直播、微聊信息、房产三网、部落视频等上千级别业务；针对每个业务集合，我们会提供一站式运营平台，包括完整的工具集、处理集、人工审核布局以及运营能力服务；在此基础上，针对多业务间并存的服务能力进行抽象，整合出包括基础工具、风险处理集合、数据增广、行为聚类、文本算法、图像算法等通用化服务能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"③ 风控团队如何对接业务侧，如何说服业务侧接入，投入产出比如何计算等？","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"答：早期所有的风控均处于风控部门，既承担裁判员的职责，又做运动员的工作，即做风控并评判风控收益，会存在与业务侧产生冲突矛盾的地方。这其中包括业务侧不认可风控的输出能力，认为引入后为带来大量的误杀，影响业务发展等各类问题。后期通过中台能力的搭建以及与业务部门责任共担，实现了双方合作共赢。投入产出比可以通过对线上数据的巡查，以及数据部门的分析，因其中包含了部分业务因素，我们也会采用三方数据、申诉反馈等综合收集信息客观反映风控能力。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"嘉宾介绍：","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"张月，58同城信息安全部架构师，2010年加入58同城。主要负责集团的风控设计以及实时引擎的实现。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接：","attrs":{}},{"type":"link","attrs":{"href":"https://mp.weixin.qq.com/s/x2JjbIc8Br7qlHTZxLn4sQ","title":""},"content":[{"type":"text","text":"58同城风控平台演进","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}