社区内容流量调控系统设计——成本与指标平衡的背后

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在闲鱼，除了可以进行闲置交易，当你点击第二个tab“会玩”后，还会发现一个好玩、有趣的内容社区。内容分发作为社区的重要环节，核心目标是实现内容与用户之间的匹配效率最大化，核心指标就是内容的点击率（CTR）和停留时长，这也正是算法推荐模型的优化方向。同时运营在内容分发环节有着各种各样的需求，主要有以下几类："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c1\/c1551d90c7cffde103ee7ea7d6ffa4e2.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 活动运营：对于社会热点、节日氛围和品牌活动相关的内容，运营需要进行提前布局，给予更多流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 社区生态：存量的内容由于历史积累的行为数据更多，更容易被算法推荐，导致流量分发存在马太效应，内容新鲜度不够。同时对于擦边球的内容，虽然CTR很高，但是从长期平台生态来看，需要限制其流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 流量扶持：对于核心内容创作者、BD作者，运营需要进行一定的流量扶持，长期来看可以促进内容供给。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果单纯依赖推荐算法实现以上需求会带来2个问题，一是开发成本高，二是有的运营需求短期会带来核心指标的下降，和算法模型的优化目标是冲突的。因此我们设计了一套流量调控系统，面向运营提供灵活的流量调控能力，在内容分发时打通推荐算法，快速实现运营需求，同时兼顾核心业务指标。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"系统整体设计"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c6\/c6115b912a03e279bef6409085e9402d.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"运营的需求整体可抽象为两类："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1.针对具体内容的流量调控任务，核心是对具体内容，在具体时间、具体分发场景、针对具体目标人群、实现具体的调控目标（曝光量）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2.基于规则的流量调控，即当特定事件发生时，自动生成流量调控任务。比如当特定等级的创作者发布了特定质量的内容时，我们在一阶段给与一定的曝光量，如果曝光目标完成后，同时CTR指标达到一定标准，会再次给予一定的曝光量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"针对上面的需求，我们设计了策略链路动态创建具体调控任务，然后通过调控链路实现调控任务的管理和调度，同时调控链路也支持运营手动创建调控任务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分发链路则打通了推荐系统，在完成调控任务的同时，尽可能的优化核心指标。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量数据处理服务负责采集用户的曝光埋点数据，为流量调控模块提供实时的调控任务状态更新，同时沉淀数据资产，供流量洞察分析使用，最终在流量策略中心运营后台为运营提供决策支撑。限于篇幅，这部分内容后续有机会再和大家分享。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"具体方案实现"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/72\/30\/7298585d9fd0a512658b10e39f90c930.jpeg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"策略链路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"策略事件接入将各种异构的消息事件进行归一化处理，形成标准的策略事件，供后续的各种策略执行器进行订阅。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/00\/0095079a0b757f425aac4e27f700ba2f.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"运营的各种调控策略被抽象为标准的策略处理器（StrategyManager），当监听到策略事件后，策略路由器将其转发给订阅了该事件的策略执行器，策略执行器处理事件，创建相应的调控任务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6d\/6d38d987f316f3427419f4e859da7bba.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"调控链路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"调控任务生成后，经过任务调度，实时同步到算法所需的调控内容池（采用图数据库存储），进行后续的内容分发。当内容在指定的分发场景曝光后，数据采集模块会实时采集客户端上报的用户行为日志埋点，进行3部分操作："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 通过处理产生曝光、点击等数据指标并存储。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 产生干预消息，实时更新调控内容池。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 判断指标是否达到目标曝光值，产生任务结束消息，结束任务调控。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"数据采集需要满足高吞吐、低延时，采用流处理平台实现。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6b\/6bb512f14515601705fa15a148c58017.png","alt":"图片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"分发链路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 召回：推荐系统在收到前台请求的时候，同时从调控内容池（冷启动内容池、保量内容池）以及大盘正常召回链路进行个性化召回，保证即使是调控内容，也和用户兴趣相匹配。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 排序：推荐系统会通过相关的核心指标预估模型，以及特定类型调控任务（如冷启动）的算法模型，对内容进行排序。投放系统会根据运营设置的优先级，进行排序。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"总结和展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前实现了运营保量投放能力，以及特定创作者的内容冷启动策略，其中"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 运营手动保量投放调控日新增任务数30+，调控曝光百万级别。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 针对特定作者的冷启动调控业务实现了对于优质创作者的流量扶持，极大地激励了创作者积极性，日均发帖量提升14.4%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"后续针对流量调控系统将进行进一步的能力升级："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 多目标调控，除曝光量外增加点击，甚至涨粉能力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 策略配置化。当前通过代码的方式实现运营策略，后续可实现调控策略配置化，热更新。实现运营策略的0代码、SOP化支持。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文转载自：闲鱼技术（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文链接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/RLGZlNtvMWeC1OZDYgU8rQ","title":"xxx","type":null},"content":[{"type":"text","text":"社区内容流量调控系统设计——成本与指标平衡的背后"}]}]}]}