社區內容流量調控系統設計——成本與指標平衡的背後

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在閒魚，除了可以進行閒置交易，當你點擊第二個tab“會玩”後，還會發現一個好玩、有趣的內容社區。內容分發作爲社區的重要環節，核心目標是實現內容與用戶之間的匹配效率最大化，核心指標就是內容的點擊率（CTR）和停留時長，這也正是算法推薦模型的優化方向。同時運營在內容分發環節有着各種各樣的需求，主要有以下幾類："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c1\/c1551d90c7cffde103ee7ea7d6ffa4e2.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 活動運營：對於社會熱點、節日氛圍和品牌活動相關的內容，運營需要進行提前佈局，給予更多流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 社區生態：存量的內容由於歷史積累的行爲數據更多，更容易被算法推薦，導致流量分發存在馬太效應，內容新鮮度不夠。同時對於擦邊球的內容，雖然CTR很高，但是從長期平臺生態來看，需要限制其流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 流量扶持：對於核心內容創作者、BD作者，運營需要進行一定的流量扶持，長期來看可以促進內容供給。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果單純依賴推薦算法實現以上需求會帶來2個問題，一是開發成本高，二是有的運營需求短期會帶來核心指標的下降，和算法模型的優化目標是衝突的。因此我們設計了一套流量調控系統，面向運營提供靈活的流量調控能力，在內容分發時打通推薦算法，快速實現運營需求，同時兼顧核心業務指標。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"系統整體設計"}]},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/c6\/c6115b912a03e279bef6409085e9402d.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運營的需求整體可抽象爲兩類："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1.針對具體內容的流量調控任務，核心是對具體內容，在具體時間、具體分發場景、針對具體目標人羣、實現具體的調控目標（曝光量）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2.基於規則的流量調控，即當特定事件發生時，自動生成流量調控任務。比如當特定等級的創作者發佈了特定質量的內容時，我們在一階段給與一定的曝光量，如果曝光目標完成後，同時CTR指標達到一定標準，會再次給予一定的曝光量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"針對上面的需求，我們設計了策略鏈路動態創建具體調控任務，然後通過調控鏈路實現調控任務的管理和調度，同時調控鏈路也支持運營手動創建調控任務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"分發鏈路則打通了推薦系統，在完成調控任務的同時，儘可能的優化核心指標。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"流量數據處理服務負責採集用戶的曝光埋點數據，爲流量調控模塊提供實時的調控任務狀態更新，同時沉澱數據資產，供流量洞察分析使用，最終在流量策略中心運營後臺爲運營提供決策支撐。限於篇幅，這部分內容後續有機會再和大家分享。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"具體方案實現"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/72\/30\/7298585d9fd0a512658b10e39f90c930.jpeg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"策略鏈路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"策略事件接入將各種異構的消息事件進行歸一化處理，形成標準的策略事件，供後續的各種策略執行器進行訂閱。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/00\/0095079a0b757f425aac4e27f700ba2f.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"運營的各種調控策略被抽象爲標準的策略處理器（StrategyManager），當監聽到策略事件後，策略路由器將其轉發給訂閱了該事件的策略執行器，策略執行器處理事件，創建相應的調控任務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6d\/6d38d987f316f3427419f4e859da7bba.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"調控鏈路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"調控任務生成後，經過任務調度，實時同步到算法所需的調控內容池（採用圖數據庫存儲），進行後續的內容分發。當內容在指定的分發場景曝光後，數據採集模塊會實時採集客戶端上報的用戶行爲日誌埋點，進行3部分操作："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 通過處理產生曝光、點擊等數據指標並存儲。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 產生干預消息，實時更新調控內容池。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 判斷指標是否達到目標曝光值，產生任務結束消息，結束任務調控。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"數據採集需要滿足高吞吐、低延時，採用流處理平臺實現。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.geekbang.org\/infoq\/6b\/6bb512f14515601705fa15a148c58017.png","alt":"圖片","title":"null","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":null,"fromPaste":true,"pastePass":true}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"分發鏈路"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 召回：推薦系統在收到前臺請求的時候，同時從調控內容池（冷啓動內容池、保量內容池）以及大盤正常召回鏈路進行個性化召回，保證即使是調控內容，也和用戶興趣相匹配。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 排序：推薦系統會通過相關的核心指標預估模型，以及特定類型調控任務（如冷啓動）的算法模型，對內容進行排序。投放系統會根據運營設置的優先級，進行排序。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結和展望"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前實現了運營保量投放能力，以及特定創作者的內容冷啓動策略，其中"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 運營手動保量投放調控日新增任務數30+，調控曝光百萬級別。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 針對特定作者的冷啓動調控業務實現了對於優質創作者的流量扶持，極大地激勵了創作者積極性，日均發帖量提升14.4%。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"後續針對流量調控系統將進行進一步的能力升級："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 多目標調控，除曝光量外增加點擊，甚至漲粉能力。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"• 策略配置化。當前通過代碼的方式實現運營策略，後續可實現調控策略配置化，熱更新。實現運營策略的0代碼、SOP化支持。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本文轉載自：閒魚技術（ID：XYtech_Alibaba）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文鏈接："},{"type":"link","attrs":{"href":"https:\/\/mp.weixin.qq.com\/s\/RLGZlNtvMWeC1OZDYgU8rQ","title":"xxx","type":null},"content":[{"type":"text","text":"社區內容流量調控系統設計——成本與指標平衡的背後"}]}]}]}