騰訊看點視頻推薦索引構建方案

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在視頻推薦場景中，一方面我們需要讓新啓用的視頻儘可能快的觸達用戶，這一點對於新聞類的內容尤爲關鍵；另一方面我們需要快速識別新物品的好壞，通過分發的流量，以及對應的後驗數據，來判斷新物品是否值得繼續分發流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而這兩點對於索引先驗數據和後驗數據的延遲都有很高的要求。下文將爲大家介紹看點視頻推薦的索引構建方案，希望和大家一同交流。文章作者：紀文忠，騰訊QQ端推薦研發工程師。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：這裏我們把視頻創建時就帶有的數據稱爲先驗數據，如tag，作者賬號id等，而把用戶行爲反饋的數據稱爲後驗數據，如曝光、點擊、播放等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、看點視頻推薦整體架構"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/03\/3d\/03d12a8dc14a3ed809b4410685a8b93d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從數據鏈路來看此架構圖，從下往上來看，首先視頻內容由內容中心通過消息隊列給到我們，經過一定的處理入庫、建索引、生成正排\/倒排數據，這時候在存儲層可召回的內容約有1千萬條。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然後經過召回層，通過用戶畫像、點擊歷史等特徵召回出數千條視頻，給到粗排層；粗排將這數千條視頻打分，取數百條給到精排層；精排再一次打分，給到重排；重排根據一定規則和策略進行打散和干預，最終取10+條給到用戶；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}