腾讯看点视频推荐索引构建方案

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"一、背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在视频推荐场景中，一方面我们需要让新启用的视频尽可能快的触达用户，这一点对于新闻类的内容尤为关键；另一方面我们需要快速识别新物品的好坏，通过分发的流量，以及对应的后验数据，来判断新物品是否值得继续分发流量。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而这两点对于索引先验数据和后验数据的延迟都有很高的要求。下文将为大家介绍看点视频推荐的索引构建方案，希望和大家一同交流。文章作者：纪文忠，腾讯QQ端推荐研发工程师。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"注：这里我们把视频创建时就带有的数据称为先验数据，如tag，作者账号id等，而把用户行为反馈的数据称为后验数据，如曝光、点击、播放等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"二、看点视频推荐整体架构"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/03\/3d\/03d12a8dc14a3ed809b4410685a8b93d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从数据链路来看此架构图，从下往上来看，首先视频内容由内容中心通过消息队列给到我们，经过一定的处理入库、建索引、生成正排\/倒排数据，这时候在存储层可召回的内容约有1千万条。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"然后经过召回层，通过用户画像、点击历史等特征召回出数千条视频，给到粗排层；粗排将这数千条视频打分，取数百条给到精排层；精排再一次打分，给到重排；重排根据一定规则和策略进行打散和干预，最终取10+条给到用户；"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}