推薦系統提供web服務的2種方式(二十四)

原創

2021-07-04 12:23

推薦web服務類型

優點

缺點

事先計算型

1. 接口響應更快

2. 整個系統有更好的魯棒性，推薦計算出問題不影響接口返回結果

3. 架構更加簡單，耦合度低，可以對接口和推薦計算分別優化升級

1. 浪費計算存儲資源

2. 對推薦結果調整的靈活度低

實時裝配型

1. 更省存儲計算資源

2. 系統更靈活，可以方便臨時調整推薦邏輯

1. 接口有更多的處理邏輯，響應相對較慢

2. 當推薦模型或模型服務出現問題時，無法給用戶提供推薦，影響用戶體驗

3. 架構相對複雜，耦合度高，推薦接口和推薦結果計算存在直接依賴關係

"}}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"表1：事先計算型和實時裝配型的優缺點對比","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"21.6 影響web服務方案的因素及選擇原則","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在上一節中，我們對兩種推薦web服務方案的優缺點進行了對比介紹，每種方案都有各自的優缺點，沒有哪一個方案是完全勝於另一個方案的。那麼在實際業務落地時，有哪些因素是會影響我們選擇具體的方案呢？我們應該怎樣選擇？有什麼判斷依據和準則嗎？在這一節中我們試圖從多個角度來回答這些問題。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"21.6.1 推薦產品形態的時效性對推薦web服務選擇的影響","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"如果推薦產品形態是T+1型推薦，由於每天只更新一次推薦結果，可以選擇事先計算型先將推薦結果計算出來。如果產品形態是實時信息流推薦，需要整合用戶的實時興趣變化，用戶的每一次行爲都會觸發更新推薦結果，這時採用臨時裝配型是更好的選擇。當然這也不是絕對的，作者公司的短視頻信息流推薦，就採用的事先計算型，事先計算型也可以做到近實時更新用戶推薦結果，前面已經提到，我們會在第28章《基於標籤的實時短視頻推薦系統》對算法原理進行詳細介紹。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"21.6.2 推薦團隊架構能力、工程實現能力對推薦web服務選擇的影響","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"實時裝配型架構相對複雜，耦合度相對更高，在推薦時需要處理的邏輯更多，因此各個子模塊都需要相當穩定，並且需要具備較高的性能，因此對整個推薦軟件系統的要求更高。因此，如果推薦團隊架構能力強，人力比較充足的情況下可以選擇實時裝配型方案。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了更好地整合用戶的實時行爲，爲用戶提供可見即所得的推薦服務，很多信息流推薦需要對推薦算法進行實時訓練，比如Google在2013年推廣的FTRL算法就是對logistic在實時推薦場景下的工程實現，具備更高的工程實現難度，因此，對推薦團隊的工程實現能力是有較高要求的。實時裝配型一般需要處理用戶的實時行爲日誌，用於挖掘用戶實時興趣，構建實時模型，這就要求整個系統有更高的實時性，需要一套完善的實時處理架構體系的支撐，這也增加了構建這類系統的複雜性。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前面也提到實時計算型一般需要有一套類似FAISS這樣的實時匹配庫，爲用戶在極短的時間內搜尋到最喜歡的標的物。而搭建這樣一套系統，需要將推薦模型做成獨立的服務，並且保證推薦模型web服務具備穩定性、高併發、可拓展性等能力，這也對架構能力有極高要求。如果希望採用容器等新技術來更好地管理推薦模型服務，這也需要新的學習成本和運維成本。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"21.6.3 推薦階段對推薦web服務選擇的影響","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們知道企業級推薦系統生成推薦結果的過程一般分爲召回、排序兩個個階段(其實還包括業務調控，業務調控更多的是運營和策略性的調整，不屬於狹義的算法範疇，參考第4章《推薦算法概述》4.2節的介紹)，先使用召回推薦算法從海量標的物中篩選出一組(一般幾百上千個)用戶可能感興趣的標的物，然後在排序階段利用更加精細化的推薦算法對結果進行重排序。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於召回是從所有標的物中篩選用戶可能感興趣的，當標的物數量龐大時(比如今日頭條有千億級文本、淘寶有上億級商品)，即使召回算法簡單，計算量也是非常大的，一般可以採用事先計算型召回策略(爲了整合用戶最近的行爲，也可以基於用戶的興趣標籤或者用戶最近瀏覽的標的物進行近實時召回，這類召回策略也屬於事先計算型，比如根據用戶最近瀏覽的標的物召回相似的標的物，每個標的物相似標的物是事先計算好的)。而對於排序推薦算法，只需要從有限的(成百上千)的標的物中過濾出用戶最喜歡的幾十個，可以在較短時間內計算完，因此排序算法可以採用實時裝配型策略。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"當然，排序階段也是可以採用事先計算型的，這就相當於先召回，再排序將推薦結果計算好，只不過整個推薦過程將事先計算拆解爲召回和排序兩個階段來進行了。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其實，直接跟推薦接口銜接的是排序階段，召回階段是不直接參與web服務的，因此根據第二節的定義，嚴格意義上事先計算型、實時裝配型是不能用於描述召回階段的。不過有些產品的標的物數量不大(比如電影只有幾萬個)，也可以將召回排序融合爲一個階段，只用一個算法就可以獲得推薦結果，或者排序可以採用簡單的規則和策略，這時排序邏輯可以整合到推薦web接口中，這兩種情況召回階段所起的作用就相當於排序階段的作用了，這時可以說召回直接跟web接口進行了交互，因此也可以用事先計算型、實時裝配型來描述召回階段。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"21.6.4 算法形態對推薦web服務選擇的影響","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"推薦算法種類繁多，從簡單的KNN、item-based協同過濾到複雜的深度學習、強化學習推薦算法，不同的算法實現方式、需要的數據來源、計算複雜度等都不一樣。這也導致了算法的使用場景不一樣。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"像深層深度學習這種模型結構非常複雜的推薦算法，即使爲單個標的物打分(即計算出用戶對標的物的偏好度)，計算時間也是簡單算法的若干倍，這時在短時間內(比如100毫秒之內)爲大量的標的物打分是不現實的，因此這類算法一般用於排序階段(排序階段只對成百上千的標的物打分)，因此比較適合實時裝配性的策略。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"簡單的推薦算法，如item-based協同過濾、矩陣分解，由於計算複雜度低，一般用於召回階段，因此是比較適合事先計算型的。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"總結","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"本章講解了推薦系統提供web服務的兩種主要方式。一種是事先計算型，提前將用戶的推薦結果計算出來並存放到NoSQL中，當用戶使用推薦模塊時，推薦web服務直接將該用戶的推薦結果取出來並組裝成合適的數據格式最終在前端展示給用戶。另一種是實時裝配型，我們需要將計算推薦結果需要的原材料準備成“半成品”(就是各種特徵)，將這些中間結果事先存起來，當用戶使用推薦服務時，推薦web服務通過簡單的組裝與計算(調用封裝好的推薦模型)，將“半成品”加工成該用戶的推薦結果，並最終給到用戶。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這兩種提供web服務的推薦方案各有優缺點，我們需要根據公司現在的技術儲備、人員能力、團隊規模、產品形態等多個維度進行評估和選擇。不管採用哪種方式，最終的目的是一樣的，我們需要爲用戶提供個性化的、響應及時的優質推薦服務。","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong","attrs":{}}],"text":"參考資料","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"1. [基於TensorFlow Serving的深度學習在線預估] ","attrs":{}},{"type":"link","attrs":{"href":"https://zhuanlan.zhihu.com/p/46591057","title":null,"type":null},"content":[{"type":"text","marks":[{"type":"underline","attrs":{}}],"text":"https://zhuanlan.zhihu.com/p/46591057","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2. [手把手教你使用TF服務將TensorFlow模型部署到生產環境] ","attrs":{}},{"type":"link","attrs":{"href":"https://zhuanlan.zhihu.com/p/60542828","title":null,"type":null},"content":[{"type":"text","marks":[{"type":"underline","attrs":{}}],"text":"https://zhuanlan.zhihu.com/p/60542828","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"3. ","attrs":{}},{"type":"link","attrs":{"href":"https://www.tensorflow.org/serving","title":null,"type":null},"content":[{"type":"text","marks":[{"type":"underline","attrs":{}}],"text":"https://www.tensorflow.org/serving","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"4. ","attrs":{}},{"type":"link","attrs":{"href":"https://github.com/facebookresearch/faiss","title":null,"type":null},"content":[{"type":"text","marks":[{"type":"underline","attrs":{}}],"text":"https://github.com/facebookresearch/faiss","attrs":{}}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" ","attrs":{}}]}]}