伴魚機器學習預測服務：設計篇

{"type":"doc","content":[{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"前言"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在伴魚，我們在多個在線場景使用機器學習提升用戶的使用體驗。例如，在伴魚繪本中，我們根據用戶的帖子瀏覽記錄，爲用戶推薦他們感興趣的帖子。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在線預測是機器學習模型發揮作用的臨門一腳，重要性不言而喻。在伴魚，我們搭建了機器學習預測服務（以下簡稱預測服務），統一地處理所有的預測請求。本文主要介紹預測服務的演進過程。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"預測服務 V1"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"目前，各個算法團隊都有一套組裝預測服務的方式。它們的架構十分相似，可以用下圖表達："}]},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"業務服務從特徵系統獲取特徵，從 AB 平臺獲取實驗分組。　"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"等待獲取結果。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"業務服務將模型 ID 和特徵向量發送給 ModelServer。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"ModelServer 根據模型 ID 和特徵向量完成推理，將推理結果返回。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/f1\/9a\/f1e805333dbe11bc7373aa5bd433919a.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"其中，ModelServer 的實現有兩種主流方式。其一，使用 TorchServe 或 TensorFlow Serving 這樣和訓練框架高度耦合的 serving 方案。其二，使用 Flask 搭建一個簡單的 HTTP 服務，將模型加載至服務的內存，在收到預測請求時調用模型的預測接口進行預測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這種方式存在幾個問題："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"性能與多框架支持難以兼得。使用 TorchServe 或 TensorFlow Serving 能保證性能，但不能提供多框架支持；而使用 Flask 搭建預測服務，儘管可以支持任意框架訓練出來的模型，但服務性能偏差。其結果是，對於不同類型的模型（LightGBM vs PyTorch），架構非常不同（Flask vs TorchServe）。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上線模型需要工程同學的配合。每個需要 ML 能力的業務服務，都需要在算法和工程同學的緊密合作下，學習、實現和維護一套與多個 ML 系統（例如特徵系統和 AB 平臺）對接的邏輯。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不規範。不基於 Go 預測服務難以接入公司自建的服務治理體系和可觀測性體系。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲了系統性地解決這幾個問題，預測服務 V2 提出了幾個設計目的："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"高性能。預測服務用於線上場景，要滿足低延遲和高吞吐的需求。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"多框架支持。我們的模型包括樹和神經網絡，至少涉及 LightGBM \/ XGBoost 和 PyTorch \/ TensorFlow 這兩類框架。預測服務需要支持多種框架的模型。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"配置化。算法工程師通過配置文件聲明預測的工作流，無需業務的工程同學額外配合。要想對接更多 ML 子系統，只需由 AI 平臺實現一次，所有算法團隊都能受益，無需不同業務線的工程同學反覆實現對接邏輯。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"合規範。可以接入公司成熟的服務治理和可觀測性體系。"}]}]}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"預測服務 V2"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"架構主要借鑑了 "},{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/05\/31\/uber-michelangelo-overview","title":null,"type":null},"content":[{"type":"text","text":"Uber"}]},{"type":"text","text":" 和 "},{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/05\/31\/doordash-prediction-service","title":null,"type":null},"content":[{"type":"text","text":"DoorDash"}]},{"type":"text","text":"：預測服務接受預測請求，根據請求的內容，進行獲取特徵、獲取 AB 實驗分組等操作，然後調用 ModelServer 進行推理，返回預測結果。詳情見下圖。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/8f\/fd\/8f61b3406875c59b8227880b3277dcfd.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"⓪ 表示在新的模型上線之前，算法工程師上傳的預測配置文件會被載入預測服務，預測服務根據配置文件的內容實例化一個工作流。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"① 到 ⑥ 代表預測請求的整個生命週期："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"業務服務（例如推薦引擎）調用預測服務的接口，需要提供模型名字、預測主體的 key（例如用戶 ID）、上下文（context）特徵。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"預測服務根據模型名字，定位到該模型所對應的工作流並執行，包括調用特徵系統、調用 AB 平臺接口等。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"預測服務調用特徵系統接口獲取特徵向量，從 AB 平臺獲取實體對應的模型 ID。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"預測服務根據模型 ID 和特徵向量，調用 ModelServer。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"預測服務從 ModelServer 獲取模型的預測值。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","text":"返回預測值，並打日誌。日誌可用於構建訓練數據集，也可以用於監控特徵和預測的質量。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在這個架構下，算法工程師要上線一個模型，只需："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"離線完成模型訓練，將模型上傳至模型倉庫。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"將模型部署至 ModelServer（選用 "},{"type":"link","attrs":{"href":"https:\/\/github.com\/SeldonIO\/seldon-core","title":null,"type":null},"content":[{"type":"text","text":"Seldon Core"}]},{"type":"text","text":" 作爲解決方案）。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"用"},{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/07\/30\/palfish-feature-system","title":null,"type":null},"content":[{"type":"text","text":"特徵系統"}]},{"type":"text","text":"開發特徵。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"用 AB 平臺創建實驗。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":5,"align":null,"origin":null},"content":[{"type":"text","text":"將預測配置文件以 Merge Request 的形式，提交到指定的代碼倉庫。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":6,"align":null,"origin":null},"content":[{"type":"text","text":"由業務方的工程同學協助，對接預測服務。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"不難看出，編寫預測配置文件是算法工程師工作流的核心。預測配置文件以 YAML 格式定義了一個完整的模型推理工作流所涉及的全部信息。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"舉一個具體的例子。假設我們有一個視頻推薦系統 Toy Recsys，它結合用戶的網絡情況（network），和用戶的短期觀看歷史（last_5_views），給用戶推薦視頻。模型的基本邏輯是：網絡情況好，就結合用戶口味，放一些用戶最可能感興趣的長視頻；網絡情況差，就結合用戶口味，放一些短視頻。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們稱 network 和 last_5_views 是 Toy Recsys 模型的兩個特徵。其中，網絡情況會附帶在請求中（這類特徵被稱爲 context 特徵），而短期觀看歷史存儲在特徵系統中。要讓預測服務知道如何獲取這兩個特徵，不需要工程同學進行額外的編碼工作，而只需由算法工程師提交如下配置文件："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"model-name: toy_recsys\nfeature-system:\n- features:\n - name: network\n source: context\n default-value: 4G\n - name: last_5_views\n source: store\n default-value: []"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"預測服務會根據算法工程師提交的預測配置文件，實例化一個獲取特徵的工作流，並開始處理該模型的請求。這個工作流會："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":null,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"從請求中獲取 user_id 和 network 參數。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"使用請求中的 user_id，去調用特徵系統的 RPC 接口，獲取 last_5_views 的特徵值。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"將 user_id, network 和 last_5_views 組裝成一個特徵向量，發送給 ModelServer 進行推理。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"值得注意的是，預測配置文件極易拓展。如果預測服務要接入 AB 平臺，我們只需要支持在配置文件中填寫 AB 實驗的信息即可。例如："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"codeblock","attrs":{"lang":"text"},"content":[{"type":"text","text":"model-name: toy_recsys\nfeature-system: ...\nab-experiment:\n- experiment-key: TOY_RECSYS"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"總結"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在完成預測服務的初步設計後，我們開始了預測服務的實現。我們期待在預測服務上線後，與大家分享預測服務的實現細節。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"參考文獻"}]},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/05\/31\/uber-michelangelo-overview","title":null,"type":null},"content":[{"type":"text","text":"Uber 機器學習平臺概述"}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/05\/31\/doordash-prediction-service","title":null,"type":null},"content":[{"type":"text","text":"DoorDash 預測服務"}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/tech.ipalfish.com\/blog\/2021\/07\/30\/palfish-feature-system","title":null,"type":null},"content":[{"type":"text","text":"機器學習特徵系統在伴魚的演進"}]}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/github.com\/SeldonIO\/seldon-core","title":null,"type":null},"content":[{"type":"text","text":"Seldon Core"}]}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作者：陳易生"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：https:\/\/tech.ipalfish.com\/blog\/2021\/08\/24\/palfish-prediction-service-design\/"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"原文：伴魚機器學習預測服務：設計篇"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"來源：伴魚技術博客"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"轉載：著作權歸作者所有。商業轉載請聯繫作者獲得授權，非商業轉載請註明出處。"}]}]}