如何將PyTorch Lightning模型部署到生產中

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"italic"}],"text":"大規模服務PyTorch Lightning模型的完整指南。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/2a/2af6ef5180b94a7f67f50706f5e45a72.jpeg","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"縱觀機器學習領域,主要趨勢之一是專注於將軟件工程原理應用於機器學習的項目激增。 例如,"},{"type":"link","attrs":{"href":"https://github.com/cortexlabs/cortex","title":null},"content":[{"type":"text","text":"Cortex"}]},{"type":"text","text":"再現了部署無服務器功能但具有推理管道的體驗。類似地,DVC實現了現代版本控制和CI / CD管道,但僅用於ML。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PyTorch Lightning具有類似的理念,僅適用於訓練。框架爲PyTorch提供了Python包裝器,可讓數據科學家和工程師編寫乾淨,可管理且性能卓越的訓練代碼。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"作爲構建 整個部署平臺的人 ,部分原因是我們討厭編寫樣板,因此我們是PyTorch Lightning的忠實擁護者。本着這種精神,我整理了將PyTorch Lightning模型部署到生產環境的指南。在此過程中,我們將研究幾種導出PyTorch Lightning模型以包括在推理管道中的選項。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"部署PyTorch Lightning模型進行推理的每種方法"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有三種方法可以導出PyTorch Lightning模型進行投放:"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將模型另存爲PyTorch檢查點"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將模型轉換爲ONNX"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"將模型導出到Torchscript"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們可以通過Cortex爲這三個服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1.直接打包和部署PyTorch Lightning模塊"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從最簡單的方法開始,讓我們部署一個沒有任何轉換步驟的PyTorch Lightning模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PyTorch Lightning Trainer是一個抽象樣板訓練代碼(思考訓練和驗證步驟)的類,它具有內置的save_checkpoint()函數,該函數會將您的模型另存爲.ckpt文件。要將模型保存爲檢查點,只需將以下代碼添加到訓練腳本中:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/02/025d681f83213cf3d7ce1654847b7be8.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在,在我們開始爲該檢查點提供服務之前,需要注意的是,雖然我一直說“ PyTorch Lightning模型”,但PyTorch Lightning是PyTorch的包裝器-項目的README字面意思是“ PyTorch Lightning只是有組織的PyTorch。” 因此,導出的模型是普通的PyTorch模型,可以相應地使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"有了保存的檢查點,我們可以在Cortex中輕鬆地爲模型提供服務。如果您不熟悉Cortex,可以 "},{"type":"link","attrs":{"href":"https://docs.cortex.dev/","title":null},"content":[{"type":"text","text":"在這裏快速熟悉一下"}]},{"type":"text","text":",但是Cortex部署過程的簡單概述是:"}]},{"type":"bulletedlist","content":[{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們使用Python爲我們的模型編寫了一個預測API"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們在YAML中定義我們的API基礎結構和行爲"}]}]},{"type":"listitem","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們使用CLI中的命令部署API"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們的預測API將使用Cortex的Python Predictor類定義一個init()函數來初始化我們的API並加載模型,並使用一個define()函數在查詢時提供預測:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/d6/d6a2224d5d2b032568e54e234730313f.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"很簡單 我們從訓練代碼中重新調整了一些代碼的用途,添加了一些推理邏輯,僅此而已。需要注意的一件事是,如果將模型上傳到S3(推薦),則需要添加一些邏輯來訪問它。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來,我們在YAML中配置基礎架構:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/dd/dd55dff424462894bebc088e789b4803.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"再次,簡單。我們給我們的API起個名字,告訴Cortex我們的預測API在哪裏,並分配一些CPU。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"接下來,我們部署它:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ab/ab1263a55e354288c69ea8c91acae5ec.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"請注意,我們還可以部署到集羣,由Cortex加速和管理:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/b8/b8c81954a32fd5b219ca9ef79961bae0.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在所有部署中,Cortex都會容器化我們的API並將其公開爲Web服務。通過雲部署,Cortex可以配置負載平衡,自動擴展,監視,更新和許多其他基礎架構功能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"就是這樣!現在,我們有一個實時Web API,可根據要求提供模型預測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2.導出到ONNX並通過ONNX運行時進行投放"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現在,我們已經部署了一個普通的PyTorch檢查點,讓事情複雜一些。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"PyTorch Lightning最近添加了一個方便的抽象,用於將模型導出到ONNX(以前,您可以使用PyTorch的內置轉換功能,儘管它們需要更多樣板)。要將模型導出到ONNX,只需將以下代碼添加到您的訓練腳本中:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/15/15ab17629a85d7908b2da8580641e3b6.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"請注意,您的輸入樣本應模仿實際模型輸入的形狀。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"導出ONNX模型後,就可以使用Cortex的ONNX Predictor爲其提供服務。代碼基本上看起來是相同的,並且過程是相同的。例如,這是一個ONNX預測API:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/eb/ebe30166a5e0eeedd0170a9658c66515.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"基本上一樣。唯一的區別是,我們不是通過直接初始化模型,而是通過onnx_client訪問該數據,這是Cortex爲服務於我們的模型而啓動的ONNX運行時容器。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我們的YAML看起來也很相似:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/ed/ed9c580485af9b3744717a2c388147b1.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我在此處添加了一個監視標誌,目的只是爲了顯示配置的簡便程度,並且有一些ONNX特定字段,但其他方面都是相同的YAML。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最後,我們使用與之前相同的$ cortex deploy命令進行部署,並且我們的ONNX API已啓用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3.使用Torchscript的JIT編譯器進行序列化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於最終部署,我們將把PyTorch Lightning模型導出到Torchscript並使用PyTorch的JIT編譯器提供服務。要導出模型,只需將其添加到您的訓練腳本中:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/31/313b86adee9ed266007585a717b42bf5.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"用於此目的的Python API與原始PyTorch示例幾乎相同:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/3c/3c407844ca45ed4f88d95ef80acfef32.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"YAML保持與以前相同,並且CLI命令當然是一致的。如果需要的話,我們實際上可以更新我們以前的PyTorch API來使用新模型,只需將新的舊的dictor.py腳本替換爲新的腳本,然後再次運行$ cortex部署:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/12/12f3991706f8eddc7a82b8b48cd97da9.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":true}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Cortex在此處自動執行滾動更新,在該更新中,新的API會被啓動,然後與舊的API交換,從而避免了模型更新之間的任何停機時間。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這就是全部。現在,您已經具有用於實時推理的完全可操作的預測API,可根據Torchscript模型提供預測。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"那麼,您應該使用哪種方法?"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"顯而易見的問題是哪種方法效果最好。事實是,這裏沒有簡單的答案,因爲這取決於您的模型。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於BERT和GPT-2等Transformer模型,ONNX可以提供令人難以置信的優化(我們測得的"},{"type":"link","attrs":{"href":"https://www.cortex.dev/post/40x-nlp-inference-with-hugging-face-and-onnx","title":null},"content":[{"type":"text","text":"CPU吞吐量提高"}]},{"type":"text","text":"了"},{"type":"link","attrs":{"href":"https://www.cortex.dev/post/40x-nlp-inference-with-hugging-face-and-onnx","title":null},"content":[{"type":"text","text":"40"}]},{"type":"text","text":"倍 )。對於其他模型,Torchscript的性能可能比香草PyTorch更好-儘管這也帶有一些警告,因爲並非所有模型都乾淨地導出到Torchscript。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"幸運的是,使用任何選項進行部署都很容易,您可以並行測試所有這三個選項,並查看哪種方式最適合您的特定API。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":4},"content":[{"type":"text","text":"如果你喜歡本文的話,歡迎點贊轉發!謝謝。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"看完別走還有驚喜!"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"我精心整理了計算機/Python/機器學習/深度學習相關的2TB視頻課與書籍,價值1W元。關注微信公衆號“計算機與AI”,點擊下方菜單即可獲取網盤鏈接。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":""}]},{"type":"image","attrs":{"src":"https://static001.geekbang.org/infoq/90/903237ffd0a3b3ae06272386f26ecb9e.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章