10+位機器學習大神測評 Amazon SageMaker 全流程實戰

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"持續三週，超過 500 個專場，150 餘項新服務，全球超 60 萬開發者註冊！這場雲計算行業的盛會，就是亞馬遜 re:Invent 2020 ！\t"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"12 月 9 日（PST 時間），AWS 全球機器學習副總裁 Swami Sivasubramanian 發表了本次大會機器學習的 Keynote，被譽爲“大殺器”的 Amazon SageMaker 今年依舊是機器學習產品線的重頭戲，相關熱門功能包括 Amazon SageMaker Data Wrangler， Amazon SageMaker Clarify， Amazon SageMaker AutoPilot， Amazon SageMaker Debugger， Amazon SageMaker Eage Manager 共 5 項。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"爲深入瞭解 Amazon SageMaker 在開發者羣體中的使用情況，近期，AWS 聯合 InfoQ 做了一次 Amazon SageMaker 的產品測評。整個測評歷時近 2 個月，從數十份開發者使用報告中精選出 15 份，以 5 份爲一組進行整理總結，分別代表初、中、高三個等級的開發者羣體使用體驗。現將整理結果分享出來，以饗讀者。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"讀完本文你將收穫："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 包含哪些功能？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 能解決什麼問題？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這個產品的使用體驗如何？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用過程中有哪些難點 \/ 注意事項？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 特性總結"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 未來發展路徑如何？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"機器學習領域的大殺器 -Amazon SageMaker\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"關注機器學習的讀者都知道，機器學習模型構建包含數據準備，模型構建、訓練、部署，最終才能將模型應用於生產中。通常一個數據科學家在進行上述工作時需要多種工具配合，理解工具細節並打通各類工具。並且在數據預處理環節常需要花費大量時間和精力，完成重複的體力工作，不利於創新。Amazon SageMaker 的出現極大改善了這一系列問題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 首度亮相於 2017 年的 Amazon re:Invent 大會，一經發布就被冠以 “大殺器” 的稱號。翻看 AWS 的官方文檔，SageMaker 的定義如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 是一項完全託管的服務，可以幫助數據科學家和開發人員快速輕鬆地構建、訓練和部署任何規模的機器學習模型。Amazon SageMaker 包含一些可同時或單獨構建、訓練和部署機器學習模型的模塊。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/05\/44\/050950a00f43209df36807e97dfa4244.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"（圖片來自："},{"type":"text","text":"https:\/\/aws.amazon.com\/cn\/blogs\/aws\/sagemaker\/）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照官方文檔的說法，Amazon SageMaker 的功能包含模型構建、訓練和部署三大部分（文檔來自：https:\/\/aws.amazon.com\/cn\/about-aws\/whats-new\/2017\/11\/introducing-amazon-sagemaker\/）："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"構建："},{"type":"text","text":"Amazon SageMaker 提供了快速連接到您的訓練數據所需的所有內容，從而可以輕鬆構建 ML 模型併爲訓練做好準備，並且還可以輕鬆爲您的應用程序選擇和優化最佳算法和框架。Amazon SageMaker 包含託管的 Jupyter 筆記本，您可以輕鬆瀏覽和可視化在 Amazon S3 中存儲的訓練數據。您可以選擇直接連接到 S3 中的數據，或者使用 AWS Glue 將數據從 Amazon RDS、Amazon DynamoDB 和 Amazon Redshift 移動到 S3 ，然後在筆記本中進行分析。爲了幫助您選擇算法，Amazon SageMaker 包含 10 種最常用的機器學習算法，這些算法已預裝好並進行過優化，與在任何其他地方運行這些算法相比，最多可以將性能提高 10 倍。Amazon SageMaker 默認配置了 TensorFlow 和 Apache MXNet，這是兩種最常見的開源框架。您也可以選擇使用自己的框架。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"訓練："},{"type":"text","text":"只需單擊一下，您就可以在 Amazon SageMaker 控制檯中開始訓練模型。Amazon SageMaker 可以管理所有底層基礎設施，並且可以輕鬆以 PB 級擴展以訓練模型。爲了使訓練過程更快更輕鬆，Amazon SageMaker 可以自動調整您的模型以達到最高的精度。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"部署："},{"type":"text","text":"在訓練並調整模型後，Amazon SageMaker 可以輕鬆在生產環境中部署該模型，以便開始針對新數據運行和生成預測（該過程稱爲推理）。Amazon SageMaker 會在跨多個可用區的 Amazon EC2 實例自動擴展集羣上部署模型以實現高性能和高可用性。Amazon SageMaker 還包含內置的 A\/B 測試功能，以幫助您測試模型並試驗不同的版本以獲得最佳效果。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"小編總結："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"上述組成部分皆可獨立使用，這意味着 Amazon SageMaker 將能夠輕鬆填補現有流程中的空白環節。換句話來說，開發人員以端到端方式使用該服務時，將能夠享受到由其提供的強大功能。同時，SageMaker 非常明智的把注意力放到訓練模型和發佈模型上， 讓數據科學家去做針對業務模型的編程工作，而數據預處理甚至是超參調優則交給機器處理，大幅提升了開發效率。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"社區開發者的聲音 -Amazon SageMaker 測評反饋\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"歷時近 2 月的測評裏，InfoQ 對這 15 位機器學習開發者的測評結果進行了整理。十餘年 IT 老兵，人工智能技術經理阿偉從數據導入，建模能力，速度，易用性，Pipeline 完整性，框架支持豐富度，生態豐富度以及可視化能力 8 個緯度測評 Amazon SageMaker，並將其與國內友商的人工智能學習平臺服務進行對比，之後得出結論："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Amazon SageMaker 在數據導入，建模能力，Pipeline 完整性，框架支持豐富度，生態豐富度以及可視化能力 6 個方面具有明顯優勢。而在速度方面，國內廠商普遍表現良好，易用性方面國內部分廠商也同樣優秀。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而對於流行的機器學習框架支持度方面，一位擁有 5 年 IT 經驗的雲計算公司基礎架構組組長測試 Amazon SageMaker 後得出結論："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/1c\/c3\/1c8997ef9af957acedcaa55091c5f3c3.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"與國內友商相比，對於 TensorFlow 和 PyTorch，國內外廠商的支持度都比較好，而其餘框架如 Apache MXNet，Scikit-learn，Spark 等，國內外廠商的支持度則各有差異。"},{"type":"text","text":"同時，各家廠商對於自家主導的開源框架擁有十分優秀的支持度，如 AWS 的 Apache MXNet。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"阿偉還提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 的優勢在於不必安裝，也不需要手動擴展，只要保證網絡暢通，有兼容的瀏覽器保證運行即可。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它提供了一個完整的機器學習套件，其中包括 IDE，API，調試、監控工具等，可以在機器學習建模的各個流程環節處理好關鍵事項。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ec\/e2\/ecdb44022428aa8b53464543deae3fe2.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"小編總結："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"總的來說，阿偉從易用性和產品完整性對 Amazon SageMaker 做了較高的評價。除阿偉外，其他社區機器學習開發者也認爲 Amazon SageMaker 體驗很好："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在一家醫療健康科技集團工作的高級算法工程師 DreamQ 提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 內置了衆多的算法，每一個算法都會在相應的環境中有一個教學模板，降低了上手門檻。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker Autopilot 是一個功能集, 可自動完成機器學習 (AutoML) 流程的關鍵任務。極大提升了模型性能和部署效率。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支撐多種編程語言，並提供了多種編程語言下的相關的統計機器學習的案例。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker Ground Truth 可將數據標記成本降低多達 70％，極大地降低人工工作量。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而就職於招商證券，專注 NLP 技術在金融領域應用的阿然認爲："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SageMaker 完全基於 web，只需要一個瀏覽器就能跨系統、跨平臺地完成機器學習工作，無論使用 windows、mac，還是手機，甚至樹莓派都能無差別地完成需要的工作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"擁有多年數據可視化經驗的 web 架構師小灰以 Xgboost 作爲算法，MNIST 作爲數據集進行測評分析，體驗後評價道："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這次技術體驗比我想象的好太多了，技術體驗耗時不到 10 分鐘，解決了最頭疼的配置和調優問題，簡化了開發流程，提升了開發效率，這個是很大的亮點，同時從功能配置方面來說也是非常強大的，在應對大規模訓練需求過程中，平臺可通過跟蹤監控，訓練調優，自動計算，彈性推理等功能，從成本和技術上展示優勢。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在數據科學領域擁有 5 年以上經驗的 PWC 數據科學家浩哥提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SageMaker 所提供的優秀的可拓展性是所有企業所非常需要的特性，它完美解決了由企業的業務增長帶來的對更多更復雜數據開發的需求問題。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可見除了易用性、產品完整性之外，Amazon SageMaker 在支持多種編程語言，自動化打標籤，功能配置靈活性和可拓展性方面也有良好的表現。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"介於解決方案與單點工具之間 - 一個端到端的機器學習服務\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如今年亞馬遜 re:Invent 大會提到的那樣，Amazon SageMaker 已經成爲一個真正意義上的端到端機器學習服務。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前不久 InfoQ 邀請了 AWS 數據分析架構師經理王曉野老師，爲亞馬遜 re:Invent 中 Werner Vogels 的 Keynote 做解析。曉野老師提到，"},{"type":"text","marks":[{"type":"strong"}],"text":"所謂“端到端”，是指從數據採集、數據清洗、數據準備，到模型搭建、模型訓練和推理，整個 AI 應用研發過程中的所有環節和領域，Amazon SageMaker 都提供瞭解決方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/3e\/1d\/3e0d3430bc8ceb32c64276ed75372f1d.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如，Amazon SageMaker Data Wrangler 可以幫助用戶準備數據，使用 Amazon SageMaker Studio 構建可視化模型，利用 Amazon SageMaker Debugger 尋找模型瓶頸、Amazon SageMaker Neo 優化模型、Amazon SageMaker Edge Manager 在邊緣設備上部署模型，等等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同時曉野老師還提到，Amazon SageMaker 發佈三年來已經贏得了大量客戶，且使用量仍在呈指數級增長。經過多年迭代，Amazon SageMaker 如今已成長爲強大的機器學習平臺，即使是欠缺 AI 領域知識的開發者也可以利用 Amazon SageMaker 完成一些機器學習領域的工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"從測評結果來看，這一端到端服務已經可以落地於實際生產應用中，雖然上手使用需要一定計算機 \/ 機器學習基礎知識，並且其官方文檔的中文支持以及英文釋義還有提升空間；但對於整個機器學習行業的開發者來說，Amazon SageMaker 無疑使機器學習技術普惠到更多開發者，並且對於中高端開發者也有很大幫助。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以預見的是，當原先由於機器學習高門檻望而卻步的開發者，以及專注於打造解決方案的業務人員都能獨立使用 Amazon SageMaker 解決生活生產問題時，遊戲規則將被徹底改變。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（本文根據 10+ 位機器學習工程師真實評測結果整理輸出，評價引文不代表 InfoQ 立場以及其所在公司或者組織的官方立場。在此感謝參與評測的（排名按姓氏順序）陳海棟、DreamQ、龔浩、郭鋒 、胡斐然、劉洋、申屠鵬會、索小輝、沈毅、吳磊、王新義、營偉、楊智凱、趙磊、朱作政老師。）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"點擊【"},{"type":"link","attrs":{"href":"https:\/\/www.amazonaws.cn\/ai_credit2_contact_us\/?trk=DevDay&trkCampaign=ai_credit_cn&sc_channel=el&sc_campaign=DevDay-ACTS&sc_outcome=Global_Marketing_Campaigns&sc_geo=CHNA&sc_country=CN","title":"xxx","type":null},"content":[{"type":"text","text":"閱讀原文"}]},{"type":"text","text":"】免費試用 Amazon SageMaker"}]}]}