10+位机器学习大神测评 Amazon SageMaker 全流程实战

{"type":"doc","content":[{"type":"blockquote","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"持续三周，超过 500 个专场，150 余项新服务，全球超 60 万开发者注册！这场云计算行业的盛会，就是亚马逊 re:Invent 2020 ！\t"}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"12 月 9 日（PST 时间），AWS 全球机器学习副总裁 Swami Sivasubramanian 发表了本次大会机器学习的 Keynote，被誉为“大杀器”的 Amazon SageMaker 今年依旧是机器学习产品线的重头戏，相关热门功能包括 Amazon SageMaker Data Wrangler， Amazon SageMaker Clarify， Amazon SageMaker AutoPilot， Amazon SageMaker Debugger， Amazon SageMaker Eage Manager 共 5 项。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"为深入了解 Amazon SageMaker 在开发者群体中的使用情况，近期，AWS 联合 InfoQ 做了一次 Amazon SageMaker 的产品测评。整个测评历时近 2 个月，从数十份开发者使用报告中精选出 15 份，以 5 份为一组进行整理总结，分别代表初、中、高三个等级的开发者群体使用体验。现将整理结果分享出来，以飨读者。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"读完本文你将收获："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 包含哪些功能？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 能解决什么问题？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这个产品的使用体验如何？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用过程中有哪些难点 \/ 注意事项？"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 特性总结"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 未来发展路径如何？"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"机器学习领域的大杀器 -Amazon SageMaker\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"关注机器学习的读者都知道，机器学习模型构建包含数据准备，模型构建、训练、部署，最终才能将模型应用于生产中。通常一个数据科学家在进行上述工作时需要多种工具配合，理解工具细节并打通各类工具。并且在数据预处理环节常需要花费大量时间和精力，完成重复的体力工作，不利于创新。Amazon SageMaker 的出现极大改善了这一系列问题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 首度亮相于 2017 年的 Amazon re:Invent 大会，一经发布就被冠以 “大杀器” 的称号。翻看 AWS 的官方文档，SageMaker 的定义如下："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 是一项完全托管的服务，可以帮助数据科学家和开发人员快速轻松地构建、训练和部署任何规模的机器学习模型。Amazon SageMaker 包含一些可同时或单独构建、训练和部署机器学习模型的模块。"}]},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/05\/44\/050950a00f43209df36807e97dfa4244.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"（图片来自："},{"type":"text","text":"https:\/\/aws.amazon.com\/cn\/blogs\/aws\/sagemaker\/）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"按照官方文档的说法，Amazon SageMaker 的功能包含模型构建、训练和部署三大部分（文档来自：https:\/\/aws.amazon.com\/cn\/about-aws\/whats-new\/2017\/11\/introducing-amazon-sagemaker\/）："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"构建："},{"type":"text","text":"Amazon SageMaker 提供了快速连接到您的训练数据所需的所有内容，从而可以轻松构建 ML 模型并为训练做好准备，并且还可以轻松为您的应用程序选择和优化最佳算法和框架。Amazon SageMaker 包含托管的 Jupyter 笔记本，您可以轻松浏览和可视化在 Amazon S3 中存储的训练数据。您可以选择直接连接到 S3 中的数据，或者使用 AWS Glue 将数据从 Amazon RDS、Amazon DynamoDB 和 Amazon Redshift 移动到 S3 ，然后在笔记本中进行分析。为了帮助您选择算法，Amazon SageMaker 包含 10 种最常用的机器学习算法，这些算法已预装好并进行过优化，与在任何其他地方运行这些算法相比，最多可以将性能提高 10 倍。Amazon SageMaker 默认配置了 TensorFlow 和 Apache MXNet，这是两种最常见的开源框架。您也可以选择使用自己的框架。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"训练："},{"type":"text","text":"只需单击一下，您就可以在 Amazon SageMaker 控制台中开始训练模型。Amazon SageMaker 可以管理所有底层基础设施，并且可以轻松以 PB 级扩展以训练模型。为了使训练过程更快更轻松，Amazon SageMaker 可以自动调整您的模型以达到最高的精度。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"部署："},{"type":"text","text":"在训练并调整模型后，Amazon SageMaker 可以轻松在生产环境中部署该模型，以便开始针对新数据运行和生成预测（该过程称为推理）。Amazon SageMaker 会在跨多个可用区的 Amazon EC2 实例自动扩展集群上部署模型以实现高性能和高可用性。Amazon SageMaker 还包含内置的 A\/B 测试功能，以帮助您测试模型并试验不同的版本以获得最佳效果。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"小编总结："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"上述组成部分皆可独立使用，这意味着 Amazon SageMaker 将能够轻松填补现有流程中的空白环节。换句话来说，开发人员以端到端方式使用该服务时，将能够享受到由其提供的强大功能。同时，SageMaker 非常明智的把注意力放到训练模型和发布模型上， 让数据科学家去做针对业务模型的编程工作，而数据预处理甚至是超参调优则交给机器处理，大幅提升了开发效率。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"社区开发者的声音 -Amazon SageMaker 测评反馈\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"历时近 2 月的测评里，InfoQ 对这 15 位机器学习开发者的测评结果进行了整理。十余年 IT 老兵，人工智能技术经理阿伟从数据导入，建模能力，速度，易用性，Pipeline 完整性，框架支持丰富度，生态丰富度以及可视化能力 8 个纬度测评 Amazon SageMaker，并将其与国内友商的人工智能学习平台服务进行对比，之后得出结论："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"Amazon SageMaker 在数据导入，建模能力，Pipeline 完整性，框架支持丰富度，生态丰富度以及可视化能力 6 个方面具有明显优势。而在速度方面，国内厂商普遍表现良好，易用性方面国内部分厂商也同样优秀。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而对于流行的机器学习框架支持度方面，一位拥有 5 年 IT 经验的云计算公司基础架构组组长测试 Amazon SageMaker 后得出结论："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/1c\/c3\/1c8997ef9af957acedcaa55091c5f3c3.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"与国内友商相比，对于 TensorFlow 和 PyTorch，国内外厂商的支持度都比较好，而其余框架如 Apache MXNet，Scikit-learn，Spark 等，国内外厂商的支持度则各有差异。"},{"type":"text","text":"同时，各家厂商对于自家主导的开源框架拥有十分优秀的支持度，如 AWS 的 Apache MXNet。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"阿伟还提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 的优势在于不必安装，也不需要手动扩展，只要保证网络畅通，有兼容的浏览器保证运行即可。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"它提供了一个完整的机器学习套件，其中包括 IDE，API，调试、监控工具等，可以在机器学习建模的各个流程环节处理好关键事项。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ec\/e2\/ecdb44022428aa8b53464543deae3fe2.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"小编总结："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"总的来说，阿伟从易用性和产品完整性对 Amazon SageMaker 做了较高的评价。除阿伟外，其他社区机器学习开发者也认为 Amazon SageMaker 体验很好："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在一家医疗健康科技集团工作的高级算法工程师 DreamQ 提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker 内置了众多的算法，每一个算法都会在相应的环境中有一个教学模板，降低了上手门槛。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker Autopilot 是一个功能集, 可自动完成机器学习 (AutoML) 流程的关键任务。极大提升了模型性能和部署效率。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"支撑多种编程语言，并提供了多种编程语言下的相关的统计机器学习的案例。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Amazon SageMaker Ground Truth 可将数据标记成本降低多达 70％，极大地降低人工工作量。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"而就职于招商证券，专注 NLP 技术在金融领域应用的阿然认为："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SageMaker 完全基于 web，只需要一个浏览器就能跨系统、跨平台地完成机器学习工作，无论使用 windows、mac，还是手机，甚至树莓派都能无差别地完成需要的工作。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"拥有多年数据可视化经验的 web 架构师小灰以 Xgboost 作为算法，MNIST 作为数据集进行测评分析，体验后评价道："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这次技术体验比我想象的好太多了，技术体验耗时不到 10 分钟，解决了最头疼的配置和调优问题，简化了开发流程，提升了开发效率，这个是很大的亮点，同时从功能配置方面来说也是非常强大的，在应对大规模训练需求过程中，平台可通过跟踪监控，训练调优，自动计算，弹性推理等功能，从成本和技术上展示优势。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在数据科学领域拥有 5 年以上经验的 PWC 数据科学家浩哥提到："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SageMaker 所提供的优秀的可拓展性是所有企业所非常需要的特性，它完美解决了由企业的业务增长带来的对更多更复杂数据开发的需求问题。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可见除了易用性、产品完整性之外，Amazon SageMaker 在支持多种编程语言，自动化打标签，功能配置灵活性和可拓展性方面也有良好的表现。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"介于解决方案与单点工具之间 - 一个端到端的机器学习服务\t"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如今年亚马逊 re:Invent 大会提到的那样，Amazon SageMaker 已经成为一个真正意义上的端到端机器学习服务。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"前不久 InfoQ 邀请了 AWS 数据分析架构师经理王晓野老师，为亚马逊 re:Invent 中 Werner Vogels 的 Keynote 做解析。晓野老师提到，"},{"type":"text","marks":[{"type":"strong"}],"text":"所谓“端到端”，是指从数据采集、数据清洗、数据准备，到模型搭建、模型训练和推理，整个 AI 应用研发过程中的所有环节和领域，Amazon SageMaker 都提供了解决方案。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/3e\/1d\/3e0d3430bc8ceb32c64276ed75372f1d.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"例如，Amazon SageMaker Data Wrangler 可以帮助用户准备数据，使用 Amazon SageMaker Studio 构建可视化模型，利用 Amazon SageMaker Debugger 寻找模型瓶颈、Amazon SageMaker Neo 优化模型、Amazon SageMaker Edge Manager 在边缘设备上部署模型，等等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"同时晓野老师还提到，Amazon SageMaker 发布三年来已经赢得了大量客户，且使用量仍在呈指数级增长。经过多年迭代，Amazon SageMaker 如今已成长为强大的机器学习平台，即使是欠缺 AI 领域知识的开发者也可以利用 Amazon SageMaker 完成一些机器学习领域的工作。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"从测评结果来看，这一端到端服务已经可以落地于实际生产应用中，虽然上手使用需要一定计算机 \/ 机器学习基础知识，并且其官方文档的中文支持以及英文释义还有提升空间；但对于整个机器学习行业的开发者来说，Amazon SageMaker 无疑使机器学习技术普惠到更多开发者，并且对于中高端开发者也有很大帮助。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"可以预见的是，当原先由于机器学习高门槛望而却步的开发者，以及专注于打造解决方案的业务人员都能独立使用 Amazon SageMaker 解决生活生产问题时，游戏规则将被彻底改变。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"（本文根据 10+ 位机器学习工程师真实评测结果整理输出，评价引文不代表 InfoQ 立场以及其所在公司或者组织的官方立场。在此感谢参与评测的（排名按姓氏顺序）陈海栋、DreamQ、龚浩、郭锋 、胡斐然、刘洋、申屠鹏会、索小辉、沈毅、吴磊、王新义、营伟、杨智凯、赵磊、朱作政老师。）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"点击【"},{"type":"link","attrs":{"href":"https:\/\/www.amazonaws.cn\/ai_credit2_contact_us\/?trk=DevDay&trkCampaign=ai_credit_cn&sc_channel=el&sc_campaign=DevDay-ACTS&sc_outcome=Global_Marketing_Campaigns&sc_geo=CHNA&sc_country=CN","title":"xxx","type":null},"content":[{"type":"text","text":"阅读原文"}]},{"type":"text","text":"】免费试用 Amazon SageMaker"}]}]}