萊斯大學和英特爾的新研究：訓練深度神經網絡，CPU 可以比 GPU 更快

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"萊斯大學（Rice University）的計算機科學家展示了一種在普通處理器上運行的人工智能軟件，它訓練深度神經網絡的速度是基於圖形處理器的平臺的 15 倍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"萊斯大學布朗工程學院計算機科學助理教授 Anshumali Shrivastava 表示：“訓練成本是人工智能的真正瓶頸，企業每星期都要花上數百萬美元，僅僅是爲了訓練和微調他們的人工智能工作負載。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Shrivastava 和來自萊斯大學與英特爾的合作者在 4 月 8 日的機器學習系統會議 "},{"type":"link","attrs":{"href":"https:\/\/mlsys.org\/","title":null,"type":null},"content":[{"type":"text","text":"MLSys"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" 上展示瞭解決這一瓶頸的研究成果。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"深度神經網絡是人工智能的一種強大形式，在某些任務上超越了人類。對於深度神經網絡的訓練通常是一系列矩陣乘法運算，而矩陣乘法運算是圖形處理單元（GPU）的理想工作負載，其成本約爲通用中央處理單元（CPU）的三倍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Shrivastava 說：“整個行業都集中在一項改進上：更快的矩陣乘法。所有人都在尋找專門的硬件和架構來推進矩陣乘法。如今，甚至有人說要爲特定種類的深度學習提供專用的軟硬件組合。與其把整個系統優化的世界都拋到昂貴的算法面前，我還不如這麼說：‘讓我們重新審視一下算法。’”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"Shrivastava 的實驗室在 2019 年完成了這項工作，他們將深度神經網絡的訓練重鑄爲一個搜索問題，並使用哈希表解決。他們的“次線性深度學習引擎”（sub-linear deep learning engine，SLIDE）是專門爲使用普通 CPU 而設計的，由 Shrivastava 和來自英特爾的合作者"},{"type":"link","attrs":{"href":"https:\/\/techxplore.com\/news\/2020-03-deep-rethink-major-obstacle-ai.html","title":null,"type":null},"content":[{"type":"text","text":"在 MLSys 2020 上發佈"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"，證明了它的性能能夠超越基於 GPU 的訓練。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/proceedings.mlsys.org\/paper\/2021\/file\/3636638817772e42b59d74cff571fbb3-Paper.pdf","title":null,"type":null},"content":[{"type":"text","text":"不久前，他們在 MLSys 2021 上發表了一項研究"}],"marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}]},{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"，探索了在現代 CPU 中使用向量化和內存優化加速器是否可以提高 SLIDE 的性能。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"該研究報告的共同作者，萊斯大學的研究生 Shabnam Daghaghi 說：“基於哈希表的加速性能已經超越了 GPU，但 CPU 也在不斷髮展，”。他說，“我們利用這些創新讓 SLIDE 更進一步，表明如果你不堅持矩陣乘法，你可以利用現代 CPU 的能力，訓練人工智能模型的速度比最好的專業硬件替代方案快 4 到 15 倍。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"研究報告的作者之一、萊斯大學本科生 Nicholas Meisburger 稱：“CPU 仍然是計算領域最普遍的硬件。在人工智能工作負載中，讓它們更有吸引力的好處是不可低估的。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":" "}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}},{"type":"strong"}],"text":"原文鏈接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"color","attrs":{"color":"#494949","name":"user"}}],"text":"https:\/\/techxplore.com\/news\/2021-04-rice-intel-optimize-ai-commodity.html"}]}]}