2021 年將是“人工智能硬件年”

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在本文中,我將討論專爲機器學習 \/ 人工智能應用開發的硬件,以及該領域的機遇。並簡要介紹英偉達是如何在機器學習硬件領域實現近乎壟斷的地位,以及爲什麼幾乎沒有人能成功挑戰它。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在過去的 10 年中,專用於機器學習應用的硬件研究迅猛發展,硬件與機器學習棧的每個部分都有關係。這種硬件可加速訓練和推理,減少延遲時間,降低功耗,並降低這些設備的零售成本。當前通用的機器學習硬件解決方案是英偉達 GPU,這使得英偉達在市場上佔據主導地位,並使其估值超越英特爾。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"隨着前景廣闊的研究不斷湧現,英偉達繼續通過出售 GPU 和它的專有 CUDA 工具箱來主導這個領域。不過,我認爲有四個因素將挑戰英偉達的統治地位,並且最快今年,也肯定會在 2~3 年內改變機器學習硬件的格局。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"numberedlist","attrs":{"start":1,"normalizeStart":1},"content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":1,"align":null,"origin":null},"content":[{"type":"text","text":"這個領域的學術研究正在成爲主流。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":2,"align":null,"origin":null},"content":[{"type":"text","text":"摩爾定律已死。隨着它的消亡,“技術和市場力量正在將計算推向相反的方向,使得計算機處理器不再是通用的,而是更加專業化的。”("},{"type":"link","attrs":{"href":"https:\/\/poseidon01.ssrn.com\/delivery.php?ID=211117027007028109012099007123091067026021000060079050028086075010069007025112025105058055039060103003114025068072124026100029114044064023023011030000001096118000084057073052125100086112090110071018005011108079091010104083101111125088093082073127085122&EXT=pdf&INDEX=TRUE","title":"","type":null},"content":[{"type":"text","text":"出處"}]},{"type":"text","text":")"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":3,"align":null,"origin":null},"content":[{"type":"text","text":"投資人和創始人都認識到,人工智能不僅能開闢新的領域,而且能增加他們的預算。"}]}]},{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":4,"align":null,"origin":null},"content":[{"type":"text","text":"人工智能產生的碳排放量過高,而且越來越高。我們需要讓計算更加節能。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"背景"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"下面是典型的機器學習管道的樣子:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/85\/d4\/854b80586b989708b78ed03bab9173d4.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於大多數數據科學工作流而言,在訓練和部署大型模型之前,通用芯片,如 CPU,就已經足夠了。GPU 在“深度學習”(涉及視覺和自然語言處理等任務的神經網絡體系結構)中幾乎總是必不可少的。爲深度學習提供 GPU 工作站的 Lambda Labs 公司估計,包括英偉達的頂級 GPU 集羣在內,"},{"type":"link","attrs":{"href":"https:\/\/lambdalabs.com\/blog\/demystifying-gpt-3\/","title":"","type":null},"content":[{"type":"text","text":"訓練 GPT-3 的費用大約爲 460 萬美元"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用 GPU 的主要優點是,與傳統 CPU 相比, GPU 可以並行地執行計算,數據吞吐量更高。計算過程中,機器學習的核心計算部分是矩陣乘法,並行運行時能大大提高運算速度。專有的英偉達"},{"type":"link","attrs":{"href":"https:\/\/developer.net\/cuda-toolkit","title":"","type":null},"content":[{"type":"text","text":"CUDA"}]},{"type":"text","text":"提供了 API 和工具,以便開發者可以利用這種並行化。像 TensorFlow 和 PyTorch 這樣的流行庫將其抽象出來,其中一行代碼會自動檢測 GPU,然後利用 CUDA 後端。若要設計一種新的算法或庫,需要利用並行計算的優勢,CUDA 提供的工具會使這一工作更加簡單。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"上世紀 90 年代初,英偉達作爲一家視頻遊戲公司起家,希望能提供能快速繪製 3D 圖像的圖像芯片。它在這一業務上取得了成功,在與另一家顯卡製造商 AMD 的不斷交鋒中,始終如一地製造出一些最強大的 GPU。巧合的是,同樣的圖形硬件竟然成了深度學習騰飛不可或缺的因素。CUDA 讓英偉達比其他 GPU 更有優勢。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/b5\/0e\/b53cf94e674d83a4820f1a7603b0850e.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2006 年,英偉達發佈了第一個 CUDA 工具包,它提供了一個 API,可以讓使用 GPU 變得非常簡單。3 年後,2009 年,斯坦福大學人工智能教授吳恩達及其合作者發表了一篇題爲《"},{"type":"link","attrs":{"href":"http:\/\/robotics.stanford.edu\/~ang\/papers\/icml09-LargeScaleUnsupervisedDeepLearningGPU.pdf","title":"","type":null},"content":[{"type":"text","text":"使用圖形處理器的大規模無監督式深度學習"}]},{"type":"text","text":"》("},{"type":"text","marks":[{"type":"italic"}],"text":"Large-scale Deep Unsupervised Learning using Graphics Processors"},{"type":"text","text":")的論文,指出如果 GPU 用於訓練,大規模的深度學習就有可能實現。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一年後,吳恩達和斯坦福大學的另一位教授,Google X 的共同創始人,Sebastian Thrun,向拉里·佩奇提出了在谷歌成立深度學習研究團隊的想法,該團隊後來成爲 Google Brain。伴隨着 Google Brain 的崛起和“"},{"type":"link","attrs":{"href":"https:\/\/qz.com\/1034972\/the-data-that-changed-the-direction-of-ai-research-and-possibly-the-world\/","title":"","type":null},"content":[{"type":"text","text":"Imagenet 時刻"}]},{"type":"text","text":"”的到來,英偉達的 GPU 已經成爲人工智能 \/ 機器學習行業事實上的計算標準。如需更多信息,請參閱這篇文章《"},{"type":"link","attrs":{"href":"https:\/\/www.forbes.com\/sites\/aarontilley\/2016\/11\/30\/nvidia-deep-learning-ai-intel\/?sh=6a1602d27ff1","title":"","type":null},"content":[{"type":"text","text":"新的英特爾:英偉達如何從驅動視頻遊戲到革新人工智能"}]},{"type":"text","text":"》("},{"type":"text","marks":[{"type":"italic"}],"text":"The New Intel: How Nvidia Went From Powering Video Games To Revolutionizing Artificial Intelligence"},{"type":"text","text":")。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"概述:現狀"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"英偉達憑藉其 GPU 在深度學習硬件領域佔據主導地位,這在很大程度上要歸功於 CUDA。據"},{"type":"link","attrs":{"href":"https:\/\/www.forbes.com\/sites\/paulteich\/2019\/06\/17\/nvidia-dominates-the-market-for-cloud-ai-accelerators-more-than-you-think\/?sh=30a782ac5edb","title":"","type":null},"content":[{"type":"text","text":"福布斯報道"}]},{"type":"text","text":",“2019 年 5 月,前四大雲計算供應商在 97.4% 的基礎設施即服務(IaaS)計算實例類型中部署了英偉達 GPU,並配備了專用加速器”。面對"},{"type":"link","attrs":{"href":"https:\/\/www.datacenterknowledge.com\/deals\/nvidia-7-billion-what-it-takes-dominate-ai-hardware","title":"","type":null},"content":[{"type":"text","text":"競爭"}]},{"type":"text","text":",它也"},{"type":"link","attrs":{"href":"https:\/\/nvidianews.nvidia.com\/news\/nvidia-to-acquire-arm-for-40-billion-creating-worlds-premier-computing-company-for-the-age-of-ai\/","title":"","type":null},"content":[{"type":"text","text":"沒有坐以待斃"}]},{"type":"text","text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"谷歌早在 2015 年就開發了專門爲神經網絡開發的人工智能加速器芯片 TPU。在其作爲特定領域加速器的狹義用例中,TPU 比 GPU 更快,也更便宜,但在谷歌的 GCP 生態系統中,TPU 被隔離起來,僅有 TensorFlow 和 PyTorch 支持(其他庫需要自己編寫 TPU 編譯器)。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"AWS 正在對自己的芯片下賭注,尤其是機器學習。到目前爲止,AWS Inferentia 芯片"},{"type":"link","attrs":{"href":"https:\/\/arstechnica.com\/gadgets\/2020\/11\/amazon-begins-shifting-alexas-cloud-ai-to-its-own-silicon\/","title":"","type":null},"content":[{"type":"text","text":"似乎是最成功的"}]},{"type":"text","text":"。這在很大程度上取決於開發者從 CUDA 切換到亞馬遜 Inferentia 和其他芯片的工具包的難易程度。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2019 年 12 月,英特爾以 20 億美元的價格收購了 Habana Labs,這是一家以色列公司,爲訓練和推理工作負載製造芯片和硬件加速器。英特爾的投資似乎得到了回報,上個月,"},{"type":"link","attrs":{"href":"https:\/\/habana.ai\/habana-gaudi-ai-processors-to-bring-lower-cost-to-train-to-amazon-ec2-customers\/","title":"","type":null},"content":[{"type":"text","text":"AWS 宣佈"}]},{"type":"text","text":"將提供運行 Habana 芯片的新 EC2 實例,“與當前基於 GPU 的 EC2 實例相比,爲機器學習工作負載提供高達 40% 的價格性能”。英特爾還推出了新的 Xeon CPU 系列,它認爲可與英偉達的 GPU 競爭。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Xilinx 是一家發明 FPGA 的上市公司,最近又涉足人工智能加速器芯片領域,2020 年 10 月被 AMD 收購。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對人工智能計算能力的需求正在加速。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"變化與機遇"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如我在上面提到的,我的設想是,到 2021 年及以後,英偉達的主導地位將會受到越來越多的挑戰和侵蝕。造成這種情況的原因有四個:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"1. 學術研究變成真正的產品"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"學術界和工業界研究人員創立的一些初創公司已經開始研究機器學習專用硬件,而且還有更多的開發空間。在這個領域發表的論文並不只是提出理論上的保證,它還展示了真正的硬件原型,這些原型實現了比商業可用選項更好的指標。("},{"type":"link","attrs":{"href":"https:\/\/eyeriss.mit.edu\/","title":"","type":null},"content":[{"type":"text","text":"實例 1"}]},{"type":"text","text":"、"},{"type":"link","attrs":{"href":"https:\/\/news.mit.edu\/2020\/thousands-artificial-brain-synapses-single-chip-0608","title":"","type":null},"content":[{"type":"text","text":"實例 2"}]},{"type":"text","text":"和"},{"type":"link","attrs":{"href":"https:\/\/ieeexplore.ieee.org\/document\/8416814","title":"","type":null},"content":[{"type":"text","text":"實例 3"}]},{"type":"text","text":")"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"芯片和硬件加速器的種類很多,每一種都有其蓬勃發展的研究社區。簡單地列舉一些:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"專用集成電路(ASIC)。谷歌 TPU 和 AWS Inferentia 都是 ASIC 的例子。ASIC 產品的研發和生產成本可能高達 5000 萬美元,但是複製產品的邊際成本通常很低。ASIC 可以被設計成低功耗的,而且不會對性能有太大的影響。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"現場可編程邏輯門陣列(FPGA)。FPGA 對於高頻交易者來說已稀鬆平常,但在機器學習方面的例子包括微軟的 Brainwave 和英特爾的 Arria。單個 FPGA 的生產成本較低,但多個 FPGA 的"},{"type":"link","attrs":{"href":"https:\/\/resources.pcb.cadence.com\/blog\/2019-fpga-vs-asic-differences-and-choosing-best-for-your-business","title":"","type":null},"content":[{"type":"text","text":"生產邊際成本要高於 ASIC"}]},{"type":"text","text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"神經形態計算。該領域試圖對人腦的生物結構進行建模,並將其轉換成硬件。儘管神經形態學的思想可以追溯到 20 世紀 80 年代,但該領域仍處於起步階段。在《自然》上有一篇很好的"},{"type":"link","attrs":{"href":"https:\/\/www.nature.com\/articles\/s41586-019-1677-2","title":"","type":null},"content":[{"type":"text","text":"綜述性論文"}]},{"type":"text","text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"更多內容請參閱此項調查報告《"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/pdf\/2009.00993.pdf","title":"","type":null},"content":[{"type":"text","text":"機器學習加速芯片綜述"}]},{"type":"text","text":"》("},{"type":"text","marks":[{"type":"italic"}],"text":"Survey of Machine Learning Accelerators"},{"type":"text","text":"),並關注"},{"type":"link","attrs":{"href":"https:\/\/www.iscas2020.org\/","title":"","type":null},"content":[{"type":"text","text":"ISCAS"}]},{"type":"text","text":"。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"使用上述研究結果的一些有前途的初創公司:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Blaize 於 2019 年"},{"type":"link","attrs":{"href":"https:\/\/www.blaize.com\/products\/","title":"","type":null},"content":[{"type":"text","text":"宣稱"}]},{"type":"text","text":"已經開發出一種完全可編程的低功耗處理器,可實現 10 倍的低延遲,並且“系統效率最高可提高 60%”。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"SambaNova Systems 是由斯坦福大學教授和甲骨文前高管創立的初創公司,由谷歌風投和英特爾資本出資組建。它"},{"type":"link","attrs":{"href":"https:\/\/sambanova.ai\/press\/sambanova-systems-ushers-in-new-era-of-computing-with-availability-of-sambanova-datascale-built-for-ai\/","title":"","type":null},"content":[{"type":"text","text":"剛剛宣佈"}]},{"type":"text","text":"了一項新產品,該產品是一個“完整、集成的軟件和硬件系統平臺,可以對從算法到芯片的數據流進行優化”。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"bulletedlist","content":[{"type":"listitem","attrs":{"listStyle":null},"content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Graphcore 是一家英國初創公司,由紅杉、微軟、寶馬和 DeepMinds 創始人領投。"}]}]}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"2. 摩爾定律已死,但無論如何,專用硬件都是未來趨勢"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/e4\/35\/e41604c155dcab2ecb7bd54641f95535.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"摩爾定律預測,集成電路上的晶體管數量每兩年就會增加一倍。自 20 世紀 70 年代以來,這在經驗上一直是正確的,並且是我們從那時起所看到的技術進步的代名詞:個人計算革命、傳感器和攝像頭的改進、移動設備的興起,以及爲人工智能提供充足資源的崛起,凡是你能想到的一切。唯一的問題是,摩爾定律即將結束,如果它還沒有結束的話。“縮小芯片的難度越來越大,這已經不是什麼祕密了,而且這樣做的好處也今非昔比了。去年,英偉達的創始人黃仁勳直言不諱地認爲,‘摩爾定律已不再可能了’。”《"},{"type":"link","attrs":{"href":"https:\/\/www.economist.com\/technology-quarterly\/2020\/06\/11\/the-cost-of-training-machines-is-becoming-a-problem","title":"","type":null},"content":[{"type":"text","text":"經濟學人"}]},{"type":"text","text":"》(The Economist)寫道。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"麻省理工學院經濟學家 Neil Thompson 在《"},{"type":"link","attrs":{"href":"https:\/\/www.technologyreview.com\/2020\/02\/24\/905789\/were-not-prepared-for-the-end-of-moores-law\/","title":"","type":null},"content":[{"type":"text","text":"麻省理工科技評論"}]},{"type":"text","text":"》(MIT Technology Review)上解釋說:“軟件和專業架構方面的進步現在將開始有選擇地針對特定的問題和商業機會,對那些有充足資金和資源的人有利,而不是像摩爾定律那樣‘水漲船高’,通過提供速度更快、成本更低的芯片來普及。”一些人,包括 Thomspon 在內的,都"},{"type":"link","attrs":{"href":"https:\/\/papers.ssrn.com\/sol3\/papers.cfm?abstract_id=3287769","title":"","type":null},"content":[{"type":"text","text":"認爲"}]},{"type":"text","text":",“這是一個消極的發展,因爲計算硬件將開始分裂爲“‘快車道’應用和‘慢車道’應用程序,前者使用功能強大的定製芯片,而後者則被卡在使用通用芯片上,而且其進展緩慢。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於這個問題,分佈式計算常常是一種解決方案:讓我們使用功能更少、成本更低的資源,但要使用大量的資源。但是,就連這種方案也越來越昂貴(更別提分佈式梯度下降算法的複雜性了)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"那麼,接下來會發生什麼呢?2018 年,CMU 的研究人員在《自然》上發表了一篇論文,題爲《"},{"type":"link","attrs":{"href":"https:\/\/www.nature.com\/articles\/s41928-017-0005-9","title":"","type":null},"content":[{"type":"text","text":"摩爾定律末期的科學研究政策"}]},{"type":"text","text":"》("},{"type":"text","marks":[{"type":"italic"}],"text":"Science and research policy at the end of Moore’s law"},{"type":"text","text":"),該論文指出,私營部門將重點放在短期盈利上,這使得摩爾定律很難找到通用的繼承者。他們呼籲公私合作,共同創造計算硬件的未來。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/f7\/0a\/f775a0548617e62d45da26163852ae0a.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"雖然我並不反對公私合作(給予他們更多的權利),但我認爲未來的計算硬件將是專用芯片的集合,當它們協同工作時,它們比現在的 CPU 更能勝任通用任務。我相信"},{"type":"link","attrs":{"href":"https:\/\/www.apple.com\/newsroom\/2020\/06\/apple-announces-mac-transition-to-apple-silicon\/","title":"","type":null},"content":[{"type":"text","text":"蘋果向自己的芯片過渡"}]},{"type":"text","text":"是朝着這個方向邁出的一步,這證明了軟硬件集成系統將優於傳統芯片。特斯拉也在自動駕駛中採用了"},{"type":"link","attrs":{"href":"https:\/\/www.theverge.com\/2019\/4\/22\/18511594\/tesla-new-self-driving-chip-is-here-and-this-is-your-best-look-yet","title":"","type":null},"content":[{"type":"text","text":"自己的硬件"}]},{"type":"text","text":"。我們需要的是大量的新玩家湧入硬件生態系統,這樣專業芯片的好處就可以實現大衆化,並分佈在昂貴的筆記本電腦、雲服務器和汽車之外。(我敢說……是時候打造了嗎?)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"3. 創始人和投資者擔心成本上漲"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Andreessen Horowitz 的 Martin Casado 和 Matt Bornstein 在去年年初發表了一篇題爲《"},{"type":"link","attrs":{"href":"https:\/\/a16z.com\/2020\/02\/16\/the-new-business-of-ai-and-how-its-different-from-traditional-software\/","title":"","type":null},"content":[{"type":"text","text":"人工智能的新業務(及其與傳統軟件的區別"}]},{"type":"text","text":"》("},{"type":"text","marks":[{"type":"italic"}],"text":"The New Business of AI (and How It’s Different From Traditional Software)"},{"type":"text","text":")的文章,他們認爲人工智能的業務與傳統軟件是不同的。說到底,一切都與利潤有關。“雲計算基礎設施對人工智能公司來說是一個巨大的成本,有時甚至是隱性成本”。正如我所提到的那樣,訓練人工智能模型可能需要花費數千美元(如果你是 OpenAI,你就得花數百萬美元),但成本並不止於這些。人工智能系統必須得到持續監控和改進。如果你的模型是“離線”訓練的,那麼它很容易出現概念漂移,即現實世界中的數據分佈隨着時間的推移與你訓練的數據發生變化。這種情況可能是自然發生的,也可能是對抗性的,比如當用戶試圖欺騙信用風險算法時。出現這種情況時,就必須對模型進行再訓練。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"對於降低概念漂移和創建與現有模型具有相同性能保證的更小的模型有一些積極的研究,但這是另一篇文章的主題。同時,該行業也正在推進更大的模型和更大的計算支出。更便宜、更專業的人工智能芯片無疑會降低這些成本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":3},"content":[{"type":"text","text":"4. 訓練大型模型有助於氣候變化"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/6e\/97\/6ec0c85f75b5ee173c078de7f02e6997.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由馬薩諸塞大學阿默斯特分校進行的"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/pdf\/1906.02243.pdf","title":"","type":null},"content":[{"type":"text","text":"一項研究"}]},{"type":"text","text":"發現,訓練一個現成的自然語言處理模型所產生的碳排放量相當於從舊金山飛往紐約的一次航班。在三大雲計算供應商中,只有谷歌的數據中心超過 50% 的能源來自可再生能源。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"但我認爲,我不必列出我們爲什麼要減少人工智能的碳排放。我想說的是,現有的芯片耗電量過大,而且研究表明,其他類型的硬件加速器,如 FPGA 和超低能耗芯片(如谷歌 TPU Edge),對於機器學習和其他任務來說,"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/pdf\/1906.11879.pdf","title":"","type":null},"content":[{"type":"text","text":"可以更加節能"}]},{"type":"text","text":"。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"即使是地理也會影響到人工智能的碳排放。"},{"type":"link","attrs":{"href":"https:\/\/hai.stanford.edu\/blog\/ais-carbon-footprint-problem","title":"","type":null},"content":[{"type":"text","text":"斯坦福大學的研究人員估計"}]},{"type":"text","text":",“在主要依賴頁岩油的愛沙尼亞舉行一次會議,其產生的碳排放量是在魁北克舉行的會議的 30 倍,而魁北克主要依靠水力發電。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"已露端倪"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"我已經提到了人工智能的硬件,但是人工智能的硬件怎麼樣?谷歌最近"},{"type":"link","attrs":{"href":"https:\/\/patents.google.com\/patent\/US20200279163A1\/en","title":"","type":null},"content":[{"type":"text","text":"申請了一項專利"}]},{"type":"text","text":",該專利是關於一種利用強化學習來確定跨多個硬件設備的機器學習模型操作的位置的方法。這項專利背後的研究人員之一是"},{"type":"link","attrs":{"href":"https:\/\/www.technologyreview.com\/innovator\/azalia-mirhoseini\/","title":"","type":null},"content":[{"type":"text","text":"Azalea Mirhoseini"}]},{"type":"text","text":",她在 Google Brain 負責機器學習硬件 \/ 系統的登月計劃。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者介紹:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Andrei Kozyrev,康奈爾大學攻讀計算機科學與政治學。研究機器學習中的公平性、隱私性和可解釋性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文鏈接:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"https:\/\/fairlydeep.substack.com\/p\/2021-will-be-the-year-of-ai-hardware"}]}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章