三位深度學習先驅聯合發文:深度學習的挑戰與未來

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度學習的三位先驅在ACM通訊期刊7月刊上發表的一篇論文中指出,深度神經網絡將在沒有來自"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2019\/11\/18\/what-is-symbolic-artificial-intelligence\/","title":"","type":null},"content":[{"type":"text","text":"符號人工智能"}]},{"type":"text","text":"的幫助的情況下邁過當下面臨的種種障礙。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2018年圖靈獎獲得者Yoshua Bengio、Geoffrey Hinton和Yann LeCun在他們的論文中解釋了深度學習當前面臨的種種"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2018\/02\/27\/limits-challenges-deep-learning-gary-marcus\/","title":"","type":null},"content":[{"type":"text","text":"挑戰"}]},{"type":"text","text":",以及它與人類和動物學習機制的區別。他們還探索了該領域的一些最新進展,這些進展可能爲未來的深度學習研究的指明方向。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這篇題爲“邁向人工智能的深度學習”的"},{"type":"link","attrs":{"href":"https:\/\/cacm.acm.org\/magazines\/2021\/7\/253464-deep-learning-for-ai\/fulltext","title":"","type":null},"content":[{"type":"text","text":"論文"}]},{"type":"text","text":"設想的未來世界中,深度學習模型可以在很少用到或不需要人類幫助的情況下學習,靈活適應環境變化,並且可以解決種類廣泛的反身和認知問題。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度學習面臨的挑戰"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/43\/1f\/43fbf604d127e2b3714a0a3a3c7e891f.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"上圖:深度學習先驅Yoshua Bengio(左)、Geoffrey Hinton(中)和Yann LeCun(右)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度學習經常被比作人類和動物的大腦。然而,過去幾年的研究已經證明,深度學習模型中使用的主要組成部分——人工神經網絡,缺乏生物大腦具備的效率、靈活性和功能多樣性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他們的論文中承認了這些缺點。“監督學習雖然在許多任務中都取得了成功,但它們通常需要大量人工標記的數據。類似地,當強化學習只基於獎勵訓練時,它就需要大量的交互,”他們寫道。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"監督學習是機器學習算法的一個流行子集。監督學習中,模型獲取很多帶標記的樣本,例如圖像列表及其對應的內容。模型經過訓練,可以在具有相似標籤的樣本中找到重複出現的模式。然後模型使用學習到的模式將新樣本與正確的標籤相關聯。監督學習對於有大量標記樣本可用的問題特別好用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"強化學習是機器學習的另一個分支。在強化學習中,“代理”學習如何在環境中最大化“獎勵”。環境可以很簡單,一個井字棋遊戲板就可以成爲一個環境,其中AI玩家排列出三個X或O就能獲得獎勵;環境也可以很複雜,比如說城市環境,其中自動駕駛汽車避免碰撞、服從交通規則、到達目的地都能獲得獎勵。代理首先會採取隨機行動。當它從環境中接收反饋後,它會找到提供更好獎勵的動作序列。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如科學家們所承認的,在這兩種情況下,機器學習模型都需要大量的勞動力。標記好的數據集很難獲得,尤其是在沒有公開、開源數據集的專業領域,這意味着它們需要人工註釋人員付出大量艱苦而昂貴的勞動。複雜的強化學習模型需要大量的計算資源來運行大量訓練集,這意味着它們只能被少數非常富有的AI實驗室和科技公司使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun也承認,當前的深度學習系統可以解決的問題範圍仍然很"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2020\/04\/09\/what-is-narrow-artificial-intelligence-ani\/","title":"","type":null},"content":[{"type":"text","text":"有限"}]},{"type":"text","text":"。這些系統在專門的任務上表現良好,但“在它們接受過訓練的狹窄領域之外往往很脆弱。”輕微的變化(例如圖像中的一些像素修改或環境中規則的微小變化)往往都會導致深度學習系統誤入歧途。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度學習系統的脆弱性很大程度上是因爲機器學習模型基於“獨立同分布”(i.i.d.)假設,也就是假設真實世界的數據與訓練數據具有相同的分佈。i.i.d還假設觀察不會相互影響(例如,硬幣或擲骰子是相互獨立的)。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科學家們寫道:“從早期開始,機器學習的理論家就專注於iid假設……不幸的是,這在真實世界中並不是一個現實的假設。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由於各種因素,現實世界的條件在不斷變化。如果沒有"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2021\/03\/15\/machine-learning-causality\/","title":"","type":null},"content":[{"type":"text","text":"因果模型"}]},{"type":"text","text":",其中許多條件實際上是不可能表示的。智能代理必須不斷地觀察它們的環境和其他代理並從中學習,並且它們必須讓自己的行爲適應變化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科學家們寫道:“當今最好的人工智能系統在從實驗室進入現場時,性能也往往會受到影響。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"i.i.d假設應用於計算機視覺和自然語言處理等領域時會變得更加脆弱,因爲這種場景中代理必須處理高熵環境。目前,許多研究人員和公司試圖用更多數據訓練神經網絡來克服深度學習的侷限性,希望更大的數據集能夠覆蓋更廣泛的分佈,並減少系統在現實世界中失敗的機率。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度學習 vs 混合AI"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"人工智能科學家的最終目標是複製人類所擁有的那種通用智能。而且我們知道人類不會被當前深度學習系統所面臨的那些問題所困擾。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他們的論文中寫道:“人類和動物似乎能夠以不依賴具體任務的方式,主要通過觀察來學習大量與世界相關的背景知識。”“這些知識塑造了常識,讓人類只需幾個小時的練習就能學會複雜的任務,比如駕駛。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在論文中科學家們還指出,“人類可以以一種不同於普通iid泛化的方式來做泛化:我們可以正確解釋現有概念的全新組合,即便這些組合在我們經受的訓練中極爲罕見也沒關係,只要它們尊重我們已經學到的高級句法和語義模式即可。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科學家們提供了各種解決方案來縮小人工智能和人類智能之間的差距。在過去幾年中被廣泛討論的一種方法是將神經網絡與經典符號系統相結合的混合人工智能。符號操作是人類推理世界能力的一個非常重要的部分。這也是深度學習系統面臨的巨大挑戰之一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun不相信混合神經網絡和符號AI。在ACM論文隨附的一段"},{"type":"link","attrs":{"href":"https:\/\/cacm.acm.org\/videos\/deep-learning-for-ai","title":"","type":null},"content":[{"type":"text","text":"視頻"}]},{"type":"text","text":"中Bengio說:“有些人認爲有一些問題是神經網絡無法解決的,於是我們必須求助於經典AI,也就是符號方法。但我們的工作表明現實並非如此。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"幾位深度學習先驅認爲,更好的神經網絡架構最終會覆蓋人類和動物智能的各個層面,包括符號操作、推理、因果推理和常識。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度學習領域頗有前景的那些進展"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他們的論文中介紹了一些深度學習領域的最新進展,這些進展可以在深度學習面臨困難的一些領域幫助推進研究工作。一個例子是"},{"type":"link","attrs":{"href":"https:\/\/jalammar.github.io\/illustrated-transformer\/","title":"","type":null},"content":[{"type":"text","text":"Transformer"}]},{"type":"text","text":"(變換器),這是一種神經網絡架構,一直是OpenAI的GPT-3和谷歌的Meena等語言模型的核心。變換器的一個好處是它們能夠在不需要標記數據的情況下進行學習。變換器可以通過無監督學習來開發表徵,然後它們可以應用這些表徵來填補不完整句子的空白,或在收到提示後生成連貫的文本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最近,研究人員表明變換器也可以應用於計算機視覺任務。變換器與卷積神經網絡結合時,可以預測遮擋區域的內容。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一種更有前途的技術是"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/abs\/2002.05709","title":"","type":null},"content":[{"type":"text","text":"對比學習"}]},{"type":"text","text":",它試圖找到缺失區域的向量表示,而不是預測精確的像素值。這是一種有趣的方法,似乎更接近人類的思維方式。當我們看到如下圖所示的圖像時,我們可能無法想象出缺失部分的精確內容,但我們的大腦可以想象出那些遮擋區域中可能發生的情況(例如門、窗等)。(我自己的觀察:這種技術可以與該領域的其他一些研究很好地結合起來,這些研究旨在讓神經網絡中的向量表示與現實世界的概念對齊。)"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"推動神經網絡減少對人類標記數據的依賴則是自監督學習的討論範疇,這是LeCun正在研究的一個概念。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ab\/8b\/ab31537d0ef1a66cb8d771d6db29a98b.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"上圖:你能猜出上圖中灰色框的後面是什麼嗎?"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"這篇論文還提到了“系統2(system 2)深度學習”,這個詞是從諾貝爾獎獲得者,心理學家Daniel Kahneman那裏借來的。系統2指的是需要有意識思考的那些大腦功能,包括符號操作、推理、多步計劃和解決複雜的數學問題等。系統2深度學習的研究仍處於早期階段,但如果它能成爲現實,就可以解決神經網絡面對的一些關鍵問題,包括分佈外泛化、因果推理、健壯遷移學習和符號操作等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科學家們所做的工作還包括研究“爲對象及對象的組成部分分配內在參考框架,並使用幾何關係來識別對象”的神經網絡。這裏引用了“膠囊網絡”,這是Hinton在過去幾年中一直關注的研究領域。膠囊網絡旨在升級神經網絡,讓它們從只檢測圖像中的特徵升級到檢測圖像中的各種對象、它們的物理特性以及它們之間的層次關係。膠囊網絡可以提供爲深度學習帶來“直覺物理學”,這種能力讓人類和動物得以理解三維環境。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“在實現真正聰明有用的神經網絡的目標之前,我們還有很長的路要走。我們希望行業會出現全新的想法,”Hinton這樣告訴ACM。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者介紹"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Ben Dickson是一名軟件工程師,也是TechTalks的創始人。他撰寫的文章涉及技術、商業和政治主題。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文鏈接:"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/venturebeat.com\/2021\/07\/05\/the-future-of-deep-learning-according-to-its-pioneers\/","title":"","type":null},"content":[{"type":"text","text":"https:\/\/venturebeat.com\/2021\/07\/05\/the-future-of-deep-learning-according-to-its-pioneers\/"}]}]}]}
發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章