三位深度学习先驱联合发文：深度学习的挑战与未来

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度学习的三位先驱在ACM通讯期刊7月刊上发表的一篇论文中指出，深度神经网络将在没有来自"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2019\/11\/18\/what-is-symbolic-artificial-intelligence\/","title":"","type":null},"content":[{"type":"text","text":"符号人工智能"}]},{"type":"text","text":"的帮助的情况下迈过当下面临的种种障碍。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"2018年图灵奖获得者Yoshua Bengio、Geoffrey Hinton和Yann LeCun在他们的论文中解释了深度学习当前面临的种种"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2018\/02\/27\/limits-challenges-deep-learning-gary-marcus\/","title":"","type":null},"content":[{"type":"text","text":"挑战"}]},{"type":"text","text":"，以及它与人类和动物学习机制的区别。他们还探索了该领域的一些最新进展，这些进展可能为未来的深度学习研究的指明方向。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这篇题为“迈向人工智能的深度学习”的"},{"type":"link","attrs":{"href":"https:\/\/cacm.acm.org\/magazines\/2021\/7\/253464-deep-learning-for-ai\/fulltext","title":"","type":null},"content":[{"type":"text","text":"论文"}]},{"type":"text","text":"设想的未来世界中，深度学习模型可以在很少用到或不需要人类帮助的情况下学习，灵活适应环境变化，并且可以解决种类广泛的反身和认知问题。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度学习面临的挑战"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/43\/1f\/43fbf604d127e2b3714a0a3a3c7e891f.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"上图：深度学习先驱Yoshua Bengio（左）、Geoffrey Hinton（中）和Yann LeCun（右）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度学习经常被比作人类和动物的大脑。然而，过去几年的研究已经证明，深度学习模型中使用的主要组成部分——人工神经网络，缺乏生物大脑具备的效率、灵活性和功能多样性。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他们的论文中承认了这些缺点。“监督学习虽然在许多任务中都取得了成功，但它们通常需要大量人工标记的数据。类似地，当强化学习只基于奖励训练时，它就需要大量的交互，”他们写道。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"监督学习是机器学习算法的一个流行子集。监督学习中，模型获取很多带标记的样本，例如图像列表及其对应的内容。模型经过训练，可以在具有相似标签的样本中找到重复出现的模式。然后模型使用学习到的模式将新样本与正确的标签相关联。监督学习对于有大量标记样本可用的问题特别好用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"强化学习是机器学习的另一个分支。在强化学习中，“代理”学习如何在环境中最大化“奖励”。环境可以很简单，一个井字棋游戏板就可以成为一个环境，其中AI玩家排列出三个X或O就能获得奖励；环境也可以很复杂，比如说城市环境，其中自动驾驶汽车避免碰撞、服从交通规则、到达目的地都能获得奖励。代理首先会采取随机行动。当它从环境中接收反馈后，它会找到提供更好奖励的动作序列。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"正如科学家们所承认的，在这两种情况下，机器学习模型都需要大量的劳动力。标记好的数据集很难获得，尤其是在没有公开、开源数据集的专业领域，这意味着它们需要人工注释人员付出大量艰苦而昂贵的劳动。复杂的强化学习模型需要大量的计算资源来运行大量训练集，这意味着它们只能被少数非常富有的AI实验室和科技公司使用。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun也承认，当前的深度学习系统可以解决的问题范围仍然很"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2020\/04\/09\/what-is-narrow-artificial-intelligence-ani\/","title":"","type":null},"content":[{"type":"text","text":"有限"}]},{"type":"text","text":"。这些系统在专门的任务上表现良好，但“在它们接受过训练的狭窄领域之外往往很脆弱。”轻微的变化（例如图像中的一些像素修改或环境中规则的微小变化）往往都会导致深度学习系统误入歧途。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"深度学习系统的脆弱性很大程度上是因为机器学习模型基于“独立同分布”（i.i.d.）假设，也就是假设真实世界的数据与训练数据具有相同的分布。i.i.d还假设观察不会相互影响（例如，硬币或掷骰子是相互独立的）。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科学家们写道：“从早期开始，机器学习的理论家就专注于iid假设……不幸的是，这在真实世界中并不是一个现实的假设。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"由于各种因素，现实世界的条件在不断变化。如果没有"},{"type":"link","attrs":{"href":"https:\/\/bdtechtalks.com\/2021\/03\/15\/machine-learning-causality\/","title":"","type":null},"content":[{"type":"text","text":"因果模型"}]},{"type":"text","text":"，其中许多条件实际上是不可能表示的。智能代理必须不断地观察它们的环境和其他代理并从中学习，并且它们必须让自己的行为适应变化。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科学家们写道：“当今最好的人工智能系统在从实验室进入现场时，性能也往往会受到影响。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"i.i.d假设应用于计算机视觉和自然语言处理等领域时会变得更加脆弱，因为这种场景中代理必须处理高熵环境。目前，许多研究人员和公司试图用更多数据训练神经网络来克服深度学习的局限性，希望更大的数据集能够覆盖更广泛的分布，并减少系统在现实世界中失败的机率。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度学习 vs 混合AI"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"人工智能科学家的最终目标是复制人类所拥有的那种通用智能。而且我们知道人类不会被当前深度学习系统所面临的那些问题所困扰。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他们的论文中写道：“人类和动物似乎能够以不依赖具体任务的方式，主要通过观察来学习大量与世界相关的背景知识。”“这些知识塑造了常识，让人类只需几个小时的练习就能学会复杂的任务，比如驾驶。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"在论文中科学家们还指出，“人类可以以一种不同于普通iid泛化的方式来做泛化：我们可以正确解释现有概念的全新组合，即便这些组合在我们经受的训练中极为罕见也没关系，只要它们尊重我们已经学到的高级句法和语义模式即可。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科学家们提供了各种解决方案来缩小人工智能和人类智能之间的差距。在过去几年中被广泛讨论的一种方法是将神经网络与经典符号系统相结合的混合人工智能。符号操作是人类推理世界能力的一个非常重要的部分。这也是深度学习系统面临的巨大挑战之一。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun不相信混合神经网络和符号AI。在ACM论文随附的一段"},{"type":"link","attrs":{"href":"https:\/\/cacm.acm.org\/videos\/deep-learning-for-ai","title":"","type":null},"content":[{"type":"text","text":"视频"}]},{"type":"text","text":"中Bengio说：“有些人认为有一些问题是神经网络无法解决的，于是我们必须求助于经典AI，也就是符号方法。但我们的工作表明现实并非如此。”"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"几位深度学习先驱认为，更好的神经网络架构最终会覆盖人类和动物智能的各个层面，包括符号操作、推理、因果推理和常识。"}]},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"深度学习领域颇有前景的那些进展"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Bengio、Hinton和LeCun在他们的论文中介绍了一些深度学习领域的最新进展，这些进展可以在深度学习面临困难的一些领域帮助推进研究工作。一个例子是"},{"type":"link","attrs":{"href":"https:\/\/jalammar.github.io\/illustrated-transformer\/","title":"","type":null},"content":[{"type":"text","text":"Transformer"}]},{"type":"text","text":"（变换器），这是一种神经网络架构，一直是OpenAI的GPT-3和谷歌的Meena等语言模型的核心。变换器的一个好处是它们能够在不需要标记数据的情况下进行学习。变换器可以通过无监督学习来开发表征，然后它们可以应用这些表征来填补不完整句子的空白，或在收到提示后生成连贯的文本。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"最近，研究人员表明变换器也可以应用于计算机视觉任务。变换器与卷积神经网络结合时，可以预测遮挡区域的内容。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"一种更有前途的技术是"},{"type":"link","attrs":{"href":"https:\/\/arxiv.org\/abs\/2002.05709","title":"","type":null},"content":[{"type":"text","text":"对比学习"}]},{"type":"text","text":"，它试图找到缺失区域的向量表示，而不是预测精确的像素值。这是一种有趣的方法，似乎更接近人类的思维方式。当我们看到如下图所示的图像时，我们可能无法想象出缺失部分的精确内容，但我们的大脑可以想象出那些遮挡区域中可能发生的情况（例如门、窗等）。（我自己的观察：这种技术可以与该领域的其他一些研究很好地结合起来，这些研究旨在让神经网络中的向量表示与现实世界的概念对齐。）"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"推动神经网络减少对人类标记数据的依赖则是自监督学习的讨论范畴，这是LeCun正在研究的一个概念。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ab\/8b\/ab31537d0ef1a66cb8d771d6db29a98b.jpg","alt":null,"title":"","style":[{"key":"width","value":"75%"},{"key":"bordertype","value":"none"}],"href":"","fromPaste":false,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":"center","origin":null},"content":[{"type":"text","text":"上图：你能猜出上图中灰色框的后面是什么吗？"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"这篇论文还提到了“系统2（system 2）深度学习”，这个词是从诺贝尔奖获得者，心理学家Daniel Kahneman那里借来的。系统2指的是需要有意识思考的那些大脑功能，包括符号操作、推理、多步计划和解决复杂的数学问题等。系统2深度学习的研究仍处于早期阶段，但如果它能成为现实，就可以解决神经网络面对的一些关键问题，包括分布外泛化、因果推理、健壮迁移学习和符号操作等。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"科学家们所做的工作还包括研究“为对象及对象的组成部分分配内在参考框架，并使用几何关系来识别对象”的神经网络。这里引用了“胶囊网络”，这是Hinton在过去几年中一直关注的研究领域。胶囊网络旨在升级神经网络，让它们从只检测图像中的特征升级到检测图像中的各种对象、它们的物理特性以及它们之间的层次关系。胶囊网络可以提供为深度学习带来“直觉物理学”，这种能力让人类和动物得以理解三维环境。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"“在实现真正聪明有用的神经网络的目标之前，我们还有很长的路要走。我们希望行业会出现全新的想法，”Hinton这样告诉ACM。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"作者介绍"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Ben Dickson是一名软件工程师，也是TechTalks的创始人。他撰写的文章涉及技术、商业和政治主题。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","marks":[{"type":"strong"}],"text":"原文链接："}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"link","attrs":{"href":"https:\/\/venturebeat.com\/2021\/07\/05\/the-future-of-deep-learning-according-to-its-pioneers\/","title":"","type":null},"content":[{"type":"text","text":"https:\/\/venturebeat.com\/2021\/07\/05\/the-future-of-deep-learning-according-to-its-pioneers\/"}]}]}]}