微软图灵通用语言表示模型 T-ULRv2 位居 XTREME 排行榜榜首

{"type":"doc","content":[{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"今天，我们很高兴地宣布，图灵多语言模型（Turing multilingual language model，T-ULRv2）目前位居 Google "},{"type":"link","attrs":{"href":"https:\/\/sites.research.google\/xtreme","title":"","type":null},"content":[{"type":"text","text":"XTREME 公共排行榜"}]},{"type":"text","text":"榜首。该模型是由 Microsoft 图灵团队与 Microsoft Research 合作开发的，该模型的平均得分高出阿里巴巴（VECO）之前的最好成绩 3.5 分。为了实现这一点，除了预训练模型之外，我们还使用了“StableTune”，这是一种基于稳定性训练的新型多语言微调技术。排行榜上的其他模型包括 XLM-R、 mBERT、XLM 等。以前最好的提交之一也是来自 Microsoft 使用 "},{"type":"link","attrs":{"href":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/filter-an-enhanced-fusion-method-for-cross-lingual-language-understanding\/","title":"","type":null},"content":[{"type":"text","text":"FILTER"}]},{"type":"text","text":" 提交的。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"image","attrs":{"src":"https:\/\/static001.infoq.cn\/resource\/image\/ac\/4d\/ac28daee75ea9f445159a2af1508e44d.png","alt":null,"title":null,"style":null,"href":null,"fromPaste":true,"pastePass":false}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"heading","attrs":{"align":null,"level":2},"content":[{"type":"text","text":"通用语言表示"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null},"content":[{"type":"text","text":"Microsoft 图灵团队长期以来，一直认为语言表示应该是通用的。在这篇发表于 2018 年的"},{"type":"link","attrs":{"href":"https:\/\/www.microsoft.com\/en-us\/research\/publication\/towards-language-agnostic-universal-representations\/","title":"","type":null},"content":[{"type":"text","text":"论文"}]},{"type":"text","text":" 中，我们提出了一种以无监督方式训练语言不可知表示的方法。这种方法允许用一种语言对训练过的模型进行微调，并以零样本学习的方式应用于另一种语言。这将克服要求标签数据以每种语言训练模型的挑战。自从这篇论文发表以来，无监督的预训练语言建模已成为所有自然语言处理模型的支柱，而基于 Transformer 的模型是所有这类创新的核心。"}]},{"type":"paragraph","attrs":{"indent":0,"number":0,"align":null,"origin":null}}]}